Should I use a CoW filesystem on my PC if I only wanted snapshot capabilities ?
I will be installing Linux on my HP Laptop and I really like ext4, its stable and time tested. But I want snapshot capabilities, or something like system restore in Windows. This is obviously for times when I shoot myself in the foot and want to restore back to a stable state.
Will filesystems like ZFS or btrfs work better in these cases rather than ext4 ?
My only requirement is snapshots, so is it worth running btrfs or ZFS despite its unstabilty (more for btrfs rather than ZFS).
Location: Asia Pacific, Cyberspace, in the Dark Dystopia
Posts: 19,118
Thanks Given: 2,351
Thanked 3,359 Times in 1,878 Posts
No.
I do not recommend those file systems.
Your are better off running ext4, a RAID configuration (I run RAID1, but do not depend on it), and doing regular backups on your data based on your risk management model (this is the most critical).
Nothing beats a strong filesystem and a very well thought out backup and recovery plan.
That is my view. YMMV
On the desktop, I run macOS and have a similar strategy. I make full backups often, based on the activity on the system. The more activity and files (and the nature of the files) created, the more frequent the backups.
If you are using LVM, you can use logical volume snapshot ability to achieve want you want.
BRTFS i have not used, but i did read some horror stories some time ago.
Probably nothing to worry about for home use, since those bugs were about raid protection.
ZFS in ubuntu, for instance, is openzfs (OpenZFS)
This is a mature and high quality file system & volume manager, but i would not use it for root just yet.
For data disks, i see no reason to reap the benefits of snapshots, compression and deduplication.
Just be sure those dedup tables fit in memory
Here is a quick example of LVM snapshots from my home box which is filesystem agnostic :
You need to have free space in volume group to create a snapshot, as i do, so lets create a snapshot.
We also specify that 5GB of total space in VG that can be consumed by system to maintain snapshot.
Hope that helps
Regards
Peasant.
These 2 Users Gave Thanks to Peasant For This Post:
Torvalds will not allow Linux kernel inclusion of ZFS support because of Oracle's position on ZFS licensure, this was important to us because we have only ZFS on Solaris 11/12 boxes. We did not want different files systems for production Linux servers - but that is what we got....ext4
How this plays out on a home desktop I cannot say exactly. I would recommend NOT using ZFS for Linux boot filesystems - as @neo said.
This User Gave Thanks to jim mcnamara For This Post:
Location: Saint Paul, MN USA / BSD, CentOS, Debian, OS X, Solaris
Posts: 2,288
Thanks Given: 430
Thanked 480 Times in 395 Posts
Hi.
Quote:
Originally Posted by sreyan32
... But I want snapshot capabilities, or something like system restore in Windows. This is obviously for times when I shoot myself in the foot and want to restore back to a stable state ...
I am a big fan of Virtual Machine technology. Here is what I do:
On my main workstation I install a small, stripped-down (i.e. no Office, etc.) Linux distribution as a host -- I prefer Debian GNU/Linux. I then install Virtualbox and create a VM. On the VM I install my day-to-day work environment -- again Debian.
Whenever I have mods to install, I use VB to take a snapshot. Then I install the mods. I leave the snapshot for some time (it's a CoW). If it works for a few days, a week, etc., then I merge the collected CoW changes into the VM (by, ironically, deleting the snapshot). If the mods fail to run, I restore the running system. I've had to do the restore perhaps 3 times in years, and it goes quite quickly, as does the creating and merging of the CoW.
Finally if the VM seems OK, then I install the mods into the host system.
For a plan for backup, we use a separate computer as a backup server, and on that we have a set of mirrored disks. We use that to run rsnapshot to backup our running, day-to-day systems. ( You might be able to run rsnapshot on the running system itself, in which case you can then use LVM, and rsnapshot will do its own LVM snapshot, do the backup, and remove its snapshot; then you could, say, tar up the resulting backup and send it to another computer ).
We have a small shop and rsnapshot helps us in many ways. The rsnapshot utility is a pull system, so the remote needs passphrase-less access to the system being backed up. The big advantage of rsnapshot is that it uses hard links, so conserving storage dramatically. For example, I backup my day-to-day system every hour, day, week, month, so 24+7+4+12 -> 47 collections, yet rarely goes over 20 GB, but oddly, if you look at any single collection, it is 20 GB, all due to the magic of hard links. The code also uses an algorithm that it transfers only the changes of a particular file, thus saving real time and network time. It also handles all renaming, copying, and removing of necessary files to accomplish the rotation of backup collection names.
We are also interested in the zfs filesystem. Our rsnapshot backup server was replaced this month with a newer model -- the old one lasted 17 years (2005..2020). In addition to being an external backup, I also installed VBox there and, as VMs, installed as guests Ubuntu 19.10 on zfs, as well as FreeBSD 12 on zfs. Since the host is on a RAID1 mirror, I didn't need the additional support for zfs mirroring (but it might be of some interest later on to experiment with them). We've had a Solaris VM for a long time on zfs:
As additional VMs, we also installed a guest that is the same as the host, and we're using that as a test bed (just as I do with my day-to-day workstation). This new install is of distribution Debian GNU/Linux buster. We also like to have the next rev available, so I installed the testing version, known as bullseye.
So VMs are what we use for experimentation, as well as to make backups, like MS System Restore Points, easy.
Location: Asia Pacific, Cyberspace, in the Dark Dystopia
Posts: 19,118
Thanks Given: 2,351
Thanked 3,359 Times in 1,878 Posts
Thanks drl,
Honestly, I have tried a similar approach on macOS, but even with a 12-core machine with 64 GB of memory on my desktop, all flavors of VMs slow the machine down to an intolerable degree. After all, running in a VM, by definition, is slower than running "on bare metal".
My guess is that your desktop VM configuration on Linux runs a lot better, since you are a bit fan.
Because I prefer the fastest speed possible (we are talking desktop operations) I find that using the built in macOS time machine to keep data based up works great; so in this configuration, I keep my machine running the fastest possible (running "bare metal" not in a VM) and at the same time my data is always backed up an external disk.
Maybe we mac users are lucky because time machine works so well and is very easy to use for users of all skill levels?
From the big wiki in the sky:
Quote:
Time Machine creates incremental backups of files that can be restored at a later date.[1] It allows the user to restore the whole system or specific files from the Recovery HD or the macOS Install DVD. It works within Mail, iWork, iLife, and several other compatible programs, making it possible to restore individual objects (e.g. emails, photos, contacts, calendar events) without leaving the application. According to an Apple support statement:
“Time Machine is a backup utility, not an archival utility, it is not intended as offline storage. Time Machine captures the most recent state of your data on your disk. As snapshots age, they are prioritized progressively lower compared to your more recent ones.”¯[2]
For backups to a network drive, Time Machine allows the user to back up Mac computers over the network, and supports backing up to certain network attached storage devices or servers, depending on the version of Time Machine. Earlier versions worked with a wide variety of NAS servers, but later versions require the server to support a recent version of Apple's Apple Filing Protocol (AFP), and Time Machine no longer works with servers using earlier versions of the Server Message Block (SMB) protocol[3] typical for Windows servers. Some of the legacy support can be re-enabled by using hand-tuned configuration options, accessed through the Terminal. Apple's Time Capsule acts as a network storage device specifically for Time Machine backups, allowing both wired and wireless backups to the Time Capsule's internal hard drive. Time Machine may also be used with any external or internal volume.
Time Machine saves hourly backups for the past 24 hours, daily backups for the past month, and weekly backups for everything older than a month until the volume runs out of space. At that point, Time Machine deletes the oldest weekly backup.
As I mentioned before, on my Linux servers in production, I run ext4 on all of them using RAID1 and make incremental and full backups of all mission critical data on a daily basis. Of course, remote servers are different compared desktops; and so that is why I prefer macOS and time machine on the desktop.
Naturally, everyone has different setups and preferences, so it is great to see people sharing their ideas openly and open-mindedly.
I do not recommend the filesystems the OP suggested (ZFS or btrfs) on Linux and as mentioned, and prefer a different backup strategy (but then again, I don't user Linux on the desktop, as mentioned, but If I used Linux on the desktop instead of macOS, I would still use ext4. This file system (ext) has served me well over the many years and I cannot recall a single problem related to the ext filesystem 27 years of Linux experience. I like "reliable" and "proven" in a filesystem... even if it is not the "most fancy SOTA out there.).
some personal experiences and experiences from others:
zfs
I'm very fond of the ease of use of zfs administration. Few simple commands which all did what I expected them to do. Compression is recommended. Data Checksumming is also a great feature. Deduplication is not as clearly recommended. It needs lots of resources(RAM). It's not that flexible as LVM or btrfs, but in a Desktop-Environment, this should not be a problem. No Problems in several years of usage. Some reading about zfs is recommended for basic configuration (ashift, blocksize values.) getting root fs to zfs is manual work. But for a data partition it's very easy to use. If I need more flexibility, I will use lvm.
btrfs
The times when btrfs had grave bugs are long gone. If you use it, make sure you do not use features that are marked as experimental. (For example the btrfs internal raid5 implementation. Use linux software raid with btrfs, if you want raid). I read some entries in another forum from someone who uses it at scale at will never change it for any other filesystem(he had experieance with all major filesystems) . It also has checksums and snapshot capabilities. The flexibility far exceeds zfs and lvm.
I checked it out and decided not to use it, for these reasons:
Some things are more complicated. You have to work your way through the documentation quite carefully. For example. You can not trust the values of du and df. The complexity of the filesystems circumvents that this is always correct and consistent. btrfs introduces own tools in addition to the tools everybody is used to.
Things do not work the way I like them to: If I have some raid and a disk is failing, I would expect the system to come up and maybe complain about that it is in degraded mode or just have some file, which would be my part to check. And if the disk is replaced, I expect an easy command or even automatic restoration of the failed disk. That seems not to be the case with btrfs. If you then use your file system in degraded mode, bad things can happen and your raid will do things like writing data only to one disk despite the fact that the device may be set up as a mirror. Those data may be kept non-replicated even if you replace and reintegrate a new disk afterwards. That of course may lead to data loss. That's not an error of the filesystem, but the ordinary procedure. One has to know correctly how btrfs is to be operated according to the documentation or you'll get into serious trouble.
I'd say btrfs is an advanced filesystem, which get's you a lot in terms of flexibility, performance, robustness, features and data security. It comes with the cost of taking your time and really get to learn the details, which can become really important.
On the other hand filesystem snapshotting ist not what you get with windows system restore points. I think that is a lot of voodoo going on there with windows system restore points(meaning it is complex under the hood). I did not ever test if filesystem-snapshots really get you a working system if you roll them back. Maybe it just works. But if you have a database, there's no guarantee, that the data will be consistent with such a snapshot.
I worked on my personal workstation within a VM too. Snapshots were possible. But the user experience was a mess. Regular Problems with the virtualization environment(virtualbox) and speed too drove me away from that and back to bare metal. For testing things virtual machines are great. Snapshotting ist great there. But not for the main workstation(for me).
Personally OS state snapshotting it's a feature, that I liked to have on windows(system restore points), but I never missed on linux, even if it would be nice to have it. I broke linux at the beginning a lot.(Because I liked experimenting). But since I'm working on a linux machine, I know what better not to do and I never had the need to reinstall the system due to a broken os.
Recommendation for the lazy: If you want to experiment: Use a virtual machine. For your workstation: Use a proper backup strategy and get to know how to validate and restore it. Backup is important!
And in opposition to windows, if it would really be necessary, it's a piece of cake to take any computer install linux on it and get your backup onto it and have every setting restored. You just do not have to endlessly reboot and klick and update.
Hi All,
How worried is everyone about the Dirty Cow Linux exploit? Has anybody experienced attacks yet?
From the research I've done it seems that the exploit is "reliable" (that is it works nearly every time on vulverable systems) which is not good news.
We all believe that Unix/Linux... (3 Replies)
Hi
I'm trying to compile my linux kernel with CONFIG_SECURITY_CAPABILITIES=y.
any idea what this thing does ??
Also another question , If I compile the kernel that I'm currently using , what'll happen ?
~cheers (3 Replies)
So I'm having a problem getting a Broadcom BCM4312 wireless controller to work under the broadcom-wl module
$uname
Linux 3.8.11-200.fc18.x86_64 #1 SMP Wed May 1 19:44:27 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
lspci -v
05:00.0 Network controller: Broadcom Corporation BCM4312 802.11b/g... (2 Replies)
I wonder if anyone could assist with some problems I'm having with Linux Capabilities and their use when using the commands "nice" and "schedtool".
I run a couple of PCs, one is an elderly AMD Sempron 2800+ (32-bit, 2GHz clock and 3GB memory) that is used as a family multimedia system running... (3 Replies)
I've got a Solaris 11 Express installed on my machine. I have created a raidz2 zpool named shares and a simple one-disc zpool named backup. I have made a script that would send a daily snapshot of shares to backup.
I use these commands
zfs snapshot shares@DDMMRRRRHHMM
zfs send -i shares@....... (10 Replies)
I'm looking for a means to ensure that servers in the two or three datacenters, connected in a ring via IP through two ISPs, can distribute load and/or replicate data among at least two SAN-class disk devices.
I want to evaluate several solutions, and I'm open to solutions ranging from free,... (6 Replies)
Hi.
I downloaded a package that could only be installed on RHEL5, and not 4 or 3, so I got the source in order to compile it on RHEL 3 so hopefully it will work on all versions.
So I have the source for a working package, but when I build it in RHEL 3 and then try to install it in RHEL 5, it... (6 Replies)
Hi there.
I've been tasked with making a new design for our Unix systems :eek:
Now the question I have is;
How many LPARs can a p570 hold WITHOUT using a VIO Server.
Many Thanks
Kees (1 Reply)
We are looking into buying a new software, billing software that is, and want to know if you can run that on the same UNIX server as another major software?
Is there a limit to the different types of software Unix can run, or is it like windows where you can install as many as you like?
... (2 Replies)