Visit Our UNIX and Linux User Community


Should I use a CoW filesystem on my PC if I only wanted snapshot capabilities ?


 
Thread Tools Search this Thread
Top Forums UNIX for Beginners Questions & Answers Should I use a CoW filesystem on my PC if I only wanted snapshot capabilities ?
# 1  
Old 02-28-2020
Should I use a CoW filesystem on my PC if I only wanted snapshot capabilities ?

I will be installing Linux on my HP Laptop and I really like ext4, its stable and time tested. But I want snapshot capabilities, or something like system restore in Windows. This is obviously for times when I shoot myself in the foot and want to restore back to a stable state.

Will filesystems like ZFS or btrfs work better in these cases rather than ext4 ?

My only requirement is snapshots, so is it worth running btrfs or ZFS despite its unstabilty (more for btrfs rather than ZFS).
# 2  
Old 02-28-2020
No.

I do not recommend those file systems.

Your are better off running ext4, a RAID configuration (I run RAID1, but do not depend on it), and doing regular backups on your data based on your risk management model (this is the most critical).

Nothing beats a strong filesystem and a very well thought out backup and recovery plan.

That is my view. YMMV

On the desktop, I run macOS and have a similar strategy. I make full backups often, based on the activity on the system. The more activity and files (and the nature of the files) created, the more frequent the backups.
# 3  
Old 02-28-2020
If you are using LVM, you can use logical volume snapshot ability to achieve want you want.

BRTFS i have not used, but i did read some horror stories some time ago.
Probably nothing to worry about for home use, since those bugs were about raid protection.

ZFS in ubuntu, for instance, is openzfs (OpenZFS)
This is a mature and high quality file system & volume manager, but i would not use it for root just yet.

For data disks, i see no reason to reap the benefits of snapshots, compression and deduplication.
Just be sure those dedup tables fit in memory Smilie

Here is a quick example of LVM snapshots from my home box which is filesystem agnostic :
Code:
root@box:~# vgdisplay  dumpvg
  --- Volume group ---
  VG Name               dumpvg
  System ID             
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  2
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                1
  Open LV               1
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               <1.82 TiB
  PE Size               4.00 MiB
  Total PE              476931
  Alloc PE / Size       262144 / 1.00 TiB
  Free  PE / Size       214787 / 839.01 GiB
  VG UUID               Qtrbm7-GEAz-CcgA-JOUV-6eZ1-qrZ6-L4ey3h
root@box:~# mount | grep dumpvg
/dev/mapper/dumpvg-dumpvol on /srv/dump type xfs (rw,noatime,attr2,inode64,noquota)
cd /srv/dump/some_files
root@box:/srv/dump/some_files# ls -rlt
total 4
-rw-r--r-- 1 root root 19 Feb 28 15:49 file1.txt
root@box:/srv/dump/some_files# cat file1.txt 
Some stuff written
root@box:/srv/dump/some_files#

You need to have free space in volume group to create a snapshot, as i do, so lets create a snapshot.
We also specify that 5GB of total space in VG that can be consumed by system to maintain snapshot.
Code:
root@box:/srv/dump/some_files# lvcreate -L 5G -s -n lv_snap_$(date "+%Y%m%d%H%M") /dev/dumpvg/dumpvol
  Logical volume "lv_snap_202002281552" created.
root@box:/srv/dump/some_files# rm -f file1.txt
root@box:/srv/dump/some_files# mount -o ro,nouuid /dev/dumpvg/lv_snap_202002281552 /srv/dump_snap/
root@box:/srv/dump/some_files# ls -lrt /srv/dump_snap/some_files/
total 4
-rw-r--r-- 1 root root 19 Feb 28 15:49 file1.txt

Hope that helps
Regards
Peasant.
These 2 Users Gave Thanks to Peasant For This Post:
# 4  
Old 02-28-2020
FWIW:
Don't Use ZFS on Linux: Linus Torvalds - It's FOSS

Torvalds will not allow Linux kernel inclusion of ZFS support because of Oracle's position on ZFS licensure, this was important to us because we have only ZFS on Solaris 11/12 boxes. We did not want different files systems for production Linux servers - but that is what we got....ext4

How this plays out on a home desktop I cannot say exactly. I would recommend NOT using ZFS for Linux boot filesystems - as @neo said.
This User Gave Thanks to jim mcnamara For This Post:
# 5  
Old 02-28-2020
Hi.
Quote:
Originally Posted by sreyan32
... But I want snapshot capabilities, or something like system restore in Windows. This is obviously for times when I shoot myself in the foot and want to restore back to a stable state ...
I am a big fan of Virtual Machine technology. Here is what I do:

On my main workstation I install a small, stripped-down (i.e. no Office, etc.) Linux distribution as a host -- I prefer Debian GNU/Linux. I then install Virtualbox and create a VM. On the VM I install my day-to-day work environment -- again Debian.

Whenever I have mods to install, I use VB to take a snapshot. Then I install the mods. I leave the snapshot for some time (it's a CoW). If it works for a few days, a week, etc., then I merge the collected CoW changes into the VM (by, ironically, deleting the snapshot). If the mods fail to run, I restore the running system. I've had to do the restore perhaps 3 times in years, and it goes quite quickly, as does the creating and merging of the CoW.

Finally if the VM seems OK, then I install the mods into the host system.

For a plan for backup, we use a separate computer as a backup server, and on that we have a set of mirrored disks. We use that to run rsnapshot to backup our running, day-to-day systems. ( You might be able to run rsnapshot on the running system itself, in which case you can then use LVM, and rsnapshot will do its own LVM snapshot, do the backup, and remove its snapshot; then you could, say, tar up the resulting backup and send it to another computer ).

We have a small shop and rsnapshot helps us in many ways. The rsnapshot utility is a pull system, so the remote needs passphrase-less access to the system being backed up. The big advantage of rsnapshot is that it uses hard links, so conserving storage dramatically. For example, I backup my day-to-day system every hour, day, week, month, so 24+7+4+12 -> 47 collections, yet rarely goes over 20 GB, but oddly, if you look at any single collection, it is 20 GB, all due to the magic of hard links. The code also uses an algorithm that it transfers only the changes of a particular file, thus saving real time and network time. It also handles all renaming, copying, and removing of necessary files to accomplish the rotation of backup collection names.

We are also interested in the zfs filesystem. Our rsnapshot backup server was replaced this month with a newer model -- the old one lasted 17 years (2005..2020). In addition to being an external backup, I also installed VBox there and, as VMs, installed as guests Ubuntu 19.10 on zfs, as well as FreeBSD 12 on zfs. Since the host is on a RAID1 mirror, I didn't need the additional support for zfs mirroring (but it might be of some interest later on to experiment with them). We've had a Solaris VM for a long time on zfs:
Code:
OS, ker|rel, machine: SunOS, 5.11, i86pc
Distribution        : Solaris 11.3 X86

As additional VMs, we also installed a guest that is the same as the host, and we're using that as a test bed (just as I do with my day-to-day workstation). This new install is of distribution Debian GNU/Linux buster. We also like to have the next rev available, so I installed the testing version, known as bullseye.

So VMs are what we use for experimentation, as well as to make backups, like MS System Restore Points, easy.

Best wishes ... cheers, drl
These 2 Users Gave Thanks to drl For This Post:
# 6  
Old 02-28-2020
Thanks drl,

Honestly, I have tried a similar approach on macOS, but even with a 12-core machine with 64 GB of memory on my desktop, all flavors of VMs slow the machine down to an intolerable degree. After all, running in a VM, by definition, is slower than running "on bare metal".

My guess is that your desktop VM configuration on Linux runs a lot better, since you are a bit fan.

Because I prefer the fastest speed possible (we are talking desktop operations) I find that using the built in macOS time machine to keep data based up works great; so in this configuration, I keep my machine running the fastest possible (running "bare metal" not in a VM) and at the same time my data is always backed up an external disk.

Maybe we mac users are lucky because time machine works so well and is very easy to use for users of all skill levels?

From the big wiki in the sky:

Quote:
Time Machine creates incremental backups of files that can be restored at a later date.[1] It allows the user to restore the whole system or specific files from the Recovery HD or the macOS Install DVD. It works within Mail, iWork, iLife, and several other compatible programs, making it possible to restore individual objects (e.g. emails, photos, contacts, calendar events) without leaving the application. According to an Apple support statement:

“Time Machine is a backup utility, not an archival utility, it is not intended as offline storage. Time Machine captures the most recent state of your data on your disk. As snapshots age, they are prioritized progressively lower compared to your more recent ones.”¯[2]

For backups to a network drive, Time Machine allows the user to back up Mac computers over the network, and supports backing up to certain network attached storage devices or servers, depending on the version of Time Machine. Earlier versions worked with a wide variety of NAS servers, but later versions require the server to support a recent version of Apple's Apple Filing Protocol (AFP), and Time Machine no longer works with servers using earlier versions of the Server Message Block (SMB) protocol[3] typical for Windows servers. Some of the legacy support can be re-enabled by using hand-tuned configuration options, accessed through the Terminal. Apple's Time Capsule acts as a network storage device specifically for Time Machine backups, allowing both wired and wireless backups to the Time Capsule's internal hard drive. Time Machine may also be used with any external or internal volume.

Time Machine saves hourly backups for the past 24 hours, daily backups for the past month, and weekly backups for everything older than a month until the volume runs out of space. At that point, Time Machine deletes the oldest weekly backup.
REF: Time Machine (macOS - Wikipedia)

As I mentioned before, on my Linux servers in production, I run ext4 on all of them using RAID1 and make incremental and full backups of all mission critical data on a daily basis. Of course, remote servers are different compared desktops; and so that is why I prefer macOS and time machine on the desktop.

Naturally, everyone has different setups and preferences, so it is great to see people sharing their ideas openly and open-mindedly.

I do not recommend the filesystems the OP suggested (ZFS or btrfs) on Linux and as mentioned, and prefer a different backup strategy (but then again, I don't user Linux on the desktop, as mentioned, but If I used Linux on the desktop instead of macOS, I would still use ext4. This file system (ext) has served me well over the many years and I cannot recall a single problem related to the ext filesystem 27 years of Linux experience. I like "reliable" and "proven" in a filesystem... even if it is not the "most fancy SOTA out there.).
# 7  
Old 02-29-2020
Hi,

some personal experiences and experiences from others:
  • zfs
    I'm very fond of the ease of use of zfs administration. Few simple commands which all did what I expected them to do. Compression is recommended. Data Checksumming is also a great feature. Deduplication is not as clearly recommended. It needs lots of resources(RAM). It's not that flexible as LVM or btrfs, but in a Desktop-Environment, this should not be a problem. No Problems in several years of usage. Some reading about zfs is recommended for basic configuration (ashift, blocksize values.) getting root fs to zfs is manual work. But for a data partition it's very easy to use. If I need more flexibility, I will use lvm.
  • btrfs
    The times when btrfs had grave bugs are long gone. If you use it, make sure you do not use features that are marked as experimental. (For example the btrfs internal raid5 implementation. Use linux software raid with btrfs, if you want raid). I read some entries in another forum from someone who uses it at scale at will never change it for any other filesystem(he had experieance with all major filesystems) . It also has checksums and snapshot capabilities. The flexibility far exceeds zfs and lvm.

    I checked it out and decided not to use it, for these reasons:
    1. Some things are more complicated. You have to work your way through the documentation quite carefully. For example. You can not trust the values of du and df. The complexity of the filesystems circumvents that this is always correct and consistent. btrfs introduces own tools in addition to the tools everybody is used to.
    2. Things do not work the way I like them to: If I have some raid and a disk is failing, I would expect the system to come up and maybe complain about that it is in degraded mode or just have some file, which would be my part to check. And if the disk is replaced, I expect an easy command or even automatic restoration of the failed disk. That seems not to be the case with btrfs. If you then use your file system in degraded mode, bad things can happen and your raid will do things like writing data only to one disk despite the fact that the device may be set up as a mirror. Those data may be kept non-replicated even if you replace and reintegrate a new disk afterwards. That of course may lead to data loss. That's not an error of the filesystem, but the ordinary procedure. One has to know correctly how btrfs is to be operated according to the documentation or you'll get into serious trouble.
    I'd say btrfs is an advanced filesystem, which get's you a lot in terms of flexibility, performance, robustness, features and data security. It comes with the cost of taking your time and really get to learn the details, which can become really important.

On the other hand filesystem snapshotting ist not what you get with windows system restore points. I think that is a lot of voodoo going on there with windows system restore points(meaning it is complex under the hood). I did not ever test if filesystem-snapshots really get you a working system if you roll them back. Maybe it just works. But if you have a database, there's no guarantee, that the data will be consistent with such a snapshot.

I worked on my personal workstation within a VM too. Snapshots were possible. But the user experience was a mess. Regular Problems with the virtualization environment(virtualbox) and speed too drove me away from that and back to bare metal. For testing things virtual machines are great. Snapshotting ist great there. But not for the main workstation(for me).

Personally OS state snapshotting it's a feature, that I liked to have on windows(system restore points), but I never missed on linux, even if it would be nice to have it. I broke linux at the beginning a lot.(Because I liked experimenting). But since I'm working on a linux machine, I know what better not to do and I never had the need to reinstall the system due to a broken os.

Recommendation for the lazy: If you want to experiment: Use a virtual machine. For your workstation: Use a proper backup strategy and get to know how to validate and restore it. Backup is important!

And in opposition to windows, if it would really be necessary, it's a piece of cake to take any computer install linux on it and get your backup onto it and have every setting restored. You just do not have to endlessly reboot and klick and update.

Last edited by stomp; 03-02-2020 at 11:33 AM..
This User Gave Thanks to stomp For This Post:

Previous Thread | Next Thread
Test Your Knowledge in Computers #219
Difficulty: Easy
When a data packet comes in to a port on a router, the router reads the network address information in the packet footer to determine the ultimate destination.
True or False?

9 More Discussions You Might Find Interesting

1. What is on Your Mind?

Anybody want to talk about Dirty Cow?

Hi All, How worried is everyone about the Dirty Cow Linux exploit? Has anybody experienced attacks yet? From the research I've done it seems that the exploit is "reliable" (that is it works nearly every time on vulverable systems) which is not good news. We all believe that Unix/Linux... (3 Replies)
Discussion started by: hicksd8
3 Replies

2. UNIX for Advanced & Expert Users

Linux capabilities discussion

Hi I'm trying to compile my linux kernel with CONFIG_SECURITY_CAPABILITIES=y. any idea what this thing does ?? Also another question , If I compile the kernel that I'm currently using , what'll happen ? ~cheers (3 Replies)
Discussion started by: leghorn
3 Replies

3. Linux

Broadcom under Fedora 18 (Spherical Cow)

So I'm having a problem getting a Broadcom BCM4312 wireless controller to work under the broadcom-wl module $uname Linux 3.8.11-200.fc18.x86_64 #1 SMP Wed May 1 19:44:27 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux lspci -v 05:00.0 Network controller: Broadcom Corporation BCM4312 802.11b/g... (2 Replies)
Discussion started by: Skrynesaver
2 Replies

4. UNIX for Advanced & Expert Users

Use of Capabilities

I wonder if anyone could assist with some problems I'm having with Linux Capabilities and their use when using the commands "nice" and "schedtool". I run a couple of PCs, one is an elderly AMD Sempron 2800+ (32-bit, 2GHz clock and 3GB memory) that is used as a family multimedia system running... (3 Replies)
Discussion started by: MikeGM
3 Replies

5. Solaris

Cannot use filesystem while sending a snapshot

I've got a Solaris 11 Express installed on my machine. I have created a raidz2 zpool named shares and a simple one-disc zpool named backup. I have made a script that would send a daily snapshot of shares to backup. I use these commands zfs snapshot shares@DDMMRRRRHHMM zfs send -i shares@....... (10 Replies)
Discussion started by: RychnD
10 Replies

6. Filesystems, Disks and Memory

Wanted: Geographically distributed filesystem solution

I'm looking for a means to ensure that servers in the two or three datacenters, connected in a ring via IP through two ISPs, can distribute load and/or replicate data among at least two SAN-class disk devices. I want to evaluate several solutions, and I'm open to solutions ranging from free,... (6 Replies)
Discussion started by: otheus
6 Replies

7. Red Hat

Adding capabilities to an RPM

Hi. I downloaded a package that could only be installed on RHEL5, and not 4 or 3, so I got the source in order to compile it on RHEL 3 so hopefully it will work on all versions. So I have the source for a working package, but when I build it in RHEL 3 and then try to install it in RHEL 5, it... (6 Replies)
Discussion started by: Boaz
6 Replies

8. UNIX for Advanced & Expert Users

p570 Capabilities

Hi there. I've been tasked with making a new design for our Unix systems :eek: Now the question I have is; How many LPARs can a p570 hold WITHOUT using a VIO Server. Many Thanks Kees (1 Reply)
Discussion started by: KeesH
1 Replies

9. UNIX for Dummies Questions & Answers

Unix Capabilities?

We are looking into buying a new software, billing software that is, and want to know if you can run that on the same UNIX server as another major software? Is there a limit to the different types of software Unix can run, or is it like windows where you can install as many as you like? ... (2 Replies)
Discussion started by: hoz
2 Replies

Featured Tech Videos