Sponsored Content
Top Forums UNIX for Beginners Questions & Answers Should I use a CoW filesystem on my PC if I only wanted snapshot capabilities ? Post 303044720 by sreyan32 on Monday 2nd of March 2020 12:59:13 PM
Old 03-02-2020
Quote:
Originally Posted by Neo
No.

I do not recommend those file systems.

Your are better off running ext4, a RAID configuration (I run RAID1, but do not depend on it), and doing regular backups on your data based on your risk management model (this is the most critical).

Nothing beats a strong filesystem and a very well thought out backup and recovery plan.

That is my view. YMMV

On the desktop, I run macOS and have a similar strategy. I make full backups often, based on the activity on the system. The more activity and files (and the nature of the files) created, the more frequent the backups.
Ok, so can you tell me something about the Timeshift program. Lets say I make certain changes to the root filesystem which makes my system unbootable. Or I install a malicious or problematic update. Then will a program like TimeShift help ?

My question is that what do I do if the system becomes unbootable, how do I recover then ?

What do you use ?
What is your strategy ?

I hope its not taking a cold boot backup with CloneZilla.

--- Post updated at 12:14 AM ---

Quote:
Originally Posted by Peasant
If you are using LVM, you can use logical volume snapshot ability to achieve want you want.

BRTFS i have not used, but i did read some horror stories some time ago.
Probably nothing to worry about for home use, since those bugs were about raid protection.

ZFS in ubuntu, for instance, is openzfs (OpenZFS)
This is a mature and high quality file system & volume manager, but i would not use it for root just yet.

For data disks, i see no reason to reap the benefits of snapshots, compression and deduplication.
Just be sure those dedup tables fit in memory Smilie

Here is a quick example of LVM snapshots from my home box which is filesystem agnostic :
Code:
root@box:~# vgdisplay  dumpvg
  --- Volume group ---
  VG Name               dumpvg
  System ID             
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  2
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                1
  Open LV               1
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               <1.82 TiB
  PE Size               4.00 MiB
  Total PE              476931
  Alloc PE / Size       262144 / 1.00 TiB
  Free  PE / Size       214787 / 839.01 GiB
  VG UUID               Qtrbm7-GEAz-CcgA-JOUV-6eZ1-qrZ6-L4ey3h
root@box:~# mount | grep dumpvg
/dev/mapper/dumpvg-dumpvol on /srv/dump type xfs (rw,noatime,attr2,inode64,noquota)
cd /srv/dump/some_files
root@box:/srv/dump/some_files# ls -rlt
total 4
-rw-r--r-- 1 root root 19 Feb 28 15:49 file1.txt
root@box:/srv/dump/some_files# cat file1.txt 
Some stuff written
root@box:/srv/dump/some_files#

You need to have free space in volume group to create a snapshot, as i do, so lets create a snapshot.
We also specify that 5GB of total space in VG that can be consumed by system to maintain snapshot.
Code:
root@box:/srv/dump/some_files# lvcreate -L 5G -s -n lv_snap_$(date "+%Y%m%d%H%M") /dev/dumpvg/dumpvol
  Logical volume "lv_snap_202002281552" created.
root@box:/srv/dump/some_files# rm -f file1.txt
root@box:/srv/dump/some_files# mount -o ro,nouuid /dev/dumpvg/lv_snap_202002281552 /srv/dump_snap/
root@box:/srv/dump/some_files# ls -lrt /srv/dump_snap/some_files/
total 4
-rw-r--r-- 1 root root 19 Feb 28 15:49 file1.txt

Hope that helps
Regards
Peasant.
Can this work if the system becomes unbootable ? If so how ? Grub doesn't have the lvm tools for restoring snapsots, AFAIK.

Also what if the drive is encrypted ? I will be going with full disk encryption.

How do you do this if the system becomes unbootable ?

--- Post updated at 12:19 AM ---

Quote:
Originally Posted by drl
Hi.

I am a big fan of Virtual Machine technology. Here is what I do:

On my main workstation I install a small, stripped-down (i.e. no Office, etc.) Linux distribution as a host -- I prefer Debian GNU/Linux. I then install Virtualbox and create a VM. On the VM I install my day-to-day work environment -- again Debian.

Whenever I have mods to install, I use VB to take a snapshot. Then I install the mods. I leave the snapshot for some time (it's a CoW). If it works for a few days, a week, etc., then I merge the collected CoW changes into the VM (by, ironically, deleting the snapshot). If the mods fail to run, I restore the running system. I've had to do the restore perhaps 3 times in years, and it goes quite quickly, as does the creating and merging of the CoW.

Finally if the VM seems OK, then I install the mods into the host system.

For a plan for backup, we use a separate computer as a backup server, and on that we have a set of mirrored disks. We use that to run rsnapshot to backup our running, day-to-day systems. ( You might be able to run rsnapshot on the running system itself, in which case you can then use LVM, and rsnapshot will do its own LVM snapshot, do the backup, and remove its snapshot; then you could, say, tar up the resulting backup and send it to another computer ).

We have a small shop and rsnapshot helps us in many ways. The rsnapshot utility is a pull system, so the remote needs passphrase-less access to the system being backed up. The big advantage of rsnapshot is that it uses hard links, so conserving storage dramatically. For example, I backup my day-to-day system every hour, day, week, month, so 24+7+4+12 -> 47 collections, yet rarely goes over 20 GB, but oddly, if you look at any single collection, it is 20 GB, all due to the magic of hard links. The code also uses an algorithm that it transfers only the changes of a particular file, thus saving real time and network time. It also handles all renaming, copying, and removing of necessary files to accomplish the rotation of backup collection names.

We are also interested in the zfs filesystem. Our rsnapshot backup server was replaced this month with a newer model -- the old one lasted 17 years (2005..2020). In addition to being an external backup, I also installed VBox there and, as VMs, installed as guests Ubuntu 19.10 on zfs, as well as FreeBSD 12 on zfs. Since the host is on a RAID1 mirror, I didn't need the additional support for zfs mirroring (but it might be of some interest later on to experiment with them). We've had a Solaris VM for a long time on zfs:
Code:
OS, ker|rel, machine: SunOS, 5.11, i86pc
Distribution        : Solaris 11.3 X86

As additional VMs, we also installed a guest that is the same as the host, and we're using that as a test bed (just as I do with my day-to-day workstation). This new install is of distribution Debian GNU/Linux buster. We also like to have the next rev available, so I installed the testing version, known as bullseye.

So VMs are what we use for experimentation, as well as to make backups, like MS System Restore Points, easy.

Best wishes ... cheers, drl
Thats interesting, but I really need restoration capabilities on my base system.

Could you tell me how rsnapshot will help in a case where lets say the system is unbootable ? How do you restore then using rsnapshot ? Does it work when the system is unbootable ?

--- Post updated at 12:23 AM ---

Quote:
Originally Posted by stomp
Hi,

some personal experiences and experiences from others:
  • zfs
    I'm very fond of the ease of use of zfs administration. Few simple commands which all did what I expected them to do. Compression is recommended. Data Checksumming is also a great feature. Deduplication is not as clearly recommended. It needs lots of resources(RAM). It's not that flexible as LVM or btrfs, but in a Desktop-Environment, this should not be a problem. No Problems in several years of usage. Some reading about zfs is recommended for basic configuration (ashift, blocksize values.) getting root fs to zfs is manual work. But for a data partition it's very easy to use. If I need more flexibility, I will use lvm.
  • btrfs
    The times when btrfs had grave bugs are long gone. If you use it, make sure you do not use features that are marked as experimental. (For example the btrfs internal raid5 implementation. Use linux software raid with btrfs, if you want raid). I read some entries in another forum from someone who uses it at scale at will never change it for any other filesystem(he had experieance with all major filesystems) . It also has checksums and snapshot capabilities. The flexibility far exceeds zfs and lvm.

    I checked it out and decided not to use it, for these reasons:
    1. Some things are more complicated. You have to work your way through the documentation quite carefully. For example. You can not trust the values of du and df. The complexity of the filesystems circumvents that this is always correct and consistent. btrfs introduces own tools in addition to the tools everybody is used to.
    2. Things do not work the way I like them to: If I have some raid and a disk is failing, I would expect the system to come up and maybe complain about that it is in degraded mode or just have some file, which would be my part to check. And if the disk is replaced, I expect an easy command or even automatic restoration of the failed disk. That seems not to be the case with btrfs. If you then use your file system in degraded mode, bad things can happen and your raid will do things like writing data only to one disk despite the fact that the device may be set up as a mirror. Those data may be kept non-replicated even if you replace and reintegrate a new disk afterwards. That of course may lead to data loss. That's not an error of the filesystem, but the ordinary procedure. One has to know correctly how btrfs is to be operated according to the documentation or you'll get into serious trouble.
    I'd say btrfs is an advanced filesystem, which get's you a lot in terms of flexibility, performance, robustness, features and data security. It comes with the cost of taking your time and really get to learn the details, which can become really important.

On the other hand filesystem snapshotting ist not what you get with windows system restore points. I think that is a lot of voodoo going on there with windows system restore points(meaning it is complex under the hood). I did not ever test if filesystem-snapshots really get you a working system if you roll them back. Maybe it just works. But if you have a database, there's no guarantee, that the data will be consistent with such a snapshot.

I worked on my personal workstation within a VM too. Snapshots were possible. But the user experience was a mess. Regular Problems with the virtualization environment(virtualbox) and speed too drove me away from that and back to bare metal. For testing things virtual machines are great. Snapshotting ist great there. But not for the main workstation(for me).

Personally OS state snapshotting it's a feature, that I liked to have on windows(system restore points), but I never missed on linux, even if it would be nice to have it. I broke linux at the beginning a lot.(Because I liked experimenting). But since I'm working on a linux machine, I know what better not to do and I never had the need to reinstall the system due to a broken os.

Recommendation for the lazy: If you want to experiment: Use a virtual machine. For your workstation: Use a proper backup strategy and get to know how to validate and restore it. Backup is important!

And in opposition to windows, if it would really be necessary, it's a piece of cake to take any computer install linux on it and get your backup onto it and have every setting restored. You just do not have to endlessly reboot and klick and update.
What would be that proper strategy ? CloneZilla ? Or LVM snapshotting ?

--- Post updated at 12:27 AM ---

Quote:
Originally Posted by Peasant
Other the licencing issues, ZFS on Linux is production quality.

I'm no zealot but name me one file system on Linux (other then BRTFS) which has all those features so to say inline :
  • Builtin in bit rot protection with raid levels integrated or non-protected with copies feature.
  • Snapshots ( accessible via one cd command) / clones / compression / dedup / encryption one command away - granularity per filesystem / transparent to user.
  • Incremental send - receive, local or remote, protocol agnostic (STDIN/STDOUT).
  • Quotas, reservations - one command away.
  • Not using standard (not to say obsolete) LRU for caching, but ARC.
  • Ability to increase performance with dedicated devices for L2ARC/ZIL.
  • Sharing volumes/files using NFS, CIFS (NAS) or ISCSI (block) - couple of commands away.
  • Compatibility between systems running Opensolaris & BSD derivatives, endianess aware.
Those are features offered by enterprise array systems for big price tag and additional licences.

Of course, much can be achieved with various other tools (LVM snaps, rsync tools, hard links, xfsdump, dump etc.)

In ZFS all is builtin, one file system to rule them all Smilie

Image

Regards
Peasant.
Interesting analogy. But doesn't zfs use more RAM than btrfs ? So isn't btrfs "the better ZFS" for desktops ?

Also this snapshot thing is really confusing me ? Restore incase of unbootable system with encrypted HDD ?

--- Post updated at 12:29 AM ---

Quote:
Originally Posted by jim mcnamara
FWIW:
Don't Use ZFS on Linux: Linus Torvalds - It's FOSS

Torvalds will not allow Linux kernel inclusion of ZFS support because of Oracle's position on ZFS licensure, this was important to us because we have only ZFS on Solaris 11/12 boxes. We did not want different files systems for production Linux servers - but that is what we got....ext4

How this plays out on a home desktop I cannot say exactly. I would recommend NOT using ZFS for Linux boot filesystems - as @neo said.
Is the licensing the only problem for Torvalds ?
 

9 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Unix Capabilities?

We are looking into buying a new software, billing software that is, and want to know if you can run that on the same UNIX server as another major software? Is there a limit to the different types of software Unix can run, or is it like windows where you can install as many as you like? ... (2 Replies)
Discussion started by: hoz
2 Replies

2. UNIX for Advanced & Expert Users

p570 Capabilities

Hi there. I've been tasked with making a new design for our Unix systems :eek: Now the question I have is; How many LPARs can a p570 hold WITHOUT using a VIO Server. Many Thanks Kees (1 Reply)
Discussion started by: KeesH
1 Replies

3. Red Hat

Adding capabilities to an RPM

Hi. I downloaded a package that could only be installed on RHEL5, and not 4 or 3, so I got the source in order to compile it on RHEL 3 so hopefully it will work on all versions. So I have the source for a working package, but when I build it in RHEL 3 and then try to install it in RHEL 5, it... (6 Replies)
Discussion started by: Boaz
6 Replies

4. Filesystems, Disks and Memory

Wanted: Geographically distributed filesystem solution

I'm looking for a means to ensure that servers in the two or three datacenters, connected in a ring via IP through two ISPs, can distribute load and/or replicate data among at least two SAN-class disk devices. I want to evaluate several solutions, and I'm open to solutions ranging from free,... (6 Replies)
Discussion started by: otheus
6 Replies

5. Solaris

Cannot use filesystem while sending a snapshot

I've got a Solaris 11 Express installed on my machine. I have created a raidz2 zpool named shares and a simple one-disc zpool named backup. I have made a script that would send a daily snapshot of shares to backup. I use these commands zfs snapshot shares@DDMMRRRRHHMM zfs send -i shares@....... (10 Replies)
Discussion started by: RychnD
10 Replies

6. UNIX for Advanced & Expert Users

Use of Capabilities

I wonder if anyone could assist with some problems I'm having with Linux Capabilities and their use when using the commands "nice" and "schedtool". I run a couple of PCs, one is an elderly AMD Sempron 2800+ (32-bit, 2GHz clock and 3GB memory) that is used as a family multimedia system running... (3 Replies)
Discussion started by: MikeGM
3 Replies

7. Linux

Broadcom under Fedora 18 (Spherical Cow)

So I'm having a problem getting a Broadcom BCM4312 wireless controller to work under the broadcom-wl module $uname Linux 3.8.11-200.fc18.x86_64 #1 SMP Wed May 1 19:44:27 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux lspci -v 05:00.0 Network controller: Broadcom Corporation BCM4312 802.11b/g... (2 Replies)
Discussion started by: Skrynesaver
2 Replies

8. UNIX for Advanced & Expert Users

Linux capabilities discussion

Hi I'm trying to compile my linux kernel with CONFIG_SECURITY_CAPABILITIES=y. any idea what this thing does ?? Also another question , If I compile the kernel that I'm currently using , what'll happen ? ~cheers (3 Replies)
Discussion started by: leghorn
3 Replies

9. What is on Your Mind?

Anybody want to talk about Dirty Cow?

Hi All, How worried is everyone about the Dirty Cow Linux exploit? Has anybody experienced attacks yet? From the research I've done it seems that the exploit is "reliable" (that is it works nearly every time on vulverable systems) which is not good news. We all believe that Unix/Linux... (3 Replies)
Discussion started by: hicksd8
3 Replies
All times are GMT -4. The time now is 03:16 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy