Filesystems, Disks and Memory

Gidday!

I'd like to setup a storage server for a friend of mine (he is a hobby photographer, and he produces about 100Gb pictures monthly). My friend has the following PC-Server-like system:
Currently, the SATA connectors are occupied as follow: 1 is used by the DVD writer, 4 by SATA HD and 1 is free. 2 SATA HDs are connected to the Adaptec RAID controller.

Using this system, we'd like to create a Linux based storage server. Basically,
the disk arrays should be used to hold the OS and the regular data (pictures). To face possible HD failures or unwanted deletions, some scripts shall backup periodically the important pictures to an external RAID-5 NAS.

The requirements for the storage solution are:
I initially thought of creating a logical volume with LVM containing the 6 drives, set-up an ext3 file system than spans the whole volume and finally mount the volume to a well known location (e.g. /srv/data). I am afraid however that with this scheme, a single HD failure causes the entire storage to fail.

What would you recommend, given the above PC configuration and requirements for the storage? Buying additional Hardware (e.g. RAID controller) could eventually be an option.

Any ideas, pointers or links are welcome.

Thanks in advance,
Loïc.

My approach would be to partition the disks in similar sized partitions, and create RAID1/RAID5 across them, and then use LVM to collect them as 1 device. Example with 4 disks:

Code:

+----------+
| 1TB      | Disk 1, 1 TB
+----------+
+----------+-----+
| 1TB      |.5TB | Disk 2, 1.5 TB
+----------+-----+
+----------+-----+
| 1TB      |.5TB | Disk 3, 1.5 TB
+----------+-----+
+----------+-----+-----+
| 1TB      |.5TB |.5TB | Disk 4, 2 TB
+----------+-----+-----+

The 4 partitions sized 1 TB would be made into 1 software RAID5 with a total usable size of about ~1.3 TB. The 500GB partitions would be assembled into a RAID5 with about 1 TB total usable space. With an LVM across both RAIDs you'd be looking at ~1.8TB total space that's protected from disk failure. The left-over 500 GB could be used for temporary space, added to the LVM as snapshot space, or to hold the OS.

Hi Pludi,

thanks for your answer.

Is there a way to go along with LVM only? Basically, if one disk fails I'd need to be able to reconstruct the volume group (after replacing the failed drive). If such a thing possible with LVM, or would you recommend against it?

TIA,
Loïc.

No, it's not possible with LVM alone. LVM is designed to simplify the management of multiple, different devices by grouping them together. Bonus is a slight speed improvement. If you loose one drive with LVM, the data on it is gone for good too, but it's easy to extend the size.

RAID, on the other hand, is designed not to simplify drive management, but to ensure data reliability. It's not trivial to manually represent multiple RAIDs as one storage, but if one device fails you'll have no data loss.

how about opensolaris with ZFS? there are many tutorials on ZFS and how to build a home NAS system...

Good Evening,

Since I am not very fluent in LVM, I set-up a virtual KVM/Qemu guest with Linux in order to play with LVM and investigate possible failures scenario. Enclosed the results of my experiments.

I have the following setup: a volume group containing the following physical volume /dev/vda6, /dev/vdb2 and /dev/vdb3. I have only one logical volume than spans the entire volume group. ext3 is used as filesystem.

Code:

+--------------------------------------------------+
|                    ext3                          |
+--------------------------------------------------+
+--------------------------------------------------+
|                    mylv                          | logical volume = 100%VG
+--------------------------------------------------+
+--------------------------------------------------+
|                    myvg                          | volume group = vda6, vdb2, vdb3 
+--------------------------------------------------+
+-------------+     +-------------+    +-----------+
| /dev/vda6   |     | /dev/vdb2   |    | /dev/vdb3 |
+-------------+     +------------+    +-----------+

It is possible to save the current volume group meta information using vgcfgbackup.
I failed /dev/vdb3, /dev/vdb2 and /dev/vda6 respectively (zeroed the partition using dd). For vdb2 and vdb3, I could restore the ext3 filesystem as follows:
- recreate the physical volume of the failed partition, giving the right uuid label.
- restore the volume group meta information using vgcfgrestore
- repairing the ext3 filesystem using fsck.
As expected, only the files from the failed partition were missing after the restore operation.

However, I failed to restore the file system if /dev/vda6 gets damaged. I used an alternate superblock for fsck (one located on vdb2 or vdb3), but no avail. I lost information about the data stored on vdb2 and vdb3 (they can be found in lost+found, but name is lost). I didn't managed so far to recover the file system if the first disk (i.e. where the primary superblock is) failed. I still need to investigate what's wrong.

What do you think about these recovery possibilities? No point however that RAID+LVM looks safer.

That could be definitively worse a try. But I'll first investigate on the system where I am most knowledgeable about.

Which is, usually, something you do not want to happen. Personally, I'd rather have a system that's a bit slower if a disk goes bad, but keeps on working, than losing files and not being able to use the system until I've got a replacement disk. But that might just be me.
That might be because of the striping done by LVM. The volume manager doesn't fill up the first disk, then the second, and so on, but first fills the first stripe (by default 4 MB) on the first disk, the first stripe on the second disk, and so on. With that scheme it might very well be that all superblocks end up on the same disk.