Linux Storage system: looking for advices Post: 302384126

Sponsored Content

Special Forums Hardware Filesystems, Disks and Memory Linux Storage system: looking for advices Post 302384126 by Loic Domaigne on Monday 4th of January 2010 05:52:11 AM

01-04-2010

Registered User

Linux Storage system: looking for advices

Gidday!

I'd like to setup a storage server for a friend of mine (he is a hobby photographer, and he produces about 100Gb pictures monthly). My friend has the following PC-Server-like system:

AMD Athlon Dual Core Processor 4850e.
ASUS M3N78-EMH HDMI motherboard with 6 SATA connectors.
3Gb RAM.
6 SATA HD of different capacity (ranging from 1.2 - 2 Tb) and manufacturers.
1 Adaptec RAID Controller 1220SA able to create RAID 0, 1, JBOD array.

Currently, the SATA connectors are occupied as follow: 1 is used by the DVD writer, 4 by SATA HD and 1 is free. 2 SATA HDs are connected to the Adaptec RAID controller.

Using this system, we'd like to create a Linux based storage server. Basically,
the disk arrays should be used to hold the OS and the regular data (pictures). To face possible HD failures or unwanted deletions, some scripts shall backup periodically the important pictures to an external RAID-5 NAS.

The requirements for the storage solution are:

to get as much storage capacity as possible,
in case of a disk crash, it is acceptable to loose the data on the disk (the important data are backup on the NAS), BUT it is not acceptable to loose the entire storage.
the storage should be flexible, i.e. allow failed drives to be replaced or allow to replace HDs with bigger HDs if required.
the storage should be easy to use. Ideally, the 6 drives should be exposed as 1 virtual drive to outside.
the storage should accessible from a MacBook (using for instance, NFS). As a matter of fact, my friend work-out his picture from his MAC and the storage should be integrated as seamlessly as possible in the "MAC world".

I initially thought of creating a logical volume with LVM containing the 6 drives, set-up an ext3 file system than spans the whole volume and finally mount the volume to a well known location (e.g. /srv/data). I am afraid however that with this scheme, a single HD failure causes the entire storage to fail.

What would you recommend, given the above PC configuration and requirements for the storage? Buying additional Hardware (e.g. RAID controller) could eventually be an option.

Any ideas, pointers or links are welcome.

Thanks in advance,
Lo�c.

Loic Domaigne

View Public Profile for Loic Domaigne

Find all posts by Loic Domaigne

LEARN ABOUT CENTOS

amzfs-sendrecv

AMZFS-SENDRECV(8)					  System Administration Commands					 AMZFS-SENDRECV(8)

NAME

       amzfs-sendrecv - Amanda script to create zfs sendrecv

DESCRIPTION

       amzfs-sendrecv is an Amanda application implementing the Application API. It should not be run by users directly. It create a zfs snapshot
       of the filesystem and backup the snapshot with 'zfs send'. Snapshot are kept after the backup is done, this increase the disk space use on
       the client but it is neccesary to be able do to incremental backup. If you want only full backup, you can disable this feature by setting
       the KEEP-SNAPSHOT property to 'NO'. Only the restoration of the complete backup is allowed, it is impossible to restore a single file.

       The application is run as the amanda user, it must have many zfs priviledge:

	   zfs allow -ldu AMANDA_USER mount,create,rename,snapshot,destroy,send,receive FILESYSTEM

       Some system doesn't have "zfs allow", but you can give the Amanda backup user the rights to manipulate ZFS filesystems by using the
       following command:

       usermod -P "ZFS File System Management,ZFS Storage Management" AMANDA_USER
       This will require that your run zfs under pfexec, set the PFEXEC property to YES.

       The format of the diskdevice in the disklist (DLE) must be one of:

       Desciption	       Example
       ----------	       -------
       Mountpoint	       /data
       ZFS pool name	       datapool
       ZFS filesystem	       datapool/database
       ZFS logical volume      datapool/dbvol

       The filesystem doesn't need to be mounted.

PROPERTIES

       This section lists the properties that control amzfs-sendrecv's functionality. See amanda-applications(7) for information on the
       Application API, application configuration.

       DF-PATH

	   Path to the 'df' binary, search in $PATH by default.

       KEEP-SNAPSHOT

	   If "YES" (the default), snapshot are kept after the backup, if set to "NO" then snapshot are no kept and incremental backup will fail.

       ZFS-PATH

	   Path to the 'zfs' binary, search in $PATH by default.

       PFEXEC-PATH

	   Path to the 'pfexec' binary, search in $PATH by default.

       PFEXEC

	   If "NO" (the default), pfexec is not used, if set to "YES" then pfexec is used.

EXAMPLE

       In this example, a dumptype is defined to use amzfs-sendrecv application to backup a zfs filesystem.
	define application-tool amzfs_sendrecv {
	  comment "amzfs-sendrecv"
	  plugin "amzfs-sendrecv"
	  #property "DF-PATH"  "/usr/sbin/df"
	  #property "KEEP-SNAPSHOT" "YES"
	  #property "ZFS-PATH" "/usr/sbin/zfs"
	  #property "PFEXEC-PATH" "/usr/sbin/pfexec"
	  #property "PFEXEC" "NO"
	}

	define dumptype user-zfs-sendrecv {
	  program "APPLICATION"
	  application "amzfs_sendrecv"
	}

SEE ALSO

       amanda(8), amanda.conf(5), amanda-client.conf(5), amanda-applications(7)

       The Amanda Wiki: : http://wiki.zmanda.com/

AUTHOR

       Jean-Louis Martineau <martineau@zmanda.com>
	   Zmanda, Inc. (http://www.zmanda.com)

Amanda 3.3.3							    01/10/2013							 AMZFS-SENDRECV(8)