Help finding a Unix friendly RAID 1 backup


 
Thread Tools Search this Thread
Special Forums Hardware Filesystems, Disks and Memory Help finding a Unix friendly RAID 1 backup
# 1  
Old 03-16-2011
Help finding a Unix friendly RAID 1 backup

First time poster and a very new Unix user, so I'll just pre-apologize for stupid questions now.

Does anybody know of a good RAID 1 hard drive backup that is Unix friendly? I want to avoid any hardcore programming. Can you recommend both NAS and non-NAS options? I need to do nightly backups from a Unix data server running SAMBA/SWAT that currently has ~300 of 420 GB used split between public and user folders. This is for an office and involves sensitive data so I need a safe and secure option.

This is what I was able to find online that seems to fit what I'm looking for:
Buffalo Technology TeraStation Duo TS-WX2.0TL/R1 2x1 TB 368.98

Synology DiskStation DS211 21002x1 TB550.99

Netgear ReadyNAS Duo 2-Bay RND2210 2x1 TB 393.6
Data Dock II DDQ-2000 2x1 TB 269.95
Do any of the above make sense? From what I can tell, only the Netgear is out of the box Unix friendly; the tech guys at Fantom couldn't tell me whether the data dock II was or not. Can you recommend any of these or other models? I don't really think I need the NAS option and it seems you pay considerably more for that. Should I be looking at an entirely different type of data storage? (Cloud storage is not an option)

In the meantime, while I figure this out, my boss wants me to backup the data asap. I was thinking about getting a consumer grade 500 GB or 1 TB external with an ethernet port and simply manually backing up the data via windows. I was thinking this would provide a good stop gap and, once the RAID 1 is setup, could simply be manually backed up weekly and provide essentially an additional disk to the RAID 1 array.

For this I was looking between these two:
Iomega Home Media 34337 1 TB 99.99
Buffalo LS-CH1.0TL 1 TB 99.99
Any help is greatly appreciated. Thank you.
# 2  
Old 03-16-2011
Define "UNIX-friendly". Which UNIX? Furthermore, what's your architecture and system? What kind of disks do you want to use?

A good stopgap a USB or ethernet drive would be, as any backup is better than no backup. However Windows has no respect for UNIX permissions so just blindly copying files could result in much hair-pull later. You could use the udpcast utility and do something like this:
Code:
# On UNIX
tar -cpf - /path/to/files/i/want/to/backup | udp-sender
# On Windows
udp-receiver > file.tar

..to just keep one giant tar which should preserve the permissions of the files inside it. To restore,
Code:
# in Windows
udp-sender < file.tar
# in UNIX
udp-receiver | tar -C /path/ -vxpf -

Of course, make sure the drive is formatted with NTFS or something which allows Windows to create >4GB files. FAT won't do.

And if you can have the UNIX system use the drive directly? All the better.

You might have troubles reusing the disk in a RAID once you want to make one, since you'd quite likely need to blank the contents before you make it part of an array.

If you want reliable hardware, I would suggest avoiding consumer-grade stuff. Especially avoid jmicron chipsets. 3ware works pretty closely with UNIX vendors.

Last edited by Corona688; 03-16-2011 at 12:35 PM..
This User Gave Thanks to Corona688 For This Post:
# 3  
Old 03-16-2011
To Corona688
By “UNIX-friendly” I meant that I am ideally looking for something that is out of the box compatible with Unix in order to minimize additional coding and therefore potential problems and headaches. From what I understand, in some cases, the internal cards that coordinate the RAID 1 aren't necessarily designed to work with a Unix system. I would like to avoid those.

To be honest, I don't know what type of Unix I am using, or what the architecture and system is, where would I find that information? (I just started this job and am essentially on my own technical wise; nobody even told me the server was on site in a closet for about a week and a half.) I do have access as the su if that can help me find this info.

What do you mean types of disks? Speed, size, company?

I think I get your point about copying from Unix to Windows. I basically need to format the drive first, and send the data over as one large file that can be restored later if needed, which would ensure individual users permissions?

Reliability is key, I will look into 3ware.

Thank you so much for your help and suggestions, I really appreciate it.
# 4  
Old 03-16-2011
Quote:
Originally Posted by c.wakeman
By “UNIX-friendly” I meant that I am ideally looking for something that is out of the box compatible with Unix in order to minimize additional coding and therefore potential problems and headaches.
Avoid software RAID then. We use that and it works decently well but took a lot of frustration to get going.

A hardware RAID on the other hand, can present multiple disks to the operating system like a single hard drive. Configuring which drives are part of an array often becomes an extended CMOS setting completely independent of the installed OS: you might get a 'press ESC to configure drives' message on boot before the OS actually loads. Assuming your server is a PC that is.
Quote:
From what I understand, in some cases, the internal cards that coordinate the RAID 1 aren’t necessarily designed to work with a Unix system. I would like to avoid those.
It's not so much that they're not designed to work with UNIX, as much as they may not have bothered writing UNIX device drivers. This may not be as important as it used to be (for PC hardware, anyway) since most disk controllers are AHCI-compliant these days and work fine with a generic driver.
Quote:
To be honest, I don’t know what type of Unix I am using, or what the architecture and system is, where would I find that information?
That's a pretty important question... "UNIX" is a completely generic term; Your wireless router might be running a kind of UNIX. So do most supercomputers. Obviously you can't run software from one on the other or fit hardware from one into the other. In a shell, try uname and uname -a to find out what your system is.

That alone won't tell you what kind of card slots this server has and which, if any, are free, so you might need to take a look at the hardware itself too.
Quote:
What do you mean types of disks? Speed, size, company?
Mostly, interface. SATA-II? SATA-III? SCSI? Probably you want SATA I think. SATA hotswap cages are cheap these days.
Quote:
I think I get your point about copying from Unix to Windows. I basically need to format the drive first, and send the data over as one large file that can be restored later if needed, which would ensure individual users permissions?
Exactly. The permissions, timestamps, users, and everything else all get bundled up along with the files when you make a tar with -p. Formatting it in NTFS is necessary for Windows if the drive came with FAT, because FAT can't hold files larger than 4 gigabytes.
Quote:
Reliability is key, I will look into 3ware.
Good deal. We tried to go cheap and tried a jmicron controller, which caused some (fortunately recoverable) data corruption. Never again.

---------- Post updated at 10:23 AM ---------- Previous update was at 10:08 AM ----------

Also: A RAID isn't exactly a backup. It's tempting since it's automatic and improves your speed too. It somewhat protects you from a single-disk failure -- that's all. (And single-disk failiures will occur more often because you're running more disks.) Any other kind of problem -- an out-of-control program, dying disk controller, murderous power supply, fire, lightning strike, utility company, theft, volcano -- are still quite capable of swallowing your data entire. A trustworthy backup is when you make a copy and mail it somewhere else.

Last edited by Corona688; 03-16-2011 at 01:19 PM..
This User Gave Thanks to Corona688 For This Post:
# 5  
Old 03-22-2011
Thanks again for your help so far Corona688.

An update so far:
I used the uname command, we're running Linux.

The decision was made to purchase a cheaper 1 TB external hard drive as a stop-gap measure to make sure the data is backed up before moving ahead with setting up an automatic backup to a dedicated raid 1 array. We purchased a Buffalo Linkstation Live LS-CHL.

The drive can be formatted to to FAT, NTFS, XFS, and HFS+. The server I want to backup is an NTFS file system. I believe the drive comes XFS standard. Should I format the drive to NTFS for compatibility? or is that a non-issue.

The user manual for the drive states the disadvantages of NTFS as:
1) Read-only from the LinkStation or a Mac.
2) Not suitable for backup from the LinkStation.

The relevant disadvantage of XFS is:
You cannot read data by directly connecting to a PC.

Once that is determined, what are the steps I need to take to make a backup .tar of the server files based upon the code you provided earlier?
Quote:
Code:
# On UNIX
tar -cpf - /path/to/files/i/want/to/backup | udp-sender
# On Windows
udp-receiver > file.tar

Does the Unix command simply go in the command line with root access? I assume I change the /path/to/files/i/want/to/backup to the path relevant to my server? What if I simply want to copy all the files on the drive? Do I designate the name of the backup file to be sent before the udp-sender or is built in to be named when received?

On the Windows end, where is that command entered? The command prompt? Can I name the file to be anything I like (probably something along the lines of backupmmddyy.tar or are there restrictions?

Sorry for, what I am sure, are elementary, if not asinine, questions; I really appreciate the responses.
# 6  
Old 03-22-2011
Quote:
Originally Posted by c.wakeman
Thanks again for your help so far Corona688.

An update so far:
I used the uname command, we're running Linux.

The decision was made to purchase a cheaper 1 TB external hard drive as a stop-gap measure to make sure the data is backed up before moving ahead with setting up an automatic backup to a dedicated raid 1 array. We purchased a Buffalo Linkstation Live LS-CHL.

The drive can be formatted to to FAT, NTFS, XFS, and HFS+.
All right so far.
Quote:
The server I want to backup is an NTFS file system.
??? I thought you were running Linux!
Quote:
I believe the drive comes XFS standard. Should I format the drive to NTFS for compatibility? or is that a non-issue.
If you intended to plug the hard drive into anything directly, Windows wouldn't understand XFS. But that should be a non-issue for network storage.
Quote:
The user manual for the drive states the disadvantages of NTFS as:
1) Read-only from the LinkStation or a Mac.
2) Not suitable for backup from the LinkStation.
I'm not sure why it says this.
Quote:
The relevant disadvantage of XFS is:
You cannot read data by directly connecting to a PC.
True for Windows, but XFS isn't completely alien -- we use it here at work for our Linux file server. Linux can, if it's configured to do so.
Quote:
Once that is determined, what are the steps I need to take to make a backup .tar of the server files based upon the code you provided earlier?
I don't have a Buffalo Linkstation Live LS-CHL so I can't tell you how you'd be able to connect to it, but once you do, you open a DOS prompt, change to the drive you attached your NAS as, and run the command. You'll need to put udpcast on the same drive or in your PATH. You can download a windows version of udp-receiver from UDP Cast
Quote:
Does the Unix command simply go in the command line with root access?
Well, you need to install udpcast first of course. And you can run it under any user with sufficient access privileges to get at all the files in question.
Quote:
I assume I change the /path/to/files/i/want/to/backup to the path relevant to my server?
Yes.
Quote:
What if I simply want to copy all the files on the drive?
All the files on which drive? Linux doesn't have a c: or d: like Windows, all your partitions are accessed through the same file tree. Some folders chosen by /etc/fstab become partitions -- files inside them reside on that partition.

You can't just do a blind copy of everything while the server's running. There's things that shouldn't be copied while in use, and lots of things it wouldn't make sense to bother copying anyway.

If you really want to do a true, blind copy of the server that you could copy back into a new drive and boot without it knowing the difference, you shouldn't do so while the server's operating, you should boot from a livecd and do so with minimal effect on the system itself. But if you don't actually know how to use Linux yet, your options are very limited.

So, what's your server actually doing? If you don't know, could you find out?
Quote:
Do I designate the name of the backup file to be sent before the udp-sender or is built in to be named when received?
Windows creates the file, Windows decides the name.
Quote:
On the Windows end, where is that command entered? The command prompt?
Yes.
Quote:
Can I name the file to be anything I like (probably something along the lines of backupmmddyy.tar or are there restrictions?
You can name it anything you want.

---------- Post updated at 03:21 PM ---------- Previous update was at 03:17 PM ----------

It may be simpler, and faster to plug the drive into the server direct, mount it, and create the tarball on it that way. Assuming your Linux server can understand XFS.
This User Gave Thanks to Corona688 For This Post:
# 7  
Old 03-23-2011
Quote:
The server I want to backup is an NTFS file system.
Quote:
??? I thought you were running Linux!
Okay so here is the setup as best I know it. We have a unix terminal running Linux (I logged in and used the uname command to check) that is used as a data server. Users in the office, mostly on pcs but some on macs, can access the drive (through windows) by mapping a network drive under Tools in My Computer and signing in as a registered user. When one does that, the details section on the left info bar lists the name and the physical address, then "Network Drive", "File System: NTFS", and then the free space and total size. I can also access the network through Samba/SWAT, SecureCRT, and the physical terminal itself.

Quote:
So, what's your server actually doing? If you don't know, could you find out?
By doing, do you mean, what is it used for? If so, the department uses it to store research data. From what I understand, most users use it to backup data from their computers, but there may be some users who save data primarily or only to the server for personal or security reasons. The server has joint shared space, where anybody has read and copy privileges but only the author of a file has edit/delete rights. In addition, each registered user should have a personal space that only they see. To be quite honest I don't know much more beyond that, nor do I think does anyone else at this point. I quite accidentally stumbled onto this problem looking for a fix for something else (trying to make a public folder on the shared space that gave all users full privileges of any files placed there) and contacted a number of current and former employees to figure out what had been done in the past in terms of backups, and that appears to be nothing.

Quote:
It may be simpler, and faster to plug the drive into the server direct, mount it, and create the tarball on it that way. Assuming your Linux server can understand XFS.
So that would be simply plugging the drive into the server via USB, mounting (with code) and creating the tarball (more code)? How can I tell if my Linux server can understand XFS or not? The uname -a command gave more info, would that help? How long approximately would that take? and vs. say doing it through the udpcast as you suggested above?

When I do the backup do I need to prevent other activity on the server?

Thanks
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. What is on Your Mind?

Mobile Friendly Version of UNIX.COM

Hello, I have noticed some problems with Google complaining our site is not "https://search.google.com/www.usearch-console/mobile-friendly" using only Tapatalk. So, after a lot of work, I have re-enabled our legacy mobile style and make some improvements and Google has declared us "mobile... (2 Replies)
Discussion started by: Neo
2 Replies

2. SCO

Backup/RAID of HD on Old UNIX Server

I need to be able to make a backup image of an OLD UNIX server HD where I can restore the complete HD from scratch if (when) the HD fails. This server runs the accounting system for a company. I can and have backed the data up via local FTP, but O/S and Apps are so old that I am not sure I could... (21 Replies)
Discussion started by: chrishouse
21 Replies

3. Red Hat

RAID Configuration for IBM Serveraid-7k SCSI RAID Controller

Hello, I want to delete a RAID configuration an old server has. Since i haven't the chance to work with the specific raid controller in the past can you please help me how to perform the configuraiton? I downloaded IBM ServeRAID Support CD but i wasn't able to configure the video card so i... (0 Replies)
Discussion started by: @dagio
0 Replies

4. AIX

SCSI PCI - X RAID Controller card RAID 5 AIX Disks disappeared

Hello, I have a scsi pci x raid controller card on which I had created a disk array of 3 disks when I type lspv ; I used to see 3 physical disks ( two local disks and one raid 5 disk ) suddenly the raid 5 disk array disappeared ; so the hardware engineer thought the problem was with SCSI... (0 Replies)
Discussion started by: filosophizer
0 Replies

5. Shell Programming and Scripting

Converting MSVC /showInclude to unix friendly path

Hi guys, I've been trying to do this for hours, and I've just been running around in circles trying to get this script made. I have a set of files outputted by an MSVC compiler that looks like this 1> helloworld.cpp 1> Note: including file: c:\dev\test\makefile\source\helloworld.h 1> ... (7 Replies)
Discussion started by: krad
7 Replies

6. UNIX for Advanced & Expert Users

User friendly Unix ksh prompt to type directories/files

Hello, I wanted to setup user friendly ksh command prompt, by typing first character of files or directories and then tab bring up whole word. No need to type whole file/directory/command names. Example: cat a file like university just typing un and then tab bring up whole university wod.... (3 Replies)
Discussion started by: abdurrouf
3 Replies

7. HP-UX

RAID 0 on Unix

I was wanting to know if anyone knew how to setup RAID 0 on an old HP Unix server. It's for where I work and my boss has two hard drives and wants the second to take over if the first one fails hence RAID 0. If anyone could help me it would be greatly appreciated. (5 Replies)
Discussion started by: zmachine04
5 Replies

8. UNIX for Advanced & Expert Users

Unix Sco 5.0.7 Raid Problems

Hello there guys , I have this problem , i have this hp smart array 641 raid card and i'm trying to install sco unix 5.0.7 and is says no root disk found right before the instalation is about to start. I know that you have to load the driver befor the install bud i really cannot find the... (0 Replies)
Discussion started by: josramon
0 Replies

9. UNIX for Dummies Questions & Answers

RAID Unix command

Is there a command used to determine whether or not a machine has been RAIDed besides using 'df -k'. I am interested b/c I am writing a script where I would like to receive some sort of notification as to whether or not a machine has been RAIDed. The simpler the info returned back, the better. ... (2 Replies)
Discussion started by: zuinc
2 Replies
Login or Register to Ask a Question