Shell Programming and Scripting

Ok, here's the problem. I need to copy about 200 GB from an old FreeBSD (5.2.1) server to a new Debian server. The old server's fastest port is the ethernet port. I set up an NFS server on the new machine so I can just copy the files over using regular commands/scripts etc. Thing is, I probably shouldn't copy dozens of GB of files using the 'cp' command. The last time I tried that it crashed the server.

My plan is to write a script that will process a list of files copying each file from the list one at a time (while making a separate list of files successfully copied that way if anything goes wrong, I can difference the two lists and pick up where it left off). More importantly, I can copy all files ahead of time and then generate a smaller list of files modified since the original copy so that on migration day I have a minimal number of files to copy minimizing downtime.

The main problem (that I can see) with this is, any list I generate will contain both files and folders, but the folders don't need to be copied, only recreated with identical owners and permissions. Unless there is a flag for the 'cp' command that copies a folder without it's contents (not that I know of), I need to find some other way to test each path to see what it is, then if it is a directory, somehow find the permissions and owner and duplicate it on the new server.

If anyone has any advice, can see any other problems or knows of a better way I might do this, I'd appreciate the help. Thanks!

sounds to me like you want to use rsync

wow, that's exactly what I want. I'm always trying to reinvent the wheel it seems

I installed rsync on my Debian box (the FreeBSD already had it) and then ran this from the BSD box:

rsync -r -H -p -o -g -t -v -e ssh /users4 root@myserver.domain:/

Everything seems to be working great. And I can't believe it actually compares parts of files. That will make transferring the large mail files so much easier. I'll have the server migrated in less than an hour during lunch with this tool

Yes, rsync is wonderful.

Now that you know that, take a look at Rsnapshot, which uses rsync to create backup sets. I found the docs a little confusing so I wrote Using Rsnapshot

Mac OS X uses a scheme just like this for their Time Machine backups. They have a few tricks rsnapshot lacks, but the basic idea is the same and it's definitely a convenient scheme - thanks to rsync!

Ok, that's even more awesome! I will definitely bookmark that and take a look once the new machine is up and running. Incremental backups would be wonderful.