Shell Programming and Scripting

Hi Folks,

I have a NAS running busybox with a ARM processor.

Several PC's backup full weekly and several daily incrementals to the NAS.

The NAS has two USB attached disks, which are swapped off site weekly.

The problem is the built in rsync only transfers at 2Mb/s to the external USB and takes about 6 hours per PC and often fails to complete.

I tried updating the built in rsync, and ran it from the command like myself with various options (-W and no-checksum) but they didnt make it any faster. I think the CPU is the bottle neck.

I am thinking of a copy script after the weekly full backup of Friday night.

The logic would be something like:

1. Delete all files older than 2 weeks from /share/external/sdu/backups
2. Copy all files from /share/backups to /share/external/sdu/backups
3. Some sort of basic verification?

The other thing that sucks is with busybox nas I dont have the cp -n option, so I cant tell it to skip existing files...grrrrrr, else I would run it nightly and just copy files that dont already exist.

I have been using find -mtime +14 to delete all files older than 2 weeks, but again their may be a better way?

Cheers

-Al

What version is the USB connection? That affects speed bigtime. We have garmin basecamp running from a small NAS and it takes 1.5+ hours to move 2GB of map data out over USB to a single hand held meter.

I doubt that it is your cpu. If you can get drivers, consider upgrading the USB.

Your find command to delete files is a very standard thing to do.

Regarding rsync - if it takes a long time to do the same backup where there are lots of identical untouched files in the archiove and the source, then it is an rsync parameter problem. By default rsync uses filetimes.

To preserve filetimes in your rsync:

Do this

Code:

rsync -avp  location1 location2

So when location1 files have the same timestamp as on location2, they are skipped over.
Also, do not use -bwlimit, it throttles throughput.

Can you please post your rsync command?

Oh - forgot to answer your question - cp -n is just a noclobber, you want a "newer" comparison
This is general I do not how your system sees the remote filesystems

Code:

remotedir=/someplace/
find ./somemountpoint -type f |
while read fname
do
      if [[ ! -f  ${remotedir}${fname}  ||  $fname  -nt ${remotedir}${fname} ]] ; then
        cp -p  $fname  ${remotedir}${fname}
      fi
done

However I still think either USB or rsync setup are the real problems here. Or both....