RSYNC script to transfer folders recursively without overwriting via FTP


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting RSYNC script to transfer folders recursively without overwriting via FTP
# 1  
Old 11-13-2009
RSYNC script to transfer folders recursively without overwriting via FTP

Hi all,

I would need a bash script to sync/transfer folders recursively via FTP/RSYNC
(I initially planned to use FTP but I heard RSYNC would fit a lot better for this job(?))

The situation:
3 different Linux servers
1. source
2. destination - Samba
3. Server where the script runs on

As the files that have to be transferred are big database files (.bak, etc) overwriting (unnecessary traffic) should be avoided. Also no prompting because the script has to be 'cronned', to run nightly.

To sketch the situation more in detail; Now there is a remote samba backup server containing lots of SQL database files: .bak, .mdf files etc.
Here, on location, we have a server that should be synced with this samba via FTP/RSYNC. File wise.
On the remote samba server the files are constantly updated and already have the filename %data%_%hour%_%databasename%.bak
Something like:

Code:
BELGsdsM_TRAININGS_23-10-2009_10h01.bak
BELGsdsM_TRAININGS_23-10-2009_18h01.bak
etc

The script runs on yet another remote server.

The actions that I would need are

1. Connect via FTP/RSYNC? to source server
2. Connect via FTP/RSYNC? to dest. server
3. Possibly compare the files between source and dest. via the filename
4. transfer the files recursively (without overwriting)
5. close connections

What I have are the 1st, 2nd and 5th step...

Thanks a lot in advance!
# 2  
Old 11-13-2009
I think you need to read the man page for rsync.

Rsync only copies what has changed. For large files, it only copies whatever bytes have changed.

Rsync is designed to do exactly what you want and yes, it can be done in cron.
# 3  
Old 11-13-2009
Quote:
Originally Posted by TonyLawrence
I think you need to read the man page for rsync.

Rsync only copies what has changed. For large files, it only copies whatever bytes have changed.

Rsync is designed to do exactly what you want and yes, it can be done in cron.
Thx. Ok good news... The intention is to write a little script and cron that.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Rsync added new folders after copy?

Hi guys, Don't really know much about unix or anything, just starting to mess around a little bit to have more understanding in general. So, I tried using rsync to copy my macbook pro backup/clone from an external drive I have to another external drive. I ended up using... "sudo rsync -a... (1 Reply)
Discussion started by: cbjeebs
1 Replies

2. Shell Programming and Scripting

recursively going through folders and subdirectorys and running delete from crontab

Hio, So I have a crontab delete of older files setup. This script works fine if I run them by each individual directory. Problem is there are so many thousands of files and hundreds of directories and sub directories that I need to recursively have it go through and delete files by directory... (2 Replies)
Discussion started by: vsekvsek
2 Replies

3. Shell Programming and Scripting

How best to remove certain characters from filenames and folders recursively

hello, I'm trying to figure out which tool is best for recursively renaming and files or folders using the characters \/*?”<>| in their name. I've tried many examples that use Bash, Python and Perl, but I'm not much of a programmer I seem to have hit a roadblock. Does anyone have any... (15 Replies)
Discussion started by: prometheon123
15 Replies

4. Shell Programming and Scripting

ftp script to copy folders recursively

hi all I want a script that will use ftp to copy folder and sub folders from source server to current server. if i use -r switch then it just copies folders for 5 level. (1 Reply)
Discussion started by: kashif.live
1 Replies

5. UNIX for Dummies Questions & Answers

delete recursively contents of folders

hi, I've a folder structure like : /home/project/LIBNAMEA/FILE1 /home/project/LIBNAMED/FILE2 /home/project/LIBNAMEC/FILE3 /home/project/LIBNAMED/FILE4 /home/project/LIBNAMEX/FILE5 (there is no relation in the letters after the project/ ) and i need to delete the files keeping... (5 Replies)
Discussion started by: jtmartins
5 Replies

6. Shell Programming and Scripting

Help needed to transfer list of files to FTP server, to different folders

Hello Unix Gurus, Help required from you. My requirement is something like this I want to create a concurrenct program in Oracle Applications using shell script to transfer files from Apps Server to destination FTP server. I have created custom program, where I will extract all the... (4 Replies)
Discussion started by: amazon
4 Replies

7. Shell Programming and Scripting

FTP run from shell script gives slow transfer rates

Hey everybody, this is my first post so be gentle. I have two Sun 5220's running Solaris 10 that are directly connected with a cross-over cable at Gig. One of these boxes is my production Oracle server which generates a 50GB dump file every evening at 10:50. The other Solaris is a devolopment... (8 Replies)
Discussion started by: Countificus
8 Replies

8. Shell Programming and Scripting

Script for FTP (transfer only new files)

Hi everybody, I just want to transfer files with FTP (mget and mput). The problem is that I dont want to overwrite any existing files and don't want to transfer them again (e.g. using the rename-function). So I only want to transfer new files with mget and mput. My first idea was to create... (3 Replies)
Discussion started by: inoxx
3 Replies

9. UNIX for Advanced & Expert Users

script to be run after every FTP transfer

Hello everyone, we are running VSFtpd as FTP Server on Redhat AS 5. I would like to have a script to be run immediately after every FTP transfer. Is is possible (or with any other FTP Server?). I thought of writing a script to monitor if FTP transfer happening and run the required-script... (2 Replies)
Discussion started by: prvnrk
2 Replies

10. UNIX for Advanced & Expert Users

how to sync remote folders without overwriting

Hi, I need to sync 2 remote folders so i setup with rsync which is working very fine. But, there comes another requirement to maintain multiple versions of files on destination (NOT to overwrite the destination file). If there is difference between source file and destination file, it must... (2 Replies)
Discussion started by: prvnrk
2 Replies
Login or Register to Ask a Question