Shell script to transfer the files from source to target server.


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Shell script to transfer the files from source to target server.
# 1  
Old 01-16-2009
Shell script to transfer the files from source to target server.

I need to write a shell script to transfer the files every hour from source - target server.
The cron job should be running every hour and shouldn't copy already copied files to the remote server ?

I was able to write intial script but not able to get the logic for (in the next run it should copy only newly created files with in that hour)

Here is the logic:-
for I in `ls`
do
echo $I - Log from cron
scp $I pluto@5.67.16.25:$REMOTE_LOC/
sleep 120
done


Thanks,
# 2  
Old 01-16-2009
When you transferred your files, just move them or rename them if possible so they will not be transferred again.
If these files may not be renamed or moved to another location, copy them from their original location to some "transfer" directory and store them there.
To save some space, you can also just write into some txt-file, which files you just transfered and filter them out to not transfer them again by checking if their names are already in the list.
There is a lot of possibilities how to do that.

Best write with [ code ] tags when displaying code, data or logs etc.
# 3  
Old 01-16-2009
Maybe you can...

For the succeeding steps, why not use find instead of ls in filtering the files that needs to be copied, that way you can use the -mtime (can be used filter files based on the last mod time), -name, -type and other useful flags
# 4  
Old 01-16-2009
Why not just use rsync? It can be a bit confusing.. but very very configurable!
# 5  
Old 01-16-2009
Here I can move the files on remote server to a different directory.
But how come the local server knows that it shouldn't transfer already transferred files in the next hour run ? The files that gets created are 500 MB each time.

here how can I use the filter command ?


Thanks,
Radhika.
# 6  
Old 01-16-2009
Hi angheloko,
why not use find instead of ls in filtering the files that needs to be copied, that way you can use the -mtime (can be used filter files based on the last mod time), -name, -type and other useful flags


I know using the mtime with days (number of days). I don't know how to use modified time like for the past 1 hour,2 hours ?

Could you please give me a hint on this ?

Thanks,
Radhika.
# 7  
Old 01-16-2009
Hi Rhije,

Why not just use rsync? It can be a bit confusing.. but very very configurable!

I don't have much idea about the rsync command ? I have just gone thru the rsync man page .


Here I am transferring the files from Solaris server to Linux server and already set up SSH between those boxes.

Thanks,
Radhika.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Automate splitting of files , scp files as each split completes and combine files on target server

i use the split command to split a one terabyte backup file into 10 chunks of 100 GB each. The files are split one after the other. While the files is being split, I will like to scp the files one after the other as soon as the previous one completes, from server A to Server B. Then on server B ,... (2 Replies)
Discussion started by: malaika
2 Replies

2. Shell Programming and Scripting

Move multiple files 4rm Source to different target folders based on a series num in the file content

Dear Experts my scenario is as follows... I have one source folder "Source" and 2 target folders "Target_123456" & "Target_789101". I have 2 series of files. 123456 series and 789101 series. Each series has got 3 types of fiels "Debit", "Refund", "Claims". All files are getting... (17 Replies)
Discussion started by: phani333
17 Replies

3. Shell Programming and Scripting

Shell scripting to transfer files from one to another server through ftps

Hi Guyz ,, I'm an ERP functional guy , I really need to automate a script which required a shell script but have a little knowledge in shell scripting. I need my generated files to be zipped first in one directory lets say (xyz) and then it needs to transfer another ftp server in... (3 Replies)
Discussion started by: Shogundion
3 Replies

4. Shell Programming and Scripting

Transfer files from one server with bash script

Hello to all, I want to copy from one server to another files of last 24 hours with size between 500MB and 2GB. The code below searches last files in 24 hours. find . -mtime -1 In order to copy faster I'd like to compress the files before copying them. How to automate the process of... (8 Replies)
Discussion started by: Ophiuchus
8 Replies

5. Shell Programming and Scripting

Help with script to transfer files from one server to another

Hi I have the following script: #!/bin/sh set -x touch -mt 201210040000 /tmp/ref1 touch -mt 201210042359 /tmp/ref2 find /fs1/bscsrtx/BSCS_ix/WORK/LOG -type f \( -newer /tmp/ref1 -a ! -newer /tmp/ref2 \) > file_lst scp $(< file_lst) root@10.100.48.76:/ano2005/fs1_2015/LOG/ but somehow its... (7 Replies)
Discussion started by: fretagi
7 Replies

6. UNIX for Dummies Questions & Answers

How to count number files successfully copied from source to target location?

Hi Guys, how to count number of files successfully copied while coping files from source to destination path ex:10 files from source to target location copying if 8 files copied successfully then echo successfully copied=8 failure=2 files if two files get error to coping files... (2 Replies)
Discussion started by: sravanreddy
2 Replies

7. Post Here to Contact Site Administrators and Moderators

How to count successfully copy files source to target location with check directory in Linux?

Hi guys...please any one help me .... how to copy files from source to target location if 5 files copied successfully out of 10 files then implement success=10 and if remaining 5 files not copied successfully then count error=5 how to implement this condition with in loop i need code linux... (0 Replies)
Discussion started by: sravanreddy
0 Replies

8. Shell Programming and Scripting

Need shell script to compare directories and delete files on target server

Hello, I need help in writing the shell script for below mentioned case. There are 2 servers(server A, server B). A cronjob syncs files between these 2 servers. Existing script is copying files from A to B. This is done using the command rsync. However, the files are not deleted... (2 Replies)
Discussion started by: SravaniVedam11
2 Replies

9. Shell Programming and Scripting

Reconcilations script between Source and Target

Hi, I am new to DB2 and in need of urgent help. Here we have about 100 queries (SQL) that have been manually executed to retrieve totals from different tables and post it to Excel spreadsheet. Is there any way I can create a shell script as a wrapper and execute these queries and create a... (5 Replies)
Discussion started by: bhaskar v
5 Replies

10. Shell Programming and Scripting

need a shell script to extract the files from source file and check whether those files existonserve

Hi, I am new to shell scripting.Please help me on this.I am using solaris 10 OS and shell i am using is # echo $0 -sh My requirement is i have source file say makefile.I need to extract files with extensions (.c |.cxx |.h |.hxx |.sc) from the makefile.after doing so i need to check whether... (13 Replies)
Discussion started by: muraliinfy04
13 Replies
Login or Register to Ask a Question