Transfer files from one server with bash script


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Transfer files from one server with bash script
# 1  
Old 02-09-2016
Transfer files from one server with bash script

Hello to all,

I want to copy from one server to another files of last 24 hours with size between 500MB and 2GB. The code below searches last files in 24 hours.

Code:
find . -mtime -1

In order to copy faster I'd like to compress the files before copying them.

How to automate the process of find the last files, then compress, copy them and finally erase the compressed files.

Maybe some could help me out.

Best regards

Last edited by Don Cragun; 02-10-2016 at 08:24 PM.. Reason: Add CODE tags.
# 2  
Old 02-09-2016
What OS and shell are you running?
# 3  
Old 02-09-2016
Hello Jim,

Both origin and destination server when I send echo $0 the answer is bash and when I send uname -a the answer is x86_64 x86_64 GNU/Linux.

Best regards

Last edited by Don Cragun; 02-10-2016 at 08:25 PM.. Reason: Add ICODE tags.
# 4  
Old 02-09-2016
Perhaps you can usersync as the recipient of find.
Code:
-a  # preserve permissions and archive
-z  # will compress for transmission
-v  # is verbose
--dry-run  # is for testing, it will show the result without doing anything. Remove it to do the real work
--remove-source-files # will delete the files after the transfer is successful
--files-from=- # will pickup the list of files to transfer from stdin.
-0 # to handle find output
-e ssh # to transfer encrypted, via the ssh protocol

Code:
find /path/ -mtime -1 -type f -printf %P\\0 | rsync -azv --dry-run --remove-source-files --files-from=- -0  /path/ -e ssh user@hostname:/destination/

If you set your ssh with private/public key this can be automatic.
# 5  
Old 02-12-2016
Hi Aia,

Sorry my lateness in response. I had issues with connection.

Source Server: is the server where the files I want to copy reside
Destination Server: is the server where I try to send the command to transfer the files from Source Server to Destination Server.

So, I'm trying to send the command you suggest in the way below in Destination Server but I get the errors below:. What I'm doing wrong?

Thanks for the help.

Code:
 find /path/ -mtime -1 -type f -printf %P\\0 | rsync -azv --dry-run --remove-source-files --files-from=- -0  /path/ -e ssh root@192.168.100.20:/destination/
 
 find: ‘/path/': No such file or directory
root@1192.168.100.20's password:
building file list ...
rsync: change_dir "/path/" failed: No such file or directory (2)
rsync error: errors selecting input/output files, dirs (code 3) at flist.c(2118) [sender=3.1.1]

# 6  
Old 02-12-2016
Quote:
Originally Posted by Ophiuchus
Hi Aia,

Sorry my lateness in response. I had issues with connection.

Source Server: is the server where the files I want to copy reside
Destination Server: is the server where I try to send the command to transfer the files from Source Server to Destination Server.

So, I'm trying to send the command you suggest in the way below in Destination Server but I get the errors below:. What I'm doing wrong?

Thanks for the help.

Code:
 find /path/ -mtime -1 -type f -printf %P\\0 | rsync -azv --dry-run --remove-source-files --files-from=- -0  /path/ -e ssh root@192.168.100.20:/destination/
 
 find: ‘/path/': No such file or directory
root@1192.168.100.20's password:
building file list ...
rsync: change_dir "/path/" failed: No such file or directory (2)
rsync error: errors selecting input/output files, dirs (code 3) at flist.c(2118) [sender=3.1.1]

/path/ was intended as an example; you have to substitute it for your own directory, the one you want to search from. Same goes with /destination/, you have to substitute for the destination directory you want.
# 7  
Old 02-15-2016
Hi Aia,

I don't know what I'm doing wrong.

The files I want to transfer/copy from are in Server A (Source Server) and it has the addres 192.168.100.20
The Server B would be the destination server

In Server A, the files I want to copy are in path "/SourceFiles"
In Server B I have the destination folder "/DestinationServerB"

I'm trying to send the command from the destination Server (Server B) in the way below but I still get the error.

Code:
find /SourceFiles/ -mtime -1 -type f -printf %P\\0 | rsync -azv --dry-run --remove-source-files --files-from=- -0  /SourceFiles/ -e ssh root@192.168.100.20:/DestinationServerB/

This command should be sent from Destination Server or from Source Server? I'd like to send the extraction script from the destination server.

Thanks again for the help
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Not able to transfer files using between server A and B

Hi All, I have generated public and private pair to avoid to enter password manually while sending files through sftp. But still I am facing issues and every-time prompt asking enter password. Below are steps I fallowed to make ssh connection between two servers. 1. Generated public and... (5 Replies)
Discussion started by: renukeswar
5 Replies

2. Shell Programming and Scripting

Making bash script allways executable when transfer ?

Does it possible to make some bash script automatic to be a executable when transfered to another pc...? (5 Replies)
Discussion started by: tomislav91
5 Replies

3. Shell Programming and Scripting

Shell scripting to transfer files from one to another server through ftps

Hi Guyz ,, I'm an ERP functional guy , I really need to automate a script which required a shell script but have a little knowledge in shell scripting. I need my generated files to be zipped first in one directory lets say (xyz) and then it needs to transfer another ftp server in... (3 Replies)
Discussion started by: Shogundion
3 Replies

4. Shell Programming and Scripting

Do I require remote login access to a windows server to transfer files from a UNIX server

Hi All I need to transfer a file from a UNIX server to a windows server. I saw that it is possible to do this using scp command by looking at the forum listed below: ... (2 Replies)
Discussion started by: vx04
2 Replies

5. Shell Programming and Scripting

Help with script to transfer files from one server to another

Hi I have the following script: #!/bin/sh set -x touch -mt 201210040000 /tmp/ref1 touch -mt 201210042359 /tmp/ref2 find /fs1/bscsrtx/BSCS_ix/WORK/LOG -type f \( -newer /tmp/ref1 -a ! -newer /tmp/ref2 \) > file_lst scp $(< file_lst) root@10.100.48.76:/ano2005/fs1_2015/LOG/ but somehow its... (7 Replies)
Discussion started by: fretagi
7 Replies

6. UNIX for Dummies Questions & Answers

Transfer file from server B to server C and running the script on server A

I have 3 servers A, B, C and server B is having some files in /u01/soa/ directory, these files i want to copy to server C, and i want to run the script from server A. Script(Server A) --> Files at Server B (Source server) --> Copy the files to Server C(Target Server). We dont have RSA key... (4 Replies)
Discussion started by: kiran_j
4 Replies

7. Shell Programming and Scripting

SFTP to transfer files from one server to another

Hello, i have to write a script to perform sftp from the remote server to another server. the files which are at the remote location are huge data log files which should be transfered to my server in a particular folder. could you please provide me the general code (simple )... (1 Reply)
Discussion started by: urfrnddpk
1 Replies

8. Shell Programming and Scripting

Shell script to transfer the files from source to target server.

I need to write a shell script to transfer the files every hour from source - target server. The cron job should be running every hour and shouldn't copy already copied files to the remote server ? I was able to write intial script but not able to get the logic for (in the next run it should... (12 Replies)
Discussion started by: radhirk
12 Replies

9. Linux

transfer files from the server 1 to server 2

I want to transfer files from the server 1 to server 2, but only information available FTP my server 2 How? Is there a script help me in the transfer of files through FTP :( (2 Replies)
Discussion started by: bomozah
2 Replies

10. UNIX for Dummies Questions & Answers

Transfer files from Unix server to Windows using FTP

Dear Friend, I don't know much about unix.I am an VB6.0 Programmer.I need to move an text files as aaa.txt from unix server to windows "D: " driver using the FTP protocol.Is it possible to do this with help of unix shell script.If possible please give some sample codes. Please answer as early... (1 Reply)
Discussion started by: gjsaravanan
1 Replies
Login or Register to Ask a Question