Copy from local to remote


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Copy from local to remote
# 1  
Old 11-04-2014
Copy from local to remote

Hi

I need a advice for writing simple bash script,

I have a file pod.txt which contains source location and remote location:
Code:
/mnt/infile/20141103/701_0001.png/remote/tmp/pk21730/p0330223723074.png
/mnt/infile/20141103/203_0001.png/remote/tmp/pk21731/p0330223723081.png

and I must copy data using above path from local to remote server:
(to better understand " scp /mnt/infile/20141020/701_0001.png /remote/tmp/pk21730/p0330223723074.png ")


I wrote this script:
Code:
#################
#/bin/bash

day=`date --date="1 day ago" +%Y%m%d`
Path1=/mnt/infile/$day
remote=node2@192.168.1.190:/tmp

awk -F '/' '{print "scp" " "  "'$Path1'" "/" $5 " " "'$remote'" "/" $8 "/" $9 }' pod.txt > tr.sh

chmod 755 tr.sh
client=`awk -F '/' '{print $8}'  pod.txt`

for file in $(echo $client);
do ssh node2@192.168.1.190 mkdir -p /tmp/$file;
done;

./tr.sh > tr.log 

##########

the script works fine but I'm afraid what will happen if the files to be copied will be about 10 000,

you have an idea for something with "for" ?





Moderator's Comments:
Mod Comment Please use code tags next time for your code and data. Thanks

Last edited by vbe; 11-04-2014 at 01:02 PM..
# 2  
Old 11-04-2014
You certainly should not login to node2 10000 times. Create a shell script with all the mkdirs, login, and execute remotely. And, I'd create the entire file and directory structure locally, e.g. while read line ; do echo mv ${line/png\//png \/tmp\/}; done < pod.txt, and then copy that over with one single scp command.
# 3  
Old 11-05-2014
Thank's RudiC

I changed the loop to create directories on a remote location
was:
Code:
for file in $(echo $client);
do ssh node2@192.168.1.190 mkdir -p /tmp/$file;
done;

is now:
Code:
ssh node2@192.168.1.190
while read line;
do mkdir -p /tmp/$line;
done; <<< $client

But I don't understand how to use one single scp command to copy all file from local server to remote server ? Can You give more details ?
# 4  
Old 11-05-2014
scp offers the -r option to recursively copy directory trees. man scp:
Quote:
-r Recursively copy entire directories. Note that scp follows symbolic links encountered in the tree traversal.
So - create the tree locally, point scp to the tree root, and let it copy all the files including directories.
Do a test with small subset first. It will not work like a greased lightning, but not logging in 10000 time will make it significantly faster.
# 5  
Old 11-05-2014
I think I can't use option -r because e.g file 1 on local server name "701_0001.png" but on remote server it must name "p0330223723074.png" (first of course I must create directory "pk21730")
# 6  
Old 11-05-2014
That's why I proposed to create the entire new structure locally. If it's on the same file system, mving files will just modify the directory entries and is very fast. When that is done, copy the entire structure over.
This User Gave Thanks to RudiC For This Post:
# 7  
Old 11-05-2014
Ok , now it is clear to me , but to use this method I must calculated If I will have enough free disk space,
ThankS very much RudiC
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Copy local files to single remote host but multiple folders using rsync

I'm trying to copy a file myfile.scr from my local Linux server to multiple folders on remote AiX server using single rsync command. Below command helps me copy the file "myfile.scr" from my localhost to a remote host folder "/app/deployment/tmpfiles" rsync --delay-updates -F --compress... (1 Reply)
Discussion started by: mohtashims
1 Replies

2. Shell Programming and Scripting

File transfer from remote to local

Hi, I came across the scenario, that I need to copy files from the remote server to my local. The files in the remote server are created by another job and its keep on generating the files in that remote folder. We can't able to use SCP command and we're using SFTP to connect the server and... (3 Replies)
Discussion started by: Janarthan
3 Replies

3. Shell Programming and Scripting

To run a local shell script in a remote machine by passing arguments to the local shell script

I need to run a local shell script on a remote machine. I am able to achieve that by executing the command > ssh -qtt user@host < test.sh However, when I try to pass arguments to test.sh it fails. Any pointers would be appreciated. (7 Replies)
Discussion started by: Sree10
7 Replies

4. UNIX for Dummies Questions & Answers

How to copy files from remote server to local?

Hi experts, I 'm newbie to unix world, now I have task to copy the latest files from remote server to my local. I believe this must be very common request in this community. I want you do it one more time for me please. My requirement is something like this: I receive files in the below... (3 Replies)
Discussion started by: parpaa
3 Replies

5. Shell Programming and Scripting

Copy a file from local host to a list of remote hosts --- perl script

Hi friends, i need to prepare a script ( in perl) i have a file called "demo.exe" in my local unix host. i have a list of remote hosts in a file "hosts.txt" now i need to push "demo.exe" file to all the hosts in "hosts.txt" file. for this i need to prepare a script(in perl, but shell... (5 Replies)
Discussion started by: siva kumar
5 Replies

6. UNIX for Dummies Questions & Answers

Copy file from RDP to Local

Hi All, I am trying to copy a file from remote desktop to my local system using copy and xcopy commands...but none of them worked..is there any batch file or commands to do the copying .if do please share with me..... (5 Replies)
Discussion started by: navsan420
5 Replies

7. Shell Programming and Scripting

Using local variable on a remote machine

Hi, I'm writing a korn shell script where the user enters a variable and I have to create a directory remotely which contains the name of that variable. Example. print 'Please enter variable:' read variable ssh user@host 'mkdir before_$variable;' Thank you. (4 Replies)
Discussion started by: jangozo
4 Replies

8. UNIX for Advanced & Expert Users

Commands to copy a tar.gz file from a Remote Unix Server to Local Desktop.

Hi, Just wanted to know, how can I ftp/transfer/copy a (design.tar.gz) archive from a Unix Server (sdmc222.sdmc.cp-srv.com) which is at a remote location, to my Windows Desktop. Obviously, it is not possible at cmd prompt on my Windows using the following commands :- ftp... (3 Replies)
Discussion started by: marconi
3 Replies

9. UNIX for Dummies Questions & Answers

File System - Remote or Local??

Is there a way to find if the file systems mounted on a AIX/Linux box is local or remote? (1 Reply)
Discussion started by: Un1xNewb1e
1 Replies

10. UNIX for Advanced & Expert Users

I like to mount a remote/local diskette...!

Hi...! I have to connect a remote machine (solaris server) from a workstation, and I want to access the local diskette. I like mount the local diskette in the workstation, not the remote diskette (in the server). The server is a Solaris 2.7 in a Sun Ultra 450. The workstations are Sun Ultra... (6 Replies)
Discussion started by: Kayron
6 Replies
Login or Register to Ask a Question