scp multiple files


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers scp multiple files
# 1  
Old 11-22-2011
scp multiple files

Hi,

I'm new to Linux. I would like to know how to scp a group of files. I have the below command, but it asks for remote password while sending each file. Is there a way to send all files (identified by - $ ls | grep '.*hrs0314a.*' | xargs -I {} grep -l '.*35663.*' {}) in one go?

$ ls | grep '.*hrs0314a.*' | xargs -I {} grep -l '.*35663.*' {} | xargs -I {} scp {} user1@example.com:/home/user1/Workplace

Thanks in advance for any suggestions.
# 2  
Old 11-22-2011
scp -vv* .xlsx username@(ip address of the machine where you are sending)
this is for the files with xlsc extention ,use same with .txt extention.

---------- Post updated at 02:31 PM ---------- Previous update was at 02:30 PM ----------

if you have 5 files for example use scp file1 file2 file3 with same
# 3  
Old 11-22-2011
Hi,
first, think about using keypair-based authentication to get rid of the password prompt.

Second, you can provide filenames with the '*' wildcard to scp in order to copy multiple files (warning! you must escape the wildcard character if using it on the remote file specification!).
# 4  
Old 11-22-2011
Hi,
first, think about using keypair-based authentication to get rid of the password prompt.

Second, you can provide filenames with the '*' wildcard to scp in order to copy multiple files (warning! you must escape the wildcard character if using it on the remote file specification!).
# 5  
Old 11-22-2011
Hi

I'm afraid I didn't explain enough what I actually want to achieve.

Let's say there are 50 files in the directory. First I'm finding out all files that has the string 'hrs0314a' in their file names. Assume there are 30 such files. In this list of 30 files, I'm looking for those files containing the string '35663'. Assume there are 20 such files.

I have the below command to get the final list of 20 files.

$ ls | grep '.*hrs0314a.*' | xargs -I {} grep -l '.*35663.*' {}

Now I want to scp these 20 files to a remote server. I tried using xargs, but it's prompting for remote password for every single file. (20 times)

$ ls | grep '.*hrs0314a.*' | xargs -I {} grep -l '.*35663.*' {} | xargs -I {} scp {} user1@example.com:/home/user1/Workplace

I'm wondering if there is a way to send these 20 files in one go by entering remote password only once.
# 6  
Old 11-22-2011
If the position of 'hrs0314a' and '35663' with respect to each other in the filename is always the same you can use wildcards in the filename, i.e. (assuming that 'hrs0314a' comes before '35663'):

Code:
$ scp *hrs0314a*35663* user1@example.com:/home/user1/Workplace

anyway, as wildcards expansion is made by the shell, I suspect (cannot test myself at the moment) that scp opens a new connection for every filename expanded, so goin' with key-based authentication may resolve the issue.

see ya
fra

Last edited by frappa; 11-22-2011 at 03:57 PM..
# 7  
Old 11-22-2011
Thanks. May be key-based authentication is the way to go.
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Automate splitting of files , scp files as each split completes and combine files on target server

i use the split command to split a one terabyte backup file into 10 chunks of 100 GB each. The files are split one after the other. While the files is being split, I will like to scp the files one after the other as soon as the previous one completes, from server A to Server B. Then on server B ,... (2 Replies)
Discussion started by: malaika
2 Replies

2. Shell Programming and Scripting

scp over multiple hops

Hi friends, I am trying to scp a file from server A to Server C via Server B (which is jump host) from Server A to Server B i have one pem key. from server B to server C I have different Pem key.. what I tried scp -r -o ProxyCommand="ssh -W %h:%p ec2-user@1.4.5.5 dmu.sh... (7 Replies)
Discussion started by: onenessboy
7 Replies

3. UNIX for Beginners Questions & Answers

scp of multiple files to remote server

Hello, I would like to scp multiple files to a remote server that requires a password for the process to be completed. I have 30 folders (x_1, x_2 ... x_30), each containing 25 files. What I want to do is scp 1 out of the 25 files (file called bvals) for all my folders to a remote server and... (3 Replies)
Discussion started by: nasia.m
3 Replies

4. Solaris

scp multiple files without pattern on Solaris

I need to transfer multiple files using scp between two solaris machines. I could not use pattern since they match with other files in the same directory. I tried the below command, it does not copy any files. $ scp -p user@machine1:/home/fid1/staging/\{a.ksh,b.ksh,c.ksh,d.ksh\} . But... (7 Replies)
Discussion started by: marecar
7 Replies

5. Shell Programming and Scripting

scp or rsync multiple files in parallel from a remote host

Hi. I'm trying to speed up an rsync command by running it in parallel. There's no real option for this other than if the files are in multiple directories (which they're not). And even then there's no way of knowing if rsync has succeeded as the process is running in the background .. and... (4 Replies)
Discussion started by: Big_Jeffrey
4 Replies

6. Shell Programming and Scripting

Need a script to scp a file to multiple boxes

Hello All, I am new to scripting and I am trying to write a script which can scp a file from one box to multiple boxes. I am thinking to do like this. 1) create a file with list of all server names 2)write a script which will pick up each server line by line from server list and copy it to... (1 Reply)
Discussion started by: sintilash
1 Replies

7. Shell Programming and Scripting

scp command for multiple file transfer.

FILE_LIST="{a.txt,b.txt,cal*}" scp -r $..$REMOTE_PATH$FILE_LIST $LOCAL_PATH This script passes only when all the three files are transfere, wat if only two file are transfered, but still I was to make the return code as pass. is it possible. (2 Replies)
Discussion started by: sangea
2 Replies

8. Shell Programming and Scripting

How to run the multiple scp from single script?

Dear Experts, how to run multiple scp commands from single scripts. In a directory oracle redo files accumulate. i would like to copy those redo logs to my standby server. For same i am using scp to copy the files. where i am monitoring that as it is sending the files sequentially most of... (1 Reply)
Discussion started by: nmadhuhb
1 Replies

9. Shell Programming and Scripting

Script to SCP a file to multiple servers

Hi All, I am a total noob to the Unix world, and i hope to learn a lot from this wonderful community. Here's my first post and question , i am trying to SCP a file to multiple servers (multiple destinations) through this little script : #!/bin/ksh # copy files # File to be copied... (7 Replies)
Discussion started by: rdlover
7 Replies

10. Shell Programming and Scripting

SCP multiple files

Hi , I am doing SCP for copying log files from different server(around 24 server) I need to copy these bulk logfiles every 15 min. How can i do multiple SCP at the same time? My current code is like this scp $CUSTCARE_USER@$CUSTCARE_SERVER:$CUSTCARE_HOME/$CUSTCARE_LOG.*... (2 Replies)
Discussion started by: scorpio
2 Replies
Login or Register to Ask a Question