Copy file from different ports in parallel


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Copy file from different ports in parallel
# 1  
Old 09-23-2017
Copy file from different ports in parallel

Hello folks,

Can you please help me to solve the below concern.

I have a source server with 2 ports and have to copy the files from both the port to destination server simultaneously in my shell script.

How can I achieve that?

Source : x.x.x.x port -22
X.x.x.x port -2222

Destination: y.y.y.y

Requirements : want to do copy files from a specific location from both ports to destination server.

I have opened the password less ssh connection to both the ports, but want to perform scp parallel from destination.

Kindly guide

Regards,
Sadique Manzar
# 2  
Old 09-24-2017
You can try something like the following:
Code:
ssh y.y.y.y
scp x.x.x.x:/path/to/files /path/to/destination &
scp -P 2222 x.x.x.x:/path/to/files /path/to/destination &
wait

The trailing ampersands make the commands run in the background, allowing for parallel operation of the two remote copy processes. The wait command makes the script wait for both processes to finish..
# 3  
Old 09-24-2017
Hello Scrutinizer,


I need to call this from destination that is from y.y.y.y need to ssh x.x.x.x.

Kindly suggest

Regards,
# 4  
Old 09-24-2017
Hi Sadique, isn't that what my suggestion is about?
# 5  
Old 09-24-2017
Note carefully: There are limits to parallel copying. Routers and switches can be set to limit throughput - Ex: you can have 10GB but you are limited only 10% of 10GB. With no limits your copying could shutdown dozens of users on a network by causing their network access to creep along. This gets lots of folks mad, fast.

So parallel copying of a large number of files is not faster than copying, say, four files at a time. And FWIW ssh compresses transmissions by default. So turning on ssh compression, for example, on our large Solaris 10 boxes actually slowed down ssh (scp) file copies. Which always seemed paradoxical to me.
This User Gave Thanks to jim mcnamara For This Post:
# 6  
Old 10-04-2017
Deae All,

My destination has open for both the port 22 and 2222.
So i need to copy the data from destination to my source.

I am using after getting help many here like this.

Here is the code:

Code:
while read -r line || [-n "$line"]
do
scp -C abc@x.x.x.x /path
scp -C -P 2222 abc@x.x.x.x /path
done < $destination_path

---------- Post updated at 08:10 AM ---------- Previous update was at 08:09 AM ----------

Please comment if any correction required.

Last edited by Scrutinizer; 10-04-2017 at 12:05 PM.. Reason: code tags
# 7  
Old 10-04-2017
Hi,
  • This is not simultaneously but sequentially (alternating) (see post #2)
  • You are reading variable line from a file whose name is contained in variable destination_path, which seems a bit strange
  • Yet this variable is not used within the loop, so you might as well leave the loop out if there is only one line in the file. Otherwise this does not make sense, since it would keep on overwriting the same file
  • [-n "$line"] is not valid code: there needs to be space around the square brackets.
Login or Register to Ask a Question

Previous Thread | Next Thread

8 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Need help grep multiple ports in a file.

I wish to grep for an entry in a file if it contains and does not start with #, Listen 443 or Listen 9443 Below is what helped me get Listen 443 but how can I tweak the below command to also include Listen 9443 port ? Note: Listen 8443 or Listen 4438 should fail in the grep. ... (2 Replies)
Discussion started by: mohtashims
2 Replies

2. Shell Programming and Scripting

Curl parallel download file list

Hello guys, first post sorry if I did some mess here =) Using Ubuntu 14.04lts 64bits server version. I have a list (url.list) with only URLs to download, one per line, that looks like this: http://domain.com/teste.php?a=2&b=3&name=1 http://domain.com/teste.php?a=2&b=3&name=2 ...... (6 Replies)
Discussion started by: tonispa
6 Replies

3. Shell Programming and Scripting

how to copy the directory but not copy certain file

Hi experts cp bin root src /mnt but not copy bin/bigfile any help? ( I post this thread in the "redhat" forum wrongly, I don't know how to withdraw that question in that wrong forum) Thanks (6 Replies)
Discussion started by: yanglei_fage
6 Replies

4. Shell Programming and Scripting

Parallel execution of command inside file

Hi all, I have the requirement to generate the file containing following command eval /path/ dsjob -logdetail projectname JOBNAME /path/ 1. The file contains the above command say about 150 times i,e only the JOBNAME changes in every command 2. The commands must be written in such a way... (2 Replies)
Discussion started by: sanjay mn
2 Replies

5. Shell Programming and Scripting

parallel while loop based on the file records

Hi, I need to execute parallel with while loop. Input File(source_file.csv) contains filenames the below source_file.csv file contains Customer1.txt Product1.txt Sales.txt Emp.txt Dept.txt Based on the number of rows that file I want to run the script ‘n' times. while... (2 Replies)
Discussion started by: onesuri
2 Replies

6. UNIX for Dummies Questions & Answers

Copy files in Parallel

Is there any out of box Command in Unix or is it possible through shell scripting to copy all the files in a directory in Parallel. Example. I am doing a COLD database backup. I have the data/dbx directory which has around 1000 data files. I was thinking is there a way to spawn the copy... (10 Replies)
Discussion started by: simonsimon
10 Replies

7. UNIX for Advanced & Expert Users

implementation of copy command in parallel

hey i have to implement copy command in parallel in c language. i dont know how to create a new directory in destination. if anything u know related to this help me (1 Reply)
Discussion started by: rajsekhar28
1 Replies

8. Windows & DOS: Issues & Discussions

TCP ports and file sharing

In using a music file sharing program (WinMx), I am told that I cannot make a primary connection (fastest downloads) because I do not have a TCP and UDP port. I am running Windows Me.What do I do? Thanks. (6 Replies)
Discussion started by: dookster5
6 Replies
Login or Register to Ask a Question