automate scp faster


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting automate scp faster
# 1  
Old 04-03-2012
automate scp faster

Hi,

I am currently using a shell script to transfer file to another machine non-stop using scp . below is my code :
Code:
#!/bin/bash
while :
do
scp /home/pc3/Desktop/b.html pc1@192.168.1.102:/home/pc1/Documents
done

After it transfer the first image, it needs 5 seconds to send the second image and need 5 seconds to send the third image ....

Is there any possible to let it send every 1seconds or even in milliseconds ?
Moderator's Comments:
Mod Comment please use code tags

Last edited by jim mcnamara; 04-04-2012 at 12:09 AM.. Reason: code tags
# 2  
Old 04-04-2012
just replace scp with rsync and run it as background process with & added at the end of your commands
Code:
#!/bin/bash
while :; do
        rsync /home/pc3/Desktop/b.html pc1@192.168.1.102:/home/pc1/Documents &
        sleep 0.03
done

# 3  
Old 04-04-2012
Code:
#!/bin/bash
cnt=0
for fname in /home/pc3/Desktop/b.html/*
do
scp -pq $fname pc1@192.168.1.102:/home/pc1/Documents/  &
cnt=$(( $cnt + 1 )) 
a=$(( $cnt % 10 ))
[ $a -eq 0 ]] & wait
done
wait

WARNING:
This sends batches of files, then waits for those all to be sent. 10 is probably kind of high depending on your ethernet bus speeds and the network throughput. You want the max throughput without over-saturating the connection(s). And maybe let other users do things as well. This could be a great way to seriously annoy friends/bosses.
# 4  
Old 04-04-2012
thanks for the quick reply ..

I am currently using cygwin in windows to run the script,

when i run the script, it give me error that

rsync: command not found

is there need to install any else program to make it run ?

and when i change the rsync in script and run in ubuntu machine, it gives error and the error is given below :

ssh_exchange_identification: Connection closed by remote host
rsync: connection unexpectedly closed (0 bytes received so far) [sender]
rsync error: unexplained error (code 255) at io.c(601) [sender=3.0.7]
ssh_exchange_identification: Connection closed by remote host



thanks in advance for the reply Smilie

Last edited by Ericyue; 04-04-2012 at 12:31 AM..
# 5  
Old 04-04-2012
Maybe you can run the CentOS on the vmware under your windows which supply you a full featured Linux environment as cygwin is just a Simulator without so many tools available
# 6  
Old 04-04-2012
thanks again for the quick reply

the CentOS VM is install under windows (when startup it don't ask for dual boot and straight to windows and the CentOS Vm is run when calling inside windows )

or it is something like install ubuntu and windows in a machine (when startup, it will ask which 1 to boot) ??

sorry for my ignorance and thanks in advance for the reply
# 7  
Old 04-04-2012
What i said just now were not to configure you computer dual boot, but just run CentOS on Vmware which were installed in your windows!
To perform dual boot and be able to choose which system to boot, just google it as it's so easy to do!
This User Gave Thanks to complex.invoke For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Automate splitting of files , scp files as each split completes and combine files on target server

i use the split command to split a one terabyte backup file into 10 chunks of 100 GB each. The files are split one after the other. While the files is being split, I will like to scp the files one after the other as soon as the previous one completes, from server A to Server B. Then on server B ,... (2 Replies)
Discussion started by: malaika
2 Replies

2. Shell Programming and Scripting

Automate scp between servers

Hello, Could someone please help me with the below requirement? I need to automate scp files between two servers. I have a file having server names and paths to the folder like below server1 /path/to/folder/ server1 /path/to/folder/ server1 /path/to/folder/ server2 /path/to/folder/... (4 Replies)
Discussion started by: Kochappa
4 Replies

3. UNIX for Dummies Questions & Answers

Which system is faster?

i'm trying to decide if to move operations from one of these hosts to the other. but i cant decide which one of them is the most powerful. each host has 8 cpus. HOSTA processor : 0 vendor_id : GenuineIntel cpu family : 6 model : 44 model name : Intel(R) Xeon(R) CPU ... (6 Replies)
Discussion started by: SkySmart
6 Replies

4. UNIX for Dummies Questions & Answers

Why is RAID0 faster?

I have read anecdotes about people installing RAID0 (RAID - Wikipedia, the free encyclopedia) on some of their machines because it gives a performance boost. Because bandwidth on the motherboard is limited, can someone explain exactly why it should be faster? (7 Replies)
Discussion started by: figaro
7 Replies

5. UNIX for Dummies Questions & Answers

Which command will be faster? y?

i)wc -c/etc/passwd|awk'{print $1}' ii)ls -al/etc/passwd|awk'{print $5}' (4 Replies)
Discussion started by: karthi_g
4 Replies

6. UNIX for Dummies Questions & Answers

How to grep faster ?

Hi I have to grep for 2000 strings in a file one after the other.Say the file name is Snxx.out which has these strings. I have to search for all the strings in the file Snxx.out one after the other. What is the fastest way to do it ?? Note:The current grep process is taking lot of time per... (7 Replies)
Discussion started by: preethgideon
7 Replies

7. Shell Programming and Scripting

Faster then cp ?

Hi , I need to copy every day about 35GB of files from one file system to another. Im using the cp command and its toke me about 25 min. I also tried to use dd command but its toke much more. Is there better option ? Regards. (6 Replies)
Discussion started by: yoavbe
6 Replies

8. UNIX for Advanced & Expert Users

faster way to loop?

Sample Log file IP.address Date&TimeStamp GET/POST URL ETC 123.45.67.89 MMDDYYYYHHMM GET myURL http://ABC.com 123.45.67.90 MMDDYYYYHHMM GET myURL http://XYZ.com I have a very huge web server log file (about 1.3GB) that contains entries like the one above. I need to get the last entries of... (9 Replies)
Discussion started by: tads98
9 Replies

9. IP Networking

Mandrake should be faster.

For some reason 8.1 Mandrake Linux seems much slower than Windows 2000 with my cable modem. DSL reports test says they conferable speed with Windows2 though. This is consistant slow with both of my boxes, at the same time. Linux used to be faster, but not with Mandrake. Any way to fix this? (17 Replies)
Discussion started by: lancest
17 Replies
Login or Register to Ask a Question