Transfer file from a server takes long time


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Transfer file from a server takes long time
# 1  
Old 12-27-2016
Hammer & Screwdriver Transfer file from a server takes long time

It takes 6 hrs for a 90 GB zip file that i am copying / transferring from serverA onto serverB.

Code:
scp user1@serverA:/opt/setup/cash.zip .
Output:
cash.zip            21%   19GB   4.7MB/s 4:11:46 ETA

Code:
uname -a
SunOS serverB 5.11 11.2 sun4v sparc sun4v

Can you please suggest if i could do something to speedup the file transfer ?
# 2  
Old 12-27-2016
its based on the network bandwidth available between 2 locations. currently i see it transfers 4.7Mb/Sec.

where is server A and B is located ?
# 3  
Old 12-27-2016
Quote:
Originally Posted by itkamaraj
its based on the network bandwidth available between 2 locations. currently i see it transfers 4.7Mb/Sec.

where is server A and B is located ?
I dont know how far are they located as they are client servers. So i may need to inquire with them as i dont know how to figure it out myself.

I was looking more so from the command prospective if we can tweak it for faster transfer based on pre-conditions.

I will live with it if there is no way out.
# 4  
Old 12-28-2016
If this is a regular transfer of a file or files that are updated (not completely re-written) you may have an improvement with rsync. This would examine the files at each end and only transmit the blocks that need changing so there should be less to transfer. Of course, if the target file does not exist, it will send the whole file, so you are stuck waiting on the network as above.

If you need to encrypt the connection (i.e. it's not a private network) then search for the string encrypt in the manual page and there are some examples there.

You can improve the network speed by paying for a faster link between the two sites, but that comes down to a business cost justification. I suppose with a complex network with many hops, it might be possible to improve things if a direct link could be set up, but this is entirely down to how the network is delivered. It might be possible to set up a VPN connection from one site to another over the public internet, but there are no guarantees that it will actually be any faster.



I hope that this helps,
Robin
# 5  
Old 12-28-2016
What is the contents of the zip file?
Were a part of its files already transferred?
If yes, that is where rsync comes into play.
# 6  
Old 12-28-2016
What are the operating systems at each end?

Last edited by hicksd8; 01-03-2017 at 11:50 AM..
# 7  
Old 01-03-2017
The option -C enable compression on the fly for scp command
This User Gave Thanks to matrixmadhan For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Wget takes a long time to complete

Hi, I wish to check the return value for wget $url. However, some urls are designed to take 45 minutes or more to return. All i need to check if the URL can be reached or not using wget. How can i get wget to return the value in a few seconds ? (8 Replies)
Discussion started by: mohtashims
8 Replies

2. UNIX and Linux Applications

database takes long time to process

Hi, we currently having a issue where when we send jobs to the server for the application lawson, it is taking a very long time to complete. here are the last few lines of the database log. 2012-09-18-10.35.55.707279-240 E244403536A576 LEVEL: Warning PID : 950492 ... (1 Reply)
Discussion started by: techy1
1 Replies

3. Red Hat

Login to server takes time

When I am trying to login to a server using Putty ssh from my windows xp machine it is taking too much time for login. My server name is http2 and when I enter this address into putty I immediately get login as: screen, but after I put my username root and enter the cursor moves downwards... (14 Replies)
Discussion started by: abhilashkrishn
14 Replies

4. Shell Programming and Scripting

sort takes a long time

Dear experts I have a 200MG text file in this format: text \tab number I try to sort using options -fd and it takes very long! is that normal or I can speed it up in some ways? I dont want to split the file since this one is already splitted. I use this command: sort -fd file >... (12 Replies)
Discussion started by: voolek
12 Replies

5. UNIX for Dummies Questions & Answers

Changing Password process takes a long time

We are running unix. After a reboot of the server we have found that changing password takes a long time. if type in passwd "username" you can type in the 1st instance of the password , press enter , then it will wait for about 3 minutes before bringing up the confirm password line typing it in... (4 Replies)
Discussion started by: AIXlewis
4 Replies

6. UNIX for Dummies Questions & Answers

time how long it takes to load a module

Hello, like the title says, how can i measure the time it takes to load a module in Linux, and how how can i measure the time it takes to load a statically compiled module. /Best Regards Olle ---------- Post updated at 01:13 PM ---------- Previous update was at 11:54 AM ---------- For... (0 Replies)
Discussion started by: ollebanan
0 Replies

7. Linux

it takes long time to login on server

Hi, I am trying to login using ssh on Red Hat Linux 5 server, The password appears immediately but after I enter the password it takes about 90 seconds to login completely. Please suggest what changes require? Regards, Manoj (4 Replies)
Discussion started by: manoj.solaris
4 Replies

8. AIX

How long does AIX resync the time with another server

Hi, I have NTP configured: vi /etc/ntp.conf broadcastclient server 128.127.1.3 driftfile /etc/ntp.drift tracefile /etc/ntp.trace # xntpdc xntpdc> sysinfo system peer: 128.127.1.3 system peer mode: client leap indicator: 00 stratum: 12 precision: ... (0 Replies)
Discussion started by: victorcheung
0 Replies

9. Shell Programming and Scripting

shell script takes long time to complete

Hi all, I wrote this shell script to validate filed numbers for input file. But it take forever to complete validation on a file. The average speed is like 9mins/MB. Can anyone tell me how to improve the performance of a shell script? Thanks (12 Replies)
Discussion started by: ozzman
12 Replies

10. Programming

fwrite takes extremely long time

After my previous thread, I think I found out what causes the long delays. I run this program on several Linux computers, and the sometimes (after the file with the arrays becomes big) the fwrite takes between 100 ms to 900 ms. This is very bad for me, as I want a timer to halt each 30 ms.... ... (5 Replies)
Discussion started by: inna
5 Replies
Login or Register to Ask a Question