Unix/Linux Go Back    


Programming Post questions about C, C++, Java, SQL, and other programming languages here.

Program or bash script to see total progress of copy

Programming


Reply    
 
Thread Tools Search this Thread Display Modes
    #15  
Old Unix and Linux 3 Days Ago
RudiC RudiC is offline Forum Staff  
Moderator
 
Join Date: Jul 2012
Last Activity: 25 September 2017, 8:39 AM EDT
Location: Aachen, Germany
Posts: 11,374
Thanks: 299
Thanked 3,527 Times in 3,244 Posts
The requestor doesn't seem to read (or try (or, at least, comment on)) ALL contributions...
Sponsored Links
    #16  
Old Unix and Linux 3 Days Ago
drl's Unix or Linux Image
drl drl is online now Forum Advisor  
Registered Voter
 
Join Date: Apr 2007
Last Activity: 25 September 2017, 9:26 AM EDT
Location: Saint Paul, MN USA / BSD, CentOS, Debian, OS X, Solaris
Posts: 2,169
Thanks: 220
Thanked 401 Times in 346 Posts
Hi.
Quote:
Originally Posted by RudiC View Post
The requestor doesn't seem to read (or try (or, at least, comment on)) ALL contributions...
I assume that this thread may be useful to others.

A set of numbers from thread linux - Copying a large directory tree locally? cp or rsync? - Server Fault

Code:
To move 532Gb of data distributed among 1,753,200 files we had those times:
    rsync took 232 minutes
    tar took 206 minutes
    cpio took 225 minutes
    rsync + parallel took 209 minutes

I wish they had added cp to the benchmark, but the theme of the thread is comparing speed, with a sub-theme of monitoring, and a few other characteristics, like re-start-ability.

So, as regards this thread, use something that does have a progress mechanism built-in or roll-your-own.

Best wishes ... cheers, drl

Last edited by drl; 3 Days Ago at 01:40 PM..
Sponsored Links
    #17  
Old Unix and Linux 3 Days Ago
Corona688 Corona688 is offline Forum Staff  
Mead Rotor
 
Join Date: Aug 2005
Last Activity: 22 September 2017, 5:42 PM EDT
Location: Saskatchewan
Posts: 22,417
Thanks: 1,126
Thanked 4,239 Times in 3,919 Posts
Of course rsync is slowest when you don't turn off checksumming.

That chart does help prove one of my biggest peeves though - "you can't multithread a hard drive". None of the options are significantly faster.
    #18  
Old Unix and Linux 3 Days Ago
drl's Unix or Linux Image
drl drl is online now Forum Advisor  
Registered Voter
 
Join Date: Apr 2007
Last Activity: 25 September 2017, 9:26 AM EDT
Location: Saint Paul, MN USA / BSD, CentOS, Debian, OS X, Solaris
Posts: 2,169
Thanks: 220
Thanked 401 Times in 346 Posts
Hi.
Quote:
Originally Posted by Corona688 View Post
... None of the options are significantly faster.
Well, 9% return, 209/232 -> 0.90, on a trivial change might be worthwhile to some folks.

Researchers at the center where I worked would moil to get even a 1% speed-up. Of course, that was most often with grad student labor Linux

Best wishes ... cheers, drl
Sponsored Links
    #19  
Old Unix and Linux 3 Days Ago
robertkwild robertkwild is offline
Registered User
 
Join Date: Apr 2016
Last Activity: 22 September 2017, 7:35 AM EDT
Posts: 74
Thanks: 7
Thanked 5 Times in 4 Posts
sorted it,


Code:
yum remove rsync


https://download.samba.org/pub/rsync/rsync-3.1.2.tar.gz



installed it by untarring it cd'd into the dir and running

Code:
./configure.sh
make
make install


and now i get the result i wanted -


Code:
[root@robw-linux data]# rsync -a --info=progress2 call_the_midwife_7_1708/ new/
 14,874,971,690  44%   27.58MB/s    0:10:48  xfr#16, to-chk=2/143)


Moderator's Comments:
Program or bash script to see total progress of copy Please start using code tags, thanks.

Last edited by zaxxon; 3 Days Ago at 08:06 AM.. Reason: code tags
Sponsored Links
    #20  
Old Unix and Linux 2 Days Ago
Corona688 Corona688 is offline Forum Staff  
Mead Rotor
 
Join Date: Aug 2005
Last Activity: 22 September 2017, 5:42 PM EDT
Location: Saskatchewan
Posts: 22,417
Thanks: 1,126
Thanked 4,239 Times in 3,919 Posts
Why did you do all that when you appear to have had it already? Did the version you have not contain --progress?
Sponsored Links
    #21  
Old Unix and Linux 2 Days Ago
Corona688 Corona688 is offline Forum Staff  
Mead Rotor
 
Join Date: Aug 2005
Last Activity: 22 September 2017, 5:42 PM EDT
Location: Saskatchewan
Posts: 22,417
Thanks: 1,126
Thanked 4,239 Times in 3,919 Posts
Quote:
Originally Posted by drl View Post
Well, 9% return, 209/232 -> 0.90, on a trivial change might be worthwhile to some folks.
and for those I suggest the trivial change -I to turn off checksumming.
The Following User Says Thank You to Corona688 For This Useful Post:
drl (2 Days Ago)
Sponsored Links
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Linux More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
Running a C/C++ program and/or bash script from a server frad UNIX for Dummies Questions & Answers 1 12-13-2013 11:50 AM
Putting bash script in C program SkySmart Programming 15 03-06-2013 05:23 PM
bash script to copy files dodgefan67 Shell Programming and Scripting 8 04-06-2011 11:40 PM
Run Program from Bash CGI-Script JMooney5115 Homework & Coursework Questions 8 01-27-2011 06:51 PM
Making a progress gauge in a bash script ccox85 Shell Programming and Scripting 1 01-31-2008 01:16 PM



All times are GMT -4. The time now is 09:28 AM.