Unix/Linux Go Back    


Programming Post questions about C, C++, Java, SQL, and other programming languages here.

Program or bash script to see total progress of copy

Programming


Reply    
 
Thread Tools Search this Thread Display Modes
    #15  
Old Unix and Linux 09-21-2017   -   Original Discussion by robertkwild
RudiC's Unix or Linux Image
RudiC RudiC is online now Forum Staff  
Moderator
 
Join Date: Jul 2012
Last Activity: 18 January 2018, 12:47 PM EST
Location: Aachen, Germany
Posts: 11,951
Thanks: 353
Thanked 3,686 Times in 3,384 Posts
The requestor doesn't seem to read (or try (or, at least, comment on)) ALL contributions...
Sponsored Links
    #16  
Old Unix and Linux 09-21-2017   -   Original Discussion by robertkwild
drl's Unix or Linux Image
drl drl is offline Forum Advisor  
Registered Voter
 
Join Date: Apr 2007
Last Activity: 18 January 2018, 11:24 AM EST
Location: Saint Paul, MN USA / BSD, CentOS, Debian, OS X, Solaris
Posts: 2,216
Thanks: 255
Thanked 417 Times in 358 Posts
Hi.
Quote:
Originally Posted by RudiC View Post
The requestor doesn't seem to read (or try (or, at least, comment on)) ALL contributions...
I assume that this thread may be useful to others.

A set of numbers from thread linux - Copying a large directory tree locally? cp or rsync? - Server Fault


Code:
To move 532Gb of data distributed among 1,753,200 files we had those times:
    rsync took 232 minutes
    tar took 206 minutes
    cpio took 225 minutes
    rsync + parallel took 209 minutes

I wish they had added cp to the benchmark, but the theme of the thread is comparing speed, with a sub-theme of monitoring, and a few other characteristics, like re-start-ability.

So, as regards this thread, use something that does have a progress mechanism built-in or roll-your-own.

Best wishes ... cheers, drl

Last edited by drl; 09-21-2017 at 02:40 PM..
Sponsored Links
    #17  
Old Unix and Linux 09-21-2017   -   Original Discussion by robertkwild
Corona688's Unix or Linux Image
Corona688 Corona688 is offline Forum Staff  
Mead Rotor
 
Join Date: Aug 2005
Last Activity: 18 January 2018, 10:06 AM EST
Location: Saskatchewan
Posts: 22,573
Thanks: 1,163
Thanked 4,293 Times in 3,961 Posts
Of course rsync is slowest when you don't turn off checksumming.

That chart does help prove one of my biggest peeves though - "you can't multithread a hard drive". None of the options are significantly faster.
    #18  
Old Unix and Linux 09-21-2017   -   Original Discussion by robertkwild
drl's Unix or Linux Image
drl drl is offline Forum Advisor  
Registered Voter
 
Join Date: Apr 2007
Last Activity: 18 January 2018, 11:24 AM EST
Location: Saint Paul, MN USA / BSD, CentOS, Debian, OS X, Solaris
Posts: 2,216
Thanks: 255
Thanked 417 Times in 358 Posts
Hi.
Quote:
Originally Posted by Corona688 View Post
... None of the options are significantly faster.
Well, 9% return, 209/232 -> 0.90, on a trivial change might be worthwhile to some folks.

Researchers at the center where I worked would moil to get even a 1% speed-up. Of course, that was most often with grad student labor Linux

Best wishes ... cheers, drl
Sponsored Links
    #19  
Old Unix and Linux 09-22-2017   -   Original Discussion by robertkwild
robertkwild's Unix or Linux Image
robertkwild robertkwild is offline
Registered User
 
Join Date: Apr 2016
Last Activity: 21 November 2017, 9:05 AM EST
Posts: 83
Thanks: 9
Thanked 8 Times in 6 Posts
sorted it,



Code:
yum remove rsync



Code:
https://download.samba.org/pub/rsync/rsync-3.1.2.tar.gz


installed it by untarring it cd'd into the dir and running


Code:
./configure.sh
make
make install


and now i get the result i wanted -



Code:
[root@robw-linux data]# rsync -a --info=progress2 call_the_midwife_7_1708/ new/
 14,874,971,690  44%   27.58MB/s    0:10:48  xfr#16, to-chk=2/143)


Moderator's Comments:
Program or bash script to see total progress of copy Please start using code tags, thanks.

Last edited by Neo; 12-07-2017 at 01:32 PM..
Sponsored Links
    #20  
Old Unix and Linux 09-22-2017   -   Original Discussion by robertkwild
Corona688's Unix or Linux Image
Corona688 Corona688 is offline Forum Staff  
Mead Rotor
 
Join Date: Aug 2005
Last Activity: 18 January 2018, 10:06 AM EST
Location: Saskatchewan
Posts: 22,573
Thanks: 1,163
Thanked 4,293 Times in 3,961 Posts
Why did you do all that when you appear to have had it already? Did the version you have not contain --progress?
Sponsored Links
    #21  
Old Unix and Linux 09-22-2017   -   Original Discussion by robertkwild
Corona688's Unix or Linux Image
Corona688 Corona688 is offline Forum Staff  
Mead Rotor
 
Join Date: Aug 2005
Last Activity: 18 January 2018, 10:06 AM EST
Location: Saskatchewan
Posts: 22,573
Thanks: 1,163
Thanked 4,293 Times in 3,961 Posts
Quote:
Originally Posted by drl View Post
Well, 9% return, 209/232 -> 0.90, on a trivial change might be worthwhile to some folks.
and for those I suggest the trivial change -I to turn off checksumming.
The Following User Says Thank You to Corona688 For This Useful Post:
drl (09-22-2017)
Sponsored Links
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Linux More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
Running a C/C++ program and/or bash script from a server frad UNIX for Dummies Questions & Answers 1 12-13-2013 12:50 PM
Putting bash script in C program SkySmart Programming 15 03-06-2013 06:23 PM
bash script to copy files dodgefan67 Shell Programming and Scripting 8 04-07-2011 12:40 AM
Run Program from Bash CGI-Script JMooney5115 Homework & Coursework Questions 8 01-27-2011 07:51 PM
Making a progress gauge in a bash script ccox85 Shell Programming and Scripting 1 01-31-2008 02:16 PM



All times are GMT -4. The time now is 01:48 PM.