Program or bash script to see total progress of copy


 
Thread Tools Search this Thread
Top Forums Programming Program or bash script to see total progress of copy
# 15  
Old 09-21-2017
The requestor doesn't seem to read (or try (or, at least, comment on)) ALL contributions...
# 16  
Old 09-21-2017
Hi.
Quote:
Originally Posted by RudiC
The requestor doesn't seem to read (or try (or, at least, comment on)) ALL contributions...
I assume that this thread may be useful to others.

A set of numbers from thread linux - Copying a large directory tree locally? cp or rsync? - Server Fault
Code:
To move 532Gb of data distributed among 1,753,200 files we had those times:
    rsync took 232 minutes
    tar took 206 minutes
    cpio took 225 minutes
    rsync + parallel took 209 minutes

I wish they had added cp to the benchmark, but the theme of the thread is comparing speed, with a sub-theme of monitoring, and a few other characteristics, like re-start-ability.

So, as regards this thread, use something that does have a progress mechanism built-in or roll-your-own.

Best wishes ... cheers, drl

Last edited by drl; 09-21-2017 at 02:40 PM..
# 17  
Old 09-21-2017
Of course rsync is slowest when you don't turn off checksumming.

That chart does help prove one of my biggest peeves though - "you can't multithread a hard drive". None of the options are significantly faster.
# 18  
Old 09-21-2017
Hi.
Quote:
Originally Posted by Corona688
... None of the options are significantly faster.
Well, 9% return, 209/232 -> 0.90, on a trivial change might be worthwhile to some folks.

Researchers at the center where I worked would moil to get even a 1% speed-up. Of course, that was most often with grad student labor Smilie

Best wishes ... cheers, drl
# 19  
Old 09-22-2017
sorted it,

Code:
yum remove rsync

installed it by untarring it cd'd into the dir and running
Code:
./configure.sh
make
make install


and now i get the result i wanted -

Code:
[root@robw-linux data]# rsync -a --info=progress2 call_the_midwife_7_1708/ new/
 14,874,971,690  44%   27.58MB/s    0:10:48  xfr#16, to-chk=2/143)


Moderator's Comments:
Mod Comment Please start using code tags, thanks.

Last edited by Neo; 12-07-2017 at 01:32 PM..
# 20  
Old 09-22-2017
Why did you do all that when you appear to have had it already? Did the version you have not contain --progress?
# 21  
Old 09-22-2017
Quote:
Originally Posted by drl
Well, 9% return, 209/232 -> 0.90, on a trivial change might be worthwhile to some folks.
and for those I suggest the trivial change -I to turn off checksumming.
This User Gave Thanks to Corona688 For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Bash script to get total size off of remainder calculated

I am working on a script to get the final total size and so far have the following and wondering if this can be improved. # Compare the desired size of each lvm to the standard size. If it is desired is larger than calculate the difference and keep that value as the amount to add to that LVM. ... (5 Replies)
Discussion started by: user3528
5 Replies

2. UNIX for Dummies Questions & Answers

Running a C/C++ program and/or bash script from a server

I wish to be able to give to a client the opportunity to : 0) Turn one of my ubuntu computers into a webserver 1) See a webpage after visiting a url where an external user/client can set a couple of variables (e.g. Number1= ?, Number2=?) 2) By pressing "run" the program runs on my machine 3)... (1 Reply)
Discussion started by: frad
1 Replies

3. Programming

Putting bash script in C program

suppose i have a bash script: #!/bin/bash echo "hello" echo "how are you" echo "today" how can i put the entire script above into a basic c program? i do not want to translate the bash code to a c code. i want C to run the bash code. is this possible? i found this on the... (15 Replies)
Discussion started by: SkySmart
15 Replies

4. Shell Programming and Scripting

Bash script to start program and answer prompts?

I'm trying to write a script the simplifies the execution of a program: After starting the program (sh ~/.mfix/model/make_mfix) I am prompted four times for options: Do you need SMP version? (y/n) Do you need DMP version? (y/n) Do you need debug version? (y/n) Force re-compilation of... (2 Replies)
Discussion started by: lanew
2 Replies

5. UNIX for Dummies Questions & Answers

Bash script to execute a program to rename files

I just can't figure it out , so please just give me a pice of advise how to: The existing Linux program foo2bar takes as its only argument the name of a single foo file and converts it to an appropriately-named bar file. Provide a script that when executed will run foo2bar against all foo... (4 Replies)
Discussion started by: raymen
4 Replies

6. Shell Programming and Scripting

bash script to check if a program is running

I'm a bit new to bash programming and I was assigned the job of writing a script that will check to see if a program server is running and to restart the program if it is not up. The script is supposed to check the program every hour (which I have looked up and I believe I know how to do) and send... (3 Replies)
Discussion started by: mcknz
3 Replies

7. Shell Programming and Scripting

Problem with bash shell script program

Hi, This is my program. #!/bin/bash today=`date +"%b-%d-%Y"` SERVICE="pbxconnect.php" if ; then echo "pbx program is running" else nohup php pbxconnect.php > logpbx-$today.txt & fi On executing using "sh myprogram.sh" , i get the following error. myprogram.sh: line 4: ' My... (7 Replies)
Discussion started by: gskumar1234
7 Replies

8. Shell Programming and Scripting

bash script to copy files

hey everyone, new here i have arch setup and i am using smbnetfs to mount some windows shares in /mnt/smbnet what i want to do is copy files from my home dir to a dir in /mnt/smbnet but i also need it to remove files if i have deleted them from my home dir seems that cp would be the... (8 Replies)
Discussion started by: dodgefan67
8 Replies

9. Homework & Coursework Questions

Run Program from Bash CGI-Script

Use and complete the template provided. The entire template must be completed. If you don't, your post may be deleted! 1. The problem statement, all variables and given/known data: This is a problem I am having with my 2 semester senior project. I have a LAMP server running Ubuntu 9.10 with... (8 Replies)
Discussion started by: JMooney5115
8 Replies

10. Shell Programming and Scripting

Making a progress gauge in a bash script

Hello once again: One thing that seems to be a nice feature is a progress gauge... so I can see how long an operation will take for a task to complete if it is requiring a lot of processing or the file is enormous. I have seen references to gauge operations, but I don't know anything about it or... (1 Reply)
Discussion started by: ccox85
1 Replies
Login or Register to Ask a Question