Sponsored Content
Top Forums Programming Program or bash script to see total progress of copy Post 303003852 by drl on Thursday 21st of September 2017 01:25:38 PM
Old 09-21-2017
Hi.
Quote:
Originally Posted by RudiC
The requestor doesn't seem to read (or try (or, at least, comment on)) ALL contributions...
I assume that this thread may be useful to others.

A set of numbers from thread linux - Copying a large directory tree locally? cp or rsync? - Server Fault
Code:
To move 532Gb of data distributed among 1,753,200 files we had those times:
    rsync took 232 minutes
    tar took 206 minutes
    cpio took 225 minutes
    rsync + parallel took 209 minutes

I wish they had added cp to the benchmark, but the theme of the thread is comparing speed, with a sub-theme of monitoring, and a few other characteristics, like re-start-ability.

So, as regards this thread, use something that does have a progress mechanism built-in or roll-your-own.

Best wishes ... cheers, drl

Last edited by drl; 09-21-2017 at 02:40 PM..
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Making a progress gauge in a bash script

Hello once again: One thing that seems to be a nice feature is a progress gauge... so I can see how long an operation will take for a task to complete if it is requiring a lot of processing or the file is enormous. I have seen references to gauge operations, but I don't know anything about it or... (1 Reply)
Discussion started by: ccox85
1 Replies

2. Homework & Coursework Questions

Run Program from Bash CGI-Script

Use and complete the template provided. The entire template must be completed. If you don't, your post may be deleted! 1. The problem statement, all variables and given/known data: This is a problem I am having with my 2 semester senior project. I have a LAMP server running Ubuntu 9.10 with... (8 Replies)
Discussion started by: JMooney5115
8 Replies

3. Shell Programming and Scripting

bash script to copy files

hey everyone, new here i have arch setup and i am using smbnetfs to mount some windows shares in /mnt/smbnet what i want to do is copy files from my home dir to a dir in /mnt/smbnet but i also need it to remove files if i have deleted them from my home dir seems that cp would be the... (8 Replies)
Discussion started by: dodgefan67
8 Replies

4. Shell Programming and Scripting

Problem with bash shell script program

Hi, This is my program. #!/bin/bash today=`date +"%b-%d-%Y"` SERVICE="pbxconnect.php" if ; then echo "pbx program is running" else nohup php pbxconnect.php > logpbx-$today.txt & fi On executing using "sh myprogram.sh" , i get the following error. myprogram.sh: line 4: ' My... (7 Replies)
Discussion started by: gskumar1234
7 Replies

5. Shell Programming and Scripting

bash script to check if a program is running

I'm a bit new to bash programming and I was assigned the job of writing a script that will check to see if a program server is running and to restart the program if it is not up. The script is supposed to check the program every hour (which I have looked up and I believe I know how to do) and send... (3 Replies)
Discussion started by: mcknz
3 Replies

6. UNIX for Dummies Questions & Answers

Bash script to execute a program to rename files

I just can't figure it out , so please just give me a pice of advise how to: The existing Linux program foo2bar takes as its only argument the name of a single foo file and converts it to an appropriately-named bar file. Provide a script that when executed will run foo2bar against all foo... (4 Replies)
Discussion started by: raymen
4 Replies

7. Shell Programming and Scripting

Bash script to start program and answer prompts?

I'm trying to write a script the simplifies the execution of a program: After starting the program (sh ~/.mfix/model/make_mfix) I am prompted four times for options: Do you need SMP version? (y/n) Do you need DMP version? (y/n) Do you need debug version? (y/n) Force re-compilation of... (2 Replies)
Discussion started by: lanew
2 Replies

8. Programming

Putting bash script in C program

suppose i have a bash script: #!/bin/bash echo "hello" echo "how are you" echo "today" how can i put the entire script above into a basic c program? i do not want to translate the bash code to a c code. i want C to run the bash code. is this possible? i found this on the... (15 Replies)
Discussion started by: SkySmart
15 Replies

9. UNIX for Dummies Questions & Answers

Running a C/C++ program and/or bash script from a server

I wish to be able to give to a client the opportunity to : 0) Turn one of my ubuntu computers into a webserver 1) See a webpage after visiting a url where an external user/client can set a couple of variables (e.g. Number1= ?, Number2=?) 2) By pressing "run" the program runs on my machine 3)... (1 Reply)
Discussion started by: frad
1 Replies

10. UNIX for Beginners Questions & Answers

Bash script to get total size off of remainder calculated

I am working on a script to get the final total size and so far have the following and wondering if this can be improved. # Compare the desired size of each lvm to the standard size. If it is desired is larger than calculate the difference and keep that value as the amount to add to that LVM. ... (5 Replies)
Discussion started by: user3528
5 Replies
SVN::Notify::Mirror::Rsync(3pm) 			User Contributed Perl Documentation			   SVN::Notify::Mirror::Rsync(3pm)

NAME
SVN::Notify::Mirror::Rsync - Mirror a repository path via Rsync SYNOPSIS
Use svnnotify in post-commit: svnnotify --repos-path "$1" --revision "$2" --handler Mirror::Rsync --to "/path/to/local/htdocs" [--svn-binary /full/path/to/svn] --rsync-host remote_server [--rsync-delete=[yes|no]] [--rsync-dest "/path/on/remote/server"] [--rsync-args arg1 [--rsync-args arg2...]] [[--rsync-ssh] [--ssh-user remote_user] [--ssh-identity /home/user/.ssh/id_rsa]] or better yet, use SVN::Notify::Config for a more sophisticated setup: #!/usr/bin/perl -MSVN::Notify::Config=$0 --- #YAML:1.0 '': PATH: "/usr/bin:/usr/local/bin" 'path/in/repository': handler: Mirror to: "/path/to/www/htdocs" 'some/other/path/in/repository': handler: Mirror to: "/path/to/local/www/htdocs" rsync-host: "remote_host" rsync-dest: "/path/on/remote/www/htdocs" ssh-user: "remote_user" ssh-identity: "/home/user/.ssh/id_rsa" DESCRIPTION
Keep a directory in sync with a portion of a Subversion repository. Typically used to keep a development web server in sync with the changes made to the repository. This directory can either be on the same box as the repository itself, or it can be remote (via SSH connection). USAGE
Depending on whether the target is a "Local Mirror" or a Remote Mirror, there are different options available. All options are available either as a commandline option to svnnotify or as a hash key in SVN::Notify::Config (see their respective documentation for more details). Working Copy on Local host Because 'svn export' is not able to be consistently updated, the local rsync'd directory must be a full working copy. The remote server will only contain the ordinary files (no Subversion admin files). The files in the working copy must be writeable (preferrably owned) by the user identity executing the hook script (this is the user identity that is running Apache or svnserve respectively). Local Mirror Please see " SVN::Notify::Mirror " for details. Remote Mirror Used for directories not located on the same machine as the repository itself. Typically, this might be a production web server located in a DMZ, so special consideration must be paid to security concerns. In particular, the remote mirror server may not be able to directly access the repository box. o rsync-host This value is required and must be the hostname or IP address of the remote host (where the mirror directories reside). o rsync-delete The default mode of operation is to delete remote files which are not present in the local working copy. NOTE: this will delete any unversioned files in the remote directory tree. Unless you have all of your files under version control, you should pass the "--no-rsync-delete" or "--rsync-delete no" option. o rsync-dest This optional value specifies the path to update on the remote host. If you do not specify this value, the same path as passed in as the "--to" parameter will be used (this may not be what you meant to do). o rsync-args This optional parameter can be used to pass additional commandline options to the rsync command. You can use this multiple times in order to pass multiple options. The default args are "--archive --compress". See the "rsync-ssh" options for using SSH instead of RSH (rather than pass those commands via "--rsync-args" o rsync-ssh This optional parameter signals that you wish to use SSH instead of whatever the default remote shell program is configured in your copy of rsync. You may need to set one or more of the "ssh-*" parameters as well. o ssh-user If the remote user is different than the local user executing the postcommit script, you can specify it with this parameter. You would often use this in conjunction with the next parameter. o ssh-identity This value may be optional and should be the full path to the local identity file being used to authenticate with the remote host. If you are setting the ssh-user to be something other than the local user name, you will typically also have to set the ssh-identity. AUTHOR
John Peacock <jpeacock@cpan.org> COPYRIGHT
Copyright (c) 2005-2008 John Peacock This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself. The full text of the license can be found in the LICENSE file included with this module. SEE ALSO
SVN::Notify, SVN::Notify::Config, SVN::Notify::Mirror perl v5.14.2 2012-07-04 SVN::Notify::Mirror::Rsync(3pm)
All times are GMT -4. The time now is 12:12 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy