Copy duration of cp Post: 302562771

Sponsored Content

Top Forums UNIX for Dummies Questions & Answers Copy duration of cp Post 302562771 by drl on Sunday 9th of October 2011 07:10:04 AM

10-09-2011

Registered User

Hi.

This is a mechanism for showing copy progress. It is a Bourne shell script, intended to be portable: Theiling Online: ASCII bar the author calls it small, the version that I have used is almost 700 lines long.

For a c code, there is pv: ivarch.com: Pipe Viewer along with a note about an OpenBSD port, and an informative blog at A Unix Utility You Should Know About: Pipe Viewer - good coders code, great reuse

Best wishes ... cheers, drl

---------- Post updated Oct 9th, 2011 at 06:10 ---------- Previous update was Oct 8th, 2011 at 06:31 ----------

Hi.

In re-visiting the web page for pv, it struck me that I could use it in a current application.

I keep several virtual machines in a large machine -- a "VM server" as it might be called. I often take snapshots with the VMWare server. However, at longer intervals I want to save the state of each VM. For that, the VM is off-powered, and I run a shell script to tar the files that VMWare creates and gzip the resulting tar file. This often takes more than 10 minutes, so I find it useful (and comforting) to see the progress.

The output looks like this after the tar file for distribution GNU/Linux mint has been created and compressed:

Code:

 Working on creating tar-gzip image for vm-mint, 4.63 GB, ( 4976613260 )
      tar: 4.63GB 0:10:49 [7.31MB/s] [======================>] 100%            
     gzip: 1.75GB 0:10:49 [2.76MB/s] [                                     <=>]

the size of the tar file was estimated with du, and 2 instances of pv were used, 1 for the tar, which produces the first progress line. The size of the data for the second process cannot be estimated, so in place of a completion bar, the symbols "<=>" are simply moved back and forth within the brackets to show activity.

The pv code is quite clever, and uses cursor position commands to move back and forth between the 2 one-line displays for updating. The first line includes measures for amount copied, time used, rate of transfer, completion bar, and ETA.

The relevant small bit of script that gets this accomplished is:

Code:

  size=$( du -sb $machine | awk '{print $1}' )
  gbytes=$( echo "scale=2;$size / (1024*1024*1024)" | bc )
  echo " Working on creating tar-gzip image for $machine, $gbytes GB, ( $size )"
  tar cvf - $machine 2>>tar.log |
  pv -cN tar -s "$size" |
  gzip |
  pv -cN gzip > $BASE/${machine}.tar.gz

I'm impressed with the thought that went into the design of pv. In my system, pv was in the repository (Debian).

Good luck ... cheers, drl

Last edited by drl; 10-08-2011 at 08:44 AM..

drl

View Public Profile for drl

Find all posts by drl

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

duration calculation

I have a file which has 3 coloumns emp_name, Joining_date, Designation. abc 12/1/2001 SSE def 2/25/2007 SE ghi 3/18/2009 SA abc 8/1/2008 SSE def 2/13/2007 SE ghi 3/24/2005 SA I need to find out the emp who has been in the company for longest period(Till date). Can I have any...

2. Solaris

ufsdump backup duration

hi, i'm trying to figure out how to tell the amount of time a ufsdump of a directory takes. i use the below command: echo "Starting Backup of u4" >> /backup/backup.log 2>&1 /usr/sbin/ufsdump 0uf /dev/rmt/0n /u4 >> /backup/backup.log 2>&1 echo "Finished Backup of u4" >> /backup/backup.log...

3. Shell Programming and Scripting

Convert duration of the process to seconds

Hi, I am looking to write a script to kill the process which are running for more than 7 days. So i have a command like "ps -eo pid,etime,args | grep -i xxxx" ( process which has xxx in it and running for more than 7 days needs to be killed ). When i exeucte the above command , i am...

4. UNIX Desktop Questions & Answers

arecord not interrupted after specified duration

I have used the arecord command like this arecord -d 1 test.wav It is keep on waiting. I need to manually interrupt it by ctrl-c. Why it is not interrupting after one second? The arecord version which I am using is : arecord: version 1.0.23 by Jaroslav Kysela

5. Shell Programming and Scripting

Sort by Duration

.......................................................................................................................... 03:40 Geonetric File from CCL Complete 03:40:59 03:41:08 00:00:09 00:00:01 N/A 005 sys_runccl ...

6. Shell Programming and Scripting

Perl ::duration of time in between dates

Hello All, I have two strings with date and time as follows.. $starttime= "06/11/2013 "; $starttime= "05:15"; $enddate="06/12/2013"; $endtime="04:45"; dates are in mm/dd/yyyy format and time in military format. and I am looking the duration of time(in minutes) in between dates. ...

7. UNIX for Beginners Questions & Answers

Process duration

Hi , How can I check that for a single process, for example pagent for how much duration this process was up or down and also I need multiple entries if this process was down or up multiple times. Please help.

8. Shell Programming and Scripting

Need help in Inix script for finding duration

I have a file with time in it. I need to find the duration between the timestamp by subtracting second row from third row and so on. and wherever it is more than 30 minutes it should display start and end time which have been subtracted file : 00:44:11 00:44:11 00:44:13 00:44:13 00:46:51...

9. Shell Programming and Scripting

Duration Calculation

I have 2 variables startTime='122717 23:20' endTime='122817 0:40' how can i get the elapsed duration as like "1 hour 20 minutes" ?

10. Shell Programming and Scripting

Function to get the duration of all videos in a folder(s)

did this function to generate the duration of all the video files in a folder or multiple folders, it works fine for my use (I am no Guru as you may have noticed) but when I give it a lot of folders the calculation get a bit borked. If any good soul had the energy to look at it and give...

LEARN ABOUT SUSE

virt-tar-out

virt-tar-out(1) 					      Virtualization Support						   virt-tar-out(1)

NAME

       virt-tar-out - Pack a virtual machine disk image directory into a tarball.

SYNOPSIS

	virt-tar-out -a disk.img /dir files.tar

	virt-tar-out -d domain /dir files.tar

	virt-tar-out -d domain /dir - | gzip --best > files.tar.gz

DESCRIPTION

       "virt-tar-out" packs a virtual machine disk image directory into a tarball.

       The first parameter is the absolute path of the virtual machine directory.  The second parameter is the tar file to write.  Use "-" to
       write to standard output.

EXAMPLES

       Download the home directories from a guest:

	virt-tar-out -d MyGuest /home - | gzip --best > homes.tar.gz

JUST A SHELL SCRIPT WRAPPER AROUND GUESTFISH

       This command is just a simple shell script wrapper around the guestfish(1) "tar-out" command.  For anything more complex than a trivial
       copy, you are probably better off using guestfish directly.

OPTIONS

       Since the shell script just passes options straight to guestfish, read guestfish(1) to see the full list of options.

SEE ALSO

       guestfish(1), virt-cat(1), virt-copy-in(1), virt-copy-out(1), virt-edit(1), virt-tar-in(1), <http://libguestfs.org/>.

AUTHORS

       Richard W.M. Jones ("rjones at redhat dot com")

COPYRIGHT

       Copyright (C) 2011 Red Hat Inc.	<http://libguestfs.org/>

       This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
       the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

       This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
       MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.

       You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation,
       Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.

libguestfs-1.18.1						    2013-12-07							   virt-tar-out(1)

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

duration calculation

Discussion started by: siba.s.nayak

2. Solaris

ufsdump backup duration

Discussion started by: pinoy43v3r

3. Shell Programming and Scripting

Convert duration of the process to seconds

Discussion started by: forums123456

4. UNIX Desktop Questions & Answers

arecord not interrupted after specified duration

Discussion started by: thillai_selvan