Sponsored Content
Top Forums Shell Programming and Scripting Start copying large file while its still being restored from tape Post 302538728 by swamik on Thursday 14th of July 2011 04:21:19 AM
Old 07-14-2011
Thanks for the reply - sounds like quite a bit of a hack to get it to work though. Does anyone know if it's possible to compile mbuffer for Mac OS X. Would be a much nicer solution....
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

copying a large filesystem

Hi there In my organisation we have a solaris network with /home being automounted from /export/home on a central file server (usual stuff) however, the guy who originally set this up only allocated 3gb to /export/home and now we are really struggling for space. I have a new 18gb disk installed... (3 Replies)
Discussion started by: hcclnoodles
3 Replies

2. Filesystems, Disks and Memory

Strange difference in file size when copying LARGE file..

Hi, Im trying to take a database backup. one of the files is 26 GB. I am using cp -pr to create a backup copy of the database. after the copying is complete, if i do du -hrs on the folders i saw a difference of 2GB. The weird fact is that the BACKUP folder was 2 GB more than the original one! ... (1 Reply)
Discussion started by: 0ktalmagik
1 Replies

3. UNIX for Dummies Questions & Answers

Writing large files to tape

I have a zipped file that is ~ 10GB. I tried tarring it off to a tape, but I receive: tar: <filename> too large to archive. Use E function modifier. The file is stored on a UFS mount, so I was unable to use ufsdump. What other options do I have? (I don't have a local file system large... (3 Replies)
Discussion started by: FredSmith
3 Replies

4. AIX

Copying to tape drive throws error

Hi All I am trying to copy files present in a partition (server 2) which is mounted to a different server (server 1) as tape drive is connected to it. I ran the below command to copy files within a partition: svr01:root:/sunfileserver> tar -cvf * a <foldername>/<filename>/<filename> a... (4 Replies)
Discussion started by: vathsan
4 Replies

5. UNIX for Dummies Questions & Answers

Copying large file problem on SVR4 Unix

We have 3 Unix servers all running SVR4 Unix 1.4. I have no problems copying files to and from 2 of the servers using either the rcp command or ftp but when i come to transfer large files to the third server the copy gives up part way through and crashes this server. Copying smaller files using RCP... (7 Replies)
Discussion started by: coatesd
7 Replies

6. Shell Programming and Scripting

Copying of large files fail

Hi, I have a process which duplicates files for different environments. As the files arrive, my script (korn shell) makes copies of them (giving a unique name) and then renames the original file so that my process won't get triggered again. I don't like it either, but it's what we were told to... (4 Replies)
Discussion started by: GoldenEye4ever
4 Replies

7. UNIX for Dummies Questions & Answers

Copying a Large File

I have a large file that I append entries to the end of every few seconds. Its grown to >150MB. Its basically a log file but a perl script is writing to it. I need to make a copy of it to a new directory. I realize the latest entries occuring while the copy is taking place will not be recorded... (1 Reply)
Discussion started by: lforum
1 Replies

8. Shell Programming and Scripting

Copying number by looking a large file

Hi All, I have a big file which looks like this: abc 34.32 cdf 343.45 computer 1.34 ladder 2.3422 I have some 100000 .TXT files which look like this: computer cdf align I have to open each of the text files and read the words from the text files. Then I have to look into that... (2 Replies)
Discussion started by: shoaibjameel123
2 Replies

9. UNIX for Dummies Questions & Answers

Copying tape-to-tape on UNIX

I am using a 4mm tape to backup my Unix system. However, I wanted to make a copy all of the files and archive headers (or just the archive headers if that's possible) created on one of my tapes to another 4mm tape. I only have one tape drive. Is there a command that will complete such task? ... (1 Reply)
Discussion started by: acoco
1 Replies

10. Shell Programming and Scripting

Copying large files in a bash script stops execution

Hello, I'm new to this forum and like to first of all say hello to everyone. I've got a really annoying problem at the moment. I'm trying to rsync some files (about 200MB with one file of 120MB) from a Raspberry PI with raspbian to a debian server via rsync. This procedure is stored in a... (3 Replies)
Discussion started by: wex_storm
3 Replies
mbuffer(1)							  console utility							mbuffer(1)

NAME
mbuffer - measuring buffer SYNTAX
mbuffer [options] DESCRIPTION
mbuffer buffers I/O operations and displays the throughput rate. It is multi-threaded, supports network connections, and offers more options than the standard buffer. OPTIONS
-i <filename> Use filename as input instead of the standard input (needs to be given for multi volume support). If filename is -, input is read from standard input. -I <port> Use network port port as input instead of the standard input. If given a hostname and a port in the form hostname:port, the first interface with the IP of hostname will be used. -o <filename> Use filename as output instead of the standard output (needs to be given for multi volume support, will enable use of sendfile if available). If filename is -, output is written to standard output. The option -o can be passed multiple times to specify multiple outputs. -O <hostname:port> Write output to hostname:port instead of the standard output (will enable use of sendfile if available). This option can be used multiple times to send data to multiple machines. -b <num> Use num blocks for buffer (default 256). -s <size> Use blocks of size bytes for buffer (default pagesize of system). -m <size> Use a total of size bytes for buffer (default 2MB) - size can be set with a trailing character (b and B for Byte, k for kByte, M for MByte, G for Gigabyte, and with % for a percentage of total physical memory). -L Lock buffer in memory - this option is not available for file-based buffers and requires mbuffer to be set-UID root (use with care). -n <num> num volumes in input device (requires use of option -i for input device specification) [currently multi volume support is EXPERIMEN- TAL] -t use a memory-mapped temporary file as buffer (use with huge buffers) -T <file> as -t but use file instead -d use block-size of device for output (needed for some devices, slows output down) -D <size> assume an output volume of size bytes (default infinite) after which a volume change will be initiated. Small values are useful for the timely testing of multi-volume runs; accurate values if your device doesn't properly signal end of media. Size can be set with a trailing character (b and B for Byte, k for kByte, M for MByte, or G for Gigabyte) -P <num> start writing after the buffer has been filled to num% (default 0 - start at once) -p <num> start reading after the buffer has dropped below fill-ratio of num% (default 100 - start at once) -l <file> log messages to file instead of standard error output -u <num> pause num microseconds after each write - might increase performance on some drives with very low performance (< 1 MB/sec) -r <rate> Set the maximum read rate to <rate>. <rate> can be given in either Bytes, kBytes, MBytes, or GBytes per second. To do so, use an appropriate suffix (i.e. k,M,G). This option is useful if you have a tape that is capable of transferring data faster than the host can handle it. In this case you can use this option to limit the transfer rate and keep the tape running. Be aware that this is both good for your tape drive, and enhances overall performance, by avoiding tape screwing. -R <rate> Same as above only for setting the transfer limit for the writer. -A <cmd> the device used is an autoloader which uses cmd to load the next volume -a <time> the device used is an autoloader which takes time seconds to load a new tape -f overwrite output file if it exists already -c write with synchronous data integrity support - This option forces all writes to complete before continuing. This enables errors to be reported earlier and more precisely, but might decrease performance. Especially systems with high level of data integrity support suffer a huge performance hit. Others might seem to be unaffected, but just neglect support for full synchronous data integrity. -v <num> set verbose level to num. Valid values are 0..6 (0 = none, 1 = errors, 2 = warnings, 4 = information messages, 5 = debugging mes- sages, 6 = I/O debugging). Higher values include lower values messages. -q quiet - do not display the status on the standard error output --direct Use O_DIRECT to open file descriptors. This option is not available on all systems. It tells the OS to bypass the page cache to improve performance when reading and writing. On Solaris this is an auto-magic option that is enabled if it is supported for the relevant file. Be aware that this option might lead to read/write failures, if the buffer isn't properly aligned for direct I/O. Additionally, open might fail with EINVAL (i.e. invalid argument) if the named file does not support O_DIRECT. -6 Force IPv6 mode for the following network I/O options on command line. -4 Force IPv4 mode for the following network I/O options on command line. -0 Choose IPv4/IPv6 mode on demand. -h, --help Output help information and exit. -H, --md5 Generate a MD5 hash of transferred data. -V, --version Output version information and exit. ENVIRONMENT VARIABLES
If TMPDIR is set, mbuffer allocates storage for file-based buffers in this directory. If TMPDIR is unset, /var/tmp will be used. FILES
/usr/bin/mbuffer /var/tmp/mbuffer-* EXAMPLES
To run this program with the default options just type: mbuffer Using mbuffer to do a backup with tar to the default tape device. Options for this example: memory-mapped temporary file with a size of 10 Megabytes, start after 80% of the buffer have been filled. tar cf - mydirectory | gzip | mbuffer -t -m 10M -P 80 -f -o $TAPE Using mbuffer with 3 tapes for input and extracting the contents in the current work directory: mbuffer -n 3 -i $TAPE | gzip -dc | tar xf - Using mbuffer to write to multiple tape volumes: tar cf - /usr | mbuffer -f -o $TAPE Write to multiple tapes and erase every tape before writing: tar cf - /usr | mbuffer -A "echo next tape; read a < /dev/tty; mt erase $TAPE" -f -o $TAPE Making a backup via network: tape server: mbuffer -I 8000 -f -o $TAPE backup client: tar zcf - /home | mbuffer -O tapeserver:8000 Distributing a directory tree to multiple machines: master: tar cf - /tree_to_clone | mbuffer -O clone0:8000 -O clone1:8000 clones: mbuffer -I master:8000 | tar xf - EXITCODE
mbuffer return 0 upon success. Any kind of failure will yield a non-zero exit code. AUTHORS
Thomas Maier-Komor <thomas@maier-komor.de> DONATIONS
If you like this software, and use it for production purposes in your company, please consider making a donation to support this work. You can donate directly via PayPal to the author's e-mail address (thomas@maier-komor.de). HOMEPAGE
http://www.maier-komor.de/mbuffer.html LICENSE
This software is published under GNU General Public License V3. See file LICENSE for details. SEE ALSO
buffer(1) Thomas Maier-Komor 20110119 mbuffer(1)
All times are GMT -4. The time now is 08:57 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy