can speed things up. More than one at once is good, too, to keep thoses queues full all down the line. A lot of UNIX is single-buffered. This uses my simple line demus to spread the load to as many as 8 streams (since it is a dumb rotation, a big file may block one stream for a while, so adding a few extras beyond net saturation is good):
Code:
find . . . | dumux 8 "cpio -oaH crc | gzip -9 | rsh that_host 'gunzip|cpio -idmH crc' "
$ cat mysrc/dumux.c
#include <stdio.h>
#include <stdlib.h>
#include <strings.h>
static void usage(){
fputs(
"\n"
"Usage: dumux <ct> <cmd> [ -l <line_ct> ]\n"
"\n"
"Runs <ct> copies of <cmd> and sends <line_ct> (default 1) lines to each\n"
"in rotation.\n"
"\n",
stderr );
exit( 1 );
}
int main( int argc, char ** argv ){
FILE **fp = NULL ;
int i, x, c, l, lct = 1 ;
if ( argc < 3
|| 2 > ( x = atoi( argv[1] ))){
usage();
}
if ( argc > 3
&& ( argc != 5
|| strcmp( argv[3], "-l" )
|| 1 > ( lct = atoi( argv[4] )))){
usage();
}
if ( !( fp = (FILE **)calloc( x, sizeof (FILE *)))){
perror( "malloc" );
exit( 2 );
}
for ( i = 0 ; i < x ; i++ ){
if ( !( fp[i] = popen( argv[2], "w" ))){
perror( "popen( $2 )" );
exit( 3 );
}
}
i = l = 0 ;
while ( EOF != ( c = getchar())){
if ( EOF == putc( c, fp[i] )){
perror( "putc( popen( $2 ))" );
exit( 4 );
}
if ( c == '\n'
&& ++l == lct ){
l = 0 ;
if ( ++i == x ){
i = 0 ;
}
}
}
if ( ferror( stdin )){
perror( "stdin" );
exit( 5 );
}
for ( i = 0 ; i < x ; i++ ){
if ( 0 > fclose( fp[i] )){
perror( "fclose( popen( $2 ))" );
}
}
exit( 0 );
}
Well, 270 gb takes a while to write, no free lunch. You could leave it compressed and write less, if you can figure how to access it on the other side -- zip files suggest themselves, being explorable, windows having compressed folders and with files clickable to open.
cp -rp to a mount would do a cpio pass for you. If the mount is over the network, it would be even slower!
With SAMBA, you can leave the data in place and let them access it on UNIX.
Solaris has a neat ClientFS that is nfs with local caching on hard disk. You start off a little slow, but then all the files re-accessed are on the local hard drive. Ditto for local mods flowing back to the NFS volume. They do not recommend high writes, but it looks like you could turn off updates during the day and then turn them on later for bulk updating.
Plan9 OS from AT&T Belll Labs had hierarchical storage, where every hard drive was a cache for the optical WORM. Never had to ask people to clean their dirs again.
Hi All,
I have dual boot on my laptop - Win8 And linuxMint.
From linuxmint i had copied all my data found on the NTFS partitions (on my laptop) to an
external hard drive (formatted with ext3). i used rsync for this.
Now after my hard disk crashed, am restoring the data back from ext3... (2 Replies)
I have WinXP Pro SP2 on first disk /dev/hda, which my BIOS currently has as first boot device.
I have CentOS-5.2-x86_64 on second disk /dev/hdb, which I can boot into if I set my BIOS to boot from this HDD. It is using LVM with vg00 and a single LV for root (/) filesystem (ext3).
How do I get... (19 Replies)
I have a question about transferring a directory to the toolman. I have a directory called assn3 that contains two txt files and one empty directory and I want to transfer the assn3 to my my tooman account. Every time I try to transfer the file it says that the assn3 is not a regular file! what... (3 Replies)
I'm having problem mounting ext3 & ntfs partitions on my PC-BSD OS.
Can anyone please help me out here.
What are the changes required to be done in fstab??
Are there any patches to be installed?? (1 Reply)
Below is a list of the file system on my Sun system. How can I transfer more disk space from the "/space" partition to the "/" partition with out rebuilding?
/ /dev/dsk/c0t0d0 6191949 5736718 393312 94%
/proc /proc 0 0 0 0%
/dev/fd fd 0 0 0 0%
/etc/mntta ... (10 Replies)
My oracle database is generating archive logs. I want to copy those archive logs over to backup server on a regular basis. I know how to create an ftp job and I can put it in my crontab. My problem is that I don't know how to send the files just once instead of sends all the files in the... (4 Replies)
Hi all.
1. How can i copy files from one unix system to another. should i use ftp? so How?
2. How can i create an archive whose extention is tar.gz? and how can i decompress them later?
3. WHat is RPM ? what does it stands for?
Thanks (5 Replies)
Dear Experts
I am using a workstatin with two screens and its OS is solaris 8. I wish to trnsfer some application windows from one screen to another one without need to closing the window and open it from other screen.
Please advise
Bests
Reza (3 Replies)
