Ext3 to NTFS - transfering data


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Ext3 to NTFS - transfering data
# 8  
Old 11-08-2010
Glad to know that post was useful to you!!!
# 9  
Old 11-08-2010
Quote:
Originally Posted by huntreilly25
However, takes forever because, after all, I am trying to transfer around 270gb.
If I were to pipe through gzip and then gunzip...
That would take even longer. It'd still have to read and write all the data, plus, it would have to compress and extract it too!
# 10  
Old 11-08-2010
Quote:
Originally Posted by Corona688
That would take even longer. It'd still have to read and write all the data, plus, it would have to compress and extract it too!
Well, it is pipeline parallel. However, if the network is a bottleneck, then something like
Code:
find . . . | cpio -oaH crc | gzip -9 | rsh that_host 'gunzip|cpio -idmH crc'

can speed things up. More than one at once is good, too, to keep thoses queues full all down the line. A lot of UNIX is single-buffered. This uses my simple line demus to spread the load to as many as 8 streams (since it is a dumb rotation, a big file may block one stream for a while, so adding a few extras beyond net saturation is good):
Code:
find . . . | dumux 8 "cpio -oaH crc | gzip -9 | rsh that_host 'gunzip|cpio -idmH crc' "
 
$ cat mysrc/dumux.c
 
#include <stdio.h>
#include <stdlib.h>
#include <strings.h>
static void usage(){
        fputs(
"\n"
"Usage: dumux <ct> <cmd> [ -l <line_ct> ]\n"
"\n"
"Runs <ct> copies of <cmd> and sends <line_ct> (default 1) lines to each\n"
"in rotation.\n"
"\n",
                stderr );
        exit( 1 );
 }
int main( int argc, char ** argv ){
        FILE **fp = NULL ;
        int i, x, c, l, lct = 1 ;
        if ( argc < 3
          || 2 > ( x = atoi( argv[1] ))){
                usage();
         }
        if ( argc > 3
          && ( argc != 5
            || strcmp( argv[3], "-l" )
            || 1 > ( lct = atoi( argv[4] )))){
                usage();
         }
        if ( !( fp = (FILE **)calloc( x, sizeof (FILE *)))){
                perror( "malloc" );
                exit( 2 );
                }
        for ( i = 0 ; i < x ; i++ ){
                if ( !( fp[i] = popen( argv[2], "w" ))){
                        perror( "popen( $2 )" );
                        exit( 3 );
                        }
                }
        i = l = 0 ;
        while ( EOF != ( c = getchar())){
                if ( EOF == putc( c, fp[i] )){
                        perror( "putc( popen( $2 ))" );
                        exit( 4 );
                        }
                if ( c == '\n'
                  && ++l == lct ){
                        l = 0 ;
                        if ( ++i == x ){
                                i = 0 ;
                         }
                 }
         }
        if ( ferror( stdin )){
                perror( "stdin" );
                exit( 5 );
                }
        for ( i = 0 ; i < x ; i++ ){
                if ( 0 > fclose( fp[i] )){
                        perror( "fclose( popen( $2 ))" );
                        }
                }
        exit( 0 );
}

 
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Errors received while restoring my data from ext3 filesystem

Hi All, I have dual boot on my laptop - Win8 And linuxMint. From linuxmint i had copied all my data found on the NTFS partitions (on my laptop) to an external hard drive (formatted with ext3). i used rsync for this. Now after my hard disk crashed, am restoring the data back from ext3... (2 Replies)
Discussion started by: coolatt
2 Replies

2. UNIX for Dummies Questions & Answers

Data files on AIX to NTFS Drive

How can I copy a data disk on an AIX system to a Windows readable format (0 Replies)
Discussion started by: Phill
0 Replies

3. Red Hat

CentOS and XP dualboot + ext3/ntfs mount

I have WinXP Pro SP2 on first disk /dev/hda, which my BIOS currently has as first boot device. I have CentOS-5.2-x86_64 on second disk /dev/hdb, which I can boot into if I set my BIOS to boot from this HDD. It is using LVM with vg00 and a single LV for root (/) filesystem (ext3). How do I get... (19 Replies)
Discussion started by: apra143
19 Replies

4. UNIX for Dummies Questions & Answers

transfering a directory to the toolman

I have a question about transferring a directory to the toolman. I have a directory called assn3 that contains two txt files and one empty directory and I want to transfer the assn3 to my my tooman account. Every time I try to transfer the file it says that the assn3 is not a regular file! what... (3 Replies)
Discussion started by: aama100
3 Replies

5. BSD

Mounting ext3 & NTFS on PC-BSD!!

I'm having problem mounting ext3 & ntfs partitions on my PC-BSD OS. Can anyone please help me out here. What are the changes required to be done in fstab?? Are there any patches to be installed?? (1 Reply)
Discussion started by: void_man()
1 Replies

6. Filesystems, Disks and Memory

Transfering size between partitions

Below is a list of the file system on my Sun system. How can I transfer more disk space from the "/space" partition to the "/" partition with out rebuilding? / /dev/dsk/c0t0d0 6191949 5736718 393312 94% /proc /proc 0 0 0 0% /dev/fd fd 0 0 0 0% /etc/mntta ... (10 Replies)
Discussion started by: meyersp
10 Replies

7. UNIX for Dummies Questions & Answers

Transfering files from one server to another.

My oracle database is generating archive logs. I want to copy those archive logs over to backup server on a regular basis. I know how to create an ftp job and I can put it in my crontab. My problem is that I don't know how to send the files just once instead of sends all the files in the... (4 Replies)
Discussion started by: Alan Bird
4 Replies

8. UNIX for Dummies Questions & Answers

Transfering files

Hi all. 1. How can i copy files from one unix system to another. should i use ftp? so How? 2. How can i create an archive whose extention is tar.gz? and how can i decompress them later? 3. WHat is RPM ? what does it stands for? Thanks (5 Replies)
Discussion started by: vbs
5 Replies

9. UNIX for Dummies Questions & Answers

Transfering a window to another screen

Dear Experts I am using a workstatin with two screens and its OS is solaris 8. I wish to trnsfer some application windows from one screen to another one without need to closing the window and open it from other screen. Please advise Bests Reza (3 Replies)
Discussion started by: Reza Nazarian
3 Replies
Login or Register to Ask a Question