Split A Large File

nbvcxzdz · #1 (**permalink**) 11-14-2005

Hi,

I have a large file(csv format) that I need to split into 2 files. The file looks something like

Original_file.txt
first name, family name, address
a, b, c,
d, e, f,
and so on for over 100,00 lines

I need to create two files from this one file. The condition is i need to ensure both files have the header row "first name, family name, address" AND the second file only starts from the 60,000th row of original_file.txt

Can some please show me the code to do this in a shell script?

Really Appreciated

Thanks

futurelet · #2 (**permalink**) 11-14-2005

Awk:

Code:

1==NR || 60000==NR {
  whichfile++
  output( "first name, family name, address" )
}
{ output( $0 ) }

function output( s )
{ print s  >( "output" whichfile ".csv" )
}

sergilda · #3 (**permalink**) 11-14-2005

Hi,

you can make the first file by using:

head -60000 filename > filename_1to60000

you can make the second one as follow:

echo 'first name, family name, address' > filename_60001to100000
tail +60001 filename >> filename_60001to100000

Kind regards

nbvcxzdz · #4 (**permalink**) 11-14-2005

I will try it out...thanks ill let you know how it goes

jerardfjay · #5 (**permalink**) 11-14-2005

split -l 60000 in_file

In case of more complex fragmentation of a file, use the context based split command csplit.

Jerardjay

More UNIX and Linux Forum Topics You Might Find Helpful
Thread	Thread Starter	Forum	Replies	Last Post
Split large file and add header and footer to each small files	ashish4422	Shell Programming and Scripting	7	07-07-2008 02:13 PM
Split large file and add header and footer to each file	ashish4422	Shell Programming and Scripting	1	04-15-2008 06:12 AM
Split a file with no pattern -- Split, Csplit, Awk	madhunk	UNIX for Dummies Questions & Answers	10	12-17-2007 12:57 PM
Strange difference in file size when copying LARGE file..	0ktalmagik	Filesystems, Disks and Memory	1	06-03-2006 07:34 PM
Need to split a large data file using a Unix script	SAIK	HP-UX	1	03-29-2006 04:05 PM

Thread Tools	Search this Thread
Show Printable Version Email this Page	Search this Thread: Advanced Search
Display Modes	Rate This Thread
Switch to Linear Mode Hybrid Mode Switch to Threaded Mode	Rate This Thread:


	Powered by