Split large file and add header and footer to each small files


Login or Register to Reply

 
Thread Tools Search this Thread
# 1  
Old 04-15-2008
Split large file and add header and footer to each small files

I have one large file, after every 200 line i have to split the file and the add header and footer to each small file?
It is possible to add different header and footer to each file?
# 2  
Old 04-15-2008
Quote:
Originally Posted by ashish4422
I have one large file, after every 200 line i have to split the file and the add header and footer to each small file?
It is possible to add different header and footer to each file?
Post sample data and example of the desired output.
# 3  
Old 04-15-2008
Content of large file:-

E106,0,1/9/1993,0,E001,E003,A,45200,3766.667,21.730769
E108,0,2/3/1995,0,E001,E003,A,15000,1250,7.211538
E109,0,06-mar-07,0,E001,E001,A,78000,6500,37.5
E110,0,09-dec-2008,0,E001,E001,A,56000,4666.667,26.923077
E104,0,06/04/1994,0,E001,E003,A,95000,7916.667,45.673077
E105,0,7/30/1993,0,E001,E003,A,87000,7250,41.826923
E106,0,1/9/1993,0,E001,E003,A,45200,3766.667,21.730769
E108,0,01-feb-2008,0,E001,E003,A,15000,1250,7.211538
E109,0,2/15/1995,0,E001,E001,A,78000,6500,37.5

small file contain:-
file1.txt
E001 start
E106,0,1/9/1993,0,E001,E003,A,45200,3766.667,21.730769
E108,0,2/3/1995,0,E001,E003,A,15000,1250,7.211538
E001 End

file2.txt
E002 Start
E109,0,06-mar-07,0,E001,E001,A,78000,6500,37.5
E110,0,09-dec-2008,0,E001,E001,A,56000,4666.667,26.923077
E002 End
# 4  
Old 04-15-2008
You can use something like this:

Code:
awk 'END { printf "E%03d end\n", c > f }
!(NR % 200) || NR == 1 { if (f) { printf "E%03d end\n", c > f; close(f) } 
printf "E%03d start\n", ++c > (f = "file" c ".txt") }
{ print > f }' large

Use nawk or /usr/xpg4/bin/awk on Solaris.
This User Gave Thanks to radoulov For This Post:
Ravichander (08-27-2012)
# 5  
Old 04-15-2008
Hi radoulov

Could you please explain you code?

what to do if header and footer remain the same in every small file.

thanks
# 6  
Old 04-15-2008
Quote:
Originally Posted by ashish4422
what to do if header and footer remain the same in every small file.
Should they remain the same across all small files?
The code I posted generates the same start and end for every single file.
# 7  
Old 04-15-2008
Perhaps you can adapt something like this to do what you want:

Code:
 $ split -d -l 3 temp.txt file && for X in file*; do { echo "$X start"; cat $X; echo "$X end"; } > $X.txt; done

Explanation:

Splits the file every three lines, naming each split-off file with "file" followed by digits. The for loop then takes each file, writes the "start" section, the contents of the file, then the "end" section, and names it with the same file name but with a ".txt" at the end.

Working example:

Code:
 $ cat temp.txt 
E106,0,1/9/1993,0,E001,E003,A,45200,3766.667,21.730769
E108,0,2/3/1995,0,E001,E003,A,15000,1250,7.211538
E109,0,06-mar-07,0,E001,E001,A,78000,6500,37.5
E110,0,09-dec-2008,0,E001,E001,A,56000,4666.667,26.923077
E104,0,06/04/1994,0,E001,E003,A,95000,7916.667,45.673077
E105,0,7/30/1993,0,E001,E003,A,87000,7250,41.826923
E106,0,1/9/1993,0,E001,E003,A,45200,3766.667,21.730769
E108,0,01-feb-2008,0,E001,E003,A,15000,1250,7.211538
E109,0,2/15/1995,0,E001,E001,A,78000,6500,37.5

 $ split -d -l 3 temp.txt file && for X in file*; do { echo "$X start"; cat $X; echo "$X end"; } > $X.txt; done

 $ cat file00.txt
file00 start
E106,0,1/9/1993,0,E001,E003,A,45200,3766.667,21.730769
E108,0,2/3/1995,0,E001,E003,A,15000,1250,7.211538
E109,0,06-mar-07,0,E001,E001,A,78000,6500,37.5
file00 end

 $ cat file01.txt
file01 start
E110,0,09-dec-2008,0,E001,E001,A,56000,4666.667,26.923077
E104,0,06/04/1994,0,E001,E003,A,95000,7916.667,45.673077
E105,0,7/30/1993,0,E001,E003,A,87000,7250,41.826923
file01 end

 $ cat file02.txt
file02 start
E106,0,1/9/1993,0,E001,E003,A,45200,3766.667,21.730769
E108,0,01-feb-2008,0,E001,E003,A,15000,1250,7.211538
E109,0,2/15/1995,0,E001,E001,A,78000,6500,37.5
file02 end

Login or Register to Reply

|
Thread Tools Search this Thread
Search this Thread:
Advanced Search

More UNIX and Linux Forum Topics You Might Find Helpful
Split large xml into mutiple files and with header and footer in file karthik Shell Programming and Scripting 36 20 Hours Ago 01:33 AM
Breaking large file into small files emily Shell Programming and Scripting 12 03-06-2015 07:30 AM
Eliminate Header and footer from EBCDIC file abhilashnair UNIX for Dummies Questions & Answers 4 12-18-2014 05:48 AM
Split and add header and trailer from input file techedipro Shell Programming and Scripting 23 07-09-2014 12:26 PM
How to split this txt file into small files? psychmyluo Shell Programming and Scripting 3 08-04-2013 05:39 PM
Split a large array into small chunks rkrish Shell Programming and Scripting 6 03-13-2013 07:36 AM
Is there a way to append both at header and footer of a file jediwannabe Shell Programming and Scripting 3 02-28-2013 06:57 AM
Removing header or footer from file sridhardwh Shell Programming and Scripting 5 06-04-2012 06:43 AM
Add header and footer with record count in footer itsranjan Shell Programming and Scripting 1 03-25-2012 12:45 AM
sort a report file having header and footer suryanarayana Shell Programming and Scripting 4 11-25-2011 10:48 PM
How to add header and footer? ken002 Shell Programming and Scripting 4 11-22-2010 11:23 PM
script to splite large file to number of small files ahmed.gad Shell Programming and Scripting 10 07-15-2008 09:52 AM
Split a file into 16 small files rrkks Shell Programming and Scripting 10 06-17-2008 05:02 PM
Split large file and add header and footer to each file ashish4422 Shell Programming and Scripting 1 04-15-2008 07:12 AM
Splitting large file into small files dncs Shell Programming and Scripting 4 06-08-2005 12:02 PM