Today (Saturday) We will make some minor tuning adjustments to MySQL.

You may experience 2 up to 10 seconds "glitch time" when we restart MySQL. We expect to make these adjustments around 1AM Eastern Daylight Saving Time (EDT) US.


Split large file and add header and footer to each small files


Login or Register to Reply

 
Thread Tools Search this Thread
# 1  
Split large file and add header and footer to each small files

I have one large file, after every 200 line i have to split the file and the add header and footer to each small file?
It is possible to add different header and footer to each file?
# 2  
Quote:
Originally Posted by ashish4422
I have one large file, after every 200 line i have to split the file and the add header and footer to each small file?
It is possible to add different header and footer to each file?
Post sample data and example of the desired output.
# 3  
Content of large file:-

E106,0,1/9/1993,0,E001,E003,A,45200,3766.667,21.730769
E108,0,2/3/1995,0,E001,E003,A,15000,1250,7.211538
E109,0,06-mar-07,0,E001,E001,A,78000,6500,37.5
E110,0,09-dec-2008,0,E001,E001,A,56000,4666.667,26.923077
E104,0,06/04/1994,0,E001,E003,A,95000,7916.667,45.673077
E105,0,7/30/1993,0,E001,E003,A,87000,7250,41.826923
E106,0,1/9/1993,0,E001,E003,A,45200,3766.667,21.730769
E108,0,01-feb-2008,0,E001,E003,A,15000,1250,7.211538
E109,0,2/15/1995,0,E001,E001,A,78000,6500,37.5

small file contain:-
file1.txt
E001 start
E106,0,1/9/1993,0,E001,E003,A,45200,3766.667,21.730769
E108,0,2/3/1995,0,E001,E003,A,15000,1250,7.211538
E001 End

file2.txt
E002 Start
E109,0,06-mar-07,0,E001,E001,A,78000,6500,37.5
E110,0,09-dec-2008,0,E001,E001,A,56000,4666.667,26.923077
E002 End
# 4  
You can use something like this:

Code:
awk 'END { printf "E%03d end\n", c > f }
!(NR % 200) || NR == 1 { if (f) { printf "E%03d end\n", c > f; close(f) } 
printf "E%03d start\n", ++c > (f = "file" c ".txt") }
{ print > f }' large

Use nawk or /usr/xpg4/bin/awk on Solaris.
This User Gave Thanks to radoulov For This Post:
# 5  
Hi radoulov

Could you please explain you code?

what to do if header and footer remain the same in every small file.

thanks
# 6  
Quote:
Originally Posted by ashish4422
what to do if header and footer remain the same in every small file.
Should they remain the same across all small files?
The code I posted generates the same start and end for every single file.
# 7  
Perhaps you can adapt something like this to do what you want:

Code:
 $ split -d -l 3 temp.txt file && for X in file*; do { echo "$X start"; cat $X; echo "$X end"; } > $X.txt; done

Explanation:

Splits the file every three lines, naming each split-off file with "file" followed by digits. The for loop then takes each file, writes the "start" section, the contents of the file, then the "end" section, and names it with the same file name but with a ".txt" at the end.

Working example:

Code:
 $ cat temp.txt 
E106,0,1/9/1993,0,E001,E003,A,45200,3766.667,21.730769
E108,0,2/3/1995,0,E001,E003,A,15000,1250,7.211538
E109,0,06-mar-07,0,E001,E001,A,78000,6500,37.5
E110,0,09-dec-2008,0,E001,E001,A,56000,4666.667,26.923077
E104,0,06/04/1994,0,E001,E003,A,95000,7916.667,45.673077
E105,0,7/30/1993,0,E001,E003,A,87000,7250,41.826923
E106,0,1/9/1993,0,E001,E003,A,45200,3766.667,21.730769
E108,0,01-feb-2008,0,E001,E003,A,15000,1250,7.211538
E109,0,2/15/1995,0,E001,E001,A,78000,6500,37.5

 $ split -d -l 3 temp.txt file && for X in file*; do { echo "$X start"; cat $X; echo "$X end"; } > $X.txt; done

 $ cat file00.txt
file00 start
E106,0,1/9/1993,0,E001,E003,A,45200,3766.667,21.730769
E108,0,2/3/1995,0,E001,E003,A,15000,1250,7.211538
E109,0,06-mar-07,0,E001,E001,A,78000,6500,37.5
file00 end

 $ cat file01.txt
file01 start
E110,0,09-dec-2008,0,E001,E001,A,56000,4666.667,26.923077
E104,0,06/04/1994,0,E001,E003,A,95000,7916.667,45.673077
E105,0,7/30/1993,0,E001,E003,A,87000,7250,41.826923
file01 end

 $ cat file02.txt
file02 start
E106,0,1/9/1993,0,E001,E003,A,45200,3766.667,21.730769
E108,0,01-feb-2008,0,E001,E003,A,15000,1250,7.211538
E109,0,2/15/1995,0,E001,E001,A,78000,6500,37.5
file02 end

Login or Register to Reply

|
Thread Tools Search this Thread
Search this Thread:
Advanced Search

More UNIX and Linux Forum Topics You Might Find Helpful
Split large xml into mutiple files and with header and footer in file
karthik
Split large xml into mutiple files and with header and footer in file tried below it splits unevenly and also i need help in adding header and footer command : csplit -s -k -f my_XML_split.xml extrfile.xml "/<Document>/" {1} sample xml <?xml version="1.0" encoding="UTF-8"?><Recipient>...... Shell Programming and Scripting
36
Shell Programming and Scripting
Split a large array into small chunks
rkrish
Hi, I need to split a large array "@sharedArray" into 10 small arrays. The arrays should be like @sharedArray1,@sharedArray2,@sharedArray3...so on.. Can anyone help me with the logic to do so :(:confused:... Shell Programming and Scripting
6
Shell Programming and Scripting
Add header and footer with record count in footer
itsranjan
This is my file(Target.txt) name|age|locaction abc|23|del xyz|24|mum jkl|25|kol The file should be like this 1|03252012 1|name|age|location 2|abc|23|del 2|xyz|24|mum 2|jkl|25|kol 2|kkk|26|hyd 3|4 Column 1 is row indicator for row 1 and 2, column indicator is 1,for data rows...... Shell Programming and Scripting
1
Shell Programming and Scripting
How to add header and footer?
ken002
Hi, Guys, I want add header and footer in a file. I can add footer using following command: echo "Footer" >>file. I don't know how to add header. Thanks in advance... Shell Programming and Scripting
4
Shell Programming and Scripting
Split large file and add header and footer to each file
ashish4422
I have one large file, after every 200 line i have to split the file and the add header and footer to each small file? It is possible to add different header and footer to each file?... Shell Programming and Scripting
1
Shell Programming and Scripting

Featured Tech Videos