File Handling


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers File Handling
# 1  
Old 01-24-2013
File Handling

Hi Team,

I am trying to cut a large file into multiple files. It has

Header

50,050 records

Trailer

-------------------------------------------

I need to cut the files into multiple files of 1000 records and should have the same header and trailer as the original files.

-------------------------------------------

Output files should be

Header

1000 Records

Trailer.

-------------------------------------------------

Can someone help me here??
# 2  
Old 01-24-2013
Code:
split -d -l 1000 filename filename_out

head -1 filename > header
tail -1 filename > trailer

for file in filename_out*
do
   cat header ${file} trailer > tmp
   mv tmp ${file}
done

# 3  
Old 01-25-2013
Thanks.

But the code has an issue.

the first file has

Header
999records..

I want to start the split from 3rd records. As my file has
Header
\n(One line Gap)
Records
\n(one line gap)
Trailer

Also for trailer I need the record count.

Say if a file has 5005 records... then first 5 files should show
Header, 1000 records, and Trailer(containing the records count of 1000)

6th file should have header, 5 records, Trailer with records count as 5.
# 4  
Old 01-25-2013
Quote:
Originally Posted by Gurkamal83
I need to cut the files into multiple files of 1000 records and should have the same header and trailer as the original files.
You said that you want the same header and trailer as the original file!

By the way I missed skipping header & trailer, which you can do using awk:
Code:
lines=$( wc -l < filename )
awk -v L=$lines 'NR>1&&NR<L' filename > filename_data
split -d -l 1000 filename_data filename_out

Use wc to calculate line count and append it as trailer to each file:
Code:
head -1 filename > header

for file in filename_out*
do
   cat header ${file} > tmp
   wc -l < ${file} >> tmp
   mv tmp ${file}
done

# 5  
Old 01-25-2013
Trying to find a awk only solution, the script has become uglier and uglier, but here it is:
Code:
$ awk  'NR==FNR {T=(T?T:"\n")$0; next}        # save trailer from tail process substitution below
        FNR<3   {H=H$0(H?"":"\n"); next}      # save header in first two lines
        /^$/    {exit}                        # find start of trailer (= empty line as specified) and quit
        !((FNR-2)%N-1) {if (fn) print T > fn; # every N records:
                        fn="file"++i;         # create new filename
                        print H > fn}
        {print > fn}                          # print to just created filename
        END {print T > fn}                    # add trailer to last file
       ' N=1000 <(tail -2 file) file          # process substitution, not in all shells available

 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

File handling

Hi All, I need to extract the data from the text file. The data of the text file is shown below #L 0.000017 4.329939 0.000017 4.716267 r7.9 P 1 1;Net=IN32 The extracted data should be IN32. Could anyone help to script in c shell.? (4 Replies)
Discussion started by: gopishrine
4 Replies

2. UNIX for Dummies Questions & Answers

File handling

I have a file 1 298167 298168 1093209 1093210 1422663 I want to write a code where in I want to read contents of above file like first read is 1 second read is 298167 Substract second read from first and if this is greater than or less than 99,999. Similarly I want to traverse thru... (3 Replies)
Discussion started by: Guru148
3 Replies

3. Shell Programming and Scripting

ksh file handling

Specifically on RHEL 5.7. When does the underlying ksh process open and close files? Every time they're accessed, or as little as possible? Say you have some script like: CreateFiles() { grep "<VALUE1>" ${infile} >> ${outfile} grep "<VALUE2>" ${infile} >> ${outfile} ...... (4 Replies)
Discussion started by: CarloM
4 Replies

4. Programming

Perl help for file handling

$# some text $$ some text $@ some text $$. some text Mg1 some text Mg2 some text . . . Mg10 some text The above 10 lines are to be extracted except the lines starting from $#,$$.,... (4 Replies)
Discussion started by: baig.abdul
4 Replies

5. Shell Programming and Scripting

UNIX File handling -Issue in reading a file

I have been doing automation of daily check activity for a server, i have been using sqls to retrive the data and while loop for reading the data from the file for several activities. BUT i got a show stopper the below one.. where the data is getting store in $temp_file, but not being read by while... (1 Reply)
Discussion started by: KuldeepSinghTCS
1 Replies

6. UNIX for Advanced & Expert Users

File handling

my input for a script is another csv file but in that file say 7 lines are there ... how can i get line by line to that input for example : >cat link.csv www.yahoo.com,yahoo www.google.com,google www.unix.com,unix another file in that file i need to ping the above links ... (2 Replies)
Discussion started by: ponmuthu
2 Replies

7. UNIX for Advanced & Expert Users

please help me in file handling

sir i have to get first line from a file for example >cat file1 abc zxc asd adsf from that file1 i need only first line expected result >abc please help me ! (1 Reply)
Discussion started by: ponmuthu
1 Replies

8. UNIX for Advanced & Expert Users

File Handling

Hi, I have a log file which runs into 3 to 5 GB. We store this typically for 6 months. When a new month starts we move the previous month into a 9 month back up log (file.9m) and delete the last month of the 9 month back up. Iam using awk to find the data and cat to join the files like... (3 Replies)
Discussion started by: baanprog
3 Replies

9. Programming

file handling

Hi all, I got a little issue here. Imagine that I have more than one process accessing one file. Is it possible to know which process(es) are accessing that file when I open the file?? Thanks for the help. Best regards, Ernesto (2 Replies)
Discussion started by: ninjanesto
2 Replies

10. Programming

File Handling in C

Hi all, I have a problem in handling files through C. here is the problem im having: i will query the database (for instance consider employees table ) for empno,ename,job,salary fields.The query returns me some 100 of rows. now i need to place them in a file in row wise pattern as they... (3 Replies)
Discussion started by: trinath
3 Replies
Login or Register to Ask a Question