File Handling | Unix Linux Forums | UNIX for Dummies Questions & Answers

  Go Back    


UNIX for Dummies Questions & Answers If you're not sure where to post a UNIX or Linux question, post it here. All UNIX and Linux newbies welcome !!

File Handling

UNIX for Dummies Questions & Answers


Tags
file handling

Closed Thread    
 
Thread Tools Search this Thread Display Modes
    #1  
Old 01-24-2013
Gurkamal83 Gurkamal83 is offline
Registered User
 
Join Date: Nov 2012
Last Activity: 11 August 2013, 9:56 AM EDT
Posts: 19
Thanks: 1
Thanked 0 Times in 0 Posts
File Handling

Hi Team,

I am trying to cut a large file into multiple files. It has

Header

50,050 records

Trailer

-------------------------------------------

I need to cut the files into multiple files of 1000 records and should have the same header and trailer as the original files.

-------------------------------------------

Output files should be

Header

1000 Records

Trailer.

-------------------------------------------------

Can someone help me here??
Sponsored Links
    #2  
Old 01-24-2013
Yoda's Avatar
Yoda Yoda is offline Forum Advisor  
Jedi Master
 
Join Date: Jan 2012
Last Activity: 22 August 2014, 1:40 PM EDT
Location: Galactic Empire
Posts: 3,362
Thanks: 230
Thanked 1,197 Times in 1,125 Posts

Code:
split -d -l 1000 filename filename_out

head -1 filename > header
tail -1 filename > trailer

for file in filename_out*
do
   cat header ${file} trailer > tmp
   mv tmp ${file}
done

Sponsored Links
    #3  
Old 01-24-2013
Gurkamal83 Gurkamal83 is offline
Registered User
 
Join Date: Nov 2012
Last Activity: 11 August 2013, 9:56 AM EDT
Posts: 19
Thanks: 1
Thanked 0 Times in 0 Posts
Thanks.

But the code has an issue.

the first file has

Header
999records..

I want to start the split from 3rd records. As my file has
Header
\n(One line Gap)
Records
\n(one line gap)
Trailer

Also for trailer I need the record count.

Say if a file has 5005 records... then first 5 files should show
Header, 1000 records, and Trailer(containing the records count of 1000)

6th file should have header, 5 records, Trailer with records count as 5.
    #4  
Old 01-24-2013
Yoda's Avatar
Yoda Yoda is offline Forum Advisor  
Jedi Master
 
Join Date: Jan 2012
Last Activity: 22 August 2014, 1:40 PM EDT
Location: Galactic Empire
Posts: 3,362
Thanks: 230
Thanked 1,197 Times in 1,125 Posts
Quote:
Originally Posted by Gurkamal83 View Post
I need to cut the files into multiple files of 1000 records and should have the same header and trailer as the original files.
You said that you want the same header and trailer as the original file!

By the way I missed skipping header & trailer, which you can do using awk :

Code:
lines=$( wc -l < filename )
awk -v L=$lines 'NR>1&&NR<L' filename > filename_data
split -d -l 1000 filename_data filename_out

Use wc to calculate line count and append it as trailer to each file:

Code:
head -1 filename > header

for file in filename_out*
do
   cat header ${file} > tmp
   wc -l < ${file} >> tmp
   mv tmp ${file}
done

Sponsored Links
    #5  
Old 01-25-2013
RudiC RudiC is offline Forum Advisor  
Registered User
 
Join Date: Jul 2012
Last Activity: 22 August 2014, 12:02 PM EDT
Location: Aachen, Germany
Posts: 4,093
Thanks: 67
Thanked 971 Times in 921 Posts
Trying to find a awk only solution, the script has become uglier and uglier, but here it is:

Code:
$ awk  'NR==FNR {T=(T?T:"\n")$0; next}        # save trailer from tail process substitution below
        FNR<3   {H=H$0(H?"":"\n"); next}      # save header in first two lines
        /^$/    {exit}                        # find start of trailer (= empty line as specified) and quit
        !((FNR-2)%N-1) {if (fn) print T > fn; # every N records:
                        fn="file"++i;         # create new filename
                        print H > fn}
        {print > fn}                          # print to just created filename
        END {print T > fn}                    # add trailer to last file
       ' N=1000 <(tail -2 file) file          # process substitution, not in all shells available

Sponsored Links
Closed Thread

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
UNIX File handling -Issue in reading a file KuldeepSinghTCS Shell Programming and Scripting 1 08-16-2010 11:47 AM
please help me in file handling ponmuthu UNIX for Advanced & Expert Users 1 07-21-2010 04:00 AM
File Handling baanprog UNIX for Advanced & Expert Users 3 10-21-2008 09:04 AM
file handling ninjanesto Programming 2 12-27-2006 10:35 AM
File Handling in C trinath Programming 3 01-19-2006 10:00 PM



All times are GMT -4. The time now is 05:36 PM.