Removing Header & Trailer from a file

karthi_gana · #1 (**permalink**) 10-02-2009

Hi All,

I am karthik. I am new to this forum. I have one requirement.

I have a file with header and footer.

Header may be like

HDR0001
or
FILE20090110

(Assume it is unknown so far, but i am sure there is a header in the file)

likewise file has the trailer too.

I just want to remove the header & footer and BCP data into a table.

Note: The file size would be 1.5 GB to 2.0 GB.

So i am looking for some good way to remove the header and trailer.

Inputs are welcome!

Franklin52 · #2 (**permalink**) 10-02-2009

This should do the trick:

Code:

awk 'NR<3{s=$0;next}{print s;s=$0}' file > newfile

or:

Code:

sed '1d;$d' file > newfile

karthi_gana · #3 (**permalink**) 10-02-2009

Thanks Franklin!

I tested the second one. It works fine.

But i don't want to create another file. I tested the script as below

filename > filename. After executing the script, the <filename> becomes empty.

so isn't possible to remove the header & footer and store it in the same file itself?

Also can you tell me what is the difference between the first one and second one? i mean which will give give good performance?

Franklin52 · #4 (**permalink**) 10-02-2009

Code:

awk 'NR<3{s=$0;next}{print s;s=$0}' file > newfile

Explanation of awk code:

Code:

NR<3{s=$0;next}

If line number < 3 assign the current line to variable s and read the next line

Code:

{print s;s=$0}

Print the previous line and assign the current line to variable s

Explanation of sed code:

Code:

sed '1d;$d'

Delete the first (1d) and the last line ($d)

thegeek · #5 (**permalink**) 10-02-2009

Answer for one of your question:

How to make the edit in the same file itself,
use -i switch in sed as

Code:

sed -i <REQUIRED-PART>

karthi_gana · #6 (**permalink**) 10-07-2009

Quote:

Originally Posted by thegeek

Answer for one of your question:

How to make the edit in the same file itself,
use -i switch in sed as

Code:

sed -i <REQUIRED-PART>

sed: illegal option -- i
$

i got the above error message.

i am using ksh.

is there any other option to do this?

binlib · #7 (**permalink**) 10-08-2009

Check the -F and -L command line options of bcp.

More UNIX and Linux Forum Topics You Might Find Helpful
Thread	Thread Starter	Forum	Replies	Last Post
Append Header and Trailer	balzzz	UNIX for Dummies Questions & Answers	2	01-06-2008 08:19 AM
Copy all the files with time stamp and remove header,trailer from file	ksrams	UNIX for Dummies Questions & Answers	35	07-30-2007 02:15 PM
Removing trailer from a flat file!!!	kumarsaravana_s	UNIX for Dummies Questions & Answers	12	06-24-2007 03:53 AM
Count No of Records in File without counting Header and Trailer Records	guiguy	Shell Programming and Scripting	2	06-07-2007 12:15 PM
Remove header(first line) and trailer(last line) in ANY given file	madhunk	Shell Programming and Scripting	2	03-13-2006 03:36 PM

Thread Tools	Search this Thread
Show Printable Version Email this Page	Search this Thread: Advanced Search
Display Modes	Rate This Thread
Switch to Linear Mode Hybrid Mode Switch to Threaded Mode	Rate This Thread:


	Powered by