sed remove two headers; writing more elegant code


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers sed remove two headers; writing more elegant code
# 1  
Old 08-24-2010
sed remove two headers; writing more elegant code

Hi there,

I have two questions. First, I was wondering how to use sed to remove two header lines or two tail lines. Here I just do the same operation twice...I'm sure there is a better way. Second, and more importantly, is there a better way to have these operations use files other than having it output these dumb ZZ* files and then rm them? There must be a more elegant way...

Thanks!

Mikey

Code:
sed '$d' ZZoutput > ZZoutput1				
sed '$d' ZZoutput1 > ZZoutfile				
sed '1d' ZZoutfile > ZZoutput2				
sed '1d' ZZoutput2 > outfile1				
rm ZZ*

Moderator's Comments:
Mod Comment Use code tags.

Last edited by zaxxon; 08-25-2010 at 04:14 AM..
# 2  
Old 08-25-2010
I'm not the sed expert, so removing the last two lines is beyond me. Removing the first two lines in one command is pretty straight forward:

Code:
sed '1,2d' <file

1,2d reads: delete the range of lines 1 through 2.

To do everything you want in one swoop, I'd turn to awk:

Code:
awk '
        NR > 4 { print buffer1; }    # print before overlaying
        NR > 2 {
                buffer1 = buffer2;    # buffer last two lines -- never printed
                buffer2 = $0;
        }
' <input-file >outputfile

You can smash it all onto one line if you want; easier to read this way I think.

It ditches all lines before line three, then buffers two lines such that the last two lines of the input are never printed.

If you want to replace your original input file, you will have to write the output to a temporary file, and them move the temp file back to the original file name. No way of getting round this, and it does prevent disaster as you can check for errors etc. before renaming the output and trashing the input file.

To answer your other question, you could have piped your commands together to not have to deal with the temp files:

Code:
sed '$d' ZZoutput |sed '$d' | sed '1d'  | sed '1d' > outfile1

I didn't test that, so with luck I've not introduced any typos.

Last edited by agama; 08-25-2010 at 12:35 AM.. Reason: added comments
This User Gave Thanks to agama For This Post:
# 3  
Old 08-25-2010
thanks AGAIN!
# 4  
Old 08-25-2010
Hi.

If you have appropriate versions of head and tail, you can use something like:
Code:
#!/usr/bin/env bash

# @(#) s1	Demonstrate head, tail for omitting first, last 2 lines.

pe() { for i;do printf "%s" "$i";done; printf "\n"; }
pl() { pe;pe "-----" ;pe "$*"; }

pl " Versions used:"
head --version | head -1 ; tail --version | head -1

cat >t1 <<EOF
a
b
c
d
e
EOF

pl " Contents of file:"
cat t1

pl " Results of head, then tail:"
head --lines=-2 t1 |
tail --lines=+3

pl " Results of tail, then head:"
tail --lines=+3 t1 |
head --lines=-2 

exit 0

producing:
Code:
% ./s1

-----
 Versions used:
head (GNU coreutils) 6.10
tail (GNU coreutils) 6.10

-----
 Contents of file:
a
b
c
d
e

-----
 Results of head, then tail:
c

-----
 Results of tail, then head:
c

See man pages for details ... cheers, drl
# 5  
Old 08-25-2010
Another way could be:
Code:
awk -v var=$(wc -l < file) 'NR > 2 && NR < var-1' file

# 6  
Old 08-25-2010
Code:
sed '1,2d;N;$d'



---------- Post updated at 07:41 PM ---------- Previous update was at 07:26 PM ----------

Sorry, it only works when the number of lines is even.

---------- Post updated at 07:55 PM ---------- Previous update was at 07:41 PM ----------

Code:
sed '1,2d;N;$d;P;D'

works with both odd and even number of lines.
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Remove white space and duplicate headers

I have a file called "dsout" with empty rows and duplicate headers. DATE TIME TOTAL_GB USED_GB %USED --------- -------- ---------- ---------- ---------- 03/05/013 12:34 PM 3151.24316 2331.56653 73.988785 ... (3 Replies)
Discussion started by: Daniel Gate
3 Replies

2. Shell Programming and Scripting

Remove headers thar dont match

Good evening I need your help please, im new at Unix and i wanted to remove the first 5 headers for 100000 records files and then create a control file .ctl that contains the number of records and all seem to work out but when i tested at production it didnt wotk. Here is the code: #!... (6 Replies)
Discussion started by: alexcol
6 Replies

3. Shell Programming and Scripting

Faster command to remove headers for files in a directory

Good evening Im new at unix shell scripting and im planning to script a shell that removes headers for about 120 files in a directory and each file contains about 200000 lines in average. i know i will loop files to process each one and ive found in this great forum different solutions... (5 Replies)
Discussion started by: alexcol
5 Replies

4. UNIX for Dummies Questions & Answers

Using sed command to remove multiple instances of repeating headers in one file?

Hi, I have catenated multiple output files (from a monte carlo run) into one big output file. Each individual file has it's own two line header. So when I catenate, there are multiple two line headers (of the same wording) within the big file. How do I use the sed command to search for the... (1 Reply)
Discussion started by: rebazon
1 Replies

5. Shell Programming and Scripting

Remove interspersed headers in .dat file with AWK

Heya there, A small selection of my data is shown below. DATE TIME FRAC_DAYS_SINCE_JAN1 2011-06-25 08:03:20.000 175.33564815 2011-06-25 08:03:25.000 175.33570602 2011-06-25 ... (4 Replies)
Discussion started by: gd9629
4 Replies

6. Shell Programming and Scripting

Remove text between headers while leaving headers intact

Hi, I'm trying to strip all lines between two headers in a file: ### BEGIN ### Text to remove, contains all kinds of characters ... Antispyware-Downloadserver.com (Germany)=http://www.antispyware-downloadserver.c om/updates/ Antispyware-Downloadserver.com #2... (3 Replies)
Discussion started by: Trones
3 Replies

7. UNIX for Dummies Questions & Answers

Remove certain headers using mailx or sendmail

Hello, So i want to send mails in any way from a solaris 5.8 system, perhaps using mailx or sendmail. My purpose is to stay clear of systems name in head data. So i want to strip at least the "Message-Id" and the "Recieved" headers of the mail. Yet this seems to be a bit of a problem. Now i... (2 Replies)
Discussion started by: congo
2 Replies

8. Shell Programming and Scripting

Please make this code elegant.

Hi All, Following is the part of my script.It does contain many for loops and is not elegant. Please feel free to suggest any changes to make this elegant. Thanks! nua7 for i in `ls $CATALINA_HOME/shared/lib/*.jar`; do LOCALCLASSPATH="$LOCALCLASSPATH:$i" done for i in... (3 Replies)
Discussion started by: nua7
3 Replies

9. Shell Programming and Scripting

Remove Headers throughout a data file

I have a data file with over 500,000 records/lines that has the header throughout the file. SEQ_ID Name Start_Date Ins_date Add1 Add2 1 Harris 04/02/08 03/02/08 333 Main Suite 101 2 Smith 02/03/08 01/23/08 287 Jenkins SEQ_ID Name ... (3 Replies)
Discussion started by: psmall
3 Replies

10. UNIX for Dummies Questions & Answers

help:how to remove headers in output file

Hi I am running a script (which compares two directory contents) for which I am getting an output of 70 pages in which few pages are blank so I was able to delete those blank lines. But I also want to delete the headers present for each page. can any one help me by providing the code... (1 Reply)
Discussion started by: raj_thota
1 Replies
Login or Register to Ask a Question