Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

sed delete pattern skipping first n lines of file.

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting sed delete pattern skipping first n lines of file.
# 1  
Old 12-22-2008
sed delete pattern skipping first n lines of file.
I have files of more than 10K lines that I need to delete lines that contain a pattern, but I want to keep the first few lines intact. Can this be done with sed?
# 2  
Old 12-22-2008
Hammer & Screwdriver I would think an awk solution would be better
Using a file from a different problem earlier today...

Code:
> cat file121
XMAS200811100100.txt
XMAS200811110100.txt
XMAS200812150105.txt
XMAS200812220100.txt
XMAS200812220200.txt
XMAS200812220199.txt
XMAS200812220177.txt
XMAS200812230177.txt
> awk '(substr($0,9,4)!="1222")||(NR<5)' file121
XMAS200811100100.txt
XMAS200811110100.txt
XMAS200812150105.txt
XMAS200812220100.txt
XMAS200812230177.txt

Thus any records less than 5 (the first four) are kept. There is a 1222 record in the first four, so it is kept. Otherwise, all 1222 records are deleted.

Without any data to look at, this is the best I could provide.
# 3  
Old 12-22-2008
you can use head to skip the first few lines.

For example, head +5 will skip the first 4 lines and start the output on the 5th line.
Padow
# 4  
Old 12-22-2008
To be more descriptive, I have a csv file that is a compilation of multiple csv files. The first three lines are column headers and I want to delete the duplicates of these lines that are in the file. I had thought something like 'sed -i '3~1/pattern/d' filename.txt' might work, but this generates an error.
# 5  
Old 12-22-2008
to get rid of duplicate lines, use sort -u
Padow
# 6  
Old 12-22-2008
Quote:
Originally Posted by Padow
to get rid of duplicate lines, use sort -u
This strips the duplicate lines but I had to use -r to keep the headers at the beginning.

Thanks
# 7  
Old 12-22-2008
you were close:
Code:
sed -i '3,$/pattern/d' filename.txt
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

sed command to delete a pattern in a file

Hi Everyone, I have an unusual requirement. Here is where i am stuck for sometime now... I have this text file.. lets say .. output.sql... it has lot of entries... here below is part of the entry... .. . . . . . . . . . . . . . . . .... (3 Replies)
Discussion started by: vivek d r
3 Replies

2. Shell Programming and Scripting

sed search pattern and delete lines

Hello, i have a question. My problem is that i have a file like: TEST JOHN ADAM MICHAEL SEBASTIAN ANDY i want find for MICHAEL and want delete lines like this: TEST (4 Replies)
Discussion started by: eightball
4 Replies

3. Shell Programming and Scripting

Sed delete blank lines upto first pattern match

Hi Im trying to do the following in sed. I want to delete any blank line at the start of a file until it matches a pattern and then stops. for example: Input output: I have got it to work within a range of two patterns with the following: sed '/1/,/pattern/{/^]*$/d}' The... (2 Replies)
Discussion started by: duonut
2 Replies

4. Shell Programming and Scripting

sed/awk : how to delete lines based on IP pattern ?

Hi, I would like to delete lines in /etc/hosts on few workstations, basically I want to delete all the lines for a list of machines like this : for HOST in $(cat stations.lst |uniq) do # echo -n "$HOST" if ping -c 1 $HOST > /dev/null 2>&1 then HOSTNAME_val=`rsh $HOST "sed... (3 Replies)
Discussion started by: albator1932
3 Replies

5. Shell Programming and Scripting

sed pattern to delete lines containing a pattern, except the first occurance

Hello sed gurus. I am using ksh on Sun and have a file created by concatenating several other files. All files contain header rows. I just need to keep the first occurrence and remove all other header rows. header for file 1111 2222 3333 header for file 1111 2222 3333 header for file... (8 Replies)
Discussion started by: gary_w
8 Replies

6. Shell Programming and Scripting

copy, then delete lines in file with sed using a pattern

I need to copy lines to a new file from files with sed using a pattern in char postions 1-3. Then after the copy, I need to delete those same lines from the input files. For example, string "ABC" in pos 1-3 (6 Replies)
Discussion started by: laksjfhoius9123
6 Replies

7. Shell Programming and Scripting

how to delete lines from a file which starts with a specific pattern

I need to delete those lines from a file, which starts with 45. How to do it? (3 Replies)
Discussion started by: mady135
3 Replies

8. Shell Programming and Scripting

delete lines in file matching a pattern

I have a text file, a sample of which is as follows: r/- * 0: WINDOWS/Microsoft.NET/Framework/v2.0.50727/ASP.NETWebAdminFiles/Images/headerGRADIENT_Tall.gif r/- * 0: WINDOWS/SoftwareDistribution/Download/cf8ec753e88561d2ddb53e183dc05c3e/backoff.jpg r/- * 0: ... (2 Replies)
Discussion started by: stumpyuk
2 Replies

9. Shell Programming and Scripting

Parsing a file using perl and skipping some lines

Hi, Consider following file with input: `YFLG:NC^Byad_insert constraint {id=600104470} {profile=GENDER == 2} {profile=BEHAVIOR == 17} {profile=SITEATTR_MULT == siteid:211051} {profile=AGE in } yad_insert ad {id=1718286093336959379} {type=R} ^AYFLG:YOO^Byad_insert constraint {id=600104471}... (1 Reply)
Discussion started by: bvids
1 Replies

10. Shell Programming and Scripting

SED: match pattern & delete matched lines

Hi all, I have the following data in a file x.csv: > ,this is some text here > ,,,,,,,,,,,,,,,,2006/11/16,0.23 > ,,,,,,,,,,,,,,,,2006/12/16,0.88 < ,,,,,,,,,,,,,,,,this shouldnt be deleted I need to use SED to match anything with a > in the line and delete that line, can someone help... (7 Replies)
Discussion started by: not4google
7 Replies
Login or Register to Ask a Question