The UNIX and Linux Forums  

Go Back   The UNIX and Linux Forums > Top Forums > UNIX for Dummies Questions & Answers
Google UNIX.COM


UNIX for Dummies Questions & Answers If you're not sure where to post a UNIX or Linux question, post it here. All UNIX and Linux newbies welcome !!

More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
regex to delete multiple blank lines in a file? fedora Shell Programming and Scripting 6 10-11-2007 01:36 PM
using sed command to delete a string spanning multiple lines radha.kalivar Shell Programming and Scripting 9 07-25-2007 07:26 AM
Need to delete multiple lines in a file. kangdom Shell Programming and Scripting 6 10-16-2006 08:02 AM
delete multiple empty lines whatisthis Shell Programming and Scripting 3 11-09-2005 01:42 PM
Delete multiple lines w/ sed bookoo Shell Programming and Scripting 2 07-25-2003 07:03 AM

Reply
 
LinkBack Thread Tools Display Modes
  #1 (permalink)  
Old 04-27-2007
Registered User
 

Join Date: Apr 2007
Posts: 4
Question Delete multiple lines containting a variable string using SED.

Good morning,
Novice scripter in Unix here, and I've run into and sed task I can't quite wrap my head around. I'm pulling my hair out fast enough as it is and thought I would go to the knowledge bank.

I have a sorted file that I'm trying to trim down by deleting any line whose first few characters are repeats of a previous line.
i.e.

1 ABCD
1 CDEF
1 EFGH
2 ACDE
2 GLKGI
2 KLIGH
.
.
.
10 ABSD
10 OIHIHN
10 OHOIN
.
.
.
XX LIHIN
XX OIHNM
XX OHINK

I need to delete any line for which the line header (first three charachters are a repeat.) So for the previous lines it would keep the first line that begins with "1 " the first line that begins with "2 " etc. So the end result of the would be

1 ABCD
2 ACDE
.
10 ABSD
.
XX LIHIN

The first three characters are always whole numbers, no more than 2 digits, and followed by a space, but the maximum number changes (it could anywhere between 11 and 40)

I suppose the other option would be to print out the first line that contains "1 ', the first line that contains "2 " etc. and drop them into a new file.
I'm more familiar with SED, but using AWK or something else would be fine too.

Thanks in advance!
Reply With Quote
Forum Sponsor
  #2 (permalink)  
Old 04-27-2007
...@...
 

Join Date: Feb 2004
Location: NM
Posts: 3,502
Code:
awk ' !arr[$1]++' oldfile > newfile
This finds the first occurrence of the first field and prints the line. Assuming I got what you want.
Reply With Quote
  #3 (permalink)  
Old 04-27-2007
Registered User
 

Join Date: Apr 2007
Posts: 4
Worked like a dream!
It's much appreciated!!
Reply With Quote
Google UNIX.COM
Reply

Thread Tools
Display Modes




All times are GMT -7. The time now is 01:50 PM.


Powered by: vBulletin, Copyright ©2000 - 2006, Jelsoft Enterprises Limited.
The UNIX and Linux Forums Content Copyright ©1993-2008 The CEP Blog All Rights Reserved -Ad Management by RedTyger Visit The Global Fact Book

Content Relevant URLs by vBSEO 3.2.0