The UNIX and Linux Forums  

Go Back   The UNIX and Linux Forums > Top Forums > UNIX for Dummies Questions & Answers
Google UNIX.COM


UNIX for Dummies Questions & Answers If you're not sure where to post a UNIX or Linux question, post it here. All UNIX and Linux newbies welcome !!

More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
concatenate all duplicate line in a file. vaskarbasak Shell Programming and Scripting 30 08-29-2008 02:31 AM
duplicate line in a text file nixguy Shell Programming and Scripting 5 04-28-2008 11:19 PM
Remove duplicate entry in one line kharen11 UNIX for Dummies Questions & Answers 5 07-05-2007 11:56 AM
Identify duplicate words in a line using command srinivasan_85 UNIX for Dummies Questions & Answers 8 04-30-2007 10:29 PM
Remove Duplicate line Student37 UNIX for Dummies Questions & Answers 1 02-22-2005 12:00 PM

Reply
 
Submit Tools LinkBack Thread Tools Display Modes
  #1  
Old 01-29-2007
Registered User
 

Join Date: Sep 2006
Posts: 25
removing line and duplicate line

Hi,

I have 3 lines in a text file that is similar to this (as a result of a diff between 2 files):

35,36d34
< DATA[0].EVENT[0].EVENT_ID.s = "3661208"
< DATA[0].EVENT[0].EVENT_ID.s = "3661208"

I am trying to get it down to just this:

DATA[0].EVENT[0].EVENT_ID.s = "3661208"

How can I do this? Please note the number "35,36d34" and "3661208" is always different because I get this file from a background process that always changes.

thanks guys
ocelot
Reply With Quote
Forum Sponsor
  #2  
Old 01-29-2007
Registered User
 

Join Date: Aug 2005
Location: Saskatchewan
Posts: 979
Do you care which, if any process number is preserved?
Reply With Quote
  #3  
Old 01-29-2007
Registered User
 

Join Date: Sep 2006
Posts: 25
I will always need the line AND the number in the quotes, i.e.

DATA[0].EVENT[0].EVENT_ID.s = "3661208"

the number here is always different. But i need it along with the rest of the line
Reply With Quote
  #4  
Old 01-29-2007
Registered User
 

Join Date: Jan 2007
Posts: 366
Could it be as simple as
sort -u
?
Reply With Quote
  #5  
Old 01-30-2007
Registered User
 

Join Date: Sep 2006
Posts: 25
i tried sort -u but it only removed one of the lines. I ended up with this:

35,36d34
< DATA[0].EVENT[0].EVENT_ID.s = "3661208"

but i am trying to just get this:

DATA[0].EVENT[0].EVENT_ID.s = "3661208"

Reply With Quote
  #6  
Old 01-30-2007
Registered User
 

Join Date: Mar 2006
Location: Bangalore,India
Posts: 1,397
Code:
diff file1 file2 | sort -u | sed -n "/^[<>]/s/^[<>] *//p"
Reply With Quote
  #7  
Old 01-30-2007
Registered User
 

Join Date: Sep 2006
Posts: 25
thanks anbu23 you saved my life!
Reply With Quote
Google The UNIX and Linux Forums
Reply

Thread Tools
Display Modes




All times are GMT -7. The time now is 07:01 AM.


Powered by: vBulletin, Copyright ©2000 - 2006, Jelsoft Enterprises Limited.
The UNIX and Linux Forums Content Copyright ©1993-2008. All Rights Reserved.Ad Management by RedTyger Visit The Complex Event Processing Blog

Content Relevant URLs by vBSEO 3.2.0