The UNIX and Linux Forums  

Go Back   The UNIX and Linux Forums > Top Forums > Shell Programming and Scripting
Google UNIX.COM


Shell Programming and Scripting Post questions about KSH, CSH, SH, BASH, PERL, PHP, SED, AWK and OTHER shell scripts here.

More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
Multiple crontab entries crashing server alexandicity Linux 2 05-12-2008 06:25 AM
Need Script to delete multiple entries from a file sidewayz Shell Programming and Scripting 1 02-14-2008 09:38 AM
problem in filtering the file adityam UNIX for Dummies Questions & Answers 0 11-22-2007 04:37 AM
Urgent: Filtering a File HItesh Shell Programming and Scripting 5 05-11-2006 01:16 AM
filtering by file size nir UNIX for Dummies Questions & Answers 2 01-17-2002 07:17 PM

Reply
 
Submit Tools LinkBack Thread Tools Search this Thread Display Modes
  #1  
Old 08-19-2006
Registered User
 

Join Date: Jan 2006
Posts: 35
Filtering multiple entries in a file

Hi,
I have a requirement to filter multiple entries in a text file.
Entries are a pair of 2 lines .Like

line1
line2
.......
........
Here line1, line2 consist of one pair. line1 and line2 contain different data.
There can be multiple entries for the same pair. And there can be 'n' such pairs having multiple entries.
Is there a method available in perl/shell scripting to filter the multiple entries.
Could any one guide me to achieve this? Any help is appreciated.

Thanks in advance.
--------------------
Suman
Reply With Quote
Forum Sponsor
  #2  
Old 08-19-2006
...@...
 

Join Date: Feb 2004
Location: NM
Posts: 4,297
You want to remove duplicate lines?
Code:
awk '!arr[$0]++' duplicatedfile > noduplicates_in_this_file
Reply With Quote
  #3  
Old 08-22-2006
Registered User
 

Join Date: Jan 2006
Posts: 35
Excellent!!!.


Could you explain me the logic in the command?

Thanks & Regards,
-------------------
Suman
Reply With Quote
Google The UNIX and Linux Forums
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes




All times are GMT -7. The time now is 12:13 AM.


Powered by: vBulletin, Copyright ©2000 - 2006, Jelsoft Enterprises Limited.
The UNIX and Linux Forums Content Copyright ©1993-2008. All Rights Reserved.Ad Management by RedTyger Visit The Complex Event Processing Blog

Content Relevant URLs by vBSEO 3.2.0