The UNIX and Linux Forums  

Go Back   The UNIX and Linux Forums > Top Forums > UNIX for Dummies Questions & Answers
.
google unix.com



UNIX for Dummies Questions & Answers If you're not sure where to post a UNIX or Linux question, post it here. All UNIX and Linux newbies welcome !!

More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
Remove duplicate text dejavu88 Shell Programming and Scripting 7 06-08-2008 05:37 PM
Remove duplicate entry in one line kharen11 UNIX for Dummies Questions & Answers 5 07-05-2007 03:56 PM
Remove duplicate sabercats Shell Programming and Scripting 2 03-31-2006 01:35 PM
Remove duplicate ??? sabercats Shell Programming and Scripting 3 03-10-2006 07:06 PM
remove duplicate kazanoova2 Shell Programming and Scripting 4 04-12-2004 01:35 AM

Closed Thread
English Japanese Spanish French German Portuguese Italian Dutch Swedish Russian Norwegian Hungarian Hebrew Danish Bulgarian Greek Powered by Powered by Google
 
LinkBack Thread Tools Search this Thread Rate Thread Display Modes
  #1 (permalink)  
Old 02-22-2005
Student37 Student37 is offline
Registered User
  
 

Join Date: Aug 2003
Location: NJ
Posts: 39
Remove Duplicate line

Hi,
I have a scenario here where I have created a flatfile with the below mentioned information. File as you can see is dispalyed in three columns
1st column is FileNameString
2nd column is Report_Name (this has spaces)
3rd column is Flag
Result file needed is, removal of duplicate Report_Name along with the entire line (which is FilenameString, Report_Name, Flag).

When I try a sort -u, issue here is that sort reads from left to right hence I cannot just capture the duplicate Report_Name

also tried this awk after searching through this archive. Here it compares second line to first line in the column (2) which is the Report_Name.
Ofcourse with no results

cat filename |
awk -F"|" '
{ A[$2] = $1}
END {for ( j in A)
print A[j], j }



19829.p3May05.58 ADJUST TRNS RPT NonNos
2315.001.p3May05.01 ADJUST TRNS RPT NonNos
6818.001.p3May05.04 ADJUST TRNS RPT NonNos
6852.p3May05.06 ADJUST TRNS RPT NonNos
1797.p2Apr05.36 EMPLEE EARN REC NonNos
1797.p2Apr05.36 EMPLEE EARN REC NonNos
283.p2Apr05.43 EMPLEE EARN REC NonNos
283.p2Apr05.43 EMPLEE EARN REC NonNos
7036.001.p2Apr05.57 WGPS VERIF RPT NonNos
7036.001.p2Apr05.57 WGPS VERIF RPT NonNos
7277.001.p2Apr05.08 WGPS VERIF RPT NonNos
69829.p3May05.58 ADJUST TRNS RPT NonNos
3315.001.p3May05.01 ADJUST TRNS RPT NonNos
9818.001.p3May05.04 ADJUST TRNS RPT NonNos

Please let me know
Ta
  #2 (permalink)  
Old 02-22-2005
vgersh99's Avatar
vgersh99 vgersh99 is offline Forum Staff  
Moderator
  
 

Join Date: Feb 2005
Location: Boston, MA
Posts: 5,131
assuming 'Report Name' is 3 words long:

Code:
sort -u -k 2,4 filename

Closed Thread

Bookmarks

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On




All times are GMT -4. The time now is 06:27 AM.


Powered by: vBulletin, Copyright ©2000 - 2006, Jelsoft Enterprises Limited. Language Translations Powered by .
vBCredits v1.4 Copyright ©2007 - 2008, PixelFX Studios
The UNIX and Linux Forums Content Copyright ©1993-2009. All Rights Reserved.Ad Management by RedTyger

Content Relevant URLs by vBSEO 3.2.0