The UNIX and Linux Forums  

Go Back   The UNIX and Linux Forums > Top Forums > UNIX for Dummies Questions & Answers
.
google unix.com



UNIX for Dummies Questions & Answers If you're not sure where to post a UNIX or Linux question, post it here. All UNIX and Linux newbies welcome !!

More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
remove an entry in crontab melanie_pfefer SUN Solaris 1 12-18-2006 05:45 AM
Remove duplicate sabercats Shell Programming and Scripting 2 03-31-2006 01:35 PM
Remove duplicate ??? sabercats Shell Programming and Scripting 3 03-10-2006 07:06 PM
Hazardous Duplicate Cron Entry? nag_sundaram HP-UX 3 07-14-2005 08:56 AM
Remove Duplicate line Student37 UNIX for Dummies Questions & Answers 1 02-22-2005 03:00 PM

Closed Thread
English Japanese Spanish French German Portuguese Italian Dutch Swedish Russian Norwegian Hungarian Hebrew Danish Bulgarian Greek Powered by Powered by Google
 
LinkBack Thread Tools Search this Thread Rate Thread Display Modes
  #1 (permalink)  
Old 07-05-2007
kharen11 kharen11 is offline
Registered User
  
 

Join Date: Mar 2007
Location: Manila
Posts: 25
Remove duplicate entry in one line

Can anyone help me how can i print only the unique entry in a line?

MI_AP MI_AP MI_CM MI_MF
RC_NAP MBS_AP SF_RAN MBS_AP NT_CAR

so that it will on output the one unique entry per line.

MI_AP MI_CM MI_MF
RC_NAP MBS_AP SF_RAN NT_CAR

I can't find the same situation on the knowledge base.
  #2 (permalink)  
Old 07-05-2007
Shell_Life's Avatar
Shell_Life Shell_Life is offline
Registered User
  
 

Join Date: Mar 2007
Location: Bahia, Brazil
Posts: 695

Code:
while read mLine
do
  echo $mLine | tr ' ' '\n' | sort | uniq | paste -d' ' -s -
done < input_file

  #3 (permalink)  
Old 07-05-2007
vgersh99's Avatar
vgersh99 vgersh99 is offline Forum Staff  
Moderator
  
 

Join Date: Feb 2005
Location: Boston, MA
Posts: 5,131
if the order matters...

nawk -f kharen.awk myFile.txt

kharen.awk:

Code:
!NF
{
  split("", arr)
  for(i=1; i<=NF; i++) {
    printf("%s%s", ($i in arr) ? "" : $i " ", (i==NF) ? "\n" : "")
    arr[$i]
  }
}


Last edited by vgersh99; 07-05-2007 at 03:34 PM..
  #4 (permalink)  
Old 07-05-2007
Shell_Life's Avatar
Shell_Life Shell_Life is offline
Registered User
  
 

Join Date: Mar 2007
Location: Bahia, Brazil
Posts: 695
Vgersh,
Your solution would not print blank or empty lines.
  #5 (permalink)  
Old 07-05-2007
vgersh99's Avatar
vgersh99 vgersh99 is offline Forum Staff  
Moderator
  
 

Join Date: Feb 2005
Location: Boston, MA
Posts: 5,131
Quote:
Originally Posted by Shell_Life View Post
Vgersh,
Your solution would not print blank or empty lines.
Ah, good catch - edited the previous post!

btw, 'sort -u' = 'sort | uniq'

Last edited by vgersh99; 07-05-2007 at 03:51 PM..
  #6 (permalink)  
Old 07-05-2007
Shell_Life's Avatar
Shell_Life Shell_Life is offline
Registered User
  
 

Join Date: Mar 2007
Location: Bahia, Brazil
Posts: 695
It works now -- great!
In any event, here is another solution in shell only:

Code:
while read mLine
do
  echo $mLine | tr ' ' '\n' > Temp
  cat -n Temp | tr '\t' '|' | sort -t'|' -k2 | uniq +7 | sort | sed 's/.*|//'
| paste -d' ' -s -
done < input_file

Closed Thread

Bookmarks

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On




All times are GMT -4. The time now is 06:00 AM.


Powered by: vBulletin, Copyright ©2000 - 2006, Jelsoft Enterprises Limited. Language Translations Powered by .
vBCredits v1.4 Copyright ©2007 - 2008, PixelFX Studios
The UNIX and Linux Forums Content Copyright ©1993-2009. All Rights Reserved.Ad Management by RedTyger

Content Relevant URLs by vBSEO 3.2.0