The UNIX and Linux Forums  

Go Back   The UNIX and Linux Forums > Top Forums > Shell Programming and Scripting
Google UNIX.COM


Shell Programming and Scripting Post questions about KSH, CSH, SH, BASH, PERL, PHP, SED, AWK and OTHER shell scripts here.

More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
finding duplicates in columns and removing lines totus Shell Programming and Scripting 14 05-16-2008 05:10 PM
using sed to get rid of duplicated columns... fedora Shell Programming and Scripting 4 04-10-2008 01:21 PM
remove duplicated columns kamel.seg Shell Programming and Scripting 6 02-21-2008 03:36 AM
Removing lines that are (same in content) based on columns adsforall UNIX for Dummies Questions & Answers 7 11-09-2007 08:13 AM
remove duplicated lines without sort lalelle Shell Programming and Scripting 6 08-21-2007 04:44 AM

Reply
 
LinkBack Thread Tools Display Modes
  #8 (permalink)  
Old 05-17-2008
era era is offline
Herder of Useless Cats
 

Join Date: Mar 2008
Location: /there/is/only/bin/sh
Posts: 3,094
If you reverse the file, the same solution can be used.

Code:
tac file |
awk '!a[$1 $2 $3]++'
This uses the three first fields to decide whether it's seen the same data before.

If you don't have the tac command, maybe you can sort the input before feeding it to awk.

There are certainly ways to make awk print the last instead of the first line; you can search the forums for a plethora of examples of this.
Reply With Quote
Forum Sponsor
  #9 (permalink)  
Old 05-17-2008
yahyaaa's Avatar
Registered User
 

Join Date: May 2008
Location: Jordan
Posts: 43
Help

Dear era,

I tried to use tac command, put the unix didnt recognize it at all. Also, when I used the awk command alone, it gave an error ( Bailing out )...
Could you tell me about the "a" in the awk, what does it stand for ?

Thanks so much for your kind help.
Reply With Quote
  #10 (permalink)  
Old 05-17-2008
Registered User
 

Join Date: Apr 2008
Location: European Union/Germany
Posts: 183
Have a look at the forum FAQ Simple rules of the UNIX.COM forums: . Duplicating and crossposting is strongly discouraged.

If you aren't able to understand above command line even after a quite similiar one has been explained in detail in another thread maybe you should start really practising shell programming (which consists of reading man pages/online ressources) instead of demanding a solution from someone beeing as kind as to sacrifice his free time for you.
Reply With Quote
  #11 (permalink)  
Old 05-17-2008
yahyaaa's Avatar
Registered User
 

Join Date: May 2008
Location: Jordan
Posts: 43
Help

Dear fabtagon,

Read the last example and you will find that it's not duplicated, it's another Question, NOT AS THE ONE THAT YOU COPIED IT's ANSWER AND PASTED IT TO MINE..... Look again if you are interested, otherwise, you have my best regards.
Reply With Quote
  #12 (permalink)  
Old 05-17-2008
era era is offline
Herder of Useless Cats
 

Join Date: Mar 2008
Location: /there/is/only/bin/sh
Posts: 3,094
There are many different variants of awk. If your awk does not understand that script, see if you can find nawk or mawk or gawk instead. On some systems (Sun, HP-UX) you might be able to find a "XPG4" version of awk which is more modern than the bare-bones "old awk".

The name of awk comes from the family names of its creators Alfred Aho, Peter Weinberger, and Brian Kernighan.

If you are unable to abide by the forum rules in spite of several remarks by forum users, perhaps these forums are not for you.
Reply With Quote
  #13 (permalink)  
Old 05-17-2008
yahyaaa's Avatar
Registered User
 

Join Date: May 2008
Location: Jordan
Posts: 43
I meant the "a" in the command you wrote (awk '!a[$1 $2 $3]++'), because it was not clear enough for me... Im new to awk and I needed a quick solution.

and I do abide the forum rules, see for your self above... I dare you if you find similar thread like this one or even close to..

Nevertheless, thanks for your help,
Reply With Quote
  #14 (permalink)  
Old 05-17-2008
era era is offline
Herder of Useless Cats
 

Join Date: Mar 2008
Location: /there/is/only/bin/sh
Posts: 3,094
The forums' own search tool stupidly treats "awk" as a stop word, so I took a detour via Google.

site:unix.com awk duplicate - Google Search

a is just the name of a variable; if the associative array already contains a value for the given key, we have already seen that key before, and suppress printing. (The default if no action is given is to print anything matching the condition.)
Reply With Quote
Google UNIX.COM
Reply

Thread Tools
Display Modes




All times are GMT -7. The time now is 03:58 PM.


Powered by: vBulletin, Copyright ©2000 - 2006, Jelsoft Enterprises Limited.
The UNIX and Linux Forums Content Copyright ©1993-2008 The CEP Blog All Rights Reserved -Ad Management by RedTyger Visit The Global Fact Book

Content Relevant URLs by vBSEO 3.2.0