![]() |
|
|
|
|
|||||||
| Forums | Portal | Register | Rules & FAQ | Contribute | Members List | Arcade | Search | Today's Posts | Mark Forums Read |
| Shell Programming and Scripting Post questions about KSH, CSH, SH, BASH, PERL, PHP, SED, AWK and OTHER shell scripts here. |
|
|
||||
| Thread | Thread Starter | Forum | Replies | Last Post |
| finding duplicates in columns and removing lines | totus | Shell Programming and Scripting | 14 | 05-16-2008 05:10 PM |
| using sed to get rid of duplicated columns... | fedora | Shell Programming and Scripting | 4 | 04-10-2008 01:21 PM |
| remove duplicated columns | kamel.seg | Shell Programming and Scripting | 6 | 02-21-2008 03:36 AM |
| Removing lines that are (same in content) based on columns | adsforall | UNIX for Dummies Questions & Answers | 7 | 11-09-2007 08:13 AM |
| remove duplicated lines without sort | lalelle | Shell Programming and Scripting | 6 | 08-21-2007 04:44 AM |
|
|
LinkBack | Thread Tools | Display Modes |
|
|||
|
If you reverse the file, the same solution can be used.
Code:
tac file | awk '!a[$1 $2 $3]++' If you don't have the tac command, maybe you can sort the input before feeding it to awk. There are certainly ways to make awk print the last instead of the first line; you can search the forums for a plethora of examples of this. |
| Forum Sponsor | ||
|
|
|
|||
|
Have a look at the forum FAQ Simple rules of the UNIX.COM forums: . Duplicating and crossposting is strongly discouraged.
If you aren't able to understand above command line even after a quite similiar one has been explained in detail in another thread maybe you should start really practising shell programming (which consists of reading man pages/online ressources) instead of demanding a solution from someone beeing as kind as to sacrifice his free time for you. |
|
||||
|
Help
Dear fabtagon,
Read the last example and you will find that it's not duplicated, it's another Question, NOT AS THE ONE THAT YOU COPIED IT's ANSWER AND PASTED IT TO MINE..... Look again if you are interested, otherwise, you have my best regards. |
|
|||
|
There are many different variants of awk. If your awk does not understand that script, see if you can find nawk or mawk or gawk instead. On some systems (Sun, HP-UX) you might be able to find a "XPG4" version of awk which is more modern than the bare-bones "old awk".
The name of awk comes from the family names of its creators Alfred Aho, Peter Weinberger, and Brian Kernighan. If you are unable to abide by the forum rules in spite of several remarks by forum users, perhaps these forums are not for you. |
|
||||
|
I meant the "a" in the command you wrote (awk '!a[$1 $2 $3]++'), because it was not clear enough for me... Im new to awk and I needed a quick solution.
and I do abide the forum rules, see for your self above... I dare you if you find similar thread like this one or even close to.. Nevertheless, thanks for your help, |
|
|||
|
The forums' own search tool stupidly treats "awk" as a stop word, so I took a detour via Google.
site:unix.com awk duplicate - Google Search a is just the name of a variable; if the associative array already contains a value for the given key, we have already seen that key before, and suppress printing. (The default if no action is given is to print anything matching the condition.) |
|||
| Google UNIX.COM |