Print row if value in column 1 is the first occurence

Raynon · #1 (**permalink**) 03-17-2008

Hi All,

I would like to have a script which is able to perform the below.
Print the whole row if column1 which is "0001" for the below example is the first occurrence. Subsequent "0001" occurrence will not be printed out and so on.

Can any expert help ?

Input:

0001 k= 40
0001 k= 2
0002 k= 1
0003 k= 1
0004 k= 77
0004 k= 1
0005 k= 88
0005 k= 6

Output:

0001 k= 40
0002 k= 1
0003 k= 1
0004 k= 77
0005 k= 88

agn · #2 (**permalink**) 03-17-2008

Code:

$ cat buf                                                                                                                                                               
0001 k= 40
0001 k= 2
0002 k= 1
0003 k= 1
0004 k= 77
0004 k= 1
0005 k= 88
0005 k= 6
$ perl -n -e '($num) = split /=/; next if $found[$num]; print; $found[$num] = 1' buf  
0001 k= 40
0002 k= 1
0003 k= 1
0004 k= 77
0005 k= 88

Shivdatta · #3 (**permalink**) 03-17-2008

$cat test
0001 k= 40
0001 k= 2
0002 k= 1
0003 k= 1
0004 k= 77
0004 k= 1
0005 k= 88
0005 k= 6

for i in `cat test|cut -d" " -f1`
do
grep "$i" test | head -1 >> out.txt
done
sort -u out.txt

0001 k= 40
0002 k= 1
0003 k= 1
0004 k= 77
0005 k= 88

fpmurphy · #4 (**permalink**) 03-17-2008

And the Python approach

Code:

#!/usr/local/bin/python

keys = {}

input = file('test')
for line in input:
   key = line.split(' ', 1)[0]
   if key not in keys:
      keys[key] = 1
      print line,

radoulov · #5 (**permalink**) 03-17-2008

Like a FAQ

Awk:

Code:

awk '!x[$1]++' file

Use nawk or /usr/xpg4/bin/awk on Solaris.

Perl:

Code:

perl -ane'print unless $x{$F[0]}++' file

agn · #6 (**permalink**) 03-17-2008

Brilliant, radoulov!

Raynon · #7 (**permalink**) 03-17-2008

Quote:

Originally Posted by radoulov

Like a FAQ

Awk:

Code:

awk '!x[$1]++' file

Use nawk or /usr/xpg4/bin/awk on Solaris.

Perl:

Code:

perl -ane'print unless $x{$F[0]}++' file

Hi radoulov,

The perl code seems to work but not the awk.
Can you help ? I am using solaris by the way.

Also, can you explain your perl code so that i can understand better. What is the function of "-ane"

Code:

$ nawk '!x[$1]++' file
x[$1]++': Event not found
$ awk '!x[$1]++' file
x[$1]++': Event not found
$ /usr/xpg4/bin/awk '!x[$1]++' file
x[$1]++': Event not found

More UNIX and Linux Forum Topics You Might Find Helpful
Thread	Thread Starter	Forum	Replies	Last Post
print line before column 1 transitions	ajp7701	Shell Programming and Scripting	2	04-17-2008 11:05 PM
how to read the column and print the values under that column	gemini106	Shell Programming and Scripting	6	03-28-2008 07:05 AM
Can we use 'tr' command to print 5th column of output of 'ls -l'	Nidhi2177	Shell Programming and Scripting	4	09-17-2007 06:53 AM
to print column using awk	cdfd123	Shell Programming and Scripting	2	07-26-2007 01:15 PM
can awk print column using a variable ??	jambesh	Shell Programming and Scripting	36	09-26-2006 07:39 AM

Thread Tools	Search this Thread
Show Printable Version Email this Page	Search this Thread: Advanced Search
Display Modes	Rate This Thread
Switch to Linear Mode Hybrid Mode Switch to Threaded Mode	Rate This Thread:


	Powered by