Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Print row if value in column 1 is the first occurence

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Print row if value in column 1 is the first occurence
# 1  
Old 03-17-2008
Print row if value in column 1 is the first occurence
Hi All,

I would like to have a script which is able to perform the below.
Print the whole row if column1 which is "0001" for the below example is the first occurrence. Subsequent "0001" occurrence will not be printed out and so on.

Can any expert help ?

Input:

0001 k= 40
0001 k= 2
0002 k= 1
0003 k= 1
0004 k= 77
0004 k= 1
0005 k= 88
0005 k= 6

Output:

0001 k= 40
0002 k= 1
0003 k= 1
0004 k= 77
0005 k= 88
# 2  
Old 03-17-2008
Code:
$ cat buf                                                                                                                                                               
0001 k= 40
0001 k= 2
0002 k= 1
0003 k= 1
0004 k= 77
0004 k= 1
0005 k= 88
0005 k= 6
$ perl -n -e '($num) = split /=/; next if $found[$num]; print; $found[$num] = 1' buf  
0001 k= 40
0002 k= 1
0003 k= 1
0004 k= 77
0005 k= 88

# 3  
Old 03-17-2008
$cat test
0001 k= 40
0001 k= 2
0002 k= 1
0003 k= 1
0004 k= 77
0004 k= 1
0005 k= 88
0005 k= 6



for i in `cat test|cut -d" " -f1`
do
grep "$i" test | head -1 >> out.txt
done
sort -u out.txt

0001 k= 40
0002 k= 1
0003 k= 1
0004 k= 77
0005 k= 88
# 4  
Old 03-17-2008
And the Python approach

Code:
#!/usr/local/bin/python

keys = {}

input = file('test')
for line in input:
   key = line.split(' ', 1)[0]
   if key not in keys:
      keys[key] = 1
      print line,

# 5  
Old 03-17-2008
Like a FAQ Smilie
Awk:

Code:
awk '!x[$1]++' file

Use nawk or /usr/xpg4/bin/awk on Solaris.

Perl:

Code:
perl -ane'print unless $x{$F[0]}++' file

# 6  
Old 03-17-2008
Brilliant, radoulov! Smilie
# 7  
Old 03-17-2008
Quote:
Originally Posted by radoulov
Like a FAQ Smilie
Awk:

Code:
awk '!x[$1]++' file

Use nawk or /usr/xpg4/bin/awk on Solaris.

Perl:

Code:
perl -ane'print unless $x{$F[0]}++' file

Hi radoulov,

The perl code seems to work but not the awk.
Can you help ? I am using solaris by the way.

Also, can you explain your perl code so that i can understand better. What is the function of "-ane"

Code:
$ nawk '!x[$1]++' file
x[$1]++': Event not found
$ awk '!x[$1]++' file
x[$1]++': Event not found
$ /usr/xpg4/bin/awk '!x[$1]++' file
x[$1]++': Event not found

Last edited by Raynon; 03-17-2008 at 10:12 PM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Print a row with the max number in a column

Hello, I have this table: chr1_16857_17742 - chr1 17369 17436 "ENST00000619216.1"; "MIR6859-1"; - 67 chr1_16857_17742 - chr1 14404 29570 "ENST00000488147.1"; "WASH7P"; - 885 chr1_16857_18061 - chr1 ... (5 Replies)
Discussion started by: coppuca
5 Replies

2. Shell Programming and Scripting

Print every alternate column in row in a text file

Hi, I have a comma separated file. I would like to print every alternate columns into a new row. Example input file: Name : John, Age : 30, DOB : 30-Oct-2018 Example output: Name,Age,DOB John,30,30-Oct-2018 (3 Replies)
Discussion started by: Lini
3 Replies

3. Shell Programming and Scripting

Print first row of column a, last row of column b if column a has the same value

I have a table with this structure: cola colb colc 1 19 lemon 20 31 lemon 32 100 lemon 159 205 cherries 210 500 cherries and need to parse it into this format: cola colb colc 1 100 lemon 159 500 cherries So I need the first row of cola and the last row of colb if colc has the... (3 Replies)
Discussion started by: coppuca
3 Replies

4. Shell Programming and Scripting

Print row on 4th column to all row

Dear All, I have input : SEG901 5173 9005 5740 SEG902 5227 5284 SEG903 5284 5346 SEG904 5346 9010 SEG905 5400 5456 SEG906 5456 5511 SEG907 5511 9011 SEG908 5572 9015 SEG909 5622 9020 SEG910 5678 5739 SEG911 5739 5796 SEG912 5796 9025 ... (3 Replies)
Discussion started by: attila
3 Replies

5. Shell Programming and Scripting

Print every 5 4th column values as separate row with different first column

Hi, I have the following file, chr1 100 200 20 chr1 201 300 22 chr1 220 345 23 chr1 230 456 33.5 chr1 243 567 90 chr1 345 600 20 chr1 430 619 21.78 chr1 870 910 112.3 chr1 914 920 12 chr1 930 999 13 My output would be peak1 20 22 23 33.5 90 peak2 20 21.78 112.3 12 13 Here the... (3 Replies)
Discussion started by: jacobs.smith
3 Replies

6. UNIX for Dummies Questions & Answers

awk to print first row with forth column and last row with fifth column in each file

file with this content awk 'NR==1 {print $4} && NR==2 {print $5}' file The error is shown with syntax error; what can be done (4 Replies)
Discussion started by: cdfd123
4 Replies

7. Shell Programming and Scripting

Print unique names in each row of a specific column using awk

Is it possible to remove redundant names in the 4th column? input cqWE 100 200 singapore;singapore AZO 300 400 brazil;america;germany;ireland;germany .... .... output cqWE 100 200 singapore AZO 300 400 brazil;america;germany;ireland (4 Replies)
Discussion started by: quincyjones
4 Replies

8. Shell Programming and Scripting

Print the row element till the next row element appear in a column

Hi all I have file with columns F3 pathway CPS F2 H2 H4 H5 H6 no pathway CMP H7 H8 H9 H10 My expected output is F3 pathway CPS F2 pathway CPS (10 Replies)
Discussion started by: Priyanka Chopra
10 Replies

9. Shell Programming and Scripting

AWK Script - Print a column - within a Row Range

Hi, Please read the whole thread. I have been working on this script below. It works fine, feel free to copy and test with the INPUT File below as well. example: PACKET DATA PROTOCOL CONTEXT DATA APNID PDPADD EQOSID VPAA PDPCH PDPTY PDPID 10 ... (6 Replies)
Discussion started by: panapty
6 Replies

10. Shell Programming and Scripting

How to print column based on row number

Hi, I want to print column value based on row number say multiple of 8. Input file: line 1 67 34 line 2 45 57 . . . . . . line 8 12 46 . . . . . . line 16 24 90 . . . . . . line 24 49 67 Output 46 90 67 (2 Replies)
Discussion started by: Surabhi_so_mh
2 Replies
Login or Register to Ask a Question