Login or Register to Ask a Question and Join Our Community


UNIX for Dummies Questions & Answers

extract first occurrence in a list

 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers extract first occurrence in a list
# 1  
Old 12-15-2009
extract first occurrence in a list
Hello everyone,

I need to extract an item the first time it appears in a list:

Input:
Code:
human pool.1 121 4e-10 75 89
human pool.1 112 2e-08 75 78
human pool.1 154 1e-07 75 72
human pool.1 122 4e-06 75 68
human pool.1 111 3e-02 75 54
Horse pool.1 222 4e-10 82 90
Horse pool.1 333 3e-09 81 80
Horse pool.1 444 2e-08 80 77
Horse pool.1 555 1e-07 69 66

output:
Code:
human pool.1 121 4e-10 75 89
Horse pool.1 222 4e-10 82 90

Does anyone know of an easy way to do this?

Thank you
Last edited by radoulov; 12-15-2009 at 01:27 PM.. Reason: Please use code tags!
# 2  
Old 12-15-2009
Code:
awk '!_[$1]++' infile

Use gawk, nawk or /usr/xpg4/bin/awk on Solaris.
# 3  
Old 12-15-2009
MySQL Thank you
Thank you, radoulov.

That worked beautifully.

viralnerd
# 4  
Old 12-16-2009
Hi radoulov,

could you please eloborate on the command.

Thanks in advance
# 5  
Old 12-16-2009
Code:
gawk '$1!=p{print}{p=$1}' file #according to your input

Last edited by ichigo; 12-16-2009 at 06:53 AM..
# 6  
Old 12-16-2009
Quote:
Originally Posted by ichigo
Code:
gawk '$1!=p{print}{p=$1}' file

Just to add that the code above won't work with not ordered input.

---------- Post updated at 11:50 AM ---------- Previous update was at 11:41 AM ----------
Quote:
Originally Posted by penchal_boddu
Hi radoulov,

could you please eloborate on the command.

Thanks in advance
An associative array named _ is created: $1 is the key, an incremented integer is the value.
Given the posted sample it yields:

Code:
% awk '{ print $0, "=>", _[$1]++ }' infile
human pool.1 121 4e-10 75 89 => 0
human pool.1 112 2e-08 75 78 => 1
human pool.1 154 1e-07 75 72 => 2
human pool.1 122 4e-06 75 68 => 3
human pool.1 111 3e-02 75 54 => 4
Horse pool.1 222 4e-10 82 90 => 0
Horse pool.1 333 3e-09 81 80 => 1
Horse pool.1 444 2e-08 80 77 => 2
Horse pool.1 555 1e-07 69 66 => 3

Consider that the NULL (empty) string "" and the numeric value 0 are evaluated as false in boolean context.

So in this case you're saying: perform the default action (print the current record) if the following expression _[$1]++ returns false (not true, hence the exclamation mark !).

Hope this helps.
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

sed print from last occurrence match until the end of last occurrence match

Hi, i have file file.txt with data like: START 03:11:30 a 03:11:40 b END START 03:13:30 eee 03:13:35 fff END jjjjjjjjjjjjjjjjjjjjj START 03:14:30 eee 03:15:30 fff END ggggggggggg iiiiiiiiiiiiiiiiiiiiiiiii I want the below output START (13 Replies)
Discussion started by: Jyotshna
13 Replies

2. Shell Programming and Scripting

awk to extract and print first occurrence of pattern in each line

I am trying to use awk to extract and print the first ocurrence of NM_ and NP_ with a : before in each line. The input file is tab-delimeted, but the output does not need to be. The below does execute but prints all the lines in the file not just the patterns. Thank you :). file tab-delimeted ... (2 Replies)
Discussion started by: cmccabe
2 Replies

3. Shell Programming and Scripting

Substitute first occurrence of keyword if occurrence between two other keywords

Assume a string that contains one or multiple occurrences of three different keywords (abbreviated as "kw"). I would like to replace kw2 with some other string, say "qux". Specifically, I would like to replace that occurrence of kw2 that is the first one that is preceded by kw1 somewhere in the... (4 Replies)
Discussion started by: M Gruenstaeudl
4 Replies

4. Shell Programming and Scripting

Extract the text between the nth occurrence of square brackets

Please can someone help with this? I have a file with lines as follows: word1 word2 word3 word4 word5 word6 word7 word8 word1 word2 word3 word4 word5 word6 word7 word8 word1 word2 word3 word4 word5 word6 word7 word8 word1 word2 word3 word4 word5 word6 word7 word8 When I use the... (7 Replies)
Discussion started by: Subhadeep_Sahu
7 Replies

5. Shell Programming and Scripting

Extract sequences based on the list

Hi, I have a file with more than 28000 records and it looks like below.. >mm10_refflat_ABCD range=chr1:1234567-2345678 tgtgcacactacacatgactagtacatgactagac....so on >mm10_refflat_BCD range=chr1:3234567-4545678... tgtgcacactacacatgactagtatgtgcacactacacatgactagta . . . . . so on ... (2 Replies)
Discussion started by: Diya123
2 Replies

6. Shell Programming and Scripting

Extract records from list

Hi Gents, I have a file 1 like this 1 1000 20 2 2000 30 3 1000 40 5 1000 50 And I have other file 1 like 2 1 I would like to get from the file 1 the complete line which are in file 2, the key to compare is the column 2 then output should be. 2 2000 30. I was trying to get it... (5 Replies)
Discussion started by: jiam912
5 Replies

7. UNIX for Advanced & Expert Users

Unix: list out Pattern occurrence (count)

Need to search a pattern occurrence (count) in a specified file. Below is the details $ cat fruits apple apple ball ball apple ball ball ball apple apple apple cat cat cat cat cat apple apple Note: If I'll use the grep command with -c option then it'll count the 1st occurrence in... (6 Replies)
Discussion started by: tp2115
6 Replies

8. UNIX for Dummies Questions & Answers

Extract out mail ID from from profile list

HI, i HAVE THE PROFILE LIST ldapsearch -D cn=root -w root -p 389 mail=dgmp_ndl@bsnl.in mail=dgmp_ndl@bsnl.in,o=Data Networks,o=Data Networks,o=Data Networks,o=Messagingdb,dc=bsnl,dc=in I NEED TO SEPERATE ONLY THE MAIL ID I m USING bash-2.05b$ ldapsearch -D cn=root -w root -p 389... (3 Replies)
Discussion started by: indoo.pathak
3 Replies

9. Shell Programming and Scripting

Extract large list of substrings

I have a very long string (millions of characters). I have a file with start location and length that is thousands of rows long: Start Length 5 10 16 21 44 100 215 37 ... I'd like to extract the substring that corresponds to the start and length from each row of the list: I tried... (7 Replies)
Discussion started by: dcfargo
7 Replies

10. Shell Programming and Scripting

extract lines with a given list of identifiers

Hi All, My question is if the simple but powerful shell scripts can extract data from a big data file by using a list of identifier. I used to put everything in the database and do joining, which sounds stupid but only way I knew. For example, my data file looks like, ... (3 Replies)
Discussion started by: mskcc
3 Replies
Login or Register to Ask a Question