Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Printing uniq first field with the the highest second field

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Printing uniq first field with the the highest second field
# 1  
Old 05-09-2013
Printing uniq first field with the the highest second field
Hi All,


I am searching for a script which will produce an output file with the uniq first field with the second field having highest value among all the duplicates..

The output file will produce only the uniqs which are duplicate 3 times..

Input file
Code:
X 9
B 5
A 1
Z 9
T 4
C 9
A 4
C 3
B 2
T 7 
C 10
A 5
B 1
X 8


Output file
Code:
B 5
A 5
C 10

I am lost in various options using awk, sort, printf etc Smilie

Help me please..
Last edited by joeyg; 05-09-2013 at 08:41 AM..
# 2  
Old 05-09-2013
Here is one solution using awk. Is the order of the output important?

Code:
 awk '($2 > array[$1] ) {array[$1] = $2} END {for(x in array)  {split(x,field); print field[1], array[x]}}' input.txt                         
A 5
B 5
C 10

# 3  
Old 05-10-2013
I ran the command using input.txt content as
Code:
X 9
B 5
A 1
Z 9
T 4
C 9
A 4
C 3
B 2
T 7 
C 10
A 5
B 1
X 8

Ouput became
Code:
A 5
B 5
C 10
T 7
X 9
Z 9

But I need output of 3 occurrence only not all..

So the expected output should be
Code:
A 5
B 5
C 10

# 4  
Old 05-10-2013
Try
Code:
$ sort -k1,1 -k2,2g file | awk 'TMP1 != $1 && A[TMP1] >= 3 {print TMP0} {TMP0=$0; TMP1=$1; A[$1]++} END {if (A[TMP1] >=3) print TMP0}'
A 5
B 5
C 10

# 5  
Old 05-10-2013
Showing the usage

Code:
usage: sort [-cmu] [-o output] [-T directory] [-S mem] [-z recsz]
        [-dfiMnr] [-b] [-t char] [-k keydef] [+pos1 [-pos2]] files...

# 6  
Old 05-10-2013
Quote:
But I need output of 3 occurrence only not all
Here is a way to do it using awk.
Code:
$ awk '{cnts[$1]++} $2 > fld2[$1] {fld2[$1]=$2; fld0[$1]=$0} END {for (key in cnts) { if (cnts[key] == 3) print fld0[key] }}' input3
A 5
B 5
C 10

This User Gave Thanks to hanson44 For This Post:
ailnilanjan
# 7  
Old 05-10-2013
Quote:
Originally Posted by ailnilanjan
Showing the usage

Code:
usage: sort [-cmu] [-o output] [-T directory] [-S mem] [-z recsz]
        [-dfiMnr] [-b] [-t char] [-k keydef] [+pos1 [-pos2]] files...

That should work, what is your OS and version?


---
Alternative for 3 or more occurrences, not exactly 3 (not sure what you require):
Code:
awk '$2>=M[$1]{M[$1]=$2} ++A[$1]==3{print $1,M[$1]}' file

**edit **
this solution is broken, since it will not always print the highest value if there are more than 3 values.. (thanks RudiC)
Last edited by Scrutinizer; 05-10-2013 at 05:48 AM..
This User Gave Thanks to Scrutinizer For This Post:
ailnilanjan
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Need help in awk: running a loop with one column and segregate data 4 each uniq value in that field

Hi All, I have a file like this(having 2 column). Column 1: like a,b,c.... Column 2: having numbers. I want to segregate those numbers based on column 1. Example: file. a 5 b 9 b 620 a 710 b 230 a 330 b 1910 (4 Replies)
Discussion started by: Raza Ali
4 Replies

2. Shell Programming and Scripting

Filter uniq field values (non-substring)

Hello, I want to filter column based on string value. All substring matches are filtered out and only unique master strings are picked up. infile: 1 abcd 2 abc 3 abcd 4 cdef 5 efgh 6 efgh 7 efx 8 fgh Outfile: 1 abcd 4 cdef 5 efgh 7 efxI have tried awk '!a++; match(a, $2)>0'... (32 Replies)
Discussion started by: yifangt
32 Replies

3. Shell Programming and Scripting

Sort field and uniq

I have a flatfile A.txt 2012/12/04 14:06:07 |trees|Boards 2, 3|denver|mekong|mekong12 2012/12/04 17:07:22 |trees|Boards 2, 3|denver|mekong|mekong12 2012/12/04 17:13:27 |trees|Boards 2, 3|denver|mekong|mekong12 2012/12/04 14:07:39 |rain|Boards 1|tampa|merced|merced11 How do i sort and get... (3 Replies)
Discussion started by: sabercats
3 Replies

4. Shell Programming and Scripting

Uniq based on first field

Hi New to unix. I want to display only the unrepeated lines from a file using first field. Ex: 1234 uname1 status1 1235 uname2 status2 1234 uname3 status3 1236 uname5 status5 I used sort filename | uniq -u output: 1234 uname1 status1 1235 uname2 status2 1234 uname3 status3 1236... (10 Replies)
Discussion started by: venummca
10 Replies

5. Shell Programming and Scripting

printing the max of each field

hi guys im very new to scripting and i have a problem. i need to use awk in my script and the script needs to print the max for each of the columns in a file. for example: numbers.txt 10 15 20 30 40 58 25 30 15 10 38 10 38 8 9 ./max numbers.txt 58 25 38 30 40 i have no clue on how to... (4 Replies)
Discussion started by: youness
4 Replies

6. Shell Programming and Scripting

How to only display lines where a field has the highest number?

Hello Wondering if anybody can advise me how I can sort the below file so it only displays lines with the latest versions of an object? As you'll see some of the scripts listed in my file have more than one version number (version number is after the file extension). E.g. cdm_bri.pkb has... (2 Replies)
Discussion started by: Glyn_Mo
2 Replies

7. UNIX for Dummies Questions & Answers

find uniq lines in file, using the first field of line

Hello all, new to unix and have just found the forum. I think I will be here quite often, and hope that in time i will be able to provide soem help, role on not being a newbie anymore :) I have a question which iI am hoping someone could help me with. If i have a file with lines in in thus... (8 Replies)
Discussion started by: grom
8 Replies

8. Shell Programming and Scripting

How to use uniq on a certain field?

How can I use uniq on a certain field or what else could I use? If I want to use uniq on the second field and the output would remove one of the lines with a 5. bob 5 hand jane 3 leg jon 4 head chris 5 lungs (1 Reply)
Discussion started by: Bandit390
1 Replies

9. UNIX for Dummies Questions & Answers

How to uniq third field in a file

Hi ; I have a question regarding the uniq command in unix How do I uniq 3rd field in a file ? original file : zoom coord 39 18652 39 18652 zoom coord 39 18653 39 18653 zoom coord 39 18818 39 18818 zoom coord 39 18840 39 18840 zoom coord 41 15096 41 15096 zoom... (1 Reply)
Discussion started by: babycakes
1 Replies

10. UNIX for Dummies Questions & Answers

Uniq using only the first field

Hi all, I have a file that contains a list of codes (shown below). I want to 'uniq' the file using only the first field. Anyone know an easy way of doing it? Cheers, Dave ##### Input File ##### 1xr1 1xws 1yxt 1yxu 1yxv 1yxx 2o3p 2o63 2o64 2o65 1xr1 1xws 1yxt 1yxv 1yxx 2o3p 2o63 2o64... (8 Replies)
Discussion started by: Digby
8 Replies
Login or Register to Ask a Question