Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

awk to count using each unique value

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting awk to count using each unique value
# 1  
Old 05-06-2014
awk to count using each unique value
Im looking for an awk script that will take the unique values in column 5, then print and count the unique values in column 6.

Code:
CA001011500  11111 11111 -9999 201301 AAA
CA001012040  11111 11111 -9999 201301 AAA
CA001012573  11111 11111 -9999 201301 BBB
CA001012710  11111 11111 -9999 201301 BBB
CA001015105  11111 11111 -9999 201301 BBB
CA001015638  11111 11111 -9999 201301 CCC
CA001011500  11111 11111  -9999 201302 AAA
CA001012040  11111 11111  -9999 201302 BBB
CA001012573  11111 11111 -9999 201302 BBB
CA001012710  11111  11111 -9999 201302 CCC
CA001015105  11111 11111  -9999 201302 CCC
CA001015638  11111 11111  -9999 201302 CCC

Output will need to be:

Code:
201301 AAA 2
201301 BBB 3
201301 CCC 1
201302 AAA 1
201302 BBB 2
201302 CCC 3

I know this will print the unique values of column 5, but how do i tie in the count of unique values in column 6?
Code:
 awk '{ a[$5]++ } END { for (b in a) { print b } }'

Thanks in advance!
# 2  
Old 05-06-2014
Use field 5 and 6 as index:
Code:
awk '{A[$5 OFS $6]++}END{for(k in A) print k, A[k]}' file

to get sorted output:
Code:
awk '{A[$5 OFS $6]++}END{for(k in A) print k, A[k]}' file | sort

This User Gave Thanks to Yoda For This Post:
ncwxpanther
# 3  
Old 05-06-2014
Quote:
Originally Posted by Yoda
Use field 5 and 6 as index:
Code:
awk '{A[$5 OFS $6]++}END{for(k in A) print k, A[k]}' file

to get sorted output:
Code:
awk '{A[$5 OFS $6]++}END{for(k in A) print k, A[k]}' file | sort

This works as expected. Thanks!
# 4  
Old 05-06-2014
try also:
Code:
awk '$0=$5 FS $6' infile | uniq -c

# 5  
Old 05-06-2014
As long as the unique values you want to count are on adjacent lines in your input file, you can easily do it all in awk without needing sort or uniq:
Code:
awk '
last != $5 OFS $6 {
        if(c) print last, c
        c = 1
        last = $5 OFS $6
        next
}
{       c++}
END {   print last, c}' file

If you want to try this on a Solaris/SunOS system, change awk to /usr/xpg4/bin/awk, /usr/xpg6/bin/awk, or nawk.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Awk: count unique elements in a field and sum their occurence across the entire file

Hi, Sure it's an easy one, but it drives me insane. input ("|" separated): 1|A,B,C,A 2|A,D,D 3|A,B,B I would like to count the occurence of each capital letters in $2 across the entire file, knowing that duplicates in each record count as 1. I am trying to get this output... (5 Replies)
Discussion started by: beca123456
5 Replies

2. UNIX for Beginners Questions & Answers

Count unique column

Hello, I am trying to count unique rows in my file based on 4 columns (2-5) and to output its frequency in a sixth column. My file is tab delimited My input file looks like this: Colum1 Colum2 Colum3 Colum4 Coulmn5 1.1 100 100 a b 1.1 100 100 a c 1.2 200 205 a d 1.3 300 301 a y 1.3 300... (6 Replies)
Discussion started by: nans
6 Replies

3. UNIX for Beginners Questions & Answers

Count unique words

Dear all, I would like to know how to list and count unique words in thousands number of text files. Please help me out thanks in advance (9 Replies)
Discussion started by: imranrasheedamu
9 Replies

4. UNIX for Beginners Questions & Answers

Awk: count unique element of array

Hi, tab-separated input: blabla_1 A,B,C,C blabla_2 A,E,G blabla_3 R,Q,A,B,C,R,Q output: blabla_1 3 blabla_2 3 blabla_3 5 After splitting $2 in an array, I am trying to store the number of unique elements in a variable, but have some difficulties resetting the variable to 0 before... (6 Replies)
Discussion started by: beca123456
6 Replies

5. Shell Programming and Scripting

Count occurrence of column one unique value having unique second column value

Hello Team, I need your help on the following: My input file a.txt is as below: 3330690|373846|108471 3330690|373846|108471 0640829|459725|100001 0640829|459725|100001 3330690|373847|108471 Here row 1 and row 2 of column 1 are identical but corresponding column 2 value are... (4 Replies)
Discussion started by: angshuman
4 Replies

6. Shell Programming and Scripting

awk unique count of partial match with semi-colon

Trying to get the unique count of the below input, but if the text in beginning of $5 is a partial match to another line in the file then it is not unique. awk awk '!seen++ {n++} END {print n}' input 7 input chr1 159174749 159174770 chr1:159174749-159174770 ACKR1 chr1 ... (2 Replies)
Discussion started by: cmccabe
2 Replies

7. UNIX for Dummies Questions & Answers

Search and count a unique string

Hi Guys, I have a file as follows. Here is my story: For each field, the string in the 5th column needs to be searched in other fields of the same column and counted if the 1st column of the field is different from that of the primary field. BTW, the unique strings of 1st column need to be... (6 Replies)
Discussion started by: a_bahreini
6 Replies

8. Shell Programming and Scripting

awk pattern match and count unique in column

Hi all I have a need of searching some pattern in file by month and then count unique records D11 G11 R11 -------> Pattern available in file S11 Jan$1 to $5 column contains some records in which I want to find unique for this purpose I have written script like below awk '/Jan/ ||... (4 Replies)
Discussion started by: nex_asp
4 Replies

9. Shell Programming and Scripting

awk count how many unique IPs have received that error

Hi all, I want to write a awk script that counts unique IPs that have received one special error. For example 25-04-2012;192.168.70.31;1254545454545417;500.0;SUCCESS 25-04-2012;192.168.70.32;355666650914;315126423993;;General_ERROR_23 30-04-2012;192.168.70.33;e;null;null;Failure... (2 Replies)
Discussion started by: arrals_vl
2 Replies

10. Shell Programming and Scripting

How to count unique strings

How do I count the total number of unique strings from a file using Perl? Any help is appreciated.. (6 Replies)
Discussion started by: my_Perl
6 Replies
Login or Register to Ask a Question