awk pattern match and count unique in column


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting awk pattern match and count unique in column
# 1  
Old 04-26-2013
awk pattern match and count unique in column

Hi all I have a need of searching some pattern in file by month and then count unique records

Code:
D11
G11
R11  -------> Pattern available in file
S11 
Jan

Code:
$1 to $5 column

contains some records in which I want to find unique

for this purpose I have written script like below

Code:
awk '/Jan/ || /JAN/ || /January/&&!/D11/&&!/G11/&&!/R11/&&!/S11/&&!_[$1$2$3$4$5]++{n++}END{if(n-1==-1)p=0;else p=n-1;printf "%s%s%s","Uniques","\t:\t",p;printf "\n"}' n=0 $file >>Quality.log

Here my code is finding unique if I remove following part from code
Code:
'!/D11/&&!/G11/&&!/R11/&&!/S11/&&

this is working
Code:
awk '!_[$1$2$3$4$5]++{n++}END{if(n-1==-1)p=0;else p=n-1;printf  "%s%s%s","Uniques","\t:\t",p;printf "\n"}' n=0 $file >>Quality.log

actual problem is it's not searching all pattern

Is there any other simple way to search pattern by monthly and to write unique count of column 1 2 3 4 and 5 to log file

right now I am pasting same code 12 times by changing month field
Code:
Jan Feb Mar .....Dec

and appending to log file, but my hard luck it's not searching all pattern
# 2  
Old 04-26-2013
I'm afraid it's not entirely clear what you want, programs which don't work for you are not nearly as illuminating as a good unabridged sample of the input data you have and the output data you want would be.
# 3  
Old 04-26-2013
Thank corona

I am trying achieve following

assume file contains 12 months records which can easily be searched like this

Code:
awk '/Jan/{print}' file

lists all rows contains string Jan

and following code prints unique records considering column 1 to 5

Code:
awk '!_[$1$2$3$4$5]++' file

with small modification to above code following gives count of unique records

Code:
awk '!_[$1$2$3$4$5]++{n++}END{print n-1}' n=1 file

along with
Code:
Jan Feb ...Dec

string file contains some more strings.. which I mentioned in #1

whenever awk finds multiple patterns in $0, I want to store that $0 in array, and finally I want to count and write number of unique records available for that particular search

The same process I have to do 12 times {Jan .....Dec}

I am sorry to say, due to the restriction from department I am unable to attach sample data

if you understand the scenario please help me.
# 4  
Old 04-26-2013
Hi there,
you would do well if you could explain the requirement in plain English. Those $0's, $1's, etc. only confuse a reader and make your intent difficult to understand.

You could post a sample by redacting the sensitive portions.
# 5  
Old 04-26-2013
Give real data example on you like your output.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

If pattern match in other column, modify column 3.

My command sed will modify everything in column 3 if i will use the command below. I want to search for a pattern then modify everything in column 3. sed -i 's/\|165\|/server1/g' file.txt Input: 01-31-2019 19:14:05|device|165|1548962040165|5c5348f9-0804-1111|file_attach|7271|587|smtp|... (6 Replies)
Discussion started by: invinzin21
6 Replies

2. Shell Programming and Scripting

Count number of unique values in each column of array

What is an efficient way of counting the number of unique values in a 400 column by 1000 row array and outputting the counts per column, assuming the unique values in the array are: A, B, C, D In other words the output should look like: Value COL1 COL2 COL3 A 50 51 52... (16 Replies)
Discussion started by: Geneanalyst
16 Replies

3. UNIX for Beginners Questions & Answers

Count unique column

Hello, I am trying to count unique rows in my file based on 4 columns (2-5) and to output its frequency in a sixth column. My file is tab delimited My input file looks like this: Colum1 Colum2 Colum3 Colum4 Coulmn5 1.1 100 100 a b 1.1 100 100 a c 1.2 200 205 a d 1.3 300 301 a y 1.3 300... (6 Replies)
Discussion started by: nans
6 Replies

4. UNIX for Beginners Questions & Answers

Awk: count unique element of array

Hi, tab-separated input: blabla_1 A,B,C,C blabla_2 A,E,G blabla_3 R,Q,A,B,C,R,Q output: blabla_1 3 blabla_2 3 blabla_3 5 After splitting $2 in an array, I am trying to store the number of unique elements in a variable, but have some difficulties resetting the variable to 0 before... (6 Replies)
Discussion started by: beca123456
6 Replies

5. Shell Programming and Scripting

Count occurrence of column one unique value having unique second column value

Hello Team, I need your help on the following: My input file a.txt is as below: 3330690|373846|108471 3330690|373846|108471 0640829|459725|100001 0640829|459725|100001 3330690|373847|108471 Here row 1 and row 2 of column 1 are identical but corresponding column 2 value are... (4 Replies)
Discussion started by: angshuman
4 Replies

6. Shell Programming and Scripting

awk unique count of partial match with semi-colon

Trying to get the unique count of the below input, but if the text in beginning of $5 is a partial match to another line in the file then it is not unique. awk awk '!seen++ {n++} END {print n}' input 7 input chr1 159174749 159174770 chr1:159174749-159174770 ACKR1 chr1 ... (2 Replies)
Discussion started by: cmccabe
2 Replies

7. Shell Programming and Scripting

awk to count using each unique value

Im looking for an awk script that will take the unique values in column 5, then print and count the unique values in column 6. CA001011500 11111 11111 -9999 201301 AAA CA001012040 11111 11111 -9999 201301 AAA CA001012573 11111 11111 -9999 201301 BBB CA001012710 11111 11111 -9999 201301... (4 Replies)
Discussion started by: ncwxpanther
4 Replies

8. Shell Programming and Scripting

Count frequency of unique values in specific column

Hi, I have tab-deliminated data similar to the following: dot is-big 2 dot is-round 3 dot is-gray 4 cat is-big 3 hot in-summer 5 I want to count the frequency of each individual "unique" value in the 1st column. Thus, the desired output would be as follows: dot 3 cat 1 hot 1 is... (5 Replies)
Discussion started by: owwow14
5 Replies

9. Shell Programming and Scripting

awk count how many unique IPs have received that error

Hi all, I want to write a awk script that counts unique IPs that have received one special error. For example 25-04-2012;192.168.70.31;1254545454545417;500.0;SUCCESS 25-04-2012;192.168.70.32;355666650914;315126423993;;General_ERROR_23 30-04-2012;192.168.70.33;e;null;null;Failure... (2 Replies)
Discussion started by: arrals_vl
2 Replies

10. UNIX for Dummies Questions & Answers

Finding Unique strings which match pattern

I need to grep for a pattern in a file. Files are huge and have several repeated occurances of the strings which match pattern. I just need the strings which contain the pattern in the output. For eg. The contents of my file are as follows. The pattern I want to match by is ABCD ... (5 Replies)
Discussion started by: tektips
5 Replies
Login or Register to Ask a Question