Sponsored Content
Top Forums Shell Programming and Scripting Count number of occurences using awk Post 302793875 by prashant2507198 on Sunday 14th of April 2013 09:51:47 AM
Old 04-14-2013
Quote:
Originally Posted by elixir_sinari
Something which works on your sample data:
Code:
awk 'FNR==NR{c[$1];next}$1 in c{++c[$1]}END{for(i in c) print i"-"c[i]}' file1 file2

I forgot to mention that file2 is quite big in GB so what would be good to split it first in smaller pieces and run on all of them parallel to get results faster?
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to count the number of occurences of this pattern?

Hi all, I have a pattern like this in a file: 123 4 56 789 234 5 67 789 121 3 56 789 222 4 65 789 321 6 90 100 478 8 40 789 243 7 80 789 How can I count the number of occurences of '789' (4th column) in this set...? Thanks for all your help! K (7 Replies)
Discussion started by: kripssmart
7 Replies

2. UNIX for Dummies Questions & Answers

Count number of occurences of a word

I want to count the number of occurences of say "200" in a file but that file also contains various stuff including dtaes like 2007 or smtg like 200.1 so count i am getting by doing grep -c "word" file is wrong Please help!!!!! (8 Replies)
Discussion started by: shikhakaul
8 Replies

3. Shell Programming and Scripting

awk counting number of occurences

Hi, I am trying to count the max number of occurences of field1 in my apache log example: 10.0.0.1 field2 field3 10.0.0.2 filed2 field3 10.0.0.1 field2 field3 10.0.0.1 field2 field3 awk result to print out only the most occurence of field1 and number of occurence and field1 is... (3 Replies)
Discussion started by: phamp008
3 Replies

4. Shell Programming and Scripting

Count number of occurences of a character in a field defined by the character in another field

Hello, I have a text file with n lines in the following format (9 column fields): Example: contig00012 149606 G C 49 68 60 18 c$cccccacccccccccc^c I need to count the number of lower-case and upper-case occurences in column 9, respectively, of the... (3 Replies)
Discussion started by: s052866
3 Replies

5. Shell Programming and Scripting

to count the number of occurences of a column value

im trying to count the number of occurences of column 2 value(starting from KKK*) of the below file, file.txt using the code cat file.txt | awk ' BEGIN { print "Category Counts"} {FS=","} {NR > 2} { cats = cats + 1} END { for(c in cats) { print c, "=", cats} } ' but its returning as ... (6 Replies)
Discussion started by: michaelrozar17
6 Replies

6. Shell Programming and Scripting

Awk to count occurences

Hi, i am in need of an awk script to accomplish the following: Input table looks like: Student1 arts Student2 science Student3 arts Student4 science Student5 science Student6 science Student7 science Student8 science Student9 science Student10 science Student11 science... (8 Replies)
Discussion started by: saint2006
8 Replies

7. UNIX for Dummies Questions & Answers

Count pattern occurences

hi, I have a text..and i need to find a pattern in the text and count to the no of times the pattern occured. i have used grep command ..but the problem is , it shows the occurrences of the pattern but doesn't count no of times the pattern occuries. (5 Replies)
Discussion started by: nvnni
5 Replies

8. Shell Programming and Scripting

awk count occurences

line number:status, market, keystation 1,SENT,EBS,1 : 1 2,DONE,REU,1 : 1 3,SENT,EBS,2 : 1 4,DONE,EBS,1 : 0 5,SENT,EBS,2 : 0 6,SENT,EBS,2 : 0 7,SENT,EBS,2 : 0 8,SENT,EBS,1 : 1 for each status, market combination I want to keep a tally of active orders. i.e if an order is SENT, then +1, if... (8 Replies)
Discussion started by: Calypso
8 Replies

9. Shell Programming and Scripting

[solved]awk count occurences in time window

Input File Time, KeyStation 00:00:00,000;KS_1 00:00:01,000;KS_1 00:00:02,000;KS_1 00:00:03,000;KS_1 00:00:04,000;KS_1 00:00:05,000;KS_1 00:00:06,000;KS_1 00:01:01,000;KS_1 00:01:02,000;KS_1 00:01:03,000;KS_1 00:01:04,000;KS_1 00:01:05,000;KS_1 00:01:06,000;KS_1 01:00:01,000;KS_1... (0 Replies)
Discussion started by: Calypso
0 Replies

10. Shell Programming and Scripting

Count and print the number of occurences

I have some files as shown below GLL ALM 654-656 654 656 SEM LYG 655-657 655 657 SEM LYG 655-657 655 657 ALM LEG 656-658 656 658 ALM LEG 656-658 656 658 ALM LEG 656-658 656 658 LEG LEG 658-660 658 660 LEG LEG 658-660 658 660 The value of GLL is... (5 Replies)
Discussion started by: arch
5 Replies
DIFF(1) 						      General Commands Manual							   DIFF(1)

NAME
diff - print differences between two files SYNOPSIS
diff [-c | -e | -C n] [-br]file1 file2 OPTIONS
-C n Produce output that contains n lines of context -b Ignore white space when comparing -c Produce output that contains three lines of context -e Produce an ed-script to convert file1 into file2 -r Apply diff recursively to files and directories of EXAMPLES
diff file1 file2 # Print differences between 2 files diff -C 0 file1 file2 # Same as above diff -C 3 file1 file2 # Output three lines of context with every diff -c file1 file2 # Same diff /etc /dev # Compares recursively the directories /etc and /dev diff passwd /etc # Compares ./passwd to /etc/passwd DESCRIPTION
the same name, when file1 and file2 are both directories" difference encountered" Diff compares two files and generates a list of lines telling how the two files differ. Lines may not be longer than 128 characters. If the two arguments on the command line are both directories, diff recursively steps through all subdirectories comparing files of the same name. If a file name is found only in one directory, a diagnostic message is written to stdout. A file that is of either block special, character special or FIFO special type, cannot be compared to any other file. On the other hand, if there is one directory and one file given on the command line, diff tries to compare the file with the same name as file in the directory directory. SEE ALSO
cdiff(1), cmp(1), comm(1), patch(1). DIFF(1)
All times are GMT -4. The time now is 04:24 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy