Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

counting the number of occurences

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting counting the number of occurences
# 1  
Old 04-28-2009
counting the number of occurences
say i've got a text file with >10million sequences:

ssss
ssss
tttttt
uuuuuu
uuuuuu
uuuuuu
...

I'd like to convert the file so that the output will report the number of occurence right by each sequence:

2 ssss
2 ssss
1 tttttt
3 uuuuuu
3 uuuuuu
3 uuuuuu
....

Is there an easy way to do this. there are 10 million lines, so I can't really use loops.

thanks!
# 2  
Old 04-28-2009
use uniq

uniq -c filename
# 3  
Old 04-28-2009
thanks for your reply.

however, that will result in:
Code:
2 ssss
1 tttttt
3 uuuuuu

what i'm doing so far is:
uniq -c, and then something like awk '{for(j=1;j<=$1;j++){print $1, $2} }' file.

however, this will take for ever to finish when dealing with a large file. is there a better way to do this??

thanks
Last edited by Yogesh Sawant; 04-28-2009 at 11:57 AM.. Reason: added code tags
# 4  
Old 04-28-2009
not sure whether below can help you some?

Code:
sort a.txt | uniq -c | awk '{for(i=1;i<=$1;i++)
        print $0
        }'

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Number of Consecutive Occurences

:b: Hi Folks, I have a requirement to find the number of consecutive occurences based on month. Here is the below sample of my requirement. I have say four columns CARUSERNAME BRAND_CODE MONTH YEAR Below is sample Data John|C|JAN|2013 Tim|C|FEB|2013 Tom|C|Mar|2013 Ron|C|Apr|2013... (4 Replies)
Discussion started by: dinesh1985
4 Replies

2. Shell Programming and Scripting

Count number of occurences using awk

Hi Guys, I have 2 files like below file1 xx yy file2 b yy b2 xx c1 yy xx yy Now I want an idea which can count occurences of text from file1 and file2 so outbout would be kind of (9 Replies)
Discussion started by: prashant2507198
9 Replies

3. Shell Programming and Scripting

Counting occurences in column

Hi guys! I have a problem writing script that would convert this input into this output: I have an awk script that counts occurences of a sign in a column, but don't know how to change it so that I would give me desired output. script awk '{count++}END{for(j in count)... (2 Replies)
Discussion started by: grincz
2 Replies

4. UNIX for Dummies Questions & Answers

Number of word occurences in a file?

Hello world, Can anybody tell me how to count how many times does a word repeat in a file? There have been many threads on this but they all are heavy loads of Scripting for a starter like me. :D So, I sat down today and after some hours of reading man pages, I found a simple one-line... (18 Replies)
Discussion started by: satish51392111
18 Replies

5. Shell Programming and Scripting

awk counting number of occurences

Hi, I am trying to count the max number of occurences of field1 in my apache log example: 10.0.0.1 field2 field3 10.0.0.2 filed2 field3 10.0.0.1 field2 field3 10.0.0.1 field2 field3 awk result to print out only the most occurence of field1 and number of occurence and field1 is... (3 Replies)
Discussion started by: phamp008
3 Replies

6. Shell Programming and Scripting

How to count the number of occurences of this pattern?

Hi all, I have a pattern like this in a file: 123 4 56 789 234 5 67 789 121 3 56 789 222 4 65 789 321 6 90 100 478 8 40 789 243 7 80 789 How can I count the number of occurences of '789' (4th column) in this set...? Thanks for all your help! K (7 Replies)
Discussion started by: kripssmart
7 Replies

7. UNIX for Dummies Questions & Answers

Counting number of occurences

Hi All, I have to count the number of occurences of the character " ; " in a given line. I had used the following awk command to achieve the same echo $KOP.dat|awk '{split($1,my,";"); for(i in my)c++ }END{print c-1}' My file KOP.dat had the following data ... (1 Reply)
Discussion started by: kingofprussia
1 Replies

8. Shell Programming and Scripting

number of occurences of a string

hi, I have a file where i need to count the occurences of a string ex) 'welcome to unix forum'. can anybody help me out (12 Replies)
Discussion started by: siddu_chittari
12 Replies

9. UNIX for Dummies Questions & Answers

Counting occurences of different strings in a file

Hi, i'd like to know if the following is possible with a shell script, and can't find the answer in the search. Suppose i have a logfile build like this: # 8 :riuyzp1028 # 38 : riuyzp1028 # 25 : riuyvzp1032 # 30 : nlkljpa0202 # 1 : nlklja0205 # 38 : riuyzp1028 # 25 :... (4 Replies)
Discussion started by: Freerider
4 Replies

10. UNIX for Dummies Questions & Answers

Counting occurences of specific charachter in a file

Hi, I need to count the number of occurences of the character " in a file that contains huge number of records. What command could I use? Please specify in detail since I am new :| Thanks much. (3 Replies)
Discussion started by: GMMike
3 Replies
Login or Register to Ask a Question