Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Word Occurrences script using awk

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Word Occurrences script using awk
# 8  
Old 11-01-2014
Unfortunately the gnu awk array sort functions destroy 1 of either the array index or the data.

So we have to copy the count and work into another array and then sort that. I then split the count and word out into v[1] and v[2]

Note the leading zeros in the printf format, this is to assure that alpha sorting will get the count in the correct order and not 1,10,100,11,...,18,19,2,20 etc.
(compare printf "%d\n" {1..100} | sort with printf "%03d\n" {1..100} | sort)

Code:
{
$0 = tolower($0)
for(i = 1; i <= NF; i++ )
freq[$i]++
}
BEGIN { printf "%-5s %20s %6s\n", "Index", "Word", "Count" }
END {
        for(word in freq)
           res[sprintf("%08i:%s",freq[word],word)];

        n = asorti(res)
        for (i=n; i; i--) {
            split(res[i],v,":")
            printf "%-5s %20s %6s\n", n-i+1, v[1]+0, v[2]
        }
}

This User Gave Thanks to Chubler_XL For This Post:
ksmarine1980
# 9  
Old 11-01-2014
Thank you so much, Chubler. Last question - now say I wanted to sort the array by the word itself (but still display the index & count), would I just switch the asorti function? Sorry to be a pain, but I want to understand how to manipulate these arrays so I can design them in the future for multiple purposes.

Output in this example would look something like:
Index Word Count
1043 the 247
1044 their 84
1045 them 15
# 10  
Old 11-02-2014
Yes as you proposed the res[] index would need to change to the sort order you would like so the:00001043. This would mean the results from the split would be v[1] = word and v[2]+0 = count
This User Gave Thanks to Chubler_XL For This Post:
ksmarine1980
# 11  
Old 11-03-2014
I must be doing something wrong. Instead of outputting the actual word, it's throwing out 00001787 for instance. The index and counts appear to be correct. Would you mind taking a look at this? Thanks!

Code:
{
        $0 = tolower($0)
        for ( i = 1; i <= NF; i++ )
        freq[$i]++
}
BEGIN { printf "%-5s %20s %6s\n", "Index", "Word", "Count"}
END {
        for (word in freq)
          res[sprintf("%08i:%s",word,freq[word])];

        n = asorti(res)
        for ( i = n; i; i--) {
        split(res[i], v, ":")
        printf "%-5s %20s %6s\n", n-i+1, v[1], v[2]+0
        }
}

Last edited by ksmarine1980; 11-03-2014 at 09:41 PM..
# 12  
Old 11-03-2014
Few things:

1: Change sprintf formats around you want %s for string followed by %08i for integer:
res[sprintf("%s:%08i",word,freq[word])];

2: Change order you want ascending order:
for ( i = 1; i<=n; i++) {

3: Change index value as we are going from 1 just display i:
printf "%-5s %20s %6s\n", i, v[1], v[2]+0
This User Gave Thanks to Chubler_XL For This Post:
ksmarine1980
# 13  
Old 11-03-2014
Chubler, you've been a tremendous help! I can't thank you enough. And thanks for making a UNIX newbie like myself feel welcome.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

awk or sed script to count number of occurrences and creating an average

Hi Friends , I am having one problem as stated file . Having an input CSV file as shown in the code U_TOP_LOGIC/U_HPB2/U_HBRIDGE2/i_core/i_paddr_reg_2_/Q,1,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,0,0... (4 Replies)
Discussion started by: kshitij
4 Replies

2. UNIX for Advanced & Expert Users

Find 2 occurrences of a word and print file names

I was thinking something like this but it always gets rid of the file location. grep -roh base. | wc -l find . -type f -exec grep -o base {} \; | wc -l Would this be a job for awk? Would I need to store the file locations in an array? (3 Replies)
Discussion started by: cokedude
3 Replies

3. Shell Programming and Scripting

awk Group By and count string occurrences

Hi Gurus, I'm scratching my head over and over and couldn't find the the right way to compose this AWK properly - PLEASE HELP :confused: Input: c,d,e,CLICK a,b,c,CLICK a,b,c,CONV c,d,e,CLICK a,b,c,CLICK a,b,c,CLICK a,b,c,CONV b,c,d,CLICK c,d,e,CLICK c,d,e,CLICK b,c,d,CONV... (6 Replies)
Discussion started by: Royi
6 Replies

4. UNIX for Dummies Questions & Answers

Awk: Counting occurrences between two files

Hi, I have two text files (1.txt and 2.txt). 2.txt contains two columns which are extracted from 1.txt using a simple if(condition) print. I want to: - count how many times the values contained in 2.txt appear in 1.txt -if they appear just one time, I have to delete the entire row in... (5 Replies)
Discussion started by: Pintug
5 Replies

5. UNIX for Dummies Questions & Answers

BASH - Counting word occurrences in a Web Page

Hi all, I have to do a script bash (for university) that counts all word occurrences in a specific web page. anyone can help me?. Thanks :) (1 Reply)
Discussion started by: piacentero
1 Replies

6. Shell Programming and Scripting

Script to count word occurrences, but exclude some?

I am trying to count the occurrences of ALL words in a file. However, I want to exclude certain words: short words (i.e. <3 chars), and words contained in an blacklist file. There is also a desire to count words that are capitalized (e.g. proper names). I am not 100% sure where the line on... (5 Replies)
Discussion started by: Cronk
5 Replies

7. Shell Programming and Scripting

Count occurrences in awk

Hello, I have an output from GDB with many entries that looks like this 0x00007ffff7dece94 39 in dl-fini.c 0x00007ffff7dece97 39 in dl-fini.c 0x00007ffff7ab356c 50 in exit.c 0x00007ffff7aed9db in _IO_cleanup () at genops.c:1022 115 in dl-fini.c 0x00007ffff7decf7b in _dl_sort_fini (l=0x0,... (6 Replies)
Discussion started by: ikke008
6 Replies

8. Homework & Coursework Questions

Du without directory and Grep for occurrences of a word

Assistance on work Use and complete the template provided. The entire template must be completed. If you don't, your post may be deleted! 1. The problem statement, all variables and given/known data: Files stored in ... (1 Reply)
Discussion started by: alindner
1 Replies

9. Shell Programming and Scripting

awk and gsub - how to replace only the first X occurrences

I have a text (text.txt) and I would like to replace only the first 2 occurrences of a word (but I might need to replace more): For example, if text is this: CAR sweet head hat red yellow CAR book brown tiger CAR cow CAR CAR milk I would like to replace the word "CAR" with word... (12 Replies)
Discussion started by: bingel
12 Replies

10. Shell Programming and Scripting

Count the number of occurrences of the word

I am a newbie in UNIX shell script and seeking help on this UNIX function. Please give me a hand. Thanks. I have a large file. Named as 'MyFile'. It was tab-delmited. I am told to write a shell function that counts the number of occurrences of the ord “mysring” in the file 'MyFile'. (1 Reply)
Discussion started by: duke0001
1 Replies
Login or Register to Ask a Question