Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Assigning the same frequency to more than one words in a file
Top Forums Shell Programming and Scripting Assigning the same frequency to more than one words in a file Post 302850865 by Don Cragun on Thursday 5th of September 2013 10:34:56 PM
Old 09-05-2013
The following awk script provides two lists (with an empty line between them). The first list provides:
Code:
NAME<tab>FREQUENCY

linesfor all NAMES on an input line and the second list provides
Code:
NAME<tab>FREQUENCY[<space>FREQUENCY]...

for all FREQUENCY entries for each NAME entry found in the input.
Code:
awk '
{       for(i = 1; i < NF; i++) {
                printf("%s\t%s\n", $i, $NF)
                ($i in a) ? a[$i] = a[$i] " " $NF : a[$i] = "\t" $NF
        }
}
END {   print ""
        for(i in a) print i a[i]
}' file

I know you're doing this on Windows, but if someone else wants to try it on a Solaris/SunOS system, they would need to use /usr/xpg4/bin/awk, /usr/xpg6/bin/awk, or nawk instead of awk. With your sample input, the output produced is:
Code:
SANDHYA	6901
DAS	6901
ARATI	6201
DAS	6201
KALPANA	4714
DAS	4714
GITA	4550
DAS	4550
BISWANATH	3949
DAS	3949
SWAPAN	3941
DAS	3941
SUKUMAR	3876
DAS	3876
GOPAL	3835
DAS	3835
SARASWATI	3769
DAS	3769
DILIP	3653
DAS	3653
TAPAN	3607
DAS	3607
ASHOKE	3604
DAS	3604
PRATIMA	3558
DAS	3558
PURNIMA	3546
DAS	3546
BASANTI	3372
DAS	3372
SHANKAR	3279
DAS	3279
SANDHYA	3254
GHOSH	3254
SANJAY	3252
DAS	3252
PRATIMA	3212
DAS	3212
KALPANA	3203
DAS	3203
ARATI	3155
GHOSH	3155
MALATI	3151
DAS	3151
SWAPAN	3138
DAS	3138
SANDHYA	3120
RANI	3120
DAS	3120
LAKSHMI	3104
DAS	3104
ANJALI	3085
DAS	3085

MALATI	3151
DILIP	3653
GOPAL	3835
TAPAN	3607
GHOSH	3254 3155
PURNIMA	3546
BASANTI	3372
SANJAY	3252
LAKSHMI	3104
RANI	3120
ARATI	6201 3155
ASHOKE	3604
KALPANA	4714 3203
SUKUMAR	3876
ANJALI	3085
GITA	4550
SARASWATI	3769
BISWANATH	3949
PRATIMA	3558 3212
DAS	6901 6201 4714 4550 3949 3941 3876 3835 3769 3653 3607 3604 3558 3546 3372 3279 3252 3212 3203 3151 3138 3120 3104 3085
SHANKAR	3279
SWAPAN	3941 3138
SANDHYA	6901 3254 3120

PS Note that the output produced here seems to match your input (but even after sorting is VERY different from the output you said you wanted). As an example, your input contains the lines:
Code:
SANDHYA GHOSH	3254
ARATI GHOSH	3155

but your output lines for GHOSH are:
Code:
GHOSH	6901
    and
GHOSH	3769

instead of:
Code:
GHOSH	3254
    and
GHOSH	3155

???
Last edited by Don Cragun; 09-05-2013 at 11:48 PM.. Reason: Note differences between my results and expected output.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Splitting Concatenated Words in Input File with Words from a Master File

Hello, I have a complex problem. I have a file in which words have been joined together: Theboy ranslowly I want to be able to correctly split the words using a lookup file in which all the words occur: the boy ran slowly slow put child ly The lookup file which is meant for look up... (21 Replies)
Discussion started by: gimley
21 Replies

2. Shell Programming and Scripting

count frequency of words in a file

I need to write a shell script "cmn" that, given an integer k, print the k most common words in descending order of frequency. Example Usage: user@ubuntu:/$ cmn 4 < example.txt :b: (3 Replies)
Discussion started by: mohit_iitk
3 Replies

3. Shell Programming and Scripting

Splitting concatenated words in input file with words from the same file

Dear all, I am working with names and I have a large file of names in which some words are written together (upto 4 or 5) and their corresponding single forms are also present in the word-list. An example would make this clear annamarie mariechristine johnsmith johnjoseph smith john smith... (8 Replies)
Discussion started by: gimley
8 Replies

4. Shell Programming and Scripting

Script to sort large file with frequency

Hello, I have a very large file of around 2 million records which has the following structure: I have used the standard awk program to sort: # wordfreq.awk --- print list of word frequencies { # remove punctuation #gsub(/_]/, "", $0) for (i = 1; i <= NF; i++) freq++ } END { for (word... (3 Replies)
Discussion started by: gimley
3 Replies

5. Shell Programming and Scripting

Sorting a file with frequency on length

Hello, I have a file which has the following structure word space Frequency The file is around 30,000 headwords each along with its frequency. The words have different lengths. What I need is a PERL or AWK script which can sort the file on length of the headword and once the file is sorted on... (12 Replies)
Discussion started by: gimley
12 Replies

6. Shell Programming and Scripting

Creating Frequency of words from a file by accessing a corpus

Hello, I have a large file of syllables /strings in Urdu. Each word is on a separate line. Example in English: be at for if being attract I need to identify the frequency of each of these strings from a large corpus (which I cannot attach unfortunately because of size limitations) and... (7 Replies)
Discussion started by: gimley
7 Replies

7. Shell Programming and Scripting

How count the number of two words associated with the two words occurring in the file?

Hi , I need to count the number of errors associated with the two words occurring in the file. It's about counting the occurrences of the word "error" for where is the word "index.js". As such the command should look like. Please kindly help. I was trying: grep "error" log.txt | wc -l (1 Reply)
Discussion started by: jmarx
1 Replies

8. UNIX for Dummies Questions & Answers

Replace the words in the file to the words that user type?

Hello, I would like to change my setting in a file to the setting that user input. For example, by default it is ONBOOT=ON When user key in "YES", it would be ONBOOT=YES -------------- This code only adds in the entire user input, but didn't replace it. How do i go about... (5 Replies)
Discussion started by: malfolozy
5 Replies

9. Shell Programming and Scripting

Frequency of Words in a File, sed script from 1980

tr -cs A-Za-z\' '\n' | tr A-Z a-z | sort | uniq -c | sort -k1,1nr -k2 | sed ${1:-25} < book7.txt This is not my script, it can be found way back from 1980 but once it worked fine to give me the most used words in a text file. Now the shell is complaining about an error in sed sed: -e... (5 Replies)
Discussion started by: 1in10
5 Replies

10. Shell Programming and Scripting

Replace particular words in file based on if finds another words in that line

Hi All, I need one help to replace particular words in file based on if finds another words in that file . i.e. my self is peter@king. i am staying at north sydney. we all are peter@king. How to replace peter to sham if it finds @king in any line of that file. Please help me... (8 Replies)
Discussion started by: Rajib Podder
8 Replies
All times are GMT -4. The time now is 06:59 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy