Sponsored Content
Top Forums Shell Programming and Scripting Finding the most frequently occurring set of words Post 302405955 by alister on Sunday 21st of March 2010 12:55:18 AM
Old 03-21-2010
Assuming that the patterns in
Code:
AIMLESSLY  EY1 M L AH0 S L IY0

are
Code:
$ echo 'AIMLESSLY  EY1 M L AH0 S L IY0' | awk '{for(i=2;i<NF;i++) {s=$i; for(j=i+1;j<=NF;j++) print s=s FS $j}}'
EY1 M
EY1 M L
EY1 M L AH0
EY1 M L AH0 S
EY1 M L AH0 S L
EY1 M L AH0 S L IY0
M L
M L AH0
M L AH0 S
M L AH0 S L
M L AH0 S L IY0
L AH0
L AH0 S
L AH0 S L
L AH0 S L IY0
AH0 S
AH0 S L
AH0 S L IY0
S L
S L IY0
L IY0

A list of of the occuring patterns sorted from most most common to least, ties sorted in increasing alphabetical order can be done with:
Code:
$ awk '{for(i=2;i<NF;i++) {s=$i; for(j=i+1;j<=NF;j++) print s=s FS $j}}' data | sort | uniq -c | sort -k1,1rn -k2
   3 EY1 M
   3 EY1 N
   3 L IY0
   2 EH1 R
   1 AE2 G
   1 AE2 G Z
   1 AH0 S
   1 AH0 S L
   1 AH0 S L IY0
   1 B AE2
   1 B AE2 G
   1 B AE2 G Z
   1 EH1 R B
   1 EH1 R B AE2
   1 EH1 R B AE2 G
   1 EH1 R B AE2 G Z
   1 EY1 L
   1 EY1 L Z
   1 EY1 M L
   1 EY1 M L AH0
   1 EY1 M L AH0 S
   1 EY1 M L AH0 S L
   1 EY1 M L AH0 S L IY0
   1 EY1 M OW2
   1 EY1 M OW2 N
   1 EY1 M Z
   1 EY1 N JH
   1 EY1 N L
   1 EY1 N L IY0
   1 EY1 N Z
   1 EY1 N Z L
   1 EY1 N Z L IY0
   1 EY1 NG
   1 G Z
   1 L AH0
   1 L AH0 S
   1 L AH0 S L
   1 L AH0 S L IY0
   1 L Z
   1 M L
   1 M L AH0
   1 M L AH0 S
   1 M L AH0 S L
   1 M L AH0 S L IY0
   1 M OW2
   1 M OW2 N
   1 M Z
   1 N JH
   1 N L
   1 N L IY0
   1 N Z
   1 N Z L
   1 N Z L IY0
   1 OW2 N
   1 R B
   1 R B AE2
   1 R B AE2 G
   1 R B AE2 G Z
   1 S L
   1 S L IY0
   1 Z L
   1 Z L IY0

If only the very first result is of interest append a head filter to the previous pipeline:
Code:
$ awk '{for(i=2;i<NF;i++) {s=$i; for(j=i+1;j<=NF;j++) print s=s FS $j}}' data | sort | uniq -c | sort -k1,1rn -k2 | head -n1
   3 EY1 M

If the count is of no interest:
Code:
$ awk '{for(i=2;i<NF;i++) {s=$i; for(j=i+1;j<=NF;j++) print s=s FS $j}}' data | sort | uniq -c | sort -k1,1rn -k2 | sed 's/ *[^ ]* *//;q'    
EY1 M

Regards,
Alister

Last edited by alister; 03-21-2010 at 02:03 AM..
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

finding no of counts the words occured

hi, cud u help me to find this. i hav 2 files. file1 has data as "ARUN ARUN is from Australia Arun likes America etc.. ARUN ARUN " file2 has "ARUN Australia America" i... (5 Replies)
Discussion started by: arunsubbhian
5 Replies

2. Shell Programming and Scripting

Display most top 10 occurring words along with number of ocurences of word inthe text

I need Display the most top 10 occurring words along with the number of occurences of those words in the given text. Sample text as below: "The Travails of Single South Indian men of conservative upbringing" or "Why we don't get any..." Yet another action packed weekend in Mumbai, full of... (2 Replies)
Discussion started by: smacherla
2 Replies

3. UNIX for Dummies Questions & Answers

Adding words after a set of words

Greetings. I am a UNIX newbies. I am currently facing difficulties dealing with a large data set and I would like to ask for helps. I have a input file like this: ak 1 AAM1 ak 2 AAM1 ak 3 AAM1 ak 11 AMM2 ak 12 AMM2 ak 13 AMM2 ak 14 AMM2 Is there any possibility for me to... (7 Replies)
Discussion started by: Amanda Low
7 Replies

4. Shell Programming and Scripting

Finding the number of unique words in a file

find the number of unique words in a file using sort com- mand. (7 Replies)
Discussion started by: abhikamune
7 Replies

5. Shell Programming and Scripting

finding and removing 2 identical consecutive words in a text

i want to write a shell script that correct a text file.for example if i have the input file: "john has has 2 apples anne has 3 oranges oranges" i want that the output file be like this: "john has 2 apples anne has 3 oranges" i've tried to read line by line from input text file into array... (11 Replies)
Discussion started by: cocostaec
11 Replies

6. Shell Programming and Scripting

Finding consecutive same words in a file

Hi All, I tried this but I am having trouble formulating this: I have a file that looks like this (this is a sample file words can be different): network router frame network router computer card host computer card One can see that in this file "network" and "router" occur... (3 Replies)
Discussion started by: shoaibjameel123
3 Replies

7. Shell Programming and Scripting

Finding my lost file by searching for words in it

Got a question for you guys...I am searching through a public directory (that has tons of files) trying to find a file that I was working on a longggggg time ago. I can't remember what it is called, but I do remember the content. It should contains words like this: Joe Pulvo botnet zeus... (5 Replies)
Discussion started by: statichazard
5 Replies

8. Shell Programming and Scripting

Finding compound words from a set of files from another set of files

Hi All, I am completely stuck here. I have a set of files (with names A.txt, B.txt until L.txt) which contain words like these: computer random access memory computer networking mouse terminal windows All the files from A.txt to L.txt have the same format i.e. complete words in... (2 Replies)
Discussion started by: shoaibjameel123
2 Replies

9. Shell Programming and Scripting

How count the number of two words associated with the two words occurring in the file?

Hi , I need to count the number of errors associated with the two words occurring in the file. It's about counting the occurrences of the word "error" for where is the word "index.js". As such the command should look like. Please kindly help. I was trying: grep "error" log.txt | wc -l (1 Reply)
Discussion started by: jmarx
1 Replies

10. Shell Programming and Scripting

Search a column a return a set of words

Hi I have two files. One is a text file consisting of sentences i.e. INPUT.txt and the second file is SEARCH.txt consisting of two or three columns. I need help to write a script to search the second column of SEARCH.txt for each set of five words (blue color as set one and green color as set... (6 Replies)
Discussion started by: my_Perl
6 Replies
IRSEND(1)							   User Commands							 IRSEND(1)

NAME
irsend - basic LIRC program to send infra-red commands SYNOPSIS
irsend [options] DIRECTIVE REMOTE CODE [CODE...] DESCRIPTION
Asks the lircd daemon to send one or more CIR (Consumer Infra-Red) commands. This is intended for remote control of electronic devices such as TV boxes, HiFi sets, etc. DIRECTIVE can be: SEND_ONCE - send CODE [CODE ...] once SEND_START - start repeating CODE SEND_STOP - stop repeating CODE LIST - list configured remote items SET_TRANSMITTERS - set transmitters NUM [NUM ...] SIMULATE - simulate IR event REMOTE is the name of a remote, as described in the lircd configuration file. CODE is the name of a remote control key of REMOTE, as it appears in the lircd configuration file. NUM is the transmitter number of the hardware device. For the LIST DIRECTIVE, REMOTE and/or CODE can be empty: LIST "" "" - list all configured remote names LIST REMOTE "" - list all codes of REMOTE LIST REMOTE CODE - list only CODE of REMOTE The SIMULATE command only works if it has been explicitly enabled in lircd. -h --help display usage summary -v --version display version -d --device use given lircd socket [/var/run/lirc/lircd] -a --address=host[:port] connect to lircd at this address -# --count=n send command n times EXAMPLES
irsend LIST DenonTuner "" irsend SEND_ONCE DenonTuner PROG-SCAN irsend SEND_ONCE OnkyoAmpli VOL-UP VOL-UP VOL-UP VOL-UP irsend SEND_START OnkyoAmpli VOL-DOWN ; sleep 3 irsend SEND_STOP OnkyoAmpli VOL-DOWN irsend SET_TRANSMITTERS 1 irsend SET_TRANSMITTERS 1 3 4 irsend SIMULATE "0000000000000476 00 OK TECHNISAT_ST3004S" FILES
/etc/lirc/lircd.conf Default lircd configuration file. It should contain all the remotes, their infra-red codes and the corresponding timing and wave- form details. DIAGNOSTICS
If lircd is not running (or /var/run/lirc/lircd lacks write permissions) irsend aborts with the following diagnostics: "irsend: could not connect to socket" "irsend: Connection refused" (or "Permission denied"). SEE ALSO
The documentation for lirc is maintained as html pages. They are located under html/ in the documentation directory. lircd(8), mode2(1), smode2(1), xmode2(1), irrecord(1), irw(1), http://www.lirc.org. irsend 0.9.0-pre1 October 2010 IRSEND(1)
All times are GMT -4. The time now is 05:33 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy