Shell scripting: frequency of specific word in a string and statistics Post: 302805795

Sponsored Content

Top Forums Shell Programming and Scripting Shell scripting: frequency of specific word in a string and statistics Post 302805795 by Skrynesaver on Saturday 11th of May 2013 05:53:33 AM

05-11-2013

Registered User

Hi Kraterions,

that wasn't a script, but rather a command line one off, as a script it would look like this, with advised addons.

Code:

#!/usr/bin/perl
use strict;

open(my $tweets, $ARGV[0])|| die "Couldn't open $ARGV[0] $!\n";
my %counts;
my %freq;
my $tot;
while(<$tweets>){
        chomp;
        my @rec=split(/,/, $_, 3);
        my @words=split/\b\s*/,$rec[2];
        map {   $counts{lc($_)}++ if /^\w+$/;
                $freq{$rec[0]}{lc($_)}++ if /^\w+$/;
                $tot++}@words;
}
my @wanted=qw(that really great day);
print "WORD,TOTFrequency,(1)Frequency,(0)Frequency,(-1)Frequency,1%Frequency,0%Frequency,-1%Frequency\n";
for (sort {$counts{b}<=>$counts{a}} @wanted){
        print "$_,$counts{$_},",$freq{1}{$_}||0,",",$freq{0}{$_}||0,",",$freq{-1}{$_}||0,sprintf("%0.2f",($counts{$_}/$tot)*100),"%,",sprintf("%0.2f",($freq{0}{$_}/$tot)*100),"%,",sprintf("%0.2f",($freq{-1}{$_}/$tot)*100),"%\n";
}

Last edited by Skrynesaver; 05-11-2013 at 07:10 AM..

This User Gave Thanks to Skrynesaver For This Post:

Skrynesaver

View Public Profile for Skrynesaver

Find all posts by Skrynesaver

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Determining Word Frequency of Specific Terms

Hello, I require a perl script that will read a .txt file that contains words like 224.199.207.IN-ADDR.ARPA. IN NS NS1.internet.com. 4.200.162.207.in-addr.arpa. IN PTR beeriftw.internet.com. arroyoeinternet.com. IN A 200.199.227.49 I want to focus on words: IN...

2. Shell Programming and Scripting

Finding a word at specific location in a string

Hi All , I have different strings (SQL queries infact) of different lengths such as: 1. "SELECT XYZ FROM ABC WHERE ABC.DEF='123' " 2. "DELETE FROM ABC WHERE ABC.DEF='567'" 3. "SELECT * FROM ABC" I need to find out the word coming after the...

3. Shell Programming and Scripting

search a word and print specific string using awk

Hi, I have list of directory paths in a variable and i want to delete those dirs and if dir does not exist then search that string and get the correct path from xml file after that delete the correct directory. i tried to use grep and it prints the entire line from the search.once i get the entire...

4. Shell Programming and Scripting

awk or sed command to print specific string between word and blank space

My source is on each line 98.194.245.255 - - "GET /disp0201.php?poc=4060&roc=1&ps=R&ooc=13&mjv=6&mov=5&rel=5&bod=155&oxi=2&omj=5&ozn=1&dav=20&cd=&daz=&drc=&mo=&sid=&lang=EN&loc=JPN HTTP/1.1" 302 - "-" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; Trident/4.0; .NET CLR 1.0.3705; .NET CLR...

5. Shell Programming and Scripting

search-word-print-specific-string

Hi, Our input xml looks like: <doc> <str name="account_id">1111</str> <str name="prd_id">DHEP155EK</str> </doc> - <doc> <str name="account_id">6666</str> <str name="prd_id">394531662</str> </doc> - <doc> <str name="account_id">6666</str> <str...

6. Shell Programming and Scripting

Parse a String for a Specific Word

Hello, I'm almost there with scripting, and I've looked at a few examples that could help me out here. But I'm still at a lost where to start. I'm looking to parse each line in the log file below and save the output like below. Log File AABBCGCAT022|242|3 AABBCGCAT023|243|4...

7. Shell Programming and Scripting

break the string and print it in a new line after a specific word

Hi Gurus I am new to this forum.. I am using HP Unix OS. I have one single string in input file as shown below Abc123 | cde | fgh | ghik| lmno | Abc456 |one |two |three | four | Abc789 | five | Six | seven | eight | Abc098 | ........ I want to achive the result in a output file as shown...

8. UNIX for Dummies Questions & Answers

How to print line starts with specific word and contains specific word using sed?

Hi, I have gone through may posts and dint find exact solution for my requirement. I have file which consists below data and same file have lot of other data. <MAPPING DESCRIPTION ='' ISVALID ='YES' NAME='m_TASK_UPDATE' OBJECTVERSION ='1'> <MAPPING DESCRIPTION ='' ISVALID ='NO'...

9. Shell Programming and Scripting

Help with calculating frequency of specific word in a string

Input file: #read_1 AWEAWQQRZZZQWQQWZ #read_2 ZZAQWRQTWQQQWADSADZZZ #read_3 POGZZZZZZADWRR . . Desired output file: #read_1 3 #read_1 1 #read_2 2 #read_2 3 #read_3 6 . .

10. UNIX for Beginners Questions & Answers

Get string before specific word in UNIX

Hi All, I'm writing unix shell script and I have these files. I need to get name before _DETL.tmp. ABC_AAA_DETL.tmp ABC_BBB_DETL.tmp ABC_CCC_DETL.tmp PQR_DETL.tmp DEF_DETL.tmp JKL_DETL.tmp YUI_DETL.tmp TG_NM_DDD_DETL.tmp TG_NM_EEE_DETL.tmp GHJ_DETL.tmp RTY_DETL.tmp output will...

LEARN ABOUT LINUX

play_sample

play_sample(3alleg4)						  Allegro manual					      play_sample(3alleg4)

NAME

       play_sample - Plays a sample. Allegro game programming library.

SYNOPSIS

       #include <allegro.h>

       int play_sample(const SAMPLE *spl, int vol, int pan, int freq, int loop);

DESCRIPTION

       Triggers  a  sample  at	the  specified volume, pan position, and frequency.  The parameters `vol' and `pan' range from 0 (min/left) to 255
       (max/right).  Frequency is relative rather than absolute: 1000 represents the frequency that the sample was  recorded  at,  2000  is  twice
       this,  etc. If `loop' is not zero, the sample will repeat until you call stop_sample(), and can be manipulated while it is playing by call-
       ing adjust_sample(). Example:

	  /* Scream from the left speaker, twice the freq. */
	  int sound = play_sample(scream, 255, 0, 2000, 0);

RETURN VALUE

       Returns the voice number that was allocated for the sample or negative if no voices were available.

SEE ALSO

       install_sound(3alleg4), load_sample(3alleg4), adjust_sample(3alleg4), stop_sample(3alleg4), exsample(3alleg4), exsprite(3alleg4)

Allegro 							   version 4.4.2					      play_sample(3alleg4)

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Determining Word Frequency of Specific Terms

Discussion started by: richsark

2. Shell Programming and Scripting

Finding a word at specific location in a string

Discussion started by: swapnil.nawale

3. Shell Programming and Scripting

search a word and print specific string using awk

Discussion started by: dragon.1431

4. Shell Programming and Scripting

awk or sed command to print specific string between word and blank space

Discussion started by: elamurugu