optimizing - to find the number of occurrence


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers optimizing - to find the number of occurrence
# 1  
Old 01-05-2007
optimizing - to find the number of occurrence

Hi,

I need to find the number of occurrence of string in a file,

for ex:
Code:
>cat filename
abc
abc
def
ghi
ghi
ghi
ghi
abc
abc

>output would be
abc 4
def 1
ghi 4

the perl code given below works perfectly.

But am sure its not really efficient, any tips/pointers for that. Smilie
Code:
#! /opt/third-party/bin/perl
                                                                                 
my $file1 = $ARGV[0];
open(fh, $file1) || die "Unable to open the file : $file1 <$!>";
                                                                                 
@ele = ();
@count = ();
$index = 0;
$change = 0;
                                                                                 
while( chomp($fileContent = <fh>) )
{
  $reccnt++;
  $index = 0;
  $change = 0;
  foreach (@ele) {
    if( $_ eq $fileContent ) {
      $count[$index]++;
      $change = 1;
      last;
    }
    $index++;
  }
  if( $change eq 0 ) {
    push(@ele, $fileContent);
    push(@count, 1);
  }
}
                                                                                 
$index=0;
foreach (@ele) {
  print "$_####$count[$index]\n";
  $index++;
}
                                                                                 
exit 0

# 2  
Old 01-07-2007
I was hoping one of our perl experts would answer this. But it looks like you're stuck with me. Here is my best shot, but I'm still learning perl....
Code:
#! /usr/local/bin/perl

open(FH, shift) || die "Unable to open the file : $file1 <$!>";

while( <FH> )
{
        @words=split;
        foreach $word (@words) {

                print "word =", $word ,  "\n";
                $counts{$word}++;
        }
}
foreach $word (sort keys %counts) {
        print "word = ", $word, "  count = ",  $counts{$word}, "\n";
}

Some notes:
I'm using split so there can be more than one word on an input line. It is a perl standard that file handles be upper case so I did that. perl guys don't initialize variables so, for now, I am doing it that way. perl guys don't call exit usually either.
# 3  
Old 01-08-2007
thanks a lot Per!

that worked with associative arrays!

Code:
#! /opt/third-party/bin/perl
                                                                              
open(FH, shift) || die "Unable to open the file : $file1 <$!>";
                                                                              
while( chomp($fileContent = <FH>) )
{
  $counts{$fileContent}++;
}
foreach $word (sort keys %counts) {
  print "$word###$counts{$word}\n";
}

My requirement should consider a line entry as a word and not to split, so I had removed 'split' functionality

thanks a lot once again Smilie
# 4  
Old 01-08-2007
i missed out something important,

Code:
close(FH);

# 5  
Old 01-08-2007
perl will close the file when it exits but I guess if you put that close between the loops it might make sense.

And when you're working in perl you don't call associative arrays "associative arrays". You call associative arrays "hashes". And when you write a perl routine you always try to sneek them in where ever you can. This book says: "Until you start thinking in terms of hashes, you aren't really thinking in Perl."
# 6  
Old 01-08-2007
If you have Python, here's an alternative:

Code:
#!/usr/bin/python
data = open("data.txt").readlines() #get data into array
data = [i.strip() for i in data]  #strip newlines for each element
unique = set(data) #get unique elements
for element in unique:
 	print "%s has %i counts." % ( element, data.count(i) )

output:
Code:
abc has 4 counts.
ghi has 4 counts.
def has 1 counts.

# 7  
Old 01-09-2007
Unix command

Hi,
I suppose the following simple unix command pipe will do the work:
cat filename | sort | uniq -c | sort -nr

In case of more than one word in a line, the pipe would change to:
cat filename | deroff -w | sort | uniq -c | sort -nr

Hope this is what you intended for. (i suspect you would have demanded only perl solutions to this issue Smilie )

Thanks
Srini

Last edited by srinivasan_85; 01-09-2007 at 08:30 AM..
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Optimizing find with many replacements

Hello, I'm looking for advice on how to optimize this bash script, currently i use the shotgun approach to avoid file io/buffering problems of forks trying to write simultaneously to the same file. i'd like to keep this as a fairly portable bash script rather than writing a C routine. in a... (8 Replies)
Discussion started by: f77hack
8 Replies

2. Shell Programming and Scripting

awk or other way to find out number of occurrence of 7th character

Hi all, I am looking for to filter out based on 7th character and list the number of occurrence based on the 7th character if p , d , o or m 1. if 7th character is p , Output should be: p_hosts = N 2. if 7th character is d , Output should be: d_hosts = N 3. if 7th character is o , Output... (10 Replies)
Discussion started by: rveri
10 Replies

3. Shell Programming and Scripting

awk to find the number of occurrence

My file contains like this on 10 th line NM1*IL*1* awk '/NM1/{print NR}' *.dat output is 10 awk '/NM1*IL*1*/{print NR}' *.dat output is Nothing but im expecting 10 on second code as well . (4 Replies)
Discussion started by: Rajesh_us
4 Replies

4. Shell Programming and Scripting

How to count the occurrence of a number?

Hi, I have a file which contained a set of numbers like Col1 col2 col3 col4 1 sa 13 0 2 sb 14 0 3 sc 15 9 4 sd 16 -9 5 sd 20 -2 6 sd 20 4 Here in last column I need to count the zeros, positive values and negative values, please help me to do that. (2 Replies)
Discussion started by: Shenbaga.d
2 Replies

5. Shell Programming and Scripting

number of occurrence

: i need a bash script to convert the displayed output 12 14 15 12 15 13 to 12 * 2 ,13 * 1,14*1,15*1 Thanks, nevil (2 Replies)
Discussion started by: nevil
2 Replies

6. Shell Programming and Scripting

Count number of occurrence at each line

Hi I have the following file ENST001 ENST002 4 4 4 88 9 9 ENST004 3 3 3 99 8 8 ENST009 ENST010 ENST006 8 8 8 77 8 8 Basically I want to count how many times ENST* is repeated in each line so the expected results is 2 1 3 Any suggestion please ? (4 Replies)
Discussion started by: fuad_
4 Replies

7. Shell Programming and Scripting

find string nth occurrence in file and print line number

Hi I have requirement to find nth occurrence in a file and capture data from with in lines (between lines) Data in File. <QUOTE> <SESSION> <ATTRIBUTE NAME='Parameter Filename' VALUE='file1.parm'/> <ATTRIBUTE NAME='Service Name' VALUE='None'/> </SESSION> <SESSION> <ATTRIBUTE... (6 Replies)
Discussion started by: tmalik79
6 Replies

8. Shell Programming and Scripting

Uniq adresses with number of occurrence

Alo I have a file with a lot of addresses where I want to list unique addresses and the number of theirs occurrence. I have this input file: 0011bd09 ea 01 0b 04 ea 01 0b 38-bd 11 00 98 15 cb 01 00 .......8........ 0011bd09 ea 11 00 98 15 cb 01 00-00 00 00 d8 3d 8d 01 94 ... (5 Replies)
Discussion started by: chitech
5 Replies

9. UNIX for Dummies Questions & Answers

line number of the i-th occurrence of a pattern

Hi all, is there a simple way to obtain the line number of the i-th occurrence of a pattern? I have OCCURRENCE=`grep -io "${STRING_NAME}" ${1}-${8}${EXT}.out_bis| wc -l` which tells me how many occurency I have. I would like to go through them and determine the line number and assign... (6 Replies)
Discussion started by: f_o_555
6 Replies

10. UNIX for Dummies Questions & Answers

Line number of an occurrence using grep

Hi All, is there a way to extract the line number of an occurrence using grep? I know that with the -n option it prints out the line number as well. I would like to assign the line number to a variable. Thanks, Sarah (5 Replies)
Discussion started by: f_o_555
5 Replies
Login or Register to Ask a Question