Counting occurences of different strings in a file


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Counting occurences of different strings in a file
# 1  
Old 07-15-2005
Counting occurences of different strings in a file

Hi,
i'd like to know if the following is possible with a shell script, and can't find the answer in the search.

Suppose i have a logfile build like this:
[15/Jul/2005:10:47:22 0100][3-I] # 8 :riuyzp1028
[15/Jul/2005:10:47:24 0100][3-I] # 38 : riuyzp1028
[15/Jul/2005:10:47:30 0100][3-I] # 25 : riuyvzp1032
[15/Jul/2005:10:47:34 0100][3-I] # 30 : nlkljpa0202
[15/Jul/2005:10:47:34 0100][3-I] # 1 : nlklja0205
[15/Jul/2005:10:47:35 0100][3-I] # 38 : riuyzp1028
[15/Jul/2005:10:47:35 0100][3-I] # 25 : riuyvzp1032
[15/Jul/2005:10:47:37 0100][3-I] # 30 : dsjfhh23
[15/Jul/2005:10:47:38 0100][3-I] # 1 : sdfjkh45

I want a script to count the number of occurences of each unique machine (where "riuyzp1028" is the machine name).
The output i'd like to see is, the machine names followed by the number of occurences...like this:
riuyzp1028 3
riuyvzp1032 2
nlkljpa0202 1
nlklja0205 1
dsjfhh23 1
sdfjkh45 1

I wan't to use this script to be able to check which machine is generating the most entries in my logfile.

Thanks for your help in advance !
# 2  
Old 07-15-2005
awk ' { print $6 }' file.txt |sort |uniq -c

to reverse string and the number of occurence
awk ' { print $6 }' file.txt |sort |uniq -c |awk ' {print $2 " " $1 } '
# 3  
Old 07-15-2005
nawk -f free.awk file.txt | sort -n -k 2,2

free.awk:
Code:
BEGIN {
  FS=":"
}

function trim(str)
{
      sub("^[ ]*", "", str);
      sub("[ ]*$", "", str);
      return str;
}

{
  arr[trim($NF)]++
}
END {
  for ( i in arr )
    printf("%s %s\n", i, arr[i])
}

# 4  
Old 07-15-2005
or better yet:

nawk -f free.awk file.txt

free.awk:
Code:
BEGIN {
  FS=":"
}

function trim(str)
{
      sub("^[ ]*", "", str);
      sub("[ ]*$", "", str);
      return str;
}

{
    f=trim($NF)
    f in count || arr[++tally] = f
    count[f]++
}
END {
  for ( i=1; i <= tally; i++ )
    printf("%s %s\n", arr[i], count[arr[i]])
}

# 5  
Old 07-15-2005
Wow you guys are fast.
Thanks a lot !
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Perl regex problem on strings with several occurences of one char

Hi all, i have the following line in a record file : retenu=non demande=non script=#vtbackup /path=/save/backup/demande position=140+70 and i want to use Perl regex to have the following output key : "retenu" value : "non" key : "demande" value "non" key : "script" value :... (2 Replies)
Discussion started by: Fundix
2 Replies

2. Shell Programming and Scripting

Count the occurences of strings

I have some text files in a folder f1 with 10 columns. The first five columns of a file are shown below. aab abb 263-455 263 455 aab abb 263-455 263 455 aab abb 263-455 263 455 bbb abb 26-455 26 455 bbb abb 26-455 26 455 bbb aka 264-266 264 266 bga bga 230-232 230 ... (10 Replies)
Discussion started by: gomez
10 Replies

3. Shell Programming and Scripting

Counting a string between 2 strings...

I have been working on this for a little while and have been unable to come to a solution. Any help would be appreciated. I am working on a UNIX workstation and have a 30-40 meg text file that I am working with. In my real file there is hundreds of Jobs. Example of input file; misc logging data... (1 Reply)
Discussion started by: ny_evan
1 Replies

4. Shell Programming and Scripting

Counting occurences in column

Hi guys! I have a problem writing script that would convert this input into this output: I have an awk script that counts occurences of a sign in a column, but don't know how to change it so that I would give me desired output. script awk '{count++}END{for(j in count)... (2 Replies)
Discussion started by: grincz
2 Replies

5. Shell Programming and Scripting

How can i delete the content between all the occurences of two strings using sed or awk command

Hi. I have to delete the content between all the occurrences of the xml tags in a single file. For example: * The tags <script>.....................</script> occurs more than once in the same file. * It follows tagging rules meaning a start tag will be followed by an end tag. Will not have... (9 Replies)
Discussion started by: satheeshkumar
9 Replies

6. Shell Programming and Scripting

awk counting number of occurences

Hi, I am trying to count the max number of occurences of field1 in my apache log example: 10.0.0.1 field2 field3 10.0.0.2 filed2 field3 10.0.0.1 field2 field3 10.0.0.1 field2 field3 awk result to print out only the most occurence of field1 and number of occurence and field1 is... (3 Replies)
Discussion started by: phamp008
3 Replies

7. Shell Programming and Scripting

counting the number of occurences

say i've got a text file with >10million sequences: ssss ssss tttttt uuuuuu uuuuuu uuuuuu ... I'd like to convert the file so that the output will report the number of occurence right by each sequence: 2 ssss 2 ssss 1 tttttt 3 uuuuuu 3 uuuuuu 3 uuuuuu .... (3 Replies)
Discussion started by: johjoh
3 Replies

8. UNIX for Dummies Questions & Answers

Counting no of lines between two strings in a file

Hi all, I'm very very new to UNIX and AWK world.Please help me in finding a solution for my problem. I'm having a file like this ----------------------------------------------------------------- ~Version Information VERS. 2.0: CWLS log ASCII Standard -VERSION 2.0 WRAP. ... (4 Replies)
Discussion started by: santyshyam
4 Replies

9. UNIX for Dummies Questions & Answers

Counting number of occurences

Hi All, I have to count the number of occurences of the character " ; " in a given line. I had used the following awk command to achieve the same echo $KOP.dat|awk '{split($1,my,";"); for(i in my)c++ }END{print c-1}' My file KOP.dat had the following data ... (1 Reply)
Discussion started by: kingofprussia
1 Replies

10. UNIX for Dummies Questions & Answers

Counting occurences of specific charachter in a file

Hi, I need to count the number of occurences of the character " in a file that contains huge number of records. What command could I use? Please specify in detail since I am new :| Thanks much. (3 Replies)
Discussion started by: GMMike
3 Replies
Login or Register to Ask a Question