Count occurances of X Y Z in a file in 1 go.


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Count occurances of X Y Z in a file in 1 go.
# 8  
Old 03-02-2009
Quote:
Originally Posted by msullivan
Is there a way I can pass a variable to the awk command so that the output will be:
Sure you can. Either by variable substitution inside awk code or by assigning the sell variable to a awk variable with the -v switch.
The GNU Awk User's Guide

Instead of looping through your log files and executing the awk code block on every loop, why don't you input all these files into awk like:
Code:
awk 'awk code block' file1 file2 file3

This would enable you to use the awk FILENAME variable to print the file name in front of the results.

Code:
awk '{total[FILENAME" "$10] += 1} END{for (i in total) print "HTTP/1.1 "i, total[i]}' file1 file2 file3


Last edited by ripat; 03-02-2009 at 04:41 AM.. Reason: add example
# 9  
Old 03-02-2009
Thanks, I'll go have a look at the user's guide.
SmilieSmilieSmilie
Thanks you so very much ripat. I came right.

for t in 00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23; do echo "Day_hour: $t"; cat /var/SP/log/apache/cre/access.log_20090302_$t | nawk -v a=$t '{total[$10] += 1} END{for (i in total) print a" HTTP/1.1 "i, total[i]}';done
std out--->
Day_hour: 00
00 HTTP/1.1 304 1457
00 HTTP/1.1 404 504
00 HTTP/1.1 500 18
00 HTTP/1.1 503 46
00 HTTP/1.1 200 30510
00 HTTP/1.1 302 167
Day_hour: 01
01 HTTP/1.1 304 701
01 HTTP/1.1 404 227
01 HTTP/1.1 500 8
01 HTTP/1.1 503 11
01 HTTP/1.1 200 17281
01 HTTP/1.1 302 98
# 10  
Old 03-02-2009
As much fun as it is to see this work on a server, I am trying to figure out how to get his working in a ssh command.

ssh readonly@100.100.100.1 'for t in 00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23; do echo "Day_hour: $t"; cat /var/SP/log/apache/cre/access.log_20090302_$t | nawk -v a=$t '{total[$10] += 1} END{for (i in total) print a" HTTP/1.1 "i, total[i]}';done'

These scripts of mine works fine:
ssh readonly@$ServerIP 'YEAR=xx;DAY=xx;MONTH=xx;LMONTH=xx;for i in 00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 \
16 17 18 19 20 21 22 23; do cat /var/SP/log/cre/access.log_$YEAR$MONTH$DAY*_$i | grep -c "HTTP/1.1\" \"500";done'

but somehow I need to pass the whole "nawk..." string to ssh, anyone have any ideas?
I keep getting
bash: syntax error near unexpected token `(i'
# 11  
Old 03-02-2009
If you don't want to do the infamous "escape dance", put your command in a file and do

Code:
ssh usr@host < command_file

Also, avoid the useless use of cat:
Code:
cat file_name | awk 'awk_code_block'

Try
Code:
awk 'awk_code_block' file_name

Useless use of cat
http://partmaps.org/era/unix/award.html
# 12  
Old 03-02-2009
Computer

I wish I could put it in a file... but I'm too lazy to do that....
but I did get rid of the cat though... and tada! some fancy escape work:

Servername=cre02 ; ServerIP=10.113.98.17;ssh readonly@$ServerIP 'for t in 00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23; do echo "Day_hour: $t"; nawk -v a=$t '\''{total[$10] += 1} END{for (i in total) print a,i, total[i]}'\'' /var/SP/log/apache/cre/access.log_20090302_$t;done'

What the poor single quotes did to deserve this treatment... I dunno

Thanks again ripat!!
# 13  
Old 03-30-2009
you can use the -v argument
Code:
nawk -v str="01" '{total[$10] += 1} END{for (i in total) print str " HTTP/1.1 "i, total[i]}'

# 14  
Old 03-30-2009
sorry. too late with the reply Smilie
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Grep pattern file and count occurances

Guys I am trying grep to read from pattern file and count occurances of each word. input is firstplace secondplace secondpot patternfile is place first second i want the following. 1-count number of times keywords in patternfile occurs in begining of each line in input file. so... (12 Replies)
Discussion started by: ahfze
12 Replies

2. Shell Programming and Scripting

Count the number of occurances for multiple files

I have some text files as shown below. I would like to add the values of each string. Your help would be appreciated!! file1.txt SUS 2 PRS 2 ALI 1 PRS 1 GLK 2 file2.txt PRS 3 GLK 6 SUS 18 Desired output SUS 20 PRS 6 (2 Replies)
Discussion started by: arch
2 Replies

3. Shell Programming and Scripting

Count the occurances of numbers

I have a file as shown below. I need to count the unique occurrences of numbers in the first and second column only if the third column is <= 10. Otherwise print the corresponding numbers as zero. Thanks in advance!:) 58 80 40.74 76 80 9.78 76 80 8 12 6 9 30 28 8.23 45 12 ... (13 Replies)
Discussion started by: andreaalex
13 Replies

4. Shell Programming and Scripting

Get number of occurances in a file

I want to get the count of occurance of a user in a file. abc abc bcd bcd bcd xyz The result should be like abc 2 bcd 3 xyz 1 (2 Replies)
Discussion started by: Sanjeev Yadav
2 Replies

5. Shell Programming and Scripting

How to count number of occurances of string in a file?

Gurus, Need little guidance. I have A.txt and B.txt file. B.txt file contains Unique strings. Sample content of B.txt file for which i cut the fourth column uniquely and output directed to B.txt file And A.txt file contains the above string as a fourth column which is last column. So A.txt... (7 Replies)
Discussion started by: Shirisha
7 Replies

6. Shell Programming and Scripting

nawk-how count the number of occurances of a pattern, when don't know the pattern

I've written a script to count the total size of SAN storage LUNs, and also display the LUN sizes. From server to server, the LUNs sizes differ. What I want to do is count the occurances as they occur and change. These are the LUN sizes: 49.95 49.95 49.95 49.95 49.95 49.95 49.95 49.95... (2 Replies)
Discussion started by: cyber111
2 Replies

7. UNIX for Dummies Questions & Answers

Count occurances in a file

Hi, I have a fixed width file in the following format Sr.No A.No Name 1 2 PPP 3 4 PPP 5 6 TTT 7 8 OOO 9 10 OOO 11 12 OOO The 3rd column starts at position 10 and ends at 15. I want to count the number of occurances of each Name and output to a file Example in the... (2 Replies)
Discussion started by: samit_9999
2 Replies

8. Shell Programming and Scripting

Count occurances of a character in a file

I want to find the number of occurences of a character in a file. How do i do it. Eg: $cat file1.txt Welcome to World of Unix. $ If i want to find the occurences of 'o' then I should be getting 3. Thanks. (6 Replies)
Discussion started by: Shivdatta
6 Replies

9. Shell Programming and Scripting

count the number chracters occurances in a line

Hi Could anybody tell me how to count the number of occurances of a character within a LINE. actually i have a single line with words seperated with '+' chracter e.g. abc+def+ghj+jkl+asd i want to separte the words above. Please provide the necessary logic in the form of a SHELL... (6 Replies)
Discussion started by: skyineyes
6 Replies

10. Shell Programming and Scripting

Find occurances of a directory and count

I am writing my first shell script to do the following: Find all occurrences of a directory from the pwd Delete the directory (which is a hidden directory) Provide feedback w/ the number of directories deleted The problems I am having are two-fold: The user may not have the appropriate... (2 Replies)
Discussion started by: stringman
2 Replies
Login or Register to Ask a Question