Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Calculate total of log by hour

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Calculate total of log by hour
# 1  
Old 09-23-2011
Calculate total of log by hour
Hi,

Just wondering, is there anyway I can get the total of logs generated by hours ? Let say I have these logs,

PHP Code:
Sep 23 04:48:43 hsbcufs: [ID 213553 kern.notice] NOTICE: realloccg /: file system full
Sep 23 04:48:47 hsbcufs: [ID 845546 kern.notice] NOTICE: alloc: /: file system full
Sep 23 04:48:51 hsbcufs: [ID 213553 kern.notice] NOTICE: realloccg /: file system full
Sep 23 04:48:55 hsbclast message repeated 1 time
Sep 23 04:48:59 hsbcufs: [ID 845546 kern.notice] NOTICE: alloc: /: file system full
Sep 23 04:50:12 hsbclast message repeated 19 times
Sep 23 04:50:16 hsbcufs: [ID 213553 kern.notice] NOTICE: realloccg /: file system full
Sep 23 04:50:20 hsbcufs: [ID 845546 kern.notice] NOTICE: alloc: /: file system full
Sep 23 04:52:22 hsbclast message repeated 30 times
Sep 23 04:52:26 hsbcufs: [ID 213553 kern.notice] NOTICE: realloccg /: file system full
Sep 23 04:52:30 hsbcufs: [ID 845546 kern.notice] NOTICE: alloc: /: file system full
Sep 23 04:53:22 hsbclast message repeated 13 times
Sep 23 04:53:26 hsbcufs: [ID 213553 kern.notice] NOTICE: realloccg /: file system full
Sep 23 04:53:30 hsbcufs: [ID 845546 kern.notice] NOTICE: alloc: /: file system full
Sep 23 04:55:03 hsbclast message repeated 23 times
Sep 23 04:55:07 hsbcufs: [ID 213553 kern.notice] NOTICE: realloccg /: file system full
Sep 23 04:55:11 hsbcufs: [ID 845546 kern.notice] NOTICE: alloc: /: file system full
Sep 23 04:55:40 hsbclast message repeated 7 times
Sep 23 04:55:44 hsbcufs: [ID 845546 kern.notice] NOTICE: alloc: /: file system full
Sep 23 04:56:44 hsbclast message repeated 15 times
Sep 23 04:56:48 hsbcufs: [ID 213553 kern.notice] NOTICE: realloccg /: file system full
Sep 23 04:56:52 hsbcufs: [ID 845546 kern.notice] NOTICE: alloc: /: file system full
Sep 23 05:00:51 hsbclast message repeated 59 times
Sep 23 05:00:55 hsbcufs: [ID 213553 kern.notice] NOTICE: realloccg /: file system full
Sep 23 05:01:03 hsbclast message repeated 2 times
Sep 23 05:01:07 hsbcufs: [ID 845546 kern.notice] NOTICE: alloc: /: file system full
Sep 23 05:02:16 hsbclast message repeated 17 times
Sep 23 05:02:20 hsbcufs: [ID 213553 kern.notice] NOTICE: realloccg /: file system full
Sep 23 05:02:24 hsbcufs: [ID 845546 kern.notice] NOTICE: alloc: /: file system full
Sep 23 05:02:52 hsbclast message repeated 7 times
Sep 23 05:02:56 hsbcufs: [ID 213553 kern.notice] NOTICE: realloccg /: file system full
Sep 23 05:03:00 hsbcufs: [ID 845546 kern.notice] NOTICE: alloc: /: file system full
Sep 23 05:03:16 hsbclast message repeated 4 times
Sep 23 05:03:20 hsbcufs: [ID 213553 kern.notice] NOTICE: realloccg /: file system full
Sep 23 05:03:24 hsbcufs: [ID 845546 kern.notice] NOTICE: alloc: /: file system full
Sep 23 05:07:27 hsbclast message repeated 60 times
Sep 23 05:07:31 hsbcufs: [ID 213553 kern.notice] NOTICE: realloccg /: file system full
Sep 23 05:07:35 hsbcufs: [ID 845546 kern.notice] NOTICE: alloc: /: file system full
Sep 23 05:08:56 hsbclast message repeated 20 times
Sep 23 05:09:00 hsbcufs: [ID 845546 kern.notice] NOTICE: alloc: /: file system full
Sep 23 05:13:59 hsbclast message repeated 74 times
Sep 23 05:14:03 hsbcufs: [ID 213553 kern.notice] NOTICE: realloccg /: file system full
Sep 23 05:14:07 hsbcufs: [ID 845546 kern.notice] NOTICE: alloc: /: file system full
Sep 23 05:15:20 hsbclast message repeated 18 times
Sep 23 05:15:24 hsbcufs: [ID 213553 kern.notice] NOTICE: realloccg /: file system full 
I would like to get the stat like ,

Hours Total line
04 ______52
05 ______ 41
06 ____ 65
07 ____ 35

and get them in minutes as well ?

Many thanks everyone !!!
# 2  
Old 09-23-2011
Code:
$ awk -F"[ :]" '{print $3}' log.log  | sort | uniq -c
     22 04
     23 05

This User Gave Thanks to itkamaraj For This Post:
dehetoxic
# 3  
Old 09-23-2011
you will need to write a script to parse the log.
This User Gave Thanks to frank_rizzo For This Post:
dehetoxic
# 4  
Old 09-23-2011
Code:
$ awk -F"[ :]" '{a[$3]++;next}END{for(i in a){print i"_--->"a[i]}}' log.log 
04_--->22
05_--->23

This User Gave Thanks to itkamaraj For This Post:
dehetoxic
# 5  
Old 09-23-2011
thanks, but the code doesn't seems to work though, i get total of line instead

PHP Code:
adm@hsbc> awk -F"[ :]" '{a[$3]++;next}END{for(i in a){print i"_--->"a[i]}}' messages
_--->274

adm@hsbc> awk -F"[ :]" '{print $3}' messages  | sort | uniq -c
 274 
# 6  
Old 09-23-2011
Code:
$ awk -F"[ :]" '{a[$3]++;next}END{for(i in a){print i"_--->"a[i]}}' log.log 
04_--->2
05_--->2

$ cat log.log
Sep 23 04:56:48 hsbcufs: [ID 213553 kern.notice] NOTICE: realloccg /: file system full
Sep 23 04:56:52 hsbcufs: [ID 845546 kern.notice] NOTICE: alloc: /: file system full
Sep 23 05:00:51 hsbclast message repeated 59 times
Sep 23 05:00:55 hsbcufs: [ID 213553 kern.notice] NOTICE: realloccg /: file system full 

$ awk -F"[ :]" '{print $3}' log.log  | sort | uniq -c
      2 04
      2 05

for me its working fine. can you post the original message contents

---------- Post updated at 07:35 PM ---------- Previous update was at 07:35 PM ----------

if you are in solaris, please use nawk
This User Gave Thanks to itkamaraj For This Post:
dehetoxic
# 7  
Old 09-26-2011
Thanks, I used nawk, and the output that I have is 00 till 59, looks like a minute count ?

PHP Code:
nawk -F"[ :]" '{print $3}' withdrawal.log  | sort | uniq -c

11340 00
10371 01
9771 02
9869 03
10782 04
12424 05
14427 06
19253 07
17638 08
16186 09
17690 10
18769 11
14854 12
10323 13
10547 14
10116 15
10626 16
10397 17
10603 18
10622 19
10541 20
9801 21
10152 22
10409 23
10288 24
9778 25
10181 26
10013 27
11081 28
9419 29
10873 30
10039 31
10041 32
9730 33
10446 34
10695 35
10562 36
10799 37
10754 38
10730 39
10573 40
10432 41
10470 42
10877 43
10152 44
10661 45
10315 46
11294 47
9727 48
9937 49
9980 50
10526 51
10148 52
9858 53
10091 54
10298 55
10307 56
10153 57
10481 58
11003 59 
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Calculate total memory using free -m

Hi I am trying to calculate memory used by Linux System free -m total used free shared buffers cached Mem: 32109 31010 1099 0 3600 7287 -/+ buffers/cache: 20121 11987 Swap: 10239 1282 8957 Now according to my requirement Im calculating memory using below cmd free -m | awk 'NR==3{printf... (2 Replies)
Discussion started by: sam@sam
2 Replies

2. Shell Programming and Scripting

Calculate the total

Hi All , I have the following script as below , I tried to modify to meet the requirement , could someone help ? very thanks ================================================================================================ while read STR NAME; do Total=0 MyString="$STR" GetData () {... (18 Replies)
Discussion started by: ust3
18 Replies

3. Shell Programming and Scripting

Calculate total value from a row

HI I have a file # cat marks.txt MARKS LIST 2013 Name english french chinese latin total_marks wer 34 45 67 23 wqa 12 39 10 56 wsy 23 90 23 78 Now i need to find the total marks of each student using... (11 Replies)
Discussion started by: Priya Amaresh
11 Replies

4. Shell Programming and Scripting

Grep string from logs of last 1 hour on files of 2 different servers and calculate count

Hi, I am trying to grep a particular string from the files of 2 different servers without copying and calculate the total count of its occurence on both files. File structure is same on both servers and for reference as follows: 27-Aug-2010... (4 Replies)
Discussion started by: poweroflinux
4 Replies

5. Shell Programming and Scripting

Calculate total space, total used space and total free space in filesystem names matching keyword

Good afternoon! Im new at scripting and Im trying to write a script to calculate total space, total used space and total free space in filesystem names matching a keyword (in this one we will use keyword virginia). Please dont be mean or harsh, like I said Im new and trying my best. Scripting... (4 Replies)
Discussion started by: bigben1220
4 Replies

6. Shell Programming and Scripting

Calculate total sum from a file

The file content is dynamic and using this format: name1 number1 name2 number2 name3 number3 name4 number4 .................... Need a smooth way to calculate the sum of all the numbers in that file (number1 + number2 + number3 + number4........ = total ) (11 Replies)
Discussion started by: TehOne
11 Replies

7. Shell Programming and Scripting

awk script to calculate total

Hi First field is the Record Type. A Record Type 5 can have multiple Record Type 6's before another Record Type 5 appears. I want to calculate the total of fields at position 8-11 on Record type 6 when Record Type 5 has a field at position 11-14 equals to '2222'. then it should delete the lines... (2 Replies)
Discussion started by: appsguy616
2 Replies

8. Shell Programming and Scripting

Awk help needed to calculate total

Hi all, I have a flat file like 10 steven 25 mike 47 Charles 127 Nancy 34 steven 23 mike 67 Charles 7761 Nancy 8 steven 54 mike 88 Charles 1267 Nancy I need to calculate the total of steven and all the members , for this I am using like grep "`sed -n 1p patterns.txt`"... (7 Replies)
Discussion started by: senthilkumar_ak
7 Replies

9. Shell Programming and Scripting

how to calculate busy hour

Dear All, please tell me any script which can able to calculate the bus hours by claculating the volume for example hour volume 1 100 2 200 3 300 4 7000 5 50 6 80 7 77 8 77 9 165 10 888 11 99 12 89 13 33 14 676 15 878 16 90 17 56 (14 Replies)
Discussion started by: shary
14 Replies
Login or Register to Ask a Question