Count events occured in the same second in awk


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Count events occured in the same second in awk
# 1  
Old 08-01-2013
Count events occured in the same second in awk

Hello to all in forum,

I need to count how many events happens within each second for 01 Aug 2013.(because in the sample appears records for 01 Aug and 02 Aug).

The output for the input below should be:

Desired Output:
Code:
[01 Aug 2013 11:54:02] --> 2 Records
[01 Aug 2013 11:54:03] --> 1 Record
[01 Aug 2013 13:24:31] --> 3 Records

Input:
Code:
[01 Aug 2013 11:54:02.200] DEBUG http-9901-1  REQUEST :MSSG-YYY--2.0 [msjFinTrans[Some text]--currentStep[1]--]
 +RESPONSE:some text
1 Xyz jjsjs l272
2 Xyz jjsjs l272
3 Xyz jjsjs l272
4 WRT MMMSK -- TimeToExecute =20
[01 Aug 2013 11:54:02.700] DEBUG http-9901-1  REQUEST :MSSG-YYY--2.0 [msjFinTrans[Some text]--currentStep[1]--]
 +RESPONSE:some text
 [01 Aug 2013 11:54:03.144] DEBUG http-9901-1  REQUEST :MSSG-YYY--2.0 [msjFinTrans[Some text]--currentStep[1]--]
 +RESPONSE:some text
1 Xyz jjsjs l272
2 Xyz jjsjs l272
3 Xyz jjsjs l272
4 WRT MMMSK -- TimeToExecute =10
[01 Aug 2013 13:24:31.444] DEBUG http-9901-1  REQUEST :MSSG-YYY--2.0 [msjFinTrans[Some text]--currentStep[1]--]
 +RESPONSE:some text
[01 Aug 2013 13:24:31.603] DEBUG http-9901-1  REQUEST :MSSG-YYY--2.0 [msjFinTrans[Some text]--currentStep[1]--]
 +RESPONSE:some text
[01 Aug 2013 13:24:31.867] DEBUG http-9901-1  REQUEST :MSSG-YYY--2.0 [msjFinTrans[Some text]--currentStep[1]--]
 +RESPONSE:some text
1 Xyz jjsjs l272
2 Xyz jjsjs l272
3 Xyz jjsjs l272
4 WRT MMMSK -- TimeToExecute =14
[02 Aug 2013 00:20:01.823] DEBUG http-9901-1  REQUEST :MSSG-YYY--2.0 [msjFinTrans[Some text]--currentStep[1]--]
 +RESPONSE:some text

Thanks in advance for any help.
# 2  
Old 08-01-2013
Code:
awk -F'[][]' '/^\[/{sub(/\..*$/,X,$2);A[$2]++}END{for(k in A) print k " --> " A[k] "Record(s)"}' file

This User Gave Thanks to Yoda For This Post:
# 3  
Old 08-01-2013
Hello Yoda,

Thanks for you help. It seems to work just fine!

Only don't understand too much how it works.

May you explain me please why the field separator is a double "[]" and not only one?

And how it works this part ? {sub(/\..*$/,X,$2);A[$2]++}

Thanks again.

---------- Post updated at 03:42 PM ---------- Previous update was at 03:11 PM ----------

Hello Yoda,

No problem. After trying your code by sections, I understand now how is the logic.

Many thanks again.Smilie
# 4  
Old 08-01-2013
Quote:
Originally Posted by Ophiuchus
May you explain me please why the field separator is a double "[]" and not only one?
It is not double, I just specified field separators ] [inside [ ]
Code:
[][]

Quote:
Originally Posted by Ophiuchus
And how it works this part ? {sub(/\..*$/,X,$2);A[$2]++}
(/\..*/,X,$2) means substitute period . followed by zero or more occurrence of any character .* in $2 value with X (which is undefined variable and hence NULL)

A[$2]++ means create an associative array A indexed by the previously modified 2nd field value and array value is incremented.
This User Gave Thanks to Yoda For This Post:
# 5  
Old 08-01-2013
Thank you Yoda for take your time to help and explain!

I understand better now.

Best regards.
# 6  
Old 08-01-2013
I'm a little bit confused. The original posting in this thread said that only entries from 01 Aug 2013 were to be considered, but the ouput from Yoda's script has an entry (shown in red below) from August 2nd:
Code:
02 Aug 2013 00:20:01 --> 1Record(s)
01 Aug 2013 13:24:31 --> 3Record(s)
01 Aug 2013 11:54:02 --> 2Record(s)

and it doesn't include output corresponding to the input line:
Code:
 [01 Aug 2013 11:54:03.144] DEBUG http-9901-1  REQUEST :MSSG-YYY--2.0 [msjFinTrans[Some text]--currentStep[1]--]

because there is a leading space on this line.

Did I misunderstand the requirements? Would the following more accurately produce the desired results?:
Code:
dtm="01 Aug 2013"
awk -v dtm="$dtm" '
BEGIN {         pat = "^ *[[]" dtm " "
                FS = pat "|[.]"
} 
$0 !~ pat {     next}
{               a[$2]++}
END {           for(i in a) printf("[%s %s] --> %d Record%s\n", dtm, i, a[i],
                                a[i] == 1 ? "" : "s")
}' Input

It produces the output:
Code:
[01 Aug 2013 13:24:31] --> 3 Records
[01 Aug 2013 11:54:02] --> 2 Records
[01 Aug 2013 11:54:03] --> 1 Record

The line in red is not produced by Yoda's script.

As always, if you're using a Solaris/SunOS system, use /usr/xpg4/bin/awk, /usr/xpg6/bin/awk, or nawk instead of /bin/awk or /usr/bin/awk.
Login or Register to Ask a Question

Previous Thread | Next Thread

8 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Awk: Print count for column in a file using awk

Hi, I have the following input in a file & need output as mentioned below(need counter of every occurance of field which is to be increased by 1). Input: 919143110065 919143110065 919143110052 918648846132 919143110012 918648873782 919143110152 919143110152 919143110152... (2 Replies)
Discussion started by: siramitsharma
2 Replies

2. Shell Programming and Scripting

awk - count character count of fields

Hello All, I got a requirement when I was working with a file. Say the file has unloads of data from a table in the form 1|121|asda|434|thesi|2012|05|24| 1|343|unit|09|best|2012|11|5| I was put into a scenario where I need the field count in all the lines in that file. It was simply... (6 Replies)
Discussion started by: PikK45
6 Replies

3. UNIX for Dummies Questions & Answers

Sort : Write Error occured while merging

Dear Friends, I am sorting the file which contain 55Lakhs of records while Sorting i am facing issues Sort :0653-657 Write Error occured while merging Could you please any one tell me how to overcome this problem Thanks in advance (5 Replies)
Discussion started by: i150371485
5 Replies

4. HP-UX

When MTU changed, system panic occured

Hello I have some hpux 11.23/31 systems ,and some other (linux,solaris) I know if i want change the mtu size ,need use lanadmin. But:if i try on 11.23 : ifconfig lan1 mtu 8000,system paniced. If i try on 11.31 :ifconfig lan1 mtu 8000,remove ip address from interface. Ithink is not a best... (1 Reply)
Discussion started by: zoldkalap
1 Replies

5. Shell Programming and Scripting

awk statement to find events from a specific date

I have a script which archives log file events which are 90-days old. Script works fine but I wanted some input on one aspect of this script. My nawk statement, bolded below, that removes events 90-days prior from today, I need it to find anything 90-days or older. The log file date pattern looks... (0 Replies)
Discussion started by: numele
0 Replies

6. UNIX for Advanced & Expert Users

Error occured while running a script

Hi All, In my application, I'm getting an error as type mismatch Let me now explain the scenario, I ran a shell script which calls some other scripts. All these scripts uses the environment variables. Take for example, this script uses a variable 'PathDir'. I initiallized the value to... (2 Replies)
Discussion started by: iamgeethuj
2 Replies

7. UNIX for Advanced & Expert Users

FATAL:exception occured with pthread_exit()

We had written an application in which we create worker thread. So the main thread will create the worker thread. After some time the child thread(Worker thread) will call pthread_exit(). This function was written in try{} and there occured an Exception and is handled in catch(...)... (0 Replies)
Discussion started by: platso
0 Replies

8. UNIX for Dummies Questions & Answers

finding no of counts the words occured

hi, cud u help me to find this. i hav 2 files. file1 has data as "ARUN ARUN is from Australia Arun likes America etc.. ARUN ARUN " file2 has "ARUN Australia America" i... (5 Replies)
Discussion started by: arunsubbhian
5 Replies
Login or Register to Ask a Question