Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Report from Log file using awk

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Report from Log file using awk
# 1  
Old 04-22-2014
Wrench Report from Log file using awk
Hi Gurus,

I've log files like below:

Code:
extint.performance.log.2014-04-10-00-00-00.13
extint.performance.log.2014-04-11-00-00-00.12
extint.performance.log.2014-04-12-00-00-00.12
extint.performance.log.2014-04-12-14-35-11.11
extint.performance.log.2014-04-13-00-00-00.10
extint.performance.log.2014-04-14-00-00-00.9
extint.performance.log.2014-04-15-00-00-00.8
extint.performance.log.2014-04-16-00-00-00.7
extint.performance.log.2014-04-17-00-00-00.6
extint.performance.log.2014-04-18-00-00-00.5

and each log file contains format like below:

Code:
Time stamp;INFO;;Ses. Meter;Service;Partner;Resp. time in Milli.Sec.;Error
2014-04-22 05:11:38,910;INFO;;Ses-1;Service-1;Partner-1;2901;0
2014-04-22 05:11:45,880;INFO;;Ses-2;Service-2;Partner-2;3249;0
2014-04-22 05:11:46,164;INFO;;Ses-3;Service-3;Partner-3;3513;0
2014-04-22 05:11:50,578;INFO;;Ses-4;Service-3;Partner-4;1123;0
2014-04-22 05:11:58,951;INFO;;Ses-5;Service-5;Partner-5;17;ERROR-TYPE-1
2014-04-22 05:12:02,306;INFO;;Ses-6;Service-6;Partner-1;2326;0
2014-04-22 05:12:07,240;INFO;;Ses-7;Service-7;Partner-7;592;0
2014-04-22 05:12:26,618;INFO;;Ses-8;Service-8;Partner-4;83;0
2014-04-22 05:12:45,139;INFO;;Ses-9;Service-3;Partner-9;28;ERROR-TYPE-2
2014-04-22 05:12:55,742;INFO;;Ses-10;Service-10;Partner-10;2603;0
2014-04-22 05:13:03,316;INFO;;Ses-11;Service-10;Partner-2;424;0

Now i am trying to get the report like this:

Code:
---<Date>---
Service;	Partner;	bad count(>1000);	good count(<1000);	ERROR;
------------	-------------------------	--------------------------	-----------------------	-------------------------------
Service-1	Partner-1		1	
Service-2	Partner-2	1		
Service-3	Partner-3,Partner-4,Partner-9	1	2	ERROR-TYPE-2(Partner-9)
Service-5	Partner-5		1	ERROR-TYPE-1(Partner-5)
Service-6	Partner-1	1		
Service-7	Partner-7		1	
Service-8	Partner-4		1	
Service-10	Partner-2,Partner-10	2

Bad count means response time($7) grater than 1000 and good count means response time($7) less than 1000.
Can any body help to get this done bye using awk ONLY?

Thanks,
Vasu
Last edited by VasuKukkapalli; 04-22-2014 at 06:37 PM..
# 2  
Old 04-22-2014
here's something to start with:
awk -f vasu.awk myFile where vasu.awk is:
Code:
BEGIN {
  FS=";"
  OFS="\t"

  FLDserv=5
  FLDpart=6
  FLDtime=7
  FLDerr=8

  fmt="%-15s%-40s%-10d%-10d%s\n"
}
FNR==2 {date=substr($1, 1, index($1, " ")-1)}
FNR>1 {
  servA[$FLDserv]=(servA[$FLDserv])? servA[$FLDserv]"," $FLDpart:$FLDpart
  if ($FLDtime < 1000)
     goodA[$FLDserv]++
  else
     badA[$FLDserv]++

  if ($FLDerr ~ /[^0-9]/)
     errorA[$FLDserv]= $FLDerr "(" $FLDpart ")"
}
END {
   print date
   printf("%-15s%-40s%-10s%-10s%s\n", "Service;", "Partner;", "badCount;", "goodCount;", "ERROR;")
   for (i in servA) {
     printf(fmt, i, servA[i], badA[i], goodA[i], errorA[i])
   }
}

This User Gave Thanks to vgersh99 For This Post:
VasuKukkapalli
# 3  
Old 04-22-2014
Hi vgersh99,

Thanks a lot for your reply, its working fine but not displaying ERROR code when same service with different partners with different error codes. for example:
Code:
Service-1 ---- Partner-1,Partner-2,Partner-3 ------ ERROR-TYPE-1(Partner-1),ERROR-TYPE-2(Partner-2),ERROR-TYPE-3(Partner-3)

Thanks for your help.

Thanks
Vasu
# 4  
Old 04-22-2014
well.... your sample file wasn't representative enough....
Code:
BEGIN {
  FS=";"
  OFS="\t"

  FLDserv=5
  FLDpart=6
  FLDtime=7
  FLDerr=8

  #fmt="%-15s%-40s%-10d%-10d%s\n"
  fmt="%-15s%-40s%-10s%-10s%s\n"
}
FNR==2 {date=substr($1, 1, index($1, " ")-1)}
FNR>1 {
  servA[$FLDserv]=(servA[$FLDserv])? servA[$FLDserv]"," $FLDpart:$FLDpart
  if ($FLDtime < 1000)
     goodA[$FLDserv]++
  else
     badA[$FLDserv]++

  if ($FLDerr ~ /[^0-9]/)
     errorA[$FLDserv]= (errorA[$FLDserv])? errorA[$FLDserv] ", " $FLDerr "(" $FLDpart ")" : $FLDerr "(" $FLDpart ")" 
}
END {
   print date
   #printf("%-15s%-40s%-10s%-10s%s\n", "Service;", "Partner;", "badCount;", "goodCount;", "ERROR;")
   printf(fmt, "Service;", "Partner;", "badCount;", "goodCount;", "ERROR;")
   for (i in servA) {
     printf(fmt, i, servA[i], badA[i], goodA[i], errorA[i])
   }
}

# 5  
Old 04-23-2014
Thanks vgersh99 for your quick turn around, Yes, My bad, some info missing.
Thanks for solving my problem. V
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Outputting data from log file to report

I have a log file that looks like this. the lines are grouped. 2 lines per entry. M: 2019-01-25 13:02:31.698 P25, received network transmission from KI4EKI to TG 10282 M: 2019-01-25 13:02:35.694 P25, network end of transmission, 4.3 seconds, 1% packet loss M: 2019-01-25 13:02:38.893 P25,... (7 Replies)
Discussion started by: ae4ml
7 Replies

2. Shell Programming and Scripting

Parsing a log file and creating a report script

The log file is huge and lot of information, i would like to parse and make a report . below is the log file looks like: REPORT DATE: Mon Aug 10 04:16:17 CDT 2017 SYSTEN VER: v1.3.0.9 TERMINAL TYPE: prod SYSTEM: nb11cu51 UPTIME: 04:16AM up 182 days 57 mins min MODEL, TYPE, and SN:... (8 Replies)
Discussion started by: amir07
8 Replies

3. Shell Programming and Scripting

Issue with awk script parsing log file

Hello All, I am trying to parse a log file and i got this code from one of the good forum colleagues, However i realised later there is a problem with this awk script, being naive to awk world wanted to see if you guys can help me out. AWK script: awk '$1 ~ "^WRITER_" {p=1;next}... (18 Replies)
Discussion started by: Ariean
18 Replies

4. Shell Programming and Scripting

Help on Log File format using sed or awk

Hello Gurus, First, i would like to know is there any way to solve my problem. i have a log file like this: INFO - ABCDRequest :: processing started for the record <0> TransactionNo <Txn#1> recordID <recID#1> INFO - ABCDRequest :: processing started for the record <0> TransactionNo... (9 Replies)
Discussion started by: VasuKukkapalli
9 Replies

5. Shell Programming and Scripting

awk to log transform on matrix file

Hi Friends, I have an input matrix file like this Col1 Col2 Col3 Col4 R1 1 2 3 4 R2 4 5 6 7 R3 5 6 7 8 I would like to consider only the numeric values without touching the column header and the row header. I looked up on the forum's search, and I found this. But, I donno how to... (3 Replies)
Discussion started by: jacobs.smith
3 Replies

6. Shell Programming and Scripting

Assistance on making a report log file

:):):):):) (0 Replies)
Discussion started by: bryan101
0 Replies

7. Shell Programming and Scripting

Extract XML message from a log file using awk

Dear all I have a log file and the content like this file name: temp.log <?xml version="1.0" encoding="cp850"?> <!DOCTYPE aaabbb SYSTEM '/dtdpath'> <aaabbb> <tranDtl> <msgId>000001</msgId> </tranDtl> ..... </aaabbb> ... ... (1 Reply)
Discussion started by: on9west
1 Replies

8. Shell Programming and Scripting

Piping tail to awk to parse a log file

Hello all, I've got what I'm pretty sure is a simple problem, but I just can't seem to work past it. I'm trying to use awk to pretty up a log file, and calculate a percentage. The log file looks like this: # tail strtovrUsage 20090531-18:15:45 RSreq - 24, RSsuc - 24, RSrun - 78, RSerr -... (4 Replies)
Discussion started by: DeCoTwc
4 Replies

9. Shell Programming and Scripting

awk script to count percentage from log file

Hi, I have a log like this : actually i want to get the log like this : where % can get from : 100 * pmTotNoRrcConnectReqSucc / pmTotNoRrcConnectReq Thanks in advance.. :) (8 Replies)
Discussion started by: justbow
8 Replies

10. Shell Programming and Scripting

awk- report generation from input file

I have input file with below content: Person: Name: Firstname1 lastname1 Address: 111, Straat City : Hilversum Person: Name : Fistname2 lastname2 Address: 222, street Cit: Bussum Person: Name : Firstname2 lastname3 Address: 333, station straat City: Amsterdam I need... (6 Replies)
Discussion started by: McLan
6 Replies
Login or Register to Ask a Question