Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

grouping log files based on counter

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting grouping log files based on counter
# 8  
Old 10-05-2012
thanks all, for amazing help.

May I ask adding little bit enhancement to the existing o/p?
can we print the existing start time-3 instead of the the time.

e.g.

Code:
first GC started at 00:18:02 and continued for 15mins
second GC started at 10:28:01 and continue for 17mins
third GC started at 17:34:01 and continue for 16mins

will be now

Code:
first GC started at  00:15:02 (00:18:02 - 3min )and continued for 15mins
second GC started at 10:25:01 (10:28:01 - 3min) and continue for 17mins
third GC started at 17:31:01  (17:34:01 - 3min )and continue for 16mins

# 9  
Old 10-05-2012
try this...

mine code looking really ugly now...Smilie

Code:
awk -v VM="first,second,third" -v GC="GC started at" -v CN="and continued for" -v MN="mins" '{split(VM,P,",")
if($0 ~ /running from last 3 min/) {
if(s){a++;split($1,T,":");print P[a],GC,T[1]":"(T[2]-3)":"T[3],k,CN,s,MN;};k=$1;}else{s=$(NF-1)}}
END{a++;split($1,T,":");print P[a],GC,T[1]":"(T[2]-3)":"T[3],k,CN,s,MN}' file

# 10  
Old 10-05-2012
You know, pamu it's alright

I have notice two points in your so called ugly code Smilie
1
o/p now looks like
first GC started at 10:25:01 00:02:02 and continued for 15 mins
second GC started at 17:31:01 10:28:01 and continued for 17 mins
third GC started at 17:44:01 17:34:01 and continued for 16 mins
extra bold time poping up and no minus of 3 is really happening here.

2
while subtracting 3mins from the time we should consider actual time line

e.g. here the min is greater than 3, so no issue what so ever
but what if the time said 00:02:02 , so minusing 3 would be 11:59:02

I tried to modify the bit
Code:
awk -v VM="first,second,third" -v GC="GC started at" -v CN="and continued for" -v MN="mins" '{split(VM,P,",")
> if($0 ~ /running from last 3 min/) {
> if(s){a++;split(k,T,":");print P[a],GC,T[1]":"(T[2]-3)":"T[3],CN,s,MN;};k=$1;}else{s=$(NF-1)}}
> END{a++;split(k,T,":");print P[a],GC,T[1]":"(T[2]-3)":"T[3],CN,s,MN}' gc.txt

first GC started at 00:-1:02 and continued for 15 mins
second GC started at 10:25:01 and continued for 17 mins
third GC started at 17:31:01 and continued for 16 mins

This what I'm looking for but not the -1 bit in first line, it should say 59.
Last edited by manas_ranjan; 10-05-2012 at 12:16 PM..
# 11  
Old 10-05-2012
actually one correction..

Code:
awk -v VM="first,second,third" -v GC="GC started at" -v CN="and continued for" -v MN="mins" '{split(VM,P,",")
if($0 ~ /running from last 3 min/) {
if(s){a++;split(k,T,":");print P[a],GC,T[1]":"(T[2]-3)":"T[3],k,CN,s,MN;};k=$1;}else{s=$(NF-1)}}
END{a++;split(k,T,":");print P[a],GC,T[1]":"(T[2]-3)":"T[3],k,CN,s,MN}' file

and about time less than 3 mins it will be more uglier add one condition to check if T[2] is greater than 3 or not..Smilie

bye..
# 12  
Old 10-05-2012
I LOVE moving targets. As you point out yourself, you would need to implement full blown date & time arithmetics to really solve your problem. This I have NOT done but left it for your exercise. At least, for your sample data, this should work:
Code:
awk     'BEGIN {FMTSTR = "%2d. GC started at %s (%s - %s min) and continued for %s mins.\n"}
         function prtit () {split (STTIME,B,":");
                            B[2]-=FSTMIN;
                            TMP = sprintf ("%02d:%02d:%02d", B[1], B[2], B[3]);
                            printf FMTSTR, ++CNT, TMP, STTIME, FSTMIN, LASTMIN
                          }
         NR  > 1 && $13 < LASTMIN {prtit ()} 
         NR == 1 || $13 < LASTMIN {STTIME = $1; FSTMIN = $13}
         { LASTMIN = $13}
         END {prtit ()}
        ' file
 1. GC started at 00:15:02 (00:18:02 - 3 min) and continued for 15 mins.
 2. GC started at 10:25:01 (10:28:01 - 3 min) and continued for 17 mins.
 3. GC started at 17:31:01 (17:34:01 - 3 min) and continued for 16 mins.

As I said, no day nor hours span taken care of...
Last edited by RudiC; 10-05-2012 at 12:38 PM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to grouping time and based on value with multiple pattern?

Hi All, need help... I have some log below : ### {"request_id":"e8395eb0-a8bd-11e9-b77b-d507ea5312aa","message":"when inquiry paybill 628524871 prevalidation cause : Invalid Transaction"} ### {"request_id":"043f2310-a8be-11e9-b57b-f9c7344998d7","message":"when inquiry paybill 62821615... (2 Replies)
Discussion started by: fajar_3t3
2 Replies

2. Shell Programming and Scripting

Counter based on 10 files or less

there are around 676 files in a directory and I want to batch every 10 and then remaining 7 files as well for a batch pressing job simultaneously. I'm trying to pick-up every 10 counter'd files -- batch these files and then move on with the next batch.. after resetting the Counters back to 0; ... (5 Replies)
Discussion started by: busyboy
5 Replies

3. Shell Programming and Scripting

Grouping files on pattern

I have this Requirement where i have to group the files, I have a folder say "temp" where many files resides...files are like this; 010020001_S-ABC-Sort-DEFAW_YYYYMMDD_HHMMSS.txt 010020004_S-PQR-Sort-DRTON_YYYYMMDD_HHMMSS.txt 010020009_S-JKL-Sort_MNOLO_YYYYMMDD_HHMMSS.txt... (8 Replies)
Discussion started by: gnnsprapa
8 Replies

4. Shell Programming and Scripting

File merging based on different counter loop

hello, File 1 main Group sub group MIT VAR_1D_DATA_TYPE 23-03-2012 MIT VAR_1D_DATA_TYPE 22-03-2012 MIT VAR_10D_DATA_TYPE 23-03-2012 MIT VAR_10D_DATA_TYPE 22-03-2012 MIT ... (0 Replies)
Discussion started by: manas_ranjan
0 Replies

5. Shell Programming and Scripting

Grouping files according to certain fields in their name

I have a list of fils stored insortedLst, and want to select certain fields to group specific files together: Example of the files would be as below: n02-z30-dsr65-ndelt0.25-varp0.002-16x12drw-run1.log n02-z30-dsr65-ndelt0.25-varp0.002-16x12drw-run2.log... (2 Replies)
Discussion started by: kristinu
2 Replies

6. Shell Programming and Scripting

grouping based on first column

I do have a tab delimited file of the following format a_1 rt a_1 st_2 a_1 st_3 a_2 bt_2 a_2 st_er b_2 st_2 b_2 st_32 S_1 rt_8 S_1 rt_64 I want to cut short the above file and group the file based on the first column like below. a_1 rt st_2 st_3 a_2 bt_2 st_er b_2 st_2... (1 Reply)
Discussion started by: Lucky Ali
1 Replies

7. UNIX for Dummies Questions & Answers

Please help me to find out maximum value of a field based on grouping of other fields.

Please help me to find out maximum value of a field based on grouping of other fields, as we do in SQL. Like in SQL if we are having below records : Client_Name Associate_Name Date1 Value C1111 A1111 2012-01-17 10 C1111 A1111 ... (1 Reply)
Discussion started by: KamalKumarKalra
1 Replies

8. Shell Programming and Scripting

Help with grouping data based on range position

Input file: data_1 1000 1290 data_4 290 234 data_2 1114 1110 data_5 534 999 data_6 900 1050 . . Desired_output_file_1_0_999: data_4 290 234 data_5 534 999 Desired_output_file_2_1000_1999: data_1 1000 1290 data_2 1114 1110 (1 Reply)
Discussion started by: perl_beginner
1 Replies

9. Shell Programming and Scripting

combine 3 files by grouping

I have a file, which is really large but i shortened it: A3059GVS 1 A 01 Plate_1 40 25.37016 14.6298 A3059GVS 2 A 01 Plate_2 40 26.642002 13.3583 A3059GVS 3 A 02 Plate_1 40 25.381462 ... (4 Replies)
Discussion started by: mykey242
4 Replies

10. Shell Programming and Scripting

Grouping files into tars

Hi all, I have a problem where i have several files in a directory which I SCP from a server to my local machine and i would like to periodically tar/gzip them based on their naming convention. Here is the scenario: I SCP files (which all end with the same ending) periodically across to a... (3 Replies)
Discussion started by: muay_tb
3 Replies
Login or Register to Ask a Question