Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Find time difference based on logfile

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Find time difference based on logfile
# 1  
Old 02-22-2015
RedHat Find time difference based on logfile
Hi All,

Firstly thank you for the forum members I need to find time difference b'w two rows of timestamp using awk/shell.

Here is the logfile:

Code:
cat business_file
start:skdjh:22:06:2010:10:30:22
sdfnskjoeirg
wregn'wergnoeirnfqoeitgherg
end:siifneworigo:22:06:2010:10:45:34
start:srsdneriieroi:24:06:2010:11:00:45
dfglkndfogn
sdgsdfgdfhdfg
end:erfqefegoieg:24:06:2010:11:46:34
oeirgoeirg\
start:sdjfhsldf:25:07:2010:12:55:43
wrgnweoigwerg
ewgjw]egpojwepr
etwasdf
gwdsdfsdf
fgpwj]pgojwth
wtr
wtwrt
end:dgnoeingwoit:25:07:2010:01:42:12

===========
The above logfile is kind of api file, and there are some rows start with "start" and "end",and the corresponding row's column 3rd to end of the row is timestamp (take delimiter as ":" ) we have to find the time difference between start and end time consecutive rows

Hope I am clear with the question,please let me know if you need more explanation.

Thx Srinivas
Last edited by Don Cragun; 02-22-2015 at 09:57 PM.. Reason: Add CODE tags.
# 2  
Old 02-23-2015
Are we to assume that by consecutive rows, you mean a row starting with start and the next row starting with end ignoring all of the lines between them that do not start with start or end? Or are the only lines to be considered from your input file:
Code:
end:siifneworigo:22:06:2010:10:45:34
start:srsdneriieroi:24:06:2010:11:00:45

since these are the only consecutive lines that start with start and end?

And, based on the data you've shown us, can we assume that all time stamps to be compared occur on the same date and the last three fields on the start and end lines are hours, minutes, and seconds from a 12-hour clock with no indication of AM/PM? Or, do some of your samples end before they start?

With the sample input you provided, what output are you hoping to get?
# 3  
Old 02-24-2015
Hi Don,

Its my bad,
Yes you have to consider the two consecutive start and end rows treated as one block.

Code:
cat business_file
start:skdjh:22:06:2010:10:30:22
end:siifneworigo:22:06:2010:10:45:34

start:srsdneriieroi:24:06:2010:11:00:45
end:erfqefegoieg:24:06:2010:11:46:34

start:sdjfhsldf:25:07:2010:12:55:43
end:dgnoeingwoit:25:07:2010:13:42:1

I just filtered out the logfile having the rows start and end.
And the date formate is 24hrs , apologies for not being clear.

the sample output is
Code:
912 sec 
6909 sec
2789 sec

Thanks
Srini
Last edited by rbatte1; 02-25-2015 at 07:04 AM.. Reason: Added CODE tags for output
# 4  
Old 02-25-2015
I don't know how you get 6909 seconds (1 hour, 55 minutes, 9 seconds) difference for the second time difference for your sample input (11:46:34 - 11:00:45), but the following brute force awk script seems to perform the desired calculations:
Code:
#!/bin/ksh
awk -F ':' -v debug=$# '
$1 == "start" {
	hs = $(NF - 2) + 0
	ms = $(NF - 1) + 0
	ss = $NF + 0
	if(debug) printf("$0=%s\n", $0)
}
$1 == "end" {
	he = $(NF - 2) + 0
	me = $(NF - 1) + 0
	se = $NF + 0
	if(se < ss) {
		me--
		se += 60
	}
	if(me < ms) {
		he--
		me += 60
	}
	if(he < hs) he += 24
	diff = (he - hs) * 3600 + (me - ms) * 60 + se - ss
	if(debug) printf("$0=%s\n", $0)
	if(debug) printf("%02d:%02d:%02d - %02d:%02d:%02d = %d\n",
			he, me, se, hs, ms, ss, diff)
	printf("%d sec\n", diff)
}' business_file

If business_file contains:
Code:
end:dgnoeingwoit:25:07:2010:13:42:12
start:1second:across:midnight:23:59:59
end:1second:across:midnight:00:00:00
start:1second:after:midnight:00:00:00
end:1second:after:midnight:00:00:01
start:full:day:00:00:00
end:full:day:23:59:59

it produces the output:
Code:
912 sec
2749 sec
2789 sec
1 sec
1 sec
86399 sec

if you invoke that script with no arguments, and adds debugging information as follows:
Code:
$0=start:skdjh:22:06:2010:10:30:22
$0=end:siifneworigo:22:06:2010:10:45:34
10:45:34 - 10:30:22 = 912
912 sec
$0=start:srsdneriieroi:24:06:2010:11:00:45
$0=end:erfqefegoieg:24:06:2010:11:46:34
11:45:94 - 11:00:45 = 2749
2749 sec
$0=start:sdjfhsldf:25:07:2010:12:55:43
$0=end:dgnoeingwoit:25:07:2010:13:42:12
12:101:72 - 12:55:43 = 2789
2789 sec
$0=start:1second:across:midnight:23:59:59
$0=end:1second:across:midnight:00:00:00
23:59:60 - 23:59:59 = 1
1 sec
$0=start:1second:after:midnight:00:00:00
$0=end:1second:after:midnight:00:00:01
00:00:01 - 00:00:00 = 1
1 sec
$0=start:full:day:00:00:00
$0=end:full:day:23:59:59
23:59:59 - 00:00:00 = 86399
86399 sec

if invoked with one or more arguments.

This was written and tested using the Korn shell, but it will work just as well with any shell that use basic Bourne shell syntax (such as ash, bash, dash, ksh, and sh).

This script will happily ignore any lines in your input file that do not start with "start:" or "end:", so you don't need to filter your input with grep.

If you want to try this on a Solaris/SunOS system, change awk to /usr/xpg4/bin/awk or nawk.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

In HP-UX how to find the date time difference ?

Hello, In HP-UX how to find the date time difference ? Start time: 28-APR-2019 21:36:01 End time : 29-APR-2019 00:36:04 ---------------------- Difference is ---------------------- Much appreciate any pointer or view on this. ... (3 Replies)
Discussion started by: Siva SQL
3 Replies

2. UNIX for Dummies Questions & Answers

Condition based on Timestamp (Date/Time based) from logfile (Epoch seconds)

Below is the sample logfile: Userids Date Time acb Checkout time: 2013-11-20 17:00 axy Checkout time: 2013-11-22 12:00 der Checkout time: 2013-11-17 17:00 xyz Checkout time: 2013-11-19 16:00 ddd Checkout time: 2013-11-21 16:00 aaa Checkout... (9 Replies)
Discussion started by: asjaiswal
9 Replies

3. Shell Programming and Scripting

Transpose timestamp based on column values and calculate time difference

Hello Expert, I need to transpose Date-Timestamp based on same column values and calculate time difference. The input file would be as below and required output is mentioned in the bottom INPUT File ======== 08/23/2012 12:36:09 JOB_5340 08/23/2012 12:36:14 JOB_5340 08/23/2012... (2 Replies)
Discussion started by: asnandhakumar
2 Replies

4. UNIX for Dummies Questions & Answers

Find time difference

I have a file wich contains time formats and i need to get the time difference TIME1 TIME2 =============== =================== 20120624192555.6Z 20120624204006.5Z which means first date 2012/6/24 19:25:55,second date 2012/6/24 20:40:06 so when i get the time... (23 Replies)
Discussion started by: wnaguib
23 Replies

5. Shell Programming and Scripting

How to find time difference?

I have a file wich contains time formats and i need to get the time difference TIME1 TIME2 ================================== 20120624192555.6Z 20120624204006.5Z which means first date 2012/6/24 19:25:55,second date 2012/6/24 20:40:06 so when i get the time... (1 Reply)
Discussion started by: wnaguib
1 Replies

6. Shell Programming and Scripting

Find time difference between two consecutive lines in same file.

Hello I have a file in following format: IV 08:09:07 NM 08:12:01 IC 08:12:00 MN 08:14:20 NM 08:14:15 I need a script to compare time on each line with previous line and show the inconsecutive line. Ex.: 08:12:00 08:14:15 A better way... (6 Replies)
Discussion started by: vilibit
6 Replies

7. Shell Programming and Scripting

Find Time difference in Unix

Hi, START_TIME :- "10-NOV-2009 00:00:04" STOP_TIME :- "10-NOV-2009 00:05:47" Please help to find difference between these two. Searched for the same topic but did not find an answer for the same time format :( Regards, Robin (3 Replies)
Discussion started by: robinbannis
3 Replies

8. AIX

How to find time difference between 2 timestamps?

HI All, can some one please help me how to fine the difference between two time stamps say a= Nov 10, 2009 9:21:25 AM b= Nov 10, 2009 10:21:25 AM I want to find difference between the a & b I googled and tried with some options but no luck. My OS is AIX (1 Reply)
Discussion started by: bandlan9
1 Replies

9. Shell Programming and Scripting

Help to find the time difference between the lines

Hi guru's, Am new to shell scripting. I am getting the below o/p from the oracle database, when I fire a query. ID JOB_ID ELAPSED_TIME FROM TO ----- ------ ------------------- -------- -------- 62663 11773 01/06/2009 09:49:13 SA CM 62664 11773 ... (4 Replies)
Discussion started by: sathik
4 Replies

10. Shell Programming and Scripting

To find the time difference between two lines of the same log file

Hello Friends, I want to write a script for the following: nlscux62:tibprod> grep "2008 Apr 30 01:" SA_EHV_SPEED_SFC_IN_03-SA_EHV_SPEED_SFC_IN_03-2.log | grep -i post | more 2008 Apr 30 01:01:23:928 GMT +2 SAPAdapter.SA_EHV_SPEED_SFC_IN_03-SA_EHV_SPEED_SFC_IN_03-2 Info AER3-000095 IDOC... (2 Replies)
Discussion started by: satyakam
2 Replies
Login or Register to Ask a Question