Visit Our UNIX and Linux User Community


Get all lines in a file after particular time


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Get all lines in a file after particular time
# 1  
Old 08-27-2009
Get all lines in a file after particular time

Hi All,

I am lookig for a way to get all the lines from a log file which has been updated 5 mins prior to the system time.

The log file will be like below:

09:01:00 Started polling
09:01:05 Checking directory test
09:02:00 Error! Cannot access directory test
09:03:00 Polling


I get the system time and calculate 5 mins before the system time. For example if the system time is 09:05, then the script will calculate the time to check as 09:00. Then a sed command will be run to get all the lines below the pattern:

sed '/$time/,$p' $logfile

Which will print all the lines below the pattern. But if the log file does not contain 09:00, no lines will be grepped. Is there a way to solve this issue?

Thanks
D
# 2  
Old 08-27-2009
why don't you first grep for 09:00 and make sure the log is written for 09:00 and if grep succeeds, you proceed with sed... in other case you can increase/decrease the time by 01 min or so.
# 3  
Old 08-27-2009
Why dont you append a line with the time every 5 minutes for you to use then?
# 4  
Old 08-27-2009
Thats a good idea. But the problem is that there are a number of log files in which the command is checking.. in that case we will need to append the time in all the log files.
# 5  
Old 08-27-2009
Code:
#!/bin/sh
five_mins_ago=$(date -d '5 min ago' +%s)
awk -v start=$five_mins_ago '{
        getlogtime="date -d "$1" +%s"
        getlogtime | getline logtime
        if (logtime > start ) print $0
}' $logfile

# 6  
Old 08-27-2009
Another awk solution:
Code:
BEGIN{"date +'%H:%m:%S' --date 'now 5 minutes ago'" | getline sys_time}
$1 >= sys_time {print}

# 7  
Old 08-28-2009
The date -d doesnt work in my version of unix.

Therefore I have been trying to follow the approach by vbe to write a specific line to all the log files in every 5 mins.

But I am facing another issue with sed:

check_param="Fri Aug 28 12:59:14 BST 2009ALERT CHECKPOINT"

Then Im using sed to get all the lines after the above string from the log files like below:

sed -n /$check_param/,$p test.log

But Im getting the error: sed: 0602-404 Function /Fri cannot be parsed.

If I use the string itself instead of the variable, im getting the correct output. I believe I am not using the variable in sed correctly.

Thanks
D

Previous Thread | Next Thread
Test Your Knowledge in Computers #475
Difficulty: Medium
A Yottabyte (YB) equals 1,208,925,819,614,629,174,706,176 bytes.
True or False?

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Check/Parse log file's lines using time difference/timestamp

I was looking at this script which outputs the two lines which differs less than one sec. #!/usr/bin/perl -w use strict; use warnings; use Time::Local; use constant SEC_MILIC => 1000; my $file='infile'; ## Open for reading argument file. open my $fh, "<", $file or die "Cannot... (1 Reply)
Discussion started by: cele_82
1 Replies

2. Shell Programming and Scripting

Read multiple lines at a time from file

Hello All, I have a file like below.... dn: cn=user1,ou=org,o=org cn=user1 uid=user1 cn=user2,ou=org,o=org cn=user2 uid=user2 cn=user3,ou=org,o=org cn=user3 cn=user33 uid=user3 cn=user4,ou=org,o=org cn=user4 uid=user4 (6 Replies)
Discussion started by: s_linux
6 Replies

3. Shell Programming and Scripting

Read two lines at time from a file

Hello community, what I need to do is read 2 rows at time from a file. I have this simple solution: File to read: LINE1 LINE2 LINE3 LINE4 LINE5 LINE6 LINE7 LINE8Read routine:#!/bin/ksh sed '1,3d' /out.txt | while read line; do read line2 echo $line $line2 doneResult:LINE1... (5 Replies)
Discussion started by: Lord Spectre
5 Replies

4. Shell Programming and Scripting

Find time difference between two consecutive lines in same file.

Hello I have a file in following format: IV 08:09:07 NM 08:12:01 IC 08:12:00 MN 08:14:20 NM 08:14:15 I need a script to compare time on each line with previous line and show the inconsecutive line. Ex.: 08:12:00 08:14:15 A better way... (6 Replies)
Discussion started by: vilibit
6 Replies

5. Shell Programming and Scripting

Searching the lines within a range of time period in a text file

Dear All, Please advice me, I have a text file with one field date and time like below given. I need to find out the lines whchi content the time stamp between Wed May 26 11:03:11 2010 and Wed May 26 11:03:52 2010 both can be included, using awk command which could be an interactive so that I... (6 Replies)
Discussion started by: chinmayadalai
6 Replies

6. Shell Programming and Scripting

Need to parse file "x" lines at a time ... awk array?

I have files that store multiple data points for the same device "vertically" and include multiple devices. It repeats a consistant pattern of lines where for each line: Column 1 is a common number for the entire file and all devices in that file Column 2 is a unique device number Column 3 is... (7 Replies)
Discussion started by: STN
7 Replies

7. Shell Programming and Scripting

Extracting lines in file based on time

Hi, anyone has any ideas on how do we extract lines from a file with format similiar to this: (based on current time) Jun 18 00:16:50 .......... ............. ............ Jun 18 00:17:59 .......... ............. ............ Jun 18 01:17:20 .......... ............. ............ Jun 18... (5 Replies)
Discussion started by: faelric
5 Replies

8. Shell Programming and Scripting

time diffrence between two lines in a log file

Hi Experts, I am writing a shell/Perl script tp get a time diffrence of two lines in a log file. I am pasting a snippet of my log file, how it look likes. Sun Apr 26 02:11:03.162 2009 Greenwich Standard Time LOW: pid 488 tid 2072: 0: 251360: calling ARRF_LIB_Evaluate. Sun Apr 26... (24 Replies)
Discussion started by: namishtiwari
24 Replies

9. Shell Programming and Scripting

Pattern matching in file and then display 10 lines above every time

hiii, i have to write a shell script like this---- i have a huge log file name abc.log .i have to search for a pattern name "pattern",it may occur 1000 times in the log file,every time it finds the pattern it should display the 10 lines above the pattern. I appericiate your help. (30 Replies)
Discussion started by: namishtiwari
30 Replies

10. Shell Programming and Scripting

To find the time difference between two lines of the same log file

Hello Friends, I want to write a script for the following: nlscux62:tibprod> grep "2008 Apr 30 01:" SA_EHV_SPEED_SFC_IN_03-SA_EHV_SPEED_SFC_IN_03-2.log | grep -i post | more 2008 Apr 30 01:01:23:928 GMT +2 SAPAdapter.SA_EHV_SPEED_SFC_IN_03-SA_EHV_SPEED_SFC_IN_03-2 Info AER3-000095 IDOC... (2 Replies)
Discussion started by: satyakam
2 Replies

Featured Tech Videos