Grep a log file starting from a specific time to the end of file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Grep a log file starting from a specific time to the end of file
# 1  
Old 08-14-2018
Grep a log file starting from a specific time to the end of file

I have a log file which have a date and time at the start of every line.

I need to search the log file starting from a specific time to the end of file.

For example:
Starting point: July 29 2018 21:00:00
End point : end of file

My concern is what if the pattern of `July 29 2018 21:00:00` does not exist, I still want to get the lines between for example `July 29 2018 21:05:11` since this is still beyond `July 29 2018 21:00:00`.
# 2  
Old 08-14-2018
Since you did not specify your environment, I assume Linux/Bash.



You may convert the timestamp to epoch(= time in seconds since 1.1.1970) and print only the lines where the value is higher:


Code:
start_date="July 29 2018 21:00:00"

start_seconds="$(date +"%s" --date="$start_date")"
while read month day year time rest;do 

   current_line_seconds=$(date +"%s" --date="$month $day $year $time")
   [ $current_line_seconds -gt $start_seconds ] && \
       echo "$month $day $year $time $rest"; 
done <your_data_file.txt

Performance of this solution is only suited for small file sizes since one process is spawned for every line.
# 3  
Old 08-14-2018
I'm not sure if this is what you meant but I am using AIX 7.1. I am getting the error below:
date: 0551-402 Invalid character in date/time specification.
Usage: date [-u] [+"Field Descriptors"]

I think it has something to do with the --date option.
# 4  
Old 08-14-2018
This does not work in AIX. It's an older date implementation there.

------ Post updated at 08:47 PM ------

I have some awk script for you, which you will need to adapt to your needs. Maybe a starting point for you to learn to get aquainted with GNU awk.


Code:
#!/usr/bin/gawk -f
#
#       Usage:
#
#       ./log_output_from "timespec" <input_file.txt
#
#               timespec is: "YYYY MM DD HH MM SS"
#
#       Example: 
#
#       ./log_output_from "2018 01 01 16 00 00" <input_file.txt
#
#
#
#
#       Sample Input data
#
#       Jul 2018 16:17:30 Some text here
#


function epoch(year,month,day,hour,min,sec) {

         return mktime(year" "month" "day" "hour" "min" "sec) 

}

BEGIN {
        months_str=("Jan_Feb_Mar_Apr_May_Jun_Jul_Aug_Sep_Oct_Nov_Dec")
        split(months_str, month_names, "_");
        for(name in month_names) {
                i=i+1
                month_number[month_names[i]]=i
        }
        start_seconds=mktime(start_time)

 }
# if the time is reached, no more checking, just print every line

 (time_reached==1) {
        print($0)
        next
}
{
        split($4,mytime,":")
        current_line_seconds=epoch($3,month_number[$1],$2,mytime[1],mytime[2],mytime[3]) 
        if(current_line_seconds > start_seconds) {
                time_reached = 1
                print($0)
        }
}


Last edited by stomp; 08-14-2018 at 03:53 PM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

How to grep a line not starting with # from a file (there are two lines starting with # and normal)?

e.g. File name: File.txt cat File.txt Result: #INBOUND_QUEUE=FAQ1 INBOUND_QUEUE=FAQ2 I want to get the value for one which is not commented out. Thanks, (3 Replies)
Discussion started by: Tanu
3 Replies

2. Shell Programming and Scripting

Extract specific line in an html file starting and ending with specific pattern to a text file

Hi This is my first post and I'm just a beginner. So please be nice to me. I have a couple of html files where a pattern beginning with "http://www.site.com" and ending with "/resource.dat" is present on every 241st line. How do I extract this to a new text file? I have tried sed -n 241,241p... (13 Replies)
Discussion started by: dejavo
13 Replies

3. Shell Programming and Scripting

Search for logs traced between specific date and time from log file

HI, I want to search for a logs which are trace between specific date and time from logs file. My logs are generated like this :- Tue Jun 18 05:00:02 EEST 2013 | file_check.sh| Message:script has files to process. Thu Jun 20 05:00:02 EEST 2013 | file_check.sh| Message:script has files to... (5 Replies)
Discussion started by: ketanraut
5 Replies

4. UNIX for Dummies Questions & Answers

Add strings from one file at the end of specific lines in text file

Hello All, this is my first post so I don't know if I am doing this right. I would like to append entries from a series of strings (contained in a text file) consecutively at the end of specifically labeled lines in another file. As an example: - the file that contains the values to be... (3 Replies)
Discussion started by: gus74
3 Replies

5. Shell Programming and Scripting

Grep in a log file within a time range (hour)

Hi, im trying to write a grep script that returns me the last inputs added in the last hour in the log file. Literally i have nothing yet but: grep 'Line im looking for' LOGFILE.log | tail -1 this only gives me the last input, but no necessarily from the last hour. Help Please. (4 Replies)
Discussion started by: blacksteel1988
4 Replies

6. Shell Programming and Scripting

need to grep contents of a file within specific time span. regex i am using is not working

Hi , I am trying to extract contents of a file between specified time stamp. but it does not seem to work. i am trying to extract output of /var/adm/messages between 15:00:00 to 15:23:59 . i have tried two regex the first one seems to kind of work. it displays some output. the second one is... (13 Replies)
Discussion started by: chidori
13 Replies

7. Shell Programming and Scripting

Grep the Content of a LOG File which has latest Date and Time

Hi All, Need a small help. I have a log file which keeps updating for every Minute with multiple number of lines. I just want to grep few properties which has latest Date and Time to it. How do i do it? I wanted to grep a property by name "Reloading cache with a maximum of" from the... (4 Replies)
Discussion started by: nvindraneel
4 Replies

8. Shell Programming and Scripting

Grep starting from a specific position

Hello people, I'm scratch my head to find a solution to my problem, I'm absolutely sure this is very simple!!! :wall: I'm using the tcpdump to show on the screen in real time the UCP traffic: tcpdump -l -i bond1 -s 1514 -nntttt -A src or dst 192.168.1.5 and port 10000 | egrep "/51/"The output... (5 Replies)
Discussion started by: Lord Spectre
5 Replies

9. Shell Programming and Scripting

Grep from a starting line till the end of the file

Hi Folks, I got to know from this forums on how to grep from a particular line say line 6 awk 'NR==6 {print;exit}' But how do i grep from line 6 till the end of the file or command output. Thanks, (3 Replies)
Discussion started by: Mr. Zer0
3 Replies

10. Shell Programming and Scripting

Print starting 3rd line until end of the file.

Hi, I want to Print starting 3rd line until end of the file. Pls let me know the command. Thanks in advance. (1 Reply)
Discussion started by: smc3
1 Replies
Login or Register to Ask a Question