Extract date ranged data from log file


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Extract date ranged data from log file
# 1  
Old 12-11-2013
Extract date ranged data from log file

Hi,

I am trying to extract lines of data within a log file on a Redhat 5 Linux system.

eg I need all the lines with a particular username over the last 3 minutes.

the log file may read like this, and I want a way to search all the lines extracting all the relevant lines over the last 3 minutes:

"Wed Dec 11 16:51:00 GMT 2013 username permissions"

Thanks in advance
# 2  
Old 12-11-2013
Here is an example using date command to get last 3 minute timestamps and using an awk program to fetch those lines:
Code:
#!/bin/bash

for c in {1..3}
do
        t=$( date -d"-${c} min" )
        [ -z "$DT" ] && DT="${t%:*}" || DT="${DT},${t%:*}"
done

awk -v D="$DT" -v U="username" '
        BEGIN {
                n = split ( D, A, "," )
        }
        {
                for ( i = 1; i <= n; i++ )
                {
                        if ( ( $0 ~ A[i] ) && ( $0 ~ U ) )
                                print $0
                }
        }
' file.log

This User Gave Thanks to Yoda For This Post:
# 3  
Old 12-11-2013
Thank you Yoda, could you please explain this as it looks abit tricky.

Also I have been told that the actual number of minutes to search will vary, so if it is possible to set a variable and value to this.

Thanks in advance.
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Extract data from log file within specified time

So, we have a script, that is supposed to have a couple of functions like showing number of failed connections, recieved bytes per IP-address, and so on. We are supposed to be able to limit the number of results to either 0-24 hours or X days back from the last data in the log file. Everything... (3 Replies)
Discussion started by: Plumpen
3 Replies

2. Shell Programming and Scripting

Extract data from log file in specified range of time

I was searching for parsing a log file and found what I need in this link http://stackoverflow.com/questions/7575267/extract-data-from-log-file-in-specified-range-of-time But the most useful answer (posted by @Kent): # this variable you could customize, important is convert to seconds. # e.g... (2 Replies)
Discussion started by: kingk110
2 Replies

3. Shell Programming and Scripting

How to extract log data based on current date and month ?

Hi Gurus, I'm using HP-UX B.11.23 operating system. I've been trying to extract this log info based on the current date and month, but was having some issues as the date column which on the 4th column has a comma and the 5th column has a dot tied to it. Here is the output from my shut... (5 Replies)
Discussion started by: superHonda123
5 Replies

4. Shell Programming and Scripting

Data Extract from XML Log File

Please help me out to extract the Data from the XML Log files. So here is the data ERROR|2010-08-26 00:05:52,958|SERIAL_ID=128279996|ST=2010-08-2600:05:52|DEVICE=113.2.21.12:601|TYPE=TransactionLog... (9 Replies)
Discussion started by: raghunsi
9 Replies

5. Shell Programming and Scripting

How to extract log data based on date

Hi Gurus, I've been having some problem in extracting the log data based on the current date and month. As shown in the sample data below, how to extract the log info for Aug 11? Sample data: root pts/ta userpc Wed Aug 11 09:46 - 20:21 (10:35) root pts/ta userpc... (13 Replies)
Discussion started by: superHonda123
13 Replies

6. Shell Programming and Scripting

need a shell script to extract data from a log file.

If I have a log like : Mon Jul 19 05:07:34 2010; TCP; eth3; 52 bytes; from abc to def Mon Jul 19 05:07:35 2010; UDP; eth3; 46 bytes; from aaa to bbb Mon Jul 19 05:07:35 2010; TCP; eth3; 52 bytes; from def to ghi I will need an output like this : Time abc to def... (1 Reply)
Discussion started by: hitha87
1 Replies

7. Shell Programming and Scripting

Extract info from log file and compute using time date stamp

Looking for a shell script or a simple perl script . I am new to scripting and not very good at it . I have 2 directories . One of them holds a text file with list of files in it and the second one is a daily log which shows the file completion time. I need to co-relate both and make a report. ... (0 Replies)
Discussion started by: breez_drew
0 Replies

8. Shell Programming and Scripting

Extract data from log file from or after the specific date

Hi , I am having a script which will start a process and appends the process related logs to a log file. The log file writes logs with every line starting with date in the format of: date +"%Y %b %d %H:%M:%S". So, in the script, before I start the process, I am storing the date as DATE=`date +"%Y... (5 Replies)
Discussion started by: chiru_h
5 Replies

9. Shell Programming and Scripting

Extract date from log file

Hello All, I just need to extract the date portion from a apache log file I am able to do it using the chain of command - Logfile contents - First record - ========================== 197.130.211.240 - - "GET /jp/index.shtml HTTP/1.1" 200 24255... (4 Replies)
Discussion started by: jambesh
4 Replies

10. Shell Programming and Scripting

shell-script which extract data from log file

give me a shell-script which extract data from log file on a server by giving date and time as input (for both start time and end time) and it will give the logs generated during the given time as output. (4 Replies)
Discussion started by: abhishek27
4 Replies
Login or Register to Ask a Question