Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Parsing log file for last 2 hours

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Parsing log file for last 2 hours
# 1  
Old 09-15-2013
Parsing log file for last 2 hours
I want to parse a log file which i am grepping root user connection but is showing whole day and previous day detail as well.

First i want to see last 2 hours log file then after that i want to search particular string. Lets suppose right now its 5:00PM, So i want to see the log of 3:00PM to 5:00PM.

my log file is

Code:
/var/log/secure

# 2  
Old 09-15-2013
Can you give more of a clue:-
1) HW if possible.
2) OS.
3) Shell type, e.g. bash.
4) A snippet of the log file in question so that we can help.
5) What have you tried so far?
This User Gave Thanks to wisecracker For This Post:
jim mcnamara
# 3  
Old 09-15-2013
I am using Fedora-17 and my shell is bash shell, My log file name i already mention, i don't want anything special.

Code:
Sep 15 01:41:33 servername
Sep 15 01:46:05 servername
Sep 15 02:46:05 servername
Sep 15 02:41:33 servername
Sep 15 02:46:05 servername
Sep 15 03:46:05 servername
Sep 15 03:41:33 servername
Sep 15 03:46:05 servername
Sep 15 04:46:05 servername
Sep 15 04:41:33 servername
Sep 15 04:46:05 servername
Sep 15 04:46:05 servername
Sep 15 05:41:33 servername
Sep 15 05:46:05 servername
Sep 15 05:46:05 servername
Sep 15 05:41:33 servername
Sep 15 06:46:05 servername
Sep 15 06:46:05 servername
Sep 15 06:41:33 servername
Sep 15 06:46:05 servername
Sep 15 07:46:05 servername
Sep 15 07:41:33 servername
Sep 15 07:46:05 servername
Sep 15 08:46:05 servername

I want log of two last hours. So i want last two hours log. Current my time is 08:50 so it show log time of 6 hour and 7 hour. Below is expected output, if i run at 12:10PM so it will show log of 10:00AM and 11:00AM vice versa.

Code:
Sep 15 06:46:05 servername
Sep 15 06:46:05 servername
Sep 15 06:41:33 servername
Sep 15 06:46:05 servername
Sep 15 07:46:05 servername
Sep 15 07:41:33 servername
Sep 15 07:46:05 servername

Last edited by learnbash; 09-15-2013 at 01:26 PM.. Reason: further detail provided
# 4  
Old 09-15-2013
You did not mention what you have tried so far, which should become a good habit so you can improve yourself.

But, as this task is soooo easy, here we go:
Code:
grep -E "Sep 15 (06|07)" file
Sep 15 06:46:05 servername
Sep 15 06:46:05 servername
Sep 15 06:41:33 servername
Sep 15 06:46:05 servername
Sep 15 07:46:05 servername
Sep 15 07:41:33 servername
Sep 15 07:46:05 servername

# 5  
Old 09-15-2013
Hi,
You can do it (in concept):
input file example:
Code:
$ cat file1.log
Sep 15 01:41:33 servername
Sep 15 01:46:05 servername
Sep 15 02:46:05 servername
Sep 15 02:41:33 servername
Sep 15 02:46:05 servername
Sep 15 03:46:05 servername
Sep 15 03:41:33 servername
Sep 15 03:46:05 servername
Sep 15 04:46:05 servername
Sep 15 04:41:33 servername
Sep 15 04:46:05 servername
Sep 15 04:46:05 servername
Sep 15 05:41:33 servername
Sep 15 05:46:05 servername
Sep 15 05:46:05 servername
Sep 15 05:41:33 servername
Sep 15 06:46:05 servername
Sep 15 06:46:05 servername
Sep 15 16:41:33 servername
Sep 15 16:46:05 servername
Sep 15 17:46:05 servername
Sep 15 17:41:33 servername
Sep 15 17:46:05 servername
Sep 15 18:46:05 servername
Sep 15 18:41:33 servername
Sep 15 18:46:05 servername
Sep 15 18:46:05 servername
Sep 15 18:41:33 servername
Sep 15 19:46:05 servername
Sep 15 19:46:05 servername

Date and hour of test:
Code:
$ LANG=C date
Sun Sep 15 19:28:13 CEST 2013

All lines 2 hours ago full to end file:
Code:
$ sed -n "/^$(LANG=C date --date='2 hours ago' '+%b %d %H:')/,\$p" file1.log
Sep 15 17:46:05 servername
Sep 15 17:41:33 servername
Sep 15 17:46:05 servername
Sep 15 18:46:05 servername
Sep 15 18:41:33 servername
Sep 15 18:46:05 servername
Sep 15 18:46:05 servername
Sep 15 18:41:33 servername
Sep 15 19:46:05 servername
Sep 15 19:46:05 servername

Or, all lines 2 hours ago full only (in this example: 17 and 18 but no 19):
Code:
$ sed -n "/^$(LANG=C date --date='2 hours ago' '+%b %d %H:')\\|^$(LANG=C date --date='1 hours ago' '+%b %d %H:')/p" file1.log
Sep 15 17:46:05 servername
Sep 15 17:41:33 servername
Sep 15 17:46:05 servername
Sep 15 18:46:05 servername
Sep 15 18:41:33 servername
Sep 15 18:46:05 servername
Sep 15 18:46:05 servername
Sep 15 18:41:33 servername

PS: I use LANG=C because my computer configuration is french...

Regards.
This User Gave Thanks to disedorgue For This Post:
learnbash
# 6  
Old 09-16-2013
Actually i modified the script and checking five hours log, but it is not working, below i have tried.

Code:
sed -n "/^$(date --date='5 hours ago' '+%b %d %H:')\\|^$(date --date='1 hours ago' '+%b %d %H:')/p" /var/log/secure

# 7  
Old 09-16-2013
It's normal, this solution work fine for two contiguous time slots.
A solution:
Code:
 sed -n "/^$(date --date='5 hours ago' '+%b %d %H:')/,\${/^$(date --date='0 hours ago' '+%b %d %H:')/q;p}" /var/log/secure

But here, you must do -1 to second hour field.

Regards.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Help 'speeding' up this 'parsing' script - taking 24+ hours to run

Hi, I've written a ksh script that read a file and parse/filter/format each line. The script runs as expected but it runs for 24+ hours for a file that has 2million lines. And sometimes, the input file has 10million lines which means it can be running for more than 2 days and still not finish.... (9 Replies)
Discussion started by: newbie_01
9 Replies

2. Shell Programming and Scripting

HELP on parsing this log file

Hi, I have a log file that looks like below and I am wanting to know if there is a better way of parsing it from how I am doing it right now. I am looking for when an application service is OFFLINE and ONLINE. This log file is getting written into every 30 minutes ... (1 Reply)
Discussion started by: newbie_01
1 Replies

3. Shell Programming and Scripting

Parsing Log File help

Hi, I am a newbie to scripting. I have multiple log files (saved as .gz) in a directory that looks like this 01-01-2013 10:00 pn: 123 01-01-2013 10:00 sn: 987 01-01-2013 10:00 Test1 01-01-2013 10:00 Result: Pass 01-01-2013 10:00 Time: 5:00 01-01-2013 10:00 Test2 01-01-2013 10:00... (3 Replies)
Discussion started by: linuxnew
3 Replies

4. Shell Programming and Scripting

Help Parsing a Log File

Hello all, I am new to scripting and I have written a script that performs an Rsync on my NAS and then moves on to send me an email with the status etc. The problem is that I think Rsync is taking to long to complete and the IF statement is timing out, as it doesn't appear to move on. Here... (1 Reply)
Discussion started by: Mongrel
1 Replies

5. Shell Programming and Scripting

Log file text parsing

I'm new to scripting and was wondering if there was a way to accomplish what I want below using shell script(s). If there is a log file as follows, where the id is the unique id of a process, with the timestamp of when the process began and completed displayed, would it be possible to find the... (3 Replies)
Discussion started by: dizydolly
3 Replies

6. UNIX for Dummies Questions & Answers

parsing a log file

I need help in parsing the following log files. 10 Apr 2009 0:16:16 * name: Tuna Belly Format: Well done, Price: 999 only 10 Apr 2009 0:16:16 * name: Roast Beef Format: Raw, Price: 55 c 10 Apr 2009 0:16:16 * name: Pasta Format: Dry, Price: 88.43 only etcetc I need to parse this... (8 Replies)
Discussion started by: izuma
8 Replies

7. UNIX for Dummies Questions & Answers

Script for parsing details in a log file to a seperate file

Hi Experts, Im a new bee for scripting, I would ned to do the following via linux shell scripting, I have an application which throws a log file, on each action of a particular work with the application, as sson as the action is done, the log file would vanish or stops updating there, the... (2 Replies)
Discussion started by: pingnagan
2 Replies

8. Shell Programming and Scripting

Last 24 hours of a log file

I'm looking to pull the last 24 hours of a log file. Here's what I've got so far: yesterday=$(TZ=$TZ+24 date +"%b %e %H:%M") today=$(date +"%b %e %H:%M") echo $yesterday $today grep -E "^$yesterday|^$today" /var/adm/syslog/syslog.log But that pulls everything from $yesterday from... (1 Reply)
Discussion started by: Bert
1 Replies

9. Shell Programming and Scripting

Help with script parsing a log file

I have a large log file, which I want to first use grep to get the specific lines then send it to awk to print out the specific column and if the result is zero, don't do anything. What I have so far is: LOGDIR=/usr/local/oracle/Transcription/log ERRDIR=/home/edixftp/errors #I want to be... (3 Replies)
Discussion started by: mevasquez
3 Replies

10. Shell Programming and Scripting

Parsing a Log file

Hi All, I'm deffently not a Unix specialist so be Gentel. I need to parse a Log file that looks like that: 2006-06-12 01:00:00,463 ERROR {cleanLoggersFiles} General Error comverse.compas.shared.exceptions.SystemParametersException: Error in reading parameter FileLocation at... (4 Replies)
Discussion started by: tbirenzweig
4 Replies
Login or Register to Ask a Question