Sponsored Content
Top Forums Shell Programming and Scripting Parsing log file for last 2 hours Post 302853613 by disedorgue on Sunday 15th of September 2013 01:39:59 PM
Old 09-15-2013
Hi,
You can do it (in concept):
input file example:
Code:
$ cat file1.log
Sep 15 01:41:33 servername
Sep 15 01:46:05 servername
Sep 15 02:46:05 servername
Sep 15 02:41:33 servername
Sep 15 02:46:05 servername
Sep 15 03:46:05 servername
Sep 15 03:41:33 servername
Sep 15 03:46:05 servername
Sep 15 04:46:05 servername
Sep 15 04:41:33 servername
Sep 15 04:46:05 servername
Sep 15 04:46:05 servername
Sep 15 05:41:33 servername
Sep 15 05:46:05 servername
Sep 15 05:46:05 servername
Sep 15 05:41:33 servername
Sep 15 06:46:05 servername
Sep 15 06:46:05 servername
Sep 15 16:41:33 servername
Sep 15 16:46:05 servername
Sep 15 17:46:05 servername
Sep 15 17:41:33 servername
Sep 15 17:46:05 servername
Sep 15 18:46:05 servername
Sep 15 18:41:33 servername
Sep 15 18:46:05 servername
Sep 15 18:46:05 servername
Sep 15 18:41:33 servername
Sep 15 19:46:05 servername
Sep 15 19:46:05 servername

Date and hour of test:
Code:
$ LANG=C date
Sun Sep 15 19:28:13 CEST 2013

All lines 2 hours ago full to end file:
Code:
$ sed -n "/^$(LANG=C date --date='2 hours ago' '+%b %d %H:')/,\$p" file1.log
Sep 15 17:46:05 servername
Sep 15 17:41:33 servername
Sep 15 17:46:05 servername
Sep 15 18:46:05 servername
Sep 15 18:41:33 servername
Sep 15 18:46:05 servername
Sep 15 18:46:05 servername
Sep 15 18:41:33 servername
Sep 15 19:46:05 servername
Sep 15 19:46:05 servername

Or, all lines 2 hours ago full only (in this example: 17 and 18 but no 19):
Code:
$ sed -n "/^$(LANG=C date --date='2 hours ago' '+%b %d %H:')\\|^$(LANG=C date --date='1 hours ago' '+%b %d %H:')/p" file1.log
Sep 15 17:46:05 servername
Sep 15 17:41:33 servername
Sep 15 17:46:05 servername
Sep 15 18:46:05 servername
Sep 15 18:41:33 servername
Sep 15 18:46:05 servername
Sep 15 18:46:05 servername
Sep 15 18:41:33 servername

PS: I use LANG=C because my computer configuration is french...

Regards.
This User Gave Thanks to disedorgue For This Post:
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Parsing a Log file

Hi All, I'm deffently not a Unix specialist so be Gentel. I need to parse a Log file that looks like that: 2006-06-12 01:00:00,463 ERROR {cleanLoggersFiles} General Error comverse.compas.shared.exceptions.SystemParametersException: Error in reading parameter FileLocation at... (4 Replies)
Discussion started by: tbirenzweig
4 Replies

2. Shell Programming and Scripting

Help with script parsing a log file

I have a large log file, which I want to first use grep to get the specific lines then send it to awk to print out the specific column and if the result is zero, don't do anything. What I have so far is: LOGDIR=/usr/local/oracle/Transcription/log ERRDIR=/home/edixftp/errors #I want to be... (3 Replies)
Discussion started by: mevasquez
3 Replies

3. Shell Programming and Scripting

Last 24 hours of a log file

I'm looking to pull the last 24 hours of a log file. Here's what I've got so far: yesterday=$(TZ=$TZ+24 date +"%b %e %H:%M") today=$(date +"%b %e %H:%M") echo $yesterday $today grep -E "^$yesterday|^$today" /var/adm/syslog/syslog.log But that pulls everything from $yesterday from... (1 Reply)
Discussion started by: Bert
1 Replies

4. UNIX for Dummies Questions & Answers

Script for parsing details in a log file to a seperate file

Hi Experts, Im a new bee for scripting, I would ned to do the following via linux shell scripting, I have an application which throws a log file, on each action of a particular work with the application, as sson as the action is done, the log file would vanish or stops updating there, the... (2 Replies)
Discussion started by: pingnagan
2 Replies

5. UNIX for Dummies Questions & Answers

parsing a log file

I need help in parsing the following log files. 10 Apr 2009 0:16:16 * name: Tuna Belly Format: Well done, Price: 999 only 10 Apr 2009 0:16:16 * name: Roast Beef Format: Raw, Price: 55 c 10 Apr 2009 0:16:16 * name: Pasta Format: Dry, Price: 88.43 only etcetc I need to parse this... (8 Replies)
Discussion started by: izuma
8 Replies

6. Shell Programming and Scripting

Log file text parsing

I'm new to scripting and was wondering if there was a way to accomplish what I want below using shell script(s). If there is a log file as follows, where the id is the unique id of a process, with the timestamp of when the process began and completed displayed, would it be possible to find the... (3 Replies)
Discussion started by: dizydolly
3 Replies

7. Shell Programming and Scripting

Help Parsing a Log File

Hello all, I am new to scripting and I have written a script that performs an Rsync on my NAS and then moves on to send me an email with the status etc. The problem is that I think Rsync is taking to long to complete and the IF statement is timing out, as it doesn't appear to move on. Here... (1 Reply)
Discussion started by: Mongrel
1 Replies

8. Shell Programming and Scripting

Parsing Log File help

Hi, I am a newbie to scripting. I have multiple log files (saved as .gz) in a directory that looks like this 01-01-2013 10:00 pn: 123 01-01-2013 10:00 sn: 987 01-01-2013 10:00 Test1 01-01-2013 10:00 Result: Pass 01-01-2013 10:00 Time: 5:00 01-01-2013 10:00 Test2 01-01-2013 10:00... (3 Replies)
Discussion started by: linuxnew
3 Replies

9. Shell Programming and Scripting

HELP on parsing this log file

Hi, I have a log file that looks like below and I am wanting to know if there is a better way of parsing it from how I am doing it right now. I am looking for when an application service is OFFLINE and ONLINE. This log file is getting written into every 30 minutes ... (1 Reply)
Discussion started by: newbie_01
1 Replies

10. Shell Programming and Scripting

Help 'speeding' up this 'parsing' script - taking 24+ hours to run

Hi, I've written a ksh script that read a file and parse/filter/format each line. The script runs as expected but it runs for 24+ hours for a file that has 2million lines. And sometimes, the input file has 10million lines which means it can be running for more than 2 days and still not finish.... (9 Replies)
Discussion started by: newbie_01
9 Replies
ATSADC(1)							       local								 ATSADC(1)

NAME
atsadc, atsa1, atsaftp, atsahttp -- counter-collection SYNOPSIS
atsadc [ t n ] [ ofile ] atsa1 [ t n ] atsaftp atsahttp DESCRIPTION
System activity-data can be gathered on special request of a user [see atsar(1) ] or automatically, on a routine basis, as described here. Usually the kernel maintains statistical counters that are incremented as various system actions occur. These include counters for CPU uti- lization, disk utilization, memory utilization and various network statistics. The program atsadc and the shell-script atsa1 are used to collect, save, and process these counters. The program atsadc (the data collector) samples system data n times with an interval of t seconds between samples, and writes in binary format to ofile or (default) to standard output. The sampling interval t should be greater than 1 second. If t and n are omitted, a special reset-record is written. This facility is used when booting to a multi-user state, to mark the time at which the counters restart from zero. For example, the reset-mark can be added to the daily data by the command: /usr/local/bin/atsadc /var/log/atsar/atsa`date +%d` Note that this entry is written to the /etc/rc.d/init.d/atsar file. The shell-script atsa1 is used to collect and store data in the binary file /var/log/atsar/atsadd where dd is the current day of the month. The arguments t and n cause records to be written n times at an interval of t seconds, or once if omitted. Furthermore this script takes care that log-files older than a week are removed once a day. A file containing following entries should be added to the /etc/cron.d directory to produce records every 20 minutes during working hours and hourly otherwise: 0 * * * 0-6 root /usr/local/bin/atsa1 20,40 8-17 * * 1-5 root /usr/local/bin/atsa1 See crontab(1) for details. The shell-script atsaftp counts the new transfers registered in the FTP-logfile(s) since the previous time this script was activated; the new counters are stored in the /var/log/atsar/ftpstat file in ASCII-format. The names of the FTP-logfiles to be watched are specified in the /etc/atsar.conf configuration-file. The shell-script atsahttp counts the new transfers registered in the HTTP-logfile(s) since the previous time this script was activated; the new counters are stored in the /var/log/atsar/httpstat file in ASCII-format. The names of the HTTP-logfiles to be watched are specified in the /etc/atsar.conf configuration-file. Both scripts must be activated just before the program atsadc is started, which also collects these counters. FILES
/var/log/atsar/atsadd Daily data file, where dd are digits representing the day of the month. SEE ALSO
atsar(1), crontab(1) AUTHOR
Gerlof Langeveld, AT Computing (gerlof@ATComputing.nl) AT Computing July 2004 ATSADC(1)
All times are GMT -4. The time now is 12:13 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy