I have logs in format
I am looking out for a script which can extract data of last 15 minutes from the last recorded data in the log file and then search a string in it and store the results in a file.
Additionally, I tried the below command but doesn't work for me
Please wrap all code, files, input & output/errors in CODE tags.
It makes them far easier to read and preserves multiple spacing, which can be critical.
in what way does in not work? Do you have incomplete output, incorrect output, no output or error messages?
What is your logic for doing this if you did it as a human? Humans recognise dates/times in records, but you might have to:-
Read a line
Get the date part from the line
Convert to seconds
Compare against current date (also in seconds)
Display the output if the difference is less than 900 seconds
It's not pretty I agree and I can see what you are trying to do, but I'm not sure that awk can do that (happy to be corrected though)
I suppose another way would be to generate all the possible matches for the date/time you need (only 900 of them) in a file and use:-
I notice from your input (now that I've put it in CODE tags for you) that there are leading spaces on some lines. That might be a little awkward, but not insurmountable.
Would either of these approaches help? If you can be more explicit in your needs, then maybe we can help a little more.
I have logs in format
I am looking out for a script which can extract data of last 15 minutes from the last recorded data in the log file and then search a string in it and store the results in a file.
Additionally, I tried the below command but doesn't work for me
The command you tried can't work for at least the following reasons (there may be others):
when comparing dates, you need to compare year, month, and day (in that order); not day, month, and year,
the month names Jan, Feb, ... do not sort into increasing date order,
the dates in your log file are based on GMT, but the dates you are producing with the date command are based on the current process' TZ setting,
some of your input lines have leading spaces,
you have a command substitution inside a command substitution causing the from variable to be set to an empty string (and it should also generate a diagnostic similar to -ksh: ####<01-Mar-2015: not found that you didn't bother mentioning),
the value you seem to be trying to store into the from variable is not a field number, and
from what you have shown us, the string WORD does not appear anywhere in the input you are processing.
There is no need to convert the timestamps to seconds since the Epoch as long as you convert the date into a string that will sort correctly when doing a string comparison (i.e., 20150301 instead of 15-Jan-2015), and awk is certainly capable of converting the date portion of the format in your log file into the format above before comparing the results to the string produced by the command:
.
If we assume that <info> is shorthand for some kind of information being logged in your log files, and if <info> NEVER contains a <newline> character, and if <info> in some of those line contain the string WORD, then the following might do what you want:
Note that this is untested because the date utility on my system does not have a -d option; AND, you didn't supply any sample data that would produce any output from your sample input.
The way this is coded, if the <info> data in your input files does contain multi-line data, it will copy the entire <info> field to the output if the 1st line meets the date requirements as long as no line in your input data contains exactly three < and or - characters in the first field of a line that is not in the specified format.
These 2 Users Gave Thanks to Don Cragun For This Post:
So, we have a script, that is supposed to have a couple of functions like showing number of failed connections, recieved bytes per IP-address, and so on. We are supposed to be able to limit the number of results to either 0-24 hours or X days back from the last data in the log file.
Everything... (3 Replies)
Hi,
I am trying to extract lines of data within a log file on a Redhat 5 Linux system.
eg I need all the lines with a particular username over the last 3 minutes.
the log file may read like this, and I want a way to search all the lines extracting all the relevant lines over the last 3... (2 Replies)
I was searching for parsing a log file and found what I need in this link
http://stackoverflow.com/questions/7575267/extract-data-from-log-file-in-specified-range-of-time
But the most useful answer (posted by @Kent):
# this variable you could customize, important is convert to seconds.
# e.g... (2 Replies)
Please help me out to extract the Data from the XML Log files.
So here is the data
ERROR|2010-08-26 00:05:52,958|SERIAL_ID=128279996|ST=2010-08-2600:05:52|DEVICE=113.2.21.12:601|TYPE=TransactionLog... (9 Replies)
Hi Gurus,
I've been having some problem in extracting the log data based on the current date and month.
As shown in the sample data below, how to extract the log info for Aug 11?
Sample data:
root pts/ta userpc Wed Aug 11 09:46 - 20:21 (10:35)
root pts/ta userpc... (13 Replies)
If I have a log like :
Mon Jul 19 05:07:34 2010; TCP; eth3; 52 bytes; from abc to def
Mon Jul 19 05:07:35 2010; UDP; eth3; 46 bytes; from aaa to bbb
Mon Jul 19 05:07:35 2010; TCP; eth3; 52 bytes; from def to ghi
I will need an output like this :
Time abc to def... (1 Reply)
Hi , I am having a script which will start a process and appends the process related logs to a log file. The log file writes logs with every line starting with date in the format of: date +"%Y %b %d %H:%M:%S".
So, in the script, before I start the process, I am storing the date as DATE=`date +"%Y... (5 Replies)
give me a shell-script which extract data from log file on a server by giving date and time as input (for both start time and end time) and it will give the logs generated during the given time as output. (4 Replies)
HI,
I have a log file (sync.log) in the following format.
05-14 05:34:56 INFO Hxxx Start: Id:xxx
05-14 05:45:32 INFO Hxxx End: Id:xxxx
05-19 11:23:21 INFO Hxxx Start: Id:xxx
05-19 11:34:45 INFO Hxxx End: Id:xxxx
05-20 02:09:47 INFO Hxxx Start: Id:xxx
05-20 02:36:03 INFO Hxxx End: Id:xxx... (5 Replies)
So I got this old webapp doing some strange format with its logging and I need to extract only specific parts.
Basically the format of the log goes:
Date:<space>timestamp
User:<space>userid
AppID:<space>applicationid
Duration:<space>time(ms)
<line>
<line>
Date:<space>timestamp... (1 Reply)