Here is a simple example of how to process a log file and save the last line for reference on the next run. It uses the entire last line from the log as the reference point, rather than a date/time stamp. Less efficient, but more accurate. I'm sure there are other ways to make this more efficient, but this might help get you started.
It does run in Kshell. Should run in bash, but for several reasons I prefer Kshell and didn't test it under bash, so beware if you want to use this as a bash script.
Code:
#!/usr/bin/env ksh
logf=$1 # log file to suss; supplied as parameter on cmd line
pat=$2 # pattern to search log for
this=${0##*/} # base name of script
lastf=/tmp/$this.data # file where we tuck our last found line away for next time
if [[ -z $logf || -z $pat ]] # error if logfile name or pattern isn't supplied
then
echo "usage: $0 log-file-name pattern"
exit 1
fi
if [[ ! -s $lastf ]] # no last run data; must check all lines from log
then
need_all=1 # this will cause us to check for pattern straight away
fi
# parse the log file writing, to stdout, any matches that haven't been seen yet
# The last line seen is written to the last data file for next time.
#
awk -v snarf=$need_all -v lastf=$lastf -v pat=$pat '
BEGIN {
if( !snarf ) # not in check all mode, must find last data
{
getline < lastf;
last_data = $0;
close( lastf );
}
}
snarf { # need all, or found the last line from prev run
if( match( $0, pat ) ) # if it contains the pattern, print it to stdout
print;
new_last = $0; # save the last line we saw
next; # go to next input line (skip remaining awk code)
}
{ # not snarfing yet, check to see if this is the last line we saw before
snarf = $0 == last_data; # start snarfing if it matches
# tricky way of saying:
# if( $0 == last_data)
# snarf = 1;
}
END {
if( new_last )
printf( "%s\n", new_last) >lastf; # save our last observed line in the data file.
}
' <$logf
exit $? # return the exit code that awk returned with
The one thing this script doesn't do is to deal with the case where the log file has been rolled off. In this case the data file would be non-empty, but we'd need to start checking for the pattern at the start of the log rather than after the previous marker is encountered.
The easy solution to this is to remove the last data file when the log is rolled. If that's not possible, then additional code will be needed to detect this condition; beyond my few minutes to spend answering questions tonight -- sorry.
Hi ,
I am trying a script which takes user input userid . I am stuck how to check whether that is a valid user id or not in the audit log files. My code is :
cd $CCP_AUDIT
cat * > /export/home/$USR/l***/files
echo "UserId:\c"
read UserId
#Date Function
echo "DATE : \c"
read xxx
I... (7 Replies)
I have a huge file and want to separate it into several subsets.
The file looks like:
C1 C2 C3 C4 ... (variable names)
1 ....
2 ....
3 ....
:
22 ....
23 ....
I want to separate the huge file using the column 1, which has numbers from 1 to 23 (but there are different amount of... (8 Replies)
I have a key file
$ cat klist
5 N:8855 CASA VERDE ROAD :32827 :ORLAND
5 N:585 MOLLY LANE :30189 :WOODST
5 N:320 NINA ROAD :32304 :TALLAH
and a data file, see example of the line below:
N:RT 15 & N 7TH STREET :17837 :U SAVE
I need to search by key (2nd field) from klist... (6 Replies)
Hi all,
I have the below script to get input but i cannot get grep to work.
input1.txt
AAAAAAAAG
input2.txt
>gi|184009.1| LEAFY-like |AAAAAAAAGSGGGDHLPY
However, when i use grep -f input1.txt input2.txt
i cannot get any output matches (note that the match is underlined).
Is it... (8 Replies)
So this is what I'm trying to do:
I have a file called registry.txt which has a list of registry entries I want to search for.
I have another file called inctrl.txt on which I want to perform the search on.
Here's the example contents of registry.txt
SOFTWARE\Microsoft\Security... (3 Replies)
Hello everybody,
I have been searching it, but it seems I am unable to find the correct information, that s why I am asking you guys, hoping somebody get an idea.
Here is my problem :
I want a script to loop until a string is identified in a log file.
Here is the script :
#!/bin/sh... (5 Replies)
Use and complete the template provided. The entire template must be completed. If you don't, your post may be deleted!
1. The problem statement, all variables and given/known data:
I need to search through the users home directories for keywords, display them. The code listed below will show... (7 Replies)
Hello -
I am running a script that is outputting to a log. Let call it output.log
I would like to monitor that log until the line "Build Successful" is found.
I think I would need to use the grep command.
How would I do that in a loop?
Thanks
Marty (1 Reply)
Hello,
I have built the following script to check if processes supplied by the argument are running or not.
#!/bin/bash
PROCLIST=$1
PROCESS="0"
ERROR_PROCS=""
IFS='+'
read -ra ADDR <<< "$PROCLIST"
for PROC in "${ADDR}"; do
if ; then
PROCESS=1
... (9 Replies)
Discussion started by: nms
9 Replies
LEARN ABOUT OSX
svk::command::log
SVK::Command::Log(3) User Contributed Perl Documentation SVK::Command::Log(3)NAME
SVK::Command::Log - Show log messages for revisions
SYNOPSIS
log DEPOTPATH
log PATH
log -r N[:M] [DEPOT]PATH
OPTIONS -r [--revision] ARG : ARG (some commands also take ARG1:ARG2 range)
A revision argument can be one of:
"HEAD" latest in repository
{DATE} revision at start of the date
NUMBER revision number
NUMBER@ interpret as remote revision number
NUM1:NUM2 revision range
Unlike other commands, negative NUMBER has no
meaning.
-l [--limit] REV : stop after displaying REV revisions
-q [--quiet] : Don't display the actual log message itself
-x [--cross] : track revisions copied from elsewhere
-v [--verbose] : print extra information
--xml : display the log messages in XML format
--filter FILTER : select revisions based on FILTER
--output FILTER : display logs using the given FILTER
DESCRIPTION
Display the log messages and other meta-data associated with revisions.
SVK provides a flexible system allowing log messages and other revision properties to be displayed and processed in many ways. This
flexibility comes through the use of "log filters." Log filters are of two types: selection and output. Selection filters determine which
revisions are included in the output, while output filters determine how the information about those revisions is displayed. Here's a
simple example. These two invocations produce equivalent output:
svk log -l 5 //local/project
svk log --filter "head 5" --output std //local/project
The "head" filter chooses only the first revisions that it encounters, in this case, the first 5 revisions. The "std" filter displays the
revisions using SVK's default output format.
Selection filters can be connected together into pipelines. For example, to see the first 3 revisions with log messages containing the
string 'needle', we might do this
svk log --filter "grep needle | head 3" //local/project
That example introduced the "grep" filter. The argument for the grep filter is a valid Perl pattern (with any '|' characters as '|' and
'' as '\'). A revision is allowed to continue to the next stage of the pipeline if the revision's log message matches the pattern. If
we wanted to search only the first 10 revisions for 'needle' we could use either of the following commands
svk log --filter "head 10 | grep needle" //local/project
svk log -l 10 --filter "grep needle" //local/project
You may change SVK's default output filter by setting the SVKLOGOUTPUT environment. See svk help environment for details.
Standard Filters
The following log filters are included with the standard SVK distribution:
Selection : grep, head, author
Output : std, xml
For detailed documentation about any of these filters, try "perldoc SVK::Log::Filter::Name" where "Name" is "Grep", "Head", "XML", etc..
Other log filters are available from CPAN <http://search.cpan.org> by searching for "SVK::Log::Filter". For details on writing log
filters, see the documentation for the SVK::Log::Filter module.
perl v5.10.0 2008-08-04 SVK::Command::Log(3)