UNIX for Dummies Questions & Answers

Folks;
I'm writing a shell script to extract some fields out of a log file & it will run periodically, how can i make it runs starting from where it left of. for example;

if the script will do the extract every 2 days, let's say the first run will extract fields until July 25, 2007 @ 11:15:22 pm
How can i make the second run start from July 25, 2007 @ 11:15:23 PM?

Here's the script:

#!/bin/sh

echo " field1 field2 field3"
egrep 'Real value' new.log | \
while read mLine
do
mFld1=`echo $mLine | cut -d'|' -f1`
mFld2=`echo $mLine | cut -d'|' -f2`
mFld3=`echo $mLine | cut -d'|' -f3`


echo "$mFld1 $mFld2 $mFld3"
done

A thought,

You could create a new log file which has the data that was not extracted and pass the new log file to the next periodic run. Backout your original log file to someother name.

Hope this helps.

Or save the date and time of the last line processed in a separate file, and, on subsequent runs skip past all records older than that.
Or even simpler, keep track of the last record number processed, just remember that if you truncate the log file, you will have to reset the last record number to zero.

Thanks folks;
Any code details will be much appreciated

I am not sure of your requirements. but i guess this could help you.

to write to a file with a timestamp you could do it this way.

Code:

echo "Hello" > filename_$(date +%Y%m%d).log

the above code will write hello to a file with the filename "filename" followed by a date tag.

I'm still looking to a way (in the code) to make the newly extracted file in the second run to start from the point where the first run ended at instead of starting from the beginning again which will cause data to be repeated & the run will take longer every time