Thank you. I will try the solution once im on the production server.
What im planning is to make this as a shell script with input arguments so its easy for me to get the data that I need just like:
"parser.sh serverIP timestamp"
also im planning to make an expect script for the automation. i will look for such example script. maybe you can refer me to a link where i can find or learn.
thanks.
Quote:
Originally Posted by yazu
Oh, yes. But because your file is really huge it's better to embed this check in awk:
You can't use grep, it will search the whole file, but you need quit after getting your lines (just imagine that your information in the first 100 kilo). But I'm afraid the above awk solution would be slow because of the string regex. But you can embed the time variable (you need a variable for easy further automation) in the awk regex literal:
But I'm afraid the above awk solution would be slow because of the string regex. But you can embed the time variable (you need a variable for easy further automation) in the awk regex literal:
Code:
$0 ~ /^'"$time"'/
Upon reading that, I was skeptical. I expected that there would be an improvement, but I didn't think it would be a large difference. Wow, was I mistaken.
Test file generation:
Code:
jot -b 'foo bar baz
bar baz foo' 5000000 > data
Trying to match lines that begin with "foo", that yields a 10 million line file whose lines alternately match and don't match.
The regular expression literal (4.738) is 38% faster than the dynamic regular expression (7.680).
I measured similar results (41% improvement) with an ancient version of mawk on a 12 yr old laptop (which still has a sticker proudly announcing "Designed for Microsoft Windows 95").
"Grepping" in awk for time is just optimization. If it doesn't work - remove it. And if you have other filed separator, "main" and "trigger" regexes, just change them:
Code:
fs=':\\| +'; main='RBL: '; trig='BIOS-INFO'
awk -F"$fs" -v main="$main" -v trig="$trig" '
$2 ~ main {
first = $0; getline; second = $0; getline; third = $0;
}
first && $2 ~ trig {
printf "%s\n%s\n%s\n", first, second, third
exit
}' INPUTFILE
===
Remember about optimization - embedding shell variables and changing dynamical regexes to static ones. But they say it's unsafe and if you are going to write a script that will get these parameters from the command line - it's very unsafe. It's possible to embed in a such parameter any shell command and execute it (though you can validate them).
Well, for this kind of problems is very hard to give you an exact solution. Something changes and you need other solution. The structure of data you give is defined a little wrong and a solution can be wrong or it can take hours instead of minutes. If you are in this buisness (text/file processing) you really should take some time and learn and understand the basics at least.
Ok, there is I was wrong and you need $7 not $6 (you have the additonal first empty field).
And about -F option - when you assign it to one symbol, you get just symbol as the field separator. When you assign it to a string, you get a regex. '|' divides your strings to 7 fields, and ':\\|' divides to two.
Hi
I have a system running uname -a
Linux cmovel-db01 2.6.32-38-server #83-Ubuntu SMP Wed Jan 4 11:26:59 UTC 2012 x86_64 GNU/Linux
I would like to capture the contents of /var/log/syslog from 11:00AM to 11:30AM and sent to this info via email.
I was thinking in set a cron entry at that... (2 Replies)
unix : sun
shell : bash
i need to select multiple rows with this format :
<special format>
10 lines
/<special format>
from log file that have lots of info
i thought of getting the number of the first line using
grep -n "special format" file | cut -d: -f1
then pass it to shell... (2 Replies)
I have a LOG file which looks like this
Import started at: Mon Jul 23 02:13:01 EDT 2012
Initialization completed in 2.146 seconds.
--------------------------------------------------------------------------------
--
Import summary for Import item: PolicyInformation... (8 Replies)
My intention is to log the output to a file as well as it should be displayed on the console > I have used tee ( tee -a ${filename} ) command for this purpose. This is working as expected for first few outputs, after some event loggin nothing is gettting logged in to the file but It is displaying... (3 Replies)
Hi Experts,
I had to edit (a particular value) in header line of a very huge file so for that i wanted to search & replace a particular value on a file which was of 24 GB in Size. I managed to do it but it took long time to complete. Can anyone please tell me how can we do it in a optimised... (7 Replies)
Hi,
I have a log file that contains information such as this:
date
id number
command1
command2
command3
command4
data
data
data
date
id number
command1
command2
command3
command4 (4 Replies)
Looking for a shell script or a simple perl script . I am new to scripting and not very good at it .
I have 2 directories . One of them holds a text file with list of files in it and the second one is a daily log which shows the file completion time. I need to co-relate both and make a report.
... (0 Replies)
I have a file with data extracted, and need to insert a header with a constant string, say: H|PayerDataExtract
if i use sed, i have to redirect the output to a seperate file like
sed ' sed commands' ExtractDataFile.dat > ExtractDataFileWithHeader.dat
the same is true for awk
and... (10 Replies)
I've got a 2.2 Gig syslog file from our Cisco firewall appliance. The problem is that we've been seeing gaps in the syslog for anywhere from 10 minutes to 2 hours. Currently I've just been using 'less' and paging through the file to see if I can find any noticeable gaps. Obviously this isn't the... (3 Replies)
