Location: Mableton, GA USA (western suburb of Atlanta)
Posts: 1
Thanks Given: 0
Thanked 0 Times in 0 Posts
Perl's buffered I/O is causing me to miss latest log file entries in log colorizer. How to fix?
I've been finding myself using a log file colorizer written in perl to reformat and colorize the output from many different programs. Mainly, however, I use it to make the output from "tail -f" commands more readable.
The base perl script I use is based on "colorlogs.pl" available from the resentment dot org web site (I would post the URL, but I am but an egg in these forums and hence not worthy ... I can mail or PM it to anyone who is curious). It's a nice little script, and it works well for busy log files. I've extended it a little, but most of it is as originally written.
Unfortunately, on our test system, the logs I'm viewing aren't very busy, and it could be several minutes .. or hours ... between changes in the file.
Because of the very low level of actitivity, I've discovered that the last several lines aren't always displayed (it wasn't obvious on a system with messages coming in every few seconds), and the culprit appears to be the fact that the perl open and print commands perform buffered I/O. This is surely good most of the time, but not in this instance.
Example calling syntax:
The code is fairly simple ... it has a list of colornames associated with various ANSI control sequences, it reads a config file which associates those colornames with various arbitrary strings to match, and then it reads STDIN to obtain the output text to colorize.
I've done some searching, read the "Suffering from Buffering" web site, and tried various things like sticking:
or
in the while loop to make STDOUT a "hot" handle, but that doesn't seem to make any difference. The issue might be STDIN. I dunno.
I also know that sysread and syswrite are unbuffered statements I could use, but I'm not sure how to adapt the existing code to do that. I'm still a relative Perl newbie ... I can tweak existing scripts to bend them to my will, but I'm still in the process of learning basic concepts.
So... How would you folks fix this? Am I on the right track with setting $| to 1, or would moving to sysread/syswrite be a better idea?
Or rewrite it in C?
Any and all comments welcome. Thanks in advance!
Last edited by rcsteiner; 05-19-2011 at 07:32 PM..
Reason: s/si/is
Hi,
I want to monitor a log file using tail -f command and search for a specific string on the most recent entry from the file. If the search string matches with the most recent or last line from the file, I want send an email to the people with the message.
tail -f service.log|tail -n 1
... (5 Replies)
Hi there
I am trying to write a script where I will need to look for a specific word in the log file and I am aware this can be done by grep for example.
As there will be multiple entries for this I want to grep the last one to enter the log... how would I go about this - would I have to use... (5 Replies)
Hello All,
I have an awk script which parses my log file and prints number grepping from a specific line/pattern, now i have to come with a shell script to continue reading the log untill the job is completed, which i would know while reading session log untill process encounters a final... (1 Reply)
Hi All,
Need a small help. I have a log file which keeps updating for every Minute with multiple number of lines. I just want to grep few properties which has latest Date and Time to it. How do i do it?
I wanted to grep a property by name "Reloading cache with a maximum of" from the... (4 Replies)
How can i print the output of a perl script on a unix console and redirect the same in a log file under same directory simultaneously ?
Like in Shell script, we use tee, is there anything in Perl or any other option ? (2 Replies)
in the below .. i want to pick the latest logfile which is having JPS.PR inside..
that means i want particularly "spgport040408041223.log:@@@@@@@@ 04:13:09 Adding: JPS.PR."
which is latest among these..
is it possible to compare the current time with logfile time ?
reptm@xblr0758rop>... (4 Replies)
Have checked the forums and couldnt locate help on this.
I want to grep a log file for a pattern using a script - I need to grep the latest log file and not sure how I am able to ensure I am greping the latest log file.
Here is sample of log files for yestersday and I effectively need to grep... (10 Replies)
everytime a new logfile get created at certain interval of time and i want a
simple shell script program which cat the lastest log file when manually excuted (1 Reply)
Hi
I have got a log file which is like:
id status
---------------------
12345 status1.true
12345 status2.true
12345 status3.false
12345 status4.true
23452 status5.true
23452 status6.true
23452 status7.false
65243 ... (2 Replies)