Sponsored Content
Top Forums Shell Programming and Scripting Performance of log parsing shell script very slow Post 302305265 by quirkasaurus on Wednesday 8th of April 2009 11:20:51 AM
Old 04-08-2009
btw -- that'll work really only if each request has an end-time record immediately following it. if this is not the case, then let me know.

there's another trick we'll have to use to preprocess the log file.

specifically, creating a key using the svcname and "Service,Start" "Service,End"
and sorting ASC,DESC based on that key.

Then processing the file again using my above script.
 

9 More Discussions You Might Find Interesting

1. Post Here to Contact Site Administrators and Moderators

Help! Slow Performance

Is the performance now very, very slow (pages take a very long time to load)? Or is it just me? Neo (6 Replies)
Discussion started by: Neo
6 Replies

2. Shell Programming and Scripting

Shell script for parsing 300mb log file..

am relatively new to Shell scripting. I have written a script for parsing a big file. The logic is: Apart from lot of other useless stuffs, there are many occurances of <abc> and corresponding </abc> tags. (All of them are properly closed) My requirement is to find a particular tag (say... (3 Replies)
Discussion started by: gurpreet470
3 Replies

3. Shell Programming and Scripting

egrep is very slow : How to improve performance

We have an egrep search in a while loop. egrep -w "$key" ${PICKUP_DIR}/new_update >> ${PICKUP_DIR}/update_record_new ${PICKUP_DIR}/new_update is 210 MB file In each iteration, the egrep on an average takes around 50-60 seconds to search. Ther'es nothing significant in the loop other... (7 Replies)
Discussion started by: hidnana
7 Replies

4. Shell Programming and Scripting

Shell script to parsing log

Hi I Have log like this : 0 234: { 3 2: 04 EE 7 14: '20081114081' 23 1: 00 79 10: '38809' 91 15: '528111510010159' 143 29: 'Streaming/downloading service' 174 3: 'MTV' 179 43: 'rtsp://172.28/MTV2GO-Loop.sdp' 224 1: 05 ... (10 Replies)
Discussion started by: justbow
10 Replies

5. Filesystems, Disks and Memory

Slow Copy(CP) performance

Hi all We have got issues with copying a 2.6 GB file from one folder to another folder. Well, this is not the first issue we are having on the box currently, i will try to explain everything we have done from the past 2 days. We got a message 2 days back saying that our Production is 98%... (3 Replies)
Discussion started by: b_sri
3 Replies

6. UNIX for Dummies Questions & Answers

Slow copy/performance... between volumes

hi guys We are seeing weird issues on my Linux Suse 10, it has lotus 8.5 and 1 filesystem for OS and another for Lotus Database. the issue is when the Lotus service starts wait on top is very high about 25% percent and in general CPU usage is very high we found that when this happens if we... (0 Replies)
Discussion started by: kopper
0 Replies

7. Shell Programming and Scripting

Slow performance filtering file

Please, I need help tuning my script. It works but it's too slow. The code reads an acivity log file with 50.000 - 100.000 lines and filters error messages from it. The data in the actlog file look similar to this: 02/08/2011 00:25:01,ANR2034E QUERY MOUNT: No match found using this criteria.... (5 Replies)
Discussion started by: Miila
5 Replies

8. Infrastructure Monitoring

99% performance wa, slow server.

There is a big problem with the server (VPS based on OpenVZ, CentOS 5, 3GB RAM). The problem is the following. The first 15-20 minutes after starting the server is operating normally, the load average is less than or about 1.0, but then begins to increase sharply% wa, then hovers around 95-99%.... (2 Replies)
Discussion started by: draiphod
2 Replies

9. Solaris

Solaris 11.1 Slow Network Performance

I have identical M5000 machines that are needing to transfer very large amounts of data between them. These are fully loaded machines, and I've already checked IO, memory usage, etc... I get poor network performance even when the machines are idle or copying via loopback. The 10 GB NICs are... (7 Replies)
Discussion started by: christr
7 Replies
BB-REP.CGI(1)						      General Commands Manual						     BB-REP.CGI(1)

NAME
bb-rep.cgi - CGI front-end to bbgen reporting SYNOPSIS
bb-rep.cgi [--noclean] [bbgen-options] DESCRIPTION
bb-rep.cgi is invoked as a CGI script via the bb-rep.sh CGI wrapper. It triggers the generation of a Xymon availability report for the timeperiod specified by the CGI paramaters. bb-rep.cgi is passed a QUERY_STRING environment variable with the following parameters: start-mon (Start month of the report) start-day (Start day-of-month of the report) start-yr (Start year of the report) end-mon (End month of the report) end-day (End day-of-month of the report) end-yr (End year of the report) style (Report style) The following non-standard parameters are handled by the bbgen version of bb-rep.cgi: suburl (Page in report to go to, if not the top page) The "month" parameters must be specified as the three-letter english month name abbreviation: Jan, Feb, Mar ... Start- and end-days are in the range 1..31; the start- and end-year must be specified including century (e.g. "2003"). End-times beyond the current time are silently replaced with the current time. The generated report will include data for the start- and end-days, i.e. the report will begin at 00:00:00 of the start-day, and end at 23:59:59 of the end-day. The "style" parameter is passed directly to bbgen(1) and should be "crit", "non-crit" or "all". Other values result in undefined behaviour. All of the processing involved in generating the report is done by invoking bbgen(1) with the proper "--reportopts" option. OPTIONS
--noclean Do not clean the BBREP directory of old reports. Makes the report-tool go a bit faster - instead, you can clean up the BBREP direc- tory e.g. via a cron-job. --env=FILENAME Load the environment from FILENAME before executing the CGI. bbgen-options All other options passed to bb-rep.cgi are passed on to the bbgen(1) program building the report files. FILES
$BBHOME/web/report_header HTML template header for the report request form $BBHOME/web/report_footer HTML template footer for the report request form $BBHOME/web/report_form HTML template report request form ENVIRONMENT VARIABLES
BBGENREPOPTS bbgen options passed by default to the bb-rep.cgi. This happens in the bb-rep.sh wrapper. BBHOME Home directory of the Xymon server installation BBREP Directory where generated reports are stored. This directory must be writable by the userid executing the CGI script, typically "www", "apache" or "nobody". Default: $BBHOME/www/rep/ BBREPURL The URL prefix to use when accessing the reports via a browser. Default: $BBWEB/rep SEE ALSO
bbgen(1), bb-hosts(5), hobbitserver.cfg(5) Xymon Version 4.2.3: 4 Feb 2009 BB-REP.CGI(1)
All times are GMT -4. The time now is 01:59 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy