Performance of log parsing shell script very slow Post: 302305262

9 More Discussions You Might Find Interesting

1. Post Here to Contact Site Administrators and Moderators

Help! Slow Performance

Is the performance now very, very slow (pages take a very long time to load)? Or is it just me? Neo

2. Shell Programming and Scripting

Shell script for parsing 300mb log file..

am relatively new to Shell scripting. I have written a script for parsing a big file. The logic is: Apart from lot of other useless stuffs, there are many occurances of <abc> and corresponding </abc> tags. (All of them are properly closed) My requirement is to find a particular tag (say...

3. Shell Programming and Scripting

egrep is very slow : How to improve performance

We have an egrep search in a while loop. egrep -w "$key" ${PICKUP_DIR}/new_update >> ${PICKUP_DIR}/update_record_new ${PICKUP_DIR}/new_update is 210 MB file In each iteration, the egrep on an average takes around 50-60 seconds to search. Ther'es nothing significant in the loop other...

4. Shell Programming and Scripting

Shell script to parsing log

Hi I Have log like this : 0 234: { 3 2: 04 EE 7 14: '20081114081' 23 1: 00 79 10: '38809' 91 15: '528111510010159' 143 29: 'Streaming/downloading service' 174 3: 'MTV' 179 43: 'rtsp://172.28/MTV2GO-Loop.sdp' 224 1: 05 ...

5. Filesystems, Disks and Memory

Slow Copy(CP) performance

Hi all We have got issues with copying a 2.6 GB file from one folder to another folder. Well, this is not the first issue we are having on the box currently, i will try to explain everything we have done from the past 2 days. We got a message 2 days back saying that our Production is 98%...

6. UNIX for Dummies Questions & Answers

Slow copy/performance... between volumes

hi guys We are seeing weird issues on my Linux Suse 10, it has lotus 8.5 and 1 filesystem for OS and another for Lotus Database. the issue is when the Lotus service starts wait on top is very high about 25% percent and in general CPU usage is very high we found that when this happens if we...

7. Shell Programming and Scripting

Slow performance filtering file

Please, I need help tuning my script. It works but it's too slow. The code reads an acivity log file with 50.000 - 100.000 lines and filters error messages from it. The data in the actlog file look similar to this: 02/08/2011 00:25:01,ANR2034E QUERY MOUNT: No match found using this criteria....

8. Infrastructure Monitoring

99% performance wa, slow server.

There is a big problem with the server (VPS based on OpenVZ, CentOS 5, 3GB RAM). The problem is the following. The first 15-20 minutes after starting the server is operating normally, the load average is less than or about 1.0, but then begins to increase sharply% wa, then hovers around 95-99%....

9. Solaris

Solaris 11.1 Slow Network Performance

I have identical M5000 machines that are needing to transfer very large amounts of data between them. These are fully loaded machines, and I've already checked IO, memory usage, etc... I get poor network performance even when the machines are idle or copying via loopback. The 10 GB NICs are...

LEARN ABOUT DEBIAN

grep-excuses

GREP-EXCUSES(1) 					      General Commands Manual						   GREP-EXCUSES(1)

NAME

       grep-excuses - search the testing excuses files for a specific maintainer

SYNOPSIS

       grep-excuses [options] [maintainer|package]

DESCRIPTION

       grep-excuses  downloads	the  update_excuses.html file and greps it for the specified maintainer name.  The libwww-perl package is required
       for this script.  If no name is given on the command line, first the environment variable DEBFULLNAME is used if it is defined, and failing
       that, the configuration variable described below is used.

OPTIONS

       --no-conf, --noconf
	      Do not read any configuration files.  This can only be used as the first option given on the command-line.

       --wipnity, -w
	      Get information from <http://release.debian.org/migration/>.  A package name must be given when using this option.

       --help Show a brief usage message.

       --version
	      Show version and copyright information.

CONFIGURATION VARIABLES

       The  two configuration files /etc/devscripts.conf and ~/.devscripts are sourced in that order to set configuration variables.  Command line
       options can be used to override configuration file settings.  Environment variable settings are ignored for this  purpose.   The  currently
       recognised variable is:

       GREP_EXCUSES_MAINTAINER
	      The default maintainer, email or package to grep for if none is specified on the command line.

SEE ALSO

       devscripts.conf(5).

AUTHOR

       Joey Hess <joeyh@debian.org>; modifications by Julian Gilbey <jdg@debian.org>.

DEBIAN
								 Debian Utilities						   GREP-EXCUSES(1)