Making script run faster


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Making script run faster
# 15  
Old 05-14-2012
Your problem is that your input file is read two often.
The solution is to parse it in one stroke.
Most speedy are awk then perl. Here is a pure shell script, just to demonstrate that the algorithm makes the difference:
Code:
#!/bin/sh
#set -x
ruler="==================================================="
nl="
"
inblock=0

while read line
do
  case $line in
  "service_description=MEMORY_CHECK")
    found1=1
    ;;
  *"CRITICAL:"*|*"UNKNOWN:"*)
    found2=1
    ;;
  'service{'*|'service {'*)
    inblock=1
    found1=0
    found2=0
    str=""
    ;;
  '}'*)
    inblock=0
    if [ $found1 -ne 0 -a $found2 -ne 0 ]
    then
      printf "%s\n%s\n\n%s\n" "$ruler" "$str" "$ruler"
    fi
    ;;
  host_name=*|check_command=*|plugin_output=*)
    str="${str}${nl}${line}"
    ;;
  esac 
done < status.log


Last edited by Franklin52; 05-14-2012 at 06:56 AM.. Reason: Replace QUOTE tags with CODE tags
# 16  
Old 05-14-2012
Quote:
Originally Posted by mirni
OK, well, that is progress.
Notice these two lines?
Code:
       
      split($N, A, "="); 
      D[A[1]]=A[2];

This splits the line (stored in $N) on '=' and stores it in array A.
The second line stores the second field only A[2].
Since nobody mentioned before that the lines could have multiple equal signs, it was assumed that lines were in the form
Code:
label=value

where the value would not contain the '=' (a very resonable assumption).

So, you need to append the rest of the fields, like e.g.:
Code:
awk 'BEGIN {
  while((getline < "server.list")>0)
     S[$0]

  FS="\n"; RS="}\n"
}

/service_description=MEMORY_CHECK/ {

  for(X in D) delete D[X];

  for(N=2; N<=NF; N++)
  {
       split($N, A, "=");
       D[A[1]] = A[2]
       i = 3;
       while (i in A) 
          D[A[1]] = D[A[1]] "=" A[i++];
  }

  if (D["host_name"] in S) 
       printf("%20s -- %50s\n", D["host_name"], D["plugin_output"])

}' $1

Now did you read the code and try to think how it works? The order is not random, at all. It processes one chunk at a time, and looks if the particular chunk's hostname is in the server list.
If you insist to process them in a fixed order, by-server, you would have to do what you did before, scan the log once for each server.
So I suggest you take the order that is given by the log file, and sort the results to get them in the order you want.

i've been playing with this code for a while but cant seem to get a handle on a seemingly minor issue.

the one thing i didn't notice about the status.log file is that there are several other chunks in it that do not have the beginning tags of "service {" and ending tags of "}"

some chunks start with "servicecomment {" and end with "}". others start with "contact {" and ends with "}".

however, the only chunks im interested in are the ones that start with "service {" and end with "}"

how can i specify the "service {" in your code?

your code works PERFECTLY. i just dont know how to modify it. i dont mean to keep pestering about this. i'm just so close to completing my task.

---------- Post updated at 09:35 PM ---------- Previous update was at 08:56 PM ----------

i got it. the fix was to add this to the script:

Code:
/service_description=MEMORY_CHECK/ && /service {/

thanks to every one!
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Script works, but I think it could be better and faster

Hi All, I'm new to the forum and to bash scripting. I did some stuff with VB.net, Batch, and VBScripting in the past, but because I shifted over to Linux, I am learning to script in Bash at this moment. So bear with me if I seem to script like a newbie, that's just because I am ;-) OK, I... (9 Replies)
Discussion started by: cornelvis
9 Replies

2. Shell Programming and Scripting

Optimize shell script to run faster

data.file: contact { contact_name=royce-rolls modified_attributes=0 modified_host_attributes=0 modified_service_attributes=0 host_notification_period=24x7 service_notification_period=24x7 last_host_notification=0 last_service_notification=0 host_notifications_enabled=1... (8 Replies)
Discussion started by: SkySmart
8 Replies

3. Shell Programming and Scripting

Making a faster alternative to a slow awk command

Hi, I have a large number of input files with two columns of numbers. For example: 83 1453 99 3255 99 8482 99 7372 83 175 I only wish to retain lines where the numbers fullfil two requirements. E.g: =83 1000<=<=2000 To do this I use the following... (10 Replies)
Discussion started by: s052866
10 Replies

4. Shell Programming and Scripting

Make script faster

Hi all, In bash scripting, I use to read files: cat $file | while read line; do ... doneHowever, it's a very slow way to read file line by line. E.g. In a file that has 3 columns, and less than 400 rows, like this: I run next script: cat $line | while read line; do ## Reads each... (10 Replies)
Discussion started by: AlbertGM
10 Replies

5. Shell Programming and Scripting

Script to parse a file faster

My example file is as given below: conn=1 uid=oracle conn=2 uid=db2 conn=3 uid=oracle conn=4 uid=hash conn=5 uid=skher conn=6 uid=oracle conn=7 uid=mpalkar conn=8 uid=anarke conn=1 op=-1 msgId=-1 - fd=104 slot=104 LDAPS connection from 10.10.5.6 to 10.18.6.5 conn=2 op=-1 msgId=-1 -... (7 Replies)
Discussion started by: sags007_99
7 Replies

6. UNIX for Advanced & Expert Users

Making things run faster

I am processing some terabytes of information on a computer having 8 processors (each with 4 cores) with a 16GB RAM and 5TB hard drive implemented as a RAID. The processing doesn't seem to be blazingly fast perhaps because of the IO limitation. I am basically running a perl script to read some... (13 Replies)
Discussion started by: Legend986
13 Replies

7. Shell Programming and Scripting

Can anyone make this script run faster?

One of our servers runs Solaris 8 and does not have "ls -lh" as a valid command. I wrote the following script to make the ls output easier to read and emulate "ls -lh" functionality. The script works, but it is slow when executed on a directory that contains a large number of files. Can anyone make... (10 Replies)
Discussion started by: shew01
10 Replies

8. UNIX for Advanced & Expert Users

Country Codes script faster response ;please help

Dear all I have group of input lines which look like this These input lines is placed in a file named phonelines.txt and there is a script which match $4 and $5 with country codes placed in another file named country-codes.txt and its contents is : Italy 39 Libyana 21892 Thuraya... (12 Replies)
Discussion started by: zanetti321
12 Replies

9. UNIX for Dummies Questions & Answers

making ssh run without password

Hello Everybody, Could anyone please tell me how to get ssh to work without asking for passwords? (i want to do a ssh <hostname> without getting a request for a password but getting connected straight away) I have attempted the following but to no avail :( ... I tried to generate a SSH... (5 Replies)
Discussion started by: rkap
5 Replies
Login or Register to Ask a Question