Sponsored Content
Full Discussion: Making script run faster
Top Forums Shell Programming and Scripting Making script run faster Post 302639387 by SkySmart on Friday 11th of May 2012 04:34:35 PM
Old 05-11-2012
Quote:
Originally Posted by Corona688
You're running grep, awk, and sed hundreds of thousands of times to process thousands of lines. I think your script may need a rewrite. You could probably do it all in one awk instance.

Can you show the input data you have, and the output you want?


here's the input data. Notice the below is what i refer to as a chunk. in this particular case, the chunk below belongs to the service called "MEMORY_CHECK".

so there's a file that contains several of these chunks for several services.

lets say the file is called status.log. for processing, I want to pull out all chunks called "service_description=MEMORY_CHECK" and then pull out other attributes of each of the chunk as needed.

attributes can be, i.e what is the name of the hostname that this MEMORY_CHECK is on? in this case, the host name would be "sky.log.net".

what is the plugin output, in this case, it would be "CRITICAL: Process was not found".

hope this helps.

Code:
service {
host_name=sky.log.net
service_description=MEMORY_CHECK
modified_attributes=1
check_command=blah!blah!blah
check_period=24x7
notification_period=24x7
event_handler=
has_been_checked=1
check_execution_time=0.453
check_latency=1.701
check_type=0
current_state=2
last_state=2
last_hard_state=2
last_event_id=2522339
current_event_id=2523376
current_problem_id=1127556
last_problem_id=0
current_attempt=1
max_attempts=1
normal_check_interval=30.000000
retry_check_interval=2.000000
state_type=1
last_state_change=1335466482
last_hard_state_change=1335466482
last_time_ok=1334123115
last_time_warning=0
last_time_unknown=1335465427
last_time_critical=1336750629
plugin_output=CRITICAL:  Process was not found
long_plugin_output=
performance_data=
last_check=1336750629
next_check=1336752429
check_options=0
notified_on_unknown=1
notified_on_warning=0
notified_on_critical=1
current_notification_number=364
current_notification_id=2530346
last_notification=0
notifications_enabled=0
active_checks_enabled=1
passive_checks_enabled=1
event_handler_enabled=1
problem_has_been_acknowledged=0
acknowledgement_type=0
flap_detection_enabled=0
failure_prediction_enabled=1
process_performance_data=1
obsess_over_service=1
is_flapping=0
percent_state_change=0.00
check_flapping_recovery_notification=0
state_history=2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2
}

 

9 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

making ssh run without password

Hello Everybody, Could anyone please tell me how to get ssh to work without asking for passwords? (i want to do a ssh <hostname> without getting a request for a password but getting connected straight away) I have attempted the following but to no avail :( ... I tried to generate a SSH... (5 Replies)
Discussion started by: rkap
5 Replies

2. UNIX for Advanced & Expert Users

Country Codes script faster response ;please help

Dear all I have group of input lines which look like this These input lines is placed in a file named phonelines.txt and there is a script which match $4 and $5 with country codes placed in another file named country-codes.txt and its contents is : Italy 39 Libyana 21892 Thuraya... (12 Replies)
Discussion started by: zanetti321
12 Replies

3. Shell Programming and Scripting

Can anyone make this script run faster?

One of our servers runs Solaris 8 and does not have "ls -lh" as a valid command. I wrote the following script to make the ls output easier to read and emulate "ls -lh" functionality. The script works, but it is slow when executed on a directory that contains a large number of files. Can anyone make... (10 Replies)
Discussion started by: shew01
10 Replies

4. UNIX for Advanced & Expert Users

Making things run faster

I am processing some terabytes of information on a computer having 8 processors (each with 4 cores) with a 16GB RAM and 5TB hard drive implemented as a RAID. The processing doesn't seem to be blazingly fast perhaps because of the IO limitation. I am basically running a perl script to read some... (13 Replies)
Discussion started by: Legend986
13 Replies

5. Shell Programming and Scripting

Script to parse a file faster

My example file is as given below: conn=1 uid=oracle conn=2 uid=db2 conn=3 uid=oracle conn=4 uid=hash conn=5 uid=skher conn=6 uid=oracle conn=7 uid=mpalkar conn=8 uid=anarke conn=1 op=-1 msgId=-1 - fd=104 slot=104 LDAPS connection from 10.10.5.6 to 10.18.6.5 conn=2 op=-1 msgId=-1 -... (7 Replies)
Discussion started by: sags007_99
7 Replies

6. Shell Programming and Scripting

Make script faster

Hi all, In bash scripting, I use to read files: cat $file | while read line; do ... doneHowever, it's a very slow way to read file line by line. E.g. In a file that has 3 columns, and less than 400 rows, like this: I run next script: cat $line | while read line; do ## Reads each... (10 Replies)
Discussion started by: AlbertGM
10 Replies

7. Shell Programming and Scripting

Making a faster alternative to a slow awk command

Hi, I have a large number of input files with two columns of numbers. For example: 83 1453 99 3255 99 8482 99 7372 83 175 I only wish to retain lines where the numbers fullfil two requirements. E.g: =83 1000<=<=2000 To do this I use the following... (10 Replies)
Discussion started by: s052866
10 Replies

8. Shell Programming and Scripting

Optimize shell script to run faster

data.file: contact { contact_name=royce-rolls modified_attributes=0 modified_host_attributes=0 modified_service_attributes=0 host_notification_period=24x7 service_notification_period=24x7 last_host_notification=0 last_service_notification=0 host_notifications_enabled=1... (8 Replies)
Discussion started by: SkySmart
8 Replies

9. Shell Programming and Scripting

Script works, but I think it could be better and faster

Hi All, I'm new to the forum and to bash scripting. I did some stuff with VB.net, Batch, and VBScripting in the past, but because I shifted over to Linux, I am learning to script in Bash at this moment. So bear with me if I seem to script like a newbie, that's just because I am ;-) OK, I... (9 Replies)
Discussion started by: cornelvis
9 Replies
pack_fopen_chunk(3alleg4)					  Allegro manual					 pack_fopen_chunk(3alleg4)

NAME
pack_fopen_chunk - Opens a sub-chunk of a file. Allegro game programming library. SYNOPSIS
#include <allegro.h> PACKFILE *pack_fopen_chunk(PACKFILE *f, int pack); DESCRIPTION
Opens a sub-chunk of a file. Chunks are primarily intended for use by the datafile code, but they may also be useful for your own file rou- tines. A chunk provides a logical view of part of a file, which can be compressed as an individual entity and will automatically insert and check length counts to prevent reading past the end of the chunk. The PACKFILE parameter is a previously opened file, and `pack' is a bool- ean parameter which will turn compression on for the sub-chunk if it is non-zero. Example: PACKFILE *output = pack_fopen("out.raw", "w!"); ... /* Create a sub-chunk with compression. */ output = pack_fopen_chunk(output, 1); if (!output) abort_on_error("Error saving data!"); /* Write some data to the sub-chunk. */ ... /* Close the sub-chunk, recovering parent file. */ output = pack_fclose_chunk(output); The data written to the chunk will be prefixed with two length counts (32-bit, a.k.a. big-endian). For uncompressed chunks these will both be set to the size of the data in the chunk. For compressed chunks (created by setting the `pack' flag), the first length will be the raw size of the chunk, and the second will be the negative size of the uncompressed data. To read the chunk, use the following code: PACKFILE *input = pack_fopen("out.raw", "rp"); ... input = pack_fopen_chunk(input, 1); /* Read data from the sub-chunk and close it. */ ... input = pack_fclose_chunk(input); This sequence will read the length counts created when the chunk was written, and automatically decompress the contents of the chunk if it was compressed. The length will also be used to prevent reading past the end of the chunk (Allegro will return EOF if you attempt this), and to automatically skip past any unread chunk data when you call pack_fclose_chunk(). Chunks can be nested inside each other by making repeated calls to pack_fopen_chunk(). When writing a file, the compression status is inherited from the parent file, so you only need to set the pack flag if the parent is not compressed but you want to pack the chunk data. If the parent file is already open in packed mode, setting the pack flag will result in data being compressed twice: once as it is written to the chunk, and again as the chunk passes it on to the parent file. RETURN VALUE
Returns a pointer to the sub-chunked PACKFILE, or NULL if there was some error (eg. you are using a custom PACKFILE vtable). SEE ALSO
pack_fclose_chunk(3alleg4), pack_fopen(3alleg4) Allegro version 4.4.2 pack_fopen_chunk(3alleg4)
All times are GMT -4. The time now is 02:26 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy