Sponsored Content
Top Forums Shell Programming and Scripting Optimize shell script to run faster Post 302930675 by SkySmart on Tuesday 6th of January 2015 11:57:28 AM
Old 01-06-2015
Optimize shell script to run faster

data.file:

Code:
contact {
contact_name=royce-rolls
modified_attributes=0
modified_host_attributes=0
modified_service_attributes=0
host_notification_period=24x7
service_notification_period=24x7
last_host_notification=0
last_service_notification=0
host_notifications_enabled=1
service_notifications_enabled=1
}
servicecomment {
host_name=pgphhram01
service_description=Free Space All Disks
entry_type=4
comment_id=32
source=0
persistent=0
entry_time=1416610977
expires=0
expire_time=0
author=hpsm
comment_data=DM02782504
}
servicecomment {
host_name=pgphhram02
service_description=Free Space All Disks
entry_type=4
comment_id=32
source=0
persistent=0
entry_time=1420561982
expires=0
expire_time=0
author=hpsm
comment_data=DM02902504
}

My data.file is about 60MB big. So i need to trim it. To trim it, i need to identify which chunks are comment chunks. and when these chunks are identified, i need to check their entry time. If their entry time is older than 60 minutes from right now, i ignore that particular chunk and move on to the next chunk. in the above data.file, there are 3 chunks in it. chunks begin with a

Code:
"   {"

and ends with a

Code:
}

here is the code i'm using:

Code:
FILE=data.file

FFNUM=$(wc -l < ${FILE})

awk '{print NR","$0}' ${FILE} | egrep " {" | awk -F"," '{print $1}' | while read CLNUM
do
        NTIME=${CLNUM}

        LINENUMS=$(while [ $NTIME -le $FFNUM ]
        do
                ENDY=$(sed -n ${NTIME}p ${FILE} | egrep "^}")

                if [ ! -z "${ENDY}" ] ; then
                        echo "${CLNUM},${NTIME}"
                        break
                fi
                NTIME=$((${NTIME} + 1))
        done)

        FOUND=$(sed -n ${LINENUMS}p ${FILE})
        ISITCOMMENT=$(echo "${FOUND}"  | egrep "comment {")
        DNOW=$(date +%s)

        if [ ! -z "${ISITCOMMENT}" ] ; then

                ENTRYTIME=$(echo "${FOUND}" | egrep "entry_time" | awk -F"=" '{print $2}')

                ELAPSEDTIME=$(awk "BEGIN{print $DNOW - $ENTRYTIME}")

                if [ ${ELAPSEDTIME} -lt ${AMINUTES} ] ; then
                        echo "${FOUND}"
                fi
        else
                echo "${FOUND}"
        fi
done

This code works and does exactly what i need. however, it runs very slow. can anyone think of anyway i can augment this script so it runs faster?
 

8 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Can anyone make this script run faster?

One of our servers runs Solaris 8 and does not have "ls -lh" as a valid command. I wrote the following script to make the ls output easier to read and emulate "ls -lh" functionality. The script works, but it is slow when executed on a directory that contains a large number of files. Can anyone make... (10 Replies)
Discussion started by: shew01
10 Replies

2. Shell Programming and Scripting

Help need to make a shell script run for ffmpeg vhook watermaking in shell

i have a small problem getting a batxh shell script to run in shell this is the code the problem seems to be centered around the ffmpeg command, something maybe to do with the ' ' wrapping around the vhook part command this is a strange problem , if i take the ffmpeg command and... (1 Reply)
Discussion started by: wingchun22
1 Replies

3. UNIX for Advanced & Expert Users

Making things run faster

I am processing some terabytes of information on a computer having 8 processors (each with 4 cores) with a 16GB RAM and 5TB hard drive implemented as a RAID. The processing doesn't seem to be blazingly fast perhaps because of the IO limitation. I am basically running a perl script to read some... (13 Replies)
Discussion started by: Legend986
13 Replies

4. UNIX for Dummies Questions & Answers

optimize shell script (snapshots)

I've a script to do some snapshots but the time it does so is very different... once i got a snapshot under 1 sec, on the other hand it took 3 sec, but nothing else changed, i didnt even move the cursor or something. I put the script on a ramdisk and its faster, but still swing from under 1... (1 Reply)
Discussion started by: mcW
1 Replies

5. Shell Programming and Scripting

Optimize shell code

#!/usr/bin/perl use strict; use warnings; use Date::Manip; my $date_converted = UnixDate(ParseDate("3 days ago"),"%e/%h/%Y"); open FILE,">$ARGV"; while(<DATA>){ my @tab_delimited_array = split(/\t/,$_); $tab_delimited_array =~ s/^\ =~ s/^\-//; my $converted_date =... (2 Replies)
Discussion started by: sandy1028
2 Replies

6. Shell Programming and Scripting

Making script run faster

Can someone help me edit the below script to make it run faster? Shell: bash OS: Linux Red Hat The point of the script is to grab entire chunks of information that concerns the service "MEMORY_CHECK". For each chunk, the beginning starts with "service {", and ends with "}". I should... (15 Replies)
Discussion started by: SkySmart
15 Replies

7. UNIX for Dummies Questions & Answers

optimize if block : shell script

Hi, I need a shell script to determine if a no. is either even, greater than 4, less than 8 SHELL : ksh OS : RHEL 6 this is the if block of the script mod=`expr $num % 2` if || || then echo "No. is either even or greater than 4 or less than 8" fi this code works... (2 Replies)
Discussion started by: sam05121988
2 Replies

8. Shell Programming and Scripting

Shell script run in a case statement call to run a php file, also Perl

Linux System having all Perl, Python, PHP (and Ruby) installed From a Shell script, can call a Perl, Python, PHP (or Ruby ?) file eg eg a Shell script run in a case statement call to run a php file, also Perl or/and Python file??? Like #!/usr/bin/bash .... .... case $INPUT_STRING... (1 Reply)
Discussion started by: hoyanet
1 Replies
NPM-RUN-SCRIPT(1)                                                                                                                NPM-RUN-SCRIPT(1)

NAME
npm-run-script - Run arbitrary package scripts SYNOPSIS
npm run-script <command> [--silent] [-- <args>...] alias: npm run DESCRIPTION
This runs an arbitrary command from a package's "scripts" object. If no "command" is provided, it will list the available scripts. run[-script] is used by the test, start, restart, and stop commands, but can be called directly, as well. When the scripts in the package are printed out, they're separated into lifecycle (test, start, restart) and directly-run scripts. As of ` https://blog.npmjs.org/post/98131109725/npm-2-0-0, you can use custom arguments when executing scripts. The special option -- is used by getopt https://goo.gl/KxMmtG to delimit the end of the options. npm will pass all the arguments after the -- directly to your script: npm run test -- --grep="pattern" The arguments will only be passed to the script specified after npm run and not to any pre or post script. The env script is a special built-in command that can be used to list environment variables that will be available to the script at run- time. If an "env" command is defined in your package, it will take precedence over the built-in. In addition to the shell's pre-existing PATH, npm run adds node_modules/.bin to the PATH provided to scripts. Any binaries provided by locally-installed dependencies can be used without the node_modules/.bin prefix. For example, if there is a devDependency on tap in your package, you should write: "scripts": {"test": "tap test/*.js"} instead of "scripts": {"test": "node_modules/.bin/tap test/*.js"} to run your tests. The actual shell your script is run within is platform dependent. By default, on Unix-like systems it is the /bin/sh command, on Windows it is the cmd.exe. The actual shell referred to by /bin/sh also depends on the system. As of ` https://github.com/npm/npm/releases/tag/v5.1.0 you can customize the shell with the script-shell configuration. Scripts are run from the root of the module, regardless of what your current working directory is when you call npm run. If you want your script to use different behavior based on what subdirectory you're in, you can use the INIT_CWD environment variable, which holds the full path you were in when you ran npm run. npm run sets the NODE environment variable to the node executable with which npm is executed. Also, if the --scripts-prepend-node-path is passed, the directory within which node resides is added to the PATH. If --scripts-prepend-node-path=auto is passed (which has been the default in npm v3), this is only performed when that node executable is not found in the PATH. If you try to run a script without having a node_modules directory and it fails, you will be given a warning to run npm install, just in case you've forgotten. You can use the --silent flag to prevent showing npm ERR! output on error. You can use the --if-present flag to avoid exiting with a non-zero exit code when the script is undefined. This lets you run potentially undefined scripts without breaking the execution chain. SEE ALSO
o npm help 7 scripts o npm help test o npm help start o npm help restart o npm help stop o npm help 7 config January 2019 NPM-RUN-SCRIPT(1)
All times are GMT -4. The time now is 03:15 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy