performance of shell script ( grep command) Post: 302382275

Sponsored Content

Top Forums Shell Programming and Scripting performance of shell script ( grep command) Post 302382275 by vikash_k on Wednesday 23rd of December 2009 12:01:08 AM

12-23-2009

Registered User

performance of shell script ( grep command)

Hi,

I have to find out the run time for 40-45 different componets. These components writes in to a genreric log file in a single directory.

eg.

directory is LOG and the log file name format is generic_log_<process_id>_<date YY_MM_DD_HH_MM_SS>.log

i am taking the run time using the time mentioned in the log file name which is actuall the log file creation time (component start time) and component end time with the last modification time of the log file.

I have tested this and it is working fine for less no of file, but when i am executing it for a month the script is taking hell lot of time.

i am searching the following patterns to check run time for a partiular component

Code:

grep -wl "SOURCE=$COMPONENT" $(grep -wl "RUN_DATE=$SEARCH_DATE" $LOG/generic_log*

but as we have 40 components which runs on daily basis, so for 30 days we'll have around 1200 log files to search for 2 patterns, thats why the script is taking lot of time.

Can any one please advice me batter way to do this.

Last edited by zaxxon; 12-23-2009 at 05:58 AM.. Reason: use code tags please,

vikash_k

View Public Profile for vikash_k

Find all posts by vikash_k

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Performance of a shell script

Hiii, I wrote a shell script for testing purpose. I have to test around 200thousand entries with the script.When i am doing only for 6000 entries its taking almost 1hour.If i test the whole testingdata it will take huge amount of time. I just want to know is it something dependent on the...

2. UNIX for Dummies Questions & Answers

how to get the output of a grep command to a file inside a shell script

hi, i wat to get the output of a grep command in a file. but when i am trying out the same grep command in the unix prompt its working fine.. i am getting the output properly.. but when i am writing the same command inside my shell script , its just creating a new output file with no contents...

3. Shell Programming and Scripting

how to redirect the output of a grep command to a file inside a shell script

4. Shell Programming and Scripting

Passing parameters to Shell script for GREP command

I am using grep to capture date from a file . Since i need to use the shell script for different dates ,is it possible to pass the date parameter to the shell script the Script is as below grep -E "08 Aug 2008|2008-08-08"* somefile.txt>test.txt The above script file greps the...

5. Shell Programming and Scripting

Grep command in shell script

Hi, I have written the following shell script - Error_String="error" var1="| grep -v note1 | grep -v note2" grep -i $Error_String /users/mqm/Pwork/Err/*.out $var1 The above script gives error saying "grep: can't open | grep: can't open grep grep: can't open -v" etc In my program...

6. Shell Programming and Scripting

can anyone help with shell script command about searching word with grep command?

i want to search in the current directory all the files that contain one word for example "hello" i want to achieve it with the grep command but not with the grep *

7. Shell Programming and Scripting

How to use grep command in shell script?

Hi, I am new to shell scripting. I had written a small code which accepts config file as parameter. In config file I mentioned a path for input files, INPUT_FILE_FOLDER=/project/dev/con/InputFile SEARCH_MASK_INPUT_FILE=ABC_CR_PQR*.* I want to use this path to get all file names from...

8. HP-UX

Performance issue with 'grep' command for huge file size

I have 2 files; one file (say, details.txt) contains the details of employees and another file (say, emp.txt) has some selected employee names. I am extracting employee details from details.txt by using emp.txt and the corresponding code is: while read line do emp_name=`echo $line` grep -e...

9. Shell Programming and Scripting

Shell Script to grep output from top command.

Hello, I want a script which would grep details from top command for specific processes. I m not sure of the PID of these processes but i know the names. $ top -c top - 16:41:55 up 160 days, 5:53, 2 users, load average: 9.36, 9.18, 8.98 Tasks: 288 total, 9 running, 279 sleeping, 0...

10. Shell Programming and Scripting

Performance issue in shell script

Hi All, I am facing performance issue while rinning the LINUX shell script. I have file1 and file 2. File one is the source file and file 2 is lookup file. Need to replace if the pattern is matching in file1 with file2. The order of lookup file is important as if any match then exit...

LEARN ABOUT OSF1

lsmsad

lsmsad(8)						      System Manager's Manual							 lsmsad(8)

NAME

       lsmsad - Starts the Storage Administrator (SA) daemon

SYNOPSIS

       /usr/sbin/lsmsad

DESCRIPTION

       The SA daemon, lsmsad, is a process required by lsmsa, the SA graphical user interface (GUI). The daemon issues commands and obtains system
       information on behalf of SA.

       The SA daemon runs on a Tru64 UNIX system on which LSM is initialized and running.  The SA client runs on any  machine  that  supports  the
       Java Runtime Environment.

       The SA daemon is automatically started at boot time. Under normal conditions, the daemon does not need to be run manually.

       If SA does not start during the boot process, enter:

       /sbin/init.d/lsmsa stop

       To restart SA, enter:

       /sbin/init.d/lsmsa start

       Only one SA daemon can be running on a system at a given time. If a second SA daemon attempts to start, it will fail.

RESTRICTIONS

       You must be root user to run lsmsad.

FILES

       The script that starts lsmsad at boot time.  The command log file that tracks SA tasks.	The access log file that tracks login to  SA.  The
       server log file that tracks server startup information and server errors.  The log maintenance shell script that saves and  compresses  log
       files.

SEE ALSO

       lsmsa(8), volintro(8)

																	 lsmsad(8)