Sponsored Content
Top Forums Shell Programming and Scripting Performance issue while using find command Post 302722405 by nanthagopal on Saturday 27th of October 2012 01:56:21 AM
Old 10-27-2012
Quote:
Originally Posted by jim mcnamara
What you posted has some problems. Since the script works correctly but slowly could you please post exactly what code you have.

Also, are you repeating the grep over the same files 3-4 times?

Code:
find /path/to/files -name '*.log'  -exec grep 'pattern'

is another way to grep files. And your xargs approach is another way as well.


Hi,

I have 4 Sub-Process. for each process i want to grep some string and finally print each sub-process results.

I have used find | xargs grep comand for grepping, the code as follows

Code:
  find . -type f ! -name "*.bz2" ! -name "*~" -name "catalina.*" | xargs grep  xxx | grep "xx\|xx\|xx\|xx" | grep "status"

It takes 7-8 minutes for grepping the entire records in the path.


Note:

For each sub-process, i have to grep the full records. so, approximately 7*4 = 28 mins to finish the entire process.

So, how to improve the performance.

Please Suggest me.

Regards,
Nanthagopal A
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

An issue with find command.

Hi all, I have a shell script(ksh) which has the code as follows. ------------------ cd $mydir for i in `find ./ -type f -mtime +$k` do echo $i done ----------------------- And in $mydir , i have some files which have space in theie names like "Case att15". The out put of the... (6 Replies)
Discussion started by: rajugp1
6 Replies

2. UNIX for Dummies Questions & Answers

Find command issue

I am currently using below command to get the 1st three characters of a file(PDM). Issue is, when i use find command in root dir, it finds all the files in sub dir also. How to limit the find command search to a given path only(ie: say only find file in apps/cmplus/datamigration/data path... (3 Replies)
Discussion started by: abhi_n123
3 Replies

3. Shell Programming and Scripting

Issue with Find Command

Hi All, I'm a bit new to Linux environment, moderately okay when it comes to Unix AIX. I'm facing an issue while trying to run a simple find command: $ for file in `find . -name *.*` > do > ls $file > done This is throwing the following error: Strangely, a few minutes... (4 Replies)
Discussion started by: adi_2_chaos
4 Replies

4. Linux

find command issue

Hi, I am not root user. I am trying to find the file which has contains pattern "fvsfile" in root directory. If i run the find cmd then i got permission denied and all the files are listed include pattern files. i cant get file name yet find . print | xargs grep -i "fvsfile" I want... (2 Replies)
Discussion started by: Mani_apr08
2 Replies

5. HP-UX

Performance issue with 'grep' command for huge file size

I have 2 files; one file (say, details.txt) contains the details of employees and another file (say, emp.txt) has some selected employee names. I am extracting employee details from details.txt by using emp.txt and the corresponding code is: while read line do emp_name=`echo $line` grep -e... (7 Replies)
Discussion started by: arb_1984
7 Replies

6. Shell Programming and Scripting

Find command issue

Guys, Here is my requirement.. Sample.cfg file="*log.gz *txt.gz" sample.sh #!/bin/sh . $HOME/Sample.cfg find . -name "$file" -mtime +20 -exec ls -la {} \; Its not finding the given *log.gz and txt.gz files. Could anyone please help me? (8 Replies)
Discussion started by: AraR87
8 Replies

7. Shell Programming and Scripting

Issue in Find and mv command

Hi I am using the below code to find mv the files. Files are moving to the Target location as expected but find is displaying some errors like below. find ./ -name "Archive*" -mtime +300 -exec mv {} /mnt/X/ARC/ \; find: `./Archive_09-30-12': No such file or directory find:... (6 Replies)
Discussion started by: rakeshkumar
6 Replies

8. Shell Programming and Scripting

Find command issue

Hi Guys, I have a file called error.logs. am just trying to display the content in the file which was modified last 1 day. I tried below command but it doesnt give the proper output. find /u/text/vinoth/bin "error.logs" -mtime -1 -exec cat {} \; >> mail.txt Any help is much... (21 Replies)
Discussion started by: Vinoth Kumar G
21 Replies

9. Shell Programming and Scripting

Performance Issue for a file search command

Hi All, This query is regarding performance improvement of a command. I have a list of IDs in a file (say file1 with single ID column) and file2 has the data rows. I need to get the IDs from file1 and search in file2, matching rows from file2 should be written to a file3. For this... (4 Replies)
Discussion started by: Tanu
4 Replies

10. Shell Programming and Scripting

Increase the performance of find command.

I'm trying to exclude 'BACKUP', 'STORE', 'LOGGER' folders while searching for all files under a directory "/tmp/moht" Once a file is found I wish to display the filename, the size of the file & the cksum value. Below is the command, I'm using: /opt/freeware/bin/find /tmp/moht -type d -name... (6 Replies)
Discussion started by: mohtashims
6 Replies
ZGREP(1)						      General Commands Manual							  ZGREP(1)

NAME
zgrep - search possibly compressed files for a regular expression SYNOPSIS
zgrep [ grep_options ] [ -e ] pattern filename... DESCRIPTION
Zgrep is used to invoke the grep on compress'ed or gzip'ed files. All options specified are passed directly to grep. If no file is speci- fied, then the standard input is decompressed if necessary and fed to grep. Otherwise the given files are uncompressed if necessary and fed to grep. If zgrep is invoked as zegrep or zfgrep then egrep or fgrep is used instead of grep. If the GREP environment variable is set, zgrep uses it as the grep program to be invoked. For example: for sh: GREP=fgrep zgrep string files for csh: (setenv GREP fgrep; zgrep string files) AUTHOR
Charles Levert (charles@comm.polymtl.ca) SEE ALSO
grep(1), egrep(1), fgrep(1), zdiff(1), zmore(1), znew(1), zforce(1), gzip(1), gzexe(1) ZGREP(1)
All times are GMT -4. The time now is 06:09 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy