xargs egrep


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting xargs egrep
# 1  
Old 07-23-2009
Network xargs egrep

Below is a sample script

Code:
jb>cat search
#!/bin/bash
set -x
PATTERN='"'`awk -F: '{printf $2 "|"} END {printf "\b"}' RULE`'"'
echo 'Pattern ['$PATTERN']'
ls -tr *$1* |xargs egrep "$PATTERN"

Sample RULE file
Code:
jb>cat RULE
P1 :pone
P2 :ptwo

Sample Output
Code:
jb>./search f
++ awk -F: '{printf $2 "|"} END {printf "\b"}' RULE
+ PATTERN='"pone|ptwo"'
+ echo 'Pattern ["pone|ptwo"]'
Pattern ["pone|ptwo"]
+ ls -tr f1 f1.txt f2 f3 f4
+ xargs egrep '"pone|ptwo"'
f2:ptwo
jb>

Directly searching

Code:
jb>ls -tr *f* |xargs egrep "pone|ptwo"
f1:000pone000
f2:ptwo

Sample file content
Code:
jb>cat f1
11
000pone000
1111
jb>cat f2
222
ptwo
22222222
jb>





Why 'search' doesn't list f1?
# 2  
Old 07-23-2009
Try the pattern without the double quotes, replace this line:

Code:
PATTERN='"'`awk -F: '{printf $2 "|"} END {printf "\b"}' RULE`'"'

with:

Code:
PATTERN=`awk -F: '{printf $2 "|"} END {printf "\b"}' RULE`

# 3  
Old 07-23-2009
Thanks,it works.
I have one more question.

I have some 20 patterns OR'ed 'pat1|pat2|.....pat20'
Each pattern of length around 20 chars
and i search around 30 files each of size 10 MB.
It takes 2 mins to complete !
Is there any optimal/fast way to do this ? Smilie
# 4  
Old 07-23-2009
You can try something like:

Code:
awk -F: '{print $2}' RULE > tmp.file

grep -f tmp.file $(ls -tr *$1*)

# 5  
Old 07-23-2009
1.Actually the RULE file names pattern,(RULE file as such is not a pattern) example
Code:
>cat RULE
SUCCESS       :Message sent successfully to
FAILURE         :Message send failed for
Acknowledged :Got ack from  
#Goes on

2.This line extract the second column ,forms a (ORed) pattern

Code:
PATTERN=`awk -F: '{printf $2 "|"} END {printf "\b"}' RULE`

#PATTERN will be  'Message sent successfully to|Message send failed for|Got ack from'

3.Search for all pattern in all file and redirect the o/p to tmp.txt (Takes 2 mins )
Code:
ls -tr *$1* |xargs egrep "$PATTERN" > tmp.txt

4.from tmp.txt do some more filtering,collect statistics (count each pattern)
(Dont have problem with this step ,takes very less time.)

Code:
Sample Output :
SUCCESS :1423
FAILURE : 432
Acknowledged : 764


But step 3 alone takes much of the time(around 2 mins)
# 6  
Old 07-23-2009
Have you tried my last 2 commands for steps 2 and 3?

Code:
awk -F: '{print $2}' RULE > tmp.file

grep -f tmp.file $(ls -tr *$1*) > tmp.txt

# 7  
Old 07-23-2009
Once again thanks,It works
Search completed in less than 30 secs.

My grep version doesn't support -f ,fgrep does.
Code:
awk -F: '{print $2}' RULE > tmp.txt
fgrep -f tmp.txt $(ls -tr *26_06_2009*) > tmp2.txt

(Sorry I thought you didn't get my question,but actually I misunderstood your solution )
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Help with xargs

Using the bash shell I'm trying to either create a command for the command line or a script that will show netstat info for a given process name. Here is an example of what I'm trying to do:$ ps aux |grep catalina |grep -v grep | awk '{print $2}' 5132 $ netstat -nlp |grep 5132 (Not all processes... (11 Replies)
Discussion started by: axiopisty
11 Replies

2. Shell Programming and Scripting

xargs

I have dir with many files ( close to 4M ) . $ ls -la total 76392 drwxr-xr-x 10 oracle dba 512 Jun 06 14:39 . drwxr-xr-x 11 oracle dba 512 Dec 20 13:21 .. drwxr-xr-x 2 oracle dba 39074816 Jun 15 14:07 ad I am trying to delete them using... (8 Replies)
Discussion started by: talashil
8 Replies

3. Shell Programming and Scripting

xargs

Dear all , any suggest on xargs to combine from (1.txt and 2.txt) to output.txt ? thanks a lot. 1.txt 0123 BUM-5M BUM-5M 93490481 63839 0124 BUM-5M BUM-5M 112112 ... (3 Replies)
Discussion started by: samoptimus
3 Replies

4. Shell Programming and Scripting

Xargs and

Hello there, Let me show you a simple example of what I am trying to achieve: 1) I have an input text file with some lines: 1 a 2 b 3 c 2) And I want to run a command with these lines as arguments (+ arbitrary extra arguments). For example: $ command "1 a" "2 b" "3 c" "bye" I... (7 Replies)
Discussion started by: tokland
7 Replies

5. Shell Programming and Scripting

Help in using xargs

Hi, I have a requirement to RCP the files from remote server to local server. Also the RCP has to run in parallel. However using 'xargs' retrives 2 file names during each loop. How do we restrict to only one file name using xargs and loop till remaining files. I use the below code for... (2 Replies)
Discussion started by: senthil3d
2 Replies

6. Shell Programming and Scripting

Using xargs

hi i just want to know that how do we use xargs command to find files which are greater than specified memory in a given directory (6 Replies)
Discussion started by: sumit the cool
6 Replies

7. UNIX for Advanced & Expert Users

xargs -P

I discovered that GNU's xargs has a -P option to allow its processes to run in parallel. Great! Is this a GNU thing, or is it supported by other platforms as well? (4 Replies)
Discussion started by: otheus
4 Replies

8. UNIX for Dummies Questions & Answers

search ")" with egrep - egrep: syntax error

Hi Guys, we have a shell script which basically query the Database which retrieves huge data and use the data with "egrep" . Now there is some data which contains characters like "abc)" and the same is used like below : "egrep (.+\|GDPRAB16\|GDPR/11702 96 abc)\|$ temp.txt" now while... (7 Replies)
Discussion started by: sagarjani
7 Replies

9. Shell Programming and Scripting

why we use xargs..

hi , can anyone help me by saying why we use xargs.. is it acing like a place holder..? thanks, Krips. (3 Replies)
Discussion started by: kripssmart
3 Replies

10. UNIX for Dummies Questions & Answers

Egrep cheat sheet anywhere? Looking for meaning of egrep -c

Hi I've been searching google and have not found what egrep -c means. Does anyone know where I can get a cheat sheet or what that -c means? thanks, Linda (2 Replies)
Discussion started by: leelm
2 Replies
Login or Register to Ask a Question