Visit Our UNIX and Linux User Community


xargs egrep


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting xargs egrep
# 1  
Old 07-23-2009
Network xargs egrep

Below is a sample script

Code:
jb>cat search
#!/bin/bash
set -x
PATTERN='"'`awk -F: '{printf $2 "|"} END {printf "\b"}' RULE`'"'
echo 'Pattern ['$PATTERN']'
ls -tr *$1* |xargs egrep "$PATTERN"

Sample RULE file
Code:
jb>cat RULE
P1 :pone
P2 :ptwo

Sample Output
Code:
jb>./search f
++ awk -F: '{printf $2 "|"} END {printf "\b"}' RULE
+ PATTERN='"pone|ptwo"'
+ echo 'Pattern ["pone|ptwo"]'
Pattern ["pone|ptwo"]
+ ls -tr f1 f1.txt f2 f3 f4
+ xargs egrep '"pone|ptwo"'
f2:ptwo
jb>

Directly searching

Code:
jb>ls -tr *f* |xargs egrep "pone|ptwo"
f1:000pone000
f2:ptwo

Sample file content
Code:
jb>cat f1
11
000pone000
1111
jb>cat f2
222
ptwo
22222222
jb>





Why 'search' doesn't list f1?
# 2  
Old 07-23-2009
Try the pattern without the double quotes, replace this line:

Code:
PATTERN='"'`awk -F: '{printf $2 "|"} END {printf "\b"}' RULE`'"'

with:

Code:
PATTERN=`awk -F: '{printf $2 "|"} END {printf "\b"}' RULE`

# 3  
Old 07-23-2009
Thanks,it works.
I have one more question.

I have some 20 patterns OR'ed 'pat1|pat2|.....pat20'
Each pattern of length around 20 chars
and i search around 30 files each of size 10 MB.
It takes 2 mins to complete !
Is there any optimal/fast way to do this ? Smilie
# 4  
Old 07-23-2009
You can try something like:

Code:
awk -F: '{print $2}' RULE > tmp.file

grep -f tmp.file $(ls -tr *$1*)

# 5  
Old 07-23-2009
1.Actually the RULE file names pattern,(RULE file as such is not a pattern) example
Code:
>cat RULE
SUCCESS       :Message sent successfully to
FAILURE         :Message send failed for
Acknowledged :Got ack from  
#Goes on

2.This line extract the second column ,forms a (ORed) pattern

Code:
PATTERN=`awk -F: '{printf $2 "|"} END {printf "\b"}' RULE`

#PATTERN will be  'Message sent successfully to|Message send failed for|Got ack from'

3.Search for all pattern in all file and redirect the o/p to tmp.txt (Takes 2 mins )
Code:
ls -tr *$1* |xargs egrep "$PATTERN" > tmp.txt

4.from tmp.txt do some more filtering,collect statistics (count each pattern)
(Dont have problem with this step ,takes very less time.)

Code:
Sample Output :
SUCCESS :1423
FAILURE : 432
Acknowledged : 764


But step 3 alone takes much of the time(around 2 mins)
# 6  
Old 07-23-2009
Have you tried my last 2 commands for steps 2 and 3?

Code:
awk -F: '{print $2}' RULE > tmp.file

grep -f tmp.file $(ls -tr *$1*) > tmp.txt

# 7  
Old 07-23-2009
Once again thanks,It works
Search completed in less than 30 secs.

My grep version doesn't support -f ,fgrep does.
Code:
awk -F: '{print $2}' RULE > tmp.txt
fgrep -f tmp.txt $(ls -tr *26_06_2009*) > tmp2.txt

(Sorry I thought you didn't get my question,but actually I misunderstood your solution )

Previous Thread | Next Thread
Test Your Knowledge in Computers #758
Difficulty: Easy
The Amiga was a family of personal computers introduced by Microsoft in 1985.
True or False?

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Help with xargs

Using the bash shell I'm trying to either create a command for the command line or a script that will show netstat info for a given process name. Here is an example of what I'm trying to do:$ ps aux |grep catalina |grep -v grep | awk '{print $2}' 5132 $ netstat -nlp |grep 5132 (Not all processes... (11 Replies)
Discussion started by: axiopisty
11 Replies

2. Shell Programming and Scripting

xargs

I have dir with many files ( close to 4M ) . $ ls -la total 76392 drwxr-xr-x 10 oracle dba 512 Jun 06 14:39 . drwxr-xr-x 11 oracle dba 512 Dec 20 13:21 .. drwxr-xr-x 2 oracle dba 39074816 Jun 15 14:07 ad I am trying to delete them using... (8 Replies)
Discussion started by: talashil
8 Replies

3. Shell Programming and Scripting

xargs

Dear all , any suggest on xargs to combine from (1.txt and 2.txt) to output.txt ? thanks a lot. 1.txt 0123 BUM-5M BUM-5M 93490481 63839 0124 BUM-5M BUM-5M 112112 ... (3 Replies)
Discussion started by: samoptimus
3 Replies

4. Shell Programming and Scripting

Xargs and

Hello there, Let me show you a simple example of what I am trying to achieve: 1) I have an input text file with some lines: 1 a 2 b 3 c 2) And I want to run a command with these lines as arguments (+ arbitrary extra arguments). For example: $ command "1 a" "2 b" "3 c" "bye" I... (7 Replies)
Discussion started by: tokland
7 Replies

5. Shell Programming and Scripting

Help in using xargs

Hi, I have a requirement to RCP the files from remote server to local server. Also the RCP has to run in parallel. However using 'xargs' retrives 2 file names during each loop. How do we restrict to only one file name using xargs and loop till remaining files. I use the below code for... (2 Replies)
Discussion started by: senthil3d
2 Replies

6. Shell Programming and Scripting

Using xargs

hi i just want to know that how do we use xargs command to find files which are greater than specified memory in a given directory (6 Replies)
Discussion started by: sumit the cool
6 Replies

7. UNIX for Advanced & Expert Users

xargs -P

I discovered that GNU's xargs has a -P option to allow its processes to run in parallel. Great! Is this a GNU thing, or is it supported by other platforms as well? (4 Replies)
Discussion started by: otheus
4 Replies

8. UNIX for Dummies Questions & Answers

search ")" with egrep - egrep: syntax error

Hi Guys, we have a shell script which basically query the Database which retrieves huge data and use the data with "egrep" . Now there is some data which contains characters like "abc)" and the same is used like below : "egrep (.+\|GDPRAB16\|GDPR/11702 96 abc)\|$ temp.txt" now while... (7 Replies)
Discussion started by: sagarjani
7 Replies

9. Shell Programming and Scripting

why we use xargs..

hi , can anyone help me by saying why we use xargs.. is it acing like a place holder..? thanks, Krips. (3 Replies)
Discussion started by: kripssmart
3 Replies

10. UNIX for Dummies Questions & Answers

Egrep cheat sheet anywhere? Looking for meaning of egrep -c

Hi I've been searching google and have not found what egrep -c means. Does anyone know where I can get a cheat sheet or what that -c means? thanks, Linda (2 Replies)
Discussion started by: leelm
2 Replies

Featured Tech Videos