Location: Saint Paul, MN USA / BSD, CentOS, Debian, OS X, Solaris
Posts: 2,288
Thanks Given: 430
Thanked 480 Times in 395 Posts
Hi.
Here is a shell script that encapsulates an awk script. The awk reads each line of each file once to ensure that each file contains at least one match to the multiple strings.
This example uses 2 strings and looks through 4 files, but can be easily extended for more strings. One advantage is that each file is read only once. Another is that when all strings have matched at least once, the operation skips to the next file. For a small number of files, the awk probably is no faster than multiple greps, but for the grep, you can imagine that for 3000 files, if the first keyword appears in 2000 files, then those 2000 files will be read at least twice, and so on. On the other hand, awk will use more CPU time than grep. For simple alpha-numeric strings, fgrep might be slightly faster than plain grep.
Both this approach and the suggestion from bartus11 will require some work to fit a specific problem. Here is the awk:
Code:
#!/usr/bin/env bash
# @(#) s1 Demonstrate single-pass to find multiple strings in files.
pe() { for _i;do printf "%s" "$_i";done; printf "\n"; }
pl() { pe;pe "-----" ;pe "$*"; }
db() { ( printf " db, ";for _i;do printf "%s" "$_i";done;printf "\n" ) >&2 ; }
db() { : ; }
C=$HOME/bin/context && [ -f $C ] && $C awk
FILES="data*"
pl " Data files:"
for i in $FILES
do
pe
pe " Data file $i:"
cat $i
done
pl " Results, files must contain \"a\" AND \"b\":"
# "c" is not used for testing, and can be eliminated.
awk '
BEGIN { a = b = c = 0 }
FNR == 1 { a = b = c = 0 }
/a/ { a++ }
/b/ { b++ }
/c/ { c++ }
a>0 && b>0 { print FILENAME ; nextfile }
' $FILES
exit 0
producing:
Code:
% ./s1
Environment: LC_ALL = C, LANG = C
(Versions displayed with local utility "version")
OS, ker|rel, machine: Linux, 2.6.26-2-amd64, x86_64
Distribution : Debian GNU/Linux 5.0.8 (lenny)
bash GNU bash 3.2.39
awk GNU Awk 3.1.5
-----
Data files:
Data file data1:
a
b
Data file data2:
b
a
c
Data file data3:
c
a
Data file data4:
a b
c
d
-----
Results, files must contain "a" AND "b":
data1
data2
data4
CW
P.S. Thanks for the awesome reply DRL, but that's a tough one to work with when the enquirer is on the other end of the phone waiting for an answer
Pretty cool tho
Hi All,
I want to use egrep on multiple files and the results should be output to multiple files. I am using the below code in my shell script(working in Ksh shell). However with this code I am not attaining the desired results.
#!/bin/ksh
(
a="/path/file1"
b="path/file2"
for file in... (4 Replies)
Hello,
I would like to ask for help with csh script.
An example of an input in .txt file is below, the number of lines varies from file to file and I have 2 or 3 columns with values. I would like to read all the values (probably one by one) and set them to independent unique variables that... (7 Replies)
Hi,
I have 5 files with two columns. I need to merge all the 5 files based on column 1. If any of them are missing then corresponding 2nd column should be substituted by missing value.
I know hoe to do this for 2 files. but how can I implement for 5 files. I tried this based on 5 files but it... (2 Replies)
Hi,
I want to grep multiple patterns from multiple files and save to multiple outputs. As of now its outputting all to the same file when I use this command.
Input : 108 files to check for 390 patterns to check for. output I need to 108 files with the searched patterns.
Xargs -I {} grep... (3 Replies)
This
for i in /dev/disco/*;do lvdisplay $i|grep -i size;done
Return me every size of lvm in vg "disco"
I want to return me,the size and the name of lvm,how to do this?
Thanks (7 Replies)
Hi,
I'd like to find the values of certain fields in multiple csv files stored in 1 directory
based upon an input search string.
An fgrep returns the complete record, I only want certain fields.
Thanks in advance for your help.
Perry (6 Replies)
Hi, every one!
I have a file with multiple strings.
file1
ATQRGNE
ASQGVKFTE
ASSQYRDRGGLET
SPEQGARSDE
ASSRDFTDT
ASSYSGGYE
ASSYTRLWNTGE
ASQGHNTD
PSLGGGNQPQH
SLDRDSYNEQF
I want to grep each string in hundreds of files in the same directory, further, I want to find out the string... (7 Replies)
Hi All,
I have multiple (5+) text files with single columns and I would like to grep the common values across all the text files and parse it to a new file. All the values are numerical. Please let me know how to do it using awk. (6 Replies)
Hi,
I will use below command for grep single string ("osuser" is search string)
ex: find . -type f | xarg grep -il osuser
but i have one more string "v$session"
here i want to grep in which file these two strings are present.
any help is appreciated,
Thanks in advance.
Gagan (2 Replies)
I want to search files (basically .cc files) in /xx folder and subfolders.
Those files (*.cc files) must contain #include "header.h" AND x() function.
I am writing it another way to make it clear,
I wanna list of *.cc files that have 'header.h' & 'x()'. They must have two strings, header.h... (2 Replies)
03-18-2012
How do you [e]grep for multiple values within multiple files?
)
