Unix/Linux Go Back    


Shell Programming and Scripting BSD, Linux, and UNIX shell scripting — Post awk, bash, csh, ksh, perl, php, python, sed, sh, shell scripts, and other shell scripting languages questions here.

Grep multiple keywords from a file

Shell Programming and Scripting


Tags
grep

Reply    
 
Thread Tools Search this Thread Display Modes
    #8  
Old Unix and Linux 05-13-2017
Scrutinizer's Unix or Linux Image
Scrutinizer Scrutinizer is offline Forum Staff  
Moderator
 
Join Date: Nov 2008
Last Activity: 19 September 2017, 11:10 PM EDT
Location: Amsterdam
Posts: 11,533
Thanks: 500
Thanked 3,335 Times in 2,943 Posts
Quote:
Originally Posted by Loc View Post
I have a script that will search for a keyword in all the log files. It work just fine.


Code:
 
 LOG_FILES={ "/Sandbox/logs/*" }
 for file in ${LOG_FILES[@]}; do
   grep $1 $file
done

[..]
This code will not work since it contains an error because curly braces are used in the assignment.
For array assignment you need normal parentheses.

Code:
 
 LOG_FILES=( "/Sandbox/logs/*" )
 for file in ${LOG_FILES[@]}; do
   grep $1 $file
done

However, since "/Sandbox/logs/*" is quoted this results in an array with a single array element "/Sandbox/logs/*".

It works because the array contents gets expanded in the loop, since the variables are used without quotes, but it will not work with files with spaces in the name.

The proper way to do this would be this:

Code:
LOG_FILES=( /Sandbox/logs/* )
for file in "${LOG_FILES[@]}"; do
  grep "$1" "$file"
done

Although if there are not too many files this would be more efficient (and thus faster):

Code:
LOG_FILES=( /Sandbox/logs/* )
grep -h "$1" "${LOG_FILES[@]}"

---

Quote:
Originally Posted by RudiC View Post
Not sure what you're after. If it's the names of the files containig ALL of the keywords, try

Code:
grep -E 'A1|B2|C3' file[1-3] | cut -d: -f1 | uniq -c | grep '^ *3' | cut -d" " -f8
file1

You will need to adapt the keyword count (3) of the last grep, and the field number (8) of the last cut.

[..]
This does not do that . It returns the files that have matches on three lines for ANY of the keywords.

---

Quote:
Originally Posted by AbelLuis View Post
Hi, you may try this:


Code:
RE="/$A1/ && /$A2/ && /$A3/"
eval awk "'" $RE "'" filename

The first line is for the expansion of parameters; the second, eval, parses and execute the built command.

Regards.
You do not need eval here, nor the quotes..

You can just use:

Code:
awk "$RE" filename

Assuming variables A[1-3] contain the strings that we are looking for..

Last edited by Scrutinizer; 05-13-2017 at 02:59 AM..
The Following 2 Users Say Thank You to Scrutinizer For This Useful Post:
AbelLuis (05-13-2017), RudiC (05-13-2017)
Sponsored Links
    #9  
Old Unix and Linux 05-13-2017
RudiC RudiC is offline Forum Staff  
Moderator
 
Join Date: Jul 2012
Last Activity: 20 September 2017, 7:26 AM EDT
Location: Aachen, Germany
Posts: 11,352
Thanks: 297
Thanked 3,521 Times in 3,240 Posts
Rats! You're right; I didn't think of duplicates. Wouldn't

Code:
grep -E 'A1|B2|C3' file[1-3] | sort -u | cut -d: -f1 | uniq -c | grep '^ *3' | cut -d" " -f8

be better?
Sponsored Links
    #10  
Old Unix and Linux 05-13-2017
Don Cragun's Unix or Linux Image
Don Cragun Don Cragun is offline Forum Staff  
Administrator
 
Join Date: Jul 2012
Last Activity: 20 September 2017, 2:57 AM EDT
Location: San Jose, CA, USA
Posts: 10,502
Thanks: 542
Thanked 3,675 Times in 3,135 Posts
Quote:
Originally Posted by RudiC View Post
Rats! You're right; I didn't think of duplicates. Wouldn't

Code:
grep -E 'A1|B2|C3' file[1-3] | sort -u | cut -d: -f1 | uniq -c | grep '^ *3' | cut -d" " -f8

be better?
Not really...
Let us imagine that file1 contains:

Code:
A1 x
A1 y
A1 z

and that file2 contains:

Code:
A1 B2 C3

and that file3 contains:

Code:
A1 x
B2 y
C3 z
A1 a
B2 b
C3 c

In this example, files file2 and file3 both contain all three strings, but the output from:

Code:
grep -E 'A1|B2|C3' file[1-3] | sort -u | cut -d: -f1 | uniq -c

is:

Code:
   3 file1
   1 file2
   6 file3

which shows that only file1 comes up with a count of three unique matching lines.

Until we get a clear description of the desired output and the log file format, I think we're wasting our time guessing at what might supply the output the OP really wants.
The Following User Says Thank You to Don Cragun For This Useful Post:
RudiC (05-13-2017)
    #11  
Old Unix and Linux 05-13-2017
Scrutinizer's Unix or Linux Image
Scrutinizer Scrutinizer is offline Forum Staff  
Moderator
 
Join Date: Nov 2008
Last Activity: 19 September 2017, 11:10 PM EDT
Location: Amsterdam
Posts: 11,533
Thanks: 500
Thanked 3,335 Times in 2,943 Posts
One more go Linux On rereading post #1, I think the OP is looking for a script kind of like this:


Code:
LOG_FILES=( /Sandbox/logs/* )
IFS=\|
awk -v par="$*" 'BEGIN{split(par,P,"|")} {for(i in P) if(! index($0,P[i]))next}1' "${LOG_FILES[@]}"

or :

Code:
LOG_FILES=( /Sandbox/logs/* )
for file in "${LOG_FILES[@]}"; do
  cat "$file"
done |
{
  IFS=\|
  awk -v par="$*" 'BEGIN{split(par,P,"|")} {for(i in P) if(! index($0,P[i]))next}1'
}

index() is used, because the search should not be interpreted as regex

Last edited by Scrutinizer; 05-13-2017 at 11:24 AM..
Sponsored Links
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Linux More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
Find keywords in multiple log files dellanicholson Shell Programming and Scripting 13 06-07-2016 09:52 PM
How to grep keywords? khchong Shell Programming and Scripting 3 04-18-2014 06:56 AM
Grep from multiple patterns multiple file multiple output Diya123 Shell Programming and Scripting 3 11-02-2013 12:38 PM
Grep Keywords one by one dashing201 Shell Programming and Scripting 3 07-18-2013 07:11 AM
finding keywords in many files using grep raghulrajan UNIX for Dummies Questions & Answers 5 03-14-2012 11:05 AM



All times are GMT -4. The time now is 09:00 AM.