Grep multiple keywords from a file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Grep multiple keywords from a file
# 8  
Old 05-13-2017
Quote:
Originally Posted by Loc
I have a script that will search for a keyword in all the log files. It work just fine.

Code:
 
 LOG_FILES={ "/Sandbox/logs/*" }
 for file in ${LOG_FILES[@]}; do
   grep $1 $file
done

[..]
This code will not work since it contains an error because curly braces are used in the assignment.
For array assignment you need normal parentheses.
Code:
 
 LOG_FILES=( "/Sandbox/logs/*" )
 for file in ${LOG_FILES[@]}; do
   grep $1 $file
done

However, since "/Sandbox/logs/*" is quoted this results in an array with a single array element "/Sandbox/logs/*".

It works because the array contents gets expanded in the loop, since the variables are used without quotes, but it will not work with files with spaces in the name.

The proper way to do this would be this:
Code:
LOG_FILES=( /Sandbox/logs/* )
for file in "${LOG_FILES[@]}"; do
  grep "$1" "$file"
done

Although if there are not too many files this would be more efficient (and thus faster):
Code:
LOG_FILES=( /Sandbox/logs/* )
grep -h "$1" "${LOG_FILES[@]}"

---

Quote:
Originally Posted by RudiC
Not sure what you're after. If it's the names of the files containig ALL of the keywords, try
Code:
grep -E 'A1|B2|C3' file[1-3] | cut -d: -f1 | uniq -c | grep '^ *3' | cut -d" " -f8
file1

You will need to adapt the keyword count (3) of the last grep, and the field number (8) of the last cut.

[..]
This does not do that . It returns the files that have matches on three lines for ANY of the keywords.

---

Quote:
Originally Posted by AbelLuis
Hi, you may try this:

Code:
RE="/$A1/ && /$A2/ && /$A3/"
eval awk "'" $RE "'" filename

The first line is for the expansion of parameters; the second, eval, parses and execute the built command.

Regards.
You do not need eval here, nor the quotes..

You can just use:
Code:
awk "$RE" filename

Assuming variables A[1-3] contain the strings that we are looking for..

Last edited by Scrutinizer; 05-13-2017 at 03:59 AM..
These 2 Users Gave Thanks to Scrutinizer For This Post:
# 9  
Old 05-13-2017
Rats! You're right; I didn't think of duplicates. Wouldn't
Code:
grep -E 'A1|B2|C3' file[1-3] | sort -u | cut -d: -f1 | uniq -c | grep '^ *3' | cut -d" " -f8

be better?
# 10  
Old 05-13-2017
Quote:
Originally Posted by RudiC
Rats! You're right; I didn't think of duplicates. Wouldn't
Code:
grep -E 'A1|B2|C3' file[1-3] | sort -u | cut -d: -f1 | uniq -c | grep '^ *3' | cut -d" " -f8

be better?
Not really...
Let us imagine that file1 contains:
Code:
A1 x
A1 y
A1 z

and that file2 contains:
Code:
A1 B2 C3

and that file3 contains:
Code:
A1 x
B2 y
C3 z
A1 a
B2 b
C3 c

In this example, files file2 and file3 both contain all three strings, but the output from:
Code:
grep -E 'A1|B2|C3' file[1-3] | sort -u | cut -d: -f1 | uniq -c

is:
Code:
   3 file1
   1 file2
   6 file3

which shows that only file1 comes up with a count of three unique matching lines.

Until we get a clear description of the desired output and the log file format, I think we're wasting our time guessing at what might supply the output the OP really wants.
This User Gave Thanks to Don Cragun For This Post:
# 11  
Old 05-13-2017
One more go Smilie On rereading post #1, I think the OP is looking for a script kind of like this:

Code:
LOG_FILES=( /Sandbox/logs/* )
IFS=\|
awk -v par="$*" 'BEGIN{split(par,P,"|")} {for(i in P) if(! index($0,P[i]))next}1' "${LOG_FILES[@]}"

or :
Code:
LOG_FILES=( /Sandbox/logs/* )
for file in "${LOG_FILES[@]}"; do
  cat "$file"
done |
{
  IFS=\|
  awk -v par="$*" 'BEGIN{split(par,P,"|")} {for(i in P) if(! index($0,P[i]))next}1'
}

index() is used, because the search should not be interpreted as regex

Last edited by Scrutinizer; 05-13-2017 at 12:24 PM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Find keywords in multiple log files

I have several problems with my program: I hope you can help me. 1) the If else statement isn't working . The IF Else syntax is: If MEMSIZE OR sasfoundation (SASEXE) OR Real Time(second) >1.0 and Filename, output column name and value to csv or else nothing Example progflag,cvs:... (13 Replies)
Discussion started by: dellanicholson
13 Replies

2. UNIX for Dummies Questions & Answers

Find keywords in multiple log files

The Problem that I am having is when the code ran and populated the progflag.csv file, columns MEMSIZE, SECOND and SASEXE were blank. The next problems are the IF else statement isn't working and the email function isn't sending the progflag.csv attachment. a. What I want the program to do is to... (2 Replies)
Discussion started by: dellanicholson
2 Replies

3. Shell Programming and Scripting

How to grep keywords?

I have below text file only with one line: vi test.txt This is the first test from a1.loa1 a1v1, b2.lob2, "c3.loc3" c3b1, loc4 but not from mot3 and second test from a5.loa5 Below should be the output that i want: a1.loa1 b2.lob2 c3.loc3 loc4 a5.loa5 alv1 and c3b1 should be... (3 Replies)
Discussion started by: khchong
3 Replies

4. Shell Programming and Scripting

Grep from multiple patterns multiple file multiple output

Hi, I want to grep multiple patterns from multiple files and save to multiple outputs. As of now its outputting all to the same file when I use this command. Input : 108 files to check for 390 patterns to check for. output I need to 108 files with the searched patterns. Xargs -I {} grep... (3 Replies)
Discussion started by: Diya123
3 Replies

5. Shell Programming and Scripting

Grep and replace multiple strings in a file with multiple filenames in a file

Hi, I have a file containing list of strings like i: Pink Yellow Green and I have file having list of file names in a directory j : a b c d Where j contains of a ,b,c,d are as follows a: Pink (3 Replies)
Discussion started by: madabhg
3 Replies

6. Shell Programming and Scripting

Grep Keywords one by one

Hi I am trying to determine number of lines having a specific keyword. So for that I am using below query: grep -i 'keyword1' filename|wc -l This give me number of lines. Perfect for me. However now the requirement is I have multiple keywords together... and I have to find number of... (3 Replies)
Discussion started by: dashing201
3 Replies

7. UNIX for Dummies Questions & Answers

finding keywords in many files using grep

Hi to all Sorry for the confusion because I did not explain the task clearly. There are many .hhr files in a folder There are so many lines in these .hhr files but I want only the following 2 lines to be transferred to the output file. The keyword No 1 and all the words in the next line They... (5 Replies)
Discussion started by: raghulrajan
5 Replies

8. Shell Programming and Scripting

searching keywords in file

hey guys, Hey all, I'm doing a project currently and want to index words in a webpage. So there would be a file with webpage content and a file with list of words, I want an output file with true and false that would show which word exists in the webpage. example: Webpage content... (2 Replies)
Discussion started by: Johanni
2 Replies

9. UNIX for Dummies Questions & Answers

grep command to find multiple strings in multiple lines in a file.

I want to search files (basically .cc files) in /xx folder and subfolders. Those files (*.cc files) must contain #include "header.h" AND x() function. I am writing it another way to make it clear, I wanna list of *.cc files that have 'header.h' & 'x()'. They must have two strings, header.h... (2 Replies)
Discussion started by: ritikaSharma
2 Replies

10. Shell Programming and Scripting

Search a file with keywords

Hi All I have a file of format asdf asf first sec endi asdk rt 123 ferf dfg ijglkp (7 Replies)
Discussion started by: mailabdulbari
7 Replies
Login or Register to Ask a Question