I have a script that will search for a keyword in all the log files. It work just fine.
Code:
LOG_FILES={ "/Sandbox/logs/*" }
for file in ${LOG_FILES[@]}; do
grep $1 $file
done
This only works for 1 keyword. What if I want to search for more then 1 keywords, say 4 or maybe even 10 keywords.
So I would enter:
Code:
./myfile.sh 120.2.1.1 Chrome 400 POST
It would check for the IP address, chrome, 400 and POST keywords in all the log files. I see on the internet, they have something like this:
Code:
egrep 'A1|A2|A3' filename
This specify that it will grep the output that is either A1 or A2 or A3 but I would like to grep the output if it is A1 AND A2 AND A3. I'm not sure how to go about doing this? Any help would be greatly appreciated.
The way to do that is to create a list of files containing A1 and passing them to a grep that will look at only those files. To search for A3 the second grep would have to produce a list as well which you could save or pipe through.
The program xargs is very helpful for this in that it reads an input file or piped input, runs your command, and places every line of your input file as the last argument(s) of your command.
Code:
grep -l A1 {2nd parm for star name files if necessary} |tee my.list1
xargs <my.list1 grep -l A2 |xargs grep A3
If you will be doing a number of searches on files containing your first search term, creating a file with a list should be useful.
In addition to what wbport suggested, there are some other alternatives in cases where the strings you are looking for all occur on a single line in your log files...
If the strings you are searching for always appear in the same order on a line (i.e., A1 followed by A2 followed by A3), you can use:
Code:
grep 'A1.*A2.*A3' file
If they can appear in any order, try one of the following:
Note that if the strings you're searching for do not all appear on a single line, the code suggested by wbport will only print lines containing A3 (not lines containing just A1 or A2). If you want to print all lines containing A1, A2, or A3 but only print those lines if the file contains all three strings, you need an extra level of grep to print the final results (i.e. read some of your files four times when looking for 3 strings; three times to find the names of files that contain each string and a final time to print all three strings (using OR instead of AND). Or, you can use awk to read the file once, gather lines that match any of your strings and keep track of which strings have been found, and then print all matching lines at the end if all of your strings have been found. If this is what you need, we can help you figure out a way to do that, but I'm not going to try to do it here if you don't need to do that. Your description of your problem isn't clear as to the extent of the problem you're trying to solve.
It would check for the IP address, chrome, 400 and POST keywords in all the log files. I see on the internet, they have something like this:
Code:
egrep 'A1|A2|A3' filename
This specify that it will grep the output that is either A1 or A2 or A3 but I would like to grep the output if it is A1 AND A2 AND A3. I'm not sure how to go about doing this? Any help would be greatly appreciated.
If all three need to be on the same line but can be in any order, just pipe the output from the first grep (without using -l) through the 2nd and 3rd. It won't be necessary to search for all permutations of what you need.
Location: Saint Paul, MN USA / BSD, CentOS, Debian, OS X, Solaris
Posts: 2,288
Thanks Given: 430
Thanked 480 Times in 395 Posts
Hi.
Here are a few alternatives; One is the use of shell expressions, the other is a relative of grep, agrep:
Code:
#!/usr/bin/env ksh
#!/usr/bin/env bash
#!/usr/bin/env zsh
# @(#) s1 Demonstrate match multiple strings AND and OR, agrep, shell.
# Utility functions: print-as-echo, print-line-with-visual-space.
LC_ALL=C ; LANG=C ; export LC_ALL LANG
pe() { for _i;do printf "%s" "$_i";done; printf "\n"; }
pl() { pe;pe "-----" ;pe "$*"; }
em() { pe "$*" >&2 ; }
db() { ( printf " db, ";for _i;do printf "%s" "$_i";done;printf "\n" ) >&2 ; }
db() { : ; }
C=$HOME/bin/context && [ -f $C ] && . $C dixf agrep
FILE=${1-data1}
pl " Data file $FILE:"
cat $FILE
pl " With agrep, \"and\":a b c :"
agrep "a;b;c" $FILE
pl " With agrep, \"or\":a b c :"
agrep "a,b,c" $FILE
# A pattern-list is a list of one or more patterns
# separated from each other with a & or |. A & signifies
# that all patterns must be matched whereas | requires
# that only one pattern be matched.
# shopt -s extglob
# shopt extglob
pl " With shell expressions, \"and\", a b c ksh (bash, zsh fail):"
while read line
do
if [[ $line == @(*a*&*b*&*c*) ]]
then
pe "Matched line: $line"
fi
done < $FILE
pl " With shell expressions, \"or\", a b c ksh (zsh fail):"
while read line
do
if [[ $line == @(*a*|*b*|*c*) ]]
then
pe "Matched line: $line"
fi
done < $FILE
pl " Some detail about agrep:"
dixf agrep
exit $?
producing:
Code:
$ ./s1
Environment: LC_ALL = C, LANG = C
(Versions displayed with local utility "version")
OS, ker|rel, machine: Linux, 3.16.0-4-amd64, x86_64
Distribution : Debian 8.7 (jessie)
ksh 93u+
dixf (local) 1.42
agrep - ( /usr/bin/agrep, 2012-04-12 )
-----
Data file data1:
also
abracadabra
chock full of beans
hello, world.
silent
-----
With agrep, "and":a b c :
abracadabra
chock full of beans
-----
With agrep, "or":a b c :
also
abracadabra
chock full of beans
-----
With shell expressions, "and", a b c ksh (bash, zsh fail):
Matched line: abracadabra
Matched line: chock full of beans
-----
With shell expressions, "or", a b c ksh (zsh fail):
Matched line: also
Matched line: abracadabra
Matched line: chock full of beans
-----
Some detail about agrep:
agrep search a file for a string or regular expression, with... (man)
Path : /usr/bin/agrep
Version : - ( /usr/bin/agrep, 2012-04-12 )
Type : ELF 64-bit LSB shared object, x86-64, version 1 (S ...)
Help : probably available with -h
Repo : Debian 8.7 (jessie)
The agrep will be much faster for files of more than trivial size.
I have several problems with my program: I hope you can help me.
1) the If else statement isn't working . The IF Else syntax is:
If MEMSIZE OR sasfoundation (SASEXE) OR Real Time(second) >1.0 and Filename, output column name and value to csv or else nothing
Example progflag,cvs:... (13 Replies)
The Problem that I am having is when the code ran and populated the progflag.csv file, columns MEMSIZE, SECOND and SASEXE were blank. The next problems are the IF else statement isn't working and the email function isn't sending the progflag.csv attachment.
a. What I want the program to do is to... (2 Replies)
I have below text file only with one line:
vi test.txt
This is the first test from a1.loa1 a1v1, b2.lob2, "c3.loc3" c3b1, loc4 but not from mot3 and second test from a5.loa5
Below should be the output that i want:
a1.loa1
b2.lob2
c3.loc3
loc4
a5.loa5
alv1 and c3b1 should be... (3 Replies)
Hi,
I want to grep multiple patterns from multiple files and save to multiple outputs. As of now its outputting all to the same file when I use this command.
Input : 108 files to check for 390 patterns to check for. output I need to 108 files with the searched patterns.
Xargs -I {} grep... (3 Replies)
Hi,
I have a file containing list of strings like
i:
Pink
Yellow
Green
and I have file having list of file names in a directory
j :
a
b
c
d
Where j contains of a ,b,c,d are as follows
a:
Pink (3 Replies)
Hi
I am trying to determine number of lines having a specific keyword.
So for that I am using below query:
grep -i 'keyword1' filename|wc -l
This give me number of lines. Perfect for me.
However now the requirement is
I have multiple keywords together... and I have to find number of... (3 Replies)
Hi to all
Sorry for the confusion because I did not explain the task clearly.
There are many .hhr files in a folder
There are so many lines in these .hhr files but I want only the following 2 lines to be transferred to the output file.
The keyword No 1 and all the words in the next line
They... (5 Replies)
hey guys,
Hey all,
I'm doing a project currently and want to index words in a webpage.
So there would be a file with webpage content and a file with list of words, I want an output file with true and false that would show which word exists in the webpage.
example:
Webpage content... (2 Replies)
I want to search files (basically .cc files) in /xx folder and subfolders.
Those files (*.cc files) must contain #include "header.h" AND x() function.
I am writing it another way to make it clear,
I wanna list of *.cc files that have 'header.h' & 'x()'. They must have two strings, header.h... (2 Replies)
