Shell script to search all files for every string in another file
Hello All
I have a pattern.txt file in source directory ((/project/source/) in linux server and data looks like:
Code:
123abc17
234cdf19
235ifg20
I have multiple log files in log directory (/project/log/) in linux server and data for one log file looks like:
Code:
<?xml version="1.0" processid is 123abc17
read successfully at 20161109093456
<?xml version="1.0" process id is 986bng21
read successfully at 20161109093459
message id aazzkk110 is 123abc17
message id aakjahsdk110 is 234cdf19
<?xml version="1.0" processid is 235ifg20
read successfully at 20161109093456
<?xml version="1.0" process id is 987skj29
I want to grep every single string in the source file - pattern.txt against all log files in log directory
and populate the output in a output.txt file in directory (/project/output/) data should look like below:
Code:
<?xml version="1.0" processid is 123abc17
read successfully at 20161109093456
message id aazzkk110 is 123abc17
message id aakjahsdk110 is 234cdf19
<?xml version="1.0" processid is 235ifg20
read successfully at 20161109093456
If a string is matched against any log file, I want to check if line starts with '<?xml version' then i want to populate second line also in the output file along with matching line
otherwise I just want to populate the matching line.
Please help me in writing a unix command / shell script to achieve it.
Last edited by Scrutinizer; 11-09-2016 at 08:37 PM..
Reason: code tags
as long as you only have regular files with names ending with .log, but I would guess that Jim accidentally just dropped the letter f to only apply grep to regular files with names ending with .log.
which should avoid any cases where the first two commands might fails with an "argument list too long" error.
Note, however, that any of these will just print the names of the files containing a line matching a line that from your pattern file. The grep utility that you requested be used does not contain sufficient power to select and print a varying number of lines following a matched pattern. For that you need something more like awk:
Code:
#!/bin/ksh
pattern_file="/project/source/pattern.txt"
log_dir="/project/log"
find "$log_dir" -type f -name '*.log' -exec awk '
FNR == NR {
patterns[++patternsc] = $0
next
}
/^<[?]xml version/ {
n = 2
}
{ for(i = 1; i <= patternsc; i++)
if($0 ~ patterns[i])
break
if(i <= patternsc)
p = ($0 ~ /^<[?]xml version/) ? 2 : 1
}
p { print
p--
}' "$pattern_file" {} +
This was written and tested using a Korn shell on macOS 10.12.1, but will work with any shell that accepts Bourne shell syntax. If you want to try this on a Solaris/SunOS system, change awk to /usr/xpg4/bin/awk or nawk.
Thank you for the script, when I executed it I was seeing all the matching lines against the patterns in the pattern.txt file but even if the line is starting with
Code:
<?xml version
its not populating the second line. Could you please help me with with that.
I have a text file which is generated when the batch job is run. This batch may take few mins to run. When completed, the last line of the text file would be process completed. I need a shell script which will wait for this file till the process completed is printed in it, once found, it would move... (2 Replies)
So, there is a large file where I have to conduct several search using bash shell scripting.
The file is like this:
TITLE and AUTHOR ETEXT NO. Aspects of plant life; with special reference to the British flora, 56900 by Robert Lloyd... (1 Reply)
Hi folks,
I am new for shell script, I hope somebody to help me to write shell script
My requirement is below steps
1. I have apache access.log i.e located in /var/log/httpd/
Ex.
127.0.0.1 - - "GET... (14 Replies)
All,
i have a file text.log:
cover6
cover3
cover2
cover4
other file is abc.log as :
0
0
1
0
Then I have a excel file result.xls that contains:
Name Path Pass
cover2
cover3
cover6
cover4 (1 Reply)
Hi,
Need your help for this scripting issue I have. I am not really good at this, so seeking your help.
I have a file looking similar to this:
Hello, i am human and name=ABCD.
How are you?
Hello, i am human and name=PQRS.
I am good.
Hello, i am human and name=ABCD.
Good bye.
Hello, i... (12 Replies)
Hi,
I am trying to create a shell script in unix platform, hence will need info on how to start and any ideas from you guys. million thx.
Objective: Shell script to search all entries from 1 file(a.out) to all other files and extract the search output to 1 file (c.out).
Situation,
1)... (8 Replies)
Have to read one file say sourcefile containing several words and having another folder containing several files.
Now read the first word of Sourcefile & search it into the folder consisting sevral files, and create another file with result. We hhave to pick the filename of the file in which... (3 Replies)
Hi,
We Perfrom Loads to the database through a Perl script which generates a statistics file. I need to read the statistics. the Statistics file looks something like below:
Process Beginning - 08-26-2010-23.41.47
DB2 CONNECTION SUCCESSFUL!
Ready to process and load file: FILENAME
# of... (2 Replies)
Hi,
I've a shell script e.g.
#!/bin/bash
echo "Enter the next hop id"
read nhid
echo "enter the IP address"
read IP
echo "enter the interface name"
read name
echo "enter the enable/disable state"
read state
exit 0
now from this script i want to search strings in another (.cam)
... (6 Replies)
I need to count the number of files which have a search string, but counting the file only once
if search string is found.
eg: File1: Please note that there are 2 occurances of "aaa"
aaa
bbb
ccc
aaa
File2: Please note that there are 3 occurances of "aaa"
aaa
bbb
ccc... (1 Reply)