Shell script to search all files for every string in another file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Shell script to search all files for every string in another file
# 8  
Old 11-10-2016
p will be set to 2 if the line matching any pattern begins with <?xml..., or 1 otherwise. When printing it is counted down until it reaches 0, thus printing 2 or 1 line, resp.
The n = 2 doesn't seem to be necessary for the correct output; maybe a relict from other attempt to the solution?
This User Gave Thanks to RudiC For This Post:
# 9  
Old 11-10-2016
Try also (shamelessly stealing from Don Cragun's proposal)
Code:
find "$log_dir" -type f -name '*.log' -exec awk '
FNR == NR       {patterns[$0]
                 next
                }
/^<\?xml v.*n/  {getline X; $0 = $0 RS X
                }
                {for (i in patterns)
                   if ($0 ~ i) print
                }
'  "$pattern_file" {} +

# 10  
Old 11-10-2016
Quote:
Originally Posted by RudiC
p will be set to 2 if the line matching any pattern begins with <?xml..., or 1 otherwise. When printing it is counted down until it reaches 0, thus printing 2 or 1 line, resp.
The n = 2 doesn't seem to be necessary for the correct output; maybe a relict from other attempt to the solution?
Hi Rüdiger,
Note that the last line of the sample input in post #1 was:
Code:
<?xml version="1.0" process id is 987skj29

If 987skj29 had been included in the pattern file my code would print that line while the code you suggested in post #9 would not. Of course both of our scripts might pick up the 1st (perhaps non-matching line) from a subsequent log file if that sample line had not been in the last log file processed in a batch by find -exec. To avoid that problem, I probably should have included an additional three lines in my script, changing:
Code:
/^<[?]xml version/ {
	n = 2
}

to:
Code:
FNR == 1 {
	p = 0
}
/^<[?]xml version/ {
	n = 2
}

to stop an erroneous printing of a continuation line when we switch to a new input log file.

Quote:
Originally Posted by pred55
Hi Don

Thank you for the script, when I executed it I was seeing all the matching lines against the patterns in the pattern.txt file but even if the line is starting with

Code:
 <?xml version

its not populating the second line. Could you please help me with with that.
Hi pred55,
With the sample input you provided in post #1, the script I suggested produces the output:
Code:
<?xml version="1.0" processid is 123abc17
read successfully at 20161109093456
message id aazzkk110 is 123abc17
message id aakjahsdk110 is 234cdf19
<?xml version="1.0" processid is 235ifg20
read successfully at 20161109093456

exactly matching the output you requested in post #1.

Please post a sample of a sequence of lines in your real data (from pattern.txt and from one of your *.log files that was not processed correctly by the code I suggested.
This User Gave Thanks to Don Cragun For This Post:
# 11  
Old 11-10-2016
Quote:
Originally Posted by Don Cragun
.
.
.
If 987skj29 had been included in the pattern file my code would print that line while the code you suggested in post #9 would not.
.
.
.
Thanks for pointing a weakness out that would need additional measures to prevent errors from happening. But: it WOULD print out that line but with a false second line - the last read X which would keep its value when getline failed.
Possible remedies: Check the getline status if (1 == getline X) $0 = $0 RS X or set X = "".
This User Gave Thanks to RudiC For This Post:
# 12  
Old 11-11-2016
Thank you all for your suggestions, I got the results.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to Search a string in a file in shell script?

I have a text file which is generated when the batch job is run. This batch may take few mins to run. When completed, the last line of the text file would be process completed. I need a shell script which will wait for this file till the process completed is printed in it, once found, it would move... (2 Replies)
Discussion started by: Lalat
2 Replies

2. Shell Programming and Scripting

Linux shell script, search by an input string

So, there is a large file where I have to conduct several search using bash shell scripting. The file is like this: TITLE and AUTHOR ETEXT NO. Aspects of plant life; with special reference to the British flora, 56900 by Robert Lloyd... (1 Reply)
Discussion started by: Philia
1 Replies

3. Shell Programming and Scripting

UNIX shell script to search a string in a file

Hi folks, I am new for shell script, I hope somebody to help me to write shell script My requirement is below steps 1. I have apache access.log i.e located in /var/log/httpd/ Ex. 127.0.0.1 - - "GET... (14 Replies)
Discussion started by: Chenchireddy
14 Replies

4. UNIX for Dummies Questions & Answers

Search for string in a file then compare it with excel files entry

All, i have a file text.log: cover6 cover3 cover2 cover4 other file is abc.log as : 0 0 1 0 Then I have a excel file result.xls that contains: Name Path Pass cover2 cover3 cover6 cover4 (1 Reply)
Discussion started by: Anamika08
1 Replies

5. Shell Programming and Scripting

Search string within a file and list common words from the line having the search string

Hi, Need your help for this scripting issue I have. I am not really good at this, so seeking your help. I have a file looking similar to this: Hello, i am human and name=ABCD. How are you? Hello, i am human and name=PQRS. I am good. Hello, i am human and name=ABCD. Good bye. Hello, i... (12 Replies)
Discussion started by: royzlife
12 Replies

6. Shell Programming and Scripting

Shell script to search all entries from 1 file to all other separated files.

Hi, I am trying to create a shell script in unix platform, hence will need info on how to start and any ideas from you guys. million thx. Objective: Shell script to search all entries from 1 file(a.out) to all other files and extract the search output to 1 file (c.out). Situation, 1)... (8 Replies)
Discussion started by: Mr_47
8 Replies

7. Shell Programming and Scripting

Need to build Shell Script to search content of a text file into a folder consist several files

Have to read one file say sourcefile containing several words and having another folder containing several files. Now read the first word of Sourcefile & search it into the folder consisting sevral files, and create another file with result. We hhave to pick the filename of the file in which... (3 Replies)
Discussion started by: mukesh.baranwal
3 Replies

8. Shell Programming and Scripting

Shell Script to Search for a particular String and copy the timestamp to a variable

Hi, We Perfrom Loads to the database through a Perl script which generates a statistics file. I need to read the statistics. the Statistics file looks something like below: Process Beginning - 08-26-2010-23.41.47 DB2 CONNECTION SUCCESSFUL! Ready to process and load file: FILENAME # of... (2 Replies)
Discussion started by: Praveenkulkarni
2 Replies

9. Shell Programming and Scripting

shell script to search a string and delete the content

Hi, I've a shell script e.g. #!/bin/bash echo "Enter the next hop id" read nhid echo "enter the IP address" read IP echo "enter the interface name" read name echo "enter the enable/disable state" read state exit 0 now from this script i want to search strings in another (.cam) ... (6 Replies)
Discussion started by: vic_mnnit
6 Replies

10. UNIX for Dummies Questions & Answers

count the number of files which have a search string, but counting the file only once

I need to count the number of files which have a search string, but counting the file only once if search string is found. eg: File1: Please note that there are 2 occurances of "aaa" aaa bbb ccc aaa File2: Please note that there are 3 occurances of "aaa" aaa bbb ccc... (1 Reply)
Discussion started by: sudheshnaiyer
1 Replies
Login or Register to Ask a Question