Matching pattern script (sed or awk?)


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Matching pattern script (sed or awk?)
# 1  
Old 08-23-2010
Matching pattern script (sed or awk?)

Hi Guys,

I am new to the forum and to scripting so bear with me.

Thanks, Gary.

I have 3 files - file1, file2, file3

I am trying to come up with a script that will check the output of these files and if the 1st nine fields are matched in all 3 files, echo "The following string had been matched in all 3 files - "string"

i.e.

file1
Code:
WARNING: The Job Test for Scheduler abc is late. A run should have been expected by
WARNING: The Tester for Scheduler def is late. A run was expected by

file2
Code:
WARNING: The Tester for Scheduler ghi is late. A run has not completed
WARNING: The Tester for Scheduler def is late. A run should have been expected by

file3
Code:
WARNING: The Tester for Scheduler def is late. A run was expected by

So, the output in the above instance would be

The following string had been matched in all 3 files - "WARNING: The Tester for Scheduler def is late. A"

Please note that abc, def and ghi can be any number of different combinations, so long as the 1st 9 fields are matched in all 3 files I want to report it.

Last edited by Scott; 08-24-2010 at 05:54 AM.. Reason: Added code tags
# 2  
Old 08-24-2010
Code:
perl -e '$hash{`head -c 9 $_`} = 1  foreach (`ls file*`);
> ((keys %hash) ==1) ? print  "all three files are same\n" : print "all three files are not same\n";'


HTH,
PL
# 3  
Old 08-24-2010
Hi, Thanks for your reply. I ran the following command and got an error -

Code:
perl -e '$hash{`head -c 9 $_`} = 1  foreach (`ls test*`); ((keys %hash) ==1) ? print  "all three files are same\n" : print "all three files are not same\n";'
 
head: illegal option -- c
head: illegal option -- c
head: illegal option -- c

all three files are same

I tried with -n instead and it compares the 3 files fine but I am not wanting to check that the test* files are all the same. I am trying to check to see if the first 9 columns are matched in any line in all 3 files. The files themselves could differ each time. I hope that makes sense.

I can run head -c on my Linux server but not this server which is Solaris 10.

Code:
 perl -e '$hash{`head -n 9 $_`} = 1  foreach (`ls test*`); ((keys %hash) ==1) ? print  "all three files are same\n" : print "all three files are not same\n";'

all three files are same

---------- Post updated at 11:08 PM ---------- Previous update was at 09:47 AM ----------

I have found a nawk command which is near to getting what I want but still needs some tweaking. e.g.

nawk 'NR==FNR{a[$1$2$3$4$5$6$7$8$9]}{if ($1$2$3$4$5$6$7$8$9 in a) print $0}' warning1.log warning2.log warning3.log > newtest

It is checking the 1st 9 fields in warning1.log and matching any of those to the same fields in warning2.log and warning3.log, then outputting to newtest.

The problem I have is that is is outputting the complete warning1.log and appending the matched lines to the bottom. Any ideas how I would get it to only output to newtest if the 9 fields are matched in all 3 seperate warning.log files and not output the entire warning1.log file?

Thanks again.
Gary.

---------- Post updated at 11:28 PM ---------- Previous update was at 11:08 PM ----------

Sorry. I also tried the following, which outputs all matches but not the one from warning1.log in this instance using the following -

nawk 'NR==FNR{a[$1$2$3$4$5$6$7$8$9]=$1$2$3$4$5$6$7$8$9;next}a[$1$2$3$4$5$6$7$8$9]' warning1.log warning2.log warning3.log > newtest

I am also only wanting it to output one match in each of the 3 warning files so if it is matched twice in one file I only need it output the once to newtest from that file.

I am essentially going to be checking that the same 1st 9 fields were reported in all 3 warning files.

Last edited by Scott; 08-24-2010 at 05:54 AM.. Reason: Please use code tags
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk script issue redirecting to multiple files after matching pattern

Hi All I am having one awk and sed requirement for the below problem. I tried multiple options in my sed or awk and right output is not coming out. Problem Description ############################################################### I am having a big file say file having repeated... (4 Replies)
Discussion started by: kshitij
4 Replies

2. Shell Programming and Scripting

Use sed to get first matching pattern

HI, I have a file: listenerport: - 1521 - 10520 - 10521 - 10522 - 10523 instances: listenerport: listenerport: - 1521 - 10540 - 10541 - 10542 - 10543 instances: ... (5 Replies)
Discussion started by: netbanker
5 Replies

3. Shell Programming and Scripting

Sed: printing lines AFTER pattern matching EXCLUDING the line containing the pattern

'Hi I'm using the following code to extract the lines(and redirect them to a txt file) after the pattern match. But the output is inclusive of the line with pattern match. Which option is to be used to exclude the line containing the pattern? sed -n '/Conn.*User/,$p' > consumers.txt (11 Replies)
Discussion started by: essem
11 Replies

4. Shell Programming and Scripting

Replacing lines matching a multi-line pattern (sed/perl/awk)

Dear Unix Forums, I am hoping you can help me with a pattern matching problem. What am I trying to do? I want to replace multiple lines of a text file (that match a multi-line pattern) with a single line of text. These patterns can span several lines and do not always have the same number of... (10 Replies)
Discussion started by: thefang
10 Replies

5. Shell Programming and Scripting

awk pattern matching in place of sed

I have written a script to parse data from some files on a Solaris 10 system and send the output to a CSV formatted file. The code snipped i am using to pull the data is as follows.... src_line=$(sed -n "/^search_pattern$/{=;}" $file) for i in $src_line do start_line1=$(( i + 9 )) nawk -v... (4 Replies)
Discussion started by: electricheadx
4 Replies

6. Shell Programming and Scripting

Help with sed pattern matching

Hi My log file is Testtmp2 cat Testtmp2 12:12:38 12:14:29 12:17:34 12:19:08 12:20:10 12:21:35 12:22:20 12:22:26 12:22:34 12:22:38 12:28:14 12:31:35 12:32:50 12:33:04 (3 Replies)
Discussion started by: rahkumar
3 Replies

7. Shell Programming and Scripting

sed - matching pattern one but not pattern two

All, I have the following file: -------------------------------------- # # /etc/pam.d/common-password - password-related modules common to all services # # This file is included from other service-specific PAM config files, # and should contain a list of modules that define the services... (2 Replies)
Discussion started by: RobertBerrie
2 Replies

8. Shell Programming and Scripting

sed pattern matching

Hi, I have a file which contains a word like ravi and ravi30. i want to replace only the word ravi with xxx for that i am using the below sed command sed -e 's/ravi/xxx/g' . but the above command out put is xxx and xxx30 but i dont need to change ravi30 please guide me how to proceed.... (4 Replies)
Discussion started by: ravi_rn
4 Replies

9. UNIX for Dummies Questions & Answers

Pattern matching New to Sed & Awk

Hello, Despite reading the Pattern Matching chapter in the O'Reilly Sed & Awk book several times and looking at numerous examples, I cannot seem to get any kind of conditional script to work in my awk scripts! I am able to do the basic awk and grep script to capture the data but when I do with... (0 Replies)
Discussion started by: pg55
0 Replies

10. Shell Programming and Scripting

Pattern matching sed

MSG="THERE WERE XX RECORDS IN ERROR TABLE,AAAA, WHEN LOADING THE BBBB TABLE WITH EXTRACT FROM CCCC INTO TABLES FOR , DATABASE DDDD." echo "$MSG" > /tmp/mplanmsg.$$.out I wan to replace XX with the content in $recordXX cat /tmp/mplanmsg.$$.out|sed 's/XX/\$recordXX/g'| sed... (3 Replies)
Discussion started by: leemjesse
3 Replies
Login or Register to Ask a Question