Searching across multiple files if pattern is available in all files searched


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Searching across multiple files if pattern is available in all files searched
# 1  
Old 02-12-2011
Searching across multiple files if pattern is available in all files searched

I have a list of pattern in a file, I want each of these pattern been searched from 4 files. I was wondering this can be done in SED / AWK.

say my 4 files to be searched are

Code:
> cat f1

abc/x(12) 1  
abc/x     3 
cde       2  
zzz       3  
fdf       4

> cat f2
fdf 4 
cde 3 
abc 2 
zzz 1 

> cat f3
cde 3  
abc 2  
fdf 1  
zzz 4  

> cat f4
zzz 3
fdf 4
abc 1
cde 2

I have file which has pattern to be search in all these 4 files. I want the output only when the search pattern is available in all the 4 files. Also I would like to have file written out patterns which is not present in all 4 files.

Code:
> cat input_pattern_file
cde
fdf
abc
zzz

Thanks.
# 2  
Old 02-12-2011
so what have you tried?
# 3  
Old 02-12-2011
I tried ... It doesn't seem to work

Code:
#!/bin/csh
foreach x ( `cat input_pattern_file` )
sed -n '/${x}/p' f1 >> out
# grep -w "$x" f1 
end

Thanks.
# 4  
Old 02-13-2011
Code:
awk 'NR==FNR{a[$1];next} $1 in a {c[$1]++}
     END{for ( i in c) if (c[i]==4) print i}' input_pattern_file f1 f2 f3 f4 > available_pattern

Code:
grep -v -f available_pattern input_pattern_file > non_pattern

# 5  
Old 02-13-2011
In the example input files I have mentioned we have "zzz" in all files, but with the given awk it is not getting printed out. Also I would need to print the values of 2nd column for the matching inputs from each of the 4 files.


Code:
> awk 'NR==FNR{a[$1];next} $1 in a {c[$1]++}  END {for ( i in c) if (c[i]==4) print i}' list f1 f2 f3 f4
cde
fdf



Thanks.
# 6  
Old 02-13-2011
I still get zzz line.
Code:
awk 'NR==FNR{a[$1];next} $1 in a {c[$1]++}
     END{for ( i in c) if (c[i]==4) print i}' input_pattern_file f1 f2 f3 f4

cde
zzz
fdf

if you need 2nd column, add one more command

Code:
grep -f available_pattern f1 f2 f3 f4

# 7  
Old 02-13-2011
MySQL

I can see it working ...

Is there a way I can publish missing cases from each of the file using the existing awk

Also do we have "SED" equivalent of "grep -f" as I need to process few millions of lines and SED is really quick against grep.

Thanks.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. BSD

Searching in multiple files

I am new to unix and I would like to search multiple log files to find earliest occurrence of a text. Ex: Say I have 10 logs file each ending with .log and I want to find the text “CustomeError” . I want to find the which log file “CustomeError” comes first and lines which surround’s ... (4 Replies)
Discussion started by: jim john
4 Replies

2. UNIX for Dummies Questions & Answers

Searching a pattern in .gz files

in a directory, I have some files(comma seperated) and some .gz files (each .gz file contain one file which again is comma seperated). I want to search and write the names of all those files which contain any particular value (say 1150) at any specified field position(say 10th field). How di I do... (1 Reply)
Discussion started by: Kumar Jivi
1 Replies

3. Red Hat

Moving of file content to another two files after searching with specific pattern

Hello, Please help me with this!! Thanks in advance!! I have a file named file.gc with the content: 1-- Mon Sep 10 08:53:09 CDT 2012 2revoke connect from FR2261; 3delete from mkt_allow where grantee = 'FR2261'; 4grant connect to FR2261 with '******'; 5alter user FR2261 comment... (0 Replies)
Discussion started by: raosr020
0 Replies

4. Shell Programming and Scripting

Searching a particular string pattern in 10000 files

Problem Statement:- I need to search a particular `String Pattern` in around `10000 files` and find the records which contains that `particular pattern`. I can use `grep` here, but it is taking lots of time. Below is the command I am using to search a `particular string pattern` after... (3 Replies)
Discussion started by: raihan26
3 Replies

5. Shell Programming and Scripting

Script for searching a pattern in 5 files and deleting the patterns given

Hi All, I have written the below script that searches for the pattern in a file and delete them if present. please can some one have a look and suggest the changes in the script. #!bin/sh # The pattern that user want to add to the files echo "Enter the pattern of the redirect" read... (4 Replies)
Discussion started by: Shazin
4 Replies

6. Shell Programming and Scripting

Searching all files that contain pattern

Hello All, i have to search a pattern in all the files in all subfolders that are present in current directory. suppose i am in d1 directory and in that sd1,sd2,sd3 are subdirectories. in sd1 i have files f1,f2 sd2 i have files f3,f4 sd3 i have file f5 i have to list out all those... (4 Replies)
Discussion started by: ravi.sadani19
4 Replies

7. UNIX for Dummies Questions & Answers

Searching for files with certain string pattern

Hello All I would like to search for files containing certain string pattern under all the directories under /vobs/vobname and print the output to a file in my home directory. How can I do this? Note: /vobs/vobname conatins several directories. Thank You in advance newbetounix (1 Reply)
Discussion started by: intrigue
1 Replies

8. Shell Programming and Scripting

Pattern searching pattern in c files

I have a problem in searching a specific pattern in c files. My requirement: I have to find all the division operator in all cfiles. The problem is, the multi line comments and single line comments will also have forward slash in it. Even after avoiding these comments also, if both... (6 Replies)
Discussion started by: murthybptl
6 Replies

9. UNIX for Dummies Questions & Answers

Pattern searching inside Variable - not looking at files

Hi, I've searched this site and not found this already, so if I missed on my search, sorry. I need to pass in a variable to a script, where the first three characters of that variable represent a calendar quarter, and the last 2 characters are the year. I.E. Q0105 for Q1, Q0205 for Q2, and... (3 Replies)
Discussion started by: Rediranch
3 Replies

10. Shell Programming and Scripting

Searching multiple files with multiple expressions

I am using a DEC ALPHA running Digital UNIX (formly DEC OSF/1) and ksh. I have a directory with hundreds of files that only share the extension .rpt. I would like to search that directory based on serial number and operation number and only files that meet both requirements to be printed out. I... (6 Replies)
Discussion started by: Anahka
6 Replies
Login or Register to Ask a Question