Grep multiple strings in multiple files


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Grep multiple strings in multiple files
# 1  
Old 11-05-2012
Grep multiple strings in multiple files

Hi, every one!

I have a file with multiple strings.
file1
HTML Code:
ATQRGNE
ASQGVKFTE
ASSQYRDRGGLET
SPEQGARSDE
ASSRDFTDT
ASSYSGGYE
ASSYTRLWNTGE
ASQGHNTD
PSLGGGNQPQH
SLDRDSYNEQF
I want to grep each string in hundreds of files in the same directory, further, I want to find out the string which occurs in more than 50 files.

Can I use grep in awk to do that? I tried but fail to import each string into grep.

Thanks in advance!
# 2  
Old 11-05-2012
I don't recommend this approach, but I hope it works:-
Code:
grep -f search_file * | awk -F":" ' { print $NF } ' | sort | uniq -c | awk ' { if($1>=50) print; } '

Note: search_file is the file with multiple strings that you mentioned.
# 3  
Old 11-05-2012
try something like this..

It may take some time.

Code:
while read line
do
c=0
while read files
do
grep "$line" "$files" && c++
done<All_file_list
if [[ "$c" -gt 50 ]]
then
echo "$line 50 times"
fi
done<search_file

This User Gave Thanks to pamu For This Post:
# 4  
Old 11-05-2012
Quote:
Originally Posted by bipinajith
I don't recommend this approach, but I hope it works:-
Code:
grep -f search_file * | awk -F":" ' { print $NF } ' | sort | uniq -c | awk ' { if($1>=50) print; } '

Note: search_file is the file with multiple strings that you mentioned.
Thanks!

Yes, this approach might not be efficiency. there are millions of strings in my file and hundreds of files to be searched. Do you have any other smart suggestions?

Thank you very much.
# 5  
Old 11-05-2012
I am not sure if I can recommend another smart approach, but I believe my approach will run much faster than pamu's script since he/she is using looping structures.

Did you try running both & check if you are getting the desired results?
This User Gave Thanks to Yoda For This Post:
# 6  
Old 11-05-2012
Quote:
Originally Posted by bipinajith
I am not sure if I can recommend another smart approach, but I believe my approach will run much faster than pamu's script since he/she is using looping structures.

Did you try running both & check if you are getting the desired results?
Yes, I'm working on that. Thank you. I will make a response later.
# 7  
Old 11-05-2012
Code:
grep -f search_file * | awk -F: '{a[$2]++} END {for (i in a) if (a[i]>=50) print i}'

This User Gave Thanks to vgersh99 For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

How to pass strings from a list of strings from another file and create multiple files?

Hello Everyone , Iam a newbie to shell programming and iam reaching out if anyone can help in this :- I have two files 1) Insert.txt 2) partition_list.txt insert.txt looks like this :- insert into emp1 partition (partition_name) (a1, b2, c4, s6, d8) select a1, b2, c4, (2 Replies)
Discussion started by: nubie2linux
2 Replies

2. Shell Programming and Scripting

Grep strings on multiple files and output to multiple files

Hi All, I want to use egrep on multiple files and the results should be output to multiple files. I am using the below code in my shell script(working in Ksh shell). However with this code I am not attaining the desired results. #!/bin/ksh ( a="/path/file1" b="path/file2" for file in... (4 Replies)
Discussion started by: am24
4 Replies

3. Shell Programming and Scripting

Grep and replace multiple strings in a file with multiple filenames in a file

Hi, I have a file containing list of strings like i: Pink Yellow Green and I have file having list of file names in a directory j : a b c d Where j contains of a ,b,c,d are as follows a: Pink (3 Replies)
Discussion started by: madabhg
3 Replies

4. Shell Programming and Scripting

Can't grep multiple strings

I have a script that periodically checks the Apache error_log to search for a specific error that causes it to hand and, if found, it restarts the service. I recently found another error that forces it to hand and won't serve pages until it is reset. What I'm trying to do is to get the script to... (3 Replies)
Discussion started by: cfjohnsn
3 Replies

5. Shell Programming and Scripting

Search & Replace: Multiple Strings / Multiple Files

I have a list of files all over a file system e.g. /home/1/foo/bar.x /www/sites/moose/foo.txtI'm looking for strings in these files and want to replace each occurrence with a replacement string, e.g. if I find: '#@!^\&@ in any of the files I want to replace it with: 655#@11, etc. There... (2 Replies)
Discussion started by: spacegoose
2 Replies

6. UNIX Desktop Questions & Answers

How do you [e]grep for multiple values within multiple files?

Hi I'm sure there's a way to do this, but I ran out of caffeine/talent before getting the answer in a long winded alternate way (don't ask ;) ) The task I was trying to do was scan a directory of files and show only files that contained 3 values: I940 5433309 2181 I tried many variations... (4 Replies)
Discussion started by: callumw
4 Replies

7. UNIX for Dummies Questions & Answers

Grep multiple strings in multiple files using single command

Hi, I will use below command for grep single string ("osuser" is search string) ex: find . -type f | xarg grep -il osuser but i have one more string "v$session" here i want to grep in which file these two strings are present. any help is appreciated, Thanks in advance. Gagan (2 Replies)
Discussion started by: gagan4599
2 Replies

8. UNIX for Dummies Questions & Answers

grep command to find multiple strings in multiple lines in a file.

I want to search files (basically .cc files) in /xx folder and subfolders. Those files (*.cc files) must contain #include "header.h" AND x() function. I am writing it another way to make it clear, I wanna list of *.cc files that have 'header.h' & 'x()'. They must have two strings, header.h... (2 Replies)
Discussion started by: ritikaSharma
2 Replies

9. UNIX for Dummies Questions & Answers

best method of replacing multiple strings in multiple files - sed or awk? most simple preferred :)

Hi guys, say I have a few files in a directory (58 text files or somthing) each one contains mulitple strings that I wish to replace with other strings so in these 58 files I'm looking for say the following strings: JAM (replace with BUTTER) BREAD (replace with CRACKER) SCOOP (replace... (19 Replies)
Discussion started by: rich@ardz
19 Replies

10. Shell Programming and Scripting

Grep Multiple Strings

Hi, Can any one pelase tell me how to grep multiple strings from multiple files in a singel folder? grep -E "string1|string2|string3|string4|string..." its taking lots of time.. can any please tell me fast grep??? URGENT (10 Replies)
Discussion started by: durgaprasad
10 Replies
Login or Register to Ask a Question

Featured Tech Videos