Script to search a large file with a list of terms in another file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Script to search a large file with a list of terms in another file
# 1  
Old 06-29-2010
Script to search a large file with a list of terms in another file

Hi-

I am trying to search a large file with a number of different search terms that are listed one per line in 3 different files. Most importantly I need to be able to do a case insensitive search.
I have tried just using egrep -f but it doesn't seam to be able to handle the -i option when selecting a file for the search terms.

Any suggestions?
# 2  
Old 06-29-2010
the following command should work:

egrep -i -f patterns file

do you get an error? or why do you say it doesn't handle the "-i"?
# 3  
Old 06-29-2010
Because using that command turns up no results when terms in the patterns file are lowercase and instances in the searched file are not.

---------- Post updated at 02:03 PM ---------- Previous update was at 01:25 PM ----------

I guess I am looking for the proper way to read in a list of search terms from a file or files and then use them one by one to grep a large file. That might allow me some flexibility to find instances in the searched file that match relatively closely. Either because of case, or different spacing for multiple word strings.
Using egrep -i -f seems to get me an all or nothing result.

Thanks for any help.
# 4  
Old 06-29-2010
Hi.


The egrep -i -f pattern-file data-file worked for me.

Please post short representative samples of the data, patterns, and expected output that you want to see ... cheers, drl
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Search spaces in 5th column in large file

i have a file having 5 columns with more than million records. And i want to search using UNIX command to find if there are any spaces in 5th column. any please help. (1 Reply)
Discussion started by: sivakumar.p
1 Replies

2. UNIX for Dummies Questions & Answers

Search file and print everything except multiple search terms

I'm trying to find a way to search a range of similar words in a file. I tried using sed but can't get it right:sed 's/\(ca01\)*//'It only removes "ca01" but leaves the rest of the word. I still want the rest of the information on the lines just not these specific words listed below. Any... (3 Replies)
Discussion started by: seekryts15
3 Replies

3. UNIX for Dummies Questions & Answers

Grep -B used with -f? (Searching a file using a list of terms, output is lines before each match)

(1 Reply)
Discussion started by: Twinklefingers
1 Replies

4. UNIX for Dummies Questions & Answers

Grep? - using a file of terms to search another file when the information is on a different line

I have a flat file that looks like this, let's call it Chromosome_9.txt: FT /Gene_Name="Guanyl-Acetylase 9" FT /Gene_Number"36952" FT /Gene_Name="Endoplasmic Luciferase" FT /Gene_Number"36953" FT ... (4 Replies)
Discussion started by: Twinklefingers
4 Replies

5. Shell Programming and Scripting

Removing duplicate terms in a file

Hi everybody I have a .txt file that contains some assembly code for optimizing it i need to remove some replicated parts. for example I have:e_li r0,-1 e_li r25,-1 e_lis r25,0000 add r31, r31 ,r0 e_li r28,-1 e_lis r28,0000 add r31, r31 ,r0 e_li r28,-1 ... (3 Replies)
Discussion started by: Behrouzx77
3 Replies

6. Shell Programming and Scripting

Script to sort large file with frequency

Hello, I have a very large file of around 2 million records which has the following structure: I have used the standard awk program to sort: # wordfreq.awk --- print list of word frequencies { # remove punctuation #gsub(/_]/, "", $0) for (i = 1; i <= NF; i++) freq++ } END { for (word... (3 Replies)
Discussion started by: gimley
3 Replies

7. Shell Programming and Scripting

Grep multiple search terms with context

I have a file that is a sort library in the format: ##def title1 content1 stuff1 content2 stuff2 ##enddef ##def title2 etc.. I want to grep def and content and pull some trailing context from content so the result would look something like: (1 Reply)
Discussion started by: Moe.Wilensky
1 Replies

8. Shell Programming and Scripting

Take a list if strings from a file and search them in a list of files and report them

I have a file 1.txt with the below contents. -----cat 1.txt----- 1234 5678 1256 1234 1247 ------------------- I have 3 more files in a folder -----ls -lrt------- A1.txt A2.txt A3.txt ------------------- The contents of those three files are similar format with different data values... (8 Replies)
Discussion started by: realspirituals
8 Replies

9. Shell Programming and Scripting

Search for multiple lines in large file

Hi, I have a requirement to search for a string in a large log file along with few lines before and after the the string. The following script was sufficient to search such an entry. STRING_TO_GREP="$1" FILE_TO_GREP="$2" NUMBER_OF_LINES_BEFORE=$3 NUMBER_OF_LINES_AFTER=$4 for i in `grep... (3 Replies)
Discussion started by: praveen123
3 Replies

10. UNIX for Dummies Questions & Answers

Large file search

How do I search through a file that is so large (over 1GB) and find out exactly on what the exact pattern is found. I've used grep and know the line exist but what it returns is not complete because there are blobs in the data and there are control characters, etc and part of the data may beyond... (2 Replies)
Discussion started by: giannicello
2 Replies
Login or Register to Ask a Question