Login or Register to Ask a Question and Join Our Community


UNIX for Dummies Questions & Answers

Pull out multiple lines with grep patternfile

 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Pull out multiple lines with grep patternfile
# 15  
Old 03-23-2013
Quote:
Originally Posted by FGPonce
Hi,
I ran it overnight but found this error this morning:

Code:
grep: memory exhausted

However I did go for gold and point a 350,000 pattern list at the 12 GB file.

I did a bit of playing and got 300 patterns looked for and written to file in 1 min 45s, 3000 patterns in 2 min 39, 30,000 stopped after 60 mins 0 bytes output. Best ratio was 5,000 patterns in 2 min 47 s.

Thought I'd have to dribble 5000 patterns in at a time somehow.

Then tried Dons version:

Code:
grep -Ff lertest.txt C2L4_all.bcf

Did all 350,000 patterns in about 5 minutes............nice one Don! What does the -F do?

FGPonce
The -F tells grep to match fixed strings instead of the default behavior which is to match basic regular expressions. I'm glad to hear that it worked for you.
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How do I use grep to pull incremental data and send to multiple files?

Hi Everyone, Im currently using the below code to pull data from a large CSV file and put it into smaller files with just the data associated with the number that I "grep". grep 'M053' test.csv > test053.csv Is there a way that I can use grep to run through my file like the example below... (6 Replies)
Discussion started by: TheStruggle
6 Replies

2. UNIX for Dummies Questions & Answers

Grep multiple lines

I want to grep multiple lines from a text file. I want to grep all lines containing X,Y and NA in a single command. How do I go about doing that? This is what my text files look like: rs1983866 0.0983 10 100016313 rs1983865 0.5994 X 100016339 rs1983864 0.3272 11 100017453 rs7077266... (2 Replies)
Discussion started by: evelibertine
2 Replies

3. UNIX for Dummies Questions & Answers

grep first occurrence but continue to next entry in patternfile

I have 1300 files (SearchFiles0001.txt, SearchFiles0002.txt, etc.) , each with 650,000 lines, tab-delimited data. I have a pattern file, with about 1000 lines with a single word. Each single word is found in the 1300 files once. If I grep -f PatternFile.txt SearchFiles*.txt >OutputFile.txt... (2 Replies)
Discussion started by: newhavendweeb
2 Replies

4. UNIX for Advanced & Expert Users

grep across multiple lines

How do you grep 'select * from table_name' string from a script if the select * and from table_name are on 2 different lines ? like select * from table_name Any help would be greatly appreciated !!! Thanks RDR (4 Replies)
Discussion started by: RDR
4 Replies

5. UNIX for Dummies Questions & Answers

grep in multiple lines

hi i have kind of below text in a file. I want to get a complete paragraph starting with START and ending with before another START) which has a particular string say XYZ or ABC START XYZ hshjghkjh 45 ljkfd fldjlj d jldf START 3493u ABC 454 4545454 4545454 45454 4545454 START ...... (3 Replies)
Discussion started by: reldb
3 Replies

6. UNIX for Dummies Questions & Answers

grep command to find multiple strings in multiple lines in a file.

I want to search files (basically .cc files) in /xx folder and subfolders. Those files (*.cc files) must contain #include "header.h" AND x() function. I am writing it another way to make it clear, I wanna list of *.cc files that have 'header.h' & 'x()'. They must have two strings, header.h... (2 Replies)
Discussion started by: ritikaSharma
2 Replies

7. Shell Programming and Scripting

How do you use pull data from multiple lines to do a for statement?

Guys I am having a problem with being able to find missing monitors in a configuration check script I am trying to create for accountability purposes for managing a large number of systems. What I am trying to do is run a script that will look at the raw config data in a file and pull all the pool... (7 Replies)
Discussion started by: scottzx7rr
7 Replies

8. Shell Programming and Scripting

grep multiple lines

Hi. I have this format on a textfile: VG Name /dev/vg00 PV Name /dev/dsk/c16t0d0 PV Name /dev/dsk/c18t0d0 PV Name /dev/dsk/c16t4d0 VG Name /dev/vg01 PV Name ... (6 Replies)
Discussion started by: jOOc
6 Replies

9. Shell Programming and Scripting

grep multiple lines

Hey guys: I've been meaning to post this question for awhile...it is regarding grep. Let's say for example that the following entry is in logxx: Wed Feb 2 07:44:11 <vsm> 91030 Line 5 Severity 1 Vps 6 Call Answered - DN:8753101 CLID:5164665761 PI:83 If I do a grep 91030... (27 Replies)
Discussion started by: cdunavent
27 Replies

10. Shell Programming and Scripting

Grep on multiple lines

I 'm trying to grep 2 fieldds on 2 differnt lines. Like this: psit > file egrep -e '(NS|ES)' $file. Not working. If this succeeds then run next cmd else exit. Pls Help Gundu (13 Replies)
Discussion started by: gundu
13 Replies
Login or Register to Ask a Question