print lines from a file containing key word


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting print lines from a file containing key word
# 1  
Old 07-07-2010
print lines from a file containing key word

i have a file containing over 1 million records,and i want to print about 300,000 line containing a some specific words.

file has content.
eg

Code:
1,rrt,234
3,fgt,678
4,crf,456
5,cde,drt
6,cfg,123

and i want to print the line with the word fgt,crf

this is just an example,my file is so huge,i have tried using
Code:
grep fgt filename

and
Code:
awk '/fgt/' filename

but it is so slow
is there a way where i can get the data faster

Last edited by vgersh99; 07-07-2010 at 03:07 PM.. Reason: code tags, please!
# 2  
Old 07-07-2010
For a sequential unsorted search, it doesn't get any faster than doing a simple pattern match. There's just no shortcut.... you could try reading the entire file into memory and searching through that... (just an idea of how you can make it faster by not using the disk).

Alternatively, you can look into a preprocessor for the file that creates indexes into the data (usually means sorting... but there are free-text indexers out there)... man are search engine like indexers... so they just are used to find full files containing matches... but look around... you never know.

Your file data looks like a spreadsheet csv... so maybe importing it as a spreadsheet might give you some advantage (??).

Just some ideas...
# 3  
Old 07-07-2010
you may get a better performance with 'fgrep' (instead of grep).
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Search for word in huge logfile and need to continue to print few lines from that line til find date

Guys i need an idea for one logic..in shell scripting am struggling with a logic...So the thing is... i need to search for a word in a huge log file and i need to continue to print few more lines from that line and the consecutive line has to end when it finds the line with date..because i know... (1 Reply)
Discussion started by: Prathi
1 Replies

2. Shell Programming and Scripting

Update a specific field in file with Variable value based on other Key Word

I have an input file with A=xyz B=pqr I would want the value in Second Field (xyz or pqr) updated with a value present in Shell Variable based on the value passed in the first field. (A or B ) while read line do NEW_VALUE = `some functionality done on $line` If $line=First Field-... (1 Reply)
Discussion started by: infernalhell
1 Replies

3. Shell Programming and Scripting

Find key pattern and print selected lines for each record

Hi, I need help on a complicated file that I am working on. I wanted to extract important info from a very huge file. It is space delimited file. I have hundred thousands of records in this file. An example content of the inputfile as below:- ## ID Ser402 Old; 23... (2 Replies)
Discussion started by: redse171
2 Replies

4. Shell Programming and Scripting

Search for a specific word and print only the word from the input file

Hi, I have a sample file as shown below, I am looking for sed or any command which prints the complete word only from the input file. Ex: $ cat "sample.log" I am searching for a word which is present in this file We can do a pattern search using grep but I need to cut only the word which... (1 Reply)
Discussion started by: mohan_kumarcs
1 Replies

5. UNIX for Dummies Questions & Answers

awk - Print lines if only matching key is found

I am looking to move matching lines (01 - 07) from File1 and 77 tab the matching string from File2, to File3.txt. I am almost done but - Currently, script is not printing lines to File3.txt in order. Thanks a lot. Any help is appreciated. Script I am using: awk 'FNR == NR && ! /^]*$/ {... (9 Replies)
Discussion started by: High-T
9 Replies

6. Shell Programming and Scripting

Shell Script @ Find a key word and If the key word matches then replace next 7 lines only

Hi All, I have a XML file which is looks like as below. <<please see the attachment >> <?xml version="1.0" encoding="UTF-8"?> <esites> <esite> <name>XXX.com</name> <storeId>10001</storeId> <module> ... (4 Replies)
Discussion started by: Rajeev_hbk
4 Replies

7. Shell Programming and Scripting

How to print few lines before and after matching word is found suing grep?

Hi, here are few lines present in the logs. I want to grep on Error and print few lines before and after Error word is found line1 Line2 Line3 Error Line4 Line5 Line6 Line7 I want the output to be Line2 Line3 Error Line5 (1 Reply)
Discussion started by: arghadeep adity
1 Replies

8. Homework & Coursework Questions

Tru to print the last name and first name using key word

I am new to shell scripting and trying to do the below Stan:Smith:Detroit:MI Jim:Jones:Farmington Hills:MI Jack:Frost:Denver:CO Sue:Apple:New York:NY Cindy:Thompson:Battle Creek:MI John:Smith:Denver:CO George:Jones:New York:NY Need to create a shell script This script will display the... (1 Reply)
Discussion started by: jakemathew
1 Replies

9. Shell Programming and Scripting

find a word and print n lines before and after the match

how to find a word and print n lines before and after the match until a blank line is encounterd (14 Replies)
Discussion started by: chidori
14 Replies

10. UNIX for Dummies Questions & Answers

Searching for key word within a file

Hello, I have a directory that holds all of my matlab codes. I am trying to run a searh on all of the matlab files that have the word "while" written inside the sytax of the code. Looking for all of the times that I did a while loop. Can someone help me do this? Thanks in advance. ... (1 Reply)
Discussion started by: moradwan
1 Replies
Login or Register to Ask a Question