Speeding/Optimizing GREP search on CSV files Post: 302450972

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Speeding up a Shell Script (find, grep and a for loop)

Hi all, I'm having some trouble with a shell script that I have put together to search our web pages for links to PDFs. The first thing I did was: ls -R | grep .pdf > /tmp/dave_pdfs.outWhich generates a list of all of the PDFs on the server. For the sake of arguement, say it looks like...

2. Shell Programming and Scripting

grep'ing and sed'ing chunks in bash... need help on speeding up a log parser.

I have a file that is 20 - 80+ MB in size that is a certain type of log file. It logs one of our processes and this process is multi-threaded. Therefore the log file is kind of a mess. Here's an example: The logfile looks like: "DATE TIME - THREAD ID - Details", and a new file is created...

3. UNIX for Dummies Questions & Answers

Using grep to search within files

Hi, At my company, we have custom web sites that we create for different clients. The folder structure is something like: <project name>/html/web/custom/ The custom folder contains a file called "category.html" Every project has the same folder structure, and same file names but, the data...

4. UNIX for Dummies Questions & Answers

Reading compressed files during a grep search

All, The bottom line is that im reading a file, storing it as variables, recursively grep searching it, and then piping it to allow word counts as well. I am unsure on how to open any .zip .tar and .gzip, search for keywords and return results. Any help would be much appreciated! Thanks

5. UNIX for Dummies Questions & Answers

pattern search using grep in specific range of files

Hi, I am trying to do the following: grep -l <pattern> <files to be searched for> In <files to be searched for> , all files should of some specific date like "Apr 8" not all files in current directory. I just to search within files Apr 8 files so that it won't search in entire list of...

6. Shell Programming and Scripting

Perl search csv fileA where two strings exist on another csv fileB

Hi I have two csv files, with the following formats: FileA.log: Application, This occured blah Application, That occured blah Application, Also this AnotherLog, Bob did this AnotherLog, Dave did that FileB.log: Uk, London, Application, datetime, LaterDateTime, Today it had'nt...

7. Shell Programming and Scripting

Speeding up search and replace in a for loop

Hello, I am using sed in a for loop to replace text in a 100MB file. I have about 55,000 entries to convert in a csv file with two entries per line. The following script works to search file.txt for the first field from conversion.csv and then replace it with the second field. While it works fine,...

8. Shell Programming and Scripting

awk read column csv and search in other csv

hi, someone to know how can i read a specific column of csv file and search the value in other csv columns if exist the value in the second csv copy entire row with all field in a new csv file. i suppose that its possible using awk but i m not expertise thanks in advance

9. Shell Programming and Scripting

Optimizing search using grep

I have a huge log file close to 3GB in size. My task is to generate some reporting based on # of times something is being logged. I need to find the number of time StringA , StringB , StringC is being called separately. What I am doing right now is: grep "StringA" server.log | wc -l...

10. Shell Programming and Scripting

Speeding up shell script with grep

HI Guys hoping some one can help I have two files on both containing uk phone numbers master is a file which has been collated over a few years ad currently contains around 4 million numbers new is a file which also contains 4 million number i need to split new nto two separate files...

LEARN ABOUT OPENDARWIN

zgrep

ZGREP(1)						      General Commands Manual							  ZGREP(1)

NAME

       zgrep - search possibly compressed files for a regular expression

SYNOPSIS

       zgrep [ grep_options ] [ -e ] pattern filename...

DESCRIPTION

       Zgrep  is used to invoke the grep on compress'ed or gzip'ed files. All options specified are passed directly to grep.  If no file is speci-
       fied, then the standard input is decompressed if necessary and fed to grep.  Otherwise the given files are uncompressed	if  necessary  and
       fed to grep.

       If  zgrep  is invoked as zegrep or zfgrep then egrep or fgrep is used instead of grep.  If the GREP environment variable is set, zgrep uses
       it as the grep program to be invoked. For example:

	   for sh:  GREP=fgrep	zgrep string files
	   for csh: (setenv GREP fgrep; zgrep string files)

AUTHOR

       Charles Levert (charles@comm.polymtl.ca)

SEE ALSO

       grep(1), egrep(1), fgrep(1), zdiff(1), zmore(1), znew(1), zforce(1), gzip(1), gzexe(1)

																	  ZGREP(1)

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Speeding up a Shell Script (find, grep and a for loop)

Discussion started by: Dave Stockdale

2. Shell Programming and Scripting

grep'ing and sed'ing chunks in bash... need help on speeding up a log parser.

Discussion started by: elinenbe

3. UNIX for Dummies Questions & Answers

Using grep to search within files

Discussion started by: miklo

4. UNIX for Dummies Questions & Answers

Reading compressed files during a grep search

Discussion started by: ryan.lee