Best way to search for patterns in huge text files Post: 302385616

10 More Discussions You Might Find Interesting

1. Solaris

Huge (repeated Entry) text files

Somebody HELP! I have a huge log file (TEXT) 76298035 bytes. It's a logfile of IMEIs and IMSIS that I get from my EIR node. Here is how the contents of the file look like: 000000, 1 33016382000913 652020100423994 1 33016382002353 652020100430743 1 33017035101003 652020100441736...

2. UNIX Desktop Questions & Answers

how to search files efficiently using patterns

hi friens, :) if i need to find files with extension .c++,.C++,.cpp,.Cpp,.CPp,.cPP,.CpP,.cpP,.c,.C wat is the pattern for finding them :confused:

3. Shell Programming and Scripting

Perl - How to search a text file with multiple patterns?

Good day, great gurus, I'm new to Perl, and programming in general. I'm trying to retrieve a column of data from my text file which spans a non-specific number of lines. So I did a regexp that will pick out the columns. However,my pattern would vary. I tried using a foreach loop unsuccessfully....

4. UNIX for Dummies Questions & Answers

script to search patterns inside list of files

>testfile while read x do if then echo $x >> testfile else fi if then echo $x >> testfile else fi done < list_of_files is there any efficient way to search abc.dml and xyz.dml ?

5. Shell Programming and Scripting

to read two files, search for patterns and store the output in third file

hello i have two files temp.txt and temp_unique.text the second file consists the unique fields from the temp.txt file the strings stored are in the following form 4,4 17,12 15,65 4,4 14,41 15,65 65,89 1254,1298i'm able to run the following script to get the total count of a...

6. SuSE

Search all files based on first and in all listed files search the second patterns

Hello Linux Masters, I am not a linux expert therefore i need help from linux gurus. Well i have a requirement where i need to search all files based on first patterns and after seraching all files then serach second pattern in all files which i have extracted based on first pattern....

7. Shell Programming and Scripting

Comparing 2 huge text files

I have this 2 files: k5login sanwar@systems.nyfix.com jjamnik@systems.nyfix.com nisha@SYSTEMS.NYFIX.COM rdpena@SYSTEMS.NYFIX.COM service/backups-ora@SYSTEMS.NYFIX.COM ivanr@SYSTEMS.NYFIX.COM nasapova@SYSTEMS.NYFIX.COM tpulay@SYSTEMS.NYFIX.COM rsueno@SYSTEMS.NYFIX.COM...

8. Shell Programming and Scripting

How to fix line breaks format text for huge files?

Hi, I need to correct line breaks for huge files (more than 1MM records in a file) and then format it properly. Except the header and trailer, each record starts with 'D'. Requirement:Scan the whole file except the header and trailer records and see if any of the records start with...

9. Shell Programming and Scripting

Search for patterns in thousands of files

Hi All, I want to search for a certain string in thousands of files and these files are distributed over different directories created daily. For that I created a small script in bash but while running it I am getting the below error: /ms.sh: xrealloc: subst.c:5173: cannot allocate...

10. Shell Programming and Scripting

Search and replace ---A huge number of files

Hello Friends, I have the below scenario in my current project. Suggest me which tool ( perl,python etc) is best to this scenario. Or should I go for Programming language ( C/Java ).. (1) I will be having a very big file ( information about 200million subscribers will be stored in it ). This...

LEARN ABOUT DEBIAN

qsgrep

QSGREP(1)							  qsgrep man page							 QSGREP(1)

NAME

       qsgrep - Print matching patterns within a file.

SYNOPSIS

       qsgrep -e <pattern> -o <sub string> [<path>]

DESCRIPTION

       qsgrep  is  a  simple tool to search patterns within files. It uses regular expressions to find patterns and prints the submatches within a
       pre-defined format string.

OPTIONS

       -e <pattern>
	      Specifes the search pattern.

       -o <string>
	      Defines the output string where $0-$9 are substituted by the submatches of the regular expression.

       <path> Defines the input file to process. qsgrep reads from from standard input if this parameter is omitted.

EXAMPLE

       Shows the IP addresses of clients causing mod_qos(031) messages):

	 qsgrep -e 'mod_qos(031).*, c=([0-9.]*)' -o 'ip=$1' error_log

SEE ALSO

       qsexec(1), qsfilter2(1), qsgeo(1), qslog(1), qslogger(1), qspng(1), qsrotate(1), qssign(1), qstail(1)

AUTHOR

       Pascal Buchbinder, http://opensource.adnovum.ch/mod_qos/

mod_qos utilities 10.8						     June 2012								 QSGREP(1)

10 More Discussions You Might Find Interesting

1. Solaris

Huge (repeated Entry) text files

Discussion started by: axl

2. UNIX Desktop Questions & Answers

how to search files efficiently using patterns

Discussion started by: arunsubbhian

3. Shell Programming and Scripting

Perl - How to search a text file with multiple patterns?

Discussion started by: Sp3ck

4. UNIX for Dummies Questions & Answers

script to search patterns inside list of files

Discussion started by: dr46014