Sponsored Content
Top Forums UNIX for Advanced & Expert Users Best way to search for patterns in huge text files Post 302385625 by Scott on Friday 8th of January 2010 05:31:48 PM
Old 01-08-2010
Hi.

Is:
Code:
grep -f pattern_file input_file[123] > output_fileX

quicker?

Also, please say WHAT doesn't work with the awk solution.

Like radoulov, I don't know what you mean by "all patterns".

Last edited by Scott; 01-08-2010 at 06:41 PM..
 

10 More Discussions You Might Find Interesting

1. Solaris

Huge (repeated Entry) text files

Somebody HELP! I have a huge log file (TEXT) 76298035 bytes. It's a logfile of IMEIs and IMSIS that I get from my EIR node. Here is how the contents of the file look like: 000000, 1 33016382000913 652020100423994 1 33016382002353 652020100430743 1 33017035101003 652020100441736... (4 Replies)
Discussion started by: axl
4 Replies

2. UNIX Desktop Questions & Answers

how to search files efficiently using patterns

hi friens, :) if i need to find files with extension .c++,.C++,.cpp,.Cpp,.CPp,.cPP,.CpP,.cpP,.c,.C wat is the pattern for finding them :confused: (2 Replies)
Discussion started by: arunsubbhian
2 Replies

3. Shell Programming and Scripting

Perl - How to search a text file with multiple patterns?

Good day, great gurus, I'm new to Perl, and programming in general. I'm trying to retrieve a column of data from my text file which spans a non-specific number of lines. So I did a regexp that will pick out the columns. However,my pattern would vary. I tried using a foreach loop unsuccessfully.... (2 Replies)
Discussion started by: Sp3ck
2 Replies

4. UNIX for Dummies Questions & Answers

script to search patterns inside list of files

>testfile while read x do if then echo $x >> testfile else fi if then echo $x >> testfile else fi done < list_of_files is there any efficient way to search abc.dml and xyz.dml ? (2 Replies)
Discussion started by: dr46014
2 Replies

5. Shell Programming and Scripting

to read two files, search for patterns and store the output in third file

hello i have two files temp.txt and temp_unique.text the second file consists the unique fields from the temp.txt file the strings stored are in the following form 4,4 17,12 15,65 4,4 14,41 15,65 65,89 1254,1298i'm able to run the following script to get the total count of a... (3 Replies)
Discussion started by: vaibhavkorde
3 Replies

6. SuSE

Search all files based on first and in all listed files search the second patterns

Hello Linux Masters, I am not a linux expert therefore i need help from linux gurus. Well i have a requirement where i need to search all files based on first patterns and after seraching all files then serach second pattern in all files which i have extracted based on first pattern.... (1 Reply)
Discussion started by: Black-Linux
1 Replies

7. Shell Programming and Scripting

Comparing 2 huge text files

I have this 2 files: k5login sanwar@systems.nyfix.com jjamnik@systems.nyfix.com nisha@SYSTEMS.NYFIX.COM rdpena@SYSTEMS.NYFIX.COM service/backups-ora@SYSTEMS.NYFIX.COM ivanr@SYSTEMS.NYFIX.COM nasapova@SYSTEMS.NYFIX.COM tpulay@SYSTEMS.NYFIX.COM rsueno@SYSTEMS.NYFIX.COM... (11 Replies)
Discussion started by: linuxgeek
11 Replies

8. Shell Programming and Scripting

How to fix line breaks format text for huge files?

Hi, I need to correct line breaks for huge files (more than 1MM records in a file) and then format it properly. Except the header and trailer, each record starts with 'D'. Requirement:Scan the whole file except the header and trailer records and see if any of the records start with... (19 Replies)
Discussion started by: kikionline
19 Replies

9. Shell Programming and Scripting

Search for patterns in thousands of files

Hi All, I want to search for a certain string in thousands of files and these files are distributed over different directories created daily. For that I created a small script in bash but while running it I am getting the below error: /ms.sh: xrealloc: subst.c:5173: cannot allocate... (17 Replies)
Discussion started by: danish0909
17 Replies

10. Shell Programming and Scripting

Search and replace ---A huge number of files

Hello Friends, I have the below scenario in my current project. Suggest me which tool ( perl,python etc) is best to this scenario. Or should I go for Programming language ( C/Java ).. (1) I will be having a very big file ( information about 200million subscribers will be stored in it ). This... (5 Replies)
Discussion started by: panyam
5 Replies
CHDB(7) 						 Miscellaneous Information Manual						   CHDB(7)

NAME
chdb - chess database browser SYNOPSIS
chdb [ file ... ] DESCRIPTION
Chdb reads the given files of chess games (hist by default) and accepts commands to search, play through, and display the games in these files. If x is the name of the file, it is looked for under the names x, x.m.out, and /lib/chess/x.m.out. After reading the files, chdb displays a chess board, a text window with a command line, and vertical and horizontal scroll bars along the edges of the board. The text window holds 6 lines of information about the game and position and, at the bottom, a command line in which to type. The displayed text contains chess symbols--use a suitable font. The vertical scroll bar scans through the games to select a game; the horizontal bar then scans through the moves of the selected game. Both scroll bars use button 1 to scan backwards, button 3 to scan forwards, and button 2 to jump to an absolute position. With buttons 1 and 3, the scroll bars are calibrated in units of 1 through 8 corresponding to the ranks and files of the board. The units measure games on the vertical scroll bar, ply (half-moves) on the horizontal scroll bar. For example, clicking button 3 on the horizontal scroll bar under the d file steps through the game two full moves. Moves may be made on the chess board by pointing with button 1. There are two methods to point at moves. For the first method, point at the piece to be moved, press button 1, point to the place to move that piece, and release button 1. In the second method, point at the place where a piece is to move and press button 1. The smallest/least-central piece that can move there is highlighted. Releasing button 1 without moving the mouse will make the highlighted move. Moving the mouse to the desired piece and releasing the button will move the selected piece. Typed lines of text are echoed in the command line and executed. The available commands are: f n n Set the format for display of moves in the text window. The first number is the verbosity, with 0 minimal. The second number is 0 for algebraic, 1 for English, and 2 for figurine (default). g n Go to the game with ordinal number n in the input files. If n is prefixed with a + or -, it is interpreted as a relative position in the current set of games (see below). N defaults to +1. p n Go to whole move n in the current game. If n is prefixed with a + or -, it is interpreted as an offset in ply from the current position. N defaults to +1. k n Mark the current set of games with tag n (see patterns, below). w type file Write the current set of games to file. Type is either a (write the games as text) or m (write the games in binary format suitable for chdb). Patterns select subsets of the games. A pattern is one of the following, in decreasing precedence order. Parentheses can be used for grouping. . The current game. * All games originally read. 'n The games previously marked with a k command with the same n. [] [number] All games in * that contain the positions that can be reached in the specified number of plies from the current position. A missing number is the same as zero, meaning just the current position. Positions are matched with black/white transpositions. /regular expression/fields This pattern matches the regular expression against the various text windows. Fields is a list of characters from the set abdefo- prw. A is for all, b for black, d for date, e for event, f for file, o for opening, p for person (white and black), r for result, and w for white. If multiple fields are given, the expression is matched on the union of the specified fields. If no field is given, p is assumed. !pattern The set subtraction of * and the given pattern. pattern-pattern The set subtraction of the given patterns. pattern+pattern pattern|pattern The set union of the given patterns. pattern pattern pattern&pattern The set intersection of the given patterns. +pattern -pattern &pattern |pattern These patterns have the current set of games as an implied first operand. EXAMPLE
Select games that Botvinnik lost: (/Botv/w/0-1/r)|(/Botv/b/1-0/r) FILES
/lib/chess directory of databases. SOURCE
/sys/src/cmd/chdb SEE ALSO
regexp(6). BUGS
The browser is only a prototype. Most of the databases are protected by copyright and not distributed. CHDB(7)
All times are GMT -4. The time now is 09:56 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy