11-29-2011
Find records with matching patterns
Hi,
I need to find records with a search string from a file. Search strings are provided in a file.
For eg. search_String.txt file is like below
chicago
mexico
newark
sanhose
and the file from where the records need to be fetched is given below
src_file:
xxx|chicago|12/31/2011|abc@xyz.com|installed
yyy|sanhose|12/30/2011|yyy@xyz.com|deleted
zzz|newark|11/30/2011|zzz@xyz.com|installed
ccc|denver|12/31/2011|ccc@xyz.com|installed
I have written a code as below using grep
#!/usr/bin/ksh
LIST=search_String.txt
while read line
do
grep ${line} src_file >> OUTPUT.TXT
done < ${LIST}
exit 0
This code works. But the problem am facing is my src_file is a very huge file having around 23 million records and I need to search for 1200 strings from this src_file and get the records from src_file.
Given this situation my code runs for hours.
Could anyone of you help me to understand if there is a better way for this search?
Thanks
10 More Discussions You Might Find Interesting
1. UNIX for Dummies Questions & Answers
IN a file I need to check for 3 patterns if all the 3 patterns are in the file. I need to send out an email. All this needs to be done in korn shell script. Please advise. (1 Reply)
Discussion started by: saibsk
1 Replies
2. Shell Programming and Scripting
In a directory, there are two different file extensions (*.txt and *.xyz) having similar names of numerical strings (*). The (*.txt) contains 5000 multiple files and the (*.xyz) also contains 5000 multiple files. Each of the files has around 4000 rows and 8 columns, with several unique string... (5 Replies)
Discussion started by: asanjuan
5 Replies
3. Shell Programming and Scripting
I have a file name in $f. If $f has "-" at the beginning, or "=", or does not have extension ".ry" or ".xt" or ".dat" then cerr would not be empty.
Tried the following but having some problems.
set cerr = `echo $f | awk '/^-|=|!.ry|!.xt|!.dat/'` (4 Replies)
Discussion started by: kristinu
4 Replies
4. Shell Programming and Scripting
file contents looks like this :
#START
line1 of record1
line2 of record1
#END
#START
line1 of record2
line2 of record2
line3 of record2
#END
#START
line1 of record3
#END
my question how should i make it a records between #START and #END .
willl i be able to get the contents of the... (5 Replies)
Discussion started by: sathish92
5 Replies
5. Shell Programming and Scripting
Hi,
I am trying to extract some patterns from a line. The input file is space delimited and i could not use column to get value after "IN" or "OUT" patterns as there could be multiple white spaces before the next digits that i need to print in the output file . I need to print 3 patterns in a... (3 Replies)
Discussion started by: redse171
3 Replies
6. Shell Programming and Scripting
Hi,
I have requirement to find the matching patterns of two files in Unix. One file is the log file and the other is the error list file. If any pattern in the log file matches the list of errors in the error list file, then I would need to find the counts of the match.
For example,
... (5 Replies)
Discussion started by: Bobby_2000
5 Replies
7. Shell Programming and Scripting
Hi,
I have multiple files in my log folder. e.g:
a_m1.log
b_1.log
c_1.log
d_1.log
b_2.log
c_2.log
d_2.log
e_m1.log
a_m2.log
e_m2.log
I need to keep latest 10 instances of each file.
I can write multiple find commands but looking if it is possible in one line.
m file are monthly... (4 Replies)
Discussion started by: wahi80
4 Replies
8. Shell Programming and Scripting
Hello.
For a given folder, I want to select any files find $PATH1 -f \( -name "*" but omit any files like pattern name ! -iname "*.jpg" ! -iname "*.xsession*" ..... \) and also omit any subfolder like pattern name -type d \( -name "/etc/gconf/gconf.*" -o -name "*cache*" -o -name "*Cache*" -o... (2 Replies)
Discussion started by: jcdole
2 Replies
9. Shell Programming and Scripting
Delete patterns matching
OS version: RHEL 7.3
Shell : Bash
I have a file like below (pattern.txt). I need to delete all lines starting with the following words (words separated by comma below) and ) character.
LOGGING, NOCOMPRESS, TABLESPACE , PCTFREE, INITRANS, MAXTRANS, STORAGE,... (3 Replies)
Discussion started by: John K
3 Replies
10. Shell Programming and Scripting
i have a file where the hostnames and variables are in same line in below format, am able extract some part variables while otherlike subscriptions and handler is missing.
can you please correct me if grep is able to perform this ?
cat /tmp/test
localhost subscriptions='' handler="genie"... (14 Replies)
Discussion started by: rakeshkumar
14 Replies
LEARN ABOUT SUSE
cpanplus::internals::search
CPANPLUS::Internals::Search(3pm) Perl Programmers Reference Guide CPANPLUS::Internals::Search(3pm)
NAME
CPANPLUS::Internals::Search
SYNOPSIS
my $aref = $cpan->_search_module_tree(
type => 'package',
allow => [qr/DBI/],
);
my $aref = $cpan->_search_author_tree(
type => 'cpanid',
data => @old_results,
verbose => 1,
allow => [qw|KANE AUTRIJUS|],
);
my $aref = $cpan->_all_installed( );
DESCRIPTION
The functions in this module are designed to find module(objects) based on certain criteria and return them.
METHODS
_search_module_tree( type => TYPE, allow => @regexes, [data => @previous_results ] )
Searches the moduletree for module objects matching the criteria you specify. Returns an array ref of module objects on success, and false
on failure.
It takes the following arguments:
type
This can be any of the accessors for the "CPANPLUS::Module" objects. This is a required argument.
allow
A set of rules, or more precisely, a list of regexes (via "qr//" or plain strings), that the "type" must adhere too. You can specify as
many as you like, and it will be treated as an "OR" search. For an "AND" search, see the "data" argument.
This is a required argument.
data
An arrayref of previous search results. This is the way to do an "AND" search -- "_search_module_tree" will only search the module
objects specified in "data" if provided, rather than the moduletree itself.
_search_author_tree( type => TYPE, allow => @regexex, [data => @previous_results ] )
Searches the authortree for author objects matching the criteria you specify. Returns an array ref of author objects on success, and false
on failure.
It takes the following arguments:
type
This can be any of the accessors for the "CPANPLUS::Module::Author" objects. This is a required argument.
allow
A set of rules, or more precisely, a list of regexes (via "qr//" or plain strings), that the "type" must adhere too. You can specify as
many as you like, and it will be treated as an "OR" search. For an "AND" search, see the "data" argument.
This is a required argument.
data
An arrayref of previous search results. This is the way to do an "and" search -- "_search_author_tree" will only search the author
objects specified in "data" if provided, rather than the authortree itself.
_all_installed()
This function returns an array ref of module objects of modules that are installed on this system.
perl v5.12.1 2010-04-26 CPANPLUS::Internals::Search(3pm)