Searching for array in large list of files


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Searching for array in large list of files
# 1  
Old 01-21-2009
Searching for array in large list of files

I tried to make the title/subject detailed, but well.. have to keep it short as well.

I am wanting to take a large list of strings, and search through a large list of files to hopefully find numerous matches. I am not sure the quickest way to do this though.

Code:
// List of files

file1.txt
file2.txt
file3.txt
file4.txt
file5.txt
file6.txt
file7.txt

// Searches

testtag
strings
more stuff
lots of stuff here

So basically take all of the strings under // Searches, and search through the files listed above it. I could imagine that the script may go something like this..

Code:
find . -type f -size -100k | xargs egrep -il "(testtag|strings|more stuff|lots of stuff here)"

While I know that will work, I would like to find a more efficient way of doing it. Any help would be greatly appreciated Smilie
# 2  
Old 01-21-2009

Put the search strings in a file and use the -f option to grep.
# 3  
Old 01-22-2009
Code:
#!/usr/bin/perl
use strict;
sub find{
	my($file,$str)=(@_);
	open FH,"<$file";
	$str=~s/,/|/g;
	while(<FH>){
		print "$file [line $.]: $_" if /($str)/;
	}
	print "---------------\n";
	close FH;
}
my @arr=("a.txt","b.txt");
my $match="152178,014052";
map {find($_,$match)} @arr;

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Bash arrays: rebin/interpolate smaller array to large array

hello, i need a bit of help on how to do this effectively in bash without a lot of extra looping or massive switch/case i have a long array of M elements and a short array of N elements, so M > N always. M is not a multiple of N. for case 1, I want to stretch N to fit M arrayHuge H = (... (2 Replies)
Discussion started by: f77hack
2 Replies

2. Shell Programming and Scripting

Split list of files into an array and pass to function

There are two parts to this. In the first part I need to read a list of files from a directory and split it into 4 arrays. I have done that with the following code, # collect list of file names STATS_INPUT_FILENAMES=($(ls './'$SET'/'$FOLD'/'*'in.txt')) # get number of files... (8 Replies)
Discussion started by: LMHmedchem
8 Replies

3. Shell Programming and Scripting

Searching a large file for short tandem repeats

Hello, I am searching large (~25gb) DNA sequence data in fasta short read format: >ReadName ACGTACGTACGT... for short tandem repeats, meaning instances of any 2-6 character based run that are repeated in tandem a number of times given as an input variable. Seems like a reasonably simple... (3 Replies)
Discussion started by: ljk
3 Replies

4. Shell Programming and Scripting

Grepping large list of files

Hi All, I need help to know the exact command when I grep large list of files. Either using ls or find command. However I do not want to find in the subdirectories as the number of subdirectories are not fixed. How do I achieve that. I want something like this: find ./ -name "MYFILE*.txt"... (2 Replies)
Discussion started by: angshuman
2 Replies

5. Shell Programming and Scripting

How to process list of files as array in the script

Hi., In my script I have written : file_list=`ls -lrt /tmp/vinay/act/files |grep "$cdate"| awk '{print $9}'` To store list of files in the directory. Now I want to access it as list. And for this I tried with : set -A filearray $file_list if }` == "" ]]; then ... (1 Reply)
Discussion started by: IND123
1 Replies

6. UNIX for Dummies Questions & Answers

Command to list large files

Looking for a line to show all of the large files on a unix server (over 300mb)... Having problems finding anything that works... TIA! (13 Replies)
Discussion started by: search66
13 Replies

7. Shell Programming and Scripting

Searching a specific line in a large file

Hey All Can any one please suggest the procedure to search a part of line in a very large file in which log entries are entered with very high speed. i have trued with grep and egrep grep 'text text text' <file-name> egrep 'text text text' <file-name> here 'text text text' is... (4 Replies)
Discussion started by: NIMISH AGARWAL
4 Replies

8. UNIX for Dummies Questions & Answers

Searching list of entries in file for actual files in dir

Hi all, I have a file entries.txt that contains a number of entries all on seperate lines e.g. test1 test2 test3 test4 Then in a directory called /TestFiles I have a number of files that could contain the above text in the file name e.g. qwertytest1.csv qwertytest2.csv... (2 Replies)
Discussion started by: not4google
2 Replies

9. UNIX for Dummies Questions & Answers

viewing and searching large file

I need to search a very large file. 13g in size. i am looking for a record that has a value in the byte 4200 . how can i view the file or how can i search for value in the byte 4200? (1 Reply)
Discussion started by: Wrightman
1 Replies

10. UNIX for Dummies Questions & Answers

List large files

Hi I need to list all files in the system: 1. Greater than specific size 2. All files sorted by size How can I do that? Thanks in advance. (2 Replies)
Discussion started by: GNMIKE
2 Replies
Login or Register to Ask a Question