Sponsored Content
Top Forums Shell Programming and Scripting Searching for array in large list of files Post 302278995 by Rhije on Wednesday 21st of January 2009 05:12:37 PM
Old 01-21-2009
Searching for array in large list of files

I tried to make the title/subject detailed, but well.. have to keep it short as well.

I am wanting to take a large list of strings, and search through a large list of files to hopefully find numerous matches. I am not sure the quickest way to do this though.

Code:
// List of files

file1.txt
file2.txt
file3.txt
file4.txt
file5.txt
file6.txt
file7.txt

// Searches

testtag
strings
more stuff
lots of stuff here

So basically take all of the strings under // Searches, and search through the files listed above it. I could imagine that the script may go something like this..

Code:
find . -type f -size -100k | xargs egrep -il "(testtag|strings|more stuff|lots of stuff here)"

While I know that will work, I would like to find a more efficient way of doing it. Any help would be greatly appreciated Smilie
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

List large files

Hi I need to list all files in the system: 1. Greater than specific size 2. All files sorted by size How can I do that? Thanks in advance. (2 Replies)
Discussion started by: GNMIKE
2 Replies

2. UNIX for Dummies Questions & Answers

viewing and searching large file

I need to search a very large file. 13g in size. i am looking for a record that has a value in the byte 4200 . how can i view the file or how can i search for value in the byte 4200? (1 Reply)
Discussion started by: Wrightman
1 Replies

3. UNIX for Dummies Questions & Answers

Searching list of entries in file for actual files in dir

Hi all, I have a file entries.txt that contains a number of entries all on seperate lines e.g. test1 test2 test3 test4 Then in a directory called /TestFiles I have a number of files that could contain the above text in the file name e.g. qwertytest1.csv qwertytest2.csv... (2 Replies)
Discussion started by: not4google
2 Replies

4. Shell Programming and Scripting

Searching a specific line in a large file

Hey All Can any one please suggest the procedure to search a part of line in a very large file in which log entries are entered with very high speed. i have trued with grep and egrep grep 'text text text' <file-name> egrep 'text text text' <file-name> here 'text text text' is... (4 Replies)
Discussion started by: NIMISH AGARWAL
4 Replies

5. UNIX for Dummies Questions & Answers

Command to list large files

Looking for a line to show all of the large files on a unix server (over 300mb)... Having problems finding anything that works... TIA! (13 Replies)
Discussion started by: search66
13 Replies

6. Shell Programming and Scripting

How to process list of files as array in the script

Hi., In my script I have written : file_list=`ls -lrt /tmp/vinay/act/files |grep "$cdate"| awk '{print $9}'` To store list of files in the directory. Now I want to access it as list. And for this I tried with : set -A filearray $file_list if }` == "" ]]; then ... (1 Reply)
Discussion started by: IND123
1 Replies

7. Shell Programming and Scripting

Grepping large list of files

Hi All, I need help to know the exact command when I grep large list of files. Either using ls or find command. However I do not want to find in the subdirectories as the number of subdirectories are not fixed. How do I achieve that. I want something like this: find ./ -name "MYFILE*.txt"... (2 Replies)
Discussion started by: angshuman
2 Replies

8. Shell Programming and Scripting

Searching a large file for short tandem repeats

Hello, I am searching large (~25gb) DNA sequence data in fasta short read format: >ReadName ACGTACGTACGT... for short tandem repeats, meaning instances of any 2-6 character based run that are repeated in tandem a number of times given as an input variable. Seems like a reasonably simple... (3 Replies)
Discussion started by: ljk
3 Replies

9. Shell Programming and Scripting

Split list of files into an array and pass to function

There are two parts to this. In the first part I need to read a list of files from a directory and split it into 4 arrays. I have done that with the following code, # collect list of file names STATS_INPUT_FILENAMES=($(ls './'$SET'/'$FOLD'/'*'in.txt')) # get number of files... (8 Replies)
Discussion started by: LMHmedchem
8 Replies

10. Shell Programming and Scripting

Bash arrays: rebin/interpolate smaller array to large array

hello, i need a bit of help on how to do this effectively in bash without a lot of extra looping or massive switch/case i have a long array of M elements and a short array of N elements, so M > N always. M is not a multiple of N. for case 1, I want to stretch N to fit M arrayHuge H = (... (2 Replies)
Discussion started by: f77hack
2 Replies
dos2unix(1)						      General Commands Manual						       dos2unix(1)

NAME
dos2unix - DOS/MAC to UNIX text file format converter SYNOPSYS
dos2unix [options] [-c convmode] [-o file ...] [-n infile outfile ...] Options: [-hkqV] [--help] [--keepdate] [--quiet] [--version] DESCRIPTION
This manual page documents dos2unix, the program that converts plain text files in DOS/MAC format to UNIX format. OPTIONS
The following options are available: -h --help Print online help. -k --keepdate Keep the date stamp of output file same as input file. -q --quiet Quiet mode. Suppress all warning and messages. -V --version Prints version information. -c --convmode convmode Sets conversion mode. Simulates dos2unix under SunOS. -o --oldfile file ... Old file mode. Convert the file and write output to it. The program default to run in this mode. Wildcard names may be used. -n --newfile infile outfile ... New file mode. Convert the infile and write output to outfile. File names must be given in pairs and wildcard names should NOT be used or you WILL lost your files. EXAMPLES
Get input from stdin and write output to stdout. dos2unix Convert and replace a.txt. Convert and replace b.txt. dos2unix a.txt b.txt dos2unix -o a.txt b.txt Convert and replace a.txt in ASCII conversion mode. Convert and replace b.txt in ISO conversion mode. Convert c.txt from Mac to Unix ascii format. dos2unix a.txt -c iso b.txt dos2unix -c ascii a.txt -c iso b.txt dos2unix -c mac a.txt b.txt Convert and replace a.txt while keeping original date stamp. dos2unix -k a.txt dos2unix -k -o a.txt Convert a.txt and write to e.txt. dos2unix -n a.txt e.txt Convert a.txt and write to e.txt, keep date stamp of e.txt same as a.txt. dos2unix -k -n a.txt e.txt Convert and replace a.txt. Convert b.txt and write to e.txt. dos2unix a.txt -n b.txt e.txt dos2unix -o a.txt -n b.txt e.txt Convert c.txt and write to e.txt. Convert and replace a.txt. Convert and replace b.txt. Convert d.txt and write to f.txt. dos2unix -n c.txt e.txt -o a.txt b.txt -n d.txt f.txt DIAGNOSTICS
BUGS
The program does not work properly under MSDOS in stdio processing mode. If you know why is that so, please tell me. AUTHORS
Benjamin Lin - <blin@socs.uts.edu.au> Bernd Johannes Wuebben (mac2unix mode) <wuebben@kde.org> MISCELLANY
Tested environment: Linux 1.2.0 with GNU C 2.5.8 SunOS 4.1.3 with GNU C 2.6.3 MS-DOS 6.20 with Borland C++ 4.02 Suggestions and bug reports are welcome. SEE ALSO
unix2dos(1) mac2unix(1) 1995.03.31 dos2unix v3.0 dos2unix(1)
All times are GMT -4. The time now is 02:04 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy