07-17-2011
Perl - work with open files or write to @lists first?
I am dealing will many thousand fairy small files.
I need to search them for various matches and depending on what I find, may need to search some files again for additional matches.
Generally speaking, is it better to write a txt file to an @array/@list and then work with it (multiple searches within it, creating $vars etc.) or open the file and go through it few times using while?
ie. Say I am searching through a file for A, B and C.
Depending what I find, I may then need to look in the same file for 1, 2, 3. Also, I will not always find A, B and C in that order.... it could be B, A, C etc.
I could just open the file and use while to look line by line for matches and depending on what I find I could use another (or more) while to go through the file again and close it when done.
OR, I could write the file to @list and search it for what I want.
Depending on what I find, some files will need a lot of searching, some very little.
Is there a rule of thumb for which approach is faster / easier on resources when dealing with thousands of files?
Thanks,
-OG-
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
I'm trying to do two different things (converting an OpenVms .com to a ksh shell script): 1) open/read/err= 2) open/write/err=
Any help? I've found some things, but can't seem to find the correct way. (1 Reply)
Discussion started by: prosserj
1 Replies
2. Shell Programming and Scripting
Hello
Can someone help me to write a perl script or kornshell reading a two files and outputting to comma format.
Here is the two files
listofdisks.txt
id, diskname, diskgroup, diskisze(GB), FC
1, CN34, GRP1, 30, FC_CN34
2, CN67, GRP5, 19,
4, VD1, GRP4, 23, FC_VD1
6, CF_D1, ... (0 Replies)
Discussion started by: deiow
0 Replies
3. Shell Programming and Scripting
I have a question regarding Perl scripting.
If I want to say open files that all look like this and assign them to a filehandle and then assign the filehandle to a variable, how do I do this?
The file names are
strand1.fa.gz.tmp
strand2.fa.gz.tmp
strand3.fa.gz.tmp
strand4.fa.gz.tmp
...... (6 Replies)
Discussion started by: japaneseguitars
6 Replies
4. Solaris
How do I find all "regular" files on solaris(8) that are open for write ( +read as well).
I tried using pfiles, and lsof commands, but not sure how to get exactly what I wanted.
ps -e | awk '{ print $1 }' | xargs -i pfiles {} 2>/dev/null (10 Replies)
Discussion started by: kchinnam
10 Replies
5. Homework & Coursework Questions
Use and complete the template provided. The entire template must be completed. If you don't, your post may be deleted!
1. The problem statement, all variables and given/known data:
I must write a shell script that calls two external Perl functions--one of which sorts the data in a file, and... (6 Replies)
Discussion started by: kowit010
6 Replies
6. Shell Programming and Scripting
Hi everybody!
I'm trying to delete some elements from a list with two elements on each row agreeing with the elements in another list. Pratically I want a perl script able to take each element of the second list (that is a single column list), compare it with both elements of each row from the... (3 Replies)
Discussion started by: gabrysfe
3 Replies
7. Shell Programming and Scripting
Hi All,
I would like to use a Perl (not Bash) script to work with multiple files of the same name in different directories (all in the same parent directory). I tried to create a loop to do so, but it isn't working.
My code so far:
while (defined(my $file = glob("./*/filename.txt")) or... (1 Reply)
Discussion started by: elgo4
1 Replies
8. Shell Programming and Scripting
Hello!
I am a real beginner in scripting, so I am struggling with a really easy task!
I want to write a script to concatenate several text files onto each other and generate a new file. I wanted the first argument to be the name of the new file, so:
./my_script.sh new_file file1.txt... (5 Replies)
Discussion started by: malajedala
5 Replies
9. Shell Programming and Scripting
help me (1 Reply)
Discussion started by: sonu pandey
1 Replies
10. UNIX for Beginners Questions & Answers
hi,
Hope you are doing good.
During my coding yesterday i got this challenge, actually not a challenge it like to optimize the code.
I am printing some statement to monitor the file progress in the log file an also to display it in the screen. so i ended up in the below statements.
... (6 Replies)
Discussion started by: mad man
6 Replies
LEARN ABOUT OPENSOLARIS
lookbib
lookbib(1) User Commands lookbib(1)
NAME
lookbib - find references in a bibliographic database
SYNOPSIS
lookbib database
DESCRIPTION
A bibliographic reference is a set of lines, constituting fields of bibliographic information. Each field starts on a line beginning with
a `%', followed by a key-letter, then a blank, and finally the contents of the field, which may continue until the next line starting with
`%'.
The lookbib utility uses an inverted index made by indxbib to find sets of bibliographic references. It reads keywords typed after the `>'
prompt on the terminal, and retrieves records containing all these keywords. If nothing matches, nothing is returned except another `>'
prompt.
It is possible to search multiple databases, as long as they have a common index made by indxbib(1). In that case, only the first argument
given to indxbib is specified to lookbib.
If lookbib does not find the index files (the .i[abc] files), it looks for a reference file with the same name as the argument, without the
suffixes. It creates a file with a .ig suffix, suitable for use with fgrep (see grep(1)). lookbib then uses this fgrep file to find refer-
ences. This method is simpler to use, but the .ig file is slower to use than the .i[abc] files, and does not allow the use of multiple
reference files.
FILES
x.ia
x.ib
x.ic index files
x.ig reference file
ATTRIBUTES
See attributes(5) for descriptions of the following attributes:
+-----------------------------+-----------------------------+
| ATTRIBUTE TYPE | ATTRIBUTE VALUE |
+-----------------------------+-----------------------------+
|Availability |SUNWdoc |
+-----------------------------+-----------------------------+
SEE ALSO
addbib(1), grep(1), indxbib(1), refer(1), roffbib(1), sortbib(1), attributes(5)
BUGS
Probably all dates should be indexed, since many disciplines refer to literature written in the 1800s or earlier.
SunOS 5.11 14 Sep 1992 lookbib(1)