07-17-2011
Perl - work with open files or write to @lists first?
I am dealing will many thousand fairy small files.
I need to search them for various matches and depending on what I find, may need to search some files again for additional matches.
Generally speaking, is it better to write a txt file to an @array/@list and then work with it (multiple searches within it, creating $vars etc.) or open the file and go through it few times using while?
ie. Say I am searching through a file for A, B and C.
Depending what I find, I may then need to look in the same file for 1, 2, 3. Also, I will not always find A, B and C in that order.... it could be B, A, C etc.
I could just open the file and use while to look line by line for matches and depending on what I find I could use another (or more) while to go through the file again and close it when done.
OR, I could write the file to @list and search it for what I want.
Depending on what I find, some files will need a lot of searching, some very little.
Is there a rule of thumb for which approach is faster / easier on resources when dealing with thousands of files?
Thanks,
-OG-
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
I'm trying to do two different things (converting an OpenVms .com to a ksh shell script): 1) open/read/err= 2) open/write/err=
Any help? I've found some things, but can't seem to find the correct way. (1 Reply)
Discussion started by: prosserj
1 Replies
2. Shell Programming and Scripting
Hello
Can someone help me to write a perl script or kornshell reading a two files and outputting to comma format.
Here is the two files
listofdisks.txt
id, diskname, diskgroup, diskisze(GB), FC
1, CN34, GRP1, 30, FC_CN34
2, CN67, GRP5, 19,
4, VD1, GRP4, 23, FC_VD1
6, CF_D1, ... (0 Replies)
Discussion started by: deiow
0 Replies
3. Shell Programming and Scripting
I have a question regarding Perl scripting.
If I want to say open files that all look like this and assign them to a filehandle and then assign the filehandle to a variable, how do I do this?
The file names are
strand1.fa.gz.tmp
strand2.fa.gz.tmp
strand3.fa.gz.tmp
strand4.fa.gz.tmp
...... (6 Replies)
Discussion started by: japaneseguitars
6 Replies
4. Solaris
How do I find all "regular" files on solaris(8) that are open for write ( +read as well).
I tried using pfiles, and lsof commands, but not sure how to get exactly what I wanted.
ps -e | awk '{ print $1 }' | xargs -i pfiles {} 2>/dev/null (10 Replies)
Discussion started by: kchinnam
10 Replies
5. Homework & Coursework Questions
Use and complete the template provided. The entire template must be completed. If you don't, your post may be deleted!
1. The problem statement, all variables and given/known data:
I must write a shell script that calls two external Perl functions--one of which sorts the data in a file, and... (6 Replies)
Discussion started by: kowit010
6 Replies
6. Shell Programming and Scripting
Hi everybody!
I'm trying to delete some elements from a list with two elements on each row agreeing with the elements in another list. Pratically I want a perl script able to take each element of the second list (that is a single column list), compare it with both elements of each row from the... (3 Replies)
Discussion started by: gabrysfe
3 Replies
7. Shell Programming and Scripting
Hi All,
I would like to use a Perl (not Bash) script to work with multiple files of the same name in different directories (all in the same parent directory). I tried to create a loop to do so, but it isn't working.
My code so far:
while (defined(my $file = glob("./*/filename.txt")) or... (1 Reply)
Discussion started by: elgo4
1 Replies
8. Shell Programming and Scripting
Hello!
I am a real beginner in scripting, so I am struggling with a really easy task!
I want to write a script to concatenate several text files onto each other and generate a new file. I wanted the first argument to be the name of the new file, so:
./my_script.sh new_file file1.txt... (5 Replies)
Discussion started by: malajedala
5 Replies
9. Shell Programming and Scripting
help me (1 Reply)
Discussion started by: sonu pandey
1 Replies
10. UNIX for Beginners Questions & Answers
hi,
Hope you are doing good.
During my coding yesterday i got this challenge, actually not a challenge it like to optimize the code.
I am printing some statement to monitor the file progress in the log file an also to display it in the screen. so i ended up in the below statements.
... (6 Replies)
Discussion started by: mad man
6 Replies
LEARN ABOUT DEBIAN
file::find::rule::procedural
File::Find::Rule::Procedural(3pm) User Contributed Perl Documentation File::Find::Rule::Procedural(3pm)
NAME
File::Find::Rule::Procedural - File::Find::Rule's procedural interface
SYNOPSIS
use File::Find::Rule;
# find all .pm files, procedurally
my @files = find(file => name => '*.pm', in => @INC);
DESCRIPTION
In addition to the regular object-oriented interface, File::Find::Rule provides two subroutines for you to use.
"find( @clauses )"
"rule( @clauses )"
"find" and "rule" can be used to invoke any methods available to the OO version. "rule" is a synonym for "find"
Passing more than one value to a clause is done with an anonymous array:
my $finder = find( name => [ '*.mp3', '*.ogg' ] );
"find" and "rule" both return a File::Find::Rule instance, unless one of the arguments is "in", in which case it returns a list of things
that match the rule.
my @files = find( name => [ '*.mp3', '*.ogg' ], in => $ENV{HOME} );
Please note that "in" will be the last clause evaluated, and so this code will search for mp3s regardless of size.
my @files = find( name => '*.mp3', in => $ENV{HOME}, size => '<2k' );
^
|
Clause processing stopped here ------/
It is also possible to invert a single rule by prefixing it with "!" like so:
# large files that aren't videos
my @files = find( file =>
'!name' => [ '*.avi', '*.mov' ],
size => '>20M',
in => $ENV{HOME} );
AUTHOR
Richard Clamp <richardc@unixbeard.net>
COPYRIGHT
Copyright (C) 2003 Richard Clamp. All Rights Reserved.
This module is free software; you can redistribute it and/or modify it under the same terms as Perl itself.
SEE ALSO
File::Find::Rule
perl v5.12.4 2011-09-19 File::Find::Rule::Procedural(3pm)