Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Perl - work with open files or write to @lists first?
Top Forums Shell Programming and Scripting Perl - work with open files or write to @lists first? Post 302539431 by OldGaf on Sunday 17th of July 2011 12:40:27 PM
Old 07-17-2011
Perl - work with open files or write to @lists first?
I am dealing will many thousand fairy small files.
I need to search them for various matches and depending on what I find, may need to search some files again for additional matches.

Generally speaking, is it better to write a txt file to an @array/@list and then work with it (multiple searches within it, creating $vars etc.) or open the file and go through it few times using while?

ie. Say I am searching through a file for A, B and C.
Depending what I find, I may then need to look in the same file for 1, 2, 3. Also, I will not always find A, B and C in that order.... it could be B, A, C etc.

I could just open the file and use while to look line by line for matches and depending on what I find I could use another (or more) while to go through the file again and close it when done.

OR, I could write the file to @list and search it for what I want.

Depending on what I find, some files will need a lot of searching, some very little.

Is there a rule of thumb for which approach is faster / easier on resources when dealing with thousands of files?

Thanks,
-OG-
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Performing an open/read, open/write

I'm trying to do two different things (converting an OpenVms .com to a ksh shell script): 1) open/read/err= 2) open/write/err= Any help? I've found some things, but can't seem to find the correct way. (1 Reply)
Discussion started by: prosserj
1 Replies

2. Shell Programming and Scripting

write a perl script or kornshell reading a two files and outputting to comma format

Hello Can someone help me to write a perl script or kornshell reading a two files and outputting to comma format. Here is the two files listofdisks.txt id, diskname, diskgroup, diskisze(GB), FC 1, CN34, GRP1, 30, FC_CN34 2, CN67, GRP5, 19, 4, VD1, GRP4, 23, FC_VD1 6, CF_D1, ... (0 Replies)
Discussion started by: deiow
0 Replies

3. Shell Programming and Scripting

Perl, open multiple files with wildcards

I have a question regarding Perl scripting. If I want to say open files that all look like this and assign them to a filehandle and then assign the filehandle to a variable, how do I do this? The file names are strand1.fa.gz.tmp strand2.fa.gz.tmp strand3.fa.gz.tmp strand4.fa.gz.tmp ...... (6 Replies)
Discussion started by: japaneseguitars
6 Replies

4. Solaris

Find all "regular" files open for write on solaris?

How do I find all "regular" files on solaris(8) that are open for write ( +read as well). I tried using pfiles, and lsof commands, but not sure how to get exactly what I wanted. ps -e | awk '{ print $1 }' | xargs -i pfiles {} 2>/dev/null (10 Replies)
Discussion started by: kchinnam
10 Replies

5. Homework & Coursework Questions

Shell script calling Perl function, sort and find data, write to new files

Use and complete the template provided. The entire template must be completed. If you don't, your post may be deleted! 1. The problem statement, all variables and given/known data: I must write a shell script that calls two external Perl functions--one of which sorts the data in a file, and... (6 Replies)
Discussion started by: kowit010
6 Replies

6. Shell Programming and Scripting

Compare two lists with perl

Hi everybody! I'm trying to delete some elements from a list with two elements on each row agreeing with the elements in another list. Pratically I want a perl script able to take each element of the second list (that is a single column list), compare it with both elements of each row from the... (3 Replies)
Discussion started by: gabrysfe
3 Replies

7. Shell Programming and Scripting

Perl: script to work with files with the same name in different directories

Hi All, I would like to use a Perl (not Bash) script to work with multiple files of the same name in different directories (all in the same parent directory). I tried to create a loop to do so, but it isn't working. My code so far: while (defined(my $file = glob("./*/filename.txt")) or... (1 Reply)
Discussion started by: elgo4
1 Replies

8. Shell Programming and Scripting

Script to open files and write into new one

Hello! I am a real beginner in scripting, so I am struggling with a really easy task! I want to write a script to concatenate several text files onto each other and generate a new file. I wanted the first argument to be the name of the new file, so: ./my_script.sh new_file file1.txt... (5 Replies)
Discussion started by: malajedala
5 Replies

9. Shell Programming and Scripting

• Write a shell script that upon invocation shows the time and date and lists all the logged-in user

help me (1 Reply)
Discussion started by: sonu pandey
1 Replies

10. UNIX for Beginners Questions & Answers

How to write in multiple output files in perl?

hi, Hope you are doing good. During my coding yesterday i got this challenge, actually not a challenge it like to optimize the code. I am printing some statement to monitor the file progress in the log file an also to display it in the screen. so i ended up in the below statements. ... (6 Replies)
Discussion started by: mad man
6 Replies

LEARN ABOUT MOJAVE

zipgrep

ZIPGREP(1L)															       ZIPGREP(1L)

NAME

       zipgrep - search files in a ZIP archive for lines matching a pattern

SYNOPSIS

       zipgrep [egrep_options] pattern file[.zip] [file(s) ...]  [-x xfile(s) ...]

DESCRIPTION

       zipgrep	will  search  files  within  a ZIP archive for lines matching the given string or pattern.  zipgrep is a shell script and requires
       egrep(1) and unzip(1L) to function.  Its output is identical to that of egrep(1).

ARGUMENTS

       pattern
	      The pattern to be located within a ZIP archive.  Any string or regular expression accepted by egrep(1) may be used.  file[.zip] Path
	      of  the  ZIP archive.  (Wildcard expressions for the ZIP archive name are not supported.)  If the literal filename is not found, the
	      suffix .zip is appended.	Note that self-extracting ZIP files are supported, as with any other ZIP archive; just	specify  the  .exe
	      suffix (if any) explicitly.

       [file(s)]
	      An  optional list of archive members to be processed, separated by spaces.  If no member files are specified, all members of the ZIP
	      archive are searched.  Regular expressions (wildcards) may be used to match multiple members:

	      *      matches a sequence of 0 or more characters

	      ?      matches exactly 1 character

	      [...]  matches any single character found inside the brackets; ranges are specified by a beginning character, a hyphen, and an  end-
		     ing character.  If an exclamation point or a caret (`!' or `^') follows the left bracket, then the range of characters within
		     the brackets is complemented (that is, anything except the characters inside the brackets is considered a match).

	      (Be sure to quote any character that might otherwise be interpreted or modified by the operating system.)

       [-x xfile(s)]
	      An optional list of archive members to be excluded from processing.  Since wildcard characters  match  directory	separators  (`/'),
	      this  option  may  be used to exclude any files that are in subdirectories.  For example, ``zipgrep grumpy foo *.[ch] -x */*'' would
	      search for the string ``grumpy'' in all C source files in the main directory of the ``foo'' archive, but none in any subdirectories.
	      Without the -x option, all C source files in all directories within the zipfile would be searched.

OPTIONS

       All options prior to the ZIP archive filename are passed to egrep(1).

SEE ALSO

       egrep(1), unzip(1L), zip(1L), funzip(1L), zipcloak(1L), zipinfo(1L), zipnote(1L), zipsplit(1L)

URL

       The Info-ZIP home page is currently at
       http://www.info-zip.org/pub/infozip/
       or
       ftp://ftp.info-zip.org/pub/infozip/ .

AUTHORS

       zipgrep was written by Jean-loup Gailly.

Info-ZIP							   20 April 2009						       ZIPGREP(1L)
All times are GMT -4. The time now is 06:00 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy