Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Perl - FileHandles that crash memory
Top Forums Shell Programming and Scripting Perl - FileHandles that crash memory Post 302168942 by KevinADC on Tuesday 19th of February 2008 10:17:47 PM
Old 02-19-2008
grep returns a list, which means it reads the entire file/folder of whatever you grep. If the files are big you are just simply running out of physical memory because perl will use all the memory the system has available. In this case you probably do have to rewrite some code to use less memory, such as:

Code:
open(FH,'file');
   while(<FH>) {
       search for whatever
   }
}
close FH;

 

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Weakness in Perl CGI causes memory dump ??

I have discovered a curious phenomenon in GCI. I need some advice from someone far more adept. Is this a bug or potential security weakness? Context: Redhat 8.0 on xx86 (pentium 3) Apache 2.0, Perl 5.80 Background: I've been trying to harden a web application which accepts user uploaded... (0 Replies)
Discussion started by: andyj
0 Replies

2. Shell Programming and Scripting

Out of memory in PERL

I dont know whether this is a common error but it seems my PERL program is running out of buffer memory. Inside a loop I am processing an array (reading elements and displaying in it--note: for every iteration of the loop the WHOLE array is being scanned once--...the array was created by reading a... (5 Replies)
Discussion started by: Abhishek Ghose
5 Replies

3. UNIX for Dummies Questions & Answers

perl memory leak detector

Could somebody give links for memory detector for perl? Thanks :) (4 Replies)
Discussion started by: matrixmadhan
4 Replies

4. Programming

Process crash when allocating memory

Hi I'm currently using C++ on a HP-UX 11i system (upgrading some libraries) and am encountering a problem with the process crashing when allocating memory via a call to new (a rather large array of objects are being created). Is there a way to find out what the sizes of the stack and heap are?... (1 Reply)
Discussion started by: themezzaman
1 Replies

5. Shell Programming and Scripting

bash/perl script that will use up all memory

Hi, Is there any known trick that will consume all of my memory and force my machine to swap? I'm scripting a monitor in perl that will alert me when available memory is dangerously low and system is swapping, and I would like to simulate the scenario. I am clueless how to make my system lack... (4 Replies)
Discussion started by: marcpascual
4 Replies

6. Shell Programming and Scripting

User perl to get memory installed in a machine

I currently have a statistics gathering script i run on my Linux servers. One of the stat i gather is total memory in the machine. The script is all perl with the exception of gathering the memory for that i use the following command: $ram = (`cat /proc/meminfo | grep "MemTotal" | awk... (1 Reply)
Discussion started by: insania
1 Replies

7. Shell Programming and Scripting

Out Of Memory in Perl

Hi , when one of the perl script is being executed i am getting the following error as out of memory, when i check memory utilisation it is showing as 60% . cp utilisation as 11%. swap is free for 19 gb. can anyone guide me to sort the above isssue.i want to know the root cause. (2 Replies)
Discussion started by: Ajoy
2 Replies

8. Shell Programming and Scripting

Perl code Out of memory problem

Hi I am working with a 3.0 GB file and running the following script to process it. After running for 15 minutes, the script stops saying Out of memory or Killed. Can the experts suggest where my code can be improved to save memory? Thanks in advance. #! usr/bin/perl use warnings; open... (3 Replies)
Discussion started by: polsum
3 Replies

9. UNIX for Dummies Questions & Answers

Fork resource unavailable error, max # filehandles open?

I wrote a perl program that simultaneously reads in data from 691 tar.gz files using zcat. I can run one instance of the program without any issues and the memory and swap sizes are negligible. However, when I attempt to run more than 1 I start to get fork: resource unavailable messages. Are... (6 Replies)
Discussion started by: aquinom85
6 Replies

10. Shell Programming and Scripting

Out of memory issue in perl

I am getting a out of memory issue while executing the perl program. Per version : /opt/acc_perl/lib/site_perl/5.14.2 Read in 54973 total records Read in 54973 table records from table. Out of memory! so the job get failed due to out of memory. need to get rid of the out of memory... (3 Replies)
Discussion started by: ramkumar15
3 Replies

LEARN ABOUT LINUX

xzgrep

XZGREP(1)							     XZ Utils								 XZGREP(1)

NAME

       xzgrep - search compressed files for a regular expression

SYNOPSIS

       xzgrep [grep_options] [-e] pattern file...
       xzegrep ...
       xzfgrep ...
       lzgrep ...
       lzegrep ...
       lzfgrep ...

DESCRIPTION

       xzgrep  invokes	grep(1)  on  files  which may be either uncompressed or compressed with xz(1), lzma(1), gzip(1), or bzip2(1).  All options
       specified are passed directly to grep(1).

       If no file is specified, then standard input is decompressed if necessary and fed to grep(1).  When reading from  standard  input,  gzip(1)
       and bzip2(1) compressed files are not supported.

       If  xzgrep  is  invoked as xzegrep or xzfgrep then egrep(1) or fgrep(1) is used instead of grep(1).  The same applies to names lzgrep, lze-
       grep, and lzfgrep, which are provided for backward compatibility with LZMA Utils.

ENVIRONMENT

       GREP   If the GREP environment variable is set, xzgrep uses it instead of grep(1), egrep(1), or fgrep(1).

SEE ALSO

       grep(1), xz(1), gzip(1), bzip2(1), zgrep(1)

Tukaani 							    2010-09-27								 XZGREP(1)
All times are GMT -4. The time now is 03:53 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy