Sponsored Content
Top Forums UNIX for Dummies Questions & Answers Question: Help need to remove blank line & sed: Couldn't re-allocate memory error. Post 302344916 by senthil.ak on Tuesday 18th of August 2009 03:43:02 AM
Old 08-18-2009
Lightbulb Question: Help need to remove blank line & sed: Couldn't re-allocate memory error.

I've shell script where i used the below command to take the line which contains patterns.

Code:
sed -n "/$year 05:/,/$year 17:/p" trace.log | grep -f patterns.txt > output.log

This was working fine for long time, but now a days this script is not working with and throwing error like

sed: Couldn't re-allocate memory

when I looked for the file size its huge now a days, usually it will be less than or equal to 1.5 GB. but now a days its

Code:
$lt trace.log
 6.9G Aug 18 00:31 trace.log

trace.log is 6.9GB in size - so i conculded that the huge file size is the root cause of the issues. when i look in to the trace.log which 7 GB in size, its contains huge blank lines. i tired to remove the blank line using below commands but failed to perform and got the below error

Code:
 
$grep -av "^$" trace.log > ~/temp.tmp
$cat temp.tmp
Binary file trace.log matches
$sed '/^$/d' /logs/gopm/trace.log > ~/temp.tmp
sed: Couldn't re-allocate memory
$

Could you please suggest some other solution to over come this issue, appericaite your help
thanks
Senthil
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Remove blank line - SED

HI, I have this list of apps like so: DivX Products.app DivX Support.app Uninstall DivX for Mac.app Build Applet.app SpringBoard.app Interface.app MobileAddressBook.app MobileSafari.app MobileSlideShow.app Preferences.app Install Flash Player 8 OSX.app Yap.app check_afp.app ... (10 Replies)
Discussion started by: pcwiz
10 Replies

2. UNIX for Dummies Questions & Answers

Newbie help with New Line & Blank Line

I have this in my script: usercount=`ldapsearch -L -b"ou=people,ou=$ou,dc=paisleyhosting,dc=com" -h"$server" -p"1391" cn=* nothing | grep -c dn` admincount=`ldapsearch -b"ou=groups,ou=$ou,dc=paisleyhosting,dc=com" -h"$server" -p"1391" cn=USERADMIN* uniquemember | grep... (5 Replies)
Discussion started by: kthatch
5 Replies

3. Shell Programming and Scripting

Remove last blank line of file

I have a number of files (arranged in directories) which have last line blank, I am trying to synchronize my code with other env and due to this blank lines, all files error out as different although only difference is that of balnk line at end of file. Is there a way I can recursively... (2 Replies)
Discussion started by: ruchimca
2 Replies

4. Shell Programming and Scripting

Perl & Sed command -- Out of Memory Error

Experts, We used to receive our source files with '~^' as row delimiter. This file contains 2500K records and two of the columns having value in HTML formats within the file. While running the below commands against the file, we are encountering out of memory, could you please help to... (3 Replies)
Discussion started by: srivijay81
3 Replies

5. Shell Programming and Scripting

Error PHP Fatal error: Allowed memory size of 67108864 bytes exhausted(tried to allocate 401 bytes)

While running script I am getting an error like Few lines in data are not being processed. After googling it I came to know that adding such line would give some memory to it ini_set("memory_limit","64M"); my input file size is 1 GB. Is that memory limit is based on RAM we have on... (1 Reply)
Discussion started by: elamurugu
1 Replies

6. Shell Programming and Scripting

"Cannot allocate memory" error when run from script

hi in my application there is a line with open(/dev/mydevice,0); it work good when it run manually, but when try to run it within script /////////////////////////////////////////// #!/bin/sh ./device_test 0 0 0 1 1 1 0 0 1 ///////////////////////////////////////// I receive 'Failed... (3 Replies)
Discussion started by: majeed
3 Replies

7. Shell Programming and Scripting

Print lines between a regExp & a blank line

Hi, I have a file, say files_list, as below (o/p of ls -R cmd) $ cat files_list /remote/dir/path/to/file: sub-dir1 sub-dir2 sub-dir3 ... /remote/dir/path/to/file/sub-dir1: remote_file1.csv.tgz <blank line 1> /remote/dir/path/to/file/sub-dir2: remote_file2.csv.tgz <blank... (3 Replies)
Discussion started by: dips_ag
3 Replies

8. Shell Programming and Scripting

How to remove blank line from a text file?

Hi All, I am creating a text file using perl. The first record I am writing as "$line" and all the other as "\n$line". At the end the file is having N number of lines. I am using this file for MLOAD (Teradata), which is reading N+1 lines in the file and failing.I am not able to find new line... (2 Replies)
Discussion started by: unankix
2 Replies

9. Shell Programming and Scripting

'Couldn't read file' error in bash script with expect, sed and awk!

Ok, so I have a bash script with an embedded expect statement. Inside of the expect statement, i'm trying to pull all of the non-comment lines from the /etc/oratab file one at a time. Here's my command: cat /etc/oratab |sed /^s*#/d\ | awk 'NR==1'|awk -F: '{print \"$1\"}'|. oraenv Now,... (0 Replies)
Discussion started by: alexdglover
0 Replies

10. Shell Programming and Scripting

Bash - sed - Remove first word from line which can begin eventually with blank

hello. How to remove first word from line. The line may or may not start with blank. NEW_PARAM1=$(magic-command " -t --protocol=TCP -P 12345-u root -h localhost ") NEW_PARAM2=$(magic-command "-t --protocol=TCP -P 12345 -u root -h localhost ") I want NEW_PARAM1 equal to NEW_PARAM2 equal ... (2 Replies)
Discussion started by: jcdole
2 Replies
FAXCRON(8C)															       FAXCRON(8C)

NAME
faxcron - HylaFAX routine maintenance script SYNOPSIS
/usr/sbin/faxcron [ -n ] [ -l lastrun ] DESCRIPTION
faxcron is a command script that does routine upkeep tasks in a HylaFAX spooling directory hierarchy. This script is intended to be invoked from cron(8C) on behalf of the fax user (often the ``fax'' user is uucp) once a day, with the standard output sent by mail to the HylaFAX administrator. For example, the following crontab might be set up for the fax user: 25 23 * * * sh /usr/sbin/faxcron | mail FaxMaster The faxcron script does the following work: o report statistics for facsimile transmitted since the last time faxcron was run. o report statistics for facsimile transmitted in the past week. o report statistics for facsimile received since the last time faxcron was run. o report statistics for facsimile received in the past week. o report any recent calls that failed suspiciously and return the associated trace log; o purge data that is 30 days or older from the remote machine information directory; o delete information that is 30 days or older from session trace log files; o remove files from the tmp directory that are older than 1 day; o remove files in the received facsimile queue that are 7 days or older; o report sites that are currently having jobs rejected; and o force all session log files to be owned by the fax user and kept with protection mode 644. OPTIONS
-n Forces faxcron to run without updating any files. -l lastrun Forces the notion of the last time that faxcron was run to be lastrun. The value of lastrun is a date and time string of the form ``MM/DD/YY HH:MM'' (the date(1) format string ``%D %H:%M''). -info n Set the expiration time for data in the info database to be n days. -log n Set the expiration time for session log data to be n days. -rcv n Set the expiration time for files in the received facsimile queue to be n days. -tmp n Set the expiration time for files in the tmp directory to be n days. -mode m Set the file protection mode for session logs to m (a command line argument to chmod(1)). NOTES
This script requires nawk(1) or gawk(1) and a date(1) program that supports the ``+format'' option. If session logs are to be kept private this script must be modified to filter out sensitive information such as calling card numbers. (It should also be run with ``-mode 0600'' so that session log files are not publicly readable.) FILES
/var/spool/fax spooling area /var/spool/fax/etc/lastrun timestamp of last run /usr/sbin/xferfaxstats for generating transmit statistics /usr/sbin/recvstats for generating receive statistics /var/spool/fax/tmp/faxcronXXXX temporary file for truncating session logs /var/spool/fax/tmp/faxlogXXXX temporary file for logs of failed calls SEE ALSO
cron(8C), faxq(8C), faxgetty(8C), faxsend(8C), xferfaxstats(8C), recvstats(8C), hylafax-server(5F) May 12, 1996 FAXCRON(8C)
All times are GMT -4. The time now is 06:30 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy