Question: Help need to remove blank line & sed: Couldn't re-allocate memory error.


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Question: Help need to remove blank line & sed: Couldn't re-allocate memory error.
Prev   Next
# 1  
Old 08-18-2009
Lightbulb Question: Help need to remove blank line & sed: Couldn't re-allocate memory error.

I've shell script where i used the below command to take the line which contains patterns.

Code:
sed -n "/$year 05:/,/$year 17:/p" trace.log | grep -f patterns.txt > output.log

This was working fine for long time, but now a days this script is not working with and throwing error like

sed: Couldn't re-allocate memory

when I looked for the file size its huge now a days, usually it will be less than or equal to 1.5 GB. but now a days its

Code:
$lt trace.log
 6.9G Aug 18 00:31 trace.log

trace.log is 6.9GB in size - so i conculded that the huge file size is the root cause of the issues. when i look in to the trace.log which 7 GB in size, its contains huge blank lines. i tired to remove the blank line using below commands but failed to perform and got the below error

Code:
 
$grep -av "^$" trace.log > ~/temp.tmp
$cat temp.tmp
Binary file trace.log matches
$sed '/^$/d' /logs/gopm/trace.log > ~/temp.tmp
sed: Couldn't re-allocate memory
$

Could you please suggest some other solution to over come this issue, appericaite your help
thanks
Senthil
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Bash - sed - Remove first word from line which can begin eventually with blank

hello. How to remove first word from line. The line may or may not start with blank. NEW_PARAM1=$(magic-command " -t --protocol=TCP -P 12345-u root -h localhost ") NEW_PARAM2=$(magic-command "-t --protocol=TCP -P 12345 -u root -h localhost ") I want NEW_PARAM1 equal to NEW_PARAM2 equal ... (2 Replies)
Discussion started by: jcdole
2 Replies

2. Shell Programming and Scripting

'Couldn't read file' error in bash script with expect, sed and awk!

Ok, so I have a bash script with an embedded expect statement. Inside of the expect statement, i'm trying to pull all of the non-comment lines from the /etc/oratab file one at a time. Here's my command: cat /etc/oratab |sed /^s*#/d\ | awk 'NR==1'|awk -F: '{print \"$1\"}'|. oraenv Now,... (0 Replies)
Discussion started by: alexdglover
0 Replies

3. Shell Programming and Scripting

How to remove blank line from a text file?

Hi All, I am creating a text file using perl. The first record I am writing as "$line" and all the other as "\n$line". At the end the file is having N number of lines. I am using this file for MLOAD (Teradata), which is reading N+1 lines in the file and failing.I am not able to find new line... (2 Replies)
Discussion started by: unankix
2 Replies

4. Shell Programming and Scripting

Print lines between a regExp & a blank line

Hi, I have a file, say files_list, as below (o/p of ls -R cmd) $ cat files_list /remote/dir/path/to/file: sub-dir1 sub-dir2 sub-dir3 ... /remote/dir/path/to/file/sub-dir1: remote_file1.csv.tgz <blank line 1> /remote/dir/path/to/file/sub-dir2: remote_file2.csv.tgz <blank... (3 Replies)
Discussion started by: dips_ag
3 Replies

5. Shell Programming and Scripting

"Cannot allocate memory" error when run from script

hi in my application there is a line with open(/dev/mydevice,0); it work good when it run manually, but when try to run it within script /////////////////////////////////////////// #!/bin/sh ./device_test 0 0 0 1 1 1 0 0 1 ///////////////////////////////////////// I receive 'Failed... (3 Replies)
Discussion started by: majeed
3 Replies

6. Shell Programming and Scripting

Error PHP Fatal error: Allowed memory size of 67108864 bytes exhausted(tried to allocate 401 bytes)

While running script I am getting an error like Few lines in data are not being processed. After googling it I came to know that adding such line would give some memory to it ini_set("memory_limit","64M"); my input file size is 1 GB. Is that memory limit is based on RAM we have on... (1 Reply)
Discussion started by: elamurugu
1 Replies

7. Shell Programming and Scripting

Perl & Sed command -- Out of Memory Error

Experts, We used to receive our source files with '~^' as row delimiter. This file contains 2500K records and two of the columns having value in HTML formats within the file. While running the below commands against the file, we are encountering out of memory, could you please help to... (3 Replies)
Discussion started by: srivijay81
3 Replies

8. Shell Programming and Scripting

Remove last blank line of file

I have a number of files (arranged in directories) which have last line blank, I am trying to synchronize my code with other env and due to this blank lines, all files error out as different although only difference is that of balnk line at end of file. Is there a way I can recursively... (2 Replies)
Discussion started by: ruchimca
2 Replies

9. UNIX for Dummies Questions & Answers

Newbie help with New Line & Blank Line

I have this in my script: usercount=`ldapsearch -L -b"ou=people,ou=$ou,dc=paisleyhosting,dc=com" -h"$server" -p"1391" cn=* nothing | grep -c dn` admincount=`ldapsearch -b"ou=groups,ou=$ou,dc=paisleyhosting,dc=com" -h"$server" -p"1391" cn=USERADMIN* uniquemember | grep... (5 Replies)
Discussion started by: kthatch
5 Replies

10. Shell Programming and Scripting

Remove blank line - SED

HI, I have this list of apps like so: DivX Products.app DivX Support.app Uninstall DivX for Mac.app Build Applet.app SpringBoard.app Interface.app MobileAddressBook.app MobileSafari.app MobileSlideShow.app Preferences.app Install Flash Player 8 OSX.app Yap.app check_afp.app ... (10 Replies)
Discussion started by: pcwiz
10 Replies
Login or Register to Ask a Question