Shell Programming and Scripting

Experts,

We used to receive our source files with '~^' as row delimiter. This file contains 2500K records and two of the columns having value in HTML formats within the file.

While running the below commands against the file, we are encountering out of memory, could you please help to identify why this error is coming out and what way can be fixed. We used to run this piece of code for several other files and it is working properly.

Code:

perl -pi -e 's/~\^/^F/g' Srik001.dat (out of memory error)

Code:

Sed 's/~\^/^F/g' Srik001.dat > Srik.dat (failed to allocate memory error)

Thanks in advance

Thanks & regards
Srik

Moderator's Comments:

There is no need to format each line on it's own. And please use code tags for command lines, listings, console output, and other similar texts

To avoid reading all the data into memory, you need to tell the program to change its idea of records:

Code:

perl -e'$/="~^";$\="\6";chomp&&print while<>' Srik001.dat

If the last record doesn't have the separator, it will not be included in the output.

Could you please provide any other alternate solution to this problem
Thanks
Srik

try this is usefull when occurs overflow mem arg buffer or pattern or hold buffer or ...

Code:

while read -r l ; do sed 's/~\^/^F/g' "$l" >>Srik.dat ; done <Srik001.dat