Shell Programming and Scripting

Hi All,

I have a 17GB file and i want to set the first column from 1234567890 to just 0, how can i do it using sed command?

thanks,

---------- Post updated at 03:09 PM ---------- Previous update was at 03:07 PM ----------

btw, my file is delimited by "|" and 1234567890 is just an example value of first column.

it's like this:

Code:

1234567890|abcddc|sfcdcd|12-12-2015|
212|fddfdgg|sdfdferruut|12-30-2014|

Is it possible for you to post five to ten lines from that 17GB and an example of how the output looks like?
That would be a better request. In that way we might know what a column means to you.

---------- Post updated at 10:53 PM ---------- Previous update was at 10:41 PM ----------

Using your last example.

Code:

sed -e 's/^1234567890\\|/0\\|/' 17GB.file > Another17GB.file && mv Another17GB.file 17GB.file.

Now, as you see, there's no way around the fact that you need a minimum of 34GB of space and cpu cycles for sed to go through.

No \\ and no \ because sed uses a simple RE (or BRE)! And a single -e option can be omitted.
If your sed has -i option

Code:

sed -i 's/^1234567890|/0|/' file

sed -i opens a temporary file AFAIK. So be sure that $TMPDIR points to a disk that has lots of free space. IF you have enough RAM, you can direct TMPDIR to the RAM drive.

Hmm, shouldn't sed -i put the temporary file near the source file, to have an efficient move(=rename)?
And the difference to Aia's explicit mv command is only that the owner/mode is preserved.

Not when TMPDIR is in kernel memory (RAM). Solaris and Linux both support this.

The answer: try tracing file open calls in a sed -i command on your system against a dummied up large file, larger than the free space on the current working directory's filesystem. This is clearly contrived but will answer the question. Be sure to define TMPDIR (or whatever )

Your point is well-taken from a UNIX system developers point of view.

My Solaris 10 system with TMPDIR defined elsewhere does this with gsed (Solaris version of GNU sed)

Code:

open64("somefile", O_RDONLY)                    = 3
open64("./sedz2dEGN", O_RDWR|O_CREAT|O_EXCL, 0600) = 4

Which is what I think is a major problem for large files when RAM drive is available.
Hmm.