Hi.

Both of the solutions, shell variable and "dd move" are about the same as far as system resources go. I used alister's amended solution; his first failed (the one that went out with an email notification).

For a file of 14,754,910 lines, about 1 GB, the times for the shell solution keeping the last 50,000 lines were:

Code:

real	0m36.738s
user	0m1.624s
sys	0m8.321s

and for the clever "dd move" solution:

Code:

real	0m44.048s
user	0m1.408s
sys	0m8.725s

There were 2 wc executions for verification in both runs. Leaving those out:

Code:

real	0m29.443s
user	0m0.560s
sys	0m7.012s

and

Code:

real	0m30.536s
user	0m0.220s
sys	0m6.460s

The results actually surprised me -- I thought the shell would be slower. I saw paging being used a few times, slightly more often with the dd. The shell was run first, so cache advantage, if any, went to dd ... cheers, drl

This system:

Code:

 OS: Debian 5.0.8 lenny
 Kernel: x86_64 Linux 2.6.26-2-amd64
 Uptime: 59d 21h 58m
 CPU: AMD Athlon 64 3000+ @ 1.802GHz
 GPU: NVidia GeForce FX 5200
 RAM: 1170MB / 3024MB