UNIX for Dummies Questions & Answers

Hi,

I am going to sort a huge flat file using sort command, this file is about 36 million lines, 179 fields delimitered by Ctrl B (002). eg.

1^B198709.....
17^B200301....
3^B196511....
.....

I want this file being sorted by the first field, the result is like :

1^B198709.....
3^B196511....
17^B200301....

I used to sort it by command like :

$sort -n -T /test filename

It worked. but I am trying to find a more effective way to sort it. I assume only sort the first field would be much quicker, then I used command like :

$sort -n -t ^B -k 1,1 -T /test filename

it worked as well, but the performance was the same.

Can anyone give me an advice ?

Thanks in advance

xli

Without having a file that huge to messa round with, it's hard for me to say
(maybe I'll generate one after this and try).

The -S option seems like a possibility, for an increased memory buffer.
I would imagine you've been down that road, but I'm just throwing it out there.

-S did seem to help in my test
scenario.

I had a file that took a little over a minute, and I added
-S 1024

And it took about 50 seconds. That is certainly not a scientific
test or anything, nor Earth shattering improvement, but it at least
seems worth exploring.

Another possibility might be using awk, but it's questionable
whether or not that could help speed things up.