UNIX for Dummies Questions & Answers

Apologies, the directory I was using is (i.e. directory1 is /powerdata0013/ALMDW/SrcFiles/ALM/QRM)

Code:

 /dev/powerdata0013 /powerdata0010/FMIS/SrcFiles/AP_578 mmfs   13 Sep 18:53 rw,mtime,atime,dev=powerdata0013

Code:

uname -a
AIX srvpoweprask04 1 6 000A713AD400

I ran a test on an old AIX 5.3 system.
Its /usr/bin/sort is a really bad implementation. It hardly uses more than 4 MB of memory.
The -y attempt did not have a measurable effect - the many GB of free RAM are not used.
Even a sort in Perl would be faster than the /usr/bin/sort.

Can't say anything about the current AIX version.
Maybe there is a GNU sort provided by IBM support or by other sources like AIX Open Source Packages | Main / AIX Open Source Packages ?

Thanks MadeInGermany. But, I don't have a clue whether we have one. Can you please let me know how to do it in perl? So that I can give it a try.

If I remember my data structures correctly, a bucket sort might help. If you can break up the file in 20 - 50 buckets based on either the first character, first few characters or word, you can put each set of values into a separate file. For example if the file begins with the lower case letters a-z you can read through the whole file once and if the current line starts with a write it to the a_line.dat file, if it is b write it to the b_line.dat file, etc. Now you have the file broken down into 26, hopefully equal files. Each file should be approximately 300-400 MB. which would better lend itself to an in memory sort. If you need to create a single sorted file, then just concatenate each of the *_line.dat files into a new sorted file in order from a to z. So this would be a divide and conquer approach.

Perl does have a bucket sort module, but if you are doing an in memory sort you are limited to the amount of free memory that you have.

Sort::Bucket - search.cpan.org