Shell Programming and Scripting

OS : Linux 2.6x
Shell : Korn

In a single file , how can I identify all the Uniqe values at a specific character position and length of each record ,
and simultaneously SPLIT the records of the file based on each of these values and write them in seperate files .

Lets say :

Code:

a) I want to know what are the distinct values in the field marked by start character position 15 , and the next three
   characters , for each record in a file
b) If there are TWO SUCH DISTINCT VALUES , how to get the records for each of the distinct values in seperate files ?

Please help , it urgent , and its not a HOMEWORK ASSIGNMENT .

Thanks
Kumarjit.

Could you provide sample input and output files?

Try

Code:

awk '{fn=substr($0, 15,3); print > fn}' file

If the number of output files gets larger and larger, you may need to close (fn) in between...

@RudiC : Danke Rudi .......
But , didnt understand what you tried to mean by saying :

Code:

you may need to close (fn) in between...

If the numbre distinct values indicated by the field whose start position is 15 th character spanning the next three characters , how to ensure that this code performs optimally , cause I had tried and awk command over 10 million records , and it was going at snails's pace.

Please validate if my undertstanding is true.

Thanks to all of you.

Regards
Kumarjit.

---------- Post updated at 03:25 AM ---------- Previous update was at 03:21 AM ----------

Actually , what I tried to mean is :

If the number distinct values indicated by the field whose start position is 15 th character spanning the next three characters is significantly on the larger side , how to ensure that this code performs optimally , cause I had tried and awk command over 10 million records , and it was going at snails's pace.



Thanks again
Kumarjit.

At some point, the number of open files per process is exhausted. Certainly, with three characters, you will be able to reach 1000+ files, which is very close to OPEN_MAX (1024 on Linux). You need to append to the files, then, using >>, and close (fn) after each write.