hi,
I have the following problem:
- 6 different files that have one key in common.
- this six files must be aggregated in one output file sorted by the key.
- the main file has to be writen twice, one in the beggining of the new output file and another in the end, for each key.
- add one identifier to the output file in order to do a sort with the expected order of the lines.
- a friend told me to use AWK to gather them, but i have never seen this and only after finishing my current task i wil check about this.
- i really have to check about performance because each file has about 800.000 lines and i have 150 files!


if someone can give me some tips or where to get one example to do my work fast i would be appreciated ( this way i would probably start my weekend at decent times).
I think the biggest headdik will be because there can the lines without key (in witch i must use the last readed key), the fact i have to read twice the first file and that the sort must be done by the identifier in order to get the expected order!

example from files to see what i have to gather:

1º:

Code:

410000000000001,dummy, data,separated,by,comma,,,,,,,,
,,,,,,,,,,,,,,,,,,,,,,,,,
,,,,,,,,,,,,,,,,,,,,,,,,,
,,,,,,,,,,,,,,,,,,,,,,,,,
410000000000002,,dummy, data,separated,by,comma,,,,,,,,
410000000000003,,dummy, data,separated,by,comma,,,,,,,,
,,,,,,,,,,,,,,,,,,,,,,,,,

2º

Code:

410000000000001,,,dummy, data,separated,by,comma,,,,,,,,
410000000000002,,,dummy, data,separated,by,comma,,,,,,,,

3º

Code:

410000000000001,,,dummy, data,separated,by,comma,,,,,,,,
410000000000002,,,dummy, data,separated,by,comma,,,,,,,,
,,,,,,,,,,,,,,,,,,,,,
,,,,,,,,,,,,,,,,,,,,,
,,,,,,,,,,,,,,,,,,,,,
410000000000003,,,dummy, data,separated,by,comma,,,,,,,,
410000000000004,,,dummy, data,separated,by,comma,,,,,,,,
,,,,,,,,,,,,,,,,,,,,,

4º

Code:

41000000000001,dummy, data,separated,by,comma,,,,,,,,
41000000000002,dummy, data,separated,by,comma,,,,,,,,
,,,,,,,
,,,,,,,
,,,,,,,
41000000000003,dummy, data,separated,by,comma,,,,,,,,
41000000000004,dummy, data,separated,by,comma,,,,,,,,
,,,,,,,

5º

Code:

410000000000001,dummy, data,separated,by,comma,,,,,,,,
410000000000002,dummy, data,separated,by,comma,,,,,,,,

6º

Code:

410000000000001,dummy, data,separated,by,comma,,,,,,,,
410000000000002,dummy, data,separated,by,comma,,,,,,,,

in:

Code:

dummy, data,separated,by,comma

i will have a variable number of fields, depending on the file

Thanks for the help.

Best regards,
Ricardo Tomás