Shell Programming and Scripting

Keeping in mind that I'm relatively comfortable with programming in general but very new to unix and korn/bourne shell scripts..

I'm using awk on a CSV file, and then performing calculations and operations on specific fields within specific records. The CSV file I'm working with has about 600 records, with 30 fields.

Since I want to be able to cross-reference one record against all other records multiple times for different/specific records that need to be checked (for example if a record orders an amendment to a previous record, I'd like to check to see if the previous record still exists), and since, I think, awk only runs through each record once, I wanted to know if it's possible, and if so, if it's even a good idea to store all the fields from each record that I need (I don't need all of them, just maybe a third of the information) into an array in awk, and then return the array itself, outside of awk.

Possible? Good idea, bad idea? I'm open to all suggestions and advice.
Note: I don't know if it's possible to run the loop for each record again once you're in the awk program, for example: a nested awk where in the middle of the loop for the host awk, you can start a second awk to check the current record in the original awk against all other records.

It's somewhat difficult to follow the thread, but.... I think I understand what you're after.

Usually you don't to store ALL the records, but rather one record per key - where the 'key' is your 'unique identifier' for the type of a record.

Awk has a notion of the 'associative arrays' - very similar to perl's hashes. You can store by a 'key', update by a 'key' and lookup by a 'key'.

You can process your CSV once, store the relative pieces of a file in the array, manipulate the array cells as you go AND post-process the content of the array in the awk's 'END' session [by iterating through the array: for( i in array)].

The associative arrays are dynamic in nature and their size is bound only by the available memory. Seems like you don't have huge files to process - you should be ok.

Sorry about the lengthy thread, I usually try to make my inquiries with as much detail as possible to avoid confusion but half the time I guess I achieve the opposite.

Yes, you understood my thread perfectly. And yes, I think that will be enough for what I wish to do now.