The UNIX and Linux Forums  

Go Back   The UNIX and Linux Forums > Top Forums > Shell Programming and Scripting
Google UNIX.COM


Shell Programming and Scripting Post questions about KSH, CSH, SH, BASH, PERL, PHP, SED, AWK and OTHER shell scripts here.

More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
help... no idea what to use dakid Shell Programming and Scripting 4 03-05-2008 09:57 PM
using case to do this might be a bad idea? Skyybugg Shell Programming and Scripting 12 04-06-2007 11:19 AM
any good idea on this? fedora Shell Programming and Scripting 7 09-29-2006 12:36 PM
an idea? /var/adm/message pressy UNIX for Advanced & Expert Users 3 08-19-2003 04:04 AM
Dual Boot a good idea? PravusMentis UNIX for Dummies Questions & Answers 8 07-18-2001 12:02 PM

Reply
 
Submit Tools LinkBack Thread Tools Display Modes
  #1  
Old 06-08-2005
Registered User
 

Join Date: Jun 2005
Location: New York City
Posts: 95
Limitations of awk? Good idea? Bad idea?

Keeping in mind that I'm relatively comfortable with programming in general but very new to unix and korn/bourne shell scripts..

I'm using awk on a CSV file, and then performing calculations and operations on specific fields within specific records. The CSV file I'm working with has about 600 records, with 30 fields.

Since I want to be able to cross-reference one record against all other records multiple times for different/specific records that need to be checked (for example if a record orders an amendment to a previous record, I'd like to check to see if the previous record still exists), and since, I think, awk only runs through each record once, I wanted to know if it's possible, and if so, if it's even a good idea to store all the fields from each record that I need (I don't need all of them, just maybe a third of the information) into an array in awk, and then return the array itself, outside of awk.

Possible? Good idea, bad idea? I'm open to all suggestions and advice.
Note: I don't know if it's possible to run the loop for each record again once you're in the awk program, for example: a nested awk where in the middle of the loop for the host awk, you can start a second awk to check the current record in the original awk against all other records.
Reply With Quote
Forum Sponsor
  #2  
Old 06-08-2005
vgersh99's Avatar
Moderator
 

Join Date: Feb 2005
Location: Boston, MA
Posts: 3,003
It's somewhat difficult to follow the thread, but.... I think I understand what you're after.

Usually you don't to store ALL the records, but rather one record per key - where the 'key' is your 'unique identifier' for the type of a record.

Awk has a notion of the 'associative arrays' - very similar to perl's hashes. You can store by a 'key', update by a 'key' and lookup by a 'key'.

You can process your CSV once, store the relative pieces of a file in the array, manipulate the array cells as you go AND post-process the content of the array in the awk's 'END' session [by iterating through the array: for( i in array)].

The associative arrays are dynamic in nature and their size is bound only by the available memory. Seems like you don't have huge files to process - you should be ok.
Reply With Quote
  #3  
Old 06-08-2005
Registered User
 

Join Date: Jun 2005
Location: New York City
Posts: 95
Thumbs up Thanks again.

Sorry about the lengthy thread, I usually try to make my inquiries with as much detail as possible to avoid confusion but half the time I guess I achieve the opposite.

Yes, you understood my thread perfectly. And yes, I think that will be enough for what I wish to do now.
Reply With Quote
Google The UNIX and Linux Forums
Reply

Thread Tools
Display Modes




All times are GMT -7. The time now is 02:32 AM.


Powered by: vBulletin, Copyright ©2000 - 2006, Jelsoft Enterprises Limited.
The UNIX and Linux Forums Content Copyright ©1993-2008. All Rights Reserved.Ad Management by RedTyger Visit The Complex Event Processing Blog

Content Relevant URLs by vBSEO 3.2.0