Programming

Well, please let us know when you do. Until your data format stops changing, writing a program for it will be a waste of time.

You could use blank lines to separate different objects perhaps. awk is pretty good at processing large amounts of text. That's what its designed for after all. This is true. It's still possible to write bad, slow code in any language, though -- not to mention, reinvent the wheel. I recall a thread like this where the poster was in essence reinventing awk slowly, painfully, and badly, refusing all suggestions of better solutions because he wanted to use a 'fast language'. Well, awk is written in C++ too.

I appreciate the feedback. I know what you mean by people trying to reinvent the wheel, it's hard to stop them once they´re convinced their idea is the solution they should go for. But if they don't even take criticism then it's even a worse thing! I think of this as some kind of syndrome where the coder just mentally shuts off and ignores all incoming information to exclusively focus on a unnecessarly complex and technical approach.

Re-writing programs in more efficient/easier to maintain languages as new powerful tools are getting released is something I´ve always kept an eye on. The challenge of course is to find a budget for purely technical upgrades. Very often you need a functional/business requirement to justify such rewrite.

About the original query, I mean I´ll just go for awk (which I´m confortable with anyway) and store all the mappings in a control file. There will be new tags, new fields added/changed in the future so I'll try to keep this configurable, maybe by making this file as a periodic report from a GUI screen where some key users can change the settings. As always, depends on how much intelligence people want to throw into that thing.

In this example, the first column, e.g., Record_key_1, did not find a home inside the xml. Many attributes, being unique at their level, are more compactly stored in Attributes of the Element, not contained in an element.

Code:

<OuterWrapper>
  <Record ID=Record_key_1>
    <Item col=1 val=A/>
    <Item col=2 val=B/>
    <Item col=3 val=C/>
   </Record>
 .
 .
 .
 </OuterWrapper>

Double quotes are necessary when attribute values have embedded whitespace, and url-encoding if containing: " < > ' & ( &quot; &lt; &gt; &apos; &amp; )
http://en.wikipedia.org/wiki/XML#Escaping