Shell Programming and Scripting

I've got a very peculiar situation. I'm trying to find out if we can compare null fields with non-null. I've output csv files from SQL and Oracle. I need to compare each field from the files, and then find out any differences. The files usualy have over 500 fields, and send the resule to DBA.
Here is the intersting part, some fields may have different value, but thats ok. Like time stamps from SQL and Oracle. What I really need is to compare fields with null and non-null values. If SQL has non-null value and corresponding Oracle also has non-null then it's good. Though the values can be different some times. But a null value and same field on other file has non-null value, then I would like to extract that and report.
Any idea or suggestions? I've been looking in to diff, cmp, and even awk.

Thanks in advance

-Nitin

Can we see an example of the input?

can you show the sample input files?

Here is a sample data:

"","","SGP0000","Yes","Support","100006;","","charters","","Charters","xxx@xx.com","","9/29/2006 10:00:37 AM","9/29/2006 10:07:36 AM","10/24/2006 8:14:18 AM","Charters"

I've truncated most of the fields. So, there are some fields which may have time and date stamps etc. Lets compare the first field, it is null. Well it has "" that but for database standpoint it is null. I guess I could strip that using vi and really make it a null. The other file may have some thing like this:

"Charter group","","SGP0000","Yes","Support","100006;","","charters","","Charters","xxx@xx.com","","10/30/2006 10:00:26 AM","10/31/2006 10:11:45 AM","10/29/2006 9:18:21 AM","Charters"

So I would like to report that the first field has a diff. However, in field 13, 14, time stamps are different, I don't want that to be reported. The problem is that I can't always tell where these time fields will be. That usually depends on the DBA and developer, how they out put the data.


Thanks,
Nitin

Thinking on this, but, how would you tell the difference between a NULL and an empty string?

Well, a completly empty string is not possible. I mean a complete empty record. We can break the string using "awk -F," and then compare each and field. So the question is, can awk compare null and non-null?
Another thing I was gonna ask is if shell can handle dynamic arrays?

Ok.. can this be done in Perl? Any one?


-Nitin

Can this be done in perl? I've some exposure to Perl, any pointers....


-Nitin