Shell Programming and Scripting

Hi,

I am looking for a ready shell script that can help in loading and validating a high volume (around 4 GB) .Dat file . The data in the file has to be validated at each of its column, like the data constraint on each of the data type on each of its 60 columns and also a few other constraints on each of the column.

After validation, the rows wiith error records need to be written to an exception report and the correct data has to be loaded separately.

If anyone has an idea or any ready to use shell script which can customised for my requirement it will be greatly appreciated.

There is no "ready shell script" for such a task. Provide a sample of the data file. Be specific about the rules for validating each column (field) for a line (record) of the data file.

I have attached zip file which contains two files.
1.) ipbcdg_check_fotp_20130909_2.dat ( This file contains data which we have to validate. Tilde sign(~) is used to seperate the different Columns values.
2.)Data_Validation_Sheet.xls ( This files contains all the rules, Against which we have to validate the Data.)

In Data validation file we have different sheet which are for different-2 interfaces e.g "cash","check" and two differnet sheets for Rules and lookup tables.

How to find for which interface we have to validate :-
In dat file name we have "check_fotp" which is our interface name.
If you open Data validation file we found one sheet with name "Check" and in that sheet for first column we have two different values . "Check_fot" and "check_fotp".

Now corresponding to "Check_fotp" values we have columns name which we have to validate in the dat file.

E.g we take first row value corresponing to "Check_fotp"
check_fotp, Transaction Reference Identifier, CHARNUM(50) ,Y, "RC_1", "RC_2", "RC_3", "RC_7"

The values "RC_1", "RC_2", "RC_3", "RC_7" have rules which we have to validate and these descriptions are present in different sheet in "Data validation file" , sheet name s"REF_CD_DETAIL".

Some time we have differnt values in the Rule column. like in 3 row corresponding to "check_fotp" we have one value "Party_Role"

This means if any value occured we have to check "lookup" sheet in the "data validation sheet " and lookup sheet we have one table corresponding to "Party_Role" which have some values.
So in dat file for this columns values should be from this "Party_Role" table only.

I know its complicated. If you need any other information please let me know.

Thanks in advance for your help.