Shell Programming and Scripting

Hi All,
We need to use & execute some several scripts before loading the data successfully using Informatica.
Can you help me the below scenario:
We will be receiving the files from different countries, and using ftp all the files will be placed onto the server. The file structure will be (.csv file which contains, file header, document header, line items)

Code:

2011032419004000_GB_004_FEED01_0001_001link002-accoounting file
2011032419000400_GB_005_FEED01_0002_001link002-reporting file

(Out of scope for some interfaces)
-The above filename specifies as Datetime in format YYYYMMDDH24MISS(Prod. date time of the file=2011032419004000)
- Next 2 letter specifes the country name(Eg:Great Britian=GB, it will change per country, IT-Italy)
-Alpha numberic Interface Id(004_FEED01)
-A run number(0001)(Sequential number)
-Next, Link code to link 2 files. The feeder systems will use link code like 001link002 and send the files.
Our requirement is:
1) Filename validation if it is successfuly, we need to execute step2, else notify the source system that file name is wrong.
2) If it's successful, again we need to check whether it is duplicate and we need to infrom to source system(country's feeders)
3)If it is unique, check whether the file is in sequence. If it is not in sequence, we need to raise the warning in the error table & continue the load.
4)If the file is in sequence, perform the priori check(wait till the arrival of cluster accounts file)
5) If priori check fails, load aborted,
6)If priori check successful, move the source file to landing pad directory
7) Read the source file and perfrom audit trail of the detail records(check whether line item total record count and balances matches with the document Header Record count & balances
8) check whether detailed records total record count and balances matches with the File Header Record Count & balances
9) If it matches, perfrom audit trail of line item total record count and ablances matches with File document Record count & balances. If it is not matched, notify the source system
10) Check the line item records total count and balances matches with document Header Record Count and balances.
11) If the line item records total record count and balances not matches with the document Header Record Count and balances, raise an appropriate error and log the error in the error table, and notify the source system & abort the load.
Please, give some ideas for implementing these scenarios using scripts.

Thx

You could use awk and especially it's substr function.

Hi zaxxon,
Thanks for the idea. If possible, Can i have the script for the above scenario's please.
Thx

This is a good topic, we'd like to help.

But before we do all works for you, can you please show us what you have done?

Hi All,
Can you please, help me out for the above scenario. If possible, provide any script for the above scenario.

Good scenarios posted by psr2009 but we do not write the entire script with the above resuirements, it would be possible through step by step.

As I feel with the above steps it is not only script but you may need to look from the Informatica (your integration part) level as well.

Hi All,
It's right. I have the whole idea on the Informatica also. We can do some of the steps through Informatica without using Unix as well. We have done some POC stuff regarding this through Informatica already, it went well. My approach is do we get enough scripts for all the steps or not. Even some step by step is also can be done by some simple scripts also. If you can share these scripts, it will be helpful for me.

Thx