Shell Programming and Scripting

Need to develop a unix shell script for the below requirement and I need your assistance:

1) search for file.log and file.bad file in a directory and read them
2) pull out "Load_Start_Time", "Data_File_Name", "Error_Type" from log file
4) concatinate each row from bad file as "Error_Description"
5) write to a output flat file record by record
6) make sure the "number of bad record count from log file = number of records written to the output flat file"
7) below are the formats of log and bad file and output file

Log File:
=========
Load started at: 1-Oct-10 11:00:00 EDT
Database: Oracle
Tablename: abc_def_123
Datafile: /abc/def/ghi/inbound/files/file.dat


Error_Type:
-------------------------------------------------
1: 123(456) [CHAR(1)] value too long for column]
2: 123(456) [CHAR(1)] value too long for column]
3: 123(456) [CHAR(1)] value too long for column]
4: 123(456) [CHAR(1)] value too long for column]

number of bad records: 4

Bad File:
========
111 456 abc 789 2010-10-1 manager US
222 456 abc 789 2010-10-1 employee US
333 456 abc 789 2010-10-1 contractor US
444 456 abc 789 2010-10-1 partner US


Output File:
===========
Load_Start_Time Data_File_Name Error_Type Error_Description
1-Oct-10 11:00:00 file.dat 1: 123(456) [CHAR(1)] value too long for column] 111 456 abc 789 2010-10-1 manager US
1-Oct-10 11:00:00 file.dat 2: 123(456) [CHAR(1)] value too long for column] 222 456 abc 789 2010-10-1 employee US
1-Oct-10 11:00:00 file.dat 3: 123(456) [CHAR(1)] value too long for column] 333 456 abc 789 2010-10-1 contractor US
1-Oct-10 11:00:00 file.dat 4: 123(456) [CHAR(1)] value too long for column] 444 456 abc 789 2010-10-1 partner US

Welcome to the UNIX and Linux forums!

Everyone at the UNIX and Linux Forums gives their best effort to reply to all questions in a timely manner. For this reason, posting questions with subjects like "Urgent!" or "Emergency" and demanding a fast reply are not permitted in the regular forums.

For members who want a higher visibility to their questions, we suggest you post in the Emergency UNIX and Linux Support Forum. This forum is given a higher priority than our regular forums.

Posting a new question in the Emergency UNIX and Linux Support Forum requires forum Bits. We monitor this forum to help people with emergencies, but we do not not guarantee response time or best answers. However, we will treat your post with a higher priority and give our best efforts to help you.

If you have posted a question in the regular forum with a subject "Urgent" "Emergency" or similar idea, we will, more-than-likely, close your thread and post this reply, redirecting you to the proper forum.

Of course, you can always post a descriptive subject text, remove words like "Urgent" etc. (from your subject and post) and post in the regular forums at any time.

Thank you.

The UNIX and Linux Forums

Got your message.

If possible please delete my thread.

Anyways I will repost it again.

Thank you for the details.

The sed tool is pretty good at this sort of thing, except line counting: the closest it gets is line numbers. PERL, C. C==, JAVA, etc. can do it, being full languages. You can use a combination of shell tools with sed to break up the information into multiple files and measure the files, processing one set of data from a file at a time, if necessary. Trying to do it all in shell itself might be too slow.