|
|
|
google unix.com
|
|||||||
| Forums | Register | Forum Rules | Links | Albums | FAQ | Members List | Calendar | Search | Today's Posts | Mark Forums Read |
| Shell Programming and Scripting Post questions about KSH, CSH, SH, BASH, PERL, PHP, SED, AWK and OTHER shell scripts and shell scripting languages here. |
More UNIX and Linux Forum Topics You Might Find Helpful
|
||||
| Thread | Thread Starter | Forum | Replies | Last Post |
| read a file in shell and put result in a line | lottiem | Shell Programming and Scripting | 6 | 07-06-2009 05:16 PM |
| read list of filenames from text file and remove these files in multiple directories | fxvisions | Shell Programming and Scripting | 5 | 08-07-2008 04:59 PM |
| Compare 2 files and output result in pop up window | hongsh | UNIX for Dummies Questions & Answers | 2 | 05-23-2008 10:50 AM |
| Need to Output result to a non-delimitted file | cosec | Shell Programming and Scripting | 1 | 04-07-2008 05:51 AM |
| any possible to run sql result and output to file | happyv | Shell Programming and Scripting | 7 | 03-02-2007 01:07 PM |
Powered by 
|
|
|
LinkBack | Thread Tools | Search this Thread | Rate Thread | Display Modes |
07-20-2009
|
||||
|
||||
|
Read multiple log files and create output file and put the result
OS : Linux 2.6.9-67 - Red Hat Enterprise Linux ES release 4 Looking for a script that reads the following log files that gets generated everynight between 2 - 5am Code:
Master_App_20090717.log Master_App1_20090717.log Master_App2_20090717.log Master_App3_20090717.log Master_App4_20090717.log Master_App5_20090717.log One exception: The log file Master_App_20090717.log gets data written into every 30 minutes in the 24hr period. This data is appended into the file and remains there. But each time the extract and transload start it start with the current time stamp in it begin line and end with current time stamp for a successful end or it could be Failed end. In any case those entries remain in the file until we fix the issue and the next load starts and the results of next load will append to the same log file for that same day and will starts with the same begin line. So we need to make sure that script should not report previous failure that happend 35 mins ago and was reported by the same script 30 minutes ago. So we need to place a check that if it does find a failed end and it is more then 35 minutes ago then ignore it and continue reading the file and then report accordingly. We will run this script in crontab One script will run every morning after 5:00 am once all the load is finished or we will run 5 script seperately after a specific time to check the status The other script will run 5 minutes pass every 30 minutes. Code:
*************************************************************************************************
Success Message:
2009-02-25 07:24:24]Master_App start_time=[20090213 000000] end_time=[20090213 002959] ... begin
[Extract-2009-02-25 07:24:24]Execute Extract_App ... Begin
[Extract-2009-02-25 07:24:24]Execute ex_nhv_104052_s ...
[Extract-2009-02-25 07:26:14]Execute ex_nhv_104052_s ... Success
[Extract-2009-02-25 07:26:14]Execute ex_nhv_104114_s ...
[Extract-2009-02-25 07:28:06]Execute ex_nhv_104114_s ... Success
[Extract-2009-02-25 07:28:07]Execute nhv_104094_s ...
[Extract-2009-02-25 07:29:58]Execute nhv_104094_s ... Success
[Extract-2009-02-25 07:29:58]Execute Extract_App ... Success
[Transload-2009-02-25 07:29:59]Execute Transload_App ... Begin
[Transload-2009-02-25 07:29:59]Execute tl_nhv_ua_merge ...
[Transload-2009-02-25 07:30:14]Execute tl_nhv_ua_merge ... Success
[Transload-2009-02-25 07:30:16]Execute tl_nhv_104094_s ...
[Transload-2009-02-25 07:39:56]Execute tl_nhv_104094_s ... Success
[Transload-2009-02-25 07:39:57]Execute Transload_APP ... Success
Archiving data files ... Success
[2009-02-25 07:39:58]Master_App start_time=[20090213 000000] end_time=[20090213 002959] [Successful end]!
*************************************************************************************************
Failure Message:
[2009-02-25 08:55:55]Master_App start_time=[20090213 003000] end_time=[20090213 005959] ... begin
[Extract-2009-02-25 08:55:55]Execute Extract_App ... Begin
[Extract-2009-02-25 08:55:55]Execute ex_nhv_104052_s ...
[Extract-2009-02-25 08:57:43]Execute ex_nhv_104052_s ... Success
[Extract-2009-02-25 08:57:43]Execute ex_nhv_104114_s ...
[Extract-2009-02-25 08:59:33]Execute ex_nhv_104114_s ... Success
[Extract-2009-02-25 08:59:33]Execute nhv_104094_s ...
[Extract-2009-02-25 09:01:34]Execute nhv_104094_s ... Success
[Extract-2009-02-25 09:01:34]Execute Extract_App ... Success
[Transload-2009-02-25 09:01:35]Execute Transload_eHealth_h ... Begin
[Transload-2009-02-25 09:01:35]Execute tl_nhv_ua_merge ...
[Transload-2009-02-25 09:01:51]Execute tl_nhv_ua_merge ... Success
[Transload-2009-02-25 09:01:52]Execute tl_nhv_104094_s ... ERROR: ORA-03114: not connected to ORACLE DECLARE Archive_Files.prg bak exp_prdmmp_dev_20090123.dmp.gz......
....................................................ERROR at line 1: ORA-03113: end-of-file on communication channel SP2-0670: Internal number conversion failed Usage: { EXIT | QUIT } [ SUCCESS | FAILURE | WARNING | n | <variable> | :<bindvariable> ] [ COMMIT | ROLLBACK ]
[Transload-2009-02-25 09:33:19]Execute tl_nhv_104094_s ... Fail
[Transload-2009-02-25 09:33:20]Execute Transload_App ... Fail
[2009-02-25 09:33:20]Master_App start_time=[20090213 003000] end_time=[20090213 005959] [Failed end]!
*************************************************************************************************
Output file : We could have 2 output files one for script that runs after 5 am and one for every 30 minute script or we could have 5 output files one for each log plus one for 30 minutes script output log so altogether 6 outfiles. In the output file First Delete the content of the file if any then Leave blank if it is successful Write "ETL Load Failed" if it finds any failed end. ---------- Post updated at 10:33 AM ---------- Previous update was at 10:32 AM ---------- Thank you in advance for your Help as i am desperately seeking a solution. Thank you and all the best Last edited by vgersh99; 07-20-2009 at 03:42 PM.. Reason: code tags, PLEASE! |
| Bookmarks |
| Thread Tools | Search this Thread |
| Display Modes | Rate This Thread |
|
|
Threaded Mode
