Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Extract repeating and check for for certain error message

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Extract repeating and check for for certain error message
# 1  
Old 10-30-2018
Extract repeating and check for for certain error message
Hi guys

I have an log from an regular application processing where the processing is done every 10 minutes and is written to the same log which is rotated on daily basis.

the block of log starts always with "Starting" and ends with a certain pattern "Completed"

The catch is I want to create an shell script for checks if an Oracle ORA error is found in this log block and if yes this is collected and send after whole log was checked to several recipients.

I have an idea how to do the whole thing what I have an issue is the part of script which checks in the log block for the ORA message


how does the log look like:

Code:
Starting
14:10:*
14:10:* some stuff
14:10:*
14:10:* blabla
Completed


Starting
14:20:*
14:20:* some stuff
14:20:* ORA- error constaint voilated
14:20:* blabla
Completed

Starting
14:30:*
14:30:* some stuff
14:30:* blabla
14:30:* blabla
Completed

Starting
14:40:*
14:40:* some stuff
14:40:* ORA- error constaint voilated
14:40:* blabla
Completed

Starting
14:50:*
14:50:* some stuff
14:50:* blabla
14:530:* blabla
Completed

the final report should contain in the blocks

Code:
Starting
14:20:*
14:20:* some stuff
14:20:* ORA- error constaint voilated
14:20:* blabla
Completed

Starting
14:40:*
14:40:* some stuff
14:40:* ORA- error constaint voilated
14:40:* blabla
Completed

keep in mind that the number of lines between Starting and Completed isnt fixed but may change

the bocks can be extracted like this: sed -n '/StartPattern/,/EndPattern/p' FileName
$ awk '/abc/{flag=1;next}/mno/{flag=0}flag' file


now I could do the grep -A50 -B50 'ORA-' file >> report but this would extract more what I like, I want to make the report clean.

One idea was maybe to get the Line number when the error ORA appers and then do an row-1 until Starting is found and row+1 until the Completed is found and get those line numbers and to an sed -n 'X,Yp' file

Any suggestion thank you in advance.

Martin
Last edited by vgersh99; 10-30-2018 at 04:40 PM.. Reason: Code tags, please!
# 2  
Old 10-30-2018
how about:
Code:
awk '/^Starting.*Completed$/ && /ORA- error/' RS=  ORS='\n\n' myFile

This User Gave Thanks to vgersh99 For This Post:
kl1ngac1k
# 3  
Old 10-30-2018
If you're only processing completed log files, vgersh99's suggestion can be simplified a little bit:
Code:
awk '/ORA- error/' RS=  ORS='\n\n' myFile

If you are processing active log files and you want to be sure that the output only contains complete entries, you could still simplify it slightly:
Code:
awk '/Completed$/ && /ORA- error/' RS=  ORS='\n\n' myFile

If the trailing empty line at the end of the output produced by vgersh99's suggestion and the above two modifications bother you, you could also try:
Code:
awk '/ORA- error/{if(c++)print "";print}' RS= myFile

or:
Code:
awk '/Completed$/ && /ORA- error/{if(c++)print "";print}' RS= myFile

This User Gave Thanks to Don Cragun For This Post:
kl1ngac1k
# 4  
Old 10-31-2018
Hi

thank you so much Smilie
Code:
awk '/ORA- error/' RS=  ORS='\n\n' myFile

and also
Code:
awk '/^Starting.*Completed$/ && /ORA- error/' RS=  ORS='\n\n' myFile

does the trick pretty good but here
Code:
awk '/^Starting.*Completed$/ && /ORA- error/' RS=  ORS='\n\n' myFile

I have to remove the ^Starting and Completed$ since they are not the first/last things in the line

Moderator's Comments:
Mod Comment Use code tags or feel the wrath of Sauron
# 5  
Old 11-02-2018
Hi guys

is is there a possibility to include the filename in the resulted report only for the log name if there contain such a lines ? thank you

will use the code block sorry about that
# 6  
Old 11-02-2018
Code:
awk '/ORA- error/{print FILENAME}' RS= myFile

# 7  
Old 11-02-2018
cool one step closer:

I want something like this

Code:
### ORA found in log: /app/logs/processing1.log
Starting
14:20:*
14:20:* some stuff
14:20:* ORA- error constaint voilated
14:20:* blabla
Completed

### ORA found in log: /app/logs/processing5.log
Starting
14:40:*
14:40:* some stuff
14:40:* ORA- error constaint voilated
14:40:* blabla
Completed
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Script to check the string in a file and print an attribute in the message

Hi, I am new to shell scripting and got a task to complete. Task is : we have a log file where in i need to traverse through the whole file to check the string "Person Type missing in message" and after that i need to get EMPLID=xxxxxx from the file and print details in a different file. ... (1 Reply)
Discussion started by: suren424
1 Replies

2. Shell Programming and Scripting

sed to extract a multiline subject from a mail message

I'm trying to extract a subject from a mail message but my subject example has 2 lines. How can I manage to extract it and write a string at the end of it? Consider this example: From: test@domain.com Subject: Re: U =?ISO-8859-1?Q?qu=EA=3F!=3F!=3F!!_wtff_=E7=E3o_=ED=F3?= ... (6 Replies)
Discussion started by: twisterbr
6 Replies

3. Shell Programming and Scripting

Shell script to extract data in repeating tags from xml

Hi, I am new to shell scripting. I need to extract data between repeating tags from an xml file and store the data in an array to process it further. <ns1:root xmlns:ns1="http://example.com/config"> <ns1:interface>in1</ns1:interface> <ns1:operation attribute1="true" attribute2="abd"... (2 Replies)
Discussion started by: sailendra
2 Replies

4. Shell Programming and Scripting

Extract certain entries from big file:Request to check

Hi all I have a big file which I have attached here. And, I have to fetch certain entries and arrange in 5 columns Name Drug DAP ID disease approved or notIn the attached file data is arranged with tab separated columns in this way: and other data is... (2 Replies)
Discussion started by: manigrover
2 Replies

5. Shell Programming and Scripting

Extract XML message from a log file using awk

Dear all I have a log file and the content like this file name: temp.log <?xml version="1.0" encoding="cp850"?> <!DOCTYPE aaabbb SYSTEM '/dtdpath'> <aaabbb> <tranDtl> <msgId>000001</msgId> </tranDtl> ..... </aaabbb> ... ... (1 Reply)
Discussion started by: on9west
1 Replies

6. UNIX for Dummies Questions & Answers

Extract repeating data from file

I want to extract the last rows of a data file, similar to that one below: C1 xxx C2 rrr C3 ttt .... Cn-1 hhh Cn bbb C1 yyy C2 sss C3 uuu ... Cn-1 iii Cn ccc ... I just want to extract the final rows between C1 and Cn at each data file. n is not a constant,... (2 Replies)
Discussion started by: natasha
2 Replies

7. UNIX for Dummies Questions & Answers

Unable to extract a tag from a very long XML message

Hi I have a log file which contain XML message. I want to extract the value between the tag : <businessEventId>13201330</businessEventId> i.e., 13201330. I tried the following commands but as the message is very long, unable to do it. Attached is the log file. Please provide inputs. --... (3 Replies)
Discussion started by: Sapna_Sai
3 Replies

8. Shell Programming and Scripting

extract message

thx thx (1 Reply)
Discussion started by: ust
1 Replies

9. Shell Programming and Scripting

Need to extract XML message from log

I need help to extract a following SOAP-ENV:Header XML message from the log. XML message need to be extracted: *************************** <SOAP-ENV:Header> <ServiceGatewayHeader> <SourceApplicationId>OXL</SourceApplicationId> <Version>1.0</Version> <UserId>TEST</UserId>... (4 Replies)
Discussion started by: tjshankar
4 Replies

10. UNIX for Advanced & Expert Users

repeating kernel message

Hi, I am getting multiple repeats of the following message in /var/log/messages Jul 26 13:36:04 linuxnol kernel: cdrom: open failed. Jul 26 13:36:11 linuxnol kernel: cdrom: open failed. Jul 26 13:38:18 linuxnol kernel: cdrom: open failed. Jul 26 13:38:26 linuxnol kernel: cdrom: open... (2 Replies)
Discussion started by: progressdll
2 Replies
Login or Register to Ask a Question