How to 'improve' this script and also 'fix' the pattern matching part?
Hi all,
Below is my script. It is currently working but I want some advice on maybe improving it and need some help on the pattern matching
So basically I pass a log file and need to search for a block of text.
As you can see from the script, I search for the string "Fatal NI connect error 12170." and add 17 to it, that will be the block of text that am after. Then I print this block of text to a file and grep the 3 basic information that I am after which is Time, Client address and the WARNING. The WARNING line sometimes exist and sometimes it doesn't.
So far, running the script does most of what I wanted.
Sample output below:
Sometimes there are some 'malform' of some sort, the ones in RED above. And this is because sometimes the log contain something like below which I am not expecting it to have
Or something like below:
As a workaround to this malform, I am changing the script to check if the 7th line is not "^Time" then I am excluding this block of text. I can't find any way of matching a block of text starting from " Fatal NI connect error 12170" to "WARNING: inbound connection timed out (ORA-3136)". Unfortunately, sometimes it does not end with "WARNING: inbound connection timed out (ORA-3136)", instead the last line is "Client address: (ADDRESS=(PROTOCOL=tcp)(HOST=11.111.22.36)(PORT=64136))" and HOST and PORT changes.
Below is the script with the 'check' for the malform. The lines in RED.
Anyway, please advise if there is any better way of doing this the way it is now.
Last edited by newbie_01; 10-22-2019 at 10:14 AM..
Reason: Updated script
Your script seems a bit intricate. E.g. when using the > redirection, you don't need to create / truncate the file upfront. And, with that many temp files, there must be a better approach. How far would
get you for the first part of your task- in the good case?
Now, if the "WARNING" line is missing sometimes, but the "client address" line is always there, why not use that and add a condition to print the "WARNING" line individually? Is the "WARNING" line always immediately following the "Client" line?
And. looks like your logger sometimes misbehaves by mixing logs of two independent events. If you can't remedy that in the originator, you'll need additional coding on the receiving side, and that can't be done in sed, but needs tools like awk, perl, or similar. Are the log lines in relative order, i.e. the second log line consistently belongs to the second event?
Your script seems a bit intricate. E.g. when using the > redirection, you don't need to create / truncate the file upfront. And, with that many temp files, there must be a better approach. How far would
get you for the first part of your task- in the good case?
Now, if the "WARNING" line is missing sometimes, but the "client address" line is always there, why not use that and add a condition to print the "WARNING" line individually? Is the "WARNING" line always immediately following the "Client" line?
And. looks like your logger sometimes misbehaves by mixing logs of two independent events. If you can't remedy that in the originator, you'll need additional coding on the receiving side, and that can't be done in sed, but needs tools like awk, perl, or similar. Are the log lines in relative order, i.e. the second log line consistently belongs to the second event?
Hi RudiC
Thanks for you reply. Very helpful as usual. Yeah I shouldn't need to pre-create the tmpfiles, just being paranoid I guess, I've since change it to use $$ as well as I need to run the script with several log files.
The sed syntax
sure speed things up and it indeed gets most of the pattern match that I am after.
Yes, the Client line is always there. The WARNING line is there most of the time, I am mainly wanting to include to check if the error that comes with the WARNING is always ORA-3136 or not. The HOST and PORT is always going to be different though in most cases. I tried using
and that works fine so I'll stick to doing that.
How do I get it to print the Time, Client and WARNING only though? At the moment, I am running the same command 2x/3x, depending on whether I want to include WARNING, and then grepping for Time, Client and WARNING and then combining the 2/3 files output.
When you say
Quote:
And. looks like your logger sometimes misbehaves by mixing logs of two independent events
, are you referring to when I am doing the if/then/else check of where I expect to see the Time, Client or WARNING line to appear? If so, that is because sometimes there is some malform in the log, where it gets too much of these errors happening an the way it get printed in the log file is not in the order that I am expecting it to look like
Hi All,
My Problem is like below.
I have a file which contains just one row and contains data like
PO_CREATE12457888888888889SK1234567878744551111111111SK89456321145789955455555SK8888888815788852222
i want to extract SK12345678
SK89456321
SK88888888
So basically SK and next 8... (4 Replies)
I have a big perl script need to fix a small pattern matching inside ..
I have patterns like
create unique index
create index
The pattern matching should look for both the pattern in the same statement, The existing matching looks for only "create unique index"
The exising code for this i... (6 Replies)
I am still learning expect, For the below script I always get This is True as the answer. Tried to debug and does not make sense to me. Not sure, where I am doing the mistake. Need Help! - Thanks
#!/usr/bin/expect -f
set timeout 10
spawn -noecho bash
expect { *$* }
send "test -d... (3 Replies)
Hi,
I am trying to write a script that list down all the files that do not match the pattern
My pattern will be like this "*.jpg|*.xml|*.sql". This pattern will be stored in a file. The script need to read this pattern from the file and delete the files that does not match this pattern. It... (7 Replies)
Hi,
I am using following command to extract string from a file. String will be after last / (slash).
awk -F\ / '{print $NF}' $FILE
but while appending the output in file in script, it dosent work. File created but of zero size... can anyone please help
`awk -F\\\/ '{print $NF}' $FILE` >... (3 Replies)
Hi,
I need to create a shell script through which i need to populate email addresses in email columns of database table in mysql. Let say if email contains yahoo, hotmail, gtalk than email addresses need to move in their respective columns.
# !/bin/sh
yim="example@yahoo.com"... (3 Replies)
Hello everyone, this is my first post so please give me a hand.
I apologize for my English, I'll try to be clear with my request.
I need to write a script (Bash) which finds all the variables defined in the file .h of the folder and then writes the name of the files .c where these variables are... (1 Reply)
I'm looking for a way to match a particular string in another string and if a match is found execute some command.
I found the case statement can be used like this;
case word in
)
command ;; ] ...
esac
If my string to find is say "foo" in the string $mystring... (1 Reply)
Hi guys, i have been struggling to achieve the following and therefor looking for some help.
I am trying to write something that will summerize the following output from my switches for daily totals.
Basicly if $1 $2 $3 doesnt change, we can keep adding up $4. Probably would use a awk print end... (3 Replies)
Hi,
I have a log file which shows the files which has been changed over the last week.
They follow this pattern:
old_file_version_number@@new_file_version_number
Now I need to know how to delete from each line parts starting from @@.
I would be issuing the command inside vi(m).
So... (5 Replies)