In the first message you said you want to set SEV to the value following the ": " on a line starting with "Severity: ".
In the last message you said you want the value of $SI printed following "Server Instance Error: ", but the error message you want this to come from had a line: Server Instance: 31415, but nothing with " Error" anywhere in the message.
I will ask again: Given the following event message text:
Code:
Sept 21, 2012 8:21:55 PM MDT - Device Main_Server_01 of type Host_Server is no longer responding to requests. This condition has persisted for more than 3 minutes.
Alert Code: Danger
Server Instance: 31415
Severity: Critical
exactly what output do you want to be produced as a result of processing this message?
And, is there always an empty line as the 2nd line of every event message?
This User Gave Thanks to Don Cragun For This Post:
In the first message you said you want to set SEV to the value following the ": " on a line starting with "Severity: ".
In the last message you said you want the value of $SI printed following "Server Instance Error: ", but the error message you want this to come from had a line: Server Instance: 31415, but nothing with " Error" anywhere in the message.
I will ask again: Given the following event message text:
Code:
Sept 21, 2012 8:21:55 PM MDT - Device Main_Server_01 of type Host_Server is no longer responding to requests. This condition has persisted for more than 3 minutes.
Alert Code: Danger
Server Instance: 31415
Severity: Critical
exactly what output do you want to be produced as a result of processing this message?
And, is there always an empty line as the 2nd line of every event message?
First question: I would want to know how to pull out any of the alert attributes, say Severity: Critical or Alert Code: Danger.
So I guess the actual data of "Danger" or "Critical" is what I am after. Please note that I am dealing with 40,000+ different event files, which means that there is the potential of 40,000 + different requirements.... In reality there is a lot less, but the point is, there is not a default or standard event file. I just pulled one out at random.
Second question: Typically there is a blank line on the 2nd line... but that rule isn't 100% followed. The greatest share of the event files that I deal with do have the space between the top line and the detail lines below it.
thanks so much for taking time in helping me out!!!!!
First question: I would want to know how to pull out any of the alert attributes, say Severity: Critical or Alert Code: Danger.
So I guess the actual data of "Danger" or "Critical" is what I am after. Please note that I am dealing with 40,000+ different event files, which means that there is the potential of 40,000 + different requirements.... In reality there is a lot less, but the point is, there is not a default or standard event file. I just pulled one out at random.
Second question: Typically there is a blank line on the 2nd line... but that rule isn't 100% followed. The greatest share of the event files that I deal with do have the space between the top line and the detail lines below it.
thanks so much for taking time in helping me out!!!!!
OK. We're starting to get somewhere, but you are making it very hard to extract a set of requirement to figure out what you want.
What you have said you want is as follows:
Before you call the script you want us to write, you will set $EventMessage to be a shell variable that contains exactly one event message.
You want to output to be a list of alert attributes contained in that EventMessage.
Is that correct?
In the message I've quoted from above, you're saying that either of two outputs will be acceptable:
Code:
Critical
or
Code:
Danger
but I can't believe that is really what you want.
Please give us an exact EventMessage (in code tags) and the exact output you want to see (in code tags) for that message. And then describe the process by which you determined that was the output you wanted to extract from that message.
If you provide a list of the strings that precede the colon on a line in an event message that identify that line as an alert attribute, we can easily do that. But, with 40,000 event message types, we have no way to guess what you consider to be an alert attribute if you don't give us a description of how we determine that a line in an event message is an alert attribute.
If you want us to read event zero or more event message from standard input, we can do that too. But, you'll need something else in the output to identify the context of the message that generated the alerts. To do that you need to explicitly specify how we find the data in the event message you want printed. (Note that saying you want "ModelType={t}, ModelName={m}" doesn't help. You used that example before when talking about the sampe event message, but the strings ModelName and ModelType don't appear anywhere in that event message, and you haven't given us a clue as to how to parse the meta-notaions {t}, {m}, {I 2}, or {S 1} (although I'm making a wild guess that {I 2} means that the data for that field is a textual representation of a two byte integer value).
This User Gave Thanks to Don Cragun For This Post:
Maybe it would be easier if I started from the beginning... I think I may have confused the issue big time.
There is a directory on my server that has 40K+ files in it. If you opened up the folder where these files are located you would see something like this:
Code:
Event00a3ff01
Event00a3ff02
Event00a3ff03
etc. etc. etc.
As you can see, every event file name starts with "Event".
The reason why there are so many event files is that our application supports 100's of manufactures in managing their devices. Each device is loaded with their manufacture's "smarts" in their firmware. An example would be a Cisco 3750 switch or an IBM BladeCenter HS22 or the 100's of others that we are managing. As you can imagine, each device sends its own type of device messages (sometimes they are in "traps" and other times they are as a result of a "polled" attribute from the management system). For every type of device message and for every type of device manufacturer I have an Event file.
Here is an example of text that is found in the event file named Event01169b15 (note: this is the raw file this is used for incoming traps for this device and alert type)
Code:
{d "%w- %d %m-, %Y - %T"} - An "iddmodAgentDownTrap" event has occurred, from a device designated as type: {t}, named: {m}.
This trap is sent when an Insight agent is detected to be down. It is then resent synchronously with the Alarm traps, every 5 minutes (default frequency) as long as the Insight agent is still detected to be down.
Agent Name = {S 1}
Agent Type = {S 3}
Agent Status = {T iddmodAlarmSummaryUpDown 4}
NOTE:
{S 1} means the String from Varbind #1 in the trap,
{S 3} means the String from the Varbind #3 in the trap,
{T iddmodAlarmSummaryUpDown 4} means it is looking up a Table named "iddmodAlarmSummaryUpDown" for its information.
When the device, in this example, an IBM Director Host, sends a trap that matches this event message the information we get within our application looks like this:
Code:
Sep 28, 2012 7:43:57 AM MDT
- An "iddmodAgentDownTrap" event has occurred, from a device designated as type Host_IBMDirector, named lpvd-sqldv01.c.utah.com.
This trap is sent when an Insight agent is detected to be down. It is then resent synchronously with the Alarm traps, every 5 minutes (default frequency) as long as the Insight agent is still detected to be down.
Agent Name = ms2008-default@lpvd-sqldv01
Agent Type = ms2005
Agent Status = down
Here's another example of another event. This event is named Event00011223 and the raw file looks like this:
Code:
{d "%w- %d %m-, %Y - %T"} - A "LiveHealthAlarm" event has occurred, from {t} device, named {m}.
eHealth Alarm Start Time - {D 2}
eHealth Alarm Occurrence - {I 10}
Element Name - {S 11}
Element Alias - {S 14}
Profile - {S 7}
Exception Type - {S 4}
Description - {S 76620}
eHealth Alarm ID - {I 6}
NOTE:
{D 2} means a DateStamp in VarBind #2 from the trap.
{I 10} means in Integer in VarBind #10 from the trap.
{S 11} means a String in VarBind #11 from the trap.
etc.
And here is what an actual alert looks like from a device that matches the criteria set up to use this event message:
Code:
Sep 27,2012 09:03:08 PM - A "LiveHealthAlarm" event has occurred, from SwCiscoIOS device, named co-lpvi-6509-Q67-1.c.utah.com.
eHealth Alarm Start Time - Sep 27,2012 09:03:08 PM
eHealth Alarm Occurrence - 305954
Element Name - co-lpvi-6509-Q67-1.c.utah.com-RH
Element Alias - co-lpvi-6509-Q67-1.c.utah.com-RH
Profile - _Router or Switch - Failure
Exception Type - basicEvent
Description - Generic Syslog message
eHealth Alarm ID - 1000042
As you can see from these two examples, the event file does not have a set standard for its format because we are dealing with different manufacturers that do things differently.
I don't need to have a script that would take into account every single permuatation of the 40k+ event files.... rather I am interested in understanding the logic to create a script for each event type that I am insterested in parsing out additional information. So I would have a script for Event01169b15 and a separate script for Event00011223. Although the Events would be different, as far as the text goes, the logic I would think would be very similar.
In the first example, (assuming this is an event that I am going to use in another application that uses the SetScript) the portion of my SetScript that deals with the actual EventMessage, it might look something like this:
Code:
...
blah blah blah....
AN=(echo "$EVENTMSG" | sed -nE 's/.*Agent\sName\s=\s(\S+)/\1/p')
AT=(echo "$EVENTMSG" | sed -nE 's/.*Agent\sType\s=\s(\S+)/\1/p')
echo "The IBM Agent that needs to be rebuilt is: " $AN
echo "The Agent Type is: " $AT
...
blah blah blah....
NOTE: I got the sed command from another post. I'm wondering if it would work?
For the other example, another SetScript, renamed to LiveHealthSetScript, would look something like this:
Code:
...
blah blah blah....
EN=(echo "$EVENTMSG" | sed -nE 's/.*Element\sName\s-\s(\S+)/\1/p')
PF=(echo "$EVENTMSG" | sed -nE 's/.*Profile\s-\s(\S+)/\1/p')
AID=(echo "$EVENTMSG" | sed -nE 's/.*Alarm\sID\s-\s(\S+)/\1/p')
echo "The eH Element Name processed is: " $EN
echo "The eH Profile used is: " $PF
echo "The eH Alarm ID was: " $AID
...
blah blah blah....
So, you see, even though I am using different event files I am using them the same way, that is, I am parsing the $EVENTMSG for the things I need on a case by case basis.
Is this what you were looking for?
And, at the risk of sounding like a broken record, THANKS!!!!!
This is a lot to absorb and your last message clears up some of the issues, but I'm still confused about several issues. I think I understand some things now. I assume that:
The 40K+ files named "Event" followed by 8 lowercase hexadecimal digits are templates for 40K+ different types of event messages.
Someplace else there are a bunch (40K+ or less) of SetScript files.
You have a way to get an individual event message stored in a shell variable referenced by $EVENTMESSAGE.
SetScript files or Eventxxxxxxxx files enable some external process to format an external event reported by some piece of hardware or software into an event message using variables that are somehow made available to the environment of that external process.
You showed the following event message in your last posting:
Code:
Sep 28, 2012 7:43:57 AM MDT
- An "iddmodAgentDownTrap" event has occurred, from a device designated as type Host_IBMDirector, named lpvd-sqldv01.c.utah.com.
This trap is sent when an Insight agent is detected to be down. It is then resent synchronously with the Alarm traps, every 5 minutes (default frequency) as long as the Insight agent is still detected to be down.
Agent Name = ms2008-default@lpvd-sqldv01
Agent Type = ms2005
Agent Status = down
which is supposed to correspond to Eventxxxxxxxx file "Event01169b15" which says that events of this type are formatted as:
Code:
{d "%w- %d %m-, %Y - %T"} - An "iddmodAgentDownTrap" event has occurred, from a device designated as type: {t}, named: {m}.
This trap is sent when an Insight agent is detected to be down. It is then resent synchronously with the Alarm traps, every 5 minutes (default frequency) as long as the Insight agent is still detected to be down.
Agent Name = {S 1}
Agent Type = {S 3}
Agent Status = {T iddmodAlarmSummaryUpDown 4}
but the 1st line of Event01169b15 turned into three lines in the event message. I believe the two extra newlines were a cut and paste error and would not appear in an actual event message.
In message #6 in this thread you showed a SetScript that contained the line:
Code:
echo "Server Instance Error: " $SI
but showed the corresponding line in the event message to be:
Code:
Server Instance: 31415
I assume it was another cut and paste error that caused " Error" to be in the SetScript file but not in the event message.
Are the above assumptions correct?
When your script is called will it always be given a single event message stored in the variable $EVENTMESSAGE, or is it given a file from which it reads multiple event messages? Is there any way for your program to determine which Eventxxxxxxxx file is associated with an event message?
Is there any way to determine which SetScript file is associated with an Eventxxxxxxxx file? Is there any way to determine which SetScript file is associated with a given event message?
And, again, more concretely, and most importantly, assuming that one of your scripts is called with EVENTMESSAGE set to:
Code:
Sep 27,2012 09:03:08 PM - A "LiveHealthAlarm" event has occurred, from SwCiscoIOS device, named co-lpvi-6509-Q67-1.c.utah.com.
eHealth Alarm Start Time - Sep 27,2012 09:03:08 PM
eHealth Alarm Occurrence - 305954
Element Name - co-lpvi-6509-Q67-1.c.utah.com-RH
Element Alias - co-lpvi-6509-Q67-1.c.utah.com-RH
Profile - _Router or Switch - Failure
Exception Type - basicEvent
Description - Generic Syslog message
eHealth Alarm ID - 1000042
exactly what output do you want your script to produce? And, does your script know which Eventxxxxxxxx file and which SetScript file are associated with that event?
The solution I posted earlier should work for this data eg:
Code:
EVENTMSG="Sept 21, 2012 8:21:55 PM MDT - Device Main_Server_01 of type Host_Server is no longer responding to requests. This condition has persisted for more than 3 minutes.
Alert Code: Danger
Server Instance: 31415
Severity: Critical"
SI=$(echo "$EVENTMSG" | sed -nE 's/.+Server Instance:\s(\S+)/\1/p')
echo $SI
Output is:
Code:
31415
If you know extended regular expressions, and I suspect you do as you posted the above RE in your first post.
You should be able to tailor the sed command to fetch any value you are after.
When using the above code I am getting an error. First, here is the code I used taken from what you gave me earlier:
Hi All,
i had to split one files into 10 equally. For that i have coded below awk.
OUTPUT_FILE=/home/sit/path/Files/file_EXPORT.lst
DIR_NM=`dirname ${OUTPUT_FILE}`
awk -v CURR_DATE="$(date +'%d-%m-%Y-%H-%M')" -v pth=$DIR_NM '{print >> pth/"tgt_file_name"CURR_DATE"_"NR%10 }' ${OUTPUT_FILE}
... (7 Replies)
i have a log file that contains something similar to this:
one two three four five six seven eight
nine ten eleven twelve thirteen fourteen
one two three four five six seven eight
nine ten eleven twelve thirteen fourteen
one two three four five six seven eight
nine ten eleven twelve... (3 Replies)
I have the following XML
<Audit_Type>1</Audit_Type><Session_Id>34505863</Session_Id>
<StatementId>1</StatementId><EntryId>1</EntryId>
<Extended_Timestamp>2012-03-06T10:25:20.789459</Extended_Timestamp>
<DB_User>KASINIY</DB_User>
<OS_User>majohn1</OS_User><OS_Process>28636</OS_Process>... (3 Replies)
Hello,
somewhere in a shell script, i am storing the output of "ls" into a variable. My question is how can i parse this variable to get each filepath. I don't want to create a temporary file to write down all the filenames and then parse it..
is there a easy way out..
here is what... (3 Replies)
Can someone help me? I have been looking in the archives as I am sure this is very simple to do, but I do not know.
I have a variable which sometimes contains a file name and sometimes contains a fully qualified file name.
I want to be able to separate the directory from the file name into 2... (3 Replies)
Hi I am new to shell scripting. I need to parse a file which contains the header and detail records and split into n of file based on dept ID, for ex.
INPUT FILE:
DEPT ID: 1
EMPNAME: XYZ
EMPAddress: XYZZZ
DEPT ID: 2
EMPNAME: ABC
EMPAddress: ABCD
DEPT ID: 1
EMPNAME: PQR
EMPAddress:... (6 Replies)
Hi,
I want to get an input from user and parse the input. The legal characters allowed in the input are alnum(a-zA-Z0-0), . , -
Also the first and las characters must be alnum only.
e.g
if the input is abc.ghh-sok.com then the script should return correct, and if the input is like... (2 Replies)
I know solution to this but I was wondering if its easier than what i think
I have to pass 20 parameters to a script, which of course is not working so I parsed $3 to be a pipe deliminated string
for instance below
a.ksh One Two Compa|Compb|Compc|compd|.............
Now i have to read... (5 Replies)
I need to pick a field out of a variable record - the field is always found 4 fields after a certain text string, but it can be on any line of the record and in any position across the record on a line. I have had no luck through any of the Unix editors being able to cut a field that isn't always... (17 Replies)
Hi all, I have a problem surfacing and I hope you all could help.
What I have to do is take a input file and fill out a fax template from that file. The biggest problem I found was I have to parse the string "//FAX(faxnumber=555-5555;style="style1";
and on and on. The string can be in any... (5 Replies)
09-27-2012
sed Error using the syntax provided
Let's see if I get it right