Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

sed error: invalid reference

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting sed error: invalid reference
# 1  
Old 10-01-2012
Question sed error: invalid reference
Hello all,
I am using sed to parse a particular part of a string and am having problems. I am getting the following error:
sed: -e expression #1, char 28: invalid reference \1 on `s' command's RHS
Here is the code I am using:
Code:
echo "Alarm SET:"
echo ""
echo "Date:            " $DATE
echo "Time:            " $TIME
echo "ModelName:       " $MNAME
TST=$(echo "$EVENTMSG" | sed -ne 's/.*odelName=(\w+[^.]+)/\1/p')
echo "Non-FQDN Name:    " $TST
Severity:                  " $SEV

Here is the output:
Code:
Alarm SET:
Date:             10/01/2012
Time:             13:26:49
ModelName:        twa-casql01.c.com
sed: -e expression #1, char 28: invalid reference \1 on `s' command's RHS
Non-FQDN Name:
Severity:         MAJOR

As you can see, something is not working when I try to parse $EVENTMSG. My expected outcome is:
Code:
Alarm SET:
Date:             10/01/2012
Time:             13:26:49
ModelName:        twa-casql01.c.com
Non-FQDN Name: twa-casql01
Severity:         MAJOR

Ultimately, I am trying to verify that the $MNAME is the non-FQDN formated name as the ModelName contained in the $EVENTMSG.

The regex syntax of:
Code:
(\w+[^.]+)

takes the FQDN (or Fully Qualified Domain Name) and pulls the dot (.) suffix off of it.

Any ideas?

MANY THANKS IN ADVANCE!!
Last edited by dlundwall; 10-01-2012 at 07:28 PM.. Reason: Unusual display??
# 2  
Old 10-01-2012
Code:
echo "Alarm SET:"
echo ""
echo "Date:            " $DATE
echo "Time:            " $TIME
echo "ModelName:       " $MNAME
echo "Non-FQDN Name:   " ${EVENTMSG%%[.]*}
echo "Severity:        " $SEV

This User Gave Thanks to rdrtx1 For This Post:
dlundwall
# 3  
Old 10-03-2012
Quote:
Originally Posted by dlundwall
Hello all,
I am using sed to parse a particular part of a string and am having problems. I am getting the following error:
sed: -e expression #1, char 28: invalid reference \1 on `s' command's RHS
Here is the code I am using:
Code:
echo "Alarm SET:"
echo ""
echo "Date:            " $DATE
echo "Time:            " $TIME
echo "ModelName:       " $MNAME
TST=$(echo "$EVENTMSG" | sed -ne 's/.*odelName=(\w+[^.]+)/\1/p')
echo "Non-FQDN Name:    " $TST
Severity:                  " $SEV

Here is the output:
Code:
Alarm SET:
Date:             10/01/2012
Time:             13:26:49
ModelName:        twa-casql01.c.com
sed: -e expression #1, char 28: invalid reference \1 on `s' command's RHS
Non-FQDN Name:
Severity:         MAJOR

As you can see, something is not working when I try to parse $EVENTMSG. My expected outcome is:
Code:
Alarm SET:
Date:             10/01/2012
Time:             13:26:49
ModelName:        twa-casql01.c.com
Non-FQDN Name: twa-casql01
Severity:         MAJOR

Ultimately, I am trying to verify that the $MNAME is the non-FQDN formated name as the ModelName contained in the $EVENTMSG.

The regex syntax of:
Code:
(\w+[^.]+)

takes the FQDN (or Fully Qualified Domain Name) and pulls the dot (.) suffix off of it.

Any ideas?

MANY THANKS IN ADVANCE!!
As usual in your postings, you leave out a lot of details that are needed to help solve your problem:
  1. What is in the shell variable EVENTMSG? For the purposes of this thread, I will assume that it contains the lines:
    Code:
     ModelName=        twa-casql01.c.com

    and
    Code:
     AlarmSeverity= Critical

    and several other lines before and/or after these lines.
    On these lines, I have no idea whether the whitespace between ...= and the following data is one or more <tab>s, one or more <space>s, or a combination of <space>s and <tab>s. The search expressions used below allow any combination of one or more <space> and/or <tab> characters.
  2. You don't say what system you're using. It is obvious why you're getting the error you're getting on your sed command (you have a request to use backreference #1 in your replacement string, but you didn't define a subexpression in your BRE), but it isn't obvious whether or not the \w would be treated as a match for whitespace (as in perl) or treated as an escaped w as specified by the standards for BREs in the sed utility. And, BREs in the standard don't treat + as special in a BRE. (It means match one or more of the previous expression in an ERE, but it just matches a <plus-sign> in a BRE.
  3. You search for odelName=, but you seem to have ModelName: in what you want output. I will assume for now that $EVENTMSG contains <equals-sign>s, but you want to translate them to <colon>s in your output.
The suggested text here should work on UNIX-, Linux-, and BSD-systems as long as you're using a POSIX compatible shell (such as bash or ksh). (I use ksh so it is what is specified here, but I'm not using any ksh specific features.)

This example is an amalgam of your original posting and the updated text in your original posting:
Code:
#!/bin/ksh
EVENTMSG='Wed 26 Sep, 2012
 ModelType= Event message testing
 ModelName=        twa-casql01.c.com
 AlarmSeverity= Critical
 ActionEvent= Evacuate Immediately
 Payload= large raw data string'

SEV=$(printf "%s\n" "$EVENTMSG" | sed -ne "s/.*AlarmSeverity=[  ]*\(.*\)/\1/p")
TST=$(printf "%s\n" "$EVENTMSG" | sed -ne 's/.*ModelName=[      ]*\([^.]\{1,\}\).*/\1/p')

printf " AlarmSeverity:\t%s\n Non-FQDN Name:\t%s\n" "$SEV" "$TST"

Note that the first bracet expression in both sed statements is the four character sequence: <left-square-bracket>, <space>, <tab>, and <right-square-bracket>.

This script produces the following output on OS X (and should be the same on any system with a sed conforming to the standards:
Code:
 AlarmSeverity:	Critical
 Non-FQDN Name:	twa-casql01

I hope this helps,
Don
This User Gave Thanks to Don Cragun For This Post:
dlundwall
# 4  
Old 10-03-2012
PHP MY Actual script output is different???
First and foremost, thank you for all the time you are putting into helping me! I am not the best scripter (I know, it's quite obvious)!!

Here is the code I used. Note, I am including everything in the shell script that has to do with my output:
Code:
echo_info()
{
echo " "
echo "============>>>    NEW Alarm Notification from SPECTRUM    <<<============"
echo "=========================================================================="
echo "      A ${SEV} ALARM JUST OCCURRED ON ${MNAME}"
echo "=========================================================================="
echo ""
echo "===>>>  ALERT TYPE: ${ONELINE}"
echo ""
echo "Date:                 " $DATE
echo "Time:                 " $TIME
echo "Alarm Severity:  " $SEV
echo "Alarm Acknowledged? " ${ACK}
echo " "
echo "###########  DEVICE/MODEL INFORMATION  ###########"
echo "${MNAME} is modeled on ${SERVER} "
echo "IPAddress:       " $IPADDRESS
echo "ModelName:       " $MNAME
echo ""
echo "Global AlarmID:  " $GLOBAL_ALARM_ID
#******************************
#        Test Section
#******************************
echo ""
DJL=$(printf "%s\n" "$EVENTMSG" | sed -ne "s/.*Device\s[  ]*\(.*\)/\1/p")
ABC=$(printf "%s\n" "$EVENTMSG" | sed -ne 's/.*of\stype\s[      ]*\([^.]\{1,\}\).*/\1/p')
printf " MODEL NAME:\t%s\n MODEL TYPE:\t%s\n" "$DJL" "$ABC"
 
###########################################################################
#  The following parameters contain values only when
#  the SPECTRUM Alarm Notification Manager is enabled.
###########################################################################
if [ "$SANM" ]
then
echo ""
echo "SANM AlarmAge Setting: ${AGE} minute(s)"
echo "NotificationData:" $NOTIFDATA
echo ""                                         # insert blank line
fi
# This variable has substituted placeholder chars, echo them inside quotes
echo "EventMessage:     $EVENTMSG"
echo ""
echo ""
}

When an alarm came in that followed the alarm filtering criteria the output is as follows:
Code:
============>>>    NEW Alarm Notification from SPECTRUM    <<<============
==========================================================================
      A MAJOR ALARM JUST OCCURRED ON lp-webtactst01
==========================================================================
 
===>>>  ALERT TYPE: MANAGEMENT AGENT LOST
 
Date:                  10/03/2012
Time:                  08:58:02
Alarm Severity:   MAJOR
Alarm Acknowledged?  No
 
###########  DEVICE/MODEL INFORMATION  ###########
lp-webtactst01 is modeled on uta-specpol01 
IPAddress:        10.10.15.250
ModelName:        lp-webtactst01
ModelHandle:      0x206dd0
DeviceType:       Net-SNMP Linux
Mtype:            Host_Device
ModelTypeHandle:  0x1160089
 
Global AlarmID:   506c527a-bfe8-1006-02cd-0050569d7aa8
 
 MODEL NAME:    is no longer responding to primary management requests (e.g. SNMP), but appears to be responsive to other communication protocol (e.g. ICMP).  This condition has persisted for an extended amount of time.  An alarm will be generated.   (event [0x00010daa])
 MODEL TYPE:     Host_Device is no longer responding to primary management requests (e
 
SANM AlarmAge Setting: 0 minute(s)
NotificationData:
 
EventMessage:     Wed 03 Oct, 2012 - 08:58:02 - Device lp-webtactst01.c.utah.com of type Host_Device is no longer responding to primary management requests (e.g. SNMP), but appears to be responsive to other communication protocol (e.g. ICMP).  This condition has persisted for an extended amount of time.  An alarm will be generated.   (event [0x00010daa])

As you can see, I am not getting my desired outcome. I was expecting the following output:
Code:
============>>>    NEW Alarm Notification from SPECTRUM    <<<============
==========================================================================
      A MAJOR ALARM JUST OCCURRED ON lp-webtactst01
==========================================================================
 
===>>>  ALERT TYPE: MANAGEMENT AGENT LOST
 
Date:                  10/03/2012
Time:                  08:58:02
Alarm Severity:   MAJOR
Alarm Acknowledged?  No
 
###########  DEVICE/MODEL INFORMATION  ###########
lp-webtactst01 is modeled on uta-specpol01 
IPAddress:        10.10.15.250
ModelName:        lp-webtactst01
ModelHandle:      0x206dd0
DeviceType:       Net-SNMP Linux
Mtype:            Host_Device
ModelTypeHandle:  0x1160089
 
Global AlarmID:   506c527a-bfe8-1006-02cd-0050569d7aa8
 
 MODEL NAME:    lp-webtactst01.c.utah.com
 MODEL TYPE:     Host_Device
 
SANM AlarmAge Setting: 0 minute(s)
NotificationData:
 
EventMessage:     Wed 03 Oct, 2012 - 08:58:02 - Device lp-webtactst01.c.utah.com of type Host_Device is no longer responding to primary management requests (e.g. SNMP), but appears to be responsive to other communication protocol (e.g. ICMP).  This condition has persisted for an extended amount of time.  An alarm will be generated.   (event [0x00010daa])

What am I doing wrong?SmilieSmilie
Last edited by Scrutinizer; 10-07-2012 at 04:09 AM.. Reason: cleaned up formatting (spurious font definitions)
# 5  
Old 10-07-2012
In message #2 in the thread titled Parsing a $VARIABLE within a script
in the forum titled Shell Programming and Scripting, you said:
Quote:
To answer your question, the $EVENTMSG could look like some 40,000 different messages taken directly from 40,000 Event files.

There is one part in every single file that is the same... the time/day/date stamp, followed by "ModelType={t}, ModelName={m}"... but I may not necessarily need to parse that out.

The {t} and {m} are variables from another program that will input the appropriate data, so when it arrives in the $EVENTMSG it has usable data.
but in the last message you show that $EVENTMSG is Wed 03 Oct, 2012 - 08:58:02 - Device lp-webtactst01.c.utah.com of type Host_Device is no longer responding to primary management requests (e.g. SNMP), but appears to be responsive to other communication protocol (e.g. ICMP). This condition has persisted for an extended amount of time. An alarm will be generated. (event [0x00010daa]) which, of course, contains neither "ModelType=" nor "ModelName=". So, I have to conclude that despite the fact that all of the threads you have started in this forum seem related, I can't rely on statements made in one thread being of any help in any other thread. Nonetheless, this and other comments about event message formats in other threads are the only specifications I have for the format of an event message.

So, back to the issues at hand. In the first message in this thread you said that the commands:
Code:
TST=$(echo "$EVENTMSG" | sed -ne 's/.*odelName=(\w+[^.]+)/\1/p')
echo "Non-FQDN Name:    " $TST

gave you the diagnostic message:
Code:
sed: -e expression #1, char 28: invalid reference \1 on `s' command's RHS

when you were expecting the output:
Code:
Non-FQDN Name: twa-casql01

As noted earlier, the diagnostic message came about because the replacement in your sed substitute command referenced subexpression 1 in the search pattern, but you didn't have any subexpressions in your search pattern. Now that you have actually shown us in message #4 in this thread what is in $EVENTMSG, we also now know that the string being searched doesn't contain the requested pattern (i.e., "odelName" doesn't appear anywhere in $EVENTMSG). To get the output you want from the value of $EVENTMSG you have supplied, the following commands will work:
Code:
TST=$(echo "$EVENTMSG" | sed -ne 's/.* Device \([^. ]*\).*/\1/p')
echo "Non-FQDN Name:     $TST"

or, without using sed:
Code:
TST=${EVENTMSG#* Device }
TST=${TST%%[. ]*}
echo "Non-FQDN Name:     $TST"

Note that the bracket expression identifying the end of the Device name is looking for <period> or <space> to end the matched name. This is because in some of your earlier posts you said that sometimes the name contained one or more <period>s (and, if so, you wanted them to be removed) and sometimes did not contain a <period>. Either of the above methods of setting $TST will terminate the string to be returned at the first <period> or <space>.

I hope this resolves this problem for you,
Don
This User Gave Thanks to Don Cragun For This Post:
dlundwall
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

sed - use back reference in 2nd command

I have data that looks like this: <Country code="US"><tag>adsf</tag><tag>bdfs</tag></Country><Country code="CA"><tag>asdf</tag><tag>bsdf</tag></Country> I want to grab the country code save it, then drop each new "<..." onto a new line with the country code added to the beginning of each So,... (9 Replies)
Discussion started by: JenniferAmon
9 Replies

2. UNIX for Dummies Questions & Answers

Extract text in sed using back reference

i have a text 20 21 22 23 24 25 26 i want to get 22 using sed back reference. I have used sed 's/{6}\(..\).*/\1/' but, it does not work. I am missing something somewhere. Please help. (5 Replies)
Discussion started by: gotamp
5 Replies

3. Shell Programming and Scripting

Pattern match and replace indirect directory reference using sed

Hi, I need a ksh script to replace indirect directory references in an .ini file with a env variable using sed or awk. The .ini file is for example as such: A=.. B=../ C=../.. D=../../ E=../bin F=../../bin G=../../bin/xml H=../../bin/xml/ Need to replace an instance of .. or... (2 Replies)
Discussion started by: andyatit
2 Replies

4. Shell Programming and Scripting

sed back reference error

I am trying to change a single line of a special file whose comment character is ! to show a path to the file in the comment. such as: !!HFSS and mcm path: \Signal_Integrity\Package_SI\Section_Models\C4toTrace\28nm\D6HS\SLC_5-2-5\GZ41_ICZ\NSSS\ to a different path and replace the !!HFSS... (1 Reply)
Discussion started by: mobrien601
1 Replies

5. Shell Programming and Scripting

Invalid null command error

Hi, I have this script which gives me output as Invalid null command set recent_file=`grep '^-.*xlsx$' $FTP_LOG |\ sed -e 's/Jan/1/g' \ -e 's/Feb/2/g' \ -e 's/Mar/3/g' \ -e... (6 Replies)
Discussion started by: juzz4fun
6 Replies

6. UNIX for Dummies Questions & Answers

Invalid back reference

The thread can be closed now :D. (3 Replies)
Discussion started by: vaz0r
3 Replies

7. Shell Programming and Scripting

Perl de-reference code reference variable

Guys, May i know how can we de reference the code reference variable.? my $a = sub{$a=shift;$b=shift;print "SUM:",($a+$b),"\n";}; print $a->(4,5); How can we print the whole function ? Please suggest me regarding this. Thanks for your time :) Cheers, Ranga :) (0 Replies)
Discussion started by: rangarasan
0 Replies

8. Shell Programming and Scripting

Shell Scripting Problem - Invalid Back Reference

Here is the question... Create a new script, sub2, taking three parameters... 1.) the string to be replaced 2.) the string with which to replace it 3.) the name of the file in which to make the substitution ...that treats the string to be replaced as plain text instead of as a regular... (1 Reply)
Discussion started by: johnhisenburg
1 Replies

9. Shell Programming and Scripting

01.30 Invalid shell error

Hi, I am getting the error 01.30 Invalid shell error I am running the bash shell script in the korn login shell. I have mentioned the #!/bin/bash statement in the my script but not sure why it is giving this error to me.. (4 Replies)
Discussion started by: mr_harish80
4 Replies

10. Shell Programming and Scripting

How to reference a variable within sed?

Hi all, How can I use sed to perform a substitution if the string that I'm going to substitute is stored in a variable: Let's say: sed 's/abcdefg/good' VS tmp="abcdefg" sed 's/$tmp/good' The second case doesn't work. Guess it's due to the single quotes on the outside. How can I... (1 Reply)
Discussion started by: rockysfr
1 Replies
Login or Register to Ask a Question