Sponsored Content
Top Forums Shell Programming and Scripting Trouble with sed and substituting a string with special characters in variable Post 302649575 by bakunin on Thursday 31st of May 2012 05:09:56 PM
Old 05-31-2012
You have told us everything - save for the contents of "$marker" and "$markerrepl", which would be the only thing necessary to know. How am i - how are we - supposed to give you advice regarding a string when we don't know the string?

I hope you aren't too dissatisfied with an answer which will be of only general nature:

Quote:
Originally Posted by ampsys
Code:
sed -i "s/$marker/$markerrepl/g" $formattedfile

When substituting from variable contents there are a few problems. The first is that this is prone to "injections" - a part of the search- or replacement-string will constitue a sed command itself. The output will probably not be what you had in mind. Example:

Code:
a="abc"
b="xyz"

sed "s/$a/$b/" /some/file       # will work as expected

a="abc/xyz"
sed "s/$a/$b/" /some/file       # will not work

A solution for this will be to escape all the characters special to sed prior to useing these strings. Escaping in sed is done with backslashes:

Code:
a="abc\/xyz"
b="xyz"

sed "s/$a/$b/" /some/file       # will work as expected


A second problem is shell-related: without quoting variables can break up lines into pieces. This can be partly overcome with proper quoting:

Code:
a="a b c"
b="x y"

sed s/$a/$b/ /some/file            # will not work
sed 's/'"$a"'/'"$b"'/g' /some/file # will work

This will not help you if variables contain line feeds.

I hope this helps.

bakunin

Last edited by bakunin; 06-01-2012 at 05:49 AM..
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Substituting with value of variable in Sed

Hi, I have a program in which i have to substitute a TAG in a file with the value of a variable. Code Snippet: ---------------- myvar=1234 sed 's/_TAG_/$myvar/' infile outfile When I run this command, the _TAG_ in the "infile" is substituted with "$myvar" but NOT the value "1234"... (1 Reply)
Discussion started by: jyotipg
1 Replies

2. Shell Programming and Scripting

Add string after another string with special characters

Hello everyone, I'm writing a script to add a string to an XML file, right after a specified string that only occurs once in the file. For testing purposes I created a file 'testfile' that looks like this: 1 2 3 4 5 6 6 7 8 9 And this is the script as far as I've managed: ... (2 Replies)
Discussion started by: heliode
2 Replies

3. Shell Programming and Scripting

Variable Manimpulation - special characters

Hello- I have a variables that contains a string like this usr/pass@SCHEMA I want to extract the usr/pass part and ignore the SCHEMA part, I tried to use this ${dbconn%%@} and apparently it will not work because @ is a special character. I tried \@ and still no go. Any idea how to solve... (1 Reply)
Discussion started by: Nomaad
1 Replies

4. Shell Programming and Scripting

Substituting Characters using SED

Can SED be used to substitute a character (y) with a character (Y) in a specified field? File has 12000 : delimeted rows as; HHC 1 BDE:Lastname, Firstname MI:firstname.mi.lastname@mil:SGT HHC 2 BDE:Lastname, Firstname MI:Firstname.MI.Lastname@mil:SGT I wish to replace the capital letters... (6 Replies)
Discussion started by: altamaha
6 Replies

5. Shell Programming and Scripting

Special characters in a bash variable in sed

Hello, I am trying the following: echo __CHANGEME__ >> testfile VAR1="&&&" sed -i "s|__CHANGEME__|${VAR1}|" testfile cat testfile This results in testfile containing __CHANGEME____CHANGEME____CHANGEME__ Whereas I want it to result in &&& I understand that if VAR1="\&\&\&" then... (3 Replies)
Discussion started by: linuxnewbeee
3 Replies

6. Shell Programming and Scripting

Trouble cutting characters into a string.

I just have a couple of quick questions. I am having trouble with this cut. I am basically trying to cut the string so that i can insert the users guess at the appropriate point in the string. $letters is the character count of the $word. What it seems to do is cut the character into the... (0 Replies)
Discussion started by: Makaer
0 Replies

7. Programming

C++ Special Characters in a String?

Hello. How can i put all of the special characters on my keyboard into a string in c++ ? I tried this but it doesn't work. string characters("~`!@#$%^&*()_-+=|\}]{ How can i accomplish this? Thanks in advance. (1 Reply)
Discussion started by: cbreiny
1 Replies

8. Shell Programming and Scripting

Sed failing for variable with special characters

This has been covered many times earlier but couldnt figure the issue myself. Can you please advise whats wrong on the below code I have a variable with special character ($) and am using that variable to replace another variable in file but however sed is failing to parse it correctly ... (7 Replies)
Discussion started by: sasiharitha
7 Replies

9. Shell Programming and Scripting

Substituting a shell variable in sed

Hi guys, I'm trying to figure out how to use a shell variable inside my sed command. I just want to remove a certain part of a path. I've tried three different combinations and none of them work. Here are the three combinations: echo $file | sed 's/'$test'//' echo $file | sed "s/$test//"... (7 Replies)
Discussion started by: chu816
7 Replies

10. Shell Programming and Scripting

getting error while substituting variable using sed..

I am using script for substitute one variable with another variable like below... below code works fine... sed 's/'$sub_fun'/'$To_sub'/g' But when i run the same code from the script getting below errors.. sed: -e expression #1, char 7: unterminated `s' command please help....... (2 Replies)
Discussion started by: pamu
2 Replies
subst(1T)						       Tcl Built-In Commands							 subst(1T)

__________________________________________________________________________________________________________________________________________________

NAME
subst - Perform backslash, command, and variable substitutions SYNOPSIS
subst ?-nobackslashes? ?-nocommands? ?-novariables? string _________________________________________________________________ DESCRIPTION
This command performs variable substitutions, command substitutions, and backslash substitutions on its string argument and returns the fully-substituted result. The substitutions are performed in exactly the same way as for Tcl commands. As a result, the string argument is actually substituted twice, once by the Tcl parser in the usual fashion for Tcl commands, and again by the subst command. If any of the -nobackslashes, -nocommands, or -novariables are specified, then the corresponding substitutions are not performed. For example, if -nocommands is specified, command substitution is not performed: open and close brackets are treated as ordinary characters with no special interpretation. Note that the substitution of one kind can include substitution of other kinds. For example, even when the -novariables option is speci- | fied, command substitution is performed without restriction. This means that any variable substitution necessary to complete the command | substitution will still take place. Likewise, any command substitution necessary to complete a variable substitution will take place, even | when -nocommands is specified. See the EXAMPLES below. | If an error occurs during substitution, then subst will return that error. If a break exception occurs during command or variable substi- | tution, the result of the whole substitution will be the string (as substituted) up to the start of the substitution that raised the excep- | tion. If a continue exception occurs during the evaluation of a command or variable substitution, an empty string will be substituted for | that entire command or variable substitution (as long as it is well-formed Tcl.) If a return exception occurs, or any other return code is | returned during command or variable substitution, then the returned value is substituted for that substitution. See the EXAMPLES below. | In this way, all exceptional return codes are ``caught'' by subst. The subst command itself will either return an error, or will complete | successfully. EXAMPLES
When it performs its substitutions, subst does not give any special treatment to double quotes or curly braces (except within command sub- stitutions) so the script set a 44 subst {xyz {$a}} returns ``xyz {44}'', not ``xyz {$a}'' and the script | set a "p} q {r" | subst {xyz {$a}} | return ``xyz {p} q {r}'', not ``xyz {p} q {r}''. | When command substitution is performed, it includes any variable substitution necessary to evaluate the script. | set a 44 | subst -novariables {$a [format $a]} | returns ``$a 44'', not ``$a $a''. Similarly, when variable substitution is performed, it includes any command substitution necessary to | retrieve the value of the variable. | proc b {} {return c} | array set a {c c [b] tricky} | subst -nocommands {[b] $a([b])} | returns ``[b] c'', not ``[b] tricky''. | The continue and break exceptions allow command substitutions to prevent substitution of the rest of the command substitution and the rest | of string respectively, giving script authors more options when processing text using subst. For example, the script | subst {abc,[break],def} | returns ``abc,'', not ``abc,,def'' and the script | subst {abc,[continue;expr 1+2],def} | returns ``abc,,def'', not ``abc,3,def''. | Other exceptional return codes substitute the returned value | subst {abc,[return foo;expr 1+2],def} | returns ``abc,foo,def'', not ``abc,3,def'' and | subst {abc,[return -code 10 foo;expr 1+2],def} | also returns ``abc,foo,def'', not ``abc,3,def''. SEE ALSO
Tcl(1T), eval(1T), break(1T), continue(1T) KEYWORDS
backslash substitution, command substitution, variable substitution ATTRIBUTES
See attributes(5) for descriptions of the following attributes: +--------------------+-----------------+ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | +--------------------+-----------------+ |Availability | SUNWTcl | +--------------------+-----------------+ |Interface Stability | Uncommitted | +--------------------+-----------------+ NOTES
Source for Tcl is available on http://opensolaris.org. Tcl 7.4 subst(1T)
All times are GMT -4. The time now is 10:28 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy