Sponsored Content
Top Forums Shell Programming and Scripting Trouble with sed and substituting a string with special characters in variable Post 302649575 by bakunin on Thursday 31st of May 2012 05:09:56 PM
Old 05-31-2012
You have told us everything - save for the contents of "$marker" and "$markerrepl", which would be the only thing necessary to know. How am i - how are we - supposed to give you advice regarding a string when we don't know the string?

I hope you aren't too dissatisfied with an answer which will be of only general nature:

Quote:
Originally Posted by ampsys
Code:
sed -i "s/$marker/$markerrepl/g" $formattedfile

When substituting from variable contents there are a few problems. The first is that this is prone to "injections" - a part of the search- or replacement-string will constitue a sed command itself. The output will probably not be what you had in mind. Example:

Code:
a="abc"
b="xyz"

sed "s/$a/$b/" /some/file       # will work as expected

a="abc/xyz"
sed "s/$a/$b/" /some/file       # will not work

A solution for this will be to escape all the characters special to sed prior to useing these strings. Escaping in sed is done with backslashes:

Code:
a="abc\/xyz"
b="xyz"

sed "s/$a/$b/" /some/file       # will work as expected


A second problem is shell-related: without quoting variables can break up lines into pieces. This can be partly overcome with proper quoting:

Code:
a="a b c"
b="x y"

sed s/$a/$b/ /some/file            # will not work
sed 's/'"$a"'/'"$b"'/g' /some/file # will work

This will not help you if variables contain line feeds.

I hope this helps.

bakunin

Last edited by bakunin; 06-01-2012 at 05:49 AM..
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Substituting with value of variable in Sed

Hi, I have a program in which i have to substitute a TAG in a file with the value of a variable. Code Snippet: ---------------- myvar=1234 sed 's/_TAG_/$myvar/' infile outfile When I run this command, the _TAG_ in the "infile" is substituted with "$myvar" but NOT the value "1234"... (1 Reply)
Discussion started by: jyotipg
1 Replies

2. Shell Programming and Scripting

Add string after another string with special characters

Hello everyone, I'm writing a script to add a string to an XML file, right after a specified string that only occurs once in the file. For testing purposes I created a file 'testfile' that looks like this: 1 2 3 4 5 6 6 7 8 9 And this is the script as far as I've managed: ... (2 Replies)
Discussion started by: heliode
2 Replies

3. Shell Programming and Scripting

Variable Manimpulation - special characters

Hello- I have a variables that contains a string like this usr/pass@SCHEMA I want to extract the usr/pass part and ignore the SCHEMA part, I tried to use this ${dbconn%%@} and apparently it will not work because @ is a special character. I tried \@ and still no go. Any idea how to solve... (1 Reply)
Discussion started by: Nomaad
1 Replies

4. Shell Programming and Scripting

Substituting Characters using SED

Can SED be used to substitute a character (y) with a character (Y) in a specified field? File has 12000 : delimeted rows as; HHC 1 BDE:Lastname, Firstname MI:firstname.mi.lastname@mil:SGT HHC 2 BDE:Lastname, Firstname MI:Firstname.MI.Lastname@mil:SGT I wish to replace the capital letters... (6 Replies)
Discussion started by: altamaha
6 Replies

5. Shell Programming and Scripting

Special characters in a bash variable in sed

Hello, I am trying the following: echo __CHANGEME__ >> testfile VAR1="&&&" sed -i "s|__CHANGEME__|${VAR1}|" testfile cat testfile This results in testfile containing __CHANGEME____CHANGEME____CHANGEME__ Whereas I want it to result in &&& I understand that if VAR1="\&\&\&" then... (3 Replies)
Discussion started by: linuxnewbeee
3 Replies

6. Shell Programming and Scripting

Trouble cutting characters into a string.

I just have a couple of quick questions. I am having trouble with this cut. I am basically trying to cut the string so that i can insert the users guess at the appropriate point in the string. $letters is the character count of the $word. What it seems to do is cut the character into the... (0 Replies)
Discussion started by: Makaer
0 Replies

7. Programming

C++ Special Characters in a String?

Hello. How can i put all of the special characters on my keyboard into a string in c++ ? I tried this but it doesn't work. string characters("~`!@#$%^&*()_-+=|\}]{ How can i accomplish this? Thanks in advance. (1 Reply)
Discussion started by: cbreiny
1 Replies

8. Shell Programming and Scripting

Sed failing for variable with special characters

This has been covered many times earlier but couldnt figure the issue myself. Can you please advise whats wrong on the below code I have a variable with special character ($) and am using that variable to replace another variable in file but however sed is failing to parse it correctly ... (7 Replies)
Discussion started by: sasiharitha
7 Replies

9. Shell Programming and Scripting

Substituting a shell variable in sed

Hi guys, I'm trying to figure out how to use a shell variable inside my sed command. I just want to remove a certain part of a path. I've tried three different combinations and none of them work. Here are the three combinations: echo $file | sed 's/'$test'//' echo $file | sed "s/$test//"... (7 Replies)
Discussion started by: chu816
7 Replies

10. Shell Programming and Scripting

getting error while substituting variable using sed..

I am using script for substitute one variable with another variable like below... below code works fine... sed 's/'$sub_fun'/'$To_sub'/g' But when i run the same code from the script getting below errors.. sed: -e expression #1, char 7: unterminated `s' command please help....... (2 Replies)
Discussion started by: pamu
2 Replies
TRS(1)								Linux User's Manual							    TRS(1)

NAME
trs - filter replacing strings SYNOPSIS
trs [-[r]e] 'REPLACE_THIS WITH_THAT [AND_THIS WITH_THAT]...' trs [-[r]f] FILE DESCRIPTION
Copy stdin to stdout replacing every occurence of given strings with other ones. This is similar to tr(1), but replaces strings, not only single chars. Rules (separated by whitespace) can be given directly after -e option, or can be read from FILE. Argument not preceded by -e or -f is guessed to be a script when it contains some whitespace, or a filename otherwise. Comments are allowed from # until the end of line. The character # in strings must be specified as #. Standard C-like escapes a  e f v \ nn are recognized. In addition, s means a space character and ! means an empty string. Sets of acceptable characters at a given position can be specified between [ and ]. ASCII ranges in sets can be shortly written as FIRST-LAST. When a set consists of only a single range, [ and ] can be omitted. When a part of the string to translate is enclosed in {...}, only that part is replaced. Any text outside {...} serves as an assertion: a string is translated only if it is preceded by the given text and followed by another one. { at the beginning or } at the end of the string can be omitted. Text outside {...} is treated as untranslated. Before the beginning of the file and after its end there are only 's. Thus, for example, {.} matches . on a line by itself, including the first line, and the last one even without the marker. A fragment of the form ?x=N, where x is a letter A-Za-z and N is a digit 0-9, contained in the target text sets the variable x to the value N when that rule succeeds. Similar fragment in the source text causes the given rule to be considered only if that variable has such value. Initially all variables have the value of 0. Several assignments or conditions can be present in one rule - they are ANDed together. OPTIONS -e Give the translation rules directly in the command line. -f Get them from the file specified. -r Reverse every rule. This affects only the next -e or -f option. Of course this doesn't have to give the reverse translation! Any rule containing any of {}[]{}- is taken in only one direction. You may force any rule to be taken in only one direction by enclosing the string to translate in {...}. --help display help and exit --version output version information and exit Multiple -e or -f options are allowed. All rules are loaded together then, and earlier ones have precedence. EXAMPLE
$ echo Leeloo |trs -e 'el n e i i aqq o} x o u' Linux DIFFERENCES FROM sed The main difference between trs and sed 's///g; ...' (excluding sed's regular expressions) is that sed takes every rule in the order speci- fied and applies it to the whole line of translated file, whereas trs examines every position and tries all rules in this place first. In sed every next rule is fed with the text produced by the previous one, whereas in trs every piece of text can be translated at most once (if more than one rule matches at a given position, the one mentioned earlier wins). That's why sed isn't well suited for translating between character sets. On the other hand, tr translates only single bytes, so it can't be used for Unicode conversions, or TeX / SGML ways for specifying extended characters. Another example: $ echo 642 |trs -e '4 7 72 66 64 4' 42 $ echo 642 |sed 's/4/7/g; s/72/66/g; s/64/4/g' 666 The string to replace can be empty; there must be something outside {} then. In this special case only one such create-from-nothing rule can success at a given position. For example, }x80-xFF @ precedes every character with high byte set with @. The rule of the form some{ thing doesn't work at the end of a file. SEE ALSO
tr(1), konwert(1) COPYRIGHT
trs is a filter replacing strings. It forms part of the konwert package. Copyright (c) 1998 Marcin 'Qrczak' Kowalczyk This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MER- CHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA AUTHOR
__("< Marcin Kowalczyk * qrczak@knm.org.pl http://qrczak.home.ml.org/ \__/ GCS/M d- s+:-- a21 C+++>+++$ UL++>++++$ P+++ L++>++++$ E->++ ^^ W++ N+++ o? K? w(---) O? M- V? PS-- PE++ Y? PGP->+ t QRCZAK 5? X- R tv-- b+>++ DI D- G+ e>++++ h! r--%>++ y- Konwert 12 Jul 1998 TRS(1)
All times are GMT -4. The time now is 03:49 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy