Sed with unescaped input


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Sed with unescaped input
# 1  
Old 04-05-2011
Sed with unescaped input

a shell script variable from user input, via read, will typically have special characters in it - i.e. sip:29384902834!someServer.com@someOtherServer.com

this value has to stay formatted like this because it being used by curl in a post to php. but it is also being used by sed to replace the previous default value with the last value provided by the user (so the script can be ran multiple times without having to re-enter everything each time)

Code:
whoAmI=${PWD}/$0
. . .
targetRoute=sip:9996101752!sbc-external.orl.example.net@sbc-internal.orl.exmaple.net
targetRoute_tmp=sip:9996101752!sbc-external.orl.example.net@sbc-internal.orl.example.net
echo "what is the outbound address?"
echo "def:${targetRoute}"
read targetRoute_in
if [ "$targetRoute_in" != "" ]; then
        targetRoute=$targetRoute_in
fi
. . .
curl "http://${driverURI}/CallXML.start?tokenid=${tokenID}&tagOUT_URI=${targetRoute}" &
. . . 
sed -e s/targetRoute=${targetRoute_tmp} /targetRoute=${targetRoute}/ \
    -e s/targetRoute_tmp=${targetRoute_tmp}/targetRoute_tmp=${targetRoute}/ \   
< ${whoAmI} > ${whoAmI}.tmp && mv ${whoAmI}.tmp ${whoAmI}
chmod +x ${whoAmI}

sed runs either after the script is done running or on trapping a number of signals.

it's a bit goofy, i know, but it works well so long as the targetRoute doesn't contain those special characters. ...

so any ideas on how to get sed to play nice here? keep in mind there are a large number of user inputs that sed is working with - this is just a sample.

edit:i've tried creating a seperate set of variables like:

targetRoute_esc=`echo ${targetRoute} | sed -e s/\!/\\\!/ -e s/\:/\\\:/ -e s/\@/\\\@/`
targetRoute_tmp_esc=`echo ${targetRoute_tmp} | sed -e s/\!/\\\!/ -e s/\:/\\\:/ -e s/\@/\\\@/`

then running sed with
-e 's/'"targetRoute=${targetRoute_tmp_esc}"'/'"targetRoute=${targetRoute_esc}"'/' \
-e 's'/"targetRoute_tmp=${targetRoute_tmp_esc}"'/'"targetRoute_tmp=${targetRoute_esc}"'/' \

and tried a large number of variations on s/\!/\\\!/ .... to s/\!/\\\\\\\!/ and everything in between, with no luck.

Last edited by Gripp; 04-05-2011 at 07:51 PM..
# 2  
Old 04-07-2011
OMG what bad way.

1) The mv command is destructive for the inode of he target file so your
Code:
mv ${whoAmI}.tmp ${whoAmI}

is not good (in my opinion).

2) If you want keep a trace of what the value of previous run between different run of your script, you may want to put it in a separated file instead of modifying the executing file.

3) If this multiple run are sequentials (one after eachother, not simultaneously) this would quite easy to implement

4) In the case if you want to run your script multiple time in parallele, you may first need clarify your real needs and design an actions' workflow to avoid overlap between different runs ...especially if you want make them be able to manage a common file that hold the target value, this could be very tedious to implement (lockfile mechanism, atomic access, enqueue...)

5) Another option would be - if this script is run interactively - to make a menu with some default values so that the user only changes what he needs, otherwise default values are taken so that it would require a minimum of typing for the user
# 3  
Old 04-16-2011
thanks for the response; though it didn't exactly answer my question.

but to reply to your concerns:
the script is for load testing. for any given test the parameters don't tend to change from the initial configuration, but you;re likely to run the test multiple times.
so, to spare myself/others from having to input all of the same parameters (there are 8 to 13 depending on the options chosen) every-single-time we ran a given test, i am having the test re-write itself to use the user's previous input as the default for each param. so the next time that test is ran the user doesn't need to enter anything at all. even more, there is a -run option which skips asking for input altogether and just runs the test with the defaults.

as a simplified example (considering there are more than 1 param for a test): for a given test you want it to run for 30 minutes, but another test you want to run 45. you make a copy of the script, call them, say, test30.sh and test45.sh ... run through the prompts in each and from then on out any time you want to run that test you simply input:
"./test30.sh -run" and everything will be ran exactly the same as the last time you ran it, without requiring the user to input anything.

or, maybe later on, you want to make a test that has all of the same params as a previous one, but with one or two changes. you simply copy that one file and run through the prompts simply accepting the defaults and only proving input for those 1 or 2 values you wish to change. and again, any future runs of that test will not require any input.... i plan on making it so you can pass -run and variables as params, and it will only replace the values you provided and keep the defualts for those you didn't...

however, i don't like overwriting the code either, but i can't find any better way to do this. i don;t want to create a separate file for each test, as making copies or moving tests wouldn't be so straight forward. and besides, i would still be overwriting that file every time the user changed something, just like now.... and, most importantly, it works.
but, if you know of some better way i'm definitely open to suggestions.

as for the original question: how to get sed to escape the user's input and have a variable defined based on name=escaped-value, i found a way. i dont have access to the script atm, but once i get back i'll post it.

Last edited by Gripp; 04-16-2011 at 10:26 AM..
# 4  
Old 04-16-2011
From a run to another, if some parameter need to be changed, in one way or another, the user must specified the new value for those parameters that are new or that have changed.

So i see 2 way of doing it :

1) setting a kind of menu that would display the parameter and ask for confirmation to run or provide some options to modify the value of these parameters

2) Set an config file storing the needed parameter and load it at the beginning of your script so that if you need to modify 1 parameter, you can just edit the config file and modify the parameters as needed.

I personnaly would go for the second solution : quick to implement, simple, and clean.
If you know that the parameter haven't changed, you can just rerun the script since the last set of parameter are in the config file will still be valid, otherwise, adjust your config file and run.

You can also do a mix : if you noticed that -let's say - 8 parameters almost never change but 3 other parameters vary very frequently, then you can
put those 8 fix args in configfile and just implement special args to call your script with some argument through which you will set the parameter when calling your script.

example :

Code:
$ cat tst.cfg
PARAM1=value1
PARAM2=value2
PARAM3=value3
PARAM4=value4
PARAM5=value5
PARAM6=value6
PARAM7=value7
PARAM8=value8

Code:
$ cat tst.ksh
#!/bin/ksh
. ./tst.cfg
PARAM9=${1:-"defaultvaluep9"}
PARAM10=${2:-"defaultvaluep10"}
 
echo " I run with the following parameter :
PARAM1=${PARAM1}
PARAM2=${PARAM2}
PARAM3=${PARAM3}
PARAM4=${PARAM4}
PARAM5=${PARAM5}
PARAM6=${PARAM6}
PARAM7=${PARAM7}
PARAM8=${PARAM8}
PARAM9=${PARAM9}
PARAM10=${PARAM10}"

Code:
$ ksh tst.ksh
 I run with the following parameter :
PARAM1=value1
PARAM2=value2
PARAM3=value3
PARAM4=value4
PARAM5=value5
PARAM6=value6
PARAM7=value7
PARAM8=value8
PARAM9=defaultvaluep9
PARAM10=defaultvaluep10

Code:
$ ksh tst.ksh newvalforp9 newvalforp10
I run with the following parameter :
PARAM1=value1
PARAM2=value2
PARAM3=value3
PARAM4=value4
PARAM5=value5
PARAM6=value6
PARAM7=value7
PARAM8=value8
PARAM9=newvalforp9
PARAM10=newvalforp10


Last edited by ctsgnb; 04-16-2011 at 11:53 AM..
# 5  
Old 04-17-2011
#2 is effectively what i'm doing - only i keep the default values at the bottom of the script, rather than a separate file. I push them out to a separate tmp file, via tail, when the script is first loaded, load that file back into to script to use those value as the default. then i push out the rest of the script to a second tmp file and append variables selected by the user, as they are selected. if the user gets through all of the prompts and either finishes or kills the test then the values they input during the last run are what are loaded as default. for the next run.

the goal of this script was to get away from editing scripts when we needed to run a test. we have thousands of test scripts, each written slightly different/by a different person and it is just painful having to figure out what each value actually does, when you simply to start the test... (which is frequently >10pm in our case, not a good time to be fighting with code when you simply need to hit 'go' ...)
even more, if you want to run a large load you'll likely need to run several versions of the same script - which generally involved copying and renaming several files, then hunting thru code of each and renaming the references to the other files. ...
so, i decided to effectively do all of this, accept those seperate files only exists while the test is runnning, and are named based on the host file's name, and the end user never has to look at....

separately, I want to clean up my code a little bit, and find that i'm repeating myself with the user prompts and selection of default 'if empty'

each one looks about like what's in the OP.
Code:
read targetRoute_in
if [ "$targetRoute_in" != "" ]; then
        targetRoute=$targetRoute_in

fi

only the variable name is different.
so i've tried a number of variations of
Code:
function getInput(){
   read "${1}_in"
   if [ "${1}_in" != "" ]; then
      ${1}="${1}_in"
   fi
}

so that i can just do each prompt like:

echo "what language is the driver application?"
getInput driverLang

but can't seem to get it working; it tends to think '$varName' is the string to be compared. yes, i've tried -z, etc. and it still doesn't evaluate the value of the variable. even in the definition it tried to do driverLang='\$driverLang_in' rather than the value of the input....

any help would be appreciated.

Last edited by Gripp; 04-17-2011 at 10:28 AM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

sed with standard input not working

I am trying use sed to replace a string in a file with input string passed, but it is not replacing the string. instead it replace as $1. Please find below the code. echo $1 sed -i.$now "s/http.*.myservice.*.war/$1/" tempfile.xml I am running above code as below myscript.sh ReplaceString... (4 Replies)
Discussion started by: sakthi.99it
4 Replies

2. Shell Programming and Scripting

sed execution with input from keyboard

> sed '' Hello hi Hello output How hi output ^D How > sed should take each line as input, process and output the result. In the above scenario the input is passed from keyboard and the output of 'Hello' as you can see is displayed on the screen after 'hi' is passed as input but not as... (1 Reply)
Discussion started by: loggedin.ksh
1 Replies

3. Shell Programming and Scripting

Script to fill in sed input

Hello I have a file with list of numbers 380634062 380907172 380733031 380733032 380896961 380896962 381067621 381067622 380995932 I would like to find a way to fill in a sed command delete pattren sed -i '/$number_from_list/d' -e '/$second_from_list/d' -e ... etc so I... (9 Replies)
Discussion started by: amlife
9 Replies

4. Shell Programming and Scripting

Reading from input with sed

I am trying to edit a file in shell script using sed. I need to get the input from command line suppose script.sh sed"/s place=/place=california/g" > /root/user/mark.txt echo " place changed " the above code searches for string place in the file mark.txt and replaces with place=... (5 Replies)
Discussion started by: sriki32
5 Replies

5. Shell Programming and Scripting

SED command using multiple input files

What is the syntax to use multiple input files in a SED command. i.e. substitute a word with a phrase in every file in a directory. for every file in /usr/include that has the word "date" in the file grep -l '\<date\>' /usr/include/*.h find each occurrence of the word "time" in the file &... (3 Replies)
Discussion started by: sheoguey
3 Replies

6. Shell Programming and Scripting

sed error: unescaped newline

Hi, i need to make a script bash that takes in input three parameters: a text file that contains an old license, a file that contains a new license, and a directory that contains source files and i have to substitute to these file the old license with the new one. So, i think tu use sed, in... (2 Replies)
Discussion started by: nolanofra
2 Replies

7. Shell Programming and Scripting

Errore Sed: unescaped newline

Salve, mi occorre realizzare uno script che prenda in ingresso tre parametri: un file di testo contenente una vecchia licenza, un file contenente una nuova licenza e una directory, contenente dei sorgenti c a cui sostituire la vecchia licenza con la nuova. Pensavo di utilizzare sed, con... (1 Reply)
Discussion started by: nolanofra
1 Replies

8. UNIX for Dummies Questions & Answers

Using grep output as input for sed command

Hi, I would like to know if this is possible, and if so what can i do to make this work. I would like to grep a line X from fileA and then use the output to replace a word Y in fileB. grep "line X" fileA | sed -e 's/Y/X/g' > outfile this statement does not work, as i do not know how to... (7 Replies)
Discussion started by: cavanac2
7 Replies

9. Shell Programming and Scripting

SED + Regex + SQL Input file

Here's the problem... I have a mysqldump file and I need to put single quotes around the date/time timestamp. So for example I have a line like: INSERT INTO attachments VALUES (1,182,2004-08-06 09:24:04,'description'... and I need it to become INSERT INTO attachments VALUES... (10 Replies)
Discussion started by: primal
10 Replies

10. Shell Programming and Scripting

sed not outputting last line of input file

I am using sed for a simple substitution (see command syntax below). Everything works fine except that the last line of the input file does not get written to the output file. Has anyone ever seen this and know of way to force the last line to be written? I don't know if it's playing a part in... (3 Replies)
Discussion started by: 2reperry
3 Replies
Login or Register to Ask a Question