Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Url encoding a string using sed
Top Forums Shell Programming and Scripting Url encoding a string using sed Post 303013247 by Paul Walker on Sunday 18th of February 2018 05:16:22 PM
Old 02-18-2018
Url encoding a string using sed
Hi I was hoping some one would know if it is possible to url encode a string using sed?
My problem is I have extracted some key value pairs from a text file with sed, and will be inserting these pairs as source variables into a curl script to automatically download some xml from our server.
My problem is that one of the value pairs needs to be url encoded. The "j_password=0NRx5KpiquE32LrlXApzrA==" needs to be url encoded
Below are my key value pairs
Code:
JSESSIONID0=1omyjon28ge2p8dnuo0ngrjps
j_password=0NRx5KpiquE32LrlXApzrA==
iv=e10d17fd73b1f684986ecc3afbe68ff3
salt=90c6ea71b949661dcfb4b6ccaf51ecc6
_uc=MDSz8K74i642sedqmp88Jg==
_ut=1518796880482
JSESSIONID=1ozzynzzoqu8o1sgbqhuoek5hy
SSO_ID=5ohshkk9is9eb
workstationId=nfFqyQ0vDc2Bs5IQdhZNQg==

These pairs will be read into a curl script, Below is part of the line from my curl script that the key value pairs will be read in as variables;
Code:
 curl 'http://192.168.1.55/epace/do-login'  lastLoggedInCompany=public;~~ JSESSIONID=$JSESSIONID0' -H 'Origin: http://192.168.1.55' -H~~'Referer: http://192.168.1.55/epace/login.html' -H 'Connection: keep-alive' --data 'j_username=ticorder&j_password=$j_password&lastLoggedInCompany=public&iv=$iv&salt=$salt&hash=' --compressed

I had read online that curl can be used to url encode. Although I am not sure how I would use this function in curl to encode only part of the curl command that I need url encoded?
The script I am using to extract the value pairs is below
Code:
sed  -i ' -e ' -n '/POST/,$p' /WebLoads/MainGo/CaptureFile-0
sed -i '' -e $'s/\t/\ /g' /WebLoads/MainGo/CaptureFile-0
sed -i '' -e $'s/\ \ /\ /g' /WebLoads/MainGo/CaptureFile-0
sed -i '' -e $'s/\ GET.*//g' /WebLoads/MainGo/CaptureFile-0
sed -i '' -e $'s/.*_uc\=/_uc\=/g' /WebLoads/MainGo/CaptureFile-0
sed -i ' -e ' -n '/.*/,// G; s/\n/&&/; /^\([ -~]*\n\).*\n\1/d; s/\n//; h; P' /WebLoads/MainGo/CaptureFile-0
sed -i '' -e $'s/\t/\ /g' /WebLoads/MainGo/CaptureFile-0
sed -i '' -e s'/.*\=public\;\ JSESSIONID\=/JSESSIONID0\=/g' /WebLoads/MainGo/CaptureFile-0
sed -i '' -e s'/Form item\:\ /\'$'\n/g' /WebLoads/MainGo/CaptureFile-0
sed -i '' -e s'/\ POST.*//g' /WebLoads/MainGo/CaptureFile-0
sed -i '' -e s'/.*j_username.*//g' /WebLoads/MainGo/CaptureFile-0
sed -i '' -e s'/.*companySelection.*//g' /WebLoads/MainGo/CaptureFile-0
sed -i '' -e s'/.*Login.*//g' /WebLoads/MainGo/CaptureFile-0
sed -i '' -e s'/.*lastLoggedInCompany.*//g' /WebLoads/MainGo/CaptureFile-0
sed -i '' -e s'/.*hash.*//g' /WebLoads/MainGo/CaptureFile-0
sed -i '' -e s'/\;\ /\'$'\n/g' /WebLoads/MainGo/CaptureFile-0
sed -i '' -e s'/\"\ =\ \"/\=/g' /WebLoads/MainGo/CaptureFile-0
sed -i '' -e s'/\"\,//g' /WebLoads/MainGo/CaptureFile-0
sed -i '' -e s'/^\"//g' /WebLoads/MainGo/CaptureFile-0
sed -i ' -e ' -n '/.*/,// G; s/\n/&&/; /^\([ -~]*\n\).*\n\1/d; s/\n//; h; P' /WebLoads/MainGo/CaptureFile-0
sed -i '' -e '/^\s*$/d' /WebLoads/MainGo/CaptureFile-0

I've looked online and it does not seem very hopeful to be able to accomplish this with sed, but It would be my first choice since my understanding of sed is better than some of the other available programs. If there is another way of accomplishing this with another program I would be more than thankful if some one was able to show me how this could be done.
Thank you
Paul
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

parsing url string

Hi, I have a url like this:- http://resource.ibab.ac.in/cgi-bin/pubmed_abstract/y.cgi?pmid=+1.10529272+&pmid=+3.8379586 I want to parse the url string.I want to get the values 1.10529272 3.8379586 How do i do that? (5 Replies)
Discussion started by: vanitham
5 Replies

2. Shell Programming and Scripting

URL encoding

Hi All, I want to do URL encoding using shell script in my project. I decided that the sed is the correct tool to do this. But I am unable achieve what I wanted using sed. kindly help me to get rid of this. My requirement is , there will be one URL with all special character, spaces etc... ... (8 Replies)
Discussion started by: Vichu
8 Replies

3. UNIX for Advanced & Expert Users

.htaccess url encoding

Hi, I tried hard to find out solution but no success. I have put together this code in .htaccess: Rewritecond %{THE_REQUEST} ^{3,9}\ /index\.php\?(+)query=(.*?)&(+)start=(.*?)&(+)\ HTTP/ Rewriterule ^index\.php$ http://subdomain.domain.com/%2/%4? RewriteRule ^(+)/(*)$... (0 Replies)
Discussion started by: LukasB
0 Replies

4. Shell Programming and Scripting

python - string encoding error

I'm trying to pull a google calendar (successful) and then put the contents into a mysql db (almost successful). On one of the field I keep getting an encode error: #!/usr/bin/python from xml.etree import ElementTree import gdata.calendar.data import gdata.calendar.client import... (12 Replies)
Discussion started by: unclecameron
12 Replies

5. Shell Programming and Scripting

How to achieve UTF-8 encoding & URL escape in an xml file?

Is there any i can achieve entity escaping, URL escaping & UTF-8 encoded for the xml generated through shell script? #! /bin/bash echo "<path>" >> file.xml for x in `ls filename*` do echo -e "\t<dir>" >> file.xml echo -e "\t\t<file>$x</file>" >> file.xml... (0 Replies)
Discussion started by: vel4ever
0 Replies

6. Shell Programming and Scripting

How to check string encoding?

I want to check if the string is WINDOWS-1251 or UTF-8 can you help me to find the string encoding??? or maybe to get URL Content-Type charset with wget? this is my function on PHP function check_utf8($str) { $len = strlen($str); for($i = 0; $i < $len; $i++){ $c =... (2 Replies)
Discussion started by: sanantonio7777
2 Replies

7. Shell Programming and Scripting

URL/HTML encoding

Hey guys, looking for a way to encode a string into URL and HTML in a bash script that I'm making to encode strings in various different digests etc. Can't find anything on it anywhere else on the forums. Any help much appreciated, still very new to bash and programming etc. (4 Replies)
Discussion started by: 3therk1ll
4 Replies

8. Shell Programming and Scripting

Replace URL using sed

Original Line {background-image:url('http://www.myoldhost.com/images/scds/tsp3.png');} Expected {background-image:url('http://www.mynewhost.com/nndn/hddh/ccdcd.png');} I am using following syntax STATIC_HOST_TEMP="http://myhost.com/temp/xyx.png" $sed -e... (1 Reply)
Discussion started by: 8055
1 Replies

9. Shell Programming and Scripting

Use curl to send a static xml file using url encoding to a web page using pos

Hi I am try to use curl to send a static xml file using url encoding to a web page using post. This has to go through a particular port on our firewall as well. This is my first exposure to curl and am not having much success, so any help you can supply, or point me in the right direction would be... (1 Reply)
Discussion started by: Paul Walker
1 Replies

10. Solaris

View file encoding then change encoding.

Hi all!! I´m using command file -i myfile.xml to validate XML file encoding, but it is just saying regular file . I´m expecting / looking an output as UTF8 or ANSI / ASCII Is there command to display the files encoding? Thank you! (2 Replies)
Discussion started by: mrreds
2 Replies
All times are GMT -4. The time now is 06:18 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy