URL encoding

Vichu · #1 (**permalink**) 04-06-2008

Hi All,

I want to do URL encoding using shell script in my project. I decided that the sed is the correct tool to do this. But I am unable achieve what I wanted using sed. kindly help me to get rid of this.

My requirement is , there will be one URL with all special character, spaces etc…

For ex. https://www.xxxxxx.com/change&$ ^this to?%checkthe@-functionality..I want to do URL encoding only after “?” mark. Final result should be https://www.xxxxxx.com/change&$ ^this to?<encoded 2nd part>

Thanks in advance

Regards
Vichu

era · #2 (**permalink**) 04-06-2008

There is nothing after the ? which requires encoding except the percent sign. Perhaps you should pick a more detailed example.

It's not going to be very elegant to do this in sed because it requires a loop, and loops are kind of tricky in sed. Basically, stash away the part you don't want to encode, loop over the remaining part, moving away everything you have already encoded by appending it to the stash.

Maybe something like this, instead?

Code:

perl -ple 's/\?(.*)//;
my $tail = $1;
$tail =~ s/([%? +&!<>()])/sprintf "%%%02x", ord($1) /ge;
s/$/$tail/'

The list of characters which require or might benefit from escaping is quite probably not complete. This assumes you have nothing after the URL which is not part of the URL, and that the first question mark separates the tail which requires encoding from the base URL.

Vichu · #3 (**permalink**) 04-06-2008

Seperating URL into two parts (? is the delimiter before ? is the first part and after ? is the second part) and I can do encoding for the second part using awk..But I don't want to split the line. Just skipping first part of URL and encoding the second part.

era · #4 (**permalink**) 04-06-2008

Did you try the code I posted?

Vichu · #5 (**permalink**) 04-07-2008

Hi era,

Thanks for your promptness..But I don't want to do it in perl...My requirement is in shell..

V3l0 · #6 (**permalink**) 04-07-2008

Use 2 files:
urlencode.sed

Code:

s/%/%25/g
s/ /%20/g
s/ /%09/g
s/!/%21/g
s/"/%22/g
s/#/%23/g
s/\$/%24/g
s/\&/%26/g
s/'\''/%27/g
s/(/%28/g
s/)/%29/g
s/\*/%2a/g
s/+/%2b/g
s/,/%2c/g
s/-/%2d/g
s/\./%2e/g
s/\//%2f/g
s/:/%3a/g
s/;/%3b/g
s//%3e/g
s/?/%3f/g
s/@/%40/g
s/\[/%5b/g
s/\\/%5c/g
s/\]/%5d/g
s/\^/%5e/g
s/_/%5f/g
s/`/%60/g
s/{/%7b/g
s/|/%7c/g
s/}/%7d/g
s/~/%7e/g
s/      /%09/g

urlencode.sh

Code:

#!/bin/ksh

STR1=$(echo "https://www.xxxxxx.com/change&$ ^this to?%checkthe@-functionality" | cut -d\? -f1)
STR2=$(echo "https://www.xxxxxx.com/change&$ ^this to?%checkthe@-functionality" | cut -d\? -f2)

OUT2=$(echo "$STR2" | sed -f urlencode.sed)

echo "$STR1?$OUT2"

Result:

Code:

./urlencode.sh
https://www.xxxxxx.com/change&$ ^this to?%25checkthe%40%2dfunctionality

OK for you ?

Vichu · #7 (**permalink**) 04-07-2008

I proposed the same procedure,whatever you have given here, using awk.But my team didnot accept it. They dont want to use extra varialbes.They would like to do encoding (IInd part) on the same variable.

My proposed steps
==============
str1=`echo "https://www.xxxxxx.com/change&$ ^this to?%checkthe@-functionality" | awk -F? '{print $1}'`

str2=`echo "https://www.xxxxxx.com/change&$ ^this to?%checkthe@-functionality" | awk -F? '{print $2}'`

str3=`echo $str2 | sed -f ./seq.sed `

echo $str?$str3

Do you have any idea?

More UNIX and Linux Forum Topics You Might Find Helpful
Thread	Thread Starter	Forum	Replies	Last Post
Encoding Problem while using "\|" (PIPE) as delimiter from Mainframe to Unix	seshendra	UNIX for Dummies Questions & Answers	1	02-20-2008 05:36 AM
File encoding in Unix	ssmallya	UNIX for Dummies Questions & Answers	6	02-04-2008 10:39 AM
character encoding in Fedora6	bsky	UNIX for Dummies Questions & Answers	1	01-04-2008 09:29 AM
encoding	palmer18	UNIX for Dummies Questions & Answers	3	08-21-2007 09:35 AM
no SOAP encoding under unix?	devotedsinner	SUN Solaris	0	11-07-2005 07:28 AM

Thread Tools	Search this Thread
Show Printable Version Email this Page	Search this Thread: Advanced Search
Display Modes	Rate This Thread
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode	Rate This Thread: