URL encoding | Unix Linux Forums | Shell Programming and Scripting

  Go Back    


Shell Programming and Scripting Post questions about KSH, CSH, SH, BASH, PERL, PHP, SED, AWK and OTHER shell scripts and shell scripting languages here.

URL encoding

Shell Programming and Scripting


Closed Thread    
 
Thread Tools Search this Thread Display Modes
    #1  
Old 04-06-2008
Vichu Vichu is offline
Registered User
 
Join Date: Apr 2008
Last Activity: 28 September 2010, 3:00 PM EDT
Posts: 10
Thanks: 0
Thanked 0 Times in 0 Posts
Question URL encoding

Hi All,

I want to do URL encoding using shell script in my project. I decided that the sed is the correct tool to do this. But I am unable achieve what I wanted using sed. kindly help me to get rid of this.

My requirement is , there will be one URL with all special character, spaces etc

For ex. https://www.xxxxxx.com/change&$ ^this to?%checkthe@-functionality..I want to do URL encoding only after ? mark. Final result should be https://www.xxxxxx.com/change&$ ^this to?<encoded 2nd part>

Thanks in advance

Regards
Vichu
Sponsored Links
    #2  
Old 04-06-2008
era era is offline Forum Advisor  
Herder of Useless Cats (On Sabbatical)
 
Join Date: Mar 2008
Last Activity: 28 March 2011, 6:41 AM EDT
Location: /there/is/only/bin/sh
Posts: 3,653
Thanks: 0
Thanked 10 Times in 8 Posts
There is nothing after the ? which requires encoding except the percent sign. Perhaps you should pick a more detailed example.

It's not going to be very elegant to do this in sed because it requires a loop, and loops are kind of tricky in sed. Basically, stash away the part you don't want to encode, loop over the remaining part, moving away everything you have already encoded by appending it to the stash.

Maybe something like this, instead?


Code:
perl -ple 's/\?(.*)//;
my $tail = $1;
$tail =~ s/([%? +&!<>()])/sprintf "%%%02x", ord($1) /ge;
s/$/$tail/'

The list of characters which require or might benefit from escaping is quite probably not complete. This assumes you have nothing after the URL which is not part of the URL, and that the first question mark separates the tail which requires encoding from the base URL.
Sponsored Links
    #3  
Old 04-06-2008
Vichu Vichu is offline
Registered User
 
Join Date: Apr 2008
Last Activity: 28 September 2010, 3:00 PM EDT
Posts: 10
Thanks: 0
Thanked 0 Times in 0 Posts
Seperating URL into two parts (? is the delimiter before ? is the first part and after ? is the second part) and I can do encoding for the second part using awk..But I don't want to split the line. Just skipping first part of URL and encoding the second part.
    #4  
Old 04-06-2008
era era is offline Forum Advisor  
Herder of Useless Cats (On Sabbatical)
 
Join Date: Mar 2008
Last Activity: 28 March 2011, 6:41 AM EDT
Location: /there/is/only/bin/sh
Posts: 3,653
Thanks: 0
Thanked 10 Times in 8 Posts
Did you try the code I posted?
Sponsored Links
    #5  
Old 04-07-2008
Vichu Vichu is offline
Registered User
 
Join Date: Apr 2008
Last Activity: 28 September 2010, 3:00 PM EDT
Posts: 10
Thanks: 0
Thanked 0 Times in 0 Posts
Hi era,

Thanks for your promptness..But I don't want to do it in perl...My requirement is in shell..
Sponsored Links
    #6  
Old 04-07-2008
V3l0 V3l0 is offline
Registered User
 
Join Date: Nov 2007
Last Activity: 10 April 2008, 10:18 AM EDT
Location: Belgium & France
Posts: 70
Thanks: 0
Thanked 0 Times in 0 Posts
Use 2 files:
urlencode.sed
Code:
s/%/%25/g
s/ /%20/g
s/ /%09/g
s/!/%21/g
s/"/%22/g
s/#/%23/g
s/\$/%24/g
s/\&/%26/g
s/'\''/%27/g
s/(/%28/g
s/)/%29/g
s/\*/%2a/g
s/+/%2b/g
s/,/%2c/g
s/-/%2d/g
s/\./%2e/g
s/\//%2f/g
s/:/%3a/g
s/;/%3b/g
s//%3e/g
s/?/%3f/g
s/@/%40/g
s/\[/%5b/g
s/\\/%5c/g
s/\]/%5d/g
s/\^/%5e/g
s/_/%5f/g
s/`/%60/g
s/{/%7b/g
s/|/%7c/g
s/}/%7d/g
s/~/%7e/g
s/      /%09/g

urlencode.sh
Code:
#!/bin/ksh

STR1=$(echo "https://www.xxxxxx.com/change&$ ^this to?%checkthe@-functionality" | cut -d\? -f1)
STR2=$(echo "https://www.xxxxxx.com/change&$ ^this to?%checkthe@-functionality" | cut -d\? -f2)

OUT2=$(echo "$STR2" | sed -f urlencode.sed)

echo "$STR1?$OUT2"

Result:
Code:
./urlencode.sh
https://www.xxxxxx.com/change&$ ^this to?%25checkthe%40%2dfunctionality

OK for you ?
Sponsored Links
    #7  
Old 04-07-2008
Vichu Vichu is offline
Registered User
 
Join Date: Apr 2008
Last Activity: 28 September 2010, 3:00 PM EDT
Posts: 10
Thanks: 0
Thanked 0 Times in 0 Posts
I proposed the same procedure,whatever you have given here, using awk.But my team didnot accept it. They dont want to use extra varialbes.They would like to do encoding (IInd part) on the same variable.

My proposed steps
==============
str1=`echo "https://www.xxxxxx.com/change&$ ^this to?%checkthe@-functionality" | awk -F? '{print $1}'`

str2=`echo "https://www.xxxxxx.com/change&$ ^this to?%checkthe@-functionality" | awk -F? '{print $2}'`

str3=`echo $str2 | sed -f ./seq.sed `

echo $str?$str3

Do you have any idea?
Sponsored Links
Closed Thread

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
Encoding problem /dev/bag Solaris 4 06-06-2011 06:27 AM
How to find the file encoding and updating the file encoding? cnraja Shell Programming and Scripting 7 05-27-2011 06:50 AM
Encoding troubles tukuyomi Shell Programming and Scripting 3 11-04-2009 04:42 PM
Araic Encoding habuzahra Shell Programming and Scripting 2 07-02-2009 08:38 PM
encoding palmer18 UNIX for Dummies Questions & Answers 3 08-21-2007 09:35 AM



All times are GMT -4. The time now is 07:39 PM.