Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: URL decoding with awk
Top Forums UNIX for Dummies Questions & Answers URL decoding with awk Post 302603629 by uiop44 on Thursday 1st of March 2012 02:22:54 PM
Old 03-01-2012
I had another go at this without using xxd.

This version uses only sh builtins, sed, jot/seq and dc.

Not thoroughly tested, but seems to work.

Handling unicode too would be useful.

Code:
# octal to hex #
o2h(){ 
echo 16o 8i $1 p|dc;
}

# print an ascii table #
ud1(){ 
{
case $(uname) in 
FreeBSD)
jot -w%03d 178 000 177;
\
 ;;
*)
seq -w 000 177;
\
 ;;
esac;
} \
 |while read a;do 
printf '\'$a'\040'$a'\n' ;
done \
 |sed '
1,/039/d;
/[89]$/d;
/[89].$/d;
$d;
/^ /s/ /+/;
/^$/d;
'

}

# rough sed script to fd1 #
ud2(){
sed 's,.* ,,' \
| while read a;
do 
printf 'sl%%';
printf $(o2h $a);
printf 'l'
printf '\'$a;
printf 'lg\n';
done;
}

# add escapes or quotes \
 # for certain specials chars \
 # and delete single quote hex 27 #
ud3(){
sed '
/20/s/ /+/;
/6C/{
s,l,/,;
s,ll,/l,;
s,lg,/g,;
};
/5C/s,.,\"&\",7;
/22/s,\",\\",;
/26/s,&,\\&,;
/27/d;
';
}

# add single quote hex 27 #
ud4(){
printf 'sl%%27l\047\"\047\"\047lg\n';
}

# function header #
ud5(){
echo "ud(){
sed '";
}

# function footer #
ud6(){
echo "'
}"
}

# put it all together #
ud7(){ 
{
ud5;
ud1 \
 |ud2 \
 |ud3;
ud4;
ud6;
};
}

Last edited by uiop44; 03-04-2012 at 09:58 PM..
 

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

url calling and parameter passing to url in script

Hi all, I need to write a unix script in which need to call a url. Then need to pass parameters to that url. please help. Regards, gander_ss (1 Reply)
Discussion started by: gander_ss
1 Replies

2. Shell Programming and Scripting

url calling and parameter passing to url in script

Hi all, I need to write a unix script in which need to call a url. Then need to pass parameters to that url. please help. Regards, gander_ss (1 Reply)
Discussion started by: gander_ss
1 Replies

3. UNIX for Dummies Questions & Answers

ReDirecting a URL to another URL - Linux

Hello, I need to redirect an existing URL, how can i do that? There's a current web address to a GUI that I have to redirect to another webaddress. Does anyone know how to do this? This is on Unix boxes Linux. example: https://m45.testing.address.net/host.php make it so the... (3 Replies)
Discussion started by: SkySmart
3 Replies

4. Shell Programming and Scripting

decoding URL encoded strings

Hi, I have a couple pages of URL encoded strings that I need to unencode (they were originally in Arabic). So the first step is to unencode the strings and then to translate them to English. They are actually lists of words so the translation from Arabic to English shouldn't be too complicated.... (1 Reply)
Discussion started by: ed111
1 Replies

5. Shell Programming and Scripting

how to judge wether a url is valid or not using awk

rt 3ks:confused: (6 Replies)
Discussion started by: rainboisterous
6 Replies

6. Shell Programming and Scripting

Extract URL from RSS Feed in AWK

Hi, I have following data file; <outline title="Matt Cutts" type="rss" version="RSS" xmlUrl="http://www.mattcutts.com/blog/feed/" htmlUrl="http://www.mattcutts.com/blog"/> <outline title="Stone" text="Stone" type="rss" version="RSS" xmlUrl="http://feeds.feedburner.com/STC-Art"... (8 Replies)
Discussion started by: fahdmirza
8 Replies

7. Web Development

Regex to rewrite URL to another URL based on HTTP_HOST?

I am trying to find a way to test some code, but I need to rewrite a specific URL only from a specific HTTP_HOST The call goes out to http://SUB.DOMAIN.COM/showAssignment/7bde10b45efdd7a97629ef2fe01f7303/jsmodule/Nevow.Athena The ID in the middle is always random due to the cookie. I... (5 Replies)
Discussion started by: EXT3FSCK
5 Replies

8. UNIX for Dummies Questions & Answers

Awk: print all URL addresses between iframe tags without repeating an already printed URL

Here is what I have so far: find . -name "*php*" -or -name "*htm*" | xargs grep -i iframe | awk -F'"' '/<iframe*/{gsub(/.\*iframe>/,"\"");print $2}' Here is an example content of a PHP or HTM(HTML) file: <iframe src="http://ADDRESS_1/?click=5BBB08\" width=1 height=1... (18 Replies)
Discussion started by: striker4o
18 Replies

9. UNIX for Dummies Questions & Answers

Decoding a string

Hi, If my input string is 3a3b4c then my result should be aaabbbcccc. Please guide me how to achieve this in a bash script. Thanks (18 Replies)
Discussion started by: pandeesh
18 Replies

10. Shell Programming and Scripting

Reading URL using Mechanize and dump all the contents of the URL to a file

Hello, Am very new to perl , please help me here !! I need help in reading a URL from command line using PERL:: Mechanize and needs all the contents from the URL to get into a file. below is the script which i have written so far , #!/usr/bin/perl use LWP::UserAgent; use... (2 Replies)
Discussion started by: scott_cog
2 Replies

LEARN ABOUT MOJAVE

seq

SEQ(1)							    BSD General Commands Manual 						    SEQ(1)

NAME

     seq -- print sequences of numbers

SYNOPSIS

     seq [-w] [-f format] [-s string] [-t string] [first [incr]] last

DESCRIPTION

     The seq utility prints a sequence of numbers, one per line (default), from first (default 1), to near last as possible, in increments of incr
     (default 1).  When first is larger than last the default incr is -1.

     All numbers are interpreted as floating point.

     Normally integer values are printed as decimal integers.

     The seq utility accepts the following options:

     -f format	   Use a printf(3) style format to print each number.  Only the E, e, f, G, g, and % conversion characters are valid, along with
		   any optional flags and an optional numeric minimum field width or precision.  The format can contain character escape sequences
		   in backslash notation as defined in ANSI X3.159-1989 (``ANSI C89'').  The default is %g.

     -s string	   Use string to separate numbers.  The string can contain character escape sequences in backslash notation as defined in ANSI
		   X3.159-1989 (``ANSI C89'').	The default is 
.

     -t string	   Use string to terminate sequence of numbers.  The string can contain character escape sequences in backslash notation as
		   defined in ANSI X3.159-1989 (``ANSI C89'').	This option is useful when the default separator does not contain a 
.

     -w 	   Equalize the widths of all numbers by padding with zeros as necessary.  This option has no effect with the -f option.  If any
		   sequence numbers will be printed in exponential notation, the default conversion is changed to %e.

     The seq utility exits 0 on success and non-zero if an error occurs.

EXAMPLES

	   # seq 1 3
	   1
	   2
	   3

	   # seq 3 1
	   3
	   2
	   1

	   # seq -w 0 .05 .1
	   0.00
	   0.05
	   0.10

SEE ALSO

     jot(1), printf(1), printf(3)

HISTORY

     The seq command first appeared in Plan 9 from Bell Labs.  A seq command appeared in NetBSD 3.0, and ported to FreeBSD 9.0.  This command was
     based on the command of the same name in Plan 9 from Bell Labs and the GNU core utilities.  The GNU seq command first appeared in the 1.13
     shell utilities release.

BUGS

     The -w option does not handle the transition from pure floating point to exponent representation very well.  The seq command is not bug for
     bug compatible with the Plan 9 from Bell Labs or GNU versions of seq.

BSD
								 February 19, 2010							       BSD
All times are GMT -4. The time now is 09:08 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy