Sponsored Content
Top Forums Shell Programming and Scripting Using Wget with Digest Authentication Post 302663501 by mtarkowski on Thursday 28th of June 2012 07:47:40 AM
Old 06-28-2012
I would like to thank everyone who has responded, a lot of good information was provided but unfortunately I still cannot connect with Wget and get the same error response. The Tomcat server has othe Basic Auth pages and I can use Wget successfully with them but not the 1 page using digest auth. Here is what I have tried without success.
- confirmed ID and password work from a browser.
- tried wrapping the ID and password in single quotes.
- tried using the --header option
- installed the latest version to eliminate the bug in earlier versions
- tried using curl -w -S -v -u admin:abc --digest <URL_here>

Thanks to Tim R., Wget is now working for the version of Digest Authentication on my server.

What Tim discovered:
Your server expects the client to use RFC 2617 Digest Access Authentication.
Wget just supports RFC 2069 Digest Access Authentication - i took a look into wget's source code.
Firefox - and maybe all other modern browsers - uses (of course) RFC 2617.

That's why Firefox works ok and wget doesn't.

To solve your problem, you can either
- configure your server to accept RFC 2069 (which is a bit unsafer)
- wait until someone extends wget (which seems not to be a big deal)

Then I received the below patch and ran it in on file http.c in directory: wget/wget-1.13.4/src/
Patch -p1 < 0001-add-support-for-RFC-2617-Digest-Access-Authenticatio.patch

cd ../
sudo make install
Wget now is able to connect to the Digest protected page.

Thanks to everyone for their help on this.

Hereis the code for the patch.

From 649b8693d699c28830fc60f5da2c11ae83fdb22b Mon Sep 17 00:00:00 2001
From: Tim R.
Date: Thu, 28 Jun 2012 17:45:18 +0200
Subject: [PATCH] * add support for RFC 2617 Digest Access Authentication

---
src/http.c | 98 ++++++++++++++++++++++++++++++++++++++++++++++--------------
1 file changed, 76 insertions(+), 22 deletions(-)

diff --git a/src/http.c b/src/http.c
index 8d4edba..9ff7f28 100644
--- a/src/http.c
+++ b/src/http.c
@@ -3655,19 +3655,23 @@ digest_authentication_encode (const char *au, const char *user,
const char *passwd, const char *method,
const char *path)
{
- static char *realm, *opaque, *nonce;
+ static char *realm, *opaque, *nonce, *qop;
static struct {
const char *name;
char **variable;
} options[] = {
{ "realm", &realm },
{ "opaque", &opaque },
- { "nonce", &nonce }
+ { "nonce", &nonce },
+ { "qop", &qop }
};
+ char cnonce[16] = "";
char *res;
+ size_t res_size;
param_token name, value;

- realm = opaque = nonce = NULL;
+
+ realm = opaque = nonce = qop = NULL;

au += 6; /* skip over `Digest' */
while (extract_param (&au, &name, &value, ','))
@@ -3683,11 +3687,19 @@ digest_authentication_encode (const char *au, const char *user,
break;
}
}
+
+ if (qop!=NULL && strcmp(qop,"auth"))
+ {
+ logprintf (LOG_NOTQUIET, _("Unsupported quality of protection '%s'.\n"), qop);
+ user=NULL; /* force freeing mem and return */
+ }
+
if (!realm || !nonce || !user || !passwd || !path || !method)
{
xfree_null (realm);
xfree_null (opaque);
xfree_null (nonce);
+ xfree_null (qop);
return NULL;
}

@@ -3716,27 +3728,69 @@ digest_authentication_encode (const char *au, const char *user,
md5_finish_ctx (&ctx, hash);
dump_hash (a2buf, hash);

- /* RESPONSE_DIGEST = H(A1BUF ":" nonce ":" A2BUF) */
- md5_init_ctx (&ctx);
- md5_process_bytes ((unsigned char *)a1buf, MD5_DIGEST_SIZE * 2, &ctx);
- md5_process_bytes ((unsigned char *)":", 1, &ctx);
- md5_process_bytes ((unsigned char *)nonce, strlen (nonce), &ctx);
- md5_process_bytes ((unsigned char *)":", 1, &ctx);
- md5_process_bytes ((unsigned char *)a2buf, MD5_DIGEST_SIZE * 2, &ctx);
- md5_finish_ctx (&ctx, hash);
+ if (!strcmp(qop,"auth"))
+ {
+ /* RFC 2617 Digest Access Authentication */
+ /* generate random hex string */
+ snprintf(cnonce, sizeof(cnonce), "%08x", random_number(INT_MAX));
+
+ /* RESPONSE_DIGEST = H(A1BUF ":" nonce ":" noncecount ":" clientnonce ":" qop ": " A2BUF) */
+ md5_init_ctx (&ctx);
+ md5_process_bytes ((unsigned char *)a1buf, MD5_DIGEST_SIZE * 2, &ctx);
+ md5_process_bytes ((unsigned char *)":", 1, &ctx);
+ md5_process_bytes ((unsigned char *)nonce, strlen (nonce), &ctx);
+ md5_process_bytes ((unsigned char *)":", 1, &ctx);
+ md5_process_bytes ((unsigned char *)"00000001", 8, &ctx); /* TODO: keep track of server nonce values */
+ md5_process_bytes ((unsigned char *)":", 1, &ctx);
+ md5_process_bytes ((unsigned char *)cnonce, strlen(cnonce), &ctx);
+ md5_process_bytes ((unsigned char *)":", 1, &ctx);
+ md5_process_bytes ((unsigned char *)qop, strlen(qop), &ctx);
+ md5_process_bytes ((unsigned char *)":", 1, &ctx);
+ md5_process_bytes ((unsigned char *)a2buf, MD5_DIGEST_SIZE * 2, &ctx);
+ md5_finish_ctx (&ctx, hash);
+ }
+ else
+ {
+ /* RFC 2069 Digest Access Authentication */
+ /* RESPONSE_DIGEST = H(A1BUF ":" nonce ":" A2BUF) */
+ md5_init_ctx (&ctx);
+ md5_process_bytes ((unsigned char *)a1buf, MD5_DIGEST_SIZE * 2, &ctx);
+ md5_process_bytes ((unsigned char *)":", 1, &ctx);
+ md5_process_bytes ((unsigned char *)nonce, strlen (nonce), &ctx);
+ md5_process_bytes ((unsigned char *)":", 1, &ctx);
+ md5_process_bytes ((unsigned char *)a2buf, MD5_DIGEST_SIZE * 2, &ctx);
+ md5_finish_ctx (&ctx, hash);
+ }
+
dump_hash (response_digest, hash);

- res = xmalloc (strlen (user)
- + strlen (user)
- + strlen (realm)
- + strlen (nonce)
- + strlen (path)
- + 2 * MD5_DIGEST_SIZE /*strlen (response_digest)*/
- + (opaque ? strlen (opaque) : 0)
- + 128);
- sprintf (res, "Digest \
-username=\"%s\", realm=\"%s\", nonce=\"%s\", uri=\"%s\", response=\"%s\"",
- user, realm, nonce, path, response_digest);
+ res_size = strlen (user)
+ + strlen (user)
+ + strlen (realm)
+ + strlen (nonce)
+ + strlen (path)
+ + 2 * MD5_DIGEST_SIZE /*strlen (response_digest)*/
+ + (opaque ? strlen (opaque) : 0)
+ + (qop ? 128: 0)
+ + 128;
+
+ res = xmalloc (res_size);
+
+ if (!strcmp(qop,"auth"))
+ {
+ snprintf (res, res_size, "Digest "\
+ "username=\"%s\", realm=\"%s\", nonce=\"%s\", uri=\"%s\", response=\"%s\""\
+ ", qop=auth, nc=00000001, cnonce=\"%s\"",
+ user, realm, nonce, path, response_digest, cnonce);
+
+ }
+ else
+ {
+ snprintf (res, res_size, "Digest "\
+ "username=\"%s\", realm=\"%s\", nonce=\"%s\", uri=\"%s\", response=\"%s\"",
+ user, realm, nonce, path, response_digest);
+ }
+
if (opaque)
{
char *p = res + strlen (res);
--
1.7.10


Last edited by mtarkowski; 06-28-2012 at 04:56 PM.. Reason: Problem Solved
These 2 Users Gave Thanks to mtarkowski For This Post:
 

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Digest MD5

Dear Guys , Am sorry i ask alot , but i do not know that much about perl , cgi , MD5 ! now i installed MD5 and Digest MD5 to my solaries 7 sparc machine . when i execute the command : $perl Makefile.PL i get the follwoing error message ,, please tell me how to fix it , i need... (11 Replies)
Discussion started by: tamemi
11 Replies

2. Shell Programming and Scripting

Help with wget

Hi, i need temperature hourly from a web page Im using wget to get the web page. I would like to save the page downloaded in a file called page. I check the file everytime i run the wget function but its not saving but instead creates a wx.php file....Each time i run it...a new wx.php file is... (2 Replies)
Discussion started by: vadharah
2 Replies

3. Shell Programming and Scripting

wget

Hi I want to download some files using wget , and want to save in a specified directory. Is there any way to save it.Please suggest me. (1 Reply)
Discussion started by: mnmonu
1 Replies

4. UNIX for Dummies Questions & Answers

Different MD5 value when using 'digest' command

Hi guys, I need to anonymise some data; Some of it in an Oracle database and some in text files. I'm using the 'digest' command on Solaris 10 OS and an Oracle function to encode the data in the database. However, as a test, when i try to encode the same string in the dataabse ans OS, I get... (4 Replies)
Discussion started by: zaff
4 Replies

5. UNIX for Dummies Questions & Answers

Wget

...... (1 Reply)
Discussion started by: hoo
1 Replies

6. Shell Programming and Scripting

WGET help!

Hi Friends, I have an url like this https://www.unix.com/help/ In this help directory, I have more than 300 directories which contains file or files. So, the 300 directories are like this http://unix.com/help/ dir1 file1 dir2 file2 dir3 file3_1 file3_2... (4 Replies)
Discussion started by: jacobs.smith
4 Replies

7. Red Hat

Wget

If I run the following command wget -r --no-parent --reject "index.html*" 10.11.12.13/backups/ A local directory named 10.11.12.13/backups with the content of web site data is created. What I want to do is have the data placed in a local directory called $HOME/backups. Thanks for... (1 Reply)
Discussion started by: popeye
1 Replies

8. Red Hat

Digest::md5 needed for RHEL 6.4

Hey, I've got a package that requires 'Digest:md5' to be installed but I can't locate a source for it from Redhat. Poking around on the internet I found some stuff that seemed to be for RHEL 6.4 but when I had yum try a local install it gives me this error. Transaction Check Error: file... (3 Replies)
Discussion started by: DustinT
3 Replies

9. Shell Programming and Scripting

Wget and gz

Can wget be used to goto a site and piped into a .gz extrated command? wget ftp://ftp.ncbi.nlm.nih.gov/pub/clinvar/vcf_GRCh37 | gunzip -d clinvar_20150603.vcf.gz (1 Reply)
Discussion started by: cmccabe
1 Replies

10. Shell Programming and Scripting

Wget - working in browser but cannot download from wget

Hi, I need to download a zip file from my the below US govt link. https://www.sam.gov/SAMPortal/extractfiledownload?role=WW&version=SAM&filename=SAM_PUBLIC_MONTHLY_20160207.ZIP I only have wget utility installed on the server. When I use the below command, I am getting error 403... (2 Replies)
Discussion started by: Prasannag87
2 Replies
All times are GMT -4. The time now is 02:14 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy