I too was able to replicate the observed behavior. Corona688 is correct in that it's not an attempt to deny access to the file. It's either apache or wget being stupid. I cannot confirm at the moment which since the wget header dump only included the server side of the conversation (@#$@@#?).

In any case, this is what's happening.

When Firefox requests the file, it indicates that it accepts gzip encoding. When wget or curl ask for it, they do not indicate this. In a bizarre attempt to be helpful, instead of sending you the compressed text file, or redirecting, or refusing to comply, the webserver sends you plain text.

That in itself seems foolish, as depending on the client headers you may download a .gz file that may or may not be a gzip'd file. Meanwhile, the Content-Type header always indicates "application/x-gzip".

(We're just getting warmed up.)

The server response, in the Content-Length header, indicates that the data (you know, the gzip'd text which is actually gunzip'd text) that it's sending you is 13258 bytes long. In its infinte wisdom, their Apache decides to close the connection one byte short of the advertised size.

(Just when you think things couldn't get more messed up ...)

When wget reconnects to finish the transfer, their webserver begins sending at the byte offset requested, but in the original, gzip compressed data file ... and continues to send until the end of that compressed data. This is why you see an identical file size that begins with text followed by "garbled data".

Using dd to skip the first 13257 bytes in the mangled file, I used cmp to compare the remaining bytes with their counterparts in the file downloaded from Firefox. They were identical.

So, in the end, the transfer received is not the 13258 bytes advertised by the first server response, but the 86777 bytes file size of the gzip'd compressed file with the first 13257 bytes as uncompressed text and the remainder as gzip'd data.

Long story short: Tell Apache that you can handle gzip'd data. Using curl, the following option works around the problem:

Code:

-H 'Accept-Encoding: gzip'

Regards,
Alister

---------- Post updated at 12:13 PM ---------- Previous update was at 11:52 AM ----------

Nah. curl is simply not retrying after the webserver closes the connection. Both curl and wget are sent plain text before the connection closes. Only wget reconnects and begins receiving gzip'd data.

Regards and welcome to the forum,
Alister