Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

BASH scripting - Preventing wget messed downloaded files

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting BASH scripting - Preventing wget messed downloaded files
# 1  
Old 06-15-2013
BASH scripting - Preventing wget messed downloaded files
hello.
How can I detect within script, that the downloaded file had not a correct size.

Code:
linux:~ # wget --limit-rate=20k  --ignore-length  -O  /Software_Downloaded/MULTIMEDIA_ADDON/skype-4.1.0.20-suse.i586.rpm   http://www.skype.com/en/download-skype/skype-for-linux/downloading/?type=suse32/skype-4.1.0.20-suse.i586.rpm
--2013-06-15 15:43:44--  http://www.skype.com/en/download-skype/skype-for-linux/downloading/?type=suse32/skype-4.1.0.20-suse.i586.rpm
Resolving www.skype.com (www.skype.com)... 91.190.216.12
Connecting to www.skype.com (www.skype.com)|91.190.216.12|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: ignored [text/html]
Saving to: `/004_SOFTWARE/MULTIMEDIA_ADDON/skype-4.1.0.20-suse.i586.rpm'

    [    <=>                                                           ] 22,246      20.8K/s   in 1.0s

But the file size file is 22M and only 22K is downloaded.
Using or not "--ignore-length" and/or "--limite-rate" is not helping.

Help is welcome.
# 2  
Old 06-15-2013
What about checking the exit status of the wget command?
I suppose that it should be different than 0, when wget is not able to download the complete file.
# 3  
Old 06-15-2013
wget is actually downloading a "Thank you for downloading Skype" page (you can view the downloaded file in a Web Browser). You will need to find a direct download link, or download the file manually (assuming there's no way to wget can work around this, which I'm not aware of).
This User Gave Thanks to Scott For This Post:
radoulov
# 4  
Old 06-15-2013
It is more complicated.
If I put the full address in my browser, I get 2 pop-up, One with the thanks screen, One with the open/save app ( open with ark in my case ).

If I click on the save option, I do not download the full file, only a partial copy. In that case I can verify the size of the file.

This arise sometimes on some kind of internet site.

This is the reason why I post this question.
# 5  
Old 06-15-2013
That annoying website is using javascript to launch a cascade of redirects.

Starting at:
Code:
Referer: http://www.skype.com/en/download-skype/skype-for-linux/downloading/?type=suse32/skype-4.1.0.20-suse.i586.rpm

Leads to:
Code:
Location: http://download.skype.com/linux/skype-4.2.0.11-suse.i586.rpm

Note that the version delivered is not the same as the version requested. They redirect to the latest version, but you can hack the Location header's URL to fetch the originally requested version.

By the way, a little time with Firefox or Chrome developer tools is very useful when diagnosing these annoyances.

Regards,
Alister
# 6  
Old 06-15-2013
No change with the latest version :
Code:
linux:~ # wget  -O  /data_temp/skype-4.2.0.11-suse.i586.rpm   http://www.skype.com/en/download-skype/skype-for-linux/downloading/?type=suse32/skype-4.2.0.11-suse.i586.rpm
--2013-06-15 23:44:23--  http://www.skype.com/en/download-skype/skype-for-linux/downloading/?type=suse32/skype-4.2.0.11-suse.i586.rpm
Resolving www.skype.com (www.skype.com)... 91.190.216.12
Connecting to www.skype.com (www.skype.com)|91.190.216.12|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 22246 (22K) [text/html]
Saving to: `/data_temp/skype-4.2.0.11-suse.i586.rpm'

100%[============================================================================================>] 22,246      --.-K/s   in 0.1s    

2013-06-15 23:44:23 (158 KB/s) - `/data_temp/skype-4.2.0.11-suse.i586.rpm' saved [22246/22246]

linux-srv:~ # 
linux-srv:~ # echo $?
0
linux-srv:~ #

The file saved is 21,7 Kb but should be 12.2 Mb
# 7  
Old 06-15-2013
Quote:
Originally Posted by jcdole
No change ...

The file saved is 21,7 Kb but should be 12.2 Mb
Because the URL you're using requires the client to support JavaScript, which wget does not. My previous post provides a working URL.

Regards,
Alister
Last edited by alister; 06-15-2013 at 10:43 PM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to move files with first part as new filenamevia bash shell scripting?

I am on linux red hat, want to use the logic within bash shell script file. I have the following type, named files in folder /staging, want to move with new filename just the firstpart upto underscore, example 20180904105056.dat, how can i get upto first part of filename and validate to chk if... (2 Replies)
Discussion started by: cplusplus1
2 Replies

2. Shell Programming and Scripting

How to join 2 text files using bash scripting?

Hi Guys, I want to combine 2 files and and put together in 1 file . See below desired output. Any help will be much appreciated. FILE AX 2134 101L 12345.00 22222.00 1 10 X 2134 101L 12345.00 22222.00 11 20 X 2134 101L 12345.00 22222.00 21 30 X 2134 111L 77777.00 ... (3 Replies)
Discussion started by: H.R
3 Replies

3. Shell Programming and Scripting

Specific image to be downloaded with wget

Hello All, I have gone through Google and came to know that we can download images from a site using wget. Now I am been asked to check whether an image is populated in a site or not. If yes, please send that image to an address as an attachment.. Say for example, the site is Wiki -... (6 Replies)
Discussion started by: sathyaonnuix
6 Replies

4. Shell Programming and Scripting

Extract urls from index.html downloaded using wget

Hi, I need to basically get a list of all the tarballs located at uri I am currently doing a wget on urito get the index.html page Now this index page contains the list of uris that I want to use in my bash script. can someone please guide me ,. I am new to Linux and shell scripting. ... (5 Replies)
Discussion started by: mnanavati
5 Replies

5. Web Development

php files are downloaded

Hello, I have setup Cherokee web server and php 5.2 in Opensolaris zone. Problem is that all .php files are downloaded from web server and not served when I use IP address instead of DNS name in web brovser. Example: test.mydomain.com <-- php works 192.168.0.10/index.php <--... (3 Replies)
Discussion started by: kreno
3 Replies

6. Shell Programming and Scripting

Help with WGET and renaming downloaded files :(

Hi everybody, I would greatly appreciate some expertise in this matter. I am trying find an efficient way to batch download files from a website and rename each file with the url it originated from (from the CLI). (ie. Instead of xyz.zip, the output file would be http://www.abc.com/xyz.zip) A... (10 Replies)
Discussion started by: o0110o
10 Replies

7. Shell Programming and Scripting

how to limit files downloaded by wget

I am trying to download a page and retrieve only wav and mp3 files via wget. the website is: Alarm Sounds | Free Sound Effects | Alarm Sound Clips | Sound Bites my command is : wget -rl 2 -e robots=off -A wav,mp3 http://soundbible.com/tags-alarm.html When not using the -A wav,mp3... (2 Replies)
Discussion started by: Narnie
2 Replies

8. Shell Programming and Scripting

Preventing whitespace to be a delimiter in a for loop (bash/sh)

Hi, I have a for loop which iterates over a list of strings, separated by whitespace: $ list="1 2 3" $ for i in $list; do echo $i; done 1 2 3 I now want to introduce some strings containing whitespace themselves ... This is straightforward if I directly iterate over the list: $ for... (4 Replies)
Discussion started by: kkkoehne
4 Replies

9. Shell Programming and Scripting

Log FILES in BASH scripting

Hi Im new to BASH scripting, I have created a simple script #! /bin/sh LOCAL=/home/scanner NOW=$(date +"%b-%d-%y") LOGFILE="log-$NOW.log" cd $LOCAL echo 'Start'; echo $LOGFILE touch $LOGFILE (2 Replies)
Discussion started by: jackdaw
2 Replies

10. UNIX for Dummies Questions & Answers

Messed up tar files

I created tar files for directories using this command: tar -cvf * >tarfile what happened was I got a file tarfile with a list of the files and it took the first file in each directory and overwrote it with the actual tar file. I've been trying to figure out since yesterday what I did... (2 Replies)
Discussion started by: Barb
2 Replies
Login or Register to Ask a Question