Wget help - unable to download pic


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Wget help - unable to download pic
# 1  
Old 12-14-2015
Wget help - unable to download pic

There is a website providing traffic camera images that gets updated every few minutes.
My aim is to download the images over time to get a view of traffic conditions through the day.

Website: CHECKPOINT.SG
Image link, as taken from site source: http://www.checkpoint.sg/sg/2701

I tried downloading the file with the code below, however all I get is a picture of a kitten.

Code:
wget -q "http://www.checkpoint.sg/sg/2701" --referer="http://www.checkpoint.sg" ‐‐execute robots=off --user-agent="Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.63" --output-document="2701.jpg"

Anyone can help me on this?
# 2  
Old 12-15-2015
The site uses a cookie to validate you are not a bot. You can verify this in chrome by opening the required image in a new tab, enabling developer tools and reloading the image.

I suspect the site owner does not want their data reused without permission.

If you believe the site owner would not object to your re-use you should open a conversation with them as they are likely to modify the image security methodology as soon as they notice someone bypassing it...
# 3  
Old 12-15-2015
Thanks for informing about using the development tools.
Found the error - need a / at the end of the referrer url.


Code:
wget -q "http://www.checkpoint.sg/sg/2701" --referer="http://www.checkpoint.sg/"  --output-document="2701.jpg"

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Wget - working in browser but cannot download from wget

Hi, I need to download a zip file from my the below US govt link. https://www.sam.gov/SAMPortal/extractfiledownload?role=WW&version=SAM&filename=SAM_PUBLIC_MONTHLY_20160207.ZIP I only have wget utility installed on the server. When I use the below command, I am getting error 403... (2 Replies)
Discussion started by: Prasannag87
2 Replies

2. Shell Programming and Scripting

How to change wget download directory?

i have a cron that mirrors a site periodically wget -r -nc --passive-ftp ftp://user:pass@123.456.789.0 i want to download this into a directory called /files but when I do this, it always create a new directory called "123.456.789.0" (the hostname) it puts it into /files/123.456.789.0 but... (3 Replies)
Discussion started by: vanessafan99
3 Replies

3. Shell Programming and Scripting

How to cancel wget download after 1%?

I am running a video download test and automating that. I wanna know how to stop a wget download session when downloads reached 1% Thanks in advance, Tamil (11 Replies)
Discussion started by: tamil.pamaran
11 Replies

4. Shell Programming and Scripting

Files download using wget

Hi, I need to implement below logic to download files daily from a URL. * Need to check if it is yesterday's file (YYYY-DD-MM.dat) * If present then download from URL (sample_url/2013-01-28.dat) * Need to implement wait logic if not present * if it still not able to find the file... (1 Reply)
Discussion started by: rakesh5300
1 Replies

5. Shell Programming and Scripting

How to download to a file using wget in perl?

Hi, I want to download some online data using wget command and write the contents to a file. For example this is the URL i want to download and store it in a file called "results.txt". #This is the URL. $url="http://www.example.com"; #retrieve data and store in a file results.txt ... (3 Replies)
Discussion started by: vanitham
3 Replies

6. UNIX and Linux Applications

download file using wget

I need to download the following srs8.3.0.1.standard.linux26_32.tar.gz file from the following website: http://downloads.biowisdomsrs.com/srs83_dist There are many gzip files along with the above one in the above site but I want to download the srs8.3.0.1.standard.linux26_32.tar.gz only from... (1 Reply)
Discussion started by: alphasahoo
1 Replies

7. Shell Programming and Scripting

download a particular file using wget

Hi All I want to download srs8.3.0.1.standard.linux24_EM64T.tar.gz file from the following website : http://downloads.biowisdomsrs.com/srs83_dist/ But this website contains lots of zipped files I want to download the above file only discarding other zipped files. When I am trying the... (1 Reply)
Discussion started by: alphasahoo
1 Replies

8. UNIX for Dummies Questions & Answers

Using wget to download a file

Hello Everyone, I'm trying to use wget recursively to download a file. Only html files are being downloaded, instead of the target file. I'm trying this for the first time, here's what I've tried: wget -r -O jdk.bin... (4 Replies)
Discussion started by: thoughts
4 Replies

9. Shell Programming and Scripting

Script to download file using wget

Hi I need a Shell script that will download a text file every second from a http server using wget. Can anyone provide me any pointers or sample scripts that will help me go about this task ??? regards techie (1 Reply)
Discussion started by: techie82
1 Replies

10. UNIX for Dummies Questions & Answers

May download a folder using wget command!

Hi All, I think wget command would not download any directories. But please confirm it. If it downloads directories, please let me know how to do it. Thank you. (1 Reply)
Discussion started by: ThrdF
1 Replies
Login or Register to Ask a Question