Wget-403-Forbidden-Error


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Wget-403-Forbidden-Error
# 1  
Old 10-04-2014
Wget-403-Forbidden-Error

Hi Friends,

I did an extensive search over the internet and tried all possible solutions that were recommended, but couldn't figure this out.

Please see this link

Code:
http://www.dli.gov.in/data6/upload/0159/808/PTIFF/00000007.tif

It works.

But, when I try the following command

Code:
wget -r -nd --no-parent -U firefox -A tif http://www.dli.gov.in/data6/upload/0159/808/PTIFF/

I get the 403 forbidden error.

Could you please suggest a way around?
# 2  
Old 10-04-2014
I cannot see that link. No such server.

In any case, there's no reason a server needs to permit you to see the index of a folder. If it's also forbidden in a browser, then it's just plain forbidden because they don't want you to do that.
# 3  
Old 10-04-2014
A quick google search of that URL suggests the image you want is part of the "Brihatkathamanjari", available here in a variety of forms:

https://archive.org/details/Brihatkathamanjari
This User Gave Thanks to Corona688 For This Post:
# 4  
Old 10-04-2014
Hi Corona,

I could access that link.

But anyways, thanks for your response.

---------- Post updated at 10:13 PM ---------- Previous update was at 10:04 PM ----------

Quote:
Originally Posted by Corona688
I cannot see that link. No such server.

In any case, there's no reason a server needs to permit you to see the index of a folder. If it's also forbidden in a browser, then it's just plain forbidden because they don't want you to do that.

I figured out that the file numbers starts with 7 preceding zeroes and for every every number greater than 0, the preceding zeroes are decreased in number.

For ex:

Code:
00000001.tif. It goes like this until 00000009.tif

And then

Code:
00000010.tif until 00000099.tif (Note the 6 preceding zeroes)


And then

Code:
00000100.tif till 00000999.tif (Note the 5 preceding zeroes)

I used this command

Code:
wget http://www.dli.gov.in/data6/upload/0159/808/PTIFF/0000000{1..94}.tif

But I could only get until 00000009.tif. Could you please suggest a for loop?

Thanks

Last edited by jacobs.smith; 10-04-2014 at 11:14 PM.. Reason: More info.
# 5  
Old 10-04-2014
It simply cannot be accessed from here. DNS returns nothing. Very very strange.

If it's somehow valid where you are, you could try playing with the referer settings:

Code:
wget --referer=http://www.dli.gov.in/ -U netscape

...which should pretend a little more to be a web browser and not a mining robot.

But actually, it would be simpler to go to http://www.dli.gov.in/data6/upload/0159/808/PTIFF/ in your browser since you say it works from there, then just save the list of URL's.
This User Gave Thanks to Corona688 For This Post:
# 6  
Old 10-04-2014
Quote:
Originally Posted by Corona688
It simply cannot be accessed from here. DNS returns nothing. Very very strange.

If it's somehow valid where you are, you could try playing with the referer settings:

Code:
wget --referer=http://www.dli.gov.in/ -U netscape

...which should pretend a little more to be a web browser and not a mining robot.

But actually, it would be simpler to go to http://www.dli.gov.in/data6/upload/0159/808/PTIFF/ in your browser and save that webpage, and get all the URL's from there.
Corona,

Actually only the tif files are made public. All the above folders are forbidden. Smilie

Could you please comment on the above for loop request?
# 7  
Old 10-04-2014
Quote:
Originally Posted by jacobs.smith
Actually only the tif files are made public. All the above folders are forbidden. Smilie
Then, for your original question, you have your answer. It won't work with wget if it won't work with your browser.

Code:
for ((N=1; N<100; N++))
do
        printf "%s/%06d.tif\n" "http://www.dli.gov.in/data6/upload/0159/808/PTIFF" $N
done | wget -I -

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Web Development

Directory index forbidden by Options directive error on specific directory with indexing disabled

I am seeing the following error appear numerous times in my Apache error log: I have my Apache config configured as below, so I would expect indexing not to occur on this directory as it falls under the parent /web directory. Strangely all the IP address, including this example, all... (5 Replies)
Discussion started by: crmpicco
5 Replies

2. UNIX for Dummies Questions & Answers

Script to if webpage is: forbidden, Error...

I have a file with about 29,000 website names. I would like to find a script that would test if each website name is current/active or not. If not it should check if the page is: Forbidden, 404 Error... (1 Reply)
Discussion started by: dcovnton
1 Replies

3. Web Development

forbidden error when accessing file(s) on a unix filesystem

greetings, i was recently asked to install apache1.3.31 on an aix5.3 server so users could have access to their output files. these two directories are nfs filesystems served by a netapp filer mounted on the aix machine. i've made no changes to the apache config and it's running as it came out of... (0 Replies)
Discussion started by: crimso
0 Replies

4. UNIX for Dummies Questions & Answers

wget error

Hello I am getting the following error please help me out as why I am unable to get wget command right. oracle@host:/home/oracle>wget /usr/lib/dld.sl: Unresolved symbol: libintl_bindtextdomain (code) from wget Abort(coredump) Thanks (15 Replies)
Discussion started by: ronick11
15 Replies

5. Web Development

Change internally 403 Error to 404 Error

I have the htaccess file within the folder in the domain I want to deny access to all users, execept one IP, after that redirects Error 403 resulting to a 404 error custom html page as shown below. # Restringting access! deny from all allow from 204.127.111.105 # Example of allowed IP... (1 Reply)
Discussion started by: cgkmal
1 Replies

6. Shell Programming and Scripting

Wget-Forbidden error

Hi , I am using shell script to connect to a https site and download all the files in a directory. eg: wget --no-check-certificate -r -np --user=<username> --password=<passwd> -O temp.txt https:<website>/<dirname> Error : HTTP request sent, awaiting response... 403 Forbidden 07:40:20... (0 Replies)
Discussion started by: mohanpadamata
0 Replies

7. Web Development

403 Forbidden / show directories

I have a directory that I want to just list the items when going to the URL instead of having an index.html page in the folder. I keep getting a 403 forbidden even though it is in my document root. I tried to add: Alias /keys/ "/var/www/html/keys/" <Directory "/var/www/html/keys"> ... (6 Replies)
Discussion started by: ippy98
6 Replies

8. UNIX for Dummies Questions & Answers

wget error

I'm trying to download / install mysql from the terminal. Here is what happens when I try to use wget: $ wget http://dev.mysql.com/get/Downloads/MySQL-5.1/MySQL-server-community-5.1.35-0.rhel3.i386.rpm/from/ftp://mirror.anl.gov/pub/mysql/ --13:29:16-- ... (6 Replies)
Discussion started by: wsetchell
6 Replies

9. Linux

Apache Forbidden Error

When i go to my browser and type both localhost or the ip, i get Forbidden, you do not have permission to access / on this machine Whys that ? (1 Reply)
Discussion started by: perleo
1 Replies

10. UNIX for Advanced & Expert Users

403 Error in Apache?

I have installed Apache lots of times before. I have recently installed apache on FreeBSD. And it gives me all the time 403 Frobidden - you don't have permittion to access / directory. I have checked permitions. I've got -rwxrwxr-x for for all Apache DocumentRoot. I've checked Files directive in... (6 Replies)
Discussion started by: solvman
6 Replies
Login or Register to Ask a Question