Wget to ignore an IP address


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Wget to ignore an IP address
# 1  
Old 12-04-2012
Wget to ignore an IP address

Hello Unix Geeks,

I am in a situation to use wget for crawling a site where the site contains 5 IP addresses. Out of 5, 4 are accessible and 1[say it as X.X.X.X] is having a problem due to firewall problems.

In this case, my wget is getting stuck with that X.X.X.X and giving up. How can I ignore this IP and redirect my wget to use the other IP out of 4.

Kindly help me on this...
# 2  
Old 12-04-2012
You can try using one of the wget "timeout" options.
# 3  
Old 12-04-2012
timeout can help you to come out of it when the time breaches, isn't it ? My case is to try the other IP address and not the X.X.X.X
# 4  
Old 12-04-2012
There are possibly other options, such as changing your hosts file or using iptables, etc, but I don't want to make any assumptions about how you are doing what you're doing. Can you explain in more detail the steps you are taking and what the goal is?

Also, please say which operating system (incl. version) and what shell you are using.
# 5  
Old 12-04-2012
Code:
 $ wget --version
GNU Wget 1.11.4 Red Hat modified

Code:
uname -a
Linux 2.6.18-308.11.1.el5 #1 SMP Fri Jun 15 15:41:53 EDT 2012 x86_64 x86_64 x86_64 GNU/Linux

Code:
$ echo $0
ksh

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Wget - working in browser but cannot download from wget

Hi, I need to download a zip file from my the below US govt link. https://www.sam.gov/SAMPortal/extractfiledownload?role=WW&version=SAM&filename=SAM_PUBLIC_MONTHLY_20160207.ZIP I only have wget utility installed on the server. When I use the below command, I am getting error 403... (2 Replies)
Discussion started by: Prasannag87
2 Replies

2. UNIX for Advanced & Expert Users

C program to detect duplicate ip address if any after assigning ip address to ethernet interface

Hi , Could someone let me know how to detect duplicate ip address after assigning ip address to ethernet interface using c program (3 Replies)
Discussion started by: Gopi Krishna P
3 Replies

3. Shell Programming and Scripting

Wget - how to ignore files in immediate directory?

i am trying to recursively save a remote FTP server but exclude the files immediately under a directory directory1 wget -r -N ftp://user:pass@hostname/directory1 I want to keep these which may have more files under them directory1/dir1/file.jpg directory1/dir2/file.jpg... (16 Replies)
Discussion started by: vanessafan99
16 Replies

4. Shell Programming and Scripting

Specifying IP address with curl/wget

Hello, I am wondering does anyone know of a method using curl/wget or other where by I could specify the IP address of the server I wish to query for a website. Something similar to editing /etc/hosts but that can be done directly from the command line. I have looked through the man pages... (4 Replies)
Discussion started by: colinireland
4 Replies

5. IP Networking

Tracing a MAC address to IP address: Solaris

Hi there I lost connectivity to one of our remote systems and when I checked the messages log I found the following: Aug 10 23:42:34 host xntpd: time reset (step) 1.681729 s Aug 16 13:20:51 host ip: WARNING: node "mac address" is using our IP address x.x.x.x on aggr1 Aug 16 13:20:51 host... (9 Replies)
Discussion started by: notreallyhere
9 Replies

6. UNIX for Dummies Questions & Answers

What would the physical address be for virtual address?

Hi guys, I got one problem which I definetily no idea. What would the physical address be for virtual address? 1) 2ABC 2) 3F4B Here is the page table:see attached Thank you sos sososososso much!! (0 Replies)
Discussion started by: lemon_06
0 Replies

7. UNIX for Dummies Questions & Answers

Panic kernal-mode address fault on user address 0x14

:) Firstly Hi all!!, im NEW!! and on here hoping that someone might be able to offer me some help... i have a server that keeps crashing every few days with the error message: PANIC KERNAL-MODE ADDRESS FAULT ON USER ADDRESS 0X14 KERNAL PAGE FAULT FROM (CS:EIP)=(100:EF71B5BD) EAX=EF822000... (10 Replies)
Discussion started by: Twix
10 Replies

8. Shell Programming and Scripting

ksh - how to list all ip address between 2 ip address

Trying to do a ksh script that needs to list all ip address between ip address a and b .. ie. Ip address A=192.168.1.200 Ip address B=192.168.2.15 So the subnet changes from 1 to 2 but I want to list all possible ip addresses between the 2.. Which would be: 192.168.1.200... (4 Replies)
Discussion started by: frustrated1
4 Replies

9. IP Networking

How to Achive IP address through MAC(Ethernet) address

Hi sir, i want to make such programe which takes MAC(Ethernet) address of any host & give me its IP address....... but i'm nt getting that how i can pass the MAC address to Frame........ Please give me an idea for making such program... Thanks & regards Krishna (3 Replies)
Discussion started by: krishnacins
3 Replies

10. UNIX for Dummies Questions & Answers

network address and broadcast address?

say I have a IP address which is 10.0.0.12, and subnet mask is 255.255.255.240, what is the network address and what is the broadcast address which host lives on? And could you explain how to get the answer? thanx in advance! (7 Replies)
Discussion started by: pnxi
7 Replies
Login or Register to Ask a Question