Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Newbie Python Url Scraper
Top Forums Shell Programming and Scripting Newbie Python Url Scraper Post 302816785 by metallica1973 on Tuesday 4th of June 2013 02:40:02 PM
Old 06-04-2013
Thanks for the reply. It didnt make a difference.
Code:
wget --spider r --server-response -U netscape http://192.168.3.3

Spider mode enabled. Check if remote file exists.
--2013-06-04 14:34:51--  (try:19)  http://192.168.3.3/
Connecting to 192.168.3.3:80... connected.
HTTP request sent, awaiting response... Read error (Connection reset by peer) in headers.
Retrying.

Spider mode enabled. Check if remote file exists.
--2013-06-04 14:35:01--  (try:20)  http://192.168.3.3/
Connecting to 192.168.3.3:80... connected.
HTTP request sent, awaiting response... Read error (Connection reset by peer) in headers.
Giving up.

I tried just a basic HTTP GET request and got this:
Code:
nc -v 192.168.3.3 80
Connection to 192.168.3.3 80 port [tcp/http] succeeded!
GET / HTTP/1.0

HTTP/1.1 400 Bad Request
Server: Netwave IP Camera
Date: Tue, 04 Jun 2013 18:37:28 GMT
Content-Type: text/html
Content-Length: 135
Connection: close

<HTML><HEAD><TITLE>400 Bad Request</TITLE></HEAD>
<BODY BGCOLOR="#cc9999"><H4>400 Bad Request</H4>
Can't parse request.
</BODY></HTML>

I will dig around. Thanks
 

9 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

url calling and parameter passing to url in script

Hi all, I need to write a unix script in which need to call a url. Then need to pass parameters to that url. please help. Regards, gander_ss (1 Reply)
Discussion started by: gander_ss
1 Replies

2. Shell Programming and Scripting

url calling and parameter passing to url in script

Hi all, I need to write a unix script in which need to call a url. Then need to pass parameters to that url. please help. Regards, gander_ss (1 Reply)
Discussion started by: gander_ss
1 Replies

3. Programming

NEWBIE QUESTION: python 3 or 2.6.x

I'm a newbie and want to learn a programming language, willy-nilly I picked python... Should I go with 2.6.x which at first glance seems extremely well documented, or should I go with 3.0, which is new and shiny?! I want...no...I'm going to NEED fantastic documentation or I'm going to fail... (2 Replies)
Discussion started by: guptaxpn
2 Replies

4. UNIX for Dummies Questions & Answers

ReDirecting a URL to another URL - Linux

Hello, I need to redirect an existing URL, how can i do that? There's a current web address to a GUI that I have to redirect to another webaddress. Does anyone know how to do this? This is on Unix boxes Linux. example: https://m45.testing.address.net/host.php make it so the... (3 Replies)
Discussion started by: SkySmart
3 Replies

5. UNIX for Dummies Questions & Answers

UNIX newbie NEWBIE question!

Hello everyone, Just started UNIX today! In our school we use solaris. I just want to know how do I setup Solaris 10 not the GUI one, the one where you have to type the commands like ECHO, ls, pwd, etc... I have windows xp and I also have vmware. I hope I am not missing anything! :p (4 Replies)
Discussion started by: Hanamachi
4 Replies

6. Web Development

Regex to rewrite URL to another URL based on HTTP_HOST?

I am trying to find a way to test some code, but I need to rewrite a specific URL only from a specific HTTP_HOST The call goes out to http://SUB.DOMAIN.COM/showAssignment/7bde10b45efdd7a97629ef2fe01f7303/jsmodule/Nevow.Athena The ID in the middle is always random due to the cookie. I... (5 Replies)
Discussion started by: EXT3FSCK
5 Replies

7. UNIX for Dummies Questions & Answers

Awk: print all URL addresses between iframe tags without repeating an already printed URL

Here is what I have so far: find . -name "*php*" -or -name "*htm*" | xargs grep -i iframe | awk -F'"' '/<iframe*/{gsub(/.\*iframe>/,"\"");print $2}' Here is an example content of a PHP or HTM(HTML) file: <iframe src="http://ADDRESS_1/?click=5BBB08\" width=1 height=1... (18 Replies)
Discussion started by: striker4o
18 Replies

8. Shell Programming and Scripting

Python Newbie Question Regex

I starting teaching myself python and am stuck on trying to understand why I am not getting the output that I want. Long story short, I am using PDB for debugging and here my function in which I am having my issue: import re ... ... ... def find_all_flvs(url): soup =... (1 Reply)
Discussion started by: metallica1973
1 Replies

9. Shell Programming and Scripting

Reading URL using Mechanize and dump all the contents of the URL to a file

Hello, Am very new to perl , please help me here !! I need help in reading a URL from command line using PERL:: Mechanize and needs all the contents from the URL to get into a file. below is the script which i have written so far , #!/usr/bin/perl use LWP::UserAgent; use... (2 Replies)
Discussion started by: scott_cog
2 Replies

LEARN ABOUT DEBIAN

fair

fair(7) 					       BSD Miscellaneous Information Manual						   fair(7)

NAME

     fair -- simple TCP load balancing service

DESCRIPTION

     fair is a load balancer for TCP connections.  It can be used to distribute incoming connections for SMTP, HTTP or any other TCP service to
     multiple hosts, distributing the load as evenly as possible.

     fair consists of two daemons.  The carrousel is the front-end; it keeps track of back-end hosts and their status, and forwards incoming con-
     nections to the back-ends in such a way that the load is distributed fairly.  The transponder runs on the back-end hosts, it registers with
     the carrousel and sends it status information.  The TCP connections forwarded by the carrousel are not sent to the transponder daemons but
     are sent directly to the desired service running on the back-end host. Both daemons share a single configuration file.

EXAMPLES

     The following example shows how to set up fair to service HTTP connections and to distribute them over back-ends in the 192.168.1.0/24 sub-
     net.

     The configuration file /etc/fair.conf contains the following:

	   WorkerService = http
	   BalancerService = http
	   AllowUDP = ^192.168.1.[0-9]+$

     On www.example.com, the front-end server receiving the incoming HTTP connections, just run:

	   carrousel

     On each of the back-ends run:

	   transponder www.example.com

SEE ALSO

     carrousel(8), transponder(8), fair.conf(5)

Debian GNU/Linux						   June 1, 2019 						  Debian GNU/Linux
All times are GMT -4. The time now is 12:47 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy