Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Newbie Python Url Scraper
Top Forums Shell Programming and Scripting Newbie Python Url Scraper Post 302816785 by metallica1973 on Tuesday 4th of June 2013 02:40:02 PM
Old 06-04-2013
Thanks for the reply. It didnt make a difference.
Code:
wget --spider r --server-response -U netscape http://192.168.3.3

Spider mode enabled. Check if remote file exists.
--2013-06-04 14:34:51--  (try:19)  http://192.168.3.3/
Connecting to 192.168.3.3:80... connected.
HTTP request sent, awaiting response... Read error (Connection reset by peer) in headers.
Retrying.

Spider mode enabled. Check if remote file exists.
--2013-06-04 14:35:01--  (try:20)  http://192.168.3.3/
Connecting to 192.168.3.3:80... connected.
HTTP request sent, awaiting response... Read error (Connection reset by peer) in headers.
Giving up.

I tried just a basic HTTP GET request and got this:
Code:
nc -v 192.168.3.3 80
Connection to 192.168.3.3 80 port [tcp/http] succeeded!
GET / HTTP/1.0

HTTP/1.1 400 Bad Request
Server: Netwave IP Camera
Date: Tue, 04 Jun 2013 18:37:28 GMT
Content-Type: text/html
Content-Length: 135
Connection: close

<HTML><HEAD><TITLE>400 Bad Request</TITLE></HEAD>
<BODY BGCOLOR="#cc9999"><H4>400 Bad Request</H4>
Can't parse request.
</BODY></HTML>

I will dig around. Thanks
 

9 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

url calling and parameter passing to url in script

Hi all, I need to write a unix script in which need to call a url. Then need to pass parameters to that url. please help. Regards, gander_ss (1 Reply)
Discussion started by: gander_ss
1 Replies

2. Shell Programming and Scripting

url calling and parameter passing to url in script

Hi all, I need to write a unix script in which need to call a url. Then need to pass parameters to that url. please help. Regards, gander_ss (1 Reply)
Discussion started by: gander_ss
1 Replies

3. Programming

NEWBIE QUESTION: python 3 or 2.6.x

I'm a newbie and want to learn a programming language, willy-nilly I picked python... Should I go with 2.6.x which at first glance seems extremely well documented, or should I go with 3.0, which is new and shiny?! I want...no...I'm going to NEED fantastic documentation or I'm going to fail... (2 Replies)
Discussion started by: guptaxpn
2 Replies

4. UNIX for Dummies Questions & Answers

ReDirecting a URL to another URL - Linux

Hello, I need to redirect an existing URL, how can i do that? There's a current web address to a GUI that I have to redirect to another webaddress. Does anyone know how to do this? This is on Unix boxes Linux. example: https://m45.testing.address.net/host.php make it so the... (3 Replies)
Discussion started by: SkySmart
3 Replies

5. UNIX for Dummies Questions & Answers

UNIX newbie NEWBIE question!

Hello everyone, Just started UNIX today! In our school we use solaris. I just want to know how do I setup Solaris 10 not the GUI one, the one where you have to type the commands like ECHO, ls, pwd, etc... I have windows xp and I also have vmware. I hope I am not missing anything! :p (4 Replies)
Discussion started by: Hanamachi
4 Replies

6. Web Development

Regex to rewrite URL to another URL based on HTTP_HOST?

I am trying to find a way to test some code, but I need to rewrite a specific URL only from a specific HTTP_HOST The call goes out to http://SUB.DOMAIN.COM/showAssignment/7bde10b45efdd7a97629ef2fe01f7303/jsmodule/Nevow.Athena The ID in the middle is always random due to the cookie. I... (5 Replies)
Discussion started by: EXT3FSCK
5 Replies

7. UNIX for Dummies Questions & Answers

Awk: print all URL addresses between iframe tags without repeating an already printed URL

Here is what I have so far: find . -name "*php*" -or -name "*htm*" | xargs grep -i iframe | awk -F'"' '/<iframe*/{gsub(/.\*iframe>/,"\"");print $2}' Here is an example content of a PHP or HTM(HTML) file: <iframe src="http://ADDRESS_1/?click=5BBB08\" width=1 height=1... (18 Replies)
Discussion started by: striker4o
18 Replies

8. Shell Programming and Scripting

Python Newbie Question Regex

I starting teaching myself python and am stuck on trying to understand why I am not getting the output that I want. Long story short, I am using PDB for debugging and here my function in which I am having my issue: import re ... ... ... def find_all_flvs(url): soup =... (1 Reply)
Discussion started by: metallica1973
1 Replies

9. Shell Programming and Scripting

Reading URL using Mechanize and dump all the contents of the URL to a file

Hello, Am very new to perl , please help me here !! I need help in reading a URL from command line using PERL:: Mechanize and needs all the contents from the URL to get into a file. below is the script which i have written so far , #!/usr/bin/perl use LWP::UserAgent; use... (2 Replies)
Discussion started by: scott_cog
2 Replies

LEARN ABOUT CENTOS

pppoe-setup

PPPOE-SETUP(8)						      System Manager's Manual						    PPPOE-SETUP(8)

NAME

       pppoe-setup - Shell script to configure Roaring Penguin PPPoE client

SYNOPSIS

       pppoe-setup

DESCRIPTION

       pppoe-setup  is a shell script which prompts you for various pieces of information and sets up an /etc/sysconfig/network-scripts/ifcfg-ppp0
       configuration script for the pppoe-start, pppoe-stop and +pppoe-connect scripts.

AUTHOR

       pppoe-setup was written by David F. Skoll <dfs@roaringpenguin.com>.

       The pppoe home page is http://www.roaringpenguin.com/pppoe/.

SEE ALSO

       pppoe(8), pppoe-start(8), pppoe-stop(8), pppoe-connect(8), pppd(8), pppoe.conf(5), pppoe-status(8), pppoe-sniff(8), pppoe-relay(8),  pppoe-
       server(8)

4th Berkeley Distribution					 21 February 2000						    PPPOE-SETUP(8)
All times are GMT -4. The time now is 08:16 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy