Sponsored Content
Top Forums Shell Programming and Scripting How to get the page size (of a url) using wget Post 302100262 by Corona688 on Monday 18th of December 2006 09:52:48 AM
Old 12-18-2006
wget --spider --server-response [url] will print any headers the server returns without downloading the page proper. Unfortunately, many pages, particularly dynamically generated ones, won't report their size, just report "Length: unspecified [text/html]".
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

wget to check an URL

I all, I wrote an script which starts a Weblogic server and waits until its loaded to deploy several apps. The way I checked was something like: while ; do wget --spider <URL>:<port>/console > /dev/null 2>&1 rc=$? done This works perfectly because it's an HTML site and when server is... (2 Replies)
Discussion started by: AlbertGM
2 Replies

2. UNIX for Dummies Questions & Answers

wget with semicolon in page name

So, I'd like to wget a webpage, as its not going to stick around forever - but the problem is the webpage has a semicolon in it. wget http://example.com/stuff/asdf;asdf obviously doesn't get the right webpage. Any good way around this? (2 Replies)
Discussion started by: Julolidine
2 Replies

3. Shell Programming and Scripting

WGET cycling on an updating page

Hello, I am experiencing an issue while downloading a few pages using wget. All of them work without a problem except one which is a page that does a tail on the log and as a result is constantly getting updated. wget here seems to run endlessly and needs to be manually killed. I wanted to... (0 Replies)
Discussion started by: prafulnama
0 Replies

4. Shell Programming and Scripting

How to extract url from html page?

for example, I have an html file, contain <a href="http://awebsite" id="awebsite" class="first">website</a>and sometime a line contains more then one link, for example <a href="http://awebsite" id="awebsite" class="first">website</a><a href="http://bwebsite" id="bwebsite"... (36 Replies)
Discussion started by: 14th
36 Replies

5. UNIX for Dummies Questions & Answers

Launch a URL,validate username and password using wget or curl

Hi All, I want to launch "ex: http://gmail.com" from the cmd window and validate the credentials with username and password, is it possible? I have found something like this "wget --http-user=USER' --http-password=PASSWORD http://gmail.com" am new to this and unable to find a solution, i... (0 Replies)
Discussion started by: harsha85
0 Replies

6. Shell Programming and Scripting

Wget and single page

Good evening to all!! I'm trying to become familiar with wget. I would like to download a page from Wikipedia with all images and CSSs but without going down to all links present in the page. It should be named index.html. I would like also to save it to /mnt/us inside a new folder. This is... (5 Replies)
Discussion started by: silver18
5 Replies

7. UNIX for Dummies Questions & Answers

Read URL data from UNIX without wget,curl,lynx,w3m.

Hi Experts, Problem statement : We have an URL for which we need to read the data and get parsed inside the shell scripts. My Aix has very limited perl utility, i cant install any utility as well. Precisely, wget,cURL,Lynx,w3m and Lwp cant be used as i got these utilities only when i googled... (0 Replies)
Discussion started by: scott_cog
0 Replies

8. Shell Programming and Scripting

Read URL data from UNIX-CLI without Wget,CURL,w3m,LWP

Hi Experts, Problem statement : We have an URL for which we need to read the data and get parsed inside the shell scripts.My Aix has very limited perl utility, i cant install any utility as well. Precisely, wget,cURL,Lynx,w3m and Lwp cant be used as i got these utilities only when i googled it.... (12 Replies)
Discussion started by: scott_cog
12 Replies

9. Post Here to Contact Site Administrators and Moderators

Page Not Found error while parsing url

Hi I just tried to post following link while answering, its not parsing properly, just try on your browser Tried to paste while answering : https://www.unix.com/302873559-post2.htmlNot operator is not coming with HTML/PHP tags so attaching file (2 Replies)
Discussion started by: Akshay Hegde
2 Replies

10. Shell Programming and Scripting

Wget fails for a valid URL

Wget Error Codes: 0 No problems occurred. 1 Generic error code. 2 Parse error—for instance, when parsing command-line options, the .wgetrc or .netrc… 3 File I/O error. 4 Network failure. 5 SSL verification failure. 6 Username/password authentication failure. ... (3 Replies)
Discussion started by: mohtashims
3 Replies
Ns_ConnReturnStatus(3aolserver) 			   AOLserver Library Procedures 			   Ns_ConnReturnStatus(3aolserver)

__________________________________________________________________________________________________________________________________________________

NAME
Ns_ConnReturnBadRequest, Ns_ConnReturnForbidden, Ns_ConnReturnInternalError, Ns_ConnReturnNoResponse, Ns_ConnReturnNotFound, Ns_ConnReturn- NotImplemented, Ns_ConnReturnNotModified, Ns_ConnReturnOk, Ns_ConnReturnStatus, Ns_ConnReturnUnauthorized, Ns_RegisterRedirect - Routines to return simple standard responses SYNOPSIS
#include "ns.h" int Ns_ConnReturnBadRequest(conn, msg) int Ns_ConnReturnForbidden(conn) int Ns_ConnReturnInternalError(conn) int Ns_ConnReturnNoResponse(conn) int Ns_ConnReturnNotFound(conn) int Ns_ConnReturnNotImplemented(conn) int Ns_ConnReturnNotModified(conn) int Ns_ConnReturnOk(conn) int Ns_ConnReturnStatus(conn, status) int Ns_ConnReturnUnauthorized(conn) void Ns_RegisterRedirect(server, status, url) ARGUMENTS
Ns_Conn conn (in) Pointer to open connection. char *msg (in) String with additional message text. int status (in) Integer HTTP status code. char *url (in) String which specifies internal redirection url. char *server(in) Virtual server. _________________________________________________________________ DESCRIPTION
These routines are used to generate complete responses, including headers, approriate status codes, content types, and possibly short HTML content messages for the most common HTTP error or status responses. They each coorespond to a particular HTTP status code, for example, Ns_ConnReturnNotFound generates an HTTP 404 "Not Found" response. They all return NS_OK if the response was sent or NS_ERROR if an under- lying routine failed. The default behavior is to return an internal, server generated response possibly with a short English language message, for example "The requested URL cannot be accessed by this server". This behavior can be modified by calling the Ns_RegisterRedirect to redirect responses internally for the cooresponding HTTP status code to another URL on the server. The "redirects" server config section can be used to map these redirects at startup int Ns_ConnReturnBadRequest(conn, msg) Returns an HTTP 400 response with the short HTML message "Invalid Request: The HTTP request presented by your browser is invalid." The optional msg string, if present, is also included in the message body. int Ns_ConnReturnForbidden(conn) Returns an HTTP 403 response with the short HTML message "Forbidden: The requested URL cannot be accessed by this server." int Ns_ConnReturnInternalError(conn) Returns an HTTP 500 response with the short HTML message "Server Error: The requested URL cannot be accessed due to a system error on this server." int Ns_ConnReturnNoResponse(conn) Equivalent to Ns_ConnReturnStatus(conn, 204). int Ns_ConnReturnNotFound(conn) Returns an HTTP 404 response with the short HTML message "Not Found: The requested URL was not found on this server." int Ns_ConnReturnNotImplemented(conn) Returns an HTTP 404 response with the short HTML message "Not Implemented: The requested URL or method is not implemented by this server." int Ns_ConnReturnNotModified(conn) Equivalent to Ns_ConnReturnStatus(conn, 304). int Ns_ConnReturnOk(conn) Equivalent to Ns_ConnReturnStatus(conn, 200). int Ns_ConnReturnStatus(conn, status) Generates a response with the given HTTP status with no content. int Ns_ConnReturnUnauthorized(conn) Returns an HTTP 401 response with the short HTML message "Access Denied: The requested URL cannot be accessed because a valid user- name and password are required." As "WWW-Authenticate: Basic realm=server realm" header is also included in the response. void Ns_RegisterRedirect(server, status, url) Redirect the above responses from the given server for the given status code from the simple internal messages described above to the given internal url. The redirect is performed using Ns_ConnRedirect. SEE ALSO
Ns_ConnFlush(3), Ns_ConnRedirect(3), Ns_ConnSetRequiredHeaders(3), Ns_ConnQueueHeaders(3), ns_return(n) KEYWORDS
connnection, response AOLserver 4.0 Ns_ConnReturnStatus(3aolserver)
All times are GMT -4. The time now is 12:56 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy