Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Methods of reducing latency with downloading webpages?
Top Forums UNIX for Dummies Questions & Answers Methods of reducing latency with downloading webpages? Post 302506780 by seagaia on Tuesday 22nd of March 2011 12:12:34 AM
Old 03-22-2011
Methods of reducing latency with downloading webpages?
I've been wondering how I can reduce latency in downloading webpages. I've been pointed to network sockets (This guide in specific, beej's guide to network programming ) as some method to do so, but I can't figure out how to apply it in the way I need it.

I am iterating through webpages like this:
www . webpage . com / page-x

where x is a number, e.g. 1, 6, 11, 16.

Currently I'm using lynx --dump. I can get an okay speed by running the scripts in parallel and having them write to different files, but that's annoying and not really faster. Here's the steps of what occurs, roughly...


1. lynx connects to page, latency occurs
2. Page downloaded, parsed and results appended to a file
3. lynx stops
4. repeat until I stop it.

Thanks for your help.
 

8 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Threaded Discussions for Webpages

Dear All, I run a website for a non-profit. Does anyone know where I can get free or cheap software to run threaded discussions for our website? Our website is obviously running off a unix platform. Thanks (4 Replies)
Discussion started by: evertk
4 Replies

2. Shell Programming and Scripting

AWK script to detect webpages from file

Hi guys I'm very new to unix and I have to create an awk script that detects webpage addresses from a file/webpage and outputs how many times each webpage was detected.e.g. if my file was: (Note: The symbol " was added to stop them being created into links) "www.google.com"... (1 Reply)
Discussion started by: ROFL
1 Replies

3. Shell Programming and Scripting

AWK script to detect webpages from file

Hi guys I'm very new to unix and I have to create an awk script that detects webpage addresses from a file/webpage and outputs how many times each webpage was detected.e.g. if my file was: www.google.com www.facebook.com www.google.com the output should be: www.google.com x2... (2 Replies)
Discussion started by: ROFL
2 Replies

4. UNIX for Dummies Questions & Answers

Hosting webpages(jspx) on Linux

Hi guys, I have developed a small website using jspx pages. Now i want to host the website to a server which is a linux machine. The linux machine which i m referring to is already hosting some web pages(jspx) through a web server Which is running at port 8888. My first question would be... (0 Replies)
Discussion started by: pinga123
0 Replies

5. Hardware

Use of SSD for serving webpages

I have seen research articles and forum postings that demonstrate that SSDs are poor at reading large files: the larger the file, the slower the SSD compared to traditional hard disk drives. The difference with hard disk drives becomes apparent at medium size files, say 20KB. Does this mean that... (2 Replies)
Discussion started by: figaro
2 Replies

6. Shell Programming and Scripting

Scripts to Interact with Webpages

Hi, Is it possible to have a script to interact with webpages. I want to create a script that logs a user into a specific site, and is able to get/post information. Would anyone give me instructions on how it's should be done, and where I can find information on starting it out. I know... (2 Replies)
Discussion started by: Pztar
2 Replies

7. Web Development

Get command-output to webpage from solaris-10-OS invoked from html webpages

Hello, I am a middleware administrator and as an admin I need to monitor a number of middleware instances (weblogic servers). But since it is a not the forum for that i would like to put my problem in simple terms and need guidance on that. I need to use apache webserver on Solaris 10 zones to... (3 Replies)
Discussion started by: poga
3 Replies

8. AIX

Latency Test

Hi every one, we have a set up in solaris 8 and 9 and running many cshell scripts.. we are migrate to AIX . Now, i want to know the latency difference between two boxes(Solaris and AIX). Kindly help me to , how to do Latency test.. (0 Replies)
Discussion started by: Madhu Siddula
0 Replies

LEARN ABOUT PHP

hman

hman(1) 						      General Commands Manual							   hman(1)

NAME

       hman - browse the on-line manual pages

SYNOPSIS

       hman [ -P browser ] [ -H host ] [ section ] name
       hman [ -P browser ] [ -H host ] [ section ] [ index ]

DESCRIPTION

       The  hman  script is an interface to man2html(1) that allows you to enter man page requests at the command line and view the output in your
       favourite browser.  The behaviour reminds of that of man(1) so that many people will be able to alias hman to man.  If the browser used	is
       netscape, and an incarnation of netscape is running already, hman will pass the request to the existing browser.

OPTIONS

       -P  browser
	      Specify  which  browser  (like  lynx, xmosaic, arena, chimera, netscape, amaya, ...) to use.  This option overrides the MANHTMLPAGER
	      environment variable.  The default is the non-httpd version of lynx, or sensible-browser if lynx cannot be found.

       -H  host
	      Specify from what host to get the man pages.  This option overrides the MANHTMLHOST environment variable.  The default is localhost.

ENVIRONMENT

       MANHTMLPAGER
	      The default browser to use is selected using this environment variable.

       MANHTMLHOST
	      The default host to use is selected using this environment variable.

SEE ALSO

       man(1), man2html(1), arena(1), lynx(1), sensible-browser(1), netscape(1), xmosaic(1), glimpse(1)

       http://www.mcom.com/newsref/std/x-remote.html

								  19 January 1998							   hman(1)
All times are GMT -4. The time now is 01:48 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy