Methods of reducing latency with downloading webpages?

Login or Register to Ask a Question and Join Our Community

Methods of reducing latency with downloading webpages?

Tags

Top Forums UNIX for Dummies Questions & Answers Methods of reducing latency with downloading webpages?

03-22-2011

Registered User

3, 0

Join Date: Jan 2011

Last Activity: 25 April 2011, 3:59 PM EDT

Posts: 3

Thanks Given: 1

Thanked 0 Times in 0 Posts

Methods of reducing latency with downloading webpages?

I've been wondering how I can reduce latency in downloading webpages. I've been pointed to network sockets (This guide in specific, beej's guide to network programming ) as some method to do so, but I can't figure out how to apply it in the way I need it.

I am iterating through webpages like this:
www . webpage . com / page-x

where x is a number, e.g. 1, 6, 11, 16.

Currently I'm using lynx --dump. I can get an okay speed by running the scripts in parallel and having them write to different files, but that's annoying and not really faster. Here's the steps of what occurs, roughly...

1. lynx connects to page, latency occurs
2. Page downloaded, parsed and results appended to a file
3. lynx stops
4. repeat until I stop it.

Thanks for your help.

seagaia

View Public Profile for seagaia

Find all posts by seagaia

Login or Register to Ask a Question

Previous Thread | Next Thread

8 More Discussions You Might Find Interesting

1. AIX

Latency Test

Hi every one, we have a set up in solaris 8 and 9 and running many cshell scripts.. we are migrate to AIX . Now, i want to know the latency difference between two boxes(Solaris and AIX). Kindly help me to , how to do Latency test..

2. Web Development

Get command-output to webpage from solaris-10-OS invoked from html webpages

Hello, I am a middleware administrator and as an admin I need to monitor a number of middleware instances (weblogic servers). But since it is a not the forum for that i would like to put my problem in simple terms and need guidance on that. I need to use apache webserver on Solaris 10 zones to...

3. Shell Programming and Scripting

Scripts to Interact with Webpages

Hi, Is it possible to have a script to interact with webpages. I want to create a script that logs a user into a specific site, and is able to get/post information. Would anyone give me instructions on how it's should be done, and where I can find information on starting it out. I know...

4. Hardware

Use of SSD for serving webpages

I have seen research articles and forum postings that demonstrate that SSDs are poor at reading large files: the larger the file, the slower the SSD compared to traditional hard disk drives. The difference with hard disk drives becomes apparent at medium size files, say 20KB. Does this mean that...

5. UNIX for Dummies Questions & Answers

Hosting webpages(jspx) on Linux

Hi guys, I have developed a small website using jspx pages. Now i want to host the website to a server which is a linux machine. The linux machine which i m referring to is already hosting some web pages(jspx) through a web server Which is running at port 8888. My first question would be...

6. Shell Programming and Scripting

AWK script to detect webpages from file

Hi guys I'm very new to unix and I have to create an awk script that detects webpage addresses from a file/webpage and outputs how many times each webpage was detected.e.g. if my file was: www.google.com www.facebook.com www.google.com the output should be: www.google.com x2...

7. Shell Programming and Scripting

AWK script to detect webpages from file

Hi guys I'm very new to unix and I have to create an awk script that detects webpage addresses from a file/webpage and outputs how many times each webpage was detected.e.g. if my file was: (Note: The symbol " was added to stop them being created into links) "www.google.com"...

8. UNIX for Dummies Questions & Answers

Threaded Discussions for Webpages

Dear All, I run a website for a non-profit. Does anyone know where I can get free or cheap software to run threaded discussions for our website? Our website is obviously running off a unix platform. Thanks

Login or Register to Ask a Question

LEARN ABOUT MOJAVE

update-rcconf-guide

UPDATE-RCCONF-GUIDE(8)                                           Debian GNU/Linux                                           UPDATE-RCCONF-GUIDE(8)

NAME

       update-rcconf-guide - Create default guide file for rcconf

SYNOPSIS

       update-rcconf-guide

DESCRIPTION

       Update-rcconf-guide creates the default guide file which rcconf uses.

       Update-rcconf-guide searches the package names corresponding to each service file in /etc/init.d directory from dpkg info
       files(/var/lib/dpkg/info/*.list) and get the description of these packages using apt-cache. Update-rcconf-guide uses Short-Description in
       priority to the description if service files has Short-Description field.  Update-rcconf-guide writes those results to
       /var/lib/rcconf/guide.default file.

       You can write your own guide in user guide file(/var/lib/rcconf/guide) by hand.  Rcconf refers Guides in /var/lib/rcconf/guide before those
       in /var/lib/rcconf/guide.default.

       If you install some packages after executed update-rcconf-guide, you need to re-create this file using update-rcconf-guide so as to refresh
       guide.default that includes new guides for installed new services.

       Notice: update-rcconf-guide was not executed when you installed rcconf
               package.

FILE

       /var/lib/rcconf/guide.default
               Guide File update-rcconf-guide generates.

       /var/lib/rcconf/guide
               Guide File user(Administrator) can define.

SEE ALSO

       rcconf(8)

AUTHOR

       Atsushi KAMOSHIDA <kamop@debian.org>

perl v5.10.1                                                        2009-01-21                                              UPDATE-RCCONF-GUIDE(8)