Sponsored Content
Top Forums Shell Programming and Scripting Reconnect and download data from online site Post 302313565 by vanitham on Wednesday 6th of May 2009 04:15:06 AM
Old 05-06-2009
Reconnect and download data from online site

Hi,

I have to connect to online site and download some data from the online site but sometimes the site which i have to connect will be busy
or it will not be working in such a case i have to connect atleast 3 times and try to download the data.

I tried like this:

Code:
use CGI::Carp qw(fatalsToBrowser);

$url="http://www.xyz.com";

$ua = new LWP::UserAgent;

$tries=0;

eval
{
$response=$ua->get($url);
};warn $@ if ($@);
if ($@) {
         die "Server error: $@.  Try again later" ;
         $tries++;
	 if($tries == "3")
	 {

		print "\nstop connecting\n"
	 } 
     }

Actually i am stuck up here if there is any error it is not displaying i.e its not
entering if($@) block . How to reconnect again???

Basically i have to reconnect and download the data when the site is not working.

How to reconnect to the site and download the data???


Any suggestions ???


Regards
Vanitha

Last edited by Franklin52; 05-06-2009 at 05:34 AM.. Reason: code tags
 

8 More Discussions You Might Find Interesting

1. Programming

Online site on Makefile

Hi, I would like to learn about Makefile. Can anybody suggest me any online site from where I can get detail information? Thanks. (2 Replies)
Discussion started by: bkausik
2 Replies

2. Programming

How to reconnect to online database?

HI, I am using perl LWP for connecting to online databases. some times i will get connection fail. I want to keep track of that error and again i want to connect. i.e when i get connection fail the program should not stop it should try to connect. I can check connection fail has... (1 Reply)
Discussion started by: vanitham
1 Replies

3. Shell Programming and Scripting

script for download files from ftp site

I'm new to scripting. I'm trying to write a script to download files from ftp site, the following is the script and the message i get after running the script. no files were downloaded :( Thanks advance! script: #!/usr/bin/ksh DAY=`date --date="-1 days" +%y%m%d` ftp -v -n "ftp.address" <<... (5 Replies)
Discussion started by: tiff-matt
5 Replies

4. Linux

shell script to download files from a site?

Hey everyone, my wife has purchased a bundle package of a bunch of images from a site, and now has to download each one of them manually. There are about 500 downloads, and it's quite a hassle to browse to each page and download them all individually. I would like to write a shell script to... (2 Replies)
Discussion started by: paqman
2 Replies

5. UNIX for Dummies Questions & Answers

Download image every 24hrs + put it in php site

Hi everyone, I am all new to this and I might not know what I am asking about :) I have access to a cPanel with cron tab scheduler. I know nothing re scripting creating php etc.. I own a ftp mainly for own email address, some storage, and a forum (ready-made one :P ) I have some knowledge... (5 Replies)
Discussion started by: zYx
5 Replies

6. Shell Programming and Scripting

Wget, download file from site's folder.

Ok, this is quite weird. wget -r mysite.com/mylink/ should get all the files recursively from the 'mylink' folder. The problem is that wget saves an index.html file! When I open this index.html with my browser I realize that it shows all the files in the current folder (plus an option to move... (3 Replies)
Discussion started by: hakermania
3 Replies

7. Shell Programming and Scripting

lftp script to connect to external sftp site and download to internal ftp and then send email

Hi there, I'm new to shell scripting and need some help if possible? I need to create a shell script (.sh) to run as a cron job on an ubuntu linux server to connect to an external sftp sites directory using credentials (which I have) and then download to our internal ftp server and then copy... (3 Replies)
Discussion started by: ghath
3 Replies

8. Shell Programming and Scripting

Download a site

A buddy of mine was telling me last night that you can write a bash script that will download an entire site in gedit?? Is this true??? I think I am going to fall in love with bash :D Any good tutorials?? (15 Replies)
Discussion started by: graphicsman
15 Replies
LWP::Authen::Wsse(3pm)					User Contributed Perl Documentation				    LWP::Authen::Wsse(3pm)

NAME
LWP::Authen::Wsse - Library for enabling X-WSSE authentication in LWP VERSION
This document describes version 0.05 of LWP::Authen::Wsse, released December 27, 2005. SYNOPSIS
use LWP::UserAgent; use HTTP::Request::Common; my $url = 'http://www.example.org/protected_page.html'; # Set up the WSSE client my $ua = LWP::UserAgent->new; $ua->credentials('example.org', '', 'username', 'password'); $request = GET $url; print "--Performing request now...----------- "; $response = $ua->request($request); print "--Done with request------------------- "; if ($response->is_success) { print "It worked!->", $response->code, " "; } else { print "It didn't work!->", $response->code, " "; } DESCRIPTION
"LWP::Authen::Wsse" allows LWP to authenticate against servers that are using the "X-WSSE" authentication scheme, as required by the Atom Authentication API. The module is used indirectly through LWP, rather than including it directly in your code. The LWP system will invoke the WSSE authentication when it encounters the authentication scheme while attempting to retrieve a URL from a server. You also need to set the credentials on the UserAgent object like this: $ua->credentials('www.company.com:80', '', "username", "password"); Alternatively, you may also subclass LWP::UserAgent and override the "get_basic_credentials()" method. See LWP::UserAgent for more details. SEE ALSO
LWP, LWP::UserAgent, lwpcook. AUTHORS
Audrey Tang <audrey@audrey.org> COPYRIGHT
Copyright 2004, 2005 by Audrey Tang <audrey@audrey.org>. This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself. See <http://www.perl.com/perl/misc/Artistic.html> perl v5.14.2 2012-11-27 LWP::Authen::Wsse(3pm)
All times are GMT -4. The time now is 12:56 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy