Sponsored Content
Full Discussion: Reading website from UNIX
Top Forums Shell Programming and Scripting Reading website from UNIX Post 302984650 by blastit.fr on Friday 28th of October 2016 03:58:47 PM
Old 10-28-2016
You should use either curl or wget .
If you're not allowed to install such software, you should try perl with LWP.
See: A Simple way to download many web pages using Perl

Last edited by blastit.fr; 10-28-2016 at 04:59 PM.. Reason: typo
This User Gave Thanks to blastit.fr For This Post:
 

9 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Unix on my website...

Hello eveybody! I'm 15mins old (in Unix years that is!) I hope this isn't a stupid question.... Is it possible to have a unix forum on my site? I've seen other sites with forums similar to the unix one. How do I got about doing this? Thankyou. Happy12:D (4 Replies)
Discussion started by: happy12
4 Replies

2. UNIX for Dummies Questions & Answers

Unix recommended reading

Hello I was wondering if anyone had a Unix book recommendation. Just one book, that can give me a good foundation in Unix and has plenty of hands on exercises to follow along with on my Unix terminal? Thank you (1 Reply)
Discussion started by: vedder191
1 Replies

3. Shell Programming and Scripting

file reading in unix

Hi , I need to read line by line from a file. Can u please tell me the most efficient way to do it. SInce i will be reading more than 1000 files in my program. i am trying the following options : 1. nawk option nawk'{ print $0 }'files.temp 2. filecount=`cat files.temp | wc -l`... (2 Replies)
Discussion started by: pradeepthanraj
2 Replies

4. Shell Programming and Scripting

Unix Practice Website

Hi , If somebody knows, Please let me know where can I practice Unix commands and Shell scripts Online. I know www.cyberspace.org. But its very slow. Is there some website which is very fast. Even I have heard there are some websites where you pay just 2-3 dollors a month and connection is... (1 Reply)
Discussion started by: satgur
1 Replies

5. Shell Programming and Scripting

UNIX file reading

i have a directory that has n files starting with ABC_00x.txt and ABC_00x.txt where x is the integer number. i want to read the contents of each file and write it into a log file. can somebody help me in writing the shell script in unix (3 Replies)
Discussion started by: vkca
3 Replies

6. Shell Programming and Scripting

Unix Script to read the XML file from Website

Hi Experts, I need a unix shell script which can copy the xml file from the below pasted website and paste in in my unix directory. http://www.westpac.co.nz/olcontent/olcontent.nsf/fx.xml Thanks in Advance... (8 Replies)
Discussion started by: phani333
8 Replies

7. Shell Programming and Scripting

feasibility of opening a website link from unix and get a response in the form of xml or html

i just wanted to know whether is it possible to open a website link and get a response in the form of xml or html format... the website is of local network... for example something like this wget http://blahblah.samplesite.com/blachblahcblach/User/jsp/ShowPerson.jsp?empid=123456 ... (2 Replies)
Discussion started by: vivek d r
2 Replies

8. Shell Programming and Scripting

Reading value from sqlplus in UNIX

I have a unix shell script that calls two Select Queries using sqlplus as shown below. more extract.sh result=$(sqlplus -s usr/swd@myhost_sid @/tmp/SELECT_QUERY.run) more @/tmp/SELECT_QUERY.run SELECT PROBLEM_TYPE, COUNT (*) FROM BATCH WHERE UPPER (STATUS) = 'NEW' AND PROB_TYPE... (1 Reply)
Discussion started by: mohtashims
1 Replies

9. What is on Your Mind?

Best Linux/UNIX-based website hosting service?

I'm currently on a shared server, with a fairly-well-known hosting provider, which has gotten progressively worse over the years, and it's time to switch to someone else. I know there are LOTS of providers out there whose infrastructure is Linux/UNIX based (actually, I'm not aware of any that... (2 Replies)
Discussion started by: markolinux
2 Replies
LWP::Simple(3)						User Contributed Perl Documentation					    LWP::Simple(3)

NAME
LWP::Simple - simple procedural interface to LWP SYNOPSIS
perl -MLWP::Simple -e 'getprint "http://www.sn.no"' use LWP::Simple; $content = get("http://www.sn.no/"); die "Couldn't get it!" unless defined $content; if (mirror("http://www.sn.no/", "foo") == RC_NOT_MODIFIED) { ... } if (is_success(getprint("http://www.sn.no/"))) { ... } DESCRIPTION
This module is meant for people who want a simplified view of the libwww-perl library. It should also be suitable for one-liners. If you need more control or access to the header fields in the requests sent and responses received, then you should use the full object-oriented interface provided by the "LWP::UserAgent" module. The following functions are provided (and exported) by this module: get($url) The get() function will fetch the document identified by the given URL and return it. It returns "undef" if it fails. The $url argument can be either a string or a reference to a URI object. You will not be able to examine the response code or response headers (like 'Content-Type') when you are accessing the web using this function. If you need that information you should use the full OO interface (see LWP::UserAgent). head($url) Get document headers. Returns the following 5 values if successful: ($content_type, $document_length, $modified_time, $expires, $server) Returns an empty list if it fails. In scalar context returns TRUE if successful. getprint($url) Get and print a document identified by a URL. The document is printed to the selected default filehandle for output (normally STDOUT) as data is received from the network. If the request fails, then the status code and message are printed on STDERR. The return value is the HTTP response code. getstore($url, $file) Gets a document identified by a URL and stores it in the file. The return value is the HTTP response code. mirror($url, $file) Get and store a document identified by a URL, using If-modified-since, and checking the Content-Length. Returns the HTTP response code. This module also exports the HTTP::Status constants and procedures. You can use them when you check the response code from getprint(), getstore() or mirror(). The constants are: RC_CONTINUE RC_SWITCHING_PROTOCOLS RC_OK RC_CREATED RC_ACCEPTED RC_NON_AUTHORITATIVE_INFORMATION RC_NO_CONTENT RC_RESET_CONTENT RC_PARTIAL_CONTENT RC_MULTIPLE_CHOICES RC_MOVED_PERMANENTLY RC_MOVED_TEMPORARILY RC_SEE_OTHER RC_NOT_MODIFIED RC_USE_PROXY RC_BAD_REQUEST RC_UNAUTHORIZED RC_PAYMENT_REQUIRED RC_FORBIDDEN RC_NOT_FOUND RC_METHOD_NOT_ALLOWED RC_NOT_ACCEPTABLE RC_PROXY_AUTHENTICATION_REQUIRED RC_REQUEST_TIMEOUT RC_CONFLICT RC_GONE RC_LENGTH_REQUIRED RC_PRECONDITION_FAILED RC_REQUEST_ENTITY_TOO_LARGE RC_REQUEST_URI_TOO_LARGE RC_UNSUPPORTED_MEDIA_TYPE RC_INTERNAL_SERVER_ERROR RC_NOT_IMPLEMENTED RC_BAD_GATEWAY RC_SERVICE_UNAVAILABLE RC_GATEWAY_TIMEOUT RC_HTTP_VERSION_NOT_SUPPORTED The HTTP::Status classification functions are: is_success($rc) True if response code indicated a successful request. is_error($rc) True if response code indicated that an error occurred. The module will also export the LWP::UserAgent object as $ua if you ask for it explicitly. The user agent created by this module will identify itself as "LWP::Simple/#.##" and will initialize its proxy defaults from the environment (by calling $ua->env_proxy). CAVEAT
Note that if you are using both LWP::Simple and the very popular CGI.pm module, you may be importing a "head" function from each module, producing a warning like "Prototype mismatch: sub main::head ($) vs none". Get around this problem by just not importing LWP::Simple's "head" function, like so: use LWP::Simple qw(!head); use CGI qw(:standard); # then only CGI.pm defines a head() Then if you do need LWP::Simple's "head" function, you can just call it as "LWP::Simple::head($url)". SEE ALSO
LWP, lwpcook, LWP::UserAgent, HTTP::Status, lwp-request, lwp-mirror perl v5.18.2 2012-02-18 LWP::Simple(3)
All times are GMT -4. The time now is 04:25 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy