Read URL data from UNIX-CLI without Wget,CURL,w3m,LWP


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Read URL data from UNIX-CLI without Wget,CURL,w3m,LWP
# 8  
Old 10-07-2013
Hi Bakunin ,

I cant use wget because , am not sure if it can be installed as an stand alone utility.

And mechanize is an module of perl-lwp , which has many more options which can be used by me for later part of the problem.

Please guide me in helping LWP installed w/o admin privillege.
# 9  
Old 10-07-2013
Quote:
Originally Posted by scott_cog
Hi Bakunin ,

I cant use wget because , am not sure if it can be installed as an stand alone utility.
wget is a stand-alone utility, and has always been a stand-alone utility, designed for this exact sort of thing.

Quote:
Please guide me in helping LWP installed w/o admin privillege.
Step 1: Get admin privilege.

If you can't, why not?
# 10  
Old 10-08-2013
Hi Corona688,

Am not sure whats the limitation i found with wget, but there is some limitation thats why i didn't kept as a option.
Fyi, weburl which am trying to access wil need username/password to login and its an HTTPS link.

Regards Admin privilege , am working for an professional organization and i have only user access , root access is out of the question, And we cant request to install as well.

With the resources which we have , i have to figure out a solution.

so the obvious choice for me to work now is Perl only- with lwp and mechanize. Hence am asking for some help in this.
# 11  
Old 10-08-2013
Quote:
Originally Posted by scott_cog
Regards Admin privilege , am working for an professional organization and i have only user access , root access is out of the question, And we cant request to install as well.
Then you are out of luck. I hope you have access to a compiler, as you may resort to building things inside home directories and creating far bigger messes than if they'd just allowed you to do your jobs in the first place.
# 12  
Old 10-09-2013
Hi Corona688,

Thanks for your reply. Perhaps, am out of luck.

After lot of searches , i found a solution for those problems, am in progress of implementing it. Am sharing the same link over here, Am sure , that it will be beneficiary for many people.

HowToInstallCpanModules < TWiki < TWiki

Installing mod_perl without superuser privileges - Perl.com

Unix.com is an great forum for solutions pertaining real world problems.
# 13  
Old 10-09-2013
You CAN pass userinfo and password using wget and/or curl.
Both are standalone utilites, specialy compared to any python or perl solution.

From their user: icyrock

Code:
# Log in to the server.  This can be done only once.
  wget --save-cookies cookies.txt \
     --post-data 'user=foo&password=bar' \
     http://server.com/auth.php
# Now grab the page or pages we care about.
  wget --load-cookies cookies.txt \
     -p http://server.com/interesting/article.php

This way you dont (should not) have to install anything.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Web Development

Wget/curl and javascript

What can I use instead of wget/curl when I need to log into websites that use javascript? Wget and curl don't handle javascript. (6 Replies)
Discussion started by: locoroco
6 Replies

2. Shell Programming and Scripting

Wget and curl to post data

i'm using this command to post data to a remote host: wget --post-data="My Data" http://<my-ip>:80 -O /dev/null -q and curl --data "My Data" http://<my-ip>:80 however, when i run the above, i see the following in my access log on the remote host: Wget: 10.10.10.10 - - "POST /... (1 Reply)
Discussion started by: SkySmart
1 Replies

3. Shell Programming and Scripting

Need help in wget or curl command in UNIX shell

Hi, I need help in wget or curl command to invoke a REST client services. This is the format i need to send request wget -H "tokenId: F6" -H "Authorization: Basic <64 bit encoded username:password>" -H "https://example.com" -H "Accept: application/Json" -o download.xml... (1 Reply)
Discussion started by: zen01234
1 Replies

4. UNIX for Dummies Questions & Answers

Read URL data from UNIX without wget,curl,lynx,w3m.

Hi Experts, Problem statement : We have an URL for which we need to read the data and get parsed inside the shell scripts. My Aix has very limited perl utility, i cant install any utility as well. Precisely, wget,cURL,Lynx,w3m and Lwp cant be used as i got these utilities only when i googled... (0 Replies)
Discussion started by: scott_cog
0 Replies

5. UNIX for Dummies Questions & Answers

Launch a URL,validate username and password using wget or curl

Hi All, I want to launch "ex: http://gmail.com" from the cmd window and validate the credentials with username and password, is it possible? I have found something like this "wget --http-user=USER' --http-password=PASSWORD http://gmail.com" am new to this and unable to find a solution, i... (0 Replies)
Discussion started by: harsha85
0 Replies

6. Shell Programming and Scripting

Specifying IP address with curl/wget

Hello, I am wondering does anyone know of a method using curl/wget or other where by I could specify the IP address of the server I wish to query for a website. Something similar to editing /etc/hosts but that can be done directly from the command line. I have looked through the man pages... (4 Replies)
Discussion started by: colinireland
4 Replies

7. Shell Programming and Scripting

Passing variables to cli curl

I have a script that uses cli curl, and it would be easier if it was possible to pass variables like this: curl -c $cookie -d $data www.********.comThis doesn't work, instead I have to generate a string and use eval. Is there another, easier way to pass variables to cli curl? (1 Reply)
Discussion started by: locoroco
1 Replies

8. Shell Programming and Scripting

proxy server with wget or cli curl

I'm using a proxy service with an ip address and a port number. How do I use the proxy with wget or cli curl? (1 Reply)
Discussion started by: locoroco
1 Replies

9. Shell Programming and Scripting

Proxy with curl/wget support

I need a proxy that would enable me to use cli curl/wget with another ip address. How do I find a paid proxy server that supports curl/wget? (1 Reply)
Discussion started by: locoroco
1 Replies

10. Shell Programming and Scripting

wget to check an URL

I all, I wrote an script which starts a Weblogic server and waits until its loaded to deploy several apps. The way I checked was something like: while ; do wget --spider <URL>:<port>/console > /dev/null 2>&1 rc=$? done This works perfectly because it's an HTML site and when server is... (2 Replies)
Discussion started by: AlbertGM
2 Replies
Login or Register to Ask a Question