Read URL data from UNIX-CLI without Wget,CURL,w3m,LWP


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Read URL data from UNIX-CLI without Wget,CURL,w3m,LWP
# 1  
Old 10-07-2013
Read URL data from UNIX-CLI without Wget,CURL,w3m,LWP

Hi Experts,
Problem statement :
We have an URL for which we need to read the data and get parsed inside the shell scripts.My Aix has very limited perl utility, i cant install any utility as well.
Precisely, wget,cURL,Lynx,w3m and Lwp cant be used as i got these utilities only when i googled it.

And also i found one solution in Unix.com itself in below thread

URL call from SHELL script

Since i dont have enough posts am not able to post the complete link sorry for that
but i have given the later part of the title that is , thread name.
and in that cfajohnson has mentioned about some bash scripting and of that only one link is accessible and other one is not accessible( 1st link).
Please dont think am dusting the old one's am really in need of that script. Please help me .
Even if we have any solution for this, would be greatly appreciated.

Any help would be really appreciated.

Moderator's Comments:
Mod Comment edit by bakunin: link to thread inserted for you.

Last edited by bakunin; 10-07-2013 at 11:56 AM..
# 2  
Old 10-07-2013
telnet

However, pretty much just http:, not https: there. How you screen scrape that could be somewhat hard without something like "expect" (though IBM also has a product for this as well).

Without add-ons, AIX (base install) is pretty sad.
# 3  
Old 10-07-2013
Honestly: while i admire ghostdog74 for his ingenuity i wouldn't use his solution in any serious task i have to carry out.

If you need a http-based communication you need a tool to do so. Either you install that yourself, get the system administrator to install it for you or declare your task to be not possible without the necessary tools. It is probably possible to do brain surgery with only a needle and some yarn - i still would recommend not to even try, especially if you have any interest in the subjects further life.

Get wget, cURL or any other CLI-based browser installed.

I hope this helps.

bakunin
# 4  
Old 10-07-2013
@ cjcox,

Thanks for your reply . Yes it is.
Can Lwp- Mechanise be installed on our own without Administration privilege.

I have all the source code copied to the machine and i tried to install with the steps available in CPAN.org. but its failing for pre-requiste.

If you guys could help me in installing the mechanize w/o root privillege, that would be great.

---------- Post updated at 09:59 AM ---------- Previous update was at 09:57 AM ----------

@ bakunin ,

Thanks for your valuable reply.

I got to know that after 2 days of struggle, so i have the perl installed just want to add few more utility to make it work. Please try to guide me in getting Mechanize installed.
# 5  
Old 10-07-2013
Quote:
Originally Posted by scott_cog
Can Lwp- Mechanise be installed on our own without Administration privilege.
Quite a few things can be installed without administration privileges. All it takes is a willingness to install in ~/bin/ instead of /bin/ and use custom paths.
# 6  
Old 10-07-2013
Quote:
Originally Posted by Corona688
Quite a few things can be installed without administration privileges. All it takes is a willingness to install in ~/bin/ instead of /bin/ and use custom paths.

Am ready to do custom installation as well, please help me in performing it.
# 7  
Old 10-07-2013
Quote:
Originally Posted by scott_cog
I got to know that after 2 days of struggle, so i have the perl installed just want to add few more utility to make it work. Please try to guide me in getting Mechanize installed.
To be honest: i never even heard of "mechanize". Why doN't you use wget, which can do all you seem to want to do right out of the box?

If your SysAdmin is just too lazy to install it: report him to your managers for being an obstacle and have him fired (yes, i really mean that). wget can be downloaded either from the IBM site:

IBM AIX toolbox for Linux applications

or Michael Perzls site, all packaged into installable rpm-packages. If it is even good enough for IBM itself it might eventually be good enough for your SysAdmin.

I hope this helps.

bakunin
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Web Development

Wget/curl and javascript

What can I use instead of wget/curl when I need to log into websites that use javascript? Wget and curl don't handle javascript. (6 Replies)
Discussion started by: locoroco
6 Replies

2. Shell Programming and Scripting

Wget and curl to post data

i'm using this command to post data to a remote host: wget --post-data="My Data" http://<my-ip>:80 -O /dev/null -q and curl --data "My Data" http://<my-ip>:80 however, when i run the above, i see the following in my access log on the remote host: Wget: 10.10.10.10 - - "POST /... (1 Reply)
Discussion started by: SkySmart
1 Replies

3. Shell Programming and Scripting

Need help in wget or curl command in UNIX shell

Hi, I need help in wget or curl command to invoke a REST client services. This is the format i need to send request wget -H "tokenId: F6" -H "Authorization: Basic <64 bit encoded username:password>" -H "https://example.com" -H "Accept: application/Json" -o download.xml... (1 Reply)
Discussion started by: zen01234
1 Replies

4. UNIX for Dummies Questions & Answers

Read URL data from UNIX without wget,curl,lynx,w3m.

Hi Experts, Problem statement : We have an URL for which we need to read the data and get parsed inside the shell scripts. My Aix has very limited perl utility, i cant install any utility as well. Precisely, wget,cURL,Lynx,w3m and Lwp cant be used as i got these utilities only when i googled... (0 Replies)
Discussion started by: scott_cog
0 Replies

5. UNIX for Dummies Questions & Answers

Launch a URL,validate username and password using wget or curl

Hi All, I want to launch "ex: http://gmail.com" from the cmd window and validate the credentials with username and password, is it possible? I have found something like this "wget --http-user=USER' --http-password=PASSWORD http://gmail.com" am new to this and unable to find a solution, i... (0 Replies)
Discussion started by: harsha85
0 Replies

6. Shell Programming and Scripting

Specifying IP address with curl/wget

Hello, I am wondering does anyone know of a method using curl/wget or other where by I could specify the IP address of the server I wish to query for a website. Something similar to editing /etc/hosts but that can be done directly from the command line. I have looked through the man pages... (4 Replies)
Discussion started by: colinireland
4 Replies

7. Shell Programming and Scripting

Passing variables to cli curl

I have a script that uses cli curl, and it would be easier if it was possible to pass variables like this: curl -c $cookie -d $data www.********.comThis doesn't work, instead I have to generate a string and use eval. Is there another, easier way to pass variables to cli curl? (1 Reply)
Discussion started by: locoroco
1 Replies

8. Shell Programming and Scripting

proxy server with wget or cli curl

I'm using a proxy service with an ip address and a port number. How do I use the proxy with wget or cli curl? (1 Reply)
Discussion started by: locoroco
1 Replies

9. Shell Programming and Scripting

Proxy with curl/wget support

I need a proxy that would enable me to use cli curl/wget with another ip address. How do I find a paid proxy server that supports curl/wget? (1 Reply)
Discussion started by: locoroco
1 Replies

10. Shell Programming and Scripting

wget to check an URL

I all, I wrote an script which starts a Weblogic server and waits until its loaded to deploy several apps. The way I checked was something like: while ; do wget --spider <URL>:<port>/console > /dev/null 2>&1 rc=$? done This works perfectly because it's an HTML site and when server is... (2 Replies)
Discussion started by: AlbertGM
2 Replies
Login or Register to Ask a Question