Visit Our UNIX and Linux User Community


how to browse a web site using script


 
Thread Tools Search this Thread
Operating Systems AIX how to browse a web site using script
# 1  
Old 10-12-2007
how to browse a web site using script

hi guys,

ok this script, i am trying to browse a http site and store the output in a text file ...

when i run the script , i get a http compatibility error.

does any1 have a good solution to contact a http server and save the html in a text file.. i know the result comes with html tags.
Thank you
# 2  
Old 10-12-2007
you can try to use wget
# 3  
Old 10-12-2007
wget is the way to go but telnet would work for simple stuff.
# 4  
Old 10-12-2007
also lynx

you can also use lynx if you wanted.

Code:
lynx -source -accept_all_cookies "http://www.foo.com">>file.html

if you wanted...
# 5  
Old 10-15-2007
thank you guys,

but the issue is i am workin on aix machine and i dont have wget tools installed. and installin wget is not an option. i am pasting my script here .. plz have a look at it ..

Code:
WEBSERVER=aix85
WEBPORT=9088
URLDIR=/DmMethods/servlet/DoMethod
TOFILE=result.txt
echo "before telnet"
telnet $WEBSERVER $WEBPORT |&   # Open pipe to web server
echo "After telnet"
IFS=~
read -p HEAD1 2>/dev/null                    # Ignore telnet headers
read -p HEAD1 2>/dev/null
read -p HEAD1 2>/dev/null
print -p "GET $URLDIR HTTP /1.1" 2>/dev/null    # Send HTTP request to server
print -p "" 2>/dev/null
read -p LINE 2>/dev/null
echo ${LINE}
if [ "${LINE}" != "HTTP/1.0 200 OK" -a "${LINE}" != "HTTP/1.1 200 OK" ]; then
  echo "ERROR: Invalid response from Web Server: ${LINE}"
  exit 8
fi
while [ "${LINE}" != "" ]                    # Skip HTTP Headers
do
  read -p LINE 2>/dev/null
done
while read -p LINE 2>/dev/null               # Process web page contents
do
  echo "${LINE}" >> result1.txt
done


Previous Thread | Next Thread
Test Your Knowledge in Computers #848
Difficulty: Medium
The IEEE standardized Bluetooth as IEEE 802.15.1, but IEEE no longer maintains the Bluetooth standard.
True or False?

8 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to block first bash script until second bash script script launches web server/site?

I'm new to utilities like socat and netcat and I'm not clear if they will do what I need. I have a "compileDeployStartWebServer.sh" script and a "StartBrowser.sh" script that are started by emacs/elisp at the same time in two different processes. I'm using Cygwin bash on Windows 10. My... (3 Replies)
Discussion started by: siegfried
3 Replies

2. Web Development

Web Site Performance Issue

Hi All, I am currently managing a web site. I find that during the peak time period,ie,between 1300hrs to 2100hrs everyday,the response slows down whenever i access the applications hosted by the server.However when i try to access the static content in the web server,it loads quickly. The... (2 Replies)
Discussion started by: Hari_Ganesh
2 Replies

3. UNIX for Dummies Questions & Answers

Browse Web pages through command line

Is there any way to browse web pages while on the command line? I know wget can download pages, but I was wondering if there was an option other than that. (2 Replies)
Discussion started by: vroomlicious
2 Replies

4. Shell Programming and Scripting

I need help in creating my first web site?

I have always wanted to create my own site, I've only had one class about the basics in creating web pages but know i would like to get into more and more detailed info about doing so anyone with alot of experience please help me out.:b: (0 Replies)
Discussion started by: kprescod4158
0 Replies

5. AIX

intranet web site

hello I would like to create an internal web site, but how do i make it with Aix ? I must to install apache and send my http pages in the declared repertory ? There are some configuration files to modify ?? thank you (2 Replies)
Discussion started by: pascalbout
2 Replies

6. UNIX for Dummies Questions & Answers

Web Site Creation - testing .*pl etc....

Hiya All, How can I test my PERL Scripts whilst making my web site? I'm hoping there is some software out there that emulates a Web Server - without all the hassle of my building/setting up a Web Server from Stratch (Never done anything like that before - my next big project! 8) ) ... (7 Replies)
Discussion started by: marty 600
7 Replies

7. UNIX for Dummies Questions & Answers

Web site mirroring

I have a unix server running RedHat Linux 9.0. I want to mirror my site on a different server with the same configuration so if the main server goes down, the other server will take over. How is this best accomplished? (1 Reply)
Discussion started by: wvmlt
1 Replies

8. UNIX for Dummies Questions & Answers

Web site setup

Hi, I have a m/c loaded with redhat linux, static IP address and this m/c intern connected to the internet thro leaseline. Now i need to setup a company website has www.abc.xyz.com. What necessary configuration i need to do so that others can view our website. Thanks Bache Gowda (5 Replies)
Discussion started by: bache_gowda
5 Replies

Featured Tech Videos