Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Read webpage from shell script

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Read webpage from shell script
# 1  
Old 03-15-2011
Read webpage from shell script
Hey experts,

I am trying to read a webpage and want to search for a text patter (case insensitive). Please help me acquire this.

If the text is found i will receive a mail which can be hardcoded in the script.

I am not a big fan of using PERL script.

Please help.

Machine: AIX

Thanks a lot in advance.
# 2  
Old 03-15-2011
You could try:
Code:
found="`egrep -i regex_pattern index.html`"
[[ "_$found" != _ ]] && echo "Found it: $found" | mail -s subject me@myemail.dom

But first you want to check that you can send mail from command line:
Code:
echo blah | mail -s testSubject my@email

# 3  
Old 03-15-2011
Code:
curl https://www.unix.com | awk '/Copyright/ {print "found the pattern!"}'

search pattern "Copyright" in unix.com homepage. once it was found, print one "found the pattern"
# 4  
Old 03-15-2011
Hi Mirni,

Thanks for you quick response. I have check and i can send mails through my command line. My issue is i have and entire link starting example The UNIX and Linux Forums - Learn UNIX and Linux from Experts some thing like this. I tried to replace ur code on index.html with my url and it failed with an error egrep: can't opn the link specified.

Please advise.

Thanks in adavance.
# 5  
Old 03-15-2011
Look at the post above. Use curl:
Code:
curl https://www.unix.com | egrep "pattern"

Alternatively you can use 'wget' to download the html file and then search... But curl sounds better/
# 6  
Old 03-15-2011
Quote:
Originally Posted by sk1418
Code:
curl https://www.unix.com | awk '/Copyright/ {print "found the pattern!"}'

search pattern "Copyright" in unix.com homepage. once it was found, print one "found the pattern"

Hi sk1418,

I tried the command you provided but unfortunately, i dont think its installed on my machine.
Below is the error i received.

Code:
 
$ curl https://www.unix.com | awk '/Copyright/ {print "found the pattern!"}'
ksh: curl:  not found

Please advice.
# 7  
Old 03-15-2011
do you have wget installed? try this
Code:
 wget -O - https://www.unix.com| [awk or grep]


--- update ---
well since you only want to know if the webpage contains the pattern, you could:

Code:
 wget -O - https://www.unix.com | grep -c "Copyright"

if the returned number > 0. means pattern was found.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Executing Shell Script from PHP Webpage

Hello , I am trying to execute a small shell script from a PHP Webpage. Below is the shell script : 1.sh $ cat /home/oracle/rahul/1.sh #!/bin/ksh echo "`date` : Report generation in progress" > /home/oracle/rahul/report.txt echo "Run successfully script"; Below is the PHP... (1 Reply)
Discussion started by: rahul2662
1 Replies

2. Shell Programming and Scripting

Launch shell script with parameters from a webpage button

I want to create a simple html page that should contain 2 fields in which the user can write the input. Then I want to have a button that should launch a shell script with the parameters inserted by user in the fields from the webpage. This is the code behind my webpage: <html> <form... (2 Replies)
Discussion started by: black_fender
2 Replies

3. UNIX for Dummies Questions & Answers

Shell script to read lines in a text file and filter user data Shell Programming and Scripting

sxsaaas (3 Replies)
Discussion started by: VikrantD
3 Replies

4. UNIX for Dummies Questions & Answers

Read version from a webpage and Upgrading the file.

Hi, I am trying to read version from a web url like <url/daily-builds/sdk-3.2.v20140312170355-osx.zip> and download and store in my filesystem if its the latest. Kindly help me how to read version 3.2.v20140312170355 and compare same with a folder named similar in my directory and... (1 Reply)
Discussion started by: pragyarastogi
1 Replies

5. Programming

Async webpage read

hi i need to asynchronous connect to webpage which has only text file. need to read its content, line by line using linux socket. any samples? (1 Reply)
Discussion started by: leo2008
1 Replies

6. Shell Programming and Scripting

Use Webpage to Start Script

I apologize if this is in the incorrect section, I'm not quite sure which section it should go in. Anyways... I've got a script that I'd like to be able to start with a webpage, something that just has a button that says "Start this Bot", which will start the bot and put it in the background.... (4 Replies)
Discussion started by: JoeGazz84
4 Replies

7. Homework & Coursework Questions

Passing shell variables to a webpage

Use and complete the template provided. The entire template must be completed. If you don't, your post may be deleted! 1. The problem statement, all variables and given/known data: This is my assignment as a whole - Use SVG to present certain dynamic (the raw data should change at least once... (5 Replies)
Discussion started by: ChedWick
5 Replies

8. Programming

socket function to read a webpage (socket.h)

Why does this socket function only read the first 1440 chars of the stream. Why not the whole stream ? I checked it with gdm and valgrind and everything seems correct... #include <stdio.h> #include <stdlib.h> #include <sys/types.h> #include <sys/stat.h> #include <string.h> #include... (3 Replies)
Discussion started by: cyler
3 Replies

9. Shell Programming and Scripting

Run a script from a webpage

Hello, I was trying to run a shell script when I click on webpage link. My webpage is hosted on Sun One web server. This is something I was looking, Chris Johnson's Test Bed I already have my shell script and it works fine when I run it from the command line. Can somebody tell me what are... (3 Replies)
Discussion started by: grajp002
3 Replies

10. Shell Programming and Scripting

Run shell commands via HTML webpage/button?

Hey guys, I got a issue here... we have a development box and a UAT box that our webmasters use, they do the webpage development on the development box and and save changed files to a particular directory on the dev machine. At a certain time of the day a cronjob kicks off and the... (3 Replies)
Discussion started by: zeekblack
3 Replies
Login or Register to Ask a Question