Monitoring an html web page changes


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Monitoring an html web page changes
# 1  
Old 11-29-2012
Monitoring an html web page changes

Hello,

I need to monitor an html web page for ANY changes and should be able to know if it's modified or not (since last query). I do not need what modifications but just notification is enough.

This is a simple web page and I don't need to parse the links any further.

Is it possible to do it using a shell script, if yes please advise me how to do it.

Thanks!

---------- Post updated at 11:07 AM ---------- Previous update was at 10:24 AM ----------

I got it done by using wget and keep downloading html files and comparing with last previously downloaded file. This way works but I just want to know if there's any better way.

Thanks
# 2  
Old 11-29-2012
some thing like
first:
Code:
wget http://domain.com/path/to/page.html
md5 page.html > previous_md5
rm page.html

then run script (from cron)
Code:
#!/bin/sh
wget http://domain.com/path/to/page.html
md5 page.html > last_md5
diff previous_md5 last_md5
if [ "$?" = "!" ] ; then 
      mail -s "page.html changed on `date`" your@mail.addr
fi
mv last_md5 previous_md5
rm page.html

# 3  
Old 11-29-2012
wget is NOT working at all because sometimes the downloaded HTML file size is getting different (few bytes) even though no changes in the web page.

It's weird and I don't think we can rely on wget for this.

Any suggestions would highly be appreciated.

Thanks!
# 4  
Old 11-29-2012
Try using lwp-download
Code:
lwp-download "http://your_URL_here.com" download.html

# 5  
Old 11-30-2012
What operating systems must be supported? Some systems have efficient notification interfaces which do not require polling. Upon notification of file modification, an email can be sent.

An example of a tool which leverages such an api: inotifywait(1) - Linux man page

Regards,
Alister
# 6  
Old 11-30-2012
Quote:
Originally Posted by prvnrk
wget is NOT working at all because sometimes the downloaded HTML file size is getting different (few bytes) even though no changes in the web page.
if file size is getting different - file changed.
wget doesn't change downloaded file by itself

if page some times differ (i.e. it has dinamic content) - you must find enother way to monitoring changes. not get page over web.

do you have access to http server or page source (svn/filesystem/other)?
do you need monitoring differing whole page or it's part?
# 7  
Old 12-20-2012
Inotifywait is to monitor file changes on LOCAL files systems.

both wget and lwp-download NOT working consistently (they show different sizes of html files even though there were no changes).

Could anyone please suggest any better solution - thanks much in advance!!
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Accessing the html page

Hi All, In our unix server we have an apache web server running. Now, I want to create my own webpage. Therefore I created webpage at /export/home/test.html file. Where do I need to place this file and what do I need mention this page in my web browser to access it. Without apache... (1 Reply)
Discussion started by: Arasu
1 Replies

2. AIX

How to Use a UNIX Shell Script to Create an HTML Web Page?

dear friends , in my work i have to monitor some system performance in hourly basis by runing some commands , for example (lpstat) to know that all the queue is ready how can i create webpage and connect it with the server (AIX operating system) and make this page refreshed every 10 second and... (12 Replies)
Discussion started by: rami abusweilei
12 Replies

3. Shell Programming and Scripting

Attached HTML page to Email

Greeting all, Not sure anyone tested to send out email with HTML page as attachment from Shell Script ? I know if I use uuencode file.html approach, the mail receive in attachment is empty. So I guess uuencode cannot be use for the html code. Appreciate if anyone can share the code to... (0 Replies)
Discussion started by: ckwan
0 Replies

4. Red Hat

Publishing HTML Page

Hi All, Thanks for reading. I am not sure if I am asking this in the correct group. But here it goes: There is a shell script which does some system checks and creates an html file called system_summary.html on my Red Hat machine say in /reports directory every hour. Now I want to view it... (6 Replies)
Discussion started by: deepakgang
6 Replies

5. UNIX for Dummies Questions & Answers

Publishing HTML Page

Hi All, Thanks for reading. I am not sure if I am asking this in the correct group. But here it goes: There is a shell script which does some system checks and creates an html file called system_summary.html on my Red Hat machine say in /reports directory every hour. Now I want to view it... (1 Reply)
Discussion started by: deepakgang
1 Replies

6. Web Development

Call shell script from HTML page - without web server

Hi, I have html page in my unix machine(server), which I will open with firefox or mozilla available in unix machine. Firefox or mozilla will be opened using x windows. Since I have access to unix machien(like other users) and this HTML page is for user having access to Unix machine, I see no... (7 Replies)
Discussion started by: vamanu9
7 Replies

7. Web Development

findstr in html page

I am planning to create an html page that will count number of connected ports, challenge for me is how to put it in a page. Thanks! (1 Reply)
Discussion started by: webmunkey23
1 Replies

8. UNIX for Dummies Questions & Answers

Accessing a HTML page

Hi All, In our unix server we have an apache web server running. I can access the default apache web page from my windows machine. Now, I want to create my own webpage. Therefore I created webpage at /export/home/myname/test.html file. Where do I need to place this file and what do I need... (2 Replies)
Discussion started by: pkm_oec
2 Replies

9. Solaris

Accessing a HTML page

Hi All, In our unix server we have an apache web server running. I can access the default apache web page from my windows machine. Now, I want to create my own webpage. Therefore I created webpage at /export/home/myname/test.html file. Where do I need to place this file and what do I need... (0 Replies)
Discussion started by: pkm_oec
0 Replies

10. UNIX and Linux Applications

Html web page to Unix Connectivity

Hi All, I need a basic overview of connecting a HTML web page to Unix I will give a brief of my exact requirement. There will be a front end HTML page - a web page which will have certain buttons. Each button will have certain functionality. For eg: There is a button for Disk Usage. When the... (1 Reply)
Discussion started by: abhilashnair
1 Replies
Login or Register to Ask a Question