Sponsored Content
Top Forums Shell Programming and Scripting Grep text matching problem with script which checks if web page contains text. Post 302817865 by gencon on Thursday 6th of June 2013 03:14:47 PM
Old 06-06-2013
Grep text matching problem with script which checks if web page contains text.

I wrote a Bash script which checks to see if a text string exists on a web page and then sends me an email if it does (or does not e.g. "Out of stock"). I run it from my crontab, it's quite handy from time to time and I've been using it for a few years now.

The script uses wget to download an url and then uses grep to match the text string which I lift from the original HTML in case of markups or new lines (though the latter has never actually occured).

Today I added the text to look for and it did not get matched even though it was present in the HTML. When I copied and pasted the, identically looking, text from the downloaded HTML file into the script and tried that it worked perfectly. [Problem solved in this case but I'd like to fix things properly.]

So the character encoding seems to be the problem. Or so I thought! Grep uses utf-8 but it turned out the source HTML was utf-8 as well. What a pain, so no easy fix by using iconv to convert all downloaded files to utf-8.

Anyone know what might be happening here and what I need to do to fix this?

Many thanks.
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Is there a way scroll text instead of page?

Is there a way to slowly scroll the output of a file instead of page or cat ? Instead of one page at a time, I would like to slowly scroll the displayed output of the file. (12 Replies)
Discussion started by: darthur
12 Replies

2. UNIX for Dummies Questions & Answers

grep multiple text files in folder into 1 text file?

How do I use the grep command to take mutiple text files in a folder and make one huge text file out of them. I'm using Mac OS X and can not find a text tool that does it so I figured I'd resort to the BSD Unix CLI for a solution... there are 5,300 files that I want to write to one huge file so... (7 Replies)
Discussion started by: coppertone
7 Replies

3. Shell Programming and Scripting

how to redirect to a web-page by shell script

Dear all, I am calling a korn shell script(CGI script) by a web-page. This shell script do some checking in a unix file and return true or false. Now within the same script, If it returns true then I want to redirect to another web-page stored in htdocs directory. Example: Login page sends a... (3 Replies)
Discussion started by: ravi18s
3 Replies

4. UNIX for Dummies Questions & Answers

Possible to download web page's text to a file?

Hi, Say there is a web page that contains just text only - that is, even the source code is just the text itself, nothing more. An example would be "http://mynasadata.larc.nasa.gov/docs/ocean_percent.txt" Is there a UNIX command that would allow me to download this text and store it in a... (1 Reply)
Discussion started by: Breanne
1 Replies

5. HP-UX

Help running a unix script from a web page

First, let me state that I am completely out of my realm with this. I have a server running HPUX. I'm not even sure if this can be considered a UNIX question and for that let me apologize in advance. I need to create a web page where a client can input 2 variables (i.e. date and phone number).... (0 Replies)
Discussion started by: grinds
0 Replies

6. Shell Programming and Scripting

Removing matching text from multiple files with a shell script

Hello all, I am in need of assistance in creating a script that will remove a specified block of text from multiple .htaccess files. (roughly 1000 files) I am attempting to help with a project to clean up a linux server that has a series of unwanted url rewrites in place, as well as some... (4 Replies)
Discussion started by: boxx
4 Replies

7. Shell Programming and Scripting

Matching text using grep

Hi folks... Relatively new to scripting, but really struggling with something that will no doubt be second nature to most people on here: Trying to get an exact match on $sub, where sub is an ip address. subnet () { clear while true do ... (18 Replies)
Discussion started by: CiCa
18 Replies

8. Shell Programming and Scripting

Script extract text from txt file with grep

All, I require a script that grabs some text from the gitHub API and will grep (or other function) for a string a characters that starts with (") quotes followed by two letters, may contain a pipe |, and ending with ) . What i have so far is below but it's not returning anything. ... (4 Replies)
Discussion started by: ChocoTaco
4 Replies

9. Shell Programming and Scripting

Web page with picture, text, php function

Hello. I'm trying to create a web page which the presentation is as follows: 1 °) at the top of page an image 2 °) below the text 3 °) to complete a php function that returns information. I tried different things but none work. Script 1: <!DOCTYPE html> <html> <head> <style> div { ... (5 Replies)
Discussion started by: jcdole
5 Replies

10. Shell Programming and Scripting

Copy text from web page and add to file

I need help to make a script for Ubuntu to OSCam that copy the text on this website that only contains "C: ip port randomUSERNAME password" and want to exclude the text "C:" and replace the rest with the old in my test.server file. (line 22) device = ip,port (line 23) user =... (6 Replies)
Discussion started by: baxarn
6 Replies
aha(1)								 Ansi HTML Adapter							    aha(1)

NAME
aha - Ansi HTML Adapter SYNOPSIS
aha [options] [-f datei] DESCRIPTION
aha takes SGR-colored Input and prints W3C conform HTML-Code. aha reads the Input from a file or stdin and writes HTML-Code to stdout. OPTIONS
--help , -h , -? A help like this --black , -b Black Background and white "standard color" --pink , -p Pink Background --iso X , -i X Uses ISO 8859-X instead of utf-8. X must be 1..16 --title X , -t X Gives the html output the title --line-fix , -l Uses a fix for inputs using control sequences to change the cursor position like htop. It's a hot fix, it may not work with any pro- gram like htop. (See EXAMPLE) EXAMPLE
aha --help | aha --black --title "the awesome aha help"> aha-help.htm Creates an HTML file with the help of aha with black background colordiff oldfile.c newfile.c | aha > colordiff.htm Creates an HTML file with a colorful diff-output of two files "oldfile.c" and "newfile.c" with white background ls --color=always | aha --pink > ls.htm Creates an HTML file with a colorful ls-output with pink background. echo a | htop | aha --black --line-fix > htop.htm Creates an HTML file with the output of htop. You have to use --line-fix due the other new-line-commands htop uses. AUTHOR
Copyleft Alexander Matthes aka Ziz 2011 zizsdl@googlemail.com SEE ALSO
http://ziz.delphigl.com/tool_aha.php August 31, 2011 aha(1)
All times are GMT -4. The time now is 04:46 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy