Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Bash script fails when run as file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Bash script fails when run as file
# 1  
Old 03-19-2014
Bash script fails when run as file
The following bash script fails with error message: "./phpquery_KNBB_html_reader.sh: line 65: syntax error near unexpected token `done'" when do ./<scriptname> in the shell. However when I copy-paste the entire contents of the file directly into a shell environment it runs ok returning the intended output. I suspect there is an issue with variables containing newlines ..

Can anyone shed some light please ?
Thx BR

See attachments for code and file data.
phpquery_KNBB_html_reader.sh
phpQueryData_1.html
phpQueryData_2.html
phpQueryOutput.txt
phpquery_KNBB_html_reader.sh (3.4 KB) 
phpQueryData_1.html (1.5 KB) 
phpQueryData_2.html (36.7 KB) 
phpQueryOutput.txt (3.7 KB) 
# 2  
Old 03-19-2014
You need to run this script with set -x. It looks like there is nothing wrong with the statement, but there is something wrong in the lines following up to the done. One of the quotes is probably off.

---------- Post updated at 11:23 AM ---------- Previous update was at 11:14 AM ----------

In the process of trying to debug this, I noticed that there were some ^M (end of line characters in the file) so I removed them by doing this....
Code:
cat phpquery_KNBB_html_reader.sh | tr -d '\r' > x.sh

I then did
Code:
chmod +x x.sh
./x.sh

and it ran fine without that error
# 3  
Old 03-19-2014
Found nothing wrong with your script or I removed the faulty char in reorganising...
It executed without complaining and got the same output...

Code:
#!/bin/bash
# -- ----------------------------------------------------------------------------------------------------------------------------
# -- File:	phpquery_KNBB_html_reader.sh
# -- Purpose:	Read dutch KNBB (biljarts bond) pages to update internal site's MySQL database
# -- Depends:	Library phpQuery: Jquery port tot PHP. Extract HTML DOM content nodes using CSS style selectors.
# -- System:	Ubuntu 13.10
# --            Linux <hostname> 3.11.0-18-generic #32-Ubuntu SMP Tue Feb 18 21:13:28 UTC 2014 i686 i686 i686 GNU/Linux
# -- History:	20140318 initial version, BurritoSolutions
# -- ----------------------------------------------------------------------------------------------------------------------------
POULE_ID=""
WEEK_NR=""
HOME_TEAM="HOME_TEAM"
AWAY_TEAM="AWAY_TEAM"
RESULT=""
DO_COUNT="false"
declare -i FIELD_COUNT
FIELD_COUNT=0
MAIN_URL="http://competitie.knbb.nl/"
DISTRICT_PATH="overview.php?District=119"
# -- test without using phpQuery; comment the phpQuery line and uncomment the "cat" line
# -- generate file: phpquery "${MAIN_URL}${DISTRICT_PATH}" --find 'table table' --contents | egrep -i "poule|overview" > phpQueryData_1.html
# -- phpquery "${MAIN_URL}${DISTRICT_PATH}" --find 'table table' --contents | egrep -i "poule|overview" | while read POULE_LINE; do
 
while read POULE_LINE 
do
  POULE_LINE=`echo $POULE_LINE | sed 's/<td>//gi' | sed 's/<\/td>//gi'`
  if [[ "$POULE_LINE" =~ "Poule" ]]
  then
    POULE_ID=`echo $POULE_LINE | sed 's/-Klasse Poule //'`
    # -- debug -- echo POULE_ID $POULE_ID
  else
    POULE_RESULTEN_PATH=`echo $POULE_LINE | cut -d'"' -f2`
    # -- debug -- echo Poule_URL ${MAIN_URL}${POULE_RESULTEN_PATH}
    # -- test without using phpQuery; comment the phpQuery line and uncomment the "cat" line
    # -- generate file: phpquery "${MAIN_URL}${POULE_RESULTEN_PATH}" --find 'div.results table' --contents | grep -i '<td' > phpQueryData_2.html
    # -- phpquery "${MAIN_URL}${POULE_RESULTEN_PATH}" --find 'div.results table' --contents | grep -i '<td' | while read RESULT_LINE; do
    #cat phpQueryData_2.html | 
      while read RESULT_LINE
      do
      # -- debug -- echo "$RESULT_LINE"
      if [[ "$RESULT_LINE" =~ "Speelronde" ]] 
      then
        WEEK_NR=`echo $RESULT_LINE | cut -d'>' -f3 | cut -d'<' -f1 | cut -c11-99`
        # -- debug -- echo Week_nr: $WEEK_NR
      fi
      if [[ "$RESULT_LINE" =~ "ma</td>" || "$RESULT_LINE" =~ "di</td>" || "$RESULT_LINE" =~ "wo</td>" ||\
     "$RESULT_LINE" =~ "do</td>" || "$RESULT_LINE" =~ "vr</td>" ]] 
      then
        DO_COUNT="true"
        FIELD_COUNT=0
      fi
      if [ "$DO_COUNT" == "true" ] 
      then
        FIELD_COUNT=$((FIELD_COUNT+1))
        # -- debug -- echo Field_Count "$FIELD_COUNT"
      fi
      # -- html td columns 5 and 6 contain team names
      if [ $FIELD_COUNT -eq 5 ]
      then
        HOME_TEAM=`echo $RESULT_LINE | cut -d'>' -f2 | cut -d'<' -f1`
      fi
      if [ $FIELD_COUNT -eq 6 ]
      then
        AWAY_TEAM=`echo $RESULT_LINE | cut -d'>' -f2 | cut -d'<' -f1`
      fi
      if [[ "$RESULT_LINE" =~ "final" ]] 
      then
        DO_COUNT="false"
      fi
      if [[ "$RESULT_LINE" =~ "cpb3" ]] 
      then
        RESULT=`echo $RESULT_LINE | cut -d'>' -f3 | cut -d'<' -f1`
        echo "$POULE_ID $WEEK_NR $HOME_TEAM $AWAY_TEAM $RESULT"
	    # -- construct SQL insert statement for internal MySQL database
      fi
    done < phpQueryData_2.html
    break # -- for test i use only one outer loop since the script still fails ..
  fi
done < phpQueryData_1.html

But when I executed the script in debug mode, it produced:
Code:
/home/vbe/wks/test/bash $ cat 0002.err|grep -v ^+
sed: Function s/<td>//gi cannot be parsed.
sed: Function s/<\/td>//gi cannot be parsed.

Those lines are the start:
Code:
+ POULE_ID=
+ WEEK_NR=
+ HOME_TEAM=HOME_TEAM
+ AWAY_TEAM=AWAY_TEAM
+ RESULT=
+ DO_COUNT=false
+ declare -i FIELD_COUNT
+ FIELD_COUNT=0
+ MAIN_URL=http://competitie.knbb.nl/
+ DISTRICT_PATH='overview.php?District=119'
+ read POULE_LINE
++ echo '<td>B1-Klasse' Poule 'A</td>'
++ sed 's/<td>//gi'
sed: Function s/<td>//gi cannot be parsed.
++ sed 's/<\/td>//gi'
sed: Function s/<\/td>//gi cannot be parsed.
+ POULE_LINE=
+ [[ '' =~ Poule ]]
++ echo
++ cut '-d"' -f2
+ POULE_RESULTEN_PATH=
+ read RESULT_LINE
+ [[ <tfoot><tr><td colspan="7">M- </td></tr></tfoot> =~ Speelronde ]]
+ [[ <tfoot><tr><td colspan="7">M- </td></tr></tfoot> =~ ma</td> ]]
+ [[ <tfoot><tr><td colspan="7">M- </td></tr></tfoot> =~ di</td> ]]
+ [[ <tfoot><tr><td colspan="7">M- </td></tr></tfoot> =~ wo</td> ]]
+ [[ <tfoot><tr><td colspan="7">M- </td></tr></tfoot> =~ do</td> ]]
+ [[ <tfoot><tr><td colspan="7">M- </td></tr></tfoot> =~ vr</td> ]]
+ '[' false == true ']'
+ '[' 0 -eq 5 ']'
+ '[' 0 -eq 6 ']'
+ [[ <tfoot><tr><td colspan="7">M- </td></tr></tfoot> =~ final ]]
+ [[ <tfoot><tr><td colspan="7">M- </td></tr></tfoot> =~ cpb3 ]]
+ read RESULT_LINE
+ [[ <tr><td colspan="7">M- </td></tr> =~ Speelronde ]]
+ [[ <tr><td colspan="7">M- </td></tr> =~ ma</td> ]]
+ [[ <tr><td colspan="7">M- </td></tr> =~ di</td> ]]
+ [[ <tr><td colspan="7">M- </td></tr> =~ wo</td> ]]
+ [[ <tr><td colspan="7">M- </td></tr> =~ do</td> ]]
+ [[ <tr><td colspan="7">M- </td></tr> =~ vr</td> ]]
+ '[' false == true ']'
+ '[' 0 -eq 5 ']'
+ '[' 0 -eq 6 ']'

etc...

Last edited by vbe; 03-19-2014 at 03:53 PM.. Reason: why cant I type without typos????
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Replace string works on command-line but fails when run from shell script

I wish to replace "\\n" with a single white space. The below does the job on command-line: $ echo '/fin/app/scripts\\n/fin/app/01/sql' | sed -e 's#\\\\n# #g'; /fin/app/scripts /fin/app/01/sql However, when i have the same code to a shell script it is not able to get me the same output:... (8 Replies)
Discussion started by: mohtashims
8 Replies

2. Shell Programming and Scripting

Script fails to run properly when run from CRONTAB

Hello all, I'm trying to write a script to gather and send data and it works just fine at the bash command line, but when executing from CRON, it does not run properly. My scripting skills are pretty limited and there's probably a better way, but as I said it works at the command line, but... (12 Replies)
Discussion started by: rusman
12 Replies

3. Shell Programming and Scripting

Mount fails (because remote machine is off) bash script stops

Hi, I'm trying to automate a couple of mounts. (I would do this in fstab, but the auto feature causes the virtual machine I'm running in to freeze when it boots up) If the machine I'm trying to connect to is OFF, I get the host unreachable error and then the bash script stops. The problem is... (3 Replies)
Discussion started by: jdilts
3 Replies

4. Shell Programming and Scripting

Bash script fails with "function: not found" error

Hello everyone, I am having problems figuring this out. This script below is supposed to create a list of file names with their "md5sum", in a file "lib-list.txt" When I run it "sh component-list.sh " I get this:component-list.sh: 4: component-list.sh: function: not found component-list.sh:... (4 Replies)
Discussion started by: joemb
4 Replies

5. Shell Programming and Scripting

Script fails with ORA-20000 when run with su

Hi, I have a ksh script that runs as root ans issues several commands as a user differente from root as 'su <user> -c "command" ' . It works fine except for one step where the command executes sql statements. That command fails with ORA-20000. Now the strangest thing, if I place a read command... (5 Replies)
Discussion started by: naf
5 Replies

6. Shell Programming and Scripting

Need help! command working ok when executed in command line, but fails when run inside a script!

Hi everyone, when executing this command in unix: echo "WM7 Fatal Alerts:", $(cat query1.txt) > a.csvIt works fine, but running this command in a shell script gives an error saying that there's a syntax error. here is content of my script: tdbsrvr$ vi hc.sh "hc.sh" 22 lines, 509... (4 Replies)
Discussion started by: 4dirk1
4 Replies

7. Shell Programming and Scripting

Run a bash script, display on the screen and save all information in a file including error info

Hi all, How to: Run a bash script, display on the screen and save all information in a file including error information. For example: I have a bash script called test.sh now I want to run the test.sh and display the output on the screen and save the output including error info to a file. ... (1 Reply)
Discussion started by: Damon sine
1 Replies

8. Shell Programming and Scripting

crontab fails to run script

OS is Ubuntu 8.04.3. When I run the command: /usr/bin/syslogMailer < /etc/syslog.pipes/criticalMessagesFrom a bash shell it works and i receive an email as per the script however when run from crontab it does not work. Can anyone explain why and how to fix it? /usr/bin/syslogMailer... (4 Replies)
Discussion started by: jelloir
4 Replies

9. Shell Programming and Scripting

how to make your bash script run on a machine with csh and bash

hi, i have a script that runs on bash and would like to run it on a machine that has csh and bash. the default setting on that machine is csh. i dont want to change my code to run it with a csh shell. is there any way i can run the script (written in bash) on this machine? in other words is there... (3 Replies)
Discussion started by: npatwardhan
3 Replies

10. Shell Programming and Scripting

Running script that sends an html formatted email fails when its run as cronjob

Hi Im very new at working with unix and this problem I simply can not understand. I know there are a lot of threads about problems with shell scripts behaving differently when run from a terminal and from a cronjob. I have tried everything(almost) but I still havent cracked this problem. Im... (15 Replies)
Discussion started by: Nightowl
15 Replies
Login or Register to Ask a Question