Hello, Unix-Board!

Normally, I would hide error messages in a shell script with

command 2> path

echo "Enter a number here"
read number
test $number -ge 0 && echo "Thank you" && sleep 1 && exit
test $number -le 0 && echo "Thank you" && sleep 1 && exit || echo "This is not a number"

Sorry, but you cannot redirect a syntax error. A numeric test expects a number. Also beware that you script had something after exit which was not going to execute.
First eliminate the non-numeric response. In this technique we try deleting all the numbers in the response and see if there is anything left (which would mean the answer was non-numeric).
( if -z is the same as test -z but modern convention).

echo "Enter a number here"
read number
if [ -z "`echo $number | tr -d [0-9]`" ]
then
            echo "Thank you"
            sleep 1
else
            echo "This is not a number"
            sleep 1
            exit
fi
# Continue processing $number here

# The next line is the last line
exit

Okay, I read some man pages about the

tr -d