Shell Programming and Scripting

how can you check if wget finishes?
i have this code to store the source of a website url into a variable:

Code:

source=$(wget --timeout=15 -qO - $site)

but this gets stuck sometimes

e.g., one site had a virus and it stopped the script
how can i check if wget finishes?




i tried to run wget in the command line for the site that stopped,
and it showed something like this at the end:

Code:

</div><!-- #wrapper -->

</body>
</html>

but then it just hangs there

like you can't type in the terminal, you have to exit out of it

do you know how to do a check for if wget is "stuck"?


thanks!

Hi,
you may check for the presence of wget in the output of

Code:

ps -eaf

if still there after a reasonable time it may mean wget is stuck and you can kill wget‘s PID.

see ya
fra

May be you can give a try at -T timeout option in wget...

--ahamed

is this for the command line?

i am trying to use wget in a script and store it into a variable, for a list of domains, so for each domain i'm trying to find a way to determine how wget is stuck, and if its stuck then to skip that iteration

but i'm not sure how to check if wget is stuck
is there a command to check this that can be put into the script?
like does wget give a certain output if it is stuck?

---------- Post updated at 09:46 AM ---------- Previous update was at 09:45 AM ----------

i'm currently using a timeout like:
source=$(wget --timeout=15 -qO - $site)

but it's not a timeout

like if there is a virus on the site, it doesn't timeout, wget still has an output but it is just "stuck", like it doesn't finish so that the next command can be entered

how can you check for this?

---------- Post updated at 09:48 AM ---------- Previous update was at 09:46 AM ----------

it seems like wget doesnt exit properly or something
it doesnt time out, just hangs
like it doesnt allow the next command

how can i check if wget is exiting properly or not in a script?

Seeing how this command should run relatively quick, perhaps you could just background it:

Code:

source=$(wget -qO - $site &)

And then sleep for "X" amount of seconds or minutes and check to see if wget is still running. If so, kill the process.

i found the problem, i think it kept on retrying, so i set tries=1 and then it exited

thanks for the responses

what i have is ok now

i was wondering, do you know how to check if wget returns an error code or a 301 redirect, etc.? like a 404 error, or 301 moved permanently, etc.? but if its complicated then nevermind, for now the basic wget check is enough

thanks