12-30-2011
i found the problem, i think it kept on retrying, so i set tries=1 and then it exited
thanks for the responses
what i have is ok now
i was wondering, do you know how to check if wget returns an error code or a 301 redirect, etc.? like a 404 error, or 301 moved permanently, etc.? but if its complicated then nevermind, for now the basic wget check is enough
thanks
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Hi All,
We now have a Log file which has time stamps of all the scripts ran. It has start
time and end time. I now have a requirement to send mail to the Admin after the script finishes running.
Can some one help me with a Script which will send mail with Start time and
End time of the... (2 Replies)
Discussion started by: srikanthgr1
2 Replies
2. Shell Programming and Scripting
The title sounds like an easy problem (maybe it is), however, the catch is that when a job 'A' is called inside a wrapper, the execution does not wait for 'A' to finish, but it goes on to the next line (right after kicking off 'A').
My question is, how will you keep the execution waiting for... (4 Replies)
Discussion started by: ChicagoBlues
4 Replies
3. Shell Programming and Scripting
Hello,
I'm using a shell script containing a wget-command that copies html-files from a website to my ISP-server. I therefore want to check if that file exist and also if the filessize is larger than e.g. 100 bytes. Probably it's done by using something like fileexist and filesize.
The wget... (3 Replies)
Discussion started by: weatherboys
3 Replies
4. Shell Programming and Scripting
I all,
I wrote an script which starts a Weblogic server and waits until its loaded to deploy several apps. The way I checked was something like:
while ; do
wget --spider <URL>:<port>/console > /dev/null 2>&1
rc=$?
done
This works perfectly because it's an HTML site and when server is... (2 Replies)
Discussion started by: AlbertGM
2 Replies
5. UNIX for Dummies Questions & Answers
If I run "sleep 10&".. when that background process is done, is there a way for unix to print to screen to let me know? Thanks. (1 Reply)
Discussion started by: jmelai
1 Replies
6. Shell Programming and Scripting
Hello,
This is my first post. Nice forum!
I have a file trls.results
small exemple of content (actually the file can be very big):
./security/htaccess.htm ./security/ipcount.log ./adhoc/sql/datamod06.sql
So there is 3 paths to 3 different files... I want to remove every string that has a... (9 Replies)
Discussion started by: Jacob106106
9 Replies
7. UNIX for Dummies Questions & Answers
Hi, I'm trying to install some libraries, when running the makefile I get an error from the "wget --no check certificate option". I had a look help and the option wasn't listed. Anyone know what I'm missing. (0 Replies)
Discussion started by: davcra
0 Replies
8. Shell Programming and Scripting
hello
check this script it jump to else part, in n both cases,
(if files exist or not)
wget $MIRROR/kkk.zip && wget $MIRROR/jjj.zip
RC="$?"
if ]
then
echo -e "$RED Ooops, Download links are broken...! $RESET"
else
echo -e "$GREEN Everything is fine, Cheers ... $RESET"
fi (4 Replies)
Discussion started by: nimafire
4 Replies
9. UNIX for Advanced & Expert Users
Hi All,
Using Wget I'm able to get the status of the server.....only when the server is completely down or up....
but problem here in script is Suppose if the server got hang I mean to say that if the server is taking long time to login, for example normally the server takes 3 seconds to login... (3 Replies)
Discussion started by: manohar2013
3 Replies
10. UNIX for Beginners Questions & Answers
Hi
I am facing a problem in my ksh.
My main script is calling 3 different child process in the background.
I am using wait to finish all and then submit another 3 child processes.
Now what i want is , whenever any one child process finishes ,i want to submit next one.so that parallel 3... (2 Replies)
Discussion started by: Sangu
2 Replies
LEARN ABOUT DEBIAN
httpindex
httpindex(1) General Commands Manual httpindex(1)
NAME
httpindex - HTTP front-end for SWISH++ indexer
SYNOPSIS
wget [ options ] URL... 2>&1 | httpindex [ options ]
DESCRIPTION
httpindex is a front-end for index++(1) to index files copied from remote servers using wget(1). The files (in a copy of the remote direc-
tory structure) can be kept, deleted, or replaced with their descriptions after indexing.
OPTIONS
wget Options
The wget(1) options that are required are: -A, -nv, -r, and -x; the ones that are highly recommended are: -l, -nh, -t, and -w. (See the
EXAMPLE.)
httpindex Options
httpindex accepts the same short options as index++(1) except for -H, -I, -l, -r, -S, and -V.
The following options are unique to httpindex:
-d Replace the text of local copies of retrieved files with their descriptions after they have been indexed. This is useful to display
file descriptions in search results without having to have complete copies of the remote files thus saving filesystem space. (See
the extract_description() function in WWW(3) for details about how descriptions are extracted.)
-D Delete the local copies of retrieved files after they have been indexed. This prevents your local filesystem from filling up with
copies of remote files.
EXAMPLE
To index all HTML and text files on a remote web server keeping descriptions locally:
wget -A html,txt -linf -t2 -rxnv -nh -w2 http://www.foo.com 2>&1 |
httpindex -d -e'html:*.html,text:*.txt'
Note that you need to redirect wget(1)'s output from standard error to standard output in order to pipe it to httpindex.
EXIT STATUS
Exits with a value of zero only if indexing completed sucessfully; non-zero otherwise.
CAVEATS
In addition to those for index++(1), httpindex does not correctly handle the use of multiple -e, -E, -m, or -M options (because the Perl
script uses the standard GetOpt::Std package for processing command-line options that doesn't). The last of any of those options ``wins.''
The work-around is to use multiple values for those options seperated by commas to a single one of those options. For example, if you want
to do:
httpindex -e'html:*.html' -e'text:*.txt'
do this instead:
httpindex -e'html:*.html,text:*.txt'
SEE ALSO
index++(1), wget(1), WWW(3)
AUTHOR
Paul J. Lucas <pauljlucas@mac.com>
SWISH++ August 2, 2005 httpindex(1)