02-24-2014
Wget takes a long time to complete
Hi,
I wish to check the return value for wget $url.
However, some urls are designed to take 45 minutes or more to return.
All i need to check if the URL can be reached or not using wget.
How can i get wget to return the value in a few seconds ?
10 More Discussions You Might Find Interesting
1. Programming
After my previous thread, I think I found out what causes the long delays.
I run this program on several Linux computers, and the sometimes (after the file with the arrays becomes big) the fwrite takes between 100 ms to 900 ms.
This is very bad for me, as I want a timer to halt each 30 ms.... ... (5 Replies)
Discussion started by: inna
5 Replies
2. Shell Programming and Scripting
Hi all,
I wrote this shell script to validate filed numbers for input file. But it take forever to complete validation on a file. The average speed is like 9mins/MB.
Can anyone tell me how to improve the performance of a shell script?
Thanks (12 Replies)
Discussion started by: ozzman
12 Replies
3. Linux
Hi,
I am trying to login using ssh on Red Hat Linux 5 server,
The password appears immediately but after I enter the password it takes about 90 seconds to login completely.
Please suggest what changes require?
Regards,
Manoj (4 Replies)
Discussion started by: manoj.solaris
4 Replies
4. UNIX for Dummies Questions & Answers
Hello,
like the title says, how can i measure the time it takes to load a module in Linux, and how how can i measure the time it takes to load a statically compiled module.
/Best Regards Olle
---------- Post updated at 01:13 PM ---------- Previous update was at 11:54 AM ----------
For... (0 Replies)
Discussion started by: ollebanan
0 Replies
5. UNIX for Dummies Questions & Answers
We are running unix. After a reboot of the server we have found that changing password takes a long time. if type in passwd "username" you can type in the 1st instance of the password , press enter , then it will wait for about 3 minutes before bringing up the confirm password line typing it in... (4 Replies)
Discussion started by: AIXlewis
4 Replies
6. Shell Programming and Scripting
Dear experts
I have a 200MG text file in this format:
text \tab number
I try to sort using options -fd and it takes very long! is that normal or I can speed it up in some ways?
I dont want to split the file since this one is already splitted.
I use this command: sort -fd file >... (12 Replies)
Discussion started by: voolek
12 Replies
7. UNIX and Linux Applications
Hi,
we currently having a issue where when we send jobs to the server for the application lawson, it is taking a very long time to complete. here are the last few lines of the database log.
2012-09-18-10.35.55.707279-240 E244403536A576 LEVEL: Warning
PID : 950492 ... (1 Reply)
Discussion started by: techy1
1 Replies
8. UNIX for Advanced & Expert Users
Hi,
Below is my find command
find /opt/app/websphere -name myfolder -perm -600 | wc -l
At time it even takes 20 mins to complete.
my OS is : SunOS mypc 5.10 Generic_150400-09 sun4v sparc SUNW,T5440 (10 Replies)
Discussion started by: mohtashims
10 Replies
9. Shell Programming and Scripting
Hi,
I have a lengthy script which i have trimmed down for a test case as below.
more run.sh
#!/bin/bash
paths="allpath.txt"
while IFS= read -r loc
do
echo "Working on $loc"
startdir=$loc
find "$startdir" -type f \( ! -name "*.log*" ! -name "*.class*" \) -print |
while read file
do... (8 Replies)
Discussion started by: mohtashims
8 Replies
10. Shell Programming and Scripting
I need to check if the files returned by ls command in the below script is a sub-string of the argument passed to the script i.e $1
The below script works fine but is too slow.
If the ls command take 12 secs to complete printing all files with while loop then; using posix substring check... (6 Replies)
Discussion started by: mohtashims
6 Replies
LEARN ABOUT DEBIAN
httpindex
httpindex(1) General Commands Manual httpindex(1)
NAME
httpindex - HTTP front-end for SWISH++ indexer
SYNOPSIS
wget [ options ] URL... 2>&1 | httpindex [ options ]
DESCRIPTION
httpindex is a front-end for index++(1) to index files copied from remote servers using wget(1). The files (in a copy of the remote direc-
tory structure) can be kept, deleted, or replaced with their descriptions after indexing.
OPTIONS
wget Options
The wget(1) options that are required are: -A, -nv, -r, and -x; the ones that are highly recommended are: -l, -nh, -t, and -w. (See the
EXAMPLE.)
httpindex Options
httpindex accepts the same short options as index++(1) except for -H, -I, -l, -r, -S, and -V.
The following options are unique to httpindex:
-d Replace the text of local copies of retrieved files with their descriptions after they have been indexed. This is useful to display
file descriptions in search results without having to have complete copies of the remote files thus saving filesystem space. (See
the extract_description() function in WWW(3) for details about how descriptions are extracted.)
-D Delete the local copies of retrieved files after they have been indexed. This prevents your local filesystem from filling up with
copies of remote files.
EXAMPLE
To index all HTML and text files on a remote web server keeping descriptions locally:
wget -A html,txt -linf -t2 -rxnv -nh -w2 http://www.foo.com 2>&1 |
httpindex -d -e'html:*.html,text:*.txt'
Note that you need to redirect wget(1)'s output from standard error to standard output in order to pipe it to httpindex.
EXIT STATUS
Exits with a value of zero only if indexing completed sucessfully; non-zero otherwise.
CAVEATS
In addition to those for index++(1), httpindex does not correctly handle the use of multiple -e, -E, -m, or -M options (because the Perl
script uses the standard GetOpt::Std package for processing command-line options that doesn't). The last of any of those options ``wins.''
The work-around is to use multiple values for those options seperated by commas to a single one of those options. For example, if you want
to do:
httpindex -e'html:*.html' -e'text:*.txt'
do this instead:
httpindex -e'html:*.html,text:*.txt'
SEE ALSO
index++(1), wget(1), WWW(3)
AUTHOR
Paul J. Lucas <pauljlucas@mac.com>
SWISH++ August 2, 2005 httpindex(1)