Login or Register to Ask a Question and Join Our Community


UNIX for Dummies Questions & Answers

Delay with Wget

 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Delay with Wget
# 1  
Old 09-12-2013
Delay with Wget
I have a list of URLs that I need to download in a file.

I know I can use the -i option to load the URLs from the file, but how can I make Wget wait a couple secs between each URL download?
# 2  
Old 09-12-2013
From the wget man page:

Code:
-w seconds --wait=seconds 
Wait the specified number of seconds between
the retrievals.  Use of this option is recommended, as it lightens the
server load by making the requests less frequent.  Instead of in seconds,
the time can be specified in minutes using the "m" suffix, in hours using
"h" suffix, or in days using "d" suffix.

Specifying a large value for this option is useful if the network or the
destination host is down, so that Wget can wait long enough to reasonably
expect the network error to be fixed before the retry.  The waiting
interval specified by this function is influenced by "--random-wait",
which see.

Last edited by in2nix4life; 09-12-2013 at 04:16 PM..
# 3  
Old 09-12-2013
Quote:
Originally Posted by in2nix4life
From the wget man page:

Code:
-w seconds --wait=seconds 
Wait the specified number of seconds between
the retrievals.  Use of this option is recommended, as it lightens the
server load by making the requests less frequent.  Instead of in seconds,
the time can be specified in minutes using the "m" suffix, in hours using
"h" suffix, or in days using "d" suffix.

Specifying a large value for this option is useful if the network or the
destination host is down, so that Wget can wait long enough to reasonably
expect the network error to be fixed before the retry.  The waiting
interval specified by this function is influenced by "--random-wait",
which see.

I tried this option but it didn't work. It seems that this delay only applies when Wget receives an error.
# 4  
Old 09-12-2013
Quote:
Originally Posted by rlopes
I tried this option but it didn't work. It seems that this delay only applies when Wget receives an error.
Nope, it works great:

Code:
wget -i list.txt -w 10

What exactly did you do? In what way did it not work? Show your script.
# 5  
Old 09-12-2013
Quote:
Originally Posted by rlopes
I know I can use the -i option...
...suggests that you did not.

If you have a loop in a script, you may need to use the shell's sleep command. Although why would you not use the -i option?
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Wget - working in browser but cannot download from wget

Hi, I need to download a zip file from my the below US govt link. https://www.sam.gov/SAMPortal/extractfiledownload?role=WW&version=SAM&filename=SAM_PUBLIC_MONTHLY_20160207.ZIP I only have wget utility installed on the server. When I use the below command, I am getting error 403... (2 Replies)
Discussion started by: Prasannag87
2 Replies

2. UNIX for Advanced & Expert Users

Trying to understand the delay

Heyas As you know me, i have scripts for about almost every aspect of my IT life. This time, i'm having issues to figure out why my script to connect to my wifi spots takes so long when started as service. The service file (the after:local-fs.target is for 'home installations'): cat... (3 Replies)
Discussion started by: sea
3 Replies

3. Shell Programming and Scripting

Loop without a delay

Hi, I am trying to understand what would happen if ther is a loop without any delay like sleep statement, I feel that would add a lot of load onto the CPU. Trying to understand how the load is reduced by the introduction of sleep(). Thanks and regards Zulfi (3 Replies)
Discussion started by: zulfi123786
3 Replies

4. IP Networking

how to delay the outgoing packets ??

hello, i am searching a way to delay the incoming packets before it goes out of the system may i know how can i approach to this problem? (3 Replies)
Discussion started by: sameer kulkarni
3 Replies

5. Shell Programming and Scripting

Time delay for awk

I have an awk script, and want to introduce a time delay. How can I do this? (3 Replies)
Discussion started by: kristinu
3 Replies

6. Shell Programming and Scripting

while loop and delay

Dear all, if we want to run a command every 5 mins to check if the process is working fine or not... like in c, we can use a simple while loop with a delay for 5 mins... how can we accomplish this is solaris 8/9 thanks br/asad (5 Replies)
Discussion started by: asadlone
5 Replies

7. Programming

Delay a process.

How to delay a process. I need to to delay a process from 3sec. At that 3sec other back ground processes also should stop. (just sit 3sec for idle & then starts execution as normally) I use sleep(3)-But it not stop the bg processes I try to use loop but it not gurantee to wait 3sec. ... (2 Replies)
Discussion started by: ugp
2 Replies

8. Programming

Introducing Delay less then a second.

Hi, I have a doubt in introducing a delay in the programs. We know that we do have a sleep() function/api using which we can bring a delay in terms of seconds. A minimum delay can be atleast 1 second. Now I'm bothered about how to introduce a delay that is just less than a second. Like... (3 Replies)
Discussion started by: S.Vishwanath
3 Replies

9. UNIX for Dummies Questions & Answers

Delay in mv

Working on AIX 4.3 I have an active exe that accepts files for processing on our RS6000. Day to day i store these files in a secure place and at the end of the day I mv them one by one. After some reading and ofcourse trial and error i figured out that this helps... mv `ls -l |head -l | awk... (2 Replies)
Discussion started by: buRst
2 Replies

10. UNIX for Dummies Questions & Answers

Login Delay

When telneting to a SUN box and incorrectly spelling your password, there is a delay of approx. 5 seconds. Is there any way to shorten this time? OS solaris 7 (1 Reply)
Discussion started by: SmartJuniorUnix
1 Replies
Login or Register to Ask a Question