Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

How to cancel wget download after 1%?

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting How to cancel wget download after 1%?
# 8  
Old 04-29-2013
Assuming you know what 1% of the target file is, why would you bother monitoring a file when you can just restrict it through a pipe with dd? Or use curl to request a byte range? Monitoring seems pointlessly inefficient.

Regards,
Aliser
# 9  
Old 05-01-2013
Thanks for the responses !!!

1% is an arbitrary value. How does pipe work? I assume wget gets complete file and passes it to the pipe, which I am not interested in.

I tried curl and its useful. But 1% is changing and hence I don't have fixed byte range for all downloads
# 10  
Old 05-01-2013
Quote:
I assume wget gets complete file and passes it to the pipe
That is not correct, as I explained in previous post on this thread. You can confirm this with wget by looking at the wget log file, for example if you run wget -o logfile ... | head and you will see that wget stops working when the pipe breaks.

If 1% is an arbitrary value, then just use something like wget ... | head -c 1MB or wget ... | head -c 1000000
# 11  
Old 05-01-2013
Quote:
Originally Posted by hanson44
If 1% is an arbitrary value, then just use something like wget ... | head -c 1MB or wget ... | head -c 1000000
If portability is a concern, be aware that some implementations of head do not offer -c.

Quote:
Originally Posted by tamil.pamaran
I tried curl and its useful. But 1% is changing and hence I don't have fixed byte range for all downloads
You can calculate the 1% byte range from the content-length header of an HTTP HEAD request. Refer to curl's -I (aka --head) option.

Regards,
Alister
# 12  
Old 05-14-2013
Linux
Hansan44 and Alister,

Thanks a lot for your responses. Smilie

/Tamil
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Wget - working in browser but cannot download from wget

Hi, I need to download a zip file from my the below US govt link. https://www.sam.gov/SAMPortal/extractfiledownload?role=WW&version=SAM&filename=SAM_PUBLIC_MONTHLY_20160207.ZIP I only have wget utility installed on the server. When I use the below command, I am getting error 403... (2 Replies)
Discussion started by: Prasannag87
2 Replies

2. Shell Programming and Scripting

Wget help - unable to download pic

There is a website providing traffic camera images that gets updated every few minutes. My aim is to download the images over time to get a view of traffic conditions through the day. Website: CHECKPOINT.SG Image link, as taken from site source: http://www.checkpoint.sg/sg/2701 I tried... (2 Replies)
Discussion started by: Leion
2 Replies

3. Shell Programming and Scripting

How to change wget download directory?

i have a cron that mirrors a site periodically wget -r -nc --passive-ftp ftp://user:pass@123.456.789.0 i want to download this into a directory called /files but when I do this, it always create a new directory called "123.456.789.0" (the hostname) it puts it into /files/123.456.789.0 but... (3 Replies)
Discussion started by: vanessafan99
3 Replies

4. Shell Programming and Scripting

Files download using wget

Hi, I need to implement below logic to download files daily from a URL. * Need to check if it is yesterday's file (YYYY-DD-MM.dat) * If present then download from URL (sample_url/2013-01-28.dat) * Need to implement wait logic if not present * if it still not able to find the file... (1 Reply)
Discussion started by: rakesh5300
1 Replies

5. Shell Programming and Scripting

How to download file without curl and wget

Hi I need a Shell script that will download a zip file every second from a http server but i can't use neither curl nor wget. Can anyone will help me go about this task ??? Thanks!! (1 Reply)
Discussion started by: rubber08
1 Replies

6. Windows & DOS: Issues & Discussions

Disappearing wget download [Windows]

I downloaded and installed wget for windows, then used cmd.exe to run it directly from its install folder. I downloaded an 8.5 GB (yes, Giga) tar file, waited a couple of days, then tried to find it only to see that it's nowhere to be found! I don't want to re-download the whole thing, especially... (3 Replies)
Discussion started by: HalfThere
3 Replies

7. Shell Programming and Scripting

How to download to a file using wget in perl?

Hi, I want to download some online data using wget command and write the contents to a file. For example this is the URL i want to download and store it in a file called "results.txt". #This is the URL. $url="http://www.example.com"; #retrieve data and store in a file results.txt ... (3 Replies)
Discussion started by: vanitham
3 Replies

8. UNIX and Linux Applications

download file using wget

I need to download the following srs8.3.0.1.standard.linux26_32.tar.gz file from the following website: http://downloads.biowisdomsrs.com/srs83_dist There are many gzip files along with the above one in the above site but I want to download the srs8.3.0.1.standard.linux26_32.tar.gz only from... (1 Reply)
Discussion started by: alphasahoo
1 Replies

9. Shell Programming and Scripting

download a particular file using wget

Hi All I want to download srs8.3.0.1.standard.linux24_EM64T.tar.gz file from the following website : http://downloads.biowisdomsrs.com/srs83_dist/ But this website contains lots of zipped files I want to download the above file only discarding other zipped files. When I am trying the... (1 Reply)
Discussion started by: alphasahoo
1 Replies

10. UNIX for Dummies Questions & Answers

Using wget to download a file

Hello Everyone, I'm trying to use wget recursively to download a file. Only html files are being downloaded, instead of the target file. I'm trying this for the first time, here's what I've tried: wget -r -O jdk.bin... (4 Replies)
Discussion started by: thoughts
4 Replies
Login or Register to Ask a Question