04-29-2013
Yes, curl has a -r option that is useful if you know the file size before starting the download.
Also, is it possible the 1% is arbitrary? Even if you don't know the file size before, for your test you could download say 1 MB or whatever you choose.
10 More Discussions You Might Find Interesting
1. UNIX for Dummies Questions & Answers
Hello Everyone,
I'm trying to use wget recursively to download a file.
Only html files are being downloaded, instead of the target file.
I'm trying this for the first time, here's what I've tried:
wget -r -O jdk.bin... (4 Replies)
Discussion started by: thoughts
4 Replies
2. Shell Programming and Scripting
Hi All
I want to download srs8.3.0.1.standard.linux24_EM64T.tar.gz file from the following website :
http://downloads.biowisdomsrs.com/srs83_dist/
But this website contains lots of zipped files
I want to download the above file only discarding other zipped files.
When I am trying the... (1 Reply)
Discussion started by: alphasahoo
1 Replies
3. UNIX and Linux Applications
I need to download the following srs8.3.0.1.standard.linux26_32.tar.gz file from the following website:
http://downloads.biowisdomsrs.com/srs83_dist
There are many gzip files along with the above one in the above site but I want to download the srs8.3.0.1.standard.linux26_32.tar.gz only from... (1 Reply)
Discussion started by: alphasahoo
1 Replies
4. Shell Programming and Scripting
Hi,
I want to download some online data using wget command and write the contents to a file.
For example this is the URL i want to download and store it in a file called "results.txt".
#This is the URL.
$url="http://www.example.com";
#retrieve data and store in a file results.txt
... (3 Replies)
Discussion started by: vanitham
3 Replies
5. Windows & DOS: Issues & Discussions
I downloaded and installed wget for windows, then used cmd.exe to run it directly from its install folder. I downloaded an 8.5 GB (yes, Giga) tar file, waited a couple of days, then tried to find it only to see that it's nowhere to be found! I don't want to re-download the whole thing, especially... (3 Replies)
Discussion started by: HalfThere
3 Replies
6. Shell Programming and Scripting
Hi
I need a Shell script that will download a zip file every second from a http server but i can't use neither curl nor wget.
Can anyone will help me go about this task ???
Thanks!! (1 Reply)
Discussion started by: rubber08
1 Replies
7. Shell Programming and Scripting
Hi,
I need to implement below logic to download files daily from a URL.
* Need to check if it is yesterday's file (YYYY-DD-MM.dat)
* If present then download from URL (sample_url/2013-01-28.dat)
* Need to implement wait logic if not present
* if it still not able to find the file... (1 Reply)
Discussion started by: rakesh5300
1 Replies
8. Shell Programming and Scripting
i have a cron that mirrors a site periodically
wget -r -nc --passive-ftp ftp://user:pass@123.456.789.0
i want to download this into a directory called /files
but when I do this, it always create a new directory called "123.456.789.0" (the hostname)
it puts it into /files/123.456.789.0
but... (3 Replies)
Discussion started by: vanessafan99
3 Replies
9. Shell Programming and Scripting
There is a website providing traffic camera images that gets updated every few minutes.
My aim is to download the images over time to get a view of traffic conditions through the day.
Website: CHECKPOINT.SG
Image link, as taken from site source: http://www.checkpoint.sg/sg/2701
I tried... (2 Replies)
Discussion started by: Leion
2 Replies
10. Shell Programming and Scripting
Hi,
I need to download a zip file from my the below US govt link.
https://www.sam.gov/SAMPortal/extractfiledownload?role=WW&version=SAM&filename=SAM_PUBLIC_MONTHLY_20160207.ZIP
I only have wget utility installed on the server.
When I use the below command, I am getting error 403... (2 Replies)
Discussion started by: Prasannag87
2 Replies
LEARN ABOUT MOJAVE
curlopt_postfieldsize_large
CURLOPT_POSTFIELDSIZE_LARGE(3) curl_easy_setopt options CURLOPT_POSTFIELDSIZE_LARGE(3)
NAME
CURLOPT_POSTFIELDSIZE_LARGE - size of POST data pointed to
SYNOPSIS
#include <curl/curl.h>
CURLcode curl_easy_setopt(CURL *handle, CURLOPT_POSTFIELDSIZE_LARGE,
curl_off_t size);
DESCRIPTION
If you want to post data to the server without having libcurl do a strlen() to measure the data size, this option must be used. When this
option is used you can post fully binary data, which otherwise is likely to fail. If this size is set to -1, the library will use strlen()
to get the size.
DEFAULT
-1
PROTOCOLS
HTTP(S)
EXAMPLE
CURL *curl = curl_easy_init();
if(curl) {
const char *data = large_chunk;
curl_off_t length_of_data; /* set somehow */
curl_easy_setopt(curl, CURLOPT_URL, "http://example.com");
/* size of the POST data */
curl_easy_setopt(curl, CURLOPT_POSTFIELDSIZE_LARGE, length_of_data);
curl_easy_setopt(curl, CURLOPT_POSTFIELDS, data);
curl_easy_perform(curl);
}
AVAILABILITY
Along with HTTP
RETURN VALUE
Returns CURLE_OK if HTTP is supported, and CURLE_UNKNOWN_OPTION if not.
SEE ALSO
CURLOPT_POSTFIELDS(3), CURLOPT_COPYPOSTFIELDS(3), CURLOPT_POSTFIELDSIZE(3),
libcurl 7.54.0 February 03, 2016 CURLOPT_POSTFIELDSIZE_LARGE(3)