As you can see, there are many lines in the file (in this case 30000). Because of that I'm using a trick to download many URLs simultaneosly with this:
Code:
cat url.list | xargs -n 1 -P 10 <<MAGIC COMMAND THAT WILL SAVE ME>>
The problem is that I'd like to rename the output file with the same value of the name field, like: 1.html, 2.html, ..., 30000.html ecc, and use curl to limit the size of the file to 50KB. So the curl command should be something like:
Code:
curl -r 0-50000 -L $URL -o $filename.html -a $filename.log
How can I have it done?
I can parse the output of the pipe with echo $URL | sed -n -e 's/^.*name=//p' but I don't know how use this in the same line grabbing the output of a pipe in 2 variables ($URL and $filename).
Basically I am needing to Download (using curl) in the background some data from(link here), with stderr redirected to /dev/null, to a file named taxcode
I was doing this,
curl & http:// name here/download/pls/Title_26.txt 2> /dev/null > taxcode
but the results were not what I was after.
... (1 Reply)
Hi
I'm trying to download an xml file from a https server using curl on a Linux machine with Ubuntu 10.4.2
I am able to connect to the remote server with my username and password but the output is only "Virtual user <username> logged in".
I am expecting to download the xml file.
My output... (4 Replies)
Hi
I need a Shell script that will download a zip file every second from a http server but i can't use neither curl nor wget.
Can anyone will help me go about this task ???
Thanks!! (1 Reply)
Hi i have a php script that works 100% however i don't want this to run on php because of server limits etc. Ideally if i could convert this simple php script to a shell script i can set it up to run on a cron. My mac server has curl on it. So i am assuming i should be using this to download the... (3 Replies)
Hello all,
I have been struggling with this issue on and off for a couple of weeks now and I just got it all working, so I wanted to share my findings in case some other poor soul needs to know how. First some background on what I'm doing. I am uploading files to different directories based on... (0 Replies)
I am using the below curl command to download a single file from client server and it is working as expected
curl --ftp-ssl -k -u ${USER}:${PASSWD} ftp://${HOST}:${PORT}/path/to/${FILE} --output ${DEST}/${FILE}
let say the client has 3 files hellofile.101, hellofile.102, hellofile.103 and I... (3 Replies)
Good Morning.
I'm trying to download a file from a server. I was able to upload to the server successfully but when i download, i see the file name in my server but with some unknow data. The file name i'm trying to download is abcd.zip in binary mode.
curl -1 -v --ftp-pasv -o abcd.zip -u... (4 Replies)
This question could be specific to the site subdivx.com In the past, I've been able to download a file following location using cURL but there is something about subdivx.com that's different and can't figure out how to get it to work.
I tried the following directly in the terminal with no... (5 Replies)
Hello,
My question is about curl command. (ubuntu14.04)
In terminal, I am able to download my mainfile with:
curl -u user1:pass1 http://11.22.33.44/*******
When I convert it into bash script like this:
#!/bin/bash
cd /root/scripts
computer_ip=11.22.33.44
curl -u $1:$2... (8 Replies)
Discussion started by: baris35
8 Replies
LEARN ABOUT CENTOS
spectool
SPECTOOL(1) User Commands SPECTOOL(1)NAME
spectool - manual page for spectool v1.0.10rpmdev1
SYNOPSIS
spectool [<options>] <specfile>
DESCRIPTION
Spectool is a tool to expand and download sources and patches from specfiles.
If you experience problems with specific specfiles, try to run
rpmbuild --nobuild --nodeps <specfile>
on the file which might give a clue why spectool fails on a file (ignore anything about missing sources or patches). The plan is to catch
errors like this in spectool itself and warn the user about it in the future.
OPTIONS
Operating mode:
-l, --lf, --list-files
lists the expanded sources/patches (default)
-g, --gf, --get-files
gets the sources/patches that are listed with a URL
-h, --help
display this help screen
Files on which to operate:
-A, --all
all files, sources and patches (default)
-S, --sources
all sources
-P, --patches
all patches
-s, --source x[,y[,...]]
specified sources
-p, --patch a[,b[,...]]
specified patches
Miscellaneous:
-d, --define 'macro value'
defines RPM macro 'macro' to be 'value'
-C, --directory dir
download into specified directory (default '.')
-R, --sourcedir
download into rpm's %{_sourcedir}
-n, --dryrun, --dry-run
don't download anything, just show what would be done
-f, --force
try to unlink and download if target files exist
-D, --debug
output debug info, don't clean up when done
FILES
/etc/rpmdevtools/curlrc
optional curl(1) configuration
spectool v1.0.10rpmdev1 June 2014 SPECTOOL(1)
05-21-2017
