Bash script idea using cUrl -- possible? Post: 302399240

Sponsored Content

Top Forums Shell Programming and Scripting Bash script idea using cUrl -- possible? Post 302399240 by fubaya on Friday 26th of February 2010 08:03:39 PM

02-26-2010

Registered User

Here is a very rough outline of one way to do it. It works but needs a lot of polishing and may not be the method you want to use. I didn't fool around with the logfile name or any error checking.

Code:

while true 
do echo "Gimme me something to download"
read LINK
[ "$LINK" = "stop" ] && exec echo "I'm exiting"
echo "File name?"
read NAME
curl -o "$NAME" "$LINK" && echo "$NAME" "$LINK" > file.log
done

"while true; do [stuff]; done" is an infinite loop. "read" just sits there and waits for you to input something, which it then puts in the LINK variable. If that something is "stop," it will say "Im exiting" and exit. The exec command is what makes it exit in this example, it runs one command then exits. For anything except "stop," it asks for a file name, then curl gets the link and puts it in that file. If curl succeeds, it echos the name and link to a logfile.

edit to add: I'm certainly not a guru and this is just a rough example. Don't take it as The Way Things Should Be Done.

fubaya

View Public Profile for fubaya

Find all posts by fubaya

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Limitations of awk? Good idea? Bad idea?

Keeping in mind that I'm relatively comfortable with programming in general but very new to unix and korn/bourne shell scripts.. I'm using awk on a CSV file, and then performing calculations and operations on specific fields within specific records. The CSV file I'm working with has about 600...

2. Shell Programming and Scripting

Idea for an "informative" bash alias or script

I had the idea come into my head that it would be good to have a single command that gave file type, file size, last modification date, and owner/group information, like what one would see in a GUI "Properties" dialog, but all in a terminal window. In my opinion, these statistics about a file...

3. Shell Programming and Scripting

bash curl escape & in url

I'm running a curl command in bash, but the & in the middle causes the second half of the line to run in the background, here's what I'm trying to do: lat="37.451" lon="-122.18" url="http://ws.geonames.org/findNearestAddress?lat=$lat&lng=$lon" curl -s "$url" I tried escaping the & with \&,...

4. Shell Programming and Scripting

Sending awk variables into curl in a bash script

Hello experts! I have a file1 with the following format (yr,day, month, hour,minute): 201201132435 201202141210 201304132030 201410100110 ... What i want to do is to assign variables and then use them in the curl command to download the text of each event from a web page. What I have...

5. Shell Programming and Scripting

Curl won't complete in a script but does from prompt. Idea?

On RHEL5 from within both a shell (sh->bash) and a csh script curl fails with a "curl: (6) Couldn't resolve host 'application'" error. This is the result whether run as a command at the shell and/or c-shell prompt and curl also fails with the same error when the scripts are "source'd" at the...

6. Shell Programming and Scripting

Curl/http 503 error with bash script

I am trying to use REST API and curl in a bash script to generate a http report. The curl command at the end of the script should generate a html file but instead I get an error "HTTP/1.1 503 Service Unavailable". This is the script #!/bin/bash export...

7. Shell Programming and Scripting

Using curl in bash script

Hello. I have pick up a script from internet to track errors from curl command. #!/bin/bash # URL_TO_TEST="http://www.xxxxxx.yyy" MY_VAR=curl_init("$URL_TO_TEST") ; curl_setopt($MY_VAR, CURLOPT_HEADER, 1); curl_setopt($MY_VAR, CURLOPT_RETURNTRANSFER, 1); curl_setopt($MY_VAR,...

8. Shell Programming and Scripting

Access a complete flow with CURL + bash shell

Hello Experts , I have an use case which needed your help . I have been using google for 2 days buy couldn`t succed , i believe i can get the help here. Here is my use case to run on bash shell 1. Access an URL -- in script , it will be mentioned as inputURL 2. Once i accessed the URL...

9. Shell Programming and Scripting

Curl , download file with user:pass in bash script

Hello, My question is about curl command. (ubuntu14.04) In terminal, I am able to download my mainfile with: curl -u user1:pass1 http://11.22.33.44/******* When I convert it into bash script like this: #!/bin/bash cd /root/scripts computer_ip=11.22.33.44 curl -u $1:$2...

LEARN ABOUT MOJAVE

curlopt_maxfilesize_large

CURLOPT_MAXFILESIZE_LARGE(3)				     curl_easy_setopt options				      CURLOPT_MAXFILESIZE_LARGE(3)

NAME

       CURLOPT_MAXFILESIZE_LARGE - maximum file size allowed to download

SYNOPSIS

       #include <curl/curl.h>

       CURLcode curl_easy_setopt(CURL *handle, CURLOPT_MAXFILESIZE_LARGE,
				 curl_off_t size);

DESCRIPTION

       Pass  a	curl_off_t  as	parameter.  This allows you to specify the maximum size (in bytes) of a file to download. If the file requested is
       found larger than this value, the transfer will not start and CURLE_FILESIZE_EXCEEDED will be returned.

       The file size is not always known prior to download, and for such files this option has no effect even if the file transfer ends  up  being
       larger than this given limit. This concerns both FTP and HTTP transfers.

DEFAULT

       None

PROTOCOLS

       FTP and HTTP

EXAMPLE

       TODO

AVAILABILITY

       Added in 7.11.0

RETURN VALUE

       Returns CURLE_OK if the option is supported, and CURLE_UNKNOWN_OPTION if not.

SEE ALSO

       CURLOPT_MAXFILESIZE(3), CURLOPT_MAX_RECV_SPEED_LARGE(3),

libcurl 7.54.0							 February 03, 2016				      CURLOPT_MAXFILESIZE_LARGE(3)