I have an alias already in my .bash_profile to download files using cUrl's -o (output to file, user provides the file name) option. I find I'm using it quite a bit, so I wanted to write a script to run "curl -o", taking the necessary inputs - file name and URL from which to download - and then returning to a "parked" state like some python scripts I've seen do, ready to take in another filename and URL. The script should also, on reading a 'buzzword' (not "quit" -- that's too literal and it might be confused by the shell), put up one last echo and exit as normal.
Then my imagination kicked in and I started to wonder, also, if it wouldn't be possible to have the script log its own activity to an external file as soon as a download is complete, naming the file with the date and time created along with its own name or some generic 'tag' such as "curldl-activity-021810-223516H.txt", and keep logging to the same file until the exit word or phrase was entered.
I have a pretty firm idea of how to write the part of the script that takes in the name and URL and invokes cUrl to do the downloading. That would probably best be handled in a function, I'm thinking. I'm a bit sketchy on setting the conditional for an exit buzzword, and while I've written scripts and executed commands to send data to outside files, where to put such a command in this script is something I'm sure I've learned somewhere but can't recall, at any rate.
So again, here's me picking the brains of the gurus and asking for advice.
Here is a very rough outline of one way to do it. It works but needs a lot of polishing and may not be the method you want to use. I didn't fool around with the logfile name or any error checking.
Code:
while true
do echo "Gimme me something to download"
read LINK
[ "$LINK" = "stop" ] && exec echo "I'm exiting"
echo "File name?"
read NAME
curl -o "$NAME" "$LINK" && echo "$NAME" "$LINK" > file.log
done
"while true; do [stuff]; done" is an infinite loop. "read" just sits there and waits for you to input something, which it then puts in the LINK variable. If that something is "stop," it will say "Im exiting" and exit. The exec command is what makes it exit in this example, it runs one command then exits. For anything except "stop," it asks for a file name, then curl gets the link and puts it in that file. If curl succeeds, it echos the name and link to a logfile.
edit to add: I'm certainly not a guru and this is just a rough example. Don't take it as The Way Things Should Be Done.
I guessed that you folks would want some "code o' my own," so I wrote some.
Did some double-checking on the rules of parameter extraction -- last time I did anything of the kind was in AppleScript three or four years ago.
So here's my script as it stands now. The echos about a correctly-formatted web address are the ones I intend to replace with the cUrl commands.
Code:
#!/bin/bash
function whatchagot() {
echo -ne "Where is the file coming from?\n"
read stringZ
dimple=$(echo ${stringZ:0:5})
if [ $dimple != "http:" ]
then
echo -ne "That was not a web address.\n"
echo -ne "I can't fill in your blanks like a web browser can, you know!\n"
else
echo -ne "Very good.\n"
echo -ne "You know what a web address is!\n"
fi
}
echo -ne "What is your file to be called?\n"
read theName
if [ $theName != "done" ]
then
whatchagot
else
echo -ne "Alright. I quit!\n"
fi
I'm taking this one step at a time. I have the function ready, allowing for some substitution of commands. Thanks for the "while true" hint. I'll add your && echo "$NAME" "$LINK" > file.log code later on.
BZT
---------- Post updated 27th Feb 2010 at 13:54 ---------- Previous update was 26th Feb 2010 at 22:42 ----------
Still a little wary of applying the "while true" loop, I made some modifications to the script as last posted, dressing up the output to the log file and making sure that the exit commands were working.
Code:
#!/bin/bash
function whatchagot() {
thisFile=$theName
echo -ne "Where is the file?\n"
read thisSource
dimple=$(echo ${thisSource:0:5})
if [ $dimple != "http:" ]
then
echo -ne "I'm exiting.\n"
else
But by avoiding the "while true" loop to this point, I may have hung myself by my own petard, as now, predictably, every time the script is run it creates a new log file with a new date & time as part of the name.
Code:
logtime=$(date "+%d.%m.%Y-%H.%M")
logname="curlactivity-$logtime.log"
touch $logname
echo -ne "What is your file to be called?\n"
read theName
if [ $theName != "done" ]
then
whatchagot
else
echo -ne "Alright. I quit!\n"
fi
I like the kind of output I'm getting, nicely formatted etc.:
Code:
27.02.2010 @ 13:37:54: tuxubuntu-cute.jpg came from http://www.cogite.info/wallpaper/ and was called Christmas_Tux_Ubuntu_1440-900.jpg
But it looks like I'll have to "dive into the while pool" to make these into real log files instead of "results" redirects with a single string of output to them. Which is not what I meant for them to be.
Would anyone advise abandoning the function part and just bulldozing through, beginning to end, with what I have now? Does that tend to work better when while loops are applied to scripts like this? Or am I asking two incompatible questions?
BZT
My complete script as of this post time:
Code:
#!/bin/bash
function whatchagot() {
thisFile=$theName
echo -ne "Where is the file?\n"
read thisSource
dimple=$(echo ${thisSource:0:5})
if [ $dimple != "http:" ]
then
echo -ne "I'm exiting.\n"
else
GRABTIME=$(date "+%d.%m.%Y @ %H:%M:%S")
thisPlace=`echo $thisSource | rev | cut -d / -f2- | rev`
thisName=`echo ${thisSource##*/}`
curl -o "$thisFile" "$thisSource" && echo -ne "$GRABTIME": "$thisFile" came from "$thisPlace/" and was called "$thisName\n">> $logname
fi
}
logtime=$(date "+%d.%m.%Y-%H.%M")
logname="curlactivity-$logtime.log"
touch $logname
echo -ne "What is your file to be called?\n"
read theName
if [ $theName != "done" ]
then
whatchagot
else
echo -ne "Alright. I quit!\n"
fi
Last edited by SilversleevesX; 02-26-2010 at 11:46 PM..
Reason: Didn't fully read the reply I was replying to.
I "dove in" and put the while true where it works. I eliminated the function, but added a timestamp as an echo redirect to the log file, just so I'd know when each log was started. So if anyone would like to use this script with their cUrl install (I'm using curl 7.20.0 as I write this), feel free to download the attachment.
Hello,
My question is about curl command. (ubuntu14.04)
In terminal, I am able to download my mainfile with:
curl -u user1:pass1 http://11.22.33.44/*******
When I convert it into bash script like this:
#!/bin/bash
cd /root/scripts
computer_ip=11.22.33.44
curl -u $1:$2... (8 Replies)
Hello Experts ,
I have an use case which needed your help . I have been using google for 2 days buy couldn`t succed , i believe i can get the help here.
Here is my use case to run on bash shell
1. Access an URL -- in script , it will be mentioned as inputURL
2. Once i accessed the URL... (1 Reply)
Hello.
I have pick up a script from internet to track errors from curl command.
#!/bin/bash
#
URL_TO_TEST="http://www.xxxxxx.yyy"
MY_VAR=curl_init("$URL_TO_TEST") ;
curl_setopt($MY_VAR, CURLOPT_HEADER, 1);
curl_setopt($MY_VAR, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($MY_VAR,... (2 Replies)
I am trying to use REST API and curl in a bash script to generate a http report. The curl command at the end of the script should generate a html file but instead I get an error "HTTP/1.1 503 Service Unavailable".
This is the script
#!/bin/bash
export... (7 Replies)
On RHEL5 from within both a shell (sh->bash) and a csh script curl fails with a "curl: (6) Couldn't resolve host 'application'" error.
This is the result whether run as a command at the shell and/or c-shell prompt and curl also fails with the same error when the scripts are "source'd" at the... (5 Replies)
Hello experts!
I have a file1 with the following format (yr,day, month, hour,minute):
201201132435
201202141210
201304132030
201410100110
...
What i want to do is to assign variables and then use them in the curl command to download the text of each event from a web page.
What I have... (6 Replies)
I'm running a curl command in bash, but the & in the middle causes the second half of the line to run in the background, here's what I'm trying to do:
lat="37.451"
lon="-122.18"
url="http://ws.geonames.org/findNearestAddress?lat=$lat&lng=$lon"
curl -s "$url"
I tried escaping the & with \&,... (4 Replies)
I had the idea come into my head that it would be good to have a single command that gave file type, file size, last modification date, and owner/group information, like what one would see in a GUI "Properties" dialog, but all in a terminal window. In my opinion, these statistics about a file... (5 Replies)
Keeping in mind that I'm relatively comfortable with programming in general but very new to unix and korn/bourne shell scripts..
I'm using awk on a CSV file, and then performing calculations and operations on specific fields within specific records. The CSV file I'm working with has about 600... (2 Replies)
