08-18-2010
Help with WGET and renaming downloaded files :(
Hi everybody, I would greatly appreciate some expertise in this matter. I am trying find an efficient way to batch download files from a website and rename each file with the url it originated from (from the CLI). (ie. Instead of xyz.zip, the output file would be
http://www.abc.com/xyz.zip) A method using WGET is preferable but not absolutely necessary. I'm just starting to get comfortable with the command line; in part because of some of the great help I've gotten here before, so once again I'm back looking for some help from this excellent community
Thanks
Last edited by o0110o; 08-18-2010 at 10:01 PM..
Reason: Revision
10 More Discussions You Might Find Interesting
1. UNIX for Advanced & Expert Users
I need to download some files from a remote server using ftp. I have ftp'd into the site. I then do an mget * to retrieve all of the data files. Everything seems to proceed normally and I am given feedback that the files were downloaded. Now if I go into the DOS Shell or Windows explorer, it list... (5 Replies)
Discussion started by: ralphisnow
5 Replies
2. Shell Programming and Scripting
I am trying to download a page and retrieve only wav and mp3 files via wget.
the website is:
Alarm Sounds | Free Sound Effects | Alarm Sound Clips | Sound Bites
my command is :
wget -rl 2 -e robots=off -A wav,mp3 http://soundbible.com/tags-alarm.html
When not using the -A wav,mp3... (2 Replies)
Discussion started by: Narnie
2 Replies
3. Web Development
Hello,
I have setup Cherokee web server and php 5.2 in Opensolaris zone. Problem is that all .php files are downloaded from web server and not served when I use IP address instead of DNS name in web brovser.
Example: test.mydomain.com <-- php works
192.168.0.10/index.php <--... (3 Replies)
Discussion started by: kreno
3 Replies
4. Shell Programming and Scripting
Hi,
I need to basically get a list of all the tarballs located at uri
I am currently doing a wget on urito get the index.html page
Now this index page contains the list of uris that I want to use in my bash script.
can someone please guide me ,.
I am new to Linux and shell scripting.
... (5 Replies)
Discussion started by: mnanavati
5 Replies
5. Shell Programming and Scripting
Hi All
I have a folder that contains hundreds of file with a names
3.msa
4.msa
21.msa
6.msa
345.msa
456.msa
98.msa
...
...
...
I need rename each of this file by adding "core_" in the begiining of each file such as
core_3.msa
core_4.msa
core_21.msa (4 Replies)
Discussion started by: Lucky Ali
4 Replies
6. Shell Programming and Scripting
Hello All,
I have gone through Google and came to know that we can download images from a site using wget.
Now I am been asked to check whether an image is populated in a site or not. If yes, please send that image to an address as an attachment..
Say for example, the site is Wiki -... (6 Replies)
Discussion started by: sathyaonnuix
6 Replies
7. Shell Programming and Scripting
hello.
How can I detect within script, that the downloaded file had not a correct size.
linux:~ # wget --limit-rate=20k --ignore-length -O /Software_Downloaded/MULTIMEDIA_ADDON/skype-4.1.0.20-suse.i586.rpm ... (6 Replies)
Discussion started by: jcdole
6 Replies
8. Shell Programming and Scripting
Need assistance in writing a for loop script or any looping method. Below is the code where i can get all the files from the URL . There are about 80 files in the URL .Every day the files get updated . Script that i wanted is the loop must keep on running till it gets 80 files. It matches the count... (5 Replies)
Discussion started by: ajayram_arya
5 Replies
9. Shell Programming and Scripting
Hi,
In sftp script to get files, I have to rename all the files which I am picking. Rename command does not work here. Is there any way to do this?
I am using #!/bin/ksh
For eg: sftp user@host <<EOF
cd /path
get *.txt
rename *.txt *.txt.done
... (7 Replies)
Discussion started by: jhilmil
7 Replies
10. Shell Programming and Scripting
Hello,
I have a server that I have to ftp files off and they all start SGRD and are followed by 6 numbers.
SGRD000001
SGRD000002
SGRD000003
The script I have will run every 10 mins to pick up files as new ones will be coming in all the time and what I want to do is delete the files I have... (7 Replies)
Discussion started by: sph90457
7 Replies
LEARN ABOUT MOJAVE
uri::url5.18
URI::URL(3) User Contributed Perl Documentation URI::URL(3)
NAME
URI::URL - Uniform Resource Locators
SYNOPSIS
$u1 = URI::URL->new($str, $base);
$u2 = $u1->abs;
DESCRIPTION
This module is provided for backwards compatibility with modules that depend on the interface provided by the "URI::URL" class that used to
be distributed with the libwww-perl library.
The following differences exist compared to the "URI" class interface:
o The URI::URL module exports the url() function as an alternate constructor interface.
o The constructor takes an optional $base argument. The "URI::URL" class is a subclass of "URI::WithBase".
o The URI::URL->newlocal class method is the same as URI::file->new_abs.
o URI::URL::strict(1)
o $url->print_on method
o $url->crack method
o $url->full_path: same as ($uri->abs_path || "/")
o $url->netloc: same as $uri->authority
o $url->epath, $url->equery: same as $uri->path, $uri->query
o $url->path and $url->query pass unescaped strings.
o $url->path_components: same as $uri->path_segments (if you don't consider path segment parameters)
o $url->params and $url->eparams methods
o $url->base method. See URI::WithBase.
o $url->abs and $url->rel have an optional $base argument. See URI::WithBase.
o $url->frag: same as $uri->fragment
o $url->keywords: same as $uri->query_keywords
o $url->localpath and friends map to $uri->file.
o $url->address and $url->encoded822addr: same as $uri->to for mailto URI
o $url->groupart method for news URI
o $url->article: same as $uri->message
SEE ALSO
URI, URI::WithBase
COPYRIGHT
Copyright 1998-2000 Gisle Aas.
perl v5.18.2 2012-02-11 URI::URL(3)