08-18-2010
Help with WGET and renaming downloaded files :(
Hi everybody, I would greatly appreciate some expertise in this matter. I am trying find an efficient way to batch download files from a website and rename each file with the url it originated from (from the CLI). (ie. Instead of xyz.zip, the output file would be
http://www.abc.com/xyz.zip) A method using WGET is preferable but not absolutely necessary. I'm just starting to get comfortable with the command line; in part because of some of the great help I've gotten here before, so once again I'm back looking for some help from this excellent community
Thanks
Last edited by o0110o; 08-18-2010 at 10:01 PM..
Reason: Revision
10 More Discussions You Might Find Interesting
1. UNIX for Advanced & Expert Users
I need to download some files from a remote server using ftp. I have ftp'd into the site. I then do an mget * to retrieve all of the data files. Everything seems to proceed normally and I am given feedback that the files were downloaded. Now if I go into the DOS Shell or Windows explorer, it list... (5 Replies)
Discussion started by: ralphisnow
5 Replies
2. Shell Programming and Scripting
I am trying to download a page and retrieve only wav and mp3 files via wget.
the website is:
Alarm Sounds | Free Sound Effects | Alarm Sound Clips | Sound Bites
my command is :
wget -rl 2 -e robots=off -A wav,mp3 http://soundbible.com/tags-alarm.html
When not using the -A wav,mp3... (2 Replies)
Discussion started by: Narnie
2 Replies
3. Web Development
Hello,
I have setup Cherokee web server and php 5.2 in Opensolaris zone. Problem is that all .php files are downloaded from web server and not served when I use IP address instead of DNS name in web brovser.
Example: test.mydomain.com <-- php works
192.168.0.10/index.php <--... (3 Replies)
Discussion started by: kreno
3 Replies
4. Shell Programming and Scripting
Hi,
I need to basically get a list of all the tarballs located at uri
I am currently doing a wget on urito get the index.html page
Now this index page contains the list of uris that I want to use in my bash script.
can someone please guide me ,.
I am new to Linux and shell scripting.
... (5 Replies)
Discussion started by: mnanavati
5 Replies
5. Shell Programming and Scripting
Hi All
I have a folder that contains hundreds of file with a names
3.msa
4.msa
21.msa
6.msa
345.msa
456.msa
98.msa
...
...
...
I need rename each of this file by adding "core_" in the begiining of each file such as
core_3.msa
core_4.msa
core_21.msa (4 Replies)
Discussion started by: Lucky Ali
4 Replies
6. Shell Programming and Scripting
Hello All,
I have gone through Google and came to know that we can download images from a site using wget.
Now I am been asked to check whether an image is populated in a site or not. If yes, please send that image to an address as an attachment..
Say for example, the site is Wiki -... (6 Replies)
Discussion started by: sathyaonnuix
6 Replies
7. Shell Programming and Scripting
hello.
How can I detect within script, that the downloaded file had not a correct size.
linux:~ # wget --limit-rate=20k --ignore-length -O /Software_Downloaded/MULTIMEDIA_ADDON/skype-4.1.0.20-suse.i586.rpm ... (6 Replies)
Discussion started by: jcdole
6 Replies
8. Shell Programming and Scripting
Need assistance in writing a for loop script or any looping method. Below is the code where i can get all the files from the URL . There are about 80 files in the URL .Every day the files get updated . Script that i wanted is the loop must keep on running till it gets 80 files. It matches the count... (5 Replies)
Discussion started by: ajayram_arya
5 Replies
9. Shell Programming and Scripting
Hi,
In sftp script to get files, I have to rename all the files which I am picking. Rename command does not work here. Is there any way to do this?
I am using #!/bin/ksh
For eg: sftp user@host <<EOF
cd /path
get *.txt
rename *.txt *.txt.done
... (7 Replies)
Discussion started by: jhilmil
7 Replies
10. Shell Programming and Scripting
Hello,
I have a server that I have to ftp files off and they all start SGRD and are followed by 6 numbers.
SGRD000001
SGRD000002
SGRD000003
The script I have will run every 10 mins to pick up files as new ones will be coming in all the time and what I want to do is delete the files I have... (7 Replies)
Discussion started by: sph90457
7 Replies
LEARN ABOUT OSX
vfs-filesystems
vfs-filesystems(n) vfs-filesystems(n)
__________________________________________________________________________________________________________________________________________________
NAME
vfs-filesystems - Filesystems provided by tclvfs
SYNOPSIS
package require Tcl 8.4
package require vfs ?1.2.1?
package require vfs::zip ?1.0?
package require vfs::mk4 ?1.6?
package require vfs::tar ?0.9?
package require vfs::ftp ?1.0?
package require vfs::ns ?1.0?
package require vfs::webdav ?0.1?
package require vfs::http ?0.5?
package require vfs::urltype ?1.0?
vfs::zip::Mount path to
vfs::mk4::Mount path to
vfs::tar::Mount path to
vfs::ftp::Mount path to
vfs::ns::Mount path to
vfs::webdav::Mount path to
vfs::http::Mount path to
vfs::urltype::Mount path to
_________________________________________________________________
DESCRIPTION
The package vfs provides not only the means to implement a virtual filesystem at the tcl level, but also a number of ready to be used
filesystems as well.
Each of these filesystem exists as its own package, and can be accessed through package require vfs::NAME. The whole set of these virtual
filesystems is known informally as 'vfslib'.
SUPPORTED VFS TYPES
Currently supported are ftp, tar, http, zip, mk4, ns, and webdav. In addition there is the ability to mount any 'urltype' as a new direc-
tory, provided an appropriate vfs is supported. This means that you can treat urls based on the schemes ftp://, http:// and file:// as
files. To do this, simply evaluate the command vfs::urltype::Mount ftp for example. Any access inside the new volume will result in an
attempt to require a package through package require vfs::${type}, which must therefore exist, or errors will be thrown. If a filesystem
is loaded, use the associated command listed below to mount a source for that filesystem as a tcl directory.
vfs::zip::Mount path to
Mount the zip file path as directory to.
vfs::mk4::Mount path to
Mount the metakit database file file path as directory to.
vfs::tar::Mount path to
Mount the tar file path as directory to.
vfs::ftp::Mount path to
Mount the ftp url path as directory to.
vfs::ns::Mount path to
Mount the tcl namespace path as directory to.
vfs::webdav::Mount path to
Mount the webdav url path as directory to.
vfs::http::Mount path to
Mount the http url path as directory to.
vfs::urltype::Mount path to
Mount the url path, of type urltype as directory to.
LIMITATIONS
Most of the vfs types listed above have not been very well debugged as yet. Please test them!
SEE ALSO
vfs, vfs-fsapi
KEYWORDS
file, filesystem, ftp, http, metakit, namespace, tar, vfs, vfslib, webdav, zip
COPYRIGHT
Copyright (c) 2001-2003 Vince Darley <vincentdarley@users.sourceforge.net>
Copyright (c) 2003 Andreas Kupries <andreas_kupries@users.sourceforge.net>
Tcl-level Virtual Filesystems 1.0 vfs-filesystems(n)