Sponsored Content
Top Forums Shell Programming and Scripting Help with WGET and renaming downloaded files :( Post 302446466 by o0110o on Wednesday 18th of August 2010 08:59:34 PM
Old 08-18-2010
Help with WGET and renaming downloaded files :(

Hi everybody, I would greatly appreciate some expertise in this matter. I am trying find an efficient way to batch download files from a website and rename each file with the url it originated from (from the CLI). (ie. Instead of xyz.zip, the output file would be http://www.abc.com/xyz.zip) A method using WGET is preferable but not absolutely necessary. I'm just starting to get comfortable with the command line; in part because of some of the great help I've gotten here before, so once again I'm back looking for some help from this excellent community Smilie

Thanks

Last edited by o0110o; 08-18-2010 at 10:01 PM.. Reason: Revision
 

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

question regarding ftp. Files downloaded are of size Zero.

I need to download some files from a remote server using ftp. I have ftp'd into the site. I then do an mget * to retrieve all of the data files. Everything seems to proceed normally and I am given feedback that the files were downloaded. Now if I go into the DOS Shell or Windows explorer, it list... (5 Replies)
Discussion started by: ralphisnow
5 Replies

2. Shell Programming and Scripting

how to limit files downloaded by wget

I am trying to download a page and retrieve only wav and mp3 files via wget. the website is: Alarm Sounds | Free Sound Effects | Alarm Sound Clips | Sound Bites my command is : wget -rl 2 -e robots=off -A wav,mp3 http://soundbible.com/tags-alarm.html When not using the -A wav,mp3... (2 Replies)
Discussion started by: Narnie
2 Replies

3. Web Development

php files are downloaded

Hello, I have setup Cherokee web server and php 5.2 in Opensolaris zone. Problem is that all .php files are downloaded from web server and not served when I use IP address instead of DNS name in web brovser. Example: test.mydomain.com <-- php works 192.168.0.10/index.php <--... (3 Replies)
Discussion started by: kreno
3 Replies

4. Shell Programming and Scripting

Extract urls from index.html downloaded using wget

Hi, I need to basically get a list of all the tarballs located at uri I am currently doing a wget on urito get the index.html page Now this index page contains the list of uris that I want to use in my bash script. can someone please guide me ,. I am new to Linux and shell scripting. ... (5 Replies)
Discussion started by: mnanavati
5 Replies

5. Shell Programming and Scripting

renaming files or adding a name in the beginning of all files in a folder

Hi All I have a folder that contains hundreds of file with a names 3.msa 4.msa 21.msa 6.msa 345.msa 456.msa 98.msa ... ... ... I need rename each of this file by adding "core_" in the begiining of each file such as core_3.msa core_4.msa core_21.msa (4 Replies)
Discussion started by: Lucky Ali
4 Replies

6. Shell Programming and Scripting

Specific image to be downloaded with wget

Hello All, I have gone through Google and came to know that we can download images from a site using wget. Now I am been asked to check whether an image is populated in a site or not. If yes, please send that image to an address as an attachment.. Say for example, the site is Wiki -... (6 Replies)
Discussion started by: sathyaonnuix
6 Replies

7. Shell Programming and Scripting

BASH scripting - Preventing wget messed downloaded files

hello. How can I detect within script, that the downloaded file had not a correct size. linux:~ # wget --limit-rate=20k --ignore-length -O /Software_Downloaded/MULTIMEDIA_ADDON/skype-4.1.0.20-suse.i586.rpm ... (6 Replies)
Discussion started by: jcdole
6 Replies

8. Shell Programming and Scripting

For loop till the files downloaded

Need assistance in writing a for loop script or any looping method. Below is the code where i can get all the files from the URL . There are about 80 files in the URL .Every day the files get updated . Script that i wanted is the loop must keep on running till it gets 80 files. It matches the count... (5 Replies)
Discussion started by: ajayram_arya
5 Replies

9. Shell Programming and Scripting

Renaming multiple files in sftp server in a get files script

Hi, In sftp script to get files, I have to rename all the files which I am picking. Rename command does not work here. Is there any way to do this? I am using #!/bin/ksh For eg: sftp user@host <<EOF cd /path get *.txt rename *.txt *.txt.done ... (7 Replies)
Discussion started by: jhilmil
7 Replies

10. Shell Programming and Scripting

Deleting multiple files off an ftp server once they have been downloaded

Hello, I have a server that I have to ftp files off and they all start SGRD and are followed by 6 numbers. SGRD000001 SGRD000002 SGRD000003 The script I have will run every 10 mins to pick up files as new ones will be coming in all the time and what I want to do is delete the files I have... (7 Replies)
Discussion started by: sph90457
7 Replies
vfs-filesystems(n)														vfs-filesystems(n)

__________________________________________________________________________________________________________________________________________________

NAME
vfs-filesystems - Filesystems provided by tclvfs SYNOPSIS
package require Tcl 8.4 package require vfs ?1.2.1? package require vfs::zip ?1.0? package require vfs::mk4 ?1.6? package require vfs::tar ?0.9? package require vfs::ftp ?1.0? package require vfs::ns ?1.0? package require vfs::webdav ?0.1? package require vfs::http ?0.5? package require vfs::urltype ?1.0? vfs::zip::Mount path to vfs::mk4::Mount path to vfs::tar::Mount path to vfs::ftp::Mount path to vfs::ns::Mount path to vfs::webdav::Mount path to vfs::http::Mount path to vfs::urltype::Mount path to _________________________________________________________________ DESCRIPTION
The package vfs provides not only the means to implement a virtual filesystem at the tcl level, but also a number of ready to be used filesystems as well. Each of these filesystem exists as its own package, and can be accessed through package require vfs::NAME. The whole set of these virtual filesystems is known informally as 'vfslib'. SUPPORTED VFS TYPES
Currently supported are ftp, tar, http, zip, mk4, ns, and webdav. In addition there is the ability to mount any 'urltype' as a new direc- tory, provided an appropriate vfs is supported. This means that you can treat urls based on the schemes ftp://, http:// and file:// as files. To do this, simply evaluate the command vfs::urltype::Mount ftp for example. Any access inside the new volume will result in an attempt to require a package through package require vfs::${type}, which must therefore exist, or errors will be thrown. If a filesystem is loaded, use the associated command listed below to mount a source for that filesystem as a tcl directory. vfs::zip::Mount path to Mount the zip file path as directory to. vfs::mk4::Mount path to Mount the metakit database file file path as directory to. vfs::tar::Mount path to Mount the tar file path as directory to. vfs::ftp::Mount path to Mount the ftp url path as directory to. vfs::ns::Mount path to Mount the tcl namespace path as directory to. vfs::webdav::Mount path to Mount the webdav url path as directory to. vfs::http::Mount path to Mount the http url path as directory to. vfs::urltype::Mount path to Mount the url path, of type urltype as directory to. LIMITATIONS
Most of the vfs types listed above have not been very well debugged as yet. Please test them! SEE ALSO
vfs, vfs-fsapi KEYWORDS
file, filesystem, ftp, http, metakit, namespace, tar, vfs, vfslib, webdav, zip COPYRIGHT
Copyright (c) 2001-2003 Vince Darley <vincentdarley@users.sourceforge.net> Copyright (c) 2003 Andreas Kupries <andreas_kupries@users.sourceforge.net> Tcl-level Virtual Filesystems 1.0 vfs-filesystems(n)
All times are GMT -4. The time now is 08:49 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy