Sponsored Content
Top Forums Shell Programming and Scripting Using md5sum to name file based on URL Post 302578950 by The undertaker on Saturday 3rd of December 2011 09:56:55 AM
Old 12-03-2011
Using md5sum to name file based on URL

I am trying to download a file and make the filename of the file be the md5sum of the URL. I know to use wgets to download the file but I do not know how to do the rest...any help would be appreciated.
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

the file: MD5SUM

i downloaded a Linux distribution from a FTP site today, and i found there is a file named MD5SUM in the same directory, with the following contents: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 c9a4d963a49e384e10dec9c2bd49ad73 valhalla-SRPMS-disc1.iso 41b03d068e84d2a17147aa27e704f79b ... (1 Reply)
Discussion started by: samprax
1 Replies

2. Shell Programming and Scripting

Restart script based on MD5sum

Ok, I run a small script, that restarts a perl script when it fails (it's a very unreliable perl script, but I can't change it, it's crucial, and I don't know perl) It outputs data into a logfile. Unfortunately, it also regularly hangs. This is a major problem because if it hangs, no data is... (6 Replies)
Discussion started by: Bakes
6 Replies

3. Shell Programming and Scripting

Duplicate file remover using md5sum--good enough?

This is not a typical question. I have a fully working script but I'm interested in optimizing it. I frequently back up photos and movies from my digital camera and cell phone to both my home/work desktops, laptops, wife's netbook, and my home NAS and often end up with multiple versions of the... (2 Replies)
Discussion started by: Michael Stora
2 Replies

4. Shell Programming and Scripting

Script to check MD5SUM on file

Hi, I currently have a shell script that takes an RPM and scp's it to a set of remote servers and installs it. What I would like to be able to do is make the script get the md5sum of the RPM locally (so get the md5sum of the rpm from where im running the script) and then scp the rpm to the... (0 Replies)
Discussion started by: tb1986
0 Replies

5. Web Development

Regex to rewrite URL to another URL based on HTTP_HOST?

I am trying to find a way to test some code, but I need to rewrite a specific URL only from a specific HTTP_HOST The call goes out to http://SUB.DOMAIN.COM/showAssignment/7bde10b45efdd7a97629ef2fe01f7303/jsmodule/Nevow.Athena The ID in the middle is always random due to the cookie. I... (5 Replies)
Discussion started by: EXT3FSCK
5 Replies

6. Shell Programming and Scripting

md5sum on a file with backslash in its name

Hi there, I found something very weird! Should I report that as a bug or is it me misusing the command? I've got a file with a backslash in its name. I know it's a horrible policy but it's not me. The file came from a mac computer because this is a backup server. Anyway, when using... (8 Replies)
Discussion started by: chebarbudo
8 Replies

7. Shell Programming and Scripting

API Based URL Parsing

Hi Friends, We have a situation where we need to pass API based URL which will return XML response ,and I need to convert those XML response to a delimited flat file. url.txt -- will have http://xyz.com/beta/xxx.xml?isDRR=True&city=London&province=England... (1 Reply)
Discussion started by: rakesh5300
1 Replies

8. Web Development

Mod_rewrite - URL rewrite based upon HTTP_REFERER

Hello, I have added following rewrite cond and rewrite rules but it does not work. RewriteCond %{HTTP_REFERER} ^http://192\.168\.1\.150/categories/.*$ RewriteRule ^(.*)$ http://www.blahblah.com/ When I hit url : http://192.168.1.150/categories/881-Goes?page=7 in my browser - it... (2 Replies)
Discussion started by: ashokvpp
2 Replies

9. Shell Programming and Scripting

Reading URL using Mechanize and dump all the contents of the URL to a file

Hello, Am very new to perl , please help me here !! I need help in reading a URL from command line using PERL:: Mechanize and needs all the contents from the URL to get into a file. below is the script which i have written so far , #!/usr/bin/perl use LWP::UserAgent; use... (2 Replies)
Discussion started by: scott_cog
2 Replies

10. Shell Programming and Scripting

Removing md5sum lines stored in text file

Hello. I'm writing a script where every file you create will generate a md5sum and store it into a text file. Say I create 2 files, it'll look like this in the text file: d41d8cd98f00b204e9800998ecf8427e /helloworld/saystheman d41d8cd98f00b204e9800998ecf8427e /helloworld/test I... (3 Replies)
Discussion started by: batarangs_
3 Replies
zsyncmake(1)							   File Transfer						      zsyncmake(1)

NAME
zsyncmake - Build control file for zsync(1) SYNTAX
zsyncmake [ { -z | -Z } ] [ -e ] [ -C ] [ -u url ] [ -U url ] [ -b blocksize ] [ -o outfile ] [ -f targetfilename ] [ -v ] filename zsync -V DESCRIPTION
Constructs a metafile for the zsync client program to use to perform partial file downloads. filename is the file that users wish to down- loads; zsyncmake constructs the appropriate metafile and writes filename.zsync in the current directory. zsync will need at least one URL from which to download the file content. If the .zsync will be in the same directory as the file to down- load, you can accept the default - zsync includes a relative URL in the control file. If not, use the -u option to specify the URL. You should also specify a URL for the uncompressed content with -U if available, as zsync can make use of this for more efficient downloads sometimes. (You can edit the .zsync file and add these afterwards - it has a simple key: value format in the header - but I suggest you only do this once you are familiar with the tool.) Note that zsyncmake itself does not (currently) verify the URLs or download any data, you must provide the file data locally and check the URLs yourself. OPTIONS
-b blocksize Specify the blocksize to the underlying rsync algorithm. A smaller blocksize may be more efficient for files where there are likely to be lots of small, scattered changes between downloads; a larger blocksize is more efficient for files with fewer or less scat- tered changes. This blocksize must be a power of two. If not specified, zsyncmake chooses one which it thinks is best for this file (currently either 2048 or 4096 depending on file size) - so normally tyou should not need to override the default. -C Tells zsyncmake not to generate any instructions in the .zsync telling the client to compress the data it receives. This is implied by -z, but this option is here in case you compress a file yourself only for the transfer, but want the client to end up with the uncompressed file (e.g. you are transferring an ISO, which is held compressed on the server, but which the client cannot use unless it is uncompressed). Without -C, zsyncmake will produce directions for the client to compress the file it receives where appropri- ate; -C is here so you can stop it telling the client to do that. -e Tells zsyncmake that the client must be able to receive the exact file that was supplied. Without this option, zsyncmake only gives a weaker guarantee - that the client will receive the data it contains (e.g. it might transfer the uncompressed version of a .gz to the client). Note that this still doesn't guarantee that the client will get it - the client could ignore the directives in the zsync file, or might be incapable of exactly reproducing the compression used. But with -e you know that zsyncmake has made it pos- sible to get the exact data - it will exit with an error if it cannot. -f filename Set the filename to include in the output file (this is what the file will be called when a user finished downloading it). -o outputfile Override the default output file name. -u url Specifies the URL from which users can download the content of the supplied file. Users need the control file in order to find out what parts of the file they already have, and they need the URLs to retrieve the parts of the file that they don't already have. You can specify multiple URLs by specifying -u multiple times. If not specified, zsync assumes that the file and the .zsync will reside in the same public directory, and includes a single relative URL. -U url Specifies a URL corresponding to the decompressed content of the file (only applicable if it is a gzip file). zsync can sometimes download more efficiently from the uncompressed data than from the compressed data - it will take advantage of this if available. If no URLs are specifies, zsync looks for a file without the .gz extension and assumes that this will be in the same public dir as the .zsync, and includes a relative URL to it. -v Enable verbose messages. -V Prints the version of zsync. -z Compress the file to transfer. Note that this overwrites any file called filename.gz without warning (if you don't give a filename, e.g. because you are reading from stdin, then zsync will use the name supplied with -f, or as a last fallback, zsync-target.gz). zsync can work with compressed data, and, in most cases where the data is not already compressed, it is more efficient to compress it first. While you can just compress the file to transfer with gzip, if you use this option then zsyncmake will compress the file for you, producing a .gz file which is optimised for zsync. This can be 30% more efficient at download time than compressing with gzip --best - but the compressed file will not be as small at that produced by gzip. -Z zsyncmake automatically looks inside gzip compressed files and exports the underlying, uncompressed data to the zsyncmake file. In testing this has proved to provide greater download efficiency. -Z overrides the default behaviour and treats gzip files as just binary data. Use this if it is essential that the user receives the compressed data (for instance because a cryptographic signature is available only for the compressed data). zsync is typically no use if you specify -Z, unless the gzip file was compressed with the special --rsync option to make it friendly to differential transfers. EXAMPLES
zsyncmake -C -u http://ftp.uk.debian.org/debian/dists/sarge/main/binary-i386/Packages.gz Packages.gz Note use of -C to save the client compressing the file on receipt; the Debian package system uses the file uncompressed. zsyncmake -z my-subversion-dump In this case there is a large, compressible file to transfer. This creates a gzipped version of the file (optimised for zsync), and a .zsync file. A URL is automatically added assuming that the two files will be served from the same directory on the web server. zsyncmake -e -u http://www.mirrorservice.org/sites/ftp.freebsd.org/pub/FreeBSD/ports/distfiles/zsync-0.2.2.tar.gz zsync-0.2.2.tar.gz This creates a zsync referring to the named source tarball, which the client should download from the given URL. This example is for down- loading a source tarball for a FreeBSD port, hence -e is specified so the client will be able to match its md5sum. AUTHORS
Colin Phipps <cph@moria.org.uk> SEE ALSO
zsync(1) Colin Phipps 0.6.2 zsyncmake(1)
All times are GMT -4. The time now is 03:54 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy