Sponsored Content
Full Discussion: MD5 File Parsing
Top Forums Shell Programming and Scripting MD5 File Parsing Post 302496372 by moshe88 on Sunday 13th of February 2011 10:47:06 PM
Old 02-13-2011
MD5 File Parsing

Hey Guys.
Here is another issue I'm struggling with: Smilie
I need wget to download only the files that has been changed. I use md5deep to generate the check sum files.

Here is a simple result; I'll call it $string for the example

Code:
0ce73169b069d11e38d6bd78796839c0  md5data
78cb11f2d943b48b8d5a00931cc454b1  md5rss
86ffb6728305730996451e43effa1738  myTextFile.txt
f385d56f26ca3859f48bd450c3e214bf  rss.txt

if I set -- $string, i can access each argument by using $# (where # is 1,2,3,etc..)

so:
echo $1 will print 0ce73169b069d11e38d6bd78796839c0
echo $2 will print md5data

however I'm not sure how to write $# in a for loop that will go through all of them.

So here is the logic of the loop I'm trying to create:

Code:
for I ; 1 to [number of arguments in the file] do; step 2
if $i != $string than download file $(i+1); end if
end for.

Any suggestions (or better ways to check md5 files and download only the files that are different?)


Thanks

Last edited by pludi; 02-14-2011 at 03:32 AM..
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Md5

Does anyone know a scipt that includes MD5. I need to run a script that includes MD5 encryption. Thanks (1 Reply)
Discussion started by: duncang3
1 Replies

2. Shell Programming and Scripting

Finding & Moving Oldest File by Parsing/Sorting Date Info in File Names

I'm trying to write a script that will look in an /exports folder for the oldest export file and move it to a /staging folder. "Oldest" in this case is actually determined by date information embedded in the file names themselves. Also, the script should only move a file from /exports to... (6 Replies)
Discussion started by: nikosey
6 Replies

3. UNIX for Dummies Questions & Answers

Script for parsing details in a log file to a seperate file

Hi Experts, Im a new bee for scripting, I would ned to do the following via linux shell scripting, I have an application which throws a log file, on each action of a particular work with the application, as sson as the action is done, the log file would vanish or stops updating there, the... (2 Replies)
Discussion started by: pingnagan
2 Replies

4. Shell Programming and Scripting

Parsing file, yaml file? Extracting specific sections

Here is a data file, which I believe is in YAML. I am trying to retrieve just the 'addon_domains" section, which doesnt seem to be as easy as I had originally thought. Any help on this would be greatly appreciated!! I have been trying to do this in awk and mostly bash scripting instead of perl... (3 Replies)
Discussion started by: Rhije
3 Replies

5. Shell Programming and Scripting

Parsing of file for Report Generation (String parsing and splitting)

Hey guys, I have this file generated by me... i want to create some HTML output from it. The problem is that i am really confused about how do I go about reading the file. The file is in the following format: TID1 Name1 ATime=xx AResult=yyy AExpected=yyy BTime=xx BResult=yyy... (8 Replies)
Discussion started by: umar.shaikh
8 Replies

6. Linux

Need Help: MD5

I am trying to compare two identical files by using md5 command, but cant get the right command parameters Please help me with any examples. All I want is to know how to compare two identical files which are residing on two different machines in my local network, for example: Host_A -... (6 Replies)
Discussion started by: greenja
6 Replies

7. Shell Programming and Scripting

parsing data from a big file using keys from another smaller file

Hi, I have 2 files format of file 1 is: a1 b2 a2 c2 d1 f3 format of file 2 is (tab delimited): a1 1.2 0.5 0.06 0.7 0.9 1 0.023 a3 0.91 0.007 0.12 0.34 0.45 1 0.7 a2 1.05 2.3 0.25 1 0.9 0.3 0.091 b1 1 5.4 0.3 9.2 0.3 0.2 0.1 b2 3 5 7 0.9 1 9 0 1 b3 0.001 1 2.3 4.6 8.9 10 0 1 0... (10 Replies)
Discussion started by: Lucky Ali
10 Replies

8. Shell Programming and Scripting

Create md5 sums and archive the resulting md5 files

Hello everyone, I am looking to basically creating md5sum files for all iso files in a directory and archive the resulting md5 files into a single archive in that very same directory. I worked out a clumsy solution such as: #find files for which md5sum are to be created and store the... (1 Reply)
Discussion started by: SurfTranquille
1 Replies

9. Shell Programming and Scripting

Apply md5 hash to a field in csv file

I have a .csv file and I want to md5 hash the second column for each row in the file. File is something like data1,foobar1,123,345 data2,foobar2,456,9393 data3,foobar3,1002,10109 Output would be like data1,6c81243028f8e455fa617dd5f0232ce1,123,345... (3 Replies)
Discussion started by: jjwags
3 Replies

10. Shell Programming and Scripting

How to reverse the b64 format(encoded b64(b64(md5($pass)))) into md5 hash format?

I have about 1500 rows (encoded b64(b64(md5($pass))) algorythm) in a file. I would like reverse the b64 into md5 hash format. How could I do that from command line? So I need only the correct md5 hash formats. These row format: 4G5qc2WQzGES6QkWAUgl5w P9tKxonBOg3ymr8vOBLnDA... (1 Reply)
Discussion started by: freeroute
1 Replies
File(3pm)						User Contributed Perl Documentation						 File(3pm)

NAME
Digest::MD5::File - Perl extension for getting MD5 sums for files and urls. SYNOPSIS
use Digest::MD5::File qw(dir_md5_hex file_md5_hex url_md5_hex); my $md5 = Digest::MD5->new; $md5->addpath('/path/to/file'); my $digest = $md5->hexdigest; my $digest = file_md5($file); my $digest = file_md5_hex($file); my $digest = file_md5_base64($file); my $md5 = Digest::MD5->new; $md5->addurl('http://www.tmbg.com/tour.html'); my $digest = $md5->hexdigest; my $digest = url_md5($url); my $digest = url_md5_hex($url); my $digest = url_md5_base64($url); my $md5 = Digest::MD5->new; $md5->adddir('/directory'); my $digest = $md5->hexdigest; my $dir_hashref = dir_md5($dir); my $dir_hashref = dir_md5_hex($dir); my $dir_hashref = dir_md5_base64($dir); DESCRIPTION
Get MD5 sums for files of a given path or content of a given url. EXPORT
None by default. You can export any file_* dir_*, or url_* function and anything Digest::MD5 can export. use Digest::MD5::File qw(md5 md5_hex md5_base64); # 3 Digest::MD5 functions print md5_hex('abc123'), " "; print md5_base64('abc123'), " "; OBJECT METHODS
addpath() my $md5 = Digest::MD5->new; $md5->addpath('/path/to/file.txt') or die "file.txt is not where you said: $!"; or you can add multiple files by specifying an array ref of files: $md5->addpath(@files); adddir() addpath()s each file in a directory recursively. Follows the same rules as the dir_* functions. my $md5 = Digest::MD5->new; $md5->adddir('/home/tmbg/') or die "See warning above to see why I bailed: $!"; addurl() my $md5 = Digest::MD5->new; $md5->addurl('http://www.tmbg.com/tour.html') or die "They Must Be not on tour"; file_* functions Get the digest in variouse formats of $file. If file does not exist or is a directory it croaks (See NOFATALS for more info) my $digest = file_md5($file) or warn "$file failed: $!"; my $digest = file_md5_hex($file) or warn "$file failed: $!"; my $digest = file_md5_base64($file) or warn "$file failed: $!"; dir_* functions Returns a hashref whose keys are files relative to the given path and the values are the MD5 sum of the file or and empty string if a directory. It recurses through the entire depth of the directory. Symlinks to files are just addpath()d and symlinks to directories are followed. my $dir_hashref = dir_md5($dir) or warn "$dir failed: $!"; my $dir_hashref = dir_md5_hex($dir) or warn "$dir failed: $!"; my $dir_hashref = dir_md5_base64($dir) or warn "$dir failed: $!"; url_* functions Get the digest in various formats of the content at $url (Including, if $url points to directory, the directory listing content). Returns undef if url fails (IE if LWP::UserAgent's $res->is_success is false) my $digest = url_md5($url) or warn "$url failed"; my $digest = url_md5_hex($url) or warn "$url failed"; my $digest = url_md5_base64($url) or warn "$url failed"; SPECIAL SETTINGS
BINMODE By default files are opened in binmode. If you do not want to do this you can unset it a variety of ways: use Digest::MD5::File qw(-nobin); or $Digest::MD5::File::BINMODE = 0; or at the function/method level by specifying its value as the second argument: $md5->addpath($file,0); my $digest = file_md5_hex($file,0); UTF8 In some cases you may want to have your data utf8 encoded, you can do this the following ways: use Digest::MD5::File qw(-utf8); or $Digest::MD5::File::UTF8 = 1; or at the function/method level by specifying its value as the third argument for files and second for urls: $md5->addpath($file,$binmode,1); my $digest = file_md5_hex($file,$binmode,1); $md5->addurl($url,1); url_md5_hex($url,1); It use's Encode's encode_utf8() function to do the encoding. So if you do not have Encode (pre 5.7.3) this won't work :) NOFATALS Instead of croaking it will return undef if you set NOFATALS to true. You can do this two ways: $Digest::MD5::File::NOFATALS = 1; or the -nofatals flag: use Digest::MD5::File qw(-nofatals); my $digest = file_md5_hex($file) or die "$file failed"; $! is not set so its not really helpful if you die(). SEE ALSO
Digest::MD5, Encode, LWP::UserAgent AUTHOR
Daniel Muey, <http://drmuey.com/cpan_contact.pl> COPYRIGHT AND LICENSE
Copyright 2005 by Daniel Muey This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself. perl v5.14.2 2012-04-04 File(3pm)
All times are GMT -4. The time now is 09:31 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy