Sponsored Content
Top Forums Shell Programming and Scripting Checking MD5 Hashes on a folder tree Post 303022597 by Pollardd on Monday 3rd of September 2018 11:42:50 PM
Old 09-04-2018
Checking MD5 Hashes on a folder tree

Hi Guys,
I have a backup program that creates incremental backups and generates a MD5 hash at the same time.
Each server backup has its own sub folder.

Each backup file has a corresponding .md5 file containing a hash and a file name like this.

3410efed13b087322de8036145230a55 *COMPANYSRV_F_VOL-b008-i052.spi

I copy these files off site periodically to a linux based NAS Drive.
The version of linux is a bit cut down but I have managed to find and install a package manager and I now have md5deep working.

My question is how can I script md5deep to validate all files in the sub folders and output any errors. Or just output everything and I can search the text for errors.
I'm surprised I can't find this as it would seem exactly what it is meant to do.

Thanks in advance
David

Update
=========
I suppose I should say what I have tried so far. Smilie
The obvious command to start with is this with the md5 and backup files in the current dir.

Code:
md5deep -m  COMPANYSRV_F_VOL-b008-i052.md5 *

I assume the * on the end is if there are multiple hashes contained within the md5. In my case there is only one hash contained in each md5.

This lists the one file if the hash is correct.
I purposely corrupted the file and tried again and there was no output.

What I really need is a success message or a list of corrupted files from the whole directory tree.

Update 2
=========
I think the command I need is going to look something like this.
Code:
find . -name "*.md5" -exec cat {} \; | md5deep -x *.md5

That is....
Find all files in the current directory (find . -name)
With a name that ends in .md5 ("*.md5")
Pipe the contents of the files to md5deep (-exec cat {} \; | md5deep)
Show the file names that don't have a matching hash. (-x *.md5)

However this doesn't seem to read the standard input and just does a md5deep -x *.md5.

If I have three files with correct matching hash files it returns two.
I'd expect to see none when the hashes match.

Last edited by Pollardd; 09-04-2018 at 04:15 AM..
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

checking missing files in side a folder

Dear all, Every hour i am receiving several data files to one folder for 24 hours each day.But some times some hours i do not have the files because of some problem.So i want to check the files inside the folder at the end of the day wether how many files i received in each hour like this.so i... (4 Replies)
Discussion started by: Nayanajith
4 Replies

2. Shell Programming and Scripting

Checking if any folder is opened or not?

Alogorithm: ============= Whenever any user open any specific directory: 1) cd /usr/lib The prompt should show an Alert message like 2) echo "You have opened this folder" How should I write shell script this one?.. It should be irrespective of users and... (11 Replies)
Discussion started by: Niroj
11 Replies

3. Shell Programming and Scripting

Continously checking folder and executing files

Hello All, I want to make a script which continously checks one folder all the time that is there is any file in it or not, and if it found any file in it than execute that file with the following command. apxrcv -text < filename > outputfile Actually my requirement is that i will put... (4 Replies)
Discussion started by: wakhan
4 Replies

4. Shell Programming and Scripting

The checking of folder existance is not working...

Hi all, I have the following code: if ; then echo 'folder not exist'; else echo 'folder exist'; fi The "testing" folder is not exist in /home/batch , but thhe result is 'folder exist'. It seems that the code cannot detect that the folder "testing" not exist. ANybody know the... (1 Reply)
Discussion started by: suigion
1 Replies

5. Web Development

Checking if a folder is empty or not using PHP

Hi, I am sharing this tip with you all.The codes given below will explain /** * Checking a folder is empty or not. * @param string $folderName * $folderName should be folder name or path * @return TRUE/FALSE (If any file or folder found/Empty folder) */ function... (1 Reply)
Discussion started by: elizas
1 Replies

6. Shell Programming and Scripting

Creating Hashes of Hashes of Array

Hi folks, I have a structure as mentioned below in a configuration file. <Component> Comp1: { item1:data,someUniqueAttribute; item2:data,someUniqueAttribute, } Comp2: { item3:data,someUniqueAttribute; ... (1 Reply)
Discussion started by: ckv84
1 Replies

7. Shell Programming and Scripting

Checking files in folder using starting string for filename

Hi, How do i check if there are any files present in the folder with some specific starting string. For eg :- I have used this where Source_File is filename parameter. if then return 2 fi But in my case the source file name is not constant. The only constant thing is... (10 Replies)
Discussion started by: chetancrsp18
10 Replies

8. Shell Programming and Scripting

Create md5 sums and archive the resulting md5 files

Hello everyone, I am looking to basically creating md5sum files for all iso files in a directory and archive the resulting md5 files into a single archive in that very same directory. I worked out a clumsy solution such as: #find files for which md5sum are to be created and store the... (1 Reply)
Discussion started by: SurfTranquille
1 Replies

9. UNIX for Dummies Questions & Answers

Script for checking the files in a folder

Hi , I am using the below script for checking for a file in a folder. if ; then echo 0 else echo 1 fi Is there any way we can check for files which are starting with GL*.csv.What I am trying to do is , I have to check in a folder for the GL*.csv files if there are any files they I... (6 Replies)
Discussion started by: wangkc
6 Replies

10. Shell Programming and Scripting

Checking of file exist in different folder

Hi All, Seeking for your assistance to compare if the file in working directory is found on the DIR2 directory and if not found move the file in DIR2. ex. WORKING_DIR=/home/dir1/ file1.txt DIR2=/home/admin/users file1.txt what i did was found_nonempty='' for file in... (4 Replies)
Discussion started by: znesotomayor
4 Replies
md5(n)							   MD5 Message-Digest Algorithm 						    md5(n)

__________________________________________________________________________________________________________________________________________________

NAME
md5 - MD5 Message-Digest Algorithm SYNOPSIS
package require Tcl 8.2 package require md5 ?2.0.7? ::md5::md5 ?-hex? [ -channel channel | -file filename | string ] ::md5::hmac ?-hex? -key key [ -channel channel | -file filename | string ] ::md5::MD5Init ::md5::MD5Update token data ::md5::MD5Final token ::md5::HMACInit key ::md5::HMACUpdate token data ::md5::HMACFinal token _________________________________________________________________ DESCRIPTION
This package is an implementation in Tcl of the MD5 message-digest algorithm as described in RFC 1321 (1). This algorithm takes an arbi- trary quantity of data and generates a 128-bit message digest from the input. The MD5 algorithm is related to the MD4 algorithm (2) but has been strengthened against certain types of potential attack. MD5 should be used in preference to MD4 for new applications. If you have critcl and have built the tcllibc package then the implementation of the hashing function will be performed by compiled code. Alternatively if you have either cryptkit or Trf then either of these can be used to accelerate the digest computation. If no suitable com- piled package is available then the pure-Tcl implementation wil be used. The programming interface remains the same in all cases. Note the previous version of this package always returned a hex encoded string. This has been changed to simplify the programming interface and to make this version more compatible with other implementations. To obtain the previous usage, either explicitly specify package ver- sion 1 or use the -hex option to the md5 command. COMMANDS
::md5::md5 ?-hex? [ -channel channel | -file filename | string ] Calculate the MD5 digest of the data given in string. This is returned as a binary string by default. Giving the -hex option will return a hexadecimal encoded version of the digest. The data to be hashed can be specified either as a string argument to the md5 command, or as a filename or a pre-opened channel. If the -filename argument is given then the file is opened, the data read and hashed and the file is closed. If the -channel argument is given then data is read from the channel until the end of file. The channel is not closed. Only one of -file, -channel or string should be given. ::md5::hmac ?-hex? -key key [ -channel channel | -file filename | string ] Calculate an Hashed Message Authentication digest (HMAC) using the MD5 digest algorithm. HMACs are described in RFC 2104 (3) and provide an MD5 digest that includes a key. All options other than -key are as for the ::md5::md5 command. PROGRAMMING INTERFACE
For the programmer, the MD5 hash can be viewed as a bucket into which one pours data. When you have finished, you extract a value that is derived from the data that was poured into the bucket. The programming interface to the MD5 hash operates on a token (equivalent to the bucket). You call MD5Init to obtain a token and then call MD5Update as many times as required to add data to the hash. To release any resources and obtain the hash value, you then call MD5Final. An equivalent set of functions gives you a keyed digest (HMAC). ::md5::MD5Init Begins a new MD5 hash. Returns a token ID that must be used for the remaining functions. ::md5::MD5Update token data Add data to the hash identified by token. Calling MD5Update $token "abcd" is equivalent to calling MD5Update $token "ab" followed by MD5Update $token "cb". See EXAMPLES. ::md5::MD5Final token Returns the hash value and releases any resources held by this token. Once this command completes the token will be invalid. The result is a binary string of 16 bytes representing the 128 bit MD5 digest value. ::md5::HMACInit key This is equivalent to the ::md5::MD5Init command except that it requires the key that will be included in the HMAC. ::md5::HMACUpdate token data ::md5::HMACFinal token These commands are identical to the MD5 equivalent commands. EXAMPLES
% md5::md5 -hex "Tcl does MD5" 8AAC1EE01E20BB347104FABB90310433 % md5::hmac -hex -key Sekret "Tcl does MD5" 35BBA244FD56D3EDF5F3C47474DACB5D % set tok [md5::MD5Init] ::md5::1 % md5::MD5Update $tok "Tcl " % md5::MD5Update $tok "does " % md5::MD5Update $tok "MD5" % md5::Hex [md5::MD5Final $tok] 8AAC1EE01E20BB347104FABB90310433 REFERENCES
[1] Rivest, R., "The MD5 Message-Digest Algorithm", RFC 1321, MIT and RSA Data Security, Inc, April 1992. (http://www.rfc-edi- tor.org/rfc/rfc1321.txt) [2] Rivest, R., "The MD4 Message Digest Algorithm", RFC 1320, MIT, April 1992. (http://www.rfc-editor.org/rfc/rfc1320.txt) [3] Krawczyk, H., Bellare, M. and Canetti, R. "HMAC: Keyed-Hashing for Message Authentication", RFC 2104, February 1997. (http://www.rfc-editor.org/rfc/rfc2104.txt) BUGS, IDEAS, FEEDBACK This document, and the package it describes, will undoubtedly contain bugs and other problems. Please report such in the category md5 of the Tcllib SF Trackers [http://sourceforge.net/tracker/?group_id=12883]. Please also report any ideas for enhancements you may have for either package and/or documentation. SEE ALSO
md4, sha1 KEYWORDS
hashing, md5, message-digest, rfc 1320, rfc 1321, rfc 2104, security CATEGORY
Hashes, checksums, and encryption COPYRIGHT
Copyright (c) 2003, Pat Thoyts <patthoyts@users.sourceforge.net> md5 2.0.7 md5(n)
All times are GMT -4. The time now is 07:18 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy