|
|
Sponsored Content
Top Forums
Shell Programming and Scripting
Checking MD5 Hashes on a folder tree
Post 303022794 by Pollardd on Friday 7th of September 2018 08:15:30 AM
09-07-2018
Hi RudiC,
I can't seem to test a single folder at the moment so we don't need to worry about sub folders yet.
The trailing * is to select the files contained within the MD5. There is only one file in each MD5 so an * is going to cover it. I get an error if I leave it out.
My MD5's files are a one to one match with the files they are a hash of.
To test I have created a folder with 3 backup files and three matching md5 files
For example one of the backup files is named COMPANYSRV_F_VOL-b008-i066.spi
The contents of the matching MD5 named COMPANYSRV_F_VOL-b008-i066.md5 is
ea068f65d93cb8cccb3a44b642192758 *COMPANYSRV_F_VOL-b008-i066.spi
Couldn't be more straightforward so far.
I want to list files where the hash no longer matches or simply return success or no errors found.
I want to run a command that will test each file against it's hash.
My backups are incremental and range in size from a few 10's ok KB to Hundreds of GB. My Test files are under 20Mb.
My tests above returned the names of two files where I would expect none.
I don't expect I just happen to pick two corrupted files for my test.
It's a long time since I piped anything to the cat so I may not have constructed that command correctly which is why I brought my question to a scripting forum.
Previously I have run this from a windows machine but it copies a TB of data across my LAN and that takes too long.
Thanks for your assistance
David
I can't seem to test a single folder at the moment so we don't need to worry about sub folders yet.
The trailing * is to select the files contained within the MD5. There is only one file in each MD5 so an * is going to cover it. I get an error if I leave it out.
My MD5's files are a one to one match with the files they are a hash of.
To test I have created a folder with 3 backup files and three matching md5 files
For example one of the backup files is named COMPANYSRV_F_VOL-b008-i066.spi
The contents of the matching MD5 named COMPANYSRV_F_VOL-b008-i066.md5 is
ea068f65d93cb8cccb3a44b642192758 *COMPANYSRV_F_VOL-b008-i066.spi
Couldn't be more straightforward so far.
I want to list files where the hash no longer matches or simply return success or no errors found.
I want to run a command that will test each file against it's hash.
My backups are incremental and range in size from a few 10's ok KB to Hundreds of GB. My Test files are under 20Mb.
My tests above returned the names of two files where I would expect none.
I don't expect I just happen to pick two corrupted files for my test.
It's a long time since I piped anything to the cat so I may not have constructed that command correctly which is why I brought my question to a scripting forum.
Previously I have run this from a windows machine but it copies a TB of data across my LAN and that takes too long.
Thanks for your assistance
David
|
|
10 More Discussions You Might Find Interesting
1. UNIX for Dummies Questions & Answers
Dear all,
Every hour i am receiving several data files to one folder for 24 hours each day.But some times some hours i do not have the files because of some problem.So i want to check the files inside the folder at the end of the day wether how many files i received in each hour like this.so i... (4 Replies)
Discussion started by: Nayanajith
4 Replies
2. Shell Programming and Scripting
Alogorithm:
=============
Whenever any user open any specific directory:
1) cd /usr/lib
The prompt should show an Alert message like
2) echo "You have opened this folder"
How should I write shell script this one?..
It should be irrespective of users and... (11 Replies)
Discussion started by: Niroj
11 Replies
3. Shell Programming and Scripting
Hello All,
I want to make a script which continously checks one folder all the time that is there is any file in it or not, and if it found any file in it than execute that file with the following command.
apxrcv -text < filename > outputfile
Actually my requirement is that i will put... (4 Replies)
Discussion started by: wakhan
4 Replies
4. Shell Programming and Scripting
Hi all,
I have the following code:
if ; then echo 'folder not exist'; else echo 'folder exist'; fi
The "testing" folder is not exist in /home/batch , but thhe result is 'folder exist'. It seems that the code cannot detect that the folder "testing" not exist.
ANybody know the... (1 Reply)
Discussion started by: suigion
1 Replies
5. Web Development
Hi,
I am sharing this tip with you all.The codes given below will explain
/**
* Checking a folder is empty or not.
* @param string $folderName
* $folderName should be folder name or path
* @return TRUE/FALSE (If any file or folder found/Empty folder)
*/
function... (1 Reply)
Discussion started by: elizas
1 Replies
6. Shell Programming and Scripting
Hi folks,
I have a structure as mentioned below in a configuration file.
<Component>
Comp1:
{
item1:data,someUniqueAttribute;
item2:data,someUniqueAttribute,
}
Comp2:
{
item3:data,someUniqueAttribute;
... (1 Reply)
Discussion started by: ckv84
1 Replies
7. Shell Programming and Scripting
Hi,
How do i check if there are any files present in the folder with some specific starting string.
For eg :- I have used this where Source_File is filename parameter.
if
then
return 2
fi
But in my case the source file name is not constant. The only constant thing is... (10 Replies)
Discussion started by: chetancrsp18
10 Replies
8. Shell Programming and Scripting
Hello everyone,
I am looking to basically creating md5sum files for all iso files in a directory and archive the resulting md5 files into a single archive in that very same directory.
I worked out a clumsy solution such as:
#find files for which md5sum are to be created and store the... (1 Reply)
Discussion started by: SurfTranquille
1 Replies
9. UNIX for Dummies Questions & Answers
Hi ,
I am using the below script for checking for a file in a folder.
if ;
then
echo 0
else
echo 1
fi
Is there any way we can check for files which are starting with GL*.csv.What I am trying to do is , I have to check in a folder for the GL*.csv files if there are any files they I... (6 Replies)
Discussion started by: wangkc
6 Replies
10. Shell Programming and Scripting
Hi All,
Seeking for your assistance to compare if the file in working directory is found on the DIR2 directory and if not found move the file in DIR2.
ex. WORKING_DIR=/home/dir1/
file1.txt
DIR2=/home/admin/users
file1.txt
what i did was
found_nonempty=''
for file in... (4 Replies)
Discussion started by: znesotomayor
4 Replies
LEARN ABOUT OSX
md5
MD5(1) BSD General Commands Manual MD5(1) NAME
md5 -- calculate a message-digest fingerprint (checksum) for a file SYNOPSIS
md5 [-pqrtx] [-s string] [file ...] DESCRIPTION
The md5 utility takes as input a message of arbitrary length and produces as output a ``fingerprint'' or ``message digest'' of the input. It is conjectured that it is computationally infeasible to produce two messages having the same message digest, or to produce any message having a given prespecified target message digest. The MD5 algorithm is intended for digital signature applications, where a large file must be ``compressed'' in a secure manner before being encrypted with a private (secret) key under a public-key cryptosystem such as RSA. MD5's designer Ron Rivest has stated "md5 and sha1 are both clearly broken (in terms of collision-resistance)". So MD5 should be avoided when creating new protocols, or implementing protocols with better options. SHA256 and SHA512 are better options as they have been more resilient to attacks (as of 2009). The following options may be used in any combination and must precede any files named on the command line. The hexadecimal checksum of each file listed on the command line is printed after the options are processed. -s string Print a checksum of the given string. -p Echo stdin to stdout and append the checksum to stdout. -q Quiet mode - only the checksum is printed out. Overrides the -r option. -r Reverses the format of the output. This helps with visual diffs. Does nothing when combined with the -ptx options. -t Run a built-in time trial. -x Run a built-in test script. EXIT STATUS
The md5 utility exits 0 on success, and 1 if at least one of the input files could not be read. SEE ALSO
cksum(1), md5(3), ripemd(3), sha(3), CC_SHA256_Init(3) Vlastimil Klima, Finding MD5 Collisions - a Toy For a Notebook, Cryptology ePrint Archive: Report 2005/075. ACKNOWLEDGMENTS
This program is placed in the public domain for free general use by RSA Data Security. BSD
June 6, 2004 BSD