Hi Guys,
I have a backup program that creates incremental backups and generates a MD5 hash at the same time.
Each server backup has its own sub folder.
Each backup file has a corresponding .md5 file containing a hash and a file name like this.
I copy these files off site periodically to a linux based NAS Drive.
The version of linux is a bit cut down but I have managed to find and install a package manager and I now have md5deep working.
My question is how can I script md5deep to validate all files in the sub folders and output any errors. Or just output everything and I can search the text for errors.
I'm surprised I can't find this as it would seem exactly what it is meant to do.
Thanks in advance
David
Update
=========
I suppose I should say what I have tried so far.
The obvious command to start with is this with the md5 and backup files in the current dir.
I assume the * on the end is if there are multiple hashes contained within the md5. In my case there is only one hash contained in each md5.
This lists the one file if the hash is correct.
I purposely corrupted the file and tried again and there was no output.
What I really need is a success message or a list of corrupted files from the whole directory tree.
Update 2
=========
I think the command I need is going to look something like this.
That is....
Find all files in the current directory (find . -name)
With a name that ends in .md5 ("*.md5")
Pipe the contents of the files to md5deep (-exec cat {} \; | md5deep)
Show the file names that don't have a matching hash. (-x *.md5)
However this doesn't seem to read the standard input and just does a md5deep -x *.md5.
If I have three files with correct matching hash files it returns two.
I'd expect to see none when the hashes match.
Hi RudiC,
I can't seem to test a single folder at the moment so we don't need to worry about sub folders yet.
The trailing * is to select the files contained within the MD5. There is only one file in each MD5 so an * is going to cover it. I get an error if I leave it out.
My MD5's files are a one to one match with the files they are a hash of.
To test I have created a folder with 3 backup files and three matching md5 files
For example one of the backup files is named COMPANYSRV_F_VOL-b008-i066.spi
The contents of the matching MD5 named COMPANYSRV_F_VOL-b008-i066.md5 is
ea068f65d93cb8cccb3a44b642192758 *COMPANYSRV_F_VOL-b008-i066.spi
Couldn't be more straightforward so far.
I want to list files where the hash no longer matches or simply return success or no errors found.
I want to run a command that will test each file against it's hash.
My backups are incremental and range in size from a few 10's ok KB to Hundreds of GB. My Test files are under 20Mb.
My tests above returned the names of two files where I would expect none.
I don't expect I just happen to pick two corrupted files for my test.
It's a long time since I piped anything to the cat so I may not have constructed that command correctly which is why I brought my question to a scripting forum.
Previously I have run this from a windows machine but it copies a TB of data across my LAN and that takes too long.
Hello Again,
After more google research I found that the MD5 files created on my windows system had one extra character apart from the * in front of the file name, that is a Carriage Return.
I also don't think I need the functionality of md5deep as "find" will search sub directories anyway.
I upgraded md5sum on my NAS device as I has some old unknown version.
Now if I use the following command
I have included a pipe through dos2unix that will convert the windows line termination to unix line termination.
I modified one of my MD5 files to cause a failure.
The Result
This is exactly what I need. I should now be able to schedule this command and send myself the output in an email.
Hope this helps someone else.
David
Last edited by Scott; 09-10-2018 at 09:56 AM..
Reason: Please use code tags
Hi All,
Seeking for your assistance to compare if the file in working directory is found on the DIR2 directory and if not found move the file in DIR2.
ex. WORKING_DIR=/home/dir1/
file1.txt
DIR2=/home/admin/users
file1.txt
what i did was
found_nonempty=''
for file in... (4 Replies)
Hi ,
I am using the below script for checking for a file in a folder.
if ;
then
echo 0
else
echo 1
fi
Is there any way we can check for files which are starting with GL*.csv.What I am trying to do is , I have to check in a folder for the GL*.csv files if there are any files they I... (6 Replies)
Hello everyone,
I am looking to basically creating md5sum files for all iso files in a directory and archive the resulting md5 files into a single archive in that very same directory.
I worked out a clumsy solution such as:
#find files for which md5sum are to be created and store the... (1 Reply)
Hi,
How do i check if there are any files present in the folder with some specific starting string.
For eg :- I have used this where Source_File is filename parameter.
if
then
return 2
fi
But in my case the source file name is not constant. The only constant thing is... (10 Replies)
Hi folks,
I have a structure as mentioned below in a configuration file.
<Component>
Comp1:
{
item1:data,someUniqueAttribute;
item2:data,someUniqueAttribute,
}
Comp2:
{
item3:data,someUniqueAttribute;
... (1 Reply)
Hi,
I am sharing this tip with you all.The codes given below will explain
/**
* Checking a folder is empty or not.
* @param string $folderName
* $folderName should be folder name or path
* @return TRUE/FALSE (If any file or folder found/Empty folder)
*/
function... (1 Reply)
Hi all,
I have the following code:
if ; then echo 'folder not exist'; else echo 'folder exist'; fi
The "testing" folder is not exist in /home/batch , but thhe result is 'folder exist'. It seems that the code cannot detect that the folder "testing" not exist.
ANybody know the... (1 Reply)
Hello All,
I want to make a script which continously checks one folder all the time that is there is any file in it or not, and if it found any file in it than execute that file with the following command.
apxrcv -text < filename > outputfile
Actually my requirement is that i will put... (4 Replies)
Alogorithm:
=============
Whenever any user open any specific directory:
1) cd /usr/lib
The prompt should show an Alert message like
2) echo "You have opened this folder"
How should I write shell script this one?..
It should be irrespective of users and... (11 Replies)
Dear all,
Every hour i am receiving several data files to one folder for 24 hours each day.But some times some hours i do not have the files because of some problem.So i want to check the files inside the folder at the end of the day wether how many files i received in each hour like this.so i... (4 Replies)