Md5sum script


Login or Register for Dates, Times and to Reply

 
Thread Tools Search this Thread
# 1  
Md5sum script

Hello,

I need to download multiple files from an FTP server but occasionally they arrive in error so I need to perform an integrity check. I've been attempting to write a bash script that does the following:
  1. Downloads all files including those in sub directories
  2. Perform md5sum using stored hash values
  3. Download any files that fail the integrity check again
  4. Repeat steps 2-3 until all files have passed the integrity check

I'm stuck on step 3. I don't know how to download the individual file so that it goes into its relevant sub-directory by itself. The relevant parts of my files and script are as follow:

Code:
checksums.txt
abcdef123456 /a/one.txt
ghijklm67890 /a/b/two.txt

Code:
file=`awk "NR==$i{print}" $checksumFile | awk -F " " '{print $2}'`

curl ftp://www.test.com/$file --user user:password

This ofcourse stores the file in the working directory. For instance, I'd like two.txt to save to /working directory/a/b/. Any ideas? Additionally, is there a better way of doing this?

Thanks
# 2  
you can check a file like that with md5sum -c, it gives output like this:

Code:
 $ md5sum -c checksum.txt 2>/dev/null
a: OK
b: FAILED
c: FAILED open or read
d: OK

so you could handle it like this:

Code:
while ! md5sum -c checksum.txt > checklist
do
        awk -F: '!/OK$/ && $0=$1' checklist | while read FILENAME
        do
                echo "Need to download $FILENAME"
        done
done

This User Gave Thanks to Corona688 For This Post:
# 3  
Thanks Corona. However, it doesn't explain how to download the file so that they go into their relevant directories. For instance (and assuming these files have failed on their first attempt):

one.txt needs to go in working directory/a/
two.txt needs to go in working directory/a/b/

Additionally, how do I get it to create the directories automatically?

Thanks
# 4  
Code:
curl --create-dirs -o a/b/c.txt

... will create the paths as it goes.
# 5  
Works, thanks.
Login or Register for Dates, Times and to Reply

Previous Thread | Next Thread
Thread Tools Search this Thread
Search this Thread:
Advanced Search

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

[md5sum] script

I am getting No such file or directory if my variable contains white spaces... Is there a way to fix this? This works x="1.md5" md5sum -c "$x" This, does not x="23\ 5\ 6\ 7\ 8\ 9\ 10.md5" md5sum -c "$x" md5sum: '23\ 5\ 6\ 7\ 8\ 9\ 10.md5': No such file or directory How do I fix... (1 Reply)
Discussion started by: soichiro
1 Replies

2. Shell Programming and Scripting

Md5sum on crontab

Hi all, I have to verify the integrity of a ISO image that is downloaded periodically on my PC. In order to do that, I've written a bash script that use the command md5sum with the aim to match the generated code with the default one created with the SIO image. The problem is that, if I start... (1 Reply)
Discussion started by: Mr. Piros
1 Replies

3. Shell Programming and Scripting

Making a script to copy files not seen before (using md5sum)

Hello, I would like to make a script that searches through a SRC folder and copies only files it's never seen before to a DEST folder. SRC = /user/.phonesync/photos-backup DST = /usr/.phonesync/photos-new So basically, I'd start with a: md5sum /user/.phonesync/photos-backup/* >... (29 Replies)
Discussion started by: nbsparks
29 Replies

4. Shell Programming and Scripting

md5sum in different linux

something strange is that i find the md5sum command in different linux generate different result, for example, i have tried the same file in CentOS and Rhel, The md5 results are different, it is quite headache, who know the tricks? (3 Replies)
Discussion started by: zbc
3 Replies

5. Shell Programming and Scripting

how to get a md5sum in perl

hi All: i write a adduser script in perl , but I don't know how to deal with the password , for it stored as md5. and i don't use the shell command passwd. give me some advice...thanks (1 Reply)
Discussion started by: kingdream
1 Replies

6. Shell Programming and Scripting

Script to check MD5SUM on file

Hi, I currently have a shell script that takes an RPM and scp's it to a set of remote servers and installs it. What I would like to be able to do is make the script get the md5sum of the RPM locally (so get the md5sum of the rpm from where im running the script) and then scp the rpm to the... (0 Replies)
Discussion started by: tb1986
0 Replies

7. Programming

Computing an MD5Sum in C

Is it possible to call the unix command md5sum from within a C program. I am trying to write a C program that scans a directory and computes the MD5Sum of all the files in the directory. Whenever I use md5sum 'filename' I get the error 'md5sum undeclared'. Is there a header file or some library... (3 Replies)
Discussion started by: snag49ers
3 Replies

8. Shell Programming and Scripting

Restart script based on MD5sum

Ok, I run a small script, that restarts a perl script when it fails (it's a very unreliable perl script, but I can't change it, it's crucial, and I don't know perl) It outputs data into a logfile. Unfortunately, it also regularly hangs. This is a major problem because if it hangs, no data is... (6 Replies)
Discussion started by: Bakes
6 Replies

9. Programming

md5sum and execve

Hello there! Is there a way to use execve() to run md5sum function? for example execve("md5sum <filename>, NULL,NULL);" thanx! (2 Replies)
Discussion started by: nicos
2 Replies

10. UNIX for Dummies Questions & Answers

What is md5sum???

Hi all, I am kinda puzzled. When and Why do we use md5sum? I've read man pages for mp5sum, but didn't get anything out of it. Please, can someone explain this to me in couple of words. Thank you all. (1 Reply)
Discussion started by: solvman
1 Replies

Featured Tech Videos