MD5 File Parsing


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting MD5 File Parsing
# 1  
Old 02-13-2011
MD5 File Parsing

Hey Guys.
Here is another issue I'm struggling with: Smilie
I need wget to download only the files that has been changed. I use md5deep to generate the check sum files.

Here is a simple result; I'll call it $string for the example

Code:
0ce73169b069d11e38d6bd78796839c0  md5data
78cb11f2d943b48b8d5a00931cc454b1  md5rss
86ffb6728305730996451e43effa1738  myTextFile.txt
f385d56f26ca3859f48bd450c3e214bf  rss.txt

if I set -- $string, i can access each argument by using $# (where # is 1,2,3,etc..)

so:
echo $1 will print 0ce73169b069d11e38d6bd78796839c0
echo $2 will print md5data

however I'm not sure how to write $# in a for loop that will go through all of them.

So here is the logic of the loop I'm trying to create:

Code:
for I ; 1 to [number of arguments in the file] do; step 2
if $i != $string than download file $(i+1); end if
end for.

Any suggestions (or better ways to check md5 files and download only the files that are different?)


Thanks

Last edited by pludi; 02-14-2011 at 03:32 AM..
# 2  
Old 02-14-2011
Ok, guess you have the md5 list file: md5_total.txt With below codes, you can find out the files list which are changed.

Code:
MD5FILE=/XXX/md5_total.txt

cd /DIR1 
find . -type f -exec md5deep {} \; > new_md5.txt

diff <(sort -k2 $MD5FILE) <(sort -k2 $new_md5.txt) |awk '/</' {print $NF}'

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to reverse the b64 format(encoded b64(b64(md5($pass)))) into md5 hash format?

I have about 1500 rows (encoded b64(b64(md5($pass))) algorythm) in a file. I would like reverse the b64 into md5 hash format. How could I do that from command line? So I need only the correct md5 hash formats. These row format: 4G5qc2WQzGES6QkWAUgl5w P9tKxonBOg3ymr8vOBLnDA... (1 Reply)
Discussion started by: freeroute
1 Replies

2. Shell Programming and Scripting

Apply md5 hash to a field in csv file

I have a .csv file and I want to md5 hash the second column for each row in the file. File is something like data1,foobar1,123,345 data2,foobar2,456,9393 data3,foobar3,1002,10109 Output would be like data1,6c81243028f8e455fa617dd5f0232ce1,123,345... (3 Replies)
Discussion started by: jjwags
3 Replies

3. Shell Programming and Scripting

Create md5 sums and archive the resulting md5 files

Hello everyone, I am looking to basically creating md5sum files for all iso files in a directory and archive the resulting md5 files into a single archive in that very same directory. I worked out a clumsy solution such as: #find files for which md5sum are to be created and store the... (1 Reply)
Discussion started by: SurfTranquille
1 Replies

4. Shell Programming and Scripting

parsing data from a big file using keys from another smaller file

Hi, I have 2 files format of file 1 is: a1 b2 a2 c2 d1 f3 format of file 2 is (tab delimited): a1 1.2 0.5 0.06 0.7 0.9 1 0.023 a3 0.91 0.007 0.12 0.34 0.45 1 0.7 a2 1.05 2.3 0.25 1 0.9 0.3 0.091 b1 1 5.4 0.3 9.2 0.3 0.2 0.1 b2 3 5 7 0.9 1 9 0 1 b3 0.001 1 2.3 4.6 8.9 10 0 1 0... (10 Replies)
Discussion started by: Lucky Ali
10 Replies

5. Linux

Need Help: MD5

I am trying to compare two identical files by using md5 command, but cant get the right command parameters Please help me with any examples. All I want is to know how to compare two identical files which are residing on two different machines in my local network, for example: Host_A -... (6 Replies)
Discussion started by: greenja
6 Replies

6. Shell Programming and Scripting

Parsing of file for Report Generation (String parsing and splitting)

Hey guys, I have this file generated by me... i want to create some HTML output from it. The problem is that i am really confused about how do I go about reading the file. The file is in the following format: TID1 Name1 ATime=xx AResult=yyy AExpected=yyy BTime=xx BResult=yyy... (8 Replies)
Discussion started by: umar.shaikh
8 Replies

7. Shell Programming and Scripting

Parsing file, yaml file? Extracting specific sections

Here is a data file, which I believe is in YAML. I am trying to retrieve just the 'addon_domains" section, which doesnt seem to be as easy as I had originally thought. Any help on this would be greatly appreciated!! I have been trying to do this in awk and mostly bash scripting instead of perl... (3 Replies)
Discussion started by: Rhije
3 Replies

8. UNIX for Dummies Questions & Answers

Script for parsing details in a log file to a seperate file

Hi Experts, Im a new bee for scripting, I would ned to do the following via linux shell scripting, I have an application which throws a log file, on each action of a particular work with the application, as sson as the action is done, the log file would vanish or stops updating there, the... (2 Replies)
Discussion started by: pingnagan
2 Replies

9. Shell Programming and Scripting

Finding & Moving Oldest File by Parsing/Sorting Date Info in File Names

I'm trying to write a script that will look in an /exports folder for the oldest export file and move it to a /staging folder. "Oldest" in this case is actually determined by date information embedded in the file names themselves. Also, the script should only move a file from /exports to... (6 Replies)
Discussion started by: nikosey
6 Replies

10. Shell Programming and Scripting

Md5

Does anyone know a scipt that includes MD5. I need to run a script that includes MD5 encryption. Thanks (1 Reply)
Discussion started by: duncang3
1 Replies
Login or Register to Ask a Question