Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Md5sum is running very slowly

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Md5sum is running very slowly
# 8  
Old 01-29-2014
This is because the separator character you listed doesn't match so it's not finding 26 fields. I might need on od -c dump of a line to get the proper value of ¬

My test file below works OK:

Code:
¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬L1F1¬L1F2¬L1F3
¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬L2F1¬L2F2¬L2F3
¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬L3F1¬L3F2¬L3F3

Edit: We can avoid perl errors and simply print invalid lines like this:
Code:
  if (scalar(@fld)>26) {
      print $line . "¬" . md5_hex($fld[24]." ".$fld[25]." ".$fld[26] . "\n") . "\n";
  } else {
      print $line . "\n";
  }

but we've really got to sort out what that separator character is to get the results you want.
Last edited by Chubler_XL; 01-29-2014 at 07:51 PM..
# 9  
Old 01-29-2014
Quote:
Originally Posted by Chubler_XL
This is because the separator character you listed doesn't match so it's not finding 26 fields. I might need on od -c dump of a line to get the proper value of ¬

My test file below works OK:

Code:
¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬L1F1¬L1F2¬L1F3
¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬L2F1¬L2F2¬L2F3
¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬L3F1¬L3F2¬L3F3

Edit: We can avoid perl errors and simply print invalid lines like this:
Code:
  if (scalar(@fld)>26) {
      print $line . "¬" . md5_hex($fld[24]." ".$fld[25]." ".$fld[26] . "\n") . "\n";
  } else {
      print $line . "\n";
  }

but we've really got to sort out what that separator character is to get the results you want.
Thanks. I noticed that the string is being generated and appended to the new file.

Here is the octal dump
Code:
0000000   X   X   X 254   X   X   X   |   1   0   0   0   0   4 254   1
0000020   6       X   X   X   X   X   X       X   X 254 254 254   X   X
0000040   X   X   X   X       X   X   X   X 254   X   X 254   2   4   X
0000060   8 254   5   1   7   8   9   1   3   1 254   X   X   X 254 254
0000100 254 254 254 254   1   6 254 254   X   X   X   X   X   x       D
0000120   r 254 254   X   X   X   x   X   X       X   X   X   X 254   X
0000140   S   W 254   1   4   2   8 254   A   U   S   T   R   A   L   I
0000160   A 254   5   X   X   X   9   1   3   1 254 254   1   6 254   A
0000200   X   X   X   X   X       D   r 254   R   9   1   3   0   0   0
0000220   0   0   0   6   0   :   A   U   S  \r  \n
0000233

# 10  
Old 01-29-2014
OK looks like your separator character is asc(172), that is 254 in octal.

Also looks like a DOS formatted file - this should get the job done. Note I'm putting the \r characters back to retain the DOS format on output.

Code:
#!/usr/bin/perl -w
use Digest::MD5 qw(md5 md5_hex md5_base64);

my $sep = chr(172);

open my $DAT, $ARGV[0] or die "Could not open $ARGV[0]: $!";

while (my $line = <$DAT>) {
  $line =~ s/\r?\n$//;
  my @fld = split $sep, $line;
  if (scalar(@fld)>26) {
      print $line . $sep . md5_hex($fld[24]." ".$fld[25]." ".$fld[26] . "\n") . "\r\n";
  } else {
      print $line . "\r\n";
  }
}

close $DAT;

Last edited by Chubler_XL; 01-29-2014 at 08:22 PM..
This User Gave Thanks to Chubler_XL For This Post:
ahmedwaseem2000
# 11  
Old 01-29-2014
Well, this is not processing anything at all. It's giving the error message as below
Code:
Can't locate object method "e" via package "Digest::MD5" (perhaps you forgot to load "Digest::MD5"?) at ./addmd5sum.pl line 1.

# 12  
Old 01-29-2014
Not getting that error with code from post #10 please check you transposed correctly.
# 13  
Old 01-29-2014
Quote:
Originally Posted by Chubler_XL
Not getting that error with code from post #10 please check you transposed correctly.
Sorry it was a copy paste issue. It works perfectly Smilie Thanks for all your help Smilie
# 14  
Old 01-29-2014
Code:
perl -MDigest::MD5=md5_hex -aF\\254 -lpe '$_=md5_hex(join(" ", @F[24..26]));'

Regards,
Alister

Edit: This is not equivalent to Chubler's suggestion and it only prints the hashes. Oh well. At least I spent a little time kicking some rust off my very puny perl muscles.
Last edited by alister; 01-29-2014 at 10:24 PM..
This User Gave Thanks to alister For This Post:
Chubler_XL
Login or Register to Ask a Question

Previous Thread | Next Thread

6 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Checking Unix Performance - Why is a process running slowly?

Hi Please can someone explain to me how they would go about monitoring the performance of a process in Unix. Lets say that a user is running a process in Unix but it seems to be taking a long time, whereas it completed a lot quicker yesterday. How would you go about investigating what is causing... (1 Reply)
Discussion started by: Sunny Sid
1 Replies

2. SuSE

SLES 10 SP2 possible kernel problem, / slowly filling up

Hello Guys I first though about posting this to emergency but cause I fixed my issue with an reboot its not as important, more is important to me what caused this situation Some facts: OS: SLES 10 x64 SP2 (Virtualized Vmware ESX 3.5) / vmware tools status OK Soft: Oracle10g LVM... (1 Reply)
Discussion started by: kl1ngac1k
1 Replies

3. Solaris

Sun StorageTek Common Array Manager 6.0 works very slowly

Hi! I have Sun StorageTek 2540 FC array and CAM works very slowly - I can wait for software response even more than 2 minutes... I run this software on Windows machine with Firefox Web Browser but speed is terrible... How can I make it works at least a little bit faster?.. (2 Replies)
Discussion started by: Sapfeer
2 Replies

4. Shell Programming and Scripting

Running md5sum on a list of files

Hello, I would like to run md5sum on a list of files saved in a text file, and save the result in another file. (ie. md5sum `cat list.txt` > md5list.txt) I have tried several things, but I am always confronted to the same problem: some of the filenames have spaces. I have run sed on the... (5 Replies)
Discussion started by: SDelroen
5 Replies

5. Solaris

Delete and copy file(s) slowly(!?)

Hi all! I have to monitor space in V890 machine, Solaris 10 weekly, because there is Oracle DB on it with many datafiles which have been taken offline to make enough size. Sometime, one or more datafiles are big, they are 20GB, 40GB etc.. The problem I have encountered is the processing of... (5 Replies)
Discussion started by: trantuananh24hg
5 Replies

6. UNIX for Dummies Questions & Answers

What is md5sum???

Hi all, I am kinda puzzled. When and Why do we use md5sum? I've read man pages for mp5sum, but didn't get anything out of it. Please, can someone explain this to me in couple of words. Thank you all. (1 Reply)
Discussion started by: solvman
1 Replies
Login or Register to Ask a Question