You know what -There was a Perl script I had got from CPAN to generate SHA1 checksum of files in a directory, I modified to suite my needs, for the project I was working at, and the script was taking around 21-22 minutes just to generate SHA1 sum of the required files across the system disk (which was most of the files of the disk except a list of certain category of files filtered by it's extension) and later would verify at boot time.
...
I was made to write a C version of the same again
...
The pleasant part was that the job which took 22 minutes (approx) was taking only around 3 seconds.
I really do expect your C++ program, if you really create, will get you to see the same kind of performance improvements.
I do not intend any personal offense, but I am compelled to say that it is absurd to tell someone that they will realize a 440x runtime improvement without any knowledge of their task, their code, or their hardware.
If the code isn't CPU-bound, the performance gain will be negligible.
Your example, by the way, of cryptographic hashing, is just the type of task that does benefit greatly from using a lower-level language. However, switching to C won't yield the massive improvements you describe because the folks who implement perl's widely-used crypto modules are already using C.
To demonstrate, on an ancient Pentium II system running at 350 MHz, let's hash all the executables in my /usr/bin directory.
Version of sha1sum (a C program):
Perl script:
Creating data file:
Comparison:
The tests were repeated several times in different orders. The times never varied significantly. 1.25x faster is a far, far cry from 440x. The small improvement is expected since both are hashing in C.
It is reasonable to conclude that whatever perl script you were using (or the modifications made to it) was seriously deficient.
Regards,
Alister
---------- Post updated at 12:43 PM ---------- Previous update was at 12:34 PM ----------
Quote:
Originally Posted by Ribosome
The task which needs to be done here is to read some log files, extract the data such as folder id, number of files it contains, processing time, no. of successfully processed files and no. of failed files etc and put them in an excel file.
The script is good and doesnt seem to need any fix but it takes some time to process millions of data. So the only concern here is to get this thing done in any other language for optimization.
For i/o and string comparisons and the like, the improvement ratio (perl-time/c-time) will be small. However, if the dataset is massive, that ratio could amount to a substantial speedup. The 1.25x improvement in the sha1 hashing in my previous post isn't much for 181 MB of data, 3.5 secs or so, but it's about 20 seconds per gigabyte and 5.5 hours per terabyte.
Hi folks,
could anyone please tell me how can i switch between two users without going thru the su(i.e. root)?
is there any such command?
thanks in advance,
thell (1 Reply)
Hi,
what is the use of the double quotes and !! in the following code segment:
su - user1 << ""!! > /dev/null 2>&1
echo "welcome user1"
EOF
!!
also what is the difference between below:
su - user1 << ""!! > /dev/null 2>&1 and
su - $USER << ""!!!> /dev/null 2>&1.
Note: $USER =... (1 Reply)
Hi,
what is the use of the double quotes and !! in the following code segment:
su - user1 << ""!! > /dev/null 2>&1
echo "welcome user1"
EOF
!!
also what is the difference between below:
su - user1 << ""!! > /dev/null 2>&1 and
su - $USER << ""!!!> /dev/null 2>&1.
Note: $USER =... (2 Replies)
Can any one tell me :
How we can switch between two users without prompting for the password. (In the SHELL SCRIPT can we fetch the USERID and PASSWORD from a specified file, without using SUDO command)? (2 Replies)
Hi all,
we have running some linux servers with sles9 and we have some problems with our dns servers. Sometimes they don't like to work. However, is there a parameter to enable faster switching between two ore more dns servers?
Thx for your help in front
Regards
frank (5 Replies)
Hi
I want to write a script which can switch between super users.But it asks for the password at the prompt.How can I manage in the script so that it didnt ask me for the password at the prompt. (1 Reply)
greetings,
i hope this hasn't been covered previously. has anyone heard of a .kext or daemon that would allow linux or (open)solaris-like vt switching? googling didn't help much..
i know os x allows a '>console' login from loginwindow.app, but i'm mainly interested in this because there are... (0 Replies)
Hi
I'm quite new with linux.
Very simple, I need to swap every 2 lines in a file.
Example
INPUT:
a a a
b b b
x x x
y y y
s s s
t t t
OUTPUT:
b b b
a a a
y y y
x x x
t t t (5 Replies)
HI,
I am using the windows 2003 server R2 in there we are using the putty as to access the different AP's
now from the primary AP i want to login to several different AP's using a script
what the script will do is :-
input a text file in which list of different ap's and the corresponding... (0 Replies)
Hello Folks
I am writing this simple program but I am stuck at this point. Here is the snippet from my script where I have issues.
3) echo "Current Directory: $(pwd) Menu 3"
echo -e "Enter a file name\n"
read fname
if
then
... (5 Replies)