Quote:
Originally Posted by
pc2001
Sorry, a noobie question....!
I want to use a linux cluster to copy a list of files. I want to split the processing over 3 nodes so that each node gets (more or less) an equal share.
Let's stop and think about this. Three CPU's can run programs faster, because they can run independently of each other. But how exactly are three
CPU's going to speed up your
disk? It doesn't matter if your CPU's can send disk-read commands faster than light, the disk can only do so much, and
One CPU is going to max that out. Three may start it trashing --
slowing it down.
Additionally -- that you're having problems copying lots of files fast enough tells me there may be another problem here, like millions of files crammed in one folder, which multithreading cannot solve either.
There are a few
very specific circumstances where multiple threads may speed this up -- a NAS with independent dedicated links, or some odd kinds of software RAID -- but I don't consider this likely without being told.
Back up and tell me more about your system and the problem you are trying to solve, please.