High Performance Computing

Hello,
I want to know how to combine the processing power of given 2 FEDORA machines in LAN.
Can you please tell me the commands,etc used to perform such an operations.Can you please give me the links where I can find more info on this topic.

First, you need to select your choice of clustering software, for example, see:

Comparison of cluster software - Wikipedia, the free encyclopedia

For Fedora, you might consider something like Nightlife:

https://fedoraproject.org/wiki/Nightlife

... which is based on Condor:

Condor Project Homepage

Hello sir,
Your link on Nightlife in fedora project was very informative.Can I download that package and check for my self.
I couldnt find the RPM file.Can you please give me the link to see the RPM file so that i can download and utilize it.

awaiting your reply !!!

The future Fedora Nightlife project is based on Condor:

Here are the installation instructions for Condor:

3.2 Installation

Here is the download site:

Download Condor

You might also consider OSCAR, which will run on Fedora as well:

OSCAR - Links

You don't have to install a "cluster" for two (or three or four) machines. Just get something like "pdsh", which will allow you to easily run the same command in parallel across different machines. The command can use its hostname to determine which configuration or data file to read. You can do things like:

Code:

pdsh -w host1,host2 find /

to get all the files on both computers. You just need to enable rsh/rlogin without passwords to work.

There's also a project called "MOSIX" which uses additional hosts' memory as if it were local memory. Processes can be migrated to other computers. No changes to the source code or executables are needed; however, it only works with Linux servers whose kernels have been patched. (Or run a special, pre-built kernel.)

Academic distributed computing usually involves an MPI implementation, such as OpenMPI . This involves writing software using a special API to enable parallelism. You can typically run MPI programs on a heterogeneous network of Linux and non-Linux servers.


Neo mentioned Condor. The main point of Condor is managing resources in a cluster. However, it alone does not enable two or machines to be a "cluster". Further, it's rather pointless to use Condor on a very small network with only one or two users -- like using a logistics company every morning to tell you what order to get up, shower, shave, eat breakfast, take out the trash, and go to work.

He also mentioned Oscar. This is a handy installation tool to help you convert a few (or many) machines into a cluster. I haven't used it, as it was still alpha ware when I last looked at it. If you don't mind starting from scratch, you can build a cluster with Rocks, which I highly recommend. Again, if we're talking only a few hosts, however, it's overkill.

If you have one system, and want to duplicate that installation to a couple of other (nearly identical) hardware, you can use "systemimager-flamethrower" which uses "rsync" to copy one sever's configuration to another.

To actually do GRID computing, you probably need the Globus Toolkit, or perhaps if you're in Europe, Glite. You'll need to get a public/private key pair for yourself that is recognized by one of the Grid communities, and probably ask to get assigned to an existing "Virtual Organization".

All these pieces fit together. The underlying cluster mechanics can be generally either MPI or MOSIX or "dumb" parallelism with pdsh (or dumber with straight rsh), while the resource management and job scheduling can be done with Condor. You use Rocks or Oscar to configure and install the software on your cluster hosts, and then you enable it to do grid/cloud computing using the Globus Toolkit.