how to do GRID COMPUTING?


 
Thread Tools Search this Thread
Special Forums UNIX and Linux Applications High Performance Computing how to do GRID COMPUTING?
# 1  
Old 01-23-2009
Bug how to do GRID COMPUTING?

Hello,
I want to know how to combine the processing power of given 2 FEDORA machines in LAN.
Can you please tell me the commands,etc used to perform such an operations.Can you please give me the links where I can find more info on this topic.
# 2  
Old 01-23-2009
First, you need to select your choice of clustering software, for example, see:

Comparison of cluster software - Wikipedia, the free encyclopedia

For Fedora, you might consider something like Nightlife:

https://fedoraproject.org/wiki/Nightlife

... which is based on Condor:

Condor Project Homepage
# 3  
Old 01-23-2009
Bug Nightlife

Hello sir,
Your link on Nightlife in fedora project was very informative.Can I download that package and check for my self.
I couldnt find the RPM file.Can you please give me the link to see the RPM file so that i can download and utilize it.

awaiting your reply !!!
# 4  
Old 01-23-2009
The future Fedora Nightlife project is based on Condor:

Here are the installation instructions for Condor:

3.2 Installation

Here is the download site:

Download Condor
# 5  
Old 01-23-2009
Oscar

You might also consider OSCAR, which will run on Fedora as well:

OSCAR - Links
# 6  
Old 01-31-2009
You don't have to install a "cluster" for two (or three or four) machines. Just get something like "pdsh", which will allow you to easily run the same command in parallel across different machines. The command can use its hostname to determine which configuration or data file to read. You can do things like:
Code:
pdsh -w host1,host2 find /

to get all the files on both computers. You just need to enable rsh/rlogin without passwords to work.

There's also a project called "MOSIX" which uses additional hosts' memory as if it were local memory. Processes can be migrated to other computers. No changes to the source code or executables are needed; however, it only works with Linux servers whose kernels have been patched. (Or run a special, pre-built kernel.)

Academic distributed computing usually involves an MPI implementation, such as OpenMPI . This involves writing software using a special API to enable parallelism. You can typically run MPI programs on a heterogeneous network of Linux and non-Linux servers.


Neo mentioned Condor. The main point of Condor is managing resources in a cluster. However, it alone does not enable two or machines to be a "cluster". Further, it's rather pointless to use Condor on a very small network with only one or two users -- like using a logistics company every morning to tell you what order to get up, shower, shave, eat breakfast, take out the trash, and go to work.

He also mentioned Oscar. This is a handy installation tool to help you convert a few (or many) machines into a cluster. I haven't used it, as it was still alpha ware when I last looked at it. If you don't mind starting from scratch, you can build a cluster with Rocks, which I highly recommend. Again, if we're talking only a few hosts, however, it's overkill.

If you have one system, and want to duplicate that installation to a couple of other (nearly identical) hardware, you can use "systemimager-flamethrower" which uses "rsync" to copy one sever's configuration to another.

To actually do GRID computing, you probably need the Globus Toolkit, or perhaps if you're in Europe, Glite. You'll need to get a public/private key pair for yourself that is recognized by one of the Grid communities, and probably ask to get assigned to an existing "Virtual Organization".

All these pieces fit together. The underlying cluster mechanics can be generally either MPI or MOSIX or "dumb" parallelism with pdsh (or dumber with straight rsh), while the resource management and job scheduling can be done with Condor. You use Rocks or Oscar to configure and install the software on your cluster hosts, and then you enable it to do grid/cloud computing using the Globus Toolkit.
Login or Register to Ask a Question

Previous Thread | Next Thread

5 More Discussions You Might Find Interesting

1. High Performance Computing

Alternative to Sun Grid Engine

Does anybody know of a good alternative to Sun Grid Engine? It seems that Oracle is now charging for this software. I am running a HPC cluster that has Solaris 10 machines and I am adding some nodes that will be running Ubuntu 10.04, eventually the Solaris machines will be migrating to Ubuntu. (0 Replies)
Discussion started by: ccj4467
0 Replies

2. Virtualization and Cloud Computing

Event Cloud Computing - IBM Turning Data Centers Into ?Computing Cloud?

Tim Bass Thu, 15 Nov 2007 23:55:07 +0000 *I predict we may experience less*debates*on the use of the term “event cloud”*related to*CEP in the future, now that both IBM and Google* have made announcements about “cloud computing” and “computing cloud”, IBM Turning Data Centers Into ‘Computing... (0 Replies)
Discussion started by: Linux Bot
0 Replies

3. High Performance Computing

Grid vs. Parallel vs. Distributed

Hello all, I was wondering if someone could either explain or maybe point me to another article somewhere that explains the difference between: distributed computing grid computing parallel computing I see these terms thrown around a lot in server and cluster environments, but I'd like a... (2 Replies)
Discussion started by: Heathe_Kyle
2 Replies

4. AIX

GRID containers technology

Hi forum, I am aware that on Hp-Superdome technology (Blade Servers) is avaliable while on Solaris GRID technology is the latest in the market. I would like to know if similar high end technology is also available for AIX ? (1 Reply)
Discussion started by: dheram
1 Replies

5. Shell Programming and Scripting

How to create a grid file

Hi everybody: I want to create a grid file for export to statistical program. My aid is create a file with both rows, one row are x coordenates and other for y coordenates. All grid obviousolly are same space. the form that i want is this: x=(400000 ........ 600000) and y=(4000000 .......... (1 Reply)
Discussion started by: tonet
1 Replies
Login or Register to Ask a Question