how to do GRID COMPUTING? Post: 302282502

Sponsored Content

Special Forums UNIX and Linux Applications High Performance Computing how to do GRID COMPUTING? Post 302282502 by otheus on Saturday 31st of January 2009 11:24:38 AM

01-31-2009

Registered User

You don't have to install a "cluster" for two (or three or four) machines. Just get something like "pdsh", which will allow you to easily run the same command in parallel across different machines. The command can use its hostname to determine which configuration or data file to read. You can do things like:

Code:

pdsh -w host1,host2 find /

to get all the files on both computers. You just need to enable rsh/rlogin without passwords to work.

There's also a project called "MOSIX" which uses additional hosts' memory as if it were local memory. Processes can be migrated to other computers. No changes to the source code or executables are needed; however, it only works with Linux servers whose kernels have been patched. (Or run a special, pre-built kernel.)

Academic distributed computing usually involves an MPI implementation, such as OpenMPI . This involves writing software using a special API to enable parallelism. You can typically run MPI programs on a heterogeneous network of Linux and non-Linux servers.

Neo mentioned Condor. The main point of Condor is managing resources in a cluster. However, it alone does not enable two or machines to be a "cluster". Further, it's rather pointless to use Condor on a very small network with only one or two users -- like using a logistics company every morning to tell you what order to get up, shower, shave, eat breakfast, take out the trash, and go to work.

He also mentioned Oscar. This is a handy installation tool to help you convert a few (or many) machines into a cluster. I haven't used it, as it was still alpha ware when I last looked at it. If you don't mind starting from scratch, you can build a cluster with Rocks, which I highly recommend. Again, if we're talking only a few hosts, however, it's overkill.

If you have one system, and want to duplicate that installation to a couple of other (nearly identical) hardware, you can use "systemimager-flamethrower" which uses "rsync" to copy one sever's configuration to another.

To actually do GRID computing, you probably need the Globus Toolkit, or perhaps if you're in Europe, Glite. You'll need to get a public/private key pair for yourself that is recognized by one of the Grid communities, and probably ask to get assigned to an existing "Virtual Organization".

All these pieces fit together. The underlying cluster mechanics can be generally either MPI or MOSIX or "dumb" parallelism with pdsh (or dumber with straight rsh), while the resource management and job scheduling can be done with Condor. You use Rocks or Oscar to configure and install the software on your cluster hosts, and then you enable it to do grid/cloud computing using the Globus Toolkit.

otheus

View Public Profile for otheus

Find all posts by otheus

5 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to create a grid file

Hi everybody: I want to create a grid file for export to statistical program. My aid is create a file with both rows, one row are x coordenates and other for y coordenates. All grid obviousolly are same space. the form that i want is this: x=(400000 ........ 600000) and y=(4000000 ..........

2. AIX

GRID containers technology

Hi forum, I am aware that on Hp-Superdome technology (Blade Servers) is avaliable while on Solaris GRID technology is the latest in the market. I would like to know if similar high end technology is also available for AIX ?

3. High Performance Computing

Grid vs. Parallel vs. Distributed

Hello all, I was wondering if someone could either explain or maybe point me to another article somewhere that explains the difference between: distributed computing grid computing parallel computing I see these terms thrown around a lot in server and cluster environments, but I'd like a...

4. Virtualization and Cloud Computing

Event Cloud Computing - IBM Turning Data Centers Into ?Computing Cloud?

Tim Bass Thu, 15 Nov 2007 23:55:07 +0000 *I predict we may experience less*debates*on the use of the term “event cloud”*related to*CEP in the future, now that both IBM and Google* have made announcements about “cloud computing” and “computing cloud”, IBM Turning Data Centers Into ‘Computing...

5. High Performance Computing

Alternative to Sun Grid Engine

Does anybody know of a good alternative to Sun Grid Engine? It seems that Oracle is now charging for this software. I am running a HPC cluster that has Solaris 10 machines and I am adding some nodes that will be running Ubuntu 10.04, eventually the Solaris machines will be migrating to Ubuntu.

LEARN ABOUT DEBIAN

grid-proxy-destroy

GRID-PROXY-DESTROY(1)						  Globus Commands					     GRID-PROXY-DESTROY(1)

NAME

       grid-proxy-destroy - Destroy the default proxy certificate

SYNOPSIS

       grid-proxy-destroy [-help] [-usage] [-version]

       grid-proxy-destroy [-debug] [-dryrun] [-default] [-all] [--] [FILENAME...]

DESCRIPTION

       The grid-proxy-destroy program removes X.509 proxy files from the local filesystem. It overwrites the data in the files and removes the
       files from the filesystem. By default, it removes the current user's default proxy (either /tmp/x509up_uUID where UID is the current POSIX
       user id, or the file pointed to by the X509_USER_PROXY environment variable) unless a list of proxy file paths are included as part of the
       command line.

       Use the -- command-line option to separate a list of proxy paths from command line options if the proxy file begins with the - character.

       The full list of command-line options to grid-proxy-destroy are:

       -help, -usage
	   Display the command-line options to grid-proxy-destroy.

       -version
	   Display the version number of the grid-proxy-destroy command

       -debug
	   Display verbose error messages.

       -dryrun
	   Do not remove the proxy, but display the path of the files that would have been removed, or the directory where they would have been
	   removed from if the -all command-line option is used.

       -default
	   Remove the default proxy in addition to the files included on the command-line. Only needed if other paths are included on the
	   command-line.

       -all
	   Remove the default proxy and all delegated proxies in the temporary file directory.

ENVIRONMENT VARIABLES

       The following environment variables affect the execution of grid-proxy-destroy:

       X509_USER_PROXY
	   Path to the default user proxy.

SEE ALSO

       grid-proxy-init(1), grid-proxy-info(1)

AUTHOR

       University of Chicago

Globus Toolkit 5.0.2						    04/25/2011						     GRID-PROXY-DESTROY(1)