how to do GRID COMPUTING? Post: 302282502

Sponsored Content

Special Forums UNIX and Linux Applications High Performance Computing how to do GRID COMPUTING? Post 302282502 by otheus on Saturday 31st of January 2009 11:24:38 AM

01-31-2009

Registered User

You don't have to install a "cluster" for two (or three or four) machines. Just get something like "pdsh", which will allow you to easily run the same command in parallel across different machines. The command can use its hostname to determine which configuration or data file to read. You can do things like:

Code:

pdsh -w host1,host2 find /

to get all the files on both computers. You just need to enable rsh/rlogin without passwords to work.

There's also a project called "MOSIX" which uses additional hosts' memory as if it were local memory. Processes can be migrated to other computers. No changes to the source code or executables are needed; however, it only works with Linux servers whose kernels have been patched. (Or run a special, pre-built kernel.)

Academic distributed computing usually involves an MPI implementation, such as OpenMPI . This involves writing software using a special API to enable parallelism. You can typically run MPI programs on a heterogeneous network of Linux and non-Linux servers.

Neo mentioned Condor. The main point of Condor is managing resources in a cluster. However, it alone does not enable two or machines to be a "cluster". Further, it's rather pointless to use Condor on a very small network with only one or two users -- like using a logistics company every morning to tell you what order to get up, shower, shave, eat breakfast, take out the trash, and go to work.

He also mentioned Oscar. This is a handy installation tool to help you convert a few (or many) machines into a cluster. I haven't used it, as it was still alpha ware when I last looked at it. If you don't mind starting from scratch, you can build a cluster with Rocks, which I highly recommend. Again, if we're talking only a few hosts, however, it's overkill.

If you have one system, and want to duplicate that installation to a couple of other (nearly identical) hardware, you can use "systemimager-flamethrower" which uses "rsync" to copy one sever's configuration to another.

To actually do GRID computing, you probably need the Globus Toolkit, or perhaps if you're in Europe, Glite. You'll need to get a public/private key pair for yourself that is recognized by one of the Grid communities, and probably ask to get assigned to an existing "Virtual Organization".

All these pieces fit together. The underlying cluster mechanics can be generally either MPI or MOSIX or "dumb" parallelism with pdsh (or dumber with straight rsh), while the resource management and job scheduling can be done with Condor. You use Rocks or Oscar to configure and install the software on your cluster hosts, and then you enable it to do grid/cloud computing using the Globus Toolkit.

otheus

View Public Profile for otheus

Find all posts by otheus

5 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to create a grid file

Hi everybody: I want to create a grid file for export to statistical program. My aid is create a file with both rows, one row are x coordenates and other for y coordenates. All grid obviousolly are same space. the form that i want is this: x=(400000 ........ 600000) and y=(4000000 ..........

2. AIX

GRID containers technology

Hi forum, I am aware that on Hp-Superdome technology (Blade Servers) is avaliable while on Solaris GRID technology is the latest in the market. I would like to know if similar high end technology is also available for AIX ?

3. High Performance Computing

Grid vs. Parallel vs. Distributed

Hello all, I was wondering if someone could either explain or maybe point me to another article somewhere that explains the difference between: distributed computing grid computing parallel computing I see these terms thrown around a lot in server and cluster environments, but I'd like a...

4. Virtualization and Cloud Computing

Event Cloud Computing - IBM Turning Data Centers Into ?Computing Cloud?

Tim Bass Thu, 15 Nov 2007 23:55:07 +0000 *I predict we may experience less*debates*on the use of the term “event cloud”*related to*CEP in the future, now that both IBM and Google* have made announcements about “cloud computing” and “computing cloud”, IBM Turning Data Centers Into ‘Computing...

5. High Performance Computing

Alternative to Sun Grid Engine

Does anybody know of a good alternative to Sun Grid Engine? It seems that Oracle is now charging for this software. I am running a HPC cluster that has Solaris 10 machines and I am adding some nodes that will be running Ubuntu 10.04, eventually the Solaris machines will be migrating to Ubuntu.

LEARN ABOUT DEBIAN

grid-cert-diagnostics

GRID-CERT-DIAGNOST(1)						  Globus Commands					     GRID-CERT-DIAGNOST(1)

NAME

       grid-cert-diagnostics - Print diagnostic information about certificates and keys

SYNOPSIS

       grid-cert-diagnostics [-h] | [-help]  [-p] [-n] [-c CERTIFICATE]

DESCRIPTION

       The grid-cert-diagnostics program displays information about the current user's security environment, including information about
       security-related environment variables, security directory search path, personal key and certificates, and trusted certificates. It is
       intended to provide information to help diagnose problems using GSIC.

       By default, grid-cert-diagnostics prints out information regarding the environment and trusted certificate directory. If the -p
       command-line option is used, then additional information about the current user's default certificate and key will be printed.

       The full set of command-line options to grid-cert-diagnostics consists of:

       -h, -help
	   Display a help message and exit.

       -p
	   Display information about the personal certificate and key that is the current user's default credential.

       -n
	   Check time synchronization with the ntpdate command.

       -c CERTIFICATE, -c -
	   Check the validity of the certificate in the file named by CERTIFICATE or standard input if the parameter to -c is -.

EXAMPLES

       In this example, we see the default mode of checking the default security environment for the system, without processing the user's key and
       certificate. Note the user receives a warning about a cog.properties and about an expired CA certificate.

	   % grid-cert-diagnostics

	   Checking Environment Variables
	   ==============================
	   Checking if X509_CERT_DIR is set... no
	   Checking if X509_USER_CERT is set... no
	   Checking if X509_USER_KEY is set... no
	   Checking if X509_USER_PROXY is set... no

	   Checking Security Directories
	   =======================
	   Determining trusted cert path... /etc/grid-security/certificates
	   Checking for cog.properties... found
	       WARNING: If the cog.properties file contains security properties,
			Java apps will ignore the security paths described in the GSI
			documentation

	   Checking trusted certificates...
	   ================================
	   Getting trusted certificate list...
	   Checking CA file /etc/grid-security/certificates/1c4f4c48.0... ok
	   Verifying certificate chain for "/etc/grid-security/certificates/1c3f2ca8.0"... ok
	   Checking CA file /etc/grid-security/certificates/9d8788eb.0... ok
	   Verifying certificate chain for "/etc/grid-security/certificates/9d8753eb.0"... failed
	       globus_credential: Error verifying credential: Failed to verify credential
	       globus_gsi_callback_module: Could not verify credential
	       globus_gsi_callback_module: The certificate has expired:
	       Credential with subject: /DC=org/DC=example/OU=grid/CN=CA has expired.

       In this example, we show a user with a mismatched private key and certificate:

	   % grid-cert-diagnostics -p

	   Checking Environment Variables
	   ==============================
	   Checking if X509_CERT_DIR is set... no
	   Checking if X509_USER_CERT is set... no
	   Checking if X509_USER_KEY is set... no
	   Checking if X509_USER_PROXY is set... no

	   Checking Security Directories
	   =======================
	   Determining trusted cert path... /etc/grid-security/certificates
	   Checking for cog.properties... not found

	   Checking Default Credentials
	   ==============================
	   Determining certificate and key file names... ok
	   Certificate Path: "/home/juser/.globus/usercert.pem"
	   Key Path: "/home/juser/.globus/userkey.pem"
	   Reading certificate... ok
	   Reading private key...
	   ok
	   Checking Certificate Subject...
	   "/O=Grid/OU=Example/OU=User/CN=Joe User"
	   Checking cert... ok
	   Checking key... ok
	   Checking that certificate contains an RSA key... ok
	   Checking that private key is an RSA key... ok
	   Checking that public and private keys have the same modulus... failed
	   Private key modulus: D294849E37F048C3B5ACEEF2CCDF97D88B679C361E29D5CB5
	   219C3E948F3E530CFC609489759E1D751F0ACFF0515A614276A0F4C11A57D92D7165B8
	   FA64E3140155DE448D45C182F4657DA13EDA288423F5B9D169DFF3822EFD81EB2E6403
	   CE3CB4CCF96B65284D92592BB1673A18354DA241B9AFD7F494E54F63A93E15DCAE2
	   Public key modulus : C002C7B329B13BFA87BAF214EACE3DC3D490165ACEB791790
	   600708C544175D9193C9BAC5AED03B7CB49BB6AE6D29B7E635FAC751E9A6D1CEA98022
	   6F1B63002902D6623A319E4682E7BFB0968DCE962CF218AAD95FAAD6A0BA5C42AA9AAF
	   7FDD32B37C6E2B2FF0E311310AA55FFB9EAFDF5B995C7D9EEAD8D5D81F3531E0AE5
	   Certificate and and private key don't match

AUTHOR

       University of Chicago

Globus Toolkit 5.0.2						    04/25/2011						     GRID-CERT-DIAGNOST(1)