Sponsored Content
Full Discussion: how to do GRID COMPUTING?
Special Forums UNIX and Linux Applications High Performance Computing how to do GRID COMPUTING? Post 302282502 by otheus on Saturday 31st of January 2009 11:24:38 AM
Old 01-31-2009
You don't have to install a "cluster" for two (or three or four) machines. Just get something like "pdsh", which will allow you to easily run the same command in parallel across different machines. The command can use its hostname to determine which configuration or data file to read. You can do things like:
Code:
pdsh -w host1,host2 find /

to get all the files on both computers. You just need to enable rsh/rlogin without passwords to work.

There's also a project called "MOSIX" which uses additional hosts' memory as if it were local memory. Processes can be migrated to other computers. No changes to the source code or executables are needed; however, it only works with Linux servers whose kernels have been patched. (Or run a special, pre-built kernel.)

Academic distributed computing usually involves an MPI implementation, such as OpenMPI . This involves writing software using a special API to enable parallelism. You can typically run MPI programs on a heterogeneous network of Linux and non-Linux servers.


Neo mentioned Condor. The main point of Condor is managing resources in a cluster. However, it alone does not enable two or machines to be a "cluster". Further, it's rather pointless to use Condor on a very small network with only one or two users -- like using a logistics company every morning to tell you what order to get up, shower, shave, eat breakfast, take out the trash, and go to work.

He also mentioned Oscar. This is a handy installation tool to help you convert a few (or many) machines into a cluster. I haven't used it, as it was still alpha ware when I last looked at it. If you don't mind starting from scratch, you can build a cluster with Rocks, which I highly recommend. Again, if we're talking only a few hosts, however, it's overkill.

If you have one system, and want to duplicate that installation to a couple of other (nearly identical) hardware, you can use "systemimager-flamethrower" which uses "rsync" to copy one sever's configuration to another.

To actually do GRID computing, you probably need the Globus Toolkit, or perhaps if you're in Europe, Glite. You'll need to get a public/private key pair for yourself that is recognized by one of the Grid communities, and probably ask to get assigned to an existing "Virtual Organization".

All these pieces fit together. The underlying cluster mechanics can be generally either MPI or MOSIX or "dumb" parallelism with pdsh (or dumber with straight rsh), while the resource management and job scheduling can be done with Condor. You use Rocks or Oscar to configure and install the software on your cluster hosts, and then you enable it to do grid/cloud computing using the Globus Toolkit.
 

5 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to create a grid file

Hi everybody: I want to create a grid file for export to statistical program. My aid is create a file with both rows, one row are x coordenates and other for y coordenates. All grid obviousolly are same space. the form that i want is this: x=(400000 ........ 600000) and y=(4000000 .......... (1 Reply)
Discussion started by: tonet
1 Replies

2. AIX

GRID containers technology

Hi forum, I am aware that on Hp-Superdome technology (Blade Servers) is avaliable while on Solaris GRID technology is the latest in the market. I would like to know if similar high end technology is also available for AIX ? (1 Reply)
Discussion started by: dheram
1 Replies

3. High Performance Computing

Grid vs. Parallel vs. Distributed

Hello all, I was wondering if someone could either explain or maybe point me to another article somewhere that explains the difference between: distributed computing grid computing parallel computing I see these terms thrown around a lot in server and cluster environments, but I'd like a... (2 Replies)
Discussion started by: Heathe_Kyle
2 Replies

4. Virtualization and Cloud Computing

Event Cloud Computing - IBM Turning Data Centers Into ?Computing Cloud?

Tim Bass Thu, 15 Nov 2007 23:55:07 +0000 *I predict we may experience less*debates*on the use of the term “event cloud”*related to*CEP in the future, now that both IBM and Google* have made announcements about “cloud computing” and “computing cloud”, IBM Turning Data Centers Into ‘Computing... (0 Replies)
Discussion started by: Linux Bot
0 Replies

5. High Performance Computing

Alternative to Sun Grid Engine

Does anybody know of a good alternative to Sun Grid Engine? It seems that Oracle is now charging for this software. I am running a HPC cluster that has Solaris 10 machines and I am adding some nodes that will be running Ubuntu 10.04, eventually the Solaris machines will be migrating to Ubuntu. (0 Replies)
Discussion started by: ccj4467
0 Replies
GRID-CERT-REQUEST(1)						  Globus Commands					      GRID-CERT-REQUEST(1)

NAME
grid-cert-request - Generate a X.509 certificate request and corresponding private key SYNOPSIS
grid-cert-request [-help] [-h] [-?] [-usage] [-version] [-versions] grid-cert-request [-cn NAME | -commonname NAME] [-dir DIRECTORY] [-prefix PREFIX] [-nopw | -nodes | -nopassphrase] [-nopw | -nodes | -nopassphrase] [-ca [HASH]] [-verbose] [-interactive | -int] [-force] grid-cert-request -host FQDN [-service SERVICE] [-dns FQDN...] [-ip IP-ADDRESS...] [-dir DIRECTORY] [-prefix PREFIX] [-ca [HASH]] [-verbose] [-interactive | -int] [-force] DESCRIPTION
The grid-cert-request program generates an X.509 Certificate Request and corresponding private key for the specified name, host, or service. It is intended to be used with a CA implemented using the globus_simple_ca package. The default behavior of grid-cert-request is to generate a certificate request and private key for the user running the command. The subject name is derived from the gecos information in the local system's password database, unless the -commonname, -cn, or -host command-line options are used. By default, grid-cert-request writes user certificate requests and keys to the $HOME/.globus directory, and host and service certificate requests and keys to /etc/grid-security. This can be overridden by using the -dir command-line option. The full set of command-line options to grid-cert-request are: -help, -h, -?, -usage Display the command-line options to grid-cert-request and exit. -version, -versions Display the version number of the grid-cert-request command. The second form includes more details. -cn NAME, -commonname NAME Create a certificate request with the common name component of the subject set to NAME. This is used to create user identity certificates. -dir DIRECTORY Write the certificate request and key to files in the directory specified by DIRECTORY. -prefix PREFIX Use the string PREFIX as the base name of the certificate, certificate_request, and key files instead of the default. For a user certificate request, this would mean creating files $HOME/.globus/PREFIXcert_request.pem, $HOME/.globus/PREFIXcert.pem, and $HOME/.globus/PREFIXkey.pem. -ca CA-HASH Use the certificate request configuration for the CA with the name hash CA-HASH instead of the default CA chosen by running grid-default-ca. -verbose Keep the output from the OpenSSL certificate request command visible after it completes, instead of clearing the screen.. -interactive, -int Prompt for each component of the subject name of the request, instead of generating the common name from other command-line options. Note that CAs may not sign certificates for subject names that don't match their signing policies. -force Overwrite any existing certificate request and private key with a new one. -nopw, -nodes, -nopassphrase Create an unencrypted private key for the certificate instead of prompting for a passphrase. This is the default behavior for host or service certificates, but not recommended for user certificates. -host FQDN Create a certificate request for use on a particular host. This option also causes the private key assoicated with the certificate request to be unencrypted. The FQDN argument to this option should be the fully qualified domain name of the host that will use this certificate. The subject name of the certificate will be derived from the FQDN and the service option if specified by the -service command-line option. If the host for the certificate has multiple names, then use either the -dns or -ip command-line options to add alternate names or addresses to the certificates. -service SERVICE Create a certificate request for a particular service on a host. The subject name of the certificate will be derived from the FQDN passed as the argument to the -host command-line option and the SERVICE string. -dns FQDN,... Create a certificate request containing a subjectAltName extension containing one or more host names. This is used when a certificate may be used by multiple virtual servers or if a host has different names when contacted within or outside a private network. Multiple DNS names can be included in the extension by separating then with a comma. -ip IP-ADDRESS,... Create a certificate request containing a subjectAltName extension containing the IP addresses named by the IP-ADDRESS strings. This is used when a certificate may be used by services listening on multiple networks. Multiple IP addresses can be included in the extension by separating then with a comma. EXAMPLES
Create a user certificate request: % grid-cert-request A certificate request and private key is being created. You will be asked to enter a PEM pass phrase. This pass phrase is akin to your account password, and is used to protect your key file. If you forget your pass phrase, you will need to obtain a new certificate. A private key and a certificate request has been generated with the subject: /O=org/OU=example/OU=grid/CN=Joe User If the CN=Joe User is not appropriate, rerun this script with the -force -cn "Common Name" options. Your private key is stored in /home/juser/.globus/userkey.pem Your request is stored in /home/juser/.globus/usercert_request.pem Please e-mail the request to the Example CA ca@grid.example.org You may use a command similar to the following: cat /home/juser/.globus/usercert_request.pem | mail ca@grid.example.org Only use the above if this machine can send AND receive e-mail. if not, please mail using some other method. Your certificate will be mailed to you within two working days. If you receive no response, contact Example CA at ca@grid.example.org Create a host certificate for a host with two names. % grid-cert-request -host grid.example.org -dns grid.example.org,grid-internal.example.org A private host key and a certificate request has been generated with the subject: /O=org/OU=example/OU=grid/CN=host/grid.example.org ---------------------------------------------------------- The private key is stored in /etc/grid-security/hostkey.pem The request is stored in /etc/grid-security/hostcert_request.pem Please e-mail the request to the Example CA ca@grid.example.org You may use a command similar to the following: cat /etc/grid-security/hostcert_request.pem | mail ca@grid.example.org Only use the above if this machine can send AND receive e-mail. if not, please mail using some other method. Your certificate will be mailed to you within two working days. If you receive no response, contact Example CA at ca@grid.example.org ENVIRONMENT VARIABLES
The following environment variables affect the execution of grid-cert-request: X509_CERT_DIR Path to the directory containing SSL configuration files for generating certificate requests. GRID_SECURITY_DIR Path to the directory containing SSL configuration files for generating certificate requests. This value is used if X509_CERT_DIR is not set. GLOBUS_LOCATION Path to the directory containing the Globus Toolkit. This is searched if neither the X509_CERT_DIR nor the GRID_SECURITY_DIR environment variables are set. FILES
$HOME/.globus/usercert_request.pem Default path to write a user certificate request. $HOME/.globus/usercert.pem Default path to write a user certificate. $HOME/.globus/userkey.pem Default path to write a user private key. /etc/grid-security/hostcert_request.pem Default path to write a host certificate request. /etc/grid-security/hostcert.pem Default path to write a host certificate. /etc/grid-security/hostkey.pem Default path to write a host private key. TRUSTED-CERT-DIR/globus-user-ssl.conf, TRUSTED-CERT-DIR/globus-user-ssl.conf.CA-HASH SSL configuration file for requesting a user certificate. The first form is the default location, the second form is used when the -ca command-line option is specified. TRUSTED-CERT-DIR/globus-host-ssl.conf, TRUSTED-CERT-DIR/globus-host-ssl.conf.CA-HASH SSL configuration file for requesting a host or service certificate. The first form is the default location, the second form is used when the -ca command-line option is specified. AUTHOR
University of Chicago Globus Toolkit 5.0.1 03/22/2010 GRID-CERT-REQUEST(1)
All times are GMT -4. The time now is 03:47 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy