Benchmarking a Beowulf Cluster


 
Thread Tools Search this Thread
Special Forums UNIX and Linux Applications High Performance Computing Benchmarking a Beowulf Cluster
# 8  
Old 05-11-2009
Quote:
Originally Posted by otheus
You are getting a different error.
Code:
HPL ERROR from process # 0, on line 621 of function HPL_pdinfo:
>>> Illegal input in file HPL.dat. Exiting ... <<<

Unfortunately, I don't have access to the source code to say what the error is. The HPL.dat file I posted was a suggested sample based on my reading of internet documentation. It could be that documentation is out-of-date or perhaps newer or perhaps just wrong. Check the README and/or sample HPL.dat files that came with you HPL and go through them line-by-line vis-a-vis your own.
Hi Otheus, same error as the other day, just that i accidently didnt copy the part of the error about there needing to be a certain number of processes for the test. The thing is, when i change the P and Q values to 1 and 1 respectively within the HPL.dat file, it works, and it performs the tests. The moment i make them anything other than 1 and 1, such as 2 and 4 to run across 8 processes on 2 nodes, then it gives me the original error message.

Been stumped for weeks now =(

Edit: The full error message being received is still the same as the other one:

"HPL ERROR from process # 0, on line 419 of function HPL_pdinfo:
>>> Need at least 8 processes for these tests <<<

HPL ERROR from process # 0, on line 621 of function HPL_pdinfo:
>>> Illegal input in file HPL.dat. Exiting ... <"

Last edited by mercthunder; 05-11-2009 at 11:27 AM.. Reason: Adding full error message
# 9  
Old 05-11-2009
Did you make sure you compiled with your MPI installation, with mpicc ??
# 10  
Old 05-11-2009
Quote:
Originally Posted by otheus
Did you make sure you compiled with your MPI installation, with mpicc ??
Hi Otheus, thanks for your reply.

By "compiled with your MPI installation", do you mean simply placing openmpi in the execution line, such as:

mpirun -np 8 -machinefile machines xhpl_em64t

That is the line i have been using, to make use of openmpi.

I apologise if this is not what you meant, i am still very new to all this, as well as linux, a total newbie i might add, but decided to take on this challenging task at uni for a research topic, and to learn about linux.

Hoping you can shed some light on what you meant by "compiled with your MPI installation", or how i go about doing this.

Thanks Otheus.
# 11  
Old 05-11-2009
How did you compile and install this program?
# 12  
Old 05-11-2009
Quote:
Originally Posted by otheus
How did you compile and install this program?
Oh the software was already installed by the university. I am using the university cluster to run these tests, with open mpi, and Linpack Benchmark all already installed and have been told by the administrator that they are all working properly, and that i just need to know how to learn to use Linpack and tune the Dat file.

Just that once the values of P and Q are other than 1 and 1 in the dat file, then the errors begin.
# 13  
Old 05-11-2009
Quote:
Oh the software was already installed by the university...have been told by the administrator that they are all working properly
Uh-hunh. I wouldn't completely trust that if I were you. Let's take it step-by-step.
  1. Use "type" or "which" or "whence" to find the full path of the linpack executable:
    Code:
    type -a hpl

  2. verify this has been compiled dynamically and not statically:
    Code:
    file <hpl path from step 1>

    You should see something like "Dynamically linked i386 object". As long as you don't see "statically linked binary" proceed to the next step. Otherwise, talk to your system admin and ask him/her very specifically how he/she compiled it.
  3. Next, run
    Code:
    ldd <hpl path from step 1>

    You should see something like "libopen-rte.so" in the output. If you do not, ask your sysadmin to point you to the correct hpl, the one this is compiled "against" (with) the openMPI runtimes.
  4. The libopen-rte.so should point to the full path of a file. If it does not, again, go to your system administrator and ask him/her for the full LD_LIBRARY_PATH that you should have to run against this hpl program.
  5. The path should be available to you by default on all machines in the cluster. If not, add the LD_LIBRARY_PATH setting into the .bashrc file and include your .bashrc from your .bash_profile (if you're using csh, god help you; if ksh, just change the names to .kshrc and .profile). Now log into the other machine and run the ldd command as above; you should see the line pointing to the full path of the MPI rte library.
  6. Make sure this all works by running:
    Code:
    mpirun -np 8 printenv LD_LIBRARY_PATH

    You should get 8 instances of the correct LD_LIBRARY_PATH.
  7. Now go back and try getting this to run for exactly 2 processes. (P=2, Q=1, -np 2)
  8. Now modify the machine file so it has two lines in it, one for each hostname, and run it again.
  9. If we're at this point, try again with 8; if it fails, there are some other things to look at and try.
# 14  
Old 05-11-2009
Quote:
Originally Posted by otheus
Uh-hunh. I wouldn't completely trust that if I were you. Let's take it step-by-step.
  1. Use "type" or "which" or "whence" to find the full path of the linpack executable:
    Code:
    type -a hpl

Hi Otheus. After typing in which -a hpl, this is what comes up:

/usr/bin/which: no hpl in (/usr/local/openmpi-1.2.6/bin:/usr/local/lam-7.1.4/bin:/usr/local/openmpi-1.2.6/bin:/usr/local/lam-7.1.4/bin:/usr/lib64/qt-3.3/bin:/usr/local/bin:/usr/bin:/bin:/usr/X11R6/bin:/usr/NX/bin:/home/jleungsh/bin:/usr/NX/bin)

Which of those directories is the correct one?

I have skipped down to step 6 in the mean time, and i get this:

[jleungsh@hydrus14 em64t]$ mpirun -np 8 printenv LD_LIBRARY_PATH
/usr/local/openmpi-1.2.6/lib:
/usr/local/openmpi-1.2.6/lib:
/usr/local/openmpi-1.2.6/lib:
/usr/local/openmpi-1.2.6/lib:
/usr/local/openmpi-1.2.6/lib:
/usr/local/openmpi-1.2.6/lib:
/usr/local/openmpi-1.2.6/lib:
/usr/local/openmpi-1.2.6/lib:



Thanks

Last edited by mercthunder; 05-11-2009 at 10:53 PM..
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Solaris

Sun cluster 4.0 - zone cluster failover doubt

Hello experts - I am planning to install a Sun cluster 4.0 zone cluster fail-over. few basic doubts. (1) Where should i install the cluster s/w binaries ?. ( global zone or the container zone where i am planning to install the zone fail-over) (2) Or should i perform the installation on... (0 Replies)
Discussion started by: NVA
0 Replies

2. UNIX and Linux Applications

Benchmarking and performance analyzing in OS

Is/Are there an/some application/applications , package/packages for benchmarking or system performance measuring which are there for almost all Linux releases and distributions? (2 Replies)
Discussion started by: nixhead
2 Replies

3. UNIX for Dummies Questions & Answers

benchmarking application

Where i get a open source benchmark program using pthread library for benchmarking our multicore system for the first stage.I need the source code too for that application ,because in later stage we need to develop our application so that i need to study pthread more. please anybody guide me . (0 Replies)
Discussion started by: sujith4u87
0 Replies

4. UNIX for Advanced & Expert Users

Benchmarking a new Solaris, with four different clients

Good morning, for the impatient: I have a new backup-server and need to monitor, what the machine can do, what's the best way of finding that out? I will tell the story right from the beginning, so you have a clue about what's going on: I have a setup of three machines: A new... (6 Replies)
Discussion started by: PatrickBaer
6 Replies

5. High Performance Computing

Tuning HPL.dat for Beowulf Cluster [Linpack]

Hi guys, I am having some issues tuning the HPL.dat file for the Linpack benchmark test across 2 nodes. I am very new to this with minimal Linux experience, however i am trying my luck. The specs for the two nodes are: 3GHZ QX6850 CORE 2 EXTREME (QUAD CORE) 4GB RAM I have been typing these... (1 Reply)
Discussion started by: mercthunder
1 Replies

6. High Performance Computing

MySQL Cluster - Designing, Evaluating and Benchmarking (reg. req'd)

Registration is required. In this white paper learn the fundamentals of how to design and select the proper components for a successful MySQL Cluster evaluation. Explore hardware, networking and software requirements. Work through basic functional testing and evaluation best practices. More... (0 Replies)
Discussion started by: Linux Bot
0 Replies

7. High Performance Computing

Building a Solaris Cluster Express cluster in a VirtualBox on OpenSolaris

Provides a description of how to set up a Solaris Cluster Express cluster in a VirtualBox on OpenSolaris. More... (0 Replies)
Discussion started by: Linux Bot
0 Replies

8. HP-UX

HP-Unix Hardware benchmarking

Hi everyone, I'm working on one HP-Unix application which we have to port completely onto Windows xp. Before that I have to compare performance of two different machines. My HP-Unix is running on HP-C8000 workstation and windows XP machine is intel Xeon. Now the problem is to evaluate the... (0 Replies)
Discussion started by: dgatkal
0 Replies

9. UNIX Benchmarks

Server and Workstation benchmarking

This is from my server AMD K6 133MHz 64Mb RAM 4GB HDD (Maxtor - ATA33) 2x10Mb NIC 1Mb Intel Graphic Card BYTE UNIX Benchmarks (Version 3.11) System -- FreeBSD sergiu.tarnita.net 5.0-RELEASE FreeBSD 5.0-RELEASE #2: Thu Mar 17 15:49:16 EET 2005... (0 Replies)
Discussion started by: Sergiu-IT
0 Replies
Login or Register to Ask a Question