Programming parallel for C++ and Python application | Unix Linux Forums | High Performance Computing

  Go Back    


High Performance Computing Message Passing Interface (MPI) programming and tuning, MPI library installation and management, parallel administration tools, cluster monitoring, cluster optimization, and more HPC topics.

Programming parallel for C++ and Python application

High Performance Computing


Tags
gpu, hpc, mpi, multi core

Closed Thread    
 
Thread Tools Search this Thread Display Modes
    #1  
Old 03-04-2013
figaro figaro is offline
Registered User
 
Join Date: Jan 2007
Last Activity: 26 August 2014, 2:37 PM EDT
Posts: 818
Thanks: 92
Thanked 25 Times in 22 Posts
Programming parallel for C++ and Python application

I am looking to have a C++ and Python application developed using the processing power of at least one GPU.
We are already using Boost, so Boost.MPI comes to mind.
We already have some experience in developing C++ applications for multi-core CPUs by using threading.
However, since we have not developed sufficient expertise in MPI, we were wondering if a Python solution would be more appropriate, because it will give the developers a better understanding of what the code does. In such a case something like mpi4py would come to mind.

Of course having a GPU specifically for simulation purposes could mean that something like PyOpenCL may be even better.
I mention "simulation", because the machine will be calculating variations on what is largely the same problem, ie not be a general purpose machine.

So in summary:
1- For raw processing speed with a multi-core CPU and a GPU, would a openCL solution or Boost.MPI solution be recommended?
2- Should the Python solution be used only for high level functionality, such as handling data upon return, or does Python compromise speed too much?
Sponsored Links
    #2  
Old 03-05-2013
jim mcnamara jim mcnamara is offline Forum Staff  
...@...
 
Join Date: Feb 2004
Last Activity: 28 August 2014, 5:04 PM EDT
Location: NM
Posts: 10,180
Thanks: 275
Thanked 785 Times in 734 Posts
What you have basically in both Boost and mpi4py is just embedded Openmpi.
IMO since mpi is straightforward (mostly just pragmas) in C/C++ you can tweak things
merely by either recompiling with -D to define some things or actually read in command line arguments.


See here for libraries and documentation:
Open MPI: Open Source High Performance Computing
Sponsored Links
    #3  
Old 03-06-2013
figaro figaro is offline
Registered User
 
Join Date: Jan 2007
Last Activity: 26 August 2014, 2:37 PM EDT
Posts: 818
Thanks: 92
Thanked 25 Times in 22 Posts
Thank you for your response. What this likely will come down to is setting up the core multi-core calculations in Boost and do some post-processing in Python. The post-processing will take place at a higher level (ie user) and is likely to be more prone to changes.
    #4  
Old 04-07-2013
figaro figaro is offline
Registered User
 
Join Date: Jan 2007
Last Activity: 26 August 2014, 2:37 PM EDT
Posts: 818
Thanks: 92
Thanked 25 Times in 22 Posts
I am getting concerned that our compiled Boost-code is getting bloated. We will be experimenting with basic C++ functions as if there was only one core and no Boost, then add MPI pragmas to the C++ code and finally add user functionality (screens, post-processing etc) in Python.
Sponsored Links
Closed Thread

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
Modifying sniffex.c to include concepts of parallel programming or threads aka.bhagvanji Programming 1 04-14-2009 02:11 PM
ftp application using socket programming toughguy2handle Programming 1 09-23-2005 10:47 PM



All times are GMT -4. The time now is 09:14 PM.