Unix/Linux Go Back    


High Performance Computing Message Passing Interface (MPI) programming and tuning, MPI library installation and management, parallel administration tools, cluster monitoring, cluster optimization, and more HPC topics.

Programming parallel for C++ and Python application

High Performance Computing


Tags
gpu, hpc, mpi, multi core

Closed Linux or Unix Question    
 
Thread Tools Search this Thread Display Modes
    #1  
Old Unix and Linux 03-04-2013
figaro figaro is offline
Registered User
 
Join Date: Jan 2007
Last Activity: 16 April 2015, 5:49 PM EDT
Posts: 840
Thanks: 93
Thanked 28 Times in 24 Posts
Programming parallel for C++ and Python application

I am looking to have a C++ and Python application developed using the processing power of at least one GPU.
We are already using Boost, so Boost.MPI comes to mind.
We already have some experience in developing C++ applications for multi-core CPUs by using threading.
However, since we have not developed sufficient expertise in MPI, we were wondering if a Python solution would be more appropriate, because it will give the developers a better understanding of what the code does. In such a case something like mpi4py would come to mind.

Of course having a GPU specifically for simulation purposes could mean that something like PyOpenCL may be even better.
I mention "simulation", because the machine will be calculating variations on what is largely the same problem, ie not be a general purpose machine.

So in summary:
1- For raw processing speed with a multi-core CPU and a GPU, would a openCL solution or Boost.MPI solution be recommended?
2- Should the Python solution be used only for high level functionality, such as handling data upon return, or does Python compromise speed too much?
Sponsored Links
    #2  
Old Unix and Linux 03-05-2013
jim mcnamara jim mcnamara is offline Forum Staff  
...@...
 
Join Date: Feb 2004
Last Activity: 18 April 2015, 9:28 AM EDT
Location: NM
Posts: 10,428
Thanks: 328
Thanked 857 Times in 796 Posts
What you have basically in both Boost and mpi4py is just embedded Openmpi.
IMO since mpi is straightforward (mostly just pragmas) in C/C++ you can tweak things
merely by either recompiling with -D to define some things or actually read in command line arguments.


See here for libraries and documentation:
Open MPI: Open Source High Performance Computing
Sponsored Links
    #3  
Old Unix and Linux 03-06-2013
figaro figaro is offline
Registered User
 
Join Date: Jan 2007
Last Activity: 16 April 2015, 5:49 PM EDT
Posts: 840
Thanks: 93
Thanked 28 Times in 24 Posts
Thank you for your response. What this likely will come down to is setting up the core multi-core calculations in Boost and do some post-processing in Python. The post-processing will take place at a higher level (ie user) and is likely to be more prone to changes.
    #4  
Old Unix and Linux 04-07-2013
figaro figaro is offline
Registered User
 
Join Date: Jan 2007
Last Activity: 16 April 2015, 5:49 PM EDT
Posts: 840
Thanks: 93
Thanked 28 Times in 24 Posts
I am getting concerned that our compiled Boost-code is getting bloated. We will be experimenting with basic C++ functions as if there was only one core and no Boost, then add MPI pragmas to the C++ code and finally add user functionality (screens, post-processing etc) in Python.
Sponsored Links
Closed Linux or Unix Question

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Unix or Linux Image More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
Modifying sniffex.c to include concepts of parallel programming or threads aka.bhagvanji Programming 1 04-14-2009 02:11 PM
ftp application using socket programming toughguy2handle Programming 1 09-23-2005 10:47 PM



All times are GMT -4. The time now is 11:42 AM.