Programming parallel for C++ and Python application


 
Thread Tools Search this Thread
Special Forums UNIX and Linux Applications High Performance Computing Programming parallel for C++ and Python application
# 1  
Old 03-04-2013
Programming parallel for C++ and Python application

I am looking to have a C++ and Python application developed using the processing power of at least one GPU.
We are already using Boost, so Boost.MPI comes to mind.
We already have some experience in developing C++ applications for multi-core CPUs by using threading.
However, since we have not developed sufficient expertise in MPI, we were wondering if a Python solution would be more appropriate, because it will give the developers a better understanding of what the code does. In such a case something like mpi4py would come to mind.

Of course having a GPU specifically for simulation purposes could mean that something like PyOpenCL may be even better.
I mention "simulation", because the machine will be calculating variations on what is largely the same problem, ie not be a general purpose machine.

So in summary:
1- For raw processing speed with a multi-core CPU and a GPU, would a openCL solution or Boost.MPI solution be recommended?
2- Should the Python solution be used only for high level functionality, such as handling data upon return, or does Python compromise speed too much?
# 2  
Old 03-05-2013
What you have basically in both Boost and mpi4py is just embedded Openmpi.
IMO since mpi is straightforward (mostly just pragmas) in C/C++ you can tweak things
merely by either recompiling with -D to define some things or actually read in command line arguments.


See here for libraries and documentation:
Open MPI: Open Source High Performance Computing
# 3  
Old 03-06-2013
Thank you for your response. What this likely will come down to is setting up the core multi-core calculations in Boost and do some post-processing in Python. The post-processing will take place at a higher level (ie user) and is likely to be more prone to changes.
# 4  
Old 04-07-2013
I am getting concerned that our compiled Boost-code is getting bloated. We will be experimenting with basic C++ functions as if there was only one core and no Boost, then add MPI pragmas to the C++ code and finally add user functionality (screens, post-processing etc) in Python.
Login or Register to Ask a Question

Previous Thread | Next Thread

7 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Redploymnet of python application on UNIX Server

I have created the python application, which was deployed on UNIX server but due to some certain issues I have to redeploy the application every time while logging in to the server I am trying to make it work permanently on that server even when I am not using the UNIX server also Any... (1 Reply)
Discussion started by: varul29
1 Replies

2. Shell Programming and Scripting

Python GNU parallel single command on multiple cores

Hello, I have a 4 core machine. Here is my initial script cd /work/ python script.py input.txt output.txt 1 2 3 This script runs for 1.5hrs. So I read across the web and figured out that you can use GNU parallel to submit multiple jobs using parallel. But I am not sure if I can run... (4 Replies)
Discussion started by: jacobs.smith
4 Replies

3. Shell Programming and Scripting

Python Programming for asn1 file

Hi. Has anyone here got an experience doing conversion from asn1 format to a readable format so that it can be processed by Oracle? I want to load the data into a table. This is a CDR file. Attached is the pairing file. Please remove the .txt at the end.... (1 Reply)
Discussion started by: aimy
1 Replies

4. UNIX for Advanced & Expert Users

Python Programming for ASN.1 file

Hi. Has anyone here got an experience doing conversion from asn1 format to a readable format so that it can be processed by Oracle? I want to load the data into a table. This is a CDR file. Attached is the pairing file. Please remove the .txt at the end. Someone said that it is possible... (1 Reply)
Discussion started by: aimy
1 Replies

5. Debian

Programming, Python.

I do some Python programming in windows, understand how to use python in windows... However I do not know how to use python in Linux. I do know the commands n such are the same but there isn't any IDLE or anything its ran straight from the terminal by typing python but I have never used python that... (2 Replies)
Discussion started by: Cryptek
2 Replies

6. Programming

Modifying sniffex.c to include concepts of parallel programming or threads

HI For the past 1 week i have been trying to include the concepts of parallel programming or thread in the sniffex.c code. Haven't been able to.. Please suggest sm appropriate modifications to the following code: /* * sniffex.c * * Sniffer example of TCP/IP packet capture using... (1 Reply)
Discussion started by: aka.bhagvanji
1 Replies

7. Programming

ftp application using socket programming

i have made a ftp application in socket programming which uses TCP/IP .. i have the problem runing the only problem is on the client side i take the user input for the file to be downloaded from the command promt. write(s, argv, strlen(argv)+1); // this is how i write in client side argv is... (1 Reply)
Discussion started by: toughguy2handle
1 Replies
Login or Register to Ask a Question