I have a program that is been implemented in C++ which I now want to add MPI support. There is an MPI binding for C++, with namespace MPI and everything.
In my case I have a specific object that is suitable to be the parallelized process into the cluster.
My questions are:
Has anyone done something like this before? Can I get some advices on how best to implement this?
How do I initialize MPI inside the constructor? After initializing MPI inside the constructor of the Class, will all the intermediate calls be parallelized too?
For example:
MyClass obj;
x = x; //this will be parallelized ?
onj.calc();
y = x++; //this will be parallelized ?
z = obj.result();
MPI doesn't parallelize anything automatically, it only gives you an interface for sending data between nodes. Your code is written and runs as usual sequential code independently on each node and every once in a while you send data to some other node or try to receive data from some other node.
I would really recommend picking up the Gropp MPI Book, it really helps for basic MPI!
Chiming in on an old thread, I found OpenMPI and Boost::MPI nice to work with. The object oriented design of the library may be a bit bolted on, but I found it much nicer to work with than pure MPI, especially with auto-serialization of many types and a rather extendable interface for gather/reduce functions as well as serialization of user types.
As background information:
Most applications that use MPI are written in Fortran or C. Every major implementation of MPI is written in C.
Support for MPI C++ bindings is sketchy at best: Some of the MPI Datatypes are not available (e.g. MPI_DOUBLE), there are issues with I/O and the order that headers are included in the source files. There are issues of name mangling if the MPI library was built with C and the application is built with Fortran or C++. mpich2 FAQ has entries to help work through these issues. I am less familiar with Open MPI and it's particular behavior with Fortran and C++.
For your specific questions:
I think that you have a fundamental mis-understanding about what MPI is and is not, and how application code should interact with the MPI libraries.
Parallel Programming with MPI is an excellent reference for learning to program with MPI. The code examples are in C, and most of the MPI API's are shown in an example. I highly recommend that you work through this book to learn what parallel programing is and is not.
Related
I would like to combine Fortran coarray with MPI inside my code. I plan to use third party softwave(HYPRE), which used MPI, for linear system solvers. For the rest of my work, I want to use the Fortran coarray (OpenCoarrays). I've already searched for a solution on Internet. But there isn't any clues about how to make it work. I wonder is that possible to mix Fortran coarray and MPI. If yes, should I use the OpenCoarrays or MPI wrapper compilers
OpenCoarrays sits on top of MPI-3 RMA (at least by default; I don’t recall the latest status of the GASNet port) so this should work, even if neither standard guarantees this. You’ll be using process-parallel execution and they should interoperate fine.
Intel Fortran also uses MPI for coarrays. Cray Fortran coarrays use DMAPP, which is compatible with MPI. Thus, the interoperability you want should cover all the widely available implementations.
In all cases, there may be some implementation quirks, particularly with respect to initialization and termination. You may find that you can’t finalize MPI until all your coarrays are deallocated, for example.
I’m sure the developers of OpenCoarrays would appreciate big reports on this topic if you have problems.
I am implementing a sort and stream compaction algorithms in CUDA C. However I have just figured that it is not that simple to implement those algorithms by myself with good performance. Given that I am working with matrices I cannot use CUDPP, so, although I was avoiding it, I will have to work with thrust library(I know nothing of C++).
I have been programing in C, and I really just want to use C++ to work with thrust, so basically I want to know if I can have most of my code in C and then have little bits of C++ code(I am guessing I will have to use the "external" function) but I wanted to be sure if it's feasible in CUDA.
Thanks in advance.
On the host code side, thrust is simple to integrate. Even though you might think that your host side code in any .cu file you compile is C, it is compiled using a C++ compiler anyway (most of CUDA internally relies on C++ features to compile). So you are actually working in C++ now without realizing it.
Yes, might complicate your build process but otherwise works fine. We use it all the time to wrap up some CUDA functions into C++ class which (and this is the REAL kicker) are then wrapped up with JNI for use in Java. If we can do it, you can do it! Have at it!
I am developing a C++ MPI application. I have some existing code that is a C MPI application which partly do what I want, so I should be able to copy some of the code (or rewrite it in a cleaner C++ way) into my new program. Since the C++ interface to MPI is being deprecated (and it is much harder to find documentation), I am seriously considering using the C interface to MPI in my C++ application. Is it a god idea to mix the C MPI interface with a C++ MPI application, or do I really need to learn to use Boost?
There is no harm in using a C API from a C++ application. Many popular APIs are written in C (the Windows API comes to mind as an example. Or POSIX. Or SQLite, zlib, Python or dozens and dozens of others).
So if that seems like the most convenient solution, go ahead and use the C API. It should be fairly easy to write some thin rappers to C++'ify it a bit yourself.
But apart from this, it is always a good idea for a C++ developer to learn and use Boost. Since Boost has a MPI library, it may, at the very least, be worth checking it out.
And in the end, go with what seems easiest to use for you, in your situation.
I'm writing a C++ project to solve the Travelling salesman problem using genetic algorithms. Naturally, I'd like to make it faster using a bunch (about 40) computers in the same LAN. The computers are all running Windows XP... So, the question is what are the ways to parallelize it using the given equipment
Update:
You've helped me to narrow my choices down to mpich an Open MPI, so the only question left is should I use boost MPI wrappers for them? Also, can you recommend a tutorial for mpich/OpenMPI?
You'll need to use some form of communication to synchronize the processes between the systems. A common API for this type of application is to use Message Passing Interface (MPI). There are quite a few implementations for MPI/MPI-2 that work on Windows XP.
MPI is one of the most used standards for such kind of projects. There are many implementations of this standard from different vendors. I think you should consider two of them:
mpich http://www.mcs.anl.gov/research/projects/mpich2/
Intel MPI http://www.intel.com/go/mpi/
Another interesting way is to use OpenMP. There are Cluster OpenMP implementations. This solution could be easier to implement, but not so scalable as MPI in some cases.
I have a fair hang of programming in various languages. I have been implementing my codes for research using MATLAB (during the past few months) and for the first time really noticed the difference in execution speed of MATLAB v$ C. (As much as I love the blazingly fast prototyping capabilities).
I am looking to pickup C++ and start using it in my research. I am aware of OOP and have programmed fair bit of Java (relatively long back) and C++ (even longer back). I would like to really get deep into C++ now and hence need suggestions for resources on the same:
What C++ things I need to pick up (STLs and. ) to really make good use of C++?
What is a good tutorial/manual to get started with?
What are the numerical/scientific libraries for C++? GSL? Is there a equivalent (features) of Scipy/Numpy for C++?
I shall be programming on Linux, so I shall be using g++ .
Any pointers to previous SO questions also appreciated.
You'll want to get to grips with parallel programming as quickly as possible. For message-passing I like this book by Karniadakis and Kirby. Of the books on OpenMP, for distributed-memory programming, this one is the best.
If you can get access to them, then Intel's Threading Building Blocks, Maths Kernel Library, and Integrated Performance Primitives are good to have. If not, there are plenty of open source alternatives, start looking at Netlib.
Oh, I almost forgot BOOST, which is a must.
In regards to numerical stuff like Numpy, you should have a look at both:
Blitz++ http://www.oonumerics.org/blitz/
and
Jama/TNT http://math.nist.gov/tnt/download.html
On the library side, check out Armadillo. It almost gives you the full extent of MATLAB's array manipulation syntax and uses LAPACK and BLAS (ATLAS) under the hood.
This tutorial absolutely rocks, but you may not want to tackle it initially.
http://www.parashift.com/c++-faq/
Make sure to read up on the STL (standard template library) and other stuff, using sites like:
http://cplusplus.com/
And, check out the Boost library:
http://www.boost.org/
To make really good use of C++, you need to learn at least the STL, that alone will save you lots of time, but as parashift mentions, C++ OOP is only programming with objects, if you don't use dynamic bindings.
TRNG is a parallel random number generation library. It allows you to create multiple independent streams and was designed for use on clusters.