As I understand, OpenMP is a standard and also a library to implement multi-threading in C++ code.
Visual C++ already has threading APIs for Windows, and UNIX has POSIX threading. I don't undertand why it is required, or in which scenario it is applicable to use OpenMP.
EDIT : Does OpenMP has improved performance too, in comparision to using CreateThread or other POSIX functions (assuming similar code was parellilized)?
System threading APIs (such as POSIX threads) require you to do an awful lot of work manually (setting up the threads, splitting up the work between the threads, synchronising when they are complete, tearing down the threads, etc.). Lots and lots of code bloat that obscures what you're really trying to do. And error-prone. And tedious. And platform-dependent.
OpenMP does all of this for you. In my opinion, it's most suitable for data-parallelism; in many cases, it's as simple as putting a #pragma omp directive before e.g. a for loop, and that loop will be automatically multi-threaded. But it can also be used for task-parallelism as well.
OpenMP doesn't improve performance, in the sense that it's always possible to write manual threading code that performs at least as well as the OpenMP version. But very often, OpenMP will get you 90%+ of theoretical optimum performance, with 5 minutes of coding effort (assuming you have written your loops in a thread-friendly way in the first place).
I recommend reading the Wikipedia article for some good examples.
When you're trying to do a portable code for example. OpenMP works both on windows and unix systems.
Moreover, it's most of the time a lot easier to use than manipulating threads.
Related
I would like to parallelize a Qt/C++ program with OpenMP, so I can compare against Qt threading tools. I have some questions.
What do I have to include, both in code and in project files to have OpenMP working properly?
Would it be painful to use OpenMP in a deliverable software project? Should each versions include updates for OpenMP and much maintenance?
What do you experienced as performance with OpenMP?
Is nested parallel work with OpenMP trustworthy?
Is OpenMP supported on the same platforms as Qt?
Any references would be appreciated. Thanks a lot!
What do I have to include, both in code and in project files to have OpenMP working properly?
You'll have to introduce OpenMP pragmas in the code, and possibly to link against the OpenMP runtime library (which will introduce limited changes in your build system).
Would it be painful to use OpenMP in a deliverable software project? Should each versions include updates for OpenMP and much maintenance?
I'm not sure what you mean by "painful". I know a lot of projects successfully using OpenMP. There might be some maintenance needed from time to time (but I guess this is also true for Qt).
What do you experienced as performance with OpenMP?
More or less what should be expected of any good thread-based parallelization tool. If the workload is sufficiently heavy, OpenMP in itself should not add much overhead to your code and Amdahl's law will be your limit.
Is nested parallel work with OpenMP trustworthy?
Yes
Is OpenMP supported on the same platforms as Qt?
Unlike Qt, which is a framework, OpenMP support is mostly done by compilers. You might find platforms on which Qt is compilable, but where the C++ compilers don't support OpenMP. Of course, it depends on what type of platforms you target.
I want to learn multi-threading in C++ but I'm not sure what type would be most useful. The ones I've seen tutorials on are:
Windows native calls
OpenMP
Boost
(I'm sure that there are probably more.)
What is each one's key features and what are they best used for?
Note: I've already done some multi-threading in C# by manually creating the threads, and more complexity of the threading will just make it more fun. :)
I'd start with pthreads if you have more of a C background, or Boost Thread if you are accustomed to more idiomatic C++. Either is reasonably portable and widely used.
How about TBB? It is portable and has easy to use parallel template patterns, concurrent containers, task scheduler and scalable memory allocaturs. TBB will let you manage threads directly, but that is not necessary in most of the cases.
Personally I would stay away from platform specific threads, unless there an urgent need to do something, well, platform specific.
Boost threads is portable and easy to use, but does have neither parallel patterns nor concurrent containers. You would need to manager threads manually, which can get ugly pretty quickly.
PThreads isn't available on Windows and its C. You really want to do multi-threading in C++, not C. RAII mixes well with mutexes and scoped locks.
Another option is PPL in Visual C++ 2010. It is similar to TBB, but as you may guess available for Windows only.
OpenMP is easy to use, but not very flexible. Since you already learned C++, you should use something more serious, such as TBB or PPL. For some strange reason Visual C++ 2010 doesn't support OpenMP 3. Too bad.
If you want to be portable, learn Posix threads. You know, all thread libraries provide more or less the same set of features, so it's up to you, but Posix will give you the basis.
openMP isn't exactly "multi-threading" as you mean it.
WinThreads (Windows) and pthreads (Linux) are POSIX threads and represent probably your best choice to get started. It is important to learn the distinction between processes and threads, then learn about the various memory access models that are associated with them. Next, try concurrency approaches like OpenMP and MPI "threads".
There are some basic concepts that will get repeated. Learn them well.
I'd like to start learning multithreading in C++. I'm learning it in Java as well. In Java, if I write a program which uses multithreading, it will work anywhere. However in C++, doesn't multithreading rely on platform-specific API's? If so, that would seem to get in the way of portability.
How can I do multithreading in C++ without causing portability issues? Is boost's thread library a good solution?
As a sidenote - how is it even possible to implement multithreading as a library? Isn't that something that has to be done by the compiler?
If you do not have a compiler that supports C++0x yet (available with visual studio c++ 2010 for example), use boost threads. (Unless you use a framework that already supports threading, which is not the case - you wouldn't ask the question otherwise -). These boost threads became actually the standard in the brand new C++. Before that time C++ itself was thread unaware.
TBB Threading Building Blocks might also be interesting to you if you want to learn other aspects in parallel programming.
Regarding Qt: if you only want threading support it is complete overkill. It has horribly slow round trip times from compiling to result. It is really well designed thought. But not an official standard like the C++0x threads from boost. Therefore I would not take it as a first choice.
In C++, yes threading is platform specific. However, many threading libraries encapsulate the nuances of threading on various platforms, providing a uniform API with which to write your threaded app, so you don't have to worry about platform specific details.
Boost threading library is a very good solution.
I also recommending checking out ACE as well.
Let's start backward:
How is it possible to implement threading in a library ?
It isn't, at least not in (pure) C++. This requires language support (the compiler is only an implementation).
At the moment 2 things are used:
assembly code for some parts (like in the pthread library)
specific compiler instructions for others (dependent on the compiler and platform)
Both are brittle and require a tremendous amount of work for portability. Basically it means lot of #ifdef portions in the code to test for the compiler and targetted architecture, test for the support of some directives etc...
That is why it was deemed necessary to add threading support in C++0x.
How do I do multithreading ?
Even before you choose a library, you should choose a method. There are 2 ways of programming multithreaded applications (and you can combine them):
Communicate by sharing: this means using mutexes, atomic operations, etc... you can use pthread on Linux platforms, but I would recommend Boost.Thread (among others) for its portability.
Share by communicating: more recent, and adapted to distributed computations, this stems from the functional languages. This means passing messages from one thread to another and not sharing any resources. You can use FastFlow or Intel's Thread Building Blocks aka TBB.
You can conceivably merge the two, but it would be better not to. Personally I have found the description of FastFlow totally awesome: it encourages lock-free programming. Also, the main advantage of the second method is that it's better adapted to multi-processes programming and scales to distributed environments.
To begin with, I would recommend focusing on either one and build some applications with it. When you're comfortable you may try out the other, but be ready to start anew, they are that different.
//This program explains how pthread works, here 5 thread are trying to update a global variable simultaneously but with locking synchronization in maintained
#include<iostream>
#include<pthread.h>
using namespace std ;
#define MAX_NO_THREAD 5
int global_sum = 0 ;
pthread_mutex_t lock ; //Declared global lock mutex object
void *print_fun(void *arg)
{
cout<<"\nThread id : "<<(int)arg;
pthread_mutex_lock(&lock) ; //aquiring lock on piece of code
for ( int j=0; j<100000000; j++)
{
global_sum++ ;
}
pthread_mutex_unlock(&lock) ; //reomving lock on peice of code
cout<<"\nGlobal Sum : "<<global_sum ;
}
int main()
{
int i = 0 ;
pthread_t threads_obj[MAX_NO_THREAD] ; //Initializing object array for thread
pthread_mutex_init(&lock, NULL) ; //Initalinzing lock object for thread
int st ;
for ( i=0; i<5; i++)
{
pthread_create(&threads_obj[i], NULL, *print_fun, (void *)i) ;//Initializing threads calling function print_fun
pthread_join(threads_obj[i], 0) ; //Forcing main thread to main until these thread complete
}
pthread_mutex_destroy(&lock) ; //Destroying lock object
}
//compile this program using -lpthread option with g++
//g++ thread.cc -lpthread
also check out
Qt
To offer a suggestion different from Boost, I use Pthreads (or Pthreads-Win32 on Windows). It's a very do-it-yourself barebones library, but provides you with all you need and nothing else. It's very lightweight compared to Boost and you can easily find C++ wrappers around it to give you higher level abstractions.
you might also consider openmp http://openmp.org. Many compilers support it, including MS, GCC/G++ and Intel. Although you don't get explicit control of threads, its higher level abstraction of parallelism is sometimes more efficient (at coding time as well as runtime), and the code is much easier to understand. It's not going to help you much if you're doing GUI work, but for scalable computation it's worth a look.
The Boost Threading Library is probably the best place to start for C++. Gives you threading contructs, as well as all the mutexes and control objects you need to write a real working multithreaded application.
With C++11, "Thread support library" is introduced under <thread> header.
More info could be found from below links
https://en.cppreference.com/w/cpp/thread
http://www.cplusplus.com/reference/thread/
If you're doing this out of interest to improve your knowledge of different programming models and language skills then the Boost library would be a fine way to go. However I would think long and hard about actually building any production applications using multi-threaded C++.
C++ is challenging enough at times to attain correctness without adding the considerable complexity of shared memory multi-threading. Even the most seasoned programmers would agree that multi-threaded programs are extremely hard to reason about and get right. Even the simplest programs can quickly become hard to test and debug when multi-threaded.
Imperative languages such as C++ or Java or C# (with their mutable variables, shared memory and locking/signalling primitives) are very often the least approachable way to try to build multi-threaded applications. There are usually perfectly good single threaded implementation options to accomplish most user-space (as opposed to kernel or embedded) application problems, including on multi-core machines.
If you really want to build reliable "multi-threaded" applications I would suggest you check out functional languages like Erlang, Haskell, F# or Clojure.
I'm experimenting with multithreading in Windows and was wondering whether I should
use Win32 API
use POSIX Threads for Windows
Learning Pthreads would be useful if I tried to develop such applications on different platforms - but am I losing anything by not learning Win32 API? Or are both similar enough so that learning one allows me to figure out the other easily?
Use Boost Threads. When C++0x comes along, we will have std::threads. Boost threads has the closest implementation to std threads.
else use pthreads. Pthreads is second closest to std::threads, and formed the main basis of std threads and boost threads.
else do windows threading directly. You can still learn how threads work, and form a mental model of things. It just tends to use synchronization primitives that are a bit non-standard.
If you're going to do much Windows programming, it will pay to learn the basic Win32 threading constructs: critical sections, interlocked functions, CreateThread, WaitFor*Object, etc. These aren't hard to understand, and they translate transparently to equivalent objects in other threading frameworks.
However, for more advanced threading constructs such as semaphores, events, etc., I would use the pthreads library, since the documentation on those tends to be clearer and examples more abundant.
If you're using C/C++, try to use the thread functions of the C/C++ runtime.
If you use the Win32 (or other non-CRT functions to create threads) the CRT might not be initialized correctly in the new thread, causing all kinds of problem (you can read about it here: http://www.codeguru.com/forum/archive/index.php/t-371305.html).
However, most thread-functions (in the CRT, Win32 or pthread) are all based around the functionality to create threads, synchronize threads and destroy threads. In practice this isn't always that easy to use.
In the last year, there is a trend to go to task-based threading (well, I call it that way, I don't know what the official name is). Instead of starting a thread and then executing some logic in it, in task-based threading you create a task and then ask the 'threading logic' to execute the task.
Systems that support this new way of working with threads are:
Visual Studio 2010 (we'll have to wait a few days for it)
Intel Threading Building Blocks
Visual Studio 2010 even has (it seems) special debugging logic to debug the 'parallel tasks'.
Take a look at std::thread
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2184.html
And a introduction
http://www.devx.com/SpecialReports/Article/38883
I've found that sticking with pthreads saves my sanity on three counts:
I don't have to fight through
WinAPI docs, which aren't habitually
of any quality.
Anyone that does much with threads
can help out with
pthreads. I've found infinitely more good sources of information about pthreads online.
Whenever I implement anything more
complicated that "Hello World" with
the WinAPI, I find it takes far
longer than one could reasonably
expect. That's just my empirical
input, though.
As far as capabilities are concerned, I've never found pthreads to be lacking in anything, so I don't think I've ever found the need to look elsewhere. There is also a great deal to be said for learning a library that you'll be able to use in any environment you tackle.
I am familiar with concurrent programming in Java which provides a lot of tools for this. However, C++ concurrent programming isn't so easy to start using.
What is the best way to start programming concurrently on C++? Are there any nice libraries which wrap concurrent programming primitives and provide you with more high-level constructs?
I tried QtConcurrent which provides you with nice MapReduce functionality but it is heavily biased towards concurrent computation using Qt so it's not a good choice if you don't want to use Qt.
Are there any other similar libraries? What do people use here?
Thanks for your help in advance,
sneg
There are several choices:
ACE which provides some concurrency constructs
Intel Threading Building Blocks
boost::threads
OpenMP
Qt Threading libraries
Morendil's suggestion (CSP - communicating sequential processes) is truly interesting to take a look at - it's a very different view of threading and one that works well once you wrap your head around it. I first encountered it in the rather esoteric Occam language for Transputers, but the idea has stuck with me.
A more conventional idea: boost::threads work quite well for building thread-based concurrent programs. It's quite low level though.
OpenMP is at a higher level than threads and also quite well-supported.
You could look at CSP, which has a C++ implementation. Way different from Java's threading primitives, though.
This question along with the answers can probably help you a little bit.
Intel's Threading Building Blocks is great for introducing concurrency at the level of individual data-parallel loops, and it takes care of managing threads and allocating work automagically. It can be used in similar ways to OpenMP, but without the need for explicit compiler support.