I am looking for lightweight multi-threading framework for C++. I found POSIX Threads.
Please, share you practical experience with POSIX threads: before I start with it I want to know its pros and cons from real people, not from wiki.
If you practically compared it with anything (maybe, better), it would be interesting to know either.
UPD: cross platform features are really important for me, so I would appreciate this direction described.
UPD2: I already have an experience with one framework for QNX / Win32, but it is not lightweight and - oh, I forgot to mention, - it is commercial, not free, but I need a free one.
I found Boost.Threads to be really nice, especially after the 1.35 rewrite. POSIX threads on Windows is not so trivial, and it's a C API, so I would definitely prefer Boost to it. It has all the stuff you need, is portable and requires little setup.
Another C thread API is GThreads from GLib. There is a 1-to-1 mapping between some gthread and pthread calls such as pthread_create, but gthreads have 2 big features that I have found very useful:
thread pools and
asynchronous queues for sending messages between threads.
The thread pools are very powerful, allowing things like dynamic resizing of the pool. See http://library.gnome.org/devel/glib/2.20/glib-Threads.html
The POSIX threading API is a C API, not C++.
What do you want to use it for? Personally, I find it to be a very clumsy and overly verbose API. But it is your best bet if you want to do cross-platform development on Unix/Linux-like operating systems. It is not natively supported on Windows.
Personally, I would not use a threading or any other OS dependent API directly in your code. Build another abstraction layer on top of it. For example, we built what we call an "OS layer"; a C++ framework for working with threads, semaphores, timers, mutexes, etc. Our code uses this exclusively. Underneath the hood, we have implementations for POSIX, Win32, INTEGRITY, and vxWorks. This lets our code work on a large variety of platforms.
If you don't want to build your own layer, you can look towards reusing many others like Boost, Qt, etc.
I used POSIX a while ago for a program I wrote. It worked fine on Linux and Solaris and it's not terribly complicated to implement. My brother on the other hand is a Windows programmer and preferred boost to Posix. I guess it depends on your target. I found boost to be a bit on the bloated side and had heard bad things about it. My brother thinks it's the greatest thing since sliced bread. I suppose it's a ford vs chevy thing. Everyone will have an opinion.
If you don't like Boost's thread API, then you might want to look at POCO's.
As you are mentioning QNX have a look at ACE. It is a vast framework that is available for many platforms (including QNX).
Others have already mentioned Boost.
You are well advised to use one of these libraries instead of the low level, non portable and error prone C APIs.
Boost threads library is probably your best bet if you work in C++. I had very positive experience with it both on Unix and win32. Avoid ACE - bad design, wrong approach. Also take a look at Intel TBB, though I haven't used it in practice.
I've found it to be pretty similar to the win32 thread API, the only (real) difference you need to be aware of is that win32 mutexes don't block when used on the same thread while posix do. Apart from that, it's a pretty straight forward API.
Related
I want to learn multi-threading in C++ but I'm not sure what type would be most useful. The ones I've seen tutorials on are:
Windows native calls
OpenMP
Boost
(I'm sure that there are probably more.)
What is each one's key features and what are they best used for?
Note: I've already done some multi-threading in C# by manually creating the threads, and more complexity of the threading will just make it more fun. :)
I'd start with pthreads if you have more of a C background, or Boost Thread if you are accustomed to more idiomatic C++. Either is reasonably portable and widely used.
How about TBB? It is portable and has easy to use parallel template patterns, concurrent containers, task scheduler and scalable memory allocaturs. TBB will let you manage threads directly, but that is not necessary in most of the cases.
Personally I would stay away from platform specific threads, unless there an urgent need to do something, well, platform specific.
Boost threads is portable and easy to use, but does have neither parallel patterns nor concurrent containers. You would need to manager threads manually, which can get ugly pretty quickly.
PThreads isn't available on Windows and its C. You really want to do multi-threading in C++, not C. RAII mixes well with mutexes and scoped locks.
Another option is PPL in Visual C++ 2010. It is similar to TBB, but as you may guess available for Windows only.
OpenMP is easy to use, but not very flexible. Since you already learned C++, you should use something more serious, such as TBB or PPL. For some strange reason Visual C++ 2010 doesn't support OpenMP 3. Too bad.
If you want to be portable, learn Posix threads. You know, all thread libraries provide more or less the same set of features, so it's up to you, but Posix will give you the basis.
openMP isn't exactly "multi-threading" as you mean it.
WinThreads (Windows) and pthreads (Linux) are POSIX threads and represent probably your best choice to get started. It is important to learn the distinction between processes and threads, then learn about the various memory access models that are associated with them. Next, try concurrency approaches like OpenMP and MPI "threads".
There are some basic concepts that will get repeated. Learn them well.
I am looking for a good and stable threadpool library for C++ that's fairly well documented. I know about the Native Windows thread pool API and the newer Vista Thread Pool API, however my program requires some backward compatibility, so perhaps an outside library I can provide with the program is better.
I have looked into Boost's threadpool and it doesn't look bad at all, unfortunatly it is not very well documented.
Does anyone know any other libraries that have a ThreadPool in C++? (for Windows)
A portable threadpool library that claims to be 'production ready'. You may want to check that out.
Intel TBB is another threading library that has some neat stuff. I find the framework for evaluating a tree of expressions in parallell especially nice.
Qt has a threading library with some nice high-level operations like map/reduce etc, as well as low-level threading stuff and thread-pool support.
Qt might be a bit big for you though, but you can use a part of it pretty easily.
Have a look at the ThreadPool and TaskManager classes from the Poco C++ libraries.
With respect to the boost thread pool: this link might be useful: http://think-async.com/Asio/Recipes
There's also ACE which does thread-pooling over networks, so it's a fair bit more complex. (but deserves mentioning here, IMO)
I'm experimenting with multithreading in Windows and was wondering whether I should
use Win32 API
use POSIX Threads for Windows
Learning Pthreads would be useful if I tried to develop such applications on different platforms - but am I losing anything by not learning Win32 API? Or are both similar enough so that learning one allows me to figure out the other easily?
Use Boost Threads. When C++0x comes along, we will have std::threads. Boost threads has the closest implementation to std threads.
else use pthreads. Pthreads is second closest to std::threads, and formed the main basis of std threads and boost threads.
else do windows threading directly. You can still learn how threads work, and form a mental model of things. It just tends to use synchronization primitives that are a bit non-standard.
If you're going to do much Windows programming, it will pay to learn the basic Win32 threading constructs: critical sections, interlocked functions, CreateThread, WaitFor*Object, etc. These aren't hard to understand, and they translate transparently to equivalent objects in other threading frameworks.
However, for more advanced threading constructs such as semaphores, events, etc., I would use the pthreads library, since the documentation on those tends to be clearer and examples more abundant.
If you're using C/C++, try to use the thread functions of the C/C++ runtime.
If you use the Win32 (or other non-CRT functions to create threads) the CRT might not be initialized correctly in the new thread, causing all kinds of problem (you can read about it here: http://www.codeguru.com/forum/archive/index.php/t-371305.html).
However, most thread-functions (in the CRT, Win32 or pthread) are all based around the functionality to create threads, synchronize threads and destroy threads. In practice this isn't always that easy to use.
In the last year, there is a trend to go to task-based threading (well, I call it that way, I don't know what the official name is). Instead of starting a thread and then executing some logic in it, in task-based threading you create a task and then ask the 'threading logic' to execute the task.
Systems that support this new way of working with threads are:
Visual Studio 2010 (we'll have to wait a few days for it)
Intel Threading Building Blocks
Visual Studio 2010 even has (it seems) special debugging logic to debug the 'parallel tasks'.
Take a look at std::thread
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2184.html
And a introduction
http://www.devx.com/SpecialReports/Article/38883
I've found that sticking with pthreads saves my sanity on three counts:
I don't have to fight through
WinAPI docs, which aren't habitually
of any quality.
Anyone that does much with threads
can help out with
pthreads. I've found infinitely more good sources of information about pthreads online.
Whenever I implement anything more
complicated that "Hello World" with
the WinAPI, I find it takes far
longer than one could reasonably
expect. That's just my empirical
input, though.
As far as capabilities are concerned, I've never found pthreads to be lacking in anything, so I don't think I've ever found the need to look elsewhere. There is also a great deal to be said for learning a library that you'll be able to use in any environment you tackle.
Having no experience with threading in the past, which threading technique in C++ will be the easiest for a beginner? boost::thread or pthreads?
I'll go in the opposite direction of everyone else - learn (or at least familiarize yourself with what is available in) pthreads.
Since boost is mainly just a wrapper around pthreads (on posix platforms) it helps to know what is going on underneath. In attempting to be generic, boost leaves the platform specific functionality unwrapped. In order to get to it you need to use the native_handle() calls. In order to use the native_handle() calls you need to know what the platform offers.
Think of it like sockets. There are dozens of socket classes and frameworks. But ultimately they wrap the underlying platform's socket API. Your understanding is always richer by knowing it - and knowing in what ways your class abstractions might have short comings.
Go for boost::thread. It's closely related to the work on the upcoming C++ standard threads, and the interface is quite easy to use and idiomatic to C++ (RAII instead of manual resource management).
boost::thread is a very nice and portable abstraction. I would certainly use it, but also learn the native thread api, like pthreads, so that you know how threading works on your platform.
Boost.Thread uses the RAII concept for locking, which makes things more exception safe and helps to avoid bugs like forgetting to release a mutex.
I'd say they're pretty close to equal in difficulty. The only big difference I see is that PThreads are pretty widely support (if you're concerned with cross platform porting). Another is that there have been quite a few good books on PThreads, though almost all the concepts will translate over to boost::thread, and many other threading libraries.
I am involved in a venture that will port some communications, parsing, data handling functionality from Win32 to Linux and both will be supported. The problem domain is very sensitive to throughput and performance.
I have very little experience with performance characteristics of boost and ACE. Specifically we want to understand which library provides the best performance for threading.
Can anyone provide some data -- documented or word-of-mouth or perhaps some links -- about the relative performance between the two?
EDIT
Thanks all. Confirmed our initial thoughts - we'll most likely choose boost for system level cross-platform stuff.
Neither library should really have any overhead compared to using native OS threading facilities. You should be looking at which API is cleaner. In my opinion the boost thread API is significantly easier to use.
ACE tends to be more "classic OO", while boost tends to draw from the design of the C++ standard library. For example, launching a thread in ACE requires creating a new class derived from ACE_Task, and overriding the virtual svc() function which is called when your thread runs. In boost, you create a thread and run whatever function you want, which is significantly less invasive.
Do yourself a favor and steer clear of ACE. It's a horrible, horrible library that should never have been written, if you ask me. I've worked (or rather HAD to work with it) for 3 years and I tell you it's a poorly designed, poorly documented, poorly implemented piece of junk using archaic C++ and built on completely brain-dead design decisions ... calling ACE "C with classes" is actually doing it a favor. If you look into the internal implementations of some of its constructs you'll often have a hard time suppressing your gag reflex.
Also, I can't stress the "poor documentation" aspect enough. Usually, ACE's notion of documenting a function consists of simply printing the function's signature. As to the meaning of its arguments, its return value and its general behavior, well you're usually left to figure that out on your own. I'm sick and tired of having to guess which exceptions a function may throw, which return value denotes success, which arguments I have to pass to make the function do what I need it to do or whether a function / class is thread-safe or not.
Boost on the other hand, is simple to use, modern C++, extremely well documented, and it just WORKS! Boost is the way to go, down with ACE!
Don't worry about the overhead of an OS-abstraction layer on threading and synchronization objects. Threading overhead literally doesn't matter at all (since it only applies to thread creation, which is already enormously slow compared to the overhead of a pimpl-ized pointer indirection). If you find that mutex ops are slowing you down, you're better off looking at atomic operations or rearranging your data access patterns to avoid contention.
Regarding boost vs. ACE, it's a matter of "new-style" vs. "old-style" programming. Boost has a lot of header-only template-based shenanigans (that are beautiful to work with, if you can appreciate it). If, on the other hand, you're used to "C with classes" style of C++, ACE will feel much more natural. I believe it's mostly a matter of personal taste for your team.
I've used ACE for numerous heavy duty production servers. It never failed me. It is rock solid and do the work for many years now. Tried to learn BOOST's ASIO network framework-Couldn't get the hang of it. While BOOST is more "modern" C++, it also harder to use for non trivial tasks - and without a "modern" C++ experience and deep STL knowledge it is difficult to use correctly
Even if ACE is a kind of old school C++, it still has many thread oriented features that boost doesn't provide yet.
At the moment I see no reason to not use both (but for different purposes). Once boost provide a simple mean to implement message queues between tasks, I may consider abandoning ACE.
When it comes to ease-of-use, boost is way better than ACE. boost-asio has a more transparent API, its abstractions are simpler and can easily provide building blocks to your application. The compile-time polymorphism is judiciously used in boost to warn/prevent illegal code. ACE's uses of templates, on the other hand, is limited to generalization and is hardly ever user-centric enough to disallow illegal operations. You're more likely to discover problems at run-time with ACE.
A simple example which I can think of is ACE_Reactor - a fairly scalable and decoupled interface- but you must remember to call its "own" function if you're running its event loop in a thread different from where it was created. I spent hours to figure this out for the first time and could've easily spent days. Ironically enough its object model shows more details than it hides - good for learning but bad for abstraction.
https://groups.google.com/forum/?fromgroups=#!topic/comp.soft-sys.ace/QvXE7391XKA
Threading is really only a small part of what boost and ACE provide, and the two aren't really comparable overall. I agree that boost is easier to use, as ACE is a pretty heavy framework.
I wouldn't call ACE "C with classes." ACE is not intuitive, but if you take your time and use the framework as intended, you will not regret it.
From what I can tell, after reading Boost's docs, I'd want to use ACE's framework and Boost's container classes.
Use ACE and boost cooperatively. ACE has better communication API, based on OO design patterns, whereas boost has like "modern C++" design and works well with containers for example.
We started to use ACE believing that it would hide the platform differences present between windows and unix in TCP sockets and the select call. Turns out, it does not. Ace's take on select, the reactor pattern, cannot mix sockets and stdin on windows, and the semantic differences between the platforms concerning socket writablility notifications are still present at the ACE level.
By the time we realized this we were already using the thread and process features of ACE (the latter of which again does not hide the platform differences to the extent we would have liked) so that our code is now tied to a huge library that actually prevents the porting of our code to 64 Bit MinGW!
I can't wait for the day when the last ACE usage in our code is finally replaced with something different.
I've been using ACE for many years (8) but I have just started investigating the use of boost again for my next project. I'm considering boost because it has a bigger tool bag (regex, etc) and parts of it are getting absorbed into the C++ standard so long term maintenance should be easier.
That said, boost is going to require some adjustment. Although Greg mentions that the thread support is less invasive as it can run any (C or static) function, if you're used to using thread classes that are more akin to the Java and C# thread classes which is what ACE_Task provides, you have to use a little finesse to get the same with boost.