I have a question about mixing and matching boost::threads with some of the c++11 standard items, does this work? I haven't actually tested anything yet but I am working with a system that uses all boost::threads and thread groups and interrupt features that you don't get out of the box with the standard, so there is no changing. Our version of boost 1.50 and doesn't have the latest std::atomic and memory ordering stuff. I was wondering if I could use the std::atomic and std:: memory ordering operations load/fectch_add etc(acquire/release,relaxed) with the boost threads and have the same results as if they were std::thread. These are all pthreads under the hood on my linux machine so I am thinking the answer is yes I can mix and match. Although,I just wanted to confirm and see if anyone has had any compatibility issues between mixing boost::thread and the std::thread apis.
That's an interesting question which I have been thinking of for a while since C++11 became widely available.
One general point, I notice that boost versions of std components often have extensions that provide more functionality than the std versions. For example, boost::bind provides more functionality than std::bind, boost <type_traits> are richer than std ones, boost::thread allows for thread cancellation/interrupts and std ones do not, etc..
With regards to boost threads vs std threads in particular, as you mention
... I am working with a system that uses all boost::threads and thread groups and interrupt features that you don't get out of the box with the standard...
I wanted to note that boost thread interruption cancellation does not come without a price, boost::condition_variable is really boost::condition_variable_any when thread cancellation is enabled in boost. boost::condition_variable_any maintains its own mutex and does more locking than the original POSIX pthread_cond_t that boost::condition_variable was designed to be a lightweight wrapper of. The thread interruption feature adds measurable 5-10% speed overhead to boost::condition_variable, condition variable: std vs boost chart.
Our version of boost 1.50 and doesn't have the latest std::atomic and memory ordering stuff. I was wondering if I could use the std::atomic and std:: memory ordering operations load/fectch_add etc(acquire/release,relaxed) with the boost threads and have the same results as if they were std::thread
std::atomic library do not use or depend on a particular threads library for certain built-in atomic types only (integers and pointers no wider than the natural platform width, e.g. 32 or 64-bit), or a platform, so you can mix and match thread with atomics libraries as you like, as long as you are careful to use std::atomic<T> where T's atomicity is supported by the hardware (again, integers and pointers), you can check that with std::atomic<T>::is_lock_free().
There isn't any technical reason why it shouldn't work. Both boost::thread and std::thread are just wrappers for native system threads, and all synchronization mechanisms are independent of what you have used to spawn the thread.
FYI, this other thread shows that there is indeed some compatibility issue between boost threads and std threads when managing signals/termination:
Preventing thread from calling std::terminate() on detach after handled exception:
Related
So I'm reading about monitors vs mutexes and finding mentions that suggest that monitors are faster mutexes because they don't lock system wide but rather only across the threads of a given process.
Is there some way in C++ to accomplish or simulate this?
Edit: I'm curious now what the difference is between system wide mutex and one restricted to a specific process.
C++ Standard does not define system-wide vs per-process primitives. So C++ does not specify whether std::mutex is system-wide.
Reasonable implementations have efficient per-process std::mutex; to have system-wide mutex you'll need to use libraries or operating system objects for your platform
The difference is that per-process mutex may use any memory operations to avoid system calls, as the process memory is shared among process's threads. Atomic operation on that memory are more efficient, and system call is often avoided via them. System-wide mutex will either start with system calls (not efficient), or will have to use shared memory (might be unsafe, also still may have some overhead).
The answer by #Alex Guteniev is as accurate as one can get (and should be considered the accepted answer). It states that the c++ standard doesn't define a system wide concept, and that mutexes for all practical purposes are per process i.e for synchronization between threads (execution agents) in a single process (and therefore according to your needs). The C++ makes it clear what a thread (std::thread) is (33.3 - ... intended to map one-to-one with OS threads (in my draft, at least...N4687)).
Microsoft post VC2015 has improved their implementation to use windows primitives as stated here. This is also indicated here in the most upvoted answer. I've also looked at the boost library implementations (which often precedes/influences the c++ standard) for microsoft and (AFAICT) it doesn't use any inter-process calls.
So to answer your question. In C++ threads and monitors are practically the same thing if this definition is to be considered accurate.
Update, stumbled across the answer to this while researching something related.
On Windows, Critical Sections can be used for single processes instead of system wide mutexes and are often faster:
Edit:
While the above statement is correct, c++ doesn't have the concept system wide mutex. This concept only exists when using OS specific primitives such as win32 CreateMutex and is not relevant to std c++.
Source:
std::mutex performance compared to win32 CRITICAL_SECTION
On Linux, pthreads are for processes.
In one of the projects I am working on migration from C to C++. A lot of the C code uses pthread and mutex associated with this library for multithreading. I want to perform possibly step by step, incremental migration. I wanted to start with the data structures as they are most obvious but they would need to be synchronized using pthread mutex. Is usage of pthread mutex safe or only the standard library threading infrastructure (like std::mutex) can guarantee proper memory interthread memory consistency?
Thread library in C++ provides higher level abstractions and other useful synchronization mechanisms, If your compiler supports C++11 or newever C++ standard features, In my opinion you should use C++11 thread library and experience beauty of RAII when managing shared resources.
I don't have much experience with pthreads, but pthread_mutex has same semantics as std::mutex and to answer your question
Is usage of pthread mutex safe or only the standard library threading infrastructure (like std::mutex) can guarantee proper memory inter thread memory consistency
As I mentioned semantically they are same and you can build a locable abstraction (class with try_lock, unlock, lock methods) on top of pthread_mutex easily, it comes down to how you use both, for example
If you are causing data races or not
If you are using lock_guard,
unique_lock or not, Because if you don't use RAII in addition to similar problems as in C for resource management,
C++ statements can throw exception which will cause threads to
deadlock if mutex is not unlocked.
Besides this there are bunch of other useful stuff available like std::recursive_mutex, std::shared_mutex, std::futures, std::promises etc.
I'm learning C++11 and have run into a threading issue. My general question: are C++11 mutexes compatible with threads not created with C++11's standard libraries?
I would like to safely share information between a thread created with C++11 and another thread created by a third-party library that I have no control over.
For example, my application uses PortAudio, which creates its own thread for audio output. I'm not sure if it's using pthreads, or OS-specific threading libraries, but I do know that PortAudio is NOT written in C++11. I want to safely share data between a GUI thread (using a C++11 thread) and the PortAudio thread using a mutex.
Similarly, can I use a C++11 mutex to synchronize QT QThreads and C++11 threads?
Are C++11 mutexes compatible with threads not created with C++11's standard libraries?
The C++ standard does not define a "thread" as something exclusively created by the C++ standard library.
1.10 Multi-threaded executions and data races [intro.multithread]
1 A thread of execution (also known as a thread) is a single flow of
control within a program, including the initial invocation of a
specific top-level function, and recursively including every function
invocation subsequently executed by the thread.
So, I would conclude the answer to your question is "yes".
Obviously, the C++ standard doesn't make any guarantees about compatebility with other systems. Part of the reason the C and C++ standards added threading facilities was to standardize on one threading system.
In practice it is expected that the C and C++ threads library is built to integrate with a platform threading system if there is one. For example, on platforms using pthreads the expectation is that pthreads are used where appropriate to buildtge standard library threading facilities (as far as I know there is no pthreads interface for the various atomic operations, i.e., the standard library may need to provide its own synchronization primitives).
The standard library classes provide access to the underlying representation through the native_handle() methods. A standard library should implement what is returned from these and, e.g., if pthreads types are provided it seems safe to assume that this particular standard library will play nice with pthreads.
The C++11 standard specifies that mutexes should work with any kind of 'execution agent', including different thread libraries. Here are some relevant quotes from the standard which I think answer the question conclusively:
Mutex requirements
A mutex object facilitates protection against data races and allows
safe synchronization of data between execution agents (30.2.5). An
execution agent owns a mutex from the time it successfully calls one
of the lock functions until it calls unlock.
Requirements for Lockable types
An execution agent is an entity such as a thread that may perform work
in parallel with other execution agents. [Note: Implementations or
users may introduce other kinds of agents such as processes or
thread-pool tasks. —end note ] The calling agent is determined by
context, e.g. the calling thread that contains the call, and so on.
It is inconceivable that C++11's threading implementation will be incompatible with the platform's native threading implementation because any practical program using C++11 threads is going to call into platform libraries, and those libraries may themselves be threaded or make thread related calls (to mutexes for example).
The C++11 library implementation for threads is not of course obliged to use the high level native threading library (say, pthreads or windows threads) but it probably will, for which purpose as has been mentioned there is a std::thread::native_handle() method to get the native handle. However, even where it does not use the high level native implementation, it will have to use the same low level kernel primitives underneath.
In all conceivable circumstances it should therefore be perfectly safe to use C++11 mutexes with thread instances created by native library calls, and vice versa, and mix any other native or C++ library synchronization calls. There may indeed be cases where it is necessary to do so. For example, the C++11 library does not at present provide thread pools or read-write locks (shared mutexes). You might want to use native read-write locks with threads started using std::thread, or use one of the many thread pool implementations provided by third party libraries in your C++ program.
The only caveat to observe is that trying to mix C++11 threads (which will in practice be obliged to use kernel threads in one way or another for the reasons mentioned above) with thread libraries which do not use kernel threads at all (for example, libraries based on green threads or "user" threads), is likely to lead you into trouble.
Edit: In support of this I notice that §30.3 of C++11 states, albeit non-normatively, that "These threads [std::thread threads] are intended to map one-to-one with operating system threads".
Linux is a new platform to me. I've coded on Windows in c++ for a number of years and have become comfortable with multithreading on that platform.
Along comes C++11 at a time when I need to learn c++ on the linux platform.
Linux appears to use pthreads for the most part - okay there's also boost::threads and QT have their own threads too. But with C++11 comes std::thread, a whole new (cross platform and C++ standard) way to do threads.
So I guess I'll have to learn pthreads and std::threads. Ultimately, std::thread seems more important, but there's a lot of legacy code out there, so I'll have to know both.
For thread synchronization on windows, I would use WaitForMultipleObjects to wait for a number of tasks to complete before continuing with further work.
Does a similar synchronization mechanism exist for pthreads? std::threads?
I've had a look at pthread_join, and it seems to have the facility to only wait on one thread at a time. Am I missing another pthread call maybe?
std::thread is boost::thread accepted into C++11 with some extras. My understanding is that if boost::thread gets replaced in code with std::thread it should still compile and work.
boost::thread is based on pthreads design, providing thin C++ wrappers over thread, mutex and condition variables. Thread cancellation though was left outside the scope of C++11, since there was no agreement how it should work in C++.
So, by learning pthreads you also learn std::thread concepts. std::thread adds mostly syntax sugar and convenience functions on top of pthreads C API.
With regards to WaitForMultipleObjects(), neither pthreads nor std::thread provide anything similar to its bWaitAll=FALSE mode, however, it's routinely simulated using pipes and select() on UNIX, or more modern eventfd() and epoll() on Linux. bWaitAll=TRUE mode can be simulated by waiting on all tasks in turn, since it doesn't proceed until all objects are ready anyway.
No, neither pthreads nor C++11 has direct equivalent of WaitForMultipleObjects (i.e. wait for any waitable "handle" type.) pthread_join can only be used to join threads, and only a single, specific thread.
The closest equivalent on posix platforms is to wait for multiple file descriptors using system calls such as select(), poll() or the linux-specific epoll(), but they require you to have a file descriptor to wait on, which is fine for I/O events but requires extra work from you to use them wait for mutexes, condition variables or other synchronisation objects. There are more general event libraries built on top of those system calls, e.g. libevent and libev and Boost ASIO, which support waiting for timers as well as I/O, but still not thread completion, mutex locks etc. with a single function like WaitForMultipleObjects
The alternatives you do have for pthreads and C++11 threads are to wait on different synchronisation types separately. You can wait for timers, wait for threads to complete, wait for mutexes, wait on condition variables, wait for asynchronous results to be ready (std::async in C++11, no direct equivalent in pthreads) ... but there's no call that will allow you to wait a heterogeneous set of those types all at once.
I could give you a really fancy answer but alas, this is where I learned them and it is a good introduction:
http://www.yolinux.com/TUTORIALS/LinuxTutorialPosixThreads.html
You use pthread_mutex_t for syncronization and pthread_join probably handles the wait for multiple tasks problem. It works exactly as you would expect.
Based on this, you must call pthread_join for each single thread you have created. Or to use mutexes, if there is a need to synchronize your threads.
Regarding WaitForMultipleObjects, this is generally called a Barrier Sync. Boost has an implementation called barrier. It uses conditional variables to implement it, in posix its a pthread_cond_t
Here is an answer I left recently explaining barrier sync.
Motivation: reason why I'm considering it is that my genius project manager thinks that boost is another dependency and that it is horrible because "you depend on it"(I tried explaining the quality of boost, then gave up after some time :( ). Smaller reason why I would like to do it is that I would like to learn c++11 features, because people will start writing code in it.
So:
Is there a 1:1 mapping between #include<thread> #include<mutex>and
boost equivalents?
Would you consider a good idea to replace boost stuff with c++11
stuff. My usage is primitive, but are there examples when std doesnt
offer what boost does? Or (blasphemy) vice versa?
P.S.
I use GCC so headers are there.
There are several differences between Boost.Thread and the C++11 standard thread library:
Boost supports thread cancellation, C++11 threads do not
C++11 supports std::async, but Boost does not
Boost has a boost::shared_mutex for multiple-reader/single-writer locking. The analogous std::shared_timed_mutex is available only since C++14 (N3891), while std::shared_mutex is available only since C++17 (N4508).
C++11 timeouts are different to Boost timeouts (though this should soon change now Boost.Chrono has been accepted).
Some of the names are different (e.g. boost::unique_future vs std::future)
The argument-passing semantics of std::thread are different to boost::thread --- Boost uses boost::bind, which requires copyable arguments. std::thread allows move-only types such as std::unique_ptr to be passed as arguments. Due to the use of boost::bind, the semantics of placeholders such as _1 in nested bind expressions can be different too.
If you don't explicitly call join() or detach() then the boost::thread destructor and assignment operator will call detach() on the thread object being destroyed/assigned to. With a C++11 std::thread object, this will result in a call to std::terminate() and abort the application.
To clarify the point about move-only parameters, the following is valid C++11, and transfers the ownership of the int from the temporary std::unique_ptr to the parameter of f1 when the new thread is started. However, if you use boost::thread then it won't work, as it uses boost::bind internally, and std::unique_ptr cannot be copied. There is also a bug in the C++11 thread library provided with GCC that prevents this working, as it uses std::bind in the implementation there too.
void f1(std::unique_ptr<int>);
std::thread t1(f1,std::unique_ptr<int>(new int(42)));
If you are using Boost then you can probably switch to C++11 threads relatively painlessly if your compiler supports it (e.g. recent versions of GCC on linux have a mostly-complete implementation of the C++11 thread library available in -std=c++0x mode).
If your compiler doesn't support C++11 threads then you may be able to get a third-party implementation such as Just::Thread, but this is still a dependency.
std::thread is largely modelled after boost::thread, with a few differences:
boost's non-copyable, one-handle-maps-to-one-os-thread, semantics are retained. But this thread is movable to allow returning thread from factory functions and placing into containers.
This proposal adds cancellation to the boost::thread, which is a significant complication. This change has a large impact not only on thread but the rest of the C++ threading library as well. It is believed this large change is justifiable because of the benefit.
The thread destructor must now call cancel prior to detaching to avoid accidently leaking child threads when parent threads are canceled.
An explicit detach member is now required to enable detaching without canceling.
The concepts of thread handle and thread identity have been separated into two classes (they are the same class in boost::thread). This is to support easier manipulation and storage of thread identity.
The ability to create a thread id which is guaranteed to compare equal to no other joinable thread has been added (boost::thread does not have this). This is handy for code which wants to know if it is being executed by the same thread as a previous call (recursive mutexes are a concrete example).
There exists a "back door" to get the native thread handle so that clients can manipulate threads using the underlying OS if desired.
This is from 2007, so some points are no longer valid: boost::thread has a native_handle function now, and, as commenters point out, std::thread doesn't have cancellation anymore.
I could not find any significant differences between boost::mutex and std::mutex.
Enterprise Case
If you are writing software for the enterprise that needs to run on a moderate to large variety of operating systems and consequently build with a variety of compilers and compiler versions (especially relatively old ones) on those operating systems, my suggestion is to stay away from C++11 altogether for now. That means that you cannot use std::thread, and I would recommend using boost::thread.
Basic / Tech Startup Case
If you are writing for one or two operating systems, you know for sure that you will only ever need to build with a modern compiler that mostly supports C++11 (e.g. VS2015, GCC 5.3, Xcode 7), and you are not already dependent on the boost library, then std::thread could be a good option.
My Experience
I am personally partial to hardened, heavily used, highly compatible, highly consistent libraries such as boost versus a very modern alternative. This is especially true for complicated programming subjects such as threading. Also, I have long experienced great success with boost::thread (and boost in general) across a vast array of environments, compilers, threading models, etc. When its my choice, I choose boost.
There is one reason not to migrate to std::thread.
If you are using static linking, std::thread becomes unusable due to these gcc bugs/features:
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=52590
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=57740
Namely, if you call std::thread::detach or std::thread::join it will lead to either exception or crash, while boost::thread works ok in these cases.
With Visual Studio 2013 the std::mutex seems to behave differently than the boost::mutex, which caused me some problems (see this question).
With regards to std::shared_mutex added in C++17
The other answers here provide a very good overview of the differences in general. However, there are several issues with std::shared_mutex that boost solves.
Upgradable mutices. These are absent from std::thread. They allow a reader to be upgraded to a writer without allowing any other writers to get in before you. These allow you to do things like pre-process a large computation (for example, reindexing a data structure) when in read mode, then upgrade to write to apply the reindex while only holding the write lock for a short time.
Fairness. If you have constant read activity with a std::shared_mutex, your writers will be softlocked indefinitely. This is because if another reader comes along, they will always be given priority. With boost:shared_mutex, all threads will eventually be given priority.(1) Neither readers nor writers will be starved.
The tl;dr of this is that if you have a very high-throughput system with no downtime and very high contention, std::shared_mutex will never work for you without manually building a priority system on top of it. boost::shared_mutex will work out of the box, although you might need to tinker with it in certain cases. I'd argue that std::shared_mutex's behavior is a latent bug waiting to happen in most code that uses it.
(1) The actual algorithm it uses is based on the OS thread scheduler. In my experience, when reads are saturated, there are longer pauses (when obtaining a write lock) on Windows than on OSX/Linux.
I tried to use shared_ptr from std instead of boost and I actually found a bug in gcc implementation of this class. My application was crashing because of destructor called twice (this class should be thread-safe and shouldn't generate such problems). After moving to boost::shared_ptr all problems disappeared. Current implementations of C++11 are still not mature.
Boost has also more features. For example header in std version doesn't provide serializer to a stream (i.e. cout << duration). Boost has many libraries that use its own , etc. equivalents, but do not cooperate with std versions.
To sum up - if you already have an application written using boost, it is safer to keep your code as it is instead of putting some effort in moving to C++11 standard.