In one of the projects I am working on migration from C to C++. A lot of the C code uses pthread and mutex associated with this library for multithreading. I want to perform possibly step by step, incremental migration. I wanted to start with the data structures as they are most obvious but they would need to be synchronized using pthread mutex. Is usage of pthread mutex safe or only the standard library threading infrastructure (like std::mutex) can guarantee proper memory interthread memory consistency?
Thread library in C++ provides higher level abstractions and other useful synchronization mechanisms, If your compiler supports C++11 or newever C++ standard features, In my opinion you should use C++11 thread library and experience beauty of RAII when managing shared resources.
I don't have much experience with pthreads, but pthread_mutex has same semantics as std::mutex and to answer your question
Is usage of pthread mutex safe or only the standard library threading infrastructure (like std::mutex) can guarantee proper memory inter thread memory consistency
As I mentioned semantically they are same and you can build a locable abstraction (class with try_lock, unlock, lock methods) on top of pthread_mutex easily, it comes down to how you use both, for example
If you are causing data races or not
If you are using lock_guard,
unique_lock or not, Because if you don't use RAII in addition to similar problems as in C for resource management,
C++ statements can throw exception which will cause threads to
deadlock if mutex is not unlocked.
Besides this there are bunch of other useful stuff available like std::recursive_mutex, std::shared_mutex, std::futures, std::promises etc.
Related
So I'm reading about monitors vs mutexes and finding mentions that suggest that monitors are faster mutexes because they don't lock system wide but rather only across the threads of a given process.
Is there some way in C++ to accomplish or simulate this?
Edit: I'm curious now what the difference is between system wide mutex and one restricted to a specific process.
C++ Standard does not define system-wide vs per-process primitives. So C++ does not specify whether std::mutex is system-wide.
Reasonable implementations have efficient per-process std::mutex; to have system-wide mutex you'll need to use libraries or operating system objects for your platform
The difference is that per-process mutex may use any memory operations to avoid system calls, as the process memory is shared among process's threads. Atomic operation on that memory are more efficient, and system call is often avoided via them. System-wide mutex will either start with system calls (not efficient), or will have to use shared memory (might be unsafe, also still may have some overhead).
The answer by #Alex Guteniev is as accurate as one can get (and should be considered the accepted answer). It states that the c++ standard doesn't define a system wide concept, and that mutexes for all practical purposes are per process i.e for synchronization between threads (execution agents) in a single process (and therefore according to your needs). The C++ makes it clear what a thread (std::thread) is (33.3 - ... intended to map one-to-one with OS threads (in my draft, at least...N4687)).
Microsoft post VC2015 has improved their implementation to use windows primitives as stated here. This is also indicated here in the most upvoted answer. I've also looked at the boost library implementations (which often precedes/influences the c++ standard) for microsoft and (AFAICT) it doesn't use any inter-process calls.
So to answer your question. In C++ threads and monitors are practically the same thing if this definition is to be considered accurate.
Update, stumbled across the answer to this while researching something related.
On Windows, Critical Sections can be used for single processes instead of system wide mutexes and are often faster:
Edit:
While the above statement is correct, c++ doesn't have the concept system wide mutex. This concept only exists when using OS specific primitives such as win32 CreateMutex and is not relevant to std c++.
Source:
std::mutex performance compared to win32 CRITICAL_SECTION
On Linux, pthreads are for processes.
I have a scenario where the shared memory area is exclusively accessed by two different processes. When I launch the processes, the first process successfully locks the mutex, updates the memory and unlock the mutex. But I observe that when the second process try to lock it, it is still in deadlock state, waiting for mutex to unlock.
Time difference between the mutex lock is 10s for first and second process.
I am using the std::mutex. Please tell me what I am missing.
Despite the citations that others have made to documentation, the classes std::mutex and std::shared_mutex actually do work across processes when used in shared memory on Linux. I've checked with GCC 8.3.1 and Clang 6.0.1.
Standard C++ implementations of these on Linux use pthreads. The pthreads provision of PTHREAD_PROCESS_SHARED and PTHREAD_PROCESS_PRIVATE as attributes for pthread_mutex_t and pthread_rwlock_t are abstracted out and defaulted
by std::mutex and std::shared_mutex. Even though POSIX documentation says that the default is PRIVATE, a pthread_mutex_t and pthread_rwlock_t allocated in shared memory will block competing processes when locked. This is because the actual implementation of pthreads on Linux uses futexes, and they are intended for use in shared memory even in cases where mapped addresses may differ across processes.
It's possible that the PTHREADS_PROCESS_PRIVATE behaviour is actually more difficult to implement given the strategy of using futexes to implement mutexes, and as such, is silently not implemented.
If you did want your mutexes to be process-private, just avoid putting them in shared memory. On the other hand if you do want to share them, be cautious that this standards discrepancy could be subject to change.
For reliability, use Boost Interprocess.
An std::mutex instance is only scoped to a single process; it is not capable of interprocess synchronization/concurrency. It is only capable of synchronizing child threads within a parent process.
Look to use Boost or an interprocess synchronization library instead.
std::mutex does not support interprocess operation but pthread library has interprocess mutex that you can use. Example here.
I have a scenario where the shared memory area is exclusively accessed by two different processes. When I launch the processes, the first process successfully locks the mutex, updates the memory and unlock the mutex. But I observe that when the second process try to lock it, it is still in deadlock state, waiting for mutex to unlock.
Time difference between the mutex lock is 10s for first and second process.
I am using the std::mutex. Please tell me what I am missing.
Despite the citations that others have made to documentation, the classes std::mutex and std::shared_mutex actually do work across processes when used in shared memory on Linux. I've checked with GCC 8.3.1 and Clang 6.0.1.
Standard C++ implementations of these on Linux use pthreads. The pthreads provision of PTHREAD_PROCESS_SHARED and PTHREAD_PROCESS_PRIVATE as attributes for pthread_mutex_t and pthread_rwlock_t are abstracted out and defaulted
by std::mutex and std::shared_mutex. Even though POSIX documentation says that the default is PRIVATE, a pthread_mutex_t and pthread_rwlock_t allocated in shared memory will block competing processes when locked. This is because the actual implementation of pthreads on Linux uses futexes, and they are intended for use in shared memory even in cases where mapped addresses may differ across processes.
It's possible that the PTHREADS_PROCESS_PRIVATE behaviour is actually more difficult to implement given the strategy of using futexes to implement mutexes, and as such, is silently not implemented.
If you did want your mutexes to be process-private, just avoid putting them in shared memory. On the other hand if you do want to share them, be cautious that this standards discrepancy could be subject to change.
For reliability, use Boost Interprocess.
An std::mutex instance is only scoped to a single process; it is not capable of interprocess synchronization/concurrency. It is only capable of synchronizing child threads within a parent process.
Look to use Boost or an interprocess synchronization library instead.
std::mutex does not support interprocess operation but pthread library has interprocess mutex that you can use. Example here.
I have a question about mixing and matching boost::threads with some of the c++11 standard items, does this work? I haven't actually tested anything yet but I am working with a system that uses all boost::threads and thread groups and interrupt features that you don't get out of the box with the standard, so there is no changing. Our version of boost 1.50 and doesn't have the latest std::atomic and memory ordering stuff. I was wondering if I could use the std::atomic and std:: memory ordering operations load/fectch_add etc(acquire/release,relaxed) with the boost threads and have the same results as if they were std::thread. These are all pthreads under the hood on my linux machine so I am thinking the answer is yes I can mix and match. Although,I just wanted to confirm and see if anyone has had any compatibility issues between mixing boost::thread and the std::thread apis.
That's an interesting question which I have been thinking of for a while since C++11 became widely available.
One general point, I notice that boost versions of std components often have extensions that provide more functionality than the std versions. For example, boost::bind provides more functionality than std::bind, boost <type_traits> are richer than std ones, boost::thread allows for thread cancellation/interrupts and std ones do not, etc..
With regards to boost threads vs std threads in particular, as you mention
... I am working with a system that uses all boost::threads and thread groups and interrupt features that you don't get out of the box with the standard...
I wanted to note that boost thread interruption cancellation does not come without a price, boost::condition_variable is really boost::condition_variable_any when thread cancellation is enabled in boost. boost::condition_variable_any maintains its own mutex and does more locking than the original POSIX pthread_cond_t that boost::condition_variable was designed to be a lightweight wrapper of. The thread interruption feature adds measurable 5-10% speed overhead to boost::condition_variable, condition variable: std vs boost chart.
Our version of boost 1.50 and doesn't have the latest std::atomic and memory ordering stuff. I was wondering if I could use the std::atomic and std:: memory ordering operations load/fectch_add etc(acquire/release,relaxed) with the boost threads and have the same results as if they were std::thread
std::atomic library do not use or depend on a particular threads library for certain built-in atomic types only (integers and pointers no wider than the natural platform width, e.g. 32 or 64-bit), or a platform, so you can mix and match thread with atomics libraries as you like, as long as you are careful to use std::atomic<T> where T's atomicity is supported by the hardware (again, integers and pointers), you can check that with std::atomic<T>::is_lock_free().
There isn't any technical reason why it shouldn't work. Both boost::thread and std::thread are just wrappers for native system threads, and all synchronization mechanisms are independent of what you have used to spawn the thread.
FYI, this other thread shows that there is indeed some compatibility issue between boost threads and std threads when managing signals/termination:
Preventing thread from calling std::terminate() on detach after handled exception:
I'm learning C++11 and have run into a threading issue. My general question: are C++11 mutexes compatible with threads not created with C++11's standard libraries?
I would like to safely share information between a thread created with C++11 and another thread created by a third-party library that I have no control over.
For example, my application uses PortAudio, which creates its own thread for audio output. I'm not sure if it's using pthreads, or OS-specific threading libraries, but I do know that PortAudio is NOT written in C++11. I want to safely share data between a GUI thread (using a C++11 thread) and the PortAudio thread using a mutex.
Similarly, can I use a C++11 mutex to synchronize QT QThreads and C++11 threads?
Are C++11 mutexes compatible with threads not created with C++11's standard libraries?
The C++ standard does not define a "thread" as something exclusively created by the C++ standard library.
1.10 Multi-threaded executions and data races [intro.multithread]
1 A thread of execution (also known as a thread) is a single flow of
control within a program, including the initial invocation of a
specific top-level function, and recursively including every function
invocation subsequently executed by the thread.
So, I would conclude the answer to your question is "yes".
Obviously, the C++ standard doesn't make any guarantees about compatebility with other systems. Part of the reason the C and C++ standards added threading facilities was to standardize on one threading system.
In practice it is expected that the C and C++ threads library is built to integrate with a platform threading system if there is one. For example, on platforms using pthreads the expectation is that pthreads are used where appropriate to buildtge standard library threading facilities (as far as I know there is no pthreads interface for the various atomic operations, i.e., the standard library may need to provide its own synchronization primitives).
The standard library classes provide access to the underlying representation through the native_handle() methods. A standard library should implement what is returned from these and, e.g., if pthreads types are provided it seems safe to assume that this particular standard library will play nice with pthreads.
The C++11 standard specifies that mutexes should work with any kind of 'execution agent', including different thread libraries. Here are some relevant quotes from the standard which I think answer the question conclusively:
Mutex requirements
A mutex object facilitates protection against data races and allows
safe synchronization of data between execution agents (30.2.5). An
execution agent owns a mutex from the time it successfully calls one
of the lock functions until it calls unlock.
Requirements for Lockable types
An execution agent is an entity such as a thread that may perform work
in parallel with other execution agents. [Note: Implementations or
users may introduce other kinds of agents such as processes or
thread-pool tasks. —end note ] The calling agent is determined by
context, e.g. the calling thread that contains the call, and so on.
It is inconceivable that C++11's threading implementation will be incompatible with the platform's native threading implementation because any practical program using C++11 threads is going to call into platform libraries, and those libraries may themselves be threaded or make thread related calls (to mutexes for example).
The C++11 library implementation for threads is not of course obliged to use the high level native threading library (say, pthreads or windows threads) but it probably will, for which purpose as has been mentioned there is a std::thread::native_handle() method to get the native handle. However, even where it does not use the high level native implementation, it will have to use the same low level kernel primitives underneath.
In all conceivable circumstances it should therefore be perfectly safe to use C++11 mutexes with thread instances created by native library calls, and vice versa, and mix any other native or C++ library synchronization calls. There may indeed be cases where it is necessary to do so. For example, the C++11 library does not at present provide thread pools or read-write locks (shared mutexes). You might want to use native read-write locks with threads started using std::thread, or use one of the many thread pool implementations provided by third party libraries in your C++ program.
The only caveat to observe is that trying to mix C++11 threads (which will in practice be obliged to use kernel threads in one way or another for the reasons mentioned above) with thread libraries which do not use kernel threads at all (for example, libraries based on green threads or "user" threads), is likely to lead you into trouble.
Edit: In support of this I notice that §30.3 of C++11 states, albeit non-normatively, that "These threads [std::thread threads] are intended to map one-to-one with operating system threads".