Problem (in short):
I'm using POSIX Shared Memory and currently just used POSIX semaphores and i need to control multiple readers, multiple writers. I need help with what variables/methods i can use to control access within the limitations described below.
I've found an approach that I want to implement but i'm unsure of what methodology i can use to implement it when using POSIX Shared memory.
What I've Found
https://stackoverflow.com/a/28140784
This link has the algorithm i'd like to use but i'm unsure how to implement it with shared memory. Do i store the class in shared memory somehow? This is where I need help please.
The reason I'm unsure is a lot of my research, points towards keeping shared memory to primitives only to avoid addressing problems and STL objects can't be used.
NOTE:
For all my multi-threading i'm using C++11 features. This shared memory will be completely seperate program executables using C++11 std::threads from which any thread of any process/executable will want access. I have avoided the Linux pthread for any of my multi-threading and will continue to do so (except if its just control variable not actual pThreads).
Solution Parameters aimed for
Must be shareable between 2+ processes which will be running multiple C++11 std::thread that may wish access. I.e. Multiple Writers (exclusive one at a time) while allowing multiple simultaneous readers when no writer wants access.
Not using BOOST libraries. Ideally native C++11 or built in linux libraries, something that will work without the need to install abstract libraries.
Not using pThread actual threads but could use some object from there that will work with C++11 std::thread.
Ideally can handle a process crash while in operation. E.g. Using POSIX semaphore if a process crashes while it has the semaphore, everyone is screwed. I have seen people using file locks?
Thanks in advance
keeping shared memory to primitives only to avoid addressing problems
You can use pointers in and to shared memory objects across programs, so long as the memory is mmaped to the same address. This is actually a straightforward proposition, especially on 64 bit. See this open source C library I wrote for implementation details: rszshm - resizable pointer-safe shared memory.
Using POSIX semaphore if a process crashes while it has the semaphore, everyone is screwed.
If you want to use OS mediated semaphores, the SysV semaphores have SEM_UNDO, which recovers in this case. OTOH pthread offers robust mutexes that can be embedded and shared in shared memory. This can be used to build more sophisticated mechanisms.
The SysV scheme of providing multiple semaphores in a semaphore set, where a group of actions must all succeed, or the call blocks, permits building sophisticated mechanism too. A read/write lock can be made with a set of three semaphores.
Related
So I'm reading about monitors vs mutexes and finding mentions that suggest that monitors are faster mutexes because they don't lock system wide but rather only across the threads of a given process.
Is there some way in C++ to accomplish or simulate this?
Edit: I'm curious now what the difference is between system wide mutex and one restricted to a specific process.
C++ Standard does not define system-wide vs per-process primitives. So C++ does not specify whether std::mutex is system-wide.
Reasonable implementations have efficient per-process std::mutex; to have system-wide mutex you'll need to use libraries or operating system objects for your platform
The difference is that per-process mutex may use any memory operations to avoid system calls, as the process memory is shared among process's threads. Atomic operation on that memory are more efficient, and system call is often avoided via them. System-wide mutex will either start with system calls (not efficient), or will have to use shared memory (might be unsafe, also still may have some overhead).
The answer by #Alex Guteniev is as accurate as one can get (and should be considered the accepted answer). It states that the c++ standard doesn't define a system wide concept, and that mutexes for all practical purposes are per process i.e for synchronization between threads (execution agents) in a single process (and therefore according to your needs). The C++ makes it clear what a thread (std::thread) is (33.3 - ... intended to map one-to-one with OS threads (in my draft, at least...N4687)).
Microsoft post VC2015 has improved their implementation to use windows primitives as stated here. This is also indicated here in the most upvoted answer. I've also looked at the boost library implementations (which often precedes/influences the c++ standard) for microsoft and (AFAICT) it doesn't use any inter-process calls.
So to answer your question. In C++ threads and monitors are practically the same thing if this definition is to be considered accurate.
Update, stumbled across the answer to this while researching something related.
On Windows, Critical Sections can be used for single processes instead of system wide mutexes and are often faster:
Edit:
While the above statement is correct, c++ doesn't have the concept system wide mutex. This concept only exists when using OS specific primitives such as win32 CreateMutex and is not relevant to std c++.
Source:
std::mutex performance compared to win32 CRITICAL_SECTION
On Linux, pthreads are for processes.
I have a general question about parallel programming in C and C++ and would appreciate it if you could answer it. As far as I know, we can declare a variable in at least one level higher (parent thread) to share it among children threads. So, I was wondering if there is any other way to share a variable among threads with the same parent thread? Is this API dependant or not?
For Posix threads, read some pthread tutorial.
For C++11, read the documentation of its thread library
All threads of the same process share the same address space in virtual memory. As commented by Marco A. consider also thread_local variables.
Notice that you share data or memory (not variables, which exist only in the source code)
In practice, you'll better protect with a mutex the shared data (for synchronization) to avoid data races.
In the simple case, the mutex and the shared data are in some global variables.
You could also use atomic operations.
BTW, you could also develop a parallel application using some message passing paradigm, e.g. using MPI (or simply using some RPC or other messages, e.g. JSON on sockets). You might consider for regular numerical applications to use the GPGPU e.g. using OpenCL. And of course you might mix all the approaches (using OpenCL, with several threads, and having your parallel software running in several such processes communicating with MPI).
Debugging a heavily parallel software can become a nightmare. Performance may depend upon the hardware system and may require tricky tuning. scalability and synchronization may becoming a growing concern.map-reduce is often a useful model.
In C++ and C any memory location (identified by a variable) can be shared among threads. The memory space is the same across all threads. There is no parent/child thread relationship with memory.
The challenge is to control or synchronize access to the memory location among the threads.
That is implementation dependent.
Any global variable is sharable among threads, since threads are light weight processes sharing the same address space. For synchronization, you need to ensure mutual exclusion while updating/accessing those global variables through semaphores or wait notify blocks.
I'm learning C++11 and have run into a threading issue. My general question: are C++11 mutexes compatible with threads not created with C++11's standard libraries?
I would like to safely share information between a thread created with C++11 and another thread created by a third-party library that I have no control over.
For example, my application uses PortAudio, which creates its own thread for audio output. I'm not sure if it's using pthreads, or OS-specific threading libraries, but I do know that PortAudio is NOT written in C++11. I want to safely share data between a GUI thread (using a C++11 thread) and the PortAudio thread using a mutex.
Similarly, can I use a C++11 mutex to synchronize QT QThreads and C++11 threads?
Are C++11 mutexes compatible with threads not created with C++11's standard libraries?
The C++ standard does not define a "thread" as something exclusively created by the C++ standard library.
1.10 Multi-threaded executions and data races [intro.multithread]
1 A thread of execution (also known as a thread) is a single flow of
control within a program, including the initial invocation of a
specific top-level function, and recursively including every function
invocation subsequently executed by the thread.
So, I would conclude the answer to your question is "yes".
Obviously, the C++ standard doesn't make any guarantees about compatebility with other systems. Part of the reason the C and C++ standards added threading facilities was to standardize on one threading system.
In practice it is expected that the C and C++ threads library is built to integrate with a platform threading system if there is one. For example, on platforms using pthreads the expectation is that pthreads are used where appropriate to buildtge standard library threading facilities (as far as I know there is no pthreads interface for the various atomic operations, i.e., the standard library may need to provide its own synchronization primitives).
The standard library classes provide access to the underlying representation through the native_handle() methods. A standard library should implement what is returned from these and, e.g., if pthreads types are provided it seems safe to assume that this particular standard library will play nice with pthreads.
The C++11 standard specifies that mutexes should work with any kind of 'execution agent', including different thread libraries. Here are some relevant quotes from the standard which I think answer the question conclusively:
Mutex requirements
A mutex object facilitates protection against data races and allows
safe synchronization of data between execution agents (30.2.5). An
execution agent owns a mutex from the time it successfully calls one
of the lock functions until it calls unlock.
Requirements for Lockable types
An execution agent is an entity such as a thread that may perform work
in parallel with other execution agents. [Note: Implementations or
users may introduce other kinds of agents such as processes or
thread-pool tasks. —end note ] The calling agent is determined by
context, e.g. the calling thread that contains the call, and so on.
It is inconceivable that C++11's threading implementation will be incompatible with the platform's native threading implementation because any practical program using C++11 threads is going to call into platform libraries, and those libraries may themselves be threaded or make thread related calls (to mutexes for example).
The C++11 library implementation for threads is not of course obliged to use the high level native threading library (say, pthreads or windows threads) but it probably will, for which purpose as has been mentioned there is a std::thread::native_handle() method to get the native handle. However, even where it does not use the high level native implementation, it will have to use the same low level kernel primitives underneath.
In all conceivable circumstances it should therefore be perfectly safe to use C++11 mutexes with thread instances created by native library calls, and vice versa, and mix any other native or C++ library synchronization calls. There may indeed be cases where it is necessary to do so. For example, the C++11 library does not at present provide thread pools or read-write locks (shared mutexes). You might want to use native read-write locks with threads started using std::thread, or use one of the many thread pool implementations provided by third party libraries in your C++ program.
The only caveat to observe is that trying to mix C++11 threads (which will in practice be obliged to use kernel threads in one way or another for the reasons mentioned above) with thread libraries which do not use kernel threads at all (for example, libraries based on green threads or "user" threads), is likely to lead you into trouble.
Edit: In support of this I notice that §30.3 of C++11 states, albeit non-normatively, that "These threads [std::thread threads] are intended to map one-to-one with operating system threads".
Currently, I have 2 processes that communicate using the message_queue and shared_memory form boost. Everything work as attended.
Now I need to make one of this process multi threaded (thanks to boost again), and I was wondering if I need to use protection mechanism between the threads (such as mutex), or if the boost::interprocess library already provides a protection mechanism ?
I did not find any info on that on the boost documentation. By the way I'm using boost 1.40.
Thanks in advance.
The shared resources from boost::interprocess (shared memory, etc) require that you provide the necessary synchronization. The reason for this is that you may not require synchronization, and it usually a somewhat expensive operation performance wise.
Say for example that you had a process that wrote to shared memory the current statistics of something in 32-bit integer format, and a few processes that read those values. Since the values are integers (and therefore on your platform the reads and writes are atomic) and you have a single process writing them and a few process reading them, no synchronization is needed for this design.
However in some examples you will require synchronization, like if the above example had multiple writers, or if instead of integers you were using string data. There are various synchronization mechanisms inside of boost (as well as non-boost ones, but since your already using boost), described here:
[Boost info for stable version: 1.48]
http://www.boost.org/doc/libs/1_48_0/doc/html/interprocess/synchronization_mechanisms.html
[Boost info for version your using: 1.40]
http://www.boost.org/doc/libs/1_40_0/doc/html/interprocess/synchronization_mechanisms.html
With shared memory it is a common practice to place the synchronization mechanism at the base of the shared memory segment, where it can be anonymous (meaning the OS kernel does not provide access to it by name). This way all the processes know how to lock the shared memory segment, and you can associate locks with their segments (if you had multiple for example)
Remember that a mutex requires the same thread of execution (inside a process) to unlock it that locked it. If you require locking and unlocking a synchronization object from different threads of execution, you need a semaphore.
Please be sure that if you choose to use a mutex that it is an interprocess mutex (http://www.boost.org/doc/libs/1_48_0/doc/html/boost/interprocess/interprocess_mutex.html) as opposed to the mutex in the boost thread library which is for a single process with multiple threads.
You have to make sure you lock the shared resources.
You can find examples in boost documentation. For example:
http://www.boost.org/doc/libs/1_40_0/doc/html/interprocess/synchronization_mechanisms.html#interprocess.synchronization_mechanisms.mutexes.mutexes_scoped_lock
I found this:
Fast interprocess synchronization method
I used to believe that a pthread mutex can only be shared between two threads in the same address space.
The question / answers there seems to imply:
If I have two separate proceses A & B. They have a shared memory region M. I can put a pThread mutex in M, lock in A, lock in B, unlock in A; and B will no longer block on the mutex. Is this correct? Can pThread mutexes be shared in two separate processes?
Edit: I'm using C++, on MacOSX.
You need to tell the mutex to be process-shared when it's inited:
http://www.opengroup.org/onlinepubs/007908775/xsh/pthread_mutexattr_setpshared.html
Note in particular, "The default value of the attribute is PTHREAD_PROCESS_PRIVATE", meaning that accessing it from different processes is undefined behaviour.
If your C/pthread library is conforming, you should be able to tell if it supports mutexes shared across multiple process by checking if the _POSIX_THREAD_PROCESS_SHARED feature test macro is defined to a value other than -1 or by querying the system configuration at run-time using sysconf(_SC_THREAD_PROCESS_SHARED) if that feature test macro is undefined.
EDIT: As Steve pointed out, you'll need to explicitly configure the mutex for sharing across processes assuming the platform supports that feature as I described above.
I was concerned that there might be a condition where a mutex in shared memory might fail to behave properly, so I did some digging and came up with some documents which treat the issue like a no-brainer:
https://computing.llnl.gov/tutorials/pthreads/
Further digging, however, showed that older versions of glibc suffered issues in shared memory mutexes: (This is an ancient change, but it illustrates the point.)
in linuxthreads/mutex.c
int __pthread_mutexattr_setpshared(...) {
/* For now it is not possible to shared a conditional variable. */
if (pshared != PTHREAD_PROCESS_PRIVATE)
return ENOSYS;
}
Without more detail on what implementation of pthread you're using, it's difficult to say whether you're safe or not.
My cause for concern is that many implementations (and some entire languages, like perl, python, and ruby) have a global lock object that manages access to shared objects. That object would not be shared between processes and therefore, while your mutexes would probably work most of the time, you might find yourself having two processes simultaneously manipulating the mutex at the same time.
I know that this flies in the face of the definition of a mutex but it is possible:
If two threads are operating at the same time in different processes, it implies that they are on different cores. Both acquire their global lock object and go to manipulate the mutex in shared memory. If the pthread implementation forces the update of the mutex through the caches, both threads could end up updating at the same time, both thinking they hold the mutex. This is just a possible failure vector that comes to mind. There could be any number of others. What are the specifics of your situation - OS, pthreads version, etc.?