I noticed that most loggers are advertised as thread safe.
What does it mean?
Are they safe against a specific threading library or can they be safe in any multithreading environment (e.g. PThread, Boost threads, C++11 threads, Win32 threads, OpenMP threads, ...)?
It means you won't get something like this in your log files:
this is the line from the firsThis is line from the second thread
t thread
Usually it means that loggers use required locking when they write to the stream in any supported environment.
If a logger is thread safe that means you can call its functions from any threads (be it pthread or boost or openmp). That is usually done by using mutexes to prevent simultaneous output. Without them your program may output mixed lines or even crash if log is used from different threads.
Related
I'm learning C++11 and have run into a threading issue. My general question: are C++11 mutexes compatible with threads not created with C++11's standard libraries?
I would like to safely share information between a thread created with C++11 and another thread created by a third-party library that I have no control over.
For example, my application uses PortAudio, which creates its own thread for audio output. I'm not sure if it's using pthreads, or OS-specific threading libraries, but I do know that PortAudio is NOT written in C++11. I want to safely share data between a GUI thread (using a C++11 thread) and the PortAudio thread using a mutex.
Similarly, can I use a C++11 mutex to synchronize QT QThreads and C++11 threads?
Are C++11 mutexes compatible with threads not created with C++11's standard libraries?
The C++ standard does not define a "thread" as something exclusively created by the C++ standard library.
1.10 Multi-threaded executions and data races [intro.multithread]
1 A thread of execution (also known as a thread) is a single flow of
control within a program, including the initial invocation of a
specific top-level function, and recursively including every function
invocation subsequently executed by the thread.
So, I would conclude the answer to your question is "yes".
Obviously, the C++ standard doesn't make any guarantees about compatebility with other systems. Part of the reason the C and C++ standards added threading facilities was to standardize on one threading system.
In practice it is expected that the C and C++ threads library is built to integrate with a platform threading system if there is one. For example, on platforms using pthreads the expectation is that pthreads are used where appropriate to buildtge standard library threading facilities (as far as I know there is no pthreads interface for the various atomic operations, i.e., the standard library may need to provide its own synchronization primitives).
The standard library classes provide access to the underlying representation through the native_handle() methods. A standard library should implement what is returned from these and, e.g., if pthreads types are provided it seems safe to assume that this particular standard library will play nice with pthreads.
The C++11 standard specifies that mutexes should work with any kind of 'execution agent', including different thread libraries. Here are some relevant quotes from the standard which I think answer the question conclusively:
Mutex requirements
A mutex object facilitates protection against data races and allows
safe synchronization of data between execution agents (30.2.5). An
execution agent owns a mutex from the time it successfully calls one
of the lock functions until it calls unlock.
Requirements for Lockable types
An execution agent is an entity such as a thread that may perform work
in parallel with other execution agents. [Note: Implementations or
users may introduce other kinds of agents such as processes or
thread-pool tasks. —end note ] The calling agent is determined by
context, e.g. the calling thread that contains the call, and so on.
It is inconceivable that C++11's threading implementation will be incompatible with the platform's native threading implementation because any practical program using C++11 threads is going to call into platform libraries, and those libraries may themselves be threaded or make thread related calls (to mutexes for example).
The C++11 library implementation for threads is not of course obliged to use the high level native threading library (say, pthreads or windows threads) but it probably will, for which purpose as has been mentioned there is a std::thread::native_handle() method to get the native handle. However, even where it does not use the high level native implementation, it will have to use the same low level kernel primitives underneath.
In all conceivable circumstances it should therefore be perfectly safe to use C++11 mutexes with thread instances created by native library calls, and vice versa, and mix any other native or C++ library synchronization calls. There may indeed be cases where it is necessary to do so. For example, the C++11 library does not at present provide thread pools or read-write locks (shared mutexes). You might want to use native read-write locks with threads started using std::thread, or use one of the many thread pool implementations provided by third party libraries in your C++ program.
The only caveat to observe is that trying to mix C++11 threads (which will in practice be obliged to use kernel threads in one way or another for the reasons mentioned above) with thread libraries which do not use kernel threads at all (for example, libraries based on green threads or "user" threads), is likely to lead you into trouble.
Edit: In support of this I notice that §30.3 of C++11 states, albeit non-normatively, that "These threads [std::thread threads] are intended to map one-to-one with operating system threads".
I don't know how to thread in C++ and I would not just wan't to know that but is there a way i can force a thread onto a different core? Also how would I find out how many cores the user has?
Binding thread to the arbitrary CPU is called setting affinity. It's platform-dependent operation.
For windows: SetProcessAffinityMask
For pthreads: pthread_attr_setaffinity_np(3) and pthread_setaffinity_np(3)
For Boost you can use native_handle() to get platform-specific thread handle to use them with functions above.
Linux is a new platform to me. I've coded on Windows in c++ for a number of years and have become comfortable with multithreading on that platform.
Along comes C++11 at a time when I need to learn c++ on the linux platform.
Linux appears to use pthreads for the most part - okay there's also boost::threads and QT have their own threads too. But with C++11 comes std::thread, a whole new (cross platform and C++ standard) way to do threads.
So I guess I'll have to learn pthreads and std::threads. Ultimately, std::thread seems more important, but there's a lot of legacy code out there, so I'll have to know both.
For thread synchronization on windows, I would use WaitForMultipleObjects to wait for a number of tasks to complete before continuing with further work.
Does a similar synchronization mechanism exist for pthreads? std::threads?
I've had a look at pthread_join, and it seems to have the facility to only wait on one thread at a time. Am I missing another pthread call maybe?
std::thread is boost::thread accepted into C++11 with some extras. My understanding is that if boost::thread gets replaced in code with std::thread it should still compile and work.
boost::thread is based on pthreads design, providing thin C++ wrappers over thread, mutex and condition variables. Thread cancellation though was left outside the scope of C++11, since there was no agreement how it should work in C++.
So, by learning pthreads you also learn std::thread concepts. std::thread adds mostly syntax sugar and convenience functions on top of pthreads C API.
With regards to WaitForMultipleObjects(), neither pthreads nor std::thread provide anything similar to its bWaitAll=FALSE mode, however, it's routinely simulated using pipes and select() on UNIX, or more modern eventfd() and epoll() on Linux. bWaitAll=TRUE mode can be simulated by waiting on all tasks in turn, since it doesn't proceed until all objects are ready anyway.
No, neither pthreads nor C++11 has direct equivalent of WaitForMultipleObjects (i.e. wait for any waitable "handle" type.) pthread_join can only be used to join threads, and only a single, specific thread.
The closest equivalent on posix platforms is to wait for multiple file descriptors using system calls such as select(), poll() or the linux-specific epoll(), but they require you to have a file descriptor to wait on, which is fine for I/O events but requires extra work from you to use them wait for mutexes, condition variables or other synchronisation objects. There are more general event libraries built on top of those system calls, e.g. libevent and libev and Boost ASIO, which support waiting for timers as well as I/O, but still not thread completion, mutex locks etc. with a single function like WaitForMultipleObjects
The alternatives you do have for pthreads and C++11 threads are to wait on different synchronisation types separately. You can wait for timers, wait for threads to complete, wait for mutexes, wait on condition variables, wait for asynchronous results to be ready (std::async in C++11, no direct equivalent in pthreads) ... but there's no call that will allow you to wait a heterogeneous set of those types all at once.
I could give you a really fancy answer but alas, this is where I learned them and it is a good introduction:
http://www.yolinux.com/TUTORIALS/LinuxTutorialPosixThreads.html
You use pthread_mutex_t for syncronization and pthread_join probably handles the wait for multiple tasks problem. It works exactly as you would expect.
Based on this, you must call pthread_join for each single thread you have created. Or to use mutexes, if there is a need to synchronize your threads.
Regarding WaitForMultipleObjects, this is generally called a Barrier Sync. Boost has an implementation called barrier. It uses conditional variables to implement it, in posix its a pthread_cond_t
Here is an answer I left recently explaining barrier sync.
Could you help me to understand how to use mutexes in multithread Linux application, where:
during data writing it is need to lock variable on write and read
during data reading from the variable it is need to lock it on write.
So it is possible to read simultaneously, but writing opertion is a single opertaion in the same time. During writing, all other operation should wait before it finishes.
You're asking about something that is a bit higher level than mutexes. A mutex is a simple, low-level device. When you lock a thread with a mutex, the CPU is either executing code in the thread that obtained the lock or it is executing some other process entirely. In other words, the mutex has locked out all other threads that belong to the same (heavyweight) process.
You are asking about a read-write lock. Read-write locks use mutexes underneath the hood. The POSIX functions that deal with read-write locks start with pthread_rwlock_. Since you are on a Linux machine, just type man pthread and look for the section marked "READ/WRITE LOCK ROUTINES".
You need a reader/writer lock to allow multiple readers/single writer.
Boost.Thread has one of these (boost::shared_mutex), if you have no other preferred threading library. This uses PThreads primitives under the covers, and will probably save you time in wrapping the raw APIs yourself.
I would not recommend implementing this yourself - it's easy to get something that appears to work, but under load either crashes or kills performance or (worst of all) silently modifies your data in a way it should not be, so you get bad results.
A simple boost::mutex can also be used here as noted by #Als, but won't allow multiple concurrent reads. That is simpler to implement, and may be sufficient for your needs, depending on your read/write access profile.
You will need to use mutexes, if you have global or static objects which are being accessed(read and written to) from different threads.
Is it possible to use fcntl() system call on a file to achieve thread/process synchronization (instead of semaphoress)?
Yes. Unix fcntl locks (and filesystem resources in general) are system-wide, so any two threads of execution (be they separate processes or not) can use them. Whether that's a good idea or not is context-dependent.
That's one way of synchronizing between processes, but if you don't want to use semaphores, you could use process shared mutexes, such as mutexes and condition variables created with the PTHREAD_PROCESS_SHARED attribute on POSIX based platforms (see pthread_mutexattr_setpshared() and pthread_condattr_setpshared()). Another option is to use an event based IPC (sockets, etc) mechanism that blocks until an event you define is demultiplexed (e.g. via select()). There are several other shared memory based options as well.
However, since you're using C++ I'd recommend using a C++ framework that greatly simplifies this sort of interprocess synchronization across multiple platforms like boost.interprocess or ACE.
The fcntl and flock are not for thread, but for process, so they cannot be used for thread synchronization.