I have an array of objects that I want to operate on in threads, but I also want to be able to access at times. This feels like a hacky way to achieve my goal, but is there a better way to do something like this?:
＊the basic goal is to have 2 locks. One that allows all individual threads to work concurrently while blocking access from the array until they are all finished, and one that allows shuts down access from the thread to ensure that no objects are touched by other threads while a function runs.
atomic<int> inThreadCount;
atomic<int> arrayLock;
map<string, MyObj*> myMap;
mutex mu1;
class MyObj{
mutex mu2;
int myInt;
public:
void update(bool shouldLowerCount){
mu2.lock();
myInt++;
if (shouldLowerCount)
inThreadCount--;
mu2.unlock();
}
}
//Some operation that requires all threads to finish first
//and doesn't allow threads to access the objects while running
void GetSnapshot(){
mu1.lock();
arrayLock++;
while (inThreadCount > 0)
Sleep(0);
map<string, MyObj *>::iterator it = myMap.begin();
auto t = time(nullptr);
auto tm = *localtime(&t);
cout << put_time(&tm, "%d-%m-%Y %H-%M-%S") << endl;
for( ; it != myMap.end(); ++it){
cout << it->first << ":" << it->second->counter);
}
arrayLock--;
mu1.unlock();
}
void updateObject(MyObj* myObj){
while (arrayLock > 0)
Sleep(0);
inThreadCount++;
async(std::launch::async, myObj->update(true));
}
PS, I realize that there is a tiny window of opportunity for error between Sleep() and arrayLock/inThreadCount++. That's part of the problem I want to solve!
I think you're asking for a shared mutex.
A shared mutex (or read-write mutex) allows many threads to lock an object in parallel while also allowing one thread at a time to lock it exclusively.
In simple terms, if a thread requests shared access it is granted unless a thread holds the object exclusively. A thread is granted exclusivity when the object isn't held (shared or exclusively) by any other thread.
It's common use is read-write exclusivity. See shared access to read and exclusive access to write. That's valid because a data race can only occur when two ore more threads access the same data and at least one of them is a write operation. Multiple readers is not a data race.
There are normally overheads in implementing a shared lock as opposed to an exclusive lock, and the model only normally helps where there are 'many' readers
that read 'frequently' and write operations are 'infrequent'. What 'many', 'frequent' and 'infrequent' mean depends on the platform and problem in hand.
That's exactly what a shared mutex is for.
C++17 supports that out of the box with std::shared_mutex but I noticed the question is tagged C++11.
Some implementations have offered that for some time (it's a classic locking strategy)
Or you can try boost::shared_mutex<>.
NB: One of the challenges in a shared-lock is to avoid live-lock on the writer.
If there are many readers that read frequently it can be easy the writer to get 'locked out' indefinitely and never progress (or progress very slowly).
A good shared lock will provide some guarantee of the writer eventually getting a turn. That may be absolute priority (no writers allowed to start after a thread starts waithing
Related
The usual pattern of using std::shared_timed_mutex is to let the 'reader' thread acquire it in shared mode and the 'writer' thread acquires it in exclusive mode. In this manner, the reads and writes cannot happen at the same time and thus the program is free from data-race/undefined behavior.
I wanted to understand if at all there's any problem if I change the mode among the threads i.e. the reader thread reads the shared variable after acquiring the lock in exclusive mode and the writer thread writes in the shared variable after taking the mutex in shared mode.
#include <iostream>
#include <thread>
#include <random>
#include <chrono>
#include <shared_mutex>
using namespace std::chrono_literals;
std::shared_timed_mutex lck;
int shared_array[5];
void writerFunc(int index);
void readerFunc();
//main thread
int main() {
std::thread writer_threads[5];
for(int i=0; i<5; ++i) {
writer_threads[i] = std::thread(writerFunc,i);
}
while(true) {
std::this_thread::sleep_for(5s);
readerFunc();
}
for(int i=0; i<5; ++i) {
writer_threads[i].join();
}
}
//function executed in writer threads.
//Each writer thread will work on it's own index in the global shared array.
void writerFunc(int index) {
std::random_device rd;
std::mt19937 mt(rd());
std::uniform_real_distribution<double> dist(1.0, 42.0);
while(true) {
{
std::shared_lock<std::shared_timed_mutex> sl(lck);
//Writing random number in shared variable.
shared_array[index] += dist(mt);
}
std::this_thread::sleep_for(100ms);
}
}
//function executed in reader thread(main).
void readerFunc() {
std::lock_guard<std::shared_timed_mutex> sl(lck);
for(int i=0; i<5 ; ++i) {
std::cout<<"\nshared_array["<<i<<"]--> "<<shared_array[i];
}
std::cout<<"\n\n";
}
Since the reader and writer thread cannot concurrently access the variable at the same time, therefore, there's no data race in the above program. Thread-sanitiser also does not report any problem with the above program.
I'm mainly having a little doubt regarding the values read by the reader thread.
Is it guaranteed by the C++ standard, irrespective of the underlying CPU architecture, that
a) the above program doesn't have any UB?
b) the reader thread can only see the latest value written by the writer thread?
******* Additional details ********
Please note that the above was a short sample program where I've tried to replicate a specific part of the design of my main project. Over there the scale is a lot bigger. e.g. the size of the array(not exactly an array but very similar) over there is ~2 million. Also the data structure is not a simple int but a custom serializable structure.
so think of something like this:
custom_serializable_struct shared_variable[2000000];
In my main program, there will be 'N' writer threads and a single reader thread. Most of the time, writer threads will be working. Since N is a lot smaller than 2 million therefore I'm using separate synchronisation(1 std::atomic_flag for each of the 2 million indexes. This is used after acquiring the shared_timed_mutex) among the writer threads(I had omitted this aspect from the design of the sample code as I felt it was not relevant to what I was asking).
Like I've said above, most of the time, writer threads will be working. Only occasionally, the reader thread will work.
Mainly, the program has following requirements:
I've to minimize the wait time of writer threads spent on the mutex while the reader thread is working.
I've to ensure that the reader thread, whenever it works, always gets the latest value written by the writer threads.
So basically this is what is happening in my main program:
N writer threads:
while (true) {
// 1. Acquire the shared_timed_mutex in shared mode.
// 2. Acquire the std::atomic_flag of the index, i, on which the thread has to work. This is required, as I mentioned, to prevent data race among writer threads.
// 3. Do changes in the custom_serializable_struct shared_variable[i]
}
1 reader thread:
while(true) {
// 1. long sleep time.
// 2. Acquire the shared_timed_mutex in exclusive mode.
// 3. read the entire 2 million values. Please note that this read is not done 1 by 1 like in a for loop. It's more like memcpy of the entire memory.
}
But Why?
I am wondering what the motivation behind changing the roles of the reader and the writer with respect to locking may be! What problem are you solving by doing so?
In a previous comment, you mentioned that you do not want contention between writers.
Looking at the code, I infer also that the update of every int in the array is independent of the others but the reader MUST see them all at once as if collectively they have ONE meaning (the reason for exclusive lock). You have not mentioned this yet - so assuming this is not the intention.
There is just one reader but many writers i.e. it looks inverted to a (some?) stereotypical case of having more readers than writers. This shouldn't be a major consideration.
Conveying an unintended meaning and surprising code should be avoided. I agree with #Nicol Bolas & suggest another approach too:
Wrong Tool - use std::atomic instead
The inverted use of std::shared_timed_mutex is a surprise here for the future maintainer (yourself?). Plus, using it is the source of misleading message to the reader and the reason for this question.
I agree with #Nicol Bolas that atomic would solve this problem:
std::atomic<int> shared_array[5];
void writerFunc(int index) {
///Other code
while(true) {
//Writing random number in shared variable.
shared_array[index].fetch_add(dist(mt));
std::this_thread::sleep_for(100ms);
}
}
void readerFunc() {
for (auto& item : shared_array) {
std::cout << item;
}
}
Better Abstraction - use libguarded::shared_guarded
The root of the sorrow looks to be the level at which you have applied the std::shared_timed_mutex lck - it controls the entire array whereas you wish to have a finer control on every element.
I highly recommend you consider using the shared_guarded of the cs_libguarded available under BSD 2-Clause "Simplified" License.
libguarded::shared_guarded<int> shared_array[5]; //Nailed it!
void writeFunc(int index) {
//Other code
while (true) {
{
auto handle = shared_array[index].lock();
auto& item = *handle;
item += dist(mt);
}
std::this_thread::sleep_for(100ms);
}
}
void readerFunc() {
for (auto& array_element : shared_array) {
auto item = array_element.lock_shared();
std::cout << *item;
}
}
The above not only ensures the correct, unsurprising use of the shared data but also ensures const correctness as it does not allow writes on lock_shared. This can work with any data-types, not just ints - a restriction that std::atomic has. As #Solomon Slow points out, the memory barriers may cause unintended results with out-of-order execution with your original approach - this code doesn't have that problem. libguarded also ensures that access to shared data is always with the correct synchronization - no accidental use of shared data.
FYI, shared_guarded is equivalent to using a mutex for every element (as below), only much cleaner, const correct and fool-proof.
std::shared_timed_mutex lck[5]; //Don't do it by hand, better use libguarded, as above
int shared_array[5];
I strongly recommend prioritizing cleaner implementation over arbitrary objectives like not wanting to have many mutexes. If you do not want contention please eliminate sharing instead of aiming to reduce mutexes. The problem is sharing and not the existence of mutex.
P.S.: You tagged the question as C++14 while libguarded needs C++17. As far as I checked, libguarded::shared_guarded should work with std::shared_timed_mutex.
unlock_shared explicitly synchronizes with subsequent lock calls on the same mutex. This would allow the reader to read data written by any of the writers. Similarly, lock_shared synchronizes with prior calls to unlock. So it is possible to use a shared_mutex backwards without a data race (note: rand is not required to be thread-safe).
But... should you?
The purpose of a mutex is to ensure data integrity, not just at the level of bytes (ie: data races), but at a higher level. You have 5 threads writing to 5 different locations. But... what is the meaning of the data? Are these data completely distinct from one another, or does the collection of data have some meaning that needs to be preserved? That is, if one thread writes to one value, does the reader get malformed information if another thread hasn't written its value yet?
If these data values are all completely, fundamentally separate, then a mutex is unnecessary (at least for basic types). What you're really doing is just atomic writes. The writers can write to an atomic<T>, and the reader will read these. Since the values are all disparate and lack any question of ordering between them, you don't need to block any thread from writing. All you need to do is ensure data integrity at the level of an individual T. Lock-free atomics will be much faster than any mutex-based solution.
But if the data has some notion of integrity to it, if the group of threads is collectively creating a single value that the reader thread should read in its entirety, then what you're looking for is a barrier, not a mutex. This object allows you to see if a group of execution agents have collectively reached a certain point. And if they have, it is safe for you to read the data. And once you're finished reading it, it is safe to release the agents to write to them once again.
My problem is quite common I suppose, but it drives me crazy:
I have a multi-threaded application with 5 threads. 4 of these threads do their job, like network communication and local file system access, and then all write their output to a data structure of this form:
struct Buffer {
std::vector<std::string> lines;
bool has_been_modified;
}
The 5th thread prints these buffer/structures to the screen:
Buffer buf1, buf2, buf3, buf4;
...
if ( buf1.has_been_modified ||
buf2.has_been_modified ||
buf3.has_been_modified ||
buf4.has_been_modified )
{
redraw_screen_from_buffers();
}
How do I protect the buffers from being overwritten while they are either being read from or written to?
I can't find a proper solution, although I think this has to be a quiet common problem.
Thanks.
You should use a mutex. The mutex class is std::mutex. With C++11 you can use std::lock_guard<std::mutex> to encapsulate the mutex using RAII. So you would change your Buffer struct to
struct Buffer {
std::vector<std::string> lines;
bool has_been_modified;
std::mutex mutex;
};
and whenever you read or write to the buffer or has_been_modified you would do
std::lock_guard<std::mutex> lockGuard(Buffer.mutex); //Do this for each buffer you want to access
... //Access buffer here
and the mutex will be automatically released by the lock_guard when it is destroyed.
You can read more about mutexes here.
You can use a mutex (or mutexes) around the buffers to ensure that they're not modified by multiple threads at the same time.
// Mutex shared between the multiple threads
std::mutex g_BufferMutex;
void redraw_screen_from_buffers()
{
std::lock_guard<std::mutex> bufferLockGuard(g_BufferMutex);
//redraw here after mutex has been locked.
}
Then your buffer modification code would have to lock the same mutex when the buffers are being modified.
void updateBuffer()
{
std::lock_guard<std::mutex> bufferLockGuard(g_BufferMutex);
// update here after mutex has been locked
}
This contains some mutex examples.
What appears you want to accomplish is to have multiple threads/workers and one observer. The latter needs to do its job only when all workers are done/signal. If this is the case then check code in this SO q/a. std::condition_variable - Wait for several threads to notify observer
mutex are a very nice thing when trying to avoid dataraces, and I'm sure the answer posted by #Phantom will satisfy most people. However, one should know that this is not scalable to large systems.
By locking you are synchronising your threads. As only one at a time can be accessing the vector, on thread writting to the container will cause the other one to wait for it to finish ... with may be good for you but causes serious performance botleneck when high performance is needed.
The best solution would be to use a more complexe lock free structure. Unfortunatelly I don't think there is any standart lockfree structure in the STL. One exemple of lockfree queue is available here
Using such a structure, your 4 working threads would be able to enqueue messages to the container while the 5th one would dequeue them, without any dataraces
More on lockfree datastructure can be found here !
I was hoping someone could advise on how multiple threads can write to a common container (eg a map). In a scenario where some threads might share the same key using Boost and C++
The map might be of type : std::map, with different threads accessing the object to modify different data members. Would each thread wait upon hitting unique_lock for the current thread to finish before proceeding?
would it be as simple as each thread entering a critical section as this example:
//somewhere within the code
boost::unique_lock mutex;
void modifyMap(const std::string& key,const unsigned int dataX,
const unsigned int dataY)
{
// would each thread wait for exclusive access?
boost::unique_lock<boost::shared_mutex> lock (mutex);
// i now have exclusive access no race conditions;
m_map.find(key)->second.setDataX(dataX);
m_map.find(key)->second.setDataX(dataY);
}
thanks in advance
You should create a thread-safe implementation of a data structure. It can be either lock-based (for example implemented by using mutexes) or lock-free (using atomic operations or memory orderings which are supported in C++11 and boost).
I can briefly describe the lock-based approach. For example, you may want to design a thread-safe linked list. If your threads perform only read operations everything is safe. On the other hand if you try to write to this data structure you might need a previous and a next node pointers in the list (if its double-linked you need to update their pointers to point to the inserted node) and while you modify them some other thread might read the incorrect pointer data so you need a lock on the two nodes between which you want to insert your new node. This creates serialization (other threads wait for mutex to be unlocked) and reduces the potential for concurrency.
The full example with a lookup table is available in the book "C++ Concurrency: Practical Multithreading" by Anthony Williams at page 171, listing 6.11. The book itself is a good start for a multithreading programming with the latest C++ standard as the author of the book also designed both boost::thread and C++11 thread libraries.
update: to make your example work for read/write (if you need more operations you need to protect them also) you're better off using boost::shared_mutex which essentially allows multiple-read single write access: if one thread wants to write than it is going acquire an exclusive lock and all other threads will have to wait. Here's some code:
template <typename mapType>
class threadSafeMap {
boost::shared_mutex map_mutex;
mapType* m_map;
public:
threadSafeMap() {
m_map = new mapType();
}
void modifyMap(std::string& key,const unsigned int dataX,
const unsigned int dataY)
{
//std::lock_guard in c++11. std::shared_mutex is going to be available in C++14
//acquire exclusive access - other threads wait
boost::lock_guard<boost::shared_mutex> lck(map_mutex);
m_map.find(key)->second.setDataX(dataX);
m_map.find(key)->second.setDataX(dataY);
}
int getValueByKey(std::string& key)
{
//std::lock_guard in c++11. std::shared_mutex is going to be available in C++11
//acquire shared access - other threads can read. If the other thread needs access it has to wait for a fully unlocked state.
boost::shared_lock<boost::shared_mutex> lck(map_mutex);
return m_map.getValue(key);
}
~threadSafeMap() {
delete m_map;
}
};
Lock-guard objects are destructed and mutex is unlocked at the end of the lifetime. mapType template can be replaced by your map type.
I am very new to Linux programming so bear with me. I have 2 thread type that perform different operations so I want each one to have it's own mutex. Here is the code I am using , is it good ? If not why ?
static pthread_mutex_t cs_mutex = PTHREAD_MUTEX_INITIALIZER;
static pthread_mutex_t cs_mutex2 = PTHREAD_MUTEX_INITALIZER;
void * Thread1(void * lp)
{
int * sock = (int*)lp;
char buffer[2024];
int bytecount = recv(*sock, buffer, 2048, 0);
while (0 == 0)
{
if ((bytecount ==0) || (bytecount == -1))
{
pthread_mutex_lock(&cs_mutex);
//Some uninteresting operations witch plays with set 1 of global variables;
pthread_mutex_unlock(&cs_mutex);
}
}
}
void * Thread2(void * lp)
{
while (0 == 0)
{
pthread_mutex_lock(&cs_mutex2);
//Some uninteresting operations witch plays with some global variables;
pthread_mutex_unlock(&cs_mutex2);
}
}
Normally, a mutex is not thread related.
It ensures that a critical area is only accessed by a single thread.
So if u have some shared areas, like processing the same array by multiple threads, then you must ensure exclusive access for this area.
That means, you do not need a mutex for each thread. You need a mutex for the critical area.
If you only have one driver, there is no advantage to having two cars. Your Thread2 code can only make useful progress while holding cs_mutex2. So there's no point to having more than one thread running that code. Only one thread can hold the mutex at a time, and the other thread can do no useful work.
So all you'll accomplish is that occasionally the thread that doesn't hold the mutex will try to run and have to wait for the other. And occasionally the thread that does hold the mutex will try to release and re-acquire it and get pre-empted by the other.
This is a completely pointless use of threads.
I see three problems here. There's a question your infinite loop, another about your intention in having multiple threads, and there's a future maintainability "gotcha" lurking.
First
int bytecount = recv(*sock, buffer, 2048, 0);
while (0 == 0)
Is that right? You read some stuff from a socket, and start an infinite loop without ever closing the socket? I can only assume that you do some more reading in the loop, but in which case you are waiting for an external event while holding the mutex. In general that's a bad pattern limiting your concurrency. A possibly pattern is to have one thread reading the data and then passing the read data to other threads which do the processing.
Next, you have two different sets of resources each protected by their own mutex. You then intend to have a set of Threads for each resource. But each thread has the pattern
take mutex
lots of processing
release mutex
tiny window (a few machine instructions)
take mutex again
lots of processing
release mutex
next tiny window
There's virtually no opportunity for two threads to work in parallel. I question whether your have need for multiple threads for each resource.
Last there's a potential maintenance issue. I'm just pointing this out for future reference, I don't think you need to do anything right now. You have two functions, intended for use by two threads, but in the end they are just functions that can be called by anyone. If later maintenance results in those functions (or refactored subsets of the functions) then you could get two threads
take mutex 1
take mutex 2
and the other
take mutex 2
take mutex 1
Bingo: deadlock.
Not an easy problem to avoid, but at the very least one can aid the maintainer by careful naming choices and refactoring.
I think your code is correct, however please note 2 things:
It is not exception safe. If exception is thrown from Some uninteresting operations then your mutex will be never unlocked -> deadlock
You could also consider using std::mutex or boost::mutex instead of raw mutexes. For mutex locking it's better to use boost::mutex::scoped_lock (or std:: analog with modern compiler)
void test()
{
// not synch code here
{
boost::mutex::scoped_lock lock(mutex_);
// synchronized code here
}
}
If you have 2 different sets of data and 2 different threads working on these sets -- why do you need mutexes at all? Usually, mutexes are used when you deal with some shared piece of data and you don't want two threads to deal with it simultaneously, so you lock it with mutex, do some stuff, unlock.
I have a problem with using std:map in my multithread application. I need to lock the map object when thread is writing to this object. Parallely another threads which reading this object should shop until writing process is finish.
Sample code:
std::map<int, int> ClientTable;
int write_function() //<-- only one thread uses this function
{
while(true)
{
//lock ClientTable
ClientTable.insert(std::pair<int, int>(1, 2)); // random values
//unlock ClientTable
//thread sleeps for 2 secs
}
}
int read_function() //<--- many thread uses this function
{
while(true)
{
int test = ClientTable[2]; // just for test
}
}
How to lock this std::map object and correctly synchronise this threads?
Looks like what you need is a typical read-write lock, allowing any number of readers but a single writer. You can have a look at boost's shared_mutex.
Additional usage examples can be found here: Example for boost shared_mutex (multiple reads/one write)?
Well, since a std::map doesn't have a builtin lock, you would have to use a separate lock that protects the map. If the map is all that's protected by that lock you could subclass map to add the lock there along with "lock" and "unlock" calls, but if the lock will be used for other items as well then it's probably not the best idea to do that.
As for "how to correctly synchronize" -- that's very specific to the application at hand. However, for the example given, you have to insert the lock/unlock calls around the read operation as well.
If you have read/write locks, this might also be a good application for one of these.