I'm using C++ 11, where I have 4 threads that are parallely inserting into a concurrent queue. I know when the threads are done processing, i.e. the expected final size of the queue.
Now, I want to perform a final aggregation operation on the contents of the queue that should be strictly performed only once. For example, say I want to aggregate the values and POST it to an external service.
How do I acquire a lock which is valid only once? I can't use a simple mutex because that would not guarantee me the only once requirement.
Pseudocode:
// inside a thread
enqueue items to concurrent_queue
if(concurrent_queue.size() == EXPECTED_SIZE) {
// do something ONLY once
}
A simple solution.
if(concurrent_queue.size() == EXPECTED_SIZE) {
// do something ONLY once
static bool doItOnce = DoItOnceOnly();
}
If you aren't concerned about the possibility of the operation failing, you can simply use an atomic<bool> that is associated with the operation. All of the threads will try to change the flag from false to true with compare_exchange_strong, but only one will succeed:
// inside a thread
enqueue items to concurrent_queue
if(concurrent_queue.size() == EXPECTED_SIZE) {
std::atomic<bool>& flag = retrieve_the_associated_bool();
bool expected = false;
if (flag.compare_exchange_strong(expected, true)) {
// do something
}
}
If the operation can fail, you should use std::call_once and a std::once_flag associated with the operation. The other threads will wait while one tries to "do something", and each try in turn until one succeeds:
// inside a thread
enqueue items to concurrent_queue
if(concurrent_queue.size() == EXPECTED_SIZE) {
std::once_flag& flag = retrieve_the_associated_once_flag();
std::call_once(flag, []{
// do something
});
// do something ONLY once
}
Check approach from Singleton implementations. The most basic:
bool executed = false;
void Execute()
{
Lock lock; // scope-based lock, released automatically when the function returns
if (executed == false)
{
executed = true;
//.. do computation
}
}
Check also solutions with atomic variables and double-checked locking for better performance:
http://preshing.com/20130930/double-checked-locking-is-fixed-in-cpp11/
Related
I have a WorkDispatcher class which holds Worker class objects as properties and launches their function in new threads.
Here is a example:
WorkDispatcher.h:
class WorkDispatcher
{
private:
std::thread m_reconstructionThread;
std::shared_ptr <Reconstruction::RGBDImageModel> m_currentRGBD;
public:
WorkDispatcher();
std::mutex m_rgbdMutex, m_commandMutex;
std::deque<Network::NetworkCommandModel> m_CommandQueue;
std::condition_variable m_RgbConditional, m_CommandConditional;
Reconstruction::SceneReconstructor m_Reconstructor;
void Initialize();
void Work();
void Stop();
};
WorkDispatcher.cpp:
void WorkDispatcher::Work()
{
m_reconstructionThread = std::thread(
&Reconstruction::SceneReconstructor::Reconstruct,
std::ref(m_Reconstructor),
std::ref(m_currentRGBD),
std::ref(m_CommandQueue),
std::ref(m_rgbdMutex),
std::ref(m_RgbConditional),
std::ref(m_commandMutex),
std::ref(m_CommandConditional)
);
}
These functions hold infinite loops and I use the condition variables to wait until work is avaible. For example my Reconstruct function:
void SceneReconstructor::Reconstruct(
std::shared_ptr<RGBDImageModel>& currentImage,
std::deque<Network::NetworkCommandModel> commandQueue,
std::mutex& rgbdMutex,
std::condition_variable& rgbdCond,
std::mutex& commandMutex,
std::condition_variable& commandConditional)
{
while (true)
{
std::unique_lock<std::mutex> rgbdLocker(rgbdMutex);
rgbdCond.wait(rgbdLocker, [this] {return m_QuitReconstructionFlag; });
// Quit flag to break out of loop
if (m_QuitReconstructionFlag)
break;
// do stuff here
}
}
So far so good, however if I want to quit the application I need to quit all of my worker threads. As seen above, for this these classes have a flag to quit, which I uses as follows:
void WorkDispatcher::Stop()
{
// for all worker threads do this
m_Reconstructor.m_QuitReconstructionFlag = true;
if (m_reconstructionThread.joinable())
m_reconstructionThread.join();
}
In theory this should stop the wait() function within a worker threads loop and then break out of the loop with the m_QuitReconstructionFlag, however this doesn't work.
What does work is the following:
remove the lambda from the wait functions
call notify_all() on the condition variables after settings the
quit-flags to true;
This works fine for me, however the question is, why doesn't the lambda work?
why doesn't the lambda work?
It works just fine, by itself.
However, C++ requires complex rules to be followed to properly synchronize multiple execution threads. Just because one execution thread sets a particular variable to a value does not guarantee you, in any way, that other execution threads will see the variable's new value. The synchronization rules govern that behavior.
So, this lambda works just fine. In its own execution thread. But if you want this lambda to observe changes to the value, made by other execution threads, this must be correctly synchronized.
Additionally, if you review your documentation of wait(), you should find a detailed explanation that says that if the condition function evaluates to false, it will not be called again until the condition variable is notified.
What does work is ... call notify_all()
Well, of course. Since wait() requires the condition variable to be notified, before it checks the waited-on condition again, then that's what you must do!
Finally, notifying the condition variable will work correctly in most cases, but, as I mentioned, synchronization rules (of which mutexes and condition variables play an important part of) have some edge cases where this, by itself, will not work. You must follow the following sequence of events strictly in order to have proper synchronization in all edge cases:
Lock the same mutex that another execution thread has locked before waiting on its condition variable.
Notify the condition variable.
Unlock the mutex
You must protect m_QuitReconstructionFlag with the same mutex used by the condition variable wait.
Or it won't work.
When using a condition variable if you do not want to learn about the C++ memory model in extreme detail, you should follow "best practices" that defend you against problems.
The best practices for a condition variable is to bundle up 3 things together.
The condition variable.
A mutex (often mutable).
A state.
Then bundle all 3 of them up behind a single abstraction of some kind.
To change the state:
Lock the mutex
Change the state
Notify the condition variable appropriately
Unlock the mutex
Do not think that the state being atomic means you don't have to lock the mutex.
When you want to wait on the condition variable:
Lock the mutex
Wait, passing a lambda that checks the state.
When exiting wait, you are free to update the state.
Unlock the mutex
In general, use a unique_lock to lock the mutex in all of the above cases, and rely on RAII to unlock it.
What, exactly, the state is, and when you notify, is up to you.
Do not interact with that mutex directly outside of this bundle and api, don't interact with the state directly outside of this bundle and api, and don't interact with the condition variable outside of this bundle and api.
Copy or move data out of the api if needed, don't hold pointers or references or iterators into it.
Your state can have more than just one variable in it. Like, you can have a queue and a bool for shutdown.
For example, suppose you have a queue.
template<class T>
struct cv_queue {
std::optional<T> pop() {
auto l = lock();
cv.wait( l, [&]{ return aborted || !queue.empty(); } );
if (aborted) return {};
auto retval = std::move(queue.front());
queue.pop_front();
return retval;
}
void push( T in ) {
auto l = lock();
queue.push_back( std::move(in) );
cv.notify_one();
}
void abort_everything() {
auto l = lock();
abort = true;
cv.notify_all();
}
bool empty() const {
auto l = lock();
return queue.empty();
}
private:
std::condition_variable cv;
mutable std::mutex m;
std::deque<T> queue;
bool aborted=false;
auto lock() const { return std::unique_lock( m ); }
};
adding pop_wait_for or try_pop isn't hard.
A simple 3-part wrapper around data or whatever isn't hard to write. Making it more generic, in my experience, doesn't add much to it being understandable.
Here the lambda returning true is not the condition to stop waiting, rather the lambda is to account for spurious wake ups. The notify or notify_all function of the conditional_variable is what is used to make the wait quit.
Rather than removing the lambda, you must simply change the stop function to
void WorkDispatcher::Stop()
{
// for all worker threads do this
m_Reconstructor.m_QuitReconstructionFlag = true;
m_RgbConditional.notify_all()
if (m_reconstructionThread.joinable())
m_reconstructionThread.join();
}
from here you can see that the wait with predicate passed to it (wait(predicate)) is equivalent to
if(!predicate())
wait()
Hence when you call Stop(), It sets the predicate to true, so when the thread is woken up wait() returns, and the predicate is checked, if it is true, the wait(predicate) returns.
In the earlier case, the predicate was set to true but the function was not woken up
I'm writing an event loop that goes to sleep when there's no work to do by waiting on a "work to do" condition variable (work_to_do). This condition variable could be notified by different threads based on various events. When an event happens in another thread it notifies on the condition variable, waking up the event loop which then checks the conditions that could have triggered the notify, loops until there's no more work to do and then waits again. One of the conditions is set by a blocking function (WaitForMessage()).
The event loop thread:
std::lock_guard<std::mutex> lock(work_to_do_lock);
for (;;) {
if (condition1) {
// Act on condition 1.
} else if (condition2) {
// Act on condition 2.
} else if (HasMessage()) {
// Act on receiving message.
} else {
work_to_do.wait(lock);
}
}
The thread that handles the notify from the blocking function:
for (;;) {
// Wait for message to be received (blocking). Once it returns you are
// guaranteed that HasMessage() will return true.
WaitForMessage();
// Wake-up the main event loop.
work_to_do.notify_one();
}
The main thread acquires a lock on the mutex guarding the condition variable (work_to_do_lock) before entering the event loop, and passes it into the wait() call when there's no work to do. To avoid lost-wakeups, the common advice is that all notifiers must hold the lock while updating their condition states. However, if you were to guard the WaitForMessage() call with work_to_do_lock you could prevent other signals from waking up the event loop.
The solution I came up with is to acquire and release the lock after WaitForMessage() but before notify_one():
for (;;) {
// Wait for message to be received (blocking). Once it returns you are
// guaranteed that HasMessage() will return true.
WaitForMessage();
{
std::lock_guard<std::mutex> lock(work_to_do_lock);
}
// Wake-up the main event loop.
work_to_do.notify_one();
}
This should avoid the lost-wakeup issue, as it is no longer possible for both the condition to become true (WaitForMessage() to return) and the notify_one() to occur in-between the condition check (HasMessage()) and the wait().
An alternative approach is to not rely on HasMessage() and just update a shared variable, which we could guard with the lock:
for (;;) {
// Wait for message to be received (blocking). Once it returns you are
// guaranteed that HasMessage() will return true.
WaitForMessage();
{
std::lock_guard<std::mutex> lock(work_to_do_lock);
has_message = true;
}
// Wake-up the main event loop.
work_to_do.notify_one();
}
Corresponding event loop that checks new condition predicate:
std::lock_guard<std::mutex> lock(work_to_do_lock);
for (;;) {
if (condition1) {
// Act on condition 1.
} else if (condition2) {
// Act on condition 2.
} else if (has_message) {
has_message = false;
// Act on receiving message.
} else {
work_to_do.wait(lock);
}
}
I've never seen the former approach before, so I was wondering if there was a flaw with the design or a reason it's typically avoided? It seems that you could use this approach as a general replacement for locking the condition variable lock before the condition state update, assuming that the specific condition state write/read itself is protected by some mutual exclusion mechanism.
Your approach works, but it’s less efficient than one which reuses whatever synchronization makes it safe to call WaitForMessage and HasMessage concurrently (or, put differently, takes your work_to_do_lock to update the HasMessage value rather than (say) using an atomic for it). Of course, if that’s inaccessible to this code, this is about the best you can do, since you need mutual exclusion for the other conditions.
I'm encountering a stuck in my c++ program using this thread pool class:
class ThreadPool {
unsigned threadCount;
std::vector<std::thread> threads;
std::list<std::function<void(void)> > queue;
std::atomic_int jobs_left;
std::atomic_bool bailout;
std::atomic_bool finished;
std::condition_variable job_available_var;
std::condition_variable wait_var;
std::mutex wait_mutex;
std::mutex queue_mutex;
std::mutex mtx;
void Task() {
while (!bailout) {
next_job()();
--jobs_left;
wait_var.notify_one();
}
}
std::function<void(void)> next_job() {
std::function<void(void)> res;
std::unique_lock<std::mutex> job_lock(queue_mutex);
// Wait for a job if we don't have any.
job_available_var.wait(job_lock, [this]()->bool { return queue.size() || bailout; });
// Get job from the queue
mtx.lock();
if (!bailout) {
res = queue.front();
queue.pop_front();
}else {
// If we're bailing out, 'inject' a job into the queue to keep jobs_left accurate.
res = [] {};
++jobs_left;
}
mtx.unlock();
return res;
}
public:
ThreadPool(int c)
: threadCount(c)
, threads(threadCount)
, jobs_left(0)
, bailout(false)
, finished(false)
{
for (unsigned i = 0; i < threadCount; ++i)
threads[i] = std::move(std::thread([this, i] { this->Task(); }));
}
~ThreadPool() {
JoinAll();
}
void AddJob(std::function<void(void)> job) {
std::lock_guard<std::mutex> lock(queue_mutex);
queue.emplace_back(job);
++jobs_left;
job_available_var.notify_one();
}
void JoinAll(bool WaitForAll = true) {
if (!finished) {
if (WaitForAll) {
WaitAll();
}
// note that we're done, and wake up any thread that's
// waiting for a new job
bailout = true;
job_available_var.notify_all();
for (auto& x : threads)
if (x.joinable())
x.join();
finished = true;
}
}
void WaitAll() {
std::unique_lock<std::mutex> lk(wait_mutex);
if (jobs_left > 0) {
wait_var.wait(lk, [this] { return this->jobs_left == 0; });
}
lk.unlock();
}
};
gdb say (when stopping the blocked execution) that the stuck was in (std::unique_lock&, ThreadPool::WaitAll()::{lambda()#1})+58>
I'm using g++ v5.3.0 with support for c++14 (-std=c++1y)
How can I avoid this problem?
Update
I've edited (rewrote) the class: https://github.com/edoz90/threadpool/blob/master/ThreadPool.h
The issue here is a race condition on your job count. You're using one mutex to protect the queue, and another to protect the count, which is semantically equivalent to the queue size. Clearly the second mutex is redundant (and improperly used), as is the job_count variable itself.
Every method that deals with the queue has to gain exclusive access to it (even JoinAll to read its size), so you should use the same queue_mutex in the three bits of code that tamper with it (JoinAll, AddJob and next_job).
Btw, splitting the code at next_job() is pretty awkward IMO. You would avoid calling a dummy function if you handled the worker thread body in a single function.
EDIT:
As other comments have already stated, you would probably be better off getting your eyes off the code and reconsidering the problem globally for a while.
The only thing you need to protect here is the job queue, so you need only one mutex.
Then there is the problem of waking up the various actors, which requires a condition variable since C++ basically does not give you any other useable synchronization object.
Here again you don't need more than one variable. Terminating the thread pool is equivalent to dequeueing the jobs without executing them, which can be done any which way, be it in the worker threads themselves (skipping execution if the termination flag is set) or in the JoinAll function (clearing the queue after gaining exclusive access).
Last but not least, you might want to invalidate AddJob once someone decided to close the pool, or else you could get stuck in the destructor while someone keeps feeding in new jobs.
I think you need to keep it simple.
you seem to be using a mutex too many. So there's queue_mutex and you use that when you add and process jobs.
Now what's the need for another separate mutex when you are waiting on reading the queue?
Why can't you use just a conditional variable with the same queue_mutex to read the queue in your WaitAll() method?
Update
I would also recommend using a lock_guard instead of the unique_lock in your WaitAll. There really isn't a need to lock the queue_mutex beyond the WaitAll under exceptional conditions. If you exit the WaitAll exceptionally it should be released regardless.
Update2
Ignore my Update above. Since you are using a condition variable you can't use a lock guard in the WaitAll. But if you are using a unique_lock always go with the try_to_lock version especially if you have more than a couple control paths
I'm refactoring some time consuming function so that it can be called from a thread, but I'm having trouble wrapping my head around the issue (not very familiar with thread programming).
At any point, the user can cancel and the function will stop. I do not want to kill the thread as soon as the user cancels since it could cause some data integrity problems. Instead, in several places in the function, I will check if the function has been cancelled and, if so, exit. I will only do that where I know it's safe to exit.
The whole code of the function will be within a mutex. This is the pseudo-code I have in mind:
SomeClass::SomeClass() {
cancelled_ = false;
}
void SomeClass::cancelBigSearch() {
cancelled_ = true;
}
void SomeClass::bigSearch() {
mutex.lock();
// ...
// Some code
// ...
// Safe to exit at this point
if (cancelled_) {
mutex.unlock();
cancelled_ = false;
return;
}
// ...
// Some more code
// ...
if (cancelled_) {
mutex.unlock();
cancelled_ = false;
return;
}
// ...
// Again more code
// ...
if (cancelled_) {
mutex.unlock();
cancelled_ = false;
return;
}
mutex.unlock();
}
So when the user starts a search, a new thread calls bigSearch(). If the user cancels, cancelBigSearch() is called and a cancelled_ flag is set. Then, when bigSearch() reaches a point where it's safe to exit, it will exit.
Any idea if this is all thread-safe?
You should lock access to cancelled_ with another mutex, so checking and setting does not happen simultaneously. Other than that, I think your approach is OK
Update: Also, make sure no exceptions can be thrown from SomeClass::bigSearch(), otherwise the mutex might remain in a locked state. To make sure that all return paths unlock the mutex, you might want to surround the processing parts of the code with if (!cancelled_) and return only at the very end of the method (where you have the one unlock() call on the mutex.
Better yet, wrap the mutex in a RAII (acronym for Resource Allocation Is Initialization) object, so no matter how the function ends (exception or otherwise), the mutex is guaranteed to be unlocked.
Yes, this is thread safe. But:
Processors can have separate cache and cache it's own copy of cancelled_, typically mutex synchronization functions applies proper cache synchronization.
Compiler generated code, can make invalid assumptions about Your data locality, this can lead to not update in time cancelled_. Some platform specific commands can help here, or you can simply use other mechanisms.
All these lead to a thread that isn't canceled in time as you wish.
Your code usage pattern is simple "signaling". So you need to transfer signal to thread. Signal patterns allows trigger multiple times same trigger (signal), and clear it later.
This can be simulated using:
atomic operations
mutex protected variables
signal synchronization primitives
It's not thread-safe, because one thread could read cancelled_ at the same time another thread writes to it, which is a data race, which is undefined behaviour.
As others suggested, either use an atomic type for cancelled_ or protect it with another mutex.
You should also use RAII types to lock the mutexes.
e.g.
void SomeClass::cancelBigSearch() {
std::lock_guard<std::mutex> lock(cxlMutex_);
cancelled_ = true;
}
bool SomeClass::cancelled() {
std::lock_guard<std::mutex> lock(cxlMutex_);
if (cancelled_) {
// reset to false, to avoid caller having to lock mutex again to reset it
cancelled_ = false;
return true;
}
return false;
}
void SomeClass::bigSearch() {
std::lock_guard<std::mutex> lock(mutex);
// ...
// Some code
// ...
// Safe to exit at this point
if (cancelled())
return;
// ...
// Some more code
// ...
if (cancelled())
return;
// ...
// Again more code
// ...
if (cancelled())
return;
}
A similar problem is this one: Are threads waiting on a lock FIFO? However, in this problem, once the lock is acquired only one thread executes the protected code, and in the end all threads will have executed the code.
What I would like to do is to execute the protected code once, but for all threads queuing for the method call at that moment, return true.
Basically, the protected code is a global checkpoint, which is relevant for all threads waiting at that moment. I.e., doing N consecutive checkpoints would not achieve more than only 1.
Note that while the checkpointing is done, there will be other calls to the method, which themselves need a new checkpoint call.
I believe what I want to do is "batch-wise" synchronized calls to the global function.
How can I achieve this in C++, perhaps with Boost?
You seem to be looking for try_lock().
Given some Boost.Thread Lockable, a call to Lockable::try_lock() will return true if it can acquire the lock at that moment, otherwise false if it cannot acquire the lock.
When your thread reaches a checkpoint, have it try to acquire this lock. If it fails, another thread is already in the function. If it succeeds, check some bool to see if the checkpoint has already been run. If it has been run, release the lock and continue. If it hasn't been run, keep the lock and run the checkpoint function and set the checkpoint bool to true.
What you seem to want looks like a barrier which is provided by boost. However, if that doesn't help you, you can make something with condition variables, also in boost
Here is pseudo-code for how I would do it. I am assuming the existing of a mutex class with lock() and unlock() operations.
// This forward declaration helps with declaration
// of the "friend" status for the nested class.
class DoItOnce;
class DoItOnce
{
private:
bool m_amFirst;
mutex m_mutex;
friend class ::DoItOnce::Op;
public:
DoItOnce()
{
m_amFirst = true;
init(m_mutex);
}
~DoItOnce() { destroy(m_mutex); }
void reset()
{
m_mutex.lock();
m_amFirst = true;
m_mutex.lock();
}
//--------
// Nested class
//--------
class Op {
public:
Op(DoItOnce & sync)
: m_sync(sync)
{
m_sync.m_mutex.lock();
m_amFirst = m_sync.m_amFirst;
m_sync.m_amFirst = false;
}
~Op() { m_sync.m_mutex.unlock(); }
bool amFirst() { return m_amFirst; }
private:
DoItOnce & m_sync;
bool m_amFirst;
}; // end of nested class
}; // end of outer class
Here is an example to illustrate its intended use. You will implement the doWork() operation and have all your threads invoke it.
class WorkToBeDoneOnce
{
private:
DoItOnce m_sync;
public:
bool doWork()
{
DoItOnce::Op scopedLock(m_sync);
if (!scopedLock.amFirst()) {
// The work has already been done.
return true;
}
... // Do the work
return true;
}
void resetAmFirstFlag()
{
m_sync.reset();
}
}
If you are confused by my use of the DoItOnce::Op nested class, then you can find an explanation of this coding idiom in my Generic Synchronisation Policies paper, which is available here in various formats (HTML, PDF and slides).