I have a groups of objects, each object has two threads: Task thread processes the data and notifies Decision thread that the data is ready, then waits for Decision thread to make the decision whether to continue operations; Decision thread waits Task thread for the data, then consumes the data and make a decision ( notify Task thread that the decision is ready to fetch ).
Task.cpp:
class Task{
public:
void DoTask(){
// process data
{
std::unique_lock<std::mutex> lck(mtx);
data_ready = true;
cv_data.notify_one();
while( decision_ready == false )
cv_decision.wait( lck );
}
if ( decision )
// continue task
else
// quit
}
void SetDecision( bool flag ) { decision = flag; }
bool GetDataFlag() const { return data_ready; }
bool SetDecisionFlag( bool flag ) { decision_ready = flag; }
std::mutex mtx;
std::condition_variable cv_data;
std::condition_variable cv_decision;
private:
bool decision;
bool data_ready;
bool decision_ready;
};
main.cpp:
void Decision ( Task *task );
int main(){
Task mytask[10];
std::thread do[10];
std::thread decision[10];
for(int i=0; i< 10; ++i)
{
do[i] = std::thread( &Task::doTask, &mytask[i] );
decision[i] = std::thread( Decision, &mytask[i] );
do[i].detach();
decision[i].detach();
}
}
void Decision( Task *task )
{
st::mutex mtx_decision;
std::unique_lock<std::mutex> lck( task->mtx );
while( task->GetDataFlag() == false )
task->cv_data.wait(lck);
std::lock_guard<std::mutex> lk(mtx_decision);
// check database and make decision
task->SetDecision( true );
task->SetDecisionFlag( true );
task->cv_decision.notify_one();
}
What is the problem with this approach? The program works well only in single thread case. If I actually open two or more threads, I get segmentation fault. I am not sure how to pass the condition variables between different scopes. And hope someone can tell me the right way to do it. Thanks.
I suppose you need the same mutex and same conditional variable to get it working. Now each class gets own mutex and condition_variable and each decision too.
The most likely reason while your application crashes is because you detach your threads and than your main() exits, killing threads in the midst of what they are doing. I strongly advice against using detached threads.
Related
I've got a Timer class that can run with both an initial time and an interval. There's an internal function internalQuit performs thread.join() before a thread is started again on the resetCallback. The thing is that each public function has it's own std::lock_guard on the mutex to prevent the data of being written. I'm now running into an issue that when using the callback to for example stop the timer in the callback, the mutex cannot be locked by stop(). I'm hoping to get some help on how to tackle this issue.
class Timer
{
public:
Timer(string_view identifier, Function &&timeoutHandler, Duration initTime, Duration intervalTime);
void start()
void stop() // for example
{
std::lock_guard lock{mutex};
running = false;
sleepCv.notify_all();
}
void setInitTime()
void setIntervalTime()
void resetCallback(Function &&timeoutHandler)
{
internalQuit();
{
std::lock_guard lock{mutex};
quit = false;
}
startTimerThread(std::forward<Function>(timeoutHandler));
}
private:
internalQuit() // performs thread join
{
{
std::lock_guard lock {mutex};
quit = true;
running = false;
sleepCv.notify_all();
}
thread.join();
}
mainLoop(Function &&timeoutHandler)
{
while(!quit)
{
std::unique_lock lock{mutex};
// wait for running with sleepCv.wait()
// handle initTimer with sleepCv.wait_until()
timeoutHandler(); // callback
// handle intervalTimer with sleepCv.wait_until()
timeoutHandler(); // callback
}
}
startTimerThread(Function &&timeoutHandler)
{
thread = std::thread([&, timeoutHandler = std::forward<Function>(timeoutHandler)](){
mainLoop(timeoutHandler);
});
}
std::thread thread{};
std::mutex mutex{};
std::condition_variable sleepCv{}
// initTime, intervalTime and some booleans for updating with sleepCv.notify_all();
}
For testing this, I have the following testcase in Gtest. I'm expecting the timer to stop in the callback. Unfortunately, the timer will hang on acquiring the mutex lock in the stop() function.
std::atomic<int> callbackCounter;
void timerCallback()
{
callbackCounter.fetch_add(1, std::memory_order_acq_rel);
}
TEST(timerTest, timerShouldStopWhenStoppedInNewCallback)
{
std::atomic<int> testCounter{0};
Timer<std::chrono::steady_clock > t{"timerstop", &timerCallback, std::chrono::milliseconds(0), std::chrono::milliseconds(100)};
t.resetCallback([&]{
testCounter += 1;
t.stop();
});
t.start();
sleepMilliSeconds(100);
ASSERT_EQ(testCounter.load(), 1); // trigger due to original interval timeout
sleepMilliSeconds(100);
ASSERT_EQ(testCounter.load(), 1); // no trigger, because stopped in new callback
}
Removing all the mutexes in each of the public fucntions, fixes the issue. But that could lead to possible race conditions for data being written to variables. Hence each function has a lock before writing to f.e. the booleans.
I've tried looking into the std::move functionality to move the thread during the resetCallback into a different variable and then call join on that one. I'm also investigating recursive_mutex but have no experience with using that.
void resetCallback(Function &&timeoutHandler)
{
internalQuit();
{
std::lock_guard lock{mutex};
quit = false;
}
auto prevThread = std::thread(std::move(this->thread));
// didn't know how to continue from here, requiring more selfstudy.
startTimerThread(std::forward<Function>(timeoutHandler));
}
It's a new subject for me, have worked with mutexes and timers before but with relatively simple stuff.
Thank you in advance.
If data race is not an issue, can I use std::condition_variable for starting (i.e., signaling) and stopping (i.e, wait) a thread for work?
For example:
std::atomic<bool> quit = false;
std::atomic<bool> work = false;
std::mutex mtx;
std::condition_variable cv;
// if work, then do computation, otherwise wait on work (or quit) to become true
// thread reads: work, quit
void thread1()
{
while ( !quit )
{
// limiting the scope of the mutex
{
std::unique_lock<std::mutex> lck(mtx);
// I want here is to wait on this lambda
cv.wait(lck, []{ return work || quit; });
}
if ( work )
{
// work can become false again while working.
// I want here is to complete the work
// then wait on the next iteration.
ComputeWork();
}
}
}
// work controller
// thread writes: work, quit
void thread2()
{
if ( keyPress == '1' )
{
// is it OK not to use a mutex here?
work = false;
}
else if ( keyPress == '2' )
{
// ... or here?
work = true;
cv.notify_all();
}
else if ( keyPress == ESC )
{
// ... or here?
quit = true;
cv.notify_all();
}
}
Update/Summary: not safe because of 'lost wakeup' scenario that Adam describes.
cv.wait(lck, predicate()); can be equivalently written as while(!predicate()){ cv.wait(lck); }.
To see the problem easier: while(!predicate()){ /*lost wakeup can occur here*/ cv.wait(lck); }
Can be fixed by putting any read/writes of predicate variables in the mutex scope:
void thread2()
{
if ( keyPress == '1' )
{
std::unique_lock<std::mutex> lck(mtx);
work = false;
}
else if ( keyPress == '2' )
{
std::unique_lock<std::mutex> lck(mtx);
work = true;
cv.notify_all();
}
else if ( keyPress == ESC )
{
std::unique_lock<std::mutex> lck(mtx);
quit = true;
cv.notify_all();
}
}
No, not safe. The waiting thread can get the mutex, check the predicate, sees nothing to wake up for. Then the signalling thread sets the bool, and signals. Next, the waiting thread blocks on the cv, and never awakens.
You must hold the mutex at some point between triggering the wakeup lambda condition, and notifying the cv, to avoid this.
The "down" case (turning off wakeup) I have not looked at, and it may depend on what behaviour exactly is ok. Without that specified in a formal sense I wouldn't do it either; in general, you should at least attempt sketches of formal proofs of correctness when fiddling with multi threaded code, or your code will be at best accidentally working.
If you can't do that, find someone who can to write that code for you.
UPD: It seems that the problem which I explain below is non-existent. I cannot reproduce it in a week already, I started suspecting that it was caused by some bugs in a compiler or corrupted memory because it is not reproducing anymore.
I tried to implement my own recursive mutex in C++, but for some reason, it fails. I tried to debug it, but I stuck. (I know that there are recursive mutex in std, but I need a custom implementation in a project where STL is not available; this implementation was just a check of an idea). I haven't thought about efficiency yet, but I don't understand why my straightforward implementation doesn't work.
First of all, here's the implementation of the RecursiveMutex:
class RecursiveMutex
{
std::mutex critical_section;
std::condition_variable cv;
std::thread::id id;
int recursive_calls = 0;
public:
void lock() {
auto thread = std::this_thread::get_id();
std::unique_lock<std::mutex> lock(critical_section);
cv.wait( lock, [this, thread]() {
return id == thread || recursive_calls == 0;
});
++recursive_calls;
id = thread;
}
void unlock() {
std::unique_lock<std::mutex> lock( critical_section );
--recursive_calls;
if( recursive_calls == 0 ) {
lock.unlock();
cv.notify_all();
}
}
};
The failing test is straightforward, it just runs two threads, both of them are locking and unlocking the same mutex (the recursive nature of the mutex is not tested here). Here it is:
std::vector<std::thread> threads;
void initThreads( int num_of_threads, std::function<void()> func )
{
threads.resize( num_of_threads );
for( auto& thread : threads )
{
thread = std::thread( func );
}
}
void waitThreads()
{
for( auto& thread : threads )
{
thread.join();
}
}
void test () {
RecursiveMutex mutex;
while (true) {
int count = 0;
initThreads(2, [&mutex] () {
for( int i = 0; i < 100000; ++i ) {
try {
mutex.lock();
++count;
mutex.unlock();
}
catch (...) {
// Extremely rarely.
// Exception is "Operation not permited"
assert(false);
}
}
});
waitThreads();
// Happens often
assert(count == 200000);
}
}
In this code I have two kinds of errors:
Extremely rarely I get an exception in RecursiveMutex::lock() which contains message "Operation not permitted" and is thrown from cv.wait. As far as I understand, this exception is thrown when wait is called on a mutex which is not owned by the thread. At the same time, I lock it just above calling the wait so this cannot be the case.
In most situations I just get a message into console "terminate called without an active exception".
My main question is what the bug is, but I'll also be happy to know how to debug and provoke race condition in such a code in general.
P.S. I use Desktop Qt 5.4.2 MinGW 32 bit.
This application is recursive multi-thread detached one. Each thread regenerate
new bunch of threads before it dies.
Option 1 (works) however it's a shared resource hence slows the application down.
Option 2 should remove this bottleneck.
Option 1 works:
std::condition_variable cv;
bool ready = false;
std::mutex mu;
// go triggers the thread's function
void go() {
std::unique_lock<std::mutex> lck( mu );
ready = true;
cv.notify_all();
}
void ThreadFunc ( ...) {
std::unique_lock<std::mutex> lck ( mu );
cv.wait(lck, []{return ready;});
do something useful
}
Option 2 does NOT trigger the thread:
std::array<std::mutex, DUToutputs*MaxGnodes> arrMutex ;
void go ( long m , long Channel )
{
std::unique_lock<std::mutex> lck( arrMutex[m+MaxGnodes*Channel] );
ready = true;
cv.notify_all();
}
void ThreadFunc ( ...) {
std::unique_lock<std::mutex> lck ( arrMutex[Inst+MaxGnodes*Channel] );
while (!ready) cv.wait(lck);
do something useful
}
How can I make option #2 work?
The code in Option 2 contains a so-called data race on the variable ready, because the read and write operations on this variable are no longer synchronized. The behaviour of programs with data races is undefined. You can remove the data race by changing bool ready to std::atomic<bool> ready.
That should already fix the problem in Option 2. However, if you use std::atomic, you can also make other optimizations:
std::atomic<bool> ready{false};
void go(long m, long Channel) {
// no lock required
ready = true;
cv.notify_all();
}
void ThreadFunc( ...) {
std::unique_lock<std::mutex> lck(arrMutex[Inst+MaxGnodes*Channel]);
cv.wait(lck, [] { return ready; });
// do something useful
}
I have a class that implements a threaded producer/consumer system using a mutex and two condition variables for synchronization. The producer signals the consumer thread when there are items to use, and the consumer signals the producer thread when it has consumed the items. The threads continue producing and consuming until the destructor requests them to quit by setting a boolean variable. Because either of the threads may be waiting on a condition variable, I have to implement a second check of the quit variable, which feels wrong and messy...
I've reduced the problem down to the following (working on GNU/Linux with g++4.7) example:
// C++11and Boost required.
#include <cstdlib> // std::rand()
#include <cassert>
#include <boost/circular_buffer.hpp>
#include <atomic>
#include <chrono>
#include <condition_variable>
#include <iostream>
#include <mutex>
#include <thread>
#include <vector>
// Creates a single producer and single consumer thread.
class prosumer
{
public:
// Create the circular buffer and start the producer and consumer thread.
prosumer()
: quit_{ false }
, buffer_{ circular_buffer_capacity }
, producer_{ &prosumer::producer_func, this }
, consumer_{ &prosumer::consumer_func, this }
{}
// Set the quit flag and wait for the threads to exit.
~prosumer()
{
quit_ = true;
producer_.join();
consumer_.join();
}
private:
// Thread entry point for the producer.
void producer_func()
{
// Value to add to the ringbuffer to simulate data.
int counter = 0;
while ( quit_ == false )
{
// Simulate the production of some data.
std::vector< int > produced_items;
const auto items_to_produce = std::rand() % circular_buffer_capacity;
for ( int i = 0; i < items_to_produce; ++i )
{
produced_items.push_back( ++counter );
}
// Get a lock on the circular buffer.
std::unique_lock< std::mutex > lock( buffer_lock_ );
// Wait for the buffer to be emptied or the quit flag to be set.
buffer_is_empty_.wait( lock, [this]()
{
return buffer_.empty() == true || quit_ != false;
} );
// Check if the thread was requested to quit.
if ( quit_ != false )
{
// Don't let the consumer deadlock.
buffer_has_data_.notify_one();
break;
}
// The buffer is locked by this thread. Put the data into it.
buffer_.insert( std::end( buffer_ ), std::begin( produced_items ), std::end( produced_items ) );
// Notify the consumer that the buffer has some data in it.
buffer_has_data_.notify_one();
}
std::cout << "producer thread quit\n";
}
// Thread entry for the consumer.
void consumer_func()
{
int counter_check = 0;
while ( quit_ == false )
{
std::unique_lock< std::mutex > lock( buffer_lock_ );
// Wait for the buffer to have some data before trying to read from it.
buffer_has_data_.wait( lock, [this]()
{
return buffer_.empty() == false || quit_ != false;
} );
// Check if the thread was requested to quit.
if ( quit_ != false )
{
// Don't let the producer deadlock.
buffer_is_empty_.notify_one();
break;
}
// The buffer is locked by this thread. Simulate consuming the data.
for ( auto i : buffer_ ) assert( i == ++counter_check );
buffer_.clear();
// Notify the producer thread that the buffer is empty.
buffer_is_empty_.notify_one();
}
std::cout << "consumer thread quit\n";
}
// How many items the circular buffer can hold.
static const int circular_buffer_capacity = 64;
// Flag set in the destructor to signal the threads to stop.
std::atomic_bool quit_;
// Circular buffer to hold items and a mutex for synchronization.
std::mutex buffer_lock_;
boost::circular_buffer< int > buffer_;
// Condition variables for the threads to signal each other.
std::condition_variable buffer_has_data_;
std::condition_variable buffer_is_empty_;
std::thread producer_;
std::thread consumer_;
};
int main( int argc, char **argv )
{
(void)argc; (void) argv;
prosumer test;
// Let the prosumer work for a little while.
std::this_thread::sleep_for( std::chrono::seconds( 3 ) );
return EXIT_SUCCESS;
}
If you look at the producer_func and consumer_func thread functions you can see that they loop until the quit variable is set by the prosumer destructor, but they also check for the quit variable again after they lock the circular buffer. If the quit variable was set, they signal each other to prevent a deadlock.
Another idea I had was to call notify_one() on the condition variables from the destructor, would that be a better solution?
Is there a better way to do this?
Update 1: I forgot to mention that in this instance, when the threads are requested to exit, the consumer does not need to consume any remaining data in the circular buffer and it's fine if the producer produces a little bit more too. As long as they both exit and don't deadlock all will be well.
In my opinion, calling notify_one (or rather notify_all if you were to extend your buffer to multiple producers/consumers) on both condition variables in the destructor before the calls to join would be the preferred solution for several reasons:
Firstly, this matches the way that conditional variables are typically used: By setting quit_, you change the state that the producer/consumer threads are interested in and wait for, so you should notify them of the state change.
Furthermore, notify_one should not be a very costly operation.
Also, in a more realistic application, it could be the case that in between the production of two elements, there is a delay; in that case you may not want to block in your destructor until the consumer notices it has to cancel as soon as the next element is enqueued; in the example code, that does not occur, as far as I can see.
In my opinion, there are two functionalities that can be separated:
message passing and dispatching
producing and consuming
It does make sense to really separate them: the 'worker' thread does nothing more than process 'messages' that could mean 'quit' or 'do_work'.
This way you can create a generic 'worker' class that aggregates the actual function. The produce and consume methods stay clean, and the worker class care only about keeping the work going.