Deadlock using std::thread and std::condition_variable - c++

I'm investigating an issue where my worker thread deadlocks when I try to stop it.
Here's the minimal version that has the problem:
#include <atomic>
#include <condition_variable>
#include <functional>
#include <iostream>
#include <memory>
#include <mutex>
#include <thread>
#include <vector>
class Worker
{
private:
std::atomic<bool> m_running;
std::condition_variable m_cond;
std::mutex m_mutex;
std::function<void()> m_workItem;
std::thread m_thread;
public:
Worker() :
m_running(true),
m_thread(std::bind(&Worker::DoWork, this))
{
}
~Worker()
{
Stop();
m_thread.join();
}
bool QueueWork(std::function<void()> item)
{
std::unique_lock<std::mutex> padlock(m_mutex);
if (m_workItem)
{
return false;
}
m_workItem = item;
padlock.unlock();
m_cond.notify_all();
return true;
}
void Stop()
{
bool expected = true;
if (m_running.compare_exchange_strong(expected, false))
{
m_cond.notify_all();
}
}
private:
void DoWork() noexcept
{
while (m_running)
{
std::unique_lock<std::mutex> padlock(m_mutex);
m_cond.wait(padlock,
[this]
()
{
return !m_running || m_workItem;
});
if (m_workItem)
{
decltype(m_workItem) workItem;
std::swap(m_workItem, workItem);
padlock.unlock();
workItem();
}
}
}
};
int main()
{
std::cout << "Start." << std::endl;
{
std::vector<std::unique_ptr<Worker>> workers;
for (int i = 0; i < 10; ++i)
{
workers.push_back(std::unique_ptr<Worker>(new Worker()));
}
workers[0]->QueueWork(
[]()
{
std::cout << "Work item" << std::endl;
});
std::this_thread::sleep_for(std::chrono::milliseconds(10));
for (auto & worker : workers)
{
worker->Stop();
}
for (auto & worker : workers)
{
worker.reset();
}
}
std::cout << "Stop." << std::endl;
return 0;
}
The idea is that when calling Worker::Stop() from the host thread, Worker::m_running is set to false, and notify_all() is called on Worker::m_cond. The worker thread wakes up from its m_cond.wait(), checks m_running and breaks out, exiting the thread function.
Sometimes, however, this deadlocks. The worker thread wakes up, sees that m_running is true (how is that possible?) and goes back to wait(). There's no additional call to m_cond.notify_all() so the thread ends up in a deadlocked state.
I spawn 10 Worker objects in this code. I don't think it has anything to do with the number of threads, but to be able to trigger the race condition (if that's what it is), I needed more threads.
What is wrong with the code?
Running gcc:
g++ --version
g++ (Ubuntu 4.9.2-0ubuntu1~14.04) 4.9.2
Copyright (C) 2014 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Compiling using:
g++ -std=c++11 -pedantic test.cpp -lpthread
Edit:
Changed the order of Worker's private members. m_thread is now last. Still the same problem.

Related

As i have made the flag volatile at some point it value of flag should change. But it is waiting infinitely on t thread only

#include <iostream>
#include<thread>
#include <initializer_list>
#include <vector>
#include <future>
#include <time.h>
using namespace std;
class Gadget{
public:
Gadget(){
flag_ = false;
cout<<"Creating new Gadgets"<<endl;
}
void wait(){
while(flag_==false){
cout<<"waiting here...."<<endl;
this_thread::sleep_for(chrono::milliseconds(1000));
}
}
void wake(){
flag_ = true;
}
private:
volatile bool flag_;
};
I am trying to make two threads and one thread will sleep for 1 sec after checking the flag value. As i have made flag volatile it should change at some point. But the program is waiting infinitely.
int main() {
Gadget g;
thread t(&Gadget::wait,g);
thread s(&Gadget::wake,g);
t.join();
s.join();
cout<<"Ending the program "<<endl;
return 0;
}
volatile isn't for variables that are changed by the program itself. It's for variables that changes outside the program's control - like if it's directly connected to hardware.
Your main problem is however that you pass g by value so the two threads are working on different copies of your original g.
So, change to
std::atomic<bool> flag_;
and
thread t(&Gadget::wait, &g);
thread s(&Gadget::wake, &g);
Also worth mentioning: The two methods will not necessarily run in the order you start them, so waiting here.... may or may not show up.
Edit:
As mentioned in the comments: When waiting for a condition you should usually use a std::condition_variable. I've made an example of how that could look. I've also moved the starting of the threads into Gadget which makes it more obvious which object the thread is working on.
#include <chrono>
#include <condition_variable>
#include <iostream>
#include <mutex>
#include <thread>
#include <vector>
class Gadget {
public:
Gadget() { std::cout << "Creating new Gadget\n"; }
// new interface for starting threads
std::thread start_wait() { return std::thread(&Gadget::wait, this); }
std::thread start_wake() { return std::thread(&Gadget::wake, this); }
private:
void wait() {
std::unique_lock<std::mutex> ul(mutex_);
std::cout << "wait: waiting here...\n";
// Read about "spurious wakeup" to understand the below:
while(not flag_) cond_.wait(ul);
// or:
// cond_.wait(ul, [this] { return flag_; });
std::cout << "wait: done\n";
}
void wake() {
// simulate some work being done for awhile
std::this_thread::sleep_for(std::chrono::milliseconds(1000));
{ // lock context start
std::lock_guard<std::mutex> lg(mutex_);
flag_ = true;
std::cout << "wake: notifying the waiting threads\n";
} // lock context end
// notify all waiting threads
cond_.notify_all();
}
std::condition_variable cond_;
std::mutex mutex_;
bool flag_ = false; // now guarded by a mutex instead
};
int main() {
Gadget g;
// start some waiting threads
std::vector<std::thread> threads(16);
for(auto& th : threads) th = g.start_wait();
// and one that wakes them up
auto th_wake = g.start_wake();
for(auto& th : threads) th.join();
th_wake.join();
std::cout << "Ending the program\n";
}

How to wait for multiple threads to finish (with c++11 threads)?

I'm trying to write a program which uses c++11 threads functionality in order to spawn multiple threads, the main thread must wait for each spawned thread to be finished, and all spawned threads must run in parallel. I've come up with the following approach:
#include <iostream>
#include <stdio.h>
#include <thread>
#include <condition_variable>
#include <mutex>
using namespace std;
class Producer
{
public:
Producer(int a_id):
m_id(a_id),
m_running(false),
m_ready(false),
m_terminate(false)
{
m_id = a_id;
m_thread = thread(&Producer::run, this);
while (!m_ready) {}
}
~Producer() {
terminate();
m_thread.join();
}
void wait() {
unique_lock<mutex> lock(m_waitForRunFinishMutex);
m_cond.wait(lock);
// avoid spurious wake up
if (m_running) {
wait();
}
lock.unlock();
cout << "wait exit " << m_id << endl;
}
void start() {
m_running = true;
m_cond.notify_all();
}
void terminate() {
start();
m_terminate = true;
}
void run() {
m_ready = true;
do {
unique_lock<mutex> lock(m_mutex);
while (!m_running) {
m_cond.wait(lock);
}
if (!m_terminate) {
cout << "running thread: " << m_id << endl;
}
m_running = false;
m_cond.notify_all();
} while (!m_terminate);
}
private:
int m_id;
bool m_running;
bool m_ready;
bool m_terminate;
thread m_thread;
mutex m_mutex;
mutex m_waitForRunFinishMutex;
condition_variable m_cond;
};
The program runs fine when testing with just one thread, i.e the following program:
int main()
{
Producer producer1(1);
producer1.start();
producer1.wait();
return 0;
}
Results in the following output:
running thread: 1
wait exit: 1
However if I test the program with 2 thread, e.g:
int main()
{
Producer producer1(1);
Producer producer2(2);
producer1.start();
producer2.start();
producer1.wait();
producer2.wait();
return 0;
}
I get the following output:
running thread: 2
running thread: 1
wait exit 1
It seems producer2 never get notified (in producer2.wait()), and therefore the program never finishes. Hopefully somebody can point out what I'm missing here.
Thanks everybody for the help in addressing the problem. Eventually the root cause of the problem is described in point (3) of the accepted answer. I've solved this by correcting the wait function as follows:
void wait() {
unique_lock<mutex> lock(m_waitForRunFinishMutex);
while (m_running) {
m_cond.wait(lock);
}
lock.unlock();
}
Here's a quick collection of issues from a glance.
wait() is recursive without unlocking its unique lock (as per the comment from Detonar)
while (!m_ready) {} Is not in a memory barrier (try compiling with some optimization and see what happens!)
If the worker thread completes before wait() is called; there is no check performed before waiting on the condition variable. Since the worker thread is complete; it will never get woken. Clearly you must check to see if the thread can get woken up within the mutex before waiting on the condition variable.

C++11 Watchdog class, test application doesn't want to exit

I am using an online C++11 compiler, link found here: cpp.sh (C++ Shell).
In my current project, I would like to have a watchdog class, to be able to check somehow the status of a thread or FSM (for example).
After some work (I'm not a C++11 guru), I finally got the code below, that compiles ok.
I also did some basic/trivial tests, but it seems the test program doesn't want to exit.
It says "Program running" and the only way to (force) exit is to hit the "Stop" button... :(
Well, my question : What am I doing wrong?
Any ideas, suggestions you can provide are highly appreciated.
Here is the full code, including my test app:
Watchdog (as MCVE):
#include <thread>
#include <atomic>
#include <chrono>
#include <condition_variable>
#include <mutex>
#include <iostream>
using namespace std::chrono;
class Watchdog
{
public:
Watchdog();
~Watchdog();
void Start(unsigned int milliseconds, std::function<void()> callback = 0);
void Stop();
void Pet();
private:
unsigned int m_interval;
std::atomic<bool> m_running;
std::thread m_thread;
std::function<void()> m_callback;
std::mutex m_mutex;
steady_clock::time_point m_lastPetTime;
std::condition_variable m_stopCondition;
void Loop();
};
Watchdog::Watchdog()
{
m_running = false;
}
Watchdog::~Watchdog()
{
Stop();
}
void Watchdog::Start(unsigned int milliseconds, std::function<void()> callback)
{
std::unique_lock<std::mutex> locker(m_mutex);
if(m_running == false)
{
m_lastPetTime = steady_clock::now();
m_interval = milliseconds;
m_callback = callback;
m_running = true;
m_thread = std::thread(&Watchdog::Loop, this);
}
}
void Watchdog::Stop()
{
std::unique_lock<std::mutex> locker(m_mutex);
if(m_running == true)
{
m_running = false;
m_stopCondition.notify_all();
m_thread.join();
}
}
void Watchdog::Pet()
{
std::unique_lock<std::mutex> locker(m_mutex);
m_lastPetTime = steady_clock::now();
m_stopCondition.notify_all();
}
void Watchdog::Loop()
{
std::unique_lock<std::mutex> locker(m_mutex);
while(m_running == true)
{
if(m_stopCondition.wait_for(locker, milliseconds(m_interval)) == std::cv_status::timeout)
{
if(m_callback != nullptr)
m_callback();
}
}
}
int main(int argc, char *argv[])
{
Watchdog wdog;
wdog.Start(3000, [] { std::cout << " WDOG TRIGGERED!!! "; });
for(auto i = 0; i < 10; i++)
{
std::cout << "[+]";
wdog.Pet();
std::this_thread::sleep_for(std::chrono::milliseconds(500));
}
}
-
You're doing a deadlock here.
void Watchdog::Stop()
{
std::unique_lock<std::mutex> locker(m_mutex);
if(m_running == true)
{
m_running = false;
m_stopCondition.notify_all();
m_thread.join();
^ ~~~~~~~~~~~~~~
m_mutex is locked; m_thread cannot continue execution
}
}
Some additional suggestion: use simple if conditions, do not compare with true or false.

Can multiple threads join the same boost::thread?

pthreads has undefined behavior if multiple threads try to join the same thread:
If multiple threads simultaneously try to join with the same thread,
the results are undefined.
Is the same true for boost::threads? The documentation does not appears to specify this.
If it is undefined, then what would be a clean way for multiple threads to wait on one thread completing?
If it is undefined, then what would be a clean way for multiple threads to wait on one thread completing?
The clean way would be for that one thread to inform the others that it is complete. A packaged_task contains a future which can be waited on, which can help us here.
Here's one way of doing that. I have used std::thread and std::packaged_task, but you could use the boost equivalents just as well.
#include <thread>
#include <mutex>
#include <future>
#include <vector>
#include <iostream>
void emit(const char* msg) {
static std::mutex m;
std::lock_guard<std::mutex> l(m);
std::cout << msg << std::endl;
std::cout.flush();
}
int main()
{
using namespace std;
auto one_task = std::packaged_task<void()>([]{
emit("waiting...");
std::this_thread::sleep_for(std::chrono::microseconds(500));
emit("wait over!");
});
// note: convert future to a shared_future so we can pass it
// to two subordinate threads simultaneously
auto one_done = std::shared_future<void>(one_task.get_future());
auto one = std::thread(std::move(one_task));
std::vector<std::thread> many;
many.emplace_back([one_done] {
one_done.wait();
// do my thing here
emit("starting thread 1");
});
many.emplace_back([one_done] {
one_done.wait();
// do my thing here
emit("starting thread 2");
});
one.join();
for (auto& t : many) {
t.join();
}
cout << "Hello, World" << endl;
return 0;
}
expected output:
waiting...
wait over!
starting thread 2
starting thread 1
Hello, World
I ended up using a boost::condition_variable... roughly:
class thread_wrapper {
boost::mutex mutex;
boost::condition_variable thread_done_condition;
bool thread_done = false;
void the_func() {
// ...
// end of the thread
{
boost:unique_lock<boost::mutex> lock(mutex);
thread_done = true;
}
thread_done_condition.notify_all();
}
void wait_until_done() {
boost::unique_lock<boost::mutex> lock(mutex);
thread_done_condition.wait(lock, [this]{ return thread_done; });
}
}
Then multiple callers can safely call wait_until_done().
It strikes me now that something like the following would also have worked:
class thread_wrapper {
public:
thread_wrapper() : thread([this]() { this->the_func(); }) { }
void wait_until_done() {
boost::unique_lock<boost::mutex> lock(join_mutex);
thread.join();
}
private:
void the_func() {
// ...
}
boost::mutex join_mutex;
boost::thread thread;
}

Getting Native Handle from Within a Thread?

I am using VS2012 and I want to set thread priority from within a running thread. The goal is to initialize all threads with the highest priority state. To do this I want to get a HANDLE to the thread.
I am having some trouble accessing the pointer that corresponds to the thread object.
Is this possible?
From the calling main thread, the pointer is valid and from the C++11 thread it is set to CCCCCCCC. Predictably dereferencing some nonsense memory location causes a crash.
The code below is a simplified version showing the problem.
#include "stdafx.h"
#include <Windows.h>
#include <thread>
#include <mutex>
#include <condition_variable>
#include <iostream>
#include <atomic>
using namespace std;
class threadContainer
{
thread* mT;
condition_variable* con;
void lockMe()
{
mutex m;
unique_lock<std::mutex> lock(m);
con->wait(lock);//waits for host thread
cout << mT << endl;//CCCCCCCC
auto h = mT->native_handle();//causes a crash
con->wait(lock);//locks forever
}
public:
void run()
{
con = new condition_variable();
mT = new thread(&threadContainer::lockMe,*this);
cout << mT << endl; //00326420
con->notify_one();// Without this line everything locks as expected
mT->join();
}
};
int _tmain(int argc, _TCHAR* argv[])
{
threadContainer mContainer;
mContainer.run();
return 0;
}
#include <mutex>
#include <condition_variable>
#include <iostream>
#include <atomic>
#include <thread>
class threadContainer {
std::thread* mT;
std::mutex m;
void lockMe() {
// wait for mT to be assigned:
{
std::unique_lock<std::mutex> lock(m);
}
std::cout << "lockMe():" << mT << "\n";
auto h = mT->native_handle();//causes a crash
std::cout << "Done lockMe!\n";
}
public:
void run() {
// release lock only after mT assigned:
{
std::unique_lock<std::mutex> lock(m);
mT = new std::thread( [&](){ this->lockMe(); } );
}
std::cout << "run():" << mT << "\n"; //00326420
mT->join();
}
};
int main() {
threadContainer mContainer;
mContainer.run();
return 0;
}
Try that.
0xcccccccc means "variable not initialized". You have a threading race bug in your code. The thread starts running before the "mT" variable is assigned. You will need additional synchronization to block the thread until the assignment is completed so you can safely use mT. This will then also ensure that the new thread can see the updated value of mT, a memory barrier is required on a multi-core machine.
This is an example code with condition_variable and mutex.
class threadContainer
{
std::thread* mT;
std::mutex m;
std::condition_variable cv;
bool flag;
void lockMe() {
// 1. you must acquire lock of mutex.
unique_lock<std::mutex> lk(m);
// 2. and wait on `cv` for `flag==true`
cv.wait(lk, [&]{ return flag; });
cout << mT << endl;
auto h = mT->native_handle();
}
public:
void run()
{
flag = false;
mT = new std::thread( [&](){ this->lockMe(); } );
{
// 3. set `flag` and signal `cv`
lock_guard<decltype(m)> lk(m);
cout << mT << endl;
flag = true;
cv.notify_one();
}
mT->join();
}
};
If what you really want to do is "initialize all threads with the highest priority state", how about this simplified code?
Anyway, changing thread priority is platform dependent and out of C++ Standard library.
class threadContainer
{
std::thread thd;
void work() {
// (1) change thread priority itself
::SetThreadPriority(::GetCurrentThread(), THREAD_PRIORITY_HIGHEST);
// do something...
}
public:
void run()
{
thd = std::thread( [&](){ this->work(); } );
// (2) or change thread priority from outside
::SetThreadPriority(thd.native_handle(), THREAD_PRIORITY_HIGHEST);
thd.join();
}
};