I need to start a loop from an event and then stop it from another event.
My idea was to call the function startDequeuing() when I press the button so that a thread with the loop start and then to terminate this loop putting "dequeuing" variable at false from the function stopDequeuing().
It's the first time I use thread, the program locks when I start the loop I think because the variable 'dequeuing' is locked and inaccessible from outside the thread, am I correct?
How can I solve this problem??
Here there is some code:
void CameraManager::startDequeuing(){
dequeuing = true;
std::thread dequeueThread(&CameraManager::dequeueLoop, this);
dequeueThread.join();
}
void CameraManager::stopDequeuing(){
dequeuing = false;
}
void *CameraManager::dequeueLoop(){
while(dequeuing){
highSpeedCamera->dequeue();
highSpeedCamera->enqueue();
}
}
The whole point of using threads is to get more than one function running in parallel. Here:
std::thread dequeueThread(&CameraManager::dequeueLoop, this);
dequeueThread.join();
You start a second thread and put the first thread to sleep, waiting for the spawned thread to return. So you still have just one thread running. If you have so kind of GUI event loop, you might lock for a possibility to add a callback that will get called, when ever that event loop is empty. This might enable you to do what you want without using threads at all.
A solution might look like this:
void CameraManager::startDequeuing(){
dequeuing = true;
dequeueThread = std::thread(&CameraManager::dequeueLoop, this);
}
void CameraManager::stopDequeuing(){
{
std::lock_guard<std::mutex> lock( mutex );
dequeuing = false;
}
dequeueThread.join();
}
bool CameraManager::keepOnDequeuing()
{
std::lock_guard<std::mutex> lock( mutex );
return dequeuing;
}
void *CameraManager::dequeueLoop(){
while( keepOnDequeuing() ){
highSpeedCamera->dequeue();
highSpeedCamera->enqueue();
}
}
Your program deadlocks because join() will block until your thread function completes; And it will never complete at that point because it's effectively executing while(true).
You want dequeueThread to be a member of your class. Why would you want it to survive only for the scope of startDequeuing?
Define dequeing as an atomic bool:
#include <atomic>
std::atomic_bool dequeing = false;
It's much faster than using a mutex and gets you the same synchronization.
Related
We have implemented TaskRunner whose functions will be called by different threads to start, stop and post tasks. TaskRunner will internally create a thread and if the queue is not empty, it will pop the task from queue and executes it. Start() will check if the thread is running. If not creates a new thread. Stop() will join the thread. The code is as below.
bool TaskRunnerImpl::PostTask(Task* task) {
tasks_queue_.push_back(task);
return true;
}
void TaskRunnerImpl::Start() {
std::lock_guard<std::mutex> lock(is_running_mutex_);
if(is_running_) {
return;
}
is_running_ = true;
runner_thread_ = std::thread(&TaskRunnerImpl::Run, this);
}
void TaskRunnerImpl::Run() {
while(is_running_) {
if(tasks_queue_.empty()) {
continue;
}
Task* task_to_run = tasks_queue_.front();
task_to_run->Run();
tasks_queue_.pop_front();
delete task_to_run;
}
}
void TaskRunnerImpl::Stop() {
std::lock_guard<std::mutex> lock(is_running_mutex_);
is_running_ = false;
if(runner_thread_.joinable()) {
runner_thread_.join();
}
}
We want to use conditional variables now otherwise the thread will be continuously checking whether the task queue is empty or not. We implemented as below.
Thread function (Run()) will wait on condition variable.
PostTask() will signal if some one posts a task.
Stop() will signal if some one calls stop.
Code is as below.
bool TaskRunnerImpl::PostTask(Task* task) {
std::lock_guard<std::mutex> taskGuard(m_task_mutex);
tasks_queue_.push_back(task);
m_task_cond_var.notify_one();
return true;
}
void TaskRunnerImpl::Start() {
std::lock_guard<std::mutex> lock(is_running_mutex_);
if(is_running_) {
return;
}
is_running_ = true;
runner_thread_ = std::thread(&TaskRunnerImpl::Run, this);
}
void TaskRunnerImpl::Run() {
while(is_running_) {
Task* task_to_run = nullptr;
{
std::unique_lock<std::mutex> mlock(m_task_mutex);
m_task_cond_var.wait(mlock, [this]() {
return !(is_running_ && tasks_queue_.empty());
});
if(!is_running_) {
return;
}
if(!tasks_queue_.empty()) {
task_to_run = tasks_queue_.front();
task_to_run->Run();
tasks_queue_.pop_front();
}
}
if(task_to_run)
delete task_to_run;
}
}
void TaskRunnerImpl::Stop() {
std::lock_guard<std::mutex> lock(is_running_mutex_);
is_running_ = false;
m_task_cond_var.notify_one();
if(runner_thread_.joinable()) {
runner_thread_.join();
}
}
I have couple of questions as below. Can some one please help me to understand these.
Condition variable m_task_cond_var is linked with mutex m_task_mutex. But Stop() already locks mutex is_running_mutex to gaurd 'is_running_'. Do I need to lock m_task_mutex before signaling? Here I am not convinced why to lock m_task_mutex as we are not protecting any thing related to task queue.
In Thread function(Run()), we are reading is_running_ without locking is_running_mutex. Is this correct?
Do I need to lock m_task_mutex before signaling [In Stop]?
When the predicate being tested in condition_variable::wait method depends on something happening in the signaling thread (which is almost always), then you should obtain the mutex before signaling. Consider the following possibility if you are not holding the m_task_mutex:
The watcher thread (TaskRunnerImpl::Run) wakes up (via spurious wakeup or being notified from elsewhere) and obtains the mutex.
The watcher thread checks its predicate and sees that it is false.
The signaler thread (TaskRunnerImpl::Stop) changes the predicate to return true (by setting is_running_ = false;).
The signaler thread signals the condition variable.
The watcher thread waits to be signaled (bad)
the signal has already come and gone
the predicate was false, so the watcher begins waiting, possibly indefinitely.
The worst that can happen if you are holding the mutex when you signal is that, the blocked thread (TaskRunnerImpl::Run) wakes up and is immediately blocked when trying to obtain the mutex. This can have some performance implications.
In [TaskRunnerImpl::Run] , we are reading is_running_ without locking is_running_mutex. Is this correct?
In general no. Even if it's of type bool. Because a boolean is typically implemented as a single byte, it's possible that one thread is writing to the byte while you are reading, resulting in a partial read. In practice, however, it's safe. That said, you should obtain the mutex before you read (and then release immediately afterwards).
In fact, it may be preferable to use std::atomic<bool> instead of a bool + mutex combination (or std::atomic_flag if you want to get fancy) which will have the same effect, but be easier to work with.
Do I need to lock m_task_mutex before signaling [In Stop]?
Yes you do. You must change condition under the same mutex and send signal either after the mutex is locked or unlocked after the change. If you do not use the same mutex, or send signal before that mutex is locked you create race condition that std::condition_variable is created to solve.
Logic is this:
Watching thread locks mutex and checks watched condition. If it did not happen it goes to sleep and unlocks the mutex atomically. So signaling thread lock the mutex, change condition and signal. If signalling thread does that before watching one locks the mutex, then watchiong one would see condition happen and would not go to sleep. If it locks before, it would go to sleep and woken when signalling thread raise the signal.
Note: you can signal condition variable before or after mutex is unlocked, both cases is correct but may affect performance. But it is incorrect to signal before locking the mutex.
Condition variable m_task_cond_var is linked with mutex m_task_mutex. But Stop() already locks mutex is_running_mutex to gaurd 'is_running_'. Do I need to lock m_task_mutex before signaling? Here I am not convinced why to lock m_task_mutex as we are not protecting any thing related to task queue.
You overcomlicated your code and made things worse. You should use only one mutex in this case and it would work as intended.
In Thread function(Run()), we are reading is_running_ without locking is_running_mutex. Is this correct?
On x86 hardware it may "work", but from language point of view this is UB.
I have used below structure to create a threadpool, now the question is how do I let all the preallocate threads end properly?
std::vector<pthread_t> preallocatedThreadsPool; // threadpool
std::queue<int> tcpQueue; // a queue to hold my task
pthread_mutex_t mutex = PTHREAD_MUTEX_INITIALIZER;
pthread_cond_t condition_var = PTHREAD_COND_INITIALIZER;
void* threadFunctionUsedByThreadsPool(void *arg);
main () {
preallocatedThreadsPool.resize(preallocatThreadsNumber);
for(pthread_t i : preallocatedThreadsPool) {
pthread_create(&i, NULL, threadFunctionUsedByThreadsPool, NULL);
}
pthread_mutex_lock(&mutex); // one thread mess with the queue at one time
tcpQueue.push(task);
pthread_cond_signal(&condition_var);
pthread_mutex_unlock(&mutex);
}
void* threadFunctionUsedByThreadsPool(void *arg) {
while (true) {
pthread_mutex_lock(&mutex);
if (tcpQueue.empty()) { // can't get work from the queue then just wait
pthread_cond_wait(&condition_var, &mutex); // wait for the signal from other thread to deal with client otherwise sleep
task = tcpQueue.front();
tcpQueue.pop();
}
pthread_mutex_unlock(&mutex);
if (task) {
// do task
}
}
return NULL;
}
I have been searching for days for this problem still can not find a decent solution, the closest one I have tried is , when the program wants to quit, push a special item into the queue, then inside threadFunctionUsedByThreadsPool, when detecting such item, I will call pthread_join, however, when I using gdb tool to debug it , those pre-allocated threads are still there, anyone could help, better with some code, for example, how do I modify the threadFunctionUsedByThreadsPool, so that I can quit all the pre-allocated threads properly?
Thanks so much!!!
TLDR: You just need a thread-safe variable that all threads can check for an exit condition in between work items. Use pthread_join to wait for a thread to exit.
First, let's get the while loop in your thread function correct with respect to condition variables.
Instead of this:
pthread_cond_wait(&condition_var, &mutex); // wait for the signal from other thread to deal with client otherwise sleep
task = tcpQueue.front();
tcpQueue.pop();
Check the state of the queue before before and after waking up on the condition variable. Spurious wake up is a real thing and there's no guarantee another thread didn't wake up and grab the last work item. You definitely don't want to be popping from an empty queue.
Better:
while (tcpQueue.empty()) {
pthread_cond_wait(&condition_var, &mutex); // wait for the signal from other thread to deal with client otherwise sleep
}
task = tcpQueue.front();
tcpQueue.pop();
With that addressed, we can introduce a new global bool that represents the stop condition:
bool stopCondition = false;
Whenever we want to tell all the threads in the pool to stop, we can set stopCondition to true and signal the condition var to alert all threads of a state change. Reading or writing stopCondition should be done under a lock. (I suppose you could also use std::atomic<bool>)
Putting it all together, your thread function becomes this:
void* threadFunctionUsedByThreadsPool(void* arg) {
pthread_mutex_lock(&mutex);
while (!stopCondition) {
// wait for a task to be queued
while (tcpQueue.empty() && !stopCondition) {
pthread_cond_wait(&condition_var, &mutex); // wait for the signal from other thread to deal with client otherwise sleep
}
if (stopCondition == false) {
task = tcpQueue.front();
tcpQueue.pop();
// exit lock while operating on a task
pthread_mutex_unlock(&mutex);
if (task) {
// do task
}
// re-acquire the lock
pthread_mutex_lock(&mutex);
}
}
// release the lock before exiting the function
pthread_mutex_unlock(&mutex);
return NULL;
}
And then a helper function to signal all the threads to exit and also waits for each thread to stop. notice that we're using pthread_cond_broadcast to notify all threads to wake up from their condition variable wait instead of pthread_cond_signal which only wakes up one thread.
void stopThreadPool()
{
// signal all threads to exit after they finish their current work item
pthread_mutex_lock(&mutex);
stopCondition = true;
pthread_cond_broadcast(&condition_var); // notify all threads
pthread_mutex_unlock(&mutex);
// wait for all threads to exit
for (auto& t : preAllocatedThreadsPool) {
pthread_join(t, nullptr);
}
preAllocatedThreadsPool.clear();
}
One last bug that I just caught - your main isn't property initializing your preAllocatedThreadsPool vector like you think it is. You're making a copy of the pthread_t, instead of using the handle actually in the vector.
Instead of this:
for(pthread_t i : preallocatedThreadsPool) {
Your loop needs to enumerate by reference:
Better:
for(pthread_t &i : preallocatedThreadsPool) {
Send a task that instructs the pool thread to requeue the task and then terminate. The poison-task will then run through all the threads in the pool, killing them all off. I have used a null as the poison, (ie, an illegal task) - it does not need to be destructed when it has killed the last thread. You may wish to purge the task queue before sending the null/whatever. If you use a null, you only need a null check in the threads, just after dequeing the task.
You need very little extra code, you don't need to know how many threads in the pool and it will work:)
Suppose I have two threads and one shared c++ 11 condition variable.
What whould happen if thread1 call notify and after that thread2 call wait?
Will thread2 block forever or it will continue it's work due to call of notify by thread1?
Edit:
enum bcLockOperation
{
bcLockOperation_Light = -1,
bcLockOperation_Medium = 50,
bcLockOperation_Heavy = 1
}
class BC_COREDLL_EXP bcCustomMutex
{
private:
bcCustomMutex(const bcCustomMutex&);
bcCustomMutex& operator=(const bcCustomMutex&);
protected:
bcAtomic<int> mFlag;
bcMutex mMutex;
bcConditionVariable mCond;
public:
bcCustomMutex() { bcAtomicOperation::bcAtomicInit(mFlag, 0); };
~bcCustomMutex() {};
void lock(bcLockOperation pLockOperation = bcLockOperation_Medium)
{
bcINT32 lNewLoopCount = static_cast<bcINT32>(pLockOperation);
bcINT32 lLoopCounter = 0;
bcINT32 lExpected = 0;
bcINT32 lLoopCount = bcAtomicOperation::bcAtomicLoad(mFlag, bcMemoryOrder_Relaxed);
while (true)
{
while(bcAtomicOperation::bcAtomicLoad(mFlag, bcMemoryOrder_Relaxed) != 0 && lLoopCounter != lLoopCount)
++lLoopCounter;
bcAtomicOperation::bcAtomicCompareExchangeStrong(
mFlag,
&lExpected,
lNewLoopCount,
bcMemoryOrder_Acquire,
bcMemoryOrder_Relaxed);
if(lExpected == 0)
{
return;
}
else if(lLoopCounter == lLoopCount)
{
bcLockGuard<bcMutex> lGuard(mMutex);
mCond.wait(mMutex);
}
else
{
continue;
}
}
void UnLock()
{
bcAtomicOperation::bcAtomicStore(mFlag, 0, bcMemoryOrder_Relaxed);
bcUniqueLock<bcMutex> lGuard(mMutex);
mCond.notifyOne();
}
bcBOOL TryLock()
{
};
};
I want to write a custom mutex such that each thread can provide an argument that represents the complexity of operations that the current thread wants to execute.
If the complexity of the operation is low other threads will be in a loop like a spin lock but if the complexity of the operation is medium each thread will iterate 50 times and then will sleep by condition variable and if operation is very complex other threads will go to sleep directly.
now assume thread1 locks this mutex and thread2 goes for waiting due to its loopCounter reaching its end and right before locking the condition variable's mutex, thread1 calls notify on the condition variable. Now thread2 will sleep until another thread locks the custom mutex and then calls unlock on it.
I am new to multithreading and I want to learn. I know that my class may contain errors or may be completely wrong, but is there any way to correct this problem or a good algorithm to write such a mutex.
Thread2 will block until someone calls notify. Calls to notify release threads that are waiting at the time of the call. If there are no threads waiting, they do nothing. They aren't saved.
Usually both the code that decides to wait and the code that decides to notify share the same mutex. So thread2 will never "miss" the notify from thread1.
Here's the classic lock-based concurrent queue example:
void push(int x)
{
lock_guard<mutex> guard{queue_mutex};
thequeue.push(x);
not_empty_condition.notify_one();
}
int pop()
{
unique_lock<mutex> guard{queue_mutex};
not_empty_condition.wait(guard, []{ return !thequeue.empty(); } );
int x = thequeue.front();
thequeue.pop();
return x;
}
Assume thread1 and thread2 are running push() and pop() respectively. Only one of them will be in the critical section at a time.
If thread2 has the lock, either it never waits because the queue is not empty (so "losing" a notify is harmless), or it sits there waiting for a notify (which won't be lost).
If thread1 got the lock, it will put an element in the queue; if thread2 was waiting, it will get notified properly; if thread2 was still waiting for the mutex, it will never wait, as there is at least one element on the queue, so losing a notify is harmless.
In this manner, a notify is only lost if it was not needed in the first place.
Now, if you have a different usage for condition variables in mind, where "losing" a notification has any consequence, I believe you either have a race condition, or are using the wrong tool altogether.
I would like to write a class that wraps around std::thread and behaves like a std::thread but without actually allocating a thread every time I need to process something async. The reason is that I need to use multi threading in a context where I'm not allow to dynamically allocate and I also don't want to have the overhead of creating a std::thread.
Instead, I want a thread to run in a loop and wait until it can start processing. The client calls invoke which wakes up the thread. The Thread locks a mutex, does it's processing and falls asleep again. A function join behaves like std::thread::join by locking until the thread frees the lock (i.e. falls asleep again).
I think I got the class to run but because of a general lack of experience in multi threading, I would like to ask if anybody can spot race conditions or if the approach I used is considered "good style". For example, I'm not sure if temporary locking the mutex is a decent way to "join" the thread.
EDIT
I found another race condition: when calling join directly after invoke, there is no reason the thread already locked the mutex and thus locks the caller of join until the thread goes to sleep. To prevent this, I had to add a check for the invoke counter.
Header
#pragma once
#include <thread>
#include <atomic>
#include <mutex>
class PersistentThread
{
public:
PersistentThread();
~PersistentThread();
// set function to invoke
// locks if thread is currently processing _func
void set(const std::function<void()> &f);
// wakes the thread up to process _func and fall asleep again
// locks if thread is currently processing _func
void invoke();
// mimics std::thread::join
// locks until the thread is finished with it's loop
void join();
private:
// intern thread loop
void loop(bool *initialized);
private:
bool _shutdownRequested{ false };
std::mutex _mutex;
std::unique_ptr<std::thread> _thread;
std::condition_variable _cond;
std::function<void()> _func{ nullptr };
};
Source File
#include "PersistentThread.h"
PersistentThread::PersistentThread()
{
auto lock = std::unique_lock<std::mutex>(_mutex);
bool initialized = false;
_thread = std::make_unique<std::thread>(&PersistentThread::loop, this, &initialized);
// wait until _thread notifies, check bool initialized to prevent spurious wakeups
_cond.wait(lock, [&] {return initialized; });
}
PersistentThread::~PersistentThread()
{
{
std::lock_guard<std::mutex> lock(_mutex);
_func = nullptr;
_shutdownRequested = true;
// wake up and let join
_cond.notify_one();
}
// join thread,
if (_thread->joinable())
{
_thread->join();
}
}
void PersistentThread::set(const std::function<void()>& f)
{
std::lock_guard<std::mutex> lock(_mutex);
this->_func = f;
}
void PersistentThread::invoke()
{
std::lock_guard<std::mutex> lock(_mutex);
_cond.notify_one();
}
void PersistentThread::join()
{
bool joined = false;
while (!joined)
{
std::lock_guard<std::mutex> lock(_mutex);
joined = (_invokeCounter == 0);
}
}
void PersistentThread::loop(bool *initialized)
{
std::unique_lock<std::mutex> lock(_mutex);
*initialized = true;
_cond.notify_one();
while (true)
{
// wait until we get the mutex again
_cond.wait(lock, [this] {return _shutdownRequested || (this->_invokeCounter > 0); });
// shut down if requested
if (_shutdownRequested) return;
// process
if (_func) _func();
_invokeCounter--;
}
}
You are asking about potential race conditions, and I see at least one race condition in the shown code.
After constructing a PersistentThread, there is no guarantee that the new thread will acquire its initial lock in its loop() before the main execution thread returns from the constructor and enters invoke(). It is possible that the main execution thread enters invoke() immediately after the constructor is complete, ends up notifying nobody, since the internal execution thread hasn't locked the mutex yet. As such, this invoke() will not result in any processing taking place.
You need to synchronize the completion of the constructor with the execution thread's initial lock acquisition.
EDIT: your revision looks right; but I also spotted another race condition.
As documented in the description of wait(), wait() may wake up "spuriously". Just because wait() returned, doesn't mean that some other thread has entered invoke().
You need a counter, in addition to everything else, with invoke() incrementing the counter, and the execution thread executing its assigned duties only when the counter is greater than zero, decrementing it. This will guard against spurious wake-ups.
I would also have the execution thread check the counter before entering wait(), and enter wait() only if it is 0. Otherwise, it decrements the counter, executes its function, and loops back.
This should plug up all the potential race conditions in this area.
P.S. The spurious wake-up also applies to the initial notification, in your correction, that the execution thread has entered the loop. You'll need to do something similar for that situation, too.
I don't understand what you're trying to ask exactly. It's a nice style you used.
It would be much safer using bools and check the single routines because void returns nothing so you could be maybe stuck caused by bugs. Check everything you can since the thread runs under the hood. Make sure the calls are running correctly, if the process had really success. Also you could read some stuff about "Thread Pooling".
Please see the following code:
std::mutex mutex;
std::condition_variable cv;
std::atomic<bool> terminate;
// Worker thread routine
void work() {
while( !terminate ) {
{
std::unique_lock<std::mutex> lg{ mutex };
cv.wait(lg);
// Do something
}
// Do something
}
}
// This function is called from the main thread
void terminate_worker() {
terminate = true;
cv.notify_all();
worker_thread.join();
}
Is the following scenario can happen?
Worker thread is waiting for signals.
The main thread called terminate_worker();
The main thread set the atomic variable terminate to true, and then signaled to the worker thread.
Worker thread now wakes up, do its job and load from terminate. At this step, the change to terminate made by the main thread is not yet seen, so the worker thread decides to wait for another signal.
Now deadlock occurs...
I wonder this is ever possible. As I understood, std::atomic only guarantees no race condition, but memory order is a different thing. Questions:
Is this possible?
If this is not possible, is this possible if terminate is not an atomic variable but is simply bool? Or atomicity has nothing to do with this?
If this is possible, what should I do?
Thank you.
I don't believe, what you describe is possible, as cv.notify_all() afaik (please correct me if I'm wrong) synchronizes with wait(), so when the worker thread awakes, it will see the change to terminate.
However:
A deadlock can happen the following way:
Worker thread (WT) determines that the terminate flag is still false.
The main thread (MT) sets the terminate flag and calls cv.notify_all().
As no one is curently waiting for the condition variable that notification gets "lost/ignored".
MT calls join and blocks.
WT goes to sleep ( cv.wait()) and blocks too.
Solution:
While you don't have to hold a lock while you call cv.notify, you
have to hold a lock, while you are modifying terminate (even if it is an atomic)
have to make sure, that the check for the condition and the actual call to wait happen while you are holding the same lock.
This is why there is a form of wait that performs this check just before it sends the thread to sleep.
A corrected code (with minimal changes) could look like this:
// Worker thread routine
void work() {
while( !terminate ) {
{
std::unique_lock<std::mutex> lg{ mutex };
if (!terminate) {
cv.wait(lg);
}
// Do something
}
// Do something
}
}
// This function is called from the main thread
void terminate_worker() {
{
std::lock_guard<std::mutex> lg(mutex);
terminate = true;
}
cv.notify_all();
worker_thread.join();
}