I want to wait for two bool variables to be true in one thread. They are changed in different places. I can use boost in my project, but not C++11.
I did find Info on how to use mutexes and condition variables, but im not sure if its possible to wait for two mutexes.
This is some pseudocode of my program.
bool job1_dataready, job2_dataready;
//t1:
void job1()
{
//do stuff
job1_dataready = true;
}
//t2:
void job2()
{
//do stuff
job2_dataready= true;
}
main()
{
boost::thread t1(job1);
boost::thread t1(job2);
if(job1_dataready&& job2_dataready)
{
//do stuff with data from both jobs
}
}
from what I see, you don't need bool variables, use std::thread::join instead:
main() {
std::thread t1(job1);
std::thread t1(job2);
t1.join();
t2.join();
// do jobs after threads t1 and t2 finish working
}
you would block on the condition variable, check your boolean values when woken, and either go back to waiting or continue processing. Your threads will signal the condition variable after they have set the boolean flag. All with appropriate mutex locking of course. You can wait on an infinite number of conditions, just check when woken after blocking on the condition.
In simple situations like this, you wait on two mutexes simply by locking them in order. First you lock the mutex from thread 1, then the mutex from thread 2. If thread 2 would finish before thread 1, the main thread would simply not block when locking mutex 2.
However, note that this is an answer you your question, but not a solution to your problem. The reason is that you have a race condition with the mutex: the main thread might lock the mutex before the worker thread even starts. So, while Andrei R.s response (std::thread::join) isn't a direct answer, it is the correct solution.
If you plan to set your two bools just before the respective threads terminate, then Andrei R.'s solution of just joining the two threads is definitely the best way to go. However, if your threads actually continue working after the dataready points are reached, and are thus not terminating yet, you need a different approach. In that case, you could use two std::future/std::promise objects, which would look something like this:
std::promise<bool> job1_dataready, job2_dataready;
//t1:
void job1()
{
//do stuff
job1_dataready.set_value(true); // The value doesn't actually matter
//do more stuff
}
//t2:
void job2()
{
//do stuff
job2_dataready.set_value(true);
//do more stuff
}
main()
{
std::future<bool> job1_future = job1_dataready.get_future();
std::future<bool> job2_future = job2_dataready.get_future();
boost::thread t1(job1);
boost::thread t2(job2);
job1_future.wait();
job2_future.wait();
if (job1_future.get() && job2_future.get()) // True unless something was aborted
{
//do stuff with data from both jobs
}
}
Related
well, actually, I'm not asking the threads must "line up" to work, but I just want to notify multiple threads. so I'm not looking for barrier.
it's kind of like the condition_variable::notify_all(), but I don't want the threads wakeup one-by-one, which may cause starvation(also the potential problem in multiple semaphore post operation). it's kind of like:
std::atomic_flag flag{ATOMIC_FLAG_INIT};
void example() {
if (!flag.test_and_set()) {
// this is the thread to do the job, and notify others
do_something();
notify_others(); // this is what I'm looking for
flag.clear();
} else {
// this is the waiting thread
wait_till_notification();
do_some_other_thing();
}
}
void runner() {
std::vector<std::threads>;
for (int i=0; i<10; ++i) {
threads.emplace_back([]() {
while(1) {
example();
}
});
}
// ...
}
so how can I do this in c/c++ or maybe posix API?
sorry, I didn't make this question clear enough, I'd add some more explaination.
it's not thunder heard problem I'm talking about, and yes, it's the re-acquire-lock that bothers me, and I tried shared_mutex, there's still some problem.
let me split the threads to 2 parts, 1 as leader thread, which do the writing job, the others as worker threads, which do the reading job.
but actually they're all equal in programme, the leader thread is the thread that 1st got access to the job( you can take it as the shared buffer is underflowed for this thread). once the job is done, the other workers just need to be notified that them have the access.
if the mutex is used here, any thread would block the others.
to give an example: the main thread's job do_something() here is a read, and it block the main thread, thus the whole system is blocked.
unfortunatly, shared_mutex won't solve this problem:
void example() {
if (!flag.test_and_set()) {
// leader thread:
lk.lock();
do_something();
lk.unlock();
flag.clear();
} else {
// worker thread
lk.shared_lock();
do_some_other_thing();
lk.shared_unlock();
}
}
// outer loop
void looper() {
std::vector<std::threads>;
for (int i=0; i<10; ++i) {
threads.emplace_back([]() {
while(1) {
example();
}
});
}
}
in this code, if the leader job was done, and not much to do between this unlock and next lock (remember they're in a loop), it may get the lock again, leave the worker jobs not working, which is why I call it starve earlier.
and to explain the blocking in do_something(), I don't want this part of job takes all my CPU time, even if the leader's job is not ready (no data arrive for read)
and std::call_once may still not be the answer to this. because, as you can see, the workers must wait till the leader's job finished.
to summarize, this is actually a one-producer-multi-consumer problem.
but I want the consumers can do the job when the product is ready for them. and any can be the producer or consumer. if any but the 1st find the product has run out, the thread should be the producer, thus others are automatically consumer.
but unfortunately, I'm not sure if this idea would work or not
it's kind of like the condition_variable::notify_all(), but I don't want the threads wakeup one-by-one, which may cause starvation
In principle it's not waking up that is serialized, but re-acquiring the lock.
You can avoid that by using std::condition_variable_any with a std::shared_lock - so long as nobody ever gets an exclusive lock on the std::shared_mutex. Alternatively, you can provide your own Lockable type.
Note however that this won't magically allow you to concurrently run more threads than you have cores, or force the scheduler to start them all running in parallel. They'll just be marked as runnable and scheduled as normal - this only fixes the avoidable serialization in your own code.
It sounds like you are looking for call_once
#include <mutex>
void example()
{
static std::once_flag flag;
bool i_did_once = false;
std::call_once(flag, [&i_did_once]() mutable {
i_did_once = true;
do_something();
});
if(! i_did_once)
do_some_other_thing();
}
I don't see how your problem relates to starvation. Are you perhaps thinking about the thundering herd problem? This may arise if do_some_other_thing has a mutex but in that case you have to describe your problem in more detail.
I would like to write a class that wraps around std::thread and behaves like a std::thread but without actually allocating a thread every time I need to process something async. The reason is that I need to use multi threading in a context where I'm not allow to dynamically allocate and I also don't want to have the overhead of creating a std::thread.
Instead, I want a thread to run in a loop and wait until it can start processing. The client calls invoke which wakes up the thread. The Thread locks a mutex, does it's processing and falls asleep again. A function join behaves like std::thread::join by locking until the thread frees the lock (i.e. falls asleep again).
I think I got the class to run but because of a general lack of experience in multi threading, I would like to ask if anybody can spot race conditions or if the approach I used is considered "good style". For example, I'm not sure if temporary locking the mutex is a decent way to "join" the thread.
EDIT
I found another race condition: when calling join directly after invoke, there is no reason the thread already locked the mutex and thus locks the caller of join until the thread goes to sleep. To prevent this, I had to add a check for the invoke counter.
Header
#pragma once
#include <thread>
#include <atomic>
#include <mutex>
class PersistentThread
{
public:
PersistentThread();
~PersistentThread();
// set function to invoke
// locks if thread is currently processing _func
void set(const std::function<void()> &f);
// wakes the thread up to process _func and fall asleep again
// locks if thread is currently processing _func
void invoke();
// mimics std::thread::join
// locks until the thread is finished with it's loop
void join();
private:
// intern thread loop
void loop(bool *initialized);
private:
bool _shutdownRequested{ false };
std::mutex _mutex;
std::unique_ptr<std::thread> _thread;
std::condition_variable _cond;
std::function<void()> _func{ nullptr };
};
Source File
#include "PersistentThread.h"
PersistentThread::PersistentThread()
{
auto lock = std::unique_lock<std::mutex>(_mutex);
bool initialized = false;
_thread = std::make_unique<std::thread>(&PersistentThread::loop, this, &initialized);
// wait until _thread notifies, check bool initialized to prevent spurious wakeups
_cond.wait(lock, [&] {return initialized; });
}
PersistentThread::~PersistentThread()
{
{
std::lock_guard<std::mutex> lock(_mutex);
_func = nullptr;
_shutdownRequested = true;
// wake up and let join
_cond.notify_one();
}
// join thread,
if (_thread->joinable())
{
_thread->join();
}
}
void PersistentThread::set(const std::function<void()>& f)
{
std::lock_guard<std::mutex> lock(_mutex);
this->_func = f;
}
void PersistentThread::invoke()
{
std::lock_guard<std::mutex> lock(_mutex);
_cond.notify_one();
}
void PersistentThread::join()
{
bool joined = false;
while (!joined)
{
std::lock_guard<std::mutex> lock(_mutex);
joined = (_invokeCounter == 0);
}
}
void PersistentThread::loop(bool *initialized)
{
std::unique_lock<std::mutex> lock(_mutex);
*initialized = true;
_cond.notify_one();
while (true)
{
// wait until we get the mutex again
_cond.wait(lock, [this] {return _shutdownRequested || (this->_invokeCounter > 0); });
// shut down if requested
if (_shutdownRequested) return;
// process
if (_func) _func();
_invokeCounter--;
}
}
You are asking about potential race conditions, and I see at least one race condition in the shown code.
After constructing a PersistentThread, there is no guarantee that the new thread will acquire its initial lock in its loop() before the main execution thread returns from the constructor and enters invoke(). It is possible that the main execution thread enters invoke() immediately after the constructor is complete, ends up notifying nobody, since the internal execution thread hasn't locked the mutex yet. As such, this invoke() will not result in any processing taking place.
You need to synchronize the completion of the constructor with the execution thread's initial lock acquisition.
EDIT: your revision looks right; but I also spotted another race condition.
As documented in the description of wait(), wait() may wake up "spuriously". Just because wait() returned, doesn't mean that some other thread has entered invoke().
You need a counter, in addition to everything else, with invoke() incrementing the counter, and the execution thread executing its assigned duties only when the counter is greater than zero, decrementing it. This will guard against spurious wake-ups.
I would also have the execution thread check the counter before entering wait(), and enter wait() only if it is 0. Otherwise, it decrements the counter, executes its function, and loops back.
This should plug up all the potential race conditions in this area.
P.S. The spurious wake-up also applies to the initial notification, in your correction, that the execution thread has entered the loop. You'll need to do something similar for that situation, too.
I don't understand what you're trying to ask exactly. It's a nice style you used.
It would be much safer using bools and check the single routines because void returns nothing so you could be maybe stuck caused by bugs. Check everything you can since the thread runs under the hood. Make sure the calls are running correctly, if the process had really success. Also you could read some stuff about "Thread Pooling".
I am a bit stuck with the problem, so it is my cry for help.
I have a manager that pushes some events to a queue, which is proceeded in another thread.
I don't want this thread to be 'busy waiting' for events in the queue, because it may be empty all the time (as well as it may always be full).
Also I need m_bShutdownFlag to stop the thread when needed.
So I wanted to try a condition_variable for this case: if something was pushed to a queue, then the thread starts its work.
Simplified code:
class SomeManager {
public:
SomeManager::SomeManager()
: m_bShutdownFlag(false) {}
void SomeManager::Initialize() {
boost::recursive_mutex::scoped_lock lock(m_mtxThread);
boost::thread thread(&SomeManager::ThreadProc, this);
m_thread.swap(thread);
}
void SomeManager::Shutdown() {
boost::recursive_mutex::scoped_lock lock(m_mtxThread);
if (m_thread.get_id() != boost::thread::id()) {
boost::lock_guard<boost::mutex> lockEvents(m_mtxEvents);
m_bShutdownFlag = true;
m_condEvents.notify_one();
m_queue.clear();
}
}
void SomeManager::QueueEvent(const SomeEvent& event) {
boost::lock_guard<boost::mutex> lockEvents(m_mtxEvents);
m_queue.push_back(event);
m_condEvents.notify_one();
}
private:
void SomeManager::ThreadProc(SomeManager* pMgr) {
while (true) {
boost::unique_lock<boost::mutex> lockEvents(pMgr->m_mtxEvents);
while (!(pMgr->m_bShutdownFlag || pMgr->m_queue.empty()))
pMgr->m_condEvents.wait(lockEvents);
if (pMgr->m_bShutdownFlag)
break;
else
/* Thread-safe processing of all the events in m_queue */
}
}
boost::thread m_thread;
boost::recursive_mutex m_mtxThread;
bool m_bShutdownFlag;
boost::mutex m_mtxEvents;
boost::condition_variable m_condEvents;
SomeThreadSafeQueue m_queue;
}
But when I test it with two (or more) almost simultaneous calls to QueueEvent, it gets locked at the line boost::lock_guard<boost::mutex> lockEvents(m_mtxEvents); forever.
Seems like the first call doesn't ever release lockEvents, so all the rest just keep waiting for its freeing.
Please, help me to find out what am I doing wrong and how to fix this.
There's a few things to point out on your code:
You may wish to join your thread after calling shutdown, to ensure that your main thread doesn't finish before your other thread.
m_queue.clear(); on shutdown is done outside of your m_mtxEvents mutex lock, meaning it's not as thread safe as you think it is.
your 'thread safe processing' of the queue should be just taking an item off and then releasing the lock while you go off to process the event. You've not shown that explicitly, but failure to do so will result in the lock preventing items from being added.
The good news about a thread blocking like this, is that you can trivially break and inspect what the other threads are doing, and locate the one that is holding the lock. It might be that as per my comment #3 you're just taking a long time to process an event. On the other hand it may be that you've got a dead lock. In any case, what you need is to use your debugger to establish exactly what you've done wrong, since your sample doesn't have enough in it to demonstrate your problem.
inside ThreadProc, while(ture) loop, the lockEvents is not unlocked in any case. try put lock and wait inside a scope.
Consider the following code:
int main() {
bool done = false;
condition_variable con;
mutex m;
thread producer([&]() {
this_thread::sleep_for(chrono::seconds(10));
done = true;
//con.notify_one();
});
thread consumer([&]() {
/*unique_lock<mutex> lock(m);
while (!done) {
con.wait(lock);
}*/
while (!done);
cout << "now my turn..."<<endl;
});
producer.join();
consumer.join();
}
if I uncomment the code in the 2 threads, I will use the condition_variable. So the consumer thread will look like this:
thread consumer([&]() {
unique_lock<mutex> lock(m);
while (!done) {
con.wait(lock);
}
// while (!done); <-this is equivalent of the above
cout << "now my turn..."<<endl;
});
It seems that I can achieve the same thing with/without condition_variable.
So my question is: why do we need condition_variable if a notifying variable ('done' variable in this case) has been used already? What is the benefit of using it? Can I do something that a notifying variable cannot do?
When waiting on a condition variable the thread is blocked (i.e. not executing). When notified the thread is put in the ready state so the OS can schedule it.
This is more efficient than the thread "busy-waiting", which is polling a variable constantly to check that it can continue. In that case the thread is using up CPU cycles that could be used for actual work instead.
Also you need to use condition variables in order to correctly protect the critical section from being accessed by multiple threads at a time. You might have 3 consumers running but only one is allowed to work at a time (the others might be doing something else until then).
I've been using boost::mutex::scoped_lock in this manner:
void ClassName::FunctionName()
{
{
boost::mutex::scoped_lock scopedLock(mutex_);
//do stuff
waitBoolean=true;
}
while(waitBoolean == true ){
sleep(1);
}
//get on with the thread's activities
}
Basically it sets waitBoolean, and the other thread signals that it is done by setting waitBoolean to false;
This doesn't seem to work, however, because the other thread can't get a lock on mutex_ !!
I was assuming that by wrapping the scoped_lock in brackets I would be terminating its lock. This isn't the case? Reading online says that it only gives up the mutex when the destructor is called. Won't it be destroyed when it goes out of that local scope?
Signaling part of code:
while(running_){
boost::mutex::scoped_lock scopedLock(mutex_);
//Run some function that need to be done...
if(waitBoolean){
waitBoolean=false;
}
}
Thanks!
To synchronize two threads use a condition variable. That is the state of the art way to synchronize two threads the way you want :
Using boost, the waiting part is something like :
void BoostSynchronisationPoint::waitSynchronisation()
{
boost::unique_lock<boost::mutex> lock(_mutex);
_synchronisationSent = false;
while(!_synchronisationSent)
{
_condition.wait(lock); // unlock and wait
}
}
The notify part is something like :
void BoostSynchronisationPoint::sendSynchronisation()
{
{
boost::lock_guard<boost::mutex> lock(_mutex);
_synchronisationSent = true;
}
_condition.notify_all();
}
The business with _synchronisationSent is to avoid spurrious wakeups : see wikipedia
The scoped_lock should indeed be released at the end of the scope. However you don't lock the waitBoolean when you're looping on it, suggesting you don't protect it properly other places as well - e.g. where it's set to false, and you'll end up with nasty race conditions.
I'd say you should use a boost::condition_variable to do this sort of things, instead of sleep + thread-unsafe checking.
Also I would suggest to mark as volatile that waitBoolean, however you have to use a condition or even better a barrier.