I'm posting several jobs to a threadpool and then waiting for it to finish. I'm wondering if I've missed something here, since occasionally my worker threads seem to freeze.
My main thread start the workers like this:
numJobsPosted = 0;
for(auto entry : list)
{
numJobsPosted++;
threadPool->post(std::bind(&Controller::workerFunc, this, entry));
}
std::unique_lock<std::mutex> lock(m_workerLock);
while(numJobsPosted > 0)
{
m_workerCondition.wait(lock);
}
Now my workerFunc looks something like this:
void Controller::workerFunc(Entry entry)
{
// do some work with entry
// notify finished
numJobsPosted--;
if(numJobsPosted <= 0)
{
// does the look need to be around the numJobsPosted-- ?
std::unique_lock<std::mutex> locker(m_workerLock);
m_workerCondition.notify_one();
}
}
Is the above code safe, or do I need to put the lock around the decrement operator?
This may depend on details of your thread pool's inner logic or setup (e.g. if you have a single thread, so jobs are actually run sequentially), but assuming that numJobsPosted is an int or similar built-in type, your code isn't thread-safe.
This line in workerFunc:
numJobsPosted--;
could very well be the subject of a race condition if it gets executed by several jobs concurrently.
Also, I'm not sure what your threadpool's post function does precisely, but if it dispatches the worker function to a thread right away and some of the worker functions can return immediately, you have another possible race condition between this line in your main thread code:
numJobsPosted++;
and this line in workerFunc:
numJobsPosted--;
To make it safe, you can for instance make numJobsPosted atomic, e.g. declare it like this (in C++11):
#include <atomic>
std::atomic_int numJobsPosted;
Making your workerFunc something like this:
void Controller::workerFunc(Entry entry)
{
// do some work with entry
// notify finished
{
std::unique_lock<std::mutex> locker(m_workerLock);
numJobsPosted--;
if(numJobsPosted <= 0)
{
m_workerCondition.notify_one();
}
}
}
may solve the first race condition case, but not the second.
(Also, I don't really understand the logic around the manipulation and testing you're doing on numJobsPosted, but I think that's beside the point of your question)
Related
well, actually, I'm not asking the threads must "line up" to work, but I just want to notify multiple threads. so I'm not looking for barrier.
it's kind of like the condition_variable::notify_all(), but I don't want the threads wakeup one-by-one, which may cause starvation(also the potential problem in multiple semaphore post operation). it's kind of like:
std::atomic_flag flag{ATOMIC_FLAG_INIT};
void example() {
if (!flag.test_and_set()) {
// this is the thread to do the job, and notify others
do_something();
notify_others(); // this is what I'm looking for
flag.clear();
} else {
// this is the waiting thread
wait_till_notification();
do_some_other_thing();
}
}
void runner() {
std::vector<std::threads>;
for (int i=0; i<10; ++i) {
threads.emplace_back([]() {
while(1) {
example();
}
});
}
// ...
}
so how can I do this in c/c++ or maybe posix API?
sorry, I didn't make this question clear enough, I'd add some more explaination.
it's not thunder heard problem I'm talking about, and yes, it's the re-acquire-lock that bothers me, and I tried shared_mutex, there's still some problem.
let me split the threads to 2 parts, 1 as leader thread, which do the writing job, the others as worker threads, which do the reading job.
but actually they're all equal in programme, the leader thread is the thread that 1st got access to the job( you can take it as the shared buffer is underflowed for this thread). once the job is done, the other workers just need to be notified that them have the access.
if the mutex is used here, any thread would block the others.
to give an example: the main thread's job do_something() here is a read, and it block the main thread, thus the whole system is blocked.
unfortunatly, shared_mutex won't solve this problem:
void example() {
if (!flag.test_and_set()) {
// leader thread:
lk.lock();
do_something();
lk.unlock();
flag.clear();
} else {
// worker thread
lk.shared_lock();
do_some_other_thing();
lk.shared_unlock();
}
}
// outer loop
void looper() {
std::vector<std::threads>;
for (int i=0; i<10; ++i) {
threads.emplace_back([]() {
while(1) {
example();
}
});
}
}
in this code, if the leader job was done, and not much to do between this unlock and next lock (remember they're in a loop), it may get the lock again, leave the worker jobs not working, which is why I call it starve earlier.
and to explain the blocking in do_something(), I don't want this part of job takes all my CPU time, even if the leader's job is not ready (no data arrive for read)
and std::call_once may still not be the answer to this. because, as you can see, the workers must wait till the leader's job finished.
to summarize, this is actually a one-producer-multi-consumer problem.
but I want the consumers can do the job when the product is ready for them. and any can be the producer or consumer. if any but the 1st find the product has run out, the thread should be the producer, thus others are automatically consumer.
but unfortunately, I'm not sure if this idea would work or not
it's kind of like the condition_variable::notify_all(), but I don't want the threads wakeup one-by-one, which may cause starvation
In principle it's not waking up that is serialized, but re-acquiring the lock.
You can avoid that by using std::condition_variable_any with a std::shared_lock - so long as nobody ever gets an exclusive lock on the std::shared_mutex. Alternatively, you can provide your own Lockable type.
Note however that this won't magically allow you to concurrently run more threads than you have cores, or force the scheduler to start them all running in parallel. They'll just be marked as runnable and scheduled as normal - this only fixes the avoidable serialization in your own code.
It sounds like you are looking for call_once
#include <mutex>
void example()
{
static std::once_flag flag;
bool i_did_once = false;
std::call_once(flag, [&i_did_once]() mutable {
i_did_once = true;
do_something();
});
if(! i_did_once)
do_some_other_thing();
}
I don't see how your problem relates to starvation. Are you perhaps thinking about the thundering herd problem? This may arise if do_some_other_thing has a mutex but in that case you have to describe your problem in more detail.
I have a class with some methods that should be thread safe, i.e. multiple threads should be able operate on the class object state. One of the methods spawns a new thread that, every 10 seconds, updates a field. Because this thread can be long-running, I'd like to be able to abort it properly.
I have implemented a solution that uses std::condition_variable.wait_for() to wait for an abortion signal inside the thread, but am not particularly sure if my solution is either optimal or correct at all.
class A
{
unsigned int value; // A value that will be updated every 10 s in another thread
bool is_being_updated; // true while value is being updated in another thread
std::thread t;
bool aborted; // true = thread should abort
mutable std::mutex m1;
mutable std::mutex m2;
std::condition_variable cv;
public:
A();
~A();
void begin_update(); // Creates a thread that periodically updates value
void abort(); // Aborts the updating thread
unsigned int get_value() const;
void set_value(unsigned int);
};
This is how I implemented the methods:
A::A() : value(0), is_being_updated(false), aborted(false) { }
A::~A()
{
// Not sure if this is thread safe?
if(t.joinable()) t.join();
}
// Updates this->value every 10 seconds
void A::begin_update()
{
std::lock_guard<std::mutex> lck(m1);
if (is_being_updated) return; // Don't allow begin_update() while updating
is_being_updated = true;
if (aborted) aborted = false;
// Create a thread that will update value periodically
t = std::thread([this] {
std::unique_lock<std::mutex> update_lock(m2);
for(int i=0; i < 10; i++)
{
cv.wait_for(update_lock, std::chrono::seconds(10), [this]{ return aborted; });
if (!aborted)
{
std::lock_guard<std::mutex> lck(m1);
this->value++; // Update value
}
else
{
break; // Break on thread abort
}
}
// Locking here would cause indefinite blocking ...
// std::lock_guard<std::mutex> lck(m1);
if(is_being_updated) is_being_updated = false;
});
}
// Aborts the thread created in begin_update()
void A::abort()
{
std::lock_guard<std::mutex> lck(m1);
is_being_updated = false;
this->value = 0; // Reset value
{
std::lock_guard<std::mutex> update_lock(m2);
aborted = true;
}
cv.notify_one(); // Signal abort ...
if(t.joinable()) t.join(); // Wait for the thread to finish
}
unsigned int A::get_value() const
{
std::lock_guard<std::mutex> lck(m1);
return this->value;
}
void A::set_value(unsigned int v)
{
std::lock_guard<std::mutex> lck(m1);
if (is_being_updated) return; // Cannot set value while thread is updating it
this->value = v;
}
This seems to work fine, but I'm uncertain about it being correct. My concerns are the following:
Is my destructor safe? Suppose that the updating thread has not been aborted and is still doing its job while A object goes out of scope. A switch to a different thread now happens while dtor's t.join() still hasn't finished, and the switched-to thread calls begin_update() on the same object. Is something like this possible? Should I introduce e.g. an extra is_being_destructed flag that I would set to true inside a destructor and that all other methods should check for being false before they can proceed? Or can no such undesired scenario happen?
Inside the thread, at the end, I'm setting is_being_updated = false without a lock, despite the variable being shared state. This can mean that other threads won't see its correct value, e.g. even after the thread is done, some other thread may still see the value as is_being_updated == true instead of false. I cannot lock the mutex, however, because abort() may have already locked it, meaning that the call will block indefinitely. I'm not sure about the best way to solve this, other than perhaps making is_being_updated atomic. Would that work?
I've read about spurious wakeups, but am not sure I the code should do anything extra to handle them. As far as I understand, the answer is no, and no problems are to be expected in this regard.
Is my thinking here correct? Did I miss anything else that I should have in mind?
This stuff is always hard to check, so don't be afraid to question me if you think I misunderstand.
Short answer, no, it's not thread safe.
As long as the thread that has scope of A is the one calling abort (and doesn't forget to call abort), you won't experience a race condition, as A::abort() will block until the thread is joined. Under these assumptions, the join in your destructor is pointless.
If abort is called by the a thread that doesn't own A, then it's definitely possible for the thread to be join-ed twice, which is bad. Using .joinable() to decide to join a thread or not is a big red flag.
Please remove one of your if(t.joinable()) t.join(); (I'm leaning towards the one in the destructor) and change the other to just t.join().
As you said, you can make is_being_updated atomic. That's a great solution.
Here's another solution. You can signal without holding the lock. (It's actually better form in general, as it helps reduce lock contention, since the first thing the woken thread needs to do is reacquire its mutex.)
void A::abort()
{
{
std::lock(m1, m2); // deadlock-proof
std::lock_guard<std::mutex> lck(m1, std::adopt_lock);
std::lock_guard<std::mutex> update_lock(m2, std::adopt_lock);
is_being_updated = false;
this->value = 0; // Reset value
aborted = true;
}
cv.notify_one(); // Signal abort ...
t.join(); // Wait for the thread to finish
}
You're good. The way you wrote the wait, you will only come back if abort==true or 10 seconds has elapsed.
1) I think this problem is inherent on your design, as it is a bool flag will not fix the problem. Maybe A shouldn't go out of scope until all the threads stop using it, in which case it should reside in a managed pointer like shared_ptr.
2) You should be using atomics for your bools and also value, this would avoid having to use the unique_lock for increasing the value and for returning it.
3) As I said in the comments the lambda in the cv handles the spurious wakeups.
The biggest bit of code smell is using a full thread to update a variable every 10 seconds. A heavy-weight OS thread with magabytes to gigabytes of address space to do one task every 10 seconds.
What more, it is updating a value without anyone being able to see the change.
You already have a get_value wrapping accessor. Simply store the start point when you want to start counting. When you call get_value calculate the time since the start point. Divide by 10 seconds. Use that to calculate the returned value.
In a real application, you'd have a timer system that lets you trigger events (either in a thread pool, or in a message pump) every period of time. You'd use that instead of a dedicated thread to do something like this, and you'd make sure that modifying that value was vulgar (allowed people to subscribe to changes in it). Then your abort would consist of deregistering the timer instead of stopping a thread.
Your system is a horrible mixture of the two, using threads for no good reason.
I am a bit stuck with the problem, so it is my cry for help.
I have a manager that pushes some events to a queue, which is proceeded in another thread.
I don't want this thread to be 'busy waiting' for events in the queue, because it may be empty all the time (as well as it may always be full).
Also I need m_bShutdownFlag to stop the thread when needed.
So I wanted to try a condition_variable for this case: if something was pushed to a queue, then the thread starts its work.
Simplified code:
class SomeManager {
public:
SomeManager::SomeManager()
: m_bShutdownFlag(false) {}
void SomeManager::Initialize() {
boost::recursive_mutex::scoped_lock lock(m_mtxThread);
boost::thread thread(&SomeManager::ThreadProc, this);
m_thread.swap(thread);
}
void SomeManager::Shutdown() {
boost::recursive_mutex::scoped_lock lock(m_mtxThread);
if (m_thread.get_id() != boost::thread::id()) {
boost::lock_guard<boost::mutex> lockEvents(m_mtxEvents);
m_bShutdownFlag = true;
m_condEvents.notify_one();
m_queue.clear();
}
}
void SomeManager::QueueEvent(const SomeEvent& event) {
boost::lock_guard<boost::mutex> lockEvents(m_mtxEvents);
m_queue.push_back(event);
m_condEvents.notify_one();
}
private:
void SomeManager::ThreadProc(SomeManager* pMgr) {
while (true) {
boost::unique_lock<boost::mutex> lockEvents(pMgr->m_mtxEvents);
while (!(pMgr->m_bShutdownFlag || pMgr->m_queue.empty()))
pMgr->m_condEvents.wait(lockEvents);
if (pMgr->m_bShutdownFlag)
break;
else
/* Thread-safe processing of all the events in m_queue */
}
}
boost::thread m_thread;
boost::recursive_mutex m_mtxThread;
bool m_bShutdownFlag;
boost::mutex m_mtxEvents;
boost::condition_variable m_condEvents;
SomeThreadSafeQueue m_queue;
}
But when I test it with two (or more) almost simultaneous calls to QueueEvent, it gets locked at the line boost::lock_guard<boost::mutex> lockEvents(m_mtxEvents); forever.
Seems like the first call doesn't ever release lockEvents, so all the rest just keep waiting for its freeing.
Please, help me to find out what am I doing wrong and how to fix this.
There's a few things to point out on your code:
You may wish to join your thread after calling shutdown, to ensure that your main thread doesn't finish before your other thread.
m_queue.clear(); on shutdown is done outside of your m_mtxEvents mutex lock, meaning it's not as thread safe as you think it is.
your 'thread safe processing' of the queue should be just taking an item off and then releasing the lock while you go off to process the event. You've not shown that explicitly, but failure to do so will result in the lock preventing items from being added.
The good news about a thread blocking like this, is that you can trivially break and inspect what the other threads are doing, and locate the one that is holding the lock. It might be that as per my comment #3 you're just taking a long time to process an event. On the other hand it may be that you've got a dead lock. In any case, what you need is to use your debugger to establish exactly what you've done wrong, since your sample doesn't have enough in it to demonstrate your problem.
inside ThreadProc, while(ture) loop, the lockEvents is not unlocked in any case. try put lock and wait inside a scope.
I want to wait for two bool variables to be true in one thread. They are changed in different places. I can use boost in my project, but not C++11.
I did find Info on how to use mutexes and condition variables, but im not sure if its possible to wait for two mutexes.
This is some pseudocode of my program.
bool job1_dataready, job2_dataready;
//t1:
void job1()
{
//do stuff
job1_dataready = true;
}
//t2:
void job2()
{
//do stuff
job2_dataready= true;
}
main()
{
boost::thread t1(job1);
boost::thread t1(job2);
if(job1_dataready&& job2_dataready)
{
//do stuff with data from both jobs
}
}
from what I see, you don't need bool variables, use std::thread::join instead:
main() {
std::thread t1(job1);
std::thread t1(job2);
t1.join();
t2.join();
// do jobs after threads t1 and t2 finish working
}
you would block on the condition variable, check your boolean values when woken, and either go back to waiting or continue processing. Your threads will signal the condition variable after they have set the boolean flag. All with appropriate mutex locking of course. You can wait on an infinite number of conditions, just check when woken after blocking on the condition.
In simple situations like this, you wait on two mutexes simply by locking them in order. First you lock the mutex from thread 1, then the mutex from thread 2. If thread 2 would finish before thread 1, the main thread would simply not block when locking mutex 2.
However, note that this is an answer you your question, but not a solution to your problem. The reason is that you have a race condition with the mutex: the main thread might lock the mutex before the worker thread even starts. So, while Andrei R.s response (std::thread::join) isn't a direct answer, it is the correct solution.
If you plan to set your two bools just before the respective threads terminate, then Andrei R.'s solution of just joining the two threads is definitely the best way to go. However, if your threads actually continue working after the dataready points are reached, and are thus not terminating yet, you need a different approach. In that case, you could use two std::future/std::promise objects, which would look something like this:
std::promise<bool> job1_dataready, job2_dataready;
//t1:
void job1()
{
//do stuff
job1_dataready.set_value(true); // The value doesn't actually matter
//do more stuff
}
//t2:
void job2()
{
//do stuff
job2_dataready.set_value(true);
//do more stuff
}
main()
{
std::future<bool> job1_future = job1_dataready.get_future();
std::future<bool> job2_future = job2_dataready.get_future();
boost::thread t1(job1);
boost::thread t2(job2);
job1_future.wait();
job2_future.wait();
if (job1_future.get() && job2_future.get()) // True unless something was aborted
{
//do stuff with data from both jobs
}
}
I'm in a scenario when I have to terminate a thread while the thread is running according to user action on GUI. I'm using Qt 4.5.2 on Windows. One way to do that is the following:
class MyThread : public QThread
{
QMutex mutex;
bool stop;
public:
MyThread() : stop(false) {}
void requestStop()
{
QMutexLocker(&mutex);
stop = true;
}
void run()
{
while(counter1--)
{
QMutexLocker(&mutex);
if (stop) return;
while(counter2--)
{
}
}
}
};
Please note that the above code is minimal. The run function can take upto 20 seconds before finish so I want to avoid locking and unlocking the mutex variable in the loop. Is there any other way faster than this method.
Thanks in advance.
It doesn't directly answer your need, but can't you scope your mutex much tighter ?
while(counter1--) {
{
QMutexLocker(&mutex);
if (stop) return;
} // End locking scope : we won't read it anymore until next time
while(counter2--)
...
Firstly it doesn't look like you need a mutex around your entire inner loop, just around the if (stop) expression as the others say, but I may be missing some of your app context to definitively say that. Maybe you need requestStop() to block until the thread exits.
If the reduced mutex scope is adequate for you, then you don't need a mutex at all if you declare your stop variable as "volatile". The "volatile" keyword causes (at least under VC++) a read/write memory barrier to be placed around accesses to stop, which means your requestStop() call is guaranteed to be communicated to your thread and not cached away. The following code should work just fine on multicore processors.
class MyThread : public QThread
{
volatile bool stop;
public:
MyThread() : stop(false) {}
void requestStop()
{
stop = true;
}
void run()
{
while(counter1--)
{
if (stop) return;
while(counter2--)
{
}
}
}
};
The main problem in your code is that you are holding the lock for much longer than you actually need. You should unlock it after you check the stop variable. That should make it much faster (depending on what is done in the inner loop). A lock-free alternative is to use QAtomicInt.
You could use a critical section instead of a mutex. They have a bit less overhead.
Otherwise you have to use this approach. If you want the worker thread to terminate within some interval t seconds, then it needs to check for a termination event at least once every t seconds.
Why not use an event that can be checked periodically and let the underlying platform worry about whether a mutex is needed or not to handle the event (I assume that Qt has event objects - I'm not all that familiar with it). If you use an event object, the platform will scope any critical section need to handle that event to as short a time period as necessary.
Also, since there's likely not going to be much contention for that mutex (the only time would be when something wants to kill the thread), grabbing and releasing the mutex will likely have little performance impact. In a loop that's taking 20 seconds to run, I'd be surprised if the impact were anything that could even be measured. But maybe I'm wrong - try measuring it by timing the thread with and without the mutex being taken. See if it's something you really need to concern yourself with.
Qt doesn't seem to have the kind of event object I'm talking about (one along the lines of Win32's event objects), but a QSemaphore can be used just as easily:
class MyThread : public QThread
{
QSemaphore stopFlag;
public:
MyThread() : stopFlag( 1) {}
void requestStop()
{
stopFlag.tryAcquire(); // decrement the flag (if it hasn't been already)
}
void run()
{
while(counter1--)
{
if (!stopFlag.available()) return;
while(counter2--)
{
}
}
}
};