I would like to have items added to a queue in one thread via in an asynchronous web request handler:
void handleRequest(item) {
toProcess.push_back(item);
}
There is a background thread that constantly processes these queue items as follows:
while(true) {
for(auto item : toProcess) { doSomething(item); }
toProcess.clear();
}
Clearly this isn't thread safe ... you might add an item to toProcess right when the for loop finishes and thus have it cleared out without being processed. What would be the best model to program something like this?
I'm going to use std::atomic<T>::wait which is a C++20 feature, there is a way to do it with condition variables too however, and they exist since C++11.
Include <atomic> and <mutex>
You will need a member atomic_bool.
std::atomic_bool RequestPassed = false;
and a member mutex
std::mutex RequestHandleMutex;
Your handleRequest function would then become
void handleRequest(item) {
std::lock_guard<std::mutex> lg(RequestHandleMutex)
toProcess.push_back(item);
RequestPassed.store(true);
RequestPassed.notify_all();
}
and your loop would be this
while(true) {
RequestPassed.wait(false);
std::lock_guard<std::mutex> lg(RequestHandleMutex)
/* handle latest item passed */
RequestPassed.store(false);
}
This way, the while thread waits instead of constantly iterating (saving cpu power and battery). If you then use handleRequest, the atomic_bool gets notified to stop waiting, the request is handled (mutex is locked so no new requests can come while this happens), RequestPassed is reset to false, and the thread waits for the next request.
Related
std::queue<double> some_q;
std::mutex mu_q;
/* an update function may be an event observer */
void UpdateFunc()
{
/* some other processing */
std::lock_guard lock{ mu_q };
while (!some_q.empty())
{
const auto& val = some_q.front();
/* update different states according to val */
some_q.pop();
}
/* some other processing */
}
/* some other thread might add some values after processing some other inputs */
void AddVal(...)
{
std::lock_guard lock{ mu_q };
some_q.push(...);
}
For this case is it okay to handle the queue this way?
Or would it be better if I try to use a lock-free queue like the boost one?
How bad it is to lock a mutex in an infinite loop or an update function
It's pretty bad. Infinite loops actually make your program have undefined behavior unless it does one of the following:
terminate
make a call to a library I/O function
perform an access through a volatile glvalue
perform a synchronization operation or an atomic operation
Acquiring the mutex lock before entering the loop and just holding it does not count as performing a synchronization operation (in the loop). Also, when holding the mutex, noone can add information to the queue, so while processing the information you extract, all threads wanting to add to the queue will have to wait - and no other worker threads wanting to share the load can extract from the queue either. It's usually better to extract one task from the queue, release the lock and then work with what you got.
The common way is to use a condition_variable that lets other threads acquire the lock and then notify other threads waiting with the same condition_variable. The CPU will be pretty close to idle while waiting and wake up to do the work when needed.
Using your program as a base, it could look like this:
#include <chrono>
#include <condition_variable>
#include <iostream>
#include <mutex>
#include <queue>
#include <thread>
std::queue<double> some_q;
std::mutex mu_q;
std::condition_variable cv_q; // the condition variable
bool stop_q = false; // something to signal the worker thread to quit
/* an update function may be an event observer */
void UpdateFunc() {
while(true) {
double val;
{
std::unique_lock lock{mu_q};
// cv_q.wait lets others acquire the lock to work with the queue
// while it waits to be notified.
while (not stop_q && some_q.empty()) cv_q.wait(lock);
if(stop_q) break; // time to quit
val = std::move(some_q.front());
some_q.pop();
} // lock released so others can use the queue
// do time consuming work with "val" here
std::cout << "got " << val << '\n';
}
}
/* some other thread might add some values after processing some other inputs */
void AddVal(double val) {
std::lock_guard lock{mu_q};
some_q.push(val);
cv_q.notify_one(); // notify someone that there's a new value to work with
}
void StopQ() { // a function to set the queue in shutdown mode
std::lock_guard lock{mu_q};
stop_q = true;
cv_q.notify_all(); // notify all that it's time to stop
}
int main() {
auto th = std::thread(UpdateFunc);
// simulate some events coming with some time apart
std::this_thread::sleep_for(std::chrono::seconds(1));
AddVal(1.2);
std::this_thread::sleep_for(std::chrono::seconds(1));
AddVal(3.4);
std::this_thread::sleep_for(std::chrono::seconds(1));
AddVal(5.6);
std::this_thread::sleep_for(std::chrono::seconds(1));
StopQ();
th.join();
}
If you really want to process everything that is currently in the queue, then extract everything first and then release the lock, then work with what you extracted. Extracting everything from the queue is done quickly by just swapping in another std::queue. Example:
#include <atomic>
std::atomic<bool> stop_q{}; // needs to be atomic in this version
void UpdateFunc() {
while(not stop_q) {
std::queue<double> work; // this will be used to swap with some_q
{
std::unique_lock lock{mu_q};
// cv_q.wait lets others acquire the lock to work with the queue
// while it waits to be notified.
while (not stop_q && some_q.empty()) cv_q.wait(lock);
std::swap(work, some_q); // extract everything from the queue at once
} // lock released so others can use the queue
// do time consuming work here
while(not stop_q && not work.empty()) {
auto val = std::move(work.front());
work.pop();
std::cout << "got " << val << '\n';
}
}
}
You can use it like you currently are assuming proper use of the lock across all threads. However, you may run into some frustrations about how you want to call updateFunc().
Are you going to be using a callback?
Are you going to be using an ISR?
Are you going to be polling?
If you use a 3rd party lib it often trivializes thread synchronization and queues
For example, if you are using a CMSIS RTOS(v2). It is a fairly straight forward process to get multiple threads to pass information between each other. You could have multiple producers, and a single consumer.
The single consumer can wait in a forever loop where it waits to receive a message before performing its work
when timeout is set to osWaitForever the function will wait for an
infinite time until the message is retrieved (i.e. wait semantics).
// Two producers
osMessageQueuePut(X,Y,Z,timeout=0)
osMessageQueuePut(X,Y,Z,timeout=0)
// One consumer which will run only once something enters the queue
osMessageQueueGet(X,Y,Z,osWaitForever)
tldr; You are safe to proceed, but using a library will likely make your synchronization problems easier.
I want the while loop in the thread to run , wait a second, then run again, so on and so on., but this don't seem to work, how would I fix it?
main(){
bool flag = true;
pthread = CreateThread(NULL, 0, ThreadFun, this, 0, &ThreadIP);
}
ThreadFun(){
while(flag == true)
WaitForSingleObject(pthread,1000);
}
This is one way to do it, I prefer using condition variables over sleeps since they are more responsive and std::async over std::thread (mainly because std::async returns a future which can send information back the the starting thread. Even if that feature is not used in this example).
#include <iostream>
#include <chrono>
#include <future>
#include <condition_variable>
// A very useful primitive to communicate between threads is the condition_variable
// despite its name it isn't a variable perse. It is more of an interthread signal
// saying, hey wake up thread something may have changed that's interesting to you.
// They come with some conditions of their own
// - always use with a lock
// - never wait without a predicate
// (https://www.modernescpp.com/index.php/c-core-guidelines-be-aware-of-the-traps-of-condition-variables)
// - have some state to observe (in this case just a bool)
//
// Since these three things go together I usually pack them in a class
// in this case signal_t which will be used to let thread signal each other
class signal_t
{
public:
// wait for boolean to become true, or until a certain time period has passed
// then return the value of the boolean.
bool wait_for(const std::chrono::steady_clock::duration& duration)
{
std::unique_lock<std::mutex> lock{ m_mtx };
m_cv.wait_for(lock, duration, [&] { return m_signal; });
return m_signal;
}
// wiat until the boolean becomes true, wait infinitely long if needed
void wait()
{
std::unique_lock<std::mutex> lock{ m_mtx };
m_cv.wait(lock, [&] {return m_signal; });
}
// set the signal
void set()
{
std::unique_lock<std::mutex> lock{ m_mtx };
m_signal = true;
m_cv.notify_all();
}
private:
bool m_signal { false };
std::mutex m_mtx;
std::condition_variable m_cv;
};
int main()
{
// create two signals to let mainthread and loopthread communicate
signal_t started; // indicates that loop has really started
signal_t stop; // lets mainthread communicate a stop signal to the loop thread.
// in this example I use a lambda to implement the loop
auto future = std::async(std::launch::async, [&]
{
// signal this thread has been scheduled and has started.
started.set();
do
{
std::cout << ".";
// the stop_wait_for will either wait 500 ms and return false
// or stop immediately when stop signal is set and then return true
// the wait with condition variables is much more responsive
// then implementing a loop with sleep (which will only
// check stop condition every 500ms)
} while (!stop.wait_for(std::chrono::milliseconds(500)));
});
// wait for loop to have started
started.wait();
// give the thread some time to run
std::this_thread::sleep_for(std::chrono::seconds(3));
// then signal the loop to stop
stop.set();
// synchronize with thread stop
future.get();
return 0;
}
While the other answer is a possible way to do it, my answer will mostly answer from a different angle trying to see what could be wrong with your code...
Well, if you don't care to wait up to one second when flag is set to false and you want a delay of at least 1000 ms, then a loop with Sleep could work but you need
an atomic variable (for ex. std::atomic)
or function (for ex. InterlockedCompareExchange)
or a MemoryBarrier
or some other mean of synchronisation to check the flag.
Without proper synchronisation, there is no guarantee that the compiler would read the value from memory and not the cache or a register.
Also using Sleep or similar function from a UI thread would also be suspicious.
For a console application, you could wait some time in the main thread if the purpose of you application is really to works for a given duration. But usually, you probably want to wait until processing is completed. In most cases, you should usually wait that threads you have started have completed.
Another problem with Sleep function is that the thread always has to wake up every few seconds even if there is nothing to do. This can be bad if you want to optimize battery usage. However, on the other hand having a relatively long timeout on function that wait on some signal (handle) might make your code a bit more robust against missed wakeup if your code has some bugs in it.
You also need a delay in some cases where you don't really have anything to wait on but you need to pull some data at regular interval.
A large timeout could also be useful as a kind of watch dog timer. For example, if you expect to have something to do and receive nothing for an extended period, you could somehow report a warning so that user could check if something is not working properly.
I highly recommand you to read a book on multithreading like Concurrency in Action before writing multithread code code.
Without proper understanding of multithreading, it is almost 100% certain that anyone code is bugged. You need to properly understand the C++ memory model (https://en.cppreference.com/w/cpp/language/memory_model) to write correct code.
A thread waiting on itself make no sense. When you wait a thread, you are waiting that it has terminated and obviously if it has terminated, then it cannot be executing your code. You main thread should wait for the background thread to terminate.
I also usually recommand to use C++ threading function over the API as they:
Make your code portable to other system.
Are usually higher level construct (std::async, std::future, std::condition_variable...) than corresponding Win32 API code.
I am writing a network module, the sending of data is carried out in a separate thread, using a concurrent queue to synchronize data in the main thread.
private:
std::mutex mutex_;
std::condition_variable blockNotification_;
moodycamel::ConcurrentQueue<Envelope> sendQueue_;
std::promise<bool> senderThreadStopped_;
void AsyncTransport::RunSender()
{
while (!drain_)
{
SendAllQueuedEnvelope();
std::unique_lock<std::mutex> lock(mutex_);
blockNotification_.wait(lock);
}
// Make sure all envelope have been sent.
SendAllQueuedEnvelope();
senderThreadStopped_.set_value(true);
assert(sendQueue_.size_approx() == 0);
}
void AsyncTransport::SendAllQueuedEnvelope()
{
auto envelope = Envelope::Wrap(nullptr);
while (sendQueue_.try_dequeue(envelope))
{
envelope = syncTransport_->Send(envelope);
}
}
Envelope AsyncTransport::Send(const Envelope& envelope) const
{
if (drain_)
{
return envelope.With<SentFaildStamp>("The current transport has drained.");
}
if (!sendQueue_.try_enqueue(envelope.CloneContent()))
{
return envelope.With<SentFaildStamp>("Send queue is full.");
}
blockNotification_.notify_all();
return envelope;
}
RunSender It runs in a separate thread, and always gets data from the concurrent queue，When all the data in the queue is sent, we wait for the thread to avoid additional cpu overhead until there is new data in the queue.
Send method call in main thread.
But I found that I had a deadlock, what did I do wrong?
I expect the sending thread to enter wait after sending data, and wake up again after there is new data in the queue.
The Send() method isn't thread-safe. I would use a std:lock_guard in a new scope to lock the mutex and ensure it is unlocked before the notify_all call like this:
Envelope AsyncTransport::Send(const Envelope& envelope) const
{
{
const std::lock_guard<std::mutex> lock(mutex_);
if (drain_)
{
return envelope.With<SentFaildStamp>("The current transport has drained.");
}
if (!sendQueue_.try_enqueue(envelope.CloneContent()))
{
return envelope.With<SentFaildStamp>("Send queue is full.");
}
}
blockNotification_.notify_all();
return envelope;
}
Since the lock_guard locks the mutex, you would have to either make the mutex mutable to be used in a const function or remove the const specifier on the function.
You should always have a condition with a wait to protect against spurious wake-ups. See CP.42. So I would change the wait to include a condition like this:
std::unique_lock<std::mutex> lock(mutex_);
blockNotification_.wait(lock, [&]() { return drain_ || !sendQueue_.empty(); });
Now the wait will only wake up once drain is true or the sendQueue has something in it.
You need a variable that is shared between threads and is the condition predicate.
You need to take a lock before reading or writing the predicate. The condition variable wait will unlock before sleeping so the other thread can lock, update the predicate and unlock. You can send the notify before or after unlocking. I prefer after, but meh.
A condition variable on its own is useless. It must always go along with a lock protected variable, or set of variables, which must be checked before continuing after waiting.
And of course it then only makes sense to update whatever that variable is before sending a notification.
Regarding the deadlock: it may happen that blockNotification_.notify_all(); happens before blockNotification_.wait(lock); and then the latter will wait forever. You need to use wait_for that checks if the queue is not-empty so that block_notification can exit when new messages are ready to be sent. Be aware of spurious wakeup.
Poring through legacy code of old and large project, I had found that there was used some odd method of creating thread-safe queue, something like this:
template < typename _Msg>
class WaitQue: public QWaitCondition
{
public:
typedef _Msg DataType;
void wakeOne(const DataType& msg)
{
QMutexLocker lock_(&mx);
que.push(msg);
QWaitCondition::wakeOne();
}
void wait(DataType& msg)
{
/// wait if empty.
{
QMutex wx; // WHAT?
QMutexLocker cvlock_(&wx);
if (que.empty())
QWaitCondition::wait(&wx);
}
{
QMutexLocker _wlock(&mx);
msg = que.front();
que.pop();
}
}
unsigned long size() {
QMutexLocker lock_(&mx);
return que.size();
}
private:
std::queue<DataType> que;
QMutex mx;
};
wakeOne is used from threads as kind of "posting" function" and wait is called from other threads and waits indefinitely until a message appears in queue. In some cases roles between threads reverse at different stages and using separate queues.
Is this even legal way to use a QMutex by creating local one? I kind of understand why someone could do that to dodge deadlock while reading size of que but how it even works? Is there a simpler and more idiomatic way to achieve this behavior?
Its legal to have a local condition variable. But it normally makes no sense.
As you've worked out in this case is wrong. You should be using the member:
void wait(DataType& msg)
{
QMutexLocker cvlock_(&mx);
while (que.empty())
QWaitCondition::wait(&mx);
msg = que.front();
que.pop();
}
Notice also that you must have while instead of if around the call to QWaitCondition::wait. This is for complex reasons about (possible) spurious wake up - the Qt docs aren't clear here. But more importantly the fact that the wake and the subsequent reacquire of the mutex is not an atomic operation means you must recheck the variable queue for emptiness. It could be this last case where you previously were getting deadlocks/UB.
Consider the scenario of an empty queue and a caller (thread 1) to wait into QWaitCondition::wait. This thread blocks. Then thread 2 comes along and adds an item to the queue and calls wakeOne. Thread 1 gets woken up and tries to reacquire the mutex. However, thread 3 comes along in your implementation of wait, takes the mutex before thread 1, sees the queue isn't empty, processes the single item and moves on, releasing the mutex. Then thread 1 which has been woken up finally acquires the mutex, returns from QWaitCondition::wait and tries to process... an empty queue. Yikes.
That you can imagine my problem i describe the usage of my design:
In the class SerialInterface there is a thread that is checking every 10ms if a message is received. The class is implemented as an Observer pattern to notify other classes about the new received message/byte.
The Notify method of the Observer pattern is blocking until every subject has done its operation. Because i want to avoid any lags, I would like to notify the subjects asynchronously.
My first thought were events (condition variables in C++11).
The implementation would look like this:
class SerialInterface: public Observer {
private:
.....
void NotifyThread() {
while (mRunThreadNotify) {
std::unique_lock<std::mutex> lock(mMutex);
mCv.wait(lock);
NotifyObservers();
}
}
std::mutex mMutex;
std::condition_variable mCv;
std::atomic_bool mRunThreadNotify;
std::thread mThreadNotify;
.....
};
Now i can notify asynchronously via mCv.notify_all();
The problem now is following:
What if the thread NotifyThread() is currently notifying the subjects, but theres a new notify event incoming at the same time. It would complete the current notification and the new state would be skipped.
So my second approach was to create a counter for notifications and let it act like a queue:
class SerialInterface: public Observer {
public:
....
private:
.....
void NotifyThread() {
while (mRunThreadNotify) {
if (mNotifications > 0) {
NotifyObservers();
mNotifications--;
} else {
std::this_thread::sleep_for(std::chrono::milliseconds(10));
}
}
}
std::atomic<size_t> mNotifications;
std::atomic_bool mRunThreadNotify;
std::thread mThreadNotify;
.....
};
Here i have to increase the variable mNotifications to notify the subjects. But for me this solution looks not perfect as i use std::this_thread::sleep_for for a fixed waiting time.
Are there any suggestions or another approaches for this problem?
It seems to me that you want to separate the real-time behavior (10mS serial poll) from the rest of the program so that the real-time thread will never be held off waiting for any other routines. Given that, my suggestion would be to split the pattern into two parts:
The real-time part, which does nothing but receive incoming serial data and append it to the end of a FIFO queue (in a thread-safe manner, of course).
The non-real-time part (running in a different thread), in which data is popped from the head of the FIFO queue and handed around to all of the software components that want to react to it. This part can be as fast or as slow as it likes, since it will not hold up the real-time thread.
The FIFO queue part is a standard producer-consumer problem; there are various ways to implement it, but the way I usually do it is with a dequeue, a lock, and a condition variable (pseudocode):
// Called by the real-time/serial thread when it received serial data
void AppendBytesToQueue(const TheBytesObject & bytes)
{
bool wasQueueEmptyBefore;
m_lock.lock();
wasQueueEmptyBefore = (m_fifo.size() == 0);
m_fifo.push_back(bytes);
m_lock.unlock();
if (wasQueueEmptyBefore) m_condition_variable.signal();
}
// Called by the non-real-time/handling thread after it was
// woken up by the condition variable's signal (outQueue should
// be a reference to an empty dequeue that gets filled by this
// method)
void GetNewBytesFromQueue(std::dequeue & outQueue)
{
m_lock.lock();
std::swap(m_fifo, outQueue); // fast O(1) operation so m_lock() will never be locked for long
m_lock.unlock();
}
... and then after calling GetNewBytesFromQueue(), the handling/non-real-time thread can iterate over the contents of its temporary dequeue and deal with each item in order, without any risk of affecting the serial thread's performance.
When a notification is received, you can check whether your requirements have been met at that time.
Meeting the requirement can be specified as a predicate in the second argument to the wait().
mCvNotifications.wait(lock, [](){return true_if_requirements_met;});
If the requirement has not been met, thread will stay in the wait stage despite the notification.