When is it more appropriate to use a pthread barrier instead of a condition wait and broadcast? - c++

I am coding a telemetry system in C++ and have been having some difficulty syncing certain threads with the standard pthread_cond_timedwait and pthread_cond_broadcast.
The problem was that I needed some way for the function that was doing the broadcasting to know if another thread acted on the broadcast.
After some hearty searching I decided I might try using a barrier for the two threads instead. However, I still wanted the timeout functionality of the pthread_cond_timedwait.
Here is basically what I came up with: (However it feels excessive)
Listen Function: Checks for a period of milliseconds to see if an event is currently being triggered.
bool listen(uint8_t eventID, int timeout)
{
int waitCount = 0;
while(waitCount <= timeout)
{
globalEventID = eventID;
if(getUpdateFlag(eventID) == true)
{
pthread_barrier_wait(&barEvent);
return true;
}
threadSleep(); //blocks for 1 millisecond
++waitCount;
}
return false;
}
Trigger Function: Triggers an event for a period of milliseconds by setting an update flag for the triggering period
bool trigger(uint8_t eventID, int timeout)
int waitCount = 0;
while(waitCount <= timeout)
{
setUpdateFlag(eventID, true); //Sets the update flag to true
if(globalEventID == eventID)
{
pthread_barrier_wait(&barEvent);
return true;
}
threadSleep(); //blocks for 1 millisecond
++waitCount;
}
setUpdateFlag(eventID, false);
return false;
}
My questions: Is another way to share information with the broadcaster, or are barriers really the only efficient way? Also, is there another way of getting timeout functionality with barriers?

Based on your described problem:
Specifically, I am trying to let thread1 know that the message it is
waiting for has been parsed and stored in a global list by thread2,
and that thread2 can continue parsing and storing because thread1 will
now copy that message from the list ensuring that thread2 can
overwrite that message with a new version and not disrupt the
operations of thread1.
It sounds like your problem can be solved by having both threads alternately wait on the condition variable. Eg. in thread 1:
pthread_mutex_lock(&mutex);
while (!message_present)
pthread_cond_wait(&cond, &mutex);
copy_message();
message_present = 0;
pthread_cond_broadcast(&cond);
pthread_mutex_unlock(&mutex);
process_message();
and in thread 2:
parse_message();
pthread_mutex_lock(&mutex);
while (message_present)
pthread_cond_wait(&cond, &mutex);
store_message();
message_present = 1;
pthread_cond_broadcast(&cond);
pthread_mutex_unlock(&mutex);

Related

Second thread is never triggered

I've been struggling with a multithreading issue for a bit. I've written some simple code to try and isolate the issue and I'm not finding it. What's happening is that the first thread is being woken up with data being sent to it, but second one never does. They each have their own condition_variable yet it doesn't seem to matter. Ultimately, what I'm trying to do is have a few long running threads that do a single dedicated task when needed, and staying in a wait state when not needed. And running them each in their own thread is important and a requirement.
Here's the code:
#include <glib.h>
#include <string>
#include <mutex>
#include <condition_variable>
#include <unistd.h>
#define NUM_THREADS 2
bool DEBUG = true;
pthread_t threads[NUM_THREADS];
std::mutex m_0;
std::mutex m_1;
std::condition_variable cov_0;
std::condition_variable cov_1;
bool dataReady_0 = false;
bool dataReady_1 = false;
bool keepRunning[NUM_THREADS] = { true };
void date_update (guint source_id, const char *json_data) {
if (DEBUG) {
start_threads(2);
sleep(2);
DEBUG = false;
}
g_print("From source id=%d\n", source_id);
switch (source_id) {
case 0:
dataReady_0 = true;
cov_0.notify_one();
break;
case 1:
dataReady_1 = true;
cov_1.notify_one();
break;
}
}
void start_threads (int thread_count) {
int rc;
switch (thread_count) {
case 2:
rc = pthread_create(&threads[1], nullptr, custom_thread_1, nullptr);
if (rc) {
g_print("Error:unable to create thread(1), return code(%d)\n", rc);
}
case 1:
rc = pthread_create(&threads[0], nullptr, custom_thread_0, nullptr);
if (rc) {
g_print("Error:unable to create thread(0), return code(%d)\n", rc);
}
}
}
void *custom_thread_0 (void *pVoid) {
g_print("Created thread for source id=0\n");
while (keepRunning[0]) {
// Wait until date_update() sends data
std::unique_lock<std::mutex> lck(m_0);
cov_0.wait(lck, [&]{return dataReady_0;});
dataReady_0 = false;
g_print("THREAD=0, DATA RECEIVED\n");
lck.unlock();
}
pthread_exit(nullptr);
}
void *custom_thread_1 (void *pVoid) {
g_print("Created thread for source id=1\n");
while (keepRunning[1]) {
// Wait until date_update() sends data
std::unique_lock<std::mutex> lck(m_1);
cov_1.wait(lck, [&]{return dataReady_1;});
dataReady_1 = false;
g_print("THREAD=1, DATA RECEIVED\n");
lck.unlock();
}
pthread_exit(nullptr);
}
Here's the output. As you can see the data_update function gets the "data" from the calling function for both source 0 and source 1, but only thread 0 ever seems to process anything. I'm at a bit of a loss as to the source of the problem.
Sending data for source id=1
From source id=1
Sending data for source id=0
From source id=0
THREAD=0, DATA RECEIVED
Sending data for source id=1
From source id=1
Sending data for source id=0
From source id=0
THREAD=0, DATA RECEIVED
Sending data for source id=1
From source id=1
Sending data for source id=0
From source id=0
THREAD=0, DATA RECEIVED
I'm sure I'm just missing a minor detail somewhere, but I'm fully willing to accept that perhaps I do not understand C/C++ threading correctly.
The 2nd thread is exiting because the keepRunning state flag is false. It's usually a good first step in debugging threads to log the start and exit of all threads.
But you have a much less obvious problem.
It does not appear that the appropriate mutex is held when the value of the condition variable's predicate is changed in date_update().
I'll break that down a bit more.
When cov_0.wait() is called, the predicate used is [&]{return dataReady_0;} (*), and the unique_lock passed is holding the mutex m_0. This means that whenever the value of the predicate might change, the mutex m_0 must be held.
This predicate is quite simple and will change value whenever the global variable dataReady_0 changes value.
In date_update() there is code to change the value of dataReady_0 and the mutex m_0 is not held when doing this. There should be a scoped_lock or unique_lock in the block that changes the global variable's state.
It will still mostly work without this, but you have a race! It will fail eventually!
The condition variable may check and see that the predicate is false, then another thread changes the predicate's value and does a notify, and then the first thread waits on the condition variable. It misses the notify because it was not yet waiting when it was sent. The use of the mutex to prevent the predicate from changing in a way that races with the notification is a critical component of what makes this work.
(*) You don't need the capture [&] here. This lambda could be stateless.
You should initialize all elements of the built-in array:
bool keepRunning[2] = { true, true };

condition_variable doesn't get notified to wake up even with a predicate

I'm having a problem where I'm having a few condition_variable's get stuck in their wait phase even though they've been notified. Each one even has a predicate that's being set just in case they miss the notify call from the main thread.
Here's the code:
unsigned int notifyCount = 10000;
std::atomic<int> threadCompletions = 0;
for (unsigned int i = 0; i < notifyCount; i++)
{
std::atomic<bool>* wakeUp = new std::atomic<bool>(false);
std::condition_variable* condition = new std::condition_variable();
// Worker thread //
std::thread([&, condition, wakeUp]()
{
std::mutex mutex;
std::unique_lock<std::mutex> lock(mutex);
condition->wait(lock, [wakeUp] { return wakeUp->load(); });
threadCompletions++;
}).detach();
// Notify //
*wakeUp = true;
condition->notify_one();
}
Sleep(5000); // Sleep for 5 seconds just in case some threads are taking a while to finish executing
// Check how many threads finished (threadCompletions should be equal to notifyCount)
Unless I'm mistaken, after the for loop is done, threadCompletions should always be equal to notifyCount. Very often though, it is not.
When running in release, I'll sometimes get just one or two out of 10000 threads that never finished, but when running in debug, I'll get 20 or more.
I thought maybe the wait call in the thread is happening after the main thread's notify_one call (meaning it missed it's notification to wake up), so I passed a predicate into wait to insure that it doesn't get stuck waiting. But it still does in some cases.
Does anyone know why this is happening?
You are assuming the call to wait() is atomic. I don't believe it is. That is why it requires the use of a mutex and a lock.
Consider the following:
Main Thread. Child Thread
// This is your wait unrolled.
while (!wakeUp->load()) {
// This is atomic
// But already checked in the
// thread.
*wakeUp = true;
// Child has not yet called wait
// So this notify_one is wasted.
condition->notify_one();
// The previous call to notify_one
// is not recorded and thus the
// thread is now locked in this wait
// never to be let free.
wait(lock);
}
// Your race condition.
Calls to notify_one() and wait() should be controlled via the same mutext to make sure they don't overlap like this.
for (unsigned int i = 0; i < notifyCount; i++)
{
std::atomic<bool>* wakeUp = new std::atomic<bool>(false);
std::mutex* mutex = new std::mutex{};
std::condition_variable* condition = new std::condition_variable();
// Worker thread //
std::thread([&]()
{
std::unique_lock<std::mutex> lock(*mutex);
condition->wait(lock, [&wakeUp] { return wakeUp->load(); });
threadCompletions++;
}).detach();
// Notify //
*wakeUp = true;
std::unique_lock<std::mutex> lock(*mutex);
condition->notify_one();
}
// Don't forget to clean up the new structures correctly/.
You have data racing. Consider following scenario:
Worker Thread: condition variable tests for whether wakeup is true - it isn't
Main Thread: wakeup is set to true and condition variable is getting notified
Worker Thread: condition_variable triggers wait but it happens after notification already occurred - impling that notification misses and the thread might never wake up.
Normally, synchronization of condition variables is done via mutexes - atomics aren't too helpful here. In C++20 there will be special mechanism for waiting/notifying in atomics.

How to abort async() if timeout has elapsed

I have a questions about async() function or any other way to solve my problem. I send to the server specified type of message and I wait for a specific
response.
I have function receive() which waits for response from server. I call this function inside async().
Sample of code:
while (true) {
future_receive = std::async(std::launch::async, [&] {
receive();
});
do {
status = future_receive.wait_for(chrono::seconds(timeLimit));
if (status == std::future_status::timeout){
//if timeout, abort async() function
}
} while (status != std::future_status::ready);
}
What is my problem? In this case, if I get "timeout", async() function will work on,
will wait until something comes, even if it will never come, and in the next cycle will be called again,
and new thread will be created. How to avoid this?
How I can abort async() when "timeout" has elapsed. Maybe any other way without async() to solve this problem. I would like to use only the standard library of C++?
The asynchronous thread has to cooperate and check whether it should continue working or give up, there is no portable way to force it to stop without its cooperation.
One way to do that is to replace the receive() call with a similar one that has a timeout, and have the thread give up after a timeout, or check a flag after a timeout to indicate whether to continue.
while (true) {
std::atomic<bool> stop{false};
future_receive = std::async(std::launch::async, [&] {
while (!stop)
try_receive(std::chrono::seconds(1));
});
do {
status = future_receive.wait_for(chrono::seconds(timeLimit));
if (status == std::future_status::timeout){
stop = true;
}
} while (status != std::future_status::ready);
}
Now the asynchronous thread will only block for up to a second, then will check if it's been told to give up, otherwise it will try receiving again.
If you're willing to sacrifice portability, something like this should work on platforms where std::thread is implemented in terms of POSIX threads:
while (true) {
std::atomic<pthread_t> tid{ pthread_self() };
future_receive = std::async(std::launch::async, [&] {
tid = pthread_self();
receive();
});
do {
status = future_receive.wait_for(chrono::seconds(timeLimit));
if (status == std::future_status::timeout){
while (tid == pthread_self())
{ /* wait for async thread to update tid */ }
pthread_cancel(tid);
}
} while (status != std::future_status::ready);
}
This assumes that there is a Pthreads cancellation point somewhere in the receive() call, so that the pthread_cancel will interrupt it.
(This is slightly more complicated than I would like. It's necessary to store some known value in the atomic initially in order to handle the situation where the async thread has not even started running yet when the calling thread gets a timeout and tries to cancel it. To handle that I store the calling thread's ID, then wait until it's changed before calling pthread_cancel.)

Waiting until another process locks and then unlocks a Win32 mutex

I am trying to tell when a producer process accesses a shared windows mutex. After this happens, I need to lock that same mutex and process the associated data. Is there a build in way in Windows to do this, short of a ridiculous loop?
I know the result of this is doable through creating a custom Windows event in the producer process, but I want to avoid changing this programs code as much as possible.
What I believe will work (in a ridiculously inefficient way) would be this (NOTE: this is not my real code, I know there are like 10 different things very wrong with this; I want to avoid doing anything like this):
#include <Windows.h>
int main() {
HANDLE h = CreateMutex(NULL, 0, "name");
if(!h) return -1;
int locked = 0;
while(true) {
if(locked) {
//can assume it wont be locked longer than a second, but even if it does should work fine
if(WaitForSingleObject(h, 1000) == WAIT_OBJECT_0) {
// do processing...
locked = 0;
ReleaseMutex(h);
}
// oh god this is ugly, and wastes so much CPU...
} else if(!(locked = WaitForSingleObject(h, 0) == WAIT_TIMEOUT)) {
ReleaseMutex(h);
}
}
return 0;
}
If there is an easier way with C++ for whatever reason, my code is actually that. This example was just easier to construct in C.
You will not be able to avoid changing the producer if efficient sharing is needed. Your design is fundamentally flawed for that.
A producer needs to be able to signal a consumer when data is ready to be consumed, and to make sure it does not alter the data while it is busy being consumed. You cannot do that with a single mutex alone.
The best way is to have the producer set an event when data is ready, and have the consumer reset the event when the data has been consumed. Use the mutex only to sync access to the data, not to signal the data's readiness.
#include <Windows.h>
int main()
{
HANDLE readyEvent = CreateEvent(NULL, TRUE, FALSE, "ready");
if (!readyEvent) return -1;
HANDLE mutex = CreateMutex(NULL, FALSE, "name");
if (!mutex) return -1;
while(true)
{
if (WaitForSingleObject(readyEvent, 1000) == WAIT_OBJECT_0)
{
if (WaitForSingleObject(mutex, 1000) == WAIT_OBJECT_0)
{
// process as needed...
ResetEvent(readyEvent);
ReleaseMutex(mutex);
}
}
}
return 0;
}
If you can't change the producer to use an event, then at least add a flag to the data itself. The producer can lock the mutex, update the data and flag, and unlock the mutex. Consumers will then have to periodically lock the mutex, check the flag and read the new data if the flag is set, reset the flag, and unlock the mutex.
#include <Windows.h>
int main()
{
HANDLE mutex = CreateMutex(NULL, FALSE, "name");
if (!mutex) return -1;
while(true)
{
if (WaitForSingleObject(mutex, 1000) == WAIT_OBJECT_0)
{
if (ready)
{
// process as needed...
ready = false;
}
ReleaseMutex(mutex);
}
}
return 0;
}
So either way, your logic will have to be tweaked in both the producer and consumer.
Otherwise, if you can't change the producer at all, then you have no choice but to change the consumer alone to simply check the data for changes peridiodically:
#include <Windows.h>
int main()
{
HANDLE mutex = CreateMutex(NULL, 0, "name");
if (!mutex) return -1;
while(true)
{
if (WaitForSingleObject(mutex, 1000) == WAIT_OBJECT_0)
{
// check data for changes
// process new data as needed
// cache results for next time...
ReleaseMutex(mutex);
}
}
return 0;
}
Tricky. I'm going to answer the underlying question: when is the memory written?
This can be observed via a four step solution:
Inject a DLL in the watched process
Add a vectored exception handler for STATUS_GUARD_PAGE_VIOLATION
Set the guard page bit on the 2 MB memory range (finding it could be a challenge)
From the vectored exception handler, inform your process and re-establish the guard bit (it's one-shot)
You may need only a single guard page if the image is always fully rewritten.

Choice between thread: time expired and user input

I'm writing a simple function that, when called, allows to execute 2 different actions (exclusive).
So there are two threads. User_choice waits until the user inserts an input and the Time_choice waits until time expires.
The choice_done shared var says that, if true, one thread has already started and blocking (it doesn't do anything!) the other one; Whereas thread_done says, if true, that thread (it doesn't matter which) has already finished, so func() waits until one thread finishes.
Here is the code.
The func procedure will be called more times during the program execution.
The various user_choice thread will be waiting forever on getline! Is it a problem? What if, after four times the program will call func() and the user doesn't insert anything, the 5th time the user inserts "yes"?
Will every user_choice thread continue the execution?? How can I kill the waiting thread? Are there other solutions?
How can I wait inside func() that a thread sets thread_done to true?
bool choice_done = false;
bool thread_done = false;
void func(){
boost::thread t1(boost::bind( time_choice() ));
boost::thread t2(boost::bind( user_choice() ));
//whait untile thread_done == true
do something...
}
// Time choice thread
void time_choice(){
sleep(5);
if(choice_done == false){
printf("Automatic choice\n");
choice_done == true;
do something...
thread_done = true;
}
}
// User choice thread
void user_choice(){
printf("Start emergency procedure?\n");
string tmp;
getline(cin, tmp);
if((tmp.compare("yes") == 0) && (choice_done == false)){
printf("Manual choice\n");
choice_done == true;
do something...
thread_done = true;
}
}
Having to create a thread for a timer is generally a sign of sub-optimal design. It does not scale well (imagine thousands of timers) and the code gets multi-threaded and more complex for no good reason. Also, sleep is not thread-safe on Linux.
Just use one thread with select and a timeout. select will wait on STDIN_FILENO for user input and timeout simultaneously.
Or, better, use a 3rd-party event-demultiplexing library, like libevent or boost::asio.