Qt5 QWaitCondition example - c++

I am getting myself familiar with QT5 concurrency library. I was looking at the QWaitCondition example (http://qt-project.org/doc/qt-5.0/qtcore/qwaitcondition.html#details).
Here, one thread (Thread B), reads user input, and all other threads (Thread A) process this input.
Thread A:
forever {
mutex.lock();
keyPressed.wait(&mutex);
++count;
mutex.unlock();
do_something();
mutex.lock();
--count;
mutex.unlock();
}
Thread B:
forever {
getchar();
mutex.lock();
// Sleep until there are no busy worker threads
while (count > 0) {
mutex.unlock();
sleep(1);
mutex.lock();
}
keyPressed.wakeAll();
mutex.unlock();
}
The reason for using count variable and the extensive mutex synchronization is to prevent symbols loss.
The problem is, I think there is still a chance that a symbol will get lost:
imagine the following scenario:
Thread A processes a symbol, and decreases the countes (--count);
the mutex is released; then Thread A stops
Thread B returns from sleep, aquires the mutex, sees, that count ==
0, and calls keyPressed.wakeAll(), and then unlocks the mutex.
However, the wakeAll() call goes to nowhere, since the Thread A is
not waiting.
Thread A starts again, aquaires the mutex and goes into wait().
Since wakeAll() was already called, the symbol is lost.
Am I right, or am I missing something? If I am right, how to correct the example to really prevent it from skipping the symbols?

you are right
to fix this the loop should be entered with the mutex already acquired:
mutex.lock();
forever {
keyPressed.wait(&mutex);
++count;
mutex.unlock();
do_something();
mutex.lock();
--count;
}
mutex.unlock();

Related

Merge two file by sorting them based by timestamp using three threads in C++

I have two file: A,B with rows as csv:timestamp, parameters.. I want to read from one thread one file and from another thread the other file and compare the timestamps in a third thread.
The goal is to construct a vector with the content of the row sorted with the timestamp of both file.
How would you approach this problem?
I did something like this: but it doesn't print out all the timestampsand maybe I'm missing something:
#include <iostream>
#include <fstream>
#include <string>
#include <mutex>
#include <condition_variable>
#include <thread>
#include <vector>
#include <chrono>
std::mutex mtx;
std::condition_variable cv;
long long timestamp1, timestamp2;
std::vector<long long> timestamps;
bool finished1 = false, finished2 = false;
void thread2() {
std::ifstream file2("a.csv");
std::string line;
while (std::getline(file2, line)) {
std::vector<std::string> values = split(line, ',');
long long current_timestamp = std::stoll(values[4]);
{
std::unique_lock<std::mutex> lock(mtx);
while (timestamp1 >= current_timestamp) {
cv.wait(lock);
}
timestamp2 = current_timestamp;
}
cv.notify_one();
}
{
std::unique_lock<std::mutex> lock(mtx);
finished2 = true;
}
cv.notify_one();
}
void thread3() {
while (!finished1 || !finished2) {
std::unique_lock<std::mutex> lock(mtx);
cv.wait(lock);
if (finished1 && finished2) {
break;
}
if (timestamp1 >= timestamp2) {
timestamps.push_back(timestamp1);
std::cout << timestamp1 <<"\n" << std::flush;
} else {
timestamps.push_back(timestamp2);
std::cout << timestamp2 <<"\n" << std::flush;
}
}
}
#include <algorithm>
int main() {
std::thread t1(thread1);
std::thread t2(thread2);
std::thread t3(thread3);
t1.join();
t2.join();
t3.join();
std::cout << std::is_sorted(timestamps.begin(),timestamps.end());
}
The race condition, that result in loss of data, is due to a common misconception that assumes that notifying a condition variable results in an immediate wakeup of any thread that's listening on the condition variable, instantly. Additionally, it's also expected that the woken execution thread is guaranteed to immediately execute all of its assigned tasks, instantly, before it gets blocked on a mutex or a condition variable, again.
This is not the case.
All that notify_one() guarantees is that any execution thread that's listening on the condition variable will be woken up at some point after notify_one() is entered, which may be before or after notify_one() returns. That's it.
So, with that in mind, let's take a look at the sequence of events in thread1 (indentation adjusted):
timestamp1 = current_timestamp;
}
cv.notify_one();
timestamp1 is updated. The mutex is unlocked. The condition variable is notified.
thread3 is now scheduled to be woken up. But you have no guarantees, whatsoever, that not only did it wake up but it also managed to successfully relock the mutex. All you have is a nebulous promises from notify_one that this will happen. Eventually.
Meanwhile, back at the ranch:
std::unique_lock<std::mutex> lock(mtx);
while (timestamp2 >= current_timestamp) {
cv.wait(lock);
}
timestamp1 = current_timestamp;
thread1 managed to read the next timestamp and relock the mutex. Modern CPUs are fast. They can accomplish quite a lot before a context switch to another execution thread. This same thread discovers that the while condition is still true.
Based on the fact that the logic waits until the shared current_timestamp is less than the current value I conclude that the timestamps must be in decreasing order. Well, the last time around the block timestamp2, from the other thread was 1000 and current_timestamp was 900; now, the next current_timestamp is 800. It's still less than 1000, so we proceed on our merry way, updatingcurrent_timestamp from 900 to 800.
Meanwhile, thread3 is still having a nice dream, and only now beginning to wake up from its slumber, as a result of the prior notify_one (which is now just a distant memory to this execution thread). And thread3 missed the 900 value completely. It was replaced by 800. It's gone forever, never to be seen again.
This is not the only flaw in the shown code's logic. The missing data you're seeing is not due to some minor, single oversight, just a few lines of code that needs to be fixed. The logic is flawed in several different ways, that results in the missing data that you're seeing. You will need to completely rework the entire logic if you still want to use this multi-threaded approach to the described task.

Little bit confused about using Condition Variable for concurrent programming

I am currently learning about Game Programming with the book 'Game Engine Architecture' authored by Jason Gregory.
In this book, he showed an example with the reason for using 'Condition Variable'
[Without Condition Variable]
Queue g_queue;
pthread_mutex_t g_mutex; bool
g_ready = false;
void* ProducerThread(void*)
{
// keep on producing forever...
while (true)
{
pthread_mutex_lock(&g_mutex);
// fill the queue with data
ProduceDataInto(&g_queue);
g_ready = true;
pthread_mutex_unlock(&g_mutex);
// yield the remainder of my timeslice
// to give the consumer a chance to run pthread_yield();
}
return nullptr;
}
void* ConsumerThread(void*)
{
// keep on consuming forever...
while (true)
{
// wait for the data to be ready
while (true)
{
// read the value into a local,
// making sure to lock the mutex
pthread_mutex_lock(&g_mutex);
const bool ready = g_ready;
pthread_mutex_unlock(&g_mutex);
if (ready) break;
}
// consume the data
pthread_mutex_lock(&g_mutex);
ConsumeDataFrom(&g_queue);
g_ready = false;
pthread_mutex_unlock(&g_mutex);
// yield the remainder of my timeslice
// to give the producer a chance to run pthread_yield();
}
return nullptr;
}
In this example, he said 'Besides the fact that this example is somewhat contrived, there’s one big problem with it: The consumer thread spins in a tight loop, polling the value of g_ready'
I found that the function 'pthread_mutex_lock(&g_mutex)' is a blocking function that if the calling thread can't acquire the mutex, it falls to asleep.
Then, isn't that the consumer thread is not on the state of 'busy-wait'?
I mean, doesn't it spin at all if it does not acquire the mutex?
Though pthread_mutex_lock is a blocking function, the producer and consumer loop will spin tightly. Because either ProduceDataInto or ConsumeDataFrom executed and returns immediately, the mutex repeats lock/unlock after each calling ProduceDataInto/ProduceDataInto.
So there must be a queue-full Condition Variable to make the producer wait and a queue-empty Condition Variable to make the consumer wait.

How to make two threads take turns executing their respective critical sections after one thread ends

In modern C++ with STL threads I want to have two worker threads that take turns doing their work. Only one can be working at a time and each may only get one turn before the other takes a turn. I have this part working.
The added constraint is that one thread needs to keep taking turns after the other thread finishes. But in my code the remaining worker thread deadlocks after the first worker thread finishes. I don't understand why, given that the last things the first worker did was unlock and notify the condition variable, which should've woken the second one up. Here's the code:
{
std::mutex mu;
std::condition_variable cv;
int turn = 0;
auto thread_func = [&](int tid, int iters) {
std::unique_lock<std::mutex> lk(mu);
lk.unlock();
for (int i = 0; i < iters; i++) {
lk.lock();
cv.wait(lk, [&] {return turn == tid; });
printf("tid=%d turn=%d i=%d/%d\n", tid, turn, i, iters);
fflush(stdout);
turn = !turn;
lk.unlock();
cv.notify_all();
}
};
auto th0 = std::thread(thread_func, 0, 20);
auto th1 = std::thread(thread_func, 1, 25); // Does more iterations
printf("Made the threads.\n");
fflush(stdout);
th0.join();
th1.join();
printf("Both joined.\n");
fflush(stdout);
}
I don't know whether this is something I don't understand about concurrency in STL threads, or whether I just have a logic bug in my code. Note that there is a question on SO that's similar to this, but without the second worker having to run longer than the first. I can't find it right now to link to it. Thanks in advance for your help.
When one thread is done, the other will wait for a notification that nobody will send. When only one thread is left, you need to either stop using the condition variable or signal the condition variable some other way.

Synchronizing producer/consumer with a barrier

I have a producer thread which produces work for three consumer threads. When work has been produced, the producer thread waits until the consumer threads have finished handling the work. The producer thread then goes on handling the results.
#include <condition_variable>
#include <mutex>
#include <boost/thread/barrier.hpp>
#include <vector>
#include <queue>
std::condition_variable cond;
std::mutex mutex;
boost::barrier barrier(4);
std::vector<std::thread> workers;
std::queue<unsigned int> work;
std::queue<unsigned int> results;
void worker();
int main()
{
// 1 producer and 3 consumers
for(unsigned int i = 0; i < 3; i++)
workers.push_back(std::thread(worker));
// Wait here so the three workers can get to cond.wait();
barrier.wait();
std::unique_lock<std::mutex> lock(mutex);
while(true)
{
// Generate work
std::cout << "gen" << std::endl;
for(unsigned int i = 0; i < 10; i++)
work.push(i);
cond.notify_all();
lock.unlock();
barrier.wait();
// Handle the results
while(results.size() > 0)
results.pop();
lock.lock();
}
return 0;
}
void worker()
{
while(true)
{
std::unique_lock<std::mutex> lock(mutex);
while(results.size() == 0)
{
lock.unlock();
barrier.wait();
lock.lock();
cond.wait(lock);
}
// Get work
unsigned int next = work.front();
work.pop();
// Store the result
results.push(next);
lock.unlock();
}
}
The problem is that I need to make sure that all consumer threads have entered cond.wait(lock) before the producer thread starts its next iteration:
All 4 threads have reached the barrier. The barrier gets released and the threads can continue.
The producer thread locks the mutex before all consumer threads have reached cond.wait(lock). Thus at least one consumer thread is blocked by lock.lock().
The producer thread starts its next iteration, creates work and notifies the consumers. Since at least one consumer thread has not yet reached cond.wait(lock) the notify_all() will be missed by at least one consumer thread. These threads now wait for the next notify_all() - which will never arrive.
The next time the barrier is reached, at least one consumer thread still waits for the next notify_all(). Thus the barrier will not be unlocked and a deadlock occured.
How can I resolve this situation?
A condition_variable should be used together with a flag, to help prevent spurious wake ups. This same flag can also be used to check if the thread should wait at all or just go straight to work.
Add a bool go_to_work=false;, then we simply add it as a predicate in the call to wait and make sure we set/unset it from the main thread.
In main thread before calling notify_all we set our bool
go_to_work=true;
cond.notify_all();
In our worker thread we add the predicate to our wait call
cond.wait(lock, [](){ return go_to_work; });
Lastly, in our main thread we want to set the flag back to false after all work has been done.
barrier.wait();
lock.lock(); // We need to lock the mutex before modifying the bool
go_to_work=false;
lock.unlock();
//Handle result...
Now if a thread reaches the wait call after the main thread has set go_to_work=true it will not wait at all and simply go ahead and do the work. As a bonus this also guards against spurious wake-ups.

Condition Variable POSIX Thread : C/C++

I am learning Multithreading. With regard to
http://www.yolinux.com/TUTORIALS/LinuxTutorialPosixThreads.html#SCHEDULING
#include <stdio.h>
#include <stdlib.h>
#include <pthread.h>
pthread_mutex_t count_mutex = PTHREAD_MUTEX_INITIALIZER;
pthread_cond_t condition_var = PTHREAD_COND_INITIALIZER;
void *functionCount1();
void *functionCount2();
int count = 0;
#define COUNT_DONE 10
#define COUNT_HALT1 3
#define COUNT_HALT2 6
main()
{
pthread_t thread1, thread2;
pthread_create( &thread1, NULL, &functionCount1, NULL);
pthread_create( &thread2, NULL, &functionCount2, NULL);
pthread_join( thread1, NULL);
pthread_join( thread2, NULL);
printf("Final count: %d\n",count);
exit(0);
}
// Write numbers 1-3 and 8-10 as permitted by functionCount2()
void *functionCount1()
{
for(;;)
{
// Lock mutex and then wait for signal to relase mutex
pthread_mutex_lock( &count_mutex );
// Wait while functionCount2() operates on count
// mutex unlocked if condition varialbe in functionCount2() signaled.
pthread_cond_wait( &condition_var, &count_mutex );
count++;
printf("Counter value functionCount1: %d\n",count);
pthread_mutex_unlock( &count_mutex );
if(count >= COUNT_DONE) return(NULL);
}
}
// Write numbers 4-7
void *functionCount2()
{
for(;;)
{
pthread_mutex_lock( &count_mutex );
if( count < COUNT_HALT1 || count > COUNT_HALT2 )
{
// Condition of if statement has been met.
// Signal to free waiting thread by freeing the mutex.
// Note: functionCount1() is now permitted to modify "count".
pthread_cond_signal( &condition_var );
}
else
{
count++;
printf("Counter value functionCount2: %d\n",count);
}
pthread_mutex_unlock( &count_mutex );
if(count >= COUNT_DONE) return(NULL);
}
}
I want to know the control flow of the code.
As pthread_cond_wait - unlocks the mutex and waits for the condition variable cond to be signaled
What I understood about the control of flow is
1) Thread One,Two are created and thread1 is passed the control (considering Single Core Processor System)
2) When it encounters pthread_cond_wait( &condition_var, &count_mutex ); in thread1 routine void *functionCount1() - it releases the lock and goes to wait state passing control to thread2 void *functionCount1()
3) In thread2 variable count is checked and since it satisfies count < COUNT_HALT1 || count > COUNT_HALT2 - it signals thread1 and restarts is to increment count
4) Steps 2 to 3 is repeated which displays 1-3 by thread1
5) For count 4-7 the thread2 is in action and there is no switching between thread1 and thread2
6)For count 8-10 again steps 2-3 are repeated.
I want to know whether my understanding is correct? Does thread1 goes to sleep and thread2 wakes it up ( i.e threads are switched) for count value 1-3 and 8-10 i.e switching between threads happen 5 times ?
EDIT
My main concern to ask this question is to know if thread1 will go to sleep state when it encounters pthread_cond_wait( &condition_var, &count_mutex ); and won't be active again unless signalled by thread2 and only then increments count i.e. it is not going to increment 1-3 in one go rather for each increment , it has to wait for signal from thread2 only then it can proceed further
First: get the book by Butenhof, and study it. The page you
cite is incorrect in several places, and the author obviously
doesn't understand threading himself.
With regards to your questions: the first thing to say is that
you cannot know about the control flow of the code. That's
a characteristic of threading, and on modern processors, you'll
often find the threads really running in parallel, with one core
executing one thread, and another core another. And within each
thread, the processor may rearrange memory accesses in
unexpected ways. This is why you need mutexes, for example.
(You mention "considering single core processing system", but in
practice, single core general purpose systems don't exist any
more.)
Second, how the threads are scheduled is up to the operating
system. In your code, for example, functionCount2 could run
until completion before functionCount1 starts, which would
mean that functionCount1 would wait forever.
Third, a thread in pthread_cond_wait may wake up spuriously.
It is an absolute rule that pthread_cond_wait be in a loop,
which checks whatever condition you're actually waiting for.
Maybe something like:
while ( count > COUNT_HALT1 && count < COUNT_HALT2 ) {
pthread_cond_wait( &condition_var, &count_mutex );
}
Finally, at times, you're accessing count in a section not
protected by the mutex. This is undefined behavior; all
accesses to count must be protected. In your case, the
locking and unlocking should probably be outside the program
loop, and both threads should wait on the conditional
variable. (But it's obviously an artificial situation—in
practice, there will almost always be a producer thread and
a consumer thread.)
In the ideal world, yes, but, in practice, not quite.
You can't predict which tread takes control first. Yes, it's likely to be thread1, but still not guaranteed. It is the first racing condition in your code.
When thread2 takes control, most likely it will finish without stopping. Regardless of how many CPUs you have. The reason is that it has no place to yield unconditionally. The fact you release mutex, doesn't mean any1 can get a lock on it. Its the second racing condition in your code.
So, the thread1 will print 11, and that is the only part guaranteed.
1) Threads are created. Control is not passed to thread1, it's system scheduler who decides which thread to execute. Both threads are active, both should receive processor time, but the order is not determined. There might be several context switches, you don't really control this.
2) Correct, thread1 comes to waiting state, thread2 continues working. Again, control is not passed explicitly.
3) Yes, thread2 notifies condition variable, so thread1 will awake and try to reacquire the mutex. Control does not go immediately to thread1.
In general you should understand that you can't really control which thread to execute; it's job os the system scheduler, who can put as many context switches as it wants.
UPD: With condition variables you can control the order of tasks execution within multithreading environment. So I think your understanding is more or less correct: thread1 is waiting on condition variable for signal from thread2. When signal is received thread1 continues execution (after it reacquires the mutex). But switching between threads - there might be many of them, 5 is just a theoretical minimum.