Condition Variable notify_one notify_all - c++

I'm trying to learn the condition variables, and I'm stuck at the following example. I thought that notify_one on consumers should unlock only one waiting consumer. But after starting it repeatedly it seems to me that this isn't the case. I've changed notify_one into notify_all and haven't noticed a change in behavior. After the producer calls notify_one on consumers I can see Get… being written on screen by more then one consumer.
Why is this happening?
#include <iostream> // std::cout
#include <thread> // std::thread
#include <mutex> // std::mutex, std::unique_lock
#include <condition_variable> // std::condition_variable
#include <chrono>
std::mutex mtx;
std::condition_variable produce,consume;
int cargo = 0; // shared value by producers and consumers
void consumer () {
std::unique_lock<std::mutex> lck(mtx);
while (cargo==0) consume.wait(lck);
std::cout << "Get" << cargo << " "<< std::this_thread::get_id() << '\n';
cargo--;
produce.notify_one();
}
void producer (int id) {
std::unique_lock<std::mutex> lck(mtx);
while (cargo!=0) produce.wait(lck);
std::cout << "Push" << id << " "<< std::this_thread::get_id() << '\n';
cargo += id;
consume.notify_one();
}
void c () {
while(1) {
consumer();
std::this_thread::sleep_for(std::chrono::milliseconds(1000));
}
}
void p(int n) {
while(1) {
producer(n);
std::this_thread::sleep_for(std::chrono::milliseconds(1000));
}
}
int main ()
{
std::thread consumers[5],producers[5];
for (int i=0; i<5; ++i) {
consumers[i] = std::thread(c);
producers[i] = std::thread(p,i+1);
}
for (int i=0; i<5; ++i) {
producers[i].join();
consumers[i].join();
}
return 0;
}

Related

notification from producer thread is not reaching to consumer thread once single item is produced

in below code snippet it looks like notification from producer thread to consumer thread is not reaching once producer produce an single item and due to this behavior once producer has finished generating items equivalent to buffer size then only consumer has started consuming items . Can anybody suggest How we should approach to fix this issue using semaphore.
#include <iostream>
#include <queue>
#include <semaphore.h>
#include <thread>
#include <functional>
const int BUFFER_SIZE = 3;
class Buffer {
public:
sem_t sem_full;
sem_t sem_empty;
std::queue<int> buffer;
Buffer() {
sem_init(&sem_full, 0, BUFFER_SIZE);
sem_init(&sem_empty, 0, 0);
}
void producer() {
while (true) {
sem_wait(&sem_full);
int item = rand() % 10;
buffer.push(item);
std::cout << "Producer added " << item << std::endl;
sem_post(&sem_empty);
if (buffer.size() == BUFFER_SIZE) {
std::cout << "Buffer is full, terminating producer thread" << std::endl;
return;
}
}
}
void consumer() {
while (true) {
sem_wait(&sem_empty);
int item = buffer.front();
buffer.pop();
std::cout << "Consumer removed " << item << std::endl;
sem_post(&sem_full);
if (buffer.empty()) {
std::cout << "Buffer is empty, terminating consumer thread" << std::endl;
return;
}
}
}
};
int main() {
Buffer buffer;
std::thread producer(std::bind(&Buffer::producer, &buffer));
std::thread consumer(std::bind(&Buffer::consumer, &buffer));
producer.join();
consumer.join();
return 0;
}
you need to use binary semaphore here to achieve this behavior without using condition variable to synchronize this.
#include <iostream>
#include <queue>
#include <semaphore.h>
#include <thread>
#include <functional>
#include <condition_variable>
#include <mutex>
#include <atomic>
const int BUFFER_SIZE = 4;
class Buffer {
public:
sem_t sem_full;
sem_t sem_empty;
std::queue<int> buffer;
std::condition_variable cv;
std::mutex m;
int buffer_full_count {0};
Buffer() {
sem_init(&sem_full, 0, 1);
sem_init(&sem_empty, 0, 0);
}
void producer() {
while (true) {
sem_wait(&sem_full);
if (buffer_full_count == BUFFER_SIZE) {
std::cout << "Buffer is full, terminating producer thread" << std::endl;
return;
}
std::unique_lock <std::mutex> lock(m);
int item = rand() % 10;
buffer.push(item);
buffer_full_count++;
std::cout << "Producer added " << item << std::endl;
std::this_thread::sleep_for(std::chrono::seconds(1));
sem_post(&sem_empty);
}
}
void consumer() {
while (buffer_full_count != BUFFER_SIZE) {
sem_wait(&sem_empty);
std::unique_lock <std::mutex> lock(m);
int item = buffer.front();
buffer.pop();
std::cout << "Consumer removed " << item << std::endl;
sem_post(&sem_full);
}
}
};
int main() {
Buffer buffer;
std::thread producer(std::bind(&Buffer::producer, &buffer));
std::thread consumer(std::bind(&Buffer::consumer, &buffer));
producer.join();
consumer.join();
return 0;
}

how to implement wait() for producer/consumer using C++17 (mutex or shared_mutex)

I have a set of consumers generating data, each with their own vector. A consumer, in this MWE the main function, should wait for one of the producers to create something, and then once it wakes up process all queues and then wait again.
With a mutex, the critical section is easy to create, but how to wait for any one of the producers to signal that data is available, and wake up main. The producers do not need to block each other, they could each add to their own queue, but each producer must block the consumer, and the consumer must block all the producers.
Here is my MWE, which is only missing the equivalent of Java obj.wait() and object.notify().
#include <thread>
#include <mutex>
#include <iostream>
#include <vector>
#include <unistd.h>
using namespace std;
mutex m;
vector<int> event1;
int count1 = 0;
void producer1() {
while(true) {
m.lock(); // enter critical section
cout << "producer1: " << count1 << '\n';
event1.push_back(count1++);
// TODO: wakeup main consumer loop
// in Java this would be obj.notify();
m.unlock();
sleep(1);
}
}
vector<int> event2;
int count2 = 0;
void producer2() {
while(true) {
m.lock(); // enter critical section
cout << "producer2: " << count2 << '\n';
event2.push_back(count2++);
// TODO: wakeup main consumer loop
// in Java this would be obj.notify();
m.unlock(); // enter critical section
sleep(2);
}
}
int main() {
thread t1(producer1);
thread t2(producer2);
while (true) {
// TODO: obj.wait();
// inside the critical section, get all the data from each producer
m.lock();
for (int i = 0; i < event1.size(); i++)
cout << "consumer1: " << event1[i] << '\n';
for (int i = 0; i < event2.size(); i++)
cout << "consumer2: " << event2[i] << '\n';
event1.clear();
event2.clear();
m.unlock();
sleep(10);
}
}

Thread mutex unlock order

Consider following code:
#include <iostream>
#include <mutex>
#include <thread>
#include <vector>
#include <chrono>
using namespace std;
const int SIZE = 10;
mutex myMutex;
std::vector<int> strange;
void add_to_vector(int i) {
lock_guard<mutex> g(myMutex);
if (strange.size() < 100) {
cout << "Dodaje do bazy" << i << std::endl;
strange.push_back(i);
cout << "Dodałem do bazy" << i << std::endl;
}
}
void f(int n) {
while (strange.size() < 100)
{
add_to_vector(n);
std::this_thread::sleep_for(std::chrono::microseconds(100));
}
}
int main()
{
thread t1(f, 1); // 0-9
thread t2(f, 2); // 10-19
thread t3(f, 3); // 20-29
thread t4(f, 4); // 30-39
thread t5(f, 5); // 40-49
t1.join();
t2.join();
t3.join();
t4.join();
t5.join();
for (auto a : strange) {
std::cout << a << ", ";
}
return 0;
}
Is threads waiting to mutex unlock, lining up ? It seems that after unlocking , access to resources gets a random thread. How to ensure that they get access to the resource threads in FIFO order ?

Producer-Consumer: Lost Wake-up issue

I was trying to write code for Producer-Consumer problem. Below code works fine most of the time but stuck sometimes because of "Lost Wake-up" (i guess). I tried thread sleep() but it didn't work. What modification is needed to handle this case in my code? Is semaphore can be helpful here ? If yes, how will i implement them here ?
#include <boost/thread/thread.hpp>
#include <boost/thread/mutex.hpp>
#include <iostream>
using namespace std;
int product = 0;
boost::mutex mutex;
boost::condition_variable cv;
boost::condition_variable pv;
bool done = false;
void consumer(){
while(done==false){
//cout << "start c" << endl
boost::mutex::scoped_lock lock(mutex);
cv.wait(lock);
//cout << "wakeup c" << endl;
if (done==false)
{
cout << product << endl;
//cout << "notify c" << endl;
pv.notify_one();
}
//cout << "end c" << endl;
}
}
void producer(){
for(int i=0;i<10;i++){
//cout << "start p" << endl;
boost::mutex::scoped_lock lock(mutex);
boost::this_thread::sleep(boost::posix_time::microseconds(50000));
++product;
//cout << "notify p" << endl;
cv.notify_one();
pv.wait(lock);
//cout << "wakeup p" << endl;
}
//cout << "end p" << endl;
cv.notify_one();
done = true;
}
int main()
{
int t = 1000;
while(t--){
/*
This is not perfect, and is prone to a subtle issue called the lost wakeup (for example, producer calls notify()
on the condition, but client hasn't really called wait() yet, then both will wait() indefinitely.)
*/
boost::thread consumerThread(&consumer);
boost::thread producerThread(&producer);
producerThread.join();
consumerThread.join();
done =false;
//cout << "process end" << endl;
}
cout << "done" << endl;
getchar();
return 0;
}
Yes, you want a way to know (in the consumer) that you "missed" a signal. A semaphore can help. There's more than one way to skin a cat, so here's my simple take on it (using just c++11 standard library features):
class semaphore
{
private:
std::mutex mtx;
std::condition_variable cv;
int count;
public:
semaphore(int count_ = 0) : count(count_) { }
void notify()
{
std::unique_lock<std::mutex> lck(mtx);
++count;
cv.notify_one();
}
void wait() { return wait([]{}); } // no-op action
template <typename F>
auto wait(F&& func = []{}) -> decltype(std::declval<F>()())
{
std::unique_lock<std::mutex> lck(mtx);
while(count == 0){
cv.wait(lck);
}
count--;
return func();
}
};
For convenience, I added a convenience wait() overload that takes a function to be executed under the lock. This makes it possible for the consumer to operate the 'semaphore' without ever manually operating the lock (and still get the value of product without data-races):
semaphore sem;
void consumer() {
do {
bool stop = false;
int received_product = sem.wait([&stop] { stop = done; return product; });
if (stop)
break;
std::cout << received_product << std::endl;
std::unique_lock<std::mutex> lock(processed_mutex);
processed_signal.notify_one();
} while(true);
}
A fully working demo: Live on Coliru:
#include <condition_variable>
#include <iostream>
#include <mutex>
#include <thread>
#include <cassert>
class semaphore
{
private:
std::mutex mtx;
std::condition_variable cv;
int count;
public:
semaphore(int count_ = 0) : count(count_) { }
void notify()
{
std::unique_lock<std::mutex> lck(mtx);
++count;
cv.notify_one();
}
void wait() { return wait([]{}); } // no-op action
template <typename F>
auto wait(F&& func = []{}) -> decltype(std::declval<F>()())
{
std::unique_lock<std::mutex> lck(mtx);
while(count == 0){
cv.wait(lck);
}
count--;
return func();
}
};
semaphore sem;
int product = 0;
std::mutex processed_mutex;
std::condition_variable processed_signal;
bool done = false;
void consumer(int check) {
do {
bool stop = false;
int received_product = sem.wait([&stop] { stop = done; return product; });
if (stop)
break;
std::cout << received_product << std::endl;
assert(++check == received_product);
std::unique_lock<std::mutex> lock(processed_mutex);
processed_signal.notify_one();
} while(true);
}
void producer() {
std::unique_lock<std::mutex> lock(processed_mutex);
for(int i = 0; i < 10; ++i) {
++product;
sem.notify();
processed_signal.wait(lock);
}
done = true;
sem.notify();
}
int main() {
int t = 1000;
while(t--) {
std::thread consumerThread(&consumer, product);
std::thread producerThread(&producer);
producerThread.join();
consumerThread.join();
done = false;
std::cout << "process end" << std::endl;
}
std::cout << "done" << std::endl;
}
You seems to ignore that the variable done is also a shared state, to the same extend as product. Which can lead to several races conditions. In your case, I see at least one scenario where consumerThread make no progress:
The loop execute has intended
consumer executes, and is waiting at cv.wait(lock);
producer has finished the for loop, and notify consumer and is preempted
consumer wakes up, read "done==false", output product, read done == false again, wait on the condition
producer set done to true and exit
consumer is stuck forever
To avoid these kind of issues you should be holding a lock when reading or writing done. Btw your implementation is quite sequential, ie the producer and the consumer can only process a single piece of data at the time...

Understanding the example of using std::condition_variable

There is example of using condition_variable taken from cppreference.com:
#include <condition_variable>
#include <mutex>
#include <thread>
#include <iostream>
#include <queue>
#include <chrono>
int main()
{
std::queue<int> produced_nums;
std::mutex m;
std::condition_variable cond_var;
bool done = false;
bool notified = false;
std::thread producer([&]() {
for (int i = 0; i < 5; ++i) {
std::this_thread::sleep_for(std::chrono::seconds(1));
std::lock_guard<std::mutex> lock(m);
std::cout << "producing " << i << '\n';
produced_nums.push(i);
notified = true;
cond_var.notify_one();
}
std::lock_guard<std::mutex> lock(m);
notified = true;
done = true;
cond_var.notify_one();
});
std::thread consumer([&]() {
while (!done) {
std::unique_lock<std::mutex> lock(m);
while (!notified) { // loop to avoid spurious wakeups
cond_var.wait(lock);
}
while (!produced_nums.empty()) {
std::cout << "consuming " << produced_nums.front() << '\n';
produced_nums.pop();
}
notified = false;
}
});
producer.join();
consumer.join();
}
If variable done comes true before the consumer thread is started, the consumer thread will not get any message. Indeed, sleep_for(seconds(1)) almost avoids such situation, but could it be possible in theory (or if don't have sleep in code)?
In my opinion correct version should look like this to force running consumer loop at least once:
std::thread consumer([&]() {
std::unique_lock<std::mutex> lock(m);
do {
while (!notified || !done) { // loop to avoid spurious wakeups
cond_var.wait(lock);
}
while (!produced_nums.empty()) {
std::cout << "consuming " << produced_nums.front() << '\n';
produced_nums.pop();
}
notified = false;
} while (!done);
});
Yes, you are absolutely right: there is a (remote) possibility that the consumer thread will not start running until after done has been set. Further, the write to done in the producer thread and the read in the consumer thread produce a race condition, and the behavior is undefined. Same problem in your version. Wrap the mutex around the entire loop in each function. Sorry, don't have the energy to write the correct code.