C++ Holding a number of threads - c++

I'm new to C++ (on Windows) and threading and I'm currently trying to find a solution to my problem using mutexes, semaphores and events.
I'm trying to create a Barrier class with a constructor and a method called Enter. The class Barrier with it's only method Enter is supposed to hold off any thread that enters it, until a number of thread have reached that method. The number of thread to wait for it recieved at the contructor.
My problem is how do I use the locks to create that effect? what I need is something like a reversed semaphore, that holds threads until a count has been reached and not like the regular semaphore works that lets threads in until a count is reached.
Any ideas as to how to go about this would be great.
Thanks,
Netanel.

Maybe:
In the ctor, store the limit count and create an empty semaphore.
When a thread calls Enter, lock a mutex first so you can twiddle inside safely. Inc a thread count toward the limit count. If the limit has not yet been reached, release the mutex and wait on the semaphore. If the limit is reached, signal the semaphore[limit-1] times in a loop, zero the thread count, (ready for next time), release the mutex and return from Enter(). Any threads that were waiting on the semaphore, and are now ready/running, should just return from their 'Enter' call.
The mutex prevents any released thread that loops around from 'getting in again' until all the threads that called 'Enter' and waited have been set running and the barrier is reset.

You can implement it with condition variable.
Here is an example:
I declare 25 threads and launch them doing the WorkerThread function.
The condition I am checking to block/unblick the threads is whether the number of threads in the section is less than 2.
(I have added some asserts to prove what my coode does).
My code is simply sleeping in the critical section and after I decrease the number of threads in the critical section.
I also added a mutex for the cout to have clean messages.
#include
#include
#include
#include
#include
#include
#include /* assert */
using namespace std;
std::mutex m;
atomic<int> NumThreadsInCritialSection=0;
int MaxNumberThreadsInSection=2;
std::condition_variable cv;
mutex coutMutex;
int WorkerThread()
{
// Wait until main() sends data
{
std::unique_lock<std::mutex> lk(m);
cv.wait(lk, []{return NumThreadsInCritialSection<MaxNumberThreadsInSection;});
}
assert (NumThreadsInCritialSection<MaxNumberThreadsInSection);
assert (NumThreadsInCritialSection>=0);
NumThreadsInCritialSection++;
{
std::unique_lock<std::mutex> lk(coutMutex);
cout<<"NumThreadsInCritialSection= "<<NumThreadsInCritialSection<<endl;
}
std::this_thread::sleep_for(std::chrono::seconds(5));
NumThreadsInCritialSection--;
{
std::unique_lock<std::mutex> lk(coutMutex);
cout<<"NumThreadsInCritialSection= "<<NumThreadsInCritialSection<<endl;
}
cv.notify_one();
return 0;
}
int main()
{
vector<thread> vWorkers;
for (int i=0;i<25;++i)
{
vWorkers.push_back(thread(WorkerThread));
}
for (auto j=vWorkers.begin(); j!=vWorkers.end(); ++j)
{
j->join();
}
return 0;
}
Hope that helps, tell me if you have any questions, I can comment or change my code.

Pseudocode outline might look like this:
void Enter()
{
Increment counter (atomically or with mutex)
if(counter >= desired_count)
{
condition_met = true; (protected if bool writes aren't atomic on your architecture)
cond_broadcast(blocking_cond_var);
}
else
{
Do a normal cond_wait loop-plus-predicate-check (waiting for the broadcast and checking condition_met each iteration to protect for spurious wakeups).
}
}

Related

Two questions on std::condition_variables

I have been trying to figure out std::condition_variables and I am particularly confused by wait() and whether to use notify_all or notify_one.
First, I've written some code and attached it below. Here's a short explanation: Collection is a class that holds onto a bunch of Counter objects. These Counter objects have a Counter::increment() method, which needs to be called on all the objects, over and over again. To speed everything up, Collection also maintains a thread pool to distribute the work over, and sends out all the work with its Collection::increment_all() method.
These threads don't need to communicate with each other, and there are usually many more Counter objects than there are threads. It's fine if one thread processes more than Counters than others, just as long as all the work gets done. Adding work to the queue is easy and only needs to be done in the "main" thread. As far as I can see, the only bad thing that can happen is if other methods (e.g. Collection::printCounts) are allowed to be called on the counters in the middle of the work being done.
#include <iostream>
#include <thread>
#include <vector>
#include <mutex>
#include <condition_variable>
#include <queue>
class Counter{
private:
int m_count;
public:
Counter() : m_count(0) {}
void increment() {
m_count ++;
}
int getCount() const { return m_count; }
};
class Collection{
public:
Collection(unsigned num_threads, unsigned num_counters)
: m_shutdown(false)
{
// start workers
for(size_t i = 0; i < num_threads; ++i){
m_threads.push_back(std::thread(&Collection::work, this));
}
// intsntiate counters
for(size_t j = 0; j < num_counters; ++j){
m_counters.emplace_back();
}
}
~Collection()
{
m_shutdown = true;
for(auto& t : m_threads){
if(t.joinable()){
t.join();
}
}
}
void printCounts() {
// wait for work to be done
std::unique_lock<std::mutex> lk(m_mtx);
m_work_complete.wait(lk); // q2: do I need a while lop?
// print all current counters
for(const auto& cntr : m_counters){
std::cout << cntr.getCount() << ", ";
}
std::cout << "\n";
}
void increment_all()
{
std::unique_lock<std::mutex> lock(m_mtx);
m_work_complete.wait(lock);
for(size_t i = 0; i < m_counters.size(); ++i){
m_which_counters_have_work.push(i);
}
}
private:
void work()
{
while(!m_shutdown){
bool action = false;
unsigned which_counter;
{
std::unique_lock<std::mutex> lock(m_mtx);
if(m_which_counters_have_work.size()){
which_counter = m_which_counters_have_work.front();
m_which_counters_have_work.pop();
action = true;
}else{
m_work_complete.notify_one(); // q1: notify_all
}
}
if(action){
m_counters[which_counter].increment();
}
}
}
std::vector<Counter> m_counters;
std::vector<std::thread> m_threads;
std::condition_variable m_work_complete;
std::mutex m_mtx;
std::queue<unsigned> m_which_counters_have_work;
bool m_shutdown;
};
int main() {
int num_threads = std::thread::hardware_concurrency()-1;
int num_counters = 10;
Collection myCollection(num_threads, num_counters);
myCollection.printCounts();
myCollection.increment_all();
myCollection.printCounts();
myCollection.increment_all();
myCollection.printCounts();
return 0;
}
I compile this on Ubuntu 18.04 with g++ -std=c++17 -pthread thread_pool.cpp -o tp && ./tp I think the code accomplishes all of those objectives, but a few questions remain:
I am using m_work_complete.wait(lk) to make sure the work is finished before I start printing all the new counts. Why do I sometimes see this written inside a while loop, or with a second argument as a lambda predicate function? These docs mention spurious wake ups. If a spurious wake up occurs, does that mean printCounts could prematurely print? If so, I don't want that. I just want to ensure the work queue is empty before I start using the numbers that should be there.
I am using m_work_complete.notify_all instead of m_work_complete.notify_one. I've read this thread, and I don't think it matters--only the main thread is going to be blocked by this. Is it faster to use notify_one just so the other threads don't have to worry about it?
std::condition_variable is not really a condition variable, it's more of a synchronization tool for reaching a certain condition. What that condition is is up to the programmer, and it should still be checked after each condition_variable wake-up, since it can wake-up spuriously, or "too early", when the desired condition isn't yet reached.
On POSIX systems, condition_variable::wait() delegates to pthread_cond_wait, which is susceptible to spurious wake-up (see "Condition Wait Semantics" in the Rationale section). On Linux, pthread_cond_wait is in turn implemented via a futex, which is again susceptible to spurious wake-up.
So yes you still need a flag (protected by the same mutex) or some other way to check that the work is actually complete. A convenient way to do this is by wrapping the check in a predicate and passing it to the wait() function, which would loop for you until the predicate is satisfied.
notify_all unblocks all threads waiting on the condition variable; notify_one unblocks just one (or at least one, to be precise). If there are more than one waiting threads, and they are equivalent, i.e. either one can handle the condition fully, and if the condition is sufficient to let just one thread continue (as in submitting a work unit to a thread pool), then notify_one would be more efficient since it won't unblock other threads unnecessarily for them to only notice no work to be done and going back to waiting. If you ever only have one waiter, then there would be no difference between notify_one and notify_all.
It's pretty simple: Use notify() when;
There is no reason why more than one thread needs to know about the event. (E.g., use notify() to announce the availability of an item that a worker thread will "consume," and thereby make the item unavailable to other workers)*AND*
There is no wrong thread that could be awakened. (E.g., you're probably safe if all of the threads are wait()ing in the same line of the same exact function.)
Use notify_all() in all other cases.

Why the program get deadlocked if I comment the "while" block? How does the "yield" line in it effect? [duplicate]

This question already has answers here:
condition variable - why calling pthread_cond_signal() before calling pthread_cond_wait() is a logical error?
(5 answers)
Closed 5 years ago.
I'm learning condition_variable and running some examples. I'm curious that why the following code get deadlock if I comment the block. It's a simple consumer and producer example using condition_variable. I think it's a deadlock problem, isn't it?
#include <iostream>
#include <thread>
#include <mutex>
#include <condition_variable>
using namespace std;
mutex mtx;
condition_variable cv;
int cargo = 0;
bool shipment_available()
{
return cargo != 0;
}
void consume(int cnt)
{
for (int i = 0; i < cnt; i++)
{
unique_lock<mutex> lck(mtx);
cv.wait(lck, shipment_available);
printf("%d\n", cargo);
cargo = 0;
}
}
int main()
{
thread consumer_thread(consume, 10);
for (int i = 0; i < 10; i++)
{
//while (shipment_available()) // Dead lock without this block
//{
// std::this_thread::yield();
//}
unique_lock<mutex> lck(mtx);
cargo = i + 1;
cv.notify_one();
}
consumer_thread.join();
}
It runs well if I uncomment the block.
So walk through this highly likely possibility:
main() starts up the consumer thread.
Before the thread can acquire the mutex, main() locks it, increments cargo, and fires the notification, then releases the mutex by scope-rotation ten times.
main() now runs down to join the consumer thread.
Consumer finally acquires the mutex, and prepares to wait on the condition variable for the given predicate (non-zero cargo).
The predicate is already true, so no wait is performed.
The consumer zeros the cargo, then releases the mutex by scope-rotation.
The consumer loops for the second time, locks the mutex, then checks the predicate, only now the predicate is false because cargo is indeed zero, so it waits on the condition variable (releasing the mutex in the process) for a signal that will never come. The only thing that ever sent that signal is main(), and it is just camped out waiting for the consumer thread to join up, which can now never happen.
In short, you seem to be of the belief that condition variables signals stack up. That simply isn't the case. if no one is actively waiting on a notification at the time it is posted, it is simply lost to the ether.
Finally, and this is important, your "fix" is completely wrong in itself. The shipment_available function examines predicate data (i.e. data that must be protected by the mutex enshrouding it, not only for modification, but likewise for examination). Checking that without the mutex locked in main is a recipe for a race condition.
My suggestion is to reduce cargo by one rather than zeroing it out. Alternatively you could increment i by the value of cargo before zeroing it, thereby hastening i's ascent to the cnt limit, but you appear to be using an ascending cargo amount, so you'll have to make other adjustments accordingly.

providing critical section for threads in c++

In this c++ program , I have created 10 threads which race to each other to acquire critical section.For this mean I'm using conditional variable as below code.The dispatcher in this program will grant one thread at a time to enter critical section.But there is a subtle problem. when dispatcher grant to a thread to enter , it will set ready_pipe variable true. If one new thread comes at this time (before consumer set ready_pipe=flase) newcomer thread will across the critical section without permission.
#include <iostream>
#include <string>
#include <thread>
#include <mutex>
#include <condition_variable>
using namespace std;
std::condition_variable con_var_pipe;
bool ready_pipe = false;
std::mutex pipe_mutex;
bool critical_section_is_free=true;
void dispatcher()
{
while(true)
{
if(critical_section_is_free)
{
critical_section_is_free=false;
// send signal to a thread to enter critical section
std::lock_guard<std::mutex> lk(pipe_mutex);
ready_pipe = true;
con_var_pipe.notify_one();
}
}
}
void consumer()
{
std::unique_lock<std::mutex> lk(pipe_mutex);
/* The Problem is Here at below line.When a new thread comes,
it will pass through this block because it see ready_pipe is true!!!*/
con_var_pipe.wait(lk, [] {return ready_pipe;});
/// critical section starts
ready_pipe=false;
/// here accessing pipe is occurring .
/// critical section ends
critical_section_is_free=true;
}
int main()
{
std::thread allThreads[10];
for(int i = 0 ; i<10 ; i++)
{
allThreads[i]=std::thread(consumer);
}
thread disp(dispatcher);
for(int i = 0 ; i< 6 ; i++)
{
allThreads[i].join();
}
disp.join();
return 0;
}
Further more , this code is also deficient because of a while(true) statement in dispatcher function and make busy waiting.
So the questions are :
1-how to create mutual exclusion when new thread comes.
2-how to avoid busy waiting in dispatcher() function.
3-and how threads be served in order as they come and register in wait().
Just use two counters and a boolean to implement a basic "take a number" scheme.
One counter, released_thread, indicates which thread may proceed. This is logically equivalent to the "now serving" indicator.
The other, next_waiter, indicates which thread waits next. This is logically equivalent to the next number that will be taken
A boolean indicates whether the thread is permitted to proceed and when it is finished, so the executive knows when to call the next number.
The algorithms are as follows:
To wait:
Acquire the lock.
Note the value of the next_waiter variable and increment it.
Broadcast (notify all) the condition variable.
Wait on the condition variable until the boolean is true and the released_thread counter is equal to the value noted in step 2.
Release the lock.
When finished, acquire the lock, set the boolean to false, broadcast the condition variable, and release the lock.
Executive:
Acquire the lock.
Wait on the condition variable until the boolean is false and next_waiter is not equal to released_thread.
Increment released_thread and set the boolean to true.
Broadcast the condition variable.
Go to step 2.

Why pthread_condition variable getting hanged?

I just started learning pthread condition variable. But below code is not working as expected.
#include<iostream>
#include<pthread.h>
using namespace std;
pthread_mutex_t mutex = PTHREAD_MUTEX_INITIALIZER;
pthread_cond_t count_threshold_cv = PTHREAD_COND_INITIALIZER;
int count=0;
void *inc_func(void *arg)
{
pthread_mutex_lock(&mutex);
int c;
while(1)
{
cin>>c;
if(c==8){
pthread_cond_signal(&count_threshold_cv);break;}
}
cout<<"inc count reached 8"<<endl;
pthread_mutex_unlock(&mutex);
}
void *watch(void *arg)
{
pthread_mutex_lock(&mutex);
while(1){
pthread_cond_wait(&count_threshold_cv,&mutex);
break;
}
cout<<"This executed"<<endl;
pthread_mutex_unlock(&mutex);
}
int main()
{
pthread_t id[26];
pthread_create(&id[0],NULL,inc_func,NULL);
pthread_create(&id[1],NULL,watch,NULL);
int i;
for(i=0;i<2;i++)
{
pthread_join(id[i],NULL);
}
}
when the input is 8 this code gets hanged at "inc count reached 8? I am not able to understand.
Where is my understanding wrong?
The correct solution to this is to make the watch thread only wait if the condition it is waiting for has not occured yet.
The condition appears to be c == 8 (since that is what is signalled), so you will need to make the c variable global so that it is shared between the threads, then change the watch thread to do:
void *watch(void *arg)
{
pthread_mutex_lock(&mutex);
while (c != 8) {
pthread_cond_wait(&count_threshold_cv, &mutex);
}
cout<<"This executed"<<endl;
pthread_mutex_unlock(&mutex);
return 0;
}
Now it doesn't matter which thread runs first: your code is correct either way. This is the right way to use conditional variables: in general, the waiter should do:
pthread_mutex_lock(&mutex);
while (!condition)
pthread_cond_wait(&cond, &mutex);
/* ... */
and the signaller should do:
pthread_mutex_lock(&mutex);
/* ... something that potentially makes condition true ... */
pthread_cond_broadcast(&cond);
pthread_mutex_unlock(&mutex);
The important thing here is that pthread_cond_signal will unblock at least one of the threads that are blocked on that condition variable (meaning which are currently blocked on a call to pthread_cond_wait on that same condition variable). If at the moment when one thread calls pthread_cond_signal there is no other thread waiting on that condition then basically nothing happens.
Keeping this in mind, the flow of your program is something like this:
create and start first thread;
first thread calls inc_func(), which locks the mutex before anything else;
inc_func() keeps waiting for the number 8 to be entered, keeping the mutex locked all this time;
sometime during this, but most times probably after the inc_func managed to lock the mutex, the second thread is created;
the second thread also tries to lock the mutex right at the start of the function, and is blocked because the first thread already has it locked;
at some point, you enter 8 and the condition gets signaled from thread 1; thread 2 is not waiting on this condition yet, so it remains blocked trying to lock the mutex;
the first thread finally releases the mutex, so thread 2 locks it and then blocks on pthread_cond_wait.
At this point, thread 1 has already finished, thread 2 is blocked waiting for the condition to be signaled, and the main thread is waiting for it to finish. There is nobody else to signal that condition, so you have a hang.
For a quick fix that will probably work most of the times, you could try changing the order in which you start the threads(start the watch thread first). But keep in mind and understand why I used bold for probably and most of the times.
The correct way to fix this would be to rethink your locking strategy: lock the mutexes in the smallest scope possible and keep them locked for the shortest time possible.
Swap the thread's execution sequence:
pthread_create(&id[1],NULL,watch,NULL);
pthread_create(&id[0],NULL,inc_func,NULL);
If you run thread0 as first, thread1 never gets past mutex lock, so it doesn't start waiting. Than thread0 ends, and only then thread1 executes pthread_cond_wait(), but there's no thread to do signal.
If you start thread1 first, it gets to the waiting part.

(C++ Threads): Creating worker threads that will be listening to jobs and executing them concurrently when wanted

Suppose we have two workers. Each worker has an id of 0 and 1. Also suppose that we have jobs arriving all the time, each job has also an identifier 0 or 1 which specifies which worker will have to do this job.
I would like to create 2 threads that are initially locked, and then when two jobs arrive, unlock them, each of them does their job and then lock them again until other jobs arrive.
I have the following code:
#include <iostream>
#include <thread>
#include <mutex>
using namespace std;
struct job{
thread jobThread;
mutex jobMutex;
};
job jobs[2];
void executeJob(int worker){
while(true){
jobs[worker].jobMutex.lock();
//do some job
}
}
void initialize(){
int i;
for(i=0;i<2;i++){
jobs[i].jobThread = thread(executeJob, i);
}
}
int main(void){
//initialization
initialize();
int buffer[2];
int bufferSize = 0;
while(true){
//jobs arrive here constantly,
//once the buffer becomes full,
//we unlock the threads(workers) and they start working
bufferSize = 2;
if(bufferSize == 2){
for(int i = 0; i<2; i++){
jobs[i].jobMutex.unlock();
}
}
break;
}
}
I started using std::thread a few days ago and I'm not sure why but Visual Studio gives me an error saying abort() has been called. I believe there's something missing however due to my ignorance I can't figure out what.
I would expect this piece of code to actually
Initialize the two threads and then lock them
Inside the main function unlock the two threads, the two threads will do their job(in this case nothing) and then they will become locked again.
But it gives me an error instead. What am I doing wrong?
Thank you in advance!
For this purpose you can use boost's threadpool class.
It's efficient and well tested. opensource library instead of you writing newly and stabilizing it.
http://threadpool.sourceforge.net/
main()
{
pool tp(2); //number of worker threads-currently its 2.
// Add some tasks to the pool.
tp.schedule(&first_task);
tp.schedule(&second_task);
}
void first_task()
{
...
}
void second_task()
{
...
}
Note:
Suggestion for your example:
You don't need to have individual mutex object for each thread. Single mutex object lock itself will does the synchronization between all the threads. You are locking mutex of one thread in executejob function and without unlocking another thread is calling lock with different mutex object leading to deadlock or undefined behaviour.
Also since you are calling mutex.lock() inside whileloop without unlocking , same thread is trying to lock itself with same mutex object infinately leading to undefined behaviour.
If you donot need to execute threads parallel you can have one global mutex object can be used inside executejob function to lock and unlock.
mutex m;
void executeJob(int worker)
{
m.lock();
//do some job
m.unlock();
}
If you want to execute job parallel use boost threadpool as I suggested earlier.
In general you can write an algorithm similar to the following. It works with pthreads. I'm sure it would work with c++ threads as well.
create threads and make them wait on a condition variable, e.g. work_exists.
When work arrives you notify all threads that are waiting on that condition variable. Then in the main thread you start waiting on another condition variable work_done
Upon receiving work_exists notification, worker threads wake up, and grab their assigned work from jobs[worker], they execute it, they send a notification on work_done variable, and then go back to waiting on the work_exists condition variable
When main thread receives work_done notification it checks if all threads are done. If not, it keeps waiting till the notification from last-finishing thread arrives.
From cppreference's page on std::mutex::unlock:
The mutex must be unlocked by all threads that have successfully locked it before being destroyed. Otherwise, the behavior is undefined.
Your approach of having one thread unlock a mutex on behalf of another thread is incorrect.
The behavior you're attempting would normally be done using std::condition_variable. There are examples if you look at the links to the member functions.