I have a function that is invoked from the main thread:
void create_thread() {
pthread_t bg_thread;
pthread_create(&bg_thread, NULL, run_in_background, NULL);
//wait here
pthread_mutex_lock(&MAIN_MUTEX);
pthread_cond_wait(&wakeUpMainThread, &MAIN_MUTEX);
pthread_mutex_unlock(&MAIN_MUTEX);
pthread_cond_signal(wakeUpBgThread);
}
Here is the short version of the function that runs in background thread:
void* run_in_background(void* v) {
pthread_mutex_t mutex;
pthread_cond_t cond;
pthread_mutex_init(&mutex, NULL);
pthread_cond_init(&cond, NULL);
//NOTE: wakeUpBgThread == cond
save_condition_and_mutex(&cond, &mutex);
pthread_mutex_lock(&mutex);
{
pthread_cond_signal(&wakeUpMainThread);
while( run_condition ) {
pthread_cond_wait(&cond, &mutex);
do_smth();
}
}
pthread_mutex_unlock(&mutex);
pthread_mutex_destroy(&mutex);
pthread_cond_destroy(&cond);
pthread_exit(NULL);
}
So the goal is:
1. Create a thread in the main one.
2. Make the main thread sleep until the signal from that thread.
3. Make the background thread sleep until the signal from the main thread.
4. Invoke the background thread from the main one.
The problem is: sometimes after the
pthread_cond_signal(&wakeUpMainThread);
scheduler switches to the main thread immediately and fires the wake up signal for the background thread. After this scheduler switches back to the background thread and it starts waiting for the signal that has already been fired, so it sleeps forever.
Question: is there any way to force background thread to execute the code until the
pthread_cond_wait(&cond, &mutex);
Your call to pthread_mutex_lock in create_thread needs to take place before pthread_create, not after it. Otherwise you have a race condition.
Use a semaphore? Semaphore signals are not lost - they just increment the count & so the background thread will run agan after the semaphore is signaled, even if it has not actually got around to waiting on it yet.
Rgds,
Martin
It sounds like your best bet is to use a condition. Have a mutex and a condition. Main initializes both, grabs the mutex, creates the thread, then goes to sleep on the condition. Child grabs the lock (after main waits on the condition) does the work (or alternatively does the work then grab the lock), and then signals the condition (you can decide whether to release the lock before or after the signal--important bit is that you grabbed it). Main then wakes up and continues processing.
pthread_cond_wait() and friends is what you look at.
You don't lock the mutex before your signal on main thread. If you want predictable behavior - you should lock the same mutex both before wait call and signal call.
Related
I have used below structure to create a threadpool, now the question is how do I let all the preallocate threads end properly?
std::vector<pthread_t> preallocatedThreadsPool; // threadpool
std::queue<int> tcpQueue; // a queue to hold my task
pthread_mutex_t mutex = PTHREAD_MUTEX_INITIALIZER;
pthread_cond_t condition_var = PTHREAD_COND_INITIALIZER;
void* threadFunctionUsedByThreadsPool(void *arg);
main () {
preallocatedThreadsPool.resize(preallocatThreadsNumber);
for(pthread_t i : preallocatedThreadsPool) {
pthread_create(&i, NULL, threadFunctionUsedByThreadsPool, NULL);
}
pthread_mutex_lock(&mutex); // one thread mess with the queue at one time
tcpQueue.push(task);
pthread_cond_signal(&condition_var);
pthread_mutex_unlock(&mutex);
}
void* threadFunctionUsedByThreadsPool(void *arg) {
while (true) {
pthread_mutex_lock(&mutex);
if (tcpQueue.empty()) { // can't get work from the queue then just wait
pthread_cond_wait(&condition_var, &mutex); // wait for the signal from other thread to deal with client otherwise sleep
task = tcpQueue.front();
tcpQueue.pop();
}
pthread_mutex_unlock(&mutex);
if (task) {
// do task
}
}
return NULL;
}
I have been searching for days for this problem still can not find a decent solution, the closest one I have tried is , when the program wants to quit, push a special item into the queue, then inside threadFunctionUsedByThreadsPool, when detecting such item, I will call pthread_join, however, when I using gdb tool to debug it , those pre-allocated threads are still there, anyone could help, better with some code, for example, how do I modify the threadFunctionUsedByThreadsPool, so that I can quit all the pre-allocated threads properly?
Thanks so much!!!
TLDR: You just need a thread-safe variable that all threads can check for an exit condition in between work items. Use pthread_join to wait for a thread to exit.
First, let's get the while loop in your thread function correct with respect to condition variables.
Instead of this:
pthread_cond_wait(&condition_var, &mutex); // wait for the signal from other thread to deal with client otherwise sleep
task = tcpQueue.front();
tcpQueue.pop();
Check the state of the queue before before and after waking up on the condition variable. Spurious wake up is a real thing and there's no guarantee another thread didn't wake up and grab the last work item. You definitely don't want to be popping from an empty queue.
Better:
while (tcpQueue.empty()) {
pthread_cond_wait(&condition_var, &mutex); // wait for the signal from other thread to deal with client otherwise sleep
}
task = tcpQueue.front();
tcpQueue.pop();
With that addressed, we can introduce a new global bool that represents the stop condition:
bool stopCondition = false;
Whenever we want to tell all the threads in the pool to stop, we can set stopCondition to true and signal the condition var to alert all threads of a state change. Reading or writing stopCondition should be done under a lock. (I suppose you could also use std::atomic<bool>)
Putting it all together, your thread function becomes this:
void* threadFunctionUsedByThreadsPool(void* arg) {
pthread_mutex_lock(&mutex);
while (!stopCondition) {
// wait for a task to be queued
while (tcpQueue.empty() && !stopCondition) {
pthread_cond_wait(&condition_var, &mutex); // wait for the signal from other thread to deal with client otherwise sleep
}
if (stopCondition == false) {
task = tcpQueue.front();
tcpQueue.pop();
// exit lock while operating on a task
pthread_mutex_unlock(&mutex);
if (task) {
// do task
}
// re-acquire the lock
pthread_mutex_lock(&mutex);
}
}
// release the lock before exiting the function
pthread_mutex_unlock(&mutex);
return NULL;
}
And then a helper function to signal all the threads to exit and also waits for each thread to stop. notice that we're using pthread_cond_broadcast to notify all threads to wake up from their condition variable wait instead of pthread_cond_signal which only wakes up one thread.
void stopThreadPool()
{
// signal all threads to exit after they finish their current work item
pthread_mutex_lock(&mutex);
stopCondition = true;
pthread_cond_broadcast(&condition_var); // notify all threads
pthread_mutex_unlock(&mutex);
// wait for all threads to exit
for (auto& t : preAllocatedThreadsPool) {
pthread_join(t, nullptr);
}
preAllocatedThreadsPool.clear();
}
One last bug that I just caught - your main isn't property initializing your preAllocatedThreadsPool vector like you think it is. You're making a copy of the pthread_t, instead of using the handle actually in the vector.
Instead of this:
for(pthread_t i : preallocatedThreadsPool) {
Your loop needs to enumerate by reference:
Better:
for(pthread_t &i : preallocatedThreadsPool) {
Send a task that instructs the pool thread to requeue the task and then terminate. The poison-task will then run through all the threads in the pool, killing them all off. I have used a null as the poison, (ie, an illegal task) - it does not need to be destructed when it has killed the last thread. You may wish to purge the task queue before sending the null/whatever. If you use a null, you only need a null check in the threads, just after dequeing the task.
You need very little extra code, you don't need to know how many threads in the pool and it will work:)
Please see the following code:
std::mutex mutex;
std::condition_variable cv;
std::atomic<bool> terminate;
// Worker thread routine
void work() {
while( !terminate ) {
{
std::unique_lock<std::mutex> lg{ mutex };
cv.wait(lg);
// Do something
}
// Do something
}
}
// This function is called from the main thread
void terminate_worker() {
terminate = true;
cv.notify_all();
worker_thread.join();
}
Is the following scenario can happen?
Worker thread is waiting for signals.
The main thread called terminate_worker();
The main thread set the atomic variable terminate to true, and then signaled to the worker thread.
Worker thread now wakes up, do its job and load from terminate. At this step, the change to terminate made by the main thread is not yet seen, so the worker thread decides to wait for another signal.
Now deadlock occurs...
I wonder this is ever possible. As I understood, std::atomic only guarantees no race condition, but memory order is a different thing. Questions:
Is this possible?
If this is not possible, is this possible if terminate is not an atomic variable but is simply bool? Or atomicity has nothing to do with this?
If this is possible, what should I do?
Thank you.
I don't believe, what you describe is possible, as cv.notify_all() afaik (please correct me if I'm wrong) synchronizes with wait(), so when the worker thread awakes, it will see the change to terminate.
However:
A deadlock can happen the following way:
Worker thread (WT) determines that the terminate flag is still false.
The main thread (MT) sets the terminate flag and calls cv.notify_all().
As no one is curently waiting for the condition variable that notification gets "lost/ignored".
MT calls join and blocks.
WT goes to sleep ( cv.wait()) and blocks too.
Solution:
While you don't have to hold a lock while you call cv.notify, you
have to hold a lock, while you are modifying terminate (even if it is an atomic)
have to make sure, that the check for the condition and the actual call to wait happen while you are holding the same lock.
This is why there is a form of wait that performs this check just before it sends the thread to sleep.
A corrected code (with minimal changes) could look like this:
// Worker thread routine
void work() {
while( !terminate ) {
{
std::unique_lock<std::mutex> lg{ mutex };
if (!terminate) {
cv.wait(lg);
}
// Do something
}
// Do something
}
}
// This function is called from the main thread
void terminate_worker() {
{
std::lock_guard<std::mutex> lg(mutex);
terminate = true;
}
cv.notify_all();
worker_thread.join();
}
I have following scenario:
condition_variable cv;
mutex mut;
// Thread 1:
void run() {
while (true) {
mut.lock();
// create_some_data();
mut.unlock();
cv.notify_all();
}
}
// Thread 2
void thread2() {
mutex lockMutex;
unique_lock<mutex> lock(lockMutex);
while (running) {
cv.wait(lock);
mut.lock();
// copy data
mut.unlock();
// process data
}
}
// Thread 3, 4... - same as Thread 2
I run thread 1 all the time to get new data. Other threads wait with condition_variable until new data is available, then copy it and do some work on it. Work perfomed by threads differs in time needed to finish, the idea is that threads will get new data only when they finished with the old one. Data got in meantime is allowed to be "missed". I don't use shared mutex (only to access data) because I don't want threads to depend on each other.
Above code works fine on Windows, but now I run it on Ubuntu and I noticed that only one thread is being notified when notify_all() is called and the other ones just hangs on wait().
Why is that? Does Linux require different approach for using condition_variable?
Your code exhibits UB immediately as it relocks the unique lock that the cv has relocked when it exits wait.
There are other problems, like not detecting spurious wakeups.
Finally cv notify all onky notified currently waiting threads. If a thread shows up later, no dice.
It's working by luck.
The mutex and the condition variable are two parts of the same construct. You can't mix and match mutexes and cvs.
try this:
void thread2() {
unique_lock<mutex> lock(mut); // use the global mutex
while (running) {
cv.wait(lock);
// mutex is already locked here
// test condition. wakeups can be spurious
// copy data
lock.unlock();
// process data
lock.lock();
}
}
Per this documentation:
Any thread that intends to wait on std::condition_variable has to
acquire a std::unique_lock, on the same mutex as used to
protect the shared variable
execute wait, wait_for, or wait_until. The wait operations atomically release the mutex and suspend the execution of the
thread.
When the condition variable is notified, a timeout expires, or a spurious wakeup occurs, the thread is awakened, and the mutex is
atomically reacquired. The thread should then check the condition
and resume waiting if the wake up was spurious.
This code
void thread2() {
mutex lockMutex;
unique_lock<mutex> lock(lockMutex);
while (running) {
doesn't do that.
I just started learning pthread condition variable. But below code is not working as expected.
#include<iostream>
#include<pthread.h>
using namespace std;
pthread_mutex_t mutex = PTHREAD_MUTEX_INITIALIZER;
pthread_cond_t count_threshold_cv = PTHREAD_COND_INITIALIZER;
int count=0;
void *inc_func(void *arg)
{
pthread_mutex_lock(&mutex);
int c;
while(1)
{
cin>>c;
if(c==8){
pthread_cond_signal(&count_threshold_cv);break;}
}
cout<<"inc count reached 8"<<endl;
pthread_mutex_unlock(&mutex);
}
void *watch(void *arg)
{
pthread_mutex_lock(&mutex);
while(1){
pthread_cond_wait(&count_threshold_cv,&mutex);
break;
}
cout<<"This executed"<<endl;
pthread_mutex_unlock(&mutex);
}
int main()
{
pthread_t id[26];
pthread_create(&id[0],NULL,inc_func,NULL);
pthread_create(&id[1],NULL,watch,NULL);
int i;
for(i=0;i<2;i++)
{
pthread_join(id[i],NULL);
}
}
when the input is 8 this code gets hanged at "inc count reached 8? I am not able to understand.
Where is my understanding wrong?
The correct solution to this is to make the watch thread only wait if the condition it is waiting for has not occured yet.
The condition appears to be c == 8 (since that is what is signalled), so you will need to make the c variable global so that it is shared between the threads, then change the watch thread to do:
void *watch(void *arg)
{
pthread_mutex_lock(&mutex);
while (c != 8) {
pthread_cond_wait(&count_threshold_cv, &mutex);
}
cout<<"This executed"<<endl;
pthread_mutex_unlock(&mutex);
return 0;
}
Now it doesn't matter which thread runs first: your code is correct either way. This is the right way to use conditional variables: in general, the waiter should do:
pthread_mutex_lock(&mutex);
while (!condition)
pthread_cond_wait(&cond, &mutex);
/* ... */
and the signaller should do:
pthread_mutex_lock(&mutex);
/* ... something that potentially makes condition true ... */
pthread_cond_broadcast(&cond);
pthread_mutex_unlock(&mutex);
The important thing here is that pthread_cond_signal will unblock at least one of the threads that are blocked on that condition variable (meaning which are currently blocked on a call to pthread_cond_wait on that same condition variable). If at the moment when one thread calls pthread_cond_signal there is no other thread waiting on that condition then basically nothing happens.
Keeping this in mind, the flow of your program is something like this:
create and start first thread;
first thread calls inc_func(), which locks the mutex before anything else;
inc_func() keeps waiting for the number 8 to be entered, keeping the mutex locked all this time;
sometime during this, but most times probably after the inc_func managed to lock the mutex, the second thread is created;
the second thread also tries to lock the mutex right at the start of the function, and is blocked because the first thread already has it locked;
at some point, you enter 8 and the condition gets signaled from thread 1; thread 2 is not waiting on this condition yet, so it remains blocked trying to lock the mutex;
the first thread finally releases the mutex, so thread 2 locks it and then blocks on pthread_cond_wait.
At this point, thread 1 has already finished, thread 2 is blocked waiting for the condition to be signaled, and the main thread is waiting for it to finish. There is nobody else to signal that condition, so you have a hang.
For a quick fix that will probably work most of the times, you could try changing the order in which you start the threads(start the watch thread first). But keep in mind and understand why I used bold for probably and most of the times.
The correct way to fix this would be to rethink your locking strategy: lock the mutexes in the smallest scope possible and keep them locked for the shortest time possible.
Swap the thread's execution sequence:
pthread_create(&id[1],NULL,watch,NULL);
pthread_create(&id[0],NULL,inc_func,NULL);
If you run thread0 as first, thread1 never gets past mutex lock, so it doesn't start waiting. Than thread0 ends, and only then thread1 executes pthread_cond_wait(), but there's no thread to do signal.
If you start thread1 first, it gets to the waiting part.
I am trying to implement very simple Windows events in Linux. Only for my scenario - 3 threads, 1 main and 2 secondary. Each of secondary threads raise 1 event by SetEvent and main thread wait it. Example:
int main()
{
void* Events[2];
Events[0] = CreateEvent();
Events[1] = CreateEvent();
pthread_start(Thread, Events[0]);
pthread_start(Thread, Events[1]);
WaitForMultipleObjects(2, Events, 30000) // 30 seconds timeout
return 0;
}
int* thread(void* Event)
{
// Do something
SetEvent(Event);
// Do something
}
So, to implement it, i use conditional variables. But my question is - is this a right way? Or i doing something wrong? My implementation:
// Actually, this function return pointer to struct with mutex and cond
// here i just simplified example
void* CreateEvent(mutex, condition)
{
pthread_mutex_init(mutex, NULL);
pthread_cond_init(condition, NULL);
}
bool SetEvent (mutex, condition)
{
pthread_mutex_lock(mutex);
pthread_cond_signal(condition);
pthread_mutex_unlock(mutex);
}
int WaitForSingleObject(mutex, condition, timeout)
{
pthread_mutex_lock(mutex);
pthread_cond_timedwait(condition, mutex, timeout);
pthread_mutex_unlock(mutex);
}
// Call WaitForSingleObject for each event.
// Yes, i know, that its a wrong way, but it should work in my example.
int WaitForMultipleObjects(count, mutex[], condition[], timeout);
And all seems good, but i think, that problem will appear when i call WaitFor.. function in Main thread before SetEvent in secondary thread will be called. In Windows, it worked well, but in Linux - only idea is described above.
Maybe you tell me the better way to solve it? Thank you.
UPD: Timeout is very important, because one of the secondary threads may not pass SetEvent().
Basing this on the description of WaitForSingleObject
The WaitForSingleObject function checks the current state of the specified object. If the object's state is nonsignaled, the calling thread enters the wait state until the object is signaled or the time-out interval elapses.
The difference between that behavior and the code is that the code will always wait on the condition variable, as it does not check a predicate. This introduces synchronization issues between the pthread_condt_timewait and pthread_cond_signal calls.
The general idiom for signalling a condition variable is:
lock mutex
set predicate
unlock mutex
signal condition variable
And when waiting for a condition variable:
lock mutex
while ( !predicate )
{
wait on condition variable
}
unlock mutex
Based on what is trying to be accomplished, a separate bool could be used as a predicate for each Event. By introducing a predicate, WaitForSingleObject should only wait on the condition variable if the Event has not been signaled. The code would look similar to the following:
bool SetEvent (mutex, condition)
{
pthread_mutex_lock(mutex); // lock mutex
bool& signalled = find_signal(condition); // find predicate
signalled = true; // set predicate
pthread_mutex_unlock(mutex); // unlock mutex
pthread_cond_signal(condition); // signal condition variable
}
int WaitForSingleObject(mutex, condition, timeout)
{
pthread_mutex_lock(mutex); // lock mutex
bool& signalled = find_signal(condition); // find predicate
while (!signalled)
{
pthread_cond_timedwait(condition, mutex, timeout);
}
signalled = false; // reset predicate
pthread_mutex_unlock(mutex); // unlock mutex
}
There was similar question on stackoverflow already: WaitForSingleObject and WaitForMultipleObjects equivalent in linux
In addition you can use semaphores:
sem_t semOne ;
sem_t semTwo ;
sem_t semMain ;
In main thread:
sem_init(semOne,0,0) ;
sem_init(semTwo,0,0) ;
sem_init(semMain,0,0) ;
...
sem_wait(&semMain);
// Thread 1
sem_wait(&semOne);
sem_post(&semMain);
// Thread 2
sem_wait(&semTwo);
sem_post(&semMain);
Detailed description and various examples could be found here: ------http://www.ibm.com/developerworks/linux/library/l-ipc2lin3/index.html
The previous link is no longer available. The most recent archived version at The Internet Archive's Wayback Machine is:
https://web.archive.org/web/20130515223326/http://www.ibm.com/developerworks/linux/library/l-ipc2lin3/index.html
I think semaphore is a better solution here, because it can be used inter-process.
You can wrap the interface, if name is not provide, then use pthread_ to initialize it for intra-process to use, which can short the resource usage, but when name used, try to use sem initialize it, for intra-process use.