I am attempting to write a thread safe task queue for multithreading in C++, using SDL2's threading library.
The thread function which runs on all threads is as follows:
int threadFunc(void * pData)
{
ThreadData* data = (ThreadData*)pData;
SDLTaskManager* pool = data->pool;
Task* task = nullptr;
while (true)
{
SDL_LockMutex(pool->mLock);
while (!pool->mRunning && pool->mCurrentTasks.empty())
{
//mutex is unlocked, then locked again when signal received
SDL_CondWait(pool->mConditionFlag, pool->mLock);
if (pool->mShuttingDown)
return 0;
}
//mutex is locked at this stage so no other threads can alter contents of deque
//code inside if block should not be executed if deque is empty
if (!pool->mCurrentTasks.empty())
{
/*out of range error here*/
task = pool->mCurrentTasks.front();
pool->mCurrentTasks.pop_front();
}
if (task != nullptr)
{
pool->notifyThreadWorking(true);
data->taskCount++;
}
else
{
pool->stop();
SDL_UnlockMutex(pool->mLock);
continue;
}
SDL_UnlockMutex(pool->mLock);
task->execute();
SDL_LockMutex(pool->mLock);
pool->notifyThreadWorking(false);
pool->mCompleteTasks.push_back(task);
SDL_UnlockMutex(pool->mLock);
task = nullptr;
}
return 0;
}
As you can see, according to the comments in the code, an out of range error occurs inside an if block, where the deque is empty. However, there is a check there to make sure that the code is only executed if the deque is not empty. The mutex is locked by SDL_CondWait so no other thread should be able to make changes to the deque, until that mutex is unlocked again.
The producer code is as follows:
SDL_LockMutex(pool->mLock);
for (int i = 0; i < numTasks; i++)
{
pool->mCurrentTasks.push_back(new Task());
}
pool->mRunning = true;
SDL_CondBroadcast(pool->mConditionFlag);
SDL_UnlockMutex(pool->mLock);
The fact that the code inside the if block is being executed, shows that at the time if (!pool->mCurrentTasks.empty()) is evaluated, the deque has member data, but not when it reaches task = pool->mCurrentTasks.front(); By my understanding of mutex' this shouldn't be possible. How can this be?
Related
I have a program that starts N number of threads (async/future). I want the main thread to set up some data, then all threads should go while the main thread waits for all of the other threads to finish, and then this needs to loop.
What I have atm is something like this
int main()
{
//Start N new threads (std::future/std::async)
while(condition)
{
//Set Up Data Here
//Send Data to threads
{
std::lock_guard<std::mutex> lock(mrun);
bRun = true;
}
run.notify_all();
//Wait for threads
{
std::unique_lock<std::mutex> lock(mrun);
run.wait(lock, [] {return bDone; });
}
//Reset bools
bRun = false;
bDone = false;
}
//Get results from futures once complete
}
int thread()
{
while(otherCondition)
{
std::unique_lock<std::mutex> lock(mrun);
run.wait(lock, [] {return bRun; });
bDone = true;
//Do thread stuff here
lock.unlock();
run.notify_all();
}
}
But I can't see any signs of either the main or the other threads waiting for each other! Any idea what I am doing wrong or how I can do this?
There are a couple of problems. First, you're setting bDone as soon as the first worker wakes up. Thus the main thread wakes immediately and begins readying the next data set. You want to have the main thread wait until all workers have finished processing their data. Second, when a worker finishes processing, it loops around and immediately checks bRun. But it can't tell if bRun == true means that the next data set is ready or if the last data set is ready. You want to wait for the next data set.
Something like this should work:
std::mutex mrun;
std::condition_variable dataReady;
std::condition_variable workComplete;
int nCurrentIteration = 0;
int nWorkerCount = 0;
int main()
{
//Start N new threads (std::future/std::async)
while(condition)
{
//Set Up Data Here
//Send Data to threads
{
std::lock_guard<std::mutex> lock(mrun);
nWorkerCount = N;
++nCurrentIteration;
}
dataReady.notify_all();
//Wait for threads
{
std::unique_lock<std::mutex> lock(mrun);
workComplete.wait(lock, [] { return nWorkerCount == 0; });
}
}
//Get results from futures once complete
}
int thread()
{
int nNextIteration == 1;
while(otherCondition)
{
std::unique_lock<std::mutex> lock(mrun);
dataReady.wait(lock, [&nNextIteration] { return nCurrentIteration==nNextIteration; });
lock.unlock();
++nNextIteration;
//Do thread stuff here
lock.lock();
if (--nWorkerCount == 0)
{
lock.unlock();
workComplete.notify_one();
}
}
}
Be aware that this solution isn't quite complete. If a worker encounters an exception, then the main thread will hang (because the dead worker will never reduce nWorkerCount). You'll likely need a strategy to deal with that scenario.
Incidentally, this pattern is called a barrier.
solved!: im copying the instance of Map in the new Thread an dont use the reference.
im learning how to use multiple threads. For this im programing a little game where i want the game to run in the main thread and the next chunk of the level shall be loaded in another thread. for this i set up a mutex around a vector to tell the loading thread what to load next. inside this mutex i also have a boolean to tell the thread when tu terminate.
initialising thread in Map::Map()
pending_orders_mutex = SDL_CreateMutex();
can_process_order = SDL_CreateCond();
chunk_loader_thread = SDL_CreateThread(Map::chunk_loader,"chunk_loader_thread",(void*)this);
loading thread
int Map::chunk_loader(void * data)
{
Map map = *(Map*)data;
bool kill_this_thread = false;
Chunk_Order actual_order;
actual_order.load_graphics = false;
actual_order.x = 0;
actual_order.y = 0;
while (!kill_this_thread)
{
SDL_LockMutex(map.pending_orders_mutex); // lock mutex
printf("3-kill_chunk_loader_thread: %d\n", map.kill_chunk_loader_thread);
kill_this_thread = map.kill_chunk_loader_thread;
printf("4-kill_chunk_loader_thread: %d\n", map.kill_chunk_loader_thread);
if (!kill_this_thread)
{
if (map.pending_orders.size())
{
actual_order = map.pending_orders.back();
map.pending_orders.pop_back();
printf("in thread processing order\n");
}
else
{
printf("in thread waiting for order\n");
SDL_CondWait(map.can_process_order, map.pending_orders_mutex);
}
}
SDL_UnlockMutex(map.pending_orders_mutex); // unlock mutex
//load actual order
}
printf("thread got killed\n");
return 0;
}
killing the thread (main thread)
SDL_LockMutex(pending_orders_mutex); // lock mutex
printf("setting kill command\n");
printf("1-kill_chunk_loader_thread: %d\n", kill_chunk_loader_thread);
kill_chunk_loader_thread = true; // send kill command
printf("2-kill_chunk_loader_thread: %d\n", kill_chunk_loader_thread);
SDL_CondSignal(can_process_order); // signal that order was pushed
SDL_UnlockMutex(pending_orders_mutex); // unlock mutex
SDL_WaitThread(chunk_loader_thread, NULL);
console output
3-kill_chunk_loader_thread: 0
4-kill_chunk_loader_thread: 0
in thread waiting for order
setting kill command
1-kill_chunk_loader_thread: 0
2-kill_chunk_loader_thread: 1
3-kill_chunk_loader_thread: 0
4-kill_chunk_loader_thread: 0
in thread waiting for order
why does the mainthread not change the "kill_chunk_loader_thread" boolean in the loading thread?
First of all, you should try to upload a minimal complete program in the question.
Its look like you set kill_chunk_loader_thread = true
but you didn't set map.kill_chunk_loader_thread = true
the section of map declaration is missig from your question, but I guess you didn't use a reference to the local or global variable, or you just perform struct copy so when you change one struct the other doesn't been affected at all.
EDIT:
Map map = *(Map*)data; copies the map structure (default copy constructor, I guess) so from now on if the source map change the copy won't.
You should keep work with the pointer, like this: Map* pMap = (Map*)data;
and check the pointer like this: kill_this_thread = pMap->kill_chunk_loader_thread; so you read from the source map.
I'm trying to do this with the C++11 concurrency support.
I have a sort of thread pool of worker threads that all do the same thing, where a master thread has an array of condition variables (one for each thread, they need to 'start' synchronized, ie not run ahead one cycle of their loop).
for (auto &worker_cond : cond_arr) {
worker_cond.notify_one();
}
then this thread has to wait for a notification of each thread of the pool to restart its cycle again. Whats the correct way of doing this? Have a single condition variable and wait on some integer each thread that isn't the master is going to increase? something like (still in the master thread)
unique_lock<std::mutex> lock(workers_mtx);
workers_finished.wait(lock, [&workers] { return workers = cond_arr.size(); });
I see two options here:
Option 1: join()
Basically instead of using a condition variable to start the calculations in your threads, you spawn a new thread for every iteration and use join() to wait for it to be finished. Then you spawn new threads for the next iteration and so on.
Option 2: locks
You don't want the main-thread to notify as long as one of the threads is still working. So each thread gets its own lock, which it locks before doing the calculations and unlocks afterwards. Your main-thread locks all of them before calling the notify() and unlocks them afterwards.
I see nothing fundamentally wrong with your solution.
Guard workers with workers_mtx and done.
We could abstract this with a counting semaphore.
struct counting_semaphore {
std::unique_ptr<std::mutex> m=std::make_unique<std::mutex>();
std::ptrdiff_t count = 0;
std::unique_ptr<std::condition_variable> cv=std::make_unique<std::condition_variable>();
counting_semaphore( std::ptrdiff_t c=0 ):count(c) {}
counting_semaphore(counting_semaphore&&)=default;
void take(std::size_t n = 1) {
std::unique_lock<std::mutex> lock(*m);
cv->wait(lock, [&]{ if (count-std::ptrdiff_t(n) < 0) return false; count-=n; return true; } );
}
void give(std::size_t n = 1) {
{
std::unique_lock<std::mutex> lock(*m);
count += n;
if (count <= 0) return;
}
cv->notify_all();
}
};
take takes count away, and blocks if there is not enough.
give adds to count, and notifies if there is a positive amount.
Now the worker threads ferry tokens between two semaphores.
std::vector< counting_semaphore > m_worker_start{count};
counting_semaphore m_worker_done{0}; // not count, zero
std::atomic<bool> m_shutdown = false;
// master controller:
for (each step) {
for (auto&& starts:m_worker_start)
starts.give();
m_worker_done.take(count);
}
// master shutdown:
m_shutdown = true;
// wake up forever:
for (auto&& starts:m_worker_start)
starts.give(std::size_t(-1)/2);
// worker thread:
while (true) {
master->m_worker_start[my_id].take();
if (master->m_shutdown) return;
// do work
master->m_worker_done.give();
}
or somesuch.
live example.
I have a weird problem with pthread_cond_wait and pthread_cond_signal. I have arranged a series of threads. They are all in sleep state when started. A wake up function will signal these threads, do some work, and wait for the results.
In the setup below, td is thread data, containing the mutex and conditions, and th is an array containing the pointer to the threads:
for (size_t i = 0; i < NUM_THREADS; i++) {
pthread_cond_init(&td[i].cond, NULL);
pthread_mutex_init(&td[i].cond_mutex, NULL);
pthread_mutex_init(&td[i].work_mutex, NULL);
pthread_mutex_lock(&td[i].cond_mutex);
pthread_mutex_lock(&td[i].work_mutex);
pthread_create(&th[i], NULL, thread_worker, (void *)&td[i]);
}
Thread worker is like this:
void*
thread_worker(void* data)
{
THREAD_DATA *td = (THREAD_DATA *)data;
while (1) {
pthread_cond_wait(&td->cond, &td->cond_mutex); // marker
// do work ...
pthread_mutex_unlock(&td->work_mutex);
}
pthread_exit(NULL);
}
This job function is supposed to wake up all the threads, do the job, and wait for them to finish:
void
job()
{
for (size_t i = 0; i < NUM_THREADS; i++) {
pthread_cond_signal(&td[i].cond);
}
for (size_t i = 0; i < NUM_THREADS; i++) {
pthread_mutex_lock(&td[i].work_mutex); // block until the work is done
}
}
In some rare situations (1 out of 1000 runs maybe), the above setup will encounter a freeze. When that happens, the 'marker' line in thread_worker will not be signaled by pthread_cond_signal, it just kept on waiting. It's very rare but it happens from time to time. I've produced numerous log messages, and I verified that pthread_cond_wait is always called before pthread_cond_signal. What am I doing wrong here?
There is nothing there that forces the pthread_cond_wait() to be called before the pthread_cond_signal(). Despite what you say about logging, it's entirely possible for the logged lines to be out-of-sequence with what really happened.
You aren't using mutexes and condition variables correctly: mutexes should only be unlocked by the same thread that locked them, and condition variables should be paired with a test over some shared state (called a predicate). The shared state is supposed to be protected by the mutex that is passed to pthread_cond_wait().
For example, your example can be reworked to correctly use mutexes and condition variables. First, add an int work_status to the THREAD_DATA structure, where 0 indicates that the thread is waiting for work, 1 indicates that work is available and 2 indicates that the work is complete.
You don't appear to need two mutexes in each THREAD_DATA, and you don't want to lock the mutex in the main thread when you're setting it up:
for (size_t i = 0; i < NUM_THREADS; i++) {
pthread_cond_init(&td[i].cond, NULL);
pthread_mutex_init(&td[i].cond_mutex, NULL);
td[i].work_status = 0;
pthread_create(&th[i], NULL, thread_worker, (void *)&td[i]);
}
Have the threads wait on work_status using the condition variable:
void*
thread_worker(void* data)
{
THREAD_DATA *td = (THREAD_DATA *)data;
while (1) {
/* Wait for work to be available */
pthread_mutex_lock(&td->cond_mutex);
while (td->work_status != 1)
pthread_cond_wait(&td->cond, &td->cond_mutex);
pthread_mutex_unlock(&td->cond_mutex);
// do work ...
/* Tell main thread that the work has finished */
pthread_mutex_lock(&td->cond_mutex);
td->work_status = 2;
pthread_cond_signal(&td->cond);
pthread_mutex_unlock(&td->cond_mutex);
}
pthread_exit(NULL);
}
...and set and wait on work_status as appropriate in job():
void
job()
{
/* Tell threads that work is available */
for (size_t i = 0; i < NUM_THREADS; i++) {
pthread_mutex_lock(&td[i].cond_mutex);
td[i].work_status = 1;
pthread_cond_signal(&td[i].cond);
pthread_mutex_unlock(&td[i].cond_mutex);
}
/* Wait for threads to signal work complete */
for (size_t i = 0; i < NUM_THREADS; i++) {
pthread_mutex_lock(&td[i].cond_mutex);
while (td[i].work_status != 2)
pthread_cond_wait(&td[i].cond, &td[i].cond_mutex);
pthread_mutex_unlock(&td[i].cond_mutex);
}
}
Some check lists:
1) Do you lock the mutex td->cond_mutex before waiting on the cond variable? Otherwise, it's undefined.
2) Do you check predicate after pthread_cond_wait() returns? Typical usage is
while(!flag) pthread_cond_wait(&cv, &mutex); //waits on flag
which is not what you have. This is to protect against spurious wake-ups and also ensure the predicate hasn't changed in the meantime.
3) pthread_cond_signal() is guaranteed to wake up at least one thread. You may want to use pthread_cond_broadcast() if there are multiple threads waiting on the same condition variable.
4) If no thread is waiting on a conditional variable then pthread_cond_signal() or pthread_cond_broadcast() has no effect.
I am using the following thread in c++ to check if a certain condition is met and if so then it should break the loop. I call the thread in a while loop so I need that to break.
The refresh token is updated by another thread.
void ThreadCheck( void* pParams )
{
if(refresh)
{
continue;
}
}
My while loop:-
while(crun)
{
refresh = false;
_beginthread( ThreadCheck, 0, NULL );
rlutil::setColor(8);
cout<<"Send>> ";
getline(cin, msg); //Make a custom function of this.
if(stricmp(msg.c_str(), "exit")==0)
{
crun = false;
}
else if(msg.empty() || stricmp(msg.c_str()," ")==0)
{
rlutil::setColor(4);
cout<<"Plz enter a valid message!\n";
continue;
} else {
manager('c', msg);
// msg.append("\n");
// chat_out<<msg;
// chat_out.close();
}
cout<<"\n";
}
You cannot modify a value in one thread while another thread is, or might be, accessing it. You need to use some form of synchronization, such as a lock.
You have 2 threads : 1) main, 2) ThreadCheck. Add a mutex so as not to update the 'crun' at the same time and inside the thread update the value to false. That's it
#include <iostream>
#include "/tbb/mutex.h"
#include "/tbb/tbb_thread.h"
using namespace tbb;
typedef mutex myMutex;
static myMutex sm;
int i = 0;
void ThreadCheck( )
{
myMutex::scoped_lock lock;//create a lock
lock.acquire(sm);//Method acquire waits until it can acquire a lock on the mutex
//***only one thread can access the lines from here...***
crun = false;;//update is safe (only one thread can execute the code in this scope) because the mutex locked above protects all lines of code until the lock release.
sleep(1);//simply creating a delay to show that no other thread can update
std::cout<<"ThreadCheck "<<"\n";
//***...to here***
lock.release();//releases the lock (duh!)
}
int main()
{
tbb_thread my_thread(ThreadCheck);//create a thread which executes 'someFunction'
// ... your code
my_thread.join();//This command causes the main thread (which is the 'calling-thread' in this case) to wait until thread1 completes its task.
}