UPD: It seems that the problem which I explain below is non-existent. I cannot reproduce it in a week already, I started suspecting that it was caused by some bugs in a compiler or corrupted memory because it is not reproducing anymore.
I tried to implement my own recursive mutex in C++, but for some reason, it fails. I tried to debug it, but I stuck. (I know that there are recursive mutex in std, but I need a custom implementation in a project where STL is not available; this implementation was just a check of an idea). I haven't thought about efficiency yet, but I don't understand why my straightforward implementation doesn't work.
First of all, here's the implementation of the RecursiveMutex:
class RecursiveMutex
{
std::mutex critical_section;
std::condition_variable cv;
std::thread::id id;
int recursive_calls = 0;
public:
void lock() {
auto thread = std::this_thread::get_id();
std::unique_lock<std::mutex> lock(critical_section);
cv.wait( lock, [this, thread]() {
return id == thread || recursive_calls == 0;
});
++recursive_calls;
id = thread;
}
void unlock() {
std::unique_lock<std::mutex> lock( critical_section );
--recursive_calls;
if( recursive_calls == 0 ) {
lock.unlock();
cv.notify_all();
}
}
};
The failing test is straightforward, it just runs two threads, both of them are locking and unlocking the same mutex (the recursive nature of the mutex is not tested here). Here it is:
std::vector<std::thread> threads;
void initThreads( int num_of_threads, std::function<void()> func )
{
threads.resize( num_of_threads );
for( auto& thread : threads )
{
thread = std::thread( func );
}
}
void waitThreads()
{
for( auto& thread : threads )
{
thread.join();
}
}
void test () {
RecursiveMutex mutex;
while (true) {
int count = 0;
initThreads(2, [&mutex] () {
for( int i = 0; i < 100000; ++i ) {
try {
mutex.lock();
++count;
mutex.unlock();
}
catch (...) {
// Extremely rarely.
// Exception is "Operation not permited"
assert(false);
}
}
});
waitThreads();
// Happens often
assert(count == 200000);
}
}
In this code I have two kinds of errors:
Extremely rarely I get an exception in RecursiveMutex::lock() which contains message "Operation not permitted" and is thrown from cv.wait. As far as I understand, this exception is thrown when wait is called on a mutex which is not owned by the thread. At the same time, I lock it just above calling the wait so this cannot be the case.
In most situations I just get a message into console "terminate called without an active exception".
My main question is what the bug is, but I'll also be happy to know how to debug and provoke race condition in such a code in general.
P.S. I use Desktop Qt 5.4.2 MinGW 32 bit.
Related
Back in days I was working on an option that would speed up my function by multithreading. The base function finished around 15seconds, and I would like to reducing it, but I cannot logicing out how to create a good and working multithreading function.
Base function, before touches:
void FirstCall()
{
MainFunction1();
MainFunction2();
}
void MainFunction1()
{
//Call another functions, MainFunction3-10 for example
}
void MainFunction2()
{
//Cann another, different functions, in a for loop
}
In this case, the time that needed to finishing the function is around 15 seconds.
That I found to speeding up this function was the multithreading idea.
Let me show how it is right now, and what is my problem with it.
//Way 1 of multithreading
void FirstCall()
{
std::vector<std::thread> threads;
threads.push_back(std::thread(&MainFunction1, this));
threads.push_back(std::thread(&MainFunction2, this));
for (auto& th : threads)
{
if (th.joinable())
{
th.join();
}
}
}
The other functions are exactly same, so that shouldnt be related to the runtime. The runtime with the function that I showed up above is around 8-10seconds, so seems it is working fine, but sometimes the application simply closing when this function is called.
//Way 2 of multithreading
void FirstCall()
{
static std::mutex s_mutex;
static std::atomic<int> thread_number = 0;
auto MainFunctions = [&](int index)
{
SwitchMainFunctions(index);
}
auto ThreadFunction = [&]()
{
std::lock_guard<std::mutex> lGuard (s_mutex);
MainFunctions(thread_number++);
}
int thread_count = std::thread::hardware_concurrency(); //8
//thread count > function count (2 functions)
std::vector<std::thread> threads;
for (int i = 0; i < 2; i++)
{
threads.push_back(std::thread(ThreadFunction));
}
for (auto& th : threads)
{
if (th.joinable())
{
th.join();
}
}
threads.clear();
}
void SwitchMainFunctions(int index)
{
switch(index)
{
case 0:
{
MainFuunction1();
}
break;
case 1:
{
MainFuunction2();
}
break;
default:
{
return;
}
break;
}
}
The function that is presented as way 2 of multithreading is working fine, my application is do not crashing anymore, but the run time is same like the untouched function time is ~15 seconds.
I think the mutex lock is forceto wait until one thread is finishing, so it is exactly same if I'd just using the default code, but I would like really speeding up the function.
I tried to speed up my function with multithreading option, but the 2 way I tried to do have different problems.
The first idea is sometimes force my application crashing when the function is called.
The second way that I created have the same run time than the default function has without multithreading.
Your second option is far more complicated first option. Here is simple
FirstCall():
void FirstCall()
{
std::vector<std::thread> threads;
threads.push_back(std::thread(MainFunction1));//this removed since MainFunction1 is void
threads.push_back(std::thread(MainFunction2));
for (auto& th : threads)
{
if (th.joinable())
{
th.join();
}
}
}
In this simple scenario main thread will block till both threads finish and join. In your first option you used this as argument for MainFunction1 in thread constructor. That implies FirstCall() to be member function of this.
In such case you should add whole class definition to your question and at least scope of MainFunction1/2. This will help to understand why application simply closing.
Your second option is worse then single threaded since lGuard will unlock only when thread finished executing all functions.
auto ThreadFunction = [&]()
{
std::lock_guard<std::mutex> lGuard (s_mutex);
MainFunctions(thread_number++);
//MainFunctions calls SwitchMainFunctions
//SwitchMainFunctions calls MainFunction
//when done lGuard unlocks on destruction
}
Another problem with second option is why do you need mutex at all. If you insist to map std::atomic thread_number to specific function simply pass result of atomic fetch_add to SwitchMainFunctions in thread constructor.
void FirstCall()
{
std::atomic<int> thread_number = 0;
std::vector<std::thread> threads;
for (int i = 0; i < 2; i++)
{
threads.push_back(std::thread(SwitchMainFunctions, thread_number++));
}
for (auto& th : threads)
{
if (th.joinable())
{
th.join();
}
}
}
I've got a Timer class that can run with both an initial time and an interval. There's an internal function internalQuit performs thread.join() before a thread is started again on the resetCallback. The thing is that each public function has it's own std::lock_guard on the mutex to prevent the data of being written. I'm now running into an issue that when using the callback to for example stop the timer in the callback, the mutex cannot be locked by stop(). I'm hoping to get some help on how to tackle this issue.
class Timer
{
public:
Timer(string_view identifier, Function &&timeoutHandler, Duration initTime, Duration intervalTime);
void start()
void stop() // for example
{
std::lock_guard lock{mutex};
running = false;
sleepCv.notify_all();
}
void setInitTime()
void setIntervalTime()
void resetCallback(Function &&timeoutHandler)
{
internalQuit();
{
std::lock_guard lock{mutex};
quit = false;
}
startTimerThread(std::forward<Function>(timeoutHandler));
}
private:
internalQuit() // performs thread join
{
{
std::lock_guard lock {mutex};
quit = true;
running = false;
sleepCv.notify_all();
}
thread.join();
}
mainLoop(Function &&timeoutHandler)
{
while(!quit)
{
std::unique_lock lock{mutex};
// wait for running with sleepCv.wait()
// handle initTimer with sleepCv.wait_until()
timeoutHandler(); // callback
// handle intervalTimer with sleepCv.wait_until()
timeoutHandler(); // callback
}
}
startTimerThread(Function &&timeoutHandler)
{
thread = std::thread([&, timeoutHandler = std::forward<Function>(timeoutHandler)](){
mainLoop(timeoutHandler);
});
}
std::thread thread{};
std::mutex mutex{};
std::condition_variable sleepCv{}
// initTime, intervalTime and some booleans for updating with sleepCv.notify_all();
}
For testing this, I have the following testcase in Gtest. I'm expecting the timer to stop in the callback. Unfortunately, the timer will hang on acquiring the mutex lock in the stop() function.
std::atomic<int> callbackCounter;
void timerCallback()
{
callbackCounter.fetch_add(1, std::memory_order_acq_rel);
}
TEST(timerTest, timerShouldStopWhenStoppedInNewCallback)
{
std::atomic<int> testCounter{0};
Timer<std::chrono::steady_clock > t{"timerstop", &timerCallback, std::chrono::milliseconds(0), std::chrono::milliseconds(100)};
t.resetCallback([&]{
testCounter += 1;
t.stop();
});
t.start();
sleepMilliSeconds(100);
ASSERT_EQ(testCounter.load(), 1); // trigger due to original interval timeout
sleepMilliSeconds(100);
ASSERT_EQ(testCounter.load(), 1); // no trigger, because stopped in new callback
}
Removing all the mutexes in each of the public fucntions, fixes the issue. But that could lead to possible race conditions for data being written to variables. Hence each function has a lock before writing to f.e. the booleans.
I've tried looking into the std::move functionality to move the thread during the resetCallback into a different variable and then call join on that one. I'm also investigating recursive_mutex but have no experience with using that.
void resetCallback(Function &&timeoutHandler)
{
internalQuit();
{
std::lock_guard lock{mutex};
quit = false;
}
auto prevThread = std::thread(std::move(this->thread));
// didn't know how to continue from here, requiring more selfstudy.
startTimerThread(std::forward<Function>(timeoutHandler));
}
It's a new subject for me, have worked with mutexes and timers before but with relatively simple stuff.
Thank you in advance.
I have two methods "log" and "measure" that should never execute at the same time.
So I tried to use a "std::mutex" to do this as follows:
void log(std::string message)
{
mtx.lock();
someLogFunctionality();
mtx.unlock();
}
void measure()
{
mtx.lock();
someMeasureFunctionality();
mtx.unlock();
}
Now it turned out that it also shall be possible to call "log" multiple times in parallel without locking and the same applies for "measure", too. (Reason: someLogFunctionality() and someMeasureFunctionality() interfere with each other but the same method may be called multiple times parallely)
I had a look at "std::shared_mutex" then, but there are two problems for me:
1.) With shared_mutex I could use lock_shared for only one of the methods (log or measure) but then the other one would have to use the exclusive lock (and could again not be executed multiple times in parallel)
void log(std::string message)
{
mtx.lock_shared();
someLogFunctionality();
mtx.unlock_shared();
}
void measure()
{
mtx.lock(); // This should also be shared but among another "group"
someMeasureFunctionality();
mtx.unlock();
}
2.) I can't use C++17 (constraint in the environment that I'm working with)
Do you have any suggestions for me how I could realize this?
Based on the reply from alexb I have written the following mutex class which currently works for me (only tried out in a simple multithreaded example application so far)
Please note that it is not protected against "starvation". In simple words: It is not ensured that that lockMeasure will ever get the lock if lockLogging is called high-frequently (and the other way round).
class MyMutex
{
private:
std::atomic<int> log_executors;
std::atomic<int> measure_executors;
std::mutex mtx;
std::condition_variable condition;
public:
MyMutex() : log_executors(0), measure_executors(0) {}
~MyMutex() {}
void lockMeasure()
{
std::unique_lock<std::mutex> lock(mtx);
while(log_executors) {
condition.wait(lock);
}
measure_executors++;
}
void unlockMeasure()
{
std::unique_lock<std::mutex> lock(mtx);
measure_executors--;
if (!measure_executors)
{
condition.notify_all();
}
}
void lockLogging()
{
std::unique_lock<std::mutex> lock(mtx);
while(measure_executors) {
condition.wait(lock);
}
log_executors++;
}
void unlockLogging()
{
std::unique_lock<std::mutex> lock(mtx);
log_executors--;
if (!log_executors)
{
condition.notify_all();
}
}
static MyMutex& getInstance()
{
static MyMutex _instance;
return _instance;
}
};
Usage:
void measure()
{
MyMutex::getInstance().lockMeasure();
someMeasureFunctionality();
MyMutex::getInstance().unlockMeasure();
}
void log()
{
MyMutex::getInstance().lockLogging();
someLogFunctionality();
MyMutex::getInstance().unlockLogging();
}
You need some barrier logic which is more complicated than shared_mutex (BTW, shared_mutex is not best choice for multiplatform compilation). For example, you can use mutex, conditional variable, and 2 variables for barrier sync. It does not take CPU and you may not use sleeps for check.
#include <mutex>
#include <condition_variable>
#include <atomic>
std::atomic<int> log_executors = 0;
std::atomic<int> measure_executors = 0;
std::mutex mutex;
std::condition_variable condition;
void log(std::string message) {
{
std::unique_lock<std::mutex> lock(mutex);
log_executors++; // Register current executor and prevent from entering new measure executors
// Wait until all measure executors will go away
while(measure_executors) {
condition.wait(lock); // wait condition variable signal. Mutex will be unlocked during wait
}
}
// here lock is freed
someLogFunctionality(); // execute logic
{
std::unique_lock<std::mutex> lock(mutex);
log_executors--; // unregister current execution
condition.notify_all(); // send signal and unlock all waiters
}
}
void measure()
{
{
std::unique_lock<std::mutex> lock(mutex);
measure_executors++; // Register current executor and prevent from entering new log executors
while(log_executors) {
condition.wait(lock); // wait until all measure executors will gone
}
}
someMeasureFunctionality();
{
std::unique_lock<std::mutex> lock(mutex);
measure_executors--; // unregister current execution
condition.notify_all(); // send signal and unlock all waiters
}
}
You can have a master lock granting access to a semaphore variable:
void log(std::string message)
{
acquire(LOG);
someLogFunctionality();
release(LOG);
}
void measure()
{
acquire(MEASURE);
someMeasureFunctionality();
release(MEASURE);
}
void acquire(int what) {
for (;;) {
mtx.lock();
if (owner == NONE) {
owner = what;
}
if (owner == what) {
// A LOG was asked while LOG is running
users[owner]++;
mtx.unlock();
return;
}
mtx.unlock();
// Some sleep would be good
usleep(5000);
}
}
void release(int what) {
mtx.lock();
if (owner != what) {
// This is an error. How could this happen?
}
if (users[what] <= 0) {
// This is an error. How could this happen?
}
users[what]--;
if (0 == users[what]) {
owner = NONE;
}
mtx.unlock();
}
In this case, for example:
owner is NONE
LOG1 acquires LOG. It can do so because owner is NONE
MEASURE1 acquires LOG. It starts spinning in place because owner != MEASURE
MEASURE2 acquires LOG. It starts spinning in place because owner != MEASURE
LOG2 acquires LOG. It can do so because owner is LOG, users[LOG]=2
LOG2 releases LOG. users[LOG]=1
LOG1 releases LOG. users[LOG]=0, so owner becomes NONE
MEASURE2 by pure chance acquires mtx before MEASURE1, finds owner=NONE and goes
MEASURE1 finds owner=MEASURE and sets users[MEASURE]=2
In the above, note that the second call to measure() actually executed a bit earlier. This should be OK. But if you want to keep the calls "serialized" even if they happen in parallel, you'll need a stack for each owner and more complex code.
I have a groups of objects, each object has two threads: Task thread processes the data and notifies Decision thread that the data is ready, then waits for Decision thread to make the decision whether to continue operations; Decision thread waits Task thread for the data, then consumes the data and make a decision ( notify Task thread that the decision is ready to fetch ).
Task.cpp:
class Task{
public:
void DoTask(){
// process data
{
std::unique_lock<std::mutex> lck(mtx);
data_ready = true;
cv_data.notify_one();
while( decision_ready == false )
cv_decision.wait( lck );
}
if ( decision )
// continue task
else
// quit
}
void SetDecision( bool flag ) { decision = flag; }
bool GetDataFlag() const { return data_ready; }
bool SetDecisionFlag( bool flag ) { decision_ready = flag; }
std::mutex mtx;
std::condition_variable cv_data;
std::condition_variable cv_decision;
private:
bool decision;
bool data_ready;
bool decision_ready;
};
main.cpp:
void Decision ( Task *task );
int main(){
Task mytask[10];
std::thread do[10];
std::thread decision[10];
for(int i=0; i< 10; ++i)
{
do[i] = std::thread( &Task::doTask, &mytask[i] );
decision[i] = std::thread( Decision, &mytask[i] );
do[i].detach();
decision[i].detach();
}
}
void Decision( Task *task )
{
st::mutex mtx_decision;
std::unique_lock<std::mutex> lck( task->mtx );
while( task->GetDataFlag() == false )
task->cv_data.wait(lck);
std::lock_guard<std::mutex> lk(mtx_decision);
// check database and make decision
task->SetDecision( true );
task->SetDecisionFlag( true );
task->cv_decision.notify_one();
}
What is the problem with this approach? The program works well only in single thread case. If I actually open two or more threads, I get segmentation fault. I am not sure how to pass the condition variables between different scopes. And hope someone can tell me the right way to do it. Thanks.
I suppose you need the same mutex and same conditional variable to get it working. Now each class gets own mutex and condition_variable and each decision too.
The most likely reason while your application crashes is because you detach your threads and than your main() exits, killing threads in the midst of what they are doing. I strongly advice against using detached threads.
I want to synchroznie threads in C++ using pthreads in smart way.
I have one global variable:
int Resources = 0;
I have two thread functions:
void *incResources(void *arg)
{
while(1)
{
pthread_mutex_lock (&resourcesMutex);
Resources += 2;
pthread_mutex_unlock (&resourcesMutex);
}
pthread_exit((void*) 0);
}
void *consumeResources(void *arg)
{
while(1)
{
pthread_mutex_lock (&resourcesMutex);
Resources--;
pthread_mutex_unlock (&resourcesMutex);
}
pthread_exit((void*) 0);
}
And in main function I intialize two consuming threads and one incrementing thread:
pthread_mutex_init(&resourcesMutex, NULL);
pthread_create(&callThd[0], &attr, incResources, (void *)i);
pthread_create(&callThd[1], &attr, consumeResources, (void *)i);
pthread_create(&callThd[2], &attr, consumeResources, (void *)i);
I feel this so unefficient and it can be done better. Can you provide me some ideas? I've tried to use wait but i dont get it :/
Thanks!
I you look for good and C++ ways, I strongly suggest to read C++ Concurrency in Action: by Anthony Williams and leave pthread behind to use futures and similar high-level thing where you can. And if you must go with manual thread fiddling you can find good examples for that too.
Your problem statement is too vague for sensible advice -- the basic idea of good threading is to NOT have shared state at all, and for handshake situation like yours is likely, use some synchronized queue made for that very purpose.
A smarter way to do this would use std::mutex and std::thread (or the Boost equivalents) so you don't need to unlock mutexes manually.
A condition variable will allow the consumers to block (without wasting CPU cycles) until there is work available for them:
struct Resource
{
int value;
std::mutex mx;
std::condition_variable cv;
};
void incResources(Resource& res)
{
while(1)
{
{
std::lock_guard<std::mutex> l(res.mx);
res.value += 2;
}
res.cv.notify_all();
}
}
void consumeResources(Resource& res)
{
while(1)
{
std::unique_lock<std::mutex> l(res.mx);
while (res.value == 0)
res.cv.wait(l);
res.value--;
}
}
and in the main thread:
Resources res;
res.value = 0;
std::thread t1(incResources, std::ref(res));
std::thread t2(consumeResources, std::ref(res));
std::thread t3(consumeResources, std::ref(res));
// ...
t1.join();
t2.join();
t3.join();
I think if you're using C++ there's no reason why to prefer native use of pthreads over the C++11 std::thread and STL synchronization classes.
If you can't use C++11 standards you should wrap the pthreads native interface to reasonable C++ class representations (see e.g. boost::thread or STTCL Posix Thread implementation).
It looks like you are attempting to implement a producer and consumer, with the += thread creating work (numbers to be reduced) and the consumer taking them away.
Rather than having the consumer in a trivial spin loop like that, take a look at condition variables.
std::queue<Job*> queue;
pthread_mutex mutex;
pthread_cond cond;
void AddJob(Job* job) {
pthread_mutex_lock(&mutex);
queue.push_back(job);
pthread_cond_signal(&cond);
pthread_mutex_unlock(&mutex);
}
void* QueueWorker(void* /*threadInfo*/) {
Job* job = NULL;
for (;;) {
pthread_mutex_lock(&mutex);
while ( queue.empty() ) {
// unlock the mutex until the cond is signal()d or broadcast() to.
// if this call succeeds, we will have the mutex locked again on the other side.
pthread_cond_wait(&cond, &mutex);
}
// take the first task and then release the lock.
job = queue.pop();
pthread_mutex_unlock(&mutex);
if ( job != NULL )
job->Execute();
}
return NULL;
}
This scales to multiple consumers.
As an aside, while it can be useful to familiarize yourself with the pthreads implementation, you should probably look at one of the threading wrappers available. C++11 introduced std::thread and std::mutex, many people swear by boost, but personally I've found the OpenSceneGraph team's "OpenThreads" library one of the easiest and most elegant to work with.
Edit: here's a complete working implementation albeit with a somewhat artificial mechanism for ending the run.
#include <queue>
#include <pthread.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
static int jobNo = 0;
class Job {
public:
Job() : m_i(++jobNo) { printf("Created job %d.\n", m_i); }
int m_i;
void Execute() { printf("Job %d executing.\n", m_i); usleep(500 * 1000); }
};
std::queue<Job*> queue;
pthread_mutex_t mutex = PTHREAD_MUTEX_INITIALIZER;
pthread_cond_t cond = PTHREAD_COND_INITIALIZER;
void AddJob(Job* job) {
pthread_mutex_lock(&mutex);
queue.push(job);
pthread_cond_signal(&cond);
pthread_mutex_unlock(&mutex);
}
void* QueueWorker(void* /*threadInfo*/) {
Job* job = NULL;
for (;;) {
pthread_mutex_lock(&mutex);
while ( queue.empty() ) {
// unlock the mutex until the cond is signal()d or broadcast() to.
// if this call succeeds, we will have the mutex locked again on the other side.
pthread_cond_wait(&cond, &mutex);
}
// take the first task and then release the lock.
job = queue.front();
queue.pop();
pthread_mutex_unlock(&mutex);
if ( job == NULL ) {
// in this demonstration, NULL ends the run, so forward to any other threads.
AddJob(NULL);
break;
}
job->Execute();
delete job;
}
return NULL;
}
int main(int argc, const char* argv[]) {
pthread_t worker1, worker2;
pthread_create(&worker1, NULL, &QueueWorker, NULL);
pthread_create(&worker2, NULL, &QueueWorker, NULL);
srand(time(NULL));
// queue 5 jobs with delays.
for ( size_t i = 0; i < 5; ++i ) {
long delay = (rand() % 800) * 1000;
printf("Producer sleeping %fs\n", (float)delay / (1000*1000));
usleep(delay);
Job* job = new Job();
AddJob(job);
}
// 5 more without delays.
for ( size_t i = 0; i < 5; ++i ) {
AddJob(new Job);
}
// null to end the run.
AddJob(NULL);
printf("Done with jobs.\n");
pthread_join(worker1, NULL);
pthread_join(worker2, NULL);
return 0;
}