Avoiding data race condition between two ROS subscriber callback funtions - c++

Let's suppose we've two ROS subscriber callback functions where the callback is called every time a new message is extracted from the queue and we want to use the value of a callback in another callback and vice versa.
I've implemented this in a class with two member variables that store my data.
I suspect a possible race condition between the two callbacks. I tried to create a simple example below.
class <class_name>
{
public:
<var_1_type> get_var_1() {
return var_1;
}
set_var_1(const <var_1_type> value) {
var_1 = value;
}
<var_2_type> get_var_2() {
return var_2;
}
set_var_2(const <var_2_type> value) {
var_2 = value;
}
private:
<var_1_type> var_1;
<var_2_type> var_2;
void callback_function_1(<msg_type> &msg_holder);
void callback_function_2(<msg_type> &msg_holder);
};
void <class_name>::callback_function_1(<msg_type> &msg_holder)
{
set_var_1(msg_holder.data);
// Use var_1 and var_2 to create a new data, e.g.,
<var_3_type> var_3 = get_var_1() * get_var_2();
// Now we can publish the var_3, which is the output of the node.
var_3_pub.publish(var_3);
}
void <class_name>::callback_function_2(<msg_type> &msg_holder)
{
// Use the var_1 and msg_holder.data to calculate var_2, e.g.,
<var_2_type> var_2_ = get_var_1() + msg_holder.data;
set_var_2(var_2_);
}
int main(int argc, char** argv)
{
// Instantiate an object of type <class_name>
// Go into ros spin and let the callback functions do all the work.
return 0;
}
In my particular application, var_1 and var_2 are in fact, 2D vectors that can be thought as a matrix, I don't want the contents of the matrix to be modified by one callback function while it's being used by the other callback function.
I'm somewhat familiar with the use of std::lock_guard<std::mutex> guard(mu);, mu.lock();, mu.unlock(). However, I cannot immediately see a way of using <mutex> in this case. Any help is appreciated.

You simply can make use of a Lock Guard using the same mutex instance in each of your callbacks like:
std::mutex mutex_;
void <class_name>::callback_function_1(<msg_type> &msg_holder)
{
const std::lock_guard<std::mutex> lock(mutex_);
//Locked code here
//The mutex is automatically released when lock goes out of scope (function left)
}
void <class_name>::callback_function_2(<msg_type> &msg_holder)
{
const std::lock_guard<std::mutex> lock(mutex_);
//Locked code here
//The mutex is automatically released when lock goes out of scope (function left)
}
The complete code in your callbacks is executed threadsafe with this solution. Note that you also are able to lock more specific parts of your function by creating a thighter scope { ... }.

Related

C++ condition variable without mutexes?

Problem
I think I'm misunderstanding the CV-Mutex design pattern because I'm creating a program that seems to not need a mutex, only CV.
Goal Overview
I am parsing a feed from a website from 2 different accounts. Alice, Bob. The parsing task is slow, so I have two separate threads each dedicated to handling the feeds from Alice and Bob.
I then have a thread that receives messages from the network and assigns the work to either the threadA or threadB, depending on who the update message is for. That way the reader/network thread isn't stalled, and the messages for Alice are in-order and the messages for Bob are in-order, too.
I don't care if Alice thread is a little bit behind Bob thread chronologically, as long as the individual account feeds are in-order.
Implementation Details
This is very similar to a thread pool, except the threads are essentially locked to a fixed-size array of size 2, and I use the same thread for each feed.
I create a AccountThread class which maintains a queue of JSON messages to be processed as soon as possible within the class. Here is the code for that:
#include <queue>
#include <string>
#include <condition_variable>
#include <mutex>
using namespace std;
class AccountThread {
public:
AccountThread(const string& name) : name(name) { }
void add_message(const string& d) {
this->message_queue.push(d);
this->cv.notify_all(); // could also do notify_one but whatever
}
void run_parsing_loop() {
while (true) {
std::unique_lock<std::mutex> mlock(lock_mutex);
cv.wait(mlock, [&] {
return this->is_dead || this->message_queue.size() > 0;
});
if (this->is_dead) { break; }
const auto message = this->message_queue.front();
this->message_queue.pop();
// Do message parsing...
}
}
void kill_thread() {
this->is_dead = true;
}
private:
const string& name;
condition_variable cv;
mutex lock_mutex;
queue<string> message_queue;
// To Kill Thread if Needed
bool is_dead;
};
I can add the main.cpp code, but it's essentially just a reader loop that calls thread.add_message(message) based on what the account name is.
Question
Why do I need the lock_mutex here? I don't see it's purpose since this class is essentially single-threaded. Is there a better design pattern for this? I feel like if I'm including a variable that I don't really need, such as the mutex then I'm using the wrong design pattern for this task.
I'm just adapting the code from some article I saw online about a threadpool implementation and was curious.
First things first: there's no condition_variable::wait without a mutex. The interface of wait requires a mutex. So regarding
I'm creating a program that seems to not need a mutex, only CV
note that the mutex is needed to protect the condition variable itself. If the notion of how you'd have a data race without the mutex doesn't immediately make sense, check Why do pthreads’ condition variable functions require a mutex.
Secondly there's multiple pain points in the code you provide. Consider this version where the problems are addressed and I'll explain the issues below:
class AccountThread {
public:
AccountThread(const string& name) : name(name)
{
consumer = std::thread(&AccountThread::run_parsing_loop, this); // 1
}
~AccountThread()
{
kill_thread(); // 2
consumer.join();
}
void add_message(const string& d) {
{
std::lock_guard lok(lock_mutex); // 3
this->message_queue.push(d);
}
this->cv.notify_one();
}
private:
void run_parsing_loop()
{
while (!is_dead) {
std::unique_lock<std::mutex> mlock(lock_mutex);
cv.wait(mlock, [this] { // 4
return is_dead || !message_queue.empty();
});
if (this->is_dead) { break; }
std::string message = this->message_queue.front();
this->message_queue.pop();
string parsingMsg = name + " is processing " + message + "\n";
std::cout << parsingMsg;
}
}
void kill_thread() {
{
std::lock_guard lock(lock_mutex);
this->is_dead = true;
}
cv.notify_one(); // 5
}
private:
string name; // 6
mutable condition_variable cv; // 7
mutable mutex lock_mutex;
std::thread consumer;
queue<string> message_queue;
bool is_dead{false}; // 8
};
Top to bottom the problems noted (in the numbered comments are):
If you have a worker thread class, like AccountThread, it's easier to get right when the class provides the thread. This way only the relevant interface is exposed and you have better control over the lifetime and workings of the consumer.
Case in point, when an AccountThread "dies" the worker should also die. In the example above I fix this dependency by killing the consumer thread inside the destructor.
add_message caused a data race in your code. Since you intend to run the parsing loop in a different thread, it's wrong to simply push to the queue without having a critical section.
It's cleaner to capture this here, e.g. you probably don't need the reference to mlock captured.
kill_thread was not correct. You need to notify the, potentially waiting, consumer thread that a change in state happened. To correctly do that you need to protect the state checked in the predicate with a lock.
The initial version with const string &name is probably not something you want. Member const references don't extend the lifetime of temporaries, and the way your constructor is written can leave an instance with dangling state. Even if you do the typical checks, overload the constructor with an r-value reference version, you'll be depending on an external string being alive longer than your AccountThread object. Better use a value member.
Remember the M&M rule.
You had undefined behavior. The is_alive member was used without being initialized.
Demo
All in all, I think the suggested changes point in the right direction. You can also check an implementation of a Go-like communication channel if you want more insight on how something like the TBB component you mention is implemented. Such a channel (or buffer queue) would simplify implementation to avoid manual usage of mutexes, CVs and alive states:
class AccountThread {
public:
AccountThread(const string& name) : name(name) {
consumer = std::thread(&AccountThread::run_parsing_loop, this);
}
~AccountThread() {
kill_thread();
consumer.join();
}
void add_message(const string& d) { _data.push(d); }
private:
void run_parsing_loop() {
try {
while (true) {
// This pop waits until there's data or the channel is closed.
auto message = _data.pop();
// TODO: Implement parsing here
}
} catch (...) {
// Single exception thrown per thread lifetime
}
}
void kill_thread() { _data.set(yap::BufferBehavior::Closed); }
private:
string name;
std::thread consumer;
yap::BufferQueue<string> _data;
};
Demo2

Is there a way to protect a smart pointer from being deallocated on one thread, when work is being done on another thread?

In our program, we have a class FooLogger which logs specific events (strings). We use the FooLogger as a unique_ptr.
We have two threads which use this unique_ptr instance:
Thread 1 logs the latest event to file in a while loop, first checking if the instance is not nullptr
Thread 2 deallocates the FooLogger unique_ptr instance when the program has reached a certain point (set to nullptr)
However, due to bad interleaving, it is possible that, while logging, the member variables of FooLogger are deallocated, resulting in an EXC_BAD_ACCESS error.
class FooLogger {
public:
FooLogger() {};
void Log(const std::string& event="") {
const float32_t time_step_s = timer_.Elapsed() - runtime_s_; // Can get EXC_BAD_ACCESS on timer_
runtime_s_ += time_step_s;
std::cout << time_step_s << runtime_s_ << event << std::endl;
}
private:
Timer timer_; // Timer is a custom class
float32_t runtime_s_ = 0.0;
};
int main() {
auto foo_logger = std::make_unique<FooLogger>();
std::thread foo_logger_thread([&] {
while(true) {
if (foo_logger)
foo_logger->Log("some event");
else
break;
}
});
SleepMs(50); // pseudo code
foo_logger = nullptr;
foo_logger_thread.join();
}
Is it possible, using some sort of thread synchronisation/locks etc. to ensure that the foo_logger instance is not deallocated while logging? If not, are there any good ways of handling this case?
The purpose of std::unique_ptr is to deallocate the instance once std::unique_ptr is out of scope. In your case, you have multiple threads each having access to the element and the owning thread might get eliminated prior to other users.
You either need to ensure that owner thread never gets deleted prior to the user threads or change ownership model from std::unique_ptr to std::shared_ptr. It is the whole purpose of std::shared_ptr to ensure that the object is alive as long as you use it.
You just need to figure out what's required for program and use the right tools to achieve it.
Use a different mechanism than the disappearance of an object for determining when to stop.
(When you use a single thing for two separate purposes, you often get into trouble.)
For instance, an atomic bool:
int main() {
FooLogger foo_logger;
std::atomic<bool> keep_going = true;
std::thread foo_logger_thread([&] {
while(keep_going) {
foo_logger.Log("some event");
}
});
SleepMs(50);
keep_going = false;
foo_logger_thread.join();
}
It sounds like std::weak_ptr can help in this case.
You can make one from a std::shared_ptr and pass it to the logger thread.
For example:
class FooLogger {
public:
void Log(std::string const& event) {
// log the event ...
}
};
int main() {
auto shared_logger = std::make_shared<FooLogger>();
std::thread foo_logger_thread([w_logger = std::weak_ptr(shared_logger)]{
while (true) {
auto logger = w_logger.lock();
if (logger)
logger->Log("some event");
else
break;
}
});
// some work ...
shared_logger.reset();
foo_logger_thread.join();
}
Use should use make_shared instead of make_unique. And change:
std::thread foo_logger_thread([&] {
to
std::thread foo_logger_thread([foo_logger] {
It will create new instance of shared_ptr.

Ensure that a thread doesn't lock a mutex twice?

Say I have a thread running a member method like runController in the example below:
class SomeClass {
public:
SomeClass() {
// Start controller thread
mControllerThread = std::thread(&SomeClass::runController, this)
}
~SomeClass() {
// Stop controller thread
mIsControllerThreadInterrupted = true;
// wait for thread to die.
std::unique_lock<std:::mutex> lk(mControllerThreadAlive);
}
// Both controller and external client threads might call this
void modifyObject() {
std::unique_lock<std::mutex> lock(mObjectMutex);
mObject.doSomeModification();
}
//...
private:
std::mutex mObjectMutex;
Object mObject;
std::thread mControllerThread;
std::atomic<bool> mIsControllerInterrupted;
std::mutex mControllerThreadAlive;
void runController() {
std::unique_lock<std::mutex> aliveLock(mControllerThreadAlive);
while(!mIsControllerInterruped) {
// Say I need to synchronize on mObject for all of these calls
std::unique_lock<std::mutex> lock(mObjectMutex);
someMethodA();
modifyObject(); // but calling modifyObject will then lock mutex twice
someMethodC();
}
}
//...
};
And some (or all) of the subroutines in runController need to modify data that is shared between threads and guarded by a mutex. Some (or all) of them, might also be called by other threads that need to modify this shared data.
With all the glory of C++11 at my disposal, how can I ensure that no thread ever locks a mutex twice?
Right now, I'm passing unique_lock references into the methods as parameters as below. But this seems clunky, difficult to maintain, potentially disastrous, etc...
void modifyObject(std::unique_lock<std::mutex>& objectLock) {
// We don't even know if this lock manages the right mutex...
// so let's waste some time checking that.
if(objectLock.mutex() != &mObjectMutex)
throw std::logic_error();
// Lock mutex if not locked by this thread
bool wasObjectLockOwned = objectLock.owns_lock();
if(!wasObjectLockOwned)
objectLock.lock();
mObject.doSomeModification();
// restore previous lock state
if(!wasObjectLockOwned)
objectLock.unlock();
}
Thanks!
There are several ways to avoid this kind of programming error. I recommend doing it on a class design level:
separate between public and private member functions,
only public member functions lock the mutex,
and public member functions are never called by other member functions.
If a function is needed both internally and externally, create two variants of the function, and delegate from one to the other:
public:
// intended to be used from the outside
int foobar(int x, int y)
{
std::unique_lock<std::mutex> lock(mControllerThreadAlive);
return _foobar(x, y);
}
private:
// intended to be used from other (public or private) member functions
int _foobar(int x, int y)
{
// ... code that requires locking
}

C++ return value on concurrent queue pushing functions

After receiving answers to a previous question on logging on a different thread, I am currently at the following bit of code (note: the concurrent_queue here is from ppl, but any other concurrent_queue should work):
class concurrentFuncQueue
{
private:
typedef std::function<void()> LambdaFunction;
mutable concurrency::concurrent_queue<LambdaFunction> functionQueue;
mutable std::atomic<bool> endcond;
LambdaFunction function;
std::thread thd;
public:
concurrentFuncQueue() : endcond(false), thd([=]{
while (endcond != true)
{
if (functionQueue.try_pop( function ))
{
function(); //note: I am popping a function and adding () to execute it
}
}
}){}
~concurrentFuncQueue() { functionQueue.push([=]{ endcond = true; }); thd.join(); }
void pushFunction(LambdaFunction function) const { functionQueue.push(function); }
};
Basically the functions I push are run on a different thread sequentially (ex. a logging function) as to avoid performance issues on the main thread.
Current usage is along the following:
static concurrentFuncQueue Logger;
vector<char> outstring(256);
Logger.pushFunction([=]{ OutputDebugString(debugString.c_str()) });
Great so far. I can push functions on to a concurrent queue that will run my functions sequentially on a separate thread.
One thing I also need to have, but currently don't are return values so that ex (pseudo-code):
int x = y = 3;
auto intReturn = Logger.pushFunction([=]()->int { return x * y; });
will push x * y on to the concurrent queue, and after the pop and completion of the function (on the other thread), returns the value calculated to the caller thread.
(I understand that I'll be blocking the caller thread until the pushed function is returned. That is exactly what I want)
I get the feeling that I might have to use something along the line of std::promise, but sadly my current low understanding of them prevent me from formulating something codable.
Any ideas? Thoughts on the above C++ code and any other comments are also much welcome (please just ignore the code completely if you feel another implementation is more appropriate or solves the problem).
You should be able to use something along the lines of:
template<typename Foo>
std::future<typename std::result_of<Foo()>::type> pushFunction(Foo&& f) {
using result_type = typename std::result_of<Foo()>::type; // change to typedef if using is not supported
std::packaged_task<result_type()> t(f);
auto ret_fut = t.get_future();
functionQueue.push(std::move(t));
return ret_fut;
}
For this to work you need to make your LambdaFunction a type-erased function handler.

C++ Critical Section not working

My critical section code does not work!!!
Backgrounder.run IS able to modify MESSAGE_QUEUE g_msgQueue and LockSections destructor hadn't been called yet !!!
Extra code :
typedef std::vector<int> MESSAGE_LIST; // SHARED OBJECT .. MUST LOCK!
class MESSAGE_QUEUE : MESSAGE_LIST{
public:
MESSAGE_LIST * m_pList;
MESSAGE_QUEUE(MESSAGE_LIST* pList){ m_pList = pList; }
~MESSAGE_QUEUE(){ }
/* This class will be shared between threads that means any
* attempt to access it MUST be inside a critical section.
*/
void Add( int messageCode ){ if(m_pList) m_pList->push_back(messageCode); }
int getLast()
{
if(m_pList){
if(m_pList->size() == 1){
Add(0x0);
}
m_pList->pop_back();
return m_pList->back();
}
}
void removeLast()
{
if(m_pList){
m_pList->erase(m_pList->end()-1,m_pList->end());
}
}
};
class Backgrounder{
public:
MESSAGE_QUEUE* m_pMsgQueue;
static void __cdecl Run( void* args){
MESSAGE_QUEUE* s_pMsgQueue = (MESSAGE_QUEUE*)args;
if(s_pMsgQueue->getLast() == 0x45)printf("It's a success!");
else printf("It's a trap!");
}
Backgrounder(MESSAGE_QUEUE* pMsgQueue)
{
m_pMsgQueue = pMsgQueue;
_beginthread(Run,0,(void*)m_pMsgQueue);
}
~Backgrounder(){ }
};
int main(){
MESSAGE_LIST g_List;
CriticalSection crt;
ErrorHandler err;
LockSection lc(&crt,&err); // Does not work , see question #2
MESSAGE_QUEUE g_msgQueue(&g_List);
g_msgQueue.Add(0x45);
printf("%d",g_msgQueue.getLast());
Backgrounder back_thread(&g_msgQueue);
while(!kbhit());
return 0;
}
#ifndef CRITICALSECTION_H
#define CRITICALSECTION_H
#include <windows.h>
#include "ErrorHandler.h"
class CriticalSection{
long m_nLockCount;
long m_nThreadId;
typedef CRITICAL_SECTION cs;
cs m_tCS;
public:
CriticalSection(){
::InitializeCriticalSection(&m_tCS);
m_nLockCount = 0;
m_nThreadId = 0;
}
~CriticalSection(){ ::DeleteCriticalSection(&m_tCS); }
void Enter(){ ::EnterCriticalSection(&m_tCS); }
void Leave(){ ::LeaveCriticalSection(&m_tCS); }
void Try();
};
class LockSection{
CriticalSection* m_pCS;
ErrorHandler * m_pErrorHandler;
bool m_bIsClosed;
public:
LockSection(CriticalSection* pCS,ErrorHandler* pErrorHandler){
m_bIsClosed = false;
m_pCS = pCS;
m_pErrorHandler = pErrorHandler;
// 0x1AE is code prefix for critical section header
if(!m_pCS)m_pErrorHandler->Add(0x1AE1);
if(m_pCS)m_pCS->Enter();
}
~LockSection(){
if(!m_pCS)m_pErrorHandler->Add(0x1AE2);
if(m_pCS && m_bIsClosed == false)m_pCS->Leave();
}
void ForceCSectionClose(){
if(!m_pCS)m_pErrorHandler->Add(0x1AE3);
if(m_pCS){m_pCS->Leave();m_bIsClosed = true;}
}
};
/*
Safe class basic structure;
class SafeObj
{
CriticalSection m_cs;
public:
void SafeMethod()
{
LockSection myLock(&m_cs);
//add code to implement the method ...
}
};
*/
#endif
Two questions in one. I don't know about the first, but the critical section part is easy to explain. The background thread isn't trying to claim the lock and so, of course, is not blocked. You need to make the critical section object crt visible to the thread so that it can lock it.
The way to use this lock class is that each section of code that you want serialised must create a LockSection object and hold on to it until the end of the serialised block:
Thread 1:
{
LockSection lc(&crt,&err);
//operate on shared object from thread 1
}
Thread 2:
{
LockSection lc(&crt,&err);
//operate on shared object from thread 2
}
Note that it has to be the same critical section instance crt that is used in each block of code that is to be serialised.
This code has a number of problems.
First of all, deriving from the standard containers is almost always a poor idea. In this case you're using private inheritance, which reduces the problems, but doesn't eliminate them entirely. In any case, you don't seem to put the inheritance to much (any?) use anyway. Even though you've derived your MESSAGE_QUEUE from MESSAGE_LIST (which is actually std::vector<int>), you embed a pointer to an instance of a MESSAGE_LIST into MESSAGE_QUEUE anyway.
Second, if you're going to use a queue to communicate between threads (which I think is generally a good idea) you should make the locking inherent in the queue operations rather than requiring each thread to manage the locking correctly on its own.
Third, a vector isn't a particularly suitable data structure for representing a queue anyway, unless you're going to make it fixed size, and use it roughly like a ring buffer. That's not a bad idea either, but it's quite a bit different from what you've done. If you're going to make the size dynamic, you'd probably be better off starting with a deque instead.
Fourth, the magic numbers in your error handling (0x1AE1, 0x1AE2, etc.) is quite opaque. At the very least, you need to give these meaningful names. The one comment you have does not make the use anywhere close to clear.
Finally, if you're going to go to all the trouble of writing code for a thread-safe queue, you might as well make it generic so it can hold essentially any kind of data you want, instead of dedicating it to one specific type.
Ultimately, your code doesn't seem to save the client much work or trouble over using the Windows functions directly. For the most part, you've just provided the same capabilities under slightly different names.
IMO, a thread-safe queue should handle almost all the work internally, so that client code can use it about like it would any other queue.
// Warning: untested code.
// Assumes: `T::T(T const &) throw()`
//
template <class T>
class queue {
std::deque<T> data;
CRITICAL_SECTION cs;
HANDLE semaphore;
public:
queue() {
InitializeCriticalSection(&cs);
semaphore = CreateSemaphore(NULL, 0, 2048, NULL);
}
~queue() {
DeleteCriticalSection(&cs);
CloseHandle(semaphore);
}
void push(T const &item) {
EnterCriticalSection(&cs);
data.push_back(item);
LeaveCriticalSection(&cs);
ReleaseSemaphore(semaphore, 1, NULL);
}
T pop() {
WaitForSingleObject(semaphore, INFINITE);
EnterCriticalSection(&cs);
T item = data.front();
data.pop_front();
LeaveCriticalSection(&cs);
return item;
}
};
HANDLE done;
typedef queue<int> msgQ;
enum commands { quit, print };
void backgrounder(void *qq) {
// I haven't quite puzzled out what your background thread
// was supposed to do, so I've kept it really simple, executing only
// the two commands listed above.
msgQ *q = (msgQ *)qq;
int command;
while (quit != (command = q->pop()))
printf("Print\n");
SetEvent(done);
}
int main() {
msgQ q;
done = CreateEvent(NULL, false, false, NULL);
_beginthread(backgrounder, 0, (void*)&q);
for (int i=0; i<20; i++)
q.push(print);
q.push(quit);
WaitForSingleObject(done, INFINITE);
return 0;
}
Your background thread needs access to the same CriticalSection object and it needs to create LockSection objects to lock it -- the locking is collaborative.
You are trying to return the last element after popping it.