Or... how to properly combine Concurrency, RAII and Polymorphism?
This is a very practical issue. We were bitten by this combination, summarized as Chandler Carruth's terrifying bug (at 1:18:45 mark)!
If you like bugs, try to catch the puzzle here (adapted from Chandler's talk):
#include <condition_variable>
#include <iostream>
#include <mutex>
#include <thread>
class A {
public:
virtual void F() = 0;
void Done() {
std::lock_guard<std::mutex> l{m};
is_done = true;
cv.notify_one();
std::cout << "Called Done..." << std::endl;
}
virtual ~A() {
std::unique_lock<std::mutex> l{m};
std::cout << "Waiting for Done..." << std::endl;
cv.wait(l, [&] {return is_done;});
std::cout << "Destroying object..." << std::endl;
}
private:
std::mutex m;
std::condition_variable cv;
bool is_done{false};
};
class B: public A {
public:
virtual void F() {}
~B() {}
};
int main() {
A *obj{new B{}};
std::thread t1{[=] {
obj->F();
obj->Done();
}};
delete obj;
t1.join();
return 0;
}
The issue (spotted when compiled via clang++ -fsanatize=thread) boils down to a race between a read of the virtual table (polymorphism) and a write on it (before entering ~A). The write being done as part of the destruction chain (so no method from B is called in A's destructor).
The recommended workaround is to move the synchronization outside the destructor, forcing each client of the class to call a WaitUntilDone/Join method. This is easy to forget and that's exactly why we wanted to use the RAII idiom in the first place.
Thus, my questions are:
Is there a nice way to enforce synchronization in base destructor?
Out of curiosity, why on earth the virtual table is even used from the destructor? I would have expected static binding here.
You can use an RAII wrapper for objects that can't support RAII semantics directly. The wrapper object's constructor can construct the inner object and then do anything that's logically part of construction but can only be done after the inner object's constructor returns. Similarly, the wrapper object's destructor can do anything that has to be done before the inner object's destructor can safely be invoked and then it can destroy the inner object.
Related
Assuming we have the classical Base class and derived class like this
class B {
public:
virtual ~B() {
// calling it here is too late, see explanations
//common_pre_cleanup_function();
}
void common_pre_cleanup_function() { }
};
class D : public B {
public:
virtual ~D() {
// What if we forget to do this call in another derived class?
common_pre_cleanup_function();
}
};
How would you make sure a function like common_pre_cleanup_function() is called in all derived Ds destructors before the members of D are destroyed but without having to explicitly call this function in every destructor-implementation of a new D?
Background
In my current project we have a base class that implements certain parallelism and threading features and will eventually start a new thread that does the actual work.
In the destructor of this base class we wanted to make sure, that the thread is always stopped and joined so that it gets cleaned up properly.
However derived classes may create members that are used by this thread in the base class. So if we destroy objects of the derived class, these members are also destroyed. But at this time the thread that is managed by the base class can still be running and now wrongfully access destroyed members.
I'm aware that this isn't the smartest approach to solve the issue and probably splitting up the threading/parallelisation parts and the "actual work" parts into separate classes might be the much smarter idea. However I'm interested if there are any approaches that don't involve an entire rewrite of the existing code base.
This code here is closer to our situation
class BackgroundTask {
public:
virtual ~BackgroundTask() {
// if we forget to call stop() in the derived classes, we will
// at this point have already destroyed any derived members
// while the thread might still run and access them; so how/where
// can we put this call?
//stop();
}
void stop() {
cancelFlag_.set();
thread_.join();
}
// more functions helping with Background tasks
private:
Thread thread_;
Condition cancelFlag_;
};
class MyTask : public BackgroundTask {
public:
virtual ~MyTask() {
// with the current case, we have to remember to call
// this function in all destructors in classes derived
// from BackgroundTask; that's what I want to avoid
stop();
}
private:
std::unique_ptr<MyClass> member;
};
Quite simply you don't. The best thing to do in this situation is to redesign how everything works to prevent this from being a problem.
But lets face it, in all likelihood you don't have the time and/or resources to achieve that. So your second best option (in my opinion) is to ensure that any call to the destroyed members of the derived class kills you application immediately with a very clear error message.
If a system must fail, fail early.
You might do something like:
template <typename TaskImpl>
class Task final : public TaskImpl
{
static_assert(std::is_base_of<BackgroundTask, TaskImpl>);
public:
virtual ~Task() { stop(); }
};
And then
class MyTaskImpl : public BackgroundTask
{
// ...
private:
std::unique_ptr<MyClass> member;
};
using MyTask = Task<MyTaskImpl>;
While I agree with comments that the design is flawed .....
Assuming that the objects are dynamically allocated, one solution is to make the destructors virtual and protected, and use a separate function to take care of calling the "pre-cleanup" before destroying the objects. For example;
class B
{
public:
void die()
{
common_pre_cleanup_function();
delete this;
};
protected:
virtual ~B() {};
private:
void common_pre_cleanup_function() { };
};
class D : public B
{
protected:
virtual ~D() {};
};
int main()
{
B *b = new D;
b->die();
}
This has a few limitations for the user of the class. In particular, behaviour is undefined if
the object is not created using a new expression;
any non-static member function of the object is called after calling die()
any non-static data member is accessed after calling die()
This also means that, if you maintain a set of objects (like a vector of pointers, B*) then it is necessary to remove the pointer from the list to ensure no usage of the object after it has died.
The protected destructors prevent a few things. Functions that are not members of friends of B or D cannot;
Create a B or a D of automatic storage duration
Use operator delete directly. For example, a statement delete b; in main() above will not compile. This also prevents destroying an object before calling the "pre-cleanup"
Edit: I realized this doesn't aswer your question but I'll leave it here for reference.
As mentioned earlier, each object should be responsible for managing its own resources so your design is a bit flawed to begin with.
Consider the following example. The TaskRunner is responsible for firing up a thread, and shutting it down when the constructor is called (textbook RAII). The Task class specifies what to do during the lifetime of the task, through pure virtual inheritance.
#include <atomic>
#include <future>
#include <iostream>
#include <memory>
struct Task {
virtual void run( ) = 0;
virtual ~Task( ) {
}
};
class TaskRunner final {
std::unique_ptr<Task> task;
std::future<void> fut;
std::atomic<bool> terminate;
public:
TaskRunner(std::unique_ptr<Task>&& task)
: task {std::move(task)}
, terminate {false} {
fut = std::async(std::launch::async, [this] {
while(!terminate) {
this->task->run( );
}
this->task.reset( );
});
}
TaskRunner(TaskRunner&&) = delete;
TaskRunner& operator=(TaskRunner&&) = delete;
TaskRunner(const TaskRunner&) = delete;
TaskRunner& operator=(const TaskRunner&) = delete;
~TaskRunner( ) {
terminate = true;
fut.wait( ); // Block until cleanup is completed
std::cout << "~TaskRunner()" << std::endl;
}
};
struct MyTask : public Task {
int i = 0;
void
run( ) {
// Do important stuf here, don't block.
std::cout << "MyTask::run() " << i++ << std::endl;
std::this_thread::sleep_for(std::chrono::milliseconds {100});
}
~MyTask( ) override {
// Clean up stuff here, run() is guaranteed to never be run again
std::cout << "~MyTask()" << std::endl;
}
};
int
main( ) {
TaskRunner t {std::make_unique<MyTask>( )};
std::this_thread::sleep_for(std::chrono::seconds {1});
}
Output
MyTask::run() 0
MyTask::run() 1
MyTask::run() 2
MyTask::run() 3
MyTask::run() 4
MyTask::run() 5
MyTask::run() 6
MyTask::run() 7
MyTask::run() 8
MyTask::run() 9
~MyTask()
~TaskRunner()
Suppose I have the following code
#include <thread>
#include <iostream>
#include <atomic>
struct FooBase {
void start(){
run_condition_ = true;
t_ = std::thread([this](){
thread_handler();
});
}
virtual ~FooBase(){
run_condition_ = false;
if(t_.joinable())
t_.join();
}
protected:
virtual void thread_handler() = 0;
std::atomic_bool run_condition_{false};
private:
std::thread t_;
};
struct Foo : FooBase {
void thread_handler() override {
while(run_condition_){
std::cout << "Foo derived thread.." << std::endl;
}
}
};
int main(){
Foo f;
f.start();
getchar();
return 0;
}
Here I think because the destructor of the derived class Foo is called before FooBase the thread_handler vtable lookup happens in the base class IF the thread has not yet joined (still running) when the destructor of Foo is done. Since FooBase::thread_handler is pure virtual I am essentially guranteed a sigabort.
How do I guard against this? I hack my way through by not having thread_handler as pure virtual
virtual void thread_handler(){}
But I am lost as to how I can guard against this in the baseclass itself, I can implement a join_thread interface in the base class and call this from every derived class, but this seems cumbersome.
There's two issues here, neither of which match precisely what you described.
Your thread only gets stopped in ~FooBase(). This means that if Foo::thread_handler ever reads or writes to any of its members, they will get destroyed out from under it before the thread is stopped.
It you get to the destructor fast enough, it's possible that start() won't have actually invoked thread_handler() on the new thread by the time Foo gets destroyed - which will lead to the pure virtual call.
Either way, you need to ensure that by the time Foo is destroyed, anything related to thread_handler is done. This implies that every derived class from FooBase has to have, in its destructor:
run_condition_ = false;
if (t_.joinable()) {
t_join();
}
Setting aside that this directly doesn't work because t_ is private (you could wrap that into a protected stop()), it's an awkward design if all of your derived classes need to do something special just to work. You could instead put FooBase into its own class that just takes an arbitrary callable as an argument:
class joining_thread {
public:
joining_thread() = default;
~joining_thread() { stop(); }
bool running() const { return run_condition_.load(); }
template <typename... Args>
void start(Args&&... args) {
run_condition_ = true;
t_ = std::thread(std::forward<Args>(args)...);
}
void stop() {
run_condition_ = false;
if (t_.joinable()) t_.join();
}
private:
std::thread t_;
std::atomic_bool run_condition_{false};
};
And then your Foo can just have that as a member:
class Foo {
public:
void start() {
t_.start([this]{
while (t_.running()) { ... }
});
}
private:
// just make me the last member, so it's destroyed first
joining_thread t_;
};
That's still a little awkward with the whole running() thing, but hopefully the idea makes sense.
What you describe is not possible. You call "start" after you have constructed the object. The object is at that stage valid. You have avoided the common problem of calling a virtual function in the constructor, which would have caused issues. There is something called a memory barrier that is implied by any thread calls, so you can count on the fact the new thread will start with a view of memory that existed at the point it was created. Any thing that existed AND was not changed, is fine.
Your problem (as described in another answer) is that you can exit and destroy the object (and it's vtable), before the thread is complete.
The simplest fix for this is use a packaged task. Calling "get" on the future ensures the task is finished before you continue. Consider the code below
#include "stdafx.h"
#include <thread>
#include <iostream>
#include <atomic>
#include <future>
int main()
{
std::atomic<bool> stop{ false };
std::future<void> sync;
std::packaged_task<void()> task([&stop]()
{
while (!stop)
{
std::cout << "Running\n";
}
});
std::thread thread([&task]() {task();});
getchar();
stop = true;
task.get_future().get();
thread.join();
return 0;
}
I have a base class that can start background thread, and stop it when needed. That thread calls two virtual methods Open() and Close(). So all inherited classes can re-implement this methods, but not starting/stoping thread routine (it more difficult than in example). I want to follow RAII principle and start/stop thid thread in constructor/destructor of base class.
The problem is, that calling virtual methods in constructor/destructor is a bad practice and didn't work in my case.
Here is a shot example of my problem:
#include <iostream>
#include <thread>
#include <atomic>
class Base {
public:
Base() {
bg_thread_ = std::thread([this] {
Open();
while(!is_stop_) {
// do stuff
}
Close();
});
}
~Base() {
is_stop_ = true;
if(bg_thread_.joinable()) {
bg_thread_.join();
}
}
private:
virtual void Open() {
std::cout << "Base open" << std::endl;
}
virtual void Close() {
std::cout << "Base close" << std::endl;
}
std::thread bg_thread_;
std::atomic<bool> is_stop_{false};
};
class Inherited : public Base {
virtual void Open() override {
std::cout << "Inherited open" << std::endl;
}
virtual void Close() override {
std::cout << "Inherited close" << std::endl;
}
};
int main() {
Inherited inherited;
std::this_thread::sleep_for(std::chrono::seconds(1));
return 0;
}
The output is:
Inherited open
Base close
And without sleep is:
Base open
Base close
My current approach is to call Start() method after constructor and Stop() before destructor, but I want solution with RAII.
void Start() {
bg_thread_ = std::thread([this] {
Open();
while(!is_stop_) {
// do stuff
}
Close();
});
}
void Stop() {
is_stop_ = true;
if(bg_thread_.joinable()) {
bg_thread_.join();
}
}
The problem is independent of threads. If you call virtual methods in the constructor of the Base, the Inherited object is not yet created so the Base implementations of the methods are called (or you get an error if they are pure virtual). If you call virtual methods in the destructor of Base the Inherited object is already destroyed so again the Base version of the virtual methods are called again.
Calling the methods from another thread does not change this behaviour. But the starting of the thread might take longer than the construction of the Inherited object so the object is fully constructed and the Inherited methods are called at the beginning of the worker thread.
One solution is to move the RAII to another object. So you don't call Start and Stop in Bases constructor and destructor. Then you can build a StartStopThing which takes a Base (by reference or by pointer) and calls Start and Stop in its constructor and destructor. Or you build a StartStopThing template class which takes Inherited as template argument, builds an Inherited object and calls the Start and Stop methods.
I ran into a weird C++ code behaviour, not sure whether it's a compiler bug or simply undefined/unspecified behaviour of my code. Here is the code:
#include <unistd.h>
#include <iostream>
#include <thread>
struct Parent {
std::thread t;
static void entry(Parent* p) {
p->init();
p->fini();
}
virtual ~Parent() { t.join(); }
void start() { t = std::thread{entry, this}; }
virtual void init() { std::cout << "Parent::init()" << std::endl; }
virtual void fini() { std::cout << "Parent::fini()" << std::endl; }
};
struct Child : public Parent {
virtual void init() override { std::cout << "Child::init()" << std::endl; }
virtual void fini() override { std::cout << "Child::fini()" << std::endl; }
};
int main() {
Child c;
c.start();
sleep(1); // <========== here is it
return 0;
}
The output of the code would be the following, which isn't surprising:
Child::init()
Child::fini()
However, if the function call "sleep(1)" is commented out, the output would be:
Parent::init()
Parent::~fini()
Tested on Ubuntu 15.04, both gcc-4.9.2 and clang-3.6.0 show the same behaviour. Compiler options:
g++/clang++ test.cpp -std=c++11 -pthread
It looks like a race condition (the vtable not fully constructed before the thread starts). Is this code ill-formed ? a compiler bug ? or it's supposed to be like this ?
#KerrekSB commented:
” The thread uses the child object, but the child object is destroyed before the thread is joined (because the joining only happens after the destruction of the child has begun).
The Child object is destroyed at the end of main. The Child destructor is executed, and effectively calls the Parent destructor, where Parent bases (no such) and data members (the thread object) are destroyed. As destructors are invoked up the chain of base classes the dynamic type of the object changes, in reverse order of how it changes during construction, so at this point the type of the object is Parent.
The virtual calls in the thread function can happen before, overlapping with or after the call of the Child destructor, and in the case of overlapping there's one thread accessing storage (in practice, the vtable pointer) that is being changed by another thread. So this is Undefined Behavior.
This is common design issue; what you tried to do is a classical anti-pattern.
Parent cannot be at the same time a thread manager, starting a thread and waiting for thread termination:
virtual ~Parent() { t.join(); }
void start() { t = std::thread{entry, this}; }
and also a thread object:
virtual void init() { std::cout << "Parent::init()" << std::endl; }
virtual void fini() { std::cout << "Parent::fini()" << std::endl; }
These are two distinct concepts, with strictly incompatible specifications.
(And thread objects are not useful in general.)
I have a weird problem with boost shared_ptr:
class A
{
A( )
: m_myObjectPtr( new MyObject( ) )
{
}
protected:
boost::shared_ptr<MyObject> m_myObjectPtr; // MyObject class is a simple class with a constructor and destructor
};
class B : A
{
B( )
{
}
void CleanMyObject( )
{
m_myObjectPtr.reset( );
}
};
class MyObject
{
MyObject( )
{
cout << "Constructed MyObject" << endl;
}
~MyObject( )
{
cout << "Destroyed MyObject" << endl;
}
};
My problem is that the destructor of MyObject is never invoked when I call B::CleanMyObject( ). "Destroyed MyObject" is never printed.
I am seeing this on iOS with arm64 build of boost 1_57 built using https://github.com/danoli3/ofxiOSBoost/blob/master/scripts/build-libc%2B%2B.sh
Any ideas?
The obvious answer is that you have multiple shared_ptrs referring to a single object, so resetting one reduces the reference count but doesn't delete the object.
This can happen even if the shared_ptr isn't referenced outside of A and B. If you assign A or B without overloading operator= or copy A or B (e.g., pass by value, return by value) without overloading the copy constructor, then this will result.
There are several ways you could investigate.
You could check boost::shared_ptr::use_count() within your CleanMyObject to see if it's greater than 1.
If you don't want sharing and reference counting, you could replace boost::shared_ptr with std::unique_ptr or boost::scoped_ptr.
If you want to make sure you're not accidentally copying or assigning A or B, you can derive A from boost::noncopyable.
Going by #Josh Kelley's suggestion, using unique_ptr solved the problem for me.
So I diverted my attention to boost::shared_ptr to understand what is the problem. It turns out that building boost for iOS with flags BOOST_AC_USE_PTHREADS and BOOST_SP_USE_PTHREADS was the original reason for the oddity. Based on:
1) Answer by Andy Weinstein: Boost threads: in IOS, thread_info object is being destructed before the thread finishes executing
2) Weakly ordered CPU tutorial (http://preshing.com/20121019/this-is-why-they-call-it-a-weakly-ordered-cpu/)
It was clear that I should be using Spin Lock i.e., BOOST_SP_USE_SPINLOCK flag for building boost.
The problem seems to be fixed after I rebuilt boost with BOOST_SP_USE_SPINLOCK flag.
The following code illustrates a working example. It uses the shared_ptr<T> from std namespace starting in C++11, but you can substitute boost::shared_ptr<T>.
MyObject::~MyObject is invoked in the call to B::CleanMyObject as you are trying to do.
#include <memory>
#include <iostream>
using namespace std;
class MyObject
{
public:
MyObject()
{
cout << "Constructed MyObject" << endl;
}
~MyObject()
{
cout << "Destroyed MyObject" << endl;
}
};
class A
{
protected:
A()
: m_myObjectPtr(new MyObject())
{
}
std::shared_ptr<MyObject> m_myObjectPtr; // MyObject class is a simple class with a constructor and destructor
};
class B : A
{
public:
B()
{
}
void CleanMyObject()
{
m_myObjectPtr.reset();
}
};
int main() {
B b;
b.CleanMyObject();
}