Recently i started to work on one legacy project and trying to fix segfaults (double delete). Many of them is happening on boost::shared_ptr destructor or operator=(on objects that contain a shared_ptr). The code contains is massive usage of shared_ptr-s, including copying, reset()-ing, assigning etc. According to boost docs we have not valid usage - its not safe to destruct/copy/reset same shared_ptr in many threads.
Locking each time seems impossible, so im searching for drop-in replacement for boost::shared_ptr. So the question is: if i replace all boost::shared_ptr with std::shared_ptr or std::tr1::shared_ptr will the solve this issue? Seems tr1 is safer version but its not clear for me. Second question - is c++0x version any better than tr1 ? (note we have gcc 4.4.6 and cannot upgrade it)
Accoring to gcc docs, c++11 std::shared_ptr should fix that, but im not sure about gcc4.4 version...
UPD: Just maked experiment and now i know all 3 implementations do segfault on this code(gcc 4.4).. seems i should make custom class or maybe other workaround...
#include <iostream>
#include <boost/thread.hpp>
#include <boost/shared_ptr.hpp>
typedef boost::shared_ptr<int> ptrtype;
ptrtype p(new int);
void test() {
for(long i=0; i<1000000; ++i) {
ptrtype p1 = p;
p = ptrtype();
p.reset( new int );
}
}
int main() {
boost::thread_group tg;
for(int i=0; i<100; ++i) tg.add_thread( new boost::thread(test) );
tg.join_all();
std::cout << "Normal exit\n";
return 0;
}
Step 1: Build a class like this, and replace usage of boost::shared_ptr<T> with it.
template<typename T>
struct trivial_ptr {
T* t;
template<typename U>
void reset( U* p ) {t=p;}
void reset( T* p = NULL ) { t=p; }
template<typename U>
trivial_ptr<T>& operator=(trivial_shared_ptr<U>const& o) {
t = o.t;
return *t;
}
explicit trivial_ptr( T* p ):t(p) {}
...
};
this class is not intended to run, but just to compile with the correct interface. Once you have compiled, you can ensure you know what parts of the boost::shared_ptr interface you are using. (Are you using custom deleters? etc -- the problem could be harder, or easier, and the above can help test it)
Once you are there, you can work out how hard it will be to write up a shared_ptr<T> that handles multiple threads accessing the same variable at the same time.
Now, this is extremely messy. If one thread resets a given shared_ptr while another thread reads from it, the pointer read from may be completely invalid by the time the reading thread has access to it. In effect, you need to guard all access to the underlying pointer in a mutex, which is utterly impractical.
On the other hand, if what you have is multiple readers, and never a reader and a writer, you are in better shape -- in theory, you could fix the problem with the use of appropriate locks on the reference count code.
However, what you actually describe seems to involve multiple threads both reading and writing to the same variable. And that is fundamentally broken in a way that mere thread safety on the shared_ptr variable is not going to fix.
The problem you appear to have is attempting to modify the same instance of a variable in two separate threads (AKA a data race). shared_ptr is no more protected against this than int's are. Your reference to the gcc docs says the same thing ("same level of thread safety as built-in types"). Attempting to modify the same instance of a shared_ptr in two different threads requires some sort of synchronization to prevent the data race. Attempting to modify two different instances of a shared_ptr that are pointing to the same object is OK (no data race, or shared_ptr must implement whatever is necessary to prevent a data race internally). Attempting to modify the object that they point to is also a data race.
std::shared_ptr may use atomic ints if the compiler/architecture/implementation supports/use it. But i wouldn't bet on it, and doing so will make your code less portable. But it may work as a temporary workaround (e.g. to have a running program so you understand what the code is supposed to do).
Writing your own shared_ptr wrapper might be an option, but your code will still need to be audited for deadlocks.
Related
Problem: Unique_ptrs express ownership well, but cannot have their object lifetimes tracked by weak_ptrs. Shared_ptrs can be tracked by weak_ptrs but do not express ownership clearly.
Proposed solution: Derive a new pointer type (I'm going to call it strong_ptr) that is simply a shared_ptr but with the copy constructor and assignment operator deleted, so that it is hard to clone them. We then create another new borrowed_ptr type (which is not easily storable) to handle the temporary lifetime extension required when the weak_ptr accesses the object, and can thereby avoid using shared_ptrs explicitly anywhere.
This question Non-ownership copies of std::unique_ptr adn this one Better shared_ptr by distinct types for "ownership" and "reference"? are both similar but in both cases the choice is framed as simply unique_ptr vs shared_ptr and the answer does not propose a satisfactory solution to my mind. (Perhaps I should be answering those questions instead of asking a new one? Not sure what the correct etiquette is in this case.)
Here's a basic stab. Note that in order to avoid the user of the weak pointer having to convert to shared_ptr to use it, I create a borrowed_ptr type (thanks rust for the name) which wraps shared_ptr but makes it hard for the user to accidentally store it. So by using differently hamstrung shared_ptr derivatives we can express the intended ownership and guide the client code into correct usage.
#include <memory>
template <typename T>
// This owns the memory
class strong_ptr : public std::shared_ptr<T> {
public:
strong_ptr() = default;
strong_ptr(T* t) : std::shared_ptr<T>(t) {}
strong_ptr(const strong_ptr&) = delete;
strong_ptr& operator=(const strong_ptr&) = delete;
};
template <typename T>
// This can temporarily extend the lifetime but is intentionally hard to store
class borrowed_ptr : public std::shared_ptr<T> {
public:
borrowed_ptr() = delete;
borrowed_ptr(const borrowed_ptr&) = delete;
borrowed_ptr& operator=(const borrowed_ptr&) = delete;
template <typename T>
static borrowed_ptr borrow(const std::weak_ptr<T>& wp)
{
return wp.lock();
}
private:
borrowed_ptr(std::shared_ptr<T> &sp) : std::shared_ptr<T>(sp) {}
};
This seems fairly simple and an improvement over shared_ptr, but I cannot find any discussion of such a technique, so I can only imagine that I have missed an obvious flaw.
Can anyone give me a concrete reason why this is a bad idea? (And yes I know this is less efficient than unique_ptr - for PIMPL and so on I would still use unique_ptr.)
Caveat: I haven't yet used this in any more than a basic example, but this compiles and runs ok:
struct xxx
{
int yyy;
double zzz;
};
struct aaa
{
borrowed_ptr<xxx> naughty;
};
void testfun()
{
strong_ptr<xxx> stp = new xxx;
stp->yyy = 123;
stp->zzz = 0.456;
std::weak_ptr<xxx> wkp = stp;
// borrowed_ptr<xxx> shp = wkp.lock(); <-- Fails to compile as planned
// aaa badStruct { borrowed_ptr<xxx>::borrow(wkp) }; <-- Fails to compile as planned
// aaa anotherBadStruct; <-- Fails to compile as planned
borrowed_ptr<xxx> brp = borrowed_ptr<xxx>::borrow(wkp); // Only way to create the borrowed pointer
// std::cout << "wkp: " << wkp->yyy << std::endl; <-- Fails to compile as planned
std::cout << "stp: " << stp->yyy << std::endl; // ok
std::cout << "bp: " << brp->yyy << std::endl; // ok
}
Unique ownership is unique, full stop. One place owns this resource and will release it when that code so chooses.
Shared ownership is shared. Multiple places can own this resource, and the resource will only be released when all of them have done so. This is a binary state: either one place owns the resource or multiple places do.
Your ownership semantics are unique... except when they're not. And rules that work a certain way except when they don't are kind of problematic.
Now, your specific implementation is full of holes. shared/weak_ptr are all explicitly part of the interface of these types, so it is exceptionally easy to just get the shared_ptr out of a strong_ptr. If you have a weak_ptr to a strong_ptr (required for borrowed_ptr::borrow), then you could just lock it and get a shared_ptr.
But even if your interface were to properly hide all of this (that is, you make your own weak_ptr-equivalent type and you stop inheriting from shared_ptr), your API cannot stop someone from storing that borrowed_ptr anywhere they want. Oh sure, they can't change it later, but it's easy enough to store it in a class member at construction time or to heap allocate one or whatever.
So at the end of the day, locking a weak pointer still represents a claim of ownership. Therefore, the ownership semantics of your pointer stack is still shared; there's simply an API encouragement to not retain shared ownership for too long.
unique_ptr doesn't have "API encouragement"; it has API enforcement. That's what gives it unique ownership. C++ does not have a mechanism to create similar enforcement of the ownership semantics you want to create.
Encouragement may be useful on some level, but it'd probably be just as useful to just have borrowed_ptr as the encouragement for those who want to express that they are only claiming ownership temporarily. And just directly use shared/weak_ptr as normal otherwise. That is, your API should explicitly recognize that it is using shared ownership, so that nobody is fooled into thinking otherwise.
I have a threaded class from which I would like to occasionally acquire a pointer an instance variable. I would like this access to be guarded by a mutex so that the thread is blocked from accessing this resource until the client is finished with its pointer.
My initial approach to this is to return a pair of objects: one a pointer to the resource and one a shared_ptr to a lock object on the mutex. This shared_ptr holds the only reference to the lock object so the mutex should be unlocked when it goes out of scope. Something like this:
void A::getResource()
{
Lock* lock = new Lock(&mMutex);
return pair<Resource*, shared_ptr<Lock> >(
&mResource,
shared_ptr<Lock>(lock));
}
This solution is less than ideal because it requires the client to hold onto the entire pair of objects. Behaviour like this breaks the thread safety:
Resource* r = a.getResource().first;
In addition, my own implementation of this is deadlocking and I'm having difficulty determining why, so there may be other things wrong with it.
What I would like to have is a shared_ptr that contains the lock as an instance variable, binding it with the means to access the resource. This seems like something that should have an established design pattern but having done some research I'm surprised to find it quite hard to come across.
My questions are:
Is there a common implementation of this pattern?
Are there issues with putting a mutex inside a shared_ptr that I'm overlooking that prevent this pattern from being widespread?
Is there a good reason not to implement my own shared_ptr class to implement this pattern?
(NB I'm working on a codebase that uses Qt but unfortunately cannot use boost in this case. However, answers involving boost are still of general interest.)
I'm not sure if there are any standard implementations, but since I like re-implementing stuff for no reason, here's a version that should work (assuming you don't want to be able to copy such pointers):
template<class T>
class locking_ptr
{
public:
locking_ptr(T* ptr, mutex* lock)
: m_ptr(ptr)
, m_mutex(lock)
{
m_mutex->lock();
}
~locking_ptr()
{
if (m_mutex)
m_mutex->unlock();
}
locking_ptr(locking_ptr<T>&& ptr)
: m_ptr(ptr.m_ptr)
, m_mutex(ptr.m_mutex)
{
ptr.m_ptr = nullptr;
ptr.m_mutex = nullptr;
}
T* operator ->()
{
return m_ptr;
}
T const* operator ->() const
{
return m_ptr;
}
private:
// disallow copy/assignment
locking_ptr(locking_ptr<T> const& ptr)
{
}
locking_ptr& operator = (locking_ptr<T> const& ptr)
{
return *this;
}
T* m_ptr;
mutex* m_mutex; // whatever implementation you use
};
You're describing a variation of the EXECUTE AROUND POINTER pattern, described by Kevlin Henney in Executing Around Sequences.
I have a prototype implementation at exec_around.h but I can't guarantee it works correctly in all cases as it's a work in progress. It includes a function mutex_around which creates an object and wraps it in a smart pointer that locks and unlocks a mutex when accessed.
There is another approach here. Far less flexible and less generic, but also far simpler. While it still seems to fit your exact scenario.
shared_ptr (both standard and Boost) offers means to construct it while providing another shared_ptr instance which will be used for usage counter and some arbitrary pointer that will not be managed at all. On cppreference.com it is the 8th form (the aliasing constructor).
Now, normally, this form is used for conversions - like providing a shared_ptr to base class object from derived class object. They share ownership and usage counter but (in general) have two different pointer values of different types. This form is also used to provide a shared_ptr to a member value based on shared_ptr to object that it is a member of.
Here we can "abuse" the form to provide lock guard. Do it like this:
auto A::getResource()
{
auto counter = std::make_shared<Lock>(&mMutex);
std::shared_ptr<Resource> result{ counter, &mResource };
return result;
}
The returned shared_ptr points to mResource and keeps mMutex locked for as long as it is used by anyone.
The problem with this solution is that it is now your responsibility to ensure that the mResource remains valid (in particular - it doesn't get destroyed) for that long as well. If locking mMutex is enough for that, then you are fine.
Otherwise, above solution must be adjusted to your particular needs. For example, you might want to have the counter a simple struct that keeps both the Lock and another shared_ptr to the A object owning the mResource.
To add to Adam Badura's answer, for a more general case using std::mutex and std::lock_guard, this worked for me:
auto A::getResource()
{
auto counter = std::make_shared<std::lock_guard<std::mutex>>(mMutex);
std::shared_ptr<Resource> ptr{ counter, &mResource} ;
return ptr;
}
where the lifetimes of std::mutex mMutex and Resource mResource are managed by some class A.
I'm using C++ in an embedded linux environment which has GCC version 2.95.
I just can't extract boost::shared_ptr files with bcp, it is just too heavy.
What I'd like would be a simple smart pointer implementation of boost::shared_ptr but without all boost overheads (if it is possible...).
I could come up with my own version reading boost source but I fear missing one or more points, it seems easy to make a faulty smart pointer and I can't afford to have a buggy implementation.
So, does a "simple" implementation or implementation example of boost::shared_ptr (or any reference counting equivalent smart pointer) exists that I could use or that I could take as an inspiration?
if you don't need mixing shared and weak ptr,and don't need coustom deletors, you can just use the quick and dirty my_shared_ptr:
template<class T>
class my_shared_ptr
{
template<class U>
friend class my_shared_ptr;
public:
my_shared_ptr() :p(), c() {}
explicit my_shared_ptr(T* s) :p(s), c(new unsigned(1)) {}
my_shared_ptr(const my_shared_ptr& s) :p(s.p), c(s.c) { if(c) ++*c; }
my_shared_ptr& operator=(const my_shared_ptr& s)
{ if(this!=&s) { clear(); p=s.p; c=s.c; if(c) ++*c; } return *this; }
template<class U>
my_shared_ptr(const my_shared_ptr<U>& s) :p(s.p), c(s.c) { if(c) ++*c; }
~my_shared_ptr() { clear(); }
void clear()
{
if(c)
{
if(*c==1) delete p;
if(!--*c) delete c;
}
c=0; p=0;
}
T* get() const { return (c)? p: 0; }
T* operator->() const { return get(); }
T& operator*() const { return *get(); }
private:
T* p;
unsigned* c;
}
For anyone interested in make_my_shared<X>, it can be trivially implemented as
template<class T, class... U>
auto make_my_shared(U&&... u)
{
return my_shared_ptr<T>(new T{std::forward<U>(u)...});
}
to be called as
auto pt = make_my_shared<T>( ... );
There is also std::tr1::shared_ptr which is just C++11 standard's lifting from boost. You could pick that if it is allowed, or write your own with reference counting.
I'd suggest you can use shared_ptr in isolation. However, if you are looking for simple implementation
with some unit tests
not thread safe
basic support for polymorphic assignment <-- let me know if you're intrested
custom deletors (i.e. array deletors, or custom functors for special resources)
Have a look here: Creating a non-thread safe shared_ptr
Which "boost overheads" are you concerned about, and in what way is shared_ptr "too heavy" for your application? There are no overheads simply from using Boost (at least if you only use header-only libraries, such as the smart pointer library); the only overheads I can think of concerning shared_ptr are:
Thread-safe reference counting; since you're using an ancient compiler, I assume you also have an ancient runtime library and kernel, so this may be very inefficient. If your application is single threaded, then you can disable this by #defining BOOST_DISABLE_THREADS.
Allocation of the reference-counting structure alongside the managed object. You can eliminate the extra allocation by creating the object using make_shared() or allocate_shared().
In many cases, you can eliminate the overhead from speed-critical code by not creating objects or copying shared pointers - pass pointers by reference and only copy them when you actually need to.
If you need thread safety for some pointers, but not others, and (after profiling, and removing all unnecessary allocations and pointer copying) you find that the use of shared pointers is still causing significant overhead, then you could consider using intrusive_ptr, and manage your own reference counting within the object.
There may also be benefit to updating to a modern GNU/Linux version, if that's feasible. Thread synchronisation is much more efficient since the introduction of futexes in Linux 2.6. You may find this helps in other ways too; there have been many improvements over the last decade. A more modern compiler would also provide the standard (TR1 or C++11) shared pointer, so you wouldn't need Boost for that.
You can use shared_ptr without all the Boost overhead: it's a header-only implementation. If you don't use anything other classes, only shared_ptr will be compiled in.
The implementation of shared_ptr is already pretty lean, but if you want to avoid the intermediate reference count block and a (potential) virtual call to deleter function, you can use boost::intrusive_ptr instead, which is more suited for embedded environments: it operates on a reference counter embedded in the object itself, you just have to provide a couple of functions to increment/decrement it. The downside is that you won't be able to use weak_ptr.
I can't comment how well gcc 2.95 would inline/fold template instantiations (it's a very old compiler), more recent versions of gcc handle it rather well, so you are on your own here.
I've recently started using Boost.Asio in a project and would like to know whether anyone knows a clean solution to transfer ownership of a newly created socket to tcp::acceptor::async_accept, which would in turn transfer this ownership to the accept handler function.
This isn't an incoherent desire, mind you, since the handler is to be called exactly once.
I have noticed I can't std::bind() an std::unique_ptr<> as parameter, since std::bind() requires its parameters to be CopyConstructible, and rightfully so. Not only that, but Boost's AcceptHandler concept is also required to be CopyConstructible.
So my options would be:
Go the deprecated std::auto_ptr<> way of moving objects with the copy constructor, potentially causing obscure bugs on new releases of Boost.Asio.
Use std::shared_ptr<> and have no way to take the shared ownership off the pointer once it's not needed anymore, i.e. when it reaches the actual handler function (this is how the job is done on the examples at http://www.boost.org/doc/libs/1_43_0/doc/html/boost_asio/examples.html as far as I've read).
or
You have a better idea for me.
I'm pretty much at a loss here. Can anyone enlighten me?
I tried to find a way to do this using the c++0x standard library, but could not.
Eventually I settled on writing my own rvalue_reference_wrapper and rvalue_ref() convenience class. As per usual with std::bind, you need to wrap your non copyable object in something that is copyable (reference_wrapper is the best example). You could also have just passed a pointer, but that means changing your interface.
This worked on my machine:
#include <iostream>
#include <functional>
#include <memory>
template< class T >
struct rvalue_reference_wrapper
{
rvalue_reference_wrapper( T&& t )
: t_(std::move(t))
{}
operator T&&() const volatile
{
return std::move(t_);
}
private:
T&& t_;
};
template< class T >
rvalue_reference_wrapper<T> rvalue_ref( T&& t )
{
return rvalue_reference_wrapper<T>(std::move(t));
}
void go( std::unique_ptr<int> i )
{
std::cout << *i << std::endl;
}
int main()
{
std::unique_ptr<int> i(new int(1));
auto b = std::bind( go, rvalue_ref(std::move(i)) );
//auto b = std::bind( go, std::ref(std::move(i)) ); // Wont work
b();
}
I have not made the code bulletproof, but welcome some discussion about the need for an rvalue_reference_wrapper, or how to simulate one using std::reference_wrapper.
Also, for your specific case, you will likely need to write a difference version of rvalue_reference_wrapper, which holds the object by value not by rvalue reference, since your original unique_ptr is likely going to leave scope (and be destroyed) since you are using async asio calls.
Is there a stock no-op deallocator in Boost to use with boost::shared_ptr for static objects, etc.
I know it's ultra-trivial to write, but I don't want to sprinkle my code with extra tiny functions if there is already one available.
Yes there is one here:
#include <boost/serialization/shared_ptr.hpp> // for null_deleter
class Foo
{
int x;
};
Foo foo;
boost::shared_ptr< Foo > sharedfoo( &foo, boost::serialization::null_deleter() );
There is, of course, a danger with the fact that you need to know the function you call doesn't store the shared_ptr for later use, as it actually goes against the policy of shared_ptr in that the underlying object remains valid until that of the last instance of the shared_ptr.
Solution uses Boost.Lambda:
#include <boost/shared_ptr.hpp>
#include <boost/lambda/lambda.hpp>
int main()
{
int *p = new int(5);
{
boost::shared_ptr<int> sp(p, boost::lambda::_1);
}
delete p;
}
'boost::lambda::_1' creates an empty functor that takes one argument.
You'll probably want to put a //comment in there to let people know why you did it, though.
If Boost <= 1.54, use boost/log/utility/empty_deleter.hpp.
If Boost == 1.55, use boost/utility/empty_deleter.hpp.
If Boost >= 1.56, use boost/core/null_deleter.hpp.
Wouldn't it be cleaner just to take an extra reference so the deallocator is never called? (Although that's still not very clean.)
I can't say that there's no function in Boost which would do the job, but it doesn't sound like something they would want to include.
EDIT: Having read the comments and a little documentation, it boils down to this:
Reference leaking. At some point, execute this:
new shared_ptr( my_global_shared_ptr );
Advantages: conceptually easy. Disadvantages: you are leaking something on the heap.
Custom deallocator. Since shared_ptr requires little of the deallocator function, an anonymous identity function like that provided in the other answer would do fine.
Advantages: leverages Boost and has absolutely no overhead. Disdvantages: requires a little documentation.
Non-static global object. If there is a global shared_ptr for your object, that should be the only means of access to it. Replace the declaration of the global with a shared_ptr intialized by new my_class. I think this is best.
There's a ticket for this on the Boost bug tracker: https://svn.boost.org/trac/boost/ticket/1913 - no activity for a long time until some murmur two weeks ago.
FWIW, this what I'm using.
I use it in unit tests to adapt a local into a shared_ptr.
// The class NoOp_sptr_Deleter can be used to construct a shared_ptr<>()
// that will NOT delete the pointee.
// This can be helpful in unit-testing. Wrapping a local as a shared_ptr.
// Do take care with the lifetimes though.
struct NoOp_sptr_Deleter
{
void operator()(void const *) const {}
};
template<typename T>
boost::shared_ptr<T> FakeSharedPtrFromRef(T& aRef)
{
return boost::shared_ptr<T>(&aRef, NoOp_sptr_Deleter() );
}