What is the correct way of passing shared pointers to stl containers in different objects, so there is no early destruction of the object?
I have multiple Systems with std::queue in them:
class System {
typedef std::shared_ptr<Event> EventPtr;
protected:
std::queue<EventPtr> mEventQueue;
static SystemManager * sSystemManager;
//this holds all the systems in the application
public:
System();
~System();
void addEventToQueue(EventPtr event) {
mEventQueue.push(event);
}
void callEventQueue() {
while(!mEventQueue.empty()) {
acceptEvent(mEventQueue.front().get());
mEventQueue.pop();
}
}
void acceptEvent(Event * event);
public:
static void sendEvent(EventPtr &event) {
for(auto system : sSystemManager->getSystems()) {
system->addEventToQueue(event);
}
}
};
I want to know if I understand it properly:
When I call System::sendEvent(std::make_shared<Event>("testEvent")); in a scope, it passes the shared pointer as a reference which doesn't create a new one and doesn't increase the reference count. However, the addEventToQueue function passes the argument as an object, so the reference count increases; If I have 5 systems, the reference count will be 6 (counting the std::make_shared itself). But where is this reference count stored? Is it the first shared pointer that's created through std::make_shared? Or is the same count in all the objects? So, when the first objects goes out scope what happens to the other objects? How do they know what the correct reference count is since they only know about the "parent" object?
All the articles I read about shared pointers, the way the reference count is shown is always common. Is the counter a static variable?
Where exactly the count is stored depends on the implementation. However, the standard prescribes that it must behave so that all instances of std::shared_ptr<T> which share ownership of one instance of T use the same reference count. In practice, this means that the reference count is allocated dynamically and a pointer to it is shared by all the relevant instances of std::shared_ptr<T>.
That's one of the reasons why std::make_shared() is the preferred way of creating shared pointers - it can allocate memory for the reference count (and other maintenance structures required) and for the object in one allocation request, instead of two separate ones. This improves performance of the allocation and perhaps also of use of the pointer (as the ref. count and object will be closer in memory and thus make cache misses less likely).
Related
I have a class that holds a shared pointer to a map.
Code snippet:
class SomeClass {
private:
shared_ptr<const map<int, string>> sptr_;
ReaderWriterLock rw_lock_;
public:
SomeClass() : sptr_(make_shared<const map<int, string>>()) {};
void UpdateSharedPointer(shared_ptr<const map<int, string>>&& new_sptr) {
if (!new_sptr) { return; }
rw_lock_.Lock(true /* is_writer */);
sptr_ = move(new_sptr);
rw_lock_.Unlock();
}
void ReadSharedPointer() const {
rw_lock_.Lock(false /* is_writer */);
// Create a copy of the pointer.
shared_ptr<const map<int, string>> local_sptr = sptr_;
rw_lock_.Unlock();
if (!local_sptr) {
return;
}
// Read the map object pointed by the local_sptr and perform some operations.
}
}
Usage pattern: An instance of SomeClass can be accessed by multiple threads. And I am okay working on a stale copy of map inside ReadSharedPointer() method.
The use of shared pointer to a constant map object leads to a less contentious code. If someone needs to update or read the map, the lock is held for only a short while. Instead, if I eliminate the shared pointer, any map update or read will need to hold the rw_lock_ for a larger time.
What I am worried about is that this extra pointer indirection might make the code fragile. For example, my current use case requires only a constant map, but if in future someone updates the map to be a non-const object and forgets to update the locking mechanism associated with the read operation, a race condition might get introduced.
Also, one might argue that eliminating the shared pointer makes the complete workflow simpler to understand. And a shared pointer might not make sense if we are not truly sharing this map but only introducing a shared pointer to reduce contention.
So basically my questions are:
How good of a design is this? What are the pros and cons? Is there any alternative to this?
I've been thinking about using shared pointers, and I know how to implement one myself--Don't want to do it, so I'm trying std::tr1::shared_ptr,and I have couple of questions...
How is the reference counting implemented? Does it use a doubly linked list? (Btw, I've already googled, but I can't find anything reliable.)
Are there any pitfalls for using the std::tr1::shared_ptr?
shared_ptr must manage a reference counter and the carrying of a deleter functor that is deduced by the type of the object given at initialization.
The shared_ptr class typically hosts two members: a T* (that is returned by operator-> and dereferenced in operator*) and a aux* where aux is a inner abstract class that contains:
a counter (incremented / decremented upon copy-assign / destroy)
whatever is needed to make increment / decrement atomic (not needed if specific platform atomic INC/DEC is available)
an abstract virtual destroy()=0;
a virtual destructor.
Such aux class (the actual name depends on the implementation) is derived by a family of templatized classes (parametrized on the type given by the explicit constructor, say U derived from T), that add:
a pointer to the object (same as T*, but with the actual type: this is needed to properly manage all the cases of T being a base for whatever U having multiple T in the derivation hierarchy)
a copy of the deletor object given as deletion policy to the explicit constructor (or the default deletor just doing delete p, where p is the U* above)
the override of the destroy method, calling the deleter functor.
A simplified sketch can be this one:
template<class T>
class shared_ptr
{
struct aux
{
unsigned count;
aux() :count(1) {}
virtual void destroy()=0;
virtual ~aux() {} //must be polymorphic
};
template<class U, class Deleter>
struct auximpl: public aux
{
U* p;
Deleter d;
auximpl(U* pu, Deleter x) :p(pu), d(x) {}
virtual void destroy() { d(p); }
};
template<class U>
struct default_deleter
{
void operator()(U* p) const { delete p; }
};
aux* pa;
T* pt;
void inc() { if(pa) interlocked_inc(pa->count); }
void dec()
{
if(pa && !interlocked_dec(pa->count))
{ pa->destroy(); delete pa; }
}
public:
shared_ptr() :pa(), pt() {}
template<class U, class Deleter>
shared_ptr(U* pu, Deleter d) :pa(new auximpl<U,Deleter>(pu,d)), pt(pu) {}
template<class U>
explicit shared_ptr(U* pu) :pa(new auximpl<U,default_deleter<U> >(pu,default_deleter<U>())), pt(pu) {}
shared_ptr(const shared_ptr& s) :pa(s.pa), pt(s.pt) { inc(); }
template<class U>
shared_ptr(const shared_ptr<U>& s) :pa(s.pa), pt(s.pt) { inc(); }
~shared_ptr() { dec(); }
shared_ptr& operator=(const shared_ptr& s)
{
if(this!=&s)
{
dec();
pa = s.pa; pt=s.pt;
inc();
}
return *this;
}
T* operator->() const { return pt; }
T& operator*() const { return *pt; }
};
Where weak_ptr interoperability is required a second counter (weak_count) is required in aux (will be incremented / decremented by weak_ptr), and delete pa must happen only when both the counters reach zero.
How is the reference counting implemented?
A smart pointer implementation could be deconstructed, using policy-based class design1, into :
Storage Policy
Ownership Policy
Conversion Policy
Checking Policy
included as template parameters.
Popular ownership strategies include: deep copy, reference counting, reference linking, and destructive copy.
Reference counting tracks the number of smart pointers pointing to (owning2) the same object. When the number goes to zero, the pointee object is deleted3. The actual counter could be:
Shared among smart pointer objects, where each smart pointer holds a pointer to the reference counter:
Included only in an additional structure that adds an extra level of indirection the pointee object. Here the space overhead of holding a counter in each smart pointer is exchanged with slower access speed:
Contained within the pointee object itself: intrusive reference counting. The disadvantage is that the object must be constructed a priori with facilities for counting:
Finally the method in your question, reference counting using doubly linked lists is called reference linking and it:
...[1] relies on the observation that you don't really need the actual count of smart pointer objects pointing to one pointee object; you only need to detect when that count goes down to zero. This leads to the idea of keeping an "ownership list" :
The advantage of reference linking over reference counting is that the former does not use extra free store, which makes it more reliable: Creating a reference-linked smart pointer cannot fail. The disadvantage is
that reference linking needs more memory for its bookkeeping (three pointers versus only one pointer plus one integer). Also, reference counting should be a bit speedier—when you copy smart pointers, only an indirection and an increment are needed. The list management is slightly more elaborate. In conclusion,
you should use reference linking only when the free store is scarce. Otherwise, prefer reference counting.
Regarding your second question:
Does it (std::shared_ptr) use a doubly linked list?
All that I could find in the C++ standard was:
20.7.2.2.6 shared_ptr creation
...
7. [ Note: These functions will typically allocate more memory than sizeof(T) to allow for internal bookkeeping structures such as the reference counts. —end note ]
Which, in my opinion, excludes doubly linked lists, as they do not contain actual count.
Your third question:
Are there any pitfalls for using the std::shared_ptr?
Reference management either counting or linking is a victim of the resource leak known as cyclic reference. Let's have an object A that holds a smart pointer to an object B. Also, object B holds a smart pointer to A. These two objects form a cyclic reference; even though you don't use any of them any more, they use each other. The reference management strategy cannot detect such cyclic references, and the two objects remain allocated forever.
Because the implementation of shared_ptr uses reference counting, cyclic references are potentially a problem. A cyclic shared_ptr chain can be broken by changing the code so that one of the references is a weak_ptr. This is done by assigning values between shared pointers and weak pointers, but a weak pointer doesn't affect the reference count. If the only pointers that point to an object are weak, the object is destroyed.
1. Each design feature with multiple implementations if formulated as policy.
2. Smart pointers similarly to pointers that point to object allocated with new, not only point to that object but also are responsible for its destruction and with the release of the memory it occupies.
3. With no further problems, if no other raw pointers are used and/or point to it.
[1] Modern C++ Design: Generic Programming and Design Patterns Applied. Andrei Alexandrescu, February 01, 2001
If you want to see all the gory details, you can have a look at the boost shared_ptr implementation:
https://github.com/boostorg/smart_ptr
The reference counting seems to usually be implemented with a counter and platform specific atomic increment/decrement instructions or explicit locking with a mutex (see the atomic_count_*.hpp files in the detail namespace).
Are there any pitfalls for using the std::tr1::shared_ptr?
Yes, If you create cycles in your shared memory pointers, then the memory being managed by the smart pointer will not be recycled when the last pointer goes out of scope because there are still references to the pointer (i.e., the cycles cause the reference count to not go down to zero).
For instance:
struct A
{
std::shared_ptr<A> ptr;
};
std::shared_ptr<A> shrd_ptr_1 = std::make_shared(A());
std::shared_ptr<B> shrd_ptr_2 = std::make_shared(A());
shrd_ptr_1->ptr = shrd_ptr_2;
shrd_ptr_2->ptr = shrd_ptr_1;
Now, even if shrd_ptr_1 and shrd_ptr_2 go out of scope, the memory they are managing is not reclaimed because the ptr member of each are pointing to each other. While this is a very naive example of such a memory cycle, it can, if you use these types of pointers without any discipline, occur in a much more nefarious and hard-to-track fashion. For instance, I could see where trying to implement a circular linked-list where each next pointer is a std::shared_ptr, if you're not too careful, could result in problems.
If I have several levels of object containment (one object defines and instantiates another object which define and instantiate another object..), is it possible to get access to upper, containing - object variables and functions, please?
Example:
class CObjectOne
{
public:
CObjectOne::CObjectOne() { Create(); };
void Create();
std::vector<ObjectTwo>vObejctsTwo;
int nVariableOne;
}
bool CObjectOne::Create()
{
CObjectTwo ObjectTwo(this);
vObjectsTwo.push_back(ObjectTwo);
}
class CObjectTwo
{
public:
CObjectTwo::CObjectTwo(CObjectOne* pObject)
{
pObjectOne = pObject;
Create();
};
void Create();
CObjectOne* GetObjectOne(){return pObjectOne;};
std::vector<CObjectTrhee>vObjectsTrhee;
CObjectOne* pObjectOne;
int nVariableTwo;
}
bool CObjectTwo::Create()
{
CObjectThree ObjectThree(this);
vObjectsThree.push_back(ObjectThree);
}
class CObjectThree
{
public:
CObjectThree::CObjectThree(CObjectTwo* pObject)
{
pObjectTwo = pObject;
Create();
};
void Create();
CObjectTwo* GetObjectTwo(){return pObjectTwo;};
std::vector<CObjectsFour>vObjectsFour;
CObjectTwo* pObjectTwo;
int nVariableThree;
}
bool CObjectThree::Create()
{
CObjectFour ObjectFour(this);
vObjectsFour.push_back(ObjectFour);
}
main()
{
CObjectOne myObject1;
}
Say, that from within CObjectThree I need to access nVariableOne in CObjectOne. I would like to do it as follows:
int nValue = vObjectThree[index].GetObjectTwo()->GetObjectOne()->nVariable1;
However, after compiling and running my application, I get Memory Access Violation error.
What is wrong with the code above(it is example, and might contain spelling mistakes)?
Do I have to create the objects dynamically instead of statically?
Is there any other way how to achieve variables stored in containing objects from withing contained objects?
When you pass a pointer that points back to the container object, this pointer is sometimes called a back pointer. I see this technique being used all the time in GUI libraries where a widget might want access to its parent widget.
That being said, you should ask yourself if there's a better design that doesn't involve circular dependencies (circular in the sense that the container depends on the containee and the containee depends on the container).
You don't strictly have to create the objects dynamically for the back pointer technique to work. You can always take the address of a stack-allocated (or statically-allocated) object. As long as the life of that object persists while others are using pointers to it. But in practice, this technique is usually used with dynamically-created objects.
Note that you might also be able to use a back-reference instead of a back-pointer.
I think I know what's causing your segmentation faults. When your vectors reallocate their memory (as the result of growing to a larger size), the addresses of the old vector elements become invalid. But the children (and grand-children) of these objects still hold the old addresses in their back-pointers!
For the back-pointer thing to work, you'll have to allocate each object dynamically and store their pointers in the vectors. This will make memory management a lot more messy, so you might want to use smart pointers or boost::ptr_containers.
After seeing the comment you made in another answer, I now have a better idea of what you're trying to accomplish. You should research generic tree structures and the composite pattern. The composite pattern is usually what's used in the widget example I cited previously.
Maybe all your object can inherit from a common interface like :
class MyObject
{
public:
virtual int getData() = 0;
}
And after you can use a std::tree from the stl library to build your structure.
As Emile said, segmentation fault is caused by reallocation. Exactly speaking -- when the local stack objects' 'this' pointer was passed to create another object, which is then copied to the vector container. Then the 'Create()' function exits, the stack frame object ceases to exist and the pointer in the container gets invalid.
For legacy reasons I need to use intrusive pointers, as I need the ability to convert raw pointers to smart pointers.
However I noticed there is no weak intrusive pointer for boost. I did find a talk about it on the boost thread list, however nothing concrete.
Does anyone know of a thread safe implementation of weak intrusive pointer?
Thanks
Rich
It does not make any sense.
To elaborate: weak_ptr points to the same instance of a counter object that shared_ptr do. When the shared_ptr goes out of scope, the instance of the counter stays (with a count effectively at 0), which allows the weak_ptr instances to check that they effectively point to a freed object.
With Intrusive Counting, the counter is integrated within the object. When the count reaches 0, the object is usually either recycled or deleted... but the point is the counter is no longer available. The rationale is that this allow for a more efficient storage (1 single chunk) and greater speed (cache locality).
If you need Weak Reference counting and do not care for the benefits of intrusive counting, you can use a combination of shared_ptr and weak_ptr.
The idea is to deassociate the counter from the objects.
class Counted
{
// bla
private:
boost::shared_ptr<int> mCounter;
};
Now you can return weak handles:
class WeakHandle
{
public:
explicit WeakHandle(Counted& c): mCounter(c.mCounter), mObject(&c) {}
bool expired() const { return mCounter.expired(); }
private:
boost::weak_ptr<int> mCounter;
Counted* mObject;
};
Here, we deassociate the lifetime of the counter from the lifetime of the object, so that it will survive the destruction of the object... partially. Thus making the weak_ptr effectively possible.
And of course, using shared_ptr and weak_ptr this is Thread Safe ;)
I didn't really like either of the previous answers so:
No, I don't know of an implementation, but I think it is possible. The standard implementation of the shared_ptr holds two reference counts, one for the "strong" and one for the "weak" references, and a pointer to the referent. In an intrusive_ptr implementation the strong count needs to be part of the object, but the weak can't be. So, it seems like you could create a "weakable" intrusive_ptr.
Define a weak pointer helper:
template<class X>
class intrusive_ptr_weak_helper {
long weak_ref_count;
X *target_instance;
};
Then record that into the object beside the reference count:
struct X {
...
intrusive_ptr_weak_helper *ref_weak_helper;
...
long ref_count;
...
};
When constructing X:
ref_count = 0;
ref_weak_helper = NULL;
The "strong" pointer, intrusive_strong_ptr, is identical to intrusive_ptr, until deletion occurs. When the strong ref count goes to zero (before deletion occurs):
if (ref_weak_helper != NULL) {
if (ref_weak_helper->weak_ref_count == 0)
delete ref_weak_helper;
else
ref_weak_helper->target_instance = NULL;
}
The "weak" version, intrusive_weak_ptr, records the pointer to the weak helper, manipulating that reference count, and accessing the target object via the target_instance pointer. When the weak_ref_count decrements to zero the status of target_instance determines whether the helper is deleted or not.
There are many details missing (concurrency concerns for instance) but this is a mixing of the shared_ptr and the intrusive_ptr. It maintains the basic benefits of the intrusive_ptr (cache optimization, reuse of 3rd party intrusive (strong) ref count, strong and weak pointer stand-ins are pointer sized) while adding extra work mainly in the weak reference path.
Current implementation of intrusive pointer is using reference counter. So deleting object delete also delete the counter, so weak_intrusive_pointer will never know that the object was deleted.
If you need to get weak_ptr from this, you probably search boost::enable_shared_from_this<T>.
OpenSceneGraph and its successor, VulkanSceneGraph, each have comprehensive implementations of intrusive strong pointers and associated weak pointers, named ref_ptr<> and observer_ptr<>, respectively.
I don't know every detail of these systems, but it seems they work using an additional object which is informed when the referent (a descendant of the class Referenced) is deleted. Weak pointers use this third object when an attempt is made to convert them to strong pointers.
VulkanSceneGraph is the next generation scene graph that is currently in development and intended to replace OpenSceneGraph, so I assume its intrusive pointer system is a more advanced implementation.
Worth checking out:
https://github.com/vsg-dev/VulkanSceneGraph/blob/master/include/vsg/core/observer_ptr.h
I have objects which create other child objects within their constructors, passing 'this' so the child can save a pointer back to its parent. I use boost::shared_ptr extensively in my programming as a safer alternative to std::auto_ptr or raw pointers. So the child would have code such as shared_ptr<Parent>, and boost provides the shared_from_this() method which the parent can give to the child.
My problem is that shared_from_this() cannot be used in a constructor, which isn't really a crime because 'this' should not be used in a constructor anyways unless you know what you're doing and don't mind the limitations.
Google's C++ Style Guide states that constructors should merely set member variables to their initial values. Any complex initialization should go in an explicit Init() method. This solves the 'this-in-constructor' problem as well as a few others as well.
What bothers me is that people using your code now must remember to call Init() every time they construct one of your objects. The only way I can think of to enforce this is by having an assertion that Init() has already been called at the top of every member function, but this is tedious to write and cumbersome to execute.
Are there any idioms out there that solve this problem at any step along the way?
Use a factory method to 2-phase construct & initialize your class, and then make the ctor & Init() function private. Then there's no way to create your object incorrectly. Just remember to keep the destructor public and to use a smart pointer:
#include <memory>
class BigObject
{
public:
static std::tr1::shared_ptr<BigObject> Create(int someParam)
{
std::tr1::shared_ptr<BigObject> ret(new BigObject(someParam));
ret->Init();
return ret;
}
private:
bool Init()
{
// do something to init
return true;
}
BigObject(int para)
{
}
BigObject() {}
};
int main()
{
std::tr1::shared_ptr<BigObject> obj = BigObject::Create(42);
return 0;
}
EDIT:
If you want to object to live on the stack, you can use a variant of the above pattern. As written this will create a temporary and use the copy ctor:
#include <memory>
class StackObject
{
public:
StackObject(const StackObject& rhs)
: n_(rhs.n_)
{
}
static StackObject Create(int val)
{
StackObject ret(val);
ret.Init();
return ret;
}
private:
int n_;
StackObject(int n = 0) : n_(n) {};
bool Init() { return true; }
};
int main()
{
StackObject sObj = StackObject::Create(42);
return 0;
}
Google's C++ programming guidelines have been criticized here and elsewhere again and again. And rightly so.
I use two-phase initialization only ever if it's hidden behind a wrapping class. If manually calling initialization functions would work, we'd still be programming in C and C++ with its constructors would never have been invented.
Depending on the situation, this may be a case where shared pointers don't add anything. They should be used anytime lifetime management is an issue. If the child objects lifetime is guaranteed to be shorter than that of the parent, I don't see a problem with using raw pointers. For instance, if the parent creates and deletes the child objects (and no one else does), there is no question over who should delete the child objects.
KeithB has a really good point that I would like to extend (in a sense that is not related to the question, but that will not fit in a comment):
In the specific case of the relation of an object with its subobjects the lifetimes are guaranteed: the parent object will always outlive the child object. In this case the child (member) object does not share the ownership of the parent (containing) object, and a shared_ptr should not be used. It should not be used for semantic reasons (no shared ownership at all) nor for practical reasons: you can introduce all sorts of problems: memory leaks and incorrect deletions.
To ease discussion I will use P to refer to the parent object and C to refer to the child or contained object.
If P lifetime is externally handled with a shared_ptr, then adding another shared_ptr in C to refer to P will have the effect of creating a cycle. Once you have a cycle in memory managed by reference counting you most probably have a memory leak: when the last external shared_ptr that refers to P goes out of scope, the pointer in C is still alive, so the reference count for P does not reach 0 and the object is not released, even if it is no longer accessible.
If P is handled by a different pointer then when the pointer gets deleted it will call the P destructor, that will cascade into calling the C destructor. The reference count for P in the shared_ptr that C has will reach 0 and it will trigger a double deletion.
If P has automatic storage duration, when it's destructor gets called (the object goes out of scope or the containing object destructor is called) then the shared_ptr will trigger the deletion of a block of memory that was not new-ed.
The common solution is breaking cycles with weak_ptrs, so that the child object would not keep a shared_ptr to the parent, but rather a weak_ptr. At this stage the problem is the same: to create a weak_ptr the object must already be managed by a shared_ptr, which during construction cannot happen.
Consider using either a raw pointer (handling ownership of a resource through a pointer is unsafe, but here ownership is handled externally so that is not an issue) or even a reference (which also is telling other programmers that you trust the referred object P to outlive the referring object C)
A object that requires complex construction sounds like a job for a factory.
Define an interface or an abstract class, one that cannot be constructed, plus a free-function that, possibly with parameters, returns a pointer to the interface, but behinds the scenes takes care of the complexity.
You have to think of design in terms of what the end user of your class has to do.
Do you really need to use the shared_ptr in this case? Can the child just have a pointer? After all, it's the child object, so it's owned by the parent, so couldn't it just have a normal pointer to it's parent?