How do pointers work with the concepts of Object oriented programming?
As I understand it (and please recognize, I'm classified as an ID-10T), the main tenet of OOP is containment and keeping management responsibility (memory/implementation/etc.) contained within the class; but when an object's method returns a pointers it seems like we are 'popping' the object. Now, somebody might need to worry about:
Are they supposed to delete the pointer's associated object?
But what if the class still needs the object?
Can they change the object? If so, how? (I recognize const might solve this issue)
and so forth...
It seems the user of the object now needs to know much more about how the class works and what the class expects of the user. It feels like a "cat's out of the bag" scenario which seems to slap in the face of OOP.
NOTE: I notice this is a language independent question; however, I was prompted to ask the question while working in a C++ environment.
What you describe are ownership issues. These are orthogonal (i.e. independent, you can have either without the other or even both) to object orientation. You have the same issues if you do not use OOP and juggle pointers to POD structs. You don't have the issue if you use OOP but solve it somehow. You can (try to) solve it using more OOP or in another way.
They are also orthogonal to the use of pointers (unless you nit pick and extend the definition of pointer). For example, the same issues arise if two separate places hold indices into an array and mutate, resize and ultimately delete the array.
In C++, the usual solution is to select the right smart pointer type (e.g. return a shared pointer when you wish to share the object, or a unique pointer to signify exclusive ownership), along with extensive documentation. Actually, the latter is a key ingredient in any language.
One OOP-related thing you can do to help this is encapsulation (of course, you can have encaptulation just fine without OOP). For instance, don't expose the object at all, only expose methods which query the object under the hood. Or don't expose raw pointers, only expose smart pointers.
For starters... You can't have polymorphism without pointers or
references. In C++, traditionally, objects are copied, and have (for
the most part) automatic storage duration. But copy doesn't work with
polymorphic objects—they tend to get sliced. And OO also often
means identity, which in turn means you don't want copy. So the
solution is for the object to be dynamically allocated, and to pass
around pointers. What you do with them is part of the design:
If the object is logically part of another object, then that object is
responsible for its lifetime, and objects which receive the pointer
should take steps to ensure that they don't use it after the owning
object disappears. (Note that this is true even in languages with
garbage collection. The object won't disappear as long as you've got a
pointer to it, but once the owning object is invalid, the owned object
may become invalid as well. The fact that the garbage collector won't
recycle the memory won't guarantee that the object you point to is
usable.)
If the object is a first class entity itself, rather than being
logically part of another object, then it should probably take care of
itself. Again, other objects which may hold a pointer to it must be
informed if it ceases to exist (or becomes invalid). The use of the
Observer pattern is the usual solution. Back when I started C++, there
was a fashion for "relationship management", with some sort of
management classes where you registered relationships, and which
supposedly ensured that everything worked out OK. In practice, they
either didn't work, or didn't do any more than the simple observer
pattern, and you don't hear any more of them today.
For the most part, your precise questions are part of the contract that
each class has to establish for each of its functions. For true OO
classes (entity objects), you should probably never delete them: that's
there business, not yours. But there are exceptions: if you're dealing
with transactions, for example, a deleted object cannot be rolled back,
so when an object decides to delete itself, it will usually register
this fact with the transaction manager, who will delete it as part of
the commit, once it's established that roll back won't be necessary. As
for changing the object, that's a question of the contract: in a lot of
applications, there are mapping objects, which are used to map an
external identifier of some sort to the object. With the goal, often,
of being able to modify the object.
From my understanding and experience, it generally revolves around what it is that you are trying to do as well as the language using pointers (e.g. C++ vs Objective-C).
Usually, though, in C++ terms, I've found that it's best to return either a reference to a smart pointer (such as std::shared_ptr) by reference (perhaps even const reference, depending on the situation), or simply hide the pointer in the class, and if it NEEDS to be accessed or used by something outside of it, use a getter method which either copies the pointer and returns that, or returns a reference to a pointer (granted, AFAIK ref-to-ptr is only possible in C++). If someone doesn't know that you shouldn't delete a ref-to-ptr in most situations (of course, if its deallocation is handled by the class internally), you should really think twice about whether or not they're ready to be doing C++ stuff on your team.
It's fairly common to just use public references for class members if they can be stack allocated (i.e., if they won't take up too much memory), while managing heap allocated objects internally. If you need to set the class member outside of the class, it's possible to just use a set method which takes the required value, rather than access it directly.
Related
C++11 introduced reference-counted smart pointers, std::shared_ptr. Being reference counted, these pointers are unable to automatically reclaim cyclic data structures. However, automatic collection of reference cycles was shown to be possible, for example by Python and PHP. To distinguish this technique from garbage collection, the rest of the question will refer to it as cycle breaking.
Given that there seem to be no proposals to add equivalent functionality to C++, is there a fundamental reason why a cycle breaker similar to the ones already deployed in other languages wouldn't work for std::shared_ptr?
Note that this question doesn't boil down to "why isn't there a GC for C++", which has been asked before. A C++ GC normally refers to a system that automatically manages all dynamically allocated objects, typically implemented using some form of Boehm's conservative collector. It has been pointed out that such a collector is not a good match for RAII. Since a garbage collector primarily manages memory, and might not even be called until there is a memory shortage, and C++ destructors manage other resources, relying on the GC to run destructors would introduce non-determinism at best and resource starvation at worst. It has also bee pointed out that a full-blown GC is largely unnecessary in the presence of the more explicit and predictable smart pointers.
However, a library-based cycle breaker for smart pointers (analogous to the one used by reference-counted interpreters) would have important differences from a general-purpose GC:
It only cares about objects managed through shared_ptr. Such objects already participate in shared ownership, and thus have to handle delayed destructor invocation, whose exact timing depends on ownership structure.
Due to its limited scope, a cycle breaker is unconcerned with patterns that break or slow down Boehm GC, such as pointer masking or huge opaque heap blocks that contain an occasional pointer.
It can be opt-in, like std::enable_shared_from_this. Objects that don't use it don't have to pay for the additional space in the control block to hold the cycle breaker metadata.
A cycle breaker doesn't require a comprehensive list of "root" objects, which is hard to obtain in C++. Unlike a mark-sweep GC which finds all live objects and discards the rest, a cycle breaker only traverses objects that can form cycles. In existing implementations, the type needs to provide help in the form of a function that enumerates references (direct or indirect) to other objects that can participate in a cycle.
It relies on regular "destroy when reference count drops to zero" semantics to destroy cyclic garbage. Once a cycle is identified, the objects that participate in it are requested to clear their strongly-held references, for example by calling reset(). This is enough to break the cycle and would automatically destroy the objects. Asking the objects to provide and clear its strongly-held references (on request) makes sure that the cycle breaker does not break encapsulation.
Lack of proposals for automatic cycle breaking indicates that the idea was rejected for practical or philosophical reasons. I am curious as what the reasons are. For completeness, here are some possible objections:
"It would introduce non-deterministic destruction of cyclic shared_ptr objects." If the programmer were in control of the cycle breaker's invocation, it would not be non-deterministic. Also, once invoked, the cycle breaker's behavior would be predictable - it would destroy all currently known cycles. This is akin to how shared_ptr destructor destroys the underlying object once its reference count drops to zero, despite the possibility of this causing a "non-deterministic" cascade of further destructions.
"A cycle breaker, just like any other form of garbage collection, would introduce pauses in program execution." Experience with runtimes that implement this feature shows that the pauses are minimal because the GC only handles cyclic garbage, and all other objects are reclaimed by reference counting. If the cycle detector is never invoked automatically, the cycle breaker's "pause" could be a predictable consequence of running it, similar to how destroying a large std::vector might run a large number of destructors. (In Python, the cyclic gc is run automatically, but there is API to disable it temporarily in code sections where it is not needed. Re-enabling the GC later will pick up all cyclic garbage created in the meantime.)
"A cycle breaker is unnecessary because cycles are not that frequent and they can be easily avoided using std::weak_ptr." Cycles in fact turn up easily in many simple data structures - e.g. a tree where children have a back-pointer to the parent, or a doubly-linked list. In some cases, cycles between heterogenous objects in complex systems are formed only occasionally with certain patterns of data and are hard to predict and avoid. In some cases it is far from obvious which pointer to replace with the weak variant.
There are a number of issues to be discussed here, so I've rewritten my post to better condense this information.
Automatic cycle detection
Your idea is to have a circle_ptr smart pointer (I know you want to add it to shared_ptr, but it's easier to talk about a new type to compare the two). The idea is that, if the type that the smart pointer is bound to derives from some cycle_detector_mixin, this activates automatic cycle detection.
This mixin also requires that the type implement an interface. It must provide the ability to enumerate all of the circle_ptr instances directly owned by that instance. And it must provide the means to invalidate one of them.
I submit that this is a highly impractical solution to this problem. It is excessively fragile and requires immense amounts of manual work from the user. And therefore, it is not appropriate for inclusion in the standard library. And here are some reasons why.
Determinism and cost
"It would introduce non-deterministic destruction of cyclic shared_ptr objects." Cycle detection only happens when a shared_ptr's reference count drops to zero, so the programmer is in control of when it happens. It would therefore not be non-deterministic. Its behavior would be predictable - it would destroy all currently known cycles from that pointer. This is akin to how shared_ptr destructor destroys the underlying object once its reference count drops to zero, despite the possibility of this causing a "non-deterministic" cascade of further destructions.
This is true, but not in a helpful way.
There is a substantial difference between the determinism of regular shared_ptr destruction and the determinism of what you suggest. Namely: shared_ptr is cheap.
shared_ptr's destructor does an atomic decrement, followed by a conditional test to see if the value was decremented to zero. If so, a destructor is called and memory is freed. That's it.
What you suggest makes this more complicated. Worst-case, every time a circle_ptr is destroyed, the code will have to walk through data structures to determine if there's a cycle. Most of the time, cycles won't exist. But it still has to look for them, just to make sure. And it must do so every single time you destroy a circle_ptr.
Python et. al. get around this problem because they are built into the language. They are able to see everything that's going on. And therefore, they can detect when a pointer is assigned at the time those assignments are made. In this way, such systems are constantly doing small amounts of work to build up cyclic chains. Once a reference goes away, it can look at its data structures and take action if that creates a cyclical chain.
But what you're suggesting is a library feature, not a language feature. And library types can't really do that. Or rather, they can, but only with help.
Remember: an instance of circle_ptr cannot know the subobject it is a member of. It cannot automatically transform a pointer to itself into a pointer to its owning class. And without that ability, it cannot update the data structures in the cycle_detector_mixin that owns it if it is reassigned.
Now, it could manually do this, but only with help from its owning instance. Which means that circle_ptr would need a set of constructors that are given a pointer to its owning instance, which derives from cycle_detector_mixin. And then, its operator= would be able to inform its owner that it has been updated. Obviously, the copy/move assignment would not copy/move the owning instance pointer.
Of course, this requires the owning instance to give a pointer to itself to every circle_ptr that it creates. In every constructor&function that creates circle_ptr instances. Within itself and any classes it owns which are not also managed by cycle_detection_mixin. Without fail. This creates a degree of fragility in the system; manual effort must be expended for each circle_ptr instance owned by a type.
This also requires that circle_ptr contain 3 pointer types: a pointer to the object you get from operator*, a pointer to the actual managed storage, and a pointer to that instance's owner. The reason that the instance must contain a pointer to its owner is that it is per-instance data, not information associated with the block itself. It is the instance of circle_ptr that needs to be able to tell its owner when it is rebound, so the instance needs that data.
And this must be static overhead. You can't know when a circle_ptr instance is within another type and when it isn't. So every circle_ptr, even those that don't use the cycle detection features, must bear this 3 pointer cost.
So not only does this require a large degree of fragility, it's also expensive, bloating the type's size by 50%. Replacing shared_ptr with this type (or more to the point, augmenting shared_ptr with this functionality) is just not viable.
On the plus side, you no longer need users who derive from cycle_detector_mixin to implement a way to fetch the list of circle_ptr instances. Instead, you have the class register itself with the circle_ptr instances. This allows circle_ptr instances that could be cyclic to talk directly to their owning cycle_detector_mixin.
So there's something.
Encapsulation and invariants
The need to be able to tell a class to invalidate one of its circle_ptr objects fundamentally changes the way the class can interact with any of its circle_ptr members.
An invariant is some state that a piece of code assumes is true because it should be logically impossible for it to be false. If you check that a const int variable is > 0, then you have established an invariant for later code that this value is positive.
Encapsulation exists to allow you to be able to build invariants within a class. Constructors alone can't do it, because external code could modify any values that the class stores. Encapsulation allows you to prevent external code from making such modifications. And therefore, you can develop invariants for various data stored by the class.
This is what encapsulation is for.
With a shared_ptr, it is possible to build an invariant around the existence of such a pointer. You can design your class so that the pointer is never null. And therefore, nobody has to check for it being null.
That's not the case with circle_ptr. If you implement the cycle_detector_mixin, then your code must be able to handle the case of any of those circle_ptr instances becoming null. Your destructor therefore cannot assume that they are valid, nor can any code that your destructor calls make that assumption.
Your class therefore cannot establish an invariant with the object pointed to by circle_ptr. At least, not if it's part of a cycle_detector_mixin with its associated registration and whatnot.
You can argue that your design does not technically break encapsulation, since the circle_ptr instances can still be private. But the class is willingly giving up encapsulation to the cycle detection system. And therefore, the class can no longer ensure certain kinds of invariants.
That sounds like breaking encapsulation to me.
Thread safety
In order to access a weak_ptr, the user must lock it. This returns a shared_ptr, which ensures that the object will remain alive (if it still was). Locking is an atomic operation, just like reference incrementing/decrementing. So this is all thread-safe.
circle_ptrs may not be very thread safe. It may be possible for a circle_ptr to become invalid from another thread, if the other thread released the last non-circular reference to it.
I'm not entirely sure about this. It may be that such circumstances only appear if you've already had a data race on the object's destruction, or are using a non-owning reference. But I'm not sure that your design can be thread safe.
Virulence factors
This idea is incredibly viral. Every other type where cyclic references can happen must implement this interface. It's not something you can put on one type. In order to get the benefits, every type that could participate in a cyclical reference must use it. Consistently and correctly.
If you try to make circle_ptr require that the object it manages implement cycle_detector_mixin, then you make it impossible to use such a pointer with any other type. It wouldn't be a replacement of (or augmentation for) shared_ptr. So there is no way for a compiler to help detect accidental misuse.
Sure, there are accidental misuses of make_shared_from_this that cannot be detected by compilers. However, that is not a viral construct. It is therefore only a problem for those who need this feature. By contrast, the only way to get a benefit from cycle_detector_mixin is to use it as comprehensively as possible.
Equally importantly, because this idea is so viral, you will be using it a lot. And therefore, you are far more likely to encounter the multiple-inheritance problem than users of make_shared_from_this. And that's not a minor issue. Especially since cycle_detector_mixin will likely use static_cast to access the derived class, so you won't be able to use virtual inheritance.
Summation
So here is what you must do, without fail, in order to detect cycles, none of which the compiler will verify:
Every class participating in a cycle must be derived from cycle_detector_mixin.
Anytime a cycle_detector_mixin-derived class constructs a circle_ptr instance within itself (either directly or indirectly, but not within a class that itself derives from cycle_detector_mixin), pass a pointer to yourself to that cycle_ptr.
Don't assume that any cycle_ptr subobject of a class is valid. Possibly even to the extent of becoming invalid within a member function thanks to threading issues.
And here are the costs:
Cycle-detecting data structures within cycle_detector_mixin.
Every cycle_ptr must be 50% bigger, even the ones that aren't used for cycle detection.
Misconceptions about ownership
Ultimately, I think this whole idea comes down to a misconception about what shared_ptr is actually for.
"A cycle detector is unnecessary because cycles are not that frequent and they can be easily avoided using std::weak_ptr." Cycles in fact turn up easily in many simple data structures - e.g. a tree where children have a back-pointer to the parent, or a doubly-linked list. In some cases, cycles between heterogenous objects in complex systems are formed only occasionally with certain patterns of data and are hard to predict and avoid. In some cases it is far from obvious which pointer to replace with the weak variant.
This is a very common argument for general-purpose GC. The problem with this argument is that it usually makes an assumption about the use of smart pointers that just isn't valid.
To use a shared_ptr means something. If a class stores a shared_ptr, that represents that the class has ownership of that object.
So explain this: why does a node in a linked list need to own both the next and previous nodes? Why does a child node in a tree need to own its parent node? Oh, they need to be able to reference the other nodes. But they do not need to control the lifetime of them.
For example, I would implement a tree node as an array of unique_ptr to their children, with a single pointer to the parent. A regular pointer, not a smart pointer. After all, if the tree is constructed correctly, the parent will own its children. So if a child node exists, it's parent node must exist; the child cannot exist without having a valid parent.
With a double linked list, I might have the left pointer be a unique_ptr, with the right being a regular pointer. Or vice-versa; one way is no better than the other.
Your mentality seems to be that we should always be using shared_ptr for things, and just let the automatic system work out how to deal with the problems. Whether it's circular references or whatever, just let the system figure it out.
That's not what shared_ptr is for. The goal of smart pointers is not that you don't think about ownership anymore; it's that you can express ownership relationships directly in code.
Overall
How is any of this an improvement over using weak_ptr to break cycles? Instead of recognizing when cycles might happen and doing extra work, you now do a bunch of extra work everywhere. Work that is exceedingly fraglile; if you do it wrong, you're no better off than if you missed a place where you should have used weak_ptr. Only it's worse, because you probably think your code is safe.
The illusion of safety is worse than no safety at all. At least the latter makes you careful.
Could you implement something like this? Possibly. Is it an appropriate type for the standard library? No. It's just too fragile. You must implement it correctly, at all times, in all ways, everywhere that cycles might appear... or you get nothing.
Authoritative references
There can be no authoritative references for something that was never proposed, suggested, or even imagined for standardization. Boost has no such type, and such constructs were never even considered for boost::shared_ptr. Even the very first smart pointer paper (PDF) never considered the possibility. The subject of expanding shared_ptr to automatically be able to handle cycles through some manual effort has never been discussed even on the standard proposal forums where far stupider ideas have been deliberated.
The closest to a reference I can provide is this paper from 1994 about a reference-counted smart pointer. This paper basically talks about making the equivalent of shared_ptr and weak_ptr part of the language (this was in the early days; they didn't even think it was possible to write a shared_ptr that allowed casting a shared_ptr<T> to a shared_ptr<U> when U is a base of T). But even so, it specifically says that cycles would not be collected. It doesn't spend much time on why not, but it does state this:
However, cycles of collected objects with clean-up
functions are problematic. If A and B are reachable from
each other, then destroying either one first will violate
the ordering guarantee, leaving a dangling pointer. If the
collector breaks the cycle arbitrarily, programmers would
have no real ordering guarantee, and subtle, time-dependent
bugs could result. To date, no one has devised a safe,
general solution to this problem [Hayes 92].
This is essentially the encapsulation/invariant issue I pointed out: making a pointer member of a type invalid breaks an invariant.
So basically, few people have even considered the possibility, and those few who did quickly discarded it as being impractical. If you truly believe that they're wrong, the single best way to prove it is by implementing it yourself. Then propose it for standardization.
std::weak_ptr is the solution to this problem. Your worry about
a tree where children have a back-pointer to the parent
can be solved by using raw pointers as the back-pointer. You have no worry of leakage if you think about it.
and your worry about
doubly-linked list
is solved by std::weak_ptr or a raw one.
I believe that the answer to your question is that, contrary to what you claim, there is no efficient way to automatically handle cyclic references. Checking for cycles must be carried out every time a "shared_ptr" is destroyed. On the other hand, introducing any deferring mechanism will inevitably result in a undetermined behavior.
The shared_ptr was not made for automatic reclamation of circular references. It existed in the boost library for some time before being copied to STL. It is a class that attaches a reference counter to any c++ object - be it an array, a class, or an int. It is a relatively lightweight and self-sufficient class. It does not know what it contains, with exception that it knows a deleter function to call when needed.
Al this cycle resolution requires too much heavy code. If you like GC, you can use another language, that was designed for GC from the beginning. Bolting it on via STL would look ugly. A language extension as in C++/CLI would be much nicer.
By reference counting what you ask for is impossible. In order to identify a circle one would have to hold identification of the references to your object. That is easy in memory managed languages since the virtual machine knows who references whom.
In c++ you can only do that by holding a list of references in the circular pointer of e.g. UUID that identifies the object referencing your resources. This would imply that the uuid is somehow passed into the structure when the object is acquired, or that the pointer has access to that resources internals.
These now become implementation specific, since you require a different pointer interface e.g copy and assignment could not be implemented as raw pointers, and demand from every platform to have a uuid source, which cannot be the case for every system. You could of course provide the memory address as a uuid .
Still to overcome the copy , and proper assignment without having a specialized assign method would probably require a single source that allocates references. This cannot be embedded in the language, but may be implemented for a specific application as global registry.
Apart from that, copying such a larger shared pointer would incurr larger performance impact, since during those operations on would have to make lookups for adding , removing, or resolving cycles. Since , doing cycle detection in a graph, from a complexity point of view, would require to traverse the graph registered and apply DFS with backtracking, which is at least proportional to the size of references, I don't see how all these do not scream GC.
I think I have a considerable experience with normal (functional) designed patters, as described e.g. in the gang of four book, which I mainly used in java and C#. In these "managed" languages this is pretty much everything you need to know to get your work done.
However, in C++ world the developer also has the control of how all the objects get allocated, passed around and deleted. I understand the principles (I read Stroutrup among other texts), but it still takes me a lot of effort to decide which mechanism is best for a given scenario - and this is where a portfolio of memory-related design patterns would be useful.
For example, yesterday I had to create a class Results, that was a container for a few objects and a collection (std::vector in this case) of yet another type of objects. So there are a few design questions I couldn't really answer:
Should I return this class by value, or by smart pointer?
Inside the class, should the vector and the objects be normal members, or should they be stored as smart pointers again?
In the vector, should I store the objects directly, or smart pointers to them again?
What should the getters defined on my Results class return (i.e. values, references or smart pointers)?
Of course, smart pointers are cool and what not, but they create syntactic clutter and I am not convinced if using malloc for every single object is optimal approach.
I would be grateful for answers for the specific points above, but even more for some longer and more general texts on memory-related design patterns - so that I can solve the problems I will have on Mondays as well!
The answer to all of your questions ends up being one and the same: it depends on whether you need reference semantics or value semantics (with some caveats to be taken into account).
If you need reference semantics, which is what you have by default in languages like Java and C# for UDTs (User-defined Data Types) declared with the class keyword, then you will have to go for smart pointers. In this scenario you want several clients to hold safe aliases to a specific object, where the word safe encapsulates these two requirements:
Avoid dangling references, so that you won't try to access an object that doesn't exist anymore;
Avoid objects which outlive all of the references to them, so that you won't leak memory.
This is what smart pointers do. If you need reference semantics (and if your algorithms are not such to make the overhead of reference counting significant where shared ownership is needed), then you should use smart pointers.
You do need reference semantics, for instance, when you want the same object to be part of several collections. When you update the object in one collection, you want the representations of the same object in all the other collections to be consistently updated. In this case, you store smart pointers to your objects in those collections. Smart pointers encapsulate the identity of an object rather than its value.
But if you do not need to create aliases, then value semantics is probably what you should rely on. This is what you get by default in C++ when you declare an object with automatic storage (i.e. on the stack).
One thing to consider is that STL collections store values, so if you have a vector<T>, then copies of T will be stored in your vector. Always supposing that you do not need reference semantics, this might become anyway an overhead if your objects are big and expensive to copy around.
To limit the likelyhood of this scenario, C++11 comes with move operations, which make it possible to efficiently transfer objects by value when the old copy of the object is no more needed.
I will now try to use the above concepts to answer your questions more directly.
1) Should I return this class by value, or by smart pointer?
It depends on whether you need reference semantics or not. What does the function do with that object? Is the object returned by that function supposed to be shared by many clients? If so, then by smart pointer. If not, is it possible to define an efficient move operation (this is almost always the case)? If so, then by value. If not, by smart pointer.
2) Inside the class, should the vector and the objects be normal members, or should they be stored as smart pointers again?
Most likely as normal members, since vectors are usually meant to be conceptually a part of your object, and their lifetime is therefore bound to the lifetime of the object that embeds them. You rarely want reference semantics in such a scenario, but if you do, then use smart pointers.
3) In the vector, should I store the objects directly, or smart pointers to them again?
Same answer as for point 1): do you need to share those objects? Are you supposed to store aliases to those objects? Do you want changes to those objects to be seen in different parts of your code which refer those objects? If so, then use shared pointers. If not, is it possible to efficiently copy and/or move those objects? If so (most of the time), store values. If not, store smart pointers.
4) What should the getters defined on my Results class return (i.e. values, references or smart pointers)?
Same answer as for point 2): it depends on what you plan to do with the returned objects: do you want them to be shared by many parts of your code? If so, return a smart pointer. If they shall be exclusively owned by just one part, return by value, unless moving/copying those objects is too expensive or not allowed at all (quite unlikely). In that case, return a smart pointer.
As a side note, please be aware that smart pointers in C++ are a bit trickier than Java/C# references: first of all, you have two main flavors of smart pointers depending on whether shared ownership (shared_ptr) or unique ownership (unique_ptr) is desired. Secondly, you need to avoid circular references of shared_ptr, which would create islands of objects that keep each other alive even though they are no more reachable by your running code. This is the reason why weak pointers (weak_ptr) exist.
These concept naturally lead to the concept of responsibility for managing the lifetime of an object or (more generally) the management of a used resource. You might want to read about the RAII idiom for instance (Resource Acquisition Is Initialization), and about exception handling in general (writing exception-safe code is one of the main reasons why these techniques exist).
I have a GUI to interact with the user, but I have an OOP design problem with this.
Through a dialog the user specifies CDiscreteDistributions and they are stored in a std::vector<CDiscreteDistribution*> in the MyAppDoc class for serialization. Through another dialog the user chooses a type of CDistribution for a particular CParameter. CDiscreteDistribution, CConstantDistribution, and CContinuousDistribution inherit CDistribution, and CParameter has a polymorphic pointer to a CDistribution member variable. MyAppDoc has a container class of CParameter. Thus the CDiscreteDistributions are pointed two twice, but only exist once.
In summary, MyAppDoc has
std::vector<CDiscreteDistribution*>
CContainer which has many CParameter which have
CDistribution* which can point to one of
CDiscreteDistribution which is one of the CDiscreteDistribution*s stored above
CConstantDistribution created/destroyed by CParameter
CContinuousDistribution created/destroyed by CParameter
This design pattern is causing me various nightmares in porting the app to use shared_ptr due to double deletes and serialization (boost). Should one of the pointers to CDiscreteDistribution be a weak_ptr? If so where should own the pointer?
Thanks for any help!
EDIT:
I re-thought the reasoning for having std::vector<CDiscreteDistribution*> and it was just to avoid copying the vector into and out of the GUI. But the objects are quite small, and so I've broken the link between them and suffer the minor performance implications. Now MyAppDoc has:
std::vector<CDiscreteDistribution>
CContainer which has many CParameter which have
CDistribution* which can point to one of
CDiscreteDistribution created/destroyed by CParameter, copied from one of the CDiscreteDistributions stored above
CConstantDistribution created/destroyed by CParameter
CContinuousDistribution created/destroyed by CParameter
I think part of the problem was boost::serialization made two shared_ptrs for each CDiscreteDistribution that weren't aware of each other's existence. Now the only issue is backwards compatibility to files created with the previous versions.
I figure this 'solution' is actually just avoiding a proper design!
The question is described not enough to understand the full situation, complications and exact problem, but in general -
I assume you want to use shared_ptr to not have to manually delete() objects
If so, see if you can solve it by not using shared_ptr, but rather using boost::ptr_vector instead of a vector of raw pointers; the ptr_vector will then handle memory management for you.
I'm not even sure what the shared_ptr would bring you - it's quite obvious, I'd say from my limited understanding of the situation, that the Doc owns the CDiscreteDistribution objects. Whoever owns the other two types of Distributions is responsible for deleting them; this can be done though a shared_ptr or otherwise. (you say 'locally instanced' but that doesn't mean much - are they instantiated on the heap or the stack? What is their lifetime? Why is their lifetime different from the DiscreteDistribution objects? What is 'local' - local to what?)
I agree with Roel that the question is not fully specified. But having done several extensive conversions from raw pointers to shared_ptr, I can give you a little advice.
weak_ptr should not be necessary unless you have circular dependencies. In other words, if an object A has a shared_ptr to object B and object B has a shared_ptr back to object A. In this case, it's impossible for the reference count of either pointer to go to 0, so you would either need to manually intervene to break the cycle, or use weak_ptr to designate one side as dependent.
I'm confused why you have issues with double deletion when using shared_ptr. One of the advantages of smart pointers in general is that you don't have to actually delete them. If you've converted all your raw pointers to shared_ptr, you shouldn't have this problem. If your CConstantDistribution and CContinuousDistribution objects are actually locals rather than allocated with new (I can't tell 100% if this is the case from your description), you can make them shared_ptr objects that are initialized in your constructor, if you can change the app's code. This would allow you to make your std::vector<CDiscreteDistribution*> a container of shared_ptr to CDiscreteDistribution instead. At that point, you shouldn't have to worry about deleting those objects at all, unless you have circular references as described above. Even if you do, you'd have converted a double-delete crash to a memory leak, which is generally less bad.
Serialization can be tough. Since you've tagged the question with MFC, I'll assume you're using MFC serialization. I generally don't have problems when wrapping everything in shared_ptr when I serialize out to a file -- I just use .get() on the smart pointer object and serialize the resulting raw pointer. When I serialize in, I read the raw pointer from the serialized file and wrap it in the shared_ptr candy coating at that point. It's a little extra code in the serialization function, but it works.
If I've guessed inaccurately on some of your situation, feel free to add a comment. I'd be happy to help further if I can.
After reading some tutorials I came to the conclusion that one should always use pointers for objects. But I have also seen a few exceptions while reading some QT tutorials (http://zetcode.com/gui/qt4/painting/) where QPaint object is created on the stack. So now I am confused. When should I use pointers?
If you don't know when you should use pointers just don't use them.
It will become apparent when you need to use them, every situation is different. It is not easy to sum up concisely when they should be used. Do not get into the habit of 'always using pointers for objects', that is certainly bad advice.
Main reasons for using pointers:
control object lifetime;
can't use references (e.g. you want to store something non-copyable in vector);
you should pass pointer to some third party function;
maybe some optimization reasons, but I'm not sure.
It's not clear to me if your question is ptr-to-obj vs stack-based-obj or ptr-to-obj vs reference-to-obj. There are also uses that don't fall into either category.
Regarding vs stack, that seems to already be covered above. Several reasons, most obvious is lifetime of object.
Regarding vs references, always strive to use references, but there are things you can do only with ptrs, for example (there are many uses):
walking through elements in an array (e.g., marching over a standard array[])
when a called function allocates something & returns it via a ptr
Most importantly, pointers (and references, as opposed to automatic/stack-based & static objects) support polymorphism. A pointer to a base class may actually point to a derived class. This is fundamental to the OO behavior supported in C++.
First off, the question is wrong: the dilemma is not between pointers and stack, but between heap and stack. You can have an object on the stack and pass the pointer to that object. I assume what you are really asking is whether you should declare a pointer to class or an instance of class.
The answer is that it depends on what you want to do with the object. If the object has to exist after the control leaves the function, then you have to use a pointer and create the object on heap. You will do this, for example, when your function has to return the pointer to the created object or add the object to a list that was created before calling your function.
On the other hand, if the objects is local to the function, then it is better to use it on stack. This enables the compiler to call the destructor when the control leaves the function.
Which tutorials would those be? Actually, the rule is that you should use pointers only when you absolutely have to, which is quite rarely. You need to read a good book on C++, like Accelerated C++ by Koenig & Moo.
Edit: To clarify a bit - two instances where you would not use a pointer (string is being used here as an exemplar - same would go for any other type):
class Person {
public:
string name; // NOT string * name;
...
};
void f() {
string value; // NOT string * value
// use vvalue
}
You usually have to use pointers in the following scenarios:
You need a collection of objects that belong to different classes (in most cases they will have a common base).
You need a stack-allocated collection of objects so large that it'll likely cause stack overflow.
You need a data structure that can rearrange objects quickly - like a linked list, tree ar similar.
You need some complex logic of lifetime management for your object.
You need a data structure that allows for direct navigation from object to object - like a linked list, tree or any other graph.
In addition to points others make (esp. w.r.t. controlling the object lifetime), if you need to handle NULL objects, you should use pointers, not references. It's possible to create a NULL reference through typecasting, but it's generally a bad idea.
Generally use pointers / references to objects when:
passing them to other methods
creating a large array (I'm not sure what the normal stack size is)
Use the stack when:
You are creating an object that lives and dies within the method
The object is the size of a CPU register or smaller
I actually use pointers in this situation:
class Foo
{
Bar* bar;
Foo(Bar& bar) : bar(&bar) { }
Bar& Bar() const { return *bar; }
};
Before that, I used reference members, initialized from the constructor, but the compiler has a problem creating copy constructors, assignment operators, and the lot.
Dave
using pointers is connected with two orthogonal things:
Dynamic allocation. In general, you should allocate dynamically, when the object is intended to live longer that the scope in which it's created. Such an object is a resource which owner have to be clearly specified (most commonly some sort of smart pointer).
Accessing by address (regardless of how the object was created). In this context pointer doesn't mean ownership. Such accessing could be needed when:
some already existing interface requires that.
association which could be null should be modeled.
copying of large objects should be avoided or copying is impossible at all, but the reference can't be used (e.g., stl collections).
The #1 and #2 can occur in different configurations, for example you can imagine dynamically allocated object accessed by pointer, but such the object could also by passed by reference to some function. You also can get pointer to some object which is created on the stack, etc.
Pass by value with well behaved copyable objects is the way to go for a large amount of your code.
If speed really matters, use pass by reference where you can, and finally use pointers.
If possible never use pointers. Rely on pass by reference or if you are going to return a structure or class, assume that your compiler has return value optimization. (You have to avoid conditional construction of the returned class however).
There is a reason why Java doesn't have pointers. C++ doesn't need them either. If you avoid their use you will get the added benefit of automatic object destruction when the object leaves scope. Otherwise your code will be generating memory errors of various types. Memory leaks can be very tricky to find and often occur in C++ due to unhandled exceptions.
If you must use pointers, consider some of the smart pointer classes like auto_ptr. Auto destruction of objects is more than just releasing the underlying memory. There is a concept called RAII. Some objects require additionally handing on destruction. e.g. mutexes and closing files etc.
Use pointers when you don't want your object to be destroyed when the stack frame is emptied.
Use references for passing parameters where possible.
Speaking about C++, objects created on the stack cannot be used when the program has left the scope it was created in. So generally, when you know you don't need a variable past a function or past a close brace, you can create it on the stack.
Speaking about Qt specifically, Qt helps the programmer by handling a lot of the memory management of heap objects. For objects that are derived from QObject (almost all classes prefixed by "Q" are), constructors take an optional parameter parent. The parent then owns the object, and when the parent is deleted, all owned objects are deleted as well. In essence, the responsibility of the children's destruction is passed to the parent object. When using this mechanism, child QObjects must be created on the heap.
In short, in Qt you can easily create objects on the heap, and as long as you set a proper parent, you'll only have to worry about destroying the parent. In general C++, however, you'll need to remember to destroy heap objects, or use smart pointers.
In a C++ project that uses smart pointers, such as boost::shared_ptr, what is a good design philosophy regarding use of "this"?
Consider that:
It's dangerous to store the raw pointer contained in any smart pointer for later use. You've given up control of object deletion and trust the smart pointer to do it at the right time.
Non-static class members intrinsically use a this pointer. It's a raw pointer and that can't be changed.
If I ever store this in another variable or pass it to another function which could potentially store it for later or bind it in a callback, I'm creating bugs that are introduced when anyone decides to make a shared pointer to my class.
Given that, when is it ever appropriate for me to explicitly use a this pointer? Are there design paradigms that can prevent bugs related to this?
Wrong question
In a C++ project that uses smart pointers
The issue has nothing to do with smart pointers actually. It is only about ownership.
Smart pointers are just tools
They change nothing WRT the concept of ownership, esp. the need to have well-defined ownership in your program, the fact that ownership can be voluntarily transferred, but cannot be taken by a client.
You must understand that smart pointers (also locks and other RAII objects) represent a value and a relationship WRT this value at the same time. A shared_ptr is a reference to an object and establishes a relationship: the object must not be destroyed before this shared_ptr, and when this shared_ptr is destroyed, if it is the last one aliasing this object, the object must be destroyed immediately. (unique_ptr can be viewed as a special case of shared_ptr where there is zero aliasing by definition, so the unique_ptr is always the last one aliasing an object.)
Why you should use smart pointers
It is recommended to use smart pointers because they express a lot with only variables and functions declarations.
Smart pointers can only express a well-defined design, they don't take away the need to define ownership. In contrast, garbage collection takes away the need to define who is responsible for memory deallocation. (But do not take away the need to define who is responsible for other resources clean-up.)
Even in non-purely functional garbage collected languages, you need to make ownership clear: you don't want to overwrite the value of an object if other components still need the old value. This is notably true in Java, where the concept of ownership of mutable data structure is extremely important in threaded programs.
What about raw pointers?
The use of a raw pointer does not mean there is no ownership. It's just not described by a variable declaration. It can be described in comments, in your design documents, etc.
That's why many C++ programmers consider that using raw pointers instead of the adequate smart pointer is inferior: because it's less expressive (I have avoided the terms "good" and "bad" on purpose). I believe the Linux kernel would be more readable with a few C++ objects to express relationships.
You can implement a specific design with or without smart pointers. The implementation that uses smart pointer appropriately will be considered superior by many C++ programmers.
Your real question
In a C++ project, what is a good design philosophy regarding use of "this"?
That's awfully vague.
It's dangerous to store the raw pointer for later use.
Why do you need to a pointer for later use?
You've given up control of object deletion and trust the responsible component to do it at the right time.
Indeed, some component is responsible for the lifetime of the variable. You cannot take the responsibility: it has to be transferred.
If I ever store this in another variable or pass it to another function which could potentially store it for later or bind it in a callback, I'm creating bugs that are introduced when anyone decides to use my class.
Obviously, since the caller is not informed that the function will hide a pointer and use it later without the control of the caller, you are creating bugs.
The solution is obviously to either:
transfer responsibility to handle the lifetime of the object to the function
ensure that the pointer is only saved and used under the control of the caller
Only in the first case, you might end up with a smart pointer in the class implementation.
The source of your problem
I think that your problem is that you are trying hard to complicate matters using smart pointers. Smart pointers are tools to make things easier, not harder. If smart pointers complicate your specification, then rethink your spec in term of simpler things.
Don't try to introduce smart pointers as a solution before you have a problem.
Only introduce smart pointers to solve a specific well-defined problem. Because you don't describe a specific well-defined problem, it is not possible to discuss a specific solution (involving smart pointers or not).
While i don't have a general answer or some idiom, there is boost::enable_shared_from_this . It allows you to get a shared_ptr managing an object that is already managed by shared_ptr. Since in a member function you have no reference to those managing shared_ptr's, enable_shared_ptr does allow you to get a shared_ptr instance and pass that when you need to pass the this pointer.
But this won't solve the issue of passing this from within the constructor, since at that time, no shared_ptr is managing your object yet.
One example of correct use is return *this; in functions like operator++() and operator<<().
When you are using a smart pointer class, you are right that is dangerous to directly expose "this". There are some pointer classes related to boost::shared_ptr<T> that may be of use:
boost::enable_shared_from_this<T>
Provides the ability to have an object return a shared pointer to itself that uses the same reference counting data as an existing shared pointer to the object
boost::weak_ptr<T>
Works hand-in-hand with shared pointers, but do not hold a reference to the object. If all the shared pointers go away and the object is released, a weak pointer will be able to tell that the object no longer exists and will return you NULL instead of a pointer to invalid memory. You can use weak pointers to get shared pointers to a valid reference-counted object.
Neither of these is foolproof, of course, but they'll at least make your code more stable and secure while providing appropriate access and reference counting for your objects.
If you need to use this, just use it explicitly. Smart pointers wrap only pointers of the objects they own - either exclusivelly (unique_ptr) or in a shared manner (shared_ptr).
I personally like to use the this pointer when accessing member variables of the class. For example:
void foo::bar ()
{
this->some_var += 7;
}
It's just a harmless question of style. Some people like it, somepeople don't.
But using the this pointer for any other thing is likely to cause problems. If you really need to do fancy things with it, you should really reconsider your design. I once saw some code that, in the constructor of a class, it assigned the this pointer to another pointer stored somewhere else! That's just crazy, and I can't ever think of a reason to do that. The whole code was a huge mess, by the way.
Can you tell us what exactly do you want to do with the pointer?
Another option is using intrusive smart pointers, and taking care of reference counting within the object itself, not the pointers. This requires a bit more work, but is actually more efficient and easy to control.
Another reason to pass around this is if you want to keep a central registry of all of the objects. In the constructor, an object calls a static method of the registry with this. Its useful for various publish/subscribe mechanisms, or when you don't want the registry to need knowledge of what objects/classes are in the system.