In our app we're about to (finally..) switch from raw pointers to using C++11 smart_ptr templates.
We do have the occasional bug in our app with (non C++) objects still keeping references to our C++ objects causing crashes in the past when accessing the then-dealloc'd objects.
Not sure if this is a silly question - but is there a way to take advantage of the smart_ptr objects and 'dump' the objects still holding on to the C++ objects when none are expected to hold a reference to one any more?
I guess what I'm asking for is a way to list all owners of smart_ptr<MyClass> at a certain point in time.
Any suggestions much appreciated!
No. Without creating your own smart pointer classes that wrap std::unique_ptr and std::shared_ptr (ignore the deprecated std::auto_ptr) that tracks this information, there is no way to do that.
The standard classes themself do not track this information (would be too costly).
Another alternative would be to modify the code of your standard library implementation to track that info. Less invasive on your code since you can keep using the standard names. But probably a bit more tricky than just wrapping the classes and use the wrappers.
Don't believe this is possible for any out of the box c++ smart pointers. You can trivially wrap a shared_ptr to achieve the same effect though.
template<typename T>
class mySmartPtr : boost::noncopyable{
public:
// This method should be the only way this object can be copied
// as the copy constructors are made private
// The newOwnerName parameter can be used to populate m_onwers.
static mySmartPtr<T> newOwner(mySmartPtr<T>&, std::string newOnwerName);
private:
std::shared_ptr<T> m_ptr;
static std::vector<std::string> m_owners;
};
We do have the occasional bug in our app with (non C++) objects still keeping references to our C++ objects causing crashes in the past when accessing the then-dealloc'd objects.
This is serious and must not be ignored.
If you're giving 3rd party library components observer status on your objects, then it stands to reason that the 3rd party component's observer must not outlive your object.
There are 3 common causes for this problem:
Improper lifetime management of the 3rd-party component (you're deleting the observed object before shutting down the 3rd party observer)
improperly detected crossing cases (resulting in 1, above)
In the case were you are the component, you must take orders on from the 3rd party framework as to when you may dispose of your object.
Before involving shared_ptrs you ought to first prove that the shared_ptr may legitimately destroy the object. If the 3rd party component has a 'deregister' method, then you can solve this by:
Ensure ownership of the 3rd party component and your observed object are controlled by the same shared_ptr, or
Use a custom deleter on the shared_ptr to cause de-registration on your controlled object on the 3rd party object before finally deleting it.
If none of this is clear, it's time to take a good long look at your object lifetimes. It often helps to draw sequence diagrams.
Related
note: this question is related to weak_ptr usage, but is not about wrapping weak_ptrs.
I am currently evaluating Swig and I have found an "inconvenience" in the usage of the wrappers by the client languages, that I have not found described online and for which I have no satisfactory solution.
In C++ if you have a complex graph of objects that are managed using shared_ptrs, you must take special care when the graph can have a cycle (i.e. if it is not a DAG) or else you will get memory leaks. By that I mean that if you must (cannot avoid) to have a cycle, it must contain at least one weak_ptr. This means that you will have to handle the cases where you cannot lock the weak_ptr, because the related shared_ptr has died. This management is something that one can expect C++ programmers to be used to deal with. Now let's look at what can happen for a user of the wrappers:
So let's take the following example:
object a, held by a shared_ptr, has a shared_ptr to b
object b, held by a shared_ptr, has a weak_ptr to a
The following could happen to a user of the wrapper:
A a
B b = a->GetB()
// here let's suppose that a gets out of scope, so it can be garbage collected
b->GetA() // fails if a has been garbage collected
The failure could be cleanly managed by propagating a C++ exception to the client code (throw if cannot lock the weak_ptr to create a shared_ptr). However this is not idiomatic to Python/C#/Java users: they do not except to have to manually keep some objects alive to access others.
I have a draft of a an alternative solution which involves creating a C++ "Co-Owner" of objects a and b that would be locked by the SWIG wrappers when any of a or b are accessed via the wrappers, thus keeping both a and b alive when any of them is accessed via the wrappers. The downsides are that this starts to look like I am implementing a proto-garbage-collection in C++, also this modifies the C++ implementation & API of objects a & b and finally the objects a & b will have to be notified by the wrappers that they are used via the wrappers (not something I think I can do without patching SWIG to add function calls in constructor and destructor of shadow objects ??).
Have I missed anything ? Is there another solution to this problem ?
Publicly available weak pointers
If weak ownership is actually part of the interface, it is possible to bind extra types manually that exhibit the behavior of weak pointers. In this example they create a type FooWeakPtr providing the relevant interface. But as you can see, this is not taking advantage of the language-specific classes in every language.
Internal weak pointers
If the weak pointers are not part of the interface, then SWIG-generated bindings should not care about them, and treat the bound objects as shared pointers (at least in Python and Java). Therefore, for as long as your objects are available from the other language, you must make sure they all stay alive.
That means it is up to you to design your hierarchy of objects in a way that clients can never face a case where an internal weak pointer is invalid, and that dropping every reference actually leads to the destruction of the hierarchy.
The solution actually resides in the details of your object hierarchy, and SWIG cannot do much about it.
I've got some code that is using shared_ptr quite widely as the standard way to refer to a particular type of object (let's call it T) in my app. I've tried to be careful to use make_shared and std::move and const T& where I can for efficiency. Nevertheless, my code spends a great deal of time passing shared_ptrs around (the object I'm wrapping in shared_ptr is the central object of the whole caboodle). The kicker is that pretty often the shared_ptrs are pointing to an object that is used as a marker for "no value"; this object is a global instance of a particular T subclass, and it lives forever since its refcount never goes to zero.
Using a "no value" object is nice because it responds in nice ways to various methods that get sent to these objects, behaving in the way that I want "no value" to behave. However, performance metrics indicate that a huge amount of the time in my code is spent incrementing and decrementing the refcount of that global singleton object, making new shared_ptrs that refer to it and then destroying them. To wit: for a simple test case, the execution time went from 9.33 seconds to 7.35 seconds if I stuck nullptr inside the shared_ptrs to indicate "no value", instead of making them point to the global singleton T "no value" object. That's a hugely important difference; run on much larger problems, this code will soon be used to do multi-day runs on computing clusters. So I really need that speedup. But I'd really like to have my "no value" object, too, so that I don't have to put checks for nullptr all over my code, special-casing that possibility.
So. Is there a way to have my cake and eat it too? In particular, I'm imagining that I might somehow subclass shared_ptr to make a "shared_immortal_ptr" class that I could use with the "no value" object. The subclass would act just like a normal shared_ptr, but it would simply never increment or decrement its refcount, and would skip all related bookkeeping. Is such a thing possible?
I'm also considering making an inline function that would do a get() on my shared_ptrs and would substitute a pointer to the singleton immortal object if get() returned nullptr; if I used that everywhere in my code, and never used * or -> directly on my shared_ptrs, I would be insulated, I suppose.
Or is there another good solution for this situation that hasn't occurred to me?
Galik asked the central question that comes to mind regarding your containment strategy. I'll assume you've considered that and have reason to rely on shared_ptr as a communical containment strategy for which no alternative exists.
I have to suggestions which may seem controversial. What you've defined is that you need a type of shared_ptr that never has a nullptr, but std::shared_ptr doesn't do that, and I checked various versions of the STL to confirm that the customer deleter provided is not an entry point to a solution.
So, consider either making your own smart pointer, or adopting one that you change to suit your needs. The basic idea is to establish a kind of shared_ptr which can be instructed to point it's shadow pointer to a global object it doesn't own.
You have the source to std::shared_ptr. The code is uncomfortable to read. It may be difficult to work with. It is one avenue, but of course you'd copy the source, change the namespace and implement the behavior you desire.
However, one of the first things all of us did in the middle 90's when templates were first introduced to the compilers of the epoch was to begin fashioning containers and smart pointers. Smart pointers are remarkably easy to write. They're harder to design (or were), but then you have a design to model (which you've already used).
You can implement the basic interface of shared_ptr to create a drop in replacement. If you used typedefs well, there should be a limited few places you'd have to change, but at least a search and replace would work reasonably well.
These are the two means I'm suggesting, both ending up with the same feature. Either adopt shared_ptr from the library, or make one from scratch. You'd be surprised how quickly you can fashion a replacement.
If you adopt std::shared_ptr, the main theme would be to understand how shared_ptr determines it should decrement. In most implementations shared_ptr must reference a node, which in my version it calls a control block (_Ref). The node owns the object to be deleted when the reference count reaches zero, but naturally shared_ptr skips that if _Ref is null. However, operators like -> and *, or the get function, don't bother checking _Ref, they just return the shadow, or _Ptr in my version.
Now, _Ptr will be set to nullptr (or 0 in my source) when a reset is called. Reset is called when assigning to another object or pointer, so this works even if using assignment to nullptr. The point is, that for this new type of shared_ptr you need, you could simply change the behavior such that whenever that happens (a reset to nullptr), you set _Ptr, the shadow pointer in shared_ptr, to the "no value global" object's address.
All uses of *,get or -> will return the _Ptr of that no value object, and will correctly behave when used in another assignment, or reset is called again, because those functions don't rely upon the shadow pointer to act upon the node, and since in this special condition that node (or control block) will be nullptr, the rest of shared_ptr would behave as though it was pointing to nullptr correctly - that is, not deleting the global object.
Obviously this sounds crazy to alter std::pointer to such application specific behavior, but frankly that's what performance work tends to make us do; otherwise strange things, like abandoning C++ occasionally in order to obtain the more raw speed of C, or assembler.
Modifying std::shared_ptr source, taken as a copy for this special purpose, is not what I would choose (and, factually, I've faced other versions of your situation, so I have made this choice several times over decades).
To that end, I suggest you build a policy based smart pointer. I find it odd I suggested this earlier on another post today (or yesterday, it's 1:40am).
I refer to Alexandrescu's book from 2001 (I think it was Modern C++...and some words I don't recall). In that he presented loki, which included a policy based smart pointer design, which is still published and freely available on his website.
The idea should have been incorporated into shared_ptr, in my opinion.
Policy based design is implemented as the paradigm of a template class deriving from one or more of it's parameters, like this:
template< typename T, typename B >
class TopClass : public B {};
In this way, you can provide B, from which the object is built. Now, B may have the same construction, it may also be a policy level which derives from it's second parameter (or multiple derivations, however the design works).
Layers can be combined to implement unique behaviors in various categories.
For example:
std::shared_ptr and std::weak_ptrare separate classes which interact as a family with others (the nodes or control blocks) to provide smart pointer service. However, in a design I used several times, these two were built by the same top level template class. The difference between a shared_ptr and a weak_ptr in that design was the attachment policy offered in the second parameter to the template. If the type is instantiated with the weak attachment policy as the second parameter, it's a weak pointer. If it's given a strong attachment policy, it's a smart pointer.
Once you create a policy designed template, you can introduce layers not in the original design (expanding it), or to "intercept" behavior and specialize it like the one you currently require - without corrupting the original code or design.
The smart pointer library I developed had high performance requirements, along with a number of other options including custom memory allocation and automatic locking services to make writing to smart pointers thread safe (which std::shared_ptr doesn't provide). The interface and much of the code is shared, yet several different kinds of smart pointers could be fashioned simply by selecting different policies. To change behavior, a new policy could be inserted without altering the existing code. At present, I use both std::shared_ptr (which I used when it was in boost years ago) and the MetaPtr library I developed years ago, the latter when I need high performance or flexible options, like yours.
If std::shared_ptr had been a policy based design, as loki demonstrates, you'd be able to do this with shared_ptr WITHOUT having to copy the source and move it to a new namespace.
In any event, simply creating a shared pointer which points the shadow pointer to the global object on reset to nullptr, leaving the node pointing to null, provides the behavior you described.
I'm trying to write a simple event manager class and listeners for a game engine. In the usual implementation (i.e. McShaffry) the event manager registers listeners which in principle saves a shared_ptr to the listener as a private member.
I have seen in many instances people saying that shared_ptr and the likes should be avoided (eg here). Thus, I'm trying to find ways to implement the event manager without sharing ownership of the listeners.
One method I've thought of, is assigning unique ids to the listeners and register their ids with the event manager. Then the listeners are responsible of 'asking' the event manager after it has updated, if any events are available under their id.
I would like to ask if there are cleaner and/or standard methods to avoid shared ownership in this case, but also generally. For example, I have the same problem with the listeners. The listeners need to store a pointer to their parent (or the object for which they are listening) so that they can call its methods when handling an event.
As Mat’s comment says, there’s no reason not to use smart pointers in general. That said, the cautionary warning does seem to apply in your situation: as far as I understand you don’t have shared ownership; the event manager has sole ownership of the listeners. A shared_ptr would thus be inappropriate here.
An alternative would be to use a unique_ptr which is in many ways the other side of the shared_ptr coin. But depending on how you model listeners even that can be avoided by simply saving concrete instances to the event manager. Without a more detailed description it’s impossible to say whether you need pointers at all but if you don’t need them then, yes, the advice applies: don’t use (smart) pointers when concrete objects would do.
Finally, if your listeners are objects whose ownership is managed elsewhere consider simply using raw pointers to those objects: in that case, the event manager isn’t at all owner of the object – neither the sole nor a shared owner. While this would be the preferred way for me, it requires careful analysis about the listeners’ life-time to ensure that the event manager doesn’t point to listeners which don’t exist any more.
shared_ptr tends to be overused; it is often recommended, for example, on SO as a solution to vaguely stated pointer problems. It is not a substitute for good design, and should not be used unless there is a design in place that is based on understanding object lifetime issues in the code being written.
From personal experience, shared_ptrs a great, but sometimes may not be the correct tool for the job. If the code is entirely under your control, 99.9% of the time, shared_ptr will likely make your life easier. You do need to make sure you don't do thinks like:
Foo *f = new Foo();
shared_ptr<Foo> fptr(f);
shared_ptr<Foo> fptr2(f);
This will cause the memory for f to be deallocated with either fptr1 or fptr2. Instead you want to do something like:
Foo *f = new Foo();
shared_ptr<Foo> fptr(f);
shared_ptr<Foo> fptr2 = fptr;
In the second case, the assignment of one shared pointer to another will increment the reference count.
Another place where you can get in trouble with shared_ptr is if you need to pass a naked pointer to a function (this might occur if you need to pass this as the first parameter to a method, or you are relying on a 3rd party library). You can get the naked pointer from the shared_ptr, but you aren't guaranteed the memory address it's pointing to will still be around, as the reference counter won't be incremented.
You can around this by keeping an additional shared_ptr, though this can be a hassle.
There are other forms of smart pointers. For example, OpenSceneGraph has a ref_ptr which is easier to work with than shared_ptr. The one caveat, is that all objects it points to must descend from Referenced. However, if you're okay with that, I think it's a lot more difficult to have really bad things happen.
In some cases shared_ptr is overkill or doesn't properly exibhit the desired semantics (for example passing ownership).
What you need to do is look at your design and see what ownership model you need. If you need/want shared ownership then just use shared_ptr to model that. If a shared/ref counted ownership is not appropriate use another smart pointer.
Wouldn't your case be a good fit for a nice use of auto_ptr described here : http://www.gotw.ca/publications/using_auto_ptr_effectively.htm (guru of the week «using auto_ptr effectively)
For what I understand, you build a listener, then give it to an event manager. So the event manager can be seen as a "sink".
With the auto_ptr technique, your event manager can cleanly and safely take full ownership of the listener you give him.
I have written a library that exposes references to several related object types. All of these objects have their lifetimes managed by the library internally via boost::shared_ptr
A user of the library would also be able to know, by nature of the library, the lifetimes of any of the exposed objects. So they could store pointers or keep references to these objects. It would be reasonable for them to do this and know when those objects are no longer valid.
But I feel guilty forcing my users to be reasonable.
Is it acceptable to have a library expose weak_ptr's to its objects? Have other libraries done this?
I have profiled this library's usage in apps and have found it to be too mission-critical to expose weak_ptr exclusively.
Would it be wiser to have matching API functions expose either a reference or a weak_ptr or to make any object capable of exposing a weak_ptr to itself?
If the smart_ptrs are already directly accessible to the library's users, then they've already got access to the weak_ptrs, simply via the corresponding weak_ptr's constructor. But if the smart_ptrs are all internal to the library, that's a different story.
In that case, I'd recommend letting each object pass out weak_ptrs to itself, in addition to any other access your library offers. That gives the users the most flexibility: if they need a weak_ptr, they've got immediate access to it; if they need a shared_ptr, they can easily get it; and if they just need access to the object itself, they can ignore the smart pointers entirely.
Of course, I don't know what your library does or how it's used or designed. That might change my recommendation.
Coming up with convoluted mechanisms to get at the objects of your library will only result in people not using your library. If the semantics of the library dictate you need to have people using weak_ptrs, there no way around the user knowing that the objects may go away at some point. Make the interface express as much information about the usage of the library as possible, keeps documentation down and makes it infinitely easier to use.
You can't design around bad/inexperienced users.
If you give your clients access to weak_ptrs they can just lock them to create shared_ptrs and end up delaying the destruction of objects. That might cause problems with your library.
I suggest wrapping a weak_ptr in some other class and giving the caller a shared_ptr to that. That way they can't just call weak_ptr<T>::lock(). You seem to have performance constraints that might influence how you implement it, but a shared_ptr<InterfaceClass> might be a good way to go, and keep the class with the weak_ptr internal to your library.
That way you also keep these implementation details out of your library interface and you can change the way you implement it without changing your interface.
I don't see any problem with exposing weak_ptrs, especially given that TR1 has similar smart pointers (PDF).
TR1 is largely implemented by Visual Studio and GCC, but not some of the other compilers out there. But when it's implemented in all the compilers you care about, you may want to rework the API to expose those smart pointers instead.
If you want to both trap invalid use of the library (trying to access the objects when they have been deleted) as well as have a high-performance API (no weak_ptr's and shared_ptr's in the API), then you could consider have a different API for debug and nondebug builds.
Let's suppose for simplicity that you have only one class of objects you expose; call this class Object. The pointer type which you return from the API for accessing the internal objects is then defined as:
#ifdef DEBUG
typedef ObjectPtrFacade ObjectPtr
#else
typedef Object * ObjectPtr;
#endif
Here the facade is class which you write. It works roughly like this:
class ObjectPtrFacade {
public:
ObjectPtrFacade(Object *o) : wptr(o) { }
// copy constructor and assignment here etc. (not written)
Object * operator -> () const { return access(); }
Object & operator * () const { return *access(); }
private:
Object * access() {
assert(wptr.use_count() > 0);
return (Object *)(wptr.lock());
}
weak_ptr<Object> wptr;
}
In this way, whenever you build a debugging build, you have a special kind of smart pointer in use which asserts before accessing the object that its use_count() is higher than zero, i.e. that the object still exists. If the object has been released, you get a failing assert, which is better than a null pointer reference.
In general of course it is so that using weak_ptr's does not help if you have "stupid" users of the API, because they could call lock() and then still make a null-pointer reference after the weak_ptr returns a shared_ptr which is empty...
I've been evaluating various smart pointer implementations (wow, there are a LOT out there) and it seems to me that most of them can be categorized into two broad classifications:
1) This category uses inheritance on the objects referenced so that they have reference counts and usually up() and down() (or their equivalents) implemented. IE, to use the smart pointer, the objects you're pointing at must inherit from some class the ref implementation provides.
2) This category uses a secondary object to hold the reference counts. For example, instead of pointing the smart pointer right at an object, it actually points at this meta data object... Who has a reference count and up() and down() implementations (and who usually provides a mechanism for the pointer to get at the actual object being pointed to, so that the smart pointer can properly implement operator ->()).
Now, 1 has the downside that it forces all of the objects you'd like to reference count to inherit from a common ancestor, and this means that you cannot use this to reference count objects that you don't have control over the source code to.
2 has the problem that since the count is stored in another object, if you ever have a situation that a pointer to an existing reference counted object is being converted into a reference, you probably have a bug (I.E., since the count is not in the actual object, there is no way for the new reference to get the count... ref to ref copy construction or assignment is fine, because they can share the count object, but if you ever have to convert from a pointer, you're totally hosed)...
Now, as I understand it, boost::shared_pointer uses mechanism 2, or something like it... That said, I can't quite make up my mind which is worse! I have only ever used mechanism 1, in production code... Does anyone have experience with both styles? Or perhaps there is another way thats better than both of these?
"What is the best way to implement smart pointers in C++"
Don't! Use an existing, well tested smart pointer, such as boost::shared_ptr or std::tr1::shared_ptr (std::unique_ptr and std::shared_ptr with C++ 11)
If you have to, then remember to:
use safe-bool idiom
provide an operator->
provide the strong exception guarantee
document the exception requirements your class makes on the deleter
use copy-modify-swap where possible to implement the strong exception guarantee
document whether you handle multithreading correctly
write extensive unit tests
implement conversion-to-base in such a way that it will delete on the derived pointer type (policied smart pointers / dynamic deleter smart pointers)
support getting access to raw pointer
consider cost/benifit of providing weak pointers to break cycles
provide appropriate casting operators for your smart pointers
make your constructor templated to handle constructing base pointer from derived.
And don't forget anything I may have forgotten in the above incomplete list.
Just to supply a different view to the ubiquitous Boost answer (even though it is the right answer for many uses), take a look at Loki's implementation of smart pointers. For a discourse on the design philosophy, the original creator of Loki wrote the book Modern C++ Design.
I've been using boost::shared_ptr for several years now and while you are right about the downside (no assignment via pointer possible), I think it was definitely worth it because of the huge amount of pointer-related bugs it saved me from.
In my homebrew game engine I've replaced normal pointers with shared_ptr as much as possible. The performance hit this causes is actually not so bad if you are calling most functions by reference so that the compiler does not have to create too many temporary shared_ptr instances.
Boost also has an intrusive pointer (like solution 1), that doesn't require inheriting from anything. It does require changing the pointer to class to store the reference count and provide appropriate member functions. I've used this in cases where memory efficiency was important, and didn't want the overhead of another object for each shared pointer used.
Example:
class Event {
public:
typedef boost::intrusive_ptr<Event> Ptr;
void addRef();
unsigned release();
\\ ...
private:
unsigned fRefCount;
};
inline void Event::addRef()
{
fRefCount++;
}
inline unsigned Event::release(){
fRefCount--;
return fRefCount;
}
inline void intrusive_ptr_add_ref(Event* e)
{
e->addRef();
}
inline void intrusive_ptr_release(Event* e)
{
if (e->release() == 0)
delete e;
}
The Ptr typedef is used so that I can easily switcth between boost::shared_ptr<> and boost::intrusive_ptr<> without changing any client code
If you stick with the ones that are in the standard library you will be fine.
Though there are a few other types than the ones you specified.
Shared: Where the ownership is shared between multiple objects
Owned: Where one object owns the object but transfer is allowed.
Unmovable: Where one object owns the object and it can not be transferred.
The standard library has:
std::auto_ptr
Boost has a couple more than have been adapted by tr1 (next version of the standard)
std::tr1::shared_ptr
std::tr1::weak_ptr
And those still in boost (which in relatively is a must have anyway) that hopefully make it into tr2.
boost::scoped_ptr
boost::scoped_array
boost::shared_array
boost::intrusive_ptr
See:
Smart Pointers: Or who owns you baby?
It seems to me this question is kind of like asking "Which is the best sort algorithm?" There is no one answer, it depends on your circumstances.
For my own purposes, I'm using your type 1. I don't have access to the TR1 library. I do have complete control over all the classes I need to have shared pointers to. The additional memory and time efficiency of type 1 might be pretty slight, but memory usage and speed are big issues for my code, so type 1 was a slam dunk.
On the other hand, for anyone who can use TR1, I'd think the type 2 std::tr1::shared_ptr class would be a sensible default choice, to be used whenever there isn't some pressing reason not to use it.
The problem with 2 can be worked around. Boost offers boost::shared_from_this for this same reason. In practice, it's not a big problem.
But the reason they went with your option #2 is that it can be used in all cases. Relying on inheritance isn't always an option, and then you're left with a smart pointer you can't use for half your code.
I'd have to say #2 is best, simply because it can be used in any circumstances.
Our project uses smart pointers extensively. In the beginning there was uncertainty about which pointer to use, and so one of the main authors chose an intrusive pointer in his module and the other a non-intrusive version.
In general, the differences between the two pointer types were not significant. The only exception being that early versions of our non-intrusive pointer implicitly converted from a raw pointer and this can easily lead to memory problems if the pointers are used incorrectly:
void doSomething (NIPtr<int> const &);
void foo () {
NIPtr<int> i = new int;
int & j = *i;
doSomething (&j); // Ooops - owned by two pointers! :(
}
A while ago, some refactoring resulted in some parts of the code being merged, and so a choice had to be made about which pointer type to use. The non-intrusive pointer now had the converting constructor declared as explicit and so it was decided to go with the intrusive pointer to save on the amount of code change that was required.
To our great surprise one thing we did notice was that we had an immediate performance improvement by using the intrusive pointer. We did not put much research into this, and just assumed that the difference was the cost of maintaining the count object. It is possible that other implementations of non-intrusive shared pointer have solved this problem by now.
What you are talking about are intrusive and non-intrusive smart pointers. Boost has both. boost::intrusive_ptr calls a function to decrease and increase the reference count of your object, everytime it needs to change the reference count. It's not calling member functions, but free functions. So it allows managing objects without the need to change the definition of their types. And as you say, boost::shared_ptr is non-intrusive, your category 2.
I have an answer explaining intrusive_ptr: Making shared_ptr not use delete. In short, you use it if you have an object that has already reference counting, or need (as you explain) an object that is already referenced to be owned by an intrusive_ptr.