Why do C++ libraries and frameworks never use smart pointers? - c++

I read in a few articles that raw pointers should almost never be used. Instead they should always be wrapped inside smart pointers, whether it's scoped or shared pointers.
However, I noticed that frameworks like Qt, wxWidgets and libraries like Boost never return nor expect smart pointers, as if they were not using them at all. Instead, they return or expect raw pointers. Is there any reason for that? Should I stay away from smart pointers when I write a public API, and why?
Just wondering why smart pointers are recommended when many major projects seem to avoid them.

Apart from the fact that many libraries were written before the advent of standard smart pointers, the biggest reason is probably the lack of a standard C++ Application Binary Interface (ABI).
If you’re writing a header-only library, you can pass around smart pointers and standard containers to your heart’s content. Their source is available to your library at compile time, so you rely on the stability of their interfaces alone, not of their implementations.
But because of the lack of standard ABI, you generally cannot pass these objects safely across module boundaries. A GCC shared_ptr is probably different from an MSVC shared_ptr, which too can differ from an Intel shared_ptr. Even with the same compiler, these classes are not guaranteed to be binary compatible between versions.
The bottom line is that if you want to distribute a prebuilt version of your library, you need a standard ABI on which to rely. C doesn’t have one, but compiler vendors are very good about interoperability between C libraries for a given platform—there are de facto standards.
The situation is not as good for C++. Individual compilers can handle interoperation between their own binaries, so you have the option of distributing a version for every supported compiler, often GCC and MSVC. But in light of this, most libraries just export a C interface—and that means raw pointers.
Non-library code should, however, generally prefer smart pointers over raw.

There can be many reasons. To list few of them:
Smart pointers became part of standard just recently. Till then they
were part of other libraries
Their primary use is to avoid memory leaks; many libraries
don't have their own memory management; Generally they provide
utilities and APIs
They are implemented as wrapper, since they are actually objects and not pointers. Which has additional time/space cost, compared to raw pointers; The users of the libraries may not want to have such overheads
Edit: Using smart pointers is a completely developer's choice. It depends on various factors.
In performance critical systems, you may not want to use smart
pointers which generates overhead
The project which needs the backward compatibility, you may not want
to use the smart pointers which has C++11 specific features
Edit2 There is a string of several downvotes in the span of 24 hours because of below passage. I fail to understand why the answer is downvoted even though below is just an add-on suggestion and not an answer.
However, C++ always facilitates you to have the options open. :) e.g.
template<typename T>
struct Pointer {
#ifdef <Cpp11>
typedef std::unique_ptr<T> type;
#else
typedef T* type;
#endif
};
And in your code use it as:
Pointer<int>::type p;
For those who say that a smart pointer and a raw pointer are different, I agree with that. The code above was just an idea where one can write a code which is interchangeable just with a #define, this is not compulsion;
For example, T* has to be deleted explicitly but a smart pointer does not. We can have a templated Destroy() to handle that.
template<typename T>
void Destroy (T* p)
{
delete p;
}
template<typename T>
void Destroy (std::unique_ptr<T> p)
{
// do nothing
}
and use it as:
Destroy(p);
In the same way, for a raw pointer we can copy it directly and for smart pointer we can use special operation.
Pointer<X>::type p = new X;
Pointer<X>::type p2(Assign(p));
Where Assign() is as:
template<typename T>
T* Assign (T *p)
{
return p;
}
template<typename T>
... Assign (SmartPointer<T> &p)
{
// use move sematics or whateve appropriate
}

There are two issues with smart pointers (pre C++11):
non-standards, so each library tend to reinvent its own (NIH syndrom & dependencies issues)
potential cost
The default smart pointer, in that it is cost-free, is unique_ptr. Unfortunately it requires C++11 move semantics, which only appeared recently. All other smart pointers have a cost (shared_ptr, intrusive_ptr) or have less than ideal semantics (auto_ptr).
With C++11 around the corner, bringing a std::unique_ptr, one would be tempted to think that it is finally over... I am not so optimistic.
Only a few major compilers implement most of C++11, and only in their recent versions. We can expect major libraries such as QT and Boost to be willing to retain compatibility with C++03 for a while, which somewhat precludes the wide adoption of the new and shiny smart pointers.

You shouldn't stay away from smart pointers, they have their use especially in applications where you have to pass a object around.
Libraries tend to either just return a value or populate a object. They don't usually have objects that need to be used in a lot of places, so there is no need for them to use smart pointers (at least not in their interface, they may use them internally).
I could take as example a library we have been working on, where after a few months of development I realized we only used pointers and smart pointers in a few classes (3-5% of all classes).
Passing variables by reference was enough in most places, we used smart pointers whenever we had a object that could be null, and raw pointers when a library that we used forced us to.
Edit (I can't comment because of my reputation):
passing variables by reference is very flexible: if you want the object to be readonly you can use a const reference (you can still do some nasty casts to be able to write the object) but you get the maximum of protection possible (it's the same with smart pointers).
But I do agree that it's much nicer to just return the object.

Qt pointlessly re-invented many parts of the Standard library in an attempt to become Java. I believe that it does actually have its own smart pointers now, but in general, it is hardly a pinnacle of design. wxWidgets, as far as I'm aware, was designed long before usable smart pointers were written.
As for Boost, I fully expect that they use smart pointers wherever appropriate. You might have to be more specific.
In addition, don't forget that smart pointers exist to enforce ownership. If the API has no ownership semantics, then why use a smart pointer?

Good question. I don't know the specific articles to which you refer, but I have read similar things from time to time. My suspicion is that the writers of such articles tend to harbor a bias against C++-style programming. If the writer programs in C++ only when he must, then returns to Java or such as soon as he can, then he doesn't really share the C++ mindset.
One suspects that some or most of the same writers prefer garbage-collecting memory managers. I don't, but I think differently than they do.
Smart pointers are great, but they have to keep reference counts. The keeping of reference counts bears costs -- often modest costs, but costs nonetheless -- at runtime. There is nothing wrong with saving these costs by using bare pointers, especially if the pointers are managed by destructors.
One of the excellent things about C++ is its support for embedded-systems programming. The use of bare pointers is part of that.
Update: A commenter has correctly observed that C++'s new unique_ptr (available since TR1) does not count references. The commenter also has a different definition of "smart pointer" than I have in mind. He may be right about the definition.
Further update: The comment thread below is illuminating. All of it is recommended reading.

There are also other types of smart pointers. You might want a specialized smart pointer for something like network replication (one that detects if it's accessed and sends any modifications to the server or some such), keeps a history of changes, marks the fact that it was accessed so it can be investigated when you save data to disk and so on. Not sure if doing that in the pointer is the best solution but using the built in smart pointer types in libraries could result in people being locked into them and loosing the flexibility.
People can have all kinds of different memory management requirements and solutions beyond smart pointers. I might want to manage memory myself, I could be allocating space for things in a memory pool so it's allocated in advance and not at runtime (useful for games). I might be using a garbage collected implementation of C++ (C++11 makes this possible although none exist yet). Or maybe I'm just not doing anything advanced enough to worry about bothering with them, I can know that I'm not going to forget to uninitialized objects and so on. Maybe I'm just confident in my ability to manage memory without the pointer crutch.
Integration with C is another issue too.
Another issue is smart pointers are part of the STL. C++ is designed to be usable without the STL.

It also depends on what domain you work in. I write game engines for a living, we avoid boost like the plague, in games the overhead of boost isn't acceptable. In our core engine we ended up writing our own version of stl (Much like the ea stl).
If i was to write a forms application, i might consider using smart pointers; but once memory management is second nature not having granular control over memory becomes quiet annoying.

Related

The std::get_temporary_buffer returns raw pointer instead of smart one

The std::get_temporary_buffer returns a std::pair holding a pointer to the beginning of the allocated storage and the number of objects allocated, and the only purpose of its counterpart: std::return_temporary_buffer is to deallocate memory previously allocated with std::get_temporary_buffer.
Both functions lies on the <memory> header which main purpose is to provide tools to enhance memory management (as it name implies) and make memory management more secure.
About the security of the memory management, the <memory> header provides also the smart pointers utility which allows to manage the memory in a RAII-like manner and hence making the memory management exception safe.
C++14 also added the std::make_unique helper function, so we can avoid using raw pointers in many cases nowadays.
With all this efforts in reducing the use of raw pointers, realizing that std::get_temporary_buffer returns a raw pointer instead of a smart pointer is pretty confusing. Thats why I want to ask:
Is there any reason for std::get_temporary_buffer to return a raw pointer instead of returning a smart one?
If there's a reason for this "old fashioned" way to allocate and deallocate memory manually, which goal it have that cannot be achieved with smart pointers?
The simple answer is that std::get_temporary_buffer was created before smart pointers were standardized, and changing the return value of std::get_temporary_buffer in C++11 would have broken code that depended on it, which is absolutely unacceptable for the C++ standard library.
Now, why haven't they standardized a new smart pointer equivalent?
Well, maybe no one was interested in having one. Personally, I find it weird to have one smart pointer own many objects. If you need a smart array, use std::vector.
If you look at the docs for the old SGI STL implementations of get_temporary_buffer et al, they say...
Note: get_temporary_buffer and return_temporary_buffer are only provided for backward compatibility. If you are writing new code, you should instead use the temporary_buffer class.
That effectively acknowledges the desirability of better automated management. GCC added temporary_buffer as an extension (see here), but it never made it into the Standard. Long and short of it is that it's just not that useful, so having a better interface won't have been a priority. The whole notion of the OS guessing at whether it should give you all the requested memory or some smaller amount flies in the face of the optimistic memory allocation strategies used by most modern Operating Systems, and once you get multiple calls requesting more than the easily available memory, being too generous with the first leaves the others a bit starved: just not a very practical notion.
If you care, you could submit a proposal for a later C++ Standard....

using non-smart pointers in modern C++

Short Version:
Is there any acceptable reason for using non-smart pointers in modern C++?
Long Version:
we have a huge product that contains lot's of old C++ code and now we are trying to refactor it to the modern C++ era. Along with all the old fashioned code, there is huge amount of pointers passing around (mostly with SAL annotations to provide some sense of security) and I was wondering if we should change all of them to smart pointers or maybe leave some of them as is?
Trying to convert some of these codes, I ended up with a code that is simply arguable for over using smart pointers.
So the question is: is there such a thing as over using smart pointers?
Or in other words: is there any acceptable scenario for non-smart pointers these days?
Smart pointers (unique_ptr and shared_ptr) should be OWNING pointers (i.e., responsible for destruction of the object). The bottom line of using them is that any object created by new should be shoved into a unique_ptr ASAP, to prevent memory leaks. After that, the unique_ptr should end up being moved:
either into a shared_ptr if ownership should be shared,
or into a unique_ptr if ownership is determined by a scope (of a block, or the lifetime of an object).
releases should be rare. If your code passes non-owning pointers around, these should be:
raw pointers if they may be null, (obtained by get)
references if they may not be null, (obtained by get)
unique_ptrs by value if the aim of the call is transferring ownership. (in which case you'll need to move them)
Factory methods should return unique_ptrs by value. (because then, if you don't assign the return value of the factory method, the object is immediately de-allocated)
And check out Ali's answer regarding links to some philosophical points of handling legacy code. (Which I totally agree on)
Short Version:
Is there any acceptable reason for using non-smart
pointers in modern C++?
Short answer:
Definitely, if they only serve for observation, that is, they don't own the pointee. However, try to use references instead of pointers even in this case; use pointers only if you really need to make them optional (initialize with null_ptr then reassign later, for example).
Long Version:
we have a huge product that contains lot's of old C++ code and now we are trying to refactor it to the modern C++ era. [...]
Long answer:
As I am reading these lines this answer comes to mind:
Advice on working with legacy code
I wish I could upvote this answer more than once. I would quote: "[...] for each re-factor we have made we can justify 'this specific change will make the actual task we are doing now easier'. Rather than 'this is now cleaner for future work'."
Long story short, don't do big refactoring unless you really need to.
So the question is: is there such a thing as over using smart pointers?
In my opinion, std::shared_ptr is overused. It is very comfortable to use, and it gives you the illusion that you don't have to think about ownership issues. But that is not the whole picture. I fully agree with Sean Parent: "a shared pointer is as good as a global variable." Shared pointers can also introduce very difficult ownership issues, etc.
On the other hand, if you need to allocate something on the heap, use a unique_ptr. You can't overuse it, if you really need heap allocation. In my experience, using unique_ptr also leads to cleaner and easier to understand code, as the ownership issues become self-explanatory.
Interesting talks from Sean Parent on how to avoid / reduce the usage of pointers are:
Inheritance Is The Base Class of Evil
Value Semantics and Concepts-based Polymorphism
Hope this helps.
Yes, raw pointers still have a uses as an "optional reference". I.e. a T* is similar to a T&. Neither implies ownership, but a T* can be a nullptr.
Check out the talks here: http://channel9.msdn.com/Events/GoingNative/2013 (especially Stroustrup's talk).
The short answer is no, assuming "modern C++" is >= c++11
The long answer is that wasn't always the case and trying to restructure a large project is almost always hard. The way we think about problems is constrained by the tools we have to solve them. There are many cases when doing such refactoring when it simply makes more sense to use pointers than try to re-express basic logic to be class and smart pointer friendly. I think it's less of a case where smart pointers are over-used and more of a case when classes are under used. YMMV ;-)
Of course there are use cases for raw pointers in modern C++:
interfaces that must be compile-able as pure C (although the implementation itself may make use of those C++ features, that are not also C features, like classes, or smart-pointers)
code that is extremely low level, so low level, that even the simplest smart-pointer proves as being to heavy-weight
Of course those are rather rare cases and for by far the most use cases of pointers smart pointers should be just fine for new code, BUT:
If the existing code works fine with raw pointers, why invest time to rewrite it and risk to add bugs when converting it to a smart-pointer using version?
Don't refactor code, that is working fine, just because the new code better follows modern programming-standards. These programming standards are not existing for their own sake, but to make working with some code easier, so don't do refactoring, which will cost you more time than they can save you in the future.
That means: If it will cost you more time to check, which of those pointers can be safely converted into smart-pointers and which can't and to hunt the bugs, that your refactoring may have introduced, than you will be able to save on future maintenance work due to the refactoring, then simply don't refactor and let it stay as it is.
If the refactoring will save more time than it costs for only some parts of the code base, then consider to only refactor those parts of the code base.

improvements/changes in pointers with time?

perhaps, pointers have been on a course of modification
regular pointers to auto_ptr to unique_ptr and shared_ptr
I just wanted to know that the new types of pointers that have been developed completely overtake over the older versions(OR the versions of the pointers that you may consider having memory leak problems)
I don't think regular pointers are dead but recently I came across auto-ptr and then I came to know a bit about shared_ptr and unique_ptr added to C++ox.I can't use them(shared_ptr and unique_ptr) though because I am having visual C++ 2008.
Also is auto_ptr completely taken over by unique_ptr and shared_ptr?
Please give some insight.
Raw pointers are still useful if you don't own the object it points at. auto_ptr is officially deprecated (so yes, it's pretty much dead for new code). VS2008 provides shared_ptr though as part of TR1, and there's always Boost. I don't see any need for any other smart pointers in my C++ life right now.
Qt offers QSharedDataPointer, which helps provide copy-on-write functionality. Its intrusive, in the sense that it only accepts types derived from QSharedData rather than any arbitrary data, however. My own personal library of utility objects includes a deep_ptr type with a copy constructor that copies its stored data rather than just aliasing the pointer. It is used exclusively as a means of implementing Pimpls such that I don't have to bother with implementing the Big 3 but still get value semantics out of the box.
But really, shared_ptr, weak_ptr, and unique_ptr do indeed provide almost all the functionality one needs.

pros and cons of smart pointers

I came to know that smart pointer is used for resource management and supports RAII.
But what are the corner cases in which smart pointer doesn't seem smart and things to be kept in mind while using it ?
Smart pointers don't help against loops in graph-like structures.
For example, object A holds a smart pointer to object B and object B - back to object A. If you release all pointers to both A and B before disconnection A from B (or B from A) both A and B will hold each other and form a happy memory leak.
Garbage collection could help against that - it could see that both object are unreachable and free them.
I would like to mention performance limitations. Smart pointers usually use atomic operations (such as InterlockedIncrement in Win32 API) for reference counting. These functions are significantly slower than plain integer arithmetic.
In most cases, this performance penalty is not a problem, just make sure you don't make too many unnecessary copies of the smart pointer object (prefer to pass the smart pointer by reference in function calls).
This is quite interesting: Smart Pointers.
It's a sample chapter from "Modern C++ Design" by Andrei Alexandrescu.
Watch out at the transitions - when assigning between raw and smart pointers. Bad smart pointers - like _com_ptr_t - make it worse by allowing implicit conversions. Most errors happen at the transition.
Watch out for cycles - as mentioned, you need weak pointers to break the cycles. However, in a complex graph that's not always easy to do.
Too much choice - most libraries offer different implementations with different advantages / drawbacks. Unfortunately, most of the time these different variants are not compatible, which becomes a probem when mixing libraries. (say, LibA uses LOKI, LibB uses boost). Having to plan ahead for enable_shared_from_this sucks, having to decide naming conventions between intrusive_ptr, shared_ptr and weak_ptr for a bunch of objects sucks.
For me, the single most e advantage of shared_ptr (or one of similar functionality) is that it is coupled to its destruction policy at creation. Both C++ and Win32 offers so many ways of getting rid of things it's not even funny. Coupling at construction time (without affecting the actual type of the pointer) means I have both policies in one place.
Beside technical limitations (already mentioned : circular dependencies), i'd say that the most important thing about smart pointers is to remember that it's still a workaround to get heap-allocated-objects deleted.
Stack allocation is the best option for most cases - along with the use of references - to manage the lifetime of objects.
The following article is a very interesting paper
Smart Pointers: Can't Live With 'Em, Can't Live Without 'Em
Here are a few things
There's no definite entity which destroys the object. Often it is desired to know exactly when and how an object is being destroyed
Circular dependencies - If they exist you'll have a memory leak. That's usually where weak_ptr come to the rescue.
There is a problem with reference counting in certain types of data structures that have cycles. There can also be problems with accessing smart pointers from multiple threads, concurrent access to reference counts can cause problems. There's a utility in boost called atomic.hpp that can mitigate this problem.
Many people run into problems when using smart pointers mixed with raw pointers (to the same objects). A typical example is when interacting with an API that uses raw pointers.
For example; in boost::shared_ptr there is a .get() function that returns the raw pointer. Good functionality if used with care, but many people seem to trip on it.
IMHO it's an example of a "leaky abstraction".
Raymond Chen is notoriously ambivalent about smart pointers. There are issues about when the destructor actually runs (please note, the destructor runs at a well-defined time in a well-defined order; it's just that once in a while you'll forget that it's after the last line in your function).
Also remember that "smart pointer" is a pretty big category. I include std::vector in that category (a std::vector is essentially a smart array).

Once you've adopted boost's smart pointers, is there any case where you use raw pointers?

I'm curious as I begin to adopt more of the boost idioms and what appears to be best practices I wonder at what point does my c++ even remotely look like the c++ of yesteryear, often found in typical examples and in the minds of those who've not been introduced to "Modern C++"?
I don't use shared_ptr almost at all, because I avoid shared ownership in general. Therefore, I use something like boost::scoped_ptr to "own" an object, but all other references to it will be raw pointers. Example:
boost::scoped_ptr<SomeType> my_object(new SomeType);
some_function(my_object.get());
But some_function will deal with a raw pointer:
void some_function(SomeType* some_obj)
{
assert (some_obj);
some_obj->whatever();
}
Just a few off the top of my head:
Navigating around in memory-mapped files.
Windows API calls where you have to over-allocate (like a LPBITMAPINFOHEADER).
Any code where you're munging around in arbitrary memory (VirtualQuery() and the like).
Just about any time you're using reinterpret_cast<> on a pointer.
Any time you use placement-new.
The common thread here is "any situation in which you need to treat a piece of memory as something other than a resource over which you have allocation control".
These days I've pretty much abandoned all use of raw pointers. I've even started looking through our code base for places where raw pointers were used and switched them to a smart pointer variant. It's amazing how much code I've been able to delete by doing this simple act. There is so much code wasted on lifetime management of raw C++ pointers.
The only places where I don't use pointers is for a couple of interop scenarios with other code bases I don't have control over.
I find the primary difference between 'modern' C++ and the old* stuff is careful use of class invariants and encapsulation. Well organised code tends naturally to have fewer pointers flying around. I'm almost as nervous swimming in shared_ptrs as I would be in news and deletes.
I'm looking forward to unique_ptr in C++0x. I think that will tidy away the few (smart) pointers that do still roam the wild.
*still unfortunately very common
Certainly any time you're dealing with a legacy library or API you'll need to pass a raw pointer, although you'll probably just extract it from your smart pointer temporarily.
In fact it is always safe to pass a raw pointer to a function, as long as the function does not try to keep a copy of the pointer in a global or member variable, or try to delete it. With these restrictions in place, the function cannot affect the lifetime of the object, and the only reason for a smart pointer is to manage the object lifetime.
I still use regular pointers in resource-sensitive code or other code that needs tiny footprint, such as certain exceptions, where I cannot assume that any data is valid and must also assume that I am running out of memory too.
Managed memory is almost always superior to raw otherwise, because it means that you don't have to deal with deleting it at the right place, but still have great control over the construction and destruction points of your pointers.
Oh, and there's one other place to use raw pointers:
boost::shared_ptr<int> ptr(new int);
I still use raw pointers on devices that have memory mapped IO, such as embedded systems, where having a smart pointer doesn't really make sense because you will never need or be able to delete it.
If you have circular data structures, e.g., A points to B and B points back to A, you can't use naively use smart pointers for both A and B, since then the objects will only be freed extra work. To free the memory, you have to manually clear the smart pointers, which is about as bad as the delete the smart pointers get rid of.
You might thing this doesn't happen very often, but suppose you have Parent object that has smart pointers to a bunch of Child objects. Somewhere along the way someone needs to look up a the Parent for a Child, so they add a smart pointer member to Child that points back to the parent. Silently, memory is no longer freed.
Some care is required. Smart pointers are not equivalent to garbage collection.
I'm writing C++ that has to co-exist with Objective C (using Objective C++ to bridge).
Because C++ objects declared as part of Objective C++ classes don't have constructors or destructors called you can't really hold them there in smart pointers.
So I tend to use raw pointers, although often with boost::intrustive_ptr and an internal ref count.
Not that I would do it, but you need raw pointers to implement, say, a linked list or a graph. But it would be much smarter to use std::list<> or boost::graph<>.