Related
I have a shared_ptr<MyProto> which I pass around. Eventually, in certain situations, I want pass the raw pointer to a function which then becomes the memory owner. In those cases the shared_ptr isn't responsible anymore for freeing the memory because the function I call took ownership. How do I get the shared_ptr to lose ownership?
The reason I want to have the shared_ptr lose ownership is that I want to use protocol buffer's AddAllocated functionality which takes an already allocated pointer and assumes ownership of it.
Example:
shared_ptr<MyProto> myProtoSharedPtr = // by this point this is the last reference to the heap allocated MyProto
// I want to add it to a collection and serialize the collection without copying
CollectionProto collectionProto;
collectionProto.mutable_my_proto().AddAllocated(myProtoSharedPtr.get()); // at this point collectionProto took ownership of the memory
std::string serialization = collectionProto.SerializeAsString();
// bad: myProtoSharedPtr.get() will be freed twice
I think you can achieve what you want to do by sharing a unique
pointer like this:
std::shared_ptr<std::unique_ptr<MyProto>> myProtoSharedUniquePtr;
Accessing it would be more indirect:
(*myProtoSharedUniquePtr)->do_stuff();
But you could take ownership like this:
CollectionProto collectionProto;
collectionProto.mutable_my_proto().AddAllocated(myProtoSharedUniquePtr->release()); // at this point collectionProto took ownership of the memory
std::string serialization = collectionProto.SerializeAsString();
However I would question why you are using a std::shared_ptr to begin with. The reason to use a std::shared_ptr is when you have no control over who will be last to access it, so each one gets to keep it alive until they are done. So it would be unusual to be able to guarantee all current std::shared_ptr instances are no longer in use.
Are you sure a std::unique_ptr would not be better for your needs?
You could use a unique_ptr, which is anyhow better suited for passing memory around:
unique_ptr<MyProto> myProtoSharedPtr = // create MyPorto object
CollectionProto collectionProto;
// unique_ptr::release returns the pointer and
// releases the ownership of the MyProto object
collectionProto.mutable_my_proto().AddAllocated(myProtoSharedPtr.release());
std::string serialization = collectionProto.SerializeAsString();
You will need to provide std::shared_ptr with a custom deleter (See constructor 4). Then you can define the deleter to do what you want. Including, not destroy your object.
Note 1: I don't recomend using shared_ptr here, but this is a way to do what you want.
Note 2: If you use make_shared to create your objects you will likely run into trouble correctly deleting the memory once the last shared_ptr is removed.
Rather than copying, you could move it into a newed object.
MyProto * myProto = new MyProto(std::move(*mySharedProto));
CollectionProto collectionProto;
collectionProto.mutable_my_proto().AddAllocated(myProto);
You could also investigate whether CollectionProto will accept it by value
CollectionProto collectionProto;
collectionProto.mutable_my_proto().Add(std::move(*mySharedProto));
You could use std::move when you want to transfer the ownership, see the following example
#include <iostream>
#include <memory>
void take_ownership(std::shared_ptr<int> ptr){
std::cout<<ptr.use_count()<<" == 2\n";
} // destroying it
int main()
{
std::shared_ptr<int> p=std::make_shared<int>(1);
std::shared_ptr<int> p2(p);
//p is valid
if(!p.get())
std::cout<<"error\n";
else
std::cout<<"OK\n";
//use p, p2
take_ownership(std::move(p));
//p is invalid
if(!p.get())
std::cout<<"OK\n";
else
std::cout<<p.use_count()<<" error\n";
}
I was thinking about a this situation not for a real implementation but to understand better how pointers works.
class foo(){
foo();
~foo();
void doComplexThings(const std::vector<int*>& v){
int* copy;
for(int i = 0; i < v.size(); i++){
copy = v[i];
// do some stuffs
}
}
}
main(){
std::vector<int*> myVector; // suppose we have 100 elements
doComplexThings(myVector);
for(int i = 0; i < myVector.size(); i++){
delete myVector[i];
}
myVector.clear();
}
Ok, I know that have no sense to copy v[i] inside an other pointer, but I was thinking: copy do a memory leak?
After the execution of doComplexThings(), copy will continue to exist and will occupy space in the heap?
After deleting all elements it will continue to exist and point to a deallocated memory?
So logically if I do this things with complex objects I'll keep occupy the memory with unreference object? Or copy is saved in the stack because I don't use new? And at the end of doComplexThings it will be deleted?
I'm a bit confused, thanks!
There is some confusion on the topic of pointers in the C++ community. While it is true that smart pointers have been added to the library to alleviate problems with dynamic memory allocation, raw pointers are not obsolete. In fact, whenever you want to inspect another object without owning it, you should use a reference or raw pointer, depending on which suits your needs. If the concept of ownership is unclear to you, think of an object as being owned by another object if the latter is responsible for cleaning up afterwards (deleting the former).
For example most uses of new and delete should be replaces with the following (omitting std for brevity):
{
auto ptr_to_T = make_unique<T>(//constructor params);
do_stuff_with_smart_ptr(ptr_to_T);
do_stuff_with_T(*ptr_to_T);
do_stuff_with_raw_ptr(ptr_to_T.get());
} // automatic release of memory allocated with make_unique()
Notice how a function that takes a T* doesn't need a smart pointer if it doesn't keep a copy of the T* it is given, because it doesn't affect the lifetime of the object. The object is guaranteed to be alive past the return point of do_stuff_with_T() and its function signature signals that it doesn't own the object by taking a raw pointer.
On the other hand, if you need to pass the pointer to an object that is allowed to keep the pointer and reference it later, it is unclear when the object will need to be destroyed and most importantly by whom. This is solved via a shared pointer.
ClassThatNeedsSharedOwnership shared_owner;
{
auto ptr_to_T = make_shared<T>(//constructor params);
shared_owner.set_T(ptr_to_T);
// do a lot of stuff
}
// At this point ptr_to_T is destroyed, but shared_owner might keep the object alive
So how does the above factor in to your code. First of all, if the vector is supposed to own (keep alive) the ints it points to, it needs to hold unique_ptr<int> or shared_ptr<int>. If it is just pointing to ints held by something else, and they are guaranteed to be alive until after the vector is destroyed, you are fine with int*. In this case, it should be evident that a delete is never necessary, because by definition your vector and the function working on the vector are not responsible for cleaning-up!
Finally, you can make your code more readable by changing the loop to this (C++11 which you've tagged in the post):
for (auto copy : v){
// equivalent to your i-indexed loop with copy = v[i];
// as long as you don't need the value of i
do_stuff_to_int_ptr(copy);
// no delete, we don't own the pointee
}
Again this is only true if some other object holds the ints and releases them, or they are on the stack but guaranteed to be alive for the whole lifetime of vector<int*> that points to them.
No additional memory is allocated on the heap when you do this:
copy = v[i];
variable copy points to the same address as v[i], but no additional array is allocated, so there would be no memory leak.
A better way of dealing with the situation is to avoid raw pointers in favor of C++ smart pointers or containers:
std::vector<std::vector<int>> myVector;
Now you can remove the deletion loop, which is an incorrect way of doing it for arrays allocated with new int[length] - it should use delete[] instead:
delete[] myVector[i];
Basically you're illustrating the problem with C pointers which lead to the introduction of C++ unique and shared pointers. If you pass a vector of allocated pointers to an opaque member function, you've no way of knowing whether that function hangs onto them or not, so you don't know whether to delete the pointer. In fact in your example you don't seem to, "copy" goes out of scope.
The real answer is that you should only seldom use allocated pointers in C++ at all. The stl vector will serve as a safer, easier to use version of malloc / new. Then you should pass them about as const & to prevent functions from changing them. If you do need an allocated pointer, make one unique_ptr() and then you know that the unique_ptr() is the "owner" of the memory.
always delete pointer even if it is just in function call stack?
Isn't it disappeared when function stack released?
// just Simple class
class CSimple{
int a;
}
// just simple function having pointer.
void simpleFunc(){
CSimple* cSimple = new CSimple();
cSimple->a = 10;
//.. do sth
delete cSimple; // <<< Always, do I have to delete 'cSimple' to prevent the leak of memory?
}
void main(){
for( int =0 ; i< 10 ; i++){
simpleFunc();
}
}
when function stack released?
It is true that "CSimple *csimple" goes away when the function returns.
However, there's a big difference between the pointer, and what it's pointed to.
When a pointer object gets destroyed, nothing happens to whatever the pointer is pointing to. There isn't just one, but two objects here:
The pointer.
What it's pointing to.
In this case, the pointer is pointing to an object in dynamic scope that was created with new.
Nothing is going to happen to this object, otherwise, so you will leak memory.
Therefore, this object needs to be deleted.
After you understand, and fully wrap your brain around this concept, your next step will be to open your C++ book to the chapter that talks about the std::unique_ptr and std::shared_ptr classes, which will take care of these pesky details, for you. You should learn how to use them. Modern C++ code rarely needs to delete something; rather these smart pointers do all the work.
Yes.
On scope exit (ex. when function exists or block { ... } finishes), all objects created on stack will be destroyed and memory will be freed.
This applies to your case, ie. the pointer will be destroyed and memory occupied by the pointer will be freed. The object pointed by the pointer will not be cleared.
This is a common problem and a nightmare when you deal with multiple flow paths (if-else ladders, many return statements) and exceptions.
To solve this problem, we employ 2 main strategies:
RAII
Smart pointers (std::unique_ptr, boost::scoped_ptr, legacy std::auto_ptr, etc).
RAII - without academic consideration - is just creating object on stack, like this:
{
std::string s;
fancy_object obj;
}
When we exit he scope, obj and s destructors will be called duing stack unwinding. Compiler ensures this for all flow paths and will keep proper order of deallocations for us.
If you need to allocate memory on heap, using new, use a smart pointer.
int foo()
{
std::unique_ptr<Object> o(new Object);
... some more code ...
if( something ) { return -1 }
... some more code ...
if( something_else ) { return -2 }
else { not yet }
return 0;
}
As you can see, we can leave the scope using 3 "exists". Normally, you'd need to clear your memory in all cases, which is prone to human errors.
Instead of clearing the object manually in all 3 palces, we rely on automatic destructor call for objects created on stack. Compiler will figure it out. When we leave the scope, std::unique_ptr destructor will be called, calling delete on Object.
Don't be affraid of smart poiners. They are not "slow", "bloat" or other nonsense. Smart poiners are designed to have no overhead on access, adding extra security.
Very similar technique is used for locks. Check out std::lock_guard class.
Yes, you must delete the data that is being pointed to.
The pointer itself is on the stack and does not need to be deleten.
You can, however, store cSimple on the stack, then you don't have to delete it:
void simpleFunc(){
CSimple cSimple; // no new
cSimple.a = 10; // "." instead of "->"
//.. do sth
// no deletion
}
I have the following code in one of my methods:
vector<Base*> units;
Base *a = new A();
Base *b = new B();
units.push_back(a);
units.push_back(b);
Should I destroy the a and b pointers before I exit the method?
Or should I somehow just destroy the units vector of pointers?
Edit 1:
This is another interesting case:
vector<Base*> units;
A a;
B b;
units.push_back(&a);
units.push_back(&b);
What about this case? Now I don't have to use delete nor smart pointers.
Thanks
If you exit the method, units will be destroyed automatically. But not a and b. Those you need to destroy explicitly.
Alternatively, you could use std::shared_ptr to do it for you, if you have C++11.
std::vector<std::shared_ptr<Base>> units;
And you just use the vector almost as you did before, but without worrying about memory leaks when the function exists. I say almost, because you'll need to use std::make_shared to assign into the vector.
A rather old-fashioned solution, that works with all compilers:
for ( vector<Base*>::iterator i = units.begin(); i != units.end(); ++i )
delete *i;
In C++11 this becomes as simple as:
for ( auto p : units )
delete p;
Your second example doesn't require pointer deallocation; actually it would be a bad error to do it. However it does require care in ensuring that a and b remain valid at least as long as units does. For this reason I would advise against that approach.
You need to iterate over the vector and delete each pointer it contains. Deleting the vector will result in memory leaks, as the objects pointed to by its elements are not deleted.
TL;DR: The objects remain, the pointers are lost == memory leak.
Yes you should destroy those pointers (assuming you aren't returning the vector elsewhere).
You could easily do it with a std::for_each as follows:
std::for_each( units.begin(), units.end(), []( Base* p ) { delete p; } );
You should not delete if this two situation match.
Created vector return to out side of the function.
Vector created outside of the function and and suppose to access from other functions.
In other situations you should delete memory pointed by pointers in vector. otherwise after you delete the pointers, no way to refer this memory locations and it calls memory leak.
vector<Base*>::iterator it;
for ( it = units.begin(); it != units.end(); ){
delete * it;
}
I would suggest that you use SmartPointers in the vector. Using smart pointers is a better practice than using raw pointers. You should use the std::unique_ptr, std::shared_ptr or std::weak_ptr smart pointers or the boost equivalents if you don't have C++11. Here is the boost library documentation for these smart pointers.
In the context of this question, yes you have to delete the pointers that are added to the vector. Else it would cause a memory leak.
You have to delete them unless you will have memory leak , in the following code if I comment the two delete lines the destructors never called, also you have to declare the destuctor of the Base class as virtual. As others mentioned is better to use smart pointers.
#include <iostream>
#include <vector>
class Base
{
public:
virtual ~Base(){std::cout << "Base destructor" << std::endl;};
};
class Derived : public Base
{
~Derived(){std::cout << "Derived destructor" << std::endl;};
};
int main()
{
std::vector<Base*> v;
Base *p=new Base();
Base *p2=new Derived();
v.push_back(p);
v.push_back(p2);
delete v.at(0);
delete v.at(1);
};
Output:
Base destructor
Derived destructor
Base destructor
Output with non-virtual base destructor (memory leak):
Base destructor
Base destructor
Yes and no. You don't need to delete them inside the function, but for other reasons than you might think.
You are essentially giving ownership of the objects to the vector, but the vector is not aware of that and therfore wont call delete on the pointers automatically. So if you store owning raw pointers in a vector, you have to manually call delete on them some time. But:
If you give the vector out of your function, you should not destroy the objects inside the function, or the vector full of pointers to freed memory would be pretty useless, so no. But in that case, you should make sure the objects are destroyed after the vector has been used outside the function.
If you don't give the vector out of the function, you should destroy the objects inside the function, but there would be no need to allocate them on the free store, so don't use pointers and new. You just push/emplace the objects themselves into the vector, it takes care of the destruction then, and therfore you don't need delete.
And besides that: Don't use plain new. Use smart pointers. Regardless what you do with them, the smart pointers will take care of a proper destruction of the objects contained. No need to use new, no need to use delete. Ever. (Except when you are writing your own low level data structures, e.g. smart pointers). So if you want to have a vector full of owning pointers, these should be smart pointers. That way you won't have to worry about wether, when and how to destroy the objects and free the memory.
The best way to store pointers in a vector will be to use smart_ptr instead of raw pointers. As soon as the vector DTOR is called and control exits the DTOR all smart_ptrs will be refernced counted. And you should never bothered about the memory leak with smart_ptrs.
In the first example, you will eventually have to delete a and b, but not necessarily when units goes out of scope. Usually you will do that just before units goes out of scope, but that is not the only possible case. It depends on what is intended.
You might (later in the same function) alias a or b, or both, because you want them to outlive units or the function scope. You might put them into two unit objects at the same time. Or, many other possible things.
What's important is that destroying the vector (automatic at scope end in this case) destroys the elements held by the vector, nothing more. The elements are pointers, and destroying a pointer does nothing. If you also want to destroy what the pointer points to (as to not leak memory), you must do that manually (for_each with a lambda would do).
If you don't want to do this work explicitly, a smart pointer can automatize that for you.
The second example (under Edit1) does not require you to delete anything (in fact that's not even possible, you would likely see a crash attempting to do that) but the approach is possibly harmful.
That code will work perfectly well as long as you never reference anything in units any more after a and b left scope. Woe if you do.
Technically, such a thing might even happen invisibly, since units is destroyed after a, but luckily, ~vector does not dereference pointer elements. It merely destroys them, which for a pointer doesn't do anything (trivial destructor).
But imagine someone was so "smart" as to extend the vector class, or maybe you apply this pattern some day in the future (because it "works fine") to another object which does just that. Bang, you're dead. And you don't even know where it came from.
What I really don't like about the code, even though it is strictly "legal" is the fact that it may lead to a condition which will crash or exhibit broken, unreproducable behaviour. However, it does not crash immediately. Code that is "broken" should crash immediately, so you see that something is wrong, and you are forced to fix it. Unluckily that's not the case here.
This will appear to work, possibly for years, until one day it doesn't. Eventually you'll have forgotten that a and b live on the current stack frame and reference the non-existing objects in the vector from some other location. Maybe you dynamically allocate the vector in a future revision of your code, since you pass it to another function. And maybe it will continue to appear working.
And then, you'll spend hours of your time (and likely the time of others) trying to find why a section of code that cannot possibly fail produces wrong results or crashes.
Warning against your second example.
This simple extension leads to undefined behavior:
class A {
public:
int m;
A(int _m): m(_m) {}
};
int main(){
std::vector<A*> units;
for (int i = 0; i < 3; ++i) {
A a(i);
units.push_back(&a);
}
for (auto i : units) std::cout << i->m << " "; // output: 2 2 2 !!!!
return 0;
}
In each loop, the pointer to each a is saved in units, but the objects that they point to go out of scope. In the case of my compiler, the memory address of each a was re-used each time, resulting in units holding three identical memory addresses -- all pointing to the final a object.
It seems to be a common sense in C/C++ that, the function which new/malloc some memory should delete/free them before the function finishes, right?
But suppose I have this kind of problem that, there is a function reader which will read chunks from a file into buffers, there is another function consumer will consume these buffer latter,
void reader(ifstream &ifs)
{
char *buf = new char[100];
ifs.read(buf, 100);
save_buffer(buf); //save this buf into a global array, say buf_array[10]
}
void consumer()
{
for(int i = 0; i < 10; i++)
consume( buf_array[i] ); //do something with those buffers read by reader
}
My problem is, many memory resources are newed inside reader, but reader cannot delete them, because these buffers haven't been used by consumer. Should consumer be responsible for deleteing those buffers?
No one said the function that allocates memory should free the memory. But generally the same component should handle it. Because your reader and consumer form a pair, it is fine for them to coordinate the memory together.
It seems to be a common sense in C/C++ that, the function which new/malloc some memory should delete/free them before the function finishes, right?
No, this is not necessarily true, you do not have to release memory in the same function, as long as you release it eventually before the program ends.
One common solution available in C++ (but not in C) is to deallocate memory in the destructor. If you pass around objects that contain dynamically allocated memory while dealing correctly with the copy/move constructors and assignment operators, the memory will be released when the destructor is called.
The principle is that "for every new, there should be a delete". That doesn't say anything about both calls having to be within the same function (obviously, that wouldn't be very useful).
There's no problem with your example with the reader allocating and the consumer freeing.
You need not free an allocated buffer in the same function in which you initialized it, as long as you carry around a pointer to the buffer.
In your case, consumer() should be responsible for deleteing the buffers allocated by consumer.
As for your secondary question, consumer doesn't know where the buffer ends; you need to tell it somehow. Instead of just storing pointers to the buffers, you might consider defining a new struct encapsulating both a buffer and its length. This has already been done: consider looking at using std::string.
You are copying the contents of buf to a global array buf_arrray which actually feeds the consumer. So, in the example above reader can free the buf.
And, it is not requrired that the function that new/mallocs should be freeing up the memory. The pointer can be passed around. The function that last uses the memory allocated needs to free it the memory.
You don't have to do new/malloc(allocation) and delete/free(release) in the same function. As long as your algorithm guarantees that every allocation does get released and only gets released once, so that it does not cause memory-leak, it is fine.
In fact, often the allocation and release do exist in separate functions.
Just remember these:
1. Use the same pointer to do the release(you can pass the pointer around of course). If you do some arithmetic on the pointer and then release the allocation using that modified pointer, it would produce an error even the pointer still points in the allocation region.
2. As mentioned above, you should guarantee that the allocation got released, and only once. Additional release causes an error.
Slightly different answer to everyone else:
Yes it is fine for the Reader not to free/delete the memory in the reading function, but I wouldn't have the consumer delete the memory. In a simple case works, but what if you have multiple consumers (e.g. to output in different formats)? If consuming the data has the side effect of freeing it, then you cannot do anything else with the data after the fist consumer does it's thing.
I would have a cleanup() type method in my reader that I explicitly call to cleanup the buffers when required. This way the module that allocates the memory is responsible for freeing it (even though it is in a different method).
e.g.
Data d = Reader.read();
Consumer1.consume(d);
Consumer2.consume(d);
Reader.cleanup(d);
// d is no longer valid.
You could also read about shared arrays http://www.boost.org/doc/libs/1_35_0/libs/smart_ptr/shared_array.htm
Using shared arrays, you can allocate them inside a function and pass them by value to other functions. The shared array keeps an internal count of how many times it has been referenced, and deletes the memory automagically when all references have gone out of scope.
eg
void bar( boost::shared_array< int > );
void foo()
{
boost::shared_array< int > x ( new int[ 100 ] );
bar( x );
// because x has been passed to bar(), its reference count is incremented
// and it will not only be deleted when everyone is finished with it!
// do more things with x
} // the memory held by x is deleted
void bar( boost::shared_array< int > y )
{
// Do things with y here
} // the memory held by y is not deleted here because foo() hasn't yet finished
In descending order of preference you'd prefer to:
Use auto storage class, so deletion happens automatically.
Have the same same code allocate and delete objects.
Pass ownership, so one component allocates and another frees.
Have shared ownership via something like shared_ptr.
3 and 4 are nearly tied, but both are quite a ways behind 2, which is quite a ways behind 1.
In this case, what you'd really prefer is for the level of function (or class, etc.) above both the producer and consumer to allocate the buffers, start the producer and consumer, and delete the buffers when both are finished:
class buffer {
std::vector<char> data;
public:
buffer() : data(100) {}
};
process_file(std::string const &name) {
std::vector<buffer> buffers(10);
std::ifstream in(name);
std::pair<std::ifstream *, std::vector<buffer> *> thread_data = {
&in, &buffers
};
prod = start_thread(producer, (void *)&thread_data);
cons = start_thread(consumer);
join(prod);
join(cons);
}
If you can do this, it can help avoid a lot of headaches in managing the memory.