Related
Is there any way to distinguish two following situations at run time:
double ptr * = new double(3.14159);
double variable = 3.14159
double * testPtr_1 = ptr;
double * testPtr_2 = &variable;
delete testPtr_1 // fine...
delete testPtr_2 // BIG RUN TIME ERROR !!!
I have find myself in situation in with I need to call delete operator for some unknown pointer. The pointer can point to anywhere (to a "local" variable or to dynamically allocated variable).
How can I find out where my "unknown" pointer points, and therefore choose when to and when not to call operator delete on it
EDIT:
Ok I see that everyone is pointing me to the smart pointers, but what if I am trying to write my own set of smart pointers (that is The reason behind my question)?
There is no way to test if a pointer is pointing to a memory area that would be valid to delete. Moreover,
There is no way to tell between pointers that must be freed with delete vs. delete[],
There is no way to tell between the pointers that have been freed and pointers that have not been freed,
There is no way to tell among pointers to an automatic variable, pointers to static variable, and pointers to dynamically allocated blocks.
The approach that you should take is tracking allocations/deallocations by some other means, such as storing flags along with your pointers. However, this is rather tedious: a much better practice is to switch to smart pointers, which would track resources for you.
You need to set some better coding practices for yourself (or for your project).
Especially since most platforms have, at the very least, a C++11-compliant compiler, there's no reason not to be using the following paradigm:
Raw Pointers (T*) should ONLY be used as non-owning pointers. If you receive a T* as the input for a function or constructor, you should assume you have no responsibility for deleting it. If you have an instance or local variable that is a T*, you should assume you have no responsibility for deleting it.
Unique Pointers (std::unique_ptr<T>) should be used as single-ownership pointers, and in general, these should be your default go-to choice for any situation where you need to dynamically allocate memory. std::make_unique<T>() should be preferred for creating any kind of Unique Pointer, as this prevents you from ever seeing the raw pointer in use, and it prevents issues like you described in your original post.
Shared Pointers (std::shared_ptr<T> and std::weak_ptr<T>) should ONLY be used in situations where it is logically correct to have multiple owners of an object. These situations occur less often than you think, by the way! std::make_shared<T>() is the preferred method of creating Shared Pointers, for the same reasons as std::make_unique, and also because std::make_shared can perform some optimizations on the allocations, improving performance.
Vectors (std::vector<T>) should be used in situations where you need to allocate multiple objects into heap space, the same as if you called new T[size]. There's no reason to use pointers at all except in very exotic situations.
It should go without saying that you need to take my rules of "ONLY do 'x'" with a grain of salt: Occasionally, you will have to break those rules, and you might be in a situation where you need a different set of rules. But for 99% of use-cases, those rules are correct and will best convey the semantics you need to prevent memory leaks and properly reason about the behavior of your code.
You cannot.
Avoid raw pointers and use smart pointers, particularly std::unique_ptr. It conveys clearly who is responsible for deleting the object, and the object will be deleted when the std::unique_ptr goes out of scope.
When creating objects, avoid using new. Wrap them in a smart pointer directly and do not take addresses of anything to wrap it in a smart pointer. This way, all raw pointers will never need freeing and all smart pointers will get cleaned up properly when their time has come.
Okay, some things you can distinguish in a very platform-specific, implementation-defined manner. I won’t go into details here, because it’s essentially insane to do (and, again, depends on the platform and implementation), but you are asking for it.
Distinguish local, global and heap variables. This is actually possible on many modern architectures, simply because those three are different ranges of the address space. Global variables live in the data section (as defined by the linker and run-time loader), local variables on the stack (usually at the end of the address space) and heap variables live in memory obtained during run-time (usually not at the end of the address space and of course not overlapping the data and code sections, a.k.a. "mostly everything else"). The memory allocator knows which range that is and can tell you details about the blocks in there, see below.
Detect already-freed variables: you can ask the memory allocator that, possibly by inspecting its state. You can even find out when a pointer points into a allocated region and then find out the block to which it belongs. This is however probably computationally expensive to do.
Distinguishing heap and stack is a bit tricky. If your stack grows large and your program is running long and some piece of heap has been returned to the OS, it is possible that an address which formerly belonged to the heap now belongs to the stack (and the opposite may be possible too). So as I mentioned, it is insane to do this.
You can't reliably. This is why owning raw pointers are dangerous, they do not couple the lifetime to the pointer but instead leave it up to you the programmers to know all the things that could happen and prepare for them all.
This is why we have smart pointers now. These pointers couple the life time to the pointer which means the pointer is only deleted once it is no longer in use anywhere. This makes dealing with pointer much more manageable.
The cpp core guildlines suggests that a raw pointer should never be deleted as it is just a view. You are just using it like a reference and it's lifetime is managed by something else.
Ok I see that everyone is pointing me to the smart pointers, but what if I am trying to write my own set of smart pointers (that is The reason behind my question)?
In that case do like the standard smart pointers do and take a deleter which you default to just using delete. That way if the user of the class wants to pass in a pointer to a stack object they can specify a do nothing deleter and you smart pointer will use that and, well, do nothing. This puts the onus on the person using the smart pointer to tell the pointer how to delete what it points to. Normally they will never need to use something other than the default but if they happen to use a custom allocator and need to use a custom deallocator they can do so using this method.
Actually you can. But memory overhead occurs.
You overload new and delete operator and then keep track of allocations and store it somewhere(void *)
#include<iostream>
#include<algorithm>
using namespace std;
void** memoryTrack=(void **)malloc(sizeof(void *)*100); //This will store address of newly allocated memory by new operator(really malloc)
int cnt=0;//just to count
//New operator overloaded
void *operator new( size_t stAllocateBlock ) {
cout<<"in new";
void *ptr = malloc(stAllocateBlock); //Allocate memory using malloc
memoryTrack[cnt] = ptr;//Store it in our memoryTrack
cnt++; //Increment counter
return ptr; //return address generated by malloc
}
void display()
{
for(int i=0;i<cnt;i++)
cout<<memoryTrack[i]<<endl;
}
int main()
{
double *ptr = new double(3.14159);
double variable = 3.14159;
double * testPtr_1 = ptr;
double * testPtr_2 = &variable;
delete testPtr_1; // fine...
delete testPtr_2;
return 0;
}
Now the most important function(You will have to work on this because it is not complete)
void operator delete( void *pvMem )
{
//Just printing the address to be searched in our memoryTrack
cout<<pvMem<<endl;
//If found free the memory
if(find(memoryTrack,memoryTrack+cnt,pvMem)!=memoryTrack+cnt)
{
//cout<<*(find(memoryTrack,memoryTrack+cnt,pvMem));
cout<<"Can be deleted\n";
free (pvMem);
//After this make that location of memoryTrack as NULL
//Also keep track of indices that are NULL
//So that you can insert next address there
//Or better yet implement linked list(Sorry was too lazy to do)
}
else
cout<<"Don't delete memory that was not allocated by you\n";
}
Output
in new
0xde1360
0xde1360
Can be deleted
0xde1360
0x7ffe4fa33f08
Dont delete memory that was not allocated by you
0xde1360
Important Node
This is just basics and just code to get you started
Open for others to edit and make necessary changes/optimization
Cannot use STL, they use new operator(if some can implement them please do,it would help to reduce and optimize the code)
In "The C++ Programming Language" book Stroustrup says:
"To deallocate space allocated by new, delete and delete[] must be able to determine the size of the object allocated. This implies that an object allocated using the standard implementation of new will occupy slightly more space than a static object. Typically, one word is used to hold the object’s size.
That means every object allocated by new has its size located somewhere in the heap. Is the location known and if it is how can I access it?
In actual fact, the typical implementation of the memory allocators store some other information too.
There is no standard way to access this information, in fact there is nothing in the standard saying WHAT information is stored either (the size in bytes, number of elements and their size, a pointer to the last element, etc).
Edit:
If you have the base-address of the object and the correct type, I suspect the size of the allocation could be relatively easily found (not necessarily "at no cost at all"). However, there are several problems:
It assumes you have the original pointer.
It assumes the memory is allocated exactly with that runtime library's allocation code.
It assumes the allocator doesn't "round" the allocation address in some way.
To illustrate how this could go wrong, let's say we do this:
size_t get_len_array(int *mem)
{
return allcoated_length(mem);
}
...
void func()
{
int *p = new int[100];
cout << get_len_array(p);
delete [] p;
}
void func2()
{
int buf[100];
cout << get_len_array(buf); // Ouch!
}
That means every object allocated by new has its size located somewhere in the heap. Is the location known and if it is how can I access it?
Not really, that is not needed for all cases. To simplify the reasoning, there are two levels at which the sizes could be needed. At the language level, the compiler needs to know what to destroy. At the allocator level, the allocator needs to know how to release the memory given only a pointer.
At the language level, only the array versions new[] and delete[] need to handle any size. When you allocate with new, you get a pointer with the type of the object, and that type has a given size.
To destroy the object the size is not needed. When you delete, either the pointer is to the correct type, or the static type of the pointer is a base and the destructor is virtual. All other cases are undefined behavior, and thus can be ignored (anything can happen). If it is the correct type, then the size is known. If it is a base with a virtual destructor, the dynamic dispatch will find the final overrider, and at that point the type is known.
There could be different strategies to manage this, the one used in the Itanium C++ ABI (used by multiple compilers in multiple platforms, although not Visual Studio) for example generates up to 3 different destructors per type, one of them being a version that takes care of releasing the memory, so although delete ptr is defined in terms of calling the appropriate destructor and then releasing the memory, in this particular ABI delete ptr call a special destructor that both destroys and releases the memory.
When you use new[] the type of the pointer is the same regardless of the number of elements in the dynamic array, so the type cannot be used to retrieve that information back. A common implementation is allocating an extra integral value and storing the size there, followed by the real objects, then returning a pointer to the first object. delete[] would then move the received pointer one integer back, read the number of elements, call the destructor for all of them and then release the memory (pointer retrieved by the allocator, not the pointer given to the program). This is really only needed if the type has a non-trivial destructor, if the type has a trivial destructor, the implementation does not need to store the size and you can avoid storing that number.
Out of the language level, the real memory allocator (think of malloc) needs to know how much memory was allocated so that the same amount can be released. In some cases that can be done by attaching the metadata to the memory buffer in the same way that new[] stores the size of the array, by acquiring a larger block, storing the metadata there and returning a pointer beyond it. The deallocator would then undo the transformation to get to the metadata.
This is, on the other hand, not always needed. A common implementation for allocators of small size is to allocate pages of memory to form pools from which the small allocations are then obtained. To make this efficient, the allocator considers only a few different sizes, and allocations that don't fit one of the sizes exactly are bumped to the next size. If you request, for example, 65 bytes, the allocator might actually give you 128 bytes (assuming pools of 64 and 128 bytes). Thus given one of the larger blocks managed by the allocator, all pointers that were allocated from it have the same size. The allocator can then find the block from which pointer was allocated and infer the size from it.
Of course, this is all implementation details that are not accessible to the C++ program in a standard portable way, and the exact implementation can differ not just based on the program, but also de execution environment. If you are interested in knowing how the information is really kept in your environment, you might be able to find the information, but I would think twice before trying to use it for anything other than learning purposes.
Your are not deleting a object directly, instead you send a pointer to delete operator.
Reference C++
You use delete by following
it with a pointer to a block of memory originally allocated with new:
int * ps = new int; // allocate memory with new
. . . // use the memory
delete ps; // free memory with delete when done
This removes the memory to which ps points; it doesn’t remove the pointer ps itself.
You can reuse ps, for example, to point to another new allocation
Someone on IRC claimed that, although allocating with new[] and deleting with delete (not delete[]) is UB, on Linux platforms (no further details about the OS) it would be safe.
Is this true? Is it guaranteed? Is it to do with something in POSIX that specifies that dynamically-allocated blocks should not have metadata at the start?
Or is it just completely untrue?
Yes, I know I shouldn't do it. I never would.I am curious about the veracity of this idea; that's it!
By "safe", I mean: "will not cause behaviour other than were the original allocation performed by new, or were the de-allocation performed by delete[]". This means that we might see 1 "element" destruction or n, but no crashing.
Of course it's not true. That person is mixing up several different concerns:
how does the OS handle allocations/deallocations
correct calls to constructors and destructors
UB means UB
On the first point, I'm sure he's correct. It is common to handle both in the same way on that level: it is simply a request for X bytes, or a request to release the allocation starting at address X. It doesn't really matter if it's an array or not.
On the second point, everything falls apart. new[] calls the constructor for each element in the allocated array. delete calls the destructor for the one element at the specified address. And so, if you allocate an array of objects, and free it with delete, only one element will have its destructor invoked. (This is easy to forget because people invariably test this with arrays of ints, in which case this difference is unnoticeable)
And then there's the third point, the catch-all. It's UB, and that means it's UB. The compiler may make optimizations based on the assumption that your code does not exhibit any undefined behavior. If it does, it may break some of these assumptions, and seemingly unrelated code might break.
Even if it happens to be safe on some environment, don't do it. There's no reason to want to do it.
Even if it did return the right memory to the OS, the destructors wouldn't be called properly.
It's definitely not true for all or even most Linuxes, your IRC friend is talking bollocks.
POSIX has nothing to do with C++. In general, this is unsafe. If it works anywhere, it's because of the compiler and library, not the OS.
This question discusses in great details when exactly mixing new[] and delete looks safe (no observable problems) on Visual C++. I suppose that by "on Linux" you actually mean "with gcc" and I've observed very similar results with gcc on ideone.com.
Please note that this requires:
global operator new() and operator new[]() functions to be implemented identically and
the compiler optimizing away the "prepend with number of elements" allocation overhead
and also only works for types with trivial destructors.
Even with these requirements met there's no guarantee it will work on a specific version of a specific compiler. You'll be much better off simply not doing that - relying on undefined behavior is a very bad idea.
It is definitely not safe as you can simply try out with the following code:
#include<iostream>
class test {
public:
test(){ std::cout << "Constructor" << std::endl; }
~test(){ std::cout << "Destructor" << std::endl; }
};
int main() {
test * t = new test[ 10 ];
delete t;
return 1;
}
Have a look at http://ideone.com/b8BiQ . It fails misserably.
It may work when you do not use classes, but only fundamental types, but even that is not guaranteed.
EDIT: Some explanations for those of you who want to know why this crashes:
new and delete mainly serve as wrappers around malloc(), hence calling free() on a newed pointer is most of the time "safe" (remember to call the destructor), but you should not rely on it. For new[] and delete[] however the situation is more complicated.
When an array of classes gets constructed using new[] each default constructor will be called in turn. When you do delete[] each destructor gets called. However each destructor also has to be supplied a this pointer to use inside as a hidden parameter. So before calling the destructor the program has to find the locations of all objects within the reserved memory, to pass these locations as this pointers to the destructor. So all information that is later needed to reconstruct this information needs to be stored somewhere.
Now the easiest way would be to have a global map somewhere around, which stores this information for all new[]ed pointers. In this case if you delete is called instead of delete[] only one of the destructors would be called and the entry would not be removed from a map. However this method is usually not used, because maps are slow and memory management should be as fast as possible.
Hence for the stdlibc++ a different solution is used. Since only a few bytes are needed as additional information, it is the fastest to just over-allocate by these few bytes, store the information at the beginning of the memory and return the pointer to the memory after the bookkeeping. So if you allocate an array of 10 objects of 10 bytes each, the programm will allocate 100+X bytes where X is the size of the data which is needed to reconstruct the this.
So in this case it looks something like this
| Bookkeeping | First Object | Second Object |....
^ ^
| This is what is returned by new[]
|
this is what is returned by malloc()
So in case you pass the pointer you have recieved from new[] to delete[] it will call all destructors, then substract X from the pointer and give that one to free(). However if you call delete instead, it will call a destructor for the first object and then immediately pass that pointer to free(), which means free() has just been passed a pointer which was never malloced, which means the result is UB.
Have a look at http://ideone.com/tIiMw , to see what gets passed to delete and delete[]. As you can see, the pointer returned from new[] is not the pointer which was allocated inside, but 4 is added to it before it is being returned to main(). When calling delete[] correctly the same four is substracted an we get the correct pointer within delete[] however this substraction is missing when calling delete and we get the wrong pointer.
In case of calling new[] on a fundamental type, the compiler immediately knows that it will not have to call any destructors later and it just optimizes the bookkeeping away. However it is definitely allowed to write bookkeeping even for fundamental types. And it is also allowed to add bookkeeping in case you call new.
This bookkeeping in front of the real pointer is actually a very good trick, in case you ever need to write your own memory allocation routines as a replacement of new and delete. There is hardly any limit on what you can store there , so one should never assume that anything returned from new or new[] was actually returned from malloc().
I expect that new[] and delete[] just boil down to malloc() and free() under Linux (gcc, glibc, libstdc++), except that the con(de)structors get called. The same for new and delete except that the con(de)structors get called differently. This means that if his constructors and destructors don't matter, then he can probably get away with it. But why try?
If I do this:
// (1.)
int* p = new int;
//...do something
delete p;
// (2.)
class sample
{
public:
sample(){}
~sample(){}
};
sample* pObj = new sample;
//...do something
delete pObj;
Then how does C++ compiler know that object following delete is built-in data type or a class object?
My other question is that if I new a pointer to an array of int's and then I delete [] then how does compiler know the size of memory block to de-allocate?
The compiler knows the type of the pointed-to object because it knows the type of the pointer:
p is an int*, therefore the pointed-to object will be an int.
pObj is a sample*, therefore the pointed-to object will be a sample.
The compiler does not know if your int* p points to a single int object or to an array (int[N]). That's why you must remember to use delete[] instead of delete for arrays.
The size of the memory block to de-allocate and, most importantly, the number of objects to destroy, are known because new[] stores them somewhere, and delete[] knows where to retrieve these values. This question from C++ FAQ Lite shows two common techniques to implement new[] and delete[].
It knows the difference between them because of the type of the pointer you pass to it: It is undefined behavior to pass a different pointer type than you allocated with (except you may pass a pointer to a base class, if the destructor is virtual, of course).
The size of an array will be stored somewhere. It's like in C where you can malloc a certain amount of memory, and free afterwards - the runtime will have to manage to know the size allocated previously.
For example it can store the count of elements prior to the buffer allocated. The Standard explicitly allows the compiler to pass a different request size to the allocation function (operator new[]) in case of array allocations - this can be used by the compiler to stick the count into, and offset the address returned by the new expression by the size of that counter.
It doesn't!
All what delete does is that it invokes the destructor of the type, which is "no action" in the case of primitive types. Then, it passes the pointer to ::operator delete (or an overloaded version if you like), and thats operator returns back the memory(a memory manager issue). i.e. You can write your own memory manager easily in C++ if you like, the language provides one by default!
The compiler knows the type of the object being deleted and writes different code for you to achieve the right results:
delete p can call the run time delete with the size of an int.
delete pObj can call pObj->~sample() first, then delete with the size of sample
I think with arrays, there is a hidden value for the size of the array, so it could be that the whole array is deleted in one go.
Then how does C++ compiler know that object following delete is built-in data type or a class object?
Because at compile time the compiler tracks the types of each object and plants the appropraite code.
My other question is that if I new a pointer to an array of int's and then I delete [] then how does compiler know the size of memory block to de-allocate?
It does not. This is kept track of by the runtime system.
When you dynamically allocate an array the runtime library associates the size of the object with the object thus when it deletes it it knows (by looking up the associated value) the size.
But I guess you want to know how it does the association?
This depends on the system and is an implementation detail. But a simple stratergy is to allocate an extra 4 bytes store the size in the first four bytes then return a pointer to the 4th byte allocated. When you delete a pointer you know that the size is the 4 bytes before the pointer. Note: I am not saying your system is using this technique but it is one stratergy.
For the first (non-array) part of the question, the answers above indicating that the compiler inserts code to de-allocate the appropriate number of bytes based on the pointer type, don't quite provide a clear answer for me... the delete operator 1) calls a destructor if applicable and then 2) calls the "operator delete()" function... it is operator delete which actually de-allocates. I can see compiler-generated code playing a role in part (1), ie. the destination address of the destructor must be inserted. But in part (2), it is a pre-existing library function handling the de-allocation, so how will it know the size of the data? The global operator delete--which, I believe is used in all cases unless a class-member/overloaded-global version is defined by the programmer--accepts only a void* argument spec'ing the start of the data, so it can't even be passed the data size.
I've read things indicating the compiler-generated code idea, as well as things suggesting that the global operator delete for non-arrays simply uses free(), ie. it knows the data size not by the pointer type, but by looking a few bytes before the data itself, where the size will have been stashed by new/malloc. The latter is the only solution that makes sense to me, but maybe someone can enlighten me differently...
I am writing a template class that takes as an input a pointer and stores it. The pointer is meant to point to an object allocated by another class, and handed to the this containing class.
Now I want to create a destructor for this container. How should I free the memory pointed to by this pointer? I have no way of knowing a priori whether it is an array or a single element.
I'm sort of new to C++, so bear with me. I've always used C, and Java is my OO language of choice, but between wanting to learn C++ and the speed requirements of my project, I've gone with C++.
Would it be a better idea to change the container from a template to a container for an abstract class that can implement its own destructor?
If you don't know whether it was allocated with new or new[], then it is not safe to delete it.
Your code may appear to work. For example, on one platform I work on, the difference only matters when you have an array of objects that have destructors. So, you do this:
// by luck, this works on my preferred platform
// don't do this - just an example of why your code seems to work
int *ints = new int[20];
delete ints;
but then you do this:
// crashes on my platform
std::string *strings = new std::string[10];
delete strings;
You must document how this class expects to be used, and always allocate as expected. You can also pass a flag to the object specifying how it should destroy. Also look at boost's smart pointers, which can handle this distinction for you.
Short answer:
If you use [] with new you want to use [] with delete.
//allocate some memory
myObject* m = new myObject[100];
//later on...destructor...
delete m; //wrong
delete[] m; //correct
That was the bare bones, the other thing you could look at is boost. Also quite difficult to answer considering you are not sure if its an array or single object. You could check this though via a flag telling your app whether to use delete or delete[].
As a general development rule, you should stick to a design where the class which calls new should also call delete
You shouldn't delete it at all. If your class takes an already initialized pointer, it is not safe to delete it. It might not even point to an object on the heap; calling either delete or delete[] could be disastrous.
The allocation and deallocation of memory should happen in the same scope. Which ever code owns and initializes the instance of your class is also presumably responsible for initializing and passing in the pointer, and that is where your delete should be.
Use delete if you allocated with new.
Use delete[] if you allocated with new[].
After these statements, if you still have a problem (maybe you want to delete an object that was created by someone else), then you are breaking the third rule:
Always delete what you created. Corollary, never delete what you did not create.
(Moving my comment into an answer, by request.)
JonH's answer is right (about using array destruction only when you used array construction), so perhaps you should offer templates: one for arrays, one not.
The other answer is to avoid arrays and instead expect a single instance that may or may not be a proper collection that cleans up after itself, such as vector<>.
edit
Stealing blatantly from Roger Pate, I'll add that you could require the use of a smart pointer, which amounts to a single-item collection.
If you have a class that takes a pointer it's going assume ownership of, then the contract for the use of the class needs to include one of a couple things. Either:
the interface needs to indicate how the object the pointer is pointing to was allocated so the new owner can know how to safely deallocate the object. This option has the advantage of keeping things simple (on one level anyway), but it's not flexible - the class can't handle taking ownership of static objects as well as dynamically allocated objects.
or
the interface needs to include a mechanism where a deallocation policy can be specified by whatever is giving the pointer to the class. This can be as simple as providing a mechanism to pass in a functor (or even a plain old function pointer) that will be called to deallocate the object (preferably in the same function/constructor that passes in the pointer itself). This makes the class arguably more complicated to use (but having a default policy of calling delete on the pointer, for example, might make it as easy to use as option 1 for the majority of uses). Now if someone wants to give the class a pointer to a statically allocated object, they can pass in a no-op functor so nothing happens when the class wants to deallocates it, or a functor to a delete[] operation if the object was allocated by new[], etc.
Since pointer in C++ does not tell us how it was allocated, yes, there's no way to decide what deallocation method to use. The solution is to give the choice to the user that hopefully knows how the memory was allocated. Take a look at Boost smart ptr library, especially at shared_ptr constructor with second parameter, for a great example.
A smart pointer like boost shared_pointer already has this covered, could you use it? linky
Put simply, given only a pointer to dynamically allocated memory there is no way of determining how to de-allocate it safely. The pointer could have been allocated in any of the the following ways:
using new
using new []
using malloc
using a user defined function
etc.
In all cases before you can deallocate the memory you have to know how it was allocated.