Someone on IRC claimed that, although allocating with new[] and deleting with delete (not delete[]) is UB, on Linux platforms (no further details about the OS) it would be safe.
Is this true? Is it guaranteed? Is it to do with something in POSIX that specifies that dynamically-allocated blocks should not have metadata at the start?
Or is it just completely untrue?
Yes, I know I shouldn't do it. I never would.I am curious about the veracity of this idea; that's it!
By "safe", I mean: "will not cause behaviour other than were the original allocation performed by new, or were the de-allocation performed by delete[]". This means that we might see 1 "element" destruction or n, but no crashing.
Of course it's not true. That person is mixing up several different concerns:
how does the OS handle allocations/deallocations
correct calls to constructors and destructors
UB means UB
On the first point, I'm sure he's correct. It is common to handle both in the same way on that level: it is simply a request for X bytes, or a request to release the allocation starting at address X. It doesn't really matter if it's an array or not.
On the second point, everything falls apart. new[] calls the constructor for each element in the allocated array. delete calls the destructor for the one element at the specified address. And so, if you allocate an array of objects, and free it with delete, only one element will have its destructor invoked. (This is easy to forget because people invariably test this with arrays of ints, in which case this difference is unnoticeable)
And then there's the third point, the catch-all. It's UB, and that means it's UB. The compiler may make optimizations based on the assumption that your code does not exhibit any undefined behavior. If it does, it may break some of these assumptions, and seemingly unrelated code might break.
Even if it happens to be safe on some environment, don't do it. There's no reason to want to do it.
Even if it did return the right memory to the OS, the destructors wouldn't be called properly.
It's definitely not true for all or even most Linuxes, your IRC friend is talking bollocks.
POSIX has nothing to do with C++. In general, this is unsafe. If it works anywhere, it's because of the compiler and library, not the OS.
This question discusses in great details when exactly mixing new[] and delete looks safe (no observable problems) on Visual C++. I suppose that by "on Linux" you actually mean "with gcc" and I've observed very similar results with gcc on ideone.com.
Please note that this requires:
global operator new() and operator new[]() functions to be implemented identically and
the compiler optimizing away the "prepend with number of elements" allocation overhead
and also only works for types with trivial destructors.
Even with these requirements met there's no guarantee it will work on a specific version of a specific compiler. You'll be much better off simply not doing that - relying on undefined behavior is a very bad idea.
It is definitely not safe as you can simply try out with the following code:
#include<iostream>
class test {
public:
test(){ std::cout << "Constructor" << std::endl; }
~test(){ std::cout << "Destructor" << std::endl; }
};
int main() {
test * t = new test[ 10 ];
delete t;
return 1;
}
Have a look at http://ideone.com/b8BiQ . It fails misserably.
It may work when you do not use classes, but only fundamental types, but even that is not guaranteed.
EDIT: Some explanations for those of you who want to know why this crashes:
new and delete mainly serve as wrappers around malloc(), hence calling free() on a newed pointer is most of the time "safe" (remember to call the destructor), but you should not rely on it. For new[] and delete[] however the situation is more complicated.
When an array of classes gets constructed using new[] each default constructor will be called in turn. When you do delete[] each destructor gets called. However each destructor also has to be supplied a this pointer to use inside as a hidden parameter. So before calling the destructor the program has to find the locations of all objects within the reserved memory, to pass these locations as this pointers to the destructor. So all information that is later needed to reconstruct this information needs to be stored somewhere.
Now the easiest way would be to have a global map somewhere around, which stores this information for all new[]ed pointers. In this case if you delete is called instead of delete[] only one of the destructors would be called and the entry would not be removed from a map. However this method is usually not used, because maps are slow and memory management should be as fast as possible.
Hence for the stdlibc++ a different solution is used. Since only a few bytes are needed as additional information, it is the fastest to just over-allocate by these few bytes, store the information at the beginning of the memory and return the pointer to the memory after the bookkeeping. So if you allocate an array of 10 objects of 10 bytes each, the programm will allocate 100+X bytes where X is the size of the data which is needed to reconstruct the this.
So in this case it looks something like this
| Bookkeeping | First Object | Second Object |....
^ ^
| This is what is returned by new[]
|
this is what is returned by malloc()
So in case you pass the pointer you have recieved from new[] to delete[] it will call all destructors, then substract X from the pointer and give that one to free(). However if you call delete instead, it will call a destructor for the first object and then immediately pass that pointer to free(), which means free() has just been passed a pointer which was never malloced, which means the result is UB.
Have a look at http://ideone.com/tIiMw , to see what gets passed to delete and delete[]. As you can see, the pointer returned from new[] is not the pointer which was allocated inside, but 4 is added to it before it is being returned to main(). When calling delete[] correctly the same four is substracted an we get the correct pointer within delete[] however this substraction is missing when calling delete and we get the wrong pointer.
In case of calling new[] on a fundamental type, the compiler immediately knows that it will not have to call any destructors later and it just optimizes the bookkeeping away. However it is definitely allowed to write bookkeeping even for fundamental types. And it is also allowed to add bookkeeping in case you call new.
This bookkeeping in front of the real pointer is actually a very good trick, in case you ever need to write your own memory allocation routines as a replacement of new and delete. There is hardly any limit on what you can store there , so one should never assume that anything returned from new or new[] was actually returned from malloc().
I expect that new[] and delete[] just boil down to malloc() and free() under Linux (gcc, glibc, libstdc++), except that the con(de)structors get called. The same for new and delete except that the con(de)structors get called differently. This means that if his constructors and destructors don't matter, then he can probably get away with it. But why try?
Related
Every time somebody asks a question about delete[] on here, there is always a pretty general "that's how C++ does it, use delete[]" kind of response. Coming from a vanilla C background what I don't understand is why there needs to be a different invocation at all.
With malloc()/free() your options are to get a pointer to a contiguous block of memory and to free a block of contiguous memory. Something in implementation land comes along and knows what size the block you allocated was based on the base address, for when you have to free it.
There is no function free_array(). I've seen some crazy theories on other questions tangentially related to this, such as calling delete ptr will only free the top of the array, not the whole array. Or the more correct, it is not defined by the implementation. And sure... if this was the first version of C++ and you made a weird design choice that makes sense. But why with $PRESENT_YEAR's standard of C++ has it not been overloaded???
It seems to be the only extra bit that C++ adds is going through the array and calling destructors, and I think maybe this is the crux of it, and it literally is using a separate function to save us a single runtime length lookup, or nullptr at end of the list in exchange for torturing every new C++ programmer or programmer who had a fuzzy day and forgot that there is a different reserve word.
Can someone please clarify once and for all if there is a reason besides "that's what the standard says and nobody questions it"?
Objects in C++ often have destructors that need to run at the end of their lifetime. delete[] makes sure the destructors of each element of the array are called. But doing this has unspecified overhead, while delete does not. This is why there are two forms of delete expressions. One for arrays, which pays the overhead and one for single objects which does not.
In order to only have one version, an implementation would need a mechanism for tracking extra information about every pointer. But one of the founding principles of C++ is that the user shouldn't be forced to pay a cost that they don't absolutely have to.
Always delete what you new and always delete[] what you new[]. But in modern C++, new and new[] are generally not used anymore. Use std::make_unique, std::make_shared, std::vector or other more expressive and safer alternatives.
Basically, malloc and free allocate memory, and new and delete create and destroy objects. So you have to know what the objects are.
To elaborate on the unspecified overhead François Andrieux's answer mentions, you can see my answer on this question in which I examined what does a specific implementation do (Visual C++ 2013, 32-bit). Other implementations may or may not do a similar thing.
In case the new[] was used with an array of objects with a non-trivial destructor, what it did was allocating 4 bytes more, and returning the pointer shifted by 4 bytes ahead, so when delete[] wants to know how many objects are there, it takes the pointer, shifts it 4 bytes prior, and takes the number at that address and treats it as the number of objects stored there. It then calls a destructor on each object (the size of the object is known from the type of the pointer passed). Then, in order to release the exact address, it passes the address that was 4 bytes prior to the passed address.
On this implementation, passing an array allocated with new[] to a regular delete results in calling a single destructor, of the first element, followed by passing the wrong address to the deallocation function, corrupting the heap. Don't do it!
Something not mentioned in the other (all good) answers is that the root cause of this is that arrays - inherited from C - have never been a "first-class" thing in C++.
They have primitive C semantics and do not have C++ semantics, and therefore C++ compiler and runtime support, which would let you or the compiler runtime systems do useful things with pointers to them.
In fact, they're so unsupported by C++ that a pointer to an array of things looks just like a pointer to a single thing. That, in particular, would not happen if arrays were proper parts of the language - even as part of a library, like string or vector.
This wart on the C++ language happened because of this heritage from C. And it remains part of the language - even though we now have std::array for fixed-length arrays and (have always had) std::vector for variable-length arrays - largely for purposes of compatibility: Being able to call out from C++ to operating system APIs and to libraries written in other languages using C-language interop.
And ... because there are truckloads of books and websites and classrooms out there teaching arrays very early in their C++ pedagogy, because of a) being able to write useful/interesting examples early on that do in fact call OS APIs, and of course because of the awesome power of b) "that's the way we've always done it".
Generally, C++ compilers and their associated runtimes build on top of the platform's C runtime. In particular in this case the C memory manager.
The C memory manager allows you to free a block of memory without knowing its size, but there is no standard way to get the size of the block from the runtime and there is no guarantee that the block that was actually allocated is exactly the size you requested. It may well be larger.
Thus the block size stored by the C memory manager can't usefully be used to enable higher-level functionality. If higher-level functionality needs information on the size of the allocation then it must store it itself. (And C++ delete[] does need this for types with destructors, to run them for every element.)
C++ also has an attitude of "you only pay for what you use", storing an extra length field for every allocation (separate from the underlying allocator's bookkeeping) would not fit well with this attitude.
Since the normal way to represent an array of unknown (at compile time) size in C and C++ is with a pointer to its first element, there is no way the compiler can distinguish between a single object allocation and an array allocation based on the type system. So it leaves it up to the programmer to distinguish.
The cover story is that delete is required because of C++'s relationship with C.
The new operator can make a dynamically allocated object of almost any object type.
But, due to the C heritage, a pointer to an object type is ambiguous between two abstractions:
being the location of a single object, and
being the base of a dynamic array.
The delete versus delete[] situation just follows from that.
However, that's does not ring true, because, in spite of the above observations being true, a single delete operator could be used. It does not logically follow that two operators are required.
Here is informal proof. The new T operator invocation (single object case) could implicitly behave as if it were new T[1]. So that is to say, every new could always allocate an array. When no array syntax is mentioned, it could be implicit that an array of [1] will be allocated. Then, there would just have to exist a single delete which behaves like today's delete[].
Why isn't that design followed?
I think it boils down to the usual: it's a goat that was sacrificed to the gods of efficiency. When you allocate an array with new [], extra storage is allocated for meta-data to keep track of the number of elements, so that delete [] can know how many elements need to be iterated for destruction. When you allocate a single object with new, no such meta-data is required. The object can be constructed directly in the memory which comes from the underlying allocator without any extra header.
It's a part of "don't pay for what you don't use" in terms of run-time costs. If you're allocating single objects, you don't have to "pay" for any representational overhead in those objects to deal with the possibility that any dynamic object referenced by pointer might be an array. However, you are burdened with the responsibility of encoding that information in the way you allocate the object with the array new and subsequently delete it.
An example might help. When you allocate a C-style array of objects, those objects may have their own destructor that needs to be called. The delete operator does not do that. It works on container objects, but not C-style arrays. You need delete[] for them.
Here is an example:
#include <iostream>
#include <stdlib.h>
#include <string>
using std::cerr;
using std::cout;
using std::endl;
class silly_string : private std::string {
public:
silly_string(const char* const s) :
std::string(s) {}
~silly_string() {
cout.flush();
cerr << "Deleting \"" << *this << "\"."
<< endl;
// The destructor of the base class is now implicitly invoked.
}
friend std::ostream& operator<< ( std::ostream&, const silly_string& );
};
std::ostream& operator<< ( std::ostream& out, const silly_string& s )
{
return out << static_cast<const std::string>(s);
}
int main()
{
constexpr size_t nwords = 2;
silly_string *const words = new silly_string[nwords]{
"hello,",
"world!" };
cout << words[0] << ' '
<< words[1] << '\n';
delete[] words;
return EXIT_SUCCESS;
}
That test program explicitly instruments the destructor calls. It’s obviously a contrived example. For one thing, a program does not need to free memory immediately before it terminates and releases all its resources. But it does demonstrate what happens and in what order.
Some compilers, such as clang++, are smart enough to warn you if you leave out the [] in delete[] words;, but if you force it to compile the buggy code anyway, you get heap corruption.
Delete is an operator that destroys array and non-array(pointer) objects which are generated by new expression.
It can be used by either using the Delete operator or Delete [ ] operator
A new operator is used for dynamic memory allocation which puts variables on heap memory.
This means the Delete operator deallocates memory from the heap.
Pointer to object is not destroyed, value or memory block pointed by the pointer is destroyed.
The delete operator has a void return type that does not return a value.
Ignoring programming style and design, is it "safe" to call delete on a variable allocated on the stack?
For example:
int nAmount;
delete &nAmount;
or
class sample
{
public:
sample();
~sample() { delete &nAmount;}
int nAmount;
}
No, it is not safe to call delete on a stack-allocated variable. You should only call delete on things created by new.
For each malloc or calloc, there should be exactly one free.
For each new there should be exactly one delete.
For each new[] there should be exactly one delete[].
For each stack allocation, there should be no explicit freeing or deletion. The destructor is called automatically, where applicable.
In general, you cannot mix and match any of these, e.g. no free-ing or delete[]-ing a new object. Doing so results in undefined behavior.
Well, let's try it:
jeremy#jeremy-desktop:~$ echo 'main() { int a; delete &a; }' > test.cpp
jeremy#jeremy-desktop:~$ g++ -o test test.cpp
jeremy#jeremy-desktop:~$ ./test
Segmentation fault
So apparently it is not safe at all.
Keep in mind that when you allocate a block of memory using new (or malloc for that matter), the actual block of memory allocated will be larger than what you asked for.
The memory block will also contain some bookkeeping information so that when you free the block, it can easily be put back into the free pool and possibly be coalesced with adjacent free blocks.
When you try to free any memory that you didn't receive from new, that bookkeeping information wont be there but the system will act like it is and the results are going to be unpredictable (usually bad).
Yes, it is undefined behavior: passing to delete anything that did not come from new is UB:
C++ standard, section 3.7.3.2.3:
The value of the first argument supplied to one of thea deallocation functions provided in the standard library may be a null pointer value; if so, and if the deallocation function is one supplied in the standard library, the call to the deallocation function has no effect. Otherwise, the value supplied to operator delete(void*) in the standard library shall be one of the values returned by a previous invocation of either operator new(std::size_t) or operator new(std::size_t, const std::nothrow_t&) in the standard library.
The consequences of undefined behavior are, well, undefined. "Nothing happens" is as valid a consequence as anything else. However, it's usually "nothing happens right away": deallocating an invalid memory block may have severe consequences in subsequent calls to the allocator.
After playing a bit with g++ 4.4 in windows, I got very interesting results:
calling delete on a stack variable doesn't seem to do anything. No errors throw, but I can access the variable without problems after deletion.
Having a class with a method with delete this successfully deletes the object if it is allocated in the heap, but not if it is allocated in the stack (if it is in the stack, nothing happens).
Nobody can know what happens. This invokes undefined behavior, so literally anything can happen. Don't do this.
No,
Memory allocated using new should be deleted using delete operator
and that allocated using malloc should be deleted using free.
And no need to deallocate the variable which are allocated on stack.
An angel loses its wings... You can only call delete on a pointer allocated with new, otherwise you get undefined behavior.
here the memory is allocated using stack so no need to delete it exernally but if you have allcoted dynamically
like
int *a=new int()
then you have to do delete a and not delete &a(a itself is a pointer), because the memory is allocated from free store.
You already answered the question yourself. delete must only be used for pointers optained through new. Doing anything else is plain and simple undefined behaviour.
Therefore there is really no saying what happens, anything from the code working fine through crashing to erasing your harddrive is a valid outcome of doing this. So please never do this.
It's UB because you must not call delete on an item that has not been dynamically allocated with new. It's that simple.
Motivation: I have two objects, A and B. I know that A has to be instantiated before B, maybe because B needs information calculated by A. Yet, I want to destruct A before B. Maybe I am writing an integration test, and I want server A to shut-down first. How do I accomplish that?
A a{};
B b{a.port()};
// delete A, how?
Solution: Don't allocate A on the stack. Instead, use std::make_unique and keep a stack-allocated smart pointer to a heap-allocated instance of A. That way is the least messy option, IMO.
auto a = std::make_unique<A>();
B b{a->port()};
// ...
a.reset()
Alternatively, I considered moving the destruction logic out of A's destructor and calling that method explicitly myself. The destructor would then call it only if it has not been called previously.
I know free() won't call the destructor, but what else will this cause besides that the member variable won't be destructed properly?
Also, what if we delete a pointer that is allocated by malloc?
It is implementation defined whether new uses malloc under the hood. Mixing new with free and malloc with delete could cause a catastrophic failure at runtime if the code was ported to a new machine, a new compiler, or even a new version of the same compiler.
I know free() won't call the destructor
And that is reason enough not to do it.
In addition, there's no requirement for a C++ implementation to even use the same memory areas for malloc and new so it may be that you're trying to free memory from a totally different arena, something which will almost certainly be fatal.
Many points:
It's undefined behaviour, and hence inherently risky and subject to change or breakage at any time and for no reason at all.
(As you know) delete calls the destructor and free doesn't... you may have some POD type and not care, but it's easy for someone else to add say a string to that type without realising there are weird limitations on its content.
If you malloc and forget to use placement new to construct an object in it, then invoke a member function as if the object existed (including delete which calls the destructor), the member function may attempt operations using pointers with garbage values
new and malloc may get memory from different heaps.
Even if new calls malloc to get its memory, there may not be a 1:1 correspondence between the new/delete and underlying malloc/free behaviour.
e.g. new may have extra logic such as small-object optimisations that have proven beneficial to typical C++ programs but harmful to typical C programs.
Someone may overload new, or link in a debug version of malloc/realloc/free, either of which could break if you're not using the functions properly.
Tools like ValGrind, Purify and Insure won't be able to differentiate between the deliberately dubious and the accidentally.
In the case of arrays, delete[] invokes all the destructors and free() won't, but also the heap memory typically has a counter of the array size (for 32-bit VC++2005 Release builds for example, the array size is in the 4 bytes immediately before the pointer value visibly returned by new[]. This extra value may or may not be be there for POD types (not for VC++2005), but if it is free() certainly won't expect it. Not all heap implementations allow you to free a pointer that's been shifted from the value returned by malloc().
An important difference is that new and delete also call the constructor and destructor of the object. Thus, you may get unexpected behavior. That is the most important thing i think.
Because it might not be the same allocator, which could lead to weird, unpredictable behaviour. Plus, you shouldn't be using malloc/free at all, and avoid using new/delete where it's not necessary.
It totally depends on the implementation -- it's possible to write an implementation where this actually works fine. But there's no guarantee that the pool of memory new allocates from is the same pool that free() wants to return the memory to. Imagine that both malloc() and new use a few bytes of extra memory at the beginning of each allocated block to specify how large the block is. Further, imagine that malloc() and new use different formats for this info -- for example, malloc() uses the number of bytes, but new uses the number of 4-byte long words (just an example). Now, if you allocate with malloc() and free with delete, the info delete expects won't be valid, and you'll end up with a corrupted heap.
I am working on modifying a relatively large C++ program, where unfortunately it is not always clear whether someone before me used C or C++ syntax (this is in the electrical engineering department at a university, and we EEs are always tempted to use C for everything, and unfortunately in this case, people can actually get away with it).
However, if someone creates an object:
Packet* thePacket = new Packet();
Does it matter whether it is destroyed with delete thePacket; or free(thePacket); ?
I realize that delete calls the destructor while free() does not, but Packet does not have a destructor. I am having a terrible time stuck in a memory management swamp here and I'm thinking this may be one of the many problems.
Yes it does matter.
For memory obtained using new you must use delete.
For memory obtained using malloc you must use free.
new and malloc may use different data structures internally to keep track of what and where it has allocated memory. So in order to free memory, you have to call that corresponding function that knows about those data structures. It is however generally a bad idea to mix these two types of memory allocation in a piece of code.
If you call free(), the destructor doesn't get called.
Also, there's no guarantee that new and free operate on the same heap.
You can also override new and delete to operate specially on a particular class. If you do so, but call free() instead of the custom delete, then you miss whatever special behavior you had written into delete. (But you probably wouldn't be asking this question if you had done that, because you'd know what behaviors you were missing..)
Packet has a destructor, even if you haven't explicitly declared one. It has a default destructor. The default destructor probably doesn't actually do much, but you can't count on that being the case. It's up to the compiler what it does.
new and malloc also may have wildly different implementations. For example, delete is always called in a context where it has perfect information about the size of the data structure it's deleting at compile time. free does not have this luxury. It's possible that the allocator that new is using may not store the bytes at the beginning of the memory area stating how many bytes it occupies. This would lead free to do entirely the wrong thing and crash your program when freeing something allocated with new.
Personally, if getting people to do the right thing or fixing the code yourself is completely impossible, I would declare my own global operator new that called malloc so then free would definitely not crash, even though it would still not call the destructor and be generally really ugly.
In short, it is as bad as undefined behavior.
This is quiet self explanatory.
C Standard ($7.20.3.2/2) - "The free
function causes the space pointed to
by ptr to be deallocated, that is,
made available for further allocation.
If ptr is a null pointer, no action
occurs. Otherwise, if the argument
does not match a pointer earlier
returned by the calloc, malloc, or
realloc function, or if the space has
been deallocated by a call to free or
realloc, the behavior is undefined."
You are absolutely right, it is NOT correct. As you said yourself, free won't call the destructor. Even if Packet doesn't have an explicit destructor, it's using an inherited one.
Using free on an object created with new is like destroying only what a shallow-copy would reach. Deep-destroying NEEDS the destructor function.
Also, I'm not sure objects created with new() are on the same memory map as malloc()'d memory. They are not guaranteed to be, I think.
if someone creates an object:
Packet* thePacket = new Packet();
Does it matter whether is is destroyed with delete thePacket; or free(thePacket); ?
Yes it does matter. free (thePacket) would invoke Undefined Behaviour but delete thePacket would not and we all know Undefined Behaviour may have disastrous consequences.
I have been working on some legacy C++ code that uses variable length structures (TAPI), where the structure size will depend on variable length strings. The structures are allocated by casting array new thus:
STRUCT* pStruct = (STRUCT*)new BYTE[sizeof(STRUCT) + nPaddingSize];
Later on however the memory is freed using a delete call:
delete pStruct;
Will this mix of array new[] and non-array delete cause a memory leak or would it depend on the compiler? Would I be better off changing this code to use malloc and free instead?
Technically I believe it could cause a problem with mismatched allocators, though in practice I don't know of any compiler that would not do the right thing with this example.
More importantly if STRUCT where to have (or ever be given) a destructor then it would invoke the destructor without having invoked the corresponding constructor.
Of course, if you know where pStruct came from why not just cast it on delete to match the allocation:
delete [] (BYTE*) pStruct;
I personally think you'd be better off using std::vector to manage your memory, so you don't need the delete.
std::vector<BYTE> backing(sizeof(STRUCT) + nPaddingSize);
STRUCT* pStruct = (STRUCT*)(&backing[0]);
Once backing leaves scope, your pStruct is no longer valid.
Or, you can use:
boost::scoped_array<BYTE> backing(new BYTE[sizeof(STRUCT) + nPaddingSize]);
STRUCT* pStruct = (STRUCT*)backing.get();
Or boost::shared_array if you need to move ownership around.
Yes it will cause a memory leak.
See this except from C++ Gotchas: http://www.informit.com/articles/article.aspx?p=30642 for why.
Raymond Chen has an explanation of how vector new and delete differ from the scalar versions under the covers for the Microsoft compiler... Here:
http://blogs.msdn.com/oldnewthing/archive/2004/02/03/66660.aspx
IMHO you should fix the delete to:
delete [] pStruct;
rather than switching to malloc/free, if only because it's a simpler change to make without making mistakes ;)
And, of course, the simpler to make change that I show above is wrong due to the casting in the original allocation, it should be
delete [] reinterpret_cast<BYTE *>(pStruct);
so, I guess it's probably as easy to switch to malloc/free after all ;)
The behaviour of the code is undefined. You may be lucky (or not) and it may work with your compiler, but really that's not correct code. There's two problems with it:
The delete should be an array delete [].
The delete should be called on a pointer to the same type as the type allocated.
So to be entirely correct, you want to be doing something like this:
delete [] (BYTE*)(pStruct);
The C++ standard clearly states:
delete-expression:
::opt delete cast-expression
::opt delete [ ] cast-expression
The first alternative is for non-array objects, and the second is for arrays. The operand shall have a pointer type, or a class type having a single conversion function (12.3.2) to a pointer type. The result has type void.
In the first alternative (delete object), the value of the operand of delete shall be a pointer to a non-array object [...] If not, the behavior is undefined.
The value of the operand in delete pStruct is a pointer to an array of char, independent of its static type (STRUCT*). Therefore, any discussion of memory leaks is quite pointless, because the code is ill-formed, and a C++ compiler is not required to produce a sensible executable in this case.
It could leak memory, it could not, or it could do anything up to crashing your system. Indeed, a C++ implementation with which I tested your code aborts the program execution at the point of the delete expression.
As highlighted in other posts:
1) Calls to new/delete allocate memory and may call constructors/destructors (C++ '03 5.3.4/5.3.5)
2) Mixing array/non-array versions of new and delete is undefined behaviour. (C++ '03 5.3.5/4)
Looking at the source it appears that someone did a search and replace for malloc and free and the above is the result. C++ does have a direct replacement for these functions, and that is to call the allocation functions for new and delete directly:
STRUCT* pStruct = (STRUCT*)::operator new (sizeof(STRUCT) + nPaddingSize);
// ...
pStruct->~STRUCT (); // Call STRUCT destructor
::operator delete (pStruct);
If the constructor for STRUCT should be called, then you could consider allocating the memory and then use placement new:
BYTE * pByteData = new BYTE[sizeof(STRUCT) + nPaddingSize];
STRUCT * pStruct = new (pByteData) STRUCT ();
// ...
pStruct->~STRUCT ();
delete[] pByteData;
#eric - Thanks for the comments. You keep saying something though, that drives me nuts:
Those run-time libraries handle the
memory management calls to the OS in a
OS independent consistent syntax and
those run-time libraries are
responsible for making malloc and new
work consistently between OSes such as
Linux, Windows, Solaris, AIX, etc....
This is not true. The compiler writer provides the implementation of the std libraries, for instance, and they are absolutely free to implement those in an OS dependent way. They're free, for instance, to make one giant call to malloc, and then manage memory within the block however they wish.
Compatibility is provided because the API of std, etc. is the same - not because the run-time libraries all turn around and call the exact same OS calls.
The various possible uses of the keywords new and delete seem to create a fair amount of confusion. There are always two stages to constructing dynamic objects in C++: the allocation of the raw memory and the construction of the new object in the allocated memory area. On the other side of the object lifetime there is the destruction of the object and the deallocation of the memory location where the object resided.
Frequently these two steps are performed by a single C++ statement.
MyObject* ObjPtr = new MyObject;
//...
delete MyObject;
Instead of the above you can use the C++ raw memory allocation functions operator new and operator delete and explicit construction (via placement new) and destruction to perform the equivalent steps.
void* MemoryPtr = ::operator new( sizeof(MyObject) );
MyObject* ObjPtr = new (MemoryPtr) MyObject;
// ...
ObjPtr->~MyObject();
::operator delete( MemoryPtr );
Notice how there is no casting involved, and only one type of object is constructed in the allocated memory area. Using something like new char[N] as a way to allocate raw memory is technically incorrect as, logically, char objects are created in the newly allocated memory. I don't know of any situation where it doesn't 'just work' but it blurs the distinction between raw memory allocation and object creation so I advise against it.
In this particular case, there is no gain to be had by separating out the two steps of delete but you do need to manually control the initial allocation. The above code works in the 'everything working' scenario but it will leak the raw memory in the case where the constructor of MyObject throws an exception. While this could be caught and solved with an exception handler at the point of allocation it is probably neater to provide a custom operator new so that the complete construction can be handled by a placement new expression.
class MyObject
{
void* operator new( std::size_t rqsize, std::size_t padding )
{
return ::operator new( rqsize + padding );
}
// Usual (non-placement) delete
// We need to define this as our placement operator delete
// function happens to have one of the allowed signatures for
// a non-placement operator delete
void operator delete( void* p )
{
::operator delete( p );
}
// Placement operator delete
void operator delete( void* p, std::size_t )
{
::operator delete( p );
}
};
There are a couple of subtle points here. We define a class placement new so that we can allocate enough memory for the class instance plus some user specifiable padding. Because we do this we need to provide a matching placement delete so that if the memory allocation succeeds but the construction fails, the allocated memory is automatically deallocated. Unfortunately, the signature for our placement delete matches one of the two allowed signatures for non-placement delete so we need to provide the other form of non-placement delete so that our real placement delete is treated as a placement delete. (We could have got around this by adding an extra dummy parameter to both our placement new and placement delete, but this would have required extra work at all the calling sites.)
// Called in one step like so:
MyObject* ObjectPtr = new (padding) MyObject;
Using a single new expression we are now guaranteed that memory won't leak if any part of the new expression throws.
At the other end of the object lifetime, because we defined operator delete (even if we hadn't, the memory for the object originally came from global operator new in any case), the following is the correct way to destroy the dynamically created object.
delete ObjectPtr;
Summary!
Look no casts! operator new and operator delete deal with raw memory, placement new can construct objects in raw memory. An explicit cast from a void* to an object pointer is usually a sign of something logically wrong, even if it does 'just work'.
We've completely ignored new[] and delete[]. These variable size objects will not work in arrays in any case.
Placement new allows a new expression not to leak, the new expression still evaluates to a pointer to an object that needs destroying and memory that needs deallocating. Use of some type of smart pointer may help prevent other types of leak. On the plus side we've let a plain delete be the correct way to do this so most standard smart pointers will work.
If you really must do this sort of thing, you should probably call operator new directly:
STRUCT* pStruct = operator new(sizeof(STRUCT) + nPaddingSize);
I believe calling it this way avoids calling constructors/destructors.
I am currently unable to vote, but slicedlime's answer is preferable to Rob Walker's answer, since the problem has nothing to do with allocators or whether or not the STRUCT has a destructor.
Also note that the example code does not necessarily result in a memory leak - it's undefined behavior. Pretty much anything could happen (from nothing bad to a crash far, far away).
The example code results in undefined behavior, plain and simple. slicedlime's answer is direct and to the point (with the caveat that the word 'vector' should be changed to 'array' since vectors are an STL thing).
This kind of stuff is covered pretty well in the C++ FAQ (Sections 16.12, 16.13, and 16.14):
http://www.parashift.com/c++-faq-lite/freestore-mgmt.html#faq-16.12
It's an array delete ([]) you're referring to, not a vector delete.
A vector is std::vector, and it takes care of deletion of its elements.
You'd could cast back to a BYTE * and the delete:
delete[] (BYTE*)pStruct;
Yes that may, since your allocating with new[] but deallocating with delelte, yes malloc/free is safer here, but in c++ you should not use them since they won't handle (de)constructors.
Also your code will call the deconstructor, but not the constructor. For some structs this may cause a memory leak (if the constructor allocated further memory, eg for a string)
Better would be to do it correctly, as this will also correctly call any constructors and deconstructors
STRUCT* pStruct = new STRUCT;
...
delete pStruct;
It's always best to keep acquisition/release of any resource as balanced as possible.
Although leaking or not is hard to say in this case. It depends on the compiler's implementation of the vector (de)allocation.
BYTE * pBytes = new BYTE [sizeof(STRUCT) + nPaddingSize];
STRUCT* pStruct = reinterpret_cast< STRUCT* > ( pBytes ) ;
// do stuff with pStruct
delete [] pBytes ;
Len: the problem with that is that pStruct is a STRUCT*, but the memory allocated is actually a BYTE[] of some unknown size. So delete[] pStruct will not de-allocate all of the allocated memory.
You're sort of mixing C and C++ ways of doing things. Why allocate more than the size of a STRUCT? Why not just "new STRUCT"? If you must do this then it might be clearer to use malloc and free in this case, since then you or other programmers might be a little less likely to make assumptions about the types and sizes of the allocated objects.
#Matt Cruikshank
You should pay attention and read what I wrote again because I never suggested not calling delete[] and just let the OS clean up. And you're wrong about the C++ run-time libraries managing the heap. If that were the case then C++ would not be portable as is today and a crashing application would never get cleaned up by the OS. (acknowledging there are OS specific run-times that make C/C++ appear non-portable). I challenge you to find stdlib.h in the Linux sources from kernel.org. The new keyword in C++ actually is talking to the same memory management routines as malloc.
The C++ run-time libraries make OS system calls and it's the OS that manages the heaps. You are partly correct in that the run-time libraries indicate when to release the memory however, they don't actually walk any heap tables directly. In other words, the runtime you link against does not add code to your application to walk heaps to allocate or deallocate. This is the case in Windows, Linux, Solaris, AIX, etc... It's also the reason you won't fine malloc in any Linux's kernel source nor will you find stdlib.h in Linux source. Understand these modern operating system have virtual memory managers that complicates things a bit further.
Ever wonder why you can make a call to malloc for 2G of RAM on a 1G box and still get back a valid memory pointer?
Memory management on x86 processors is managed within Kernel space using three tables. PAM (Page Allocation Table), PD (Page Directories) and PT (Page Tables). This is at the hardware level I'm speaking of. One of the things the OS memory manager does, not your C++ application, is to find out how much physical memory is installed on the box during boot with help of BIOS calls. The OS also handles exceptions such as when you try to access memory your application does not have rights too. (GPF General Protection Fault).
It may be that we are saying the same thing Matt, but I think you may be confusing the under hood functionality a bit. I use to maintain a C/C++ compiler for a living...
#ericmayo - cripes. Well, experimenting with VS2005, I can't get an honest leak out of scalar delete on memory that was made by vector new. I guess the compiler behavior is "undefined" here, is about the best defense I can muster.
You've got to admit though, it's a really lousy practice to do what the original poster said.
If that were the case then C++ would
not be portable as is today and a
crashing application would never get
cleaned up by the OS.
This logic doesn't really hold, though. My assertion is that a compiler's runtime can manage the memory within the memory blocks that the OS returns to it. This is how most virtual machines work, so your argument against portability in this case don't make much sense.
#Matt Cruikshank
"Well, experimenting with VS2005, I can't get an honest leak out of scalar delete on memory that was made by vector new. I guess the compiler behavior is "undefined" here, is about the best defense I can muster."
I disagree that it's a compiler behavior or even a compiler issue. The 'new' keyword gets compiled and linked, as you pointed out, to run-time libraries. Those run-time libraries handle the memory management calls to the OS in a OS independent consistent syntax and those run-time libraries are responsible for making malloc and new work consistently between OSes such as Linux, Windows, Solaris, AIX, etc.... This is the reason I mentioned the portability argument; an attempt to prove to you that the run-time does not actually manage memory either.
The OS manages memory.
The run-time libs interface to the OS.. On Windows, this is the virtual memory manager DLLs. This is why stdlib.h is implemented within the GLIB-C libraries and not the Linux kernel source; if GLIB-C is used on other OSes, it's implementation of malloc changes to make the correct OS calls. In VS, Borland, etc.. you will never find any libraries that ship with their compilers that actually manage memory either. You will, however, find OS specific definitions for malloc.
Since we have the source to Linux, you can go look at how malloc is implemented there. You will see that malloc is actually implemented in the GCC compiler which, in turn, basically makes two Linux system calls into the kernel to allocate memory. Never, malloc itself, actually managing memory!
And don't take it from me. Read the source code to Linux OS or you can see what K&R say about it... Here is a PDF link to the K&R on C.
http://www.oberon2005.ru/paper/kr_c.pdf
See near end of Page 149:
"Calls to malloc and free may occur in any order; malloc calls
upon the operating system to obtain more memory as necessary. These routines illustrate some of the considerations involved in writing machine-dependent code in a relatively machineindependent way, and also show a real-life application of structures, unions and typedef."
"You've got to admit though, it's a really lousy practice to do what the original poster said."
Oh, I don't disagree there. My point was that the original poster's code was not conducive of a memory leak. That's all I was saying. I didn't chime in on the best practice side of things. Since the code is calling delete, the memory is getting free up.
I agree, in your defense, if the original poster's code never exited or never made it to the delete call, that the code could have a memory leak but since he states that later on he sees the delete getting called. "Later on however the memory is freed using a delete call:"
Moreover, my reason for responding as I did was due to the OP's comment "variable length structures (TAPI), where the structure size will depend on variable length strings"
That comment sounded like he was questioning the dynamic nature of the allocations against the cast being made and was consequentially wondering if that would cause a memory leak. I was reading between the lines if you will ;).
In addition to the excellent answers above, I would also like to add:
If your code runs on linux or if you can compile it on linux then I would suggest running it through Valgrind. It is an excellent tool, among the myriad of useful warnings it produces it also will tell you when you allocate memory as an array and then free it as a non-array ( and vice-versa ).
Use operator new and delete:
struct STRUCT
{
void *operator new (size_t)
{
return new char [sizeof(STRUCT) + nPaddingSize];
}
void operator delete (void *memory)
{
delete [] reinterpret_cast <char *> (memory);
}
};
void main()
{
STRUCT *s = new STRUCT;
delete s;
}
I think the is no memory leak.
STRUCT* pStruct = (STRUCT*)new BYTE [sizeof(STRUCT) + nPaddingSize];
This gets translated into a memory allocation call within the operating system upon which a pointer to that memory is returned. At the time memory is allocated, the size of sizeof(STRUCT) and the size of nPaddingSize would be known in order to fulfill any memory allocation requests against the underlying operating system.
So the memory that is allocated is "recorded" in the operating system's global memory allocation tables. Memory tables are indexed by their pointers. So in the corresponding call to delete, all memory that was originally allocated is free. (memory fragmentation a popular subject in this realm as well).
You see, the C/C++ compiler is not managing memory, the underlying operating system is.
I agree there are cleaner methods but the OP did say this was legacy code.
In short, I don't see a memory leak as the accepted answer believes there to be one.
Rob Walker reply is good.
Just small addition, if you don't have any constructor or/and distructors, so you basically need allocate and free a chunk of raw memory, consider using free/malloc pair.
ericmayo.myopenid.com is so wrong, that someone with enough reputation should downvote him.
The C or C++ runtime libraries are managing the heap which is given to it in blocks by the Operating System, somewhat like you indicate, Eric. But it is the responsibility of the developer to indicate to the compiler which runtime calls should be made to free memory, and possibly destruct the objects that are there. Vector delete (aka delete[]) is necessary in this case, in order for the C++ runtime to leave the heap in a valid state. The fact that when the PROCESS terminates, the OS is smart enough to deallocate the underlying memory blocks is not something that developers should rely on. This would be like never calling delete at all.