Related
Someone on IRC claimed that, although allocating with new[] and deleting with delete (not delete[]) is UB, on Linux platforms (no further details about the OS) it would be safe.
Is this true? Is it guaranteed? Is it to do with something in POSIX that specifies that dynamically-allocated blocks should not have metadata at the start?
Or is it just completely untrue?
Yes, I know I shouldn't do it. I never would.I am curious about the veracity of this idea; that's it!
By "safe", I mean: "will not cause behaviour other than were the original allocation performed by new, or were the de-allocation performed by delete[]". This means that we might see 1 "element" destruction or n, but no crashing.
Of course it's not true. That person is mixing up several different concerns:
how does the OS handle allocations/deallocations
correct calls to constructors and destructors
UB means UB
On the first point, I'm sure he's correct. It is common to handle both in the same way on that level: it is simply a request for X bytes, or a request to release the allocation starting at address X. It doesn't really matter if it's an array or not.
On the second point, everything falls apart. new[] calls the constructor for each element in the allocated array. delete calls the destructor for the one element at the specified address. And so, if you allocate an array of objects, and free it with delete, only one element will have its destructor invoked. (This is easy to forget because people invariably test this with arrays of ints, in which case this difference is unnoticeable)
And then there's the third point, the catch-all. It's UB, and that means it's UB. The compiler may make optimizations based on the assumption that your code does not exhibit any undefined behavior. If it does, it may break some of these assumptions, and seemingly unrelated code might break.
Even if it happens to be safe on some environment, don't do it. There's no reason to want to do it.
Even if it did return the right memory to the OS, the destructors wouldn't be called properly.
It's definitely not true for all or even most Linuxes, your IRC friend is talking bollocks.
POSIX has nothing to do with C++. In general, this is unsafe. If it works anywhere, it's because of the compiler and library, not the OS.
This question discusses in great details when exactly mixing new[] and delete looks safe (no observable problems) on Visual C++. I suppose that by "on Linux" you actually mean "with gcc" and I've observed very similar results with gcc on ideone.com.
Please note that this requires:
global operator new() and operator new[]() functions to be implemented identically and
the compiler optimizing away the "prepend with number of elements" allocation overhead
and also only works for types with trivial destructors.
Even with these requirements met there's no guarantee it will work on a specific version of a specific compiler. You'll be much better off simply not doing that - relying on undefined behavior is a very bad idea.
It is definitely not safe as you can simply try out with the following code:
#include<iostream>
class test {
public:
test(){ std::cout << "Constructor" << std::endl; }
~test(){ std::cout << "Destructor" << std::endl; }
};
int main() {
test * t = new test[ 10 ];
delete t;
return 1;
}
Have a look at http://ideone.com/b8BiQ . It fails misserably.
It may work when you do not use classes, but only fundamental types, but even that is not guaranteed.
EDIT: Some explanations for those of you who want to know why this crashes:
new and delete mainly serve as wrappers around malloc(), hence calling free() on a newed pointer is most of the time "safe" (remember to call the destructor), but you should not rely on it. For new[] and delete[] however the situation is more complicated.
When an array of classes gets constructed using new[] each default constructor will be called in turn. When you do delete[] each destructor gets called. However each destructor also has to be supplied a this pointer to use inside as a hidden parameter. So before calling the destructor the program has to find the locations of all objects within the reserved memory, to pass these locations as this pointers to the destructor. So all information that is later needed to reconstruct this information needs to be stored somewhere.
Now the easiest way would be to have a global map somewhere around, which stores this information for all new[]ed pointers. In this case if you delete is called instead of delete[] only one of the destructors would be called and the entry would not be removed from a map. However this method is usually not used, because maps are slow and memory management should be as fast as possible.
Hence for the stdlibc++ a different solution is used. Since only a few bytes are needed as additional information, it is the fastest to just over-allocate by these few bytes, store the information at the beginning of the memory and return the pointer to the memory after the bookkeeping. So if you allocate an array of 10 objects of 10 bytes each, the programm will allocate 100+X bytes where X is the size of the data which is needed to reconstruct the this.
So in this case it looks something like this
| Bookkeeping | First Object | Second Object |....
^ ^
| This is what is returned by new[]
|
this is what is returned by malloc()
So in case you pass the pointer you have recieved from new[] to delete[] it will call all destructors, then substract X from the pointer and give that one to free(). However if you call delete instead, it will call a destructor for the first object and then immediately pass that pointer to free(), which means free() has just been passed a pointer which was never malloced, which means the result is UB.
Have a look at http://ideone.com/tIiMw , to see what gets passed to delete and delete[]. As you can see, the pointer returned from new[] is not the pointer which was allocated inside, but 4 is added to it before it is being returned to main(). When calling delete[] correctly the same four is substracted an we get the correct pointer within delete[] however this substraction is missing when calling delete and we get the wrong pointer.
In case of calling new[] on a fundamental type, the compiler immediately knows that it will not have to call any destructors later and it just optimizes the bookkeeping away. However it is definitely allowed to write bookkeeping even for fundamental types. And it is also allowed to add bookkeeping in case you call new.
This bookkeeping in front of the real pointer is actually a very good trick, in case you ever need to write your own memory allocation routines as a replacement of new and delete. There is hardly any limit on what you can store there , so one should never assume that anything returned from new or new[] was actually returned from malloc().
I expect that new[] and delete[] just boil down to malloc() and free() under Linux (gcc, glibc, libstdc++), except that the con(de)structors get called. The same for new and delete except that the con(de)structors get called differently. This means that if his constructors and destructors don't matter, then he can probably get away with it. But why try?
I am working on modifying a relatively large C++ program, where unfortunately it is not always clear whether someone before me used C or C++ syntax (this is in the electrical engineering department at a university, and we EEs are always tempted to use C for everything, and unfortunately in this case, people can actually get away with it).
However, if someone creates an object:
Packet* thePacket = new Packet();
Does it matter whether it is destroyed with delete thePacket; or free(thePacket); ?
I realize that delete calls the destructor while free() does not, but Packet does not have a destructor. I am having a terrible time stuck in a memory management swamp here and I'm thinking this may be one of the many problems.
Yes it does matter.
For memory obtained using new you must use delete.
For memory obtained using malloc you must use free.
new and malloc may use different data structures internally to keep track of what and where it has allocated memory. So in order to free memory, you have to call that corresponding function that knows about those data structures. It is however generally a bad idea to mix these two types of memory allocation in a piece of code.
If you call free(), the destructor doesn't get called.
Also, there's no guarantee that new and free operate on the same heap.
You can also override new and delete to operate specially on a particular class. If you do so, but call free() instead of the custom delete, then you miss whatever special behavior you had written into delete. (But you probably wouldn't be asking this question if you had done that, because you'd know what behaviors you were missing..)
Packet has a destructor, even if you haven't explicitly declared one. It has a default destructor. The default destructor probably doesn't actually do much, but you can't count on that being the case. It's up to the compiler what it does.
new and malloc also may have wildly different implementations. For example, delete is always called in a context where it has perfect information about the size of the data structure it's deleting at compile time. free does not have this luxury. It's possible that the allocator that new is using may not store the bytes at the beginning of the memory area stating how many bytes it occupies. This would lead free to do entirely the wrong thing and crash your program when freeing something allocated with new.
Personally, if getting people to do the right thing or fixing the code yourself is completely impossible, I would declare my own global operator new that called malloc so then free would definitely not crash, even though it would still not call the destructor and be generally really ugly.
In short, it is as bad as undefined behavior.
This is quiet self explanatory.
C Standard ($7.20.3.2/2) - "The free
function causes the space pointed to
by ptr to be deallocated, that is,
made available for further allocation.
If ptr is a null pointer, no action
occurs. Otherwise, if the argument
does not match a pointer earlier
returned by the calloc, malloc, or
realloc function, or if the space has
been deallocated by a call to free or
realloc, the behavior is undefined."
You are absolutely right, it is NOT correct. As you said yourself, free won't call the destructor. Even if Packet doesn't have an explicit destructor, it's using an inherited one.
Using free on an object created with new is like destroying only what a shallow-copy would reach. Deep-destroying NEEDS the destructor function.
Also, I'm not sure objects created with new() are on the same memory map as malloc()'d memory. They are not guaranteed to be, I think.
if someone creates an object:
Packet* thePacket = new Packet();
Does it matter whether is is destroyed with delete thePacket; or free(thePacket); ?
Yes it does matter. free (thePacket) would invoke Undefined Behaviour but delete thePacket would not and we all know Undefined Behaviour may have disastrous consequences.
First of all, using delete for anything allocated with new[] is undefined behaviour according to C++ standard.
In Visual C++ 7 such pairing can lead to one of the two consequences.
If the type new[]'ed has trivial constructor and destructor VC++ simply uses new instead of new[] and using delete for that block works fine - new just calls "allocate memory", delete just calls "free memory".
If the type new[]'ed has a non-trivial constructor or destructor the above trick can't be done - VC++7 has to invoke exactly the right number of destructors. So it prepends the array with a size_t storing the number of elements. Now the address returned by new[] points onto the first element, not onto the beginning of the block. So if delete is used it only calls the destructor for the first element and the calls "free memory" with the address different from the one returned by "allocate memory" and this leads to some error indicaton inside HeapFree() which I suspect refers to heap corruption.
Yet every here and there one can read false statements that using delete after new[] leads to a memory leak. I suspect that anything size of heap corruption is much more important than a fact that the destructor is called for the first element only and possibly the destructors not called didn't free heap-allocated sub-objects.
How could using delete after new[] possibly lead only to a memory leak on some C++ implementation?
Suppose I'm a C++ compiler, and I implement my memory management like this: I prepend every block of reserved memory with the size of the memory, in bytes. Something like this;
| size | data ... |
^
pointer returned by new and new[]
Note that, in terms of memory allocation, there is no difference between new and new[]: both just allocate a block of memory of a certain size.
Now how will delete[] know the size of the array, in order to call the right number of destructors? Simply divide the size of the memory block by sizeof(T), where T is the type of elements of the array.
Now suppose I implement delete as simply one call to the destructor, followed by the freeing of the size bytes, then the destructors of the subsequent elements will never be called. This results in leaking resources allocated by the subsequent elements. Yet, because I do free size bytes (not sizeof(T) bytes), no heap corruption occurs.
The fairy tale about mixing new[] and delete allegedly causing a memory leak is just that: a fairy tale. It has absolutely no footing in reality. I don't know where it came from, but by now it acquired a life of its own and survives like a virus, propagating by the word of mouth from one beginner to another.
The most likely rationale behind this "memory leak" nonsense is that from the innocently naive point of view the difference between delete and delete[] is that delete is used to destroy just one object, while delete[] destroys an array of objects ("many" objects). A naive conclusion that is usually derived from this is that the first element of the array will be destroyed by delete, while the rest will persist, thus creating the alleged "memory leak". Of course, any programmer with at least basic understanding of typical heap implementations would immediately understand that the most likely consequence of that is heap corruption, not a "memory leak".
Another popular explanation for the naive "memory leak" theory is that since the wrong number of destructors gets called, the secondary memory owned by the objects in the array does not get deallocated. This might be true, but it is obviously a very forced explanation, which bears little relevance in the face of much more serious problem with heap corruption.
In short, mixing different allocation functions is one of those error that lead to solid, unpredictable and very practical undefined behavior. Any attempts to impose some concrete limits on the manifestations of this undefined behavior are just waste of time and sure sign of the lack of basic understanding.
Needless to add, new/delete and new[]/delete[] are in fact two independent memory management mechanisms, which are independently customizable. Once they get customized (by replacing raw memory management functions) there's absolutely no way to even begin to predict what might happen if they get mixed.
It seems that your question is really "why heap corruption doesn't happen?". The answer to that one is "because the heap manager keeps track of allocated block sizes". Let's go back to C for a minute: if you want to allocate a single int in C you would do int* p = malloc(sizeof(int)), if you want to allocate array of size n you can either write int* p = malloc(n*sizeof(int)) or int* p = calloc(n, sizeof(int)). But in any case you'll free it by free(p), no matter how you allocated it. You never pass size to free(), free() just "knows" how much to free, because the size of a malloc()-ed block is saved somewhere "in front" of the block. Back to C++, new/delete and new[]/delete[] are usually implemented in terms of malloc (although they don't have to be, you shouldn't rely on that). This is why new[]/delete combination doesn't corrupt the heap - delete will free the right amount of memory, but, as explained by everyone before me, you can get leaks by not calling the right number of destructors.
That said, reasoning about undefined behavior in C++ is always pointless exercise. Why does it matter if new[]/delete combination happens to work, "only" leaks or causes heap corruption? You shouldn't code like that, period! And, in practice, I would avoid manual memory management whenever possible - STL & boost are there for a reason.
If the non-trivial destructor that are not called for all but the first element in the array are supposed to free some memory you get a memory leak as these objects are not cleaned up properly.
It will lead to a leak in ALL implementations of C++ in any case where the destructor frees memory, because the destructor never gets called.
In some cases it can cause much worse errors.
memory leak might happen if new() operator is overridden but new[] is not. same goes to the delete / delete[] operator
Apart from resulting in undefined behavior, the most straightforward cause of leaks lies in the implementation not calling the destructor for all but the first object in the array. This will obviously result in leaks if the objects have allocated resources.
This is the simplest possible class I could think of resulting in this behaviour:
struct A {
char* ch;
A(): ch( new char ){}
~A(){ delete ch; }
};
A* as = new A[10]; // ten times the A::ch pointer is allocated
delete as; // only one of the A::ch pointers is freed.
PS: note that constructors fail to get called in lots of other programming mistakes, too: non-virtual base class destructors, false reliance on smart pointers, ...
Late for an answer, but...
If your delete mechanism is simply to call the destructor and put the freed pointer, together with the size implied by sizeof, onto a free stack, then calling delete on a chunk of memory allocated with new[] will result memory being lost -- but not corruption.
More sophisticated malloc structures could corrupt on, or detect, this behaviour.
Why can't the answer be that it causes both?
Obviously memory is leaked whether heap corruption occurs or not.
Or rather, since I can re-implement new and delete..... can't it not cause anything at all. Technically I can cause new and delete to perform new[] and delete[].
HENCE: Undefined Behavior.
I was answering a question which was marked off as a duplicate, so i'll just copy it here in case it metters. It was said well before me the way memory allocation works, i`ll just explain the cause & effects.
Just a little thing right off google: http://en.cppreference.com/w/cpp/memory/new/operator_delete
Anyhow, delete is a function for a single object. It frees the instance from the pointer, and leaves;
delete[] is a function used in order to deallocate arrays. That means, it doesnt just free the pointer; It declares the whole memory block of that array as garbage.
That's all cool in practice, but you tell me your application works. You are probably wondering... why?
The solution is C++ does not fix memory leaks. If you`ll use delete without the parenthesises, it'll delete just the array as an object - a proccess which might cause a memory leak.
cool story, memory leak, why should i care?
Memory leak happens when allocated memory doesn't get deleted. That memory then requires unneccessary disk-space, which will make you lose useful memory for pretty much no reason. That's bad programming, and you should probably fix it in your systems.
When, if ever, can delete and free be used interchangeably in C++?
My concern is as follows: Say there is an incorrect mixup in the use of malloc/ free and
new/ delete (not to mention new[]/ delete[]). However delete and free doing the same thing;
Fortuitously so this goes uncaught in testing. Later this may lead to a crash in production.
How can I enforce some kind of check to prevent this? Can I be warned if the two are mixed up?
If not at compile time, perhaps some code instrumentation at run time? How would I approach
this?
The intention of this question is to find ways to avoid inadvertent mix up in the usages.
The easy way to not get them mixed up is never to use malloc(), then you will never be tempted to call free(). The infrastructure to create to avoid this problem is called "code review", though in this case a quick "grep malloc(" or "grep free(" on the codebase will probably suffice.
Never. If it works it's by sheer accident of implementation. Do not rely on this behavior.
To answer the second question, if you control both malloc/free and operator new/delete, you can stash extra information to associate with pointers returned by both that tell you how they were allocated. When a pointer is passed to free or operator delete, check to see that it was allocated by the appropriate function. If not, assert or raise an exception or do whatever it is you do to report the mismatch.
Usually this is done by allocating extra memory, e.g., given malloc(size) or operator new(size), you allocate size + additional space and shove extra information in there.
The only way you can ensure you never get them mixed up is by either:
Never using malloc/free in the first place, or
Rely on RAII for your memory allocations. Protect every memory allocation in a RAII object which ensures the memory get correctly and consistently freed when it goes out of scope, or wrap the allocation in a smart pointer.
Manually calling delete or free is just an invitation for bugs.
You should always use new and delete in C++, as only new and delete will call the object's constructor and destructor.
If you find you must use both (for instance, if you're interfacing with a C library), thorough code reviews should carefully scrutinize any uses of free() to determine whether or not they correspond to a malloc(), and whether or not they are being used in a C context.
If I had to codify it, I'd put in the style guide something like this:
free() may be called only on a private pointer field of an object.
malloc()ed buffers (or buffers returned from a C API which caller must free()) must be assigned to a private pointer field of an object.
private pointer fields which hold free()-able buffers must only be used for that purpose.
if you use hungarian notation, add a letter for it (and if you don't, don't).
generally free() will be called only in a destructor, with exceptions when the free()-able buffer is replaced during the lifetime of the object. In that case you can call free() on a value recently copied out of a private field during replacement, rather than on the field value directly.
In other words, stick a wrapper around anything that uses malloc/free. This wrapper could be a single template everyone uses, or you could allow smart pointers with the deletor function set to free(). Then in code review, if you see a call to malloc/free anywhere else, it's wrong.
Basically the best way to stop this being a problem is to be on top of your resource handling in general. In C, people do not have a major problem with accidentally calling free() on streams instead of fclose().
You should always use delete or delete[] when freeing things allocated with new. The same goes for malloc and free.
If using free for deleting new:ed classes the destructor won't be properly called.
Also new and delete doesn't necessarily use malloc/free for its allocations so you might end up corrupting your heap as well.
always use delete for things allocated with new, delete [] for things allocated with new [] and free() for things allocated using malloc().
new and new[] allocate from different heaps than malloc() and using the wrong free()/delete will attempt to deallocate from the wrong heap.
Never mix new/delete with new[]/delete[] or with malloc()/free(). Atop of this, the use of malloc()/free() in C++ is questionable at least.
The easiest way to make sure you never do this accidentally is to not to manage memory manually. In C++ there strings and other containers as well as smart pointers to take care of memory management. There simply isn't a need to do this manually. (I'm not sure whether the last time I remember me typing delete really was the last time I typed it. But if my memory doesn't fail me, this must have been 2001 or 2002.)
You could write your own version of the functions which allocate some extra memory in new/new[]/malloc to track which one did the allocation. Then check in delete/delete[]/free that the right function was used to re-claim the memory. You can also check for things like mixing new[] and delete (without the []) this way. You might want to only use these versions in the debug build.
You could try running valgrind on your application to see if it catches anything. The manual specifically mentions its ability to catch uses of the wrong deallocation function for a block of memory. I don't think there's a way to do this at compile time, however.
One way to avoid misusage of malloc/free or new/delete is that DO NOT call these function directly, call them by a wrapper layer. In the wrapper layer, you can manage the pointers distributed by your own hands and guarantee that misusage will get an error explicitly.
Why dont you just count the total number of malloc statments and tally that with the total count of free? Do the same for New and delete. The process can be automated with regular expressions.
One thing I've done relating to this is to reimplement malloc and free
in C++, so that it calls new/delete under the hood. It's not
optimal, but it serves well enough on a smaller project.
I have been working on some legacy C++ code that uses variable length structures (TAPI), where the structure size will depend on variable length strings. The structures are allocated by casting array new thus:
STRUCT* pStruct = (STRUCT*)new BYTE[sizeof(STRUCT) + nPaddingSize];
Later on however the memory is freed using a delete call:
delete pStruct;
Will this mix of array new[] and non-array delete cause a memory leak or would it depend on the compiler? Would I be better off changing this code to use malloc and free instead?
Technically I believe it could cause a problem with mismatched allocators, though in practice I don't know of any compiler that would not do the right thing with this example.
More importantly if STRUCT where to have (or ever be given) a destructor then it would invoke the destructor without having invoked the corresponding constructor.
Of course, if you know where pStruct came from why not just cast it on delete to match the allocation:
delete [] (BYTE*) pStruct;
I personally think you'd be better off using std::vector to manage your memory, so you don't need the delete.
std::vector<BYTE> backing(sizeof(STRUCT) + nPaddingSize);
STRUCT* pStruct = (STRUCT*)(&backing[0]);
Once backing leaves scope, your pStruct is no longer valid.
Or, you can use:
boost::scoped_array<BYTE> backing(new BYTE[sizeof(STRUCT) + nPaddingSize]);
STRUCT* pStruct = (STRUCT*)backing.get();
Or boost::shared_array if you need to move ownership around.
Yes it will cause a memory leak.
See this except from C++ Gotchas: http://www.informit.com/articles/article.aspx?p=30642 for why.
Raymond Chen has an explanation of how vector new and delete differ from the scalar versions under the covers for the Microsoft compiler... Here:
http://blogs.msdn.com/oldnewthing/archive/2004/02/03/66660.aspx
IMHO you should fix the delete to:
delete [] pStruct;
rather than switching to malloc/free, if only because it's a simpler change to make without making mistakes ;)
And, of course, the simpler to make change that I show above is wrong due to the casting in the original allocation, it should be
delete [] reinterpret_cast<BYTE *>(pStruct);
so, I guess it's probably as easy to switch to malloc/free after all ;)
The behaviour of the code is undefined. You may be lucky (or not) and it may work with your compiler, but really that's not correct code. There's two problems with it:
The delete should be an array delete [].
The delete should be called on a pointer to the same type as the type allocated.
So to be entirely correct, you want to be doing something like this:
delete [] (BYTE*)(pStruct);
The C++ standard clearly states:
delete-expression:
::opt delete cast-expression
::opt delete [ ] cast-expression
The first alternative is for non-array objects, and the second is for arrays. The operand shall have a pointer type, or a class type having a single conversion function (12.3.2) to a pointer type. The result has type void.
In the first alternative (delete object), the value of the operand of delete shall be a pointer to a non-array object [...] If not, the behavior is undefined.
The value of the operand in delete pStruct is a pointer to an array of char, independent of its static type (STRUCT*). Therefore, any discussion of memory leaks is quite pointless, because the code is ill-formed, and a C++ compiler is not required to produce a sensible executable in this case.
It could leak memory, it could not, or it could do anything up to crashing your system. Indeed, a C++ implementation with which I tested your code aborts the program execution at the point of the delete expression.
As highlighted in other posts:
1) Calls to new/delete allocate memory and may call constructors/destructors (C++ '03 5.3.4/5.3.5)
2) Mixing array/non-array versions of new and delete is undefined behaviour. (C++ '03 5.3.5/4)
Looking at the source it appears that someone did a search and replace for malloc and free and the above is the result. C++ does have a direct replacement for these functions, and that is to call the allocation functions for new and delete directly:
STRUCT* pStruct = (STRUCT*)::operator new (sizeof(STRUCT) + nPaddingSize);
// ...
pStruct->~STRUCT (); // Call STRUCT destructor
::operator delete (pStruct);
If the constructor for STRUCT should be called, then you could consider allocating the memory and then use placement new:
BYTE * pByteData = new BYTE[sizeof(STRUCT) + nPaddingSize];
STRUCT * pStruct = new (pByteData) STRUCT ();
// ...
pStruct->~STRUCT ();
delete[] pByteData;
#eric - Thanks for the comments. You keep saying something though, that drives me nuts:
Those run-time libraries handle the
memory management calls to the OS in a
OS independent consistent syntax and
those run-time libraries are
responsible for making malloc and new
work consistently between OSes such as
Linux, Windows, Solaris, AIX, etc....
This is not true. The compiler writer provides the implementation of the std libraries, for instance, and they are absolutely free to implement those in an OS dependent way. They're free, for instance, to make one giant call to malloc, and then manage memory within the block however they wish.
Compatibility is provided because the API of std, etc. is the same - not because the run-time libraries all turn around and call the exact same OS calls.
The various possible uses of the keywords new and delete seem to create a fair amount of confusion. There are always two stages to constructing dynamic objects in C++: the allocation of the raw memory and the construction of the new object in the allocated memory area. On the other side of the object lifetime there is the destruction of the object and the deallocation of the memory location where the object resided.
Frequently these two steps are performed by a single C++ statement.
MyObject* ObjPtr = new MyObject;
//...
delete MyObject;
Instead of the above you can use the C++ raw memory allocation functions operator new and operator delete and explicit construction (via placement new) and destruction to perform the equivalent steps.
void* MemoryPtr = ::operator new( sizeof(MyObject) );
MyObject* ObjPtr = new (MemoryPtr) MyObject;
// ...
ObjPtr->~MyObject();
::operator delete( MemoryPtr );
Notice how there is no casting involved, and only one type of object is constructed in the allocated memory area. Using something like new char[N] as a way to allocate raw memory is technically incorrect as, logically, char objects are created in the newly allocated memory. I don't know of any situation where it doesn't 'just work' but it blurs the distinction between raw memory allocation and object creation so I advise against it.
In this particular case, there is no gain to be had by separating out the two steps of delete but you do need to manually control the initial allocation. The above code works in the 'everything working' scenario but it will leak the raw memory in the case where the constructor of MyObject throws an exception. While this could be caught and solved with an exception handler at the point of allocation it is probably neater to provide a custom operator new so that the complete construction can be handled by a placement new expression.
class MyObject
{
void* operator new( std::size_t rqsize, std::size_t padding )
{
return ::operator new( rqsize + padding );
}
// Usual (non-placement) delete
// We need to define this as our placement operator delete
// function happens to have one of the allowed signatures for
// a non-placement operator delete
void operator delete( void* p )
{
::operator delete( p );
}
// Placement operator delete
void operator delete( void* p, std::size_t )
{
::operator delete( p );
}
};
There are a couple of subtle points here. We define a class placement new so that we can allocate enough memory for the class instance plus some user specifiable padding. Because we do this we need to provide a matching placement delete so that if the memory allocation succeeds but the construction fails, the allocated memory is automatically deallocated. Unfortunately, the signature for our placement delete matches one of the two allowed signatures for non-placement delete so we need to provide the other form of non-placement delete so that our real placement delete is treated as a placement delete. (We could have got around this by adding an extra dummy parameter to both our placement new and placement delete, but this would have required extra work at all the calling sites.)
// Called in one step like so:
MyObject* ObjectPtr = new (padding) MyObject;
Using a single new expression we are now guaranteed that memory won't leak if any part of the new expression throws.
At the other end of the object lifetime, because we defined operator delete (even if we hadn't, the memory for the object originally came from global operator new in any case), the following is the correct way to destroy the dynamically created object.
delete ObjectPtr;
Summary!
Look no casts! operator new and operator delete deal with raw memory, placement new can construct objects in raw memory. An explicit cast from a void* to an object pointer is usually a sign of something logically wrong, even if it does 'just work'.
We've completely ignored new[] and delete[]. These variable size objects will not work in arrays in any case.
Placement new allows a new expression not to leak, the new expression still evaluates to a pointer to an object that needs destroying and memory that needs deallocating. Use of some type of smart pointer may help prevent other types of leak. On the plus side we've let a plain delete be the correct way to do this so most standard smart pointers will work.
If you really must do this sort of thing, you should probably call operator new directly:
STRUCT* pStruct = operator new(sizeof(STRUCT) + nPaddingSize);
I believe calling it this way avoids calling constructors/destructors.
I am currently unable to vote, but slicedlime's answer is preferable to Rob Walker's answer, since the problem has nothing to do with allocators or whether or not the STRUCT has a destructor.
Also note that the example code does not necessarily result in a memory leak - it's undefined behavior. Pretty much anything could happen (from nothing bad to a crash far, far away).
The example code results in undefined behavior, plain and simple. slicedlime's answer is direct and to the point (with the caveat that the word 'vector' should be changed to 'array' since vectors are an STL thing).
This kind of stuff is covered pretty well in the C++ FAQ (Sections 16.12, 16.13, and 16.14):
http://www.parashift.com/c++-faq-lite/freestore-mgmt.html#faq-16.12
It's an array delete ([]) you're referring to, not a vector delete.
A vector is std::vector, and it takes care of deletion of its elements.
You'd could cast back to a BYTE * and the delete:
delete[] (BYTE*)pStruct;
Yes that may, since your allocating with new[] but deallocating with delelte, yes malloc/free is safer here, but in c++ you should not use them since they won't handle (de)constructors.
Also your code will call the deconstructor, but not the constructor. For some structs this may cause a memory leak (if the constructor allocated further memory, eg for a string)
Better would be to do it correctly, as this will also correctly call any constructors and deconstructors
STRUCT* pStruct = new STRUCT;
...
delete pStruct;
It's always best to keep acquisition/release of any resource as balanced as possible.
Although leaking or not is hard to say in this case. It depends on the compiler's implementation of the vector (de)allocation.
BYTE * pBytes = new BYTE [sizeof(STRUCT) + nPaddingSize];
STRUCT* pStruct = reinterpret_cast< STRUCT* > ( pBytes ) ;
// do stuff with pStruct
delete [] pBytes ;
Len: the problem with that is that pStruct is a STRUCT*, but the memory allocated is actually a BYTE[] of some unknown size. So delete[] pStruct will not de-allocate all of the allocated memory.
You're sort of mixing C and C++ ways of doing things. Why allocate more than the size of a STRUCT? Why not just "new STRUCT"? If you must do this then it might be clearer to use malloc and free in this case, since then you or other programmers might be a little less likely to make assumptions about the types and sizes of the allocated objects.
#Matt Cruikshank
You should pay attention and read what I wrote again because I never suggested not calling delete[] and just let the OS clean up. And you're wrong about the C++ run-time libraries managing the heap. If that were the case then C++ would not be portable as is today and a crashing application would never get cleaned up by the OS. (acknowledging there are OS specific run-times that make C/C++ appear non-portable). I challenge you to find stdlib.h in the Linux sources from kernel.org. The new keyword in C++ actually is talking to the same memory management routines as malloc.
The C++ run-time libraries make OS system calls and it's the OS that manages the heaps. You are partly correct in that the run-time libraries indicate when to release the memory however, they don't actually walk any heap tables directly. In other words, the runtime you link against does not add code to your application to walk heaps to allocate or deallocate. This is the case in Windows, Linux, Solaris, AIX, etc... It's also the reason you won't fine malloc in any Linux's kernel source nor will you find stdlib.h in Linux source. Understand these modern operating system have virtual memory managers that complicates things a bit further.
Ever wonder why you can make a call to malloc for 2G of RAM on a 1G box and still get back a valid memory pointer?
Memory management on x86 processors is managed within Kernel space using three tables. PAM (Page Allocation Table), PD (Page Directories) and PT (Page Tables). This is at the hardware level I'm speaking of. One of the things the OS memory manager does, not your C++ application, is to find out how much physical memory is installed on the box during boot with help of BIOS calls. The OS also handles exceptions such as when you try to access memory your application does not have rights too. (GPF General Protection Fault).
It may be that we are saying the same thing Matt, but I think you may be confusing the under hood functionality a bit. I use to maintain a C/C++ compiler for a living...
#ericmayo - cripes. Well, experimenting with VS2005, I can't get an honest leak out of scalar delete on memory that was made by vector new. I guess the compiler behavior is "undefined" here, is about the best defense I can muster.
You've got to admit though, it's a really lousy practice to do what the original poster said.
If that were the case then C++ would
not be portable as is today and a
crashing application would never get
cleaned up by the OS.
This logic doesn't really hold, though. My assertion is that a compiler's runtime can manage the memory within the memory blocks that the OS returns to it. This is how most virtual machines work, so your argument against portability in this case don't make much sense.
#Matt Cruikshank
"Well, experimenting with VS2005, I can't get an honest leak out of scalar delete on memory that was made by vector new. I guess the compiler behavior is "undefined" here, is about the best defense I can muster."
I disagree that it's a compiler behavior or even a compiler issue. The 'new' keyword gets compiled and linked, as you pointed out, to run-time libraries. Those run-time libraries handle the memory management calls to the OS in a OS independent consistent syntax and those run-time libraries are responsible for making malloc and new work consistently between OSes such as Linux, Windows, Solaris, AIX, etc.... This is the reason I mentioned the portability argument; an attempt to prove to you that the run-time does not actually manage memory either.
The OS manages memory.
The run-time libs interface to the OS.. On Windows, this is the virtual memory manager DLLs. This is why stdlib.h is implemented within the GLIB-C libraries and not the Linux kernel source; if GLIB-C is used on other OSes, it's implementation of malloc changes to make the correct OS calls. In VS, Borland, etc.. you will never find any libraries that ship with their compilers that actually manage memory either. You will, however, find OS specific definitions for malloc.
Since we have the source to Linux, you can go look at how malloc is implemented there. You will see that malloc is actually implemented in the GCC compiler which, in turn, basically makes two Linux system calls into the kernel to allocate memory. Never, malloc itself, actually managing memory!
And don't take it from me. Read the source code to Linux OS or you can see what K&R say about it... Here is a PDF link to the K&R on C.
http://www.oberon2005.ru/paper/kr_c.pdf
See near end of Page 149:
"Calls to malloc and free may occur in any order; malloc calls
upon the operating system to obtain more memory as necessary. These routines illustrate some of the considerations involved in writing machine-dependent code in a relatively machineindependent way, and also show a real-life application of structures, unions and typedef."
"You've got to admit though, it's a really lousy practice to do what the original poster said."
Oh, I don't disagree there. My point was that the original poster's code was not conducive of a memory leak. That's all I was saying. I didn't chime in on the best practice side of things. Since the code is calling delete, the memory is getting free up.
I agree, in your defense, if the original poster's code never exited or never made it to the delete call, that the code could have a memory leak but since he states that later on he sees the delete getting called. "Later on however the memory is freed using a delete call:"
Moreover, my reason for responding as I did was due to the OP's comment "variable length structures (TAPI), where the structure size will depend on variable length strings"
That comment sounded like he was questioning the dynamic nature of the allocations against the cast being made and was consequentially wondering if that would cause a memory leak. I was reading between the lines if you will ;).
In addition to the excellent answers above, I would also like to add:
If your code runs on linux or if you can compile it on linux then I would suggest running it through Valgrind. It is an excellent tool, among the myriad of useful warnings it produces it also will tell you when you allocate memory as an array and then free it as a non-array ( and vice-versa ).
Use operator new and delete:
struct STRUCT
{
void *operator new (size_t)
{
return new char [sizeof(STRUCT) + nPaddingSize];
}
void operator delete (void *memory)
{
delete [] reinterpret_cast <char *> (memory);
}
};
void main()
{
STRUCT *s = new STRUCT;
delete s;
}
I think the is no memory leak.
STRUCT* pStruct = (STRUCT*)new BYTE [sizeof(STRUCT) + nPaddingSize];
This gets translated into a memory allocation call within the operating system upon which a pointer to that memory is returned. At the time memory is allocated, the size of sizeof(STRUCT) and the size of nPaddingSize would be known in order to fulfill any memory allocation requests against the underlying operating system.
So the memory that is allocated is "recorded" in the operating system's global memory allocation tables. Memory tables are indexed by their pointers. So in the corresponding call to delete, all memory that was originally allocated is free. (memory fragmentation a popular subject in this realm as well).
You see, the C/C++ compiler is not managing memory, the underlying operating system is.
I agree there are cleaner methods but the OP did say this was legacy code.
In short, I don't see a memory leak as the accepted answer believes there to be one.
Rob Walker reply is good.
Just small addition, if you don't have any constructor or/and distructors, so you basically need allocate and free a chunk of raw memory, consider using free/malloc pair.
ericmayo.myopenid.com is so wrong, that someone with enough reputation should downvote him.
The C or C++ runtime libraries are managing the heap which is given to it in blocks by the Operating System, somewhat like you indicate, Eric. But it is the responsibility of the developer to indicate to the compiler which runtime calls should be made to free memory, and possibly destruct the objects that are there. Vector delete (aka delete[]) is necessary in this case, in order for the C++ runtime to leave the heap in a valid state. The fact that when the PROCESS terminates, the OS is smart enough to deallocate the underlying memory blocks is not something that developers should rely on. This would be like never calling delete at all.