I am working on modifying a relatively large C++ program, where unfortunately it is not always clear whether someone before me used C or C++ syntax (this is in the electrical engineering department at a university, and we EEs are always tempted to use C for everything, and unfortunately in this case, people can actually get away with it).
However, if someone creates an object:
Packet* thePacket = new Packet();
Does it matter whether it is destroyed with delete thePacket; or free(thePacket); ?
I realize that delete calls the destructor while free() does not, but Packet does not have a destructor. I am having a terrible time stuck in a memory management swamp here and I'm thinking this may be one of the many problems.
Yes it does matter.
For memory obtained using new you must use delete.
For memory obtained using malloc you must use free.
new and malloc may use different data structures internally to keep track of what and where it has allocated memory. So in order to free memory, you have to call that corresponding function that knows about those data structures. It is however generally a bad idea to mix these two types of memory allocation in a piece of code.
If you call free(), the destructor doesn't get called.
Also, there's no guarantee that new and free operate on the same heap.
You can also override new and delete to operate specially on a particular class. If you do so, but call free() instead of the custom delete, then you miss whatever special behavior you had written into delete. (But you probably wouldn't be asking this question if you had done that, because you'd know what behaviors you were missing..)
Packet has a destructor, even if you haven't explicitly declared one. It has a default destructor. The default destructor probably doesn't actually do much, but you can't count on that being the case. It's up to the compiler what it does.
new and malloc also may have wildly different implementations. For example, delete is always called in a context where it has perfect information about the size of the data structure it's deleting at compile time. free does not have this luxury. It's possible that the allocator that new is using may not store the bytes at the beginning of the memory area stating how many bytes it occupies. This would lead free to do entirely the wrong thing and crash your program when freeing something allocated with new.
Personally, if getting people to do the right thing or fixing the code yourself is completely impossible, I would declare my own global operator new that called malloc so then free would definitely not crash, even though it would still not call the destructor and be generally really ugly.
In short, it is as bad as undefined behavior.
This is quiet self explanatory.
C Standard ($7.20.3.2/2) - "The free
function causes the space pointed to
by ptr to be deallocated, that is,
made available for further allocation.
If ptr is a null pointer, no action
occurs. Otherwise, if the argument
does not match a pointer earlier
returned by the calloc, malloc, or
realloc function, or if the space has
been deallocated by a call to free or
realloc, the behavior is undefined."
You are absolutely right, it is NOT correct. As you said yourself, free won't call the destructor. Even if Packet doesn't have an explicit destructor, it's using an inherited one.
Using free on an object created with new is like destroying only what a shallow-copy would reach. Deep-destroying NEEDS the destructor function.
Also, I'm not sure objects created with new() are on the same memory map as malloc()'d memory. They are not guaranteed to be, I think.
if someone creates an object:
Packet* thePacket = new Packet();
Does it matter whether is is destroyed with delete thePacket; or free(thePacket); ?
Yes it does matter. free (thePacket) would invoke Undefined Behaviour but delete thePacket would not and we all know Undefined Behaviour may have disastrous consequences.
Related
Someone on IRC claimed that, although allocating with new[] and deleting with delete (not delete[]) is UB, on Linux platforms (no further details about the OS) it would be safe.
Is this true? Is it guaranteed? Is it to do with something in POSIX that specifies that dynamically-allocated blocks should not have metadata at the start?
Or is it just completely untrue?
Yes, I know I shouldn't do it. I never would.I am curious about the veracity of this idea; that's it!
By "safe", I mean: "will not cause behaviour other than were the original allocation performed by new, or were the de-allocation performed by delete[]". This means that we might see 1 "element" destruction or n, but no crashing.
Of course it's not true. That person is mixing up several different concerns:
how does the OS handle allocations/deallocations
correct calls to constructors and destructors
UB means UB
On the first point, I'm sure he's correct. It is common to handle both in the same way on that level: it is simply a request for X bytes, or a request to release the allocation starting at address X. It doesn't really matter if it's an array or not.
On the second point, everything falls apart. new[] calls the constructor for each element in the allocated array. delete calls the destructor for the one element at the specified address. And so, if you allocate an array of objects, and free it with delete, only one element will have its destructor invoked. (This is easy to forget because people invariably test this with arrays of ints, in which case this difference is unnoticeable)
And then there's the third point, the catch-all. It's UB, and that means it's UB. The compiler may make optimizations based on the assumption that your code does not exhibit any undefined behavior. If it does, it may break some of these assumptions, and seemingly unrelated code might break.
Even if it happens to be safe on some environment, don't do it. There's no reason to want to do it.
Even if it did return the right memory to the OS, the destructors wouldn't be called properly.
It's definitely not true for all or even most Linuxes, your IRC friend is talking bollocks.
POSIX has nothing to do with C++. In general, this is unsafe. If it works anywhere, it's because of the compiler and library, not the OS.
This question discusses in great details when exactly mixing new[] and delete looks safe (no observable problems) on Visual C++. I suppose that by "on Linux" you actually mean "with gcc" and I've observed very similar results with gcc on ideone.com.
Please note that this requires:
global operator new() and operator new[]() functions to be implemented identically and
the compiler optimizing away the "prepend with number of elements" allocation overhead
and also only works for types with trivial destructors.
Even with these requirements met there's no guarantee it will work on a specific version of a specific compiler. You'll be much better off simply not doing that - relying on undefined behavior is a very bad idea.
It is definitely not safe as you can simply try out with the following code:
#include<iostream>
class test {
public:
test(){ std::cout << "Constructor" << std::endl; }
~test(){ std::cout << "Destructor" << std::endl; }
};
int main() {
test * t = new test[ 10 ];
delete t;
return 1;
}
Have a look at http://ideone.com/b8BiQ . It fails misserably.
It may work when you do not use classes, but only fundamental types, but even that is not guaranteed.
EDIT: Some explanations for those of you who want to know why this crashes:
new and delete mainly serve as wrappers around malloc(), hence calling free() on a newed pointer is most of the time "safe" (remember to call the destructor), but you should not rely on it. For new[] and delete[] however the situation is more complicated.
When an array of classes gets constructed using new[] each default constructor will be called in turn. When you do delete[] each destructor gets called. However each destructor also has to be supplied a this pointer to use inside as a hidden parameter. So before calling the destructor the program has to find the locations of all objects within the reserved memory, to pass these locations as this pointers to the destructor. So all information that is later needed to reconstruct this information needs to be stored somewhere.
Now the easiest way would be to have a global map somewhere around, which stores this information for all new[]ed pointers. In this case if you delete is called instead of delete[] only one of the destructors would be called and the entry would not be removed from a map. However this method is usually not used, because maps are slow and memory management should be as fast as possible.
Hence for the stdlibc++ a different solution is used. Since only a few bytes are needed as additional information, it is the fastest to just over-allocate by these few bytes, store the information at the beginning of the memory and return the pointer to the memory after the bookkeeping. So if you allocate an array of 10 objects of 10 bytes each, the programm will allocate 100+X bytes where X is the size of the data which is needed to reconstruct the this.
So in this case it looks something like this
| Bookkeeping | First Object | Second Object |....
^ ^
| This is what is returned by new[]
|
this is what is returned by malloc()
So in case you pass the pointer you have recieved from new[] to delete[] it will call all destructors, then substract X from the pointer and give that one to free(). However if you call delete instead, it will call a destructor for the first object and then immediately pass that pointer to free(), which means free() has just been passed a pointer which was never malloced, which means the result is UB.
Have a look at http://ideone.com/tIiMw , to see what gets passed to delete and delete[]. As you can see, the pointer returned from new[] is not the pointer which was allocated inside, but 4 is added to it before it is being returned to main(). When calling delete[] correctly the same four is substracted an we get the correct pointer within delete[] however this substraction is missing when calling delete and we get the wrong pointer.
In case of calling new[] on a fundamental type, the compiler immediately knows that it will not have to call any destructors later and it just optimizes the bookkeeping away. However it is definitely allowed to write bookkeeping even for fundamental types. And it is also allowed to add bookkeeping in case you call new.
This bookkeeping in front of the real pointer is actually a very good trick, in case you ever need to write your own memory allocation routines as a replacement of new and delete. There is hardly any limit on what you can store there , so one should never assume that anything returned from new or new[] was actually returned from malloc().
I expect that new[] and delete[] just boil down to malloc() and free() under Linux (gcc, glibc, libstdc++), except that the con(de)structors get called. The same for new and delete except that the con(de)structors get called differently. This means that if his constructors and destructors don't matter, then he can probably get away with it. But why try?
I know free() won't call the destructor, but what else will this cause besides that the member variable won't be destructed properly?
Also, what if we delete a pointer that is allocated by malloc?
It is implementation defined whether new uses malloc under the hood. Mixing new with free and malloc with delete could cause a catastrophic failure at runtime if the code was ported to a new machine, a new compiler, or even a new version of the same compiler.
I know free() won't call the destructor
And that is reason enough not to do it.
In addition, there's no requirement for a C++ implementation to even use the same memory areas for malloc and new so it may be that you're trying to free memory from a totally different arena, something which will almost certainly be fatal.
Many points:
It's undefined behaviour, and hence inherently risky and subject to change or breakage at any time and for no reason at all.
(As you know) delete calls the destructor and free doesn't... you may have some POD type and not care, but it's easy for someone else to add say a string to that type without realising there are weird limitations on its content.
If you malloc and forget to use placement new to construct an object in it, then invoke a member function as if the object existed (including delete which calls the destructor), the member function may attempt operations using pointers with garbage values
new and malloc may get memory from different heaps.
Even if new calls malloc to get its memory, there may not be a 1:1 correspondence between the new/delete and underlying malloc/free behaviour.
e.g. new may have extra logic such as small-object optimisations that have proven beneficial to typical C++ programs but harmful to typical C programs.
Someone may overload new, or link in a debug version of malloc/realloc/free, either of which could break if you're not using the functions properly.
Tools like ValGrind, Purify and Insure won't be able to differentiate between the deliberately dubious and the accidentally.
In the case of arrays, delete[] invokes all the destructors and free() won't, but also the heap memory typically has a counter of the array size (for 32-bit VC++2005 Release builds for example, the array size is in the 4 bytes immediately before the pointer value visibly returned by new[]. This extra value may or may not be be there for POD types (not for VC++2005), but if it is free() certainly won't expect it. Not all heap implementations allow you to free a pointer that's been shifted from the value returned by malloc().
An important difference is that new and delete also call the constructor and destructor of the object. Thus, you may get unexpected behavior. That is the most important thing i think.
Because it might not be the same allocator, which could lead to weird, unpredictable behaviour. Plus, you shouldn't be using malloc/free at all, and avoid using new/delete where it's not necessary.
It totally depends on the implementation -- it's possible to write an implementation where this actually works fine. But there's no guarantee that the pool of memory new allocates from is the same pool that free() wants to return the memory to. Imagine that both malloc() and new use a few bytes of extra memory at the beginning of each allocated block to specify how large the block is. Further, imagine that malloc() and new use different formats for this info -- for example, malloc() uses the number of bytes, but new uses the number of 4-byte long words (just an example). Now, if you allocate with malloc() and free with delete, the info delete expects won't be valid, and you'll end up with a corrupted heap.
This question was asked to me in an interview:
In C++,
what if we allocate memory using malloc and use delete to free that allocated memory?
what if we allocate the memory using new and free it using free?
What are the problems that we would face if the above things are used in the code?
My answer was there is no difference. Was I right in saying so?
If you do so you will run into undefined behavior. Never try that. Although new might be implemented through malloc() and delete might be implemented through free() there's no guarantee that they are really implemented that way and also the user can overload new and delete at his discretion. You risk running into heap corruption.
Other than that don't forget that when you call malloc() you get raw memory - no constructor is invoked - and when you call free() no destructor is invoked. This can as well lead to undefined behavior and improper functioning of the program.
The bottom line is... never do this.
1) Undefined behaviour but will probably "work" though. Destructors will get called on the memory being freed that pobably doesn't want to be deconstructed.
2) Undefined behaviour but will probably "work" though. Destructors will NOT get called.
ie IF it works, and there is no guarantee of that, then it will only, likely, work exactly as required for basic builtin data types.
First of all, using delete for anything allocated with new[] is undefined behaviour according to C++ standard.
In Visual C++ 7 such pairing can lead to one of the two consequences.
If the type new[]'ed has trivial constructor and destructor VC++ simply uses new instead of new[] and using delete for that block works fine - new just calls "allocate memory", delete just calls "free memory".
If the type new[]'ed has a non-trivial constructor or destructor the above trick can't be done - VC++7 has to invoke exactly the right number of destructors. So it prepends the array with a size_t storing the number of elements. Now the address returned by new[] points onto the first element, not onto the beginning of the block. So if delete is used it only calls the destructor for the first element and the calls "free memory" with the address different from the one returned by "allocate memory" and this leads to some error indicaton inside HeapFree() which I suspect refers to heap corruption.
Yet every here and there one can read false statements that using delete after new[] leads to a memory leak. I suspect that anything size of heap corruption is much more important than a fact that the destructor is called for the first element only and possibly the destructors not called didn't free heap-allocated sub-objects.
How could using delete after new[] possibly lead only to a memory leak on some C++ implementation?
Suppose I'm a C++ compiler, and I implement my memory management like this: I prepend every block of reserved memory with the size of the memory, in bytes. Something like this;
| size | data ... |
^
pointer returned by new and new[]
Note that, in terms of memory allocation, there is no difference between new and new[]: both just allocate a block of memory of a certain size.
Now how will delete[] know the size of the array, in order to call the right number of destructors? Simply divide the size of the memory block by sizeof(T), where T is the type of elements of the array.
Now suppose I implement delete as simply one call to the destructor, followed by the freeing of the size bytes, then the destructors of the subsequent elements will never be called. This results in leaking resources allocated by the subsequent elements. Yet, because I do free size bytes (not sizeof(T) bytes), no heap corruption occurs.
The fairy tale about mixing new[] and delete allegedly causing a memory leak is just that: a fairy tale. It has absolutely no footing in reality. I don't know where it came from, but by now it acquired a life of its own and survives like a virus, propagating by the word of mouth from one beginner to another.
The most likely rationale behind this "memory leak" nonsense is that from the innocently naive point of view the difference between delete and delete[] is that delete is used to destroy just one object, while delete[] destroys an array of objects ("many" objects). A naive conclusion that is usually derived from this is that the first element of the array will be destroyed by delete, while the rest will persist, thus creating the alleged "memory leak". Of course, any programmer with at least basic understanding of typical heap implementations would immediately understand that the most likely consequence of that is heap corruption, not a "memory leak".
Another popular explanation for the naive "memory leak" theory is that since the wrong number of destructors gets called, the secondary memory owned by the objects in the array does not get deallocated. This might be true, but it is obviously a very forced explanation, which bears little relevance in the face of much more serious problem with heap corruption.
In short, mixing different allocation functions is one of those error that lead to solid, unpredictable and very practical undefined behavior. Any attempts to impose some concrete limits on the manifestations of this undefined behavior are just waste of time and sure sign of the lack of basic understanding.
Needless to add, new/delete and new[]/delete[] are in fact two independent memory management mechanisms, which are independently customizable. Once they get customized (by replacing raw memory management functions) there's absolutely no way to even begin to predict what might happen if they get mixed.
It seems that your question is really "why heap corruption doesn't happen?". The answer to that one is "because the heap manager keeps track of allocated block sizes". Let's go back to C for a minute: if you want to allocate a single int in C you would do int* p = malloc(sizeof(int)), if you want to allocate array of size n you can either write int* p = malloc(n*sizeof(int)) or int* p = calloc(n, sizeof(int)). But in any case you'll free it by free(p), no matter how you allocated it. You never pass size to free(), free() just "knows" how much to free, because the size of a malloc()-ed block is saved somewhere "in front" of the block. Back to C++, new/delete and new[]/delete[] are usually implemented in terms of malloc (although they don't have to be, you shouldn't rely on that). This is why new[]/delete combination doesn't corrupt the heap - delete will free the right amount of memory, but, as explained by everyone before me, you can get leaks by not calling the right number of destructors.
That said, reasoning about undefined behavior in C++ is always pointless exercise. Why does it matter if new[]/delete combination happens to work, "only" leaks or causes heap corruption? You shouldn't code like that, period! And, in practice, I would avoid manual memory management whenever possible - STL & boost are there for a reason.
If the non-trivial destructor that are not called for all but the first element in the array are supposed to free some memory you get a memory leak as these objects are not cleaned up properly.
It will lead to a leak in ALL implementations of C++ in any case where the destructor frees memory, because the destructor never gets called.
In some cases it can cause much worse errors.
memory leak might happen if new() operator is overridden but new[] is not. same goes to the delete / delete[] operator
Apart from resulting in undefined behavior, the most straightforward cause of leaks lies in the implementation not calling the destructor for all but the first object in the array. This will obviously result in leaks if the objects have allocated resources.
This is the simplest possible class I could think of resulting in this behaviour:
struct A {
char* ch;
A(): ch( new char ){}
~A(){ delete ch; }
};
A* as = new A[10]; // ten times the A::ch pointer is allocated
delete as; // only one of the A::ch pointers is freed.
PS: note that constructors fail to get called in lots of other programming mistakes, too: non-virtual base class destructors, false reliance on smart pointers, ...
Late for an answer, but...
If your delete mechanism is simply to call the destructor and put the freed pointer, together with the size implied by sizeof, onto a free stack, then calling delete on a chunk of memory allocated with new[] will result memory being lost -- but not corruption.
More sophisticated malloc structures could corrupt on, or detect, this behaviour.
Why can't the answer be that it causes both?
Obviously memory is leaked whether heap corruption occurs or not.
Or rather, since I can re-implement new and delete..... can't it not cause anything at all. Technically I can cause new and delete to perform new[] and delete[].
HENCE: Undefined Behavior.
I was answering a question which was marked off as a duplicate, so i'll just copy it here in case it metters. It was said well before me the way memory allocation works, i`ll just explain the cause & effects.
Just a little thing right off google: http://en.cppreference.com/w/cpp/memory/new/operator_delete
Anyhow, delete is a function for a single object. It frees the instance from the pointer, and leaves;
delete[] is a function used in order to deallocate arrays. That means, it doesnt just free the pointer; It declares the whole memory block of that array as garbage.
That's all cool in practice, but you tell me your application works. You are probably wondering... why?
The solution is C++ does not fix memory leaks. If you`ll use delete without the parenthesises, it'll delete just the array as an object - a proccess which might cause a memory leak.
cool story, memory leak, why should i care?
Memory leak happens when allocated memory doesn't get deleted. That memory then requires unneccessary disk-space, which will make you lose useful memory for pretty much no reason. That's bad programming, and you should probably fix it in your systems.
When, if ever, can delete and free be used interchangeably in C++?
My concern is as follows: Say there is an incorrect mixup in the use of malloc/ free and
new/ delete (not to mention new[]/ delete[]). However delete and free doing the same thing;
Fortuitously so this goes uncaught in testing. Later this may lead to a crash in production.
How can I enforce some kind of check to prevent this? Can I be warned if the two are mixed up?
If not at compile time, perhaps some code instrumentation at run time? How would I approach
this?
The intention of this question is to find ways to avoid inadvertent mix up in the usages.
The easy way to not get them mixed up is never to use malloc(), then you will never be tempted to call free(). The infrastructure to create to avoid this problem is called "code review", though in this case a quick "grep malloc(" or "grep free(" on the codebase will probably suffice.
Never. If it works it's by sheer accident of implementation. Do not rely on this behavior.
To answer the second question, if you control both malloc/free and operator new/delete, you can stash extra information to associate with pointers returned by both that tell you how they were allocated. When a pointer is passed to free or operator delete, check to see that it was allocated by the appropriate function. If not, assert or raise an exception or do whatever it is you do to report the mismatch.
Usually this is done by allocating extra memory, e.g., given malloc(size) or operator new(size), you allocate size + additional space and shove extra information in there.
The only way you can ensure you never get them mixed up is by either:
Never using malloc/free in the first place, or
Rely on RAII for your memory allocations. Protect every memory allocation in a RAII object which ensures the memory get correctly and consistently freed when it goes out of scope, or wrap the allocation in a smart pointer.
Manually calling delete or free is just an invitation for bugs.
You should always use new and delete in C++, as only new and delete will call the object's constructor and destructor.
If you find you must use both (for instance, if you're interfacing with a C library), thorough code reviews should carefully scrutinize any uses of free() to determine whether or not they correspond to a malloc(), and whether or not they are being used in a C context.
If I had to codify it, I'd put in the style guide something like this:
free() may be called only on a private pointer field of an object.
malloc()ed buffers (or buffers returned from a C API which caller must free()) must be assigned to a private pointer field of an object.
private pointer fields which hold free()-able buffers must only be used for that purpose.
if you use hungarian notation, add a letter for it (and if you don't, don't).
generally free() will be called only in a destructor, with exceptions when the free()-able buffer is replaced during the lifetime of the object. In that case you can call free() on a value recently copied out of a private field during replacement, rather than on the field value directly.
In other words, stick a wrapper around anything that uses malloc/free. This wrapper could be a single template everyone uses, or you could allow smart pointers with the deletor function set to free(). Then in code review, if you see a call to malloc/free anywhere else, it's wrong.
Basically the best way to stop this being a problem is to be on top of your resource handling in general. In C, people do not have a major problem with accidentally calling free() on streams instead of fclose().
You should always use delete or delete[] when freeing things allocated with new. The same goes for malloc and free.
If using free for deleting new:ed classes the destructor won't be properly called.
Also new and delete doesn't necessarily use malloc/free for its allocations so you might end up corrupting your heap as well.
always use delete for things allocated with new, delete [] for things allocated with new [] and free() for things allocated using malloc().
new and new[] allocate from different heaps than malloc() and using the wrong free()/delete will attempt to deallocate from the wrong heap.
Never mix new/delete with new[]/delete[] or with malloc()/free(). Atop of this, the use of malloc()/free() in C++ is questionable at least.
The easiest way to make sure you never do this accidentally is to not to manage memory manually. In C++ there strings and other containers as well as smart pointers to take care of memory management. There simply isn't a need to do this manually. (I'm not sure whether the last time I remember me typing delete really was the last time I typed it. But if my memory doesn't fail me, this must have been 2001 or 2002.)
You could write your own version of the functions which allocate some extra memory in new/new[]/malloc to track which one did the allocation. Then check in delete/delete[]/free that the right function was used to re-claim the memory. You can also check for things like mixing new[] and delete (without the []) this way. You might want to only use these versions in the debug build.
You could try running valgrind on your application to see if it catches anything. The manual specifically mentions its ability to catch uses of the wrong deallocation function for a block of memory. I don't think there's a way to do this at compile time, however.
One way to avoid misusage of malloc/free or new/delete is that DO NOT call these function directly, call them by a wrapper layer. In the wrapper layer, you can manage the pointers distributed by your own hands and guarantee that misusage will get an error explicitly.
Why dont you just count the total number of malloc statments and tally that with the total count of free? Do the same for New and delete. The process can be automated with regular expressions.
One thing I've done relating to this is to reimplement malloc and free
in C++, so that it calls new/delete under the hood. It's not
optimal, but it serves well enough on a smaller project.