Related
From what is written here, new allocates in free store while malloc uses heap and the two terms often mean the same thing.
From what is written here, realloc may move the memory block to a new location. If free store and heap are two different memory spaces, does it mean any problem then?
Specifically I'd like to know if it is safe to use
int* data = new int[3];
// ...
int* mydata = (int*)realloc(data,6*sizeof(int));
If not, is there any other way to realloc memory allocated with new safely? I could allocate new area and memcpy the contents, but from what I understand realloc may use the same area if possible.
You can only realloc that which has been allocated via malloc (or family, like calloc).
That's because the underlying data structures that keep track of free and used areas of memory, can be quite different.
It's likely but by no means guaranteed that C++ new and C malloc use the same underlying allocator, in which case realloc could work for both. But formally that's in UB-land. And in practice it's just needlessly risky.
C++ does not offer functionality corresponding to realloc.
The closest is the automatic reallocation of (the internal buffers of) containers like std::vector.
The C++ containers suffer from being designed in a way that excludes use of realloc.
Instead of the presented code
int* data = new int[3];
//...
int* mydata = (int*)realloc(data,6*sizeof(int));
… do this:
vector<int> data( 3 );
//...
data.resize( 6 );
However, if you absolutely need the general efficiency of realloc, and if you have to accept new for the original allocation, then your only recourse for efficiency is to use compiler-specific means, knowledge that realloc is safe with this compiler.
Otherwise, if you absolutely need the general efficiency of realloc but is not forced to accept new, then you can use malloc and realloc. Using smart pointers then lets you get much of the same safety as with C++ containers.
The only possibly relevant restriction C++ adds to realloc is that C++'s malloc/calloc/realloc must not be implemented in terms of ::operator new, and its free must not be implemented in terms of ::operator delete (per C++14 [c.malloc]p3-4).
This means the guarantee you are looking for does not exist in C++. It also means, however, that you can implement ::operator new in terms of malloc. And if you do that, then in theory, ::operator new's result can be passed to realloc.
In practice, you should be concerned about the possibility that new's result does not match ::operator new's result. C++ compilers may e.g. combine multiple new expressions to use one single ::operator new call. This is something compilers already did when the standard didn't allow it, IIRC, and the standard now does allow it (per C++14 [expr.new]p10). That means that even if you go this route, you still don't have a guarantee that passing your new pointers to realloc does anything meaningful, even if it's no longer undefined behaviour.
In general, don't do that. If you are using user defined types with non-trivial initialization, in case of reallocation-copy-freeing, the destructor of your objects won't get called by realloc. The copy constructor won't be called too, when copying. This may lead to undefined behavior due to an incorrect use of object lifetime (see C++ Standard §3.8 Object lifetime, [basic.life]).
1 The lifetime of an object is a runtime property of the object. An object is said to have non-trivial initialization if it is of a class or aggregate type and it or one of its members is initialized by a constructor other than a trivial default constructor. [ Note: initialization by a trivial copy/move constructor is non-trivial initialization. —end note ]
The lifetime of an object of type T begins when:
— storage with the proper alignment and size for type T is obtained, and
— if the object has non-trivial initialization, its initialization is complete.
The lifetime of an object of type T ends when:
— if T is a class type with a non-trivial destructor (12.4), the destructor call starts, or
— the storage which the object occupies is reused or released.
And later (emphasis mine):
3 The properties ascribed to objects throughout this International Standard apply for a given object only during its lifetime.
So, you really don't want to use an object out of its lifetime.
It is not safe, and it's not elegant.
It might be possible to override new/delete to support the reallocation, but then you may as well consider to use the containers.
In general, no.
There are a slew of things which must hold to make it safe:
Bitwise copying the type and abandoning the source must be safe.
The destructor must be trivial, or you must in-place-destruct the elements you want to deallocate.
Either the constructor is trivial, or you must in-place-construct the new elements.
Trivial types satisfy the above requirements.
In addition:
The new[]-function must pass the request on to malloc without any change, nor do any bookkeeping on the side. You can force this by replacing global new[] and delete[], or the ones in the respective classes.
The compiler must not ask for more memory in order to save the number of elements allocated, or anything else.
There is no way to force that, though a compiler shouldn't save such information if the type has a trivial destructor as a matter of Quality of Implementation.
Yes - if new actually called malloc in the first place (for example, this is how VC++ new works).
No otherwise. do note that once you decide to reallocate the memory (because new called malloc), your code is compiler specific and not portable between compilers anymore.
(I know this answer may upset many developers, but I answer depends on real facts, not just idiomaticy).
That is not safe. Firstly the pointer you pass to realloc must have been obtained from malloc or realloc: http://en.cppreference.com/w/cpp/memory/c/realloc.
Secondly the result of new int [3] need not be the same as the result of the allocation function - extra space may be allocated to store the count of elements.
(And for more complex types than int, realloc wouldn't be safe since it doesn't call copy or move constructors.)
You may be able to (not in all cases), but you shouldn't. If you need to resize your data table, you should use std::vector instead.
Details on how to use it are listed in an other SO question.
These function is mostly used in C.
memset sets the bytes in a block of memory to a specific value.
malloc allocates a block of memory.
calloc, same as malloc. Only difference is that it initializes the bytes to zero.
In C++ the preferred method to allocate memory is to use new.
C: int intArray = (int*) malloc(10 *sizeof(int));
C++: int intArray = new int[10];
C: int intArray = (int*) calloc(10 *sizeof(int));
C++: int intArray = new int10;
I am using few library functions that return a pointer created either using malloc or new.
So, I have my own customer deallocator based on what type of allocation was used.
E.g
shared_ptr<int> ptr1(LibFunctA(), &MallocDeleter); //LibFunctA returns pointer created using malloc
shared_ptr<int> ptr2(LibFunctB(), &newDeleter); //LibFunctB returns pointer created using new
Now, I understand this is a very naive use of deallocator above but what other scenarios is it heavily used for ?
Also, how can one use a customer allocator ? I tried to assign a custom allocator as below but now how do I actually get it called ? Where does this kind of feature help ?
shared_ptr<int> ptr3(nullptr_t, &CustomDeleter, &CustomAllocator); //assume both functs are defined somewhere.
I don't see anything "naive" about using deleters that way. It is the main purpose of the feature after all; to destroy pointer objects that aren't allocated using the standard C++ methods.
Allocators are for when you need control of how the shared_ptr's control block of memory is allocated and deleted. For example, you might have a pool of memory that you want these things to come from, or if you're in a memory-limited situation where allocation of memory via new is simply not acceptable. And since the type of the control block is up to shared_ptr, there's no other way to be able to control how it is allocated except with some kind of allocator.
Custom deleters for shared_ptr are very useful for wrapping some (usually) C resource that you need to later call a freeing function on. For example, you might do something like:
shared_ptr<void> file(::CreateFileW(...), ::CloseHandle);
Examples like this abound in C libraries. This saves from having to manually free the resource later and take care of possible exceptions and other nasties.
I think the custom allocator will be used to allocate space for the "shared count" object, that stores a copy of the deallocator (deleter) and the reference counter.
As for what a custom deleter can be used for...
One use was already mentioned: make shared_ptr compatible with objects that must be deleted by some special function (like FILE which is deleted by fclose), without having to wrap it into a helper-class that takes care of the proper deletion.
Another use for a custom deleter is pools. The pool can hand out shared_ptr<T> that were initialized with a "special" deleter, which doesn't really delete anything, but returns the object to the pool instead.
And one other thing: the deleter is already necessary to implement some shared_ptr features. E.g. the type that's deleted is always fixed at creation time, and independent of the type of the shared_ptr that's being initialized.
Vou can create a shared_ptr<Base> by actually initializing it with a Derived. shared_ptr guarantees that when the object is deleted, it will be deleted as a Derived, even if Base does not have a virtual dtor. To make this possible, shared_ptr already has to store some information about how the object shall be deleted. So allowing the user to specify a completely custom deleter doesn't cost anything (in terms of runtime performance), and doesn't require much additional code either.
There are probably dozens of other scenarios where one can make good use of the custom deleter, that's just what I have come up with so far.
I’ve heard some words of warning against shipping an object to another memory location via memcpy, but I don’t know the specific reasons. Unless its contained members do tricky things that depend on memory location, this should be perfectly safe … or not?
EDIT: The contemplated use case is a data structure like a vector, which stores objects (not pointers to objects) in a continuous chunk of memory (i.e. an array). To insert a new object at the n-th position, all objects starting at position n and beyond will need to be moved to make room for the object to be inserted.
One primary reason why you should not do this is destructors. When you memcpy a C++ object to another location in memory, you will end up with 2 versions of the object in memory for which only 1 constructor has been run. This will destroy the resource freeing logic of pretty much every single C++ class out there.
It's not allowed by the language specification. It is undefined behavior. That is, ultimately, what's wrong with it. In practice, it tends to mess with virtual function calls, and it means the destructor will be run twice (and more often than the constructors), member objects are shallow copied (so if, for example, if you try this stunt with a std::vector, it blows up, as multiple objects end up pointing to the same internal array.)
The exception is POD types. They don't have (copy) constructors, destructors, virtual functions, base classes or anything else that might cause this to break, so with those, you're allowed to use memcpy to copy them.
For the sake of discussion, I assume you mean moving to mean that the original object "dropped" (is no longer used, didn't have it's destructor run) rather than have two copies (which would lead to a lot more problems, reference counts being off, etc). I generally refer to the property of being able to do this being bitwise movable.
In the code bases I work on, the majority of objects are bitwise movable, as they don't store self references. However, some data structures aren't bitwise movable (I believe that gcc's std::set wasn't bitwise movable; other examples would be linked list nodes). In general, I would avoid attempting to use this property as it can lead to some very hard to debug errors, and prefer the object oriented calling copy constructors.
Edited to add:
There seems to be some confusion on how/why someone would do this: here's a comment I made on the how:
Normally, I see the above on alternate
implementations of vector. The memory
is allocated via
malloc(sizeof(Class)*size) and the
objects are constructed in place via
explicitly called constructors and
destructors. Sometimes (like during
reallocation) they have to be moved,
so the option is to do std::vector's
repeated calling of copy constructors
on new memory and destructors on the
old, or use memcopy and just "free"
the old block. Most times the latter
just "works", but doesn't for all
objects.
As to why, a memcopy (or realloc) approach can be significantly faster.
Yes, it invokes undefined behavior, but it also just tends to work for a majority of objects. Some people consider the speed worth it. If you were really set on using this approach, I would suggest implementing a bitwise_movable type trait to allow types this works for to be whitelisted, and fall back on the traditional copy for objects not in the whitelist, much like the example here.
If the object had no pointers within it, and no virtual functions, no children with any of the same, you might get away with it. It is not recommended!!!
This should be done using a copy or deepcopy function or overridden operators.
In the method you would call a new contructor and copy it's contained data items one by one.
for a shallow copy you would copy pointers / references so you would have two object pointing to the same contained elements.... a potential memory leak nightmare.
for a deep copy you would traverse the contained objects and references making new copies of them also.
To move an object you would copy it and delete the original.
Short answer: std::memcpy() is for moving memory, not for moving objects. Using it nonetheless will invoke undefined behavior.
Somewhat longer answer: A C++ object that isn't a POD might contain resources that need to be freed and which are kept in handles that cannot be easily copied. (A popular resource is memory, where the handle is a pointer.) It also might contain stuff inserted by the implementation (virtual base class instance pointers) that shouldn't be copied as if it were memory.
The only right way to move an object in C++98 and C++03 is to copy-construct it to its new location and invoke the destructor in the old. (In C++1x there will be move semantic so things might get more interesting in certain cases.)
Off the top of my head : If you just do a memcpy you end up doing a shallow copy. If you need a deep-copy then this won't work.
What's wrong with the copy constructor and the assignment operators anyway?
In general (and in all languages, not just C++), in order to safely move an object, you also need to rewrite ALL pointers/references to that object to point at the new location. That's a problem in C++, because there's no easy way to tell if any object in the system has a 'hidden' pointer to the object you're moving. As you've noted, some classes may contain hidden pointers to themselves. Other classes may have hidden pointers in a factory object that tracks all instances. Its also possible for seemingly unrelated classes to cache pointers to objects for various reasons of their own.
The only way to do it safely is if you have some sort of reflective access to all objects in the system so that you can find all the pointers to the object and rewrite them. This is a potentially very expensive operation in any case, so systems that need it (such as copying garbage collectors) tend to be very carefully organized to do the copying of many objects at once and/or bound the places that need to be searched for pointers with write barriers and such.
I am writing a template class that takes as an input a pointer and stores it. The pointer is meant to point to an object allocated by another class, and handed to the this containing class.
Now I want to create a destructor for this container. How should I free the memory pointed to by this pointer? I have no way of knowing a priori whether it is an array or a single element.
I'm sort of new to C++, so bear with me. I've always used C, and Java is my OO language of choice, but between wanting to learn C++ and the speed requirements of my project, I've gone with C++.
Would it be a better idea to change the container from a template to a container for an abstract class that can implement its own destructor?
If you don't know whether it was allocated with new or new[], then it is not safe to delete it.
Your code may appear to work. For example, on one platform I work on, the difference only matters when you have an array of objects that have destructors. So, you do this:
// by luck, this works on my preferred platform
// don't do this - just an example of why your code seems to work
int *ints = new int[20];
delete ints;
but then you do this:
// crashes on my platform
std::string *strings = new std::string[10];
delete strings;
You must document how this class expects to be used, and always allocate as expected. You can also pass a flag to the object specifying how it should destroy. Also look at boost's smart pointers, which can handle this distinction for you.
Short answer:
If you use [] with new you want to use [] with delete.
//allocate some memory
myObject* m = new myObject[100];
//later on...destructor...
delete m; //wrong
delete[] m; //correct
That was the bare bones, the other thing you could look at is boost. Also quite difficult to answer considering you are not sure if its an array or single object. You could check this though via a flag telling your app whether to use delete or delete[].
As a general development rule, you should stick to a design where the class which calls new should also call delete
You shouldn't delete it at all. If your class takes an already initialized pointer, it is not safe to delete it. It might not even point to an object on the heap; calling either delete or delete[] could be disastrous.
The allocation and deallocation of memory should happen in the same scope. Which ever code owns and initializes the instance of your class is also presumably responsible for initializing and passing in the pointer, and that is where your delete should be.
Use delete if you allocated with new.
Use delete[] if you allocated with new[].
After these statements, if you still have a problem (maybe you want to delete an object that was created by someone else), then you are breaking the third rule:
Always delete what you created. Corollary, never delete what you did not create.
(Moving my comment into an answer, by request.)
JonH's answer is right (about using array destruction only when you used array construction), so perhaps you should offer templates: one for arrays, one not.
The other answer is to avoid arrays and instead expect a single instance that may or may not be a proper collection that cleans up after itself, such as vector<>.
edit
Stealing blatantly from Roger Pate, I'll add that you could require the use of a smart pointer, which amounts to a single-item collection.
If you have a class that takes a pointer it's going assume ownership of, then the contract for the use of the class needs to include one of a couple things. Either:
the interface needs to indicate how the object the pointer is pointing to was allocated so the new owner can know how to safely deallocate the object. This option has the advantage of keeping things simple (on one level anyway), but it's not flexible - the class can't handle taking ownership of static objects as well as dynamically allocated objects.
or
the interface needs to include a mechanism where a deallocation policy can be specified by whatever is giving the pointer to the class. This can be as simple as providing a mechanism to pass in a functor (or even a plain old function pointer) that will be called to deallocate the object (preferably in the same function/constructor that passes in the pointer itself). This makes the class arguably more complicated to use (but having a default policy of calling delete on the pointer, for example, might make it as easy to use as option 1 for the majority of uses). Now if someone wants to give the class a pointer to a statically allocated object, they can pass in a no-op functor so nothing happens when the class wants to deallocates it, or a functor to a delete[] operation if the object was allocated by new[], etc.
Since pointer in C++ does not tell us how it was allocated, yes, there's no way to decide what deallocation method to use. The solution is to give the choice to the user that hopefully knows how the memory was allocated. Take a look at Boost smart ptr library, especially at shared_ptr constructor with second parameter, for a great example.
A smart pointer like boost shared_pointer already has this covered, could you use it? linky
Put simply, given only a pointer to dynamically allocated memory there is no way of determining how to de-allocate it safely. The pointer could have been allocated in any of the the following ways:
using new
using new []
using malloc
using a user defined function
etc.
In all cases before you can deallocate the memory you have to know how it was allocated.
I know that when delete [] will cause destruction for all array elements and then releases the memory.
I initially thought that compiler wants it just to call destructor for all elements in the array, but I have also a counter - argument for that which is:
Heap memory allocator must know the size of bytes allocated and using sizeof(Type) its possible to find no of elements and to call appropriate no of destructors for an array to prevent resource leaks.
So my assumption is correct or not and please clear my doubt on it.
So I am not getting the usage of [] in delete [] ?
Scott Meyers says in his Effective C++ book: Item 5: Use the same form in corresponding uses of new and delete.
The big question for delete is this: how many objects reside in the memory being deleted? The answer to that determines how many destructors must be called.
Does the pointer being deleted point to a single object or to an array of objects? The only way for delete to know is for you to tell it. If you don't use brackets in your use of delete, delete assumes a single object is pointed to.
Also, the memory allocator might allocate more space that required to store your objects and in this case dividing the size of the memory block returned by the size of each object won't work.
Depending on the platform, the _msize (windows), malloc_usable_size (linux) or malloc_size (osx) functions will tell you the real length of the block that was allocated. This information can be exploited when designing growing containers.
Another reason why it won't work is that Foo* foo = new Foo[10] calls operator new[] to allocate the memory. Then delete [] foo; calls operator delete[] to deallocate the memory. As those operators can be overloaded, you have to adhere to the convention otherwise delete foo; calls operator delete which may have an incompatible implementation with operator delete []. It's a matter of semantics, not just keeping track of the number of allocated object to later issue the right number of destructor calls.
See also:
[16.14] After p = new Fred[n], how does the compiler know there are n objects to be destructed during delete[] p?
Short answer: Magic.
Long answer: The run-time system stores the number of objects, n, somewhere where it can be retrieved if you only know the pointer, p. There are two popular techniques that do this. Both these techniques are in use by commercial-grade compilers, both have tradeoffs, and neither is perfect. These techniques are:
Over-allocate the array and put n just to the left of the first Fred object.
Use an associative array with p as the key and n as the value.
EDIT: after having read #AndreyT comments, I dug into my copy of Stroustrup's "The Design and Evolution of C++" and excerpted the following:
How do we ensure that an array is correctly deleted? In particular, how do we ensure that the destructor is called for all elements of an array?
...
Plain delete isn't required to handle both individual objects an arrays. This avoids complicating the common case of allocating and deallocating individual objects. It also avoids encumbering individual objects with information necessary for array deallocation.
An intermediate version of delete[] required the programmer to specify the number of elements of the array.
...
That proved too error prone, so the burden of keeping track of the number of elements was placed on the implementation instead.
As #Marcus mentioned, the rational may have been "you don't pay for what you don't use".
EDIT2:
In "The C++ Programming Language, 3rd edition", §10.4.7, Bjarne Stroustrup writes:
Exactly how arrays and individual objects are allocated is implementation-dependent. Therefore, different implementations will react differently to incorrect uses of the delete and delete[] operators. In simple and uninteresting cases like the previous one, a compiler can detect the problem, but generally something nasty will happen at run time.
The special destruction operator for arrays, delete[], isn’t logically necessary. However, suppose the implementation of the free store had been required to hold sufficient information for every object to tell if it was an individual or an array. The user could have been relieved of a burden, but that obligation would have imposed significant time and space overheads on some C++ implementations.
The main reason why it was decided to keep separate delete and delete[] is that these two entities are not as similar as it might seem at the first sight. For a naive observer they might seem to be almost the same: just destruct and deallocate, with the only difference in the potential number of objects to process. In reality, the difference is much more significant.
The most important difference between the two is that delete might perform polymorphic deletion of objects, i.e. the static type of the object in question might be different from its dynamic type. delete[] on the other hand must deal with strictly non-polymorphic deletion of arrays. So, internally these two entities implement logic that is significantly different and non-intersecting between the two. Because of the possibility of polymorphic deletion, the functionality of delete is not even remotely the same as the functionality of delete[] on an array of 1 element, as a naive observer might incorrectly assume initially.
Contrary to the strange claims made in some other answers, it is, of course, perfectly possible to replace delete and delete[] with just a single construct that would branch at the very early stage, i.e. it would determine the type of the memory block (array or not) using the household information that would be stored by new/new[], and then jump to the appropriate functionality, equivalent to either delete or delete[]. However, this would be a rather poor design decision, since, once again, the functionality of the two is too different. Forcing both into a single construct would be akin to creating a Swiss Army Knife of a deallocation function. Also, in order to be able to tell an array from a non-array we'd have to introduce an additional piece of household information even into a single-object memory allocations done with plain new. This might easily result in notable memory overhead in single object allocations.
But, once again, the main reason here is the functional difference between delete and delete[]. These language entities possess only apparent skin-deep similarity that exists only at the level of naive specification ("destruct and free memory"), but once one gets to understand in detail what these entities really have to do one realizes that they are too different to be merged into one.
P.S. This is BTW one of the problems with the suggestion about sizeof(type) you made in the question. Because of the potentially polymorphic nature of delete, you don't know the type in delete, which is why you can't obtain any sizeof(type). There are more problems with this idea, but that one is already enough to explain why it won't fly.
The heap itself knows the size of allocated block - you only need the address. Look like free() works - you only pass the address and it frees memory.
The difference between delete (delete[]) and free() is that the former two first call the destructors, then free memory (possibly using free()). The problem is that delete[] also has only one argument - the address and having only that address it need to know the number of objects to run destructors on. So new[] uses som implementation-defined way of writing somewhere the number of elements - usually it prepends the array with the number of elements. Now delete[] will rely on that implementation-specific data to run destructors and then free memory (again, only using the block address).
delete[] just calls a different implementation (function);
There's no reason an allocator couldn't track it (in fact, it would be easy enough to write your own).
I don't know the reason they did not manage it, or the history of the implementation, if I were to guess: Many of these 'well, why wasn't this slightly simpler?' questions (in C++) came down to one or more of:
compatibility with C
performance
In this case, performance. Using delete vs delete[] is easy enough, I believe it could all be abstracted from the programmer and be reasonably fast (for general use). delete[] only requires only a few additional function calls and operations (omitting destructor calls), but that is per call to delete, and unnecessary because the programmer generally knows the type he/she is dealing with (if not, there's likely a bigger problem at hand). So it just avoids calling through the allocator. Additionally, these single allocations may not need to be tracked by the allocator in as much detail; Treating every allocation as an array would require additional entries for count for trivial allocations, so it is multiple levels of simple allocator implementation simplifications which are actually important for many people, considering it is a very low level domain.
This is more complicated.
The keyword and the convention to use it to delete an array was invented for the convenience of implementations, and some implementations do use it (I don't know which though. MS VC++ does not).
The convenience is this:
In all other cases, you know the exact size to be freed by other means. When you delete a single object, you can have the size from compile-time sizeof(). When you delete a polymorphic object by base pointer and you have a virtual destructor, you can have the size as a separate entry in vtbl. If you delete an array, how would you know the size of memory to be freed, unless you track it separately?
The special syntax would allow tracking such size only for an array - for instance, by putting it before the address that is returned to the user. This takes up additional resources and is not needed for non-arrays.