Assuming I have some array in heap doesn't matter constructed by malloc or new. I need the most efficient way to enlarge it. I mean if it has enough free space which lying after already allocated data can I keep my data untouched. Is it possible to maintain in C++?
Does realloc work in such manner?
Yes, realloc is what you are looking for. Note that it won't work with new, you will have to use malloc (or, say, calloc). Also, sometimes it is just impossible to extend memory, so realloc will try to do it for you, but if it couldn't — it will resort to allocating new memory, copying your contents to a new place and freeing the old memory.
yes, realloc works like that, though the link says that it is not guaranteed, I think this is for cases where memory is fragmented and there is not enough room to expand the memory block in-situ.
Related
Quick question here:
template <class T>
T* allocate(std::size_t n){
return new T[n];
}
So in the above code, when n==1, we're doing new T[1], so I have two worries:
1.I heard that when allocating array, extra memory are used to store the length of the array (unsure though), so would this be malicious when having new T[1] a lot, wasting a lot of memory?
2.Should I free this using delete[] or simply delete?
Yes, in a typical implementation when you use new T[n] some extra memory will be required to store the exact length of the array, but only for types with non-trivial destructors.
E.g. in a typical implementation new int[1] carries no memory overhead compared to new int, but new std::string[1] will carry memory overhead compared to new std::string.
The extra memory is just an extra size_t field, meaning that percentage-wise it depends on the size of the object you allocated. If sizeof(T) is comparable to sizeof(size_t) the overhead might be considerable.
But it all might also depend on the additional details of the implementation-specific memory allocation mechanism.
In other words, if this a part of application-specific code it makes sense to try it and see whether it has any negative impact on your program's memory consumption. Maybe it is not a problem at all. But if you are writing a generic library, then things like that are worth paying attention to.
Yes, you should use delete [].
RAM is cheap. It is true that if all your program does, all the time, is allocate arrays of 1 object, over and over again, then this would be slightly wasteful, true. But an occasional allocation of a 1-element array is not going to be the end of the world.
Use delete[]. If you used new[], then you have to use delete[]. Doesn't matter how much was newed. This is a fairly cut-and-try rule. No exceptions.
No memory is used to store the length of the array, or at least no memory you are allowed to access. The memory manager that granted you the memory likely has some hidden bookkeeping. Should you find this bookkeeping, leave it alone. Anything you do with it is certainly non-portable. It is also possible you will be provided memory in fixed size blocks. You will be given a block that is at least the size of T. Any extra memory in this block is effectively wasted.
Anything allocated with new[] must be released with delete[] or you you invoke undefined behaviour. Once again, do not mess with the low-level memory manager.
This question already has answers here:
How do malloc() and free() work?
(13 answers)
Closed 7 years ago.
In C++, how may operator new save information that a piece of memory is allocated? AFAIK, it does not work for constant time and have to search for free memory in heap. Or, maybe, it is not about C++, but about OS?
P.S. I do not know whether it is specified by standard or not, whether it is managed by OS or by C++, but how may it in fact be implemented?
There's no simple, standard answer. Most implementations of operator
new/operator delete ultimately forward to malloc/free, but there are a
lot of different algorithms which can be used for those. The only thing that's
more or less universal is that allocation will typically allocate a little bit
more than requested, and use the extra memory (normally in front of the address
actually returned) to maintain some additional information: either the actual
size allocated or a pointer to the end of the block (or the start of the next
block). Except that some algorithms will keep allocations of the same size
together, and be able to determine the size from the address. There is no
single answer to your question.
new is oftentimes implemented on basis of malloc/free.
How does malloc/free implement it? The answer is: It depends on the implementation. Surprisingly: Malloc oftentimes does not keep track of the allocated blocks at all! The only thing, malloc is doing most of the time, is adding a little bit of information containing the size of the block "before" the allocated block. Meaning, that when you allocate 40 bytes, it will allocate 44 bytes (on 32bit machines) and writes the size in the first 4 bytes. It will return the address of this chunk+4 to you.
Malloc/free keeps track of a freelist. A freelist is a list of freed memory chunks that is not (yet) be given back to the operating system. Malloc searches the freelist, when a new block is needed and when a fitting block is available uses that.
But a more exhausting answer about malloc/free, I have given here:
How do malloc() and free() work?
One additional information:
One implication of the fact, that many allocators don't track allocated blocks: When you return memory by free or delete and pass a pointer in, that was not allocated before, you will corrupt your heap, since the system is not able to check if the pointer is valid. The really ugly thing about it is, that in such a case, your program will not dump immediately, but any time after the error-cause occured ... and thus this error would be really ugly to find. That is one reason, memory handling in C/C++ is so hard!
new maintains a data structure to keep track of individually allocated blocks. There are plenty of ways for doing that. Usually, some kind of linked list is used.
Here a small article to illustrate this.
For example I have an array of 200 integers. What I want to do is convert it to two arrays of 80 integers, removing the 40 integers in between. The goal of course is to use the existing memory block without allocating two new arrays of length 80 integers and copying from the first array, what I want is to cut the initial array from 80 to 120 and treat what is left as two separate arrays.
Move semantics uses a similar low level approach to avoid unnecessary copy of rvalues, so my question is if there is a low level approach to achieve a similar effect but assign the already existing data to multiple objects.
For example assigning pointers to the raw memory addresses of where the cuts are and the new elements begin, forcing them to act as arrays which use the same data, already allocated and filled in by the initial array?
Naturally, I could also delete the initial array and get its address and use that to assign its memory area to a new element, but is it possible to tell C++ on which exact address to allocate a new object? And also, is there a way to guarantee the data of the initial array wont be corrupted in between its deletion and the reallocation of that same memory area to a new object?
Such an approach is absent from any of the books on c++ I've read, but I get the feeling there might very well be some low level trickery to achieve the desired result.
Yes, this is possible using placement-new. However, there is no way to guarantee that the content of a memory-location will not be changed between delete and a reallocation. Anyway, placement-new only enables you to construct an object in memory that you already own. So you'd have to allocate a pool of memory and then do your own memory management inside that pool.
You can use placement new but it's generally not a good idea to go this low-level, unless you are really constrained.
Using it is a good idea when:
You implement a memory pool
You need to write to a precise memory location because the platform requires that (for example, memory mapped I/O on some embedded device).
Disadvantages of using it:
You need to make sure that the allocated memory is large enough to hold the object.
You need to explicitly call the destructor when the object is not needed anymore.
is there a way to guarantee the data of the initial array wont be corrupted in between its deletion and the reallocation of that same memory area to a new object
No, that's up to the OS. It might be possible with a platform-specific API if it exists.
What I'd simply do is pass pointers to the relevant slices:
void i_need_80(int arr[80]); // same as "int*"
std::array<int, 200> the_array;
i_need_80(the_array);
i_need_80(the_array + 120);
In place of std::array<int, 200> you can also have a dynamic std::vector<int> v, in which case you'd say v.data() and v.data() + 120.
Don't use new.
The only thing you can do is call realloc() to resize (and possibly move) a block allocated with malloc() of realloc(). If the block was allocated using new[], even that isn't possible. The things you want require a rewrite (re-invention) of the memory management functions malloc(), free() and realloc().
This question was asked to me in an interview.
Suppose char *p=malloc(n) assigns more than n,say N bytes of memory are allocated and free(p) is used to free the memory allocated to p.
can heap manager perform such faulty allocation ?
what happens now, will n bytes are freed or N bytes are freed?
is there any method to find how much memory is freed?
EDIT
is there any method to find how much memory is freed?
better than nothing,
mallinfo() can shed some light as pointed by "Fred Larson"
Yes, that's what happens almost every time do you a malloc(). The malloc block header contains information about the the size of the block, and when free() is called, it returns that amount back to the heap. It's not faulty, it's expected operation.
A simple implementation might, for instance, store just the size of the block in the space immediately preceding the returned pointer. Then, free() would look something like this:
void free(void *ptr)
{
size_t *size = (size_t *)ptr - 1;
return_to_heap(ptr, *size);
}
Where return_to_heap() is used here to mean a function that does the actual work of returning the specified block of memory to the heap for future use.
Yes, the heap manager is allowed to return a block of more than n bytes. It is completely safe (and required!) to free the returned pointer using free, and free will deallocate all of it.
Many heap implementations track their allocations by inserting blocks of metadata into the heap. free will look for that metadata to determine how much memory to deallocate. This is implementation-specific, though, so there's no way to know how much malloc gave you, and generally, you shouldn't care.
Generally the heap manager will free whatever it allocated. It stores this info somewhere, and looks it up when free() is called.
A heap manager is not "faulty" if it allocates more memory than was requested. Heap managers often work with fixed block sizes, and will round up to the next appropriate block size when satisfying a request. The heap manager's job is to be as efficient as possible, and often big efficiencies result from a few small inefficiencies.
This is the default behavior of malloc. It will return NULL or a pointer to a section of memory at least as long as the one you asked for. So yes free must be able to handle getting rid of memory longer than what was asked for.
Finding out how much memory was actually free or allocated is a platform specific question.
Other answers have explained well how the block size is handled. To find out how much memory is freed, the only solution I can think of is to call mallinfo() before and after the free.
Let's say I have a pointer allocated to hold 4096 bytes. How would one deallocate the last 1024 bytes in C? What about in C++? What if, instead, I wanted to deallocate the first 1024 bytes, and keep the rest (in both languages)? What about deallocating from the middle (it seems to me that this would require splitting it into two pointers, before and after the deallocated region).
Don't try and second-guess memory management. It's usually cleverer than you ;-)
The only thing you can achieve is the first scenario to 'deallocate' the last 1K
char * foo = malloc(4096);
foo = realloc(foo, 4096-1024);
However, even in this case, there is NO GUARANTEE that "foo" will be unchanged. Your entire 4K may be freed, and realloc() may move your memory elsewhere, thus invalidating any pointers to it that you may hold.
This is valid for both C and C++ - however, use of malloc() in C++ is a bad code smell, and most folk would expect you to use new() to allocate storage. And memory allocated with new() cannot be realloc()ed - or at least, not in any kind of portable way. STL vectors would be a much better approach in C++
If you have n bytes of mallocated memory, you can realloc m bytes (where m < n) and thus throw away the last n-m bytes.
To throw away from the beginning, you can malloc a new, smaller buffer and memcpy the bytes you want and then free the original.
The latter option is also available using C++ new and delete. It can also emulate the first realloc case.
You don't have "a pointer allocated to hold 4096 bytes", you have a pointer to an allocated block of 4096 bytes.
If your block was allocated with malloc(), realloc() will allow you to reduce or increase the size of the block. The start address of the block won't necessarily stay the same, though.
You can't change the start address of a malloc'd memory block, which is really what your second scenario is asking. There's also no way to split a malloc'd block.
This is a limitation of the malloc/calloc/realloc/free API -- and implementations may rely on these limitations (for example, keeping bookkeeping information about the allocation immediately before the start address, which would make moving the start address difficult.)
Now, malloc isn't the only allocator out there -- your platform or libraries might provide other ones, or you could write your own (which gets memory from the system via malloc, mmap, VirtualAlloc or some other mechanism) and then hands it out to your program in whatever fashion you desire.
For C++, if you allocate memory with std::malloc, the information above applies. If you're using new and delete, you're allocating storage for and constructing objects, and so changing the size of an allocated block doesn't make sense -- objects in C++ are a fixed size.
You can make it shorter with realloc(). I don't think the rest is possible.
You can use realloc() to apparently make the memory shorter. Note that for some implementations such a call will actually do nothing. You can't free the first bit of the block and retain the last bit.
If you find yourself needing this kind of functionality, you should consider using a more complex data structure. An array is not the correct answer to every programming problem.
http://en.wikipedia.org/wiki/New_(C%2B%2B)
SUMMARY:In contrast to C's realloc, it
is not possible to directly reallocate
memory allocated with new[]. To extend
or reduce the size of a block, one
must allocate a new block of adequate
size, copy over the old memory, and
delete the old block. The C++ standard
library provides a dynamic array that
can be extended or reduced in its
std::vector template.