I'm currently trying to create vector-like container. It uses memory allocated by new[] as it's base. the problem arises when I need to expand the array. I allocate a bigger chunk of memory with new[], then memcpy old memory into there and delete[] the old memory. Thing is, trying to store any pointer or any pointer-containing object inside result in memory corruption. So I need a way to free the memory used without destroying objects inside
Edit: Some code to understand the problem:
template<typename T>
class myvector
{
private:
T* _data;
size_t _size, _capacity;
static constexpr float multiplier = 1.5;
public:
void expand()
{
size_t e_size = sizeof(T);
size_t old_capacity = this->_capacity;
this->_capacity = (unsigned long)(float(this->_capacity) * myvector::multiplier);
T *tmp = new T[this->_capacity];
memcpy(tmp, this->_data, e_size * (old_capacity));
// this will destroy all the objects inside the container
// which will result in destruction of any allocated memory
delete[] this->_data;
// so now we have an array of invalid pointers. fun time
this->_data = tmp;
}
}
How to free memory allocated by new[]?
Using delete[]. This must be done before the pointer value is lost, and it must be done after the last use of the pointer value. And it must be done exactly once. And it must not be done for any other pointer than one that was returned by array new.
Thing is, trying to store any pointer or any pointer-containing object inside result in memory corruption.
Then there is a bug in how you use the objects. Storing such objects in your template should by itself not be a problem.
So I need a way to free the memory used without destroying objects inside
That is simply not possible. An object cannot exist without storage (except for special cases that don't apply here).
delete[] this->_data;
// so now we have an array of invalid pointers. fun time
Why would there be an array of invalid pointers? Do the pointers in the array point to this->_data?
Indeed, your data structure does not have stable addresses for its elements. Expansion will invalidate any references to the elements. If you need such stability, then you must use a node based data structure such as a linked list.
Your template does have a limitation that it is well defined only for trivially copyable classes. Perhaps you've overlooked this limitation. It is easy to get rid of this limitation: Simply use std::copy (or perhaps std::move from <algorithm>, depending on exception safety guarantees that you need) instead of std::memcpy.
Related
This question is similar to Problem with delete[], how to partially delete the memory?
I understand that deleting an array after incrementing its pointer is not possible as it loses the track of how many bytes to clean. But, I am not able to understand why one-by-one delete/deallocation of a dynamic array doesn't work either.
int main()
{
int n = 5;
int *p = new int[n];
for(int i=0;i<n;++i){
delete &p[i];
}
}
I believe this should work, but in clang 12.0 it fails with the invalid pointer error. Can anyone explain why?
An array is a contiguous object in memory of a specific size. It is one object where you can place your data in and therefore you can only free/delete it as one object.
You are thinking that an array is a list of multiple objects, but that's not true. That would be true for something like a linked list, where you allocate individual objects and link them together.
You allocated one object of the type int[n] (one extent of memory for an array) using the operator new
int *p = new int[n];
Elements of the array were not allocated dynamically separately.
So to delete it you just need to write
delete []p;
If for example you allocated an array of pointers like
int **p = new int *[n];
and then for each pointer of the array you allocated an object of the type int like
for ( int i = 0;i < n;++i )
{
p[i] = new int( i );
}
then to delete all the allocated objects you need to write
for ( int i = 0; i < n; ++i )
{
delete p[i];
}
delete []p;
That is the number of calling of the operator delete or delete [] one to one corresponds to the number of calling operator new or new [].
One new always goes with one delete. Just as that.
In detail, when we request an array using new, what we actually do is to get a pointer that controls a contiguous & fixed block on the memory. Whatever we do with that array, we do it through that pointer and this pointer associates strictly with the array itself.
Furthermore, let's assume that you were able to delete an elemnent in the middle of that array. After the deletion, that array would fall apart and they are not contiguous anymore! By then, an array would not really be an array!
Because of that, we can not 'chop off' an array into separate pieces. We must always treat an array as one thing, not distinctive elements scattered around the memory.
Greatly simplyfyinh: in most systems memory is allocated in logical blocks which are described by the starting pointer of the allocated block.
So if you allocate an array:
int* array = new int[100];
OS stores the information of that allocation as a pair (simplifying) (block_begin, size) -> (value of array ptr, 100)
Thus when you deallocate the memory you don't need to specify how much memory you allocated i.e:
// you use
delete[] array; // won't go into detail why you do delete[] instead of delete - mostly it is due to C++ way of handling destruction of objects
// instead of
delete[100] array;
In fact in bare C you would do this with:
int* array = malloc(100 * sizeof(int))
[...]
free(array)
So in most OS'es it is not possible due to the way they are implemented.
However theoretically allocating large chunk of memory in fact allocate many smaller blocks which could be deallocated this way, but still it would deallocate smaller blocks at a time not one-by-one.
All of new or new[] and even C's malloc do exactly the same in respect to memory: requesting a fix block of memory from the operating system.
You cannot split up this block of memory and return it partially to the operating system, that's simply not supported, thus you cannot delete a single element from the array either. Only all or none…
If you need to remove an element from an array all you can do is copy the subsequent elements one position towards front, overwriting the element to delete and additionally remember how many elements actually are valid – the elements at the end of the array stay alive!
If these need to be destructed immediately you might call the destructor explicitly – and then assure that it isn't called again on an already destructed element when delete[]ing the array (otherwise undefined behaviour!) – ending in not calling new[] and delete[] at all but instead malloc, placement new for each element, std::launder any pointer to any element created that way and finally explicitly calling the constructor when needed.
Sounds like much of a hassle, doesn't it? Well, there's std::vector doing all this stuff for you! You should this one it instead…
Side note: You could get similar behaviour if you use an array of pointers; you then can – and need to – maintain (i.e. control its lifetime) each object individually. Further disadvantages are an additional level of pointer indirection whenever you access the array members and the array members indeed being scattered around the memory (though this can turn into an advantage if you need to move objects around your array and copying/moving objects is expensive – still you would to prefer a std::vector, of pointers this time, though; insertions, deletions and managing the pointer array itself, among others, get much safer and much less complicated).
I've seen a lot of similar questions regarding this topic, but I have seen no explicit answer to this question. Consider the following code:
typedef struct Student
{
int id;
} Student;
vector<Student> students();
for( int i = 0; i < 10; i++ )
{
Student s();
s.id = i
students.push_back( s );
}
How does this vector allocate memory? From my understanding, each Student s should have it's memory on the stack, and be de-allocated when the loop iterates, so this should produce undefined behaviour if I try and access this data later on. But if I had done the same thing with a vector<int> it would not produce undefined behaviour.
Is this correct? Either way, my goal is create a vector<Student> in which the objects are allocated on the heap, without having to use a vector<Student*>. Is this possible?
Create the std::unique_ptr<Student> ...
Also use the std::vector<std::unique_ptr<Student>> type to hold the data.
This way you do not have to worry about deleting the allocated memory on heap storage by unique_ptr.
According to vector in cppreference, a vector required two template parameters which are T (the type of elements in vector) and Allocator which is set by default to std::allocator<T> and according to std::allocator's allocate function, which will have to be called by vector in order to allocate memory:
Allocates n * sizeof(T) bytes of uninitialized storage by calling ::operator new(std::size_t) or ::operator new(std::size_t, std::align_val_t) (since C++17), but it is unspecified when and how this function is called. The pointer hint may be used to provide locality of reference: the allocator, if supported by the implementation, will attempt to allocate the new memory block as close as possible to hint.
So whenever there is no given allocator to vector, all the newly members are stored in heap. Although most of the allocators must be using heap to allocate memory.
I have
int * array=new int[2];
and I would like to free the memory of the last element, thus reducing the allocated memory to only 1 element. I tried to call
delete array+1;
but it gives error
*** glibc detected *** skuska:
free(): invalid pointer: 0x000000000065a020 *
Can this be done in C++03 without explicit reallocation?
Note: If I wanted to use a class instead a primitive datatype (like int), how can I free the memory so that the destructor of the class is called too?
Note2: I am trying to implement vector::pop_back
Don't use new[] expression for this. That's not how vector works. What you do is allocate a chunk of raw memory. You could use malloc for this, or you could use operator new, which is different from the new expression. This is essentially what the reserve() member function of std::vector does, assuming you've used the default allocator. It doesn't create any actual objects the way the new[] expression does.
When you want to construct an element, you use placement new, passing it a location somewhere in the raw memory you've allocated. When you want to destoy an element, you call its destructor directly. When you are done, instead of using the delete[] expression, you use operator delete if you used operator new, or you use free() if you used malloc.
Here's an example creating 10 objects, and destoying them in reverse order. I could destroy them in any order, but this is how you would do it in a vector implementation.
int main()
{
void * storage = malloc(sizeof(MyClass) * 10);
for (int i=0; i<10; ++i)
{
// this is placement new
new ((MyClass*)storage + i) MyClass;
}
for (int i=9; i>=0; --i)
{
// calling the destructor directly
((MyClass*)storage + i)->~MyClass();
}
free(storage);
}
pop_back would be implemented by simply calling the destructor of the last element, and decrementing the size member variable by 1. It wouldn't, shouldn't (and couldn't, without making a bunch of unnecessary copies) free any memory.
There is no such option. Only way to resize array is allocate new array with size old_size - 1, copy content of old array and then delete old array.
If you want free object memory why not create array of pointers?
MyClass **arr = new MyClass*[size];
for(int i = 0; i < size; i++)
arr[i] = new MyClass;
// ...
delete arr[size-1];
std::vector::pop_back doesn't reallocate anything — it simply updates the internal variable determining data size, reducing it by one. The old last element is still there in memory; the vector simply doesn't let you access it through its public API. *
This, as well as growing re-allocation being non-linear, is the basis of why std::vector::capacity() is not equivalent to std::vector::size().
So, if you're really trying to re-invent std::vector for whatever reason, the answer to your question about re-allocation is don't.
* Actually for non-primitive data types it's a little more complex, since such elements are semantically destroyed even though their memory will not be freed.
Since you are using C++03, you have access to the std::vector data type. Use that and it's one call:
#include <vector>
//...
std::vector<int> ary(3);
//...
ary.erase(ary.begin() + (ary.size() - 1));
or
#include <vector>
//...
std::vector<int> ary(3);
//...
ary.pop_back();
EDIT:
Why are you trying to re-invent the wheel? Just use vector::pop_back.
Anyway, the destructor is called on contained data types ONLY if the contained data type IS NOT a pointer. If it IS a pointer you must manually call delete on the object you want to delete, set it to nullptr or NULL (because attempting to call delete on a previously deleted object is bad, calling delete on a null pointer is a non-op), then call erase.
I saw an example of using the function: delete in cpp and I didn't completely understand it.
the code is:
class Name {
const char* s;
//...
};
class Table {
Name* p;
size_t sz;
public:
Table(size_t s = 15){p = new Name[sz = s]; }
~Table { delete[] p; }
};
What is the exact action of the command: delete[] p;?
I think the aim was to delete all the pointers in the container Table.
The brackets in delete[] give me a clue that it deletes an array of pointers to Name but the size of the array is not specified, so how does the destructor "know" how many pointers to delete?
delete isn't a function, it's an operator.
A delete expression using [] destroys objects created with new ... [] and releases the associated memory. delete[] must be used for pointers returned by new ... []; non-array delete only on pointers returned by non-array new. Using the non-matching delete form is always incorrect.
The delete expression in ~Table() (missing () in your code) will destroy the dynamically created array of Name objects ensuring that the Name destructor is called for each member of the array.
It is up the the implementation to implement some mechanism of recording the number of elements in arrays allocated with new ... [] the programmer doesn't have to worry about this.
In many implementations, where the array elements have non-trivial destructors, a new[] expression will allocate extra space to record the element count before the space for all the array members. This hidden count is then looked up when delete[] is used to ensure the correct number of destructors are called. This is just an implementation detail, though, other implementations are possible.
In short, delete[] knows the size of the array it is deleting because it is required to.
Because the C++ language standard states that it must know.
So when you allocate an array, it is up to the system to store the size somewhere where delete[] can find it.
One option is to allocate a few bytes more than needed. Use the first bytes to specify the size, and then instead of returning a pointer to the first allocated byte, return a pointer to the first byte past the size field.
Then delete[] just has to subtract a few bytes from the pointer in order to find the size.
Another option could be to have a global map<void*, int>, where the key is the pointer to a memory allocation, and the value is the size of that allocation. There are plenty of other ways in which delete[] can find out the size. The C++ standard doesn't specify which one to use. It just says that delete[] must know the size, and leaves it up to the implementers to figure out how to pull it off.
Below, I'm not declaring my_ints as a pointer. I don't know where the memory will be allocated. Please educate me here!
#include <iostream>
#include <vector>
class FieldStorage
{
private:
std::vector<int> my_ints;
public:
FieldStorage()
{
my_ints.push_back(1);
my_ints.push_back(2);
}
void displayAll()
{
for (int i = 0; i < my_ints.size(); i++)
{
std::cout << my_ints[i] << std::endl;
}
}
};
And in here, I'm declaring the field my_ints as a pointer:
#include <iostream>
#include <vector>
class FieldStorage
{
private:
std::vector<int> *my_ints;
public:
FieldStorage()
{
my_ints = new std::vector<int>();
my_ints->push_back(1);
my_ints->push_back(2);
}
void displayAll()
{
for (int i = 0; i < my_ints->size(); i++)
{
std::cout << (*my_ints)[i] << std::endl;
}
}
~FieldStorage()
{
delete my_ints;
}
};
main() function to test:
int main()
{
FieldStorage obj;
obj.displayAll();
return 0;
}
Both of them produces the same result. What's the difference?
In terms of memory management, these two classes are virtually identical. Several other responders have suggested that there is a difference between the two in that one is allocating storage on the stack and other on the heap, but that's not necessarily true, and even in the cases where it is true, it's terribly misleading. In reality, all that's different is where the metadata for the vector is allocated; the actual underlying storage in the vector is allocated from the heap regardless.
It's a little bit tricky to see this because you're using std::vector, so the specific implementation details are hidden. But basically, std::vector is implemented like this:
template <class T>
class vector {
public:
vector() : mCapacity(0), mSize(0), mData(0) { }
~vector() { if (mData) delete[] mData; }
...
protected:
int mCapacity;
int mSize;
T *mData;
};
As you can see, the vector class itself only has a few members -- capacity, size and a pointer to a dynamically allocated block of memory that will store the actual contents of the vector.
In your example, the only difference is where the storage for those few fields comes from. In the first example, the storage is allocated from whatever storage you use for your containing class -- if it is heap allocated, so too will be those few bits of the vector. If your container is stack allocated, so too will be those few bits of the vector.
In the second example, those bits of the vector are always heap allocated.
In both examples, the actual meat of the vector -- the contents of it -- are allocated from the heap, and you cannot change that.
Everybody else has pointed out already that you have a memory leak in your second example, and that is also true. Make sure to delete the vector in the destructor of your container class.
You have to release ( to prevent memory leak ) memory allocated for vector in the second case in the FieldStorage destructor.
FieldStorage::~FieldStorage()
{
delete my_ints;
}
As Mykola Golubyev pointed out, you need to delete the vector in the second case.
The first will possibly build faster code, as the optimiser knows the full size of FieldStorage including the vector, and could allocate enough memory in one allocation for both.
Your second implementation requires two separate allocations to construct the object.
I think you are really looking for the difference between the Stack and the Heap.
The first one is allocated on the stack while the second one is allocated on the heap.
In the first example, the object is allocated on the stack.
The the second example, the object is allocated in the heap, and a pointer to that memory is stored on the stack.
the difference is that the second allocates the vector dynamically. there are few differences:
you have to release memory occupied by the vector object (the vector object itself, not the object kept in the vector because it is handled correctly by the vector). you should use some smart pointer to keep the vector or make (for example in the destructor):
delete my_ints;
the first one is probably more efficient because the object is allocated on the stack.
access to the vector's method have different syntax :)
The first version of FieldStorage contains a vector. The size of the FieldStorage class includes enough bytes to hold a vector. When you construct a FieldStorage, the vector is constructed right before the body of FieldStorage's constructor is executed. When a FieldStorage is destructed, so is the vector.
This does not necessarily allocate the vector on the stack; if you heap-allocate a FieldStorage object, then the space for the vector comes from that heap allocation, not the stack. If you define a global FieldStorage object, then the space for the vector comes from neither the stack nor the heap, but rather from space designated for global objects (e.g. the .data or .bss section on some platforms).
Note that the vector performs heap allocations to hold the actual data, so it is likely to only contain a few pointers, or a pointer and couple of lengths, but it may contain whatever your compiler's STL implementation needs it to.
The second version of FieldStorage contains a pointer to a vector. The size of the FieldStorage class includes room for a pointer to a vector, not an actual vector. You are allocating storage for the vector using new in the body of FieldStorage's constructor, and you leaking that storage when FieldStorage is destructed, because you didn't define a destructor that deletes the vector.
First way is the prefereable one to use, you don't need pointer on vector and have forgot to delete it.
In the first case, the std::vector is being put directly into your class (and it is handling any memory allocation and deallocation it needs to grow and shrink the vector and free the memory when your object is destroyed; it is abstracting the memory management from you so that you don't have to deal with it). In the second case, you are explicitly allocating the storage for the std::vector in the heap somewhere, and forgetting to ever delete it, so your program will leak memory.
The size of the object is going to be different. In the second case the Vector<>* only takes up the size of a pointer (4bytes on 32bit machines). In the first case, your object will be larger.
One practical difference is that in your second example, you never release either the memory for the vector, or its contents (because you don't delete it, and therefore call its destructor).
But the first example will automatically destroy the vector (and free its contents) upon destruction of your FieldStorage object.