I was watching a youtube video on how to implement your own vector/dynamic_array in C++.
I understood everything except one like of code, I'm new to c++ and trying to understand the underline data structure implementation.
But I didn't see such a line of code like that one.
The Code:
template<typename... Args>
T& EmplaceBack(Args&&... args) {
if (m_Size >= m_Capacity) {
ReAlloc(m_Capacity + m_Capacity / 2);
}
new(&m_Data[m_Size])T(std::forward<Args>(args)...);
return m_Data[m_Size++];
}
The line I didn't understand:
new(&m_Data[m_Size])T(std::forward<Args>(args)...);
What that line of code is really doing ?
Btw, I don't really know what std::forward is.
Thx for any explanation. :)
This line of code constructs an object of type T with arguments args... at the address &m_Data[m_Size]. No memory allocation involved.
new(&m_Data[m_Size])T(std::forward<Args>(args)...);
Lets assume we call a "normal" new operator:
new T((std::forward<Args>(args)...);
This will allocate memory which size will be equal to sizeof(T) and call a constructor of type T at that memory address. std::forward is used in the technique called perfect forwarding. In short: no copies, no moves, no passing in any way - arguments are just given to the constructor as if you called it directly.
But what if you have your memory already preallocated - the main performance boosting point of many containers, like the vector? Placement new operator is your friend here. You provide it with the address of the preallocated memory and it only constructs the object. And no delete is required later, because you didn't allocate anything with placement new!
What about the destruction? Normally destructors are called automatically when you delete the object. In this case you'll have to call the object's destructor directly, like this - t->~T();
Related
I have done a very simple implementation of vector. I admit that I have cheated a lot and the only instance variables are for example size_t sz and T * elem. The one of many problems (but this one is the only problem I know of which may cause a crash) is the problem with pop_back().
template <typename T>
void vec<T>::pop_back() {
if (sz == 0)
return;
elem[sz-1].~T();
--sz;
}
The vector elem is a dynamically allocated array, but the objects in the array may or may not be. The problem I can see here is that this may crash in case I destroy an object in this array. This can happen in case I would destroy elements multiple times. So to say, in case I have used pop_back. This does not seem to happen for my compiler (which seems strange), but I have heard about someone having this problem with this function. I have been trying to do some research on the web and found another alternative. The zero check is not done here (probably laziness), and this example is using reinterpret_cast
template<class T>
void Vector<T>::pop_back() {
// You way want to do a check for empty() before calling the destructor.
// Call the destructor.
(reinterpret_cast<T*>(buffer)[_size-1]).~T();
// It is usual to prefer the pre decrement (and pre increment).
--_size;
}
The question is, Can I really use reinterpret_cast to be able to destroy non dynamically allocated objects in my vector class?
EDIT
By request I will show my push_back, which I was unsure how to properly write with the short time span I could spend on this. I admit that the main purpose of the class was not efficiency but a good way to handle resources instead of raw dynamic arrays (I can of course use a vector but this is rather a question about a smaller scope than general strategy so I would appreciate if the use-std::vector discussion was left out)
template <typename T>
void vec<T>::push_back(const T& obj) {
T* tmp = new T[sz+1];
if (sz > 0){
uninitialized_copy(elem, elem+sz, tmp);
delete[] elem;
}
elem = tmp;
elem[sz] = T{obj};
++sz;
}
The main problem was the capacity. I realized the capacity part would require a lot of work and without that I could just create new elements.
This answer is inspired by the stack implementation in the book Exceptional C++ by Herb Sutter (Item 12):
You can allocate memory for your vec, if you initialize your T * elem variable with
elem = static_cast<T*>(sz == 0 ? nullptr : operator new(sizeof(T)*sz));
This will give you memory on which no object has yet been constructed.
Edit(thanks to Cheers and hth. - Alf): You can also use the allocator instead of the above code by calling a.allocate(sz).
If you want to add new elements to the vector, you can use construct [1]. To call the destructor of an object you can use destroy [2].
If you now keep track on how many elements there are allocated in your vector in a variable used, you can deallocate the last by calling destroy(elem+used);
construct and destroy use the placement new and an explicit destructor call inside them. I would recommend using these functions instead of raw placement news and destructor calls.
I'm having a problem which I can't wrap my head around.
I'm writing my own container which is more or less like an std::vector<T> and I don't know how to solve the problem around allocating memory for the objects.
Lets say for instance that I write a wrapper around an array and want to allocate data like this:
T* cArray = new T[size];
cArray[index] = std::move(obj);
If the obj doesn't have a default constructor which doesn't take any parameters Ill get a error: "Class: no appropriate default constructor available".
So I though i could solve the problem by not calling the obj's constructor by using operator new: static_cast<T*>(::operator new(sizeof(T)*this->cap)) and this works only if I use built in types like int, double and floats as members for my Test obj.
If I do something like this:
struct Test
{
Test(int x){}
std::string s;
double d;
}
MyVector<Test> vec;
vec.push_back(Test(1));
I get a runtime error if I try to assign data to the location in my push_back function which contains the following line:
cArray[index] = std::move(obj);
in file: xmemory0 on line 106:
Expression: "(_Ptr_user & (_BIG_ALLOCATION_ALIGNMENT -1)) == 0
Again, this is only a problem when I'm not using built-in types. If I remove the std::string as a member everything works as expected. I have no idea what the cause of the problem might be nor do I know how to solve it after hours of searching.
Do you guys know how to solve this problem?
One answer is to not store T[] but instead store something of the same size and alignment requirements (the choice is most often aligned_storage). Then you use the placement new operator to construct items within it, e.g.
new (&carray[index]) T(...args)
When you are done with items you need to manually call the destructor on each item.
You have the right idea, at least the first half of it. Which is to allocate raw memory, rather than using new Test[N]. But you still need to construct your objects, which you can do using placement new. Given a pointer p which points to a location where you want to construct an object, you can do this:
new (p) Test(...constructor args...);
Also note that when you are done with the object, you will need to call the destructor manually:
p->~Test();
That's the right way to allocate the memory. But you still need to call the constructor eventually, to turn the empty bytes into an object. (And later on you need to call the destructor.)
You'll need to use "placement new" to invoke the constructor:
new (&carray[index]) Test(obj)
Let us suposse I have a stack of "unused" dynamic objects. Each time I need a new object, I use one of this "unused" objects as placeholder for the new petition. Something like:
template<typename T>
class my_list
{
public:
template<typename... Args>
T* newObject(Args&&... args)
{
if (unused.empty())
return new T(forward<Args>(args)...);
else {
auto* newbie = unused.top();
unused.pop();
newbie = new (newbie) T(forward<Args>(args)...); // l. X
return newbie;
}
}
private:
stack<T*> unused;
};
Question is, in line X, what is more efficient, the written sentence, or:
*newbie = std::move(T(forward<Args>(args)...));
That means, what is more time-efficient of both, a call of a new with newbie as address (avoiding petitions of new memory) or simply a movement of a new temporary object overwritting the original?
Here's another viable option. It has the advantage that while C++11 is still required for perfect forwarding, it works with library code that hasn't been updated to implement move construction and assignment.
T(forward<Args>(args)...).swap(*newbie);
Besides, your move assignment call already has a temporary object, no need to explicitly make it movable.
*newbie = T(forward<Args>(args)...);
Either of these will make it much easier to provide exception safety than the "first destroy, then construct in-place" approach.
The move is unnecessary, since a constructor call creates a prvalue temporary. Your question comes down to which is more efficient:
newbie->~T();
new (newbie) T(std::forward<Args>(args)...);
or
*newbie = T(std::forward<Args>(args)...);
Assuming T has a properly-written move assignment operator, there is likely to be little difference if any; in terms of side effects the destructor of T is called before the constructor in the first case, and after in the second, but there is no reason to believe that this would affect performance one way more than the other. The latter could result in more primitive value copies, but any decent optimising compiler will eliminate them.
In the absence of performance testing indicating that the former significantly improves performance, you should prefer the latter as it is far more readable, and easier to make exception safe.
I have a class Bullet that takes several arguments for its construction. However, I am using a dynamic memory array to store them. I am using C++ so i want to conform to it's standard by using the new operator to allocate the memory. The problem is that the new operator is asking for the constructor arguments when I'm allocating the array, which I don't have at that time. I can accomplish this using malloc to get the right size then fill in form there, but that's not what i want to use :) any ideas?
pBulletArray = (Bullet*) malloc(iBulletArraySize * sizeof(Bullet)); // Works
pBulletArray = new Bullet[iBulletArraySize]; // Requires constructor arguments
Thanks.
You can't.
And if you truly want to conform to C++ standards, you should use std::vector.
FYI, it would probably be even more expensive than what you're trying to achieve. If you could do this, new would call a constructor. But since you'll modify the object later on anyway, the initial construction is useless.
1) std::vector
A std::vector really is the proper C++ way to do this.
std::vector<Bullet> bullets;
bullets.reserve(10); // allocate memory for bullets without constructing any
bullets.push_back(Bullet(10.2,"Bang")); // put a Bullet in the vector.
bullets.emplace_back(10.2,"Bang"); // (C++11 only) construct a Bullet in the vector without copying.
2) new [] operator
It is also possible to do this with new, but you really shouldn't. Manually managing resources with new/delete is an advanced task, similar to template meta-programming in that it's best left to library builders, who'll use these features to build efficient, high level libraries for you. In fact to do this correctly you'll basically be implementing the internals of std::vector.
When you use the new operator to allocate an array, every element in the array is default initialized. Your code could work if you added a default constructor to Bullet:
class Bullet {
public:
Bullet() {} // default constructor
Bullet(double,std::string const &) {}
};
std::unique_ptr<Bullet[]> b = new Bullet[10]; // default construct 10 bullets
Then, when you have the real data for a Bullet you can assign it to one of the elements of the array:
b[3] = Bullet(20.3,"Bang");
Note the use of unique_ptr to ensure that proper clean-up occurs, and that it's exception safe. Doing these things manually is difficult and error prone.
3) operator new
The new operator initializes its objects in addition to allocating space for them. If you want to simply allocate space, you can use operator new.
std::unique_ptr<Bullet,void(*)(Bullet*)> bullets(
static_cast<Bullet*>(::operator new(10 * sizeof(Bullet))),
[](Bullet *b){::operator delete(b);});
(Note that the unique_ptr ensures that the storage will be deallocated but no more. Specifically, if we construct any objects in this storage we have to manually destruct them and do so in an exception safe way.)
bullets now points to storage sufficient for an array of Bullets. You can construct an array in this storage:
new (bullets.get()) Bullet[10];
However the array construction again uses default initialization for each element, which we're trying to avoid.
AFAIK C++ doesn't specify any well defined method of constructing an array without constructing the elements. I imagine this is largely because doing so would be a no-op for most (all?) C++ implementations. So while the following is technically undefined, in practice it's pretty well defined.
bool constructed[10] = {}; // a place to mark which elements are constructed
// construct some elements of the array
for(int i=0;i<10;i+=2) {
try {
// pretend bullets points to the first element of a valid array. Otherwise 'bullets.get()+i' is undefined
new (bullets.get()+i) Bullet(10.2,"Bang");
constructed = true;
} catch(...) {}
}
That will construct elements of the array without using the default constructor. You don't have to construct every element, just the ones you want to use. However when destroying the elements you have to remember to destroy only the elements that were constructed.
// destruct the elements of the array that we constructed before
for(int i=0;i<10;++i) {
if(constructed[i]) {
bullets[i].~Bullet();
}
}
// unique_ptr destructor will take care of deallocating the storage
The above is a pretty simple case. Making non-trivial uses of this method exception safe without wrapping it all up in a class is more difficult. Wrapping it up in a class basically amounts to implementing std::vector.
4) std::vector
So just use std::vector.
It's possible to do what you want -- search for "operator new" if you really want to know how. But it's almost certainly a bad idea. Instead, use std::vector, which will take care of all the annoying details for you. You can use std::vector::reserve to allocate all the memory you'll use ahead of time.
Bullet** pBulletArray = new Bullet*[iBulletArraySize];
Then populate pBulletArray:
for(int i = 0; i < iBulletArraySize; i++)
{
pBulletArray[i] = new Bullet(arg0, arg1);
}
Just don't forget to free the memory using delete afterwards.
The way C++ new normally works is allocating the memory for the class instance and then calling the constructor for that instance. You basically have already allocated the memory for your instances.
You can call only the constructor for the first instance like this:
new((void*)pBulletArray) Bullet(int foo);
Calling the constructor of the second one would look like this (and so on)
new((void*)pBulletArray+1) Bullet(int bar);
if the Bullet constructor takes an int.
If what you're really after here is just fast allocation/deallocation, then you should look into "memory pools." I'd recommend using boost's implementation, rather than trying to roll your own. In particular, you would probably want to use an "object_pool".
I've recently come across this rant.
I don't quite understand a few of the points mentioned in the article:
The author mentions the small annoyance of delete vs delete[], but seems to argue that it is actually necessary (for the compiler), without ever offering a solution. Did I miss something?
In the section 'Specialized allocators', in function f(), it seems the problems can be solved with replacing the allocations with: (omitting alignment)
// if you're going to the trouble to implement an entire Arena for memory,
// making an arena_ptr won't be much work. basically the same as an auto_ptr,
// except that it knows which arena to deallocate from when destructed.
arena_ptr<char> string(a); string.allocate(80);
// or: arena_ptr<char> string; string.allocate(a, 80);
arena_ptr<int> intp(a); intp.allocate();
// or: arena_ptr<int> intp; intp.allocate(a);
arena_ptr<foo> fp(a); fp.allocate();
// or: arena_ptr<foo>; fp.allocate(a);
// use templates in 'arena.allocate(...)' to determine that foo has
// a constructor which needs to be called. do something similar
// for destructors in '~arena_ptr()'.
In 'Dangers of overloading ::operator new[]', the author tries to do a new(p) obj[10]. Why not this instead (far less ambiguous):
obj *p = (obj *)special_malloc(sizeof(obj[10]));
for(int i = 0; i < 10; ++i, ++p)
new(p) obj;
'Debugging memory allocation in C++'. Can't argue here.
The entire article seems to revolve around classes with significant constructors and destructors located in a custom memory management scheme. While that could be useful, and I can't argue with it, it's pretty limited in commonality.
Basically, we have placement new and per-class allocators -- what problems can't be solved with these approaches?
Also, in case I'm just thick-skulled and crazy, in your ideal C++, what would replace operator new? Invent syntax as necessary -- what would be ideal, simply to help me understand these problems better.
Well, the ideal would probably be to not need delete of any kind. Have a garbage-collected environment, let the programmer avoid the whole problem.
The complaints in the rant seem to come down to
"I liked the way malloc does it"
"I don't like being forced to explicitly create objects of a known type"
He's right about the annoying fact that you have to implement both new and new[], but you're forced into that by Stroustrups' desire to maintain the core of C's semantics. Since you can't tell a pointer from an array, you have to tell the compiler yourself. You could fix that, but doing so would mean changing the semantics of the C part of the language radically; you could no longer make use of the identity
*(a+i) == a[i]
which would break a very large subset of all C code.
So, you could have a language which
implements a more complicated notion of an array, and eliminates the wonders of pointer arithmetic, implementing arrays with dope vectors or something similar.
is garbage collected, so you don't need your own delete discipline.
Which is to say, you could download Java. You could then extend that by changing the language so it
isn't strongly typed, so type checking the void * upcast is eliminated,
...but that means that you can write code that transforms a Foo into a Bar without the compiler seeing it. This would also enable ducktyping, if you want it.
The thing is, once you've done those things, you've got Python or Ruby with a C-ish syntax.
I've been writing C++ since Stroustrup sent out tapes of cfront 1.0; a lot of the history involved in C++ as it is now comes out of the desire to have an OO language that could fit into the C world. There were plenty of other, more satisfying, languages that came out around the same time, like Eiffel. C++ seems to have won. I suspect that it won because it could fit into the C world.
The rant, IMHO, is very misleading and it seems to me that the author does understand the finer details, it's just that he appears to want to mislead. IMHO, the key point that shows the flaw in argument is the following:
void* operator new(std::size_t size, void* ptr) throw();
The standard defines that the above function has the following properties:
Returns: ptr.
Notes: Intentionally performs no other action.
To restate that - this function intentionally performs no other action. This is very important, as it is the key to what placement new does: It is used to call the constructor for the object, and that's all it does. Notice explicitly that the size parameter is not even mentioned.
For those without time, to summarise my point: everything that 'malloc' does in C can be done in C++ using "::operator new". The only difference is that if you have non aggregate types, ie. types that need to have their destructors and constructors called, then you need to call those constructor and destructors. Such types do not explicitly exist in C, and so using the argument that "malloc does it better" is not valid. If you have a struct in 'C' that has a special "initializeMe" function which must be called with a corresponding "destroyMe" then all points made by the author apply equally to that struct as they do to a non-aggregate C++ struct.
Taking some of his points explicitly:
To implement multiple inheritance, the compiler must actually change the values of pointers during some casts. It can't know which value you eventually want when converting to a void * ... Thus, no ordinary function can perform the role of malloc in C++--there is no suitable return type.
This is not correct, again ::operator new performs the role of malloc:
class A1 { };
class A2 { };
class B : public A1, public A2 { };
void foo () {
void * v = ::operator new (sizeof (B));
B * b = new (v) B(); // Placement new calls the constructor for B.
delete v;
v = ::operator new (sizeof(int));
int * i = reinterpret_cast <int*> (v);
delete v'
}
As I mention above, we need placement new to call the constructor for B. In the case of 'i' we can cast from void* to int* without a problem, although again using placement new would improve type checking.
Another point he makes is about alignment requirements:
Memory returned by new char[...] will not necessarily meet the alignment requirements of a struct intlist.
The standard under 3.7.3.1/2 says:
The pointer returned shall be suitably aligned so that it can be converted to a
pointer of any complete object type and then used to access the object or array in the storage allocated (until
the storage is explicitly deallocated by a call to a corresponding deallocation function).
That to me appears pretty clear.
Under specialized allocators the author describes potential problems that you might have, eg. you need to use the allocator as an argument to any types which allocate memory themselves and the constructed objects will need to have their destructors called explicitly. Again, how is this different to passing the allocator object through to an "initalizeMe" call for a C struct?
Regarding calling the destructor, in C++ you can easily create a special kind of smart pointer, let's call it "placement_pointer" which we can define to call the destructor explicitly when it goes out of scope. As a result we could have:
template <typename T>
class placement_pointer {
// ...
~placement_pointer() {
if (*count == 0) {
m_b->~T();
}
}
// ...
T * m_b;
};
void
f ()
{
arena a;
// ...
foo *fp = new (a) foo; // must be destroyed
// ...
fp->~foo ();
placement_pointer<foo> pfp = new (a) foo; // automatically !!destructed!!
// ...
}
The last point I want to comment on is the following:
g++ comes with a "placement" operator new[] defined as follows:
inline void *
operator new[](size_t, void *place)
{
return place;
}
As noted above, not just implemented this way - but it is required to be so by the standard.
Let obj be a class with a destructor. Suppose you have sizeof (obj[10]) bytes of memory somewhere and would like to construct 10 objects of type obj at that location. (C++ defines sizeof (obj[10]) to be 10 * sizeof (obj).) Can you do so with this placement operator new[]? For example, the following code would seem to do so:
obj *
f ()
{
void *p = special_malloc (sizeof (obj[10]));
return new (p) obj[10]; // Serious trouble...
}
Unfortunately, this code is incorrect. In general, there is no guarantee that the size_t argument passed to operator new[] really corresponds to the size of the array being allocated.
But as he highlights by supplying the definition, the size argument is not used in the allocation function. The allocation function does nothing - and so the only affect of the above placement expression is to call the constructor for the 10 array elements as you would expect.
There are other issues with this code, but not the one the author listed.