Iterating through objects with a common base class in contigious memory - c++

Question

I'm trying to figure out how I can iterate through a container (like std::vector) of objects that share a common base parent class contiguously in memory.
To demonstrate the problem let's use the following examples.
class Base
{
public:
Base();
virtual void doStuff() = 0;
};
class DerivedA : public Base
{
private:
//specific A member variables
public:
DerivedA();
virtual void doStuff();
};
class DerivedB : public Base
{
private:
//specific B member variables
public:
DerivedB();
virtual void doStuff();
};
Now, using std::vector to iterate would keep the objects in contiguous memory but we would experience slicing as there isn't room for the derived properties.
So we have to use polymorphic technique using pointers like so
int main ()
{
std::vector<Base*> container;
container.push_back(new DerivedA());
container.push_back(new DerivedB());
for (std::vector<Base*>::iterator i = container.begin(); i!=container.end(); i++)
{
(*(*i)).doStuff();
}
}
As far as I know that should work fine given that the classes are implemented.
Problem:
Now, the vector contains pointers in contiguous memory but that does not mean that the addresses they are pointing to are.
So if I want to be able to delete and insert objects into the vector on the fly at any time, the objects will be spread all over the place in memory.
Question:
It seems like everyone suggests doing it the std::vector way but why isn't it considered problematic that it isn't iterable contiguously in memory (assuming we actually use the pointer)?
Am I forced to do it the copy-pasta way?
int main ()
{
std::vector<DerivedA> containerA;
DerivedA a;
containerA.push_back(a);
std::vector<DerivedB> containerB;
DerivedB b;
containerB.push_back(b);
for (std::vector<DerivedA>::iterator i = containerA.begin(); i!=container.end(); i++)
{
(*i).doStuff();
}
for (std::vector<DerivedB>::iterator i = containerB.begin(); i!=container.end(); i++)
{
(*i).doStuff();
}
}
I'm guessing there might not be a real solution to this since keeping objects of various sizes linearly in memory doesn't really make sense but if anyone can give me some advice I'd appreciate it.

Answer 1

Let's take the questions in order.
Q: How I can create a contiguous, heterogeneous container?
A: You can't.
Suppose you used some
placement new
shenanigans to arrange your objects in memory like this:
[B ][DA ][DB ][B ][B ][DB ][DA ]
How would the iteration mechanism know how far to advance the iteration
pointer from one object to the next? The number of bytes from the first
element to the second is different from the second to the third.
The reason contiguous arrays have to be homogenous is so the distance
from one object to the next is the same for all elements. You might
try to embed a size in each object, but then you basically have a linked
list rather than an array (albeit one with good
locality).
This reasoning leads to the idea to use an array of pointers, about which
you have posed the next question:
Q: Why isn't it considered problematic that it isn't iterable contiguously
A: It is not as slow as you think.
Your concern seems to be the performance of following pointers to
scattered memory locations. But the cost of following these pointers is
unlikely to be dominant. Don't get hung up on micro-optimizations like
memory layout until you have solid evidence they are needed.
Q: Am I forced to do it the copy-pasta way?
A: No!
Here, the concern seems to be maintainability rather than performance.
Maintainability is much more important in my opinion, and a good thing
to think about early.
For maintainability, you already have a fine solution: maintain a
vector of Base*.
If you really want to use multiple vectors, there is still a better way
than copy and paste: use a template function, like this (untested):
template <class T>
void doStuffToVector(std::vector<T> &vec)
{
for (std::vector<T>::iterator i = vec.begin(); i!=vec.end(); ++i) {
(*i).doStuff();
}
}
Then call it on each container:
doStuffToVector(containerA);
doStuffToVector(containerB);
If your only concern is maintainability, either the vector of pointers
or the template function should suffice.
Q: Any advice?
A: For starters, ignore performance, at least as far as constant
factors are concerned. Concentrate on correctness and maintainability.
Then, measure performance. Observe that this question did not begin
with a statement of current and desired speed. You don't yet have an
actual problem to solve!
After measurement, if you conclude it's too slow, use a
profiler
to find out where the slow spots are. They are almost never where you
think they will be.
Case in point: this entire question and answers have been focused on
the iteration, but no one has raised the point that the virtual function
calls to doStuff are much more likely to be the bottleneck! Virtual
function calls are expensive because they are indirect control flow,
which causes major problems for the
pipeline;
indirect data access is much less expensive because the
data cache is usually very
effective at satisfying data access requests quickly.
Q: (Implied) How would I optimize this?
A: After careful measurement, you might possibly find that this code
(the iteration itself, including virtual function dispatch; not what's
inside doStuff) is a bottleneck. That must mean it's being executed
for billions of iterations, minimum.
First, look into algorithmic improvements that will reduce the number
of iterations required.
Next, eliminate the virtual function call, for example by embedding an
explicit indicator of object type and testing it with if or switch.
That will allow the processor's
branch predictor to
be much more effective.
Finally, yes, you'd probably want to put all of the elements into one
contiguous array to improve locality and eliminate the indirect data
access. That will mean eliminating the class hierarchy too so all
objects are the same type, either combining all of the fields into a
single class and/or using a union. This will harm your program's
maintainability! That is sometimes one of the costs of writing high
performance code, but is actually necessary only very rarely.

Answer 2

A very simple solution is to sort your pointer array by address value. Then if you iterate your vector, they will be in memory order. Perhaps not contiguous, but in order nonetheless, which reduces cache misses.
The only way to truly have contiguous memory is to allocate it as such, for example have vectors of objects of derived type stored in their own container, which you then reference in your pointer vector.

Answer 3

It seems like everyone suggests doing it the std::vector way but why
isn't it considered problematic that it isn't iterable contiguously in
memory (assuming we actually use the pointer)?
I don't know who considers it problematic or not. As in the other answers, in a lot of cases you just don't care. Do the profiling and you'll see if you have to optimize it or not.
In most of the cases, people would recomment you to use a std::vector<std::unique_ptr<...>>.
In many cases though, it is very important to have your objects in contiguous memory. Gaming is one of those cases. I write a lot of computational code (finite element libraries), where it is also very important. You could read about how to organize your data in a different way in order to have everything line up. For instance, it might be interesting to store all Arm objects in a std::vector rather than store each Arm in a Hero object and access the Arm objects through the Hero object.
Anyways, here is an easy way to store your objects from your example in a contiguous container.
For the base class, use alignas to fix the size of the object. Make sure it's big enough so all derived objects fit in it. In my below example, DerivedA has size 16, DerivedB has size 24. The specified align size must be a power of 2, so we choose 32.
struct alignas(32) Base
{
virtual void print() const {}
};
struct DerivedA : Base
{
void print() const final override { std::cout << "num: " << i << std::endl; }
int i = 1;
};
struct DerivedB : Base
{
void print() const final override { std::cout << "num: " << i << std::endl; }
int i = 2;
double j = 100.0;
};
Now we can write instances of DerivedA and DerivedB with placement new:
int main ()
{
std::vector<Base> v(2);
new (&v[0]) DerivedA();
new (&v[1]) DerivedB();
for (const auto& e : v)
e.print();
return 0;
}
EDIT
The problem here is that you need to manage the sizes manually. Also, as has been pointed out to me recently, alignas is designed for positioning the object in memory, not to allocate size. Maybe a better way would just be using std::variant.
int main()
{
std::vector<std::variant<DerivedA, DerivedB>> vec;
vec.emplace_back(DerivedA());
vec.emplace_back(DerivedB());
for (const auto& e : vec)
std::visit(VisitPackage(), e);
return 0;
}
where VisitPackage could be something like this:
struct VisitPackage
{
void operator()(const DerivedA& d) { d.print(); }
void operator()(const DerivedB& d) { d.print(); }
};
Below is a full and short example of how to get what you want using std::variant.
#include <iostream>
#include <vector>
#include <variant>
struct Base { virtual void print() const = 0; };
struct DerivedA : Base { void print() const final override { std::cout << "DerivedA\n"; } };
struct DerivedB : Base { void print() const final override { std::cout << "DerivedB\n"; } };
struct Print
{
template <typename T>
// note that the operator() calls print from DerivedA or DerivedB directly
void operator()(const T& obj) const { obj.print(); }
};
int main ()
{
using var_t = std::variant<DerivedA, DerivedB>;
std::vector<var_t> vec { DerivedA(), DerivedB() };
for (auto& e : vec)
std::visit(Print(), e);
return 0;
}

Answer 4

If we have to store objects in array, their type must be fixed. Then we have these variants:
allocate dynamically and store pointers - if pointed objects are required to be continuous in memory, use custom allocator
use polymorphic type of a fixed size, e.g. union, as a storage type
For the second variant, the code could be something like this:
#include <new>
struct A {
A() {}
virtual void f() {}
};
struct B : A {
B() {}
void f() override {}
};
union U {
A a;
B b;
U() {}
};
int main() {
U u[2];
new (&u[0]) A;
new (&u[1]) B;
((A*)&u[0])->f(); // A::f
((A*)&u[1])->f(); // B::f
}

Answer 5

std::vector<T> iterators assume that the objects in the contiguous memory are of type T, std::vector<T>::iterator::operator++ considers sizeof T to be invariant - that is, it doesn't consult the specific instance for size data.
In essence, you can think of vector and vector::iterator as a thin facade over a T* m_data pointer, such that iterator++ is really just a basic pointer operation.
You will likely need to use a custom allocator and in-place new to prepare your data, accompanied by either indexing, linking etc. Perhaps consider something like http://www.boost.org/doc/libs/1_58_0/doc/html/intrusive/slist.html
See also boost::stable_vector

Answer 6

std::vector allocateS the objects in continuous memory but the object pointers you are storing within the vector are not. This is how you iterate through vector. Following code is written in c++14. The problem described is not solvable by this solution as the object pointers are stored in continuous memory but not the actual objects.
#include <iostream>
#include <memory>
#include <vector>
#include <algorithm>
using namespace std;
class Base
{
public:
Base() {}
virtual void doStuff() = 0;
};
class DerivedA : public Base
{
private:
//specific A member variables
public:
DerivedA() : Base() {}
virtual void doStuff() {
std::cout << "Derived Class A - Do Stuff" << std::endl;
}
};
class DerivedB : public Base
{
private:
//specific B member variables
public:
DerivedB() : Base() {}
virtual void doStuff() {
std::cout << "Derived Class B - Do Stuff" << std::endl;
}
};
int main() {
// your code goes here
std::vector<std::unique_ptr<Base> > container;
container.push_back(std::make_unique<DerivedA>());
container.push_back(std::make_unique<DerivedB>());
std::for_each(container.begin(), container.end(),[](std::unique_ptr<Base> & b) {
b->doStuff();
});
return 0;
}
Live Demo here.