Avoiding performance concerns of Runtime Polymorphism - c++

Question

In a numerical code run on thousands of processors for 10s of hours, I have a base class (Mesh) whose methods are hit 100s to 1000s of millions of times. There are currently two (Mesh_A, Mesh_B) derived classes, but eventually this will expand to three or four. User code cannot know until runtime whether its pointer to Mesh is actually a Mesh_A or Mesh_B, but for the rest of the run, it never changes.
Current Implementation:
// Base class
class Mesh {
...
virtual const Point& cell_centroid(int c) = 0;
}
// derived class A
class MeshA : public Mesh {
...
Point& cell_centroid(int c) { return cell_centroids_[c]; }
}
// derived class B
class MeshB : public Mesh {
...
Point& cell_centroid(int c) { return other_framework_->cell_centroid(c); }
}
// typical user class
class User {
User(Mesh* mesh) : mesh_(mesh) {}
void evalFunction() {
for (int c=0; c!=mesh_->num_cells(); ++c) {
double result = func(mesh_->cell_centroid(c));
...
}
}
// Other methods which use mesh_->cell_centroid() very often, and in different ways.
}
Previously, MeshA was the only Mesh, and there was no base class, and the heavily hit methods were all inlined. Profiling shows that the change to runtime polymorphism (likely thanks to the loss of inlining?) with virtual methods has resulted in a ~15% hit, which just isn't going to fly.
I've been pouring over static polymorphism and other ideas, but I'd love to hear thoughts on how one might avoid this hit in a reasonably sustainable way.
Idea 1: Coarsen the virtual function to amortize overhead. One thought was to try to encapsulate all the "calling patterns" of these methods inside a virtual method, lifting the virtual to a coarser level while keeping the fine-grained methods as non-virtual. For example, in the above example, one could pass a function pointer to a new virtual method of Mesh that implemented the loop, returning an array of doubles and called a non-virtual, inlined cell_centroid() method inside of this.
// Base class
class Mesh {
...
virtual void evalFunction(double (*func)(Point&), std::vector<double>* result) = 0;
}
// derived class A
class MeshA : public Mesh {
...
void evalFunction(double (*func)(Point&), std::vector<double>* result) {
for (int c=0; c!=num_cells(); ++c) (*result)[c] = (*func)(cell_centroid(c));
}
Point& cell_centroid(int c) { return cell_centroids_[c]; }
}
// similar for B
// typical user class
class User {
User(Mesh* mesh) : mesh_(mesh) {}
void evalFunction() {
m_->evalFunction();
}
}
I'm a little nervous that this will make the Mesh interface huge -- I don't have a single access pattern (like the example) that could easily be encapsulated. My guess is that, for every virtual method in the current Mesh class (15-20), I'd have 3 or 4 different "calling patterns", and the interface for Mesh would explode. There are a variety of "User" classes and, while the sometimes use Mesh the same way, they don't always, and I don't want to limit myself to a few patterns.
Idea 2: Template all user code with Mesh_T. Write a factory that creates User<MeshA> or User<MeshB> instances depending upon runtime information. This is a little concerning because this will effectively mean that my entire code is templated code, compile times will blow up, errors will be harder to debug etc. A large code base would be touched.
Idea 3: It seems to me that one ought to be able to resolve, at the start of the run, that the Mesh pointer User gets is actually a MeshA or MeshB, and not need to do the virtual table lookups and regain the inlined A or B implementation. I don't know of an elegant way of doing this that wouldn't basically be worse than Idea 1, i.e. a bunch of duplicated code in User with case/switch. But if there were an elegant way of doing this, it would be my first choice.
Any thoughts on a good choice, a better idea, or other comments on runtime polymorphism of a high level class without virtual low-level methods would be appreciated!

Answer 1

Provided I understood you correctly that mesh_ will always be either MeshA or MeshB and not a mix of them.
// typical user class
class User {
User(Mesh* mesh) : mesh_(mesh) {}
template<class dType>
void evalFunction() {
dType *myMesh = dynamic_cast<dType *>(mesh_);
for (int c=0; c!=myMesh _->num_cells(); ++c) {
double result = func(myMesh _->cell_centroid(c));
...
}
}
void evalFunction() {
if (dynamic_cast<MeshA *>(mesh_))
evalFunction<MeshA>();
if (dynamic_cast<MeshB *>(mesh_))
evalFunction<MeshB>();
}
}
evalFunction chooses either A or B template.
Alternatively
class User {
User(Mesh* mesh) : mesh_(mesh) {}
template<class dType>
void evalFunction(dType *myMesh) {
for (int c=0; c!=myMesh _->num_cells(); ++c) {
double result = func(myMesh _->cell_centroid(c));
...
}
}
void evalFunction() {
MeshA *meshA = dynamic_cast<MeshA *>(mesh_);
if (meshA)
evalFunction<MeshA>(meshA);
MeshB *meshB = dynamic_cast<MeshB *>(mesh_);
if (meshB)
evalFunction<MeshB>(meshB);
}
}