Virtual function performance: one large class vs many smaller subclasses - c++

Question

I am in the process of refactoring a c++ OpenGL app I made (technically, an app that makes heavy use of the thin OpenGL wrapper in Qt's QQuickItem class). My app is performing ok but could be better.
One of the issues I'm curious about relates to the use of virtual functions in very time-sensitive (frame rate) algorithms. My OpenGL drawing code calls many virtual functions on the various objects that need drawing. Since this happens many times per second, I am wondering if the virtual dispatch could bring down the frame rate.
I am thinking about changing to this structure instead which avoids inheritance by keeping everything in one base class, but where the previously virtual functions now simply contain switch statements to call the appropriate routine based on the class's "type" which is really just a typedef enum:
Previously:
struct Base{
virtual void a()=0;
virtual void b()=0;
}
struct One : public Base{
void a(){...}
void b(){...}
}
Considering:
struct Combined{
MyEnumTypeDef t; //essentially holds what "type" of object this is
void a(){
switch (t){
case One:
....
break;
case Two:
....
break;
}
}
}
When function a() is called very often in OpenGL drawing routines, I am tempted to think that the Combined class will be considerably more efficient since it does not require the dynamic dispatch on virtual tables.
I'd appreciate some advice on this issue, if this is wise or not.

Answer 1

In your case, it probably does not matter. I say probably because, and I mean this constructively, the fact that you did not specify performance requirements and did not specify how often the function in question is called indicates that you may not have enough information to make a judgment now - the "don't speculate: profile" blanket response actually only intends to make sure you have all the necessary information required, because premature micro-optimizations are very common and our real goal is to help you out in the big picture.
Jeremy Friesner really hit the nail on the head with his comment on another answer here:
If you don't understand why it is slow you won't be able to speed it up.
So, with all that in mind, assuming that either A) Your performance requirements are already being met (e.g. you are pulling 4000 FPS - far higher than any display refresh rate) or B) You are struggling to meet performance requirements and this function is only called a few (say < 1000-ish) times per frame) or C) You are struggling to meet performance requirements and this function is called often but does a lot of other significant work (and thus function call overhead is negligible), then:
Using a virtual function may, at most, end up in one extra lookup in a table somewhere (and possibly some cache misses - but not so much if it is accessed repeatedly in, say, an inner loop), which is a few CPU clock cycles worst case (and most likely still less than a switch, although that is really moot here), and this is completely insignificant compared to your target frame rate, the effort required to render a frame, and any other algorithms and logic you are performing. If you'd like to prove it to yourself, profile.
What you should do is use whatever technique leads to the clearest, cleanest, most maintainable and readable code. Micro-optimizations such as this are not going to have an effect, and the cost in code maintainability, even if it is minor, is not worth the benefit, which is essentially zero.
What you should also do is sit down and get a handle on your actual situation. Do you need to improve performance? Is this function called enough to actually have a significant impact or should you be concentrating on other techniques (e.g. higher level algorithms, other design strategies, off-loading computation to the GPU or using machine-specific optimizations e.g. bulk operations with SSE, etc.)?
One thing you could do, in the absence of concrete information, is try both approaches. While performance will differ from machine to machine, you could at least get a rough idea of the impact of this particular bit of code on your overall performance (e.g. if you're shooting for 60 FPS, and these two options give you 23.2 FPS vs. 23.6 FPS, then this isn't where you want to focus, and possible sacrifices made by choosing one of these strategies over the other may not be worth it).
Also consider using call lists, vertex index buffers, etc. OpenGL provides many facilities for optimizing drawing of objects where certain aspects remain constant. For example, if you have a huge surface model with small parts whose vertex coordinates change often, divide the model into sections, using call lists, and only update the call list for the section that changed when it has changed since the last redraw. Leave e.g. coloring and texturing out of the call list (or use coordinate arrays) if they change often. That way you can avoid calling your functions altogether.
If you're curious, here is a test program (which probably does not represent your actual usage, again, this is not possible to answer with the information given - this test is the one requested in comments below). This does not mean these results will be reflected in your program and, again, you need to have concrete information about your actual requirements. But, here this is just for giggles:
This test program compares a switch-based operation vs. a virtual-function based operation vs. pointer-to-member (where member is called from another class member function) vs. pointer-to-member (where member is called directly from test loop). It also performs three types of tests: A run on a dataset with just one operator, a run that alternates back and forth between two operators, and a run that uses a random mix of two operators.
Output when compiled with gcc -O0, for 1,000,000,000 iterations:
$ g++ -O0 tester.cpp
$ ./a.out
--------------------
Test: time=6.34 sec (switch add) [-358977076]
Test: time=6.44 sec (switch subtract) [358977076]
Test: time=6.96 sec (switch alternating) [-281087476]
Test: time=18.98 sec (switch mixed) [-314721196]
Test: time=6.11 sec (virtual add) [-358977076]
Test: time=6.19 sec (virtual subtract) [358977076]
Test: time=7.88 sec (virtual alternating) [-281087476]
Test: time=19.80 sec (virtual mixed) [-314721196]
Test: time=10.96 sec (ptm add) [-358977076]
Test: time=10.83 sec (ptm subtract) [358977076]
Test: time=12.53 sec (ptm alternating) [-281087476]
Test: time=24.24 sec (ptm mixed) [-314721196]
Test: time=6.94 sec (ptm add (direct)) [-358977076]
Test: time=6.89 sec (ptm subtract (direct)) [358977076]
Test: time=9.12 sec (ptm alternating (direct)) [-281087476]
Test: time=21.19 sec (ptm mixed (direct)) [-314721196]
Output when compiled with gcc -O3, for 1,000,000,000 iterations:
$ g++ -O3 tester.cpp ; ./a.out
--------------------
Test: time=0.87 sec (switch add) [372023620]
Test: time=1.28 sec (switch subtract) [-372023620]
Test: time=1.29 sec (switch alternating) [101645020]
Test: time=7.71 sec (switch mixed) [855607628]
Test: time=2.95 sec (virtual add) [372023620]
Test: time=2.95 sec (virtual subtract) [-372023620]
Test: time=14.74 sec (virtual alternating) [101645020]
Test: time=9.39 sec (virtual mixed) [855607628]
Test: time=4.20 sec (ptm add) [372023620]
Test: time=4.21 sec (ptm subtract) [-372023620]
Test: time=13.11 sec (ptm alternating) [101645020]
Test: time=9.32 sec (ptm mixed) [855607628]
Test: time=3.37 sec (ptm add (direct)) [372023620]
Test: time=3.37 sec (ptm subtract (direct)) [-372023620]
Test: time=13.08 sec (ptm alternating (direct)) [101645020]
Test: time=9.74 sec (ptm mixed (direct)) [855607628]
Note that -O3 does a lot, and without looking at the assembler, we cannot use this as a 100% accurate representation of the problem at hand.
In the unoptimized case, we notice:
Virtual outperforms switch in runs of a single operator.
Switch outperforms virtual in cases where multiple operators used.
Pointer-to-member when the member is called directly (object->*ptm_) is similar to, but slower than, virtual.
Pointer-to-member when the member is called through another method (object->doit() where doit() calls this->*ptm_) takes a little under twice the time.
As expected, "mixed" case performance suffers due to branch prediction failures.
In the optimized case:
Switch outperforms virtual in all cases.
Similar characteristics for pointer-to-member as unoptimized case.
All "alternating" cases that involve function pointers at some point are slower than with -O0 and slower than "mixed" for reasons I do not understand. This does not occur on my PC at home.
What is especially significant here is how much the effects of e.g. branch prediction outweigh any choice of "virtual" vs. "switch". Again, be sure you understand your code and are optimizing the right thing.
The other significant thing here is this represents time differences on the order of 1-14 nanoseconds per operation. This difference could be significant for large numbers of operations, but is likely negligible compared to other things you are doing (note that these functions perform only a single arithmetic operation, any more than that will quickly dwarf the effects of virtual vs. switch).
Note also that while calling the pointer-to-member directly showed an "improvement" over calling it through another class member, this has potentially large impacts on overall design as such an implementation (at least in this case, where something outside the class calls the member directly) could not be dropped in as a direct replacement for another implementation due to different syntax for calling pointer-to-member functions (-> vs. ->*). I had to create a whole separate set of test cases just to handle that, for example.
Conclusion
Performance difference will easily be dwarfed by even a couple extra arithmetic operations. Note also that branch prediction has a far more significant impact in all but the "virtual alternating" case with -O3. However, test is also unlikely to be representative of actual application (which the OP has kept a secret), and -O3 introduces even more variables, and so results must be taken with a grain of salt and are unlikely to be applicable to other scenarios (in other words, test may be interesting, but is not particularly meaningful).
Source:
// === begin timing ===
#ifdef __linux__
# include <sys/time.h>
typedef struct timeval Time;
static void tick (Time &t) {
gettimeofday(&t, 0);
}
static double delta (const Time &a, const Time &b) {
return
(double)(b.tv_sec - a.tv_sec) +
(double)(b.tv_usec - a.tv_usec) / 1000000.0;
}
#else // windows; untested, working from memory; sorry for compile errors
# include <windows.h>
typedef LARGE_INTEGER Time;
static void tick (Time &t) {
QueryPerformanceCounter(&t);
}
static double delta (const Time &a, const Time &b) {
LARGE_INTEGER freq;
QueryPerformanceFrequency(&freq);
return (double)(b.QuadPart - a.QuadPart) / (double)freq.QuadPart;
}
#endif
// === end timing
#include <cstdio>
#include <cstdlib>
#include <ctime>
using namespace std;
// Size of dataset.
static const size_t DATASET_SIZE = 10000000;
// Repetitions per test.
static const unsigned REPETITIONS = 100;
// Class performs operations with a switch statement.
class OperatorSwitch {
public:
enum Op { Add, Subtract };
explicit OperatorSwitch (Op op) : op_(op) { }
int perform (int a, int b) const {
switch (op_) {
case Add: return a + b;
case Subtract: return a - b;
}
}
private:
Op op_;
};
// Class performs operations with pointer-to-member.
class OperatorPTM {
public:
enum Op { Add, Subtract };
explicit OperatorPTM (Op op) {
perform_ = (op == Add) ?
&OperatorPTM::performAdd :
&OperatorPTM::performSubtract;
}
int perform (int a, int b) const { return (this->*perform_)(a, b); }
int performAdd (int a, int b) const { return a + b; }
int performSubtract (int a, int b) const { return a - b; }
//private:
int (OperatorPTM::*perform_) (int, int) const;
};
// Base class for virtual-function test operator.
class OperatorBase {
public:
virtual ~OperatorBase () { }
virtual int perform (int a, int b) const = 0;
};
// Addition
class OperatorAdd : public OperatorBase {
public:
int perform (int a, int b) const { return a + b; }
};
// Subtraction
class OperatorSubtract : public OperatorBase {
public:
int perform (int a, int b) const { return a - b; }
};
// No base
// Addition
class OperatorAddNoBase {
public:
int perform (int a, int b) const { return a + b; }
};
// Subtraction
class OperatorSubtractNoBase {
public:
int perform (int a, int b) const { return a - b; }
};
// Processes the dataset a number of times, using 'oper'.
template <typename T>
static void test (const int *dataset, const T *oper, const char *name) {
int result = 0;
Time start, stop;
tick(start);
for (unsigned n = 0; n < REPETITIONS; ++ n)
for (size_t i = 0; i < DATASET_SIZE; ++ i)
result = oper->perform(result, dataset[i]);
tick(stop);
// result is computed and printed so optimizations do not discard it.
printf("Test: time=%.2f sec (%s) [%i]\n", delta(start, stop), name, result);
fflush(stdout);
}
// Processes the dataset a number of times, alternating between 'oper[0]'
// and 'oper[1]' per element.
template <typename T>
static void testalt (const int *dataset, const T * const *oper, const char *name) {
int result = 0;
Time start, stop;
tick(start);
for (unsigned n = 0; n < REPETITIONS; ++ n)
for (size_t i = 0; i < DATASET_SIZE; ++ i)
result = oper[i&1]->perform(result, dataset[i]);
tick(stop);
// result is computed and printed so optimizations do not discard it.
printf("Test: time=%.2f sec (%s) [%i]\n", delta(start, stop), name, result);
fflush(stdout);
}
// Processes the dataset a number of times, choosing between 'oper[0]'
// and 'oper[1]' randomly (based on value in dataset).
template <typename T>
static void testmix (const int *dataset, const T * const *oper, const char *name) {
int result = 0;
Time start, stop;
tick(start);
for (unsigned n = 0; n < REPETITIONS; ++ n)
for (size_t i = 0; i < DATASET_SIZE; ++ i) {
int d = dataset[i];
result = oper[d&1]->perform(result, d);
}
tick(stop);
// result is computed and printed so optimizations do not discard it.
printf("Test: time=%.2f sec (%s) [%i]\n", delta(start, stop), name, result);
fflush(stdout);
}
// Same as test() but calls perform_() pointer directly.
static void test_ptm (const int *dataset, const OperatorPTM *oper, const char *name) {
int result = 0;
Time start, stop;
tick(start);
for (unsigned n = 0; n < REPETITIONS; ++ n)
for (size_t i = 0; i < DATASET_SIZE; ++ i)
result = (oper->*(oper->perform_))(result, dataset[i]);
tick(stop);
// result is computed and printed so optimizations do not discard it.
printf("Test: time=%.2f sec (%s) [%i]\n", delta(start, stop), name, result);
fflush(stdout);
}
// Same as testalt() but calls perform_() pointer directly.
static void testalt_ptm (const int *dataset, const OperatorPTM * const *oper, const char *name) {
int result = 0;
Time start, stop;
tick(start);
for (unsigned n = 0; n < REPETITIONS; ++ n)
for (size_t i = 0; i < DATASET_SIZE; ++ i) {
const OperatorPTM *op = oper[i&1];
result = (op->*(op->perform_))(result, dataset[i]);
}
tick(stop);
// result is computed and printed so optimizations do not discard it.
printf("Test: time=%.2f sec (%s) [%i]\n", delta(start, stop), name, result);
fflush(stdout);
}
// Same as testmix() but calls perform_() pointer directly.
static void testmix_ptm (const int *dataset, const OperatorPTM * const *oper, const char *name) {
int result = 0;
Time start, stop;
tick(start);
for (unsigned n = 0; n < REPETITIONS; ++ n)
for (size_t i = 0; i < DATASET_SIZE; ++ i) {
int d = dataset[i];
const OperatorPTM *op = oper[d&1];
result = (op->*(op->perform_))(result, d);
}
tick(stop);
// result is computed and printed so optimizations do not discard it.
printf("Test: time=%.2f sec (%s) [%i]\n", delta(start, stop), name, result);
fflush(stdout);
}
int main () {
int *dataset = new int[DATASET_SIZE];
srand(time(NULL));
for (int n = 0; n < DATASET_SIZE; ++ n)
dataset[n] = rand();
OperatorSwitch *switchAdd = new OperatorSwitch(OperatorSwitch::Add);
OperatorSwitch *switchSub = new OperatorSwitch(OperatorSwitch::Subtract);
OperatorSwitch *switchAlt[2] = { switchAdd, switchSub };
OperatorBase *virtAdd = new OperatorAdd();
OperatorBase *virtSub = new OperatorSubtract();
OperatorBase *virtAlt[2] = { virtAdd, virtSub };
OperatorPTM *ptmAdd = new OperatorPTM(OperatorPTM::Add);
OperatorPTM *ptmSub = new OperatorPTM(OperatorPTM::Subtract);
OperatorPTM *ptmAlt[2] = { ptmAdd, ptmSub };
while (true) {
printf("--------------------\n");
test(dataset, switchAdd, "switch add");
test(dataset, switchSub, "switch subtract");
testalt(dataset, switchAlt, "switch alternating");
testmix(dataset, switchAlt, "switch mixed");
test(dataset, virtAdd, "virtual add");
test(dataset, virtSub, "virtual subtract");
testalt(dataset, virtAlt, "virtual alternating");
testmix(dataset, virtAlt, "virtual mixed");
test(dataset, ptmAdd, "ptm add");
test(dataset, ptmSub, "ptm subtract");
testalt(dataset, ptmAlt, "ptm alternating");
testmix(dataset, ptmAlt, "ptm mixed");
test_ptm(dataset, ptmAdd, "ptm add (direct)");
test_ptm(dataset, ptmSub, "ptm subtract (direct)");
testalt_ptm(dataset, ptmAlt, "ptm alternating (direct)");
testmix_ptm(dataset, ptmAlt, "ptm mixed (direct)");
}
}

Answer 2

The model of "lots of objects that draw themselves" is attractive, but bad in a sneaky way. It's not the virtual function call overhead (which exists, but is small), it encourages a rendering anti-pattern: letting every object draw itself in isolation. It sounds like one of those things touted in "software engineering best practices", but it's not, it's very bad. Every object would make a lot of expensive API calls (such as binding shaders and textures). Now, I don't really know what your code looks like, maybe it doesn't work like this, the objects aren't necessarily bad, it's how they're used.
Anyway, here are some suggestions.
Sort your objects by the state (shader, texture, vertex buffer, in that order) they want (actually, don't sort - put them in buckets and iterate of those). This is easy, everyone does it, and it may be enough.
Merge states, so there's nothing to switch between. Use übershaders. Use texture arrays, or better, bindless textures (which doesn't have the problem that all slices have to be the same format/size/etc). Use a huge vertex buffer that you put everything into. Use uniform buffers. Use persistent mapping for the dynamic buffers.
And finally, glMultiDrawElementsIndirect. If you've put everything into buffers anyway, as per the previous suggestion, then you will need only very few calls to glMultiDrawElementsIndirect. Very few, you can do a lot with just one call. What you'd probably use otherwise is a bunch of glDrawArrays with no binding in between, which isn't bad either, but it's not much effort to make it even better.
The end result is that the actual drawing code has almost disappeared. Almost all API calls are gone, replaced by writes to buffers.

Answer 3

It would be faster to not use virtual functions, but whether the difference is significant is hard to say. You should run your program through a profiler to see where it's spending its time. You might discover that the cpu cycles are spent on something entirely different and you'd be wasting your time (and degrading your design) by messing with the virtuals.
Unix: How can I profile C++ code running in Linux?
Windows: What's the best free C++ profiler for Windows?
Another option to consider might be to use the Curiously Recurring Template Pattern: http://en.wikipedia.org/wiki/Curiously_recurring_template_pattern to get similar polymorphism without using virtuals.