Profiling my C++ code with gprof, I discovered that a significant portion of my time is spent calling one virtual method over and over. The method itself is short and could probably be inlined if it wasn't virtual.
What are some ways I could speed this up short of rewriting it all to not be virtual?
Are you sure the time is all call-related? Could it be the function itself where the cost is? If this is the case simply inlining things might make the function vanish from your profiler but you won't see much speed-up.
Assuming it really is the overhead of making so many virtual calls there's a limit to what you can do without making things non-virtual.
If the call has early-outs for things like time/flags then I'll often use a two-level approach. The checking is inlined with a non-virtual call, with the class-specific behavior only called if necessary.
E.g.
class Foo
{
public:
inline void update( void )
{
if (can_early_out)
return;
updateImpl();
}
protected:
virtual void updateImpl( void ) = 0;
};
If the virtual calling really is the bottleneck give CRTP a try.
Is the time being spent in the actual function call, or in the function itself?
A virtual function call is noticeably slower than a non-virtual call, because the virtual call requires an extra dereference. (Google for 'vtable' if you want to read all the hairy details.) )Update: It turns out the Wikipedia article isn't bad on this.
"Noticeably" here, though, means a couple of instructions If it's consuming a significant part of the total computation including time spent in the called function, that sounds like a marvelous place to consider unvirtualizing and inlining.
But in something close to 20 years of C++, I don't think I've ever seen that really happen. I'd love to see the code.
Please be aware that "virtual" and "inline" are not opposites -- a method can be both. The compiler will happily inline a virtual function if it can determine the type of the object at compile time:
struct B {
virtual int f() { return 42; }
};
struct D : public B {
virtual int f() { return 43; }
};
int main(int argc, char **argv) {
B b;
cout << b.f() << endl; // This call will be inlined
D d;
cout << d.f() << endl; // This call will be inlined
B& rb = rand() ? b : d;
cout << rb.f() << endl; // Must use virtual dispatch (i.e. NOT inlined)
return 0;
}
[UPDATE: Made certain rb's true dynamic object type cannot be known at compile time -- thanks to MSalters]
If the type of the object can be determined at compile time but the function is not inlineable (e.g. it is large or is defined outside of the class definition), it will be called non-virtually.
It's sometimes instructive to consider how you'd write the code in good old 'C' if you didn't have C++'s syntactic sugar available. Sometimes the answer isn't using an indirect call. See this answer for an example.
You might be able get a little better performance from the virtual call by changing the calling convention. The old Borland compiler had a __fastcall convention which passed arguments in cpu registers instead of on the stack.
If you're stuck with the virtual call and those few operations really count, then check your compiler documentation for supported calling conventions.
Here is one possible way to do it using RTTI.
Related
Virtual function calls can be slow due to virtual calls requiring an extra indexed deference to the v-table, which can result in a data cache miss as well as an instruction cache miss... Not good for performance critical applications.
So I have been thinking of a way to overcome this performance issue of virtual functions yet still having some of the same functionality that virtual functions provide.
I am confident that this has been done before, but I devised a simple test that allows the base class to store a member function pointer that can be set by any the derived class. And when I call Foo() on any derived class, it will call the appropriate member function without having to traverse the v-table...
I am just wondering if this method is a viable replacement for the virtual-call paradigm, if so, why is it not more ubiquitous?
Thanks in advance for your time! :)
class BaseClass
{
protected:
// member function pointer
typedef void(BaseClass::*FooMemFuncPtr)();
FooMemFuncPtr m_memfn_ptr_Foo;
void FooBaseClass()
{
printf("FooBaseClass() \n");
}
public:
BaseClass()
{
m_memfn_ptr_Foo = &BaseClass::FooBaseClass;
}
void Foo()
{
((*this).*m_memfn_ptr_Foo)();
}
};
class DerivedClass : public BaseClass
{
protected:
void FooDeriveddClass()
{
printf("FooDeriveddClass() \n");
}
public:
DerivedClass() : BaseClass()
{
m_memfn_ptr_Foo = (FooMemFuncPtr)&DerivedClass::FooDeriveddClass;
}
};
int main(int argc, _TCHAR* argv[])
{
DerivedClass derived_inst;
derived_inst.Foo(); // "FooDeriveddClass()"
BaseClass base_inst;
base_inst.Foo(); // "FooBaseClass()"
BaseClass * derived_heap_inst = new DerivedClass;
derived_heap_inst->Foo();
return 0;
}
I did a test, and the version using virtual function calls was faster on my system with optimization.
$ time ./main 1
Using member pointer
real 0m3.343s
user 0m3.340s
sys 0m0.002s
$ time ./main 2
Using virtual function call
real 0m2.227s
user 0m2.219s
sys 0m0.006s
Here is the code:
#include <cstdlib>
#include <cstring>
#include <iostream>
#include <stdio.h>
struct BaseClass
{
typedef void(BaseClass::*FooMemFuncPtr)();
FooMemFuncPtr m_memfn_ptr_Foo;
void FooBaseClass() { }
BaseClass()
{
m_memfn_ptr_Foo = &BaseClass::FooBaseClass;
}
void Foo()
{
((*this).*m_memfn_ptr_Foo)();
}
};
struct DerivedClass : public BaseClass
{
void FooDerivedClass() { }
DerivedClass() : BaseClass()
{
m_memfn_ptr_Foo = (FooMemFuncPtr)&DerivedClass::FooDerivedClass;
}
};
struct VBaseClass {
virtual void Foo() = 0;
};
struct VDerivedClass : VBaseClass {
virtual void Foo() { }
};
static const size_t count = 1000000000;
static void f1(BaseClass* bp)
{
for (size_t i=0; i!=count; ++i) {
bp->Foo();
}
}
static void f2(VBaseClass* bp)
{
for (size_t i=0; i!=count; ++i) {
bp->Foo();
}
}
int main(int argc, char** argv)
{
int test = atoi(argv[1]);
switch (test) {
case 1:
{
std::cerr << "Using member pointer\n";
DerivedClass d;
f1(&d);
break;
}
case 2:
{
std::cerr << "Using virtual function call\n";
VDerivedClass d;
f2(&d);
break;
}
}
return 0;
}
Compiled using:
g++ -O2 main.cpp -o main
with g++ 4.7.2.
Virtual function calls can be slow due to virtual calls having to traverse the v-table,
That's not quite correct. The vtable should be computed on object construction, with each virtual function pointer set to the most specialized version in the hierarchy. The process of calling a virtual function does not iterate pointers but call something like *(vtbl_address + 8)(args);, which is computed in constant time.
which can result in a data cache miss as well as an instruction cache miss... Not good for performance critical applications.
Your solution is not good for performance critical applications (in general) either, because it is generic.
As a rule, performance critical applications are optimized on a per-case basis (measure, pick code with worst performance problems within module and optimize).
With this per-case approach, you will probably never have a case where your code is slow because the compiler has to traverse a vtbl. If that is the case, the slowness would probably come from calling functions through pointers instead of directly (i.e. the problem would be solved by inlining, not by adding an extra pointer in the base class).
All this is academic anyway, until you have a concrete case to optimize (and you have measured that your worst offender is virtual function calls).
Edit:
I am just wondering if this method is a viable replacement for the virtual-call paradigm, if so, why is it not more ubiquitous?
Because it looks like a generic solution (applying it ubiquitously would decrease performance instead of improving it), solving a non-existent problem (your application is generally not slowed down due to virtual function calls).
Virtual functions do not "traverse" the table, just do a single fetch of a pointer from a location and call that address. That as if you had a manual implementation of a pointer-to-funciton and used that for a call instead of a direct one.
So your work is only good for obfuscation, and sabotage the cases where the compiler can issue nonvirtual direct call.
Using a pointer-to-memberfunction is probably even worse than PTF, it will likely use the same VMT structure for an similar offseted access, just a variable one instead of fixed.
Mostly because it doesn't work. Most modern CPUs are better at branch prediction and speculative execution than you think. However I have yet to see a CPU that do speculative execution beyond a non-static branch.
Furthermore in a modern CPU you are more likely to have a cache miss because you had a context switch just prior to the call and another program took over the cache than you are because of a v-table, even this scenario is a very remote possiblity.
Actually some compilers may use thunks, which translate to ordinary function pointers themselves, so basically the compiler does for you what you are trying to do manually (and probably confuse the hell out of people).
Also, having a pointer to virtual function table, the space complexity of virtual function is O(1) (just the pointer). On the other hand, if you store function pointers within the class, then the complexity is O(N) (your class now contains as many pointers as there are "virtual" functions). If there are many functions, you are paying toll for that - when pre-fetching your object, you are loading all the pointers in the cache line, instead of just a single pointer and the first few members which you are likely to need. That sounds like a waste.
The virtual function table, on the other hand, sits in one place for all the objects of one type and is likely never pushed out of the cache while your code calls some short virtual functions in a loop (which is presumably the problem when virtual function cost would become the bottleneck).
As to the branch prediction, in some cases a simple decision tree over object type and inlined functions for each particular type give good performance (then you store type information instead of a pointer). This is not applicable to all types of problems and would be mostly a premature optimization.
As a rule of thumb, don't worry about the language constructs because they seem unfamiliar. Worry and optimize only after you have measured and identified where the bottleneck really is.
I'm developing an application which is heavily based on math (sin, cos, sqrt etc).
These functions take some time to run but have high precision.
Some of my clients don't need that high precision, but they need it to be as fast as possible.
So I have my Sin function which is a simple array (which is created before the program starts to run) that takes a degree between 0 and 360 and returns its sin (let's say the array has 360 values).
I want to create an interface:
interface MyMath
{
double PreciseSin(double x);
double PreciseCos(double x);
}
It will be inherited by
"Precise math" which implementation will call the normal sin,cos function.
"Fast math" which will use the array trick I explained earlier.
My code will use a variable of type "mymath" to do the calculations, and in the beginning it will be initialized with preciseMath or fastMath.
Finally my questions are:
How much penalty in time I will pay for calling a virtual function that calls "Math.sin" instead of calling it directly?
Will the compiler will be able to optimize it and understand that in case I initialize MyMath with PriciseMath all I want is to call the normal Sin and Cos functions?
Can I change my design to help the compiler understand and optimize my code?
Odds are that your sqrt, and trig functions will have a much higher cost then the function call, even if its virtual. However this looks like the perfect place to use templates. If you use them correctly you can completely remove the run time cost of the function calls as all of them can be inlined.
class PreciseMath{
public:
inline double sin(double sin){
//code goes here
}
inline double cos(double sin){
//code goes here
}
inline double sqrt(double sin){
//code goes here
}
};
class FastMath{
public:
inline double sin(double sin){
//code goes here
}
inline double cos(double sin){
//code goes here
}
inline double sqrt(double sin){
//code goes here
}
};
template<class T>
class ExpensiveOP{
public:
T math;
void do(){
double x = math.sin(9);
x=math.cos(x);
//etc
}
}
ExpensiveOP<PreciseMath> preciseOp;
ExpensiveOP<FastMath> fasterOp;
For one and two:
You'll pay the same amount as calling a function through a function pointer. This isn't very much.
No, the compiler won't optimise virtual function calls into static function calls because it can't know that the type won't change at runtime somehow (like getting a pointer from some outside code that it doesn't know anything about). Delnan informed me in the comments that in very simple cases such as A* a = new A; a->func() the compiler can see that a will never be anything other than an A, so it can perform "devirtualisation" and optimise the virtual function call into a static function call. However, the cases where it can do this are somewhat rare, and if you got the pointer from, say, the argument to a function, it can't do this because it could actually be a derived type.
I don't know of any design that could make your code faster than that, besides "compile-time virtual functions" (aka. CRTP) but you lose polymorphism if you go this way. Try it with virtual functions and profile it; if it is too slow for you, then you can try another route, but don't waste time trying to make it faster without knowing how fast it already is.
How much penalty in time I will pay for calling a virtual function that calls "Math.sin" instead of calling it directly?
A virtual call is implemented by dereferencing the virtual table pointer, getting the function pointer from the appropriate offset in the virtual table and calling the function through that pointer.
This is slightly costlier than the static call, but is still considered very cheap for "normal" usage. If you need to squeeze every last drop of performance, consider specifying all your types at compile-time to allow usage of non-virtual functions.
Will the compiler will be able to optimize it and understand that in case I initialize MyMath with PriciseMath all I want is to call the normal Sin and Cos functions?
If compiler can prove (to itself) that an object will have a particular type at the run-time, then it will be able to emit a static function call, even when the function itself is declared as virtual.
However, the compiler is not guaranteed to be smart enough to actually do it. The only way to guarantee the static call is to use the non-virtual function.
Can I change my design to help the compiler understand and optimize my code?
Eliminate virtual call overhead: If there is no need to change the implementation at run-time, then specify the types at compile-time and stop using virtual functions altogether. Templates can be indispensable in doing so in a generic way.
Eliminate the static function call overhead: Provide function bodies in headers, allowing compiler to inline function calls.
(The RTS's answer is a nice illustration of both these techniques.)
And finally, if performance is really important to you, don't just rely on advice of other people (including me) - always perform the measurements yourself!
If you can provide different binaries, just do conditional compilation:
namespace MyMath {
#ifdef FAST_MATH
double sin(double x) { /* do it fast */ }
double sos(double x) { /* do it fast */ }
#else
double sin(double x) { /* do it precise */ }
double sos(double x) { /* do it precise */ }
#endif
}
And then call your compiler with -DFAST_MATH to generate the fast binary and without for the exact binary.
As I understand it, the compiler can inline a virtual function call when it knows at compile time what the type of the object will be at runtime (C++ faq).
What happens, however, when one is implementing a pure virtual method from a base class? Do the same rules apply? Will the following function call be inlined?
class base
{
public:
virtual void print() = 0;
virtual void callPrint()
{
print(); // will this be inline?
}
};
class child : public base
{
public:
void print() { cout << "hello\n"; }
};
int main()
{
child c;
c.callPrint();
return 0;
}
EDIT:
I think my original example code was actually a poor representation of what I wanted to ask. I've updated the code, but the question remains the same.
The compiler is never required to inline a function call. In this case, it is permitted to inline the function call, because it knows the concrete type of c (since it is not indirected through a pointer or reference, the compiler can see where it was allocated as a child). As such, the compiler knows which implementation of print() is used, and can choose not to perform vtable indirection, and further choose to inline the implementation of the function.
However, the compiler is also free to not inline it; it might insert a direct call to child::print(), or indirect through the vtable, if it decides to do so.
These optimizations in general boil down to the 'as-if' rule - the compiler must behave as-if it was doing a full vtable indirect - this means that the result must be the same, but the compiler can choose a different method of achieving the result if the result is the same. This includes inlining, etc.
The answer is of course "it depends", but in principle there's no obstruction to optimization. In fact, you're not even doing anything polymorphic here, so this is really straight-forward.
The question would be more interesting if you had code like this:
child c;
base & b = c;
b.print();
The point is that the compiler knows at this point what the ultimate target of the dynamic dispatch will be (namly child::print()), so this is eligible for optimization. (There are two separate opportunities for optimization, of course: one by avoiding the dynamic dispatch, and one coming from having the function body of the target visible in the TU.)
There are only a couple of rules you should be aware of:
1) The compiler is never forced to inline - even using the directive or defining a method in the header.
2) Polymorphism MUST ALWAYS WORK. This means that the compiler will prefer calling the function via the vftable rather than inlining it when the possibility of dynamic calls exists.
Comparing virtual functions in C++ and virtual tables in C, do compilers in general (and for sufficiently large projects) do as good a job at devirtualization?
Naively, it seems like virtual functions in C++ have slightly more semantics, thus may be easier to devirtualize.
Update: Mooing Duck mentioned inlining devirtualized functions. A quick check shows missed optimizations with virtual tables:
struct vtab {
int (*f)();
};
struct obj {
struct vtab *vtab;
int data;
};
int f()
{
return 5;
}
int main()
{
struct vtab vtab = {f};
struct obj obj = {&vtab, 10};
printf("%d\n", obj.vtab->f());
}
My GCC will not inline f, although it is called directly, i.e., devirtualized. The equivalent in C++,
class A
{
public:
virtual int f() = 0;
};
class B
{
public:
int f() {return 5;}
};
int main()
{
B b;
printf("%d\n", b.f());
}
does even inline f. So there's a first difference between C and C++, although I don't think that the added semantics in the C++ version are relevant in this case.
Update 2: In order to devirtualize in C, the compiler has to prove that the function pointer in the virtual table has a certain value. In order to devirtualize in C++, the compiler has to prove that the object is an instance of a particular class. It would seem that the proof is harder in the first case. However, virtual tables are typically modified in only very few places, and most importantly: just because it looks harder, doesn't mean that compilers aren't as good in it (for otherwise you might argue that xoring is generally faster than adding two integers).
The difference is that in C++, the compiler can guarantee that the virtual table address never changes. In C then it's just another pointer and you could wreak any kind of havoc with it.
However, virtual tables are typically modified in only very few places
The compiler doesn't know that in C. In C++, it can assume that it never changes.
I tried to summarize in http://hubicka.blogspot.ca/2014/01/devirtualization-in-c-part-2-low-level.html why generic optimizations have hard time to devirtualize. Your testcase gets inlined for me with GCC 4.8.1, but in slightly less trivial testcase where you pass pointer to your "object" out of main it will not.
The reason is that to prove that the virtual table pointer in obj and the virtual table itself did not change the alias analysis module has to track all possible places you can point to it. In a non-trivial code where you pass things outside of the current compilation unit this is often a lost game.
C++ gives you more information on when type of object may change and when it is known. GCC makes use of it and it will make a lot more use of it in the next release. (I will write on that soon, too).
Yes, if it is possible for the compiler to deduce the exact type of a virtualized type, it can "devirtualize" (or even inline!) the call. A compiler can only do this if it can guarantee that no matter what, this is the function needed.
The major concern is basically threading. In the C++ example, the guarantees hold even in a threaded environment. In C, that can't be guaranteed, because the object could be grabbed by another thread/process, and overwritten (deliberately or otherwise), so the function is never "devirtualized" or called directly. In C the lookup will always be there.
struct A {
virtual void func() {std::cout << "A";};
}
struct B : A {
virtual void func() {std::cout << "B";}
}
int main() {
B b;
b.func(); //this will inline in optimized builds.
}
It depends on what you are comparing compiler inlining to. Compared to link time or profile guided or just in time optimizations, compilers have less information to use. With less information, the compile time optimizations will be more conservative (and do less inlining overall).
A compiler will still generally be pretty decent at inlining virtual functions as it is equivalent to inlining function pointer calls (say, when you pass a free function to an STL algorithm function like sort or for_each).
Having at least one virtual method in a C++ class (or any of its parent classes) means that the class will have a virtual table, and every instance will have a virtual pointer.
So the memory cost is quite clear. The most important is the memory cost on the instances (especially if the instances are small, for example if they are just meant to contain an integer: in this case having a virtual pointer in every instance might double the size of the instances. As for the memory space used up by the virtual tables, I guess it is usually negligible compared to the space used up by the actual method code.
This brings me to my question: is there a measurable performance cost (i.e. speed impact) for making a method virtual? There will be a lookup in the virtual table at runtime, upon every method call, so if there are very frequent calls to this method, and if this method is very short, then there might be a measurable performance hit? I guess it depends on the platform, but has anyone run some benchmarks?
The reason I am asking is that I came across a bug that happened to be due to a programmer forgetting to define a method virtual. This is not the first time I see this kind of mistake. And I thought: why do we add the virtual keyword when needed instead of removing the virtual keyword when we are absolutely sure that it is not needed? If the performance cost is low, I think I will simply recommend the following in my team: simply make every method virtual by default, including the destructor, in every class, and only remove it when you need to. Does that sound crazy to you?
I ran some timings on a 3ghz in-order PowerPC processor. On that architecture, a virtual function call costs 7 nanoseconds longer than a direct (non-virtual) function call.
So, not really worth worrying about the cost unless the function is something like a trivial Get()/Set() accessor, in which anything other than inline is kind of wasteful. A 7ns overhead on a function that inlines to 0.5ns is severe; a 7ns overhead on a function that takes 500ms to execute is meaningless.
The big cost of virtual functions isn't really the lookup of a function pointer in the vtable (that's usually just a single cycle), but that the indirect jump usually cannot be branch-predicted. This can cause a large pipeline bubble as the processor cannot fetch any instructions until the indirect jump (the call through the function pointer) has retired and a new instruction pointer computed. So, the cost of a virtual function call is much bigger than it might seem from looking at the assembly... but still only 7 nanoseconds.
Edit: Andrew, Not Sure, and others also raise the very good point that a virtual function call may cause an instruction cache miss: if you jump to a code address that is not in cache then the whole program comes to a dead halt while the instructions are fetched from main memory. This is always a significant stall: on Xenon, about 650 cycles (by my tests).
However this isn't a problem specific to virtual functions because even a direct function call will cause a miss if you jump to instructions that aren't in cache. What matters is whether the function has been run before recently (making it more likely to be in cache), and whether your architecture can predict static (not virtual) branches and fetch those instructions into cache ahead of time. My PPC does not, but maybe Intel's most recent hardware does.
My timings control for the influence of icache misses on execution (deliberately, since I was trying to examine the CPU pipeline in isolation), so they discount that cost.
There is definitely measurable overhead when calling a virtual function - the call must use the vtable to resolve the address of the function for that type of object. The extra instructions are the least of your worries. Not only do vtables prevent many potential compiler optimizations (since the type is polymorphic the compiler) they can also thrash your I-Cache.
Of course whether these penalties are significant or not depends on your application, how often those code paths are executed, and your inheritance patterns.
In my opinion though, having everything as virtual by default is a blanket solution to a problem you could solve in other ways.
Perhaps you could look at how classes are designed/documented/written. Generally the header for a class should make quite clear which functions can be overridden by derived classes and how they are called. Having programmers write this documentation is helpful in ensuring they are marked correctly as virtual.
I would also say that declaring every function as virtual could lead to more bugs than just forgetting to mark something as virtual. If all functions are virtual everything can be replaced by base classes - public, protected, private - everything becomes fair game. By accident or intention subclasses could then change the behavior of functions that then cause problems when used in the base implementation.
It depends. :) (Had you expected anything else?)
Once a class gets a virtual function, it can no longer be a POD datatype, (it may not have been one before either, in which case this won't make a difference) and that makes a whole range of optimizations impossible.
std::copy() on plain POD types can resort to a simple memcpy routine, but non-POD types have to be handled more carefully.
Construction becomes a lot slower because the vtable has to be initialized. In the worst case, the difference in performance between POD and non-POD datatypes can be significant.
In the worst case, you may see 5x slower execution (that number is taken from a university project I did recently to reimplement a few standard library classes. Our container took roughly 5x as long to construct as soon as the data type it stored got a vtable)
Of course, in most cases, you're unlikely to see any measurable performance difference, this is simply to point out that in some border cases, it can be costly.
However, performance shouldn't be your primary consideration here.
Making everything virtual is not a perfect solution for other reasons.
Allowing everything to be overridden in derived classes makes it much harder to maintain class invariants. How does a class guarantee that it stays in a consistent state when any one of its methods could be redefined at any time?
Making everything virtual may eliminate a few potential bugs, but it also introduces new ones.
If you need the functionality of virtual dispatch, you have to pay the price. The advantage of C++ is that you can use a very efficient implementation of virtual dispatch provided by the compiler, rather than a possibly inefficient version you implement yourself.
However, lumbering yourself with the overhead if you don't needx it is possibly going a bit too far. And most classesare not designed to be inherited from - to create a good base class requires more than making its functions virtual.
Virtual dispatch is an order of magnitude slower than some alternatives - not due to indirection so much as the prevention of inlining. Below, I illustrate that by contrasting virtual dispatch with an implementation embedding a "type(-identifying) number" in the objects and using a switch statement to select the type-specific code. This avoids function call overhead completely - just doing a local jump. There is a potential cost to maintainability, recompilation dependencies etc through the forced localisation (in the switch) of the type-specific functionality.
IMPLEMENTATION
#include <iostream>
#include <vector>
// virtual dispatch model...
struct Base
{
virtual int f() const { return 1; }
};
struct Derived : Base
{
virtual int f() const { return 2; }
};
// alternative: member variable encodes runtime type...
struct Type
{
Type(int type) : type_(type) { }
int type_;
};
struct A : Type
{
A() : Type(1) { }
int f() const { return 1; }
};
struct B : Type
{
B() : Type(2) { }
int f() const { return 2; }
};
struct Timer
{
Timer() { clock_gettime(CLOCK_MONOTONIC, &from); }
struct timespec from;
double elapsed() const
{
struct timespec to;
clock_gettime(CLOCK_MONOTONIC, &to);
return to.tv_sec - from.tv_sec + 1E-9 * (to.tv_nsec - from.tv_nsec);
}
};
int main(int argc)
{
for (int j = 0; j < 3; ++j)
{
typedef std::vector<Base*> V;
V v;
for (int i = 0; i < 1000; ++i)
v.push_back(i % 2 ? new Base : (Base*)new Derived);
int total = 0;
Timer tv;
for (int i = 0; i < 100000; ++i)
for (V::const_iterator i = v.begin(); i != v.end(); ++i)
total += (*i)->f();
double tve = tv.elapsed();
std::cout << "virtual dispatch: " << total << ' ' << tve << '\n';
// ----------------------------
typedef std::vector<Type*> W;
W w;
for (int i = 0; i < 1000; ++i)
w.push_back(i % 2 ? (Type*)new A : (Type*)new B);
total = 0;
Timer tw;
for (int i = 0; i < 100000; ++i)
for (W::const_iterator i = w.begin(); i != w.end(); ++i)
{
if ((*i)->type_ == 1)
total += ((A*)(*i))->f();
else
total += ((B*)(*i))->f();
}
double twe = tw.elapsed();
std::cout << "switched: " << total << ' ' << twe << '\n';
// ----------------------------
total = 0;
Timer tw2;
for (int i = 0; i < 100000; ++i)
for (W::const_iterator i = w.begin(); i != w.end(); ++i)
total += (*i)->type_;
double tw2e = tw2.elapsed();
std::cout << "overheads: " << total << ' ' << tw2e << '\n';
}
}
PERFORMANCE RESULTS
On my Linux system:
~/dev g++ -O2 -o vdt vdt.cc -lrt
~/dev ./vdt
virtual dispatch: 150000000 1.28025
switched: 150000000 0.344314
overhead: 150000000 0.229018
virtual dispatch: 150000000 1.285
switched: 150000000 0.345367
overhead: 150000000 0.231051
virtual dispatch: 150000000 1.28969
switched: 150000000 0.345876
overhead: 150000000 0.230726
This suggests an inline type-number-switched approach is about (1.28 - 0.23) / (0.344 - 0.23) = 9.2 times as fast. Of course, that's specific to the exact system tested / compiler flags & version etc., but generally indicative.
COMMENTS RE VIRTUAL DISPATCH
It must be said though that virtual function call overheads are something that's rarely significant, and then only for oft-called trivial functions (like getters and setters). Even then, you might be able to provide a single function to get and set a whole lot of things at once, minimising the cost. People worry about virtual dispatch way too much - so do do the profiling before finding awkward alternatives. The main issue with them is that they perform an out-of-line function call, though they also delocalise the code executed which changes the cache utilisation patterns (for better or (more often) worse).
The extra cost is virtually nothing in most scenarios. (pardon the pun). ejac has already posted sensible relative measures.
The biggest thing you give up is possible optimizations due to inlining. They can be especially good if the function is called with constant parameters. This rarely makes a real difference, but in a few cases, this can be huge.
Regarding optimizations:
It is important to know and consider the relative cost of constructs of your language. Big O notation is onl half of the story - how does your application scale. The other half is the constant factor in front of it.
As a rule of thumb, I wouldn't go out of my way to avoid virtual functions, unless there are clear and specific indications that it is a bottle neck. A clean design always comes first - but it is only one stakeholder that should not unduly hurt others.
Contrived Example: An empty virtual destructor on an array of one million small elements may plow through at least 4MB of data, thrashing your cache. If that destructor can be inlined away, the data won't be touched.
When writing library code, such considerations are far from premature. You never know how many loops will be put around your function.
While everyone else is correct about the performance of virtual methods and such, I think the real problem is whether the team knows about the definition of the virtual keyword in C++.
Consider this code, what is the output?
#include <stdio.h>
class A
{
public:
void Foo()
{
printf("A::Foo()\n");
}
};
class B : public A
{
public:
void Foo()
{
printf("B::Foo()\n");
}
};
int main(int argc, char** argv)
{
A* a = new A();
a->Foo();
B* b = new B();
b->Foo();
A* a2 = new B();
a2->Foo();
return 0;
}
Nothing surprising here:
A::Foo()
B::Foo()
A::Foo()
As nothing is virtual. If the virtual keyword is added to the front of Foo in both A and B classes, we get this for the output:
A::Foo()
B::Foo()
B::Foo()
Pretty much what everyone expects.
Now, you mentioned that there are bugs because someone forgot to add a virtual keyword. So consider this code (where the virtual keyword is added to A, but not B class). What is the output then?
#include <stdio.h>
class A
{
public:
virtual void Foo()
{
printf("A::Foo()\n");
}
};
class B : public A
{
public:
void Foo()
{
printf("B::Foo()\n");
}
};
int main(int argc, char** argv)
{
A* a = new A();
a->Foo();
B* b = new B();
b->Foo();
A* a2 = new B();
a2->Foo();
return 0;
}
Answer: The same as if the virtual keyword is added to B? The reason is that the signature for B::Foo matches exactly as A::Foo() and because A's Foo is virtual, so is B's.
Now consider the case where B's Foo is virtual and A's is not. What is the output then? In this case, the output is
A::Foo()
B::Foo()
A::Foo()
The virtual keyword works downwards in the hierarchy, not upwards. It never makes the base class methods virtual. The first time a virtual method is encountered in the hierarchy is when the polymorphism begins. There isn't a way for later classes to make previous classes have virtual methods.
Don't forget that virtual methods mean that this class is giving future classes the ability to override/change some of its behaviors.
So if you have a rule to remove the virtual keyword, it may not have the intended effect.
The virtual keyword in C++ is a powerful concept. You should make sure each member of the team really knows this concept so that it can be used as designed.
Depending on your platform, the overhead of a virtual call can be very undesirable. By declaring every function virtual you're essentially calling them all through a function pointer. At the very least this is an extra dereference, but on some PPC platforms it will use microcoded or otherwise slow instructions to accomplish this.
I'd recommend against your suggestion for this reason, but if it helps you prevent bugs then it may be worth the trade off. I can't help but think that there must be some middle ground that is worth finding, though.
It will require just a couple of extra asm instruction to call virtual method.
But I don't think you worry that fun(int a, int b) has a couple of extra 'push' instructions compared to fun(). So don't worry about virtuals too, until you are in special situation and see that it really leads to problems.
P.S. If you have a virtual method, make sure you have a virtual destructor. This way you'll avoid possible problems
In response to 'xtofl' and 'Tom' comments. I did small tests with 3 functions:
Virtual
Normal
Normal with 3 int parameters
My test was a simple iteration:
for(int it = 0; it < 100000000; it ++) {
test.Method();
}
And here the results:
3,913 sec
3,873 sec
3,970 sec
It was compiled by VC++ in debug mode. I did only 5 tests per method and computed the mean value (so results may be pretty inaccurate)... Any way, the values are almost equal assuming 100 million calls. And the method with 3 extra push/pop was slower.
The main point is that if you don't like the analogy with the push/pop, think of extra if/else in your code? Do you think about CPU pipeline when you add extra if/else ;-) Also, you never know on what CPU the code will be running... Usual compiler can generates code more optimal for one CPU and less optimal for an other (Intel C++ Compiler)