Generic type vector performances - c++

Question

I need to have a vector class that is able to manage as set of predefined types and have the same performance as the standard std::vector.
I can't just rely on a simple template because the type isn't defined until run time. Bellow are two implementations I tested, but the first is about 4 times slower and the second 3 times slower than a simple std::vector. I also tested having a function pointer to the proper setValue that is assigned in the constructor, but it was 10 times slower.
#include <chrono>
#include <iostream>
#include <vector>
class TypeBase
{
public:
virtual void setValue(int id, int value) = 0;
virtual void setValue(int id, float value) = 0;
};
class TypeInt : public TypeBase
{
public:
int* data;
TypeInt(char* ptr)
{
data = (int*)ptr;
}
void setValue(int id, int value)
{
data[id] = value;
}
void setValue(int id, float value)
{
data[id] = (int)value;
}
};
class TypeFloat : public TypeBase
{
public:
float* data;
TypeFloat(char* ptr)
{
data = (float*)ptr;
}
void setValue(int id, int value)
{
data[id] = (float)value;
}
void setValue(int id, float value)
{
data[id] = value;
}
};
class genericTypeVector1
{
public:
// Allow to get the right setValue
TypeBase* typeMng = nullptr;
// Vector storage
std::vector<char> data;
enum class Types {Int, Float};
genericTypeVector1(Types type, size_t size)
{
switch (type)
{
case Types::Int:
data.resize(size * sizeof(int));
typeMng = new TypeInt(data.data());
break;
case Types::Float:
data.resize(size * sizeof(float));
typeMng = new TypeFloat(data.data());
break;
}
}
~genericTypeVector1()
{
if (typeMng != nullptr)
delete typeMng;
}
void setValue(int id, int value)
{
typeMng->setValue(id, value);
}
void setValue(int id, float value)
{
typeMng->setValue(id, value);
}
};
class genericTypeVector2
{
public:
// Vector storage
std::vector<char> data;
enum class Types { Int, Float };
// Current type of the vector
Types type;
genericTypeVector2(Types type, size_t size)
:type(type)
{
switch (type)
{
case Types::Int:
data.resize(size * sizeof(int));
break;
case Types::Float:
data.resize(size * sizeof(float));
break;
}
}
void setValue(int id, int value)
{
switch (type)
{
case Types::Int:
((int*)data.data())[id] = value;
break;
case Types::Float:
((float*)data.data())[id] = (int)value;
break;
}
}
void setValue(int id, float value)
{
switch (type)
{
case Types::Int:
((int*)data.data())[id] = (float)value;
break;
case Types::Float:
((float*)data.data())[id] = value;
break;
}
}
};
template<class T>
inline void setValue(T* ptr, int id, float value)
{
ptr[id] = value;
}
int main()
{
int N = 100000000;
{
std::vector<float> v(N);
auto begin = std::chrono::steady_clock::now();
for (int i = 0; i < N; ++i)
v[i] = 2 * i - N;
std::cout << "std::vector : " << std::chrono::duration_cast<std::chrono::milliseconds>(std::chrono::steady_clock::now() - begin).count() << "ms" << std::endl;
}
{
genericTypeVector1 a(genericTypeVector1::Types::Float, N);
auto begin = std::chrono::steady_clock::now();
for (int i = 0; i < N; ++i)
a.setValue(i, 2 * i - N);
std::cout << "genericTypeVector with type manager : " << std::chrono::duration_cast<std::chrono::milliseconds>(std::chrono::steady_clock::now() - begin).count() << "ms" << std::endl;
}
{
genericTypeVector2 a(genericTypeVector2::Types::Float, N);
auto begin = std::chrono::steady_clock::now();
for (int i = 0; i < N; ++i)
a.setValue(i, 2 * i - N);
std::cout << "genericTypeVector with switch-case in setValue : " << std::chrono::duration_cast<std::chrono::milliseconds>(std::chrono::steady_clock::now() - begin).count() << "ms" << std::endl;
}
{
genericTypeVector2 a(genericTypeVector2::Types::Float, N);
auto begin = std::chrono::steady_clock::now();
switch (a.type)
{
case genericTypeVector2::Types::Int:
for (int i = 0; i < N; ++i)
setValue((int*)a.data.data(), i, 2 * i - N);
break;
case genericTypeVector2::Types::Float:
for (int i = 0; i < N; ++i)
setValue((float*)a.data.data(), i, 2 * i - N);
break;
}
std::cout << "genericTypeVector with switch case outside for loop : " << std::chrono::duration_cast<std::chrono::milliseconds>(std::chrono::steady_clock::now() - begin).count() << "ms" << std::endl;
}
}
I don't know what to try now. Perhaps I'm on a completely wrong path.
A added a last example that shows that it's possible to have the same performances as with a raw std::vector, but I don't want to have this switch-case every time I call setValue.

Answer 1

[Edit]
I redo my whole answer now that I've looked at the code and ran it! One thing BTW from a benchmarking standpoint -- you have code like this:
genericTypeVector2 a(genericTypeVector2::Types::Float, N);
auto begin = std::chrono::steady_clock::now();
for (int i = 0; i < N; ++i)
a.setValue(i, 2 * i - N);
std::cout << "genericTypeVector with switch-case in setValue : " << std::chrono::duration_cast<std::chrono::milliseconds>(std::chrono::steady_clock::now() - begin).count() << "ms" << std::endl;
The time measurements don't begin until after the data structure is initialized. I was wondering why on earth your final version was beating std::vector when it still involved dynamic dispatch:
std::vector : 46ms
genericTypeVector with type manager : 182ms
genericTypeVector with switch-case in setValue : 115ms
genericTypeVector with switch case outside for loop : 44ms
.. until I looked at your code more carefully and realized you weren't factoring in times to construct the structure.std::vector is often aggresively optimized in terms of the functions it uses for constructing and destroying PODs, able to do things like a calloc if the type is trivially default-constructible (although it still often just uses a plain old placement new loop for fill constructors) in which case it might not even need to touch the memory until you begin to access it via operator[]. So with the way you're measuring, the bulk of the overhead of paging and cache misses are likely incurred by vector in its timing (I need to profile to make sure but this seems most likely), while your version incurs them outside of when you actually begin measuring the times.
So I fixed the test and got these times which is closer to my expectations from having repeatedly benchmarked and profiled similar code:
std::vector : 109ms
genericTypeVector with type manager : 276ms
genericTypeVector with switch-case in setValue : 191ms
genericTypeVector with switch case outside for loop : 131ms
And that's not too bad unless you have a real-world case where just setting values of array elements in a loop is showing up as one of your bigger hotspots. It's easy to get cross-eyed trying to do these ultra granular performance tests and think 3-4x slower is so bad when most domains involve more meaty processing over each element (unless we're talking about image processing or something like that).
But the main overhead I suspect is just the extra branching for the dynamic dispatch in all your cases (two layers for version 1, one for version 2) + switch overhead.
One thing you can possibly do about the switch versions is to use __assume(0) in MSVC or goto labels in GCC. It's a trick I learned optimizing interpreters. You can get a much improvement in performance if the switch branching is dominating the times since both of these solutions eliminate an extra branch required for what would otherwise be a default case when you handle all possible cases.