How to avoid de-reference overhead of pointers, using references in containers? - c++

Question

I'm encountering a design challenge. There is a huge std::vector<int> called O say of size 10000. There are two many objects of type Foo, f_1...f_n. Each Foo has an internal std::vector<int> which is a suborder of O. For example:
O = 1, 2, ..., 100000
f_1.order = 1, 2, 3
f_2.order = 1, 4, 16
f_3.order = 100, 101, 102
// ...
The main requirement is to update corresponding values of O when a f_n changes its values. Length and contents of all Foo objects are knows at construction time and not supposed to change during their lifetime. For example it's known that f_1 holds first, second and third elements of O.
The obvious solution is to use pointers of course. Foo may hold a std::vector<int*> which each element points to underlying data of original order (O).
On the other hand, my program do some heavy calculations using Foo objects. So I'm looking for a method to remove overhead of pointer dereferencing. It would be nice if design allow me to use some sort of std::vector<int&> but it's not possible (I guess because vector<T> needs presence of T*).
A colleague suggested to use boost::ptr_vector. Another suggested holding indexes in a vector<size_t>...

Answer 1

I would say that optimizing for pointer dereferencing overhead is pointless. Let's look at some example code:
void bar(int i);
void foo(int* p, int i)
{
bar(*p);
bar(i);
}
And now let's look at the assembly of it:
void foo(int* p, int i)
{
push rbx
mov ebx, esi
bar(*p);
mov edi, DWORD PTR [rdi]
call a <foo+0xa>
bar(i);
mov edi, ebx
call 11 <foo+0x11>
}
There's an "overhead" of one memory read.
As for using references, it's not gonna do anything useful. References may have different semantics to pointers, but underneath, they're still pointers:
void foo(int& r)
{
bar(r);
mov edi,DWORD PTR [rbx]
call 20 <_Z3fooPiiRi+0x20>
}
There's the same memory read happening.
I'm not sure if this counts as an "answer", but seriously: don't bother.

Answer 2

This cannot be stressed enough - do not optimize your code before you know you have a problem. Pointer dereferencing is not costly, and is usually not the main bottleneck in a program.
Note that references are implemented using pointer dereferencing, so even if you could do std::vector<int&>, it would not help.
If you really, really feel you must do something - even though I'm really, totally sure it can't possibly help your performance in any meaningful sense - you could try overlaying the memory. That is, you could define it like this (note that I'm not in any way endorsing this - I'm only pointing it out so that you don't do something worse):
std::vector<int> O;
struct Foo {
int *myData;
int &operator[](int offset) { return myData[offset]; }
};
O.resize(1000000, 0);
Foo f_1, f_2, ...;
f_1.myData = &(O[0]);
f_2.myData = &(O[3]);
O[0] = 5;
cout << f_1[0]; // prints 5
Also, BTW - please, please, please, do not use the name O as a variable. Please. It looks like a zero.

Answer 3

It sound as premature optimization. Dereference a pointer is ridiculously cheap.
The obvious solution is to use pointers of course. Foo may hold a std::vector which each element points to underlying data of original order (O).
Here a solution, deducting what you need, not using pointers, using std::reference_wrapper and std::ref:
struct Foo
{
Foo(std::vector<int>& _data) : dataFull(_data)
{ ; }
void add(int index)
{
assert(index < dataFull.size());
if(index < references.size())
{
// replace
references[index] = std::ref(dataFull[index]);
}
else
{
// add n times, need sync with index
while(index >= references.size())
{
references.push_back(std::ref(dataFull[index]));
}
}
// mark as valid index
indexes.push_back(index);
// sort for can find with binary_search
std::sort(indexes.begin(), indexes.end());
}
int* get(int index)
{
if(std::binary_search(indexes.begin(), indexes.end(), index))
{
return &references[index].get();
}
else
{
return NULL;
}
}
protected:
std::vector<int>& dataFull;
std::vector<std::reference_wrapper<int> > references;
std::vector<int> indexes;
};
int main()
{
const int size = 1000000;
std::vector<int> O;
O.resize(1000000, 0);
Foo f_1(O);
f_1.add(1);
f_1.add(2);
f_1.add(3);
Foo f_2(O);
f_2.add(1);
f_2.add(4);
f_2.add(16);
Foo f_3(O);
f_3.add(100);
f_3.add(101);
f_3.add(102);
// index 1 is changed, it must affect to all "Foo" that use this index (f_1 and f_2)
O[1] = 666;
// checking if it changed
assert( *f_1.get(1) == 666 );
assert( *f_2.get(1) == 666 );
assert( f_3.get(1) == NULL );
return 0;
}
EDIT:
Performance is same that if you use pointer, but std::reference_wrapper can be integrated best in templated code because you have T& and don't need have code for T* and T&.
Have indexes in other vector, only is useful if your struct is ordered by multiple criteria.
I show a example with a vector, where T is a struct complex with two fields. I can reorder this vector with 2 criterias, without touch the original.
template <typename T>
struct Index
{
typedef typename bool(*Comparator)(const T&, const T&);
Index(std::vector<T>& _data, Comparator _comp)
: dataFull(_data)
, comp(_comp)
{
for(unsigned int i = 0; i < dataFull.size(); ++i)
{
add(i);
}
commit();
}
void commit()
{
std::sort(references.begin(), references.end(), comp);
}
std::vector<std::reference_wrapper<T> >& getReference() {return references;}
protected:
void add(int index)
{
assert(index < dataFull.size());
references.push_back(std::ref(dataFull[index]));
}
protected:
std::vector<T>& dataFull;
std::vector<std::reference_wrapper<T> > references;
Comparator comp;
};
int main()
{
struct ComplexData
{
int field1;
int field2;
};
// Generate vector
const int size = 10;
std::vector<ComplexData> data;
data.resize(size);
for(unsigned int i = 0; i < size; ++i)
{
ComplexData& c = data[i];
c.field1 = i;
c.field2 = size - i;
}
// Vector reordered without touch original
std::cout << "Vector data, ordered by field1" << std::endl;
{
Index<ComplexData> f_1(data,
[](const ComplexData& a, const ComplexData& b){return a.field1 < b.field1;});
auto it = f_1.getReference().begin();
auto ite = f_1.getReference().end();
for(; it != ite; ++it)
{
std::cout << "-> " << it->get().field1 << " - " << it->get().field2 << std::endl;
}
}
// Vector reordered without touch original
std::cout << "Vector data, ordered by field2" << std::endl;
{
Index<ComplexData> f_2(data,
[](const ComplexData& a, const ComplexData& b){return a.field2 < b.field2;});
auto it = f_2.getReference().begin();
auto ite = f_2.getReference().end();
for(; it != ite; ++it)
{
std::cout << "-> " << it->get().field1 << " - " << it->get().field2 << std::endl;
}
}
return 0;
}