C++ Efficient Growing Vector of Fix-Sized Vector - c++

Question

In my program, I had a std::vector<std::array<float, n_channels>> vecvec, where n_channels was a constant integer known at compile time. In the program, vecvec grows over time.
I now want to lift the constraint that n_channels must be known at compile time, so I changed the definition to std::vector<std::vector<float>> vecvec. n_channels is still a fixed value, which is known before vecvec is constructed (all elements of vecvec have the same length).
However, now my program is suddenly about 2.5x slower.
I assume this is because the memory of vecvec is suddenly fragmented, because it doesn't "know" that every element of vecvec will have the same size.
Is there a way I can have my cake and eat it too?

Answer 1

Do you want to have your cake and eat it too? Implement your own row-resizable 2D array class TODAY!!!
You can write your own 2D array class. By making rows contiguous in memory, you get all the benefits of using std::vector<std::array<...>>, but without the fixed compiletime sizes! To simplify the implementation, you can make it wrap std::vector.
In order to achieve full functionality, we should also create two "helper" classes. One of them represents a row in the array, and the other represents an iterator for that row. When we iterate through the 2D array, we'll be iterating over the rows of the array.
Row class
This is pretty straight-forward. It just contains a beginning and end pointer. The array is stored contiguously, so we don't actually store Rows, but it's still convenient to have them so we have a type to iterate over.
Because the Row class just represents a view of a row in the matrix, the Row class should NOT allocate or delete any memory. Additionally, I made all the member functions of the Row class constant so that operations can be performed on Rows returned directly from the RowIterator.
template<class T>
struct Row {
T* _start;
size_t _size;
// These are const because if we need the elements to be const
// We just make T const
T* begin() const noexcept { return _start; }
T* end() const noexcept { return _start + _size; }
size_t size() const noexcept { return _size; }
T& operator[](size_t index) const noexcept {
return _start[index];
}
// Implicitly convertible to Row<T const>
operator Row<T const>() const noexcept {
return {_start, _size};
}
};
RowIterator class
This one just implements the basic abilities of a random-access iterator. You can move it forward, backward, index into it, add or subtract integers from it, etc. If I subtract 5, for example, it moves back 5 rows.
template<class T>
struct RowIterator {
using value_type = Row<T>;
using element_type = Row<T>;
using reference_type = Row<T>;
using const_reference_type = Row<T>;
// Add other iterator traits as needed
Row<T> current;
void operator++() noexcept {
current._start += current._size;
}
void operator--() noexcept {
current._start -= current._size;
}
RowIterator<T> operator+(intptr_t rows) const noexcept {
return { Row<T>{current._start + rows * current._size, current._size } };
}
RowIterator<T> operator-(intptr_t rows) const noexcept {
return { Row<T>{current._start - rows * current._size, current._size } };
}
RowIterator<T>& operator+=(intptr_t rows) noexcept {
current._start += rows * current._size;
return *this;
}
RowIterator<T>& operator-=(intptr_t rows) noexcept {
current._start -= rows * current._size;
return *this;
}
Row<T> operator*() const noexcept {
return current;
}
bool operator==(RowIterator<T> other) const noexcept {
return current._start == other.current._start && current._size == other.current._size;
}
bool operator!=(RowIterator<T> other) const noexcept {
return current._start != other.current._start || current._size != other.current._size;
}
Row<T> operator[](intptr_t index) {
return (*this + index).current;
}
};
vector2D class
The 2D vector class stores it's elements contiguously in a vector, but to access them or iterator over them it returns Rows and RowIterators. Because a Row is just two values (a pointer and a size), this is really cheap to do, and the compiler should be able to optimize it easily.
Note that to preserve const correctness, I use Row<T const>, which creates a Row with constant elements. (This greatly simplifies the implementation of Row).
template<class T>
class vector2D : private std::vector<T> {
size_t rows;
size_t columns;
using std::vector<T>::data;
public:
size_t size() const noexcept {
return rows;
}
// Gets a particular row
Row<T> operator[](size_t index) noexcept {
return { data() + columns * index, columns };
}
// Get a particular row when const
Row<T const> operator[](size_t index) const noexcept {
return { data() + columns * index, columns };
}
RowIterator<T> begin() noexcept {
return { Row<T>{ data() , columns } };
}
RowIterator<T> end() noexcept {
return { Row<T>{ data() + columns * rows, columns } };
}
RowIterator<T const> begin() const noexcept {
return { Row<T const>{ data() , columns } };
}
RowIterator<T const> end() const noexcept {
return { Row<T const>{ data() + columns * rows, columns } };
}
template<size_t N>
void push_back(std::array<T, N> const& arr) {
if(arr.size() == columns) {
insert(end(), arr.begin(), arr.end());
rows++;
}
else
throw std::invalid_argument("Bad number of columns");
}
void push_back(Row<T> arr) {
if(arr.size() == columns) {
insert(end(), arr.begin(), arr.end());
rows++;
}
else
throw std::invalid_argument("Bad number of columns");
}
void push_back(Row<T const> arr) {
if(arr.size() == columns) {
insert(end(), arr.begin(), arr.end());
rows++;
}
else
throw std::invalid_argument("Bad number of columns");
}
void push_back(std::initializer_list<T> arr) {
if(arr.size() == columns) {
insert(end(), arr.begin(), arr.end());
rows++;
}
else
throw std::invalid_argument("Bad number of columns");
}
vector2D(size_t rows, size_t columns)
: std::vector<T>(rows * columns)
, rows(rows)
, columns(columns) {}
};
Benchmark results
Run the benchmark here
The benchmark results are in, and vector2D is just as fast as using a vector of arrays!!!
The test
The test has two parts:
Fill the 2D array with values
Sum all the values
To make things as general as possible, these are the functions I used. They can be used with std::vector<std::vector<...>>, std::vector<std::array<...>>, or our very own vector2D!
template<class List>
auto calculateSum2D(List const& list) {
using elem_t = std::decay_t<decltype(list[0][0])>;
elem_t initial = 0;
for(auto const& row : list) {
for(auto& elem : row) {
initial += elem;
}
}
return initial;
}
template<class List>
void fill(List& list, int rows, int cols) {
for(int i = 0; i < rows; i++) {
for(int j = 0; j < cols; j++) {
list[i][j] = i * j;
}
}
}
The results
We used Quickbench to obtain the results, and vector2D was 4.5 times faster than using a vector of vectors!
These results were obtained using the corresponding functions, written using quick bench!
// Benchmark using a vector of vectors
static void sumVector(benchmark::State& state) {
// Code inside this loop is measured repeatedly
for (auto _ : state) {
std::vector<std::vector<double>> vect(rows, std::vector<double>(cols));
fill(vect, rows, cols);
auto sum = calculateSum2D(vect);
benchmark::DoNotOptimize(sum);
}
}
// Register the function as a benchmark
BENCHMARK(sumVector);
// Benchmark using a vector of arrays
static void sumArray(benchmark::State& state) {
// Code inside this loop is measured repeatedly
for (auto _ : state) {
std::vector<std::array<double, cols>> vect(rows, std::array<double, cols>());
fill(vect, rows, cols);
auto sum = calculateSum2D(vect);
benchmark::DoNotOptimize(sum);
}
}
// Register the function as a benchmark
BENCHMARK(sumArray);
// Benchmark using vector2D implementation
static void sumvector2D(benchmark::State& state) {
// Code inside this loop is measured repeatedly
for (auto _ : state) {
vector2D<double> vect(rows, cols);
fill(vect, rows, cols);
auto sum = calculateSum2D(vect);
benchmark::DoNotOptimize(sum);
}
}
// Register the function as a benchmark
BENCHMARK(sumvector2D);
Benchmarks v2: no repeated allocation
View benchmark 2 here
As it turns out, in the initial benchmark, most of the cost came from repeated allocations (in all cases, the object was re-allocated each iteration of the benchmark). To fix this, I moved the declaration out of the loop, so that it'd only occur once. I also adjusted the number of rows and columns so that there were more rows and fewer columns, so as to get a more realistic scenario where not the entire thing fits in the cache.
Once again, vector2D and vector<array> perform nearly identically, however this time vector<vector> does a much better job, and the gap is not nearly so impressive.
The reason for the difference in speedup is that this time around, the only differences were the result of poor cache locality, as each object was only being allocated once.
Summary
Based on the benchmark results, vector2D should bump your performance back up to what it was originally. Because your code presumably contains a mix of allocations and usage, you got a result somewhere between the two benchmarks (2.5 times slower for vector of vector). Because vector2D is contiguous and avoids the repeated heap allocations that plagued the vector-of-vector approach, it should be just as fast as a vector of arrays.