Related
I try to write an iterator that iterates over a container by index.
A It and a const It both allow changing the content of the container.
A Const_it and a const Const_it both forbid changing the content of the container.
After that, I try to write a span<T> over a container.
For a type T that is not const, both const span<T> and span<T> allows changing the content of the container.
Both const span<const T> and span<const T> forbid changing the content of the container.
The code does not compile because:
// *this is const within a const method
// But It<self_type> requires a non-const *this here.
// So the code does not compile
It<self_type> begin() const { return It<self_type>(*this, 0); }
If I make the constructor of It to accept a const container, it doesn't look right because the iterator can modify the content of the container.
If I get rid of the const of the method, then for a non-const type T, a const span<T> cannot modify the container.
It inherits from Const_it to allow implicit conversion from It to Const_it during template instantiation.
I use a pointer instead of a reference to in the iterators (const C* container_;) for allowing assigning one iterator to another iterator.
I suspect something is very wrong here because I even think about:
Does cast away const of *this cause undefined behavior?
But I don't know how to fix it.
Test:
#include <vector>
#include <numeric>
#include <iostream>
template<typename C>
class Const_it {
typedef Const_it<C> self_type;
public:
Const_it(const C& container, const int ix)
: container_(&container), ix_(ix) {}
self_type& operator++() {
++ix_;
return *this;
}
const int& operator*() const {
return ref_a()[ix_];
}
bool operator!=(const self_type& rhs) const {
return ix_ != rhs.ix_;
}
protected:
const C& ref_a() const { return *container_; }
const C* container_;
int ix_;
};
template<typename C>
class It : public Const_it<C> {
typedef Const_it<C> Base;
typedef It<C> self_type;
public:
//It(const C& container.
It(C& container, const int ix)
: Base::Const_it(container, ix) {}
self_type& operator++() {
++ix_;
return *this;
}
int& operator*() const {
return mutable_a()[ix_];
}
private:
C& mutable_a() const { return const_cast<C&>(ref_a()); }
using Base::ref_a;
using Base::container_;
using Base::ix_;
};
template <typename V>
class span {
typedef span<V> self_type;
public:
explicit span(V& v) : v_(v) {}
It<self_type> begin() { return It<self_type>(*this, 0); }
// *this is const within a const method
// But It<self_type> requires a non-const *this here.
// So the code does not compile
It<self_type> begin() const { return It<self_type>(*this, 0); }
It<self_type> end() { return It<self_type>(*this, v_.size()); }
It<self_type> end() const { return It<self_type>(*this, v_.size()); }
int& operator[](const int ix) {return v_[ix];}
const int& operator[](const int ix) const {return v_[ix];}
private:
V& v_;
};
int main() {
typedef std::vector<int> V;
V v(10);
std::iota(v.begin(), v.end(), 0);
std::cout << v.size() << "\n";
const span<V> s(v);
for (auto&& x : s) {
x = 4;
std::cout << x << "\n";
}
}
There are two main notes to be said to make this work. First:
If I make the constructor of It to accept a const container, it doesn't look right because the iterator can modify the content of the container.
Not really, because C in your template<typename C> class It is not the actual container, but the span<V>. In other words, take a look at:
It<self_type> begin() const { return It<self_type>(*this, 0); }
Here self_type means const span<V>, therefore you are returning a It<const span<V>>. Thus, your iterator can do whatever can be done with a const span -- but the container is still non-const. The variable name container_ is not fortunate, then.
For a type T that is not const, both const span<T> and span<T> allows changing the content of the container. Both const span<const T> and span<const T> forbid changing the content of the container.
In addition, since you want that const span is allowed to modify the contents, then what you should write inside span itself is (note the const):
int& operator[](const int ix) const {return v_[ix];}
// Removing the other `const` version:
// const int& operator[](const int ix) const {return v_[ix];}
With those two bits clarified, you can then construct a working example. Here is one based from your code and simplified to solve the issue at hand:
#include <vector>
#include <iostream>
template<typename S>
class It {
typedef It<S> self_type;
const S& span_;
int ix_;
public:
It(const S& span, const int ix)
: span_(span), ix_(ix) {}
self_type& operator++() {
++ix_;
return *this;
}
int& operator*() const {
return span_[ix_];
}
bool operator!=(const self_type& rhs) const {
return &span_ != &rhs.span_ or ix_ != rhs.ix_;
}
};
template <typename V>
class span {
typedef span<V> self_type;
public:
explicit span(V& v) : v_(v) {}
It<self_type> begin() const { return It<self_type>(*this, 0); }
It<self_type> end() const { return It<self_type>(*this, v_.size()); }
int& operator[](const int ix) const {return v_[ix];}
private:
V& v_;
};
int main() {
typedef std::vector<int> V;
V v(10);
const span<V> s(v);
for (auto&& x : s) {
x = 4;
std::cout << x << "\n";
}
}
Take a look as well at the corrected implementation of operator!= and at the fact that there is no need for the non-const version of begin() and end(). You can also throw there a cbegin() and cend(). Then you have to work on adding back the const iterator cases.
By the way, in case it saves some confusion for anybody: in the near future, std::span (proposed for C++20) might be added; and it will be just a (pointer-to-first-element, index) pair -- rather than your (pointer-to-container, index) version.
In other words, as its template parameter, it will take the elements' type, rather than a container:
span<std::vector<int>> s(v);
// vs
std::span<int> s(v);
This allows consumers of std::span to avoid knowing about which container is behind the scenes (or even no container: a contiguous memory area or an array).
Finally, you may want to take a look at GSL's implementation of std::span to get some inspiration on how to fully implement it (including the second template parameter about the extent).
After studying Acorn's solution, I found another fix.
This allows using the same iterator template for both a std::vector<T> and a span.
The two cases are different.
For a non-const type T, a const std::vector<T> forbids changing its element. A const span<T> allows changing its elements.
The main difference is It<const self_type>(*this, 0); in the span class and S& span_; instead of const S& span_; in the It class.
Fix:
#include <vector>
#include <iostream>
template<typename S>
class It {
typedef It<S> self_type;
public:
It(S& span, const int ix)
: span_(span), ix_(ix) {}
self_type& operator++() {
++ix_;
return *this;
}
int& operator*() const {
return span_[ix_];
}
bool operator!=(const self_type& rhs) const {
return &span_ != &rhs.span_ or ix_ != rhs.ix_;
}
private:
S& span_;
int ix_;
};
template<typename V>
class span {
typedef span<V> self_type;
public:
explicit span(V& v) : v_(v) {}
It<const self_type> begin() const {
return It<const self_type>(*this, 0);
}
It<const self_type> end() const {
return It<const self_type>(*this, v_.size());
}
int& operator[](const int ix) const { return v_[ix]; }
private:
V& v_;
};
int main() {
// Test adding iterator to a span
typedef std::vector<int> V;
V v(10);
const span<V> s(v);
for (auto&& x : s) {
x = 4;
std::cout << x << " ";
}
std::cout << "\n";
// Test adding iterator to a std::vector
const It<V> begin(v, 0);
const It<V> end(v, v.size());
for (auto it = begin; it != end; ++it) {
*it = 10;
std::cout << *it << " ";
}
std::cout << "\n";
}
Say I have a virtual base class Base, which will in part behave like a container, with two derived classes VectorLike and RangeLike.
I want to achieve something like the following:
class VectorLike : public Base {
std::vector<int> data;
public:
virtual std::vector<int>::const_iterator cbegin() { return data.cbegin() }
virtual std::vector<int>::const_iterator cend() { return data.cend() }
}
class RangeLike : public Base {
int min, max;
class const_iterator {
int x;
public:
int operator++() { return ++x }
bool operator==( const_iterator rhs ) { return x == rhs.x }
const_iterator( int y ) { x = y }
}
public:
virtual const_iterator cbegin() { return const_iterator( min ); }
virtual const_iterator cend() { return const_iterator( max ); }
}
This code will not compile, since std::vector<int>::const_iterator and RangeLike::const_iterator aren't identical or covariant.
To achieve the second, I would need an iterator base class from which both std::vector<int>::const_iterator and RangeLike::const_iterator will derive. But then still cbegin() and cend() will have to return pointers to iterators, which will make an even bigger mess.
My question is, is it possible to achieve something like the above code and if so how?
Here is an implementation of a polymorphic const int iterator. You can construct it with any iterator type (including pointers) where std::iterator_traits<Iter>::value_type resolves to int.
This should be the case for both std::vector<int> and your_range_type<int>.
This should get you started.
#include <iostream>
#include <vector>
#include <array>
#include <memory>
#include <algorithm>
struct poly_const_iterator
{
using value_type = int;
struct concept {
virtual void next(int n) = 0;
virtual const value_type& deref() const = 0;
virtual bool equal(const void* other) const = 0;
virtual std::unique_ptr<concept> clone() const = 0;
virtual const std::type_info& type() const = 0;
virtual const void* address() const = 0;
virtual ~concept() = default;
};
template<class Iter>
struct model : concept
{
model(Iter iter) : _iter(iter) {}
void next(int n) override { _iter = std::next(_iter, n); }
const value_type& deref() const override { return *_iter; }
bool equal(const void* rp) const override { return _iter == static_cast<const model*>(rp)->_iter; }
std::unique_ptr<concept> clone() const override { return std::make_unique<model>(*this); }
const std::type_info& type() const override { return typeid(_iter); }
const void* address() const override { return this; }
Iter _iter;
};
std::unique_ptr<concept> _impl;
public:
// interface
// todo: constrain Iter to be something that iterates value_type
template<class Iter>
poly_const_iterator(Iter iter) : _impl(std::make_unique<model<Iter>>(iter)) {};
poly_const_iterator(const poly_const_iterator& r) : _impl(r._impl->clone()) {};
const value_type& operator*() const {
return _impl->deref();
}
poly_const_iterator& operator++() {
_impl->next(1);
return *this;
}
bool operator==(const poly_const_iterator& r) const {
return _impl->type() == r._impl->type()
and _impl->equal(r._impl->address());
}
bool operator != (const poly_const_iterator& r) const {
return not(*this == r);
}
};
void emit(poly_const_iterator from, poly_const_iterator to)
{
std::copy(from, to, std::ostream_iterator<int>(std::cout, ", "));
std::cout << std::endl;
}
int main()
{
std::vector<int> v = { 1, 2, 3, 4, 5 };
std::array<int, 5> a = { 6, 7,8, 9, 0 };
emit(std::begin(v), std::end(v));
emit(std::begin(a), std::end(a));
return 0;
}
expected results:
1, 2, 3, 4, 5,
6, 7, 8, 9, 0,
Here's a C++20 based solution (partial implementation)... just thought I'd put it out there.
It uses a variant with stack storage rather than polymorphism and dynamic allocation, so it might have better performance.
It should work for any given set of iterator types that have a common value_type and whose reference type is value_type&. It should also be fairly easy to adapt for other types of iterator and to include operator->.
Not claiming this is better than the implementation in the original answer. Merely an interesting alternative...
PS. normal in the names below simply indicates a basic pointer based iterator.
template <typename _Iterator, typename _Value, typename _Reference, typename _Difference>
concept forward_iterator_for
= std::forward_iterator<_Iterator>
&& std::same_as<std::iter_value_t<_Iterator>, _Value>
&& std::same_as<std::iter_reference_t<_Iterator>, _Reference>
&& std::same_as<std::iter_difference_t<_Iterator>, _Difference>;
template <typename _Iterator, typename _Value>
concept normal_forward_iterator_for //
= std::same_as<std::remove_cvref_t<_Value>, std::remove_const_t<_Value>>
&& forward_iterator_for<_Iterator, std::remove_const_t<_Value>, _Value&, std::ptrdiff_t>;
template <typename _Value, normal_forward_iterator_for<_Value> ... _Iterator>
class normal_forward_iterator_variant {
public:
using iterator_category = std::forward_iterator_tag;
using difference_type = std::ptrdiff_t;
using value_type = std::remove_cv_t<_Value>;
using reference = _Value&;
using iterator_type = std::variant<_Iterator...>;
private:
iterator_type _m_iter;
public:
normal_forward_iterator_variant() = default;
template <normal_forward_iterator_for<_Value> _Iter, typename... _Args>
normal_forward_iterator_variant(std::in_place_type_t<_Iter>, _Args&&... args)
noexcept (std::is_nothrow_constructible_v<_Iter, _Args&&...>)
: _m_iter(std::in_place_type<_Iter>, std::forward<_Args>(args)...) {}
normal_forward_iterator_variant(iterator_type const& iter)
noexcept (std::is_nothrow_copy_constructible_v<iterator_type>)
: _m_iter(iter) {}
normal_forward_iterator_variant(normal_forward_iterator_variant const&) = default;
normal_forward_iterator_variant(normal_forward_iterator_variant&&) = default;
constexpr normal_forward_iterator_variant&
operator=(normal_forward_iterator_variant const& iter)
noexcept (std::is_nothrow_copy_assignable_v<iterator_type>) //
requires std::is_copy_assignable_v<iterator_type> {
_m_iter = iter._m_iter;
return *this;
}
constexpr normal_forward_iterator_variant&
operator=(normal_forward_iterator_variant&& iter)
noexcept (std::is_nothrow_move_assignable_v<iterator_type>) //
requires std::is_move_assignable_v<iterator_type> {
_m_iter = std::move(iter._m_iter);
return *this;
}
template <typename _Tp>
constexpr normal_forward_iterator_variant&
operator=(_Tp const& x)
noexcept (std::is_nothrow_assignable_v<iterator_type, _Tp&>) //
requires std::is_assignable_v<iterator_type, _Tp&> {
_m_iter = x;
return *this;
}
template <typename _Tp>
constexpr normal_forward_iterator_variant&
operator=(_Tp&& x)
noexcept (std::is_nothrow_assignable_v<iterator_type, _Tp&&>) //
requires std::is_assignable_v<iterator_type, _Tp&&> {
_m_iter = std::move(x);
return *this;
}
[[nodiscard]] constexpr reference
operator*() const noexcept {
return std::visit([](auto&& iter) -> reference {
return *iter;
}, _m_iter);
}
constexpr normal_forward_iterator_variant&
operator++() noexcept {
std::visit([](auto&& iter) {
++iter;
}, _m_iter);
return *this;
}
constexpr normal_forward_iterator_variant
operator++(int) noexcept {
normal_forward_iterator_variant rv(*this);
++*this;
return rv;
}
[[nodiscard]] friend constexpr bool
operator==(normal_forward_iterator_variant const& a, normal_forward_iterator_variant const& b) noexcept {
return (a._m_iter == b._m_iter);
}
};
Example use:
#include <iostream>
#include <iomanip>
#include <vector>
#include <list>
#include <iterator.h>
int
main() {
using vector = std::vector<int>;
using list = std::list<int>;
using iterator = normal_forward_iterator_variant<const int, vector::const_iterator, list::const_iterator>;
iterator iter;
vector v{ 0, 1, 2 };
iter = v.cbegin();
std::cout << *iter++ << std::endl;
std::cout << *iter << std::endl;
std::cout << *++iter << std::endl;
list l{ 3, 4, 5 };
iter = l.cbegin();
std::cout << *iter++ << std::endl;
std::cout << *iter << std::endl;
std::cout << *++iter << std::endl;
}
Output is 1, 2, ..., 6 as expected.
After answering this question and reading this talk and looking at this code, I want to implement constexpr find with just simple array class.
Consider following example:
#include <cstddef>
template <class It, class T>
constexpr auto constexpr_find(const It& b, const It& e, T value) {
auto begin = b;
while (begin != e) {
if (*begin == value) break;
++begin;
}
return *begin;
}
template<typename T, size_t N>
class array
{
public:
typedef T* iterator;
typedef const T* const_iterator;
constexpr auto begin() const { return const_iterator(array_); }
constexpr auto end() const { return const_iterator(array_ + N); }
T array_[N];
static constexpr size_t size = N;
};
int main()
{
constexpr array<int, 3> array{{0,2,3}};
static_assert(constexpr_find(array.begin(), array.end(), 0) == 0, "");
}
compiles as expected
And with custom constexpr iterator:
template<class T>
class array_iterator
{
public:
constexpr array_iterator(const T* v) : iterator(v)
{
}
constexpr const T& operator * () const { return *iterator; }
constexpr array_iterator& operator ++()
{
++iterator;
return *this;
}
constexpr bool operator != (const array_iterator& other) const { return iterator != other.iterator; }
private:
const T* iterator;
};
In array class:
typedef const array_iterator<const T> const_iterator;
that's the only difference, compiler give me error:
in constexpr expansion of
constexpr_find<array_iterator<const int>, int>(array.array<T,
N>::begin<int, 3u>(), array.array<T, N>::end<int, 3u>(), 0)
error: (((const int*)(& array.array<int, 3u>::array_)) != (((const
int*)(& array.array<int, 3u>::array_)) + 12u)) is not a constant
expression
Live example
Is this gcc bug, since clang compile this fine, or there is difference in two snippets?
I cannot say for sure, but you store pointers for array's member into external iterator class, it may be the reason for that error.
--------- update start ---------
Here is the minimal snippet that demonstrates the problem:
constexpr const struct A { int i[2]; } a {{0,0}};
int main ()
{
static_assert (nullptr != a.i , ""); // ok
static_assert (nullptr != a.i+0, ""); // ok
static_assert (nullptr != a.i+1, ""); // error
}
It seems to be forbidden to have pointers to array elements (with non-zero offset) in constant expressions.
--------- update end ---------
The workaround is trivial - store the pointer to array object and offset.
Live
#include <cstddef>
template <class It, class T>
constexpr auto constexpr_find(const It& b, const It& e, T value) {
auto begin = b, end = e;
while (begin != end) {
if (*begin == value) break;
++begin;
}
return *begin;
}
template<class Array>
class array_iterator
{
public:
constexpr array_iterator(const Array& a, size_t pos=0u) : array_(&a), pos_ (pos)
{
}
constexpr const typename Array::value_type&
operator * () const { return (*array_)[pos_]; }
constexpr array_iterator& operator ++()
{
++pos_;
return *this;
}
constexpr bool operator != (const array_iterator& other) const
{ return array_ != other.array_ || pos_ != other.pos_; }
private:
const Array* array_;
size_t pos_;
};
template<typename T, size_t N>
class array
{
public:
typedef T value_type;
typedef const array_iterator<array> const_iterator;
constexpr T const& operator[] (size_t idx) const { return array_[idx]; }
constexpr auto begin() const { return const_iterator(*this); }
constexpr auto end() const { return const_iterator(*this, N); }
T array_[N];
static constexpr size_t size = N;
};
int main()
{
constexpr array<int, 3> array{{0,2,3}};
static_assert(constexpr_find(array.begin(), array.end(), 0) == 0, "");
}
By the way, it is possible to implement C++11 version of constexpr enabled find:
Live
#include <cstddef>
#include <cassert>
#if !defined(__clang__) && __GNUC__ < 5
// TODO: constexpr asserts does not work in gcc4, but we may use
// "thow" workaround from
// http://ericniebler.com/2014/09/27/assert-and-constexpr-in-cxx11/
# define ce_assert(x) ((void)0)
#else
# define ce_assert(x) assert(x)
#endif
namespace my {
template <class It, class T>
inline constexpr It
find (It begin, It end, T const& value) noexcept
{
return ! (begin != end && *begin != value)
? begin
: find (begin+1, end, value);
}
template<class Array>
class array_iterator
{
public:
using value_type = typename Array::value_type;
constexpr array_iterator(const Array& array, size_t size = 0u) noexcept
: array_ (&array)
, pos_ (size)
{}
constexpr const value_type operator* () const noexcept
{
return ce_assert (pos_ < Array::size), (*array_) [pos_];
}
#if __cplusplus >= 201402L // C++14
constexpr
#endif
array_iterator& operator ++() noexcept
{
return ce_assert (pos_ < Array::size), ++pos_, *this;
}
constexpr array_iterator operator+ (size_t n) const noexcept
{
return ce_assert (pos_+n <= Array::size), array_iterator (*array_, pos_+n);
}
friend constexpr bool
operator != (const array_iterator& i1, const array_iterator& i2) noexcept
{
return i1.array_ != i2.array_ || i1.pos_ != i2.pos_;
}
friend constexpr size_t
operator- (array_iterator const& i1, array_iterator const& i2) noexcept
{
return ce_assert (i1.array_ == i2.array_), i1.pos_ - i2.pos_;
}
private:
const Array* array_;
size_t pos_;
};
template<typename T, size_t N>
class array
{
public:
using value_type = T;
using const_iterator = const array_iterator<array>;
constexpr value_type const&
operator[] (size_t idx) const noexcept
{ return array_[idx]; }
constexpr const_iterator begin() const noexcept
{ return const_iterator(*this); }
constexpr const_iterator end() const noexcept
{ return const_iterator(*this, N); }
T array_[N];
static constexpr size_t size = N;
};
}
int main()
{
static constexpr my::array<int, 3> array{{0,2,3}};
static_assert (
find (array.begin(), array.end(), 2) - array.begin () == 1,
"error");
}
You may also be interested to check Sprout library, it contains a lot of constexpr data structures and algorithms.
I'm writing a library that uses expression templates with CRTP. The source files can be found here: https://github.com/mspraggs/pyQCD/tree/master/lib/include/base
The expression templates are based on the example given in the Wikipedia article on the subject. I list the code here in case the Wiki article changes in future:
#include <vector>
#include <cassert>
template <typename E>
// A CRTP base class for Vecs with a size and indexing:
class VecExpression {
public:
typedef std::vector<double> container_type;
typedef container_type::size_type size_type;
typedef container_type::value_type value_type;
typedef container_type::reference reference;
size_type size() const { return static_cast<E const&>(*this).size(); }
value_type operator[](size_type i) const { return static_cast<E const&>(*this)[i]; }
operator E&() { return static_cast< E&>(*this); }
operator E const&() const { return static_cast<const E&>(*this); }
};
// The actual Vec class:
class Vec : public VecExpression<Vec> {
container_type _data;
public:
reference operator[](size_type i) { return _data[i]; }
value_type operator[](size_type i) const { return _data[i]; }
size_type size() const { return _data.size(); }
Vec(size_type n) : _data(n) {} // Construct a given size:
// Construct from any VecExpression:
template <typename E>
Vec(VecExpression<E> const& vec) {
E const& v = vec;
_data.resize(v.size());
for (size_type i = 0; i != v.size(); ++i) {
_data[i] = v[i];
}
}
};
template <typename E1, typename E2>
class VecDifference : public VecExpression<VecDifference<E1, E2> > {
E1 const& _u;
E2 const& _v;
public:
typedef Vec::size_type size_type;
typedef Vec::value_type value_type;
VecDifference(VecExpression<E1> const& u, VecExpression<E2> const& v) : _u(u), _v(v) {
assert(u.size() == v.size());
}
size_type size() const { return _v.size(); }
value_type operator[](Vec::size_type i) const { return _u[i] - _v[i]; }
};
template <typename E>
class VecScaled : public VecExpression<VecScaled<E> > {
double _alpha;
E const& _v;
public:
VecScaled(double alpha, VecExpression<E> const& v) : _alpha(alpha), _v(v) {}
Vec::size_type size() const { return _v.size(); }
Vec::value_type operator[](Vec::size_type i) const { return _alpha * _v[i]; }
};
// Now we can overload operators:
template <typename E1, typename E2>
VecDifference<E1,E2> const
operator-(VecExpression<E1> const& u, VecExpression<E2> const& v) {
return VecDifference<E1,E2>(u,v);
}
template <typename E>
VecScaled<E> const
operator*(double alpha, VecExpression<E> const& v) {
return VecScaled<E>(alpha,v);
}
What I want to do is add another expression template that allows assignment to part of the original template object (the Vec class in the code above, and the LatticeBase class in the code I've linked to). Possible usage:
Vec myvector(10);
Vec another_vector(5);
myvector.head(5) = another_vector; // Assign first 5 elements on myvector
myvector.head(2) = another_vector.head(2); // EDIT
So I'd create a new function Vec::head that would a return an expression template for a portion of the Vec object. I don't know how this would fit into the framework I currently have. In particular I have the following questions/comments:
I've seen examples of what I want to achieve in expression templates that don't use CRTP. What do I gain by using CRTP in this case? Is there any point? Should I ditch it and follow the other examples I've found?
In the current framework, the assignment to the _data member in the Vec class is handled by a copy constructor in the Vec class. This won't work if I want to use the expression template returned by Vec::head, since the assignment happens within the class that holds the data, not the expression template.
I've tried creating an assignment operator within the new expression template, but that won't work with the code above as all the expression template members are const references, and so the assignment operator is deleted at compile time. Can I just switch the members to being values instead of references? Will this impact on performance if additional storage is needed? Will this even work (if I change a stored copy of the expression rather than the expression itself)?
Overall I'm confused about how to go about adding an expression template that can be used as an lvalue in the code above. Any guidance on this would be greatly appreciated.
Try this:
#include <vector>
#include <cassert>
template <typename E>
// A CRTP base class for Vecs with a size and indexing:
class VecExpression {
public:
typedef std::vector<double> container_type;
typedef container_type::size_type size_type;
typedef container_type::value_type value_type;
typedef container_type::reference reference;
size_type size() const { return static_cast<E const&>(*this).size(); }
value_type operator[](size_type i) const { return static_cast<E const&>(*this)[i]; }
operator E&() { return static_cast<E&>(*this); }
operator E const&() const { return static_cast<const E&>(*this); }
};
class VecHead;
// The actual Vec class:
class Vec : public VecExpression<Vec> {
container_type _data;
public:
reference operator[](size_type i) { return _data[i]; }
value_type operator[](size_type i) const { return _data[i]; }
size_type size() const { return _data.size(); }
Vec(size_type n) : _data(n) {} // Construct a given size:
// Construct from any VecExpression:
template <typename E>
Vec(VecExpression<E> const& vec) {
E const& v = vec;
_data.resize(v.size());
for (size_type i = 0; i != v.size(); ++i) {
_data[i] = v[i];
}
}
VecHead head(size_type s);
};
class VecHead : public VecExpression< VecHead >
{
Vec::size_type _s;
Vec& _e;
public:
typedef Vec::size_type size_type;
typedef Vec::value_type value_type;
VecHead(std::size_t s, Vec& e)
: _s(s)
, _e(e)
{
assert(_e.size() >= _s);
}
size_type size() const { return _s; }
value_type operator[](Vec::size_type i) const { assert(i < _s); return _e[i]; }
VecHead& operator = (const VecHead& rhs)
{
return operator=(static_cast<const VecExpression<VecHead>&>(rhs));
}
template <typename E>
VecHead& operator = (const VecExpression<E>& rhs)
{
assert(rhs.size() >= _s);
for (size_type i = 0; i < _s && i < rhs.size(); ++i)
_e[i] = rhs[i];
return *this;
}
};
VecHead Vec::head(size_type s)
{
VecHead aHead(s, *this);
return aHead;
}
template <typename E1, typename E2>
class VecDifference : public VecExpression<VecDifference<E1, E2> > {
E1 const& _u;
E2 const& _v;
public:
typedef Vec::size_type size_type;
typedef Vec::value_type value_type;
VecDifference(VecExpression<E1> const& u, VecExpression<E2> const& v) : _u(u), _v(v) {
assert(u.size() == v.size());
}
size_type size() const { return _v.size(); }
value_type operator[](Vec::size_type i) const { return _u[i] - _v[i]; }
};
template <typename E>
class VecScaled : public VecExpression<VecScaled<E> > {
double _alpha;
E const& _v;
public:
VecScaled(double alpha, VecExpression<E> const& v) : _alpha(alpha), _v(v) {}
Vec::size_type size() const { return _v.size(); }
Vec::value_type operator[](Vec::size_type i) const { return _alpha * _v[i]; }
};
// Now we can overload operators:
template <typename E1, typename E2>
VecDifference<E1, E2> const
operator-(VecExpression<E1> const& u, VecExpression<E2> const& v) {
return VecDifference<E1, E2>(u, v);
}
template <typename E>
VecScaled<E> const
operator*(double alpha, VecExpression<E> const& v) {
return VecScaled<E>(alpha, v);
}
int main()
{
Vec myvector(10);
Vec another_vector(5);
for (int i = 0; i < 5; ++i)
another_vector[i] = i;
myvector.head(5) = another_vector; // Assign first 5 elements on myvector
assert(myvector.head(5).size() == 5);
for (int i = 0; i < 10; ++i)
{
assert(myvector[i] == (i < 5 ? static_cast<double>(i) : 0.));
}
//! Added test due to comment vec1.head(2) = vec2.head(2) doesn't work.
Vec vec1(10), vec2(10);
for (int i = 0; i < 10; ++i)
vec2[i] = 2 * (vec1[i] = i);
vec1.head(2) = vec2.head(2);
for (int i = 0; i < 10; ++i)
{
if (i < 2)
{
assert(vec1[i] == vec2[i]);
}
else
{
assert(vec1[i] != vec2[i]);
}
}
return 0;
}
For supercomputing simulation purpose, I have a structure that contains two big (billions of elements) std::vector: one std::vector of "keys" (64 bits integers) and one std::vector of "values". I cannot use a std::map because in the simulations I consider, vectors are far more optimal than std::map. Moreover, I cannot use a vector of pairs because of some optimization and cache efficiency provided by separate vectors. Moreover I cannot use any extra memory.
So, considering these constaints, what is the most optimized way to sort the two vectors by increasing values of the keys ? (template metaprogramming and crazy compile-time tricks are welcome)
Two ideas off the top of my head:
Take a quicksort implementation and apply it to the "key" vector; but modify the code so that every time it does a swap on the key vector, it also performs the same swap on the value vector.
Or, perhaps more in keeping with the spirit of C++, write a custom "wrapper" iterator which iterates over both vectors at once (returning a std::pair when dereferenced). Perhaps Boost has one? You could then combine this with std::sort and a custom comparison function which considers only the "key".
EDIT:
I've used the first suggestion here for a similar problem back in a past life as a C programmer. It's far from ideal for obvious reasons, but it's possibly the quickest way to get something going.
I haven't tried a wrapper iterator like this with std::sort, but TemplateRex in the comments says it won't work, and I'm happy to defer to him on that one.
I think problem may be splitted into 2 independent parts:
How to make effective iterator for virtual map
Which sorting alorithm to use
Iterator
Implementing iterator the main problem how to return pair of key/value not creating
unnecessary copies. We can achieve it by using different types for value_type & reference. My implementation is here.
template <typename _Keys, typename _Values>
class virtual_map
{
public:
typedef typename _Keys::value_type key_type;
typedef typename _Values::value_type mapped_type;
typedef std::pair<key_type, mapped_type> value_type;
typedef std::pair<key_type&, mapped_type&> proxy;
typedef std::pair<const key_type&, const mapped_type&> const_proxy;
class iterator :
public boost::iterator_facade < iterator, value_type, boost::random_access_traversal_tag, proxy >
{
friend class boost::iterator_core_access;
public:
iterator(virtual_map *map_, size_t offset_) :
map(map_),
offset(offset_)
{}
iterator(const iterator &other_)
{
this->map = other_.map;
this->offset = other_.offset;
}
private:
bool equal(const iterator &other) const
{
assert(this->map == other.map);
return this->offset == other.offset;
}
void increment() { ++offset; }
void decrement() { --offset; }
void advance(difference_type n) { offset += n; }
reference dereference() const { return reference(map->keys[offset], map->values[offset]); }
difference_type distance_to(const iterator &other_) const { return other_.offset - this->offset; }
private:
size_t offset;
virtual_map *map;
};
public:
virtual_map(_Keys &keys_, _Values &values_) :
keys(keys_),
values(values_)
{
if(keys_.size() != values_.size())
throw std::runtime_error("different size");
}
public:
iterator begin() { return iterator(this, 0); }
iterator end() { return iterator(this, keys.size()); }
protected:
_Keys &keys;
_Values &values;
};
usage sample:
int main(int argc, char* const argv[])
{
std::vector<int> keys_ = { 17, 2, 13, 4, 51, 78, 49, 37, 1 };
std::vector<std::string> values_ = { "17", "2", "13", "4", "51", "78", "49", "37", "1" };
typedef virtual_map<std::vector<int>, std::vector<std::string>> map;
map map_(keys_, values_);
std::sort(std::begin(map_), std::end(map_), [](map::const_proxy left_, map::const_proxy right_)
{
return left_.first < right_.first;
});
return 0;
}
Sorting algorithm
Its very hard to reason which method better without additional details. What memory restriction do you have? Is it possible to use concurrency?
There are some issues:
Iterating both sequences together requires a pair representing
references to the sequence elements - that pair, itself, is no
reference. Hence, algorithms working on references will not work.
Performance will degenerate (the sequences are loosely coupled) -
An implementation using a pair of references and std::sort:
// Copyright (c) 2014 Dieter Lucking. Distributed under the Boost
// software License, Version 1.0. (See accompanying file
// LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
#include <algorithm>
#include <chrono>
#include <memory>
#include <iostream>
// None
// ============================================================================
/// A void type
struct None {
None()
{}
/// Explicit conversion to None.
template <typename T>
explicit None(const T&)
{}
template <typename T>
None& operator = (const T&) {
return *this;
}
/// Never null.
None* operator & () const;
};
extern None& none();
inline None* None::operator & () const { return &none(); }
None& none() {
static None result;
return result;
}
// IteratorAdaptorTraits
// ============================================================================
namespace Detail {
// IteratorAdaptorTraits
// =====================
template <typename Iterator, typename ReturnType, bool IsReference>
struct IteratorAdaptorTraits;
// No reference
// ============
template <typename Iterator, typename ReturnType>
struct IteratorAdaptorTraits<Iterator, ReturnType, false>
{
typedef Iterator iterator_type;
typedef ReturnType return_type;
typedef ReturnType value_type;
typedef None reference;
typedef None pointer;
static_assert(
! std::is_base_of<None, return_type>::value,
"None as return type.");
template <typename Accessor>
static return_type iterator_value(const Accessor& accessor, const Iterator& iterator) {
return accessor.value(iterator);
}
template <typename Accessor>
static pointer iterator_pointer(const Accessor& accessor, const Iterator& iterator) {
return &none();
}
};
// Reference
// =========
template <typename Iterator, typename ReturnType>
struct IteratorAdaptorTraits<Iterator, ReturnType, true>
{
typedef Iterator iterator_type;
typedef ReturnType return_type;
typedef typename std::remove_reference<ReturnType>::type value_type;
typedef ReturnType reference;
typedef value_type* pointer;
static_assert(
! std::is_base_of<None, return_type>::value,
"None as return type.");
template <typename Accessor>
static return_type iterator_value(const Accessor& accessor, const Iterator& iterator) {
return accessor.value(iterator);
}
template <typename Accessor>
static pointer iterator_pointer(const Accessor& accessor, const Iterator& iterator) {
return &accessor.value(iterator);
}
};
} // namespace Detail
// RandomAccessIteratorAdaptor
// ============================================================================
/// An adaptor around a random access iterator.
/// \ATTENTION The adaptor will not fulfill the standard iterator requierments,
/// if the accessor does not support references: In that case, the
/// reference and pointer type are None.
template <typename Iterator, typename Accessor>
class RandomAccessIteratorAdaptor
{
// Types
// =====
private:
static_assert(
! std::is_base_of<None, Accessor>::value,
"None as accessor.");
static_assert(
! std::is_base_of<None, typename Accessor::return_type>::value,
"None as return type.");
typedef typename Detail::IteratorAdaptorTraits<
Iterator,
typename Accessor::return_type,
std::is_reference<typename Accessor::return_type>::value
> Traits;
public:
typedef typename Traits::iterator_type iterator_type;
typedef Accessor accessor_type;
typedef typename std::random_access_iterator_tag iterator_category;
typedef typename std::ptrdiff_t difference_type;
typedef typename Traits::return_type return_type;
typedef typename Traits::value_type value_type;
typedef typename Traits::reference reference;
typedef typename Traits::pointer pointer;
typedef typename accessor_type::base_type accessor_base_type;
typedef RandomAccessIteratorAdaptor<iterator_type, accessor_base_type> base_type;
// Tag
// ===
public:
struct RandomAccessIteratorAdaptorTag {};
// Construction
// ============
public:
explicit RandomAccessIteratorAdaptor(
iterator_type iterator, const accessor_type& accessor = accessor_type())
: m_iterator(iterator), m_accessor(accessor)
{}
template <typename IteratorType, typename AccessorType>
explicit RandomAccessIteratorAdaptor(const RandomAccessIteratorAdaptor<
IteratorType, AccessorType>& other)
: m_iterator(other.iterator()), m_accessor(other.accessor())
{}
// Element Access
// ==============
public:
/// The underlaying accessor.
const accessor_type& accessor() const { return m_accessor; }
/// The underlaying iterator.
const iterator_type& iterator() const { return m_iterator; }
/// The underlaying iterator.
iterator_type& iterator() { return m_iterator; }
/// The underlaying iterator.
operator iterator_type () const { return m_iterator; }
/// The base adaptor.
base_type base() const {
return base_type(m_iterator, m_accessor.base());
}
// Iterator
// ========
public:
return_type operator * () const {
return Traits::iterator_value(m_accessor, m_iterator);
}
pointer operator -> () const {
return Traits::iterator_pointer(m_accessor, m_iterator);
}
RandomAccessIteratorAdaptor increment() const {
return ++RandomAccessIteratorAdaptor(*this);
}
RandomAccessIteratorAdaptor increment_n(difference_type n) const {
RandomAccessIteratorAdaptor tmp(*this);
tmp.m_iterator += n;
return tmp;
}
RandomAccessIteratorAdaptor decrement() const {
return --RandomAccessIteratorAdaptor(*this);
}
RandomAccessIteratorAdaptor decrement_n(difference_type n) const {
RandomAccessIteratorAdaptor tmp(*this);
tmp.m_iterator -= n;
return tmp;
}
RandomAccessIteratorAdaptor& operator ++ () {
++m_iterator;
return *this;
}
RandomAccessIteratorAdaptor operator ++ (int) {
RandomAccessIteratorAdaptor tmp(*this);
++m_iterator;
return tmp;
}
RandomAccessIteratorAdaptor& operator += (difference_type n) {
m_iterator += n;
return *this;
}
RandomAccessIteratorAdaptor& operator -- () {
--m_iterator;
return *this;
}
RandomAccessIteratorAdaptor operator -- (int) {
RandomAccessIteratorAdaptor tmp(*this);
--m_iterator;
return tmp;
}
RandomAccessIteratorAdaptor& operator -= (difference_type n) {
m_iterator -= n;
return *this;
}
bool equal(const RandomAccessIteratorAdaptor& other) const {
return this->m_iterator == other.m_iterator;
}
bool less(const RandomAccessIteratorAdaptor& other) const {
return this->m_iterator < other.m_iterator;
}
bool less_equal(const RandomAccessIteratorAdaptor& other) const {
return this->m_iterator <= other.m_iterator;
}
bool greater(const RandomAccessIteratorAdaptor& other) const {
return this->m_iterator > other.m_iterator;
}
bool greater_equal(const RandomAccessIteratorAdaptor& other) const {
return this->m_iterator >= other.m_iterator;
}
private:
iterator_type m_iterator;
accessor_type m_accessor;
};
template <typename Iterator, typename Accessor>
inline RandomAccessIteratorAdaptor<Iterator, Accessor> operator + (
const RandomAccessIteratorAdaptor<Iterator, Accessor>& i,
typename RandomAccessIteratorAdaptor<Iterator, Accessor>::difference_type n) {
return i.increment_n(n);
}
template <typename Iterator, typename Accessor>
inline RandomAccessIteratorAdaptor<Iterator, Accessor> operator - (
const RandomAccessIteratorAdaptor<Iterator, Accessor>& i,
typename RandomAccessIteratorAdaptor<Iterator, Accessor>::difference_type n) {
return i.decrement_n(n);
}
template <typename Iterator, typename Accessor>
inline typename RandomAccessIteratorAdaptor<Iterator, Accessor>::difference_type
operator - (
const RandomAccessIteratorAdaptor<Iterator, Accessor>& a,
const RandomAccessIteratorAdaptor<Iterator, Accessor>& b) {
return a.iterator() - b.iterator();
}
template <typename Iterator, typename Accessor>
inline bool operator == (
const RandomAccessIteratorAdaptor<Iterator, Accessor>& a,
const RandomAccessIteratorAdaptor<Iterator, Accessor>& b) {
return a.equal(b);
}
template <typename Iterator, typename Accessor>
inline bool operator != (
const RandomAccessIteratorAdaptor<Iterator, Accessor>& a,
const RandomAccessIteratorAdaptor<Iterator, Accessor>& b) {
return ! a.equal(b);
}
template <typename Iterator, typename Accessor>
inline bool operator < (
const RandomAccessIteratorAdaptor<Iterator, Accessor>& a,
const RandomAccessIteratorAdaptor<Iterator, Accessor>& b) {
return a.less(b);
}
template <typename Iterator, typename Accessor>
inline bool operator <= (
const RandomAccessIteratorAdaptor<Iterator, Accessor>& a,
const RandomAccessIteratorAdaptor<Iterator, Accessor>& b) {
return a.less_equal(b);
}
template <typename Iterator, typename Accessor>
inline bool operator > (
const RandomAccessIteratorAdaptor<Iterator, Accessor>& a,
const RandomAccessIteratorAdaptor<Iterator, Accessor>& b) {
return a.greater(b);
}
template <typename Iterator, typename Accessor>
inline bool operator >= (
const RandomAccessIteratorAdaptor<Iterator, Accessor>& a,
const RandomAccessIteratorAdaptor<Iterator, Accessor>& b) {
return a.greater_equal(b);
}
// ElementPair
// ============================================================================
/// A pair of references which can mutate to a pair of values.
/// \NOTE If the key is one or two the pair is less comparable
/// regarding the first or second element.
template <typename First, typename Second, unsigned Key = 0>
class ElementPair
{
// Types
// =====
public:
typedef First first_type;
typedef Second second_type;
// Construction
// ============
public:
/// Reference
/// \POSTCONDITION reference() returns true
ElementPair(first_type& first, second_type& second)
: m_first(&first), m_second(&second)
{}
/// Copy construction
/// \POSTCONDITION reference() returns false
ElementPair(const ElementPair& other)
: m_first(new(m_first_storage) first_type(*other.m_first)),
m_second(new(&m_second_storage) second_type(*other.m_second))
{}
/// Move construction
/// \POSTCONDITION reference() returns false
ElementPair(ElementPair&& other)
: m_first(new(m_first_storage) first_type(std::move(*other.m_first))),
m_second(new(m_second_storage) second_type(std::move(*other.m_second)))
{}
~ElementPair() {
if( ! reference()) {
reinterpret_cast<first_type*>(m_first_storage)->~first_type();
reinterpret_cast<second_type*>(m_second_storage)->~second_type();
}
}
// Assignment
// ==========
public:
/// Swap content.
void swap(ElementPair& other) {
std::swap(*m_first, *other.m_first);
std::swap(*m_second, *other.m_second);
}
/// Assign content.
ElementPair& operator = (const ElementPair& other) {
if(&other != this) {
*m_first = *other.m_first;
*m_second = *other.m_second;
}
return *this;
}
/// Assign content.
ElementPair& operator = (ElementPair&& other) {
if(&other != this) {
*m_first = std::move(*other.m_first);
*m_second = std::move(*other.m_second);
}
return *this;
}
// Element Access
// ==============
public:
/// True if the pair holds references to external elements.
bool reference() {
return (m_first != reinterpret_cast<first_type*>(m_first_storage));
}
const first_type& first() const { return *m_first; }
first_type& first() { return *m_first; }
const second_type& second() const { return *m_second; }
second_type& second() { return *m_second; }
private:
first_type* m_first;
typename std::aligned_storage<
sizeof(first_type),
std::alignment_of<first_type>::value>::type
m_first_storage[1];
second_type* m_second;
typename std::aligned_storage<
sizeof(second_type),
std::alignment_of<second_type>::value>::type
m_second_storage[1];
};
// Compare
// =======
template <typename First, typename Second>
inline bool operator < (
const ElementPair<First, Second, 1>& a,
const ElementPair<First, Second, 1>& b)
{
return (a.first() < b.first());
}
template <typename First, typename Second>
inline bool operator < (
const ElementPair<First, Second, 2>& a,
const ElementPair<First, Second, 2>& b)
{
return (a.second() < b.second());
}
// Swap
// ====
namespace std {
template <typename First, typename Second, unsigned Key>
inline void swap(
ElementPair<First, Second, Key>& a,
ElementPair<First, Second, Key>& b)
{
a.swap(b);
}
}
// SequencePairAccessor
// ============================================================================
template <typename FirstSequence, typename SecondSequence, unsigned Keys = 0>
class SequencePairAccessor
{
// Types
// =====
public:
typedef FirstSequence first_sequence_type;
typedef SecondSequence second_sequence_type;
typedef typename first_sequence_type::size_type size_type;
typedef typename first_sequence_type::value_type first_type;
typedef typename second_sequence_type::value_type second_type;
typedef typename first_sequence_type::iterator iterator;
typedef None base_type;
typedef ElementPair<first_type, second_type, Keys> return_type;
// Construction
// ============
public:
SequencePairAccessor(first_sequence_type& first, second_sequence_type& second)
: m_first_sequence(&first), m_second_sequence(&second)
{}
// Element Access
// ==============
public:
base_type base() const { return base_type(); }
return_type value(iterator pos) const {
return return_type(*pos, (*m_second_sequence)[pos - m_first_sequence->begin()]);
}
// Data
// ====
private:
first_sequence_type* m_first_sequence;
second_sequence_type* m_second_sequence;
};
This test shows a degenaration of performance (on my system) by a factor of 1.5 for const char* and a factor of 3.4 for a std::string (compared to a single vector holding std::pair(s)).
// Test
// ============================================================================
#define SAMPLE_SIZE 1e1
#define VALUE_TYPE const char*
int main() {
const unsigned samples = SAMPLE_SIZE;
typedef int key_type;
typedef VALUE_TYPE value_type;
typedef std::vector<key_type> key_sequence_type;
typedef std::vector<value_type> value_sequence_type;
typedef SequencePairAccessor<key_sequence_type, value_sequence_type, 1> accessor_type;
typedef RandomAccessIteratorAdaptor<
key_sequence_type::iterator,
accessor_type>
iterator_adaptor_type;
key_sequence_type keys;
value_sequence_type values;
keys.reserve(samples);
values.reserve(samples);
const char* words[] = { "Zero", "One", "Two", "Three", "Four", "Five", "Six", "Seven", "Eight", "Nine" };
for(unsigned i = 0; i < samples; ++i) {
key_type k = i % 10;
keys.push_back(k);
values.push_back(words[k]);
}
accessor_type accessor(keys, values);
std::random_shuffle(
iterator_adaptor_type(keys.begin(), accessor),
iterator_adaptor_type(keys.end(), accessor)
);
if(samples <= 10) {
std::cout << "\nRandom:\n"
<< "======\n";
for(unsigned i = 0; i < keys.size(); ++i)
std::cout << keys[i] << ": " << values[i] << '\n';
}
typedef std::pair<key_type, value_type> pair_type;
std::vector<pair_type> ref;
for(const auto& k: keys) {
ref.push_back(pair_type(k, words[k]));
}
struct Less {
bool operator () (const pair_type& a, const pair_type& b) const {
return a.first < b.first;
}
};
auto ref_start = std::chrono::system_clock::now();
std::sort(ref.begin(), ref.end(), Less());
auto ref_end = std::chrono::system_clock::now();
auto ref_elapsed = double((ref_end - ref_start).count())
/ std::chrono::system_clock::period::den;
auto start = std::chrono::system_clock::now();
std::sort(
iterator_adaptor_type(keys.begin(), accessor),
iterator_adaptor_type(keys.end(), accessor)
);
auto end = std::chrono::system_clock::now();
auto elapsed = double((end - start).count())
/ std::chrono::system_clock::period::den;;
if(samples <= 10) {
std::cout << "\nSorted:\n"
<< "======\n";
for(unsigned i = 0; i < keys.size(); ++i)
std::cout << keys[i] << ": " << values[i] << '\n';
}
std::cout << "\nDuration sorting " << double(samples) << " samples:\n"
<< "========\n"
<< " One Vector: " << ref_elapsed << '\n'
<< "Two Vectors: " << elapsed << '\n'
<< " Factor: " << elapsed/ref_elapsed << '\n'
<< '\n';
}
(Please adjust SAMPLE_SIZE and VALUE_TYPE)
My conclusion is a sorted view into a sequence of unsorted data might be more aprropiate (but that violates the requirement of the question).