Inside an algorithm, I want to create a lambda that accepts an element by reference-to-const:
template<typename Iterator>
void solve_world_hunger(Iterator it)
{
auto lambda = [](const decltype(*it)& x){
auto y = x; // this should work
x = x; // this should fail
};
}
The compiler does not like this code:
Error: »const«-qualifier cannot be applied to »int&« (translated manually from German)
Then I realized that decltype(*it) is already a reference, and of course those cannot be made const. If I remove the const, the code compiles, but I want x = x to fail.
Let us trust the programmer (which is me) for a minute and get rid of the const and the explicit &, which gets dropped due to reference collapsing rules, anyways. But wait, is decltype(*it) actually guaranteed to be a reference, or should I add the explicit & to be on the safe side?
If we do not trust the programmer, I can think two solutions to solve the problem:
(const typename std::remove_reference<decltype(*it)>::type& x)
(const typename std::iterator_traits<Iterator>::value_type& x)
You can decide for yourself which one is uglier. Ideally, I would want a solution that does not involve any template meta-programming, because my target audience has never heard of that before. So:
Question 1: Is decltype(*it)& always the same as decltype(*it)?
Question 2: How can I pass an element by reference-to-const without template meta-programming?
Question 1: no, the requirement on InputIterator is merely that *it is convertible to T (table 72, in "Iterator requirements").
So decltype(*it) could for example be const char& for an iterator whose value_type is int. Or it could be int. Or double.
Using iterator_traits is not equivalent to using decltype, decide which you want.
For the same reason, auto value = *it; does not necessarily give you a variable with the value type of the iterator.
Question 2: might depend what you mean by template meta-programming.
If using a traits type is TMP, then there's no way of specifying "const reference to the value type of an iterator" without TMP, because iterator_traits is the only means to access the value type of an arbitrary iterator.
If you want to const-ify the decltype then how about this?
template<typename Iterator>
void solve_world_hunger(Iterator it)
{
const auto ret_type = *it;
auto lambda = [](decltype(ret_type)& x){
auto y = x; // this should work
x = x; // this should fail
};
}
You might have to capture ret_type in order to use its type, I can't easily check at the moment.
Unfortunately it dereferences the iterator an extra time. You could probably write some clever code to avoid that, but the clever code would end up being an alternative version of remove_reference, hence TMP.
Related
I have a function with a template parameter T and would like to pass a value of type const T by reference.
The C++ compiler throws an error, (kind of) understandably so. Hence I was wondering if there exists a way to do this in a safe and concise way?
I created a very small example that reflects the issue I am having in my project.
(in my project the issue appears in a constant member function of some class, but from my experiments the issue should be "faithfully" reflected in the example below by use of a constant variable of int instead for simplicity's sake).
I am aware that I could theoretically use a separate template parameter "cT", but that would be horribly unsafe, as the caller of this function need not pass an object of "const T" as second argument ...
I also understand that I could simply refrain from using templates at all and just specify this for every type.
I was just wondering if what I am trying to achieve below can be done with templates.
Thanks and have a nice day! :)
template<typename T>
bool ContainsElement(std::list<T>& findList, const T& elem)
{
for (auto& entry : findList)
{
if (entry == elem)
return true;
}
return false;
}
int main()
{
std::list<int*> myList;
const int testConst = 6;
auto pointerToTestConst = &testConst;
ContainsElement(myList, pointerToTestConst); // compiler screams
}
The issue is in incompatibility between pointers:
pointerToTestConst is of type const int* - non-const pointer to const integer. Therefore T=const int*
myList is of type list<int*>, deducing T=int*.
Since those types are not the same, compilation fails. Rightfully so, because elem would allow changing testConst if T=int*.
The issue manifests regardles you passing elem by (const) reference or value.
horribly unsafe, as the caller of this function need not pass an object of "const T" as second argument
So what? They get a compiler error about the comparison. It is no more unsafe than your code right now. I would argue that a generic ContainsElement should not care what you pass to is as long as it compares equal to some element in the list, it is a match.
Of course STL already offers this for std::find and std::ranges::find which also does not care and it is not called unsafe because of it.
To "normalize" the type for pointers and non-pointers, std::conditional may be used, finally your template still defines single T parameter:
#include <type_traits>
template<typename T>
bool ContainsElement(std::list<T>& findList,
const std::conditional_t<
std::is_pointer<T>::value,
std::add_pointer_t<
std::add_const_t<
std::remove_pointer_t<T>
>
>,
std::add_const_t<T>
>& elem)
{
for (auto& entry : findList)
{
if (entry == elem)
return true;
}
return false;
}
Above example uses helper types with postfix _t introduced in C++14, for C++11, use std::conditional::type.
It works for std::list<int> and std::list<int*>.
Of course in case of std::list<int*> it will check whether list contains exactly the same pointer address, not the pointed value.
I noticed that std::ranges::sort cannot sort std::vector<bool>:
<source>:6:51: error: no match for call to '(const std::ranges::__sort_fn) (std::vector<bool, std::allocator<bool> >)'
6 | std::ranges::sort(std::vector{false, true, true});
|
Is this allowed? Should we need a specialization of std::ranges::sort for std::vector<bool>? Is there any information about how the committee consider this?
As an update, now that zip was adopted for c++23, part of that paper added const-assignment to vector<bool>::reference, which allows that that type to satisfy indirectly_writable, and thus std::ranges::sort on a vector<bool> works in C++23.
Correct.
More generally, std::ranges::sort cannot sort proxy references. The direct reason is that sort requires sortable (surprising, right) which if we follow that chain up requires permutable which requires indirectly_movable_storable which requires indirectly_movable which requires indirectly_writable.
And indirectly_writeable is a very peculiar looking concept.
template<class Out, class T>
concept indirectly_writable =
requires(Out&& o, T&& t) {
*o = std::forward<T>(t); // not required to be equality-preserving
*std::forward<Out>(o) = std::forward<T>(t); // not required to be equality-preserving
const_cast<const iter_reference_t<Out>&&>(*o) =
std::forward<T>(t); // not required to be equality-preserving
const_cast<const iter_reference_t<Out>&&>(*std::forward<Out>(o)) =
std::forward<T>(t); // not required to be equality-preserving
};
I want to specifically draw your attention to:
const_cast<const iter_reference_t<Out>&&>(*o) = std::forward<T>(t);
Wait, we require const assignability?
This particular issue has a long history. You can start with #573, in which a user demonstrated this problem:
struct C
{
explicit C(std::string a) : bar(a) {}
std::string bar;
};
int main()
{
std::vector<C> cs = { C("z"), C("d"), C("b"), C("c") };
ranges::sort(cs | ranges::view::transform([](const C& x) {return x.bar;}));
for (const auto& c : cs) {
std::cout << c.bar << std::endl;
}
}
The expectation of course was that it would print b, c, d, z in that order. But it didn't. It printed z, d, b, c. The order didn't change. The reason here is that because this is a range of prvalues, the elements we're swapping as part of the sort. Well, they're temporaries. This has no effect on cs whatsoever.
This obviously can't work. The user has a bug - they intended to sort the Cs by the bars (i.e. use a projection) but instead they're just sorting the bars (even if the lambda returned a reference, they'd be sorting just the bars and not the Cs anyway -- in this case there is only one member of C anyway but in the general case this is clearly not the intended behavior).
But the goal then is really: how do we make this bug not compile? That's the dream. The problem is that C++ added ref-qualifications in C++11, but implicit assignment has always existed. And implicit operator= has no ref-qualifier, you can assign to an rvalue just fine, even if that makes no sense whatsoever:
std::string("hello") = "goodbye"; // fine, but pointless, probably indicative of a bug
Assigning to an rvalue is really only okay if the ravlue itself handles this correctly. Ideally, we could just check to make sure a type has an rvalue-qualified operator=. Proxy types (such as vector<bool>::reference) would then qualify their assignment operators, that's what we would check, and everybody's happy.
But we can't do that - because basically every type is rvalue-assignable, even if very few types actually meaningfully are. So what Eric and Casey came up with is morally equivalent to adding a type trait to a type that says "I am, legitimately, for real, rvalue-assignable." And unlike most type traits where you would do something like:
template <>
inline constexpr bool for_real_rvalue_assignable<T> = true;
This one is just spelled:
T& operator=(Whatever) const;
Even though the const equality operator will not actually be invoked as part of the algorithm. It just has to be there.
You might ask at this point - wait, what about references? For "normal" ranges (say, vector<int>, the iter_reference_t<Out> gives you int&, and const iter_reference_t<Out>&& is... still just int&. That's why this just works. For ranges that yield glvalues, these const-assignment requirements basically duplicate the normal assignment requirements. The const-assignability issue is _only_for prvalues.
This issue was also the driver of why views::zip isn't in C++20. Because zip also yields a prvalue range and a tuple<T&...> is precisely the kind of proxy reference that we would need to handle here. And to handle that, we would have to make a change to std::tuple to allow this kind of const-assignability.
As far as I'm aware, this is still the direction that it's intended (given that we have already enshrined that requirement into a concept, a requirement that no standard library proxy types actually satisfy). So when views::zip is added, tuple<T&...> will be made const-assignable as well as vector<bool>::reference.
The end result of that work is that:
std::ranges::sort(std::vector{false, true, true});
will actually both compile and work correctly.
What is the correct practice capturing return value of boost::make_iterator_range in following scenarios:
When used in range based for loop:
for (auto&& val : boost::make_iterator_range(...))// <-- is `auto&&` the right way?
When used as formal parameter:
DoStuff(boost::make_iterator_range(...));
template<class Iterator>
void DoStuff(Iterator&& it){}// <-- is Iterator&& the right way?
for (auto&& val : boost::make_iterator_range(...))// <-- is `auto&&` the right way?
The return value of make_iterator_range is not stored in val. val simply stores the return value of the dereferencing of the iterators: the element, not the entire range.
auto&& creates a forwarding reference to the element.
DoStuff(boost::make_iterator_range(...));
template<class Iterator>
void DoStuff(Iterator&& it){}// <-- is Iterator&& the right way?
Using the name Iterator belies a misunderstanding: make_itetator_range is not an iterator, it is a range defined by iterators.
template<class Range>
void DoStuff(Range&& r){}
This does not change the behaviour of the code, but it was like having a variable of type int named Double: pointlessly or aggressively confusing.
In any case, Range&& r in this context is also a forwarding reference.
A forwarding reference in a type deduction context can bind to either an lvalue or rvalue. It can cause reference lifetime extension as well.
In both of these situations, auto&& and Range&& are reasonable approaches, but so would auto const&, auto& or auto in the first case, and in the second Range const& or Range have their advantages. They just mean different things.
So your solutions are one way that can be right, but there is no "one right way" for what to put there. I use forwarding references by default, but that is a matter of taste: others might choose to default to const& or values. And in some contexts, auto& would be proper (where your loop modifies the value and intends that modification to persist).
I can see why the auto type in C++11 improves correctness and maintainability. I've read that it can also improve performance (Almost Always Auto by Herb Sutter), but I miss a good explanation.
How can auto improve performance?
Can anyone give an example?
auto can aid performance by avoiding silent implicit conversions. An example I find compelling is the following.
std::map<Key, Val> m;
// ...
for (std::pair<Key, Val> const& item : m) {
// do stuff
}
See the bug? Here we are, thinking we're elegantly taking every item in the map by const reference and using the new range-for expression to make our intent clear, but actually we're copying every element. This is because std::map<Key, Val>::value_type is std::pair<const Key, Val>, not std::pair<Key, Val>. Thus, when we (implicitly) have:
std::pair<Key, Val> const& item = *iter;
Instead of taking a reference to an existing object and leaving it at that, we have to do a type conversion. You are allowed to take a const reference to an object (or temporary) of a different type as long as there is an implicit conversion available, e.g.:
int const& i = 2.0; // perfectly OK
The type conversion is an allowed implicit conversion for the same reason you can convert a const Key to a Key, but we have to construct a temporary of the new type in order to allow for that. Thus, effectively our loop does:
std::pair<Key, Val> __tmp = *iter; // construct a temporary of the correct type
std::pair<Key, Val> const& item = __tmp; // then, take a reference to it
(Of course, there isn't actually a __tmp object, it's just there for illustration, in reality the unnamed temporary is just bound to item for its lifetime).
Just changing to:
for (auto const& item : m) {
// do stuff
}
just saved us a ton of copies - now the referenced type matches the initializer type, so no temporary or conversion is necessary, we can just do a direct reference.
Because auto deduces the type of the initializing expression, there is no type conversion involved. Combined with templated algorithms, this means that you can get a more direct computation than if you were to make up a type yourself – especially when you are dealing with expressions whose type you cannot name!
A typical example comes from (ab)using std::function:
std::function<bool(T, T)> cmp1 = std::bind(f, _2, 10, _1); // bad
auto cmp2 = std::bind(f, _2, 10, _1); // good
auto cmp3 = [](T a, T b){ return f(b, 10, a); }; // also good
std::stable_partition(begin(x), end(x), cmp?);
With cmp2 and cmp3, the entire algorithm can inline the comparison call, whereas if you construct a std::function object, not only can the call not be inlined, but you also have to go through the polymorphic lookup in the type-erased interior of the function wrapper.
Another variant on this theme is that you can say:
auto && f = MakeAThing();
This is always a reference, bound to the value of the function call expression, and never constructs any additional objects. If you didn't know the returned value's type, you might be forced to construct a new object (perhaps as a temporary) via something like T && f = MakeAThing(). (Moreover, auto && even works when the return type is not movable and the return value is a prvalue.)
There are two categories.
auto can avoid type erasure. There are unnamable types (like lambdas), and almost unnamable types (like the result of std::bind or other expression-template like things).
Without auto, you end up having to type erase the data down to something like std::function. Type erasure has costs.
std::function<void()> task1 = []{std::cout << "hello";};
auto task2 = []{std::cout << " world\n";};
task1 has type erasure overhead -- a possible heap allocation, difficulty inlining it, and virtual function table invocation overhead. task2 has none. Lambdas need auto or other forms of type deduction to store without type erasure; other types can be so complex that they only need it in practice.
Second, you can get types wrong. In some cases, the wrong type will work seemingly perfectly, but will cause a copy.
Foo const& f = expression();
will compile if expression() returns Bar const& or Bar or even Bar&, where Foo can be constructed from Bar. A temporary Foo will be created, then bound to f, and its lifetime will be extended until f goes away.
The programmer may have meant Bar const& f and not intended to make a copy there, but a copy is made regardless.
The most common example is the type of *std::map<A,B>::const_iterator, which is std::pair<A const, B> const& not std::pair<A,B> const&, but the error is a category of errors that silently cost performance. You can construct a std::pair<A, B> from a std::pair<const A, B>. (The key on a map is const, because editing it is a bad idea)
Both #Barry and #KerrekSB first illustrated these two principles in their answers. This is simply an attempt to highlight the two issues in one answer, with wording that aims at the problem rather than being example-centric.
The existing three answers give examples where using auto helps “makes it less likely to unintentionally pessimize” effectively making it "improve performance".
There is a flip side to the the coin. Using auto with objects that have operators that don't return the basic object can result in incorrect (still compilable and runable) code. For example, this question asks how using auto gave different (incorrect) results using the Eigen library, i.e. the following lines
const auto resAuto = Ha + Vector3(0.,0.,j * 2.567);
const Vector3 resVector3 = Ha + Vector3(0.,0.,j * 2.567);
std::cout << "resAuto = " << resAuto <<std::endl;
std::cout << "resVector3 = " << resVector3 <<std::endl;
resulted in different output. Admittedly, this is mostly due to Eigens lazy evaluation, but that code is/should be transparent to the (library) user.
While performance hasn't been greatly affected here, using auto to avoid unintentional pessimization might be classified as premature optimization, or at least wrong ;).
I generally prefer constness, but recently came across a conundrum with const iterators that shakes my const attitude annoys me about them:
MyList::const_iterator find( const MyList & list, int identifier )
{
// do some stuff to find identifier
return retConstItor; // has to be const_iterator because list is const
}
The idea that I'm trying to express here, of course, is that the passed in list cannot/willnot be changed, but once I make the list reference const I then have to use 'const_iterator's which then prevent me from doing anything with modifing the result (which makes sense).
Is the solution, then, to give up on making the passed in container reference const, or am I missing another possibility?
This has always been my secret reservation about const: That even if you use it correctly, it can create issues that it shouldn't where there is no good/clean solution, though I recognize that this is more specifically an issue between const and the iterator concept.
Edit: I am very aware of why you cannot and should not return a non-const iterator for a const container. My issue is that while I want a compile-time check for my container which is passed in by reference, I still want to find a way to pass back the position of something, and use it to modify the non-const version of the list. As mentioned in one of the answers it's possible to extract this concept of position via "advance", but messy/inefficient.
If I understand what you're saying correctly, you're trying to use const to indicate to the caller that your function will not modify the collection, but you want the caller (who may have a non-const reference to the collection) to be able to modify the collection using the iterator you return. If so, I don't think there's a clean solution for that, unless the container provides a mechanism for turning a const interator into a non-const one (I'm unaware of a container that does this). Your best bet is probably to have your function take a non-const reference. You may also be able to have 2 overloads of your function, one const and one non-const, so that in the case of a caller who has only a const reference, they will still be able to use your function.
It's not a trap; it's a feature. (:-)
In general, you can't return a non-const "handle" to your data from a const method. For example, the following code is illegal.
class Foo
{
public:
int& getX() const {return x;}
private:
int x;
};
If it were legal, then you could do something like this....
int main()
{
const Foo f;
f.getX() = 3; // changed value of const object
}
The designers of STL followed this convention with const-iterators.
In your case, what the const would buy you is the ability to call it on const collections. In which case, you wouldn't want the iterator returned to be modifiable. But you do want to allow it to be modifiable if the collection is non-const. So, you may want two interfaces:
MyList::const_iterator find( const MyList & list, int identifier )
{
// do some stuff to find identifier
return retConstItor; // has to be const_iterator because list is const
}
MyList::iterator find( MyList & list, int identifier )
{
// do some stuff to find identifier
return retItor;
}
Or, you can do it all with one template function
template<typename T>
T find(T start, T end, int identifier);
Then it will return a non-const iterator if the input iterators are non-const, and a const_iterator if they are const.
What I've done with wrapping standard algorithms, is have a metaobject for determining the type of container:
namespace detail
{
template <class Range>
struct Iterator
{
typedef typename Range::iterator type;
};
template <class Range>
struct Iterator<const Range>
{
typedef typename Range::const_iterator type;
};
}
This allows to provide a single implementation, e.g of find:
template <class Range, class Type>
typename detail::Iterator<Range>::type find(Range& range, const Type& value)
{
return std::find(range.begin(), range.end(), value);
}
However, this doesn't allow calling this with temporaries (I suppose I can live with it).
In any case, to return a modifiable reference to the container, apparently you can't make any guarantees what your function does or doesn't do with the container. So this noble principle indeed breaks down: don't get dogmatic about it.
I suppose const correctness is more of a service for the caller of your functions, rather that some baby-sitting measure that is supposed to make sure you get your simple find function right.
Another question is: how would you feel if I defined a following predicate and then abused the standard find_if algorithm to increment all the values up to the first value >= 3:
bool inc_if_less_than_3(int& a)
{
return a++ < 3;
}
(GCC doesn't stop me, but I couldn't tell if there's some undefined behaviour involved pedantically speaking.)
1) The container belongs to the user. Since allowing modification through the predicate in no way harms the algorithm, it should be up to the caller to decide how they use it.
2) This is hideous!!! Better implement find_if like this, to avoid this nightmare (best thing to do, since, apparently, you can't choose whether the iterator is const or not):
template <class Iter, class Pred>
Iter my_find_if(Iter first, Iter last, Pred fun)
{
while (first != last
&& !fun( const_cast<const typename std::iterator_traits<Iter>::value_type &>(*first)))
++first;
return first;
}
Although I think your design is a little confusing (as others have pointed iterators allow changes in the container, so I don't see your function really as const), there's a way to get an iterator out of a const_iterator. The efficiency depends on the kind of iterators.
#include <list>
int main()
{
typedef std::list<int> IntList;
typedef IntList::iterator Iter;
typedef IntList::const_iterator ConstIter;
IntList theList;
ConstIter cit = function_returning_const_iter(theList);
//Make non-const iter point to the same as the const iter.
Iter it(theList.begin());
std::advance(it, std::distance<ConstIter>(it, cit));
return 0;
}
Rather than trying to guarantee that the list won't be changed using the const keyword, it is better in this case to guarantee it using a postcondition. In other words, tell the user via comments that the list won't be changed.
Even better would be using a template that could be instantiated for iterators or const_iterators:
template <typename II> // II models InputIterator
II find(II first, int identifier) {
// do stuff
return iterator;
}
Of course, if you're going to go to that much trouble, you might as well expose the iterators of MyList to the user and use std::find.
If you're changing the data directed by the iterator, you're changing the list.
The idea that I'm trying to express here, of course, is that the passed in list cannot/willnot be changed, but once I make the list reference const I then have to use 'cons_iterator's which then prevent me from doing anything with the result.
What is "dong anything"? Modifying the data? That's changing the list, which is contradictory to your original intentions. If a list is const, it (and "it" includes its data) is constant.
If your function were to return a non-const iterator, it would create a way of modifying the list, hence the list wouldn't be const.
You are thinking about your design in the wrong way. Don't use const arguments to indicate what the function does - use them to describe the argument. In this case, it doesn't matter that find() doesn't change the list. What matters is that find() is intended to be used with modifiable lists.
If you want your find() function to return a non-const iterator, then it enables the caller to modify the container. It would be wrong for it to accept a const container, because that would provide the caller with a hidden way of removing the const-ness of the container.
Consider:
// Your function
MyList::iterator find(const MyList& list, int identifier);
// Caller has a const list.
const MyList list = ...
// but your function lets them modify it.
*( find(list,3) ) = 5;
So, your function should take a non-const argument:
MyList::iterator find(MyList& list, int identifier);
Now, when the caller tries to use your function to modify her const list, she'll get a compilation error. That's a much better outcome.
If you're going to return a non-const accessor to the container, make the function non-const as well. You're admitting the possibility of the container being changed by a side effect.
This is a good reason the standard algorithms take iterators rather than containers, so they can avoid this problem.