Do gcc and clang STL implementations violate rules about allocator? - c++

In 23.2.1p3 C++11 Standart we can read:
For the components affected by this subclause that declare an allocator_type, objects stored in these components shall be constructed using the allocator_traits<allocator_type>::construct function and destroyed using the allocator_traits<allocator_type>::destroy function (20.6.8.2). These functions are called only for the container’s element type, not for internal types used by the container. [ Note: This means, for example, that a node-based container might need to construct nodes containing aligned buffers and call construct to place the element into the buffer. —end note ]
allocator_traits<allocator_type>::construct just call passed allocator's construct method, if allocator defines one. I tried to use this and create allocator, which use list-initialization for construction, so I can utilize emplace for aggregate initialization:
#include <memory>
#include <vector>
#include <string>
#include <iostream>
#include <cmath>
template<typename T>
struct init_list_allocator : public std::allocator<T> {
template<typename... Args>
void construct(T* p, Args&&... args)
{ ::new((void *)p) T{std::forward<Args>(args)...}; }
// Fix copy-constructors usage for aggregates
void construct(T* p, T& copy_construct_arg)
{ std::allocator<T>::construct(p, copy_construct_arg); }
void construct(T* p, const T& copy_construct_arg)
{ std::allocator<T>::construct(p, copy_construct_arg); }
void construct(T* p, const T&& copy_construct_arg)
{ std::allocator<T>::construct(p, std::move(copy_construct_arg)); }
void construct(T *p, T&& move_construct_arg)
{ std::allocator<T>::construct(p, std::move(move_construct_arg)); }
};
template<class T>
using improved_vector = std::vector<T, init_list_allocator<T>>;
struct A {
int x;
double y;
const char* z;
};
int main()
{
using namespace std;
vector<string> strings;
improved_vector<A> v;
for (int i = 0; i < 21; ++i) {
strings.emplace_back(to_string(i*i));
v.emplace_back(i, sqrt(i), strings.back().c_str());
};
for (const auto& elem : v)
cout << elem.x << ' ' << elem.y << ' ' << elem.z << '\n';
}
However, at least in gcc and clang, this doesn't work. The problem is, that their implementations of vector use Allocator::rebind<T>::other::construct instead of Allocator::construct. And, because of our inheritance from std::allocator, this rebind gives std::allocator<T>::construct. Ok, no problem, just add
template<typename U>
struct rebind {
using other = init_list_allocator<U>;
};
in our allocator's definition and this code will work. Great, now let's change vector to list. Here we have the unsolvable problem, because instead of Allocator::construct object is initialized inside std::_List_node<_Tp> constuctor in direct-initialization form (form with brackets).
Are this 2 issues a standard violations or I miss something?

To my understanding, libstdc++ and MSVC++ are correct here. The point of rebind is, as the note indicates, that containers may be required to construct things that are not T. For example, std::list<T> needs to construct a list node containing T, not T. Similar cases exist for the associative and unordered containers. That's why the rebind structure exists in the first place. Your allocator was nonconforming before that was in place.
For the second issue, your reference
These functions are called only for the container’s element type, not for internal types used by the container.
seems to indicate that standard library implementations aren't allowed to call construct for rebound allocators, however. This may be a bug in libstdc++.
As for the actual solution to this problem, give A a constructor that has the behavior you want, and don't bother with allocators for this purpose. People may want to create instances of A outside of a container with the special allocator:
#include <vector>
struct A {
int x;
double y;
const char* z;
A() = default; // This allows A to still be a POD because the default constructor
// is not "user-provided", see 8.4.2 [dcl.fct.def.default]/4
A(int x_, double y_, char const* z_) : x(x_), y(y_), z(z_) {}
};
int main()
{
using namespace std;
vector<string> strings;
vector<A> v;
for (int i = 0; i < 21; ++i) {
strings.emplace_back(to_string(i*i));
v.emplace_back(i, sqrt(i), strings.back().c_str());
};
for (const auto& elem : v)
cout << elem.x << ' ' << elem.y << ' ' << elem.z << '\n';
}

After some research, I have found the answer myself and want to provide it.
For the first issue (using Allocator::rebind<T>::other::construct instead of Allocator::construct), my first allocator implementation (second one is OK) doesn't satisfy Allocator requirements in part of rebind, see 17.6.3.5 Table 28:
+------------------+-------------+-------------------------------------+
| Expression | Return type | Assertion/note pre-/post- condition |
+------------------+-------------+-------------------------------------+
| typename | Y | For all U (including T), |
| X::template | | Y::template rebind<T>::other is X. |
| rebind<U>::other | | |
+------------------+-------------+-------------------------------------+
For the second issue: GCC has old, pre-C++11 implementation of std::list, which will be fixed only in GCC 5.0, because this change breaks ABI (see Should std::list::size have constant complexity in C++11? for more info)
However, the quoted standard requirement, that container must call construct function for exactly allocator_type and not for some rebinded type seems like a standard defect (http://cplusplus.github.io/LWG/lwg-active.html#2218). Libstdc++ implementations of std::set, multiset, map and multimap rely on this fact and use rebinded allocator for construct (https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64096).

Related

How do I iterate over a group of vectors by their references? [duplicate]

I am trying to iterate over a number of std::lists, sorting each of them. This is the naive approach:
#include<list>
using namespace std;
int main(void){
list<int> a,b,c;
for(auto& l:{a,b,c}) l.sort();
}
producing
aa.cpp:5:25: error: no matching member function for call to 'sort'
for(auto& l:{a,b,c}) l.sort();
~~^~~~
/usr/bin/../lib64/gcc/x86_64-linux-gnu/4.9/../../../../include/c++/4.9/bits/stl_list.h:1586:7: note:
candidate function not viable: 'this' argument has type 'const
std::list<int, std::allocator<int> >', but method is not marked const
sort();
^
/usr/bin/../lib64/gcc/x86_64-linux-gnu/4.9/../../../../include/c++/4.9/bits/stl_list.h:1596:9: note:
candidate function template not viable: requires 1 argument, but 0 were
provided
sort(_StrictWeakOrdering);
^
1 error generated.
Am I correctly guessing that brace-initializer is creating copy of those lists? And is there a way to not copy them, and make them modifiable inside the loop? (other than making list of pointers to them, which is my current workaround).
You are guessing correctly. std::initializer_list elements are always const (which makes sort()ing them impossible, as sort() is a non-const member function) and its elements are always copied (which would make sort()-ing them meaningless even if they weren't const). From [dcl.init.list], emphasis mine:
An object of type std::initializer_list<E> is constructed from an initializer list as if the implementation
allocated a temporary array of N elements of type const E, where N is the number of elements in the
initializer list. Each element of that array is copy-initialized with the corresponding element of the initializer
list, and the std::initializer_list<E> object is constructed to refer to that array. [ Note: A constructor
or conversion function selected for the copy shall be accessible (Clause 11) in the context of the initializer
list. —end note ] If a narrowing conversion is required to initialize any of the elements, the program is
ill-formed. [ Example:
struct X {
X(std::initializer_list<double> v);
};
X x{ 1,2,3 };
The initialization will be implemented in a way roughly equivalent to this:
const double __a[3] = {double{1}, double{2}, double{3}};
X x(std::initializer_list<double>(__a, __a+3));
assuming that the implementation can construct an initializer_list object with a pair of pointers. —end
example ]
There is no way to make them non-const or non-copied. The pointer solution works:
for (auto l : {&a, &b, &c}) l->sort();
because it's the pointer that's const, not the element it's pointing to. The other alternative would be to write a variadic function template:
template <typename... Lists>
void sortAll(Lists&&... lists) {
// before C++17
using expander = int[];
expander{0, (void(lists.sort()), 0)...};
// C++17 or later
(lists.sort(), ...);
}
sortAll(a, b, c);
You could also, I guess, write a helper to wrap your lists into an array of reference_wrapper to list<int> (since you can't have an array of references), but this is probably more confusing than helpful:
template <typename List, typename... Lists>
std::array<std::reference_wrapper<List>, sizeof...(Lists) + 1>
as_array(List& x, Lists&... xs) {
return {x, xs...};
}
for (list<int>& l : as_array(a, b, c)) { // can't use auto, that deduces
l.sort(); // reference_wrapper<list<int>>,
} // so would need l.get().sort()
It is possible to write a function ref_range which allows you to do this:
for(auto& l : ref_range(a,b,c)) {
l.sort();
}
As others have said, once you write {a,b,c} you are stuck with an initializer_list, and such a list always takes copies of its arguments. The copies are const (hence your error), but even if you could get a non-const reference you would be modifying the copies of a, b and c instead of the originals.
Anyway, here is ref_range. It builds a vector of reference_wrapper.
// http://stackoverflow.com/questions/31724863/range-based-for-with-brace-initializer-over-non-const-values
#include<list>
#include<functional>
#include<array>
template<typename T, std:: size_t N>
struct hold_array_of_refs {
using vec_type = std:: array< std:: reference_wrapper<T>, N >;
vec_type m_v_of_refs;
hold_array_of_refs(vec_type && v_of_refs) : m_v_of_refs(std::move(v_of_refs)) { }
~hold_array_of_refs() { }
struct iterator {
typename vec_type :: const_iterator m_it;
iterator(typename vec_type :: const_iterator it) : m_it(it) {}
bool operator != (const iterator &other) {
return this->m_it != other.m_it;
}
iterator& operator++() { // prefix
++ this->m_it;
return *this;
}
T& operator*() {
return *m_it;
}
};
iterator begin() const {
return iterator(m_v_of_refs.begin());
}
iterator end() const {
return iterator(m_v_of_refs.end());
}
};
template<typename... Ts>
using getFirstTypeOfPack = typename std::tuple_element<0, std::tuple<Ts...>>::type;
template<typename ...T>
auto ref_range(T&... args) -> hold_array_of_refs< getFirstTypeOfPack<T...> , sizeof...(args)> {
return {{{ std:: ref(args)... }}}; // Why does clang prefer three levels of {} ?
}
#include<iostream>
int main(void){
std:: list<int> a,b,c;
// print the addresses, so we can verify we're dealing
// with the same objects
std:: cout << &a << std:: endl;
std:: cout << &b << std:: endl;
std:: cout << &c << std:: endl;
for(auto& l : ref_range(a,b,c)) {
std:: cout << &l << std:: endl;
l.sort();
}
}
The {...} syntax is actually creating an std::initializer_list. As the linked page states :
A std::initializer_list object is automatically constructed when:
[...]
a braced-init-list is bound to auto, including in a ranged for loop
And :
An object of type std::initializer_list<T> is a lightweight proxy object that provides access to an array of objects of type const T.
Thus, you can't modify the objects accessed through this initialize_list. Your solutions with the pointers looks like the easiest one to me.
Direct answer to your question:
Am I correctly guessing that brace-initializer is creating copy of
those lists?
Yes, this is the first problem. Your code would create copies of your lists, sort those copies, and finally forget the sorted copies.
However, that alone would just result in non-working code. The compiler error hints to a second problem: The implicit type of l is list<int> const&, rather than list<int>&. So the compiler complains that sort() tries to modify constant lists.
You can work around this second problem with a nasty const_cast:
#include <list>
#include <iostream>
using namespace std;
int main(void){
list<int> a,b,c;
a.push_back(2);
a.push_back(0);
a.push_back(1);
for(auto& l:{a,b,c}) const_cast<list<int>&>(l).sort();
for(auto i:a) cout << i << endl;
}
However, that will then trigger the first problem: Your list of lists contains copies, and only those copies are sorted. So the final output is not what you want:
2
0
1
The easiest workaround is to create a list of pointers to your lists:
#include <list>
#include <iostream>
using namespace std;
int main(void){
list<int> a,b,c;
a.push_back(2);
a.push_back(0);
a.push_back(1);
for(auto l:{&a,&b,&c}) l->sort();
for(auto i:a) cout << i << endl;
}
This will produce the desired result:
0
1
2
Others have already mentioned std::reference_wrapper, but they then used it to create an STL container instead of sticking with the brace-initializer list.
So all you need to do is:
for(auto& l:{std::ref(a),std::ref(b),std::ref(c)}) l.get().sort();
Of course, this is very similiar to the pointer solution already suggested.

What does it mean to inherit from lambda?

Found this code which looks to be interesting:
auto a = [](){};
class B : decltype(a)
{
};
I want to know what it does. Can this be useful in any way?
Well, that code will compile, but the problem is that you will be unable to default construct any object of that class1, because the constructor of the lambda isn't accessible (other than copy/move constructors). The only constructors guaranteed by a lambda type is a defaulted copy/move constructor. And there's no default constructor
[expr.prim.lambda/21]
The closure type associated with a lambda-expression has no default
constructor and a deleted copy assignment operator. It has a defaulted
copy constructor and a defaulted move constructor ([class.copy]). [
Note: These special member functions are implicitly defined as usual,
and might therefore be defined as deleted. — end note ]
or from cppreference:
//ClosureType() = delete; //(until C++14)
ClosureType(const ClosureType& ) = default; //(since C++14)
ClosureType(ClosureType&& ) = default; //(since C++14)
The history of the lambda constructor being inaccessible dates back to its early days proposal, found here
In section 3, second paragraph, and I quote:
In this translation, __some_unique_name is a new name, not used
elsewhere in the program in a way that would cause conflicts with its
use as a closure type. This name, and the constructor for the class,
do not need to be exposed to the user—the only features that the user
can rely on in the closure type are a copy constructor (and a move
constructor if that proposal is approved) and the function call
operator. Closure types do not need default constructors, assignment
operators, or any other means of access beyond function calls. It may
be worthwhile for implementability to forbid creating derived classes
from closure types. ...
As you can see, the proposal even suggested that creating derived classes from closure types should be forbidden.
1 of course you can copy-initialize the base class with a in order to initialize an object of type B. See this
Now, to your question:
Can this be useful in any way?
Not in your exact form. Your's will only be instantiable with the instance a.
However, if you inherit from a generic Callable Class such as a lambda type, there are two cases I can think of.
Create a Functor that calls a group of functors in a given inheritance sequence:
A simplified example:
template<typename TFirst, typename... TRemaining>
class FunctionSequence : public TFirst, FunctionSequence<TRemaining...>
{
public:
FunctionSequence(TFirst first, TRemaining... remaining)
: TFirst(first), FunctionSequence<TRemaining...>(remaining...)
{}
template<typename... Args>
decltype(auto) operator () (Args&&... args){
return FunctionSequence<TRemaining...>::operator()
( TFirst::operator()(std::forward<Arg>(args)...) );
}
};
template<typename T>
class FunctionSequence<T> : public T
{
public:
FunctionSequence(T t) : T(t) {}
using T::operator();
};
template<typename... T>
auto make_functionSequence(T... t){
return FunctionSequence<T...>(t...);
}
example usage:
int main(){
//note: these lambda functions are bug ridden. Its just for simplicity here.
//For correct version, see the one on coliru, read on.
auto trimLeft = [](std::string& str) -> std::string& { str.erase(0, str.find_first_not_of(' ')); return str; };
auto trimRight = [](std::string& str) -> std::string& { str.erase(str.find_last_not_of(' ')+1); return str; };
auto capitalize = [](std::string& str) -> std::string& { for(auto& x : str) x = std::toupper(x); return str; };
auto trimAndCapitalize = make_functionSequence(trimLeft, trimRight, capitalize);
std::string str = " what a Hullabaloo ";
std::cout << "Before TrimAndCapitalize: str = \"" << str << "\"\n";
trimAndCapitalize(str);
std::cout << "After TrimAndCapitalize: str = \"" << str << "\"\n";
return 0;
}
output
Before TrimAndCapitalize: str = " what a Hullabaloo "
After TrimAndCapitalize: str = "WHAT A HULLABALOO"
See it Live on Coliru
Create a Functor with an overloaded operator()(...), overloaded with all base classes' operator()(...):
Nir Friedman has already given a good instance of that in his answer to this question.
I have also drafted out a similar and simplified example, drawn from His. See it on Coliru
Jason Lucas also demonstrated its practical applications in his CppCon 2014 presentation "Polymorphism with Unions". You can find the Repo here, one of exact location in source code here (Thanks Cameron DaCamara)
Another cool trick: Since the resulting type from make_functionSequence(...) is a callable class. You can append more lambda's or callable to it at a later time.
//.... As previously seen
auto trimAndCapitalize = make_functionSequence(trimLeft, trimRight, capitalize);
auto replace = [](std::string& str) -> std::string& { str.replace(0, 4, "Whaaaaat"); return str; };
//Add more Functors/lambdas to the original trimAndCapitalize
auto replaced = make_functionSequence(trimAndCapitalize, replace /*, ... */);
replaced(str2);
Lambdas are function objects underneath, with additional syntatic sugar. a evaluates to something like that (with the MyLambda name being a random name, just like when you make namespace {} - namespace name will be random):
class MyLambda {
public:
void operator()() {
}
}
So when you inherit from a lambda, what you're doing is inheriting from an anonymous class / structure.
As for usefulness, well, it's as useful as any other inheritance. You can have the functionality of multiple lambdas with multiple inheritance in one object, you can add new methods to it to extend it. I can't think of any real application at the moment, but I'm sure there are many.
Refer to this question for more information.
This can actually be quite useful, but it depends how direct you want to be about the whole thing. Consider the following code:
#include <boost/variant.hpp>
#include <iostream>
#include <string>
#include <unordered_map>
template <class R, class T, class ... Ts>
struct Inheritor : public T, Inheritor<R, Ts...>
{
using T::operator();
using Inheritor<R, Ts...>::operator();
Inheritor(T t, Ts ... ts) : T(t), Inheritor<R, Ts...>(ts...) {}
};
template <class R, class T>
struct Inheritor<R, T> : public boost::static_visitor<R>, T
{
using T::operator();
Inheritor(T t) : T(t) {}
};
template <class R, class V, class ... T>
auto apply_visitor_inline(V& v, T ... t)
{
Inheritor<R, T...> i(t...);
return boost::apply_visitor(i, v);
}
int main()
{
boost::variant< int, std::string > u("hello world");
boost::variant< int, std::string > u2(5);
auto result = apply_visitor_inline<int64_t>(u, [] (int i) { return i;}, [] (const std::string& s) { return s.size();});
auto result2 = apply_visitor_inline<int64_t>(u2, [] (int i) { return i;}, [] (const std::string& s) { return s.size();});
std::cout << result;
std::cout << result2;
}
The snippet in your question does not show up in exact form anywhere. But you can see that the types of lambdas are being inferred in apply_visitor_inline. A class is then instantiated that inherits from all of these lambdas. The purpose? We are able to combine multiple lambdas into a single one, for the purpose of things like apply_visitor. This function expects to receive a single function object that defines multiple operator() and distinguish between them based on overloading. But sometimes it's more convenient to define a lambda that operates on each of the types we have to cover. In that case, inheritance from lambdas provides a mechanism for combining.
I got the inline visitor idea from here: https://github.com/exclipy/inline_variant_visitor, though I did not look at the implementation there, so this implementation is my own (but I guess its very similar).
Edit: the originally posted code only worked due to a bug in clang. According to this question (Overloaded lambdas in C++ and differences between clang and gcc), the lookup for multiple operator() in base classes is ambiguous, and indeed the code I posted originally did not compile in gcc. The new code compiles in both and should be compliant. Sadly, there is no way seemingly to do a variadic using statement, so recursion must be used.

Range-based for with brace-initializer over non-const values?

I am trying to iterate over a number of std::lists, sorting each of them. This is the naive approach:
#include<list>
using namespace std;
int main(void){
list<int> a,b,c;
for(auto& l:{a,b,c}) l.sort();
}
producing
aa.cpp:5:25: error: no matching member function for call to 'sort'
for(auto& l:{a,b,c}) l.sort();
~~^~~~
/usr/bin/../lib64/gcc/x86_64-linux-gnu/4.9/../../../../include/c++/4.9/bits/stl_list.h:1586:7: note:
candidate function not viable: 'this' argument has type 'const
std::list<int, std::allocator<int> >', but method is not marked const
sort();
^
/usr/bin/../lib64/gcc/x86_64-linux-gnu/4.9/../../../../include/c++/4.9/bits/stl_list.h:1596:9: note:
candidate function template not viable: requires 1 argument, but 0 were
provided
sort(_StrictWeakOrdering);
^
1 error generated.
Am I correctly guessing that brace-initializer is creating copy of those lists? And is there a way to not copy them, and make them modifiable inside the loop? (other than making list of pointers to them, which is my current workaround).
You are guessing correctly. std::initializer_list elements are always const (which makes sort()ing them impossible, as sort() is a non-const member function) and its elements are always copied (which would make sort()-ing them meaningless even if they weren't const). From [dcl.init.list], emphasis mine:
An object of type std::initializer_list<E> is constructed from an initializer list as if the implementation
allocated a temporary array of N elements of type const E, where N is the number of elements in the
initializer list. Each element of that array is copy-initialized with the corresponding element of the initializer
list, and the std::initializer_list<E> object is constructed to refer to that array. [ Note: A constructor
or conversion function selected for the copy shall be accessible (Clause 11) in the context of the initializer
list. —end note ] If a narrowing conversion is required to initialize any of the elements, the program is
ill-formed. [ Example:
struct X {
X(std::initializer_list<double> v);
};
X x{ 1,2,3 };
The initialization will be implemented in a way roughly equivalent to this:
const double __a[3] = {double{1}, double{2}, double{3}};
X x(std::initializer_list<double>(__a, __a+3));
assuming that the implementation can construct an initializer_list object with a pair of pointers. —end
example ]
There is no way to make them non-const or non-copied. The pointer solution works:
for (auto l : {&a, &b, &c}) l->sort();
because it's the pointer that's const, not the element it's pointing to. The other alternative would be to write a variadic function template:
template <typename... Lists>
void sortAll(Lists&&... lists) {
// before C++17
using expander = int[];
expander{0, (void(lists.sort()), 0)...};
// C++17 or later
(lists.sort(), ...);
}
sortAll(a, b, c);
You could also, I guess, write a helper to wrap your lists into an array of reference_wrapper to list<int> (since you can't have an array of references), but this is probably more confusing than helpful:
template <typename List, typename... Lists>
std::array<std::reference_wrapper<List>, sizeof...(Lists) + 1>
as_array(List& x, Lists&... xs) {
return {x, xs...};
}
for (list<int>& l : as_array(a, b, c)) { // can't use auto, that deduces
l.sort(); // reference_wrapper<list<int>>,
} // so would need l.get().sort()
It is possible to write a function ref_range which allows you to do this:
for(auto& l : ref_range(a,b,c)) {
l.sort();
}
As others have said, once you write {a,b,c} you are stuck with an initializer_list, and such a list always takes copies of its arguments. The copies are const (hence your error), but even if you could get a non-const reference you would be modifying the copies of a, b and c instead of the originals.
Anyway, here is ref_range. It builds a vector of reference_wrapper.
// http://stackoverflow.com/questions/31724863/range-based-for-with-brace-initializer-over-non-const-values
#include<list>
#include<functional>
#include<array>
template<typename T, std:: size_t N>
struct hold_array_of_refs {
using vec_type = std:: array< std:: reference_wrapper<T>, N >;
vec_type m_v_of_refs;
hold_array_of_refs(vec_type && v_of_refs) : m_v_of_refs(std::move(v_of_refs)) { }
~hold_array_of_refs() { }
struct iterator {
typename vec_type :: const_iterator m_it;
iterator(typename vec_type :: const_iterator it) : m_it(it) {}
bool operator != (const iterator &other) {
return this->m_it != other.m_it;
}
iterator& operator++() { // prefix
++ this->m_it;
return *this;
}
T& operator*() {
return *m_it;
}
};
iterator begin() const {
return iterator(m_v_of_refs.begin());
}
iterator end() const {
return iterator(m_v_of_refs.end());
}
};
template<typename... Ts>
using getFirstTypeOfPack = typename std::tuple_element<0, std::tuple<Ts...>>::type;
template<typename ...T>
auto ref_range(T&... args) -> hold_array_of_refs< getFirstTypeOfPack<T...> , sizeof...(args)> {
return {{{ std:: ref(args)... }}}; // Why does clang prefer three levels of {} ?
}
#include<iostream>
int main(void){
std:: list<int> a,b,c;
// print the addresses, so we can verify we're dealing
// with the same objects
std:: cout << &a << std:: endl;
std:: cout << &b << std:: endl;
std:: cout << &c << std:: endl;
for(auto& l : ref_range(a,b,c)) {
std:: cout << &l << std:: endl;
l.sort();
}
}
The {...} syntax is actually creating an std::initializer_list. As the linked page states :
A std::initializer_list object is automatically constructed when:
[...]
a braced-init-list is bound to auto, including in a ranged for loop
And :
An object of type std::initializer_list<T> is a lightweight proxy object that provides access to an array of objects of type const T.
Thus, you can't modify the objects accessed through this initialize_list. Your solutions with the pointers looks like the easiest one to me.
Direct answer to your question:
Am I correctly guessing that brace-initializer is creating copy of
those lists?
Yes, this is the first problem. Your code would create copies of your lists, sort those copies, and finally forget the sorted copies.
However, that alone would just result in non-working code. The compiler error hints to a second problem: The implicit type of l is list<int> const&, rather than list<int>&. So the compiler complains that sort() tries to modify constant lists.
You can work around this second problem with a nasty const_cast:
#include <list>
#include <iostream>
using namespace std;
int main(void){
list<int> a,b,c;
a.push_back(2);
a.push_back(0);
a.push_back(1);
for(auto& l:{a,b,c}) const_cast<list<int>&>(l).sort();
for(auto i:a) cout << i << endl;
}
However, that will then trigger the first problem: Your list of lists contains copies, and only those copies are sorted. So the final output is not what you want:
2
0
1
The easiest workaround is to create a list of pointers to your lists:
#include <list>
#include <iostream>
using namespace std;
int main(void){
list<int> a,b,c;
a.push_back(2);
a.push_back(0);
a.push_back(1);
for(auto l:{&a,&b,&c}) l->sort();
for(auto i:a) cout << i << endl;
}
This will produce the desired result:
0
1
2
Others have already mentioned std::reference_wrapper, but they then used it to create an STL container instead of sticking with the brace-initializer list.
So all you need to do is:
for(auto& l:{std::ref(a),std::ref(b),std::ref(c)}) l.get().sort();
Of course, this is very similiar to the pointer solution already suggested.

Use of rvalue references in function parameter of overloaded function creates too many combinations

Imagine you have a number of overloaded methods that (before C++11) looked like this:
class MyClass {
public:
void f(const MyBigType& a, int id);
void f(const MyBigType& a, string name);
void f(const MyBigType& a, int b, int c, int d);
// ...
};
This function makes a copy of a (MyBigType), so I want to add an optimization by providing a version of f that moves a instead of copying it.
My problem is that now the number of f overloads will duplicate:
class MyClass {
public:
void f(const MyBigType& a, int id);
void f(const MyBigType& a, string name);
void f(const MyBigType& a, int b, int c, int d);
// ...
void f(MyBigType&& a, int id);
void f(MyBigType&& a, string name);
void f(MyBigType&& a, int b, int c, int d);
// ...
};
If I had more parameters that could be moved, it would be unpractical to provide all the overloads.
Has anyone dealt with this issue? Is there a good solution/pattern to solve this problem?
Thanks!
Herb Sutter talks about something similar in a cppcon talk
This can be done but probably shouldn't. You can get the effect out using universal references and templates, but you want to constrain the type to MyBigType and things that are implicitly convertible to MyBigType. With some tmp tricks, you can do this:
class MyClass {
public:
template <typename T>
typename std::enable_if<std::is_convertible<T, MyBigType>::value, void>::type
f(T&& a, int id);
};
The only template parameter will match against the actual type of the parameter, the enable_if return type disallows incompatible types. I'll take it apart piece by piece
std::is_convertible<T, MyBigType>::value
This compile time expression will evaluate to true if T can be converted implicitly to a MyBigType. For example, if MyBigType were a std::string and T were a char* the expression would be true, but if T were an int it would be false.
typename std::enable_if<..., void>::type // where the ... is the above
this expression will result in void in the case that the is_convertible expression is true. When it's false, the expression will be malformed, so the template will be thrown out.
Inside the body of the function you'll need to use perfect forwarding, if you are planning on copy assigning or move assigning, the body would be something like
{
this->a_ = std::forward<T>(a);
}
Here's a coliru live example with a using MyBigType = std::string. As Herb says, this function can't be virtual and must be implemented in the header. The error messages you get from calling with a wrong type will be pretty rough compared to the non-templated overloads.
Thanks to Barry's comment for this suggestion, to reduce repetition, it's probably a good idea to create a template alias for the SFINAE mechanism. If you declare in your class
template <typename T>
using EnableIfIsMyBigType = typename std::enable_if<std::is_convertible<T, MyBigType>::value, void>::type;
then you could reduce the declarations to
template <typename T>
EnableIfIsMyBigType<T>
f(T&& a, int id);
However, this assumes all of your overloads have a void return type. If the return type differs you could use a two-argument alias instead
template <typename T, typename R>
using EnableIfIsMyBigType = typename std::enable_if<std::is_convertible<T, MyBigType>::value,R>::type;
Then declare with the return type specified
template <typename T>
EnableIfIsMyBigType<T, void> // void is the return type
f(T&& a, int id);
The slightly slower option is to take the argument by value. If you do
class MyClass {
public:
void f(MyBigType a, int id) {
this->a_ = std::move(a); // move assignment
}
};
In the case where f is passed an lvalue, it will copy construct a from its argument, then move assign it into this->a_. In the case that f is passed an rvalue, it will move construct a from the argument and then move assign. A live example of this behavior is here. Note that I use -fno-elide-constructors, without that flag, the rvalue cases elides the move construction and only the move assignment takes place.
If the object is expensive to move (std::array for example) this approach will be noticeably slower than the super-optimized first version. Also, consider watching this part of Herb's talk that Chris Drew links to in the comments to understand when it could be slower than using references. If you have a copy of Effective Modern C++ by Scott Meyers, he discusses the ups and downs in item 41.
You may do something like the following.
class MyClass {
public:
void f(MyBigType a, int id) { this->a = std::move(a); /*...*/ }
void f(MyBigType a, string name);
void f(MyBigType a, int b, int c, int d);
// ...
};
You just have an extra move (which may be optimized).
My first thought is that you should change the parameters to pass by value. This covers the existing need to copy, except the copy happens at the call point rather than explicitly in the function. It also allows the parameters to be created by move construction in a move-able context (either unnamed temporaries or by using std::move).
Why you would do that
These extra overloads only make sense, if modifying the function paramers in the implementation of the function really gives you a signigicant performance gain (or some kind of guarantee). This is hardly ever the case except for the case of constructors or assignment operators. Therefore, I would advise you to rethink, whether putting these overloads there is really necessary.
If the implementations are almost identical...
From my experience this modification is simply passing the parameter to another function wrapped in std::move() and the rest of the function is identical to the const & version. In that case you might turn your function into a template of this kind:
template <typename T> void f(T && a, int id);
Then in the function implementation you just replace the std::move(a) operation with std::forward<T>(a) and it should work. You can constrain the parameter type T with std::enable_if, if you like.
In the const ref case: Don't create a temporary, just to to modify it
If in the case of constant references you create a copy of your parameter and then continue the same way the move version works, then you may as well just pass the parameter by value and use the same implementation you used for the move version.
void f( MyBigData a, int id );
This will usually give you the same performance in both cases and you only need one overload and implementation. Lots of plusses!
Significantly different implementations
In case the two implementations differ significantly, there is no generic solution as far as I know. And I believe there can be none. This is also the only case, where doing this really makes sense, if profiling the performance shows you adequate improvements.
You might introduce a mutable object:
#include <memory>
#include <type_traits>
// Mutable
// =======
template <typename T>
class Mutable
{
public:
Mutable(const T& value) : m_ptr(new(m_storage) T(value)) {}
Mutable(T& value) : m_ptr(&value) {}
Mutable(T&& value) : m_ptr(new(m_storage) T(std::move(value))) {}
~Mutable() {
auto storage = reinterpret_cast<T*>(m_storage);
if(m_ptr == storage)
m_ptr->~T();
}
Mutable(const Mutable&) = delete;
Mutable& operator = (const Mutable&) = delete;
const T* operator -> () const { return m_ptr; }
T* operator -> () { return m_ptr; }
const T& operator * () const { return *m_ptr; }
T& operator * () { return *m_ptr; }
private:
T* m_ptr;
char m_storage[sizeof(T)];
};
// Usage
// =====
#include <iostream>
struct X
{
int value = 0;
X() { std::cout << "default\n"; }
X(const X&) { std::cout << "copy\n"; }
X(X&&) { std::cout << "move\n"; }
X& operator = (const X&) { std::cout << "assign copy\n"; return *this; }
X& operator = (X&&) { std::cout << "assign move\n"; return *this; }
~X() { std::cout << "destruct " << value << "\n"; }
};
X make_x() { return X(); }
void fn(Mutable<X>&& x) {
x->value = 1;
}
int main()
{
const X x0;
std::cout << "0:\n";
fn(x0);
std::cout << "1:\n";
X x1;
fn(x1);
std::cout << "2:\n";
fn(make_x());
std::cout << "End\n";
}
This is the critical part of the question:
This function makes a copy of a (MyBigType),
Unfortunately, it is a little ambiguous. We would like to know what is the ultimate target of the data in the parameter. Is it:
1) to be assigned to an object that existing before f was called?
2) or instead, stored in a local variable:
i.e:
void f(??? a, int id) {
this->x = ??? a ???;
...
}
or
void f(??? a, int id) {
MyBigType a_copy = ??? a ???;
...
}
Sometimes, the first version (the assignment) can be done without any copies or moves. If this->x is already long string, and if a is short, then it can efficiently reuse the existing capacity. No copy-construction, and no moves. In short, sometimes assignment can be faster because we can skip the copy contruction.
Anyway, here goes:
template<typename T>
void f(T&& a, int id) {
this->x = std::forward<T>(a); // is assigning
MyBigType local = std::forward<T>(a); // if move/copy constructing
}
If the move version will provide any optimization then the implementation of the move overloaded function and the copy one must be really different. I don't see a way to get around this without providing implementations for both.

Type erasure techniques

(With type erasure, I mean hiding some or all of the type information regarding a class, somewhat like Boost.Any.)
I want to get a hold of type erasure techniques, while also sharing those, which I know of. My hope is kinda to find some crazy technique that somebody thought of in his/her darkest hour. :)
The first and most obvious, and commonly taken approach, that I know, are virtual functions. Just hide the implementation of your class inside an interface based class hierarchy. Many Boost libraries do this, for example Boost.Any does this to hide your type and Boost.Shared_ptr does this to hide the (de)allocation mechanic.
Then there is the option with function pointers to templated functions, while holding the actual object in a void* pointer, like Boost.Function does to hide the real type of the functor. Example implementations can be found at the end of the question.
So, for my actual question:
What other type erasure techniques do you know of? Please provide them, if possible, with an example code, use cases, your experience with them and maybe links for further reading.
Edit
(Since I wasn't sure wether to add this as an answer, or just edit the question, I'll just do the safer one.)
Another nice technique to hide the actual type of something without virtual functions or void* fiddling, is the one GMan employs here, with relevance to my question on how exactly this works.
Example code:
#include <iostream>
#include <string>
// NOTE: The class name indicates the underlying type erasure technique
// this behaves like the Boost.Any type w.r.t. implementation details
class Any_Virtual{
struct holder_base{
virtual ~holder_base(){}
virtual holder_base* clone() const = 0;
};
template<class T>
struct holder : holder_base{
holder()
: held_()
{}
holder(T const& t)
: held_(t)
{}
virtual ~holder(){
}
virtual holder_base* clone() const {
return new holder<T>(*this);
}
T held_;
};
public:
Any_Virtual()
: storage_(0)
{}
Any_Virtual(Any_Virtual const& other)
: storage_(other.storage_->clone())
{}
template<class T>
Any_Virtual(T const& t)
: storage_(new holder<T>(t))
{}
~Any_Virtual(){
Clear();
}
Any_Virtual& operator=(Any_Virtual const& other){
Clear();
storage_ = other.storage_->clone();
return *this;
}
template<class T>
Any_Virtual& operator=(T const& t){
Clear();
storage_ = new holder<T>(t);
return *this;
}
void Clear(){
if(storage_)
delete storage_;
}
template<class T>
T& As(){
return static_cast<holder<T>*>(storage_)->held_;
}
private:
holder_base* storage_;
};
// the following demonstrates the use of void pointers
// and function pointers to templated operate functions
// to safely hide the type
enum Operation{
CopyTag,
DeleteTag
};
template<class T>
void Operate(void*const& in, void*& out, Operation op){
switch(op){
case CopyTag:
out = new T(*static_cast<T*>(in));
return;
case DeleteTag:
delete static_cast<T*>(out);
}
}
class Any_VoidPtr{
public:
Any_VoidPtr()
: object_(0)
, operate_(0)
{}
Any_VoidPtr(Any_VoidPtr const& other)
: object_(0)
, operate_(other.operate_)
{
if(other.object_)
operate_(other.object_, object_, CopyTag);
}
template<class T>
Any_VoidPtr(T const& t)
: object_(new T(t))
, operate_(&Operate<T>)
{}
~Any_VoidPtr(){
Clear();
}
Any_VoidPtr& operator=(Any_VoidPtr const& other){
Clear();
operate_ = other.operate_;
operate_(other.object_, object_, CopyTag);
return *this;
}
template<class T>
Any_VoidPtr& operator=(T const& t){
Clear();
object_ = new T(t);
operate_ = &Operate<T>;
return *this;
}
void Clear(){
if(object_)
operate_(0,object_,DeleteTag);
object_ = 0;
}
template<class T>
T& As(){
return *static_cast<T*>(object_);
}
private:
typedef void (*OperateFunc)(void*const&,void*&,Operation);
void* object_;
OperateFunc operate_;
};
int main(){
Any_Virtual a = 6;
std::cout << a.As<int>() << std::endl;
a = std::string("oh hi!");
std::cout << a.As<std::string>() << std::endl;
Any_Virtual av2 = a;
Any_VoidPtr a2 = 42;
std::cout << a2.As<int>() << std::endl;
Any_VoidPtr a3 = a.As<std::string>();
a2 = a3;
a2.As<std::string>() += " - again!";
std::cout << "a2: " << a2.As<std::string>() << std::endl;
std::cout << "a3: " << a3.As<std::string>() << std::endl;
a3 = a;
a3.As<Any_Virtual>().As<std::string>() += " - and yet again!!";
std::cout << "a: " << a.As<std::string>() << std::endl;
std::cout << "a3->a: " << a3.As<Any_Virtual>().As<std::string>() << std::endl;
std::cin.get();
}
All type erasure techniques in C++ are done with function pointers (for behaviour) and void* (for data). The "different" methods simply differ in the way they add semantic sugar. Virtual functions, e.g., are just semantic sugar for
struct Class {
struct vtable {
void (*dtor)(Class*);
void (*func)(Class*,double);
} * vtbl
};
iow: function pointers.
That said, there's one technique I particularly like, though: It's shared_ptr<void>, simply because it blows the minds off of people who don't know you can do this: You can store any data in a shared_ptr<void>, and still have the correct destructor called at the end, because the shared_ptr constructor is a function template, and will use the type of the actual object passed for creating the deleter by default:
{
const shared_ptr<void> sp( new A );
} // calls A::~A() here
Of course, this is just the usual void*/function-pointer type erasure, but very conveniently packaged.
Fundamentally, those are your options: virtual functions or function pointers.
How you store the data and associate it with the functions can vary. For example, you could store a pointer-to-base, and have the derived class contain the data and the virtual function implementations, or you could store the data elsewhere (e.g. in a separately allocated buffer), and just have the derived class provide the virtual function implementations, which take a void* that points to the data. If you store the data in a separate buffer, then you could use function pointers rather than virtual functions.
Storing a pointer-to-base works well in this context, even if the data is stored separately, if there are multiple operations that you wish to apply to your type-erased data. Otherwise you end up with multiple function pointers (one for each of the type-erased functions), or functions with a parameter that specifies the operation to perform.
I would also consider (similar to void*) the use of "raw storage": char buffer[N].
In C++0x you have std::aligned_storage<Size,Align>::type for this.
You can store anything you want in there, as long as it's small enough and you deal with the alignment properly.
Stroustrup, in The C++ programming language (4th edition) §25.3, states:
Variants of the technique of using a single runt-time representation for values of a number of types and relying on the (static) type system to ensure that they are used only according to their declared type has been called type erasure.
In particular, no use of virtual functions or function pointers is needed to perform type erasure if we use templates. The case, already mentioned in other answers, of the correct destructor call according to the type stored in a std::shared_ptr<void> is an example of that.
The example provided in Stroustrup's book is just as enjoyable.
Think about implementing template<class T> class Vector, a container along the lines of std::vector. When you will use your Vector with a lot of different pointers types, as it often happens, the compiler will supposedly generate different code for every pointer type.
This code bloat can be prevented by defining a specialization of Vector for void* pointers and then using this specialization as a common base implementation of Vector<T*> for all others types T:
template<typename T>
class Vector<T*> : private Vector<void*>{
// all the dirty work is done once in the base class only
public:
// ...
// static type system ensures that a reference of right type is returned
T*& operator[](size_t i) { return reinterpret_cast<T*&>(Vector<void*>::operator[](i)); }
};
As you can see, we have a strongly typed container but Vector<Animal*>, Vector<Dog*>, Vector<Cat*>, ..., will share the same (C++ and binary) code for the implementation, having their pointer type erased behind void*.
See this series of posts for a (fairly short) list of type erasure techniques and the discussion about the trade-offs:
Part I,
Part II,
Part III,
Part IV.
The one I haven't seen mentioned yet is Adobe.Poly, and Boost.Variant, which can be considered a type erasure to some extent.
As stated by Marc, one can use cast std::shared_ptr<void>.
For example store the type in a function pointer, cast it and store in a functor of only one type:
#include <iostream>
#include <memory>
#include <functional>
using voidFun = void(*)(std::shared_ptr<void>);
template<typename T>
void fun(std::shared_ptr<T> t)
{
std::cout << *t << std::endl;
}
int main()
{
std::function<void(std::shared_ptr<void>)> call;
call = reinterpret_cast<voidFun>(fun<std::string>);
call(std::make_shared<std::string>("Hi there!"));
call = reinterpret_cast<voidFun>(fun<int>);
call(std::make_shared<int>(33));
call = reinterpret_cast<voidFun>(fun<char>);
call(std::make_shared<int>(33));
// Output:,
// Hi there!
// 33
// !
}