Converting a lambda to a std::tr1::function - c++

Using visual studio 2008 with the tr1 service pack and Intel C++ Compiler 11.1.071 [IA-32], this is related to my other question
I'm attempting to write a functional map for c++ which would work somewhat like the ruby version
strings = [2,4].map { |e| e.to_s }
So i've defined the following function in the VlcFunctional namespace
template<typename Container, typename U>
vector<U> map(const Container& container, std::tr1::function<U(Container::value_type)> f)
{
vector<U> transformedValues(container.size());
int index = -1;
BOOST_FOREACH(const auto& element, container)
{
transformedValues.at(++index) = f(element);
}
return transformedValues;
}
and you can call this like so (Note that the function template arguments are defined explicitly):
vector<int> test;
test.push_back(2); test.push_back(4);
vector<string> mappedData2 = VlcFunctional::map<vector<int>,string>(test, [](int i) -> string
{
return ToString(i);
});
Or like so (Note that the function template arguments aren't defined explicitly)
std::tr1::function f = [](int i) -> string { return ToString(i); };
vector<string> mappedData2 = VlcFunctional::map<vector<int>,string>(test, f);
But crucially, NOT LIKE THIS
vector<string> mappedData2 = VlcFunctional::map(test, [](int i) -> string { return ToString(i); });
Without the explicit definition of hte template arguments, it doesn't know which template to use and falls over with a compile error
..\tests\VlcFunctional_test.cpp(106): error: no instance of function template "VlcFunctional::map" matches the argument list, argument types are: (std::vector<int, std::allocator<int>>, __lambda3)
Having to define the template arguments makes it a much more bulky syntax and I'm aiming for minimal cruft at the call site - any ideas on why it doesn't know how do the conversion? Is this a compiler issue or does the language not allow for this type of template argument inference?

The problem is that a lambda is not a std::function even if it can be converted. When deducing type arguments, the compiler is not allowed to perform conversions on the actual provided arguments. I would look for a way to have the compiler detect the type U and let the second argument free for the compiler to deduce:
template <typename Container, typename Functor>
std::vector< XXX > VlcFunctional::map( Container &, Functor )...
Now the issue is what to write in XXX. I don't have the same compiler that you do, and all C++0x features are still a little tricky. I would first try to use decltype:
template <typename Container, typename Functor>
auto VlcFunctional::map( Container & c, Functor f ) -> std::vector< decltype(f(*c.begin())) > ...
Or maybe type traits if the compiler does not support decltype yet.
Also note that the code you are writting is quite unidiomatic in C++. Usually when manipulating containers the functions are implemented in terms of iterators, and your whole map is basically the old std::transform:
std::vector<int> v = { 1, 2, 3, 4, 5 };
std::vector<std::string> s;
std::transform( v.begin(), v.end(), std::back_inserter(s), [](int x) { return ToString(x); } );
Where std::transform is the C++ version of your map function. While the syntax is more cumbersome, the advantage is that you can apply it to any container, and produce the output to any other container, so the transformed container is not fixed to std::vector.
EDIT:
A third approach, probably easier to implement with your current compiler support is manually providing just the return type of the lambda as template argument, and letting the compiler deduce the rest:
template <typename LambdaReturn, typename Container, typename Functor>
std::vector<LambdaReturn> map( Container const & c, Functor f )
{
std::vector<LambdaReturn> ret;
std::transform( c.begin(), c.end(), std::back_inserter(ret), f );
return ret;
}
int main() {
std::vector<int> v{ 1, 2, 3, 4, 5 };
auto strs = map<std::string>( v, [](int x) {return ToString(x); });
}
Even if you want to add syntactic sugar to your map function, there is no need to manually implement it when you can use existing functionality.

Related

Can type arguments be made deduceable for function templates using std container?

I found this implementation of a few common features of functional programming, e.g. map / reduce:
(I'm aware stuff like that is aparently coming or partially present in new C++ versions)
github link
A part of the code:
template <typename T, typename U>
U foldLeft(const std::vector<T>& data,
const U& initialValue,
const std::function<U(U,T)>& foldFn) {
typedef typename std::vector<T>::const_iterator Iterator;
U accumulator = initialValue;
Iterator end = data.cend();
for (Iterator it = data.cbegin(); it != end; ++it) {
accumulator = foldFn(accumulator, *it);
}
return accumulator;
}
template <typename T, typename U>
std::vector<U> map(const std::vector<T>& data, const std::function<U(T)> mapper) {
std::vector<U> result;
foldLeft<T, std::vector<U>&>(data, result, [mapper] (std::vector<U>& res, T value) -> std::vector<U>& {
res.push_back(mapper(value));
return res;
});
return result;
}
Usage example:
std::vector<int> biggerInts = map<int,int>(test, [] (int num) { return num + 10; });
The type arguments T,U have to be fully qualified for this to compile, as shown in the example, with e.g. map< int,int >( ... ).
This implementation is for C++11, as mentioned on the linked-to page.
Is it possible with newer C++ versions (or even 11) now to make the use of this less verbose, i.e. making the types U,T deduce automatically?
I have googled for that and only found that there is apparently some improvement for class template, as opposed to function template, argument deduction in C++17.
But since I only ever used templates in a rather basic manner, I was wondering whether there is something in existence that I'm not aware of which could improve this implementation verboseness-wise.
You can rewrite map signature to be:
template <typename T, typename M, typename U = decltype(std::declval<M>()(T{}))>
std::vector<U> map(const std::vector<T>& data, const M mapper)
then T will be deduced as value_type of vector's items.
M is any callable object.
U is deduced as return type of M() functor when called for T{}.
Below
std::vector<int> biggerInts = map(test, [] (int num) { return num + 10; });
^^^^ empty template arguments list
works fine.
Live demo
More general templates make template argument deduction easier.
One principle: it is often a mistake to use a std::function as a templated function's parameter. std::function is a type erasure, for use when something needs to store some unknown invokable thing as a specific type. But templates already have the ability to handle any arbitrary invokable type. So if we just use a generic typename FuncT template parameter, it can be deduced for a raw pointer-to-function, a lambda, or another class with operator() directly.
We might as well also get more general and accept any input container instead of just vector, then determine T from it, if it's even directly needed.
So for C++11 I would rewrite these:
// C++20 is adding std::remove_cvref, but it's trivial to implement:
template <typename T>
using remove_cvref_t =
typename std::remove_cv<typename std::remove_reference<T>::type>::type;
template <typename Container, typename U, typename FuncT>
remove_cvref_t<U> foldLeft(
const Container& data,
U&& initialValue,
const FuncT& foldFn) {
remove_cvref_t<U> accumulator = std::forward<U>(initialValue);
for (const auto& elem : data) {
accumulator = foldFn(std::move(accumulator), elem);
}
return accumulator;
}
template <typename Container, typename FuncT>
auto map(const Container& data, const FuncT& mapper)
-> std::vector<remove_cvref_t<decltype(mapper(*std::begin(data)))>>
{
using T = remove_cvref_t<decltype(*std::begin(data))>;
using ResultT = std::vector<remove_cvref_t<decltype(mapper(std::declval<const T&>()))>>;
ResultT result;
foldLeft(data, std::ref(result), [&mapper] (ResultT &res, const T& value) -> ResultT& {
res.push_back(mapper(value));
return res;
});
return result;
}
See the working program on coliru.
There was one unfortunate thing about the old map: it potentially copied the result vector at every iteration. The = in accumulator = foldFn(accumulator, *it); is a self-assignment, which might do nothing, or might allocate new memory, copy contents, then free the old memory and update the container. So instead I've changed the U for foldLeft in this case to a std::reference_wrapper. The = in that case will still "rebind" the wrapper to the same object, but that will at least be quick.
In C++14 and later, you could do away with finding T within map by using a generic lambda: [&mapper] (std::vector<U>& res, const auto& value) ...

Range view to std::vector

In the proposed C++20 (The One) Ranges TS, what is the proposed method for converting the view into a std::vector?
The following code does not compile:
int
main() {
std::vector<float> values = {1.0, 2.0, 3.0, 4.0, 5.2, 6.0, 7.0, 8.0, 9.0};
//fmt::print("{}\n", std::experimental::ranges::views::filter(values, [] (float v) { return v < 5.f; }));
std::vector<float> foo = vw::filter(values, [] (float v) { return v < 5.f; });
fmt::print("{}\n", foo);
}
with the error
../src/view.cpp:19:40: error: conversion from ‘std::experimental::ranges::v1::filter_view<std::experimental::ranges::v1::ref_view<std::vector<float> >, main()::<lambda(float)> >’ to non-scalar type ‘std::vector<float>’ requested
std::vector<float> foo = vw::filter(values, [] (float v) { return v < 5.f; });
(the commented line will also not compile due to some CV constraints).
So how do I do anything with a view except for using a range-based for loop?
Also some bonus questions:
Is the cmcstl2 implementation I used even following the proposal? The ranges-v3 seems not to be.
Is there any documentation on the Ranges TS? The proposal PDF I found is pretty much an awfully formatted code dump in diff style. In fact directly reading the cmcstl2 sources was way easier to read for me. The cppreference seems to be lacking as well...
The C++20 method to convert a view to a std::vector (or indeed any other container) is to pass the range's begin and end members to the vector constructor that accepts 2 iterators (and an optional allocator).
I was also looking for an answer to this question. What I really wanted is an overload of the constructor of std::vector accepting a range. Approximately:
template <std::ranges::input_range R>
vector(R&& r) : vector(r.begin(), r.end()) {
}
but that isn't in C++20.
First, I implemented this:
namespace rng = std::ranges;
template <rng::range R>
constexpr auto to_vector(R&& r) {
using elem_t = std::decay_t<rng::range_value_t<R>>;
return std::vector<elem_t>{r.begin(), r.end()};
}
which works, but isn't very "rangy": https://godbolt.org/z/f2xAcd
I then did it a bit better:
namespace detail {
// Type acts as a tag to find the correct operator| overload
template <typename C>
struct to_helper {
};
// This actually does the work
template <typename Container, rng::range R>
requires std::convertible_to<rng::range_value_t<R>, typename Container::value_type>
Container operator|(R&& r, to_helper<Container>) {
return Container{r.begin(), r.end()};
}
}
// Couldn't find an concept for container, however a
// container is a range, but not a view.
template <rng::range Container>
requires (!rng::view<Container>)
auto to() {
return detail::to_helper<Container>{};
}
https://godbolt.org/z/G8cEGqeq6
No doubt one can do better for sized_ranges and containers like std::vector that have a reserve member function.
There is a proposal to add a to function to C++23 (https://wg21.link/p1206) which will do a better job than this, I'm sure.

How to provide the function signature for a function taking iterators of stl containers?

I want to write a function my_func that can be called as so, but does not care that v is a std::vector, it could be any STL container. A bit like std::for_each:
std::vector<std::string> v = {...};
my_func(v.begin(), v.end());
But I cannot figure out the function signature.
void my_func(??? i1, ??? i2)
{
std::for_each(i1, i2, ...); // dumb example implementation
}
I am not great at template programming so even looking at the function declaration for std::for_each is not helping me.
Is there an easy implementation or is this fundamentally going to get messy with template vars?
It depends on how generic you want the function to be. If the iterator types have to match, then
template <typename T>
void my_func(T i1, T i2)
{
std::for_each(i1,i2,...); //dumb example implementation
}
is all you need. If you want them to be able to be different, then you just need another template parameter like
template <typename T, typename U>
void my_func(T i1, U i2)
{
std::for_each(i1,i2,...); //dumb example implementation
}
Finally, if you don't like dealing with templates you can use a lambda instead and let the compiler take care of this for you. That would give you
auto my_func = [](auto i1, auto i2)
{
std::for_each(i1,i2,...); //dumb example implementation
};
You could write a templated function
template<typename Iterator>
void my_func(Iterator startIter, const Iterator endIter)
{
std::for_each(startIter, endIter, /* lambda */);
}
In case of wondering, how to pass the third parameter of the std::for_each, you could provide one more template parameter
const auto defaultCallable = [](auto element){ }; // does nothing
template<typename Iterator, typename Callable = decltype(defaultCallable)>
void my_func(Iterator startIter, const Iterator endIter, Callable func = {})
{
std::for_each(startIter, endIter, func);
}
The syntax is not too obscure! The following way uses the range for at the point of use:
template <template<typename...> class Iterable, typename T>
void foo(
const Iterable<T>& y // the container
){
for (auto&& e : y){
// e is the 'thingy' in the container.
}
}
and you can pass any iterable container of arbitrary type to foo.

Mapping a vector of one type to another using lambda

I have a bit of code that looks like
B Convert(const A& a) {
B b;
// implementation omitted.
return b;
}
vector<B> Convert(const vector<A>& to_convert) {
vector<B> ret;
for (const A& a : to_convert) {
ret.push_back(Convert(a));
}
retun ret;
}
I was trying to rewrite this using lambdas but the code does not look more concise or more clear at all:
vector<B> Convert(const vector<A>& to_convert) {
vector<B> ret;
std::transform(to_convert.begin(),
to_convert.end(),
std::back_inserter(ret),
[](const A& a) -> B { return Convert(a); });
retun ret;
}
What I would really like to do is something like:
vector<B> Convert(const vector<A>& to_convert) {
return map(to_convert, [](const A& a) -> B { return Convert(a); });
}
Where map is a functional style map function that could be implemented as:
template<typename T1, typename T2>
vector<T2> map(const vector<T1>& to_convert,
std::function<T2(const T1&)> converter) {
vector<T2> ret;
std::transform(to_convert.begin(),
to_convert.end(),
std::back_inserter(ret),
converter);
retun ret;
}
Obviously the above is limited because it only works with vector, ideally one would want similar functions for all container types. At the end of the day, the above is still not better than my original code.
Why isn't there something like this (that I could find) in the stl?
You said it yourself, this map is not generic enough. std::transform on the other hand is, at the cost of more verbose interface. Another reason is that map, unlike std::transform forces new allocation, which is not always desirable.
template<class F, class R, class Out>
struct convert_t {
F f;
R r;
// TODO: upgrade std::begin calls with Koenig lookup
template<class D>
operator D()&&{
D d;
std::transform(
std::begin(std::forward<R>(r)),
std::end(std::forward<R>(r)),
std::back_inserter(d),
std::forward<F>(f)
);
return d;
}
template<template<class...>class Z, class Result=Z<
typename std::decay<Out>::type
>>
Result to()&&{return std::move(*this);}
};
template<class F, class R,
class dF=typename std::decay<F>::type,
class dR=typename std::decay<R>::type,
class R_T=decltype(*std::begin(std::declval<dR>())),
class Out=typename std::result_of<dF&(R_T)>::type
>
convert_t<dF,dR,Out>
convert( F&& f, R&& r ) { return {std::forward<F>(f), std::forward<R>(r)}; }
which gives us this:
std::vector<int> vec{1,2,3};
auto r = convert(Convert, vec).to<std::vector>();
for (auto&& x:r)
std::cout << x << '\n';
std::vector<double> r2 = convert(Convert, vec);
for (auto&& x:r)
std::cout << x << '\n';
live example.
This only handles sequence container output, as std::back_inserter would have to be swapped for std::inserter or somesuch for associative containers.
Also, some associative containers (like map) don't like being passed a pair -- they want Key,Value. Expressing that generically is tricky.
The standard library takes care to separate containers, and their traversal. By having std algorithms take containers directly, you'd lose the possibility of using different iterator types for different methods of traversal.
For example, Boost.Iterator uses specifically this clean separation to provide a neat collection of every traversal method you could dream of.
Note also that not all iterators traverse actual containers : std::back_inserter (which you should use instead of ret.begin() if you don't want to fall into unallocated space) actually constructs the container as it goes, and std::ostream_iterator is completely unrelated to any container as it pushes what's assigned to it to a stream.
Of course, nothing stops you from making a thin wrapper for the classic begin/end traversal :
template <
template <class...> class Container,
class Transform,
class ContainerT,
class... ContainerParams
>
auto map(Container<ContainerT, ContainerParams...> const &container, Transform &&transform) {
using DestT = std::result_of_t<Transform(ContainerT const&)>;
Container<DestT, ContainerParams...> res;
using std::begin;
using std::end;
std::transform(
begin(container),
end(container),
std::inserter(res, end(res)),
std::forward<Transform>(transform)
);
return res;
}
(live on Coliru)

template argument type deduction from std::function return type with lambda

First of, I'm using C++11 (and my topic sucks).
What I'm trying to do is write a generic template function that implements something usually called sort_by in other programming languages. It involves calculating an arbitrary criterion for each member of a range exactly once and then sorting that range according to those criteria. Such a criterion doesn't have to be a POD, all it has to be is less-than-comparable. For things for which std::less doesn't work the caller should be able to provide her own comparison functor.
I've successfully written said function which uses the following signature:
template< typename Tcriterion
, typename Titer
, typename Tcompare = std::less<Tcriterion>
>
void
sort_by(Titer first, Titer last,
std::function<Tcriterion(typename std::iterator_traits<Titer>::value_type const &)> criterion_maker,
Tcompare comparator = Tcompare()) {
}
It can be used e.g. like this:
struct S { int a; std::string b; double c; };
std::vector<S> s_vec{
{ 42, "hello", 0.5 },
{ 42, "moo!", 1.2 },
{ 23, "fubar", 0.2 },
};
sort_by1< std::pair<int, double> >(
s_vec.begin(), s_vec.end(),
[](S const &one_s) { return std::make_pair(one_s.a, one_s.c); }
);
What I don't like about this approach is that I have to provide the Tcriterion argument myself because the compiler cannot deduce that type from the lambda expression. Therefore this does not work:
sort_by1(s_vec.begin(), s_vec.end(), [](S const &one_s) { return std::make_pair(one_s.a, one_s.c); });
clang 3.1 and gcc 4.7.1 both bark on this (gcc 4.7.1 even barks on the code above, so I guess I'm really doing something wrong here).
However, if I assign the lambda to a std::function first then at least clang 3.1 can deduce the argument, meaning this works:
typedef std::pair<int, double> criterion_type;
std::function<criterion_type(S const &)> criterion_maker = [](S const &one_s) {
return std::make_pair(one_s.a, one_s.c);
};
sort_by1(s_vec.begin(), s_vec.end(), criterion_maker);
So my questions are: How do I have to change my function signature so that I don't need to specify that one argument? And (probably related) how would I fix my example to have it working with gcc?
Don't use std::function in tandem with template argument deduction. In fact, there's very likely no reason to use std::function in a function or function template argument list. More often than not, you should not use std::function; it is a very specialized tool that is very good at solving one particular problem. The rest of the time, you can dispense with it altogether.
In your case you don't need template argument deduction if you use a polymorphic functor to order things:
struct less {
template<typename T, typename U>
auto operator()(T&& t, U&& u) const
-> decltype( std::declval<T>() < std::declval<U>() )
{ return std::forward<T>(t) < std::forward<U>(u); }
// operator< is not appropriate for pointers however
// the Standard defines a 'composite pointer type' that
// would be very helpful here, left as an exercise to implement
template<typename T, typename U>
bool operator()(T* t, U* u) const
{ return std::less<typename std::common_type<T*, U*>::type> {}(t, u); }
};
You can then declare:
template<typename Iter, typename Criterion, typename Comparator = less>
void sort_by(Iter first, Iter last, Criterion crit, Comparator comp = less {});
and comp(*ita, *itb) will do the right thing, as well as comp(crit(*ita), crit(*itb)) or anything else as long as it makes sense.
How about something like this:
template< typename Titer
, typename Tmaker
, typename Tcompare
>
void
sort_by(Titer first, Titer last,
Tmaker criterion_maker,
Tcompare comparator)
{
typedef decltype(criterion_maker(*first)) Tcriterion;
/*
Now that you know the actual type of your criterion,
you can do the real work here
*/
}
The problem is that you can obviously not use a default for the comparator with this, but you can easily overcome that by providing an overload that doesn't take a comparator and fills in std::less internally.
To do it like you originally suggested, the compiler would have to be able to "invert" the template instantiation process. I.e. for a given std::function<> instantiation, what parameter do I have to supply as the result to get it. This "looks" easy, but it is not!
You can use also something like this.
template< typename Titer
, typename Tmaker
, typename TCriterion = typename
std::result_of
<
Tmaker
(
decltype(*std::declval<Titer>())
)
>::type
, typename Tcompare = std::less<TCriterion>
>
void
sort_by(Titer first, Titer last,
Tmaker criterion_maker, Tcompare comparator = Tcompare())
{
}
http://liveworkspace.org/code/0aacc8906ab4102ac62ef0e45a37707d