Are there any parts of the standard library that would support the following use case:
You have N collections, with each collection potentially being a type of collection (C1, C2 ..., Cn) all of which support begin(), end() and iteration, ie (vector, deque, etc).
Each of these collections can contain a objects of different types ie, collections are C1, C2, C3, and are of different sizes.
In addition, all of these types can be ordered via a timestamp, but the way that each of these items store a timestamp is different. For example, type A has member A.timestamp, B has a member B.TimeStamp, C has a function C.GetTimestamp().
Each of the collections are already ordered by this function.
What I want to do is iterate over all items in all collections, in order, using the order functions, and call another function, i.e. a visit function, std::function<void(A &)> for collections of A, std::function<void(B&)> for items of type B, etc.
I then want to, make calls for each item in timestamp order. For example:
class A
{
public:
time_t timeStamp;
int length;
};
class B
{
public:
B(time_t _tm, std::string _name):timestamp(_tm), name(_name){}
time_t GetTimestamp() { return timeStamp; }
std::string GetName() { return name; }
private:
time_t timeStamp;
std::string name;
}
std::vector<A> listA {1, 4}, {5, 7}, {8,9});
std::deque<B> listB { B(0,"bob"), B(3, "Frank") };
// iterate over listA and listB in time sequential order
// for items in A call [](const A&a) { std::cout << a.length << std::endl; }
// for items in B call [](const B&b) { std::cout << b.name << std::endl; }
// Output would be:
// bob
// 4
// Frank
// 7
// 9
My thought on implementation would be to, define a base class which is templated by the type that the ordering function returns:
template<classname O>
class VisitedCollection
{
public:
virtual bool end() = 0; // returns if we are at the end of collection
virtual O next_order_measure() = 0; returns an instance of a class that can be used for ordering
virtual void visit();
};
class VisitedCollectionA: VisitedCollection<time_t>
{
public:
VisitedCollectionA(std::vector<A> &&a): items(std::move(a))
{
next_item = items.begin();
}
virtual bool end() override { return next_item == items.end(); }
virtual time_t next_order_meaure() override { return nextItem->timeStamp;}
virtual void visit() override { std::cout << nextItem->length << std::endl;}
private:
std::vector<A> &items;
std::vector<A>::iterator next_item;
}
... similar for class B
... could also add a class C, D, etc
Now I can create a collection of VisitedCollection<time_t>, and add VisitedCollectionA, and VisitedCollectionB. This collection of collections would:
Start by looking at the return value of the ordering function for the first item in each collection. Which ever one has the smallest value, call it's visitor function. Then find the collection which whose next item has the lowest ordering value. On ties on the ordering function, iterate collection which comes first in the "collection" of collections. Once a collection hits 'end', it's removed from the iteration.
I'm considering rolling my own, but wanted to know if there was already something like this in the standard library. I could even make visit() a lambda, which would allow the VisitedCollectionA types be templates, which take an ordering type and a collection type, that would allow the creation of the main visitor to be initialized with something like
{
VisitedCollection<time_t, std::vector<A>>
(
vecA,
[](){ return next_item->timeStamp; },
[](const A&a) { std::cout << a << std::endl; }
),
VisitedCollection<time_t, std::deque<B>>
(
deqB,
[](){ return next_item->GetTimestamp(); },
[](const B&b) { std::cout << a << std::endl; }
)
}
This feels a bit like a mixture of variant and ranges
Does something like this exist?
There's no straightforward solution in the C++ std (or any other std, I guess?) but some support for a self-made solution can be found there anyway. Even though suboptimal, it should be easy to understand and rewrite if needed.
First of all, you need const getters (omitted here) and a unified interface for getting timestamps out of the types iterated:
time_t timestamp(A const& a) { return a.timeStamp; }
time_t timestamp(B const& b) { return b.GetTimestamp(); }
template<typename... Ts> time_t timestamp(std::variant<Ts...> const& v) {
return visit([](auto&& e) { return timestamp(e); }, v);
} // see below
The basic idea is putting references to all elements into a single container (as variants), sorting and then visiting them:
void visit_sorted_timestamps(auto visitor, auto&&... ranges) {
std::vector<std::variant<
std::reference_wrapper<std::ranges::range_value_t<decltype(ranges)>>...
>> mixed;
mixed.reserve((... + size(ranges)));
(..., mixed.insert(end(mixed), begin(ranges), end(ranges)));
std::sort(begin(mixed), end(mixed), [](auto&& v1, auto&& v2) {
return timestamp(v1) < timestamp(v2);
});
for (auto&& v: mixed) visit(visitor, v);
}
Usage example:
int main() {
visit_sorted_timestamps(Overload{
[](A const& a) { std::cout << a.length << '\n'; },
[](B const& b) { std::cout << b.GetName() << '\n'; }
}, listA, listB);
}
If you don't have the lambda "overloading" struct yet, here it goes:
template<typename... Fs> struct Overload: Fs... { using Fs::operator()...; };
template<typename... Fs> Overload(Fs...) -> Overload<Fs...>;
I want to implement a math parser with user-defined function.
There are several problems to be solved.
For example, int eg(int a,int b){return a+b;} is the function I want to add to the parser.
First: How to store all the functions into a container?
std::map<std::string,boost::any> func_map may be a choose (by func_map["eg"]=eg". However, It's very hard to call the function in this kind of map, for I have to use any_cast<T> to get the real function from the wrapper of boost::any.
Second: How to handle the overloaded function?
It's true that I can distinguish the overloaded functions by the method of typeid, but it's far from a real implementation.
Parsering expressions is not a difficult skill and the hardest part is described above.
muparserx provides an interesting solution for this problem, but I'm finding another method.
I'm not familiar with lambda expressions but may be it's an acceptable way.
Update:
I need something like this:
int eg(int a,int b){ return a+b;}
int eg(int a,int b, string c){return a+b+c.length();}
double eh(string a){return length.size()/double(2);}
int main(){
multimap<string,PACKED_FUNC> func_map;
func_map.insert(make_pair("eg",pack_function<int,int>(eg));
func_map.insert(make_pair("eg",pack_function<int,int,string>(eg));
func_map.insert(make_pair("eh",pack_function<string>(eh));
auto p1=make_tuple(1,2);
int result1=apply("eg",PACK_TUPLE(p1));//result1=3
auto p2=tuple_cat(p1,make_tuple("test"));
int result2=apply("eg",PACK_TUPLE(p2));//result2=7
auto p3=make_tuple("testagain");
double result3=apply("eh",PACK_TUPLE(p3));//result3=4.5
return 0;
}
How to store all the functions into a container?
To store then inside some container, they must be of the same type. The std::function wrapper is a good choice, since this allows you to use even stateful function objects. Since you probably don't want all functions to take the same number of arguments, you need to "extract" the arity of the functions from the static host type system. An easy solution is to use functions that accept a std::vector:
// Arguments type to the function "interface"
using Arguments = std::vector<int> const &;
// the interface
using Function = std::function<int (Arguments)>;
But you don't want your users to write functions that have to unpack their arguments manually, so it's sensible to automate that.
// Base case of packing a function.
// If it's taking a vector and no more
// arguments, then there's nothing left to
// pack.
template<
std::size_t N,
typename Fn>
Function pack(Fn && fn) {
return
[fn = std::forward<decltype(fn)>(fn)]
(Arguments arguments)
{
if (N != arguments.size()) {
throw
std::string{"wrong number of arguments, expected "} +
std::to_string(N) +
std::string{" but got "} +
std::to_string(arguments.size());
}
return fn(arguments);
};
}
The above code handles the easy case: A function that already accepts a vector. For all other functions they need to be wrapped and packed into a newly created function. Doing this one argument a time makes this relatively easy:
// pack a function to a function that takes
// it's arguments from a vector, one argument after
// the other.
template<
std::size_t N,
typename Arg,
typename... Args,
typename Fn>
Function pack(Fn && fn) {
return pack<N+1, Args...>(
[fn = std::forward<decltype(fn)>(fn)]
(Arguments arguments, Args const &... args)
{
return fn(
arguments,
arguments[N],
args...);
});
}
The above only works with (special) functions that already take a vector. For normal functions we need an function to turn them into such special functions:
// transform a function into one that takes its
// arguments from a vector
template<
typename... Args,
typename Fn>
Function pack_function(Fn && fn) {
return pack<0, Args...>(
[fn = std::forward<decltype(fn)>(fn)]
(Arguments arguments, Args const &... args)
{
return fn(args...);
});
}
Using this, you can pack any function up to be the same type:
Function fn =
pack_function<int, int>([] (auto lhs, auto rhs) {return lhs - rhs;});
You can then have them in a map, and call them using some vector, parsed from some input:
int main(int, char**) {
std::map<std::string, Function> operations;
operations ["add"] = pack_function<int, int>(add);
operations ["sub"] = pack_function<int, int>(
[](auto lhs, auto rhs) { return lhs - rhs;});
operations ["sum"] = [] (auto summands) {
int result = 0;
for (auto e : summands) {
result += e;
}
return result;
};
std::string line;
while (std::getline(std::cin, line)) {
std::istringstream command{line};
std::string operation;
command >> operation;
std::vector<int> arguments {
std::istream_iterator<int>{command},
std::istream_iterator<int>{} };
auto function = operations.find(operation);
if (function != operations.end ()) {
std::cout << line << " = ";
try {
std::cout << function->second(arguments);
} catch (std::string const & error) {
std::cout << error;
}
std::cout << std::endl;
}
}
return 0;
}
A live demo of the above code is here.
How to handle the overloaded function? It's true that I can distinguish the overloaded functions by the method of typeid, but it's far from a real implementation.
As you see, you don't need to, if you pack the relevant information into the function. Btw, typeid shouldn't be used for anything but diagnostics, as it's not guaranteed to return different strings with different types.
Now, finally, to handle functions that don't only take a different number of arguments, but also differ in the types of their arguments, you need to unify those types into a single one. That's normally called a "sum type", and very easy to achieve in languages like Haskell:
data Sum = IVal Int | SVal String
-- A value of type Sum is either an Int or a String
In C++ this is a lot harder to achieve, but a simple sketch could look such:
struct Base {
virtual ~Base() = 0;
};
inline Base::~Base() {}
template<typename Target>
struct Storage : public Base {
Target value;
};
struct Any {
std::unique_ptr<Base const> value;
template<typename Target>
Target const & as(void) const {
return
dynamic_cast<Storage<Target> const &>(*value).value;
}
};
template<typename Target>
auto make_any(Target && value) {
return Any{std::make_unique<Storage<Target>>(value)};
}
But this is only a rough sketch, since there's boost::any which should work perfectly for this case. Note that the above and also boost::any are not quite like a real sum type (they can be any type, not just one from a given selection), but that shouldn't matter in your case.
I hope this gets you started :)
Since you had problems adding multi type support I expanded a bit on the above sketch and got it working. The code is far from being production ready, though: I'm throwing strings around and don't talk to me about perfect forwarding :D
The main change to the above Any class is the use of a shared pointer instead of a unique one. This is only because it saved me from writing copy and move constructors and assignment operators.
Apart from that I added a member function to be able to print an Any value to a stream and added the respective operator:
struct Base {
virtual ~Base() = 0;
virtual void print_to(std::ostream &) const = 0;
};
inline Base::~Base() {}
template<typename Target>
struct Storage : public Base {
Target value;
Storage (Target t) // screw perfect forwarding :D
: value(std::forward<Target>(t)) {}
void print_to(std::ostream & stream) const {
stream << value;
}
};
struct Any {
std::shared_ptr<Base const> value;
template<typename Target>
Target const & as(void) const {
return
dynamic_cast<Storage<Target> const &>(*value).value;
}
template<typename T>
operator T const &(void) const {
return as<T>();
}
friend std::ostream & operator<<(std::ostream& stream, Any const & thing) {
thing.value->print_to(stream);
return stream;
}
};
template<typename Target>
Any make_any(Target && value) {
return Any{std::make_shared<Storage<typename std::remove_reference<Target>::type> const>(std::forward<Target>(value))};
}
I also wrote a small "parsing" function which shows how to turn a raw literal into an Any value containing (in this case) either an integer, a double or a string value:
Any parse_literal(std::string const & literal) {
try {
std::size_t next;
auto integer = std::stoi(literal, & next);
if (next == literal.size()) {
return make_any (integer);
}
auto floating = std::stod(literal, & next);
if (next == literal. size()) {
return make_any (floating);
}
} catch (std::invalid_argument const &) {}
// not very sensible, string literals should better be
// enclosed in some form of quotes, but that's the
// job of the parser
return make_any<std:: string> (std::string{literal});
}
std::istream & operator>>(std::istream & stream, Any & thing) {
std::string raw;
if (stream >> raw) {
thing = parse_literal (raw);
}
return stream;
}
By also providing operator>> it's possible to keep using istream_iterators for input.
The packing functions (or more precisely the functions returned by them) are also modified: When passing an element from the arguments vector to the next function, an conversion from Any to the respective argument type is performed. This may also fail, in which case a std::bad_cast is caught and an informative message rethrown. The innermost function (the lambda created inside pack_function) wraps its result into an make_any call.
add 5 4 = 9
sub 3 2 = 1
add 1 2 3 = wrong number of arguments, expected 2 but got 3
add 4 = wrong number of arguments, expected 2 but got 1
sum 1 2 3 4 = 10
sum = 0
sub 3 1.5 = argument 1 has wrong type
addf 3 3.4 = argument 0 has wrong type
addf 3.0 3.4 = 6.4
hi Pete = Hello Pete, how are you?
An example similar to the previous one can be found here. I need to add that this Any type doesn't support implicit type conversions, so when you have an Any with an int stored, you cannot pass that to an function expecting a double. Though this can be implemented (by manually providing a lot of conversion rules).
But I also saw your update, so I took that code and applied the necessary modifications to run with my presented solution:
Any apply (multimap<string, Function> const & map, string const & name, Arguments arguments) {
auto range = map.equal_range(name);
for (auto function = range.first;
function != range.second;
++function) {
try {
return (function->second)(arguments);
} catch (string const &) {}
}
throw string {" no such function "};
}
int eg(int a,int b){ return a+b;}
int eg(int a,int b, string c){return a+b+c.length();}
double eh(string a){return a.size()/double(2);}
int main(){
multimap<string, Function> func_map;
func_map.insert(make_pair(
"eg",pack_function<int,int>(
static_cast<int(*)(int, int)>(&eg))));
func_map.insert(make_pair(
"eg",pack_function<int,int,string>(
static_cast<int (*)(int, int, string)>(&eg))));
func_map.insert(make_pair(
"eh",pack_function<string>(eh)));
// auto p1=make_tuple(1,2);
// if you want tuples, just write a
// function to covert them to a vector
// of Any.
Arguments p1 =
{make_any (1), make_any (2)};
int result1 =
apply(func_map, "eg", p1).as<int>();
vector<Any> p2{p1};
p2.push_back(make_any<string> ("test"));
int result2 =
apply(func_map, "eg", p2).as<int>();
Arguments p3 = {make_any<string>("testagain")};
double result3 =
apply(func_map, "eh", p3).as<double>();
cout << result1 << endl;
cout << result2 << endl;
cout << result3 << endl;
return 0;
}
It doesn't use tuples, but you could write a (template recursive) function to access each element of a tuple, wrap it into an Any and pack it inside a vector.
Also I'm not sure why the implicit conversion from Any doesn't work when initialising the result variables.
Hm, converting it to use boost::any shouldn't be that difficult. First, the make_any would just use boost::any's constructor:
template<typename T>
boost::any make_any(T&& value) {
return boost::any{std::forward<T>(value)};
}
In the pack function, the only thing that I'd guess needs to be changed is the "extraction" of the correct type from the current element in the arguments vector. Currently this is as simple as arguments.at(N), relying on implicit conversion to the required type. Since boost::any doesn't support implicit conversion, you need to use boost::any_cast to get to the underlying value:
template<
std::size_t N,
typename Arg,
typename... Args,
typename Fn>
Function pack(Fn && fn) {
return pack<N+1, Args...>(
[fn = std::forward<decltype(fn)>(fn)]
(Arguments arguments, Args const &... args)
{
try {
return fn(
arguments,
boost::any_cast<Arg>(arguments.at(N)),
args...);
} catch (boost::bad_any_cast const &) { // throws different type of exception
throw std::string{"argument "} + std::to_string (N) +
std::string{" has wrong type "};
}
});
}
And of course, if you use it like in the example you provided you also need to use boost::any_cast to access the result value.
This should (in theory) do it, eventually you need to add some std::remove_reference "magic" to the template parameter of the boost::any_cast calls, but I doubt that this is neccessary.
(typename std::remove_reference<T>::type instead of just T)
Though I currently cannot test any of the above.
I am trying to create a function in c++, I am wondering if I can create it such that it is able to return different types of vectors. e.g based on different case it returns vector string, int, double or ...anything.
Is it possible in c++? (I do not want to use overload function with different arg(S) and different returns)
I am very new to C++ and my question may seem to be stupid.
here is a piece of my code:
//zero here means intersection
std::vector<??????> findZeros(const mesh::Region& s, char *model) const
{
//Point
if( model == "point" )
{
std::vector<Vertex> zeros;
for(Region::pointIterator it = s.beginPoint(); itv != s.endPoint(); ++itv )
{
if( abs(Val(*it)) < 1.e-12 )
zeros.push_back(*it);
}
std::vector<point> zerosP(zeros.begin(), zeros.end());
return zerosP;
}
//line
else if (EntityS == "line")
{
std::vector<line> zerosE;
std::vector<Point&> PointE;
for(Region::lineIterator ite = s.beginLine(); ite != s.endLine(); ++ite )
{
Line ed = *ite;
Point P0 = ed.point(0);
Point P1 = e.point(1);
if( ......... ) zerosE.push_back(ed);
else if ( ....... )
{
PontE.push_back( P0, P1);
zerosE.push_back(ed);
}
}
//here I want to return "point" or "line with its points" or in upper level our surface.
//I want to do all in one function!
}
Templates
Try this:
template <typename T>
std::vector<T> func( /* arguments */ )
{
std::vector<T> v;
// ... do some stuff to the vector ...
return v;
}
You can call this function with different type in this way:
std::vector<int> func<int>( args );
std::vector<double> func<double>( args );
Alternatives
This is one way, if you know the types at compile-time. If you don't know the type at compile-time but at run-time only, then you have different choices:
Use unions. I can only recommend this, if you have very simple C-struct-like types which are called PODs (plain old data) in the C++ standard.
Use some type of variant. For example there is boost::variant from the Boost libraries or QVariant from the Qt library. They are a safe kind of unions on more general types. They also allow some conversions between different types. For example setting something to an integer value will make it possible to read the same value as floating point number.
Use boost::any which can wrap any type but does not allow conversions between them.
Use inheritance and polymorphism. For this case you need a common base class, say Base. Then you create an array of pointers to that base preferably with std::shared_ptrs. So the array type would be std::vector<std::shared_ptr<Base>>. The std::shared_ptr is better than built in pointers in this case because the manage your memory automagically by reference counting.
Use a dynamic language that doesn't care about types and performance.
C++17 Update
If you known the type at compile time, you can use templates as illustrated in this answer.
If the type is known at runtime only, with c++17 as an alternative to boost::variant we have the std::variant.
Here is a working example:
#include <iostream>
#include <string>
#include <type_traits>
#include <variant>
#include <vector>
using variant_vector = std::variant<std::vector<int>, std::vector<std::string>>;
auto get_vector(int i)
{
if (i < 0)
return variant_vector(std::vector<int>(3, 1));
else
return variant_vector(std::vector<std::string>(3, "hello"));
}
int main()
{
auto visit_vec = [](const auto& vec) {
using vec_type = typename std::remove_reference_t<decltype(vec)>::value_type;
if constexpr (std::is_same_v<vec_type, int>)
std::cout << "vector of int:" << std::endl;
else if constexpr (std::is_same_v<vec_type, std::string>)
std::cout << "vector of string:" << std::endl;
for (const auto& x : vec)
std::cout << x << std::endl;
};
std::visit(visit_vec, get_vector(-1));
std::visit(visit_vec, get_vector(1));
return 0;
}
See it live on Coliru.
In the code above, the function get_vector returns a std::variant object that either holds a std::vector<int> or a std::vector<std::string>. The contents of the returned object are inspected using std::visit.
It depends on exactly what you're trying to accomplish, but there multiple possibilities for how to do this. Here are a few that come to mind:
If one of a specific list of return types is decided inside the function:
Since you edited your question, this seems to be what you want. You might try boost::variant:
boost::variant<int, double, std::string> foo() {
if (something)
//set type to int
else if (something else)
//set type to double
else
//set type to std::string
}
If the return type depends on a template argument:
You can use SFINAE to manipulate overload resolution:
template<typename T, typename = typename std::enable_if<std::is_integral<T>::value, T>::type>
std::vector<int> foo() {...}
template<typename T, typename = typename std::enable_if<std::is_floating_point<T>::value, T>::type>
std::vector<std::string> foo() {...}
If the return type can be anything:
A boost::any would work well:
boost::any foo() {...}
If the return type is always derived from a specific class:
Return a smart pointer to the base class:
std::unique_ptr<Base> foo() {
if (something)
return std::unique_ptr<Base>{new Derived1};
if (something else)
return std::unique_ptr<Base>{new Derived2};
}
You can use templates, if you know what type to return before you call the function. But you can't have a function, which internally decide to return some type.
What you can do is to create a class which will be a container for returned data, fill object of this class with desired data and then return this object.
typedef enum { VSTRING, VINT, V_WHATEVER ... } datatype;
class MyReturnClass {
datatype d;
// now either
vector<string> * vs;
vector<int> * vi;
// or
void * vector;
}
MyReturnClass * thisIsTheFunction () {
MyReturnClass * return_me = new MyReturnClass();
return_me->datatype = VSTRING;
return_me->vs = new Vector<String>;
return return_me;
}
To update #chris' answer, since C++17 you can use std::variant:
#include <variant>
std::variant<int, double, std::string> foo() {
if (something)
//set type to int
else if (something else)
//set type to double
else
//set type to std::string
}
auto result = foo();
if (std::holds_alternative<int>(result)) {
int value = std::get<int>(result);
}
I have a structure containing 3 fields, two ints (let's call them A and B) and a bool (C).
I want to create a sort of array of that struct and be able to access it through any of the keys (A or B), getting the hole object (with A, B and C) in return.
I won't need to do something like "getting all the object for which the bool is true", if that makes any difference.
Obviously, both key are unique and the bool can't be, but I thought I'd mention it for the sake of clarity.
If there was no A or B, it would be a simple std::map<int, bool>.
The only solution I currently see is to make a wrapper containing 2 sets and a vector.
Is there any way to make my life easier?
NB: It will contain at most a hundred tuples, so performance should not be an issue. Linear access is acceptable.
To make it even clearer, here is what I'd like to be able to do:
foobar<int, int, bool> array; // or something along those lines
array.add(1, 101, true);
array.add(2, 102, false);
array.getA(1); // returns first object
array.getA(2); // returns second object
array.getB(102); // returns second object again
I believe what you're looking for is boost::multi_index. It'll allow you to declare a container with multiple indices.
struct MultiIDStruct
{
size_t idA;
size_t idB;
std::string name;
};
namespace mul = boost::multi_index;
boost::multi_index_container< MultiIDStruct,
mul::indexed_by<
mul::ordered_unique< mul::member< MultiIDStruct, size_t, &MultiIDStruct::idA > >,
mul::ordered_unique< mul::member< MultiIDStruct, size_t, &MultiIDStruct::idB > >
> > data;
(Used namespace "shortcut" as per Rapptz suggestion)
For example here you have a multi_index container of MultiIDStruct for which there are two unique orderings, one on idA (which is a member of MultiIDStruct) and a second on idB (which is also a member).
The template parameters seem like a handful at first but they're not so bad once you understand how they work.
The suggestion to split it up into two maps is certainly a bit simpler, but if you want some more flexibility and can use C++11 for features like std::tuple, you might try something of the form:
#include <iostream>
#include <map>
#include <tuple>
template <typename T1, typename T2, typename T3>
class foobar
{
public:
void add(T1 t1, T2 t2, T3 t3)
{
m1[t1] = std::make_tuple(t1, t2, t3);
m2[t2] = std::make_tuple(t1, t2, t3);
}
std::tuple<T1,T2,T3> getA(T1 t1)
{
return m1[t1];
}
std::tuple<T1,T2,T3> getB(T2 t2)
{
return m2[t2];
}
private:
std::map<T1,std::tuple<T1,T2,T3>> m1;
std::map<T2,std::tuple<T1,T2,T3>> m2;
};
int main()
{
foobar<int, int, bool> array; // or something along those lines
array.add(1, 101, true);
array.add(2, 102, false);
auto res1 = array.getA(1); // returns first object
auto res2 = array.getA(2); // returns second object
auto res3 = array.getB(102); // returns second object again
std::cout << std::get<0>(res1) << std::endl;
std::cout << std::get<1>(res2) << std::endl;
std::cout << std::get<2>(res3) << std::endl;
return 0;
}
A working example gives the output 1, 102, 0 (false).
I know I didn't give the detail implementation. But I'm just suggesting a logic with two maps. What's wrong with it? Why do I get downvoted?
struct s
{
int i;
int j;
bool b;
};
std::map<int, int> mapA;
std::map<int, s> mapB;
const s& getA(int i)
{
return mapB[mapA[i]];
}
const s& getB(int j)
{
return mapB[j];
}
void add(int i, int j, bool b)
{
s p;
p.i=i;
p.j=j;
p.b=b;
mapB[j]=p;
mapA[i]=j;
}
Having the same problem, and a different solution!
Have two hash functions on A and B, giving h1(A) and h2(B) such, that they do not give equal values. Example:
uint32_t A;
uint32_t B;
uint64_t hashA(uint32_t value)
{ return ((uint64_t)value) << 32; }
uint64_t hashB(uint32_t value)
{ return (uint64_t)value; }
Put all your stuff into std::map so that hashA and hashB have the same value for bool. Access it with either hashA or hashB.
Example: A = 0x10000001, B = 0x20000002, C = true
hashA(A): 0x1000000100000000
hashB(B): 0x0000000020000002
map:
0x1000000100000000 -> true
0x0000000020000002 -> true
C++ does not have native support for lazy evaluation (as Haskell does).
I'm wondering if it is possible to implement lazy evaluation in C++ in a reasonable manner. If yes, how would you do it?
EDIT: I like Konrad Rudolph's answer.
I'm wondering if it's possible to implement it in a more generic fashion, for example by using a parametrized class lazy that essentially works for T the way matrix_add works for matrix.
Any operation on T would return lazy instead. The only problem is to store the arguments and operation code inside lazy itself. Can anyone see how to improve this?
I'm wondering if it is possible to implement lazy evaluation in C++ in a reasonable manner. If yes, how would you do it?
Yes, this is possible and quite often done, e.g. for matrix calculations. The main mechanism to facilitate this is operator overloading. Consider the case of matrix addition. The signature of the function would usually look something like this:
matrix operator +(matrix const& a, matrix const& b);
Now, to make this function lazy, it's enough to return a proxy instead of the actual result:
struct matrix_add;
matrix_add operator +(matrix const& a, matrix const& b) {
return matrix_add(a, b);
}
Now all that needs to be done is to write this proxy:
struct matrix_add {
matrix_add(matrix const& a, matrix const& b) : a(a), b(b) { }
operator matrix() const {
matrix result;
// Do the addition.
return result;
}
private:
matrix const& a, b;
};
The magic lies in the method operator matrix() which is an implicit conversion operator from matrix_add to plain matrix. This way, you can chain multiple operations (by providing appropriate overloads of course). The evaluation takes place only when the final result is assigned to a matrix instance.
EDIT I should have been more explicit. As it is, the code makes no sense because although evaluation happens lazily, it still happens in the same expression. In particular, another addition will evaluate this code unless the matrix_add structure is changed to allow chained addition. C++0x greatly facilitates this by allowing variadic templates (i.e. template lists of variable length).
However, one very simple case where this code would actually have a real, direct benefit is the following:
int value = (A + B)(2, 3);
Here, it is assumed that A and B are two-dimensional matrices and that dereferencing is done in Fortran notation, i.e. the above calculates one element out of a matrix sum. It's of course wasteful to add the whole matrices. matrix_add to the rescue:
struct matrix_add {
// … yadda, yadda, yadda …
int operator ()(unsigned int x, unsigned int y) {
// Calculate *just one* element:
return a(x, y) + b(x, y);
}
};
Other examples abound. I've just remembered that I have implemented something related not long ago. Basically, I had to implement a string class that should adhere to a fixed, pre-defined interface. However, my particular string class dealt with huge strings that weren't actually stored in memory. Usually, the user would just access small substrings from the original string using a function infix. I overloaded this function for my string type to return a proxy that held a reference to my string, along with the desired start and end position. Only when this substring was actually used did it query a C API to retrieve this portion of the string.
Boost.Lambda is very nice, but Boost.Proto is exactly what you are looking for. It already has overloads of all C++ operators, which by default perform their usual function when proto::eval() is called, but can be changed.
What Konrad already explained can be put further to support nested invocations of operators, all executed lazily. In Konrad's example, he has an expression object that can store exactly two arguments, for exactly two operands of one operation. The problem is that it will only execute one subexpression lazily, which nicely explains the concept in lazy evaluation put in simple terms, but doesn't improve performance substantially. The other example shows also well how one can apply operator() to add only some elements using that expression object. But to evaluate arbitrary complex expressions, we need some mechanism that can store the structure of that too. We can't get around templates to do that. And the name for that is expression templates. The idea is that one templated expression object can store the structure of some arbitrary sub-expression recursively, like a tree, where the operations are the nodes, and the operands are the child-nodes. For a very good explanation i just found today (some days after i wrote the below code) see here.
template<typename Lhs, typename Rhs>
struct AddOp {
Lhs const& lhs;
Rhs const& rhs;
AddOp(Lhs const& lhs, Rhs const& rhs):lhs(lhs), rhs(rhs) {
// empty body
}
Lhs const& get_lhs() const { return lhs; }
Rhs const& get_rhs() const { return rhs; }
};
That will store any addition operation, even nested one, as can be seen by the following definition of an operator+ for a simple point type:
struct Point { int x, y; };
// add expression template with point at the right
template<typename Lhs, typename Rhs> AddOp<AddOp<Lhs, Rhs>, Point>
operator+(AddOp<Lhs, Rhs> const& lhs, Point const& p) {
return AddOp<AddOp<Lhs, Rhs>, Point>(lhs, p);
}
// add expression template with point at the left
template<typename Lhs, typename Rhs> AddOp< Point, AddOp<Lhs, Rhs> >
operator+(Point const& p, AddOp<Lhs, Rhs> const& rhs) {
return AddOp< Point, AddOp<Lhs, Rhs> >(p, rhs);
}
// add two points, yield a expression template
AddOp< Point, Point >
operator+(Point const& lhs, Point const& rhs) {
return AddOp<Point, Point>(lhs, rhs);
}
Now, if you have
Point p1 = { 1, 2 }, p2 = { 3, 4 }, p3 = { 5, 6 };
p1 + (p2 + p3); // returns AddOp< Point, AddOp<Point, Point> >
You now just need to overload operator= and add a suitable constructor for the Point type and accept AddOp. Change its definition to:
struct Point {
int x, y;
Point(int x = 0, int y = 0):x(x), y(y) { }
template<typename Lhs, typename Rhs>
Point(AddOp<Lhs, Rhs> const& op) {
x = op.get_x();
y = op.get_y();
}
template<typename Lhs, typename Rhs>
Point& operator=(AddOp<Lhs, Rhs> const& op) {
x = op.get_x();
y = op.get_y();
return *this;
}
int get_x() const { return x; }
int get_y() const { return y; }
};
And add the appropriate get_x and get_y into AddOp as member functions:
int get_x() const {
return lhs.get_x() + rhs.get_x();
}
int get_y() const {
return lhs.get_y() + rhs.get_y();
}
Note how we haven't created any temporaries of type Point. It could have been a big matrix with many fields. But at the time the result is needed, we calculate it lazily.
I have nothing to add to Konrad's post, but you can look at Eigen for an example of lazy evaluation done right, in a real world app. It is pretty awe inspiring.
I'm thinking about implementing a template class, that uses std::function. The class should, more or less, look like this:
template <typename Value>
class Lazy
{
public:
Lazy(std::function<Value()> function) : _function(function), _evaluated(false) {}
Value &operator*() { Evaluate(); return _value; }
Value *operator->() { Evaluate(); return &_value; }
private:
void Evaluate()
{
if (!_evaluated)
{
_value = _function();
_evaluated = true;
}
}
std::function<Value()> _function;
Value _value;
bool _evaluated;
};
For example usage:
class Noisy
{
public:
Noisy(int i = 0) : _i(i)
{
std::cout << "Noisy(" << _i << ")" << std::endl;
}
Noisy(const Noisy &that) : _i(that._i)
{
std::cout << "Noisy(const Noisy &)" << std::endl;
}
~Noisy()
{
std::cout << "~Noisy(" << _i << ")" << std::endl;
}
void MakeNoise()
{
std::cout << "MakeNoise(" << _i << ")" << std::endl;
}
private:
int _i;
};
int main()
{
Lazy<Noisy> n = [] () { return Noisy(10); };
std::cout << "about to make noise" << std::endl;
n->MakeNoise();
(*n).MakeNoise();
auto &nn = *n;
nn.MakeNoise();
}
Above code should produce the following message on the console:
Noisy(0)
about to make noise
Noisy(10)
~Noisy(10)
MakeNoise(10)
MakeNoise(10)
MakeNoise(10)
~Noisy(10)
Note that the constructor printing Noisy(10) will not be called until the variable is accessed.
This class is far from perfect, though. The first thing would be the default constructor of Value will have to be called on member initialization (printing Noisy(0) in this case). We can use pointer for _value instead, but I'm not sure whether it would affect the performance.
Johannes' answer works.But when it comes to more parentheses ,it doesn't work as wish. Here is an example.
Point p1 = { 1, 2 }, p2 = { 3, 4 }, p3 = { 5, 6 }, p4 = { 7, 8 };
(p1 + p2) + (p3+p4)// it works ,but not lazy enough
Because the three overloaded + operator didn't cover the case
AddOp<Llhs,Lrhs>+AddOp<Rlhs,Rrhs>
So the compiler has to convert either (p1+p2) or(p3+p4) to Point ,that's not lazy enough.And when compiler decides which to convert ,it complains. Because none is better than the other .
Here comes my extension: add yet another overloaded operator +
template <typename LLhs, typename LRhs, typename RLhs, typename RRhs>
AddOp<AddOp<LLhs, LRhs>, AddOp<RLhs, RRhs>> operator+(const AddOp<LLhs, LRhs> & leftOperandconst, const AddOp<RLhs, RRhs> & rightOperand)
{
return AddOp<AddOp<LLhs, LRhs>, AddOp<RLhs, RRhs>>(leftOperandconst, rightOperand);
}
Now ,the compiler can handle the case above correctly ,and no implicit conversion ,volia!
As it's going to be done in C++0x, by lambda expressions.
Anything is possible.
It depends on exactly what you mean:
class X
{
public: static X& getObjectA()
{
static X instanceA;
return instanceA;
}
};
Here we have the affect of a global variable that is lazily evaluated at the point of first use.
As newly requested in the question.
And stealing Konrad Rudolph design and extending it.
The Lazy object:
template<typename O,typename T1,typename T2>
struct Lazy
{
Lazy(T1 const& l,T2 const& r)
:lhs(l),rhs(r) {}
typedef typename O::Result Result;
operator Result() const
{
O op;
return op(lhs,rhs);
}
private:
T1 const& lhs;
T2 const& rhs;
};
How to use it:
namespace M
{
class Matrix
{
};
struct MatrixAdd
{
typedef Matrix Result;
Result operator()(Matrix const& lhs,Matrix const& rhs) const
{
Result r;
return r;
}
};
struct MatrixSub
{
typedef Matrix Result;
Result operator()(Matrix const& lhs,Matrix const& rhs) const
{
Result r;
return r;
}
};
template<typename T1,typename T2>
Lazy<MatrixAdd,T1,T2> operator+(T1 const& lhs,T2 const& rhs)
{
return Lazy<MatrixAdd,T1,T2>(lhs,rhs);
}
template<typename T1,typename T2>
Lazy<MatrixSub,T1,T2> operator-(T1 const& lhs,T2 const& rhs)
{
return Lazy<MatrixSub,T1,T2>(lhs,rhs);
}
}
In C++11 lazy evaluation similar to hiapay's answer can be achieved using std::shared_future. You still have to encapsulate calculations in lambdas but memoization is taken care of:
std::shared_future<int> a = std::async(std::launch::deferred, [](){ return 1+1; });
Here's a full example:
#include <iostream>
#include <future>
#define LAZY(EXPR, ...) std::async(std::launch::deferred, [__VA_ARGS__](){ std::cout << "evaluating "#EXPR << std::endl; return EXPR; })
int main() {
std::shared_future<int> f1 = LAZY(8);
std::shared_future<int> f2 = LAZY(2);
std::shared_future<int> f3 = LAZY(f1.get() * f2.get(), f1, f2);
std::cout << "f3 = " << f3.get() << std::endl;
std::cout << "f2 = " << f2.get() << std::endl;
std::cout << "f1 = " << f1.get() << std::endl;
return 0;
}
C++0x is nice and all.... but for those of us living in the present you have Boost lambda library and Boost Phoenix. Both with the intent of bringing large amounts of functional programming to C++.
Lets take Haskell as our inspiration - it being lazy to the core.
Also, let's keep in mind how Linq in C# uses Enumerators in a monadic (urgh - here is the word - sorry) way.
Last not least, lets keep in mind, what coroutines are supposed to provide to programmers. Namely the decoupling of computational steps (e.g. producer consumer) from each other.
And lets try to think about how coroutines relate to lazy evaluation.
All of the above appears to be somehow related.
Next, lets try to extract our personal definition of what "lazy" comes down to.
One interpretation is: We want to state our computation in a composable way, before executing it. Some of those parts we use to compose our complete solution might very well draw upon huge (sometimes infinite) data sources, with our full computation also either producing a finite or infinite result.
Lets get concrete and into some code. We need an example for that! Here, I choose the fizzbuzz "problem" as an example, just for the reason that there is some nice, lazy solution to it.
In Haskell, it looks like this:
module FizzBuzz
( fb
)
where
fb n =
fmap merge fizzBuzzAndNumbers
where
fizz = cycle ["","","fizz"]
buzz = cycle ["","","","","buzz"]
fizzBuzz = zipWith (++) fizz buzz
fizzBuzzAndNumbers = zip [1..n] fizzBuzz
merge (x,s) = if length s == 0 then show x else s
The Haskell function cycle creates an infinite list (lazy, of course!) from a finite list by simply repeating the values in the finite list forever. In an eager programming style, writing something like that would ring alarm bells (memory overflow, endless loops!). But not so in a lazy language. The trick is, that lazy lists are not computed right away. Maybe never. Normally only as much as subsequent code requires it.
The third line in the where block above creates another lazy!! list, by means of combining the infinite lists fizz and buzz by means of the single two elements recipe "concatenate a string element from either input list into a single string". Again, if this were to be immediately evaluated, we would have to wait for our computer to run out of resources.
In the 4th line, we create tuples of the members of a finite lazy list [1..n] with our infinite lazy list fizzbuzz. The result is still lazy.
Even in the main body of our fb function, there is no need to get eager. The whole function returns a list with the solution, which itself is -again- lazy. You could as well think of the result of fb 50 as a computation which you can (partially) evaluate later. Or combine with other stuff, leading to an even larger (lazy) evaluation.
So, in order to get started with our C++ version of "fizzbuzz", we need to think of ways how to combine partial steps of our computation into larger bits of computations, each drawing data from previous steps as required.
You can see the full story in a gist of mine.
Here the basic ideas behind the code:
Borrowing from C# and Linq, we "invent" a stateful, generic type Enumerator, which holds
- The current value of the partial computation
- The state of a partial computation (so we can produce subsequent values)
- The worker function, which produces the next state, the next value and a bool which states if there is more data or if the enumeration has come to an end.
In order to be able to compose Enumerator<T,S> instance by means of the power of the . (dot), this class also contains functions, borrowed from Haskell type classes such as Functor and Applicative.
The worker function for enumerator is always of the form: S -> std::tuple<bool,S,T where S is the generic type variable representing the state and T is the generic type variable representing a value - the result of a computation step.
All this is already visible in the first lines of the Enumerator class definition.
template <class T, class S>
class Enumerator
{
public:
typedef typename S State_t;
typedef typename T Value_t;
typedef std::function<
std::tuple<bool, State_t, Value_t>
(const State_t&
)
> Worker_t;
Enumerator(Worker_t worker, State_t s0)
: m_worker(worker)
, m_state(s0)
, m_value{}
{
}
// ...
};
So, all we need to create a specific enumerator instance, we need to create a worker function, have the initial state and create an instance of Enumerator with those two arguments.
Here an example - function range(first,last) creates a finite range of values. This corresponds to a lazy list in the Haskell world.
template <class T>
Enumerator<T, T> range(const T& first, const T& last)
{
auto finiteRange =
[first, last](const T& state)
{
T v = state;
T s1 = (state < last) ? (state + 1) : state;
bool active = state != s1;
return std::make_tuple(active, s1, v);
};
return Enumerator<T,T>(finiteRange, first);
}
And we can make use of this function, for example like this: auto r1 = range(size_t{1},10); - We have created ourselves a lazy list with 10 elements!
Now, all is missing for our "wow" experience, is to see how we can compose enumerators.
Coming back to Haskells cycle function, which is kind of cool. How would it look in our C++ world? Here it is:
template <class T, class S>
auto
cycle
( Enumerator<T, S> values
) -> Enumerator<T, S>
{
auto eternally =
[values](const S& state) -> std::tuple<bool, S, T>
{
auto[active, s1, v] = values.step(state);
if (active)
{
return std::make_tuple(active, s1, v);
}
else
{
return std::make_tuple(true, values.state(), v);
}
};
return Enumerator<T, S>(eternally, values.state());
}
It takes an enumerator as input and returns an enumerator. Local (lambda) function eternally simply resets the input enumeration to its start value whenever it runs out of values and voilà - we have an infinite, ever repeating version of the list we gave as an argument:: auto foo = cycle(range(size_t{1},3)); And we can already shamelessly compose our lazy "computations".
zip is a good example, showing that we can also create a new enumerator from two input enumerators. The resulting enumerator yields as many values as the smaller of either of the input enumerators (tuples with 2 element, one for each input enumerator). I have implemented zip inside class Enumerator itself. Here is how it looks like:
// member function of class Enumerator<S,T>
template <class T1, class S1>
auto
zip
( Enumerator<T1, S1> other
) -> Enumerator<std::tuple<T, T1>, std::tuple<S, S1> >
{
auto worker0 = this->m_worker;
auto worker1 = other.worker();
auto combine =
[worker0,worker1](std::tuple<S, S1> state) ->
std::tuple<bool, std::tuple<S, S1>, std::tuple<T, T1> >
{
auto[s0, s1] = state;
auto[active0, newS0, v0] = worker0(s0);
auto[active1, newS1, v1] = worker1(s1);
return std::make_tuple
( active0 && active1
, std::make_tuple(newS0, newS1)
, std::make_tuple(v0, v1)
);
};
return Enumerator<std::tuple<T, T1>, std::tuple<S, S1> >
( combine
, std::make_tuple(m_state, other.state())
);
}
Please note, how the "combining" also ends up in combining the state of both sources and the values of both sources.
As this post is already TL;DR; for many, here the...
Summary
Yes, lazy evaluation can be implemented in C++. Here, I did it by borrowing the function names from haskell and the paradigm from C# enumerators and Linq. There might be similarities to pythons itertools, btw. I think they followed a similar approach.
My implementation (see the gist link above) is just a prototype - not production code, btw. So no warranties whatsoever from my side. It serves well as demo code to get the general idea across, though.
And what would this answer be without the final C++ version of fizzbuz, eh? Here it is:
std::string fizzbuzz(size_t n)
{
typedef std::vector<std::string> SVec;
// merge (x,s) = if length s == 0 then show x else s
auto merge =
[](const std::tuple<size_t, std::string> & value)
-> std::string
{
auto[x, s] = value;
if (s.length() > 0) return s;
else return std::to_string(x);
};
SVec fizzes{ "","","fizz" };
SVec buzzes{ "","","","","buzz" };
return
range(size_t{ 1 }, n)
.zip
( cycle(iterRange(fizzes.cbegin(), fizzes.cend()))
.zipWith
( std::function(concatStrings)
, cycle(iterRange(buzzes.cbegin(), buzzes.cend()))
)
)
.map<std::string>(merge)
.statefulFold<std::ostringstream&>
(
[](std::ostringstream& oss, const std::string& s)
{
if (0 == oss.tellp())
{
oss << s;
}
else
{
oss << "," << s;
}
}
, std::ostringstream()
)
.str();
}
And... to drive the point home even further - here a variation of fizzbuzz which returns an "infinite list" to the caller:
typedef std::vector<std::string> SVec;
static const SVec fizzes{ "","","fizz" };
static const SVec buzzes{ "","","","","buzz" };
auto fizzbuzzInfinite() -> decltype(auto)
{
// merge (x,s) = if length s == 0 then show x else s
auto merge =
[](const std::tuple<size_t, std::string> & value)
-> std::string
{
auto[x, s] = value;
if (s.length() > 0) return s;
else return std::to_string(x);
};
auto result =
range(size_t{ 1 })
.zip
(cycle(iterRange(fizzes.cbegin(), fizzes.cend()))
.zipWith
(std::function(concatStrings)
, cycle(iterRange(buzzes.cbegin(), buzzes.cend()))
)
)
.map<std::string>(merge)
;
return result;
}
It is worth showing, since you can learn from it how to dodge the question what the exact return type of that function is (as it depends on the implementation of the function alone, namely how the code combines the enumerators).
Also it demonstrates that we had to move the vectors fizzes and buzzes outside the scope of the function so they are still around when eventually on the outside, the lazy mechanism produces values. If we had not done that, the iterRange(..) code would have stored iterators to the vectors which are long gone.
Using a very simple definition of lazy evaluation, which is the value is not evaluated until needed, I would say that one could implement this through the use of a pointer and macros (for syntax sugar).
#include <stdatomic.h>
#define lazy(var_type) lazy_ ## var_type
#define def_lazy_type( var_type ) \
typedef _Atomic var_type _atomic_ ## var_type; \
typedef _atomic_ ## var_type * lazy(var_type); //pointer to atomic type
#define def_lazy_variable(var_type, var_name ) \
_atomic_ ## var_type _ ## var_name; \
lazy_ ## var_type var_name = & _ ## var_name;
#define assign_lazy( var_name, val ) atomic_store( & _ ## var_name, val )
#define eval_lazy(var_name) atomic_load( &(*var_name) )
#include <stdio.h>
def_lazy_type(int)
void print_power2 ( lazy(int) i )
{
printf( "%d\n", eval_lazy(i) * eval_lazy(i) );
}
typedef struct {
int a;
} simple;
def_lazy_type(simple)
void print_simple ( lazy(simple) s )
{
simple temp = eval_lazy(s);
printf("%d\n", temp.a );
}
#define def_lazy_array1( var_type, nElements, var_name ) \
_atomic_ ## var_type _ ## var_name [ nElements ]; \
lazy(var_type) var_name = _ ## var_name;
int main ( )
{
//declarations
def_lazy_variable( int, X )
def_lazy_variable( simple, Y)
def_lazy_array1(int,10,Z)
simple new_simple;
//first the lazy int
assign_lazy(X,111);
print_power2(X);
//second the lazy struct
new_simple.a = 555;
assign_lazy(Y,new_simple);
print_simple ( Y );
//third the array of lazy ints
for(int i=0; i < 10; i++)
{
assign_lazy( Z[i], i );
}
for(int i=0; i < 10; i++)
{
int r = eval_lazy( &Z[i] ); //must pass with &
printf("%d\n", r );
}
return 0;
}
You'll notice in the function print_power2 there is a macro called eval_lazy which does nothing more than dereference a pointer to get the value just prior to when it's actually needed. The lazy type is accessed atomically, so it's completely thread-safe.