c++ auto-comparator for structs - c++

There is many primitive structs (several hundreds), that are used to transfer data between two components (for example a player and a server). There are no methods in them, just raw data.
The task is to write all requests and answers to be able to replay a player scenario without a server (we remember all question and all answers, that are pure functions).
So the task is put this structs in map without comparator. Now we use memcmp, it allows not to think about changes in this structs and it is compact, but there are too many problems with padding and etc.
Is it possible to get smth like getHashValue or any default comparator with metaprogramming in c++?
Conditions:
1) I do not want to create a comparator for each struct.
2) I want to have an error if a field was added or deleted if it breaks existing behavior and needs fix.
3) I don't want to change header files with struct definitions.
Example of a struct
struct A {
int a;
int b;
c c;
}
bool operator<(const A& a1, const A& a2)
{
if (a1.a != a2.a) return a1.a < a2.a;
if (a1.b != a2.b) return a1.b < a2.b;
if (a1.c != a2.c) return a1.c < a2.c;
return false;
}
I can consider other languages to implement this exact part (collect questions/answers), if it will not require to describe all this structs on that language again.

In C++17 you can pull this off if you are willing to (A) hard code how many elements are in each struct somewhere, and (B) write or generate code for each count of number of elements in the struct.
template<std::size_t N>
using size_k = std::integral_constant<std::size_t, N>;
template<class T>
auto auto_tie( size_k<0>, T&& t ) {
return std::tie();
}
template<class T>
auto auto_tie( size_k<1>, T&& t ) {
auto& [ x0 ] = std::forward<T>(t);
return std::tie( x0 );
}
template<class T>
auto auto_tie( size_k<2>, T&& t ) {
auto& [ x0, x1 ] = std::forward<T>(t);
return std::tie( x0, x1 );
}
// etc
now, in the namespace of the struct in question, write
struct foo {
int x;
};
struct bar {
int a, b;
};
size_k<1> elems( foo const& ) { return {}; }
size_k<2> elems( bar const& ) { return {}; }
an elems function that return the size_k counting how many elements.
Now in the namespace of the structs, write:
template<class T, class Size=decltype(elems(std::declval<T const&>()))>
bool operator<( T const& lhs, T const& rhs ) {
return auto_tie( Size{}, lhs ) < auto_tie( Size{}, rhs );
}
and you are done.
Test code:
foo f0{1}, f1{2};
bar b0{1,2}, b1{-7, -3};
std::cout << (f0<f1) << (f1<f0) << (f0<f0) << "\n";
std::cout << (b0<b1) << (b1<b0) << (b0<b0) << "\n";
Live example.
Getting further than this will require writing 3rd party tools or waiting for reflection extension to C++, maybe in C++20 or 23.
If you get elems wrong, I believe the structured bindings code in auto_tie should generate an error.

I suppose you could write your own compare operator based upon memcmp.
bool operator<(const A &lhs, const A &rhs) {
return memcmp(&lhs, &rhs, sizeof(A)) < 0;
}
Off course, writing these for each object might be a burden, so you could write a template for this. Though without some SFINAE it will cover too much types.
#include <type_traits>
#include <cstring>
template<typename T>
std::enable_if_t<std::is_pod_v<std::decay_t<T> //< Check if POD
&& !std::is_fundamental_v<std::decay_t<T>>>, //< Don't override for fundamental types like 'int'
bool>
operator<(const T &lhs, const T &rhs) {
return memcmp(&lhs, &rhs, sizeof(std::decay_t<T>)) < 0;
}
EDIT: Note that this technique requires you to zero-initialize the structs.

Looks like the best way to do it is to write a generator, that will generate .h and .cpp with bool operator< for all types in this header file. Then add this project as pre-build step to the main.
It doesn't look like a nice solution, but it allows to avoid hand-written code duplication and will support adding/removing new structs/fields. So I didn't find a better way.

Related

Is there a 'requires' replacement for 'void_t'?

void_t is a nice hack to detect compilability of certain expressions, but I wonder if there is some way to do that check with requires (or requires requires 😉 ) since I really do not like void_t from the readability perspective.
For example, for a certain type I want to check if some expression is fine (i.e. compilable) or not, including negations.
Ideally I would wish that this works, but it does not, probably since lambdas are not templated...
#include <unordered_set>
int main() {
auto a = []() requires requires(int x) {x<x;} {};
auto b = []() requires !requires(std::unordered_set<int> x) {x<x;} {};
}
If this use seems weird, my real motivation is to check that something does not compile, for example that my nontemplated type does not have operator< or that it is not constructible from int or...
P.S.: I know boost::hana has a way to do this, I am looking for vanilla C++20 solution.
my real motivation is to check that something does not compile, for example that my nontemplated type does not have operator <
This is possible with concepts, perhaps I am misunderstanding?
template<class T>
concept has_less_than = requires(const T& x, const T& y)
{
{x < y} -> std::same_as<bool>;
};
struct Has_Less
{
bool operator<(const Has_Less& other) const
{
return true;
}
};
struct Nope{};
int main()
{
static_assert(has_less_than<Has_Less>);
static_assert(!has_less_than<Nope>);
}
Live Demo
While the answer of #AndyG is perfectly fine, I would like to make an addition.
You can make an ad-hoc concept as well, like you can see in the next example program (includes are missing):
void func( auto v )
{
// unnamed concept in constexpr if-branch!
if constexpr( requires { { v < v }->std::same_as<bool>; } ) {
// use operator
puts( "HAVE operator <" );
} else {
// cannot use operator
puts( "no operator <" );
}
}
struct X {};
int main()
{
func( 2 );
func( X{} );
return EXIT_SUCCESS;
}
Tested with VisualStudio 2022 (17.3.5) in C++20 mode.
This technique is named "Design by introspection". I also recently wrote a short blog post to this (can be found in my profile).

Generic comparison operator for structs

In many of my unit tests I need to compare the contents of simple structs having only data members:
struct Object {
int start;
int stop;
std::string message;
}
Now, if I want to write something like:
CHECK(object1==object2);
I always have to implement:
bool operator==(const Object& lhs, const Object& rhs) {
return lhs.start==rhs.start && lhs.stop==rhs.stop && lhs.message=rhs.message;
}
Writing all these comparison functions becomes tedious, but is also prone to errors. Just imagine, what will happen if I add a new data member to Object, but the comparison operator will not be updated.
Then I remembered my knowledge in Haskell and the magic deriving(Eq) directive, which just generates a sane comparison function for free.
How, could I derive something similar in C++?
Happily, I figured out that C++17 comes with a generic operator== and that every struct should be easily convertible to an std::tuple by the virtue of std::make_tuple.
So I boldly tried the following:
#include <tuple>
#include <iostream>
#include <tuple>
template<typename T>
bool operator==(const T& lhs, const T& rhs)
{
auto leftTuple = std::make_tuple(lhs);
auto rightTuple = std::make_tuple(rhs);
return leftTuple==rightTuple;
}
struct Object
{
std::string s;
int i;
double d;
};
int main(int arg, char** args)
{
std::cout << (Object{ "H",1,2. } == Object{ "H",1,2. }) << std::endl;
std::cout << (Object{ "A",2,3. } == Object{ "H",1,2. }) << std::endl;
return EXIT_SUCCESS;
}
But, unfortunately it just doesn't compile and I really don't know why. Clang tells me:
main.cpp:11:18: error: use of overloaded operator '==' is ambiguous (with operand types
'std::tuple<Object>' and 'std::tuple<Object>')
return leftTuple==rightTuple;
Can I possibly fix this compile error to get my desired behavior?
No, since comparing tuples reverts to comparing the elements of the tuple, so leftTuple == rightTuple tries to compare two Objects which is not possible.
that every struct should be easily convertible to an std::tuple by the virtue of std::make_tuple
No, you'll just get a tuple with one element, the struct.
The trick is to use std::tie:
std::tie(lhs.mem1, lhs.mem2) == std::tie(rhs.mem1, rhs.mem2)
but that has the same problem as your original solution. Unfortunately C++17 doesn't have any facility to avoid this problemyou could write a macro :). But in C++20 you will be able to do:
struct Object
{
std::string s;
int i;
double d;
bool operator==(const Object &) const = default;
};
which will generate the correct comparison operators for Object.

Efficient and simple comparison operators for structs

The application I am working on currently has a large number structs which contain data which is input from various sources such as data bases and files. For example like this:
struct A
{
float val1;
std::string val2;
int val3;
bool operator < (const A& other) const;
};
For processing, these structs are stored up in STL-containers, such as maps and therefore need a comparison operator. These are all the same and using simple boolean logic they can be written like this:
bool A:operator < (const A& o) const {
return val1 < o.val1 ||
(val1 == o.val1 && ( val2 < o.val2 ||
(val2 == o.val2 && ( val3 < o.val3 ) ) );
}
This seems efficient, but has several drawbacks:
These expressions get huge if the structs as a dozen or more members.
It is cumbersome to write and maintain if members change.
It needs to be done for every struct separately.
Is there a more maintainable way to compare structs like this?
You can use the builtin comparison that ships with <tuple> like this:
#include <tuple>
bool A::operator < (const A& rhs) const {
return std::tie(val1, val2, val3) < std::tie(rhs.val1, rhs.val2, rhs.val3);
}
This doesn't scale when more and more data members are added to the struct, but this might also be a hint that you could create intermediate structs that implement operator < and hence play well with the above implementation of a top-level operator <.
Let me add three additional comments on operator <.
Once you have operator <, clients will expect that all other comparison operators are provided, too. Before we have the three-way comparison in C++20, you can avoid unnecessary boilerplate code by e.g. using the Boost operator library:
#include <boost/operators.hpp>
struct A : private boost::totally_ordered<A> { /* ... */ };
which generates all operators based on operator < and operator == for you.
In your example, there is no need for the operator to be a member of A. You can make it a free function, which is preferable (see here for the rationale).
If there is no intrinsic ordering related to A and you just need operator < to store instances as keys in a std::map, consider providing a named predicate.
Great answer by lubgr.
One further refinement I perform is the creation of a member function as_tuple on any object which is to be ordered by its members:
#include <string>
#include <tuple>
#include <iostream>
struct A
{
float val1;
std::string val2;
int val3;
// provide easy conversion to tuple
auto as_tuple() const
{
return std::tie(val1, val2, val3);
}
};
Which often gives rise to thoughts of a general system of making objects and tuples interchangeable in terms of comparisons
template<class T> auto as_tuple(T&& l) -> decltype(l.as_tuple())
{
return l.as_tuple();
}
template<class...Ts>
auto as_tuple(std::tuple<Ts...> const& tup)
-> decltype(auto)
{
return tup;
}
template<class L, class R>
auto operator < (L const& l, R const& r)
-> decltype(as_tuple(l), void(), as_tuple(r), void(), bool())
{
return as_tuple(l) < as_tuple(r);
}
Which allows such code as:
int main()
{
auto a = A { 1.1, "foo", 0 };
auto b = A { 1.1, "foo", 1 };
auto test1 = a < b;
std::cout << test1 << std::endl;
auto test2 = a < std::make_tuple(1.1, "bar", 0);
std::cout << test2 << std::endl;
auto test3 = std::make_tuple(1.0, "bar", 0) < std::make_tuple(1.1, "bar", 0);
std::cout << test3 << std::endl;
auto test4 = a < std::make_tuple(2l, std::string("bar"), 0);
std::cout << test4 << std::endl;
}
example: http://coliru.stacked-crooked.com/a/ead750f3f65e3ee9

Sorting just two elements using STL

Quite often I have two variables foo1 and foo2 which are numeric types. They represent the bounds of something.
A user supplies values for them, but like a recalcitrant musician, not necessarily in the correct order!
So my code is littered with code like
if (foo2 < foo1){
std::swap(foo2, foo1);
}
Of course, this is an idiomatic sort with two elements not necessarily contiguous in memory. Which makes me wonder: is there a STL one-liner for this?
I suggest to take a step back and let the type system do the job for you: introduce a type like Bounds (or Interval) which takes care of the issue. Something like
template <typename T>
class Interval {
public:
Interval( T start, T end ) : m_start( start ), m_end( end ) {
if ( m_start > m_end ) {
std::swap( m_start, m_end );
}
}
const T &start() const { return m_start; }
const T &end() const { return m_end; }
private:
T m_start, m_end;
};
This not only centralizes the swap-to-sort code, it also helps asserting the correct order very early on so that you don't pass around two elements all the time, which means that you don't even need to check the order so often in the first place.
An alternative approach to avoid the issue is to express the boundaries as a pair of 'start value' and 'length' where the 'length' is an unsigned value.
No, but when you notice you wrote the same code twice it's time to write a function for it:
template<typename T, typename P = std::less<T>>
void swap_if(T& a, T& b, P p = P()) {
if (p(a, b)) {
using std::swap;
swap(a, b);
}
}
 
std::minmax returns pair of smallest and largest element. Which you can use with std::tie.
#include <algorithm>
#include <tuple>
#include <iostream>
int main()
{
int a = 7;
int b = 5;
std::tie(a, b) = std::minmax({a,b});
std::cout << a << " " << b; // output: 5 7
}
Note that this isn't the same as the if(a < b) std::swap(a,b); version. For example this doesn't work with move-only elements.
if the data type of your value that you're going to compare is not already in c++. You need to overload the comparison operators.
For example, if you want to compare foo1 and foo2
template <class T>
class Foo {
private:
int value; // value
public:
int GetValue() const {
return value;
}
};
bool operator<(const Foo& lhs, const Foo& rhs) {
return (lhs.GetValue() < rhs.GetValue());
}
If your value is some type of int, or double. Then you can use the std::list<>::sort member function.
For example:
std::list<int> integer_list;
int_list.push_back(1);
int_list.push_back(8);
int_list.push_back(9);
int_list.push_back(7);
int_list.sort();
for(std::list<int>::iterator list_iter = int_list.begin(); list_iter != int_list.end(); list_iter++)
{
std::cout<<*list_iter<<endl;
}

Lazy evaluation in C++

C++ does not have native support for lazy evaluation (as Haskell does).
I'm wondering if it is possible to implement lazy evaluation in C++ in a reasonable manner. If yes, how would you do it?
EDIT: I like Konrad Rudolph's answer.
I'm wondering if it's possible to implement it in a more generic fashion, for example by using a parametrized class lazy that essentially works for T the way matrix_add works for matrix.
Any operation on T would return lazy instead. The only problem is to store the arguments and operation code inside lazy itself. Can anyone see how to improve this?
I'm wondering if it is possible to implement lazy evaluation in C++ in a reasonable manner. If yes, how would you do it?
Yes, this is possible and quite often done, e.g. for matrix calculations. The main mechanism to facilitate this is operator overloading. Consider the case of matrix addition. The signature of the function would usually look something like this:
matrix operator +(matrix const& a, matrix const& b);
Now, to make this function lazy, it's enough to return a proxy instead of the actual result:
struct matrix_add;
matrix_add operator +(matrix const& a, matrix const& b) {
return matrix_add(a, b);
}
Now all that needs to be done is to write this proxy:
struct matrix_add {
matrix_add(matrix const& a, matrix const& b) : a(a), b(b) { }
operator matrix() const {
matrix result;
// Do the addition.
return result;
}
private:
matrix const& a, b;
};
The magic lies in the method operator matrix() which is an implicit conversion operator from matrix_add to plain matrix. This way, you can chain multiple operations (by providing appropriate overloads of course). The evaluation takes place only when the final result is assigned to a matrix instance.
EDIT I should have been more explicit. As it is, the code makes no sense because although evaluation happens lazily, it still happens in the same expression. In particular, another addition will evaluate this code unless the matrix_add structure is changed to allow chained addition. C++0x greatly facilitates this by allowing variadic templates (i.e. template lists of variable length).
However, one very simple case where this code would actually have a real, direct benefit is the following:
int value = (A + B)(2, 3);
Here, it is assumed that A and B are two-dimensional matrices and that dereferencing is done in Fortran notation, i.e. the above calculates one element out of a matrix sum. It's of course wasteful to add the whole matrices. matrix_add to the rescue:
struct matrix_add {
// … yadda, yadda, yadda …
int operator ()(unsigned int x, unsigned int y) {
// Calculate *just one* element:
return a(x, y) + b(x, y);
}
};
Other examples abound. I've just remembered that I have implemented something related not long ago. Basically, I had to implement a string class that should adhere to a fixed, pre-defined interface. However, my particular string class dealt with huge strings that weren't actually stored in memory. Usually, the user would just access small substrings from the original string using a function infix. I overloaded this function for my string type to return a proxy that held a reference to my string, along with the desired start and end position. Only when this substring was actually used did it query a C API to retrieve this portion of the string.
Boost.Lambda is very nice, but Boost.Proto is exactly what you are looking for. It already has overloads of all C++ operators, which by default perform their usual function when proto::eval() is called, but can be changed.
What Konrad already explained can be put further to support nested invocations of operators, all executed lazily. In Konrad's example, he has an expression object that can store exactly two arguments, for exactly two operands of one operation. The problem is that it will only execute one subexpression lazily, which nicely explains the concept in lazy evaluation put in simple terms, but doesn't improve performance substantially. The other example shows also well how one can apply operator() to add only some elements using that expression object. But to evaluate arbitrary complex expressions, we need some mechanism that can store the structure of that too. We can't get around templates to do that. And the name for that is expression templates. The idea is that one templated expression object can store the structure of some arbitrary sub-expression recursively, like a tree, where the operations are the nodes, and the operands are the child-nodes. For a very good explanation i just found today (some days after i wrote the below code) see here.
template<typename Lhs, typename Rhs>
struct AddOp {
Lhs const& lhs;
Rhs const& rhs;
AddOp(Lhs const& lhs, Rhs const& rhs):lhs(lhs), rhs(rhs) {
// empty body
}
Lhs const& get_lhs() const { return lhs; }
Rhs const& get_rhs() const { return rhs; }
};
That will store any addition operation, even nested one, as can be seen by the following definition of an operator+ for a simple point type:
struct Point { int x, y; };
// add expression template with point at the right
template<typename Lhs, typename Rhs> AddOp<AddOp<Lhs, Rhs>, Point>
operator+(AddOp<Lhs, Rhs> const& lhs, Point const& p) {
return AddOp<AddOp<Lhs, Rhs>, Point>(lhs, p);
}
// add expression template with point at the left
template<typename Lhs, typename Rhs> AddOp< Point, AddOp<Lhs, Rhs> >
operator+(Point const& p, AddOp<Lhs, Rhs> const& rhs) {
return AddOp< Point, AddOp<Lhs, Rhs> >(p, rhs);
}
// add two points, yield a expression template
AddOp< Point, Point >
operator+(Point const& lhs, Point const& rhs) {
return AddOp<Point, Point>(lhs, rhs);
}
Now, if you have
Point p1 = { 1, 2 }, p2 = { 3, 4 }, p3 = { 5, 6 };
p1 + (p2 + p3); // returns AddOp< Point, AddOp<Point, Point> >
You now just need to overload operator= and add a suitable constructor for the Point type and accept AddOp. Change its definition to:
struct Point {
int x, y;
Point(int x = 0, int y = 0):x(x), y(y) { }
template<typename Lhs, typename Rhs>
Point(AddOp<Lhs, Rhs> const& op) {
x = op.get_x();
y = op.get_y();
}
template<typename Lhs, typename Rhs>
Point& operator=(AddOp<Lhs, Rhs> const& op) {
x = op.get_x();
y = op.get_y();
return *this;
}
int get_x() const { return x; }
int get_y() const { return y; }
};
And add the appropriate get_x and get_y into AddOp as member functions:
int get_x() const {
return lhs.get_x() + rhs.get_x();
}
int get_y() const {
return lhs.get_y() + rhs.get_y();
}
Note how we haven't created any temporaries of type Point. It could have been a big matrix with many fields. But at the time the result is needed, we calculate it lazily.
I have nothing to add to Konrad's post, but you can look at Eigen for an example of lazy evaluation done right, in a real world app. It is pretty awe inspiring.
I'm thinking about implementing a template class, that uses std::function. The class should, more or less, look like this:
template <typename Value>
class Lazy
{
public:
Lazy(std::function<Value()> function) : _function(function), _evaluated(false) {}
Value &operator*() { Evaluate(); return _value; }
Value *operator->() { Evaluate(); return &_value; }
private:
void Evaluate()
{
if (!_evaluated)
{
_value = _function();
_evaluated = true;
}
}
std::function<Value()> _function;
Value _value;
bool _evaluated;
};
For example usage:
class Noisy
{
public:
Noisy(int i = 0) : _i(i)
{
std::cout << "Noisy(" << _i << ")" << std::endl;
}
Noisy(const Noisy &that) : _i(that._i)
{
std::cout << "Noisy(const Noisy &)" << std::endl;
}
~Noisy()
{
std::cout << "~Noisy(" << _i << ")" << std::endl;
}
void MakeNoise()
{
std::cout << "MakeNoise(" << _i << ")" << std::endl;
}
private:
int _i;
};
int main()
{
Lazy<Noisy> n = [] () { return Noisy(10); };
std::cout << "about to make noise" << std::endl;
n->MakeNoise();
(*n).MakeNoise();
auto &nn = *n;
nn.MakeNoise();
}
Above code should produce the following message on the console:
Noisy(0)
about to make noise
Noisy(10)
~Noisy(10)
MakeNoise(10)
MakeNoise(10)
MakeNoise(10)
~Noisy(10)
Note that the constructor printing Noisy(10) will not be called until the variable is accessed.
This class is far from perfect, though. The first thing would be the default constructor of Value will have to be called on member initialization (printing Noisy(0) in this case). We can use pointer for _value instead, but I'm not sure whether it would affect the performance.
Johannes' answer works.But when it comes to more parentheses ,it doesn't work as wish. Here is an example.
Point p1 = { 1, 2 }, p2 = { 3, 4 }, p3 = { 5, 6 }, p4 = { 7, 8 };
(p1 + p2) + (p3+p4)// it works ,but not lazy enough
Because the three overloaded + operator didn't cover the case
AddOp<Llhs,Lrhs>+AddOp<Rlhs,Rrhs>
So the compiler has to convert either (p1+p2) or(p3+p4) to Point ,that's not lazy enough.And when compiler decides which to convert ,it complains. Because none is better than the other .
Here comes my extension: add yet another overloaded operator +
template <typename LLhs, typename LRhs, typename RLhs, typename RRhs>
AddOp<AddOp<LLhs, LRhs>, AddOp<RLhs, RRhs>> operator+(const AddOp<LLhs, LRhs> & leftOperandconst, const AddOp<RLhs, RRhs> & rightOperand)
{
return AddOp<AddOp<LLhs, LRhs>, AddOp<RLhs, RRhs>>(leftOperandconst, rightOperand);
}
Now ,the compiler can handle the case above correctly ,and no implicit conversion ,volia!
As it's going to be done in C++0x, by lambda expressions.
Anything is possible.
It depends on exactly what you mean:
class X
{
public: static X& getObjectA()
{
static X instanceA;
return instanceA;
}
};
Here we have the affect of a global variable that is lazily evaluated at the point of first use.
As newly requested in the question.
And stealing Konrad Rudolph design and extending it.
The Lazy object:
template<typename O,typename T1,typename T2>
struct Lazy
{
Lazy(T1 const& l,T2 const& r)
:lhs(l),rhs(r) {}
typedef typename O::Result Result;
operator Result() const
{
O op;
return op(lhs,rhs);
}
private:
T1 const& lhs;
T2 const& rhs;
};
How to use it:
namespace M
{
class Matrix
{
};
struct MatrixAdd
{
typedef Matrix Result;
Result operator()(Matrix const& lhs,Matrix const& rhs) const
{
Result r;
return r;
}
};
struct MatrixSub
{
typedef Matrix Result;
Result operator()(Matrix const& lhs,Matrix const& rhs) const
{
Result r;
return r;
}
};
template<typename T1,typename T2>
Lazy<MatrixAdd,T1,T2> operator+(T1 const& lhs,T2 const& rhs)
{
return Lazy<MatrixAdd,T1,T2>(lhs,rhs);
}
template<typename T1,typename T2>
Lazy<MatrixSub,T1,T2> operator-(T1 const& lhs,T2 const& rhs)
{
return Lazy<MatrixSub,T1,T2>(lhs,rhs);
}
}
In C++11 lazy evaluation similar to hiapay's answer can be achieved using std::shared_future. You still have to encapsulate calculations in lambdas but memoization is taken care of:
std::shared_future<int> a = std::async(std::launch::deferred, [](){ return 1+1; });
Here's a full example:
#include <iostream>
#include <future>
#define LAZY(EXPR, ...) std::async(std::launch::deferred, [__VA_ARGS__](){ std::cout << "evaluating "#EXPR << std::endl; return EXPR; })
int main() {
std::shared_future<int> f1 = LAZY(8);
std::shared_future<int> f2 = LAZY(2);
std::shared_future<int> f3 = LAZY(f1.get() * f2.get(), f1, f2);
std::cout << "f3 = " << f3.get() << std::endl;
std::cout << "f2 = " << f2.get() << std::endl;
std::cout << "f1 = " << f1.get() << std::endl;
return 0;
}
C++0x is nice and all.... but for those of us living in the present you have Boost lambda library and Boost Phoenix. Both with the intent of bringing large amounts of functional programming to C++.
Lets take Haskell as our inspiration - it being lazy to the core.
Also, let's keep in mind how Linq in C# uses Enumerators in a monadic (urgh - here is the word - sorry) way.
Last not least, lets keep in mind, what coroutines are supposed to provide to programmers. Namely the decoupling of computational steps (e.g. producer consumer) from each other.
And lets try to think about how coroutines relate to lazy evaluation.
All of the above appears to be somehow related.
Next, lets try to extract our personal definition of what "lazy" comes down to.
One interpretation is: We want to state our computation in a composable way, before executing it. Some of those parts we use to compose our complete solution might very well draw upon huge (sometimes infinite) data sources, with our full computation also either producing a finite or infinite result.
Lets get concrete and into some code. We need an example for that! Here, I choose the fizzbuzz "problem" as an example, just for the reason that there is some nice, lazy solution to it.
In Haskell, it looks like this:
module FizzBuzz
( fb
)
where
fb n =
fmap merge fizzBuzzAndNumbers
where
fizz = cycle ["","","fizz"]
buzz = cycle ["","","","","buzz"]
fizzBuzz = zipWith (++) fizz buzz
fizzBuzzAndNumbers = zip [1..n] fizzBuzz
merge (x,s) = if length s == 0 then show x else s
The Haskell function cycle creates an infinite list (lazy, of course!) from a finite list by simply repeating the values in the finite list forever. In an eager programming style, writing something like that would ring alarm bells (memory overflow, endless loops!). But not so in a lazy language. The trick is, that lazy lists are not computed right away. Maybe never. Normally only as much as subsequent code requires it.
The third line in the where block above creates another lazy!! list, by means of combining the infinite lists fizz and buzz by means of the single two elements recipe "concatenate a string element from either input list into a single string". Again, if this were to be immediately evaluated, we would have to wait for our computer to run out of resources.
In the 4th line, we create tuples of the members of a finite lazy list [1..n] with our infinite lazy list fizzbuzz. The result is still lazy.
Even in the main body of our fb function, there is no need to get eager. The whole function returns a list with the solution, which itself is -again- lazy. You could as well think of the result of fb 50 as a computation which you can (partially) evaluate later. Or combine with other stuff, leading to an even larger (lazy) evaluation.
So, in order to get started with our C++ version of "fizzbuzz", we need to think of ways how to combine partial steps of our computation into larger bits of computations, each drawing data from previous steps as required.
You can see the full story in a gist of mine.
Here the basic ideas behind the code:
Borrowing from C# and Linq, we "invent" a stateful, generic type Enumerator, which holds
- The current value of the partial computation
- The state of a partial computation (so we can produce subsequent values)
- The worker function, which produces the next state, the next value and a bool which states if there is more data or if the enumeration has come to an end.
In order to be able to compose Enumerator<T,S> instance by means of the power of the . (dot), this class also contains functions, borrowed from Haskell type classes such as Functor and Applicative.
The worker function for enumerator is always of the form: S -> std::tuple<bool,S,T where S is the generic type variable representing the state and T is the generic type variable representing a value - the result of a computation step.
All this is already visible in the first lines of the Enumerator class definition.
template <class T, class S>
class Enumerator
{
public:
typedef typename S State_t;
typedef typename T Value_t;
typedef std::function<
std::tuple<bool, State_t, Value_t>
(const State_t&
)
> Worker_t;
Enumerator(Worker_t worker, State_t s0)
: m_worker(worker)
, m_state(s0)
, m_value{}
{
}
// ...
};
So, all we need to create a specific enumerator instance, we need to create a worker function, have the initial state and create an instance of Enumerator with those two arguments.
Here an example - function range(first,last) creates a finite range of values. This corresponds to a lazy list in the Haskell world.
template <class T>
Enumerator<T, T> range(const T& first, const T& last)
{
auto finiteRange =
[first, last](const T& state)
{
T v = state;
T s1 = (state < last) ? (state + 1) : state;
bool active = state != s1;
return std::make_tuple(active, s1, v);
};
return Enumerator<T,T>(finiteRange, first);
}
And we can make use of this function, for example like this: auto r1 = range(size_t{1},10); - We have created ourselves a lazy list with 10 elements!
Now, all is missing for our "wow" experience, is to see how we can compose enumerators.
Coming back to Haskells cycle function, which is kind of cool. How would it look in our C++ world? Here it is:
template <class T, class S>
auto
cycle
( Enumerator<T, S> values
) -> Enumerator<T, S>
{
auto eternally =
[values](const S& state) -> std::tuple<bool, S, T>
{
auto[active, s1, v] = values.step(state);
if (active)
{
return std::make_tuple(active, s1, v);
}
else
{
return std::make_tuple(true, values.state(), v);
}
};
return Enumerator<T, S>(eternally, values.state());
}
It takes an enumerator as input and returns an enumerator. Local (lambda) function eternally simply resets the input enumeration to its start value whenever it runs out of values and voilà - we have an infinite, ever repeating version of the list we gave as an argument:: auto foo = cycle(range(size_t{1},3)); And we can already shamelessly compose our lazy "computations".
zip is a good example, showing that we can also create a new enumerator from two input enumerators. The resulting enumerator yields as many values as the smaller of either of the input enumerators (tuples with 2 element, one for each input enumerator). I have implemented zip inside class Enumerator itself. Here is how it looks like:
// member function of class Enumerator<S,T>
template <class T1, class S1>
auto
zip
( Enumerator<T1, S1> other
) -> Enumerator<std::tuple<T, T1>, std::tuple<S, S1> >
{
auto worker0 = this->m_worker;
auto worker1 = other.worker();
auto combine =
[worker0,worker1](std::tuple<S, S1> state) ->
std::tuple<bool, std::tuple<S, S1>, std::tuple<T, T1> >
{
auto[s0, s1] = state;
auto[active0, newS0, v0] = worker0(s0);
auto[active1, newS1, v1] = worker1(s1);
return std::make_tuple
( active0 && active1
, std::make_tuple(newS0, newS1)
, std::make_tuple(v0, v1)
);
};
return Enumerator<std::tuple<T, T1>, std::tuple<S, S1> >
( combine
, std::make_tuple(m_state, other.state())
);
}
Please note, how the "combining" also ends up in combining the state of both sources and the values of both sources.
As this post is already TL;DR; for many, here the...
Summary
Yes, lazy evaluation can be implemented in C++. Here, I did it by borrowing the function names from haskell and the paradigm from C# enumerators and Linq. There might be similarities to pythons itertools, btw. I think they followed a similar approach.
My implementation (see the gist link above) is just a prototype - not production code, btw. So no warranties whatsoever from my side. It serves well as demo code to get the general idea across, though.
And what would this answer be without the final C++ version of fizzbuz, eh? Here it is:
std::string fizzbuzz(size_t n)
{
typedef std::vector<std::string> SVec;
// merge (x,s) = if length s == 0 then show x else s
auto merge =
[](const std::tuple<size_t, std::string> & value)
-> std::string
{
auto[x, s] = value;
if (s.length() > 0) return s;
else return std::to_string(x);
};
SVec fizzes{ "","","fizz" };
SVec buzzes{ "","","","","buzz" };
return
range(size_t{ 1 }, n)
.zip
( cycle(iterRange(fizzes.cbegin(), fizzes.cend()))
.zipWith
( std::function(concatStrings)
, cycle(iterRange(buzzes.cbegin(), buzzes.cend()))
)
)
.map<std::string>(merge)
.statefulFold<std::ostringstream&>
(
[](std::ostringstream& oss, const std::string& s)
{
if (0 == oss.tellp())
{
oss << s;
}
else
{
oss << "," << s;
}
}
, std::ostringstream()
)
.str();
}
And... to drive the point home even further - here a variation of fizzbuzz which returns an "infinite list" to the caller:
typedef std::vector<std::string> SVec;
static const SVec fizzes{ "","","fizz" };
static const SVec buzzes{ "","","","","buzz" };
auto fizzbuzzInfinite() -> decltype(auto)
{
// merge (x,s) = if length s == 0 then show x else s
auto merge =
[](const std::tuple<size_t, std::string> & value)
-> std::string
{
auto[x, s] = value;
if (s.length() > 0) return s;
else return std::to_string(x);
};
auto result =
range(size_t{ 1 })
.zip
(cycle(iterRange(fizzes.cbegin(), fizzes.cend()))
.zipWith
(std::function(concatStrings)
, cycle(iterRange(buzzes.cbegin(), buzzes.cend()))
)
)
.map<std::string>(merge)
;
return result;
}
It is worth showing, since you can learn from it how to dodge the question what the exact return type of that function is (as it depends on the implementation of the function alone, namely how the code combines the enumerators).
Also it demonstrates that we had to move the vectors fizzes and buzzes outside the scope of the function so they are still around when eventually on the outside, the lazy mechanism produces values. If we had not done that, the iterRange(..) code would have stored iterators to the vectors which are long gone.
Using a very simple definition of lazy evaluation, which is the value is not evaluated until needed, I would say that one could implement this through the use of a pointer and macros (for syntax sugar).
#include <stdatomic.h>
#define lazy(var_type) lazy_ ## var_type
#define def_lazy_type( var_type ) \
typedef _Atomic var_type _atomic_ ## var_type; \
typedef _atomic_ ## var_type * lazy(var_type); //pointer to atomic type
#define def_lazy_variable(var_type, var_name ) \
_atomic_ ## var_type _ ## var_name; \
lazy_ ## var_type var_name = & _ ## var_name;
#define assign_lazy( var_name, val ) atomic_store( & _ ## var_name, val )
#define eval_lazy(var_name) atomic_load( &(*var_name) )
#include <stdio.h>
def_lazy_type(int)
void print_power2 ( lazy(int) i )
{
printf( "%d\n", eval_lazy(i) * eval_lazy(i) );
}
typedef struct {
int a;
} simple;
def_lazy_type(simple)
void print_simple ( lazy(simple) s )
{
simple temp = eval_lazy(s);
printf("%d\n", temp.a );
}
#define def_lazy_array1( var_type, nElements, var_name ) \
_atomic_ ## var_type _ ## var_name [ nElements ]; \
lazy(var_type) var_name = _ ## var_name;
int main ( )
{
//declarations
def_lazy_variable( int, X )
def_lazy_variable( simple, Y)
def_lazy_array1(int,10,Z)
simple new_simple;
//first the lazy int
assign_lazy(X,111);
print_power2(X);
//second the lazy struct
new_simple.a = 555;
assign_lazy(Y,new_simple);
print_simple ( Y );
//third the array of lazy ints
for(int i=0; i < 10; i++)
{
assign_lazy( Z[i], i );
}
for(int i=0; i < 10; i++)
{
int r = eval_lazy( &Z[i] ); //must pass with &
printf("%d\n", r );
}
return 0;
}
You'll notice in the function print_power2 there is a macro called eval_lazy which does nothing more than dereference a pointer to get the value just prior to when it's actually needed. The lazy type is accessed atomically, so it's completely thread-safe.