I have written a forward iterator that iterates over the nodes of a graph in order of a (preorder/postorder/inorder) DFS spanning tree. Since it is quite complicated compared to writing a simple DFS and calling a callback for each encountered node, I thought I could use C++20 coroutines to simplify the code of the iterator.
However, C++20 coroutines are not copyable (much less so if they are stateful) but iterators should better be copyable!
Is there any way I could still use some coroutine-like code for my iterator?
Note: I want all iterators to iterate independently from each other. If I copy an iterator and then call ++ on it, then only the iterator should have advanced, but not its copy.
I figured that, with the Miro Knejp "goto-hack", something resembling copyable co-routines is possible as follows (this toy example just counts "+1" "*2" until a certain value but it illustrates the point).
(1) this is just a simple wrapper for the actual function
template<class Func, class Data>
struct CopyableCoroutine {
Data data;
Func func;
CopyableCoroutine(const Data& _data, const Func& _func): data(_data), func(_func) {}
bool done() const { return data.done(); }
template<class... Args> decltype(auto) operator()(Args&&... args) {
return func(data, std::forward<Args>(args)...);
}
};
(2) this is where the magic happens
struct my_stack {
int n;
int next_label_idx = 0;
bool done() const { return n > 30; }
};
int wierd_count(my_stack& coro_data) {
static constexpr void* jump_table[] = {&&beginning, &&before_add, &&after_add, &&the_end};
goto* jump_table[coro_data.next_label_idx];
beginning:
while(coro_data.n <= 32) {
coro_data.next_label_idx = 1;
return coro_data.n;
before_add:
coro_data.next_label_idx = 2;
++coro_data.n; // odd steps: add one
return coro_data.n;
after_add:
coro_data.next_label_idx = 3;
coro_data.n *= 2; // even steps: multiply 2
}
the_end:
return -1;
}
play with it here.
NOTE: unfortunately, this "dirty hack" requires some extra work and I would be super happy if that could be avoided somehow. I'd really love to see C++ native coroutines that can be copied if the user promises that its stack is copyable.
Related
Let's say I have two classes which implement the same basic API, and I want to test that they are "stochastically equivalent"1, over at least over a subset of their methods.
E.g., I write my own "list" class foo:list and rather than painstakingly writing a bunch of unit tests for it, I want to compare it to std::list as a reference. That is, any sequence of operations on foo::list should produce the same results as the same sequence of std::list.
I'm OK listing the names of the operations, but hopefully not much more boilerplate than that. A generic solution that can be applied to other pairs of "behaviorally equivalent" classes is ideal.
1 By "stochastically equivalent" I mean that no differences are observed over many series of operations, which obviously falls short of a complete proof of equivalence.
In Short
Construct one foo::list and one std::list and then compare them as you perform operations on them. Really the only difference from a normal unit test is that you have two containers and instead of directly using REQUIRE() for each operation on the type you are testing, you perform the operation on the type you are testing and the reference type and then compare them. For this we assume that std::list or whatever is bug-free. We then use it as our reference point for not failing. In other words, if the operation succeeds with std::list and succeeds with foo::list, and they compare equal, the operation succeeded.
An Example
You know what the subset of operations are that you can use to compare state and I do not, so here's a mock comparison function
template <class T, class U>
bool compare_types(const T &t, const U &u)
{
bool equivalent = true;
//Generic comparisons here, like iterating over the elements to compare their values.
//Of course update equal or just return false if the comparison fails.
//If your types are not containers, perform whatever is needed to test equivalent state.
return equivalent;
}
As Jarod42 pointed out, this can get more fun and more generic, particularly if the Op f below is a lambda (C++14 needed for generic lambdas):
template <class ValueT, class RefT, class TestT, class Op>
bool compare_op_with_value(RefT &t, TestT &u, Op f, const ValueT &value)
{
if (!compare_types(t, u))
return false;
f(t, value);
f(u, value);
return compare_types(t, u);
}
Your function may return a value:
template <class ValueT, class RefT, class TestT, class Op>
bool compare_op_with_ret(RefT &t, TestT &u, Op f)
{
if (!compare_types(t, u))
return false;
ValueT ret1 = f(t);
ValueT ret2 = f(u);
return ret1 == ret2 && compare_types(t, u);
}
...and so on for dereferenceable return types, etc. You'll need to write a new comparison function for each kind of test, but that's pretty trivial. You'll need to add another template parameter for return types that are not the same (e.g. an iterator).
Then you need your test case (I subbed in std::vector as foo::list for exposition)...
TEMPLATE_TEST_CASE("StdFooCompare", "[list]", int)
{
using std_type = std::list<TestType>;
using foo_type = std::vector<TestType>;
auto initializer = {0,1,2,3,4};
std_type list1 = initializer;
foo_type list2 = initializer;
//testing insertion, using auto since insert() returns iterators
auto insert_f = [](auto list, TestType value) -> auto {
return list.insert(list.begin(), value);
};
REQUIRE(compare_op_with_value(list1, list2, insert_f, -1));
//testing front(), being explicit about the return type rather than using auto
auto front_f = [](auto list) -> TestType & {
return list.front();
};
REQUIRE(compare_op_with_ret<TestType>(list1, list2, front_f));
//continue testing along these lines
}
I could spend a couple more hours on this, but I hope you get the idea. I spent more time on this.
Note: I did not actually run this code, so consider it all pseudo-code to get the idea across, e.g. I may have missed a semicolon or some such.
I am working on a part of a program that applies different types of filters onto an already read data part of a bitmap image. The method in question gets the data stored in a 2-dim std::vector and furthermore a pointer to the function in which the filter is applied as arguments. By that we can generically apply different filters by using this method.
My question is, are function pointers the only way to achieve this, or does C++ offer a more beautiful and more readable solution to achieve it?
This is the method this question is about. Second argument is the function pointer that is used to access the function in the if/else statement inside the for loops.
void SteganoMessage::genFilter(std::vector<std::vector<uint32_t>> *d, uint32_t (*f)(uint32_t, size_t)){
int count = 0;
int pixel = getPixel();
for(auto itOuter = d->begin(); itOuter != d->end(); ++itOuter){
for(auto itInner = itOuter->begin(); itInner != itOuter->end(); ++itInner){
if(mode == true)
*itInner = f(*itInner, sizeof(*itInner));
else
*itInner = f(*itInner, this->getImage()->getBitmapHeader()->getBitCount()/8);
displayProgress(count, pixel);
}
}
displayProgress(0);
}
Call of genFilter function:
//...
{
genFilter(data, substB);
}
//...
While substB is a function of course.
Would be very thankful for a hint that leads me into the right direction where I could research or a code snippet that shows a possible more C++ like way to do it.
Type-preserving
The usual way to pass a function (or things that can be INVOKEd) in C++ is by using a template parameter:
// version #1
template <typename F>
void func(F f)
{
static_assert(std::is_invocable_v<F, std::uint32_t, std::size_t>);
// instead of f(*itInner, sizeof(*itInner))
std::invoke(f, *itInner, sizeof(*itInner));
}
You can also use SFINAE to prevent postponing the error to instantiation time. This also enables overloading:
// version #2
template <typename F>
std::enable_if_t<std::is_invocable_v<F, std::uint32_t, std::size_t>>
func(F f)
{
// no need to static_assert here
std::invoke(f, *itInner, sizeof(*itInner));
}
Since C++20, we can use concepts:
// version #3
template <std::Invocable<std::uint32_t, std::size_t> F>
void func(F f)
{
// same as above
}
Which can be simplified further, using an abbreviated function template, to:
// version #4
void func(std::Invocable<std::uint32_t, std::size_t> auto f)
{
// same as above
}
(This is still a function template rather than an ordinary function. It is equivalent to version #3.)
Type-erasing
You can also use std::function for type erasure:
// version #5
void func(std::function<void(std::uint32_t, std::size_t)> f)
{
// ...
f(*itInner, sizeof(*itInner));
}
Compared to the type-preserving alternatives (versions #1–4), this approach may reduce code bloat, but may incur runtime overhead for virtual function calling.
I agree with the already suggested comments and answer.
But if your question was about finding a way to avoid to pass a raw pointer to function as arguments and gain more control over the given filters, I think you can create a wrapping class with a functor that will handle the applied filters.
What is the motivation behind doing this ? Because a raw pointer to function does not give you the guarantee that the function is what you expect. You can pass any function which respect the prototype but is not a real filter and can do anything.
You can solve this problem this way (explanation below the code):
enum class FILTER_TYPE {MY_FILTER, MY_OTHER_FILTER};
class Filter
{
protected:
FILTER_TYPE f_type;
public:
Filter(FILTER_TYPE ft) : f_type(ft)
{}
uint32_t operator()(uint32_t a, size_t b) const
{
switch(f_type)
{
case FILTER_TYPE::MY_FILTER: return my_filter(a, b);
case FILTER_TYPE::MY_OTHER_FILTER: return my_other_filter(a, b);
}
}
private:
uint32_t my_filter(uint32_t a, size_t b) const
{
return a+static_cast<uint32_t>(b); // completely arbitrary
}
uint32_t my_other_filter(uint32_t a, size_t b) const
{
return a*static_cast<uint32_t>(b); // completely arbitrary
}
};
As you can see, you define all your different filters in the private section. Then you redefine the operator() in order to call the proper filter (selected by the FILTER_TYPE attribute).
Then, you can write your function this way:
void SteganoMessage::genFilter(std::vector <std::vector <uint32_t>> & data, const Filter & filter)
{
int count = 0;
int pixel = getPixel();
for(auto itOuter = data.begin(); itOuter != data.end(); ++itOuter)
{
for(auto itInner = itOuter->begin(); itInner != itOuter->end(); ++itInner)
{
if(mode == true)
*itInner = filter(*itInner, sizeof(*itInner));
else
*itInner = filter(*itInner, this->getImage()->getBitmapHeader()->getBitCount()/8);
displayProgress(count, pixel);
}
}
displayProgress(0);
}
This way, you have the guarantee that the argument is a well-defined filter, and you avoid the use of raw pointer to function (that make the code more readable).
I redefined the operator() in order to use the Filter instance as a function. It makes the code more intuitive in my opinion.
Last thing, I passed the data by reference instead of the address directly.
I hope it can be a good additional information.
Consider a case of a lock-free concurrent data structure where a pop() operation needs to return an item or false if the cointainer is empty (rather than blocking or throwing). The data structure is templated on a user type T, which can potentially be large (but also could be lightweight, and I want things to be efficient in either case). T has to be at least movable, but I don't want it to have to be copyable.
I was thinking that the function signature would be bool DS<T>::pop(T &item) so the item is extracted as an out-parameter rather than return value (which instead is used to indicate success or failure). However, how do I actually pass it out? Assume there's an underlying buffer. Would I do item = std::move(_buff[_tail])—does it make sense to move into a reference out-parameter? A downside is that the user would have to pass in a default-constructed T, which goes a bit against effective RAII because the result is an object that hasn't actually initialized its resources if the function fails.
Another option is returning an std::pair<bool, T> rather than using an out-parameter, but there again needs to be a default-constructible T which holds no resource in the case of failure, for the return std::make_pair(false, T).
A third option would be returning the item as std::unique_ptr<T>, but this incurs useless overhead in the case of T being a pointer or another lightweight type. While I could store just pointers in the data structure with the actual items stored externally, that incurs not just the penalty of additional dereferences and cache misses, but also removes the natural padding that items stored directly in the buffer add and help minimize producer and consumer threads from hitting the same cache lines.
#include <boost/optional.hpp>
#include <string>
template<class T>
struct atomic_queue
{
using value_type = T;
auto pop() -> boost::optional<T>
{
boost::optional<T> result;
/*
* insert atomic ops here, optionally filling result
*/
return result;
};
auto push(T&& arg) -> bool
{
/*
* insert atomic ops here, optionally stealing arg
*/
return true;
};
static auto make_empty_result() {
return boost::optional<T>();
}
};
struct difficult {
difficult(std::string);
difficult() = delete;
difficult(difficult const&) = delete;
difficult& operator=(difficult const&) = delete;
difficult(difficult &&) = default;
difficult& operator=(difficult &&) = default;
};
extern void spin();
int main()
{
atomic_queue<difficult> q;
auto d = difficult("arg");
while(not q.push(std::move(d)))
spin();
auto popped = q.make_empty_result();
while(not (popped = q.pop()))
spin();
auto& val = popped.get();
}
I'm trying to create a wrapper for an std::vector (or any other container from STL, if possible) that can "lock" and "unlock" the const state of a vector that it's holding.
For example, if I create an object of that wrapper, I want to be able to do something like this:
int main()
{
ConstLockVectorWrapper<int> myWrapper(std::vector<int>{}); // Here I pass an empty vector in the constructor parameters,
// which means that my wrapper will be holding an empty vector
// By default the vector inside my wrapper is not locked,
// I can change its size and the values that it holds
myWrapper.get().push_back(10); // ok
myWrapper.get().push_back(20); // ok
myWrapper.get().at(0) = 5; // ok
print(myWrapper.get()); // Prints 5 20
myWrapper.lock(); // Now I made the vector inside my wrapper unchangable
myWrapper.get().push_back(30); // error, the vector is locked
myWrapper.get().at(0) = 55; // error
print(myWrapper.get()); // ok
myWrapper.unlock(); // Now I can change my vector's size and its values again
_getch();
return 0;
}
The only solution (that's not working, unfortunately) I've got, is to create a const reference (const std::vector<T> &) and a regular reference (td::vector<T> &) inside a wrapper class, and bound them to the main vector in our wrapper class.
So, this is what I've done:
template <typename T>
class ConstLockVectorWrapper {
public:
ConstLockVectorWrapper(const std::vector<T> & vec)
: wrappedVector(vec), wrappedVectorRef(wrappedVector), wrappedVectorConstRef(wrappedVector), constLock(false)
{}
void lock()
{
if (constLock) // if the vector is already locked, we just exit the function
return;
// else we lock the vector
constLock = true;
}
void unlock()
{
if (!constLock) // if the vector is already unlocked (changable), we just exit the function
return;
// else we unlock the vector
constLock = false;
}
return_type get() // I need to return a const std::vector<T> & if constLock == true, and std::vector<T> & otherwise, what return type should I put in here?
{
if (constLock)
return wrappedVectorConstRef;
else
return wrappedVectorRef;
}
private:
bool constLock;
std::vector<T> wrappedVector;
// refs
std::vector<T> & wrappedVectorRef;
const std::vector<T> & wrappedVectorConstRef;
};
Of course, it doesn't work. Just because I don't know what to put in the return type of my get() fucntion.
I've tried using trailing return type, didn't work:
template <typename T>
class ConstLockVectorWrapper {
public:
// ...
private:
bool constLock;
std::vector<T> wrappedVector;
// refs
std::vector<T> & wrappedVectorRef;
const std::vector<T> & wrappedVectorConstRef;
public:
auto get() -> decltype((constLock ? wrappedVectorConstRef : wrappedVectorRef))
{
if (constLock)
return wrappedVectorConstRef;
else
return wrappedVectorRef;
}
};
I can't come up with any solution that will actually work, because I'm not so good at C++ yet.
So I'm asking for your help with my problem. Any suggestions or hints to solve this problem would be appreciated!
Thanks
PS
My main goal is to make my wrapper container-type-independent, so it can "lock" and "unlock" the const state of the container it's holding, independently of its type.
And here's the print() function I used in the first code snippet:
template <typename Container>
void print(const Container & c)
{
for (const auto & var : c)
std::cout << var << std::endl;
}
Fundamentally, a method always returns the same thing. The same type. Every time. It's not possible, in C++, to have a method sometimes return one type, and another type at other times. C++ does not work this way.
So, the initial approach would be to have get() return a proxy object with a state. Using, roughly, the same classes and names from your question:
class return_type {
bool is_const;
std::vector<T> &wrapped_vec;
public:
return_type(bool is_constArg,
std::vector<T> &wrapped_vecArg)
: is_const(is_constArg), wrapped_vec(wrapped_vecArg)
{
}
void push_back(T &&t)
{
if (is_const)
throw std::runtime_error(); // Or, whatever...
wrapped_vec.push_back(std::forward<T>(t));
}
// return_type will have to implement, and baby-sit all other
// methods you wish to invoke on the underlying vector.
};
return_type get()
{
return return_type(constLock);
}
This is simple, but crude and somewhat tedious. You would have to implement every std::vector method you need to use in the return_type proxy.
A better approach would be to take advantage of C++11 lambdas. This will avoid the need to reimplement every wheel, at an expense of some additional code bloat. But, big deal. RAM is cheap, these days. Instead of get() and return_type, you will now be implementing two template methods in your wrapper: get_const() and get_mutable(). Each one of them takes a lambda parameter and invokes it and, if all goes well, passing it the wrapped vector as an argument:
template<typename lambda>
void get_mutable(lambda &&l)
{
if (constLock)
throw std::runtime_error(); // Or, whatever...
l(wrapped_vec);
}
template<typename lambda>
void get_const(lambda &&l)
{
l(const_cast<const std::vector<T> &>(wrapped_vec));
}
The only thing you now need to decide is whether you need access a mutable or a constant vector, and pick the right getter:
myWrapper.get_mutable( [&](std::vector<int> &v) { v.push_back(10); } );
get_mutable() throws an exception if the vector is locked at this time. Otherwise it passes the vector to your lambda. Your lambda does whatever the heck it wants with it, which can be push_back(), or anything else, then returns.
But if you only need read-only access to the vector, use get_const():
int s;
myWrapper.get_const( [&](const std::vector<int> &v) { s=v.size(); } );
Note that get_const() takes care to const_cast the vector, before invoking the lambda, so the lambda will not be able to modify it. This will be enforced at compile-time.
With some additional work, it would also be possible to clean this up a little bit, and have the getter also return whatever lambda returns to the caller, making it possible to do something like this:
int s=myWrapper.get_const( [&](const std::vector<int> &v) { return v.size(); } );
It's possible to have get_const() and get_mutable() be smart enough to figure out if the lambda returns something, and happily pass it back to the caller, whatever it is. And how to do that, I suppose, will have to be another question on stackoverflow.com
P.S. If you don't have C++11, you can just have get_const() and get_mutable() return the wrapped vector (with get_mutable() verifying that it's not locked). This really accomplishes the same thing. The key point is that due to the way that C++ works, you will have to disambiguate, in advance, whether you need constant or mutable access.
I was working on a similar problem a while back. In multithreaded environment sometimes its more efficient to have different types of lock depending on whether you are reading or writing. But the locking is entirely cooperative. It is possible to obtain a read-only lock but still accidentally write to the object.
One solution I am exploring is, instead of obtaining a read-only lock from an object, getting a read-only wrapper of my object so that not only is the object read-only locked it is also only possible to call read-only (const) methods on the object.
The basic wrapper I used was something like this:
template<typename T>
class ConstWrapper
{
T& v;
public:
ConstWrapper(T& v): v(v) {}
T const& operator* () const { return v; } // return const reference
T const* operator->() const { return &v;} // return const pointer
};
By overloading the * and -> operators you get a kind of pass through ability to call the enclosed objects methods - but using pointer semantics (though its not a pointer).
std::vector<int> v {1, 2, 3, 4}; // not const
ConstWrapper<std::vector<int>> cv(v); // const wrapper
std::cout << cv->at(0) << '\n'; // okay at() is a const method
cv->push_back(8); // ILLEGAL!! push_back() is not a const method
I'm writing a custom OrderedTree class I want to use as a key to an unordered_set.
I want to do a couple things when hashing the Tree:
calculate the hash lazily and cache it as needed (since this may be an expensive operation),
maybe balance the tree.
Neither of these operations change the semantic equality or hash value of the object, but they do modify some private fields.
Unfortunately, trying to modify any members in OrderedTree while inside std::hash<Tree>::operator() seems to violate const correctness that unordered_set expects.
Can I use my OrderedTree with unordered_set? If so, how?
EDIT:
As per request in the comments, minimal proof of concept:
#include <unordered_set>
std::size_t hash_combine(std::size_t a, std::size_t b) {
// TODO: Copy from boost source or something
return 0;
}
struct Node {
int value;
Node *left, *right, *parent;
std::size_t hash(std::size_t seed) const {
if (left != nullptr)
seed = left->hash(seed);
std::hash<int> hasher;
seed = hash_combine(seed, hasher(value));
if (right != nullptr)
seed = right->hash(seed);
return seed;
}
};
struct Tree {
Tree(): hash_(0), root(nullptr) {}
Node *root;
std::size_t hash() const {
if (hash_ == 0 && root != nullptr) {
hash_ = root->hash(7);
}
return hash_;
}
private:
std::size_t hash_;
};
namespace std {
template<>
struct hash<Tree> {
std::size_t operator()(const Tree& t) const {
return t.hash();
}
};
}
int main() {
std::unordered_set<Tree> set;
}
When I try to compile I get:
Sample.cc:31:13: error: cannot assign to non-static data member within const member function 'hash'
hash_ = root->hash(7);
~~~~~ ^
Sample.cc:29:15: note: member function 'Tree::hash' is declared const here
std::size_t hash() const {
~~~~~~~~~~~~^~~~~~~~~~~~
There is a guarantee that std containers will only call const members when doing const or logically const operations. If those const operations are multiple-reader safe, then so is the container; contrawise, if they are not, neither is the container.
The immutability of the hash value and equality (or < on ordered containers) are the only things you need guarantee in a key type in an associative container. Actual const gives the above multiple-reader guarantee, which can be quite useful. What more, violating it costs you using this in the future, and/or subtle buts when someone does presume const means immutable.
You could carefully synchonize the write operation internally to keep the multiple-reader guarantee, or you can give it up.
To violate const, typically you use mutable. A const method that uses casting to bypass const risks Undefined Behaviour if the object was actually const, and not just a const view of a non-const object.
In general, be careful before using this kind of optimizaton; it can easily increase code complexity (hance bugs, maintenance, etc) more than it adds speed. And speeding up code is fungible: make sure you identify this as slow code and this part as a bottlenecm prior to investing in it. And if you are going to balance in hash, why wait for hash? Balance before insert!