I've implemented an LRU cache (code) that I would like to use for a multithreaded matching problem with N elements and full N^2 (all pairs) matching. Ideally, I would just get a reference to each element directly from the cache to save memory.
The time is takes to match two elements (lets call them A and B) can greatly vary, and I am worried that if one pair of elements takes a long time to match then another thread (that is working very fast and processing many pairs) will cause either A or B to be evicted from the cache, making the references invalid.
A simple solution is to just not use references, but I'm wondering if there is a better way to ensure that elements won't be evicted if they are "currently used" or have a reference to them?
To avoid evicting objects that are in use it is possible to use the reference-counting functionality of std::shared_ptr. Consider the following implementation:
#include <iostream>
#include <string>
#include <memory>
#include <map>
#include <algorithm>
template <typename K, typename V> class cache
{
public:
cache() {}
static const constexpr int max_cache_size = 2;
std::shared_ptr<V> getValue(const K& k)
{
auto iter = cached_values.find(k);
if (iter == cached_values.end()) {
if (cached_values.size() == max_cache_size) {
auto evictIter =
std::find_if(cached_values.begin(), cached_values.end(),
[](const auto& kv) { return kv.second.second.unique(); });
if (evictIter == cached_values.end()) {
std::cout << "Nothing to evict\n";
return nullptr;
}
cached_values.erase(evictIter);
}
static V next;
iter = cached_values.insert(std::make_pair(k, std::make_pair(++next, nullptr))).first;
iter->second.second = std::shared_ptr<V>(&iter->second.first, [](const auto&) {});
}
return iter->second.second;
}
std::map<K, std::pair<V, std::shared_ptr<V>>> cached_values;
};
int main()
{
cache<int, int> c;
std::cout << *c.getValue(10) << "\n";
std::cout << *c.getValue(20) << "\n";
std::cout << *c.getValue(30) << "\n";
auto useOne = c.getValue(10);
auto useTwo = c.getValue(20);
std::cout << *c.getValue(20) << "\n"; // We can use stuff that is still in cache
std::cout << c.getValue(30); // Cache is full, also note no dereferencing
}
Basically, as long as anyone outside the cache uses the returned value, the std::shared_ptr::unique will return false, making the cache entry non-evictable.
Live example
Related
I have the following code where I implement dispatching on runtime value to interpret the data in certain way(in this toy example data can be either uint8_t or short).
Code seems to work, but I am wondering if I can somehow microoptimize the code so that when I have a hit(processing function matches) processing is stopped (currently even if first element of tuple is a "handler" entire tuple is iterated over at runtime).
#include <boost/mp11/tuple.hpp>
#include <iostream>
uint8_t data[4] = {0,1,100,2};
template<int runtimeId, typename T>
struct kindToType{
static constexpr int id = runtimeId;
using type = T;
};
const auto print =[]<typename T> (const T* data){
if constexpr(std::is_same_v<short, std::remove_cvref_t<T>>){
const short* values = (const short*)data;
std::cout << values[0] << " " << values[1] << std::endl;
} else if constexpr(std::is_same_v<uint8_t, std::remove_cvref_t<T>>){
const uint8_t* values = (const uint8_t*)data;
std::cout << (int)values[0] << " " << (int)values[1]<< " " << (int)values[2] << " " << (int)values[3] << std::endl;;
}
};
static constexpr std::tuple<kindToType<10, uint8_t>, kindToType<11, short>> mappings{};
void dispatch(int kind){
boost::mp11::tuple_for_each(mappings, [kind]<typename Mapping>(const Mapping&) {
if (Mapping::id == kind)
{
print((typename Mapping::type*)data);
}
});
}
int main()
{
// no guarantee that kind is index like(e.g. for two values
// it can have values 47 and 1701)
dispatch(10);
dispatch(11);
}
Notes:
I can not/want to use std::variant.
I do not want to use std::map or std::unordered map(where value is std::function)
I know this is premature optimization(even 10 integer comparisons is cheap assuming handlers do nontrivial amount of work).
my handlers are unique, i.e. it is std::map like thing, not std::multimap like thing so it is fine to break;.
kind of id used for runtime values is not guaranteed to have values in [0, n-1].
I am fine with C++20 solution as long as it is implemented in at least 1 compiler.
The runtime performance of this heavily depends on the size of your tuple. You can make your own for_each_tuple implementation that does an early out when your function gets executed:
template<typename FuncTuple, typename Selector>
void tuple_for_each(FuncTuple const& funcTuple, Selector selector)
{
std::apply([selector](auto const& ...funcs)
{
(void)(selector(funcs) || ...);
}, funcTuple);
}
your dispatch would then look like this:
void dispatch(int kind)
{
tuple_for_each(mappings, [kind]<typename Mapping>(const Mapping&)
{
std::cout << "loop, ";
if (Mapping::id == kind)
{
print((typename Mapping::type*)data);
return true;
}
return false;
});
}
If you get rid of the template in your lambda and use auto instead this code will compile with C++17. We use operator short circuiting to our advantage so the compiler will provide an early out for us. Here is the full code.
Also, note that the cast (const short*)data is UB.
I have a class that stores a std::vector of stuff. In my program, I create a std::unordered_set of std::shared_ptr to objects of this class (see code below). I defined custom functions to compute hashes and equality so that the unordered_set "works" with the objects instead of the pointers. This means: Two different pointers to different objects that have the same content should be treated as equal, let's call it "equivalent".
So far everything worked as expected but now I stumbled across a strange behaviour: I add a pointer to an object to the unordered_set and create a different pointer to a different object with the same content. As said I would expect that my_set.find(different_object) would return a valid iterator to the equivalent pointer stored in the set. But it doesn't.
Here is a minimal working code example.
#include <boost/functional/hash.hpp>
#include <cstdlib>
#include <functional>
#include <iostream>
#include <memory>
#include <unordered_set>
#include <vector>
class Foo {
public:
Foo() {}
bool operator==(Foo const & rhs) const {
return bar == rhs.bar;
}
std::vector<int> bar;
};
struct FooHash {
size_t operator()(std::shared_ptr<Foo> const & foo) const {
size_t seed = 0;
for (size_t i = 0; i < foo->bar.size(); ++i) {
boost::hash_combine(seed, foo->bar[i]);
}
return seed;
}
};
struct FooEq {
bool operator()(std::shared_ptr<Foo> const & rhs,
std::shared_ptr<Foo> const & lhs) const {
return *lhs == *rhs;
}
};
int main() {
std::unordered_set<std::shared_ptr<Foo>, FooHash, FooEq> fooSet;
auto empl = fooSet.emplace(std::make_shared<Foo>());
(*(empl.first))->bar.emplace_back(0);
auto baz = std::make_shared<Foo>();
baz->bar.emplace_back(0);
auto eqFun = fooSet.key_eq();
auto hashFun = fooSet.hash_function();
if (**fooSet.begin() == *baz) {
std::cout << "Objects equal" << std::endl;
}
if (eqFun(*fooSet.begin(), baz)) {
std::cout << "Keys equal" << std::endl;
}
if (hashFun(*fooSet.begin()) == hashFun(baz)) {
std::cout << "Hashes equal" << std::endl;
}
if (fooSet.find(baz) != fooSet.end()) {
std::cout << "Baz in fooSet" << std::endl;
} else {
std::cout << "Baz not in fooSet" << std::endl;
}
return 0;
}
Output
Objects equal
Keys equal
Hashes equal
And here is the problem:
Baz not in fooSet
What am I missing here? Why does the set not find the equivalent object?
Possibly of interest: I played around with this and found that if my class stores a plain int instead of a std::vector, it works. If I stick to the std::vector but change my constructor to
Foo(int i) : bar{i} {}
and initialize my objects with
std::make_shared<Foo>(0);
it also works. If I remove the whole pointer stuff, It breaks as std::unordered_set::find returns constant iterators and thus modification of objects in the set cannot be done (this way). However, none of these changes is applicable in my real program, anyway.
I compile with g++ version 7.3.0 using -std=c++17
You can't modify an element of a set (and expect the set to work). Because you have provided FooHash and FooEq which inspect the referent's value, that makes the referent part of the value from the point of view of the set!
If we change the initialisation of fooSet to set up the element before inserting it, we get the result you want/expect:
std::unordered_set<std::shared_ptr<Foo>, FooHash, FooEq> fooSet;
auto e = std::make_shared<Foo>();
e->bar.emplace_back(0); // modification is _before_
fooSet.insert(e); // insertion
Looking up the object in the set depends on the hash value not changing. If we really need to modify a member after it has been added, we need to remove it, make the changes, then add the modified object - see Yakk's answer.
To avoid running into issues like this, it may be safer to use std::shared_ptr<const Foo> as elements, which will prevent modification of the pointed-at Foo through the set (although you're still responsible for the use of any non-const pointers you may also have).
Any operation such that the hash or == result of an element in an unordered_set violates the rules of unordered_set is bad; the result is undefined behavior.
You changed the result of a hash of an element in an unordered_set, because your elements are shared pointers, but their hash and == is based off of the value pointed to. And your code changes the value pointed to.
Make all std::shared_ptr<Foo> in your code std::shared_ptr<Foo const>.
This includes the equals and hash code and unordered set code.
auto empl = fooSet.emplace(std::make_shared<Foo>());
(*(empl.first))->bar.emplace_back(0);
this code is right out, and it will (afterwards) fail to compile, as is safe.
If you want to mutate an element in a fooSet,
template<class C, class It, class F>
void mutate(C& c, It it, F&& f) {
auto e = *it->first;
f(e); // do this before erasing, more exception-safe
auto new_elem = std::make_shared<decltype(e)>(std::move(e));
c.erase(it);
c.insert( new_elem ); // could throw, but hard to avoid.
}
now the code reads:
auto empl = fooSet.emplace(std::make_shared<Foo>());
mutate(fooSet, empl.first, [&](auto&& elem) {
elem.emplace_back(0);
});
mutate copies an element out, removes the pointer from the set, calls the function on it, then reinserts it back into the fooSet.
Of course in this case it is dumb; we just put it in and now we take it out mutate it and put it back.
But in a more general case it will be less dumb.
Here you add an object and it's stored using its current hash value.
auto empl = fooSet.emplace(std::make_shared<Foo>());
Here you change the hash value:
(*(empl.first))->bar.emplace_back(0);
The set now has an object stored using the old/wrong hash value. If you need to change anything in an object that affects its hash value, you need to extract the object, change it and re-insert it. If all mutable members of the class are used to calculate the hash value, make it a set of <const Foo> instead.
To make future declarations of sets of shared_ptr<const Foo> easier, you may also extend the std namespace with your specializations.
class Foo {
public:
Foo() {}
size_t hash() const {
size_t seed = 0;
for (auto& b : bar) {
boost::hash_combine(seed, b);
}
return seed;
}
bool operator==(Foo const & rhs) const {
return bar == rhs.bar;
}
std::vector<int> bar;
};
namespace std {
template<>
struct hash<Foo> {
size_t operator()(const Foo& foo) const {
return foo.hash();
}
};
template<>
struct hash<std::shared_ptr<const Foo>> {
size_t operator()(const std::shared_ptr<const Foo>& foo) const {
/* A version using std::hash<Foo>:
std::hash<Foo> hasher;
return hasher(*foo);
*/
return foo->hash();
}
};
template<>
struct equal_to<std::shared_ptr<const Foo>> {
bool operator()(std::shared_ptr<const Foo> const & rhs,
std::shared_ptr<const Foo> const & lhs) const {
return *lhs == *rhs;
}
};
}
With that in place, you can simply declare your unordered_set like this:
std::unordered_set<std::shared_ptr<const Foo>> fooSet;
which now is the same as declaring it like this:
std::unordered_set<
std::shared_ptr<const Foo>,
std::hash<std::shared_ptr<const Foo>>,
std::equal_to<std::shared_ptr<const Foo>>
> fooSet;
How can I efficiently tell if an element is at the beginning of an intrusive set or rbtree? I would like to define a simple function prev that returns a pointer to the previous item in a tree, or nullptr if there is no previous item. An analogous next function is easy to write, using iterator_to and comparing to end(). However, there is no equivalent reverse_iterator_to function that would allow me to compare to rend(). Moreover, I specifically do not want to compare to begin(), because that's not constant time in a red-black tree.
One thing that certainly seems to work is decrementing an iterator and comparing it to end(). That works fine with the implementation, but I can find no support for this in the documentation. What's the best way to implement prev in the following minimal working example?
#include <iostream>
#include <string>
#include <boost/intrusive/set.hpp>
using namespace std;
using namespace boost::intrusive;
struct foo : set_base_hook<> {
string name;
foo(const char *n) : name(n) {}
friend bool operator<(const foo &a, const foo &b) { return a.name < b.name; }
};
rbtree<foo> tree;
foo *
prev(foo *fp)
{
auto fi = tree.iterator_to(*fp);
return --fi == tree.end() ? nullptr : &*fi;
}
int
main()
{
tree.insert_equal(*new foo{"a"});
tree.insert_equal(*new foo{"b"});
tree.insert_equal(*new foo{"c"});
for (foo *fp = &*tree.find("c"); fp; fp = prev(fp))
cout << fp->name << endl;
}
Update: Okay, so what I was missing, which is probably what sehe was getting at indirectly, is that in STL begin() is actually guaranteed to be constant-time. So even though a generic red-black tree requires log(n) time to find the minimum element, an STL map doesn't--an STL std::map implementation is required to cache the first element. And I think the point sehe is making is that even though boost is not documented, it is fair to assume that boost::intrusive containers behave sort of like STL containers. Given that assumption, it is perfectly fine to say:
foo *
prev(foo *fp)
{
auto fi = tree.iterator_to(*fp);
return fi == tree.begin() ? nullptr : &*--fi;
}
As the comparison to tree.begin() shouldn't be too costly.
You can get the reverse-iterator from iterator_to.
Also, note that there is rbtree<>::container_from_iterator(iterator it) so you don't have to have a "global" state for your prev function.
You can just create the corresponding reverse_iterator. You'll have to +1 the iterator to get the expected address:
So my take on this would be (bonus: without memory leaks):
Live On Coliru
#include <boost/intrusive/set.hpp>
#include <iostream>
#include <string>
#include <vector>
using namespace boost::intrusive;
struct foo : set_base_hook<> {
std::string name;
foo(char const* n) : name(n) {}
bool operator<(const foo &b) const { return name < b.name; }
};
int main()
{
std::vector<foo> v;
v.emplace_back("a");
v.emplace_back("b");
v.emplace_back("c");
using Tree = rbtree<foo>;
Tree tree;
tree.insert_unique(v.begin(), v.end());
for (auto key : { "a", "b", "c", "missing" })
{
std::cout << "\nusing key '" << key << "': ";
auto start = tree.iterator_to(*tree.find(key));
if (start != tree.end()) {
for (auto it = Tree::reverse_iterator(++start); it != tree.rend(); ++it)
std::cout << it->name << " ";
}
}
}
Which prints
using key 'a': a
using key 'b': b a
using key 'c': c b a
using key 'missing':
I want to store a floating point value for an unordered pair of an integers. I am unable to find any kind of easy to understand tutorials for this. E.g for the unordered pair {i,j} I want to store a floating point value f. How do I insert, store and retrieve values like this?
Simple way to handle unordered int pairs is using std::minmax(i,j) to generate std::pair<int,int>. This way you can implement your storage like this:
std::map<std::pair<int,int>,float> storage;
storage[std::minmax(i,j)] = 0.f;
storage[std::minmax(j,i)] = 1.f; //rewrites storage[(i,j)]
Admittedly proper hashing would give you some extra performance, but there is little harm in postponing this kind of optimization.
Here's some indicative code:
#include <iostream>
#include <unordered_map>
#include <utility>
struct Hasher
{
int operator()(const std::pair<int, int>& p) const
{
return p.first ^ (p.second << 7) ^ (p.second >> 3);
}
};
int main()
{
std::unordered_map<std::pair<int,int>, float, Hasher> m =
{ { {1,3}, 2.3 },
{ {2,3}, 4.234 },
{ {3,5}, -2 },
};
// do a lookup
std::cout << m[std::make_pair(2,3)] << '\n';
// add more data
m[std::make_pair(65,73)] = 1.23;
// output everything (unordered)
for (auto& x : m)
std::cout << x.first.first << ',' << x.first.second
<< ' ' << x.second << '\n';
}
Note that it relies on the convention that you store the unordered pairs with the lower number first (if they're not equal). You might find it convenient to write a support function that takes a pair and returns it in that order, so you can use that function when inserting new values in the map and when using a pair as a key for trying to find a value in the map.
Output:
4.234
3,5 -2
1,3 2.3
65,73 1.23
2,3 4.234
See it on ideone.com. If you want to make a better hash function, just hunt down an implementation of hash_combine (or use boost's) - plenty of questions here on SO explaining how to do that for std::pair<>s.
You implement a type UPair with your requirements and overload ::std::hash (which is the rare occasion that you are allowed to implement something in std).
#include <utility>
#include <unordered_map>
template <typename T>
class UPair {
private:
::std::pair<T,T> p;
public:
UPair(T a, T b) : p(::std::min(a,b),::std::max(a,b)) {
}
UPair(::std::pair<T,T> pair) : p(::std::min(pair.first,pair.second),::std::max(pair.first,pair.second)) {
}
friend bool operator==(UPair const& a, UPair const& b) {
return a.p == b.p;
}
operator ::std::pair<T,T>() const {
return p;
}
};
namespace std {
template <typename T>
struct hash<UPair<T>> {
::std::size_t operator()(UPair<T> const& up) const {
return ::std::hash<::std::size_t>()(
::std::hash<T>()(::std::pair<T,T>(up).first)
) ^
::std::hash<T>()(::std::pair<T,T>(up).second);
// the double hash is there to avoid the likely scenario of having the same value in .first and .second, resulinting in always 0
// that would be a problem for the unordered_map's performance
}
};
}
int main() {
::std::unordered_map<UPair<int>,float> um;
um[UPair<int>(3,7)] = 3.14;
um[UPair<int>(8,7)] = 2.71;
return 10*um[::std::make_pair(7,3)]; // correctly returns 31
}
Is there a convenient way to get the index of the current container entry in a C++11 foreach loop, like enumerate in python:
for idx, obj in enumerate(container):
pass
I could imagine an iterator that can also return the index or similar.
Of course I could have a counter, but often iterators don't give guarantees of the order they iterate over a container.
A good implementation of the feature you are requested can be found here:
https://github.com/ignatz/pythonic
The idea behind is, that you build a wrapper struct with a custom iterator that does the counting. Below is a very minimal exemplary implementation to illustrate the idea:
// Distributed under the terms of the GPLv2 or newer
#include <iostream>
#include <vector>
#include <tuple>
// Wrapper class
template <typename T>
class enumerate_impl
{
public:
// The return value of the operator* of the iterator, this
// is what you will get inside of the for loop
struct item
{
size_t index;
typename T::value_type & item;
};
typedef item value_type;
// Custom iterator with minimal interface
struct iterator
{
iterator(typename T::iterator _it, size_t counter=0) :
it(_it), counter(counter)
{}
iterator operator++()
{
return iterator(++it, ++counter);
}
bool operator!=(iterator other)
{
return it != other.it;
}
typename T::iterator::value_type item()
{
return *it;
}
value_type operator*()
{
return value_type{counter, *it};
}
size_t index()
{
return counter;
}
private:
typename T::iterator it;
size_t counter;
};
enumerate_impl(T & t) : container(t) {}
iterator begin()
{
return iterator(container.begin());
}
iterator end()
{
return iterator(container.end());
}
private:
T & container;
};
// A templated free function allows you to create the wrapper class
// conveniently
template <typename T>
enumerate_impl<T> enumerate(T & t)
{
return enumerate_impl<T>(t);
}
int main()
{
std::vector<int> data = {523, 1, 3};
for (auto x : enumerate(data))
{
std::cout << x.index << ": " << x.item << std::endl;
}
}
What about a simple solution like:
int counter=0;
for (auto &val: container)
{
makeStuff(val, counter);
counter++;
}
You could make a bit more "difficult" to add code after the counter by adding a scope:
int counter=0;
for (auto &val: container)
{{
makeStuff(val, counter);
}counter++;}
As #graham.reeds pointed, normal for loop is also a solution, that could be as fast:
int counter=0;
for (auto it=container.begin(); it!=container.end(); ++it, ++counter)
{
makeStuff(val, counter);
}
And finally, a alternative way using algorithm:
int counter = 0;
std::for_each(container.begin(), container.end(), [&counter](int &val){
makeStuff(val, counter++);
});
Note: the order between range loop and normal loop is guaranteed by the standard 6.5.4. Meaning the counter is able to be coherent with the position in the container.
If you have access to Boost its range adaptors can be used like this:
#include <boost/range/adaptor/indexed.hpp>
using namespace boost::adaptors;
for (auto const& elem : container | indexed(0))
{
std::cout << elem.index() << " - " << elem.value() << '\n';
}
Source (where there are also other examples)
C++17 and structured bindings makes this look OK - certainly better than some ugly mutable lambda with a local [i = 0](Element&) mutable or whatever I've done before admitting that probably not everything should be shoehorned into for_each() et al. - and than other solutions that require a counter with scope outside the for loop.
for (auto [it, end, i] = std::tuple{container.cbegin(), container.cend(), 0};
it != end; ++it, ++i)
{
// something that needs both `it` and `i`ndex
}
You could make this generic, if you use this pattern often enough:
template <typename Container>
auto
its_and_idx(Container&& container)
{
using std::begin, std::end;
return std::tuple{begin(container), end(container), 0};
}
// ...
for (auto [it, end, i] = its_and_idx(foo); it != end; ++it, ++i)
{
// something
}
C++ Standard proposal P2164 proposes to add views::enumerate, which would provide a view of a range giving both reference-to-element and index-of-element to a user iterating it.
We propose a view enumerate whose value type is a struct with 2 members index and value representing respectively the position and value of the elements in the adapted range.
[ . . .]
This feature exists in some form in Python, Rust, Go (backed into the language), and in many C++ libraries: ranges-v3, folly, boost::ranges (indexed).
The existence of this feature or lack thereof is the subject of recurring stackoverflow questions.
Hey, look! We're famous.
If you need the index then a traditional for works perfectly well.
for (int idx=0; idx<num; ++idx)
{
// do stuff
}