Number of buckets of std::unordered_map grows unexpectedly - c++

Question

I'd like to use std::unordered map as a software cache with a limited capacity. Namely, I set the number of buckets in the constructor (doesn't mind that it might become actually larger) and insert new data (if not already there) if the following way:
If the bucket where the data belong is not empty, I replace its node with the inserted data (by C++17 extraction-insertion pattern).
Otherwise, I simply insert data.
The minimal example that simulates this approach is as follows:
#include <iostream>
#include <unordered_map>
std::unordered_map<int, int> m(2);
void insert(int a) {
auto idx = m.bucket(a);
if (m.bucket_size(idx) > 0) {
const auto& key = m.begin(idx)->first;
auto nh = m.extract(key);
nh.key() = a;
nh.mapped() = a;
m.insert(std::move(nh));
}
else
m.insert({a, a});
}
int main() {
for (int i = 0; i < 1000; i++) {
auto bc1 = m.bucket_count();
insert(i);
auto bc2 = m.bucket_count();
if (bc1 != bc2) std::cerr << bc2 << std::endl;
}
}
The problem is, that with GCC 8.1 (that is available for me in the production environment), the bucket count is not fixed and grows instead; the output reads:
7
17
37
79
167
337
709
1493
Live demo: https://wandbox.org/permlink/c8nnEU52NsWarmuD
Updated info: the bucket count is always increased in the else branch: https://wandbox.org/permlink/p2JaHNP5008LGIpL.
However, when I use GCC 9.1 or Clang 8.0, the bucket count remains fixed (no output is printed in the error stream).
My question is whether this is a bug in the older version of libstdc++, or my approach isn't correct and I cannot use std::unordered_map this way.
Moreover, I found out that the problem disappears when I set the max_load_factor to some very high number, such as
m.max_load_factor(1e20f);
But I don't want to rely on such a "fragile" solution in the production code.

Answer 1

Unfortunately the problem you're having appears to be a bug in older implementations of std::unordered_map. This problem disappears in g++-9, but if you're limited to g++-8, I recommend rolling your own hash-cache.
Rolling our own hash-cache
Thankfully, the type of cache you want to write is actually simpler than writing a full hash-table, mainly because it's fine if values occasionally get dropped from the table. To see how difficult it'd be, I wrote my own version.
So what's it look like?
Let's say you have an expensive function you want to cache. The fibbonacci function, when written using the recursive implementation, is notorious for requiring exponential time in terms of the input because it calls itself.
// Uncached version
long long fib(int n) {
if(n <= 1)
return n;
else
return fib(n - 1) + fib(n - 2);
}
Let's transform it to the cached version, using the Cache class which I'll show you in a moment. We actually only need to add one line of code to the function:
// Cached version; much faster
long long fib(int n) {
static auto fib = Cache(::fib, 1024); // fib now refers to the cache, instead of the enclosing function
if(n <= 1)
return n;
else
return fib(n - 1) + fib(n - 2); // Invokes cache
}
The first argument is the function you want to cache (in this case, fib itself), and the second argument is the capacity. For n == 40, the uncached version takes 487,000 microseconds to run. And the cached version? Just 16 microseconds to initialize the cache, fill it, and return the value! You can see it run here.. After that initial access, retrieving a stored value from the cache takes around 6 nanoseconds.
(If Compiler Explorer shows the assembly instead of the output, click on the tab next to it.)
How would we write this Cache class?
Here's a compact implementation of it. The Cache class stores the following
An array of bools, which keeps track of which buckets have values
An array of keys
An array of values
A bitmask & hash function
A function to calculate values that aren't in the table
In order to calculate a value, we:
Check if the key is stored in the table
If the key is not in the table, calculate and store the value
Return the stored value
Here's the code:
template<class Key, class Value, class Func>
class Cache {
static size_t calc_mask(size_t min_cap) {
size_t actual_cap = 1;
while(actual_cap <= min_cap) {
actual_cap *= 2;
}
return actual_cap - 1;
}
size_t mask = 0;
std::unique_ptr<bool[]> isEmpty;
std::unique_ptr<Key[]> keys;
std::unique_ptr<Value[]> values;
std::hash<Key> hash;
Func func;
public:
Cache(Cache const& c)
: mask(c.mask)
, isEmpty(new bool[mask + 1])
, keys(new Key[mask + 1])
, values(new Value[mask + 1])
, hash(c.hash)
, func(c.func)
{
std::copy_n(c.isEmpty.get(), capacity(), isEmpty.get());
std::copy_n(c.keys.get(), capacity(), keys.get());
std::copy_n(c.values.get(), capacity(), values.get());
}
Cache(Cache&&) = default;
Cache(Func func, size_t cap)
: mask(calc_mask(cap))
, isEmpty(new bool[mask + 1])
, keys(new Key[mask + 1])
, values(new Value[mask + 1])
, hash()
, func(func) {
std::fill_n(isEmpty.get(), capacity(), true);
}
Cache(Func func, size_t cap, std::hash<Key> const& hash)
: mask(calc_mask(cap))
, isEmpty(new bool[mask + 1])
, keys(new Key[mask + 1])
, values(new Value[mask + 1])
, hash(hash)
, func(func) {
std::fill_n(isEmpty.get(), capacity(), true);
}
Value operator()(Key const& key) const {
size_t index = hash(key) & mask;
auto& value = values[index];
auto& old_key = keys[index];
if(isEmpty[index] || old_key != key) {
old_key = key;
value = func(key);
isEmpty[index] = false;
}
return value;
}
size_t capacity() const {
return mask + 1;
}
};
template<class Key, class Value>
Cache(Value(*)(Key), size_t) -> Cache<Key, Value, Value(*)(Key)>;