I am looking for a container like std::unordered_map that does not use any dynamic allocation. I believe this would be the case for any associate container with a fixed number of keys, or even keys that have to be chosen at compile time.
I am not looking for a constexpr or compile time hash map because I would like to be able to update the values in the map.
Example use case:
FixedCapacityMap<std::string_view, int> fruits {
{"n_apples", 0},
{"n_pairs", 0}
}
fruits["n_apples"] += 1;
fruits["n_pairs"] += 1;
Does anyone know if such a library exists, and if not how something like this could be implemented?
A necessary consequence of the "no dynamic allocation" rule is that the underlying data is embedded in your type, so you need to specify the number of keys as a template parameter as well.
If the keys are known at compile time you can construct a fixed-size hash table over that.
In general, the next best thing is either chained hashing or binary search. Here is a small implementation that uses binary search over a std::array<std::pair<K,V>, N>:
template <class K, class V, size_t N>
class FixedCapacityMap {
public:
using value_type = std::pair<K,V>;
FixedCapacityMap(std::initializer_list<value_type> init) {
assert(init.size() == N);
std::copy(cbegin(init), cend(init), begin(store));
}
V& operator[](const K& key) {
auto it = std::lower_bound(begin(store), end(store), std::pair{key, V()});
if (it == end(store) || it.first != key)
throw std::out_of_range(key);
return it.second;
}
private:
std::array<value_type, N> store;
}
I was able to find a library with this functionality:
https://github.com/serge-sans-paille/frozen
It allows for constexpr and constinit ordered and unordered maps with (newly added by me) the ability to update values in runtime.
Related
I want to have a collection of items which are searchable based on a key (an unsigned value), but I want the elements to be sorted based on a different criteria i.e. the last accessed time (Which is part of the value).
How can I achieve this in C++? I can sort them separately on demand, but can I create the container itself such that sorting happens automatically?
Are there ready made containers (in boost) that can have similar feature built into them?
You could probably implement something of this kind, using std::list and std::unordered_map pointing to each other.
#include <list>
#include <unordered_map>
template <typename A>
struct Cache {
using key = unsigned;
struct Composite {
Composite(A &_a, std::list<key>::iterator _it) : a(_a), it(_it) {}
A &a;
std::list<key>::iterator it;
};
std::unordered_map<key, Composite> map;
std::list <key> list;
void insert(key k, A &a) { // Assuming inserting contains accessing
list.emplace_front(k);
map[k] = Composite(a, list.front());
}
A &operator[](key k) {
list.erase(map[k].it);
list.emplace_front(k);
return map[k].a;
}
A &last_accessed() { // or whatever else you wish to implement
assert(!list.empty());
return map[list.front()].a;
}
};
This solution is optimized for keeping track of which element was accessed last. If you want to sort given a different attribute, you can follow a similar process but use an std::set to store the values with your comparison function, and then iterators to that from an std::unordered_map hashed with a key of your choice.
Say we have a map with larger objects and an index value. The index value is also part of the larger object.
What I would like to know is whether it is possible to replace the map with a set, extracting the index value.
It is fairly easy to create a set that sorts on a functor comparing two larger objects by extracting the index value.
Which leaves searching by index value, which is not supported by default in a set, I think.
I was thinking of using std::find_if, but I believe that searches linearly, ignoring the fact we have set.
Then I thought of using std::binary_search with a functor comparing the larger object and the value, but I believe that it doesn't work in this case as it wouldn't make use of the structure and would use traversal as it doesn't have a random access iterator. Is this correct? Or are there overloads which correctly handle this call on a set?
And then finally I was thinking of using a boost::containter::flat_set, as this has an underlying vector and thus presumably should be able to work well with std::binary_search?
But maybe there is an all together easier way to do this?
Before you answer just use a map where a map ought to be used - I am actually using a vector that is manually sorted (well std::lower_bound) and was thinking of replacing it with boost::containter::flat_set, but it doesn't seem to be easily possible to do so, so I might just stick with the vector.
C++14 will introduce the ability to lookup by a key that does not require the construction of the entire stored object. This can be used as follows:
#include <set>
#include <iostream>
struct StringRef {
StringRef(const std::string& s):x(&s[0]) { }
StringRef(const char *s):x(s) { std::cout << "works: " << s << std::endl; }
const char *x;
};
struct Object {
long long data;
std::size_t index;
};
struct ObjectIndexer {
ObjectIndexer(Object const& o) : index(o.index) {}
ObjectIndexer(std::size_t index) : index(index) {}
std::size_t index;
};
struct ObjComp {
bool operator()(ObjectIndexer a, ObjectIndexer b) const {
return a.index < b.index;
}
typedef void is_transparent; //Allows the comparison with non-Object types.
};
int main() {
std::set<Object, ObjComp> stuff;
stuff.insert(Object{135, 1});
std::cout << stuff.find(ObjectIndexer(1))->data << "\n";
}
More generally, these sorts of problems where there are multiple ways of indexing your data can be solved using Boost.MultiIndex.
Use boost::intrusive::set which can utilize the object's index value directly. It has a find(const KeyType & key, KeyValueCompare comp) function with logarithmic complexity. There are also other set types based on splay trees, AVL trees, scapegoat trees etc. which may perform better depending on your requirements.
If you add the following to your contained object type:
less than operator that only compares the object indices
equality operator that only compares the object indices
a constructor that takes your index type and initializes a dummy object with that value for the index
then you can pass your index type to find, lower_bound, equal_range, etc... and it will act the way you want. When you pass your index to the set's (or flat_set's) find methods it will construct a dummy object of the contained type to use for the comparisons.
Now if your object is really big, or expensive to construct, this might not be the way you want to go.
I'm using an std::unordered_map<key,value> in my implementation. i will be using any of the STL containers as the key. I was wondering if it is possible to create a generic hash function for any container being used.
This question in SO offers generic print function for all STL containers. While you can have that, why cant you have something like a Hash function that defines everything ? And yeah, a big concern is also that it needs to fast and efficient.
I was considering doing a simple hash function that converts the values of the key to a size_t and do a simple function like this.
Can this be done ?
PS : Please don't use boost libraries. Thanks.
We can get an answer by mimicking Boost and combining hashes.
Warning: Combining hashes, i.e. computing a hash of many things from many hashes of the things, is not a good idea generally, since the resulting hash function is not "good" in the statistical sense. A proper hash of many things should be build from the entire raw data of all the constituents, not from intermediate hashes. But there currently isn't a good standard way of doing this.
Anyway:
First off, we need the hash_combine function. For reasons beyond my understanding it's not been included in the standard library, but it's the centrepiece for everything else:
template <class T>
inline void hash_combine(std::size_t & seed, const T & v)
{
std::hash<T> hasher;
seed ^= hasher(v) + 0x9e3779b9 + (seed << 6) + (seed >> 2);
}
Using this, we can hash everything that's made up from hashable elements, in particular pairs and tuples (exercise for the reader).
However, we can also use this to hash containers by hashing their elements. This is precisely what Boost's "range hash" does, but it's straight-forward to make that yourself by using the combine function.
Once you're done writing your range hasher, just specialize std::hash and you're good to go:
namespace std
{
template <typename T, class Comp, class Alloc>
struct hash<std::set<T, Comp, Alloc>>
{
inline std::size_t operator()(const std::set<T, Comp, Alloc> & s) const
{
return my_range_hash(s.begin(), s.end());
}
};
/* ... ditto for other containers */
}
If you want to mimic the pretty printer, you could even do something more extreme and specialize std::hash for all containers, but I'd probably be more careful with that and make an explicit hash object for containers:
template <typename C> struct ContainerHasher
{
typedef typename C::value_type value_type;
inline size_t operator()(const C & c) const
{
size_t seed = 0;
for (typename C::const_iterator it = c.begin(), end = c.end(); it != end; ++it)
{
hash_combine<value_type>(seed, *it);
}
return seed;
}
};
Usage:
std::unordered_map<std::set<int>, std::string, ContainerHasher<std::set<int>>> x;
I know that STL has a HashMap API, but I cannot find any good and thorough documentation with good examples regarding this.
Any good examples will be appreciated.
The standard library includes the ordered and the unordered map (std::map and std::unordered_map) containers. In an ordered map (std::map) the elements are sorted by the key, insert and access is in O(log n). Usually the standard library internally uses red black trees for ordered maps. But this is just an implementation detail. In an unordered map (std::unordered_map) insert and access is in O(1). It is just another name for a hashtable.
An example with (ordered) std::map:
#include <map>
#include <iostream>
#include <cassert>
int main(int argc, char **argv)
{
std::map<std::string, int> m;
m["hello"] = 23;
// check if key is present
if (m.find("world") != m.end())
std::cout << "map contains key world!\n";
// retrieve
std::cout << m["hello"] << '\n';
std::map<std::string, int>::iterator i = m.find("hello");
assert(i != m.end());
std::cout << "Key: " << i->first << " Value: " << i->second << '\n';
return 0;
}
Output:
23
Key: hello Value: 23
If you need ordering in your container and are fine with the O(log n) runtime then just use std::map.
Otherwise, if you really need a hash-table (O(1) insert/access), check out std::unordered_map, which has a similar to std::map API (e.g. in the above example you just have to search and replace map with unordered_map).
The unordered_map container was introduced with the C++11 standard revision. Thus, depending on your compiler, you have to enable C++11 features (e.g. when using GCC 4.8 you have to add -std=c++11 to the CXXFLAGS).
Even before the C++11 release GCC supported unordered_map - in the namespace std::tr1. Thus, for old GCC compilers you can try to use it like this:
#include <tr1/unordered_map>
std::tr1::unordered_map<std::string, int> m;
It is also part of boost, i.e. you can use the corresponding boost-header for better portability.
A hash_map is an older, unstandardized version of what for standardization purposes is called an unordered_map (originally in TR1, and included in the standard since C++11). As the name implies, it's different from std::map primarily in being unordered -- if, for example, you iterate through a map from begin() to end(), you get items in order by key1, but if you iterate through an unordered_map from begin() to end(), you get items in a more or less arbitrary order.
An unordered_map is normally expected to have constant complexity. That is, an insertion, lookup, etc., typically takes essentially a fixed amount of time, regardless of how many items are in the table. An std::map has complexity that's logarithmic on the number of items being stored -- which means the time to insert or retrieve an item grows, but quite slowly, as the map grows larger. For example, if it takes 1 microsecond to lookup one of 1 million items, then you can expect it to take around 2 microseconds to lookup one of 2 million items, 3 microseconds for one of 4 million items, 4 microseconds for one of 8 million items, etc.
From a practical viewpoint, that's not really the whole story though. By nature, a simple hash table has a fixed size. Adapting it to the variable-size requirements for a general purpose container is somewhat non-trivial. As a result, operations that (potentially) grow the table (e.g., insertion) are potentially relatively slow (that is, most are fairly fast, but periodically one will be much slower). Lookups, which cannot change the size of the table, are generally much faster. As a result, most hash-based tables tend to be at their best when you do a lot of lookups compared to the number of insertions. For situations where you insert a lot of data, then iterate through the table once to retrieve results (e.g., counting the number of unique words in a file) chances are that an std::map will be just as fast, and quite possibly even faster (but, again, the computational complexity is different, so that can also depend on the number of unique words in the file).
1 Where the order is defined by the third template parameter when you create the map, std::less<T> by default.
Here's a more complete and flexible example that doesn't omit necessary includes to generate compilation errors:
#include <iostream>
#include <unordered_map>
class Hashtable {
std::unordered_map<const void *, const void *> htmap;
public:
void put(const void *key, const void *value) {
htmap[key] = value;
}
const void *get(const void *key) {
return htmap[key];
}
};
int main() {
Hashtable ht;
ht.put("Bob", "Dylan");
int one = 1;
ht.put("one", &one);
std::cout << (char *)ht.get("Bob") << "; " << *(int *)ht.get("one");
}
Still not particularly useful for keys, unless they are predefined as pointers, because a matching value won't do! (However, since I normally use strings for keys, substituting "string" for "const void *" in the declaration of the key should resolve this problem.)
Evidence that std::unordered_map uses a hash map in GCC stdlibc++ 6.4
This was mentioned at: https://stackoverflow.com/a/3578247/895245 but in the following answer: What data structure is inside std::map in C++? I have given further evidence of such for the GCC stdlibc++ 6.4 implementation by:
GDB step debugging into the class
performance characteristic analysis
Here is a preview of the performance characteristic graph described in that answer:
How to use a custom class and hash function with unordered_map
This answer nails it: C++ unordered_map using a custom class type as the key
Excerpt: equality:
struct Key
{
std::string first;
std::string second;
int third;
bool operator==(const Key &other) const
{ return (first == other.first
&& second == other.second
&& third == other.third);
}
};
Hash function:
namespace std {
template <>
struct hash<Key>
{
std::size_t operator()(const Key& k) const
{
using std::size_t;
using std::hash;
using std::string;
// Compute individual hash values for first,
// second and third and combine them using XOR
// and bit shifting:
return ((hash<string>()(k.first)
^ (hash<string>()(k.second) << 1)) >> 1)
^ (hash<int>()(k.third) << 1);
}
};
}
For those of us trying to figure out how to hash our own classes whilst still using the standard template, there is a simple solution:
In your class you need to define an equality operator overload ==. If you don't know how to do this, GeeksforGeeks has a great tutorial https://www.geeksforgeeks.org/operator-overloading-c/
Under the standard namespace, declare a template struct called hash with your classname as the type (see below). I found a great blogpost that also shows an example of calculating hashes using XOR and bitshifting, but that's outside the scope of this question, but it also includes detailed instructions on how to accomplish using hash functions as well https://prateekvjoshi.com/2014/06/05/using-hash-function-in-c-for-user-defined-classes/
namespace std {
template<>
struct hash<my_type> {
size_t operator()(const my_type& k) {
// Do your hash function here
...
}
};
}
So then to implement a hashtable using your new hash function, you just have to create a std::map or std::unordered_map just like you would normally do and use my_type as the key, the standard library will automatically use the hash function you defined before (in step 2) to hash your keys.
#include <unordered_map>
int main() {
std::unordered_map<my_type, other_type> my_map;
}
This is my code
map<string,int> persons;
persons["B"] = 123;
persons["A"] = 321;
for(map<string,int>::iterator i = persons.begin();
i!=persons.end();
++i)
{
cout<< (*i).first << ":"<<(*i).second<<endl;
}
Expected output:
B:123
A:321
But output it gives is:
A:321
B:123
I want it to maintain the order in which keys and values were inserted in the map<string,int>.
Is it possible? Or should I use some other STL data structure? Which one?
There is no standard container that does directly what you want. The obvious container to use if you want to maintain insertion order is a vector. If you also need look up by string, use a vector AND a map. The map would in general be of string to vector index, but as your data is already integers you might just want to duplicate it, depending on your use case.
Like Matthieu has said in another answer, the Boost.MultiIndex library seems the right choice for what you want. However, this library can be a little tough to use at the beginning especially if you don't have a lot of experience with C++. Here is how you would use the library to solve the exact problem in the code of your question:
struct person {
std::string name;
int id;
person(std::string const & name, int id)
: name(name), id(id) {
}
};
int main() {
using namespace::boost::multi_index;
using namespace std;
// define a multi_index_container with a list-like index and an ordered index
typedef multi_index_container<
person, // The type of the elements stored
indexed_by< // The indices that our container will support
sequenced<>, // list-like index
ordered_unique<member<person, string,
&person::name> > // map-like index (sorted by name)
>
> person_container;
// Create our container and add some people
person_container persons;
persons.push_back(person("B", 123));
persons.push_back(person("C", 224));
persons.push_back(person("A", 321));
// Typedefs for the sequence index and the ordered index
enum { Seq, Ord };
typedef person_container::nth_index<Seq>::type persons_seq_index;
typedef person_container::nth_index<Ord>::type persons_ord_index;
// Let's test the sequence index
persons_seq_index & seq_index = persons.get<Seq>();
for(persons_seq_index::iterator it = seq_index.begin(),
e = seq_index.end(); it != e; ++it)
cout << it->name << ":"<< it->id << endl;
cout << "\n";
// And now the ordered index
persons_ord_index & ord_index = persons.get<Ord>();
for(persons_ord_index::iterator it = ord_index.begin(),
e = ord_index.end(); it != e; ++it)
cout << it->name << ":"<< it->id << endl;
cout << "\n";
// Thanks to the ordered index we have fast lookup by name:
std::cout << "The id of B is: " << ord_index.find("B")->id << "\n";
}
Which produces the following output:
B:123
C:224
A:321
A:321
B:123
C:224
The id of B is: 123
Map is definitely not right for you:
"Internally, the elements in the map are sorted from lower to higher key value following a specific strict weak ordering criterion set on construction."
Quote taken from here.
Unfortunately there is no unordered associative container in the STL, so either you use a nonassociative one like vector, or write your own :-(
I had the same problem every once in a while and here is my solution: https://github.com/nlohmann/fifo_map. It's a header-only C++11 solution and can be used as drop-in replacement for a std::map.
For your example, it can be used as follows:
#include "fifo_map.hpp"
#include <string>
#include <iostream>
using nlohmann::fifo_map;
int main()
{
fifo_map<std::string,int> persons;
persons["B"] = 123;
persons["A"] = 321;
for(fifo_map<std::string,int>::iterator i = persons.begin();
i!=persons.end();
++i)
{
std::cout<< (*i).first << ":"<<(*i).second << std::endl;
}
}
The output is then
B:123
A:321
Besides Neil's recommendation of a combined vector+map if you need both to keep the insertion order and the ability to search by key, you can also consider using boost multi index libraries, that provide for containers addressable in more than one way.
maps and sets are meant to impose a strict weak ordering upon the data. Strick weak ordering maintains that no entries are equavalent (different to being equal).
You need to provide a functor that the map/set may use to perform a<b. With this functor the map/set sorts its items (in the STL from GCC it uses a red-black tree). It determines weather two items are equavalent if !a<b && !b<a -- the equavelence test.
The functor looks like follows:
template <class T>
struct less : binary_function<T,T,bool> {
bool operator() (const T& a, const T& b) const {
return a < b;
}
};
If you can provide a function that tells the STL how to order things then the map and set can do what you want. For example
template<typename T>
struct ItemHolder
{
int insertCount;
T item;
};
You can then easily write a functor to order by insertCount. If your implementation uses red-black trees your underlying data will remain balanced -- however you will get a lot of re-balancing since your data will be generated based on incremental ordering (vs. Random) -- and in this case a list with push_back would be better. However you cannot access data by key as fast as you would with a map/set.
If you want to sort by string -- provide the functor to search by string, using the insertCount you could potentiall work backwards. If you want to search by both you can have two maps.
map<insertcount, string> x; // auxhilary key
map<string, item> y; //primary key
I use this strategy often -- however I have never placed it in code that is run often. I'm considering boost::bimap.
Well, there is no STL container which actually does what you wish, but there are possibilities.
1. STL
By default, use a vector. Here it would mean:
struct Entry { std::string name; int it; };
typedef std::vector<Entry> container_type;
If you wish to search by string, you always have the find algorithm at your disposal.
class ByName: std::unary_function<Entry,bool>
{
public:
ByName(const std::string& name): m_name(name) {}
bool operator()(const Entry& entry) const { return entry.name == m_name; }
private:
std::string m_name;
};
// Use like this:
container_type myContainer;
container_type::iterator it =
std::find(myContainer.begin(), myContainer.end(), ByName("A"));
2. Boost.MultiIndex
This seems way overkill, but you can always check it out here.
It allows you to create ONE storage container, accessible via various indexes of various styles, all maintained for you (almost) magically.
Rather than using one container (std::map) to reference a storage container (std::vector) with all the synchro issues it causes... you're better off using Boost.
For preserving all the time complexity constrains you need map + list:
struct Entry
{
string key;
int val;
};
typedef list<Entry> MyList;
typedef MyList::iterator Iter;
typedef map<string, Iter> MyMap;
MyList l;
MyMap m;
int find(string key)
{
Iter it = m[key]; // O(log n)
Entry e = *it;
return e.val;
}
void put(string key, int val)
{
Entry e;
e.key = key;
e.val = val;
Iter it = l.insert(l.end(), e); // O(1)
m[key] = it; // O(log n)
}
void erase(string key)
{
Iter it = m[key]; // O(log n)
l.erase(it); // O(1)
m.erase(key); // O(log n)
}
void printAll()
{
for (Iter it = l.begin(); it != l.end(); it++)
{
cout<< it->key << ":"<< it->val << endl;
}
}
Enjoy
You could use a vector of pairs, it is almost the same as unsorted map container
std::vector<std::pair<T, U> > unsorted_map;
Use a vector. It gives you complete control over ordering.
I also think Map is not the way to go. The keys in a Map form a Set; a single key can occur only once. During an insert in the map the map must search for the key, to ensure it does not exist or to update the value of that key. For this it is important (performance wise) that the keys, and thus the entries, have some kind of ordering. As such a Map with insert ordering would be highly inefficient on inserts and retrieving entries.
Another problem would be if you use the same key twice; should the first or the last entry be preserved, and should it update the insert order or not?
Therefore I suggest you go with Neils suggestion, a vector for insert-time ordering and a Map for key-based searching.
Yes, the map container is not for you.
As you asked, you need the following code instead:
struct myClass {
std::string stringValue;
int intValue;
myClass( const std::string& sVal, const int& iVal ):
stringValue( sVal ),
intValue( iVal) {}
};
std::vector<myClass> persons;
persons.push_back( myClass( "B", 123 ));
persons.push_back( myClass( "A", 321 ));
for(std::vector<myClass>::iterator i = persons.begin();
i!=persons.end();
++i)
{
std::cout << (*i).stringValue << ":" << (*i).intValue << std::endl;
}
Here the output is unsorted as expected.
Map is ordered collection (second parametr in template is a order functor), as set. If you want to pop elements in that sequenses as pushd you should use deque or list or vector.
In order to do what they do and be efficient at it, maps use hash tables and sorting. Therefore, you would use a map if you're willing to give up memory of insertion order to gain the convenience and performance of looking up by key.
If you need the insertion order stored, one way would be to create a new type that pairs the value you're storing with the order you're storing it (you would need to write code to keep track of the order). You would then use a map of string to this new type for storage. When you perform a look up using a key, you can also retrieve the insertion order and then sort your values based on insertion order.
One more thing: If you're using a map, be aware of the fact that testing if persons["C"] exists (after you've only inserted A and B) will actually insert a key value pair into your map.
Instead of map you can use the pair function with a vector!
ex:
vector<::pair<unsigned,string>> myvec;
myvec.push_back(::pair<unsigned,string>(1,"a"));
myvec.push_back(::pair<unsigned,string>(5,"b"));
myvec.push_back(::pair<unsigned,string>(3,"aa"));`
Output:
myvec[0]=(1,"a"); myvec[1]=(5,"b"); myvec[2]=(3,"aa");
or another ex:
vector<::pair<string,unsigned>> myvec2;
myvec2.push_back(::pair<string,unsigned>("aa",1));
myvec2.push_back(::pair<string,unsigned>("a",3));
myvec2.push_back(::pair<string,unsigned>("ab",2));
Output: myvec2[0]=("aa",1); myvec2[1]=("a",3); myvec2[2]=("ab",2);
Hope this can help someone else in the future who was looking for non sorted maps like me!
struct Compare : public binary_function<int,int,bool> {
bool operator() (int a, int b) {return true;}
};
Use this to get all the elements of a map in the reverse order in which you entered (i.e.: the first entered element will be the last and the last entered element will be the first). Not as good as the same order but it might serve your purpose with a little inconvenience.
Use a Map along with a vector of iterators as you insert in Map. (Map iterators are guaranteed not to be invalidated)
In the code below I am using Set
set<string> myset;
vector<set<string>::iterator> vec;
void printNonDuplicates(){
vector<set<string>::iterator>::iterator vecIter;
for(vecIter = vec.begin();vecIter!=vec.end();vecIter++){
cout<<(*vecIter)->c_str()<<endl;
}
}
void insertSet(string str){
pair<set<string>::iterator,bool> ret;
ret = myset.insert(str);
if(ret.second)
vec.push_back(ret.first);
}
If you don't want to use boost::multi_index, I have put a proof of concept class template up for review here:
https://codereview.stackexchange.com/questions/233157/wrapper-class-template-for-stdmap-stdlist-to-provide-a-sequencedmap-which
using std::map<KT,VT> and std::list<OT*> which uses pointers to maintain the order.
It will take O(n) linear time for the delete because it needs to search the whole list for the right pointer. To avoid that would need another cross reference in the map.
I'd vote for typedef std::vector< std::pair< std::string, int > > UnsortedMap;
Assignment looks a bit different, but your loop remains exactly as it is now.
There is std::unordered_map that you can check out. From first view, it looks like it might solve your problem.