In my application, I have a CMap of CString values. After adding the elements in the Map, if I retrieve the elements in some other place, am not getting the elements in the order of insertion.Suppose I retrieve the third element, I get the fifth like that. Is it a behavior of CMap. Why this happens?
You asked for "why", so here goes:
A Map provides for an efficient way to retrieve values by key. It does this by using a clever datastructure that is faster for this than a list or an array would be (where you have to search through the whole list before you know if an element is in there or not). There are trade-offs, such as increased memory usage, and the inability to do some other things (such as knowing in which order things were inserted).
There are two common ways to implement this
a hashmap, which puts keys into buckets by hash value.
a treemap, which arranges keys into a binary tree, according to how they are sorted
You can iterate over maps, but it will be according to how they are stored internally, either in key order (treemap) or completely unpredictable (hashmap). Your CMap seems to be a hashmap.
Either way, insertion order is not preserved. If you want that, you need an extra datastructure (such as a list).
How about read documentation to CMap?
http://msdn.microsoft.com/ru-ru/library/s897094z%28v=vs.71%29.aspx
It's unordered map really. How you retrieve elements? By GetStartPosition and GetNextAssoc? http://msdn.microsoft.com/ru-ru/library/d82fyybt%28v=vs.71%29.aspx read Remark here
Remarks
The iteration sequence is not predictable; therefore, the "first element in the map" has no special significance.
CMap is a dictionary collection class that maps unique keys to values. Once you have inserted a key-value pair (element) into the map, you can efficiently retrieve or delete the pair using the key to access it. You can also iterate over all the elements in the map.
Related
When iterating over an unordered map of values, std::unordered_map<Foo, Bar>, it will use an iterator pointing to values instd::pair<Foo, Bar>. This makes it seem like std::unordered_map stores its values internally as std::pairs of values.
If I'm not mistaken, std::unordered_map works by hashing the key and using that for lookups. So, if the internal structure is something like a list of pairs, how does it know which pair has the hashed key? Wouldn't it need to hash the whole pair, value included?
If the internal structure does not hold pairs, does calling std::unordered_map::begin() then create a std::pair object using the data in the map? Then how does modifying the data in the pair also modify the data in the actual map itself?
Thank you.
Let's pretend that the map is really a single vector:
std::vector<std::pair<Foo, Bar>> container;
It's easy to visualize that iterating over this container iterates over std::pairs of two classes.
The real unordered map works the same way, except instead of a vector the map stores std::pair<Foo, Bar>s in a hash table, with additional pointers that stitch the whole hash table together, that are not exposed via the map's iterators.
The way that associative containers, like maps, get loosely explained and described -- as a key/value lookup -- makes it sound like in the maps keys and values are stored separately: here's a hash table of keys, and each key points to its corresponding value. However that is not the case. The key and its value are kept together, tightly coupled in discrete std::pair objects, and the map's internal structure arranges them in a hashed table that the iterator knows how to iterate over.
So, if the internal structure is something like a list of pairs, how does it know which pair has the hashed key?
Neither. An unordered map can be loosely described as a:
std::vector<std::list<std::pair<Key, Value>>>
A key's hash is the index in the vector. All Keys in the ith position in this vector have the same hash value. All keys with the same hash are stored in a single linked list.
I would like to implement a map whose number of elements never exceeds a certain limit L. When the L+1-th element is inserted, the oldest entry should be removed from the map to empty the space.
I found something similar: Data Structure for Queue using Map Implementations in Java with Size limit of 5. There it is suggested to use a linked hash map, i.e., a hash map that also keeps a linked list of all elements. Unfortunately, that is for java, and I need a solution for C++. I could not find anything like this in the standard library nor in the boost libraries.
Same story here: Add and remove from MAP with limited size
A possible solution for C++ is given here, but it does not address my questions below: C++ how to mix a map with a circular buffer?
I would implement it in a very similar way to what is described there. An hash map to store the key-value pairs and a linked list, or a double-ended queue, of keys to keep the index of the entries. To insert a new value I would add it to the hash map and its key at the end of the index; if the size of the has at this point exceeds the limit, I would pop the first element of the index and remove the entry with that key from the has. Simple, same complexity as adding to the hash map.
Removing an entry requires an iteration over the index to remove the key from there, which has linear complexity for both linked lists and double-ended queue. (Double-ended queues also have the disadvantage that removing an element has itself linear complexity.) So it looks like the remove operation on such a data structure does not preserve the complexity as the underlying has map.
The question is: Is this increase in complexity necessary, or there are some clever ways to implement a limited map data structure so that both insertion and removal keep the same complexity?
Ooops, just posted and immediately realized something important. The size of the index is also limited. If that limit is constant, then the complexity of iterating over it can also be considered constant.
Well, the limit gives an upper bound to the cost of the removal operation.
If the limit is very high one might still prefer a solution that does not involve a linear iteration over the index.
I would still use an associative container to have direct access and a sequential one to allow easy removal of the older item. Let us look at the required access methods:
access to an element given its key => ok, the associative container allows direct access
add a new key-value pair
if the map is not full it is easy: push_back on the sequence container, and simple addition to the associative one
if the map is full, above action must happen, but the oldest element must be removed => front on the sequence container will give that element, and pop_front and erase will remove it, provided the key is contained in the sequence container
remove an element given by its key => trivial to remove from an associative container, but only list allows for constant time removal of an element provided you have an iterator on it. The good news is that removing or inserting an element in a list does not invalidate iterators pointing on other elements.
As you did not give any requirement for keeping keys sorted, I would use an unordered_map for the associative container and a list for the sequence one. The additional requirements is that the list must contain the key, and that the unordered_map must contain an iterator to its corresponding element in the list. The value can be in either container. As I assume that the major access will be the direct one, I would store the value in the map.
It boils down to:
a list<K> to allow identification of the oldest key
an unordered_map<K, pair<V, list<K>::iterator>>
It doubles the storage for key and adds an additional iterator. But keys are expected not to be too big and list::iterator normally contains little more than a pointer: this changes a small amount of memory for speed.
That should be enough to provide constant time
key access
insertion of a new item
key removal of an item
You may want to take a look at Boost.MultiIndex MRU example.
I inserted the elements to the unordered_map with this code:
myMap.insert(std::make_pair("A", 10));
myMap.insert(std::make_pair("B", 11));
myMap.insert(std::make_pair("C", 12));
myMap.insert(std::make_pair("D", 13));
But when I used this command to print the keys
for (const auto i : myMap)
{
cout << i.first << std::endl;
}
they are not in the same order as I inserted them.
Is it possible to keep the order?
No, it is not possible.
Usage of std::unordered_map doesn't give you any guarantee on element order.
If you want to keep elements sorted by map keys (as seems from your example) you should use std::map.
If you need to keep list of ordered pairs you can use std::vector<std::pair<std::string,int>>.
Not with an unordered associative data structure. However, other data structures preserve order, such as std::map which keeps the data sorted by their keys. If you search Stackoverflow a little, you will find many solutions for a data structure with fast key-based lookup and ordered access, e.g. using boost::multi_index.
If it is just about adding values to a container, and taking them out in the order of insertion, then you can go with something that models a queue, e.g. std::dequeue. Just push_back to add a new value, and pop_front to remove the oldest value. If there is no need to remove the values from the container then just go with a std::vector.
Remarkably common request without too many clean,simple solutions. But here they are:
The big library: https://www.boost.org/doc/libs/1_71_0/libs/multi_index/doc/tutorial/index.html Use unique index std::string (or is it char?) and sequenced index to retain the order.
Write it yourself using 2 STL containers: Use a combination of std::unordered_map (or std::map if you want sorted key order traverse as well) and a vector to retain the sequenced order. There are several ways to set this up, depending on the types on your keys/values. Usually keys in the map and the values in the vector. then the map is map<"key_type",int> where the int points to the element in the vector and therefore the value.
I might have a play with a simple template for a wrapper to tie the 2 STL containers together and post it here later...
I have put a proof of concept up for review here:
I went with std::list to store the order in the end, because I wanted efficient delete. But You might choose std::vector if you wanted random access by insertion order.
I used a a list of Pair pointers to avoid duplicate key storage.
I have a map which stores <int, char *>. Now I want to retrieve the elements in the order they have been inserted. std::map returns the elements ordered by the key instead. Is it even possible?
If you are not concerned about the order based on the int (IOW you only need the order of insertion and you are not concerned with the normal key access), simply change it to a vector<pair<int, char*>>, which is by definition ordered by the insertion order (assuming you only insert at the end).
If you want to have two indices simultaneously, you need, well, Boost.MultiIndex or something similar. You'd probably need to keep a separate variable that would only count upwards (would be a steady counter), though, because you could use .size()+1 as new "insertion time key" only if you never erased anything from your map.
Now I want to retrieve the elements in the order they have been inserted. [...] Is it even possible?
No, not with std::map. std::map inserts the element pair into an already ordered tree structure (and after the insertion operation, the std::map has no way of knowing when each entry was added).
You can solve this in multiple ways:
use a std::vector<std::pair<int,char*>>. This will work, but not provide the automatic sorting that a map does.
use Boost stuff (#BartekBanachewicz suggested Boost.MultiIndex)
Use two containers and keep them synchronized: one with a sequential insert (e.g. std::vector) and one with indexing by key (e.g. std::map).
use/write a custom container yourself, so that supports both type of indexing (by key and insert order). Unless you have very specific requirements and use this a lot, you probably shouldn't need to do this.
A couple of options (other than those that Bartek suggested):
If you still want key-based access, you could use a map, along with a vector which contains all the keys, in insertion order. This gets inefficient if you want to delete elements later on, though.
You could build a linked list structure into your values: instead of the values being char*s, they're structs of a char* and the previously- and nextly-inserted keys**; a separate variable stores the head and tail of the list. You'll need to do the bookkeeping for that on your own, but it gives you efficient insertion and deletion. It's more or less what boost.multiindex will do.
** It would be nice to store map iterators instead, but this leads to circular definition problems.
Java has a LinkedHashSet, which is a set with a predictable iteration order. What is the closest available data structure in C++?
Currently I'm duplicating my data by using both a set and a vector. I insert my data into the set. If the data inserted successfully (meaning data was not already present in the set), then I push_back into the vector. When I iterate through the data, I use the vector.
If you can use it, then a Boost.MultiIndex with sequenced and hashed_unique indexes is the same data structure as LinkedHashSet.
Failing that, keep an unordered_set (or hash_set, if that's what your implementation provides) of some type with a list node in it, and handle the sequential order yourself using that list node.
The problems with what you're currently doing (set and vector) are:
Two copies of the data (might be a problem when the data type is large, and it means that your two different iterations return references to different objects, albeit with the same values. This would be a problem if someone wrote some code that compared the addresses of the "same" elements obtained in the two different ways, expecting the addresses to be equal, or if your objects have mutable data members that are ignored by the order comparison, and someone writes code that expects to mutate via lookup and see changes when iterating in sequence).
Unlike LinkedHashSet, there is no fast way to remove an element in the middle of the sequence. And if you want to remove by value rather than by position, then you have to search the vector for the value to remove.
set has different performance characteristics from a hash set.
If you don't care about any of those things, then what you have is probably fine. If duplication is the only problem then you could consider keeping a vector of pointers to the elements in the set, instead of a vector of duplicates.
To replicate LinkedHashSet from Java in C++, I think you will need two vanilla std::map (please note that you will get LinkedTreeSet rather than the real LinkedHashSet instead which will get O(log n) for insert and delete) for this to work.
One uses actual value as key and insertion order (usually int or long int) as value.
Another ones is the reverse, uses insertion order as key and actual value as value.
When you are going to insert, you use std::map::find in the first std::map to make sure that there is no identical object exists in it.
If there is already exists, ignore the new one.
If it does not, you map this object with the incremented insertion order to both std::map I mentioned before.
When you are going to iterate through this by order of insertion, you iterate through the second std::map since it will be sorted by insertion order (anything that falls into the std::map or std::set will be sorted automatically).
When you are going to remove an element from it, you use std::map::find to get the order of insertion. Using this order of insertion to remove the element from the second std::map and remove the object from the first one.
Please note that this solution is not perfect, if you are planning to use this on the long-term basis, you will need to "compact" the insertion order after a certain number of removals since you will eventually run out of insertion order (2^32 indexes for unsigned int or 2^64 indexes for unsigned long long int).
In order to do this, you will need to put all the "value" objects into a vector, clear all values from both maps and then re-insert values from vector back into both maps. This procedure takes O(nlogn) time.
If you're using C++11, you can replace the first std::map with std::unordered_map to improve efficiency, you won't be able to replace the second one with it though. The reason is that std::unordered map uses a hash code for indexing so that the index cannot be reliably sorted in this situation.
You might wanna know that std::map doesn't give you any sort of (log n) as in "null" lookup time. And using std::tr1::unordered is risky business because it destroys any ordering to get constant lookup time.
Try to bash a boost multi index container to be more freely about it.
The way you described your combination of std::set and std::vector sounds like what you should be doing, except by using std::unordered_set (equivalent to Java's HashSet) and std::list (doubly-linked list). You could also use std::unordered_map to store the key (for lookup) along with an iterator into the list where to find the actual objects you store (if the keys are different from the objects (or only a part of them)).
The boost library does provide a number of these types of combinations of containers and look-up indices. For example, this bidirectional list with fast look-ups example.