Using std::map::extract to modify key - c++

My implementation uses std::map to store data. When I started my code it seemed like the best option. Now I came to a point where I have to change the key values of all objects inside the map.
The problem is that each object points to another object inside the map:
class AND : public node{
vector <node*> inputs;
vector <node*> outputs;
}
And the map is declared like this:
map<unsigned int, AND> all_ANDs;
My question is: If I use map::extract from C++17 to modify the key values in all_ANDs map, will my pointers (E.g. the ones inside the attribute inputs) keep pointing to the right places?
In other words: If I change the value of "first" element with extract, the address of "second" will keep intact?
I noticed from this link that the string "papaya" stays the same (and works gracefully). But I wanted to be sure about pointers.

YES
The reference you have already quoted in your posts clearly states that no elements are copied or moved. (This assumes that node in your code snippet does not refer to map::node_type).
The same holds for the insert operation of the map-node (after modifying its key):
If the insertion is successful, pointers and references to the element obtained while it is held in the node handle are invalidated, and pointers and references obtained to that element before it was extracted become valid. (since C++17)
However, accessing the object between extract()ion and re-insert()ion has undefined behaviour and its address taken whilst in the extracted state is of limited use. Quoting from the standard:
The extract members invalidate only iterators to the removed element;
pointers and references to the removed element remain valid. However,
accessing the element through such pointers and references while the
element is owned by a node_­type is undefined behavior. References and
pointers to an element obtained while it is owned by a node_­type are
invalidated if the element is successfully inserted.
Explanation
Essentially, a map<> is implemented as a tree of nodes, each holding a key and T (which are exposed as pair<const Key, T> to the user). Nodes are allocated (typically) on the heap and the address of your object is related to that of a node. map::extract() un-links a node from its tree and returns a node handle (an object holding a pointer to a map node), but AFAIK the node itself is not re-allocated, moved, or copied. Upon map::insert(handle), the node is re-linked into the tree according to its (new) key. Again, this involves no re-allocation, move, or copy of the node.
Remark
The above is a rough sketch. How things are actually done is likely more complex and also implementation defined. As explained here a node_handle does allow to alter the key through the member function
Key &node_handle::key() const;
How this is done under the hood is not specified and I speculate that the implementation uses a union or some cast to allow this. Of course, the map has to present to users a pair<const Key,T> in order to prevent them from changing the key and hence breaking the map, but this is not of any concern for an element extracted from the map.

My above answer addresses your immediate question. However, as I have suggested in a comment, this appears to be a XY problem. What I suspect:
You have some structure of AND objects which are interlinked via their inputs and outputs fields. This linkage must not be broken by any re-allocation, so you cannot store them in a growing vector<AND> with re-allocation.
You also want to order these objects according to some key and have therefore stored them in a map<Key,AND>, which indeed does not re-allocate when grown.
You now want to order them according to another key (and/or change all the keys).
(If you're actually are not interested in ordering but merely in finding your objects by their key, you should have used unordered_map instead of map, which supports find() in O(n) rather than O(log(n)) operations.)
I suggest a different layout of your data:
You store your AND objects in a way that allows growing their number without re-allocation. An obvious choice here is deque<AND>, since
insertion and deletion at either end of a deque never invalidates
pointers or references to the rest of the elements
You may also make AND non-copyable and non-movable, ensuring that once allocated their address never changes (and pointers to them remain valid until destruction).
You can support any find-by-key or order-by-key operations by actually working on pointers to the stored objects, either by sorting a vector of pair<key,AND*> or by using a map<key,AND*> or unordered_map<key,AND*>. You can even simultaneously have various keys per object (and a map for each).
When you must re-key all objects, simply forget the old map and make a new one: since the map only stores pointers and not the objects, this does not affect your linkages.

Your map holds actual AND objects, not pointers to objects. So, if the AND* pointers stored inside your vectors are pointing at the map's AND objects, then those pointers WILL become invalid once those objects are erased from the map.
However, extraction merely unlinks a specified node from the map, the node and thus its key and value are still valid in memory. The node can be re-inserted into a map without affecting the addresses of the node's key and value. In this regard, the pointers in the vectors WILL NOT become invalid (although it is undefined to dereference them while the node is detached from the container).
Another option is to change your map to hold AND* pointers instead. Or better, consider using std::shared_ptr<AND> in the map and std::shared_ptr<node> in the vectors, instead of raw pointers. Then it won't matter whether the map entries are erased or extracted, the AND objects will remain valid as long as there are active shared_ptr references to them.

Related

std::unordered_map insert invalidates only iterators but not references and pointers to the element node

Can somebody explain why insertion into std::unordered_map container only invalidates iterators but not references and pointers. Also I am not able to understand what the below statement from https://en.cppreference.com/w/cpp/container/unordered_map/insert mean
If the insertion is successful, pointers and references to the element obtained while it is held in the node handle are invalidated, and pointers and references obtained to that element before it was extracted become valid.
Insertion of unordered_map doesn't invalidate references because it doesn't move the data, however the underlying data structure might change rather significantly. Details of how exactly it is implemented aren't specified and different compilers do it differently. For instance, MSVC has a linked list for data storage and, I believe, a vector for the look-up. And insert can cause rehash, meaning look-up gets changed completely and the linked list gets reorded significantly - but original data is not moved.
The iterators refer to this underlying structure and any changes to it can cause iterators to invalidate. Basically, they contain more info than pointers and references and subsequently can get invalidated.
The confusing passage is about insert of a node_type - nodes that were previously extracted. Checkout the method unordered_map::extract
https://en.cppreference.com/w/cpp/container/unordered_map/extract
For some reason it is forbidden to use pointers/references while the node is extracted. After inserting it is allowed to use them again. I don't know why it is so.
In terms of the second part of the question, it is referring to the Node handle introduced in C++17. It is a move-only type, that has direct ownership of the underlying key and value. It can be used to change the key of an element without reallocation and transfer element ownership without copy or move.
Since it's allowed to change const-like data(such as key), I personally think it makes sense to only allow such edit to happen when it is isolated from the container, ie when it is in the node form; which is why pointer and reference to it underlying data should be invalidated once they are insert back to the container.
Similarly, since the insertion does not incur any reallocations, once the node is inserted back to the container, pointer and references that were point to the data before they were extract will be valid again.

Is it safe to have pointers to elements in Data Structures? (c++ with QT)

I have the following structure on the software I am developing:
ClassA:
QHash<int, ClassB>
ClassB:
QHash<int, ClassC>
ClassC:
QMap<ID, QSharedPointer<ClassD> > (this is because I need to have the items ordered)
QHash<int, QSharedPointer<ClassD> > (this exists so I can access an item via id)
My question is if it is safe to have a pointer, that will be edited, to an element inside a data structure. I have been getting errors while trying to debug in which the debugger is unable to stop at a break point and I get a SIGTRAP error, but I am not sure if it is related to a memory issue on this.
To give a better example, related to the software I'm developing I have a QHash<int, Take> that represents a list of videos. The user will be editing only one video at a time, so I have a pointer to the current video, which is a Take inside the Hash. Each Take has a bunch of parameters that can be edited but the most common is a QMap of Notes. Is is safe to do something like this?
Take *currentTake = &takes[currentTakeId];
----//---
currentTake->addNote(new Note());
currentTake->changeSomeParameter();
etc
Whether (or how long) it is safe to keep a pointer/reference to an element of a collection is up to that collection. For example, a std::vector invalidates all pointers into it on reallocation, and removal/insertion without reallocation invalidates (well, changes what they point to) all pointers beyond the insertion/removal point. A std::list on the other hand is stable; pointers only get invalidated if the specific element they point to is removed.
A collection should generally document its invalidation behavior. Unfortunately, the Qt collections don't. Reading their documentation tells us that QMap is a red-black balanced binary tree, and QHash is a separate chaining hash table, so they should both have the std::list behavior for invalidation, but there's no guarantee of that. (For example, QHash could store the head entry directly in the bucket list, which would mean that rehashing would invalidate most pointers, and removing an element could invalidate pointers to elements in the same bucket.)
Use references, the data container of your example is storing values of object and returns reference to the object when you do takes[currentTakeId].
You can do this
Take &currentTake = takes.value(currentTakeId]);
// same as 'takes[currentTakeId]'
// but avoid to create empty element if 'currentTakeId' has no element
----//---
currentTake.addNote(new Note());
currentTake.changeSomeParameter();
// do not change 'takes' as long as you use 'currentTake'
Your idea is fine (and more or less the same), you can retrieve the address and work on the pointer as long as the container is not modified when you are using this pointer. Because the data container might copy your element if resizing, or delete it, and then the pointer would be invalid.
If your objects Take are quite big you can lose performance because of the copy when the container is resized or passed by value (and copied). Then storing pointer can be a solution, or use the implicit sharing pattern of Qt on your data class.

Container that guarantee fixed position for items

Is there C++ container that guarantees a fixed pointer for items what ever changes happened?
For example, std::vector may change the address of an item if a push_back or erase happened. So the address of the item will be rubbish after this change. So, is there a container that does not change items address in memory while the container changing?
P.S. compile time size or fixed size is not an option
EDIT:
As #Joachim Pileborg stated it is XY problem or in actual it is XYZ one! The Z is this question. The Y is the following one:
Keeping vector of iterators of the data
The original one:
I have data which is set of Points(x,y). This Points will go into a pipeline. The result should be:
set of Lines
set of Points for each line... in other word, set of set of Points
I do not want to copy the point and return them by value. I know a Point with just x and y is nothing to worry about copying it. However, in my it is templated problem which may be much bigger object in some cases.
Is there C++ container that guarantees a fixed pointer for items what ever changes happened?
If by what ever you include erasing the item that is pointed to, then only std::array is such container because you cannot remove elements from it.
If you mean that anything else but erasing the pointed item, then all node based containers have that property, as pointed out in the comments. Such standard containers are std::list, std::forward_list, std::map std::multimap, std::set and std::multiset. Erasing or modifying (if modifying is possible) an item from any of those containers does not invalidate iterators nor pointers or references to elements.
Also, if you store pointers in std::vector or other containers that don't have the property, then the stored pointer to the object still remains valid even though indices, pointers, references and iterators to the stored pointer become invalid. There is a stable_vector template in boost that stores pointers to the element and does not invalidate iterators or pointers to the element when the container is modified. The indices do of course become invalid if elements are removed and obviously it doesn't have the advantage of contiguous memory.
About your original problem:
Given your requirements, returning a set of set of iterators/pointers to the original container seems indeed appropriate. And if the iterators must remain valid when the original container is be modified later, say by adding more points or by removing points that are not referred by any partition, then the type of the original container must indeed be such as discussed on this page.
You can use a std::map with the pointers as the key and since the keys are unique, no matter whatever changes you do, the addresses will not change.
eg:
std::map<int*, list<int>> x;

Should I manipulate a C++ map's value via a pointer or by updating the record?

I'm using a C++ std::map to hold a large collection of entities:
using std::map;
map {structureEntityID, classEntityRecord} tableEntityRecords; //(replace {}s with arrows)
I will be frequently modifying the entities in my table (many times a second). Is it better to modify those records via a pointer or is it better to make a local copy, modify it, and then update the table?
For example...
Via a pointer:
classEntityRecord* getEntityRecord(structureEntityID entityID)
{
map {structureEntityID, classEntityRecord}::iterator iteratorEntityRecord;
iteratorEntityRecord = tableEntityRecords.find(entityID);
return &iteratorEntityRecord->second;
}
classEntityRecord *entityRecord;
entityRecord = getEntityRecord(entityID);
entityRecord->x = 15;
Via a copy/modify/update:
classEntityRecord getEntityRecord(structureEntityID entityID)
{
map {structureEntityID, classEntityRecord}::iterator iteratorEntityRecord;
iteratorEntityRecord = tableEntityRecords.find(entityID);
return iteratorEntityRecord->second;
}
classEntityRecord entityRecord;
entityRecord = getEntityRecord(entityID);
entityRecord.x = 15;
tableEntityRecords[entityID] = entityRecord;
I would think it is better to use a pointer, but this is my first time with C++ maps so I don't fully understand how they work.
My big worry is if I take a pointer to one of the values in my table, is it possible for the C++ map to be reordered and have that pointer no longer be valid? The program is multithreaded so entities can be added to the table while others are being modified.
I appreciate the help!
You should be thinking of references (&) not pointers (*). As it stands your getEntityRecord returns a copy of the map element value, hence changes to the returned value will not be seen via the map. If you change this function to return a reference, it will do what you want.
classEntityRecord& getEntityRecord(structureEntityID entityID)
map::find returns you an iterator, the second field of which contains a reference to the map member value. You can use this reference to safely modify the map member value without worrying about the map being modified, provided nobody else deletes that entry.
From the map documentation for SGI STL
Map has the important property that
inserting a new element into a map
does not invalidate iterators that
point to existing elements. Erasing an
element from a map also does not
invalidate any iterators, except, of
course, for iterators that actually
point to the element that is being
erased.
As for your worry about "if I take a pointer to one of the values in my table, is it possible for the C++ map to be reordered and have that pointer no longer be valid?", the answer is that you probably don't need to worry much about it - but do need to take some care (assuming that you use iterators rather than pointers, which should be easy, since iterators act like pointers by design).
Here's what the standard says about the validity of iterators and references to objects in associative containers like std::map (23.1.2/8 "Associative containers"):
The insert members shall not affect the validity of iterators and references to the container, and the erase members shall invalidate only iterators and references to the erased elements.

Storing iterators inside containers

I am building a DLL that another application would use. I want to store the current state of some data globally in the DLL's memory before returning from the function call so that I could reuse state on the next call to the function.
For doing this, I'm having to save some iterators. I'm using a std::stack to store all other data, but I wasn't sure if I could do that with the iterators also.
Is it safe to put list iterators inside container classes? If not, could you suggest a way to store a pointer to an element in a list so that I can use it later?
I know using a vector to store my data instead of a list would have allowed me to store the subscript and reuse it very easily, but unfortunately I'm having to use only an std::list.
Iterators to list are invalidated only if the list is destroyed or the "pointed" element is removed from the list.
Yes, it'll work fine.
Since so many other answers go on about this being a special quality of list iterators, I have to point out that it'd work with any iterators, including vector ones. The fact that vector iterators get invalidated if the vector is modified is hardly relevant to a question of whether it is legal to store iterators in another container -- it is. Of course the iterator can get invalidated if you do anything that invalidates it, but that has nothing to do with whether or not the iterator is stored in a stack (or any other data structure).
It should be no problem to store the iterators, just make sure you don't use them on a copy of the list -- an iterator is bound to one instance of the list, and cannot be used on a copy.
That is, if you do:
std::list<int>::iterator it = myList.begin ();
std::list<int> c = myList;
c.insert (it, ...); // Error
As noted by others: Of course, you should also not invalidate the iterator by removing the pointed-to element.
This might be offtopic, but just a hint...
Be aware, that your function(s)/data structure would probably be thread unsafe for read operations. There is a kind of basic thread safety where read operations do not require synchronization. If you are going to store the sate how much the caller read from your structure it will make the whole concept thread unsafe and a bit unnatural to use. Because nobody assumes a read to be state-full operation.
If two threads are going to call it they will either need to synchronize the calls or your data structure might end-up in a race condition. The problem in such a design is that both threads must have access to a common synchronization variable.
I would suggest making two overloaded functions. Both are stateless, but one of them should accept a hint iterator, where to start next read/search/retrieval etc. This is e.g. how Allocator in STL is implemented. You can pass to allocator a hint pointer (default 0) so that it quicker finds a new memory chunk.
Regards,
Ovanes
Storing the iterator for the list should be fine. It will not get invalidated unless you remove the same element from the list for which you have stored the iterator. Following quote from SGI site:
Lists have the important property that
insertion and splicing do not
invalidate iterators to list elements,
and that even removal invalidates only
the iterators that point to the
elements that are removed
However, note that the previous and next element of the stored iterator may change. But the iterator itself will remain valid.
The same rule applies to an iterator stored in a local variable as in a longer lived data structure: it will stay valid as long as the container allows.
For a list, this means: as long as the node it points to is not deleted, the iterator stays valid. Obviously the node gets deleted when the list is destructed...