Is there C++ container that guarantees a fixed pointer for items what ever changes happened?
For example, std::vector may change the address of an item if a push_back or erase happened. So the address of the item will be rubbish after this change. So, is there a container that does not change items address in memory while the container changing?
P.S. compile time size or fixed size is not an option
EDIT:
As #Joachim Pileborg stated it is XY problem or in actual it is XYZ one! The Z is this question. The Y is the following one:
Keeping vector of iterators of the data
The original one:
I have data which is set of Points(x,y). This Points will go into a pipeline. The result should be:
set of Lines
set of Points for each line... in other word, set of set of Points
I do not want to copy the point and return them by value. I know a Point with just x and y is nothing to worry about copying it. However, in my it is templated problem which may be much bigger object in some cases.
Is there C++ container that guarantees a fixed pointer for items what ever changes happened?
If by what ever you include erasing the item that is pointed to, then only std::array is such container because you cannot remove elements from it.
If you mean that anything else but erasing the pointed item, then all node based containers have that property, as pointed out in the comments. Such standard containers are std::list, std::forward_list, std::map std::multimap, std::set and std::multiset. Erasing or modifying (if modifying is possible) an item from any of those containers does not invalidate iterators nor pointers or references to elements.
Also, if you store pointers in std::vector or other containers that don't have the property, then the stored pointer to the object still remains valid even though indices, pointers, references and iterators to the stored pointer become invalid. There is a stable_vector template in boost that stores pointers to the element and does not invalidate iterators or pointers to the element when the container is modified. The indices do of course become invalid if elements are removed and obviously it doesn't have the advantage of contiguous memory.
About your original problem:
Given your requirements, returning a set of set of iterators/pointers to the original container seems indeed appropriate. And if the iterators must remain valid when the original container is be modified later, say by adding more points or by removing points that are not referred by any partition, then the type of the original container must indeed be such as discussed on this page.
You can use a std::map with the pointers as the key and since the keys are unique, no matter whatever changes you do, the addresses will not change.
eg:
std::map<int*, list<int>> x;
Related
Can somebody explain why insertion into std::unordered_map container only invalidates iterators but not references and pointers. Also I am not able to understand what the below statement from https://en.cppreference.com/w/cpp/container/unordered_map/insert mean
If the insertion is successful, pointers and references to the element obtained while it is held in the node handle are invalidated, and pointers and references obtained to that element before it was extracted become valid.
Insertion of unordered_map doesn't invalidate references because it doesn't move the data, however the underlying data structure might change rather significantly. Details of how exactly it is implemented aren't specified and different compilers do it differently. For instance, MSVC has a linked list for data storage and, I believe, a vector for the look-up. And insert can cause rehash, meaning look-up gets changed completely and the linked list gets reorded significantly - but original data is not moved.
The iterators refer to this underlying structure and any changes to it can cause iterators to invalidate. Basically, they contain more info than pointers and references and subsequently can get invalidated.
The confusing passage is about insert of a node_type - nodes that were previously extracted. Checkout the method unordered_map::extract
https://en.cppreference.com/w/cpp/container/unordered_map/extract
For some reason it is forbidden to use pointers/references while the node is extracted. After inserting it is allowed to use them again. I don't know why it is so.
In terms of the second part of the question, it is referring to the Node handle introduced in C++17. It is a move-only type, that has direct ownership of the underlying key and value. It can be used to change the key of an element without reallocation and transfer element ownership without copy or move.
Since it's allowed to change const-like data(such as key), I personally think it makes sense to only allow such edit to happen when it is isolated from the container, ie when it is in the node form; which is why pointer and reference to it underlying data should be invalidated once they are insert back to the container.
Similarly, since the insertion does not incur any reallocations, once the node is inserted back to the container, pointer and references that were point to the data before they were extract will be valid again.
My implementation uses std::map to store data. When I started my code it seemed like the best option. Now I came to a point where I have to change the key values of all objects inside the map.
The problem is that each object points to another object inside the map:
class AND : public node{
vector <node*> inputs;
vector <node*> outputs;
}
And the map is declared like this:
map<unsigned int, AND> all_ANDs;
My question is: If I use map::extract from C++17 to modify the key values in all_ANDs map, will my pointers (E.g. the ones inside the attribute inputs) keep pointing to the right places?
In other words: If I change the value of "first" element with extract, the address of "second" will keep intact?
I noticed from this link that the string "papaya" stays the same (and works gracefully). But I wanted to be sure about pointers.
YES
The reference you have already quoted in your posts clearly states that no elements are copied or moved. (This assumes that node in your code snippet does not refer to map::node_type).
The same holds for the insert operation of the map-node (after modifying its key):
If the insertion is successful, pointers and references to the element obtained while it is held in the node handle are invalidated, and pointers and references obtained to that element before it was extracted become valid. (since C++17)
However, accessing the object between extract()ion and re-insert()ion has undefined behaviour and its address taken whilst in the extracted state is of limited use. Quoting from the standard:
The extract members invalidate only iterators to the removed element;
pointers and references to the removed element remain valid. However,
accessing the element through such pointers and references while the
element is owned by a node_type is undefined behavior. References and
pointers to an element obtained while it is owned by a node_type are
invalidated if the element is successfully inserted.
Explanation
Essentially, a map<> is implemented as a tree of nodes, each holding a key and T (which are exposed as pair<const Key, T> to the user). Nodes are allocated (typically) on the heap and the address of your object is related to that of a node. map::extract() un-links a node from its tree and returns a node handle (an object holding a pointer to a map node), but AFAIK the node itself is not re-allocated, moved, or copied. Upon map::insert(handle), the node is re-linked into the tree according to its (new) key. Again, this involves no re-allocation, move, or copy of the node.
Remark
The above is a rough sketch. How things are actually done is likely more complex and also implementation defined. As explained here a node_handle does allow to alter the key through the member function
Key &node_handle::key() const;
How this is done under the hood is not specified and I speculate that the implementation uses a union or some cast to allow this. Of course, the map has to present to users a pair<const Key,T> in order to prevent them from changing the key and hence breaking the map, but this is not of any concern for an element extracted from the map.
My above answer addresses your immediate question. However, as I have suggested in a comment, this appears to be a XY problem. What I suspect:
You have some structure of AND objects which are interlinked via their inputs and outputs fields. This linkage must not be broken by any re-allocation, so you cannot store them in a growing vector<AND> with re-allocation.
You also want to order these objects according to some key and have therefore stored them in a map<Key,AND>, which indeed does not re-allocate when grown.
You now want to order them according to another key (and/or change all the keys).
(If you're actually are not interested in ordering but merely in finding your objects by their key, you should have used unordered_map instead of map, which supports find() in O(n) rather than O(log(n)) operations.)
I suggest a different layout of your data:
You store your AND objects in a way that allows growing their number without re-allocation. An obvious choice here is deque<AND>, since
insertion and deletion at either end of a deque never invalidates
pointers or references to the rest of the elements
You may also make AND non-copyable and non-movable, ensuring that once allocated their address never changes (and pointers to them remain valid until destruction).
You can support any find-by-key or order-by-key operations by actually working on pointers to the stored objects, either by sorting a vector of pair<key,AND*> or by using a map<key,AND*> or unordered_map<key,AND*>. You can even simultaneously have various keys per object (and a map for each).
When you must re-key all objects, simply forget the old map and make a new one: since the map only stores pointers and not the objects, this does not affect your linkages.
Your map holds actual AND objects, not pointers to objects. So, if the AND* pointers stored inside your vectors are pointing at the map's AND objects, then those pointers WILL become invalid once those objects are erased from the map.
However, extraction merely unlinks a specified node from the map, the node and thus its key and value are still valid in memory. The node can be re-inserted into a map without affecting the addresses of the node's key and value. In this regard, the pointers in the vectors WILL NOT become invalid (although it is undefined to dereference them while the node is detached from the container).
Another option is to change your map to hold AND* pointers instead. Or better, consider using std::shared_ptr<AND> in the map and std::shared_ptr<node> in the vectors, instead of raw pointers. Then it won't matter whether the map entries are erased or extracted, the AND objects will remain valid as long as there are active shared_ptr references to them.
when you push_back() items into an std::vector, and retain the pointers to the objects in the vector via the back() reference -- are you guaranteed (assuming no deletions occur) that the address of the objects in the vector will remain the same?
It seems like my vector changes the pointers of the objects I use, such that if I push 10 items into it, and retain the pointers to those 10 items by remembering the back() reference after each push_back.
if your vector is to store objects, not pointers to objects, are the addresses of those objects subject to constant change upon pushing more items?
Any method that causes the vector to resize itself will invalidate all iterators, pointers, and references to the elements contained within. This can be avoided by reserving mememory, or using boost::stable_vector.
23.3.6.5/1:
Remarks: Causes reallocation if the new size is greater than the old capacity. If no reallocation happens,
all the iterators and references before the insertion point remain valid.
No, std::vector is not a stable container, i.e. pointers and iterators may get invalidated by resizing the vector (or, better, by the corresponding reallocation). If you want to avoid this behaviour, use boost::stable_vector or std::list or std::deque instead (I would prefer the last). Or, more easily, you can simply store your locations by indices.
For more information, consider also the answer to this question here.
It's not guaranteed. If you push_back enough items to exceed the size of the memory buffer that's the backing store of the vector, a new buffer will be created, all the contents will be copied to the new location, and the old buffer will be deleted. At that point, old pointers (as well as iterators!) will be invalid.
If you know exactly how much maximum space you will ever need, you could set the size of the vector's buffer to that size when you create it, to avoid reallocation. However, I prefer to store "references" to elements of a vector as a reference to the vector and a size_t index into the vector, instead of using pointers. It's not necessarily slower than pointers (depending on the CPU type) but, even if it is, it won't be much slower and in my opinion it's worth it for the peace of mind knowing that no matter how the vector is used in the future or reallocated, it'll still refer to the proper element.
I want to insert a bit from a bitset in the beginning of a vector. I am having a hard time understanding how to do that. Here is how I think I can do it:
keyRej.insert(x, inpSeq[0]);
I don't know what to put in the place of x?
I don't know what to put in the place of x?
An iterator to the position you want to insert in:
keyRej.insert(keyRej.begin(), inpSeq[0]);
Semantically, the inserted element goes before the iterator passed as first argument. But this will result in all elements of the vector having to be moved across one position, and may also incur a re-allocation of the vector's internal data storage block. It also means that all iterators or references to the vector's elements are invalidated.
See this reference for std::vector::insert for more information.
Note that there are containers, such as std::deque, for which appending elements to the front is cheap, and reference (but not iterator) validity is maintained.
x is an iterator according to the documentation you probably read here, the new object is insert just before it.
keyRej.insert(keyRej.begin(), inpSeq[0]);
What are recommended methods of avoiding errors, when iterating through a vector; where any number of elements in the vector may (directly, or indirectly) cause insertions or removals of elements - invaliding the iterators?
Specifically, I'm asking in relation to games programming, with elements in the vector being game objects; some of which can spawn other objects, and some of which will be killed and need removed when they are updated. Because of this, reserving a large capacity before iteration is also not possible, as it's entirely unknown how many elements can be expected to be added.
It's not possible to say in general. Part of the reason that the iterators are invalidated is that there's no way to read your mind to know what element you would like the invalidated iterators to refer to.
For example, I have an iterator to element 3 of 5. I erase element 2. Should the iterator point to the new element 2 (because it's "the same value moved down"), or should it point to the new element 3 (because it's "the same element of the vector with a different value moved down into it")?
Practically, your options are:
Do not insert/erase elements while someone else has iterators. This means altering your code to make the changes at another time.
Use indexes instead of iterators. This results in the second option above ("index refers to the same element with a new value"). Beware that indexes can also become invalid, if you erase enough elements that the index is off the end.
Use a different data structure with different iterator invalidation rules. For example a std::list would give you the first option above ("iterator refers to the same element in a new position")
The iterator that you're using to iterate should never be harmfully invalidated by insertions/erases of a single element at that position. It is invalidated, but both functions return a new iterator that you can use to continue iterating. See their documentation for what element that new iterator refers to. That's why the problem only arises when you're using multiple iterators on the same container.
Here are some ideas:
Iterate over the vector using an integer index. This won't get invalidated, but you need to take care to adjust the current index when inserting/removing elements.
Iterate over a copy of the vector.
Instead of making changes to the vector as you go along, keep track of what needs to be inserted/removed, and apply the changes after you've finished iterating.
Use a different container, for example a linked list.
Vector might not be the best container to support intensive insertion/removal (especially when the size is large).
But if you have to use vector, IMHO the safest way would be to work not with an iterator but with an index and keep track (and adjust the index) of items being inserted/deleted before the current position. This way you will at least avoid problems with reallocations.
And recalculate v.size()/v.end() after each iteration.
Use a separate vector and std::move the elements which don't get removed to it along with any created new elements. Then drop the old vector.
This way you don't have to remove/insert anything from/to the original vector and the iterators don't get invalidated.
You can, of course, go around the problem by using a container that doesn't invalidate the iterators on insertion and removal.
I'd like to recommend a different approach. Essentially you're iterating over the vector to create a new state in the game world. Instead of changing the state of the world as you traverse the vector, store what needs to change in a new vector. Then, after you've completely examined the old state, apply the changes you stored in the new vector.
This approach has one conceptual advantage: every object's new state depends on the old state only. If you apply the changes as you traverse the old state, and decisions about one object can depend on other objects, the order of traversal can affect the outcome, which can give "earlier" or "later" objects an unfair advantage.