I want to build up a map of devices such that the map contains:
QString 'DeviceID' and QVector 'Command List'
Currently I have the QMap as follows:
QMap<QString, QVector<QString> *> devices;
QVector<QString> *pCommands= new QVector<QString>;
// :
// Fill pCommands with lots of data here
// :
devices.insert(RadioID, pCommands);
But I am wondering if this is actually any better then:
QMap<QString, QVector<QString>> devices;
QVector<QString> commands;
// :
// Fill commands with lots of data here
// :
devices.insert(RadioID, commands);
I am sure that I read somewhere that Qt does something quite efficient when copying data. I am not seeing many people using pointers with Qt and it seems messy that I have to go through the QMap deleting all the QVector's at the end...
I am using c++11, so maybe some kind of move semantic may work here?
EDIT
I have modified the comments in the code to show that the vector is not empty.
Also I would state that I do not need to change the data once it is stored.
There is no reason to consider manually allocating the vectors to be better.
Sure, you only need to copy a pointer, rather than a vector, but an empty vector is still quite fast to copy. The biggest gain of storing objects rather than pointers is that you don't need to do manual memory management.
I am using c++11, so maybe some kind of move semantic may work here?
If QMap::insert supports move semantics, and if QVector is move-constructible like their standard library counterparts, then you could indeed move the vector into the map. But moving an empty vector is just as fast as copying.
If QMap has an emplace like function std::map does, then you could even construct the vector in-place without even a move.
I'm not familiar with Qt, though so you'll need to verify those details from the documentation. Whether Qt supports move semantics doesn't change the fact that manual memory management is a pain.
Edit: according to the documentation QVector appears to be movable, but QMap does not support move semantics. However, as Arpegius and the documentation point out, QVector does copy-on-write optimization, so as long as the copied vector is not modified, then the data won't be copied. None of this matters really, when copying an empty vector.
Edit again
If the added vectors are full of data, then copying is indeed quite expensive unless it remains unmodified. Moving would remedy that, but QMap appears not to support that. There is another trick, though: Insert an empty vector, and then swap the full vector with the empty one in the map.
The simplest and pretty much idiomatic way to do it would be:
QMap<QString, QVector<QString>> devices;
// Get the vector constructed in the map itself.
auto & commands = devices[RadioID];
// Fill commands with lots of data here - a bit faster
commands.resize(2);
commands[0] = "Foo";
commands[1] = "Bar";
// Fill commands with lots of data here - a bit slower
commands.append("Baz");
commands.append("Fan");
You'll see this pattern often in Qt and other code, and it's the simplest way to do it that works equally well on C++98 and C++11 :)
This way you're working on the vector that's already in the map. The vector itself is never copied. The members of the vector are copied, but they are implicitly shared, so copying is literally a pointer copy and an atomic reference count increment.
Related
In my application there's a part that requires me to make a copy of a container. At the moment I use std::vector (but might consider using something else). The application is very sensitive with regard to the latency. So, I was looking for a way to make a copy of a vector as fast as possible. I found that memcpy does it better than anything else. However, it does not change the internal size of the vector. I.e. vector.size() would still give me 0.
I know what slippery path I am on. I do not mind throwing away safety checks. I do know how many elements are being copied. I do not want to use vector.resize() to change the size of the destination vector (it is a slow operation).
Question:
std::vector<my_struct> destination_vector;
destination_vector.reserve(container_length);
std::memcpy(destination_vector.data(), original_vector.data(), sizeof(my_struct)*container_length);
After the code above I need to tell my destination_vector what size it is. How do I do this?
Thank you.
You must to actually resize() the vector before you copy stuff to it using memcpy():
destination_vector.resize(container_length);
But it would be better to avoid the use of memcpy() in the first place and use the mechanisms to copy vector content which is offered by vector, as suggested in the other answers:
std::vector<my_struct> destination_vector(original_vector);
or if the destination_vector instance already exists:
destination_vector.insert(destination_vector.begin(), original_vector.begin(), original_vector.end);
or, the fastest if you do not need the original content any more:
destination_vector.swap(original_vector);
All of these variants will be as fast or even faster than your memcpy()variant. If you experience slowness then see 2.:
You probably have a non-trivial default constructor in my_struct. Remove it, or insert a trivial (empty) default constructor to speed things up (to avoid construction of many elements which you never use).
If my_structcontains non-POD data members (like std::string) you cannot use memcpy() at all.
(Side note: You rarely want to call reserve(). The vector maintains its own internal storage in such a way that is always allocates more than is actually needed, exponentially, to avoid frequent resizes/copying when frequently appending elements.)
resize() is not a slow operation. It is as fast as any memory allocation.
Does my_struct have a non-trivial default constructor? Remove it and take care of initialization manually. This might be the reason why you say resize() is slow. It will actually construct your objects. But since you can apparently memcpy() your objects you can probably get away with a trivial (empty) default constructor.
How to manually assign vector's size?
You can't. You can only modify vector's size through the modification functions that add or remove elements such as insert, clear, resize etc.
After the code above I need to tell my destination_vector what size it is. How do I do this?
The mentioned code above has undefined behaviour, so it doesn't matter what you do after it.
A simple and efficient way to copy a vector is:
std::vector<my_struct> destination_vector(original_vector);
Your snippet has undefined behaviour, you can't memcpy into an empty vector, even if you have reserved space. It may also be undefined behaviour to memcpy any my_struct objects, if it isn't a TriviallyCopyable type.
You can construct the vector as a copy of the source directly. Most likely your compiler will emit code identical (or faster) than your original snippet, if my_struct is TriviallyCopyable.
std::vector<my_struct> destination_vector(original_vector.begin(), original_vector.begin() + container_length);
I need to create a two-way mapping from ints to an object. I can't use the boost::bimap because my objects are modified after being placed in the mapping (they are being modified in ways that do not affect the mapping).
The simple solution is two use a vector and an unordered_map:
vector<MyClass> _vector;
unordered_map<MyClass, size_t> _map;
However, this maintains two copies of each MyClass, and I don't want that.
I can keep MyClass * pointers in one of the containers, and use the storage of the other, but I'm afraid either vector or unordered_map can move the instances around (when reallocating the vector, or resizing the hash table).
Any help would be appreciated.
And can't you just store your objects in one vector and save the mapped indexes in another?
std::vector<MyClass> vC;
std::vector<unsigned int> vM;
Then vC[vM[i]] is the mapped class of the vC[i] object.
Though if you give more details about what you are trying to do (is the map reflexive? All classes have a mapped class or just some? How often you need to modify your objects?) we could help a bit more.
You can use a std::shared_ptr<MyClass> in your main container, and std::weak_ptr<MyClass> in the referencing one.
Unfortunately you didn't give enough context or requirements, to give you a concise example. Anyway, you should also have some synchronization management, that removes entries from the referencing container, as soon these are erased from the main container, though std::weak_ptr makes this easier to implement.
Not only this is more memory efficient, it also relieves you from keeping otherwise unrelated copies of MyClass in sync.
I have a vector containing objects of type STL map, and I do vector.push_back(some map).
This unfortunately calls the map copy constructor, and wastes a lot of time. I understand that i can get around this by keeping a vector of (smart) pointers to maps - but this got me wondering - I read that STL anyway keeps its data on the heap and not on the stack - so why is the copy ctor not O(1) time, by simply copying pointers?
If you don't need the original map anymore after pushing back a copy back into the vector, write:
some_vector.push_back(std::move(some_map));
If you don't have a C++11 compiler yet, add an empty map and then swap that with the original:
some_vector.resize(some_vector.size() + 1);
some_vector.back().swap(some_map);
To answer your question directly: to do that, it would have to start with some sort of copy on write mechanism -- when you put something into a vector, it's required to be a copy of the original (or at least act like one). For example, if I push a map onto my vector, and then remove an item from the original map, that item should still be there in the copy of the map that was pushed onto the vector.
Then it would have to keep track of all the pointers, and ensure that the pointee (the map in this case) remained valid until all those pointers were themselves destroyed. It's certainly possible to do that. Quite a few languages, for example, provide garbage collection largely for this reason. Most of those change the semantics of things, so when/if you (for example) create a vector of maps, putting a map into the vector has reference semantics -- i.e., when you modify the original map, that's supposed to change any "copies" of it that you put into other collections.
As you've observed, you can do any/all of the above in C++ if you really want. The reason it doesn't right now is that most of the C++ standard library is built around value semantics instead of reference semantics. Either is (IMO, anyway) a perfectly valid and reasonable approach -- some languages take one, others take the other. Either/both can work just fine, but value semantics happens to be the choice that was made in C++.
If you want to copy pointers, create a vector of pointers to map. You can do that.
std::vector<std::map<A,B>* > x;
It doesn't do this automatically because it can't know who you want to manage the memory. Should the objects of the map be destroyed when the vector goes out of scope. What if the original map is still in scope?
My question is best illustrated with a code sample, so let's just start off with that:
class Game
{
// All this vector does is establish ownership over the Card objects
// It is initialized with data when Game is created and then is never
// changed.
vector<shared_ptr<Card> > m_cards;
// And then we have a bunch of pointers to the Cards.
// All these pointers point to Cards from m_cards.
// These could have been weak_ptrs, but at the moment, they aren't
vector<Card*> m_ptrs;
// Note: In my application, m_ptrs isn't there, instead there are
// pointers all over the place (in objects that are stored in member
// variables of Game.
// Also, in my application, each Card in m_cards will have a pointer
// in m_ptrs (or as I said, really just somewhere), while sometimes
// there is more than one pointer to a Card.
}
Now what I want to do is to make a deep copy of this Game class. I make a new vector with new shared_ptrs in it, which point to new Card objects which are copies of the original Card objects. That part is easy.
Then the trouble starts, the pointers of m_ptrs should be updated to point to the cards in m_cards, which is no simple task.
The only way I could think of to do this is to create a map and fill it during the copying of m_cards (with map[oldPtr] = newPtr) and then to use that to update m_ptrs. However, this is only O(m * log(n)) (m = m_ptrs.size(); n = m_cards.size()). As this is going to be a pretty regular operation* I would like to do this efficiently, and I have the feeling that it should be possible in O(m) using custom pointers. However, I can't seem to find an efficient way of doing this. Anybody who does?
*it's used to create a testbed for the AI, letting it "try out" different moves
Edit: I would like to add a bit on accepting an answer, as I haven't yet. I am waiting until I get back to this project (I got on a side track as I had worked too much on this project - if you do it for fun it's got to stay fun), so it may be a while longer before I accept an answer. Nevertheless, I will accept an answer some time, so don't worry :P
Edit nr 2: I still haven't gotten back to this project. Right now, I am thinking about just taking the O(m * log(n)) way and not complaining, then seeing later if it needs to be faster. However, as I have recently taken some time to learn my patterns, I am also thinking that I really need to refactor this project some time. Oh, and that I might just spend some time working on this problem with all the new knowledge I have under my belt. Since there isn't an answer that says "just stick with the hashmap and see later if it really needs to be faster" (and I would actually be pretty disappointed if there was, as it's not an answer to my question), I am postponing the picking of an answer yet a bit more till I do get back to this project.
Edit nr 3: I still didn't get back to this project. More precisely, it has been shelved indefinitely. I am pretty sure I just wouldn't get my head too bent over the O(m * log(n))right now, and then perhaps look at it later if it turned out to be a problem. However, that would just not have been a good answer to my question, as I explicitly asked for better performance. Not wanting to leave the answers unaccepted any longer, I chose the most helpful answer and accepted it.
Store the pointers as indexes.
As you say they all point to m_Cards which is a vector that can be indexed (is that correct English?).
Either you do that only for storing and convert them back to pointers at loading.
Or you may think of using indices generally instead of pointers.
What about keeping cards elements index instead of pointer:
vector<int> m_indexes;
...
Card* ptr = &m_cards[m_indexes[0]];
Vector with indexes can be copied without changes.
I recently encountered the very similar problem: cloning the class internal structure implemented by pointers and std::vector as an objects storage.
First of all (unrelated to the question though), I'd suggest either stick with smart pointers or with plain structures. In your case it means that it makes much more sense to use vector<weak_ptr<Card> > m_ptrs instead of raw pointers.
About the question itself - one more possible workaround is using pointer differences in the copy constructor. I will demonstrate it for vector of objects, but working with shared pointers will utilize the same principle, the only difference will be in copying of m_cards (you should not simply use assignment if you want objects clones but copy the m_cards element-by-element).
It is very important that the method works only for containers where the elements are guaranteed to be stored consequently (vector, array).
Another very important moment is that the m_ptrs elements should represent only internal Card structrure, i. e. they must point only to the internal m_cards elements.
// assume we store objects directly, not in shared pointers
// the only difference for shared pointers will be in
// m_cards assignment
// and using m_cards[...].get() instead of &m_cards[...]
vector<Card> m_cards;
vector<Card*> m_ptrs;
In that case your array of pointers can be easily computed by using pointers arithmetic taking linear time. In that case your copy constructor will look like this:
Game::Game(const Game &rhs) {
if (rhs.m_cards.empty())
return;
m_cards = rhs.m_cards;
// if we have vector of shared pointers
// and we need vector of pointers to objects clones
// something like this should be done
// for (auto p: rhs.m_cards) {
// // we must be certain here that dereferencing is safe,
// // i. e. object must exist. If not, additional check is required.
// // copy constructor will be called here:
// m_cards.push_back(std::make_shared<Card>(*p));
// }
Card *first = &rhs.m_cards[0];
for (auto p: rhs.m_ptrs) {
m_ptrs.push_back(&m_cards[p - first]);
}
}
Basically in this deepcopy method you will be still working with indexes, but you preserve the convenience of working with pointers in other class methods without storing your indexes separately.
Anyway, for using that kind of structure you should exactly know what you are doing with the class members and why, that requires much more manual control (for example, at least adding/removing elements to/from m_cards should be done consciously, in other case m_ptrs can easily become broken even without copying the object).
I want to be able to have a vector of vectors of some type such as:
vector<vector<MyStruct> > vecOfVec;
I then create a vector of MyStruct, and populate it.
vector<MyStruct> someStructs;
// Populate it with data
Then finally add someStructs to vecOfVec;
vecOfVec.push_back(someStructs);
What I want to do is avoid having the copy constructor calls when pushing the vector. I know this can be accomplished by using a vector of pointers, but I'd like to avoid that if possible.
One strategy I've thought of seems to work, but I don't know if I'm over-engineering this problem.
// Push back an empty vector
vecOfVec.push_back(vector<MyStruct>());
// Swap the empty with the filled vector (constant time)
vecOfVec.back().swap(someStructs);
This seems like it would add my vector without having to do any copies, but this seems like something a compiler would already be doing during optimization.
Do you think this is a good strategy?
Edit: Simplified my swap statement due to some suggestions.
The swap trick is as good as it gets with C++03. In C++0x, you'll be able to use the vector's move constructor via std::move to achieve the same thing in a more obvious way.
Another option is to not create a separate vector<MyStruct>, but instead have the code that creates it accept it a a vector<MyStruct>& argument, and operate on it. Then, you add a new empty element to your outer vector<vector<MyStruct>>, and pass a reference to the code that will fill it.
I know this can be accomplished by
using a vector of pointers, but I'd
like to avoid that if possible.
Why?
That would be the most intuitive/readable/maintainable solution and would be much better than any weird hacks anyone comes up with (such as the swap you show).
Tim,
There's a common pattern to solve this. This is called smart pointers, and the best one to use is boost::shared_ptr.
Then, never pass vector by value or store it. Instead, store boost::shared_ptr >. You don't need to care about allocations/deallocations (when the containing vector is destroyed, so will be the others, just as in your code), and you can access the inner members almost the same way. The copy is, however, avoided by means of the smart pointer object's reference counting mechanism.
Let me show you how.
using boost::shared_ptr;
vector<shared_ptr<vector<MyStruct> > vecOfVecs;
shared_ptr<vector<MyStruct> > someStructs(new vector<MyStruct>);
// fill in the vector MyStructs
MyStructs->push_back(some struct.... as you usually do).
//...
vecOfVecs.push_back(someStructs); // Look! No copy!
If you do not already use boost::shared_ptr, I recommend downloading it from boost.org rather than implementing your own. It is really irreplaceable tool, soon to be in the C++ standard library.
You can either do something like vect.push_back(vector<MyStruct>()); and do vect.back().push_back(MyStruct()); or use smart pointers and have a vector of smart pointers to vector<MyStruct>
I think the swap idea is already fine, but can be written much easier:
vecOfVec.push_back(vector<MyStruct>());
vecOfVec.back().swap(someStructs);