I have a memory leak in my application which I have been writing with Qt (C++) . And I suspect problem is with that line.
for(int i=0; i<DATA_LENGTH;i++){
cdata1->replace(i,data->at(i));
}
cdata1 is a QVector, data is a QList .
The reason I'm using replace(), I have constant length of data. And I didn't want to create a QVector each time. QVector is initialized on the object constructor with that line:
cdata1 = new QVector<double>(DATA_LENGTH,0);
Qt documentation says
Note that using non-const operators can cause QVector to do a deep
copy.
What I'm asking does replace() function causes a deep copy or how can I understand that?
Deep copy means the whole container, not the elements. As linked just after the sentence you quoted, QVector uses implicit sharing, also known as copy-on-write. read-only copies of the container are cheap, as the internals are shared, until one of the copy is modified:
QVector<A> vec1;
...
QVector<A> vec2 = vec1; //cheap, only copies a pointer internally. vec1 and vec2
int siz2 = vec2.size(); //cheap, doesn't modify vec2, vec1 and vec2 are still the same object, internally
vec2[0] = something; //potentially expensive: modifies vec2, thus vec2 "detaches" from vec1, getting its own copy of vec1's internal data, which is then modified by the assignment.
That's also the reason why creating containers on the heap is rather nonsensical (and unidiomatic) in almost all cases and you should create them on the stack instead.
Yes, it copies your double values. But I don't think double values can be subject to Memory Leaks unless you create them with new?
Anyway, dependent on your surrounding code you can maybe replace that whole block by using
cdata1 = data->toVector();
(see http://doc.qt.nokia.com/latest/qlist.html#toVector)
As an additional hint, you should start using more readable variable names than cdata1 or data. Your variables should describe what they are storing, e.g. if you have a list storing temperature data points it should be called something like temperatureDataPoints or temperatureDataPointList. It involves more typing than "data" of course, but you won't regret using more readable names if you look at your code in a year or so.
Related
I want to build up a map of devices such that the map contains:
QString 'DeviceID' and QVector 'Command List'
Currently I have the QMap as follows:
QMap<QString, QVector<QString> *> devices;
QVector<QString> *pCommands= new QVector<QString>;
// :
// Fill pCommands with lots of data here
// :
devices.insert(RadioID, pCommands);
But I am wondering if this is actually any better then:
QMap<QString, QVector<QString>> devices;
QVector<QString> commands;
// :
// Fill commands with lots of data here
// :
devices.insert(RadioID, commands);
I am sure that I read somewhere that Qt does something quite efficient when copying data. I am not seeing many people using pointers with Qt and it seems messy that I have to go through the QMap deleting all the QVector's at the end...
I am using c++11, so maybe some kind of move semantic may work here?
EDIT
I have modified the comments in the code to show that the vector is not empty.
Also I would state that I do not need to change the data once it is stored.
There is no reason to consider manually allocating the vectors to be better.
Sure, you only need to copy a pointer, rather than a vector, but an empty vector is still quite fast to copy. The biggest gain of storing objects rather than pointers is that you don't need to do manual memory management.
I am using c++11, so maybe some kind of move semantic may work here?
If QMap::insert supports move semantics, and if QVector is move-constructible like their standard library counterparts, then you could indeed move the vector into the map. But moving an empty vector is just as fast as copying.
If QMap has an emplace like function std::map does, then you could even construct the vector in-place without even a move.
I'm not familiar with Qt, though so you'll need to verify those details from the documentation. Whether Qt supports move semantics doesn't change the fact that manual memory management is a pain.
Edit: according to the documentation QVector appears to be movable, but QMap does not support move semantics. However, as Arpegius and the documentation point out, QVector does copy-on-write optimization, so as long as the copied vector is not modified, then the data won't be copied. None of this matters really, when copying an empty vector.
Edit again
If the added vectors are full of data, then copying is indeed quite expensive unless it remains unmodified. Moving would remedy that, but QMap appears not to support that. There is another trick, though: Insert an empty vector, and then swap the full vector with the empty one in the map.
The simplest and pretty much idiomatic way to do it would be:
QMap<QString, QVector<QString>> devices;
// Get the vector constructed in the map itself.
auto & commands = devices[RadioID];
// Fill commands with lots of data here - a bit faster
commands.resize(2);
commands[0] = "Foo";
commands[1] = "Bar";
// Fill commands with lots of data here - a bit slower
commands.append("Baz");
commands.append("Fan");
You'll see this pattern often in Qt and other code, and it's the simplest way to do it that works equally well on C++98 and C++11 :)
This way you're working on the vector that's already in the map. The vector itself is never copied. The members of the vector are copied, but they are implicitly shared, so copying is literally a pointer copy and an atomic reference count increment.
Assume I have some object, such as:
std::map<int, std::vector<double> > some_map;
Simple question: is it more efficient to do the following
std::vector<double> vec = some_map[some_index];
or referencing it
std::vector<double>& vec = some_map[some_index];
Can anyone explain in short what typically happens behind the scenes here?
Thanks very much in advance!
The two have different semantics, and aren't interchangeable.
The first gives you a copy, which you can modify however you
wish, without changing anything in the map. The second gives
you a reference to the data element in the map; any
modifications modify the contents of the map. Also, although
probably not an issue, be aware that if the map is destructed
before the reference goes out of scope, the reference will
dangle.
With regards to performance, it depends on what's in the vector,
and what you do with it later; in most cases, the reference will
probably have better performance, but you shouldn't worry about
it until the profiler says you have to. (And if you do use the
reference, make it const, unless you really do want to be able
to modify the contents of the map.)
Creating a reference is more efficient, but you should note that these two statements are different in semantics and have different behaviors.
If you do
std::vector<double> vec = some_map[some_index];
The copy constructor of std::vector is called to copy the whole vector some_map[some_index] into vec. In this way, you get a fresh new vector vec. They are independent objects and any changes to vec does not affect the original map.
If you use
std::vector<double>& vec = some_map[some_index];
then vec refers directly to some_map[some_index] and copy is avoided. However, be aware that if you later change vec, the change will be reflected in both vec and some_map[some_index] since they refer to the same object. To prevent undesirable changes, it is safer to use a const reference:
const std::vector<double>& vec = some_map[some_index];
Referencing is much more efficient, both in terms of memory used and cpu cycles. Your first line of code makes a copy of the vector, which includes copying every item in the vector. In the second, you're simply referring to the existing vector. No copies are made.
I'd like to use a std::vector to control a given piece of memory. First of all I'm pretty sure this isn't good practice, but curiosity has the better of me and I'd like to know how to do this anyway.
The problem I have is a method like this:
vector<float> getRow(unsigned long rowIndex)
{
float* row = _m->getRow(rowIndex); // row is now a piece of memory (of a known size) that I control
vector<float> returnValue(row, row+_m->cols()); // construct a new vec from this data
delete [] row; // delete the original memory
return returnValue; // return the new vector
}
_m is a DLL interface class which returns an array of float which is the callers responsibility to delete. So I'd like to wrap this in a vector and return that to the user.... but this implementation allocates new memory for the vector, copies it, and then deletes the returned memory, then returns the vector.
What I'd like to do is to straight up tell the new vector that it has full control over this block of memory so when it gets deleted that memory gets cleaned up.
UPDATE: The original motivation for this (memory returned from a DLL) has been fairly firmly squashed by a number of responders :) However, I'd love to know the answer to the question anyway... Is there a way to construct a std::vector using a given chunk of pre-allocated memory T* array, and the size of this memory?
The obvious answer is to use a custom allocator, however you might find that is really quite a heavyweight solution for what you need. If you want to do it, the simplest way is to take the allocator defined (as the default scond template argument to vector<>) by the implementation, copy that and make it work as required.
Another solution might be to define a template specialisation of vector, define as much of the interface as you actually need and implement the memory customisation.
Finally, how about defining your own container with a conforming STL interface, defining random access iterators etc. This might be quite easy given that underlying array will map nicely to vector<>, and pointers into it will map to iterators.
Comment on UPDATE: "Is there a way to construct a std::vector using a given chunk of pre-allocated memory T* array, and the size of this memory?"
Surely the simple answer here is "No". Provided you want the result to be a vector<>, then it has to support growing as required, such as through the reserve() method, and that will not be possible for a given fixed allocation. So the real question is really: what exactly do you want to achieve? Something that can be used like vector<>, or something that really does have to in some sense be a vector, and if so, what is that sense?
Vector's default allocator doesn't provide this type of access to its internals. You could do it with your own allocator (vector's second template parameter), but that would change the type of the vector.
It would be much easier if you could write directly into the vector:
vector<float> getRow(unsigned long rowIndex) {
vector<float> row (_m->cols());
_m->getRow(rowIndex, &row[0]); // writes _m->cols() values into &row[0]
return row;
}
Note that &row[0] is a float* and it is guaranteed for vector to store items contiguously.
The most important thing to know here is that different DLL/Modules have different Heaps. This means that any memory that is allocated from a DLL needs to be deleted from that DLL (it's not just a matter of compiler version or delete vs delete[] or whatever). DO NOT PASS MEMORY MANAGEMENT RESPONSIBILITY ACROSS A DLL BOUNDARY. This includes creating a std::vector in a dll and returning it. But it also includes passing a std::vector to the DLL to be filled by the DLL; such an operation is unsafe since you don't know for sure that the std::vector will not try a resize of some kind while it is being filled with values.
There are two options:
Define your own allocator for the std::vector class that uses an allocation function that is guaranteed to reside in the DLL/Module from which the vector was created. This can easily be done with dynamic binding (that is, make the allocator class call some virtual function). Since dynamic binding will look-up in the vtable for the function call, it is guaranteed that it will fall in the code from the DLL/Module that originally created it.
Don't pass the vector object to or from the DLL. You can use, for example, a function getRowBegin() and getRowEnd() that return iterators (i.e. pointers) in the row array (if it is contiguous), and let the user std::copy that into its own, local std::vector object. You could also do it the other way around, pass the iterators begin() and end() to a function like fillRowInto(begin, end).
This problem is very real, although many people neglect it without knowing. Don't underestimate it. I have personally suffered silent bugs related to this issue and it wasn't pretty! It took me months to resolve it.
I have checked in the source code, and boost::shared_ptr and boost::shared_array use dynamic binding (first option above) to deal with this.. however, they are not guaranteed to be binary compatible. Still, this could be a slightly better option (usually binary compatibility is a much lesser problem than memory management across modules).
Your best bet is probably a std::vector<shared_ptr<MatrixCelType>>.
Lots more details in this thread.
If you're trying to change where/how the vector allocates/reallocates/deallocates memory, the allocator template parameter of the vector class is what you're looking for.
If you're simply trying to avoid the overhead of construction, copy construction, assignment, and destruction, then allow the user to instantiate the vector, then pass it to your function by reference. The user is then responsible for construction and destruction.
It sounds like what you're looking for is a form of smart pointer. One that deletes what it points to when it's destroyed. Look into the Boost libraries or roll your own in that case.
The Boost.SmartPtr library contains a whole lot of interesting classes, some of which are dedicated to handle arrays.
For example, behold scoped_array:
int main(int argc, char* argv[])
{
boost::scoped_array<float> array(_m->getRow(atoi(argv[1])));
return 0;
}
The issue, of course, is that scoped_array cannot be copied, so if you really want a std::vector<float>, #Fred Nurk's is probably the best you can get.
In the ideal case you'd want the equivalent to unique_ptr but in array form, however I don't think it's part of the standard.
I just started learning about pointers in C++, and I'm not very sure on when to use pointers, and when to use actual objects.
For example, in one of my assignments we have to construct a gPolyline class, where each point is defined by a gVector. Right now my variables for the gPolyline class looks like this:
private:
vector<gVector3*> points;
If I had vector< gVector3 > points instead, what difference would it make? Also, is there a general rule of thumb for when to use pointers? Thanks in advance!
The general rule of thumb is to use pointers when you need to, and values or references when you can.
If you use vector<gVector3> inserting elements will make copies of these elements and the elements will not be connected any more to the item you inserted. When you store pointers, the vector just refers to the object you inserted.
So if you want several vectors to share the same elements, so that changes in the element are reflected in all the vectors, you need the vectors to contain pointers. If you don't need such functionality storing values is usually better, for example it saves you from worrying about when to delete all these pointed to objects.
Pointers are generally to be avoided in modern C++. The primary purpose for pointers nowadays revolves around the fact that pointers can be polymorphic, whereas explicit objects are not.
When you need polymorphism nowadays though it's better to use a smart pointer class -- such as std::shared_ptr (if your compiler supports C++0x extensions), std::tr1::shared_ptr (if your compiler doesn't support C++0x but does support TR1) or boost::shared_ptr.
Generally, it's a good idea to use pointers when you have to, but references or alternatively objects objects (think of values) when you can.
First you need to know if gVector3 fulfils requirements of standard containers, namely if the type gVector3 copyable and assignable. It is useful if gVector3 is default constructible as well (see UPDATE note below).
Assuming it does, then you have two choices, store objects of gVector3 directly in std::vector
std::vector<gVector3> points;
points.push_back(gVector(1, 2, 3)); // std::vector will make a copy of passed object
or manage creation (and also destruction) of gVector3 objects manually.
std::vector points;
points.push_back(new gVector3(1, 2, 3));
//...
When the points array is no longer needed, remember to talk through all elements and call delete operator on it.
Now, it's your choice if you can manipulate gVector3 as objects (you can assume to think of them as values or value objects) because (if, see condition above) thanks to availability of copy constructor and assignment operator the following operations are possible:
gVector3 v1(1, 2, 3);
gVector3 v2;
v2 = v1; // assignment
gVector3 v3(v2); // copy construction
or you may want or need to allocate objects of gVector3 in dynamic storage using new operator. Meaning, you may want or need to manage lifetime of those objects on your own.
By the way, you may be also wondering When should I use references, and when should I use pointers?
UPDATE: Here is explanation to the note on default constructibility. Thanks to Neil for pointing that it was initially unclear. As Neil correctly noticed, it is not required by C++ standard, however I pointed on this feature because it is an important and useful one. If type T is not default constructible, what is not required by the C++ standard, then user should be aware of potential problems which I try to illustrate below:
#include <vector>
struct T
{
int i;
T(int i) : i(i) {}
};
int main()
{
// Request vector of 10 elements
std::vector<T> v(10); // Compilation error about missing T::T() function/ctor
}
You can use pointers or objects - it's really the same at the end of the day.
If you have a pointer, you'll need to allocate space for the actual object (then point to it) any way. At the end of the day, if you have a million objects regardless of whether you are storing pointers or the objects themselves, you'll have the space for a million objects allocated in the memory.
When to use pointers instead? If you need to pass the objects themselves around, modify individual elements after they are in the data structure without having to retrieve them each and every time, or if you're using a custom memory manager to manage the allocation, deallocation, and cleanup of the objects.
Putting the objects themselves in the STL structure is easier and simpler. It requires less * and -> operators which you may find to be difficult to comprehend. Certain STL objects would need to have the objects themselves present instead of pointers in their default format (i.e. hashtables that need to hash the entry - and you want to hash the object, not the pointer to it) but you can always work around that by overriding functions, etc.
Bottom line: use pointers when it makes sense to. Use objects otherwise.
Normally you use objects.
Its easier to eat an apple than an apple on a stick (OK 2 meter stick because I like candy apples).
In this case just make it a vector<gVector3>
If you had a vector<g3Vector*> this implies that you are dynamically allocating new objects of g3Vector (using the new operator). If so then you need to call delete on these pointers at some point and std::Vector is not designed to do that.
But every rule is an exception.
If g3Vector is a huge object that costs a lot to copy (hard to tell read your documentation) then it may be more effecient to store as a pointer. But in this case I would use the boost::ptr_vector<g3Vector> as this automatically manages the life span of the object.
I am extending the Visual Studio 2003 debugger using autoexp.dat and a DLL to improve the way it displays data in the watch window. The main reason I am using a DLL rather than just the basic autoexp.dat functionality is that I want to be able to display things conditionally. e.g. I want to be able to say "If the name member is not an empty string, display name, otherwise display [some other member]"
I'm quite new to OOP and haven't got any experience with the STL. So it might be that I'm missing the obvious.
I'm having trouble displaying vector members because I don't know how to get the pointer to the memory the actual values are stored in.
Am I right in thinking the values are stored in a contiguous block of memory? And is there any way to get access to the pointer to that memory?
Thanks!
[edit:] To clarify my problem (I hope):
In my DLL, which is called by the debugger, I use a function called ReadDebuggeeMemory which makes a copy of the memory used by an object. It doesn't copy the memory the object points to. So I need to know the actual address value of the internal pointer in order to be able to call ReadDebuggeeMemory on that as well. At the moment, the usual methods of getting the vector contents are returning garbage because that memory hasn't been copied yet.
[update:]
I was getting garbage, even when I was looking at the correct pointer _Myfirst because I was creating an extra copy of the vector, when I should have been using a pointer to a vector. So the question then becomes: how do you get access to the pointer to the vector's memory via a pointer to the vector? Does that make sense?
The elements in a standard vector are allocated as one contiguous memory chunk.
You can get a pointer to the memory by taking the address of the first element, which can be done is a few ways:
std::vector<int> vec;
/* populate vec, e.g.: vec.resize(100); */
int* arr = vec.data(); // Method 1, C++11 and beyond.
int* arr = &vec[0]; // Method 2, the common way pre-C++11.
int* arr = &vec.front(); // Method 3, alternative to method 2.
However unless you need to pass the underlying array around to some old interfaces, generally you can just use the operators on vector directly.
Note that you can only access up to vec.size() elements of the returned value. Accessing beyond that is undefined behavior (even if you think there is capacity reserved for it).
If you had a pointer to a vector, you can do the same thing above just by dereferencing:
std::vector<int>* vecptr;
int* arr = vecptr->data(); // Method 1, C++11 and beyond.
int* arr = &(*vecptr)[0]; // Method 2, the common way pre-C++11.
int* arr = &vec->front(); // Method 3, alternative to method 2.
Better yet though, try to get a reference to it.
About your solution
You came up with the solution:
int* vMem = vec->_Myfirst;
The only time this will work is on that specific implementation of that specific compiler version. This is not standard, so this isn't guaranteed to work between compilers, or even different versions of your compiler.
It might seem okay if you're only developing on that single platform & compiler, but it's better to do the the standard way given the choice.
Yes, The values are stored in a contiguous area of memory, and you can take the address of the first element to access it yourself.
However, be aware that operations which change the size of the vector (eg push_back) can cause the vector to be reallocated, which means the memory may move, invalidating your pointer. The same happens if you use iterators.
vector<int> v;
v.push_back(1);
int* fred = &v[0];
for (int i=0; i<100; ++i)
v.push_back(i);
assert(fred == &v[0]); // this assert test MAY fail
I couldn't implement the solutions suggested by Gman and Roddy once I changed from having a vector variable to having a pointer to a vector, quite probably because I'm just too clueless.
I have found the pointer I was looking for though: _Myfirst
So my code works when I use
std::vector<int>* vec;
int *vMem = vec->_Myfirst;
I'm surprised that I've got access to _Myfirst. I would have expected it to be a private member. But obviously not...
You couldn't implement the solutions suggested by Gman and Roddy because they answers, seemingly correct, but has nothing to do with you situation.
When developing an Addin Dll for Expression Evaluation (EE) to use with autoexp.dat, you are getting one raw pointer to the vector object from your process. But the Dll is running in the process of Visual Studio Debugger, so the only way to access the vector's data is through the ReadDebuggeeMemory function.
You should use that raw pointer and ReadDebuggeeMemory() to read the memory occupied by an object in Debugee process to the local buffer/object. (local for Debugger process, meaning your dll). Then get necessary data from local object, if it is a pointer, than you have to use ReadDebuggeeMemory() again, to read the object pointed by it to another local location. And so on.
For vector's (I haven't done it myself to be more specific), you should
1. Read (ReadDebuggeeMemory()) the vector object to the local one.
2. Get the size of the vector (I assume it is a non pointer data in vector class)
3. Get the pointer to that the contiguous memory location, and read (ReadDebuggeeMemory()) that block to the local buffer/block. Block size is sizeof(VectorType) * VectorSize in bytes. (if it is not continuous, then things will be more complicated, but the idea is the same).
4. Since your vector contains pointers, you have to read (ReadDebuggeeMemory()) every item separately into local memory.