This code causes a memory leak because of the vector inside the for a loop. I am currently aware that this can be avoided by not using pointers in C++. However, I need to use pointers in my code. This is an equivalent (proof of concept) of my actual code in C in which I have no other option, but to use pointers. I used C++ for my question because it makes it shorter and easier to visualize. In my actual code, I use a TreeMap instead of a vector and a HashMap instead of a queue from the Libgee library. I don't write my actual code because to make sense of it, I would have to post over a thousand lines of code and that makes it unfeasible for my question. This is a minified equivalent that programmatically causes the same problem of memory leak and has the same logic. In the end, I will use the queue (HashMap in my code) until the user decides to run the loop again to get new values. The reason for which I cannot declare containerVector outside of the loop is that each containerVector needs to be different for each iteration of i.
TL;DR Variables that are created inside a for loop will be needed outside the loop, but references get lost and cause memory leaks. Variables need to be pointers.
Is there a way to avoid a memory leak here while still using pointers? I don't know beforehand how many vectors (which here stand for a TreeMap in my C code) I will need and they must be created as a pointer inside the for a loop.
#include <time.h>
#include <cstdlib>
#include <vector>
#include <queue>
using namespace std;
int main()
{
srand(time(NULL));
queue< vector<int>* >* containerQueue = new queue< vector<int>* >;
// Unknown for loop size before running program
for (int i = 0; i < rand() % 1000 + 10000; i++) {
vector<int>* containerVector = new vector<int>;
// Unknown nested loop size before running the program
for (int j = 0; j < rand() % 100 + 50; j++) {
// If a condition is met
if (j % 2) {
// Add a value to the vector
containerVector->push_back(j);
}
}
// Release memory pointed by containerVector if it was not used
if (containerVector->empty()) delete containerVector;
// Add the vector to the queue
containerQueue->push(containerVector);
// If containerVector were to be freed here, the memory leak issue is solved,
// But containerQueue would then become useless
}
// Use containerQueue here
// The values of containerVector are needed in this part of the code
return 0;
}
Updated to be copy/pasted and run. My bad.
I know I'm probably going to get the whole "this question was asked already" but I spent sometime looking and couldn't find a matching problem. It's very possible I just don't know enough to look in the right place.
When I call InitSortedArray() it runs through a seemingly random number of elements before throwing exception: write access violation. Everytime I run it it stops at a different element number. Any ideas?
#include <array>
#include <iostream>
using namespace std;
int * toSort;
const int SIZE = 100000;
void InitSortedArray()
{
srand(0);
toSort[0] = rand() % 5;
cout << toSort[0];
for (int i = 1; i < SIZE - 1; i++)
{
srand(0);
toSort[i] = toSort[i - 1] + rand() % 5;
cout << toSort[i] << endl;
}
}
void Search()
{
toSort[SIZE];
InitSortedArray();
}
int main()
{
Search();
}
int * toSort;
allocates a pointer to some data yet to be assigned to it. No data is ever assigned. You could
int * toSort = new int[100000];
but that picks up some memory management work you don't need. Any time you use new[] sooner or later you must delete[]. Instead use
const int SIZE = 100000; // place first so we can use it below
int toSort[SIZE];
or the more modern
const int SIZE = 100000; // place first so we can use it below
std::array<int, SIZE> toSort;
to declare an array.
toSort[100000];
in Search does nothing helpful (and is in fact harmful as it invokes Undefined Behaviour by accessing outside the bounds of toSort) and should be removed.
Extra stuff:
srand reseeds and restarts the random number generator. It is only in truly rare circumstances that you want to call it more than once, and in those cases there are many better options than srand and rand.
Place a single call to srand at the top of main and make absolutely certain you want srand(0) as this will always generate the exact same numbers on a given computer. It's great for testing, but not so good if you want a different sequence every time. Typical use is srand(time(NULL)) to seed the generator based on the ever-changing flow of time. That's still not all that good, but good enough for most cases where rand is in use.
It looks like you're using an uninitialized pointer that points to random space, and trying to store elements and access elements in it. Also, your inclusion of "array" doesn't make any sense here. I believe what you want to do here is initialize your toSort array to actually point to a section of memory that you intend to point it to:
int toSort[SIZE];
instead of
int * toSort;
If you're looking to use the STL array (which is likely highly recommendable) then you need to explicitly use it:
std::array<int, SIZE> toSort;
The nice thing about using the STL is it takes care of a lot of the memory access issues you can run into like memory access violation. Another helpful thing from the STL would be vector:
#include <vector>
std::vector<int> toSort;
then later: (this adds an item to the back of the vector)
toSort.push_back(<some number>);
and to access:
int somethingElse = toSort[<index number>];
Arrays: http://en.cppreference.com/w/cpp/container/array
Vectors: http://en.cppreference.com/w/cpp/container/vector
I have an application that is performing some processing on some images.
Given that I know the width/height/format etc. (I do), and thinking just about defining a buffer to store the pixel data:
Then, rather than using new and delete [] on an unsigned char* and keeping a separate note of the buffer size, I'm thinking of simplifying things by using a std::vector.
So I would declare my class something like this:
#include <vector>
class MyClass
{
// ... etc. ...
public:
virtual void OnImageReceived(unsigned char *pPixels,
unsigned int uPixelCount);
private:
std::vector<unsigned char> m_pImageBuffer; // buffer for 8-bit pixels
// ... etc. ...
};
Then, when I received a new image (of some variable size - but don't worry about those details here), I can just resize the vector (if necessary) and copy the pixels:
void MyClass::OnImageReceived(unsigned char *pPixels, unsigned int uPixelCount)
{
// called when a new image is available
if (m_pImageBuffer.size() != uPixelCount)
{
// resize image buffer
m_pImageBuffer.reserve(uPixelCount);
m_pImageBuffer.resize(uPixelCount, 0);
}
// copy frame to local buffer
memcpy_s(&m_pImageBuffer[0], m_pImageBuffer.size(), pPixels, uPixelCount);
// ... process image etc. ...
}
This seems fine to me, and I like that fact that I don't have to worry about the memory management, but it raises some questions:
Is this a valid application of std::vector or is there a more suitable container?
Am I doing the right thing performance-wise by calling reserve and resize?
Will it always be the case that the underlying memory is consecutive so I can use memcpy_s as shown?
Any additional comment, criticism or advice would be very welcome.
Sure, this'll work fine. The one thing you need to worry about is ensuring that the buffer is correctly aligned, if your class relies on a particular alignment; in this case you may want to use a vector of the datatype itself (like float).
No, reserve is not necessary here; resize will automatically grow the capacity as necessary, in exactly the same way.
Before C++03, technically not (but in practice yes). Since C++03, yes.
Incidentally, though, memcpy_s isn't the idiomatic approach here. Use std::copy instead. Keep in mind that a pointer is an iterator.
Starting in C++17, std::byte is the idiomatic unit of opaquely typed storage such as you are using here. char will still work, of course, but allows unsafe usages (as char!) which byte does not.
Besides what other answers mention, I would recommend you to use std::vector::assign rather than std::vector::resize and memcpy:
void MyClass::OnImageReceived(unsigned char *pPixels, unsigned int uPixelCount)
{
m_pImageBuffer.assign(pPixels, pPixels + uPixelCount);
}
That will resize if necessary, and you would be avoiding the unnecessary 0 initialization of the buffer caused by std::vector::resize.
Using a vector in this case is fine. In C++ the storage is guaranteed to be contigious.
I would not both resize and reserve, nor would I memcpy to copy the data in. Instead, all you need to do is reserve to make sure you don't have to reallocate many times, then clear out the vector using clear. If you resize, it will go through and set the values of every element to their defaults -- this is unnecesarry here because you're just going to overwrite it anyway.
When you're ready to copy the data in, don't use memcpy. Use copy in conjunction with back_inserter into an empty vector:
std::copy (pPixels, pPixels + uPixelCount, std::back_inserter(m_pImageBuffer));
I would consider this idiom to be much closer to canonical than the memcpy method you are employing. There might be faster or more efficient methods, but unless you can prove that this is a bottleneck in your code (which it likely won't be; you'll have much bigger fish to fry elsewhere) I would stick with idiomatic methods and leave the premature micro-optimizations to someone else.
I would avoid std::vector as a container for storing an unstructured buffer, as std::vector is profoundly slow when used as a buffer
Consider this (C++14) example (for C++11, you can used shared instead of unique ptrs, but you'll notice slight performance hit in the array example that you don't get from the vectors when running at -O3 or -O2):
#include <array>
#include <chrono>
#include <ctime>
#include <iostream>
#include <memory>
#include <vector>
namespace {
std::unique_ptr<std::array<unsigned char, 4000000>> allocateWithPtr() {
return std::make_unique<std::array<unsigned char, 4000000>>();
}
std::vector<unsigned char> allocateWithVector() {
return std::vector<unsigned char>(4000000);
}
} // namespace
int main() {
auto start = std::chrono::system_clock::now();
for (long i = 0; i < 1000; i++) {
auto myBuff = allocateWithPtr();
}
auto ptr_end = std::chrono::system_clock::now();
for (long i = 0; i < 1000; i++) {
auto myBuff = allocateWithVector();
}
auto vector_end = std::chrono::system_clock::now();
std::cout << "std::unique_ptr = " << (ptr_end - start).count() / 1000.0
<< " ms." << std::endl;
std::cout << "std::vector = " << (vector_end - ptr_end).count() / 1000.0
<< " ms." << std::endl;
}
Output:
bash % clang++ -O3 -std=gnu++14 test.cpp && ./a.out
std::unique_ptr = 0 ms.
std::vector = 0 ms
bash % clang++ -O2 -std=gnu++14 test.cpp && ./a.out
std::unique_ptr = 0 ms.
std::vector = 0 ms.
bash % clang++ -O1 -std=gnu++14 test.cpp && ./a.out
std::unique_ptr = 89.945 ms.
std::vector = 14135.3 ms.
bash % clang++ -O0 -std=gnu++14 test.cpp && ./a.out
std::unique_ptr = 80.945 ms.
std::vector = 67521.1 ms.
Even with no writes or reallocations, std::vector is over 800 times slower than just using a new with a unique_ptr at -O0 and 150 times slower at -O1. What's going on here?
As #MartinSchlott points out, it is not designed for this task. A vector is for holding a set object instances, not an unstructured (from an array standpoint) buffer. Objects have destructors and constructors.
When the vector is destroyed, it calls the destructor for each element in it, even vector will call a destructor for each char in your vector.
You can see how much time it takes just to "destroy" the unsigned chars in this vector with this example:
#include <chrono>
#include <ctime>
#include <iostream>
#include <memory>
#include <vector>
std::vector<unsigned char> allocateWithVector() {
return std::vector<unsigned char>(4000000); }
}
int main() {
auto start = std::chrono::system_clock::now();
for (long i = 0; i < 100; i++) {
auto leakThis = new std::vector<unsigned char>(allocateWithVector());
}
auto leak_end = std::chrono::system_clock::now();
for (long i = 0; i < 100; i++) {
auto myBuff = allocateWithVector();
}
auto vector_end = std::chrono::system_clock::now();
std::cout << "leaking vectors: = "
<< (leak_end - start).count() / 1000.0 << " ms." << std::endl;
std::cout << "destroying vectors = "
<< (vector_end - leak_end).count() / 1000.0 << " ms." << std::endl;
}
Output:
leaking vectors: = 2058.2 ms.
destroying vectors = 3473.72 ms.
real 0m5.579s
user 0m5.427s
sys 0m0.135s
Even when removing the destruction of the vector, it's still taking 2 seconds to just construct 100 of these things.
If you don't need dynamic resizing, or construction & destruction of the elements making up your buffer, don't use std::vector.
std::vector was MADE to be used in such cases. So, yes.
Yes, it is.
reserve is unnecessary in your case.
Yes, it will.
In addition - to ensure a minimum of allocated memory:
void MyClass::OnImageReceived(unsigned char *pPixels, unsigned int uPixelCount)
{
m_pImageBuffer.swap(std::vector<unsigned char>(
pPixels, pPixels + uPixelCount));
// ... process image etc. ...
}
vector::assign does not change the amount of memory allocated, if the capacity is bigger than the amount needed:
Effects:
erase(begin(), end());
insert(begin(), first, last);
Please, consider this:
void MyClass::OnImageReceived(unsigned char *pPixels, unsigned int uPixelCount)
{
// called when a new image is available
if (m_pImageBuffer.size() != uPixelCount) // maybe just < ??
{
std::vector<unsigned char> temp;
temp.reserve(uPixelCount); // no initialize
m_pImageBuffer.swap(temp) ; // no copy old data
}
m_pImageBuffer.assign(pPixels, pPixels + uPixelCount); // no reallocate
// ... process image etc. ...
}
My point is that if you have a big picture and need a litter bigger pic, your old pic will get copy during the reserve and/or resize into the new allocated memmory, the excess of memmory initialized, and then rewrited with the new pic. You colud directly assing, but then you will no be able to use the info you have about the new size to avoid posible reallocations (maybe the implementation of assign is allready optimize for this simple case ????).
It depends.
If you access the data only through iterators and the [] operator, than its okay to use a vector.
If you have to give a pointer to functions which expect a buffer of e.g. bytes. It is not in my opinion. In this case You should use something like
unique_ptr<unsigned char[]> buf(new unsigned char[size])
is it as save as a vector, but instead of a vector you have maximum control of the buffer. A vector may reallocate a buffer or during a method/function call you may unintentionally make a copy of your whole vector. A easily made mistake.
The rule (for me) is. If you have a vector, use it like a vector. If you need a memory buffer, use a memory buffer.
As in a comment pointed out, the vector has a data method. This is C++. The freedom of using a vector as a raw buffer does not mend that you should use it as a raw buffer. In my humble opinion, the intention of a vector was to have a type save buffer with type save access system. For compatibility you can use the internal buffer for calls. The intention was not to use the vector as a smart pointer buffer container. For that, I use the pointer templates, signaling other user of my code that I use this buffer in a raw way. If I use vectors, I use them in the way they are intended to, not the possible ways they offer.
AS I got some blame here for my opinion (not recommendation) I want to add some words to the actual problem the op described.
If he expect always the same picture size, he should, in my opinion, use a unique_ptr, because that's what he is doing with it in my opinion. Using
m_pImageBuffer.resize(uPixelCount, 0);
zeros the buffer first before he copy the pPixel to it, a unnecessary time penalty.
If the pictures he is expecting of different size, he should, in my opinion, not use a vector during following reason. Especially in his code:
// called when a new image is available
if (m_pImageBuffer.size() != uPixelCount)
{
// resize image buffer
m_pImageBuffer.reserve(uPixelCount);
m_pImageBuffer.resize(uPixelCount, 0);
}
he will resize the vector, which is in fact a malloc and copy as long as the images are getting bigger. A realloc in my experience always leads to malloc and copy.
That is the reason I, especially in this situation, recommand the use of a unique_ptr instead of a vector.
I need a functionality of std::set for each element in an array. How can I achieve this functionality?
I started with allocating dynamic array of std set in C++ as follows:
set<int>* entry;
followed by allocation:
entry = (set<int>*)malloc(sizeof(set<int>)*32);
No compilation problem, but the runtime fails with segmentation fault when any element is accessed:
entry[0].insert(23);
Any help is highly appreciated.
What about
#include <set>
#include <vector>
int main()
{
std::vector < std::set<int> > entry(32); // std::vector constructor makes 32 calls to std::set<int> constructor
entry[0].insert(23);
// std::vector destructor makes 32 calls to std::set<int> destructor
}
In c++ you allocate memory with new. The difference from malloc here is that the constructor is called to initialize the memory.
entry = new set<int>[32];
Even though you've allocated storage for 32 std::set's you haven't initalized this span of memory (ie. the constructor of your std::set's has not been called) therefore the memory you are trying to operate on/access in entry[0].insert (23) will cause undefined behavior.
Mixing C++ objects with malloc and it's equivalent is normally (I'm tempted to write "always") considered to be bad practice.
Instead turn to operator new which will allocate memory and handle construction of your object in a proper manner, also remember to delete the memory allocated to release the memory back to your system (and make the object(s) destruct in a true manner).
The proper way to do it in C++
Some answers will contain text saying that you are better of using a std::vector<std::set>, though that isn't really an answer to your question so I'll leave you with this example snippet
int
main (int argc, char *argv[])
{
std::set<int> *entries = new std::set<int> [32];
entries[0].insert (123);
delete [] entries;
}
This is a good question, whose answer is not immediately obvious. The trouble is that each set object wants to be initialized before it is used, whereas your code only allocates raw memory for each set. This fixes it:
#include <vector>
#include <set>
using std::vector;
using std::set;
const int N = 32;
int main() {
vector< set<int> > entry(N);
entry[0].insert(23);
return 0;
}
Not try to use malloc/calloc/realloc etc with c++ classes. Use new.
std::map<int, int> * mp = new std::map<int, int>;
for(int i = 0; i < 999999; i++){
mp->insert(std::pair<int, int>(i, 999999-i ));
}
p("created");
//mp->clear(); - doesn't help either
delete mp;
p("freed");
The problem is: "delete mp" doesn't do anything. To compare:
std::vector<int> * vc = new std::vector<int>;
for(int i = 0; i < 9999999; i++){
vc->push_back(i);
}
p("created");
delete vc;
p("freed");
releases memory. How to release memory from map?
PS: p("string") just pauses program and waits for input.
The RAM used by the application is not a precise way to tell if the memory has been semantically freed.
Freed memory is memory that you can reuse. Sometimes though, you don't observe this freed memory directly in what the OS reports as memory used by our app.
You know that the memory is freed because the semantics of the language say so.
Actually, if the following code doesn't leak:
{
std::map<int, int> some_map;
}
The following code shouldn't leak as well:
{
std::map<int, int>* some_map = new std::map<int, int>();
/* some instructions that do not throw or exit the function */
delete some_map;
}
This applies whatever the type you use with new, as long as the type is well written. And std::map is probably very well written.
I suggest you use valgrind to check for your leaks. I highly doubt that what you observed was a real leak.
As Daniel mentions, RAM used by an application is not necessarily an indicator of a memory leak.
With respect to the behavior you notice regarding vectors, a vector guarantees that the memory layout is contiguous & hence when you create a vector of size 99999999, all the 99999999 elements are laid out in sequence & would constitute a pretty sizable chunk of memory. Deleting this would definitely impact process size. A map behaves differently (as per Wikipedia, its often implemented as a self balancing binary tree) so I guess deleting it causes fragmentation in the process memory space & maybe due to that the process memory does not drop immediately.
The best way to detect memory leaks would be to use a tool like valgrind which will clearly indicate whats wrong.
To pinpoint if you are releasing memory or not, try adding observable effects to the destructors of your object and... observe them.
For instance, instead of a map, create a custom class which emits output when the destructor is invoked.
Something like this:
#include <map>
#include <iostream>
#include <utility>
class Dtor{
int counter;
public:
explicit Dtor(int c):counter(c) {std::cout << "Constructing counter: " << counter << std::endl; }
Dtor(const Dtor& d):counter(d.counter) {std::cout << "Copy Constructing counter: " << counter << std::endl; }
~Dtor(){ std::cout << "Destroying counter: " << counter << std::endl; }
};
int main(){
std::map<int, const Dtor&> * mp = new std::map<int, const Dtor&>;
for (int i = 0; i < 10; ++i){
mp -> insert(std::make_pair(i, Dtor(i)));
}
delete mp;
return 0;
}
You will witness that deleting the pointer invokes the destructor of your objects, as expected.