Additional thanks extend to Daniel Newby for answering my memory usage question (and Martin York for explaining it a bit more). It is definitely the answer I was looking for, but more of my other questions were answered by others.
Thanks everyone
for clearing up all of my concerns. Very pleased to see things running how I expect them to run.
I've run into something that I'm not exactly sure about.
In my program, I'm not using malloc() or free(). I'm making instances of my classes with new and I've made sure each one runs it's destructor when it's delete'd, however, there are no free() calls or even setting their pointers (to things inside a global scope, or other classes) to NULL or 0.
What I mean by "I've made sure", is not that I call each destructor. I only use delete to call on the destructor to run, but I have variables that increase by 1 everytime an object is created, and everytime it's destructor is run. This is how I've made sure the amount of objects I created are equal to the amount of destructors called.
Should I be using malloc() and free() anyway? Should I be NULLing pointers to things that I still want to exist?
A second question is why, when I look at my task manager, does my process never "drop" memory? It used to never stop gaining, and then I started deleting everything properly. Or so I thought.
Wouldn't free() or delete make the memory usage go down?
What practices should I pursue about malloc'ing and free'ing memory with linked lists?
There's rarely a reason to use malloc() and free() in a C++ program. Stick with new and delete. Note that unlike languages with garbage collection, setting a pointer to NULL or 0 in C++ has nothing to do with deallocating the memory.
you should be using delete with a new and free with a malloc. delete will call the class' destructor so you don't have to explicitly call it. The purpose of the destructor is to release and resources the class might have and delete will free the memory as well.
The only time you should explicetly use the destructor is when you have initialized your object through placement new. You should put yourself in a position where the compiler generated code releases your resources -- read this article on the C++ idiom : resource acquisition is initialization.
Also setting the pointer of a class to null does nothing, there is no garbage collector in the background cleaning up your memory. If you don't free dynamic memory in C++ it will be "leaked" memory -- i.e., there are no links to the memory and it will never be reclaimed till the process exits.
p.s., once again do not mix the pairs of the memory allocation functions.
edt: don't implement linked lists, use the containers provided by the Standard template library. If you feel you need better performance use the intrusive containers from boost.
You should use new and delete in preference to malloc()/calloc()/realloc() and free().
If you're creating linked lists you should use std::list. Also, look into std::vector
As far as the apparent memory usage of your application: it's quite likely that memory is not returned to the system until the application exits.
new and delete can more or less to be considered the C++ versions of malloc and free. So stick within one pair or another, i.e. if a pointer was created with new, it should be released with delete, which ensures the destructor call you mentioned. The malloc/free pair are not C++ aware, and just allocate and release a block of memory with no attached constructor/destructor semantics.
Yes indeed I consider it good form to set pointers to NULL or zero when they've been freed or deleted. During debugging I also sometimes set them to something characteristic like 0xdeadbeef so they stand out.
It's likely that the OS "memory usage" is reflecting the entire size of your process' heap, rather than your memory manager's idea of how much memory is allocated. When the allocator discovers that it doesn't have enough heap space, it grows the heap, and this will be reflected in the "memory usage" you're looking at. In theory it would be possible to shrink the heap accordingly on memory release, but this doesn't always happen. Thus you may only see the memory usage grow, and never shrink.
"Should I be using malloc() and free() anyway?"
No, in most cases. Stick with one form of memory management only. Trying to use both means that you will inevitably screw up and delete a malloc()ed item, or free() a newed item, giving a subtle bug. Stick with one form and you fix these bugs in advance.
"Wouldn't free() or delete make the memory usage go down?"
The OS allocates memory in pages, often 4 kiB in size. As long as a single byte of the page is still in use, it will not be returned to the OS. You are probably allocating many small items, but not deallocating all of them. (There are ways to help with a custom new/delete, but they are generally not worth the effort.)
In my program, I'm not using malloc()or free(). I'm making instances of my classes with new and I've made sure each one runs it's destructor when it's delete'd,
That is scary. You should not need to make anything run its destructor. It is allocated (new) it is destroyed (delete) the constructor is run atomatically on new and the destructor is run automatically on delete.
however, there are no free() calls or even setting their pointers (to things inside a global scope, or other classes) to NULL or 0.
There is no need to use malloc/free in C++ code (there are a few situations where you are using C libs that require malloced memory but they are explicitly documented and few).
Technically there is no need to set a pointer to NULL after you delete it.
It is good technique for a variable to go out of scope just after it is deleted so it can not accidently be re-used. If for some reason the pointer variable lives (ie does not go out of scope) for a long time after you call delete then it is usefull to set it to NULL so that it is no accidently re-used.
Should I be using malloc() and free() anyway? Should I be NULLing pointers to things that I still want to exist?
No and No.
Note: C++ unlike Java does not keep track of how many pointers point at an object.
If you have more than one pointer pointing at an object you need to use smart pointers (you should be using smart pointers anyway).
A second question is why, when I look at my task manager, does my process never "drop" memory? It used to never stop gaining, and then I started deleting everything properly. Or so I thought.
The application never releases back to the OS (on most OS's in normal situations).
So the memory will never go down (until the application exits).
Internally the memory management tracks all the frees so that the memory can be re-used.
But if it runs out it will ask the OS for more and thus in the task manager the memory allocation will go up (this will not be returned to the OS).
Wouldn't free() or delete make the memory usage go down?
No.
What practices should I pursue about malloc'ing and free'ing memory with linked lists?
You should use Smart Pointers so you don't need to worry about when to delete an object.
They also make your code exception safe.
But if you using pointers. call delete to remove an element then set is value to NULL (the pointer scope is alive long after the delete is called).
Note: A boost:shared_pointer is very similar to a Java pointer. It tracks the number of pointers and deletes the object when the last reference to the object is destroyed. No need for you to do any deleting (just like Java) and all you actually need to do is call new (just like Java).
Related
I just had a quick question about using vectors when programming in c++. If you create a vector do you need to clear it or delete it before the program closes or will the vector be deleted and the memory freed when you close the program?
Thanks
Steven
This depends on how you declared your vector.
If you use RAII (your vector is a stack object, std::vector<...> myVec;)
Or if you created it on the heap (std::vector<...>* myVec = new std::vector<...>();)
In the first case you will not have to do anything, because as soon as the vector runs out of scope (which is always the case when the application terminates), its desctructor will be called and it will clean up its ressoruces.
In the second case youll have to call delete myVec; so 1) the vector becomes uninitialized (the same as in the first case, destructor gets called etc.) and 2) the memory of your vector becomes freed.
If you simply declare a vector, eg.
std::vector<int> vec;
It has automatic storage duration and the destructor will be called when it goes out of scope.
If you have used new, then you should use delete.
If you didn't create it with new (which, usually, you shouldn't), then there's no need to do anything. It will be destroyed automatically according at the end of its lifetime, and the destructor will automatically release its memory.
If you created it with new then, for good order, you should make sure you delete it before the program ends. If you don't, then most modern operating systems will release the memory anyway; but allowing it to leak might make it harder to track down problematic leaks, and could cause issues on exotic operating systems.
In either case, there's no need to clear it; the destructor will do that for you.
Your question touches two different things. OS and life of object within the program
OS part: Generally speaking, in modern OSes you do not have to do anything. Any decent modern operating system dealloactes memory after process ends. Even the dynamically allocated/requested.
This is one of the reasons why windows 95 was so faulty and unstable over time. It did not do it. So if the application would leak out memory (and apps did and do it pretty much all the time), it would be hogged permanently.
c++: Now when you have modern c++ containers and well designed and implemented objects, they all have written destructors. So they are polite and nice and they free their own memory. And they are destructed basically on two conditions. If you do not allocate it dynamically e.g. via new compiler knows when the life of object ends. It goes out of scope. And it calls the destructor automatically. If you use new you need to call delete afterwards.
So the answer is. The object can clean up after it self not only when application closes, but when the object dies. If you however leak memory, you still should be safe, then the process finishes.
When you "close a program" it is the responsibility of the operating system to free any memory the program was using. No matter how you created your vector, when you "close the program" all the memory will be restored. You do not have to worry about anything.
However, in large programs, you should not rely on a user "closing the program" to free up memory used. If you use the keyword "new" you are responsible for calling "delete." If you declared a vector without "new" you do not have to do anything as the destructor will automatically handle memory deallocation for you.
It is far more common to see something like this:
std::vector<MyType*> container;
Rather than something like this:
std::vector<MyType> *container = new std::vector<MyType>;
In fact, I have never in my 15+ years of programming seen the latter. The latter would defeat the purpose of an STL container in my opinion. I guess it is possible that someone might still be doing some programming for windows 95, but that seems highly unlikely. The most common issue with any STL container is when you have a container of pointers. Since containers cannot be heterogeneous, it is fairly common to see or use containers of pointers to the base, interface class of objects. In that case, you would have to loop and delete every object prior to erasing things from the vector. If the vector loses scope it will not know how to delete the objects that are pointed to. It will only clean up the memory for the pointers that have been added.
If you do the following, in a function or in a class declaration then you have no memory cleanup obligations. The STL implementation either works properly or it doesn't. Either way, there is nothing more that the programmer could do except to define your own custom allocator:
std::vector<MyType> container; // then add some stuff to it
I'm not going to get into boost or C++11, but FYI there are special container types within boost that are designed to make the user of containers of pointers easier. I'm not sure if there are any new C++ 11 features that provide that capability. Personally, I would recommend that you not worry about whether OSs will do the right thing when a program leaks memory. Don't leak memory. If you are working with an embedded OS, read the documentation and make sure you know whether its STL implementation has any limitations.
In some classes I have an static std::map with pointers inside. My question is if I need to delete at the end of the program or this memory is automatically freed. My concern is if the pointers stored inside are correctly deleted through our destructors when std::map is deleted.
Thanks.
If the map contains pointers that were allocated with new (or new[], or malloc), then each pointer needs a corresponding delete (or delete[], or free).
The map's destructor wont know what to do with a bald pointer. Consider using a smart pointer that has appropriate move semantics like a boost smart pointer or if you've got a very new compiler, one of the C++0x smart pointers. However, do not use the current standard's std::auto_ptr, inside of STL containers. See this thread for why.
Edit:
As Billy ONeal pointed out, boost::ptr_map is also designed exactly for this purpose.
If I understand the situation correctly, you don't delete the map itself. But you probably need to delete the objects the map is pointing to. It would probably be a really good idea to use a smart pointer such as Boost shared_ptr in your map instead of native pointers. Then the objects would be cleaned up automatically.
Edit:
Using Boost ptr_map might be an even better idea.
The memory is "automatically freed", in the sense that the entire process memory is freed, but the destructors of the objects pointed to will not be called. This can cause a resource leak, if you use RAII.
std::map never calls delete on it's members. Assuming you're working with a relatively recent operating system, the OS will reclaim the memory occupied by the members on process termination, but the destructors will not run.
If you have a map of pointers, then the answer is 'no', your destructors will not be called, but 'yes', the memory will be freed at the end of process execution. All memory allocated by a process is always freed by the Operating System when a process exits (even if it crashes), but destructors might not be called.
A memory "leak" is where memory is unintentionally not deleted over a period of time, and ends up reducing as the process continues. If it is a type of process that runs for a very long period of time, eg a server that is rarely restarted, this can be a major problem.
Memory leak detectors will pick up on any memory that is allocated and not deleted by a programming call, so valgrind, etc. will report this as a leak.
It is as well to check your code with programs like valgrind, and therefore the less that "gets in the way", the easier it will be to spot real leaks. Therefore my advice is not do just let the system clean up the memory, or singletons, etc, for you when you have allocated a pointer with new (or malloc or new[]).
You can have a "clean-up" routine to do this. Just have an object in the scope of your map that has a deleter (as it will be deleted when it exits) that will clean up the pointers in the map. As you need your object to be deleted first it should be declared later than the map.
Like in any case of storage class which stores pointers: you are responsible to deallocate memory they point to. Storage class is responsible only to clean its own resources. Relying on reclaiming memory by OS at the process termination is a bad practice.
How to check if memory to which pointer p points has been succesfully deallocated?
In few words: you can't.
Check out tools like Valgrind to help you debugging memory leaks issues.
Some other things you should consider:
Use smart pointers so that you don't have do think about memory management,
Set your pointers to 0 after you free them, so that a further delete has no effect,
Use standard classes (vector, ...) instead of rolling your own,
Finally, don't use pointers (actually you almost can)
Sorry, very short answer "You can't"
Use IBM rational purify tool to check correct deallocation of memory.
Define successfully! Define deallocated!
After deallocating memory (whether it is free or delete) you must not use that pointer again. All other assumptions are irrelevant.
After all, you call the C/C++ runtime to deallocate memory, but the C/C++ runtime also calls functions of the operating system to free the page. You could even have a custom memory allocator on top of the C/C++ runtime that e.g. uses caching to implement a faster memory allocation algorithm.
All of these layers may keep the deallocated memory for themselves (because of fragmentation or just because they like to keep it to themselves) or may tell the underlying layer to deallocate it. Anything can happen, just don't use that pointer anymore.
Some tools which are doing static code analysis can point some problems regarding the memory deallocation.
Use valgrind to check whether you have memory leaks
Avoid raw pointers - use smart pointers instead
Exception handling. I.e. try/catch blocks.
In C++, you can safely assume deallocation never fails. Destructors must not throw exceptions, and the actual memory reserved should never fail to be released, so given those two points, nothing can go wrong.
However, if you delete a pointer that has already been deleted, your program will probably crash. This isn't a problem with deallocation itself though - the original delete worked successfully. It's a problem with your program's memory management if it tries to delete a pointer twice, but that's rarely necessary with modern STL and smart pointers like std::vector, std::unique_ptr, etc...
In C++, when you make a new variable on the heap like this:
int* a = new int;
you can tell C++ to reclaim the memory by using delete like this:
delete a;
However, when your program closes, does it automatically free the memory that was allocated with new?
Yes, it is automatically reclaimed, but if you intend to write a huge program that makes use of the heap extensively and not call delete anywhere, you are bound to run out of heap memory quickly, which will crash your program.
Therefore, it is a must to carefully manage your memory and free dynamically allocated data with a matching delete for every new (or delete [] if using new []), as soon as you no longer require the said variable.
When the process is terminated the memory is reclaimed back by the OS. Of course this argument shouldn't in any case be used to not perform proper memory management by the program.
Don't let people tell you yes. C++ has no concept of an OS, so to say "yes the OS will clean it up" is no longer talking about C++ but about C++ running on some environment, which may not be yours.
That is, if you dynamically allocate something but never free it you've leaked. It can only end its lifetime once you call delete/delete[] on it. On some OS's (and almost all desktop OS's), memory will be reclaimed (so other programs may use it.) But memory is not the same as resource! The OS can free all the memory it wants, if you have some socket connection to close, some file to finish writing to, etc, the OS might not do it. It's important not to let resources leak. I've heard of some embedded platforms that won't even reclaim the memory you've not freed, resulting in a leak until the platform is reset.
Instead of dynamically allocating things raw (meaning you're the one that has to explicitly delete it), wrap them into automatically allocated (stack allocated) containers; not doing so is considered bad practice, and makes your code extremely messy.
So don't use new T[N], use std::vector<T> v(N);. The latter won't let a resource leak occur. Don't use new T;, use smart_ptr p(new T);. The smart pointer will track the object and delete it when it's know longer used. This is called Scope-bound Resource Management (SBRM, also known as the dumber name Resource-Acquisition is Initialization, or RAII.)
Note there is no single "smart_ptr". You have to pick which one is best. The current standard includes std::auto_ptr, but it's quite unwieldy. (It cannot be used in standard containers.) Your best bet is to use the smart pointers part of Boost, or TR1 if your compiler supports it. Then you get shared_ptr, arguably the most useful smart pointer, but there are many others.
If every pointer to dynamically allocated memory is in an object that will destruct (i.e., not another object that is dynamically allocated), and that object knows to free the memory, that pointer is guaranteed to be freed. This question shouldn't even be a problem, since you should never be in a position to leak.
No, when the program exits ("closes") the dynamically allocated memory is left as is
EDIT:
Reading the other answers, I should be more precise. The destructors of dynamically allocated objects will not run but the memory will be reclaimed anyway by any decent OS.
PS: The first line should read
int* a = new int;
No, it's your responsibility to free it. Also, a must be a pointer, so it should be:
int *a = new int;
delete a;
This excellent answer by Brian R. Bondy details why it's good practice to free the memory allocated by a.
It is important to explicitly call
delete because you may have some code
in the destructor that you want to
execute. Like maybe writing some data
to a log file. If you let the OS free
your memory for you, your code in your
destructor will not be executed.
Most operating systems will deallocate
the memory when your program ends. But
it is good practice to deallocate it
yourself and like I said above the OS
won't call your destructor.
As for calling delete in general, yes
you always want to call delete, or
else you will have a memory leak in
your program, which will lead to new
allocations failing.
When your process terminates, the OS does regain control of all resources the process was using, including memory. However, that, of course, will not cause C++'s destructors to be necessarily run, so it's not a panacea for not explicitly freeing said resources (though it won't be a problem for int or other types with noop dtors, of course;-).
When, if ever, can delete and free be used interchangeably in C++?
My concern is as follows: Say there is an incorrect mixup in the use of malloc/ free and
new/ delete (not to mention new[]/ delete[]). However delete and free doing the same thing;
Fortuitously so this goes uncaught in testing. Later this may lead to a crash in production.
How can I enforce some kind of check to prevent this? Can I be warned if the two are mixed up?
If not at compile time, perhaps some code instrumentation at run time? How would I approach
this?
The intention of this question is to find ways to avoid inadvertent mix up in the usages.
The easy way to not get them mixed up is never to use malloc(), then you will never be tempted to call free(). The infrastructure to create to avoid this problem is called "code review", though in this case a quick "grep malloc(" or "grep free(" on the codebase will probably suffice.
Never. If it works it's by sheer accident of implementation. Do not rely on this behavior.
To answer the second question, if you control both malloc/free and operator new/delete, you can stash extra information to associate with pointers returned by both that tell you how they were allocated. When a pointer is passed to free or operator delete, check to see that it was allocated by the appropriate function. If not, assert or raise an exception or do whatever it is you do to report the mismatch.
Usually this is done by allocating extra memory, e.g., given malloc(size) or operator new(size), you allocate size + additional space and shove extra information in there.
The only way you can ensure you never get them mixed up is by either:
Never using malloc/free in the first place, or
Rely on RAII for your memory allocations. Protect every memory allocation in a RAII object which ensures the memory get correctly and consistently freed when it goes out of scope, or wrap the allocation in a smart pointer.
Manually calling delete or free is just an invitation for bugs.
You should always use new and delete in C++, as only new and delete will call the object's constructor and destructor.
If you find you must use both (for instance, if you're interfacing with a C library), thorough code reviews should carefully scrutinize any uses of free() to determine whether or not they correspond to a malloc(), and whether or not they are being used in a C context.
If I had to codify it, I'd put in the style guide something like this:
free() may be called only on a private pointer field of an object.
malloc()ed buffers (or buffers returned from a C API which caller must free()) must be assigned to a private pointer field of an object.
private pointer fields which hold free()-able buffers must only be used for that purpose.
if you use hungarian notation, add a letter for it (and if you don't, don't).
generally free() will be called only in a destructor, with exceptions when the free()-able buffer is replaced during the lifetime of the object. In that case you can call free() on a value recently copied out of a private field during replacement, rather than on the field value directly.
In other words, stick a wrapper around anything that uses malloc/free. This wrapper could be a single template everyone uses, or you could allow smart pointers with the deletor function set to free(). Then in code review, if you see a call to malloc/free anywhere else, it's wrong.
Basically the best way to stop this being a problem is to be on top of your resource handling in general. In C, people do not have a major problem with accidentally calling free() on streams instead of fclose().
You should always use delete or delete[] when freeing things allocated with new. The same goes for malloc and free.
If using free for deleting new:ed classes the destructor won't be properly called.
Also new and delete doesn't necessarily use malloc/free for its allocations so you might end up corrupting your heap as well.
always use delete for things allocated with new, delete [] for things allocated with new [] and free() for things allocated using malloc().
new and new[] allocate from different heaps than malloc() and using the wrong free()/delete will attempt to deallocate from the wrong heap.
Never mix new/delete with new[]/delete[] or with malloc()/free(). Atop of this, the use of malloc()/free() in C++ is questionable at least.
The easiest way to make sure you never do this accidentally is to not to manage memory manually. In C++ there strings and other containers as well as smart pointers to take care of memory management. There simply isn't a need to do this manually. (I'm not sure whether the last time I remember me typing delete really was the last time I typed it. But if my memory doesn't fail me, this must have been 2001 or 2002.)
You could write your own version of the functions which allocate some extra memory in new/new[]/malloc to track which one did the allocation. Then check in delete/delete[]/free that the right function was used to re-claim the memory. You can also check for things like mixing new[] and delete (without the []) this way. You might want to only use these versions in the debug build.
You could try running valgrind on your application to see if it catches anything. The manual specifically mentions its ability to catch uses of the wrong deallocation function for a block of memory. I don't think there's a way to do this at compile time, however.
One way to avoid misusage of malloc/free or new/delete is that DO NOT call these function directly, call them by a wrapper layer. In the wrapper layer, you can manage the pointers distributed by your own hands and guarantee that misusage will get an error explicitly.
Why dont you just count the total number of malloc statments and tally that with the total count of free? Do the same for New and delete. The process can be automated with regular expressions.
One thing I've done relating to this is to reimplement malloc and free
in C++, so that it calls new/delete under the hood. It's not
optimal, but it serves well enough on a smaller project.