I'm allocating memory using new as and when I receive data in one of my methods and in the destructor, I'm releasing all the allocated memory using delete.
However, after releasing the memory, from the task manager, when I look at mem usage under the process tab, the memory usage still remains the same. It doesn't give an impression that the memory is being released.
So, when does the memory actually get released? And what is the best way to find out the actual memory being used by a process.
Thanks
In most cases, it's never given back to the OS while the app is running. Afterwards, of course, all resources are recovered by the OS.
[Edited after the comments rightly pointed out that 'never' is a long time ...]
The OS allocates a default heap to your application. This heap is allocated during the initialization of your process. Thus, your new's and delete's won't affect the bar you see in the task manager.
However, if you try to initialize a big buffer and the allocated heap isn't enough, the OS will allocate more memory for your application - and this should be reflected in the task manager...
Related
I am sharing a memory between my driver and application.
After I did Allocating and Mapping, I clean up with Unmapping and Free.
I found that Allocating and Mapping would allocate a brand new virtual memory page in the application, which can be accessed by both driver and application, but it does not disappear after I did the clean up.
I want to know if I should expect the the page to be free after I did clean up.
I saw that MmAllocatePagesForMdlEx and MmMapLockedPagesSpecifyCache will allocate a brand new page of virtual memory in application, which did not appear in the memory list before. After I clean up with MmUnmapLockedPages and MmFreePagesFromMdl, I still see that allocated memory in the list, which is still able to be read and write by the VC++ debugger and application.
I have think of if the clean up failed, but another thing makes me confused. The clean up would zero the memory page, which makes me think it actually is working.
And there is one more thing I did. I use a tool called CE to read the memory but the result is it cannot be read. CE is using ReadProcessMemory to read the memory. So I again get confused.
To summarize, I did
Allocate - MmAllocatePagesForMdlEx
Map - MmMapLockedPagesSpecifyCache
Unmap - MmUnmapLockedPages
Free - MmFreePagesFromMdl
The result is
VC++ debugger can read the cleaned up memory
Application itself can read the cleaned up memory
CE which use standard API cannot read the cleaned up memory
CE which use standard API can see the memory info in the memory list
The method I used to test reading cleaned up memory
VC++ debugger - in memory tab, goto address
Application - value = ^(ULONG64^)addr; //star disabled
The main code are as below:
mdl = MmAllocatePagesForMdlEx(least, most, least, totalBytes, MmCached, 0);
MmGetSystemAddressForMdlSafe(mdl, NormalPagePriority);
MmUnmapLockedPages(userAddr, mdl);
MmFreePagesFromMdl(mdl);
I expected that after I did the last two code mentioned above, the allocated memory would be free because even the mapped physical memory is being free.
So again my question is, should I expect the cleaned up memory become No Access in application / usermode space.
When I use the top terminal program at Linux, I can't see the result of free.
My expectation is:
free map and list.
The memory usage that I can see at the top(Linux function) or /proc/meminfo
get smaller than past.
sleep is start.
program exit.
But
The usage of memory only gets smaller when the program ends.
Would you explain the logic of free function?
Below is my code.
for(mapIter = bufMap->begin(); mapIter != bufMap -> end();mapIter++)
{
list<buff> *buffList = mapIter->second;
list<buff>::iterator listIter;
for(listIter = buffList->begin(); listIter != buffList->end();listIter++)
{
free(listIter->argu1);
free(listIter->argu2);
free(listIter->argu3);
}
delete buffList;
}
delete bufMap;
printf("Free Complete!\n");
sleep(10);
printf("endend\n");
Thanks you.
Memory is allocated onto a heap.
When you request some memory in your program (with a new() or malloc() etc.) Your program requests some memory from its heap, which in turn requests it from the operating system{1}. Since this is an expensive operation, it gets a chunk of memory from the OS, not just what you ask for. The memory manager puts everything it gets into the heap, just returning to you the perhaps small amount you asked for. When you free() or delete() this memory, it simply gets returned to the heap, not the OS.
It's absolutely normal for that memory to not be returned to the operating system until your program exits, as you may request further memory later on.
If your program design relies on this memory be recycled, it may be achievable using multiple copies of your program (by fork()~ing) which run and exit.
{1} The heap is probably non-empty on program start, but assuming it's not illustrates my point.
The heap typically uses operating system functions to manage its memory. The heap’s
size may be fixed when the program is created, or it may be allowed to grow. However,
the heap manager does not necessarily return memory to the operating system when
the free function is called. The deallocated memory is simply made available for subsequent use by the application. Thus, when a program allocates and then frees up memory, the deallocation of memory is not normally reflected in the application’s memory
usage as seen from the operating system perspective.
How can I free the virtual memory that is left up after calling TerminateThread? Can it be done via VirtualFree and how of course. I fully understand the "Dangers" of TerminateThread.
In an unmanaged process, there's no realistic way to tidy up memory from the outside.
Memory can be allocated in many different ways. Ultimately it all starts with calls to VirtualAlloc, VirtualAllocEx etc. But in practice runtime libraries invariably use sub allocating heap managers. These heap allocators will get memory by calls to VirtualAlloc, but then will hand out sub-blocks. And heap managers are generally shared between threads in a process. So you've no way from the outside of knowing how to free those sub-blocks.
And even if we did not have sub-allocators, how could you know which blocks handed out by VirtualAlloc you were allowed to destroy? A thread may allocate memory with a call to VirtualAlloc and require that the memory out lives the allocating thread and is destroyed by another thread.
But if you are happy to let all of that go, and just want the stack to be destroyed (as per your comments), then this article shows you how to do so with RtlFreeUserThreadStack: http://www.nicklowe.org/2012/01/thread-termination-dont-leak-the-stack/
I have a c++ code. But it is not releasing memory properly. Tell me where I am wrong, here is my code
1 void MyClass::MyFunction(void)
2 {
3 for (int i=0; i<count; i++)
4 {
5 _bstr_t xml = GetXML(i);
6 // some work
7 SysFreeString(xml);
8 }
9 }
GetXML (line 5) returns me a BSTR. At this memory of program increases. But after SysFreeString (line 7) memory does not release. What I am doing wrong here?
First:
// This makes a copy.
// This is where the leak is. You are leaking the original string.
_bstr_t xml = GetXML();
// You want to use this, to attach the BSTR to the _bstr_t
_bstr_t xml = _bstr_t(GetXML(), false);
Second, don't do this:
SysFreeString(xml);
The _bstr_t class will do that for you.
Third, BSTR will not release the memory to the OS immediately, it caches recently used strings in order to make SysAllocString faster. You shouldn't expect to see memory usage go straight down after SysFreeString.
You can control this behaviour for debugging purposes:
http://support.microsoft.com/default.aspx?scid=kb;en-us;Q139071
Lastly, when viewing memory usage in Task Manager you need to look at the column "Commit Size" not "Working Set". Go to Menu->View->Select Columns to show the column. And also note that this really only helps over a period of time - the memory may not be released to the OS immediately, but if you have no leaks, it shouln't go up forever, over a course of hours.
I suppose you should use :
xml.Attach(GetXML(i));
operator= looks like it is actually assigning new value - which means copying it. That value returned by GetXML stays unfreed.
also there should be no need for SysFreeString(xml);
Task Manager only provides the amount of memory allocated to the process. When C++ releases memory ( C's free) it does not necessarily return the memory to the Operating system so Task Manager will not necessarily show memory going doem until the process ends.
What Task Manager can show is if you keep allocating memory and not releasing it then the memory size of the process will keep increasing, if this happens you probably hava a memory leak.
When programming you need to use memory profilers to see if you are releasing memory. In Windows I used Rational's Purify to give me this information but it costs a lot. MS C runtime can be used to track memory. MSDN provides an overview here, read and follow th links.
As to your code and as per other comments and answers one of the points of using _bstr_t class is to do the memory and other resource management for you so you should not call SysFreeString
The destructor of _bstr_t will call SysFreeString(xml), so you don't need to call SysFreeString(xml) again. An additional memory free will result in crash.
Every process can use heap memory to store and share data within the process. We have a rule in programming whenever we take some space in heap memory, we need to release it once job is done, else it leads to memory leaks.
int *pIntPtr = new int;
.
.
.
delete pIntPtr;
My question: Is heap memory per-process?
If YES,
then memory leak is possible only when a process is in running state.
If NO,
then it means OS is able to retain data in a memory somewhere. If so, is there a way to access this memory by another process. Also this may become a way for inter-process communication.
I suppose answer to my question is YES. Please provide your valuable feedback.
On almost every system currently in use, heap memory is per-process. On older systems without protected memory, heap memory was system-wide. (In a nutshell, that's what protected memory does: it makes your heap and stack private to your process.)
So in your example code on any modern system, if the process terminates before delete pIntPtr is called, pIntPtr will still be freed (though its destructor, not that an int has one, would not be called.)
Note that protected memory is an implementation detail, not a feature of the C++ or C standards. A system is free to share memory between processes (modern systems just don't because it's a good way to get your butt handed to you by an attacker.)
In most modern operating systems each process has its own heap that is accessible by that process only and is reclaimed once the process terminates - that "private" heap is usually used by new. Also there might be a global heap (look at Win32 GlobalAlloc() family functions for example) which is shared between processes, persists for the system runtime and indeed can be used for interprocess communications.
Generally the allocation of memory to a process happens at a lower level than heap management.
In other words, the heap is built within the process virtual address space given to the process by the operating system and is private to that process. When the process exits, this memory is reclaimed by the operating system.
Note that C++ does not mandate this, this is part of the execution environment in which C++ runs, so the ISO standards do not dictate this behaviour. What I'm discussing is common implementation.
In UNIX, the brk and sbrk system calls were used to allocate more memory from the operating system to expand the heap. Then, once the process finished, all this memory was given back to the OS.
The normal way to get memory which can outlive a process is with shared memory (under UNIX-type operating systems, not sure about Windows). This can result in a leak but more of system resources rather than process resources.
There are some special purpose operating systems that will not reclaim memory on process exit. If you're targeting such an OS you likely know.
Most systems will not allow you to access the memory of another process, but again...there are some unique situations where this is not true.
The C++ standard deals with this situation by not making any claim about what will happen if you fail to release memory and then exit, nor what will happen if you attempt to access memory that isn't explicitly yours to access. This is the very essence of what "undefined behavior" means and is the core of what it means for a pointer to be "invalid". There are more issues than just these two, but these two play a part.
Normally the O/S will reclaim any leaked memory when the process terminates.
For that reason I reckon it's OK for C++ programmers to never explicitly free any memory which is needed until the process exits; for example, any 'singletons' within a process are often not explicitly freed.
This behaviour may be O/S-specific, though (although it's true for e.g. both Windows and Linux): not theoretically part of the C++ standard.
For practical purposes, the answer to your question is yes. Modern operating systems will generally release memory allocated by a process when that process is shut down. However, to depend on this behavior is a very shoddy practice. Even if we can be assured that operating systems will always function this way, the code is fragile. If some function that fails to free memory suddenly gets reused for another purpose, it might translate to an application-level memory leak.
Nevertheless, the nature of this question and the example posted requires, ethically, for me to point you and your team to look at RAII.
int *pIntPtr = new int;
...
delete pIntPtr;
This code reeks of memory leaks. If anything in [...] throws, you have a memory leak. There are several solutions:
int *pIntPtr = 0;
try
{
pIntPtr = new int;
...
}
catch (...)
{
delete pIntPtr;
throw;
}
delete pIntPtr;
Second solution using nothrow (not necessarily much better than first, but allows sensible initialization of pIntPtr at the time it is defined):
int *pIntPtr = new(nothrow) int;
if (pIntPtr)
{
try
{
...
}
catch (...)
{
delete pIntPtr;
throw;
}
delete pIntPtr;
}
And the easy way:
scoped_ptr<int> pIntPtr(new int);
...
In this last and finest example, there is no need to call delete on pIntPtr as this is done automatically regardless of how we exit this block (hurray for RAII and smart pointers).