Imagine that I use C++11 threads. The thread will run a function that do malloc. After that I will use join without free (the memory). So, I killed the thread. It is expected that the memory frees automatically?
No, it is not. The memory is freed only after the whole application is terminated. The whole benefit of using multiple threads (as opposed to processes) is that they share the same memory, so they collectively own all the memory allocated in one of them.
Related
Heap (HeapAlloc) Corruption in release mode only
I can just guess but why wouldn't you check the result of HeapFree? Because according the documentation it could be the reason. Try to use HEAP_NO_SERIALIZE flag when you allocate heap.
You should not refer in any way to memory that has been freed by HeapFree. After that memory is freed, any information that may have been in it is gone forever. If you require information, do not free memory containing the information. Function calls that return information about memory (such as HeapSize) may not be used with freed memory, as they may return bogus data. Calling HeapFree twice with the same pointer can cause heap corruption, resulting in subsequent calls to HeapAlloc returning the same pointer twice.
Serialization ensures mutual exclusion when two or more threads attempt to simultaneously allocate or free blocks from the same heap. There is a small performance cost to serialization, but it must be used whenever multiple threads allocate and free memory from the same heap. Setting the HEAP_NO_SERIALIZE value eliminates mutual exclusion on the heap. Without serialization, two or more threads that use the same heap handle might attempt to allocate or free memory simultaneously, likely causing corruption in the heap. The HEAP_NO_SERIALIZE value can, therefore, be safely used only in the following situations:
The process has only one thread.
The process has multiple threads, but only one thread calls the heap functions for a specific heap.
The process has multiple threads, and the application provides its own mechanism for mutual exclusion to a specific heap.
How can I free the virtual memory that is left up after calling TerminateThread? Can it be done via VirtualFree and how of course. I fully understand the "Dangers" of TerminateThread.
In an unmanaged process, there's no realistic way to tidy up memory from the outside.
Memory can be allocated in many different ways. Ultimately it all starts with calls to VirtualAlloc, VirtualAllocEx etc. But in practice runtime libraries invariably use sub allocating heap managers. These heap allocators will get memory by calls to VirtualAlloc, but then will hand out sub-blocks. And heap managers are generally shared between threads in a process. So you've no way from the outside of knowing how to free those sub-blocks.
And even if we did not have sub-allocators, how could you know which blocks handed out by VirtualAlloc you were allowed to destroy? A thread may allocate memory with a call to VirtualAlloc and require that the memory out lives the allocating thread and is destroyed by another thread.
But if you are happy to let all of that go, and just want the stack to be destroyed (as per your comments), then this article shows you how to do so with RtlFreeUserThreadStack: http://www.nicklowe.org/2012/01/thread-termination-dont-leak-the-stack/
Lets say I have allocated some memory in my background thread, that is, a thread stack is holding a pointer to that memory. Now I want do terminate background thread execution by calling pthread_cancel on it. Will that memory be released or not? (My platform is iOS, compiler is gcc 4.2)
Each thread by necessity requires its own stack; however there is typically only one heap per process. When a thread is destroyed, there is no automatic mechanism to free the memory allocated on the heap. All you end up with is a memory leak.
As a general rule, avoid using pthread_cancel since it is hard to ensure that pthread_cancel will run safely. Rather build in some mechanism where you can pass a message to the thread to destroy itself (after freeing any resources that it owns).
The thread stack will be removed once the thread exits. But there will be no process or code that will look into your thread stack and release any references for the objects that you've allocated on the heap. Also, typically, thread stacks don't hold any references to the memory, the thread stack is an independent space that is given to the thread to use for generic program stack, any reference will only be on the stack for as long as you are inside the function that pushed such a reference onto the stack, typically because you are referencing it with a local variable.
by default, no -- see other answers, which are more specific to the answer you seek. there is however, such a thing as a thread-specific allocator; if you were using one, you'd know.
No, it won't be deleted or freed automatically. If you're very lucky, it might be garbage collected sometime if you're running a collector. File handles, shared memory ids, mutexes etc. won't be released/deallocated either. Async cancellation is safe for e.g. pure maths calculations on data still owned by another thread, but very risky in general - that's why some threading APIs have experimented with and removed the function completely.
Every process can use heap memory to store and share data within the process. We have a rule in programming whenever we take some space in heap memory, we need to release it once job is done, else it leads to memory leaks.
int *pIntPtr = new int;
.
.
.
delete pIntPtr;
My question: Is heap memory per-process?
If YES,
then memory leak is possible only when a process is in running state.
If NO,
then it means OS is able to retain data in a memory somewhere. If so, is there a way to access this memory by another process. Also this may become a way for inter-process communication.
I suppose answer to my question is YES. Please provide your valuable feedback.
On almost every system currently in use, heap memory is per-process. On older systems without protected memory, heap memory was system-wide. (In a nutshell, that's what protected memory does: it makes your heap and stack private to your process.)
So in your example code on any modern system, if the process terminates before delete pIntPtr is called, pIntPtr will still be freed (though its destructor, not that an int has one, would not be called.)
Note that protected memory is an implementation detail, not a feature of the C++ or C standards. A system is free to share memory between processes (modern systems just don't because it's a good way to get your butt handed to you by an attacker.)
In most modern operating systems each process has its own heap that is accessible by that process only and is reclaimed once the process terminates - that "private" heap is usually used by new. Also there might be a global heap (look at Win32 GlobalAlloc() family functions for example) which is shared between processes, persists for the system runtime and indeed can be used for interprocess communications.
Generally the allocation of memory to a process happens at a lower level than heap management.
In other words, the heap is built within the process virtual address space given to the process by the operating system and is private to that process. When the process exits, this memory is reclaimed by the operating system.
Note that C++ does not mandate this, this is part of the execution environment in which C++ runs, so the ISO standards do not dictate this behaviour. What I'm discussing is common implementation.
In UNIX, the brk and sbrk system calls were used to allocate more memory from the operating system to expand the heap. Then, once the process finished, all this memory was given back to the OS.
The normal way to get memory which can outlive a process is with shared memory (under UNIX-type operating systems, not sure about Windows). This can result in a leak but more of system resources rather than process resources.
There are some special purpose operating systems that will not reclaim memory on process exit. If you're targeting such an OS you likely know.
Most systems will not allow you to access the memory of another process, but again...there are some unique situations where this is not true.
The C++ standard deals with this situation by not making any claim about what will happen if you fail to release memory and then exit, nor what will happen if you attempt to access memory that isn't explicitly yours to access. This is the very essence of what "undefined behavior" means and is the core of what it means for a pointer to be "invalid". There are more issues than just these two, but these two play a part.
Normally the O/S will reclaim any leaked memory when the process terminates.
For that reason I reckon it's OK for C++ programmers to never explicitly free any memory which is needed until the process exits; for example, any 'singletons' within a process are often not explicitly freed.
This behaviour may be O/S-specific, though (although it's true for e.g. both Windows and Linux): not theoretically part of the C++ standard.
For practical purposes, the answer to your question is yes. Modern operating systems will generally release memory allocated by a process when that process is shut down. However, to depend on this behavior is a very shoddy practice. Even if we can be assured that operating systems will always function this way, the code is fragile. If some function that fails to free memory suddenly gets reused for another purpose, it might translate to an application-level memory leak.
Nevertheless, the nature of this question and the example posted requires, ethically, for me to point you and your team to look at RAII.
int *pIntPtr = new int;
...
delete pIntPtr;
This code reeks of memory leaks. If anything in [...] throws, you have a memory leak. There are several solutions:
int *pIntPtr = 0;
try
{
pIntPtr = new int;
...
}
catch (...)
{
delete pIntPtr;
throw;
}
delete pIntPtr;
Second solution using nothrow (not necessarily much better than first, but allows sensible initialization of pIntPtr at the time it is defined):
int *pIntPtr = new(nothrow) int;
if (pIntPtr)
{
try
{
...
}
catch (...)
{
delete pIntPtr;
throw;
}
delete pIntPtr;
}
And the easy way:
scoped_ptr<int> pIntPtr(new int);
...
In this last and finest example, there is no need to call delete on pIntPtr as this is done automatically regardless of how we exit this block (hurray for RAII and smart pointers).
I am using malloc_stats() to print malloc related statistics in which I am finding "Arena 0" for some programs and "Arena 0 and Arena 1" for some other programs.
What do these arenas represent?
The heap code resides inside the glibc component, and is packaged in the libc.so.x shared library. The current implementation of the heap uses multiple independent sub-heaps called arenas. Each arena has its own mutex for concurrency protection. Thus if there are sufficient arenas within a process' heap, and a mechanism to distribute the threads' heap accesses evenly between them, then the potential for contention for the mutexes should be minimal. It turns out that this works well for allocations. In malloc(), a test is made to see if the mutex for current target arena for the current thread is free (trylock). If so then the arena is now locked and the allocation proceeds. If the mutex is busy then each remaining arena is tried in turn and used if the mutex is not busy. In the event that no arena can be locked without blocking, a fresh new arena is created. This arena by definition is not already locked, so the allocation can now proceed without blocking. Lastly, the ID of the arena last used by a thread is retained in thread local storage, and subsequently used as the first arena to try when malloc() is next called by that thread. Therefore all calls to malloc() will proceed without blocking.
See link text. It looks like heap is a collection of arenas ("sub-heaps") to handle memory allocation between several threads, thus reducing contention.
In certain malloc implementations, an "arena" is a pool of memory from which individual allocations are made. The algorithms to determine which arena is used will differ between implementations, so it's not possible for us to explain why you see a difference. One common factor is allocation size.
Everything is there: http://www.gnu.org/software/libc/manual/html_node/Statistics-of-Malloc.html
int arena
This is the total size of memory allocated with sbrk by malloc, in bytes.