I am developing the software system for an embedded system of a student satellite. Our code is a mix of C/C++, running on an an AT32UC3A3256S 32-bit AVR microcontroller. We are running the FreeRTOS operating system on the hardware which is working fine. We also have a need for a somewhat specialized memory management scheme due to the physical memory layout, and concept of operations for the mission.
I have been attempting to use a dynamic memory implementation called dlmalloc, largely due to the availability of mspaces, which allows dynamic memory allocation into contained and tracked sections. I have some code that wraps around dlmalloc, in order to create mspaces in certain places in memory, and cause allocations to be tied to these mspaces depending on the FreeRTOS task making the request. The end product is a memory management system that amount of memory a given task has allocated, and if it has gone over it's imposed limit, will reset the task and free it's memory.
I have created a test task that essentially is a big memory leak, continuously allocating memory without freeing it. The memory management system in place should periodically reset this task as it overflows its limit, freeing all memory that would otherwise be leaked. This works perfectly for a single task running, however fails in very odd ways if two similar copies of this task run simultaneously, leading me to believe that the memory allocation is not thread safe.
I have surrounded every call to memory allocation routines with FreeRTOS routines that ensure that only the task allocating memory will run for the duration of the memory request. To me this seems like it should provide the thread-safeness needed, but obviously something else is wrong.
Does anybody have any ideas on what I might be missing to make this system thread-safe, on how to port dlmalloc to the hardware I am using, on any other concurrent memory allocators I could possibly use, or any advice at all? I can provide much more information if necessary but not did want to bloat the original post more than I already have.
Related
I'm working on a project that contains a real-time software component, using the RT PREEMPT patch on Linux.
During "idle" operation the software just sits waiting for incoming TCP connections and requests. Depending on the request, the software may create a real-time thread that runs for a period of time. So the entire application doesn't need to operate in real-time, only this one thread from time to time.
My question is this: I'm well aware that dynamic memory allocation is non-deterministic and is detrimental to real-time code. However, is accessing existing memory on the heap also detrimental to real-time constraints?
I ask because I'm considering a situation where the program starts up, allocates any required structures on the heap, then triggers a real-time thread that accesses the heap.
EDIT: Once the real-time thread has started, other threads are prevented from writing to variables the real-time thread needs to access using locks (well, except for one variable that must be updated, but access is still restricted using a separate lock).
EDIT2: I forgot to mention that the program will ultimately be deployed on a system that doesn't have any swap space, so I don't think the paging of memory should be an issue. (Though of course this doesn't avoid the issue of page-faults through memory that hasn't yet been provisioned by the OS.)
It is possible that the virtual memory manager might move your memory to swap making your thread generate a major page fault when it runs. You need to lock the memory pages using mlock(). I also recommend allocating memory in chunks and writing to all of it with memset() before using it to avoid minor page faults at run time and use placement new instead of the regular one to create your objects in the already allocated memory.
is accessing existing memory on the heap also detrimental to real-time constraints?
No, unless your system is thrashing.
BTW, you could consider writing your own allocation (e.g. above mmap(2)...) and using mlock(2) for the memory that ought to be in RAM.
I have seen that lot of code bases specially server codes have basic (sometimes advanced) memory managers. Is the real purpose of memory manager is to reduce number of malloc calls or mainly for the purpose of memory analysis, corruption check or may be other application centric purposes.
Is the argument of saving malloc calls reasonable enough as malloc in itself is a memory manager. The only performance gain I can reason is when we know that system always ask for same size memory.
Or the reason for having memory manager is that free does not return memory to OS but saves in the list. So over the lifetime of the process, the heap usage of the process may increase if we keep on doing malloc/free because of fragmentation.
mallocis a general purpose allocator - "not slow" is more important than "always fast".
Consider a feature that would be a 10% improvement in many common cases, but might cause significant performance degradation in a few rare cases. An application specific allocator can avoid the rare case and reap the benefits. A general purpose allocator should not.
Besides number of calls to malloc, there are other relevant attributes:
locality of allocations
On current hardware, this easily the most important factor for performance. An application has more knowledge of the access patterns and can optimize the allocations accordingly.
multithreading
A general purpose allocator must allow calls to malloc and free from different threads. This usually requires a lock or similar concurrency handling. If the heap is very busy, this leads to massive contention.
An application that knows that some high-frequency alloc/frees come only from one thread can use its own thread-specific heap, which not only avoids contention for these allocations, but also increases their locality and takes load off the default allocator.
fragmentation
This is still a problem for long running applications on systems with limited physical memory or address space. Fragmentation may require more and more memory or address space from the OS, even without the actual working set increasing. This is a significant problem for applications that need to run uninterrupted.
Last time I looked deeper into allocators (which is probably half a decade past), the consensus was that naive attempts to reduce fragmentation often conflict with the never slow rule.
Again, an application that knows (some of its) allocation patterns can take a lot of load from the default allocator. One very common use case is building a syntax tree or something similar: there are gazillions of small allocations which are never freed individually, only as a whole. Such a pattern can be served efficiently with a very trivial allocator.
resilence and diagnostics
Last not least the diagnostic and self-protection capabilities of the default allocator may not be sufficient for many applications.
Why do we have custom memory managers rather than the built-in ones?
Number one reason is probably that the codebase was originaly written 20-30years ago when the provided one wasn't any good and nobody dares change it.
But otherwise, as you say because the application needs to manage fragmentation, grab memory at startup to ensure that memory will always be available, for security or a bunch of other reasons - most of which could be acheived by correct use of the built-in manager.
C and C++ are designed to be stripped down. They don't do much that is not explicitly asked for, so when a program asks for memory, it gets the minimum possible effort required to deliver that memory.
In other words, if you don't need it, you don't pay for it.
If finer-grained control of the memory is required, that's the domain of the programmer. If the programmer wishes to trade bare metal speed for a system that will provide higher performance on the target hardware in conjunction with the program's often unique goals, better debugging support, or simply likes the look and feel and warm fuzzies that come from using a manager, that is up to them. The programmer either writes something smarter or finds a third party library to do what they want.
You briefly touched on a lot of the different reasons why you would use a memory manager in your question.
Is the real purpose of a memory manager to reduce the number of malloc calls or mainly for the purpose of memory analysis, corruption check or other application centric purposes?
This is the big question. A memory manager in any application can be generic (like malloc) or it can be more specific. The more specialized the memory manager becomes it is likely to be more efficient at the specific task it is supposed to accomplish.
Take this overly-simplified example:
#define MAX_OBJECTS 1000
Foo globalObjects[MAX_OBJECTS];
int main(int argc, char ** argv)
{
void * mallocObjects[MAX_OBJECTS] = {0};
void * customObjects[MAX_OBJECTS] = {0};
for(int i = 0; i < 1000; ++i)
{
mallocObjects[i] = malloc(sizeof(Foo));
customObjects[i] = &globalObjects[i];
}
}
In the above I am pretending that this global object list is our "custom memory allocator." This is just to simplify what I am explaining.
When you allocate with malloc there is no guarantee it is right next to the previous allocation. Malloc is a general purpose allocator and does a good job at that but doesn't necessarily make the most efficient choice for every application.
With a custom allocator you might be able to up front allocate room for 1000 custom objects and since they are a fixed size return the exact amount of memory you need to prevent fragmentation and to efficiently allocate that block.
There is also the difference between memory abstraction and custom memory allocators. STL allocators are arguably an abstraction model and not a custom memory allocator.
Take a look at this link for some more information on custom allocators and why they are useful: gamedev.net link
There are many reasons why we would want to do this and it really depends on the application itself. In fact all the reasons you mentioned are valid.
I once built a very simple memory manager that kept track of shared_ptr allocations in order for me to see what was not being released properly on application end.
I would say stick to your runtime unless you need something that it does not provide.
Memory managers are used basically to manage efficiently your memory reservation. Normally processes have access to a limited amount of memory (4GB in 32bits systems), from this you have to subtract the virtual memory space reserved for the kernel (1GB or 2GB depending on your OS configuration). Thus, virtually the process has access let's say to 3GB of memory that will be used to hold all of its segments (code, data, bss, heap and stack).
Memory managers (malloc for example) try to fulfill the different memory reservation requests issued by the process by requesting new memory pages to the OS (using sbrk or mmap system calls). Every time this happens it implies an extra cost on the program execution since the OS has to look for a suitable memory page to be assigned to the process (Physical memory is limited and all the running processes want to use it), update the process tables (TMP, etc). These operations are time consuming and hit the process execution and performance. Thus, the memory manager normally try to request the needed pages to fulfill the process reservations cleverly. For example it could ask for some more pages to avoid calling more mmap calls in the near future. Additionally, it tries to deal with issues like fragmentation, memory alignment, etc. This basically unloads the process from this responsibility, otherwise everybody writing some program that needs dynamic memory allocation has to perform this manually!
Actually, there are cases where one could be interested in doing the memory management manually. This is the case for embedded or high availability systems which have to run for 24/365. In these cases even if the memory fragmentation is low it could become a problem after very long period of running (1 year for example). So, one of the solutions that are used in this case is to use a memory pool to allocate before hand the memory for the application objects. After-wards each time you need memory for some object you just use the already reserved memory.
For server based or any application that needs to run for long periods of time or indefinitely, the main issue is paged memory fragmentation. After a long series of mallocs / new and free / delete, paged memory can end up with gaps in the pages that waste space and could eventually run out of virtual address space. Microsoft deals with this with it's .NET framework, by occasionally pausing a process to repack paged memory for a process.
To avoid slowdown during repacking of memory in a process, a server type application can use multiple processes for the application, so that during repacking of one process, the other process(es) take more of the load.
I need to manage memory that's in a separate memory space. Another program has a large slab of contiguous memory that my code cannot directly access, and notifies my code of its size during initialization. The other program will ask my code to "allocate" X bytes from the slab, and will later notify my code to deallocate the allocated blocks. The allocations and deallocations will be more or less unpredictable, much like the use of regular malloc and free. It's up to my code to manage the dynamic memory for the other process. The intended use has to do with device memory on a GPU, but I would rather not make the question specific to that. I'd like an answer that's generic, maybe even enough that I could theoretically use it as a back-end for a network API for managing virtual memory on a remote machine.
The basic functionality I need is to be able initialize the manager with the slab size, perform the equivalent of malloc() and free(), and get a few stats like remaining available memory and maybe maximum allocatable size.
Obviously, I can't use malloc() and free() 'abstractly'. I also don't want the management of my actual process memory to interfere with the management of my abstract slab.
What would be the best way to go about this? And, more specifically, does the standard library or Boost have this kind of a facility?
Notes:
Allocation strategy is not extremely important at this point. I mean, maybe it is, but first I need to figure out how I'm going to go about this business.
I'm going to be using this for allocating between tens and several thousand buffers per second, with different sizes; some as small as several bytes, some as large as gigabytes. There will be no buildup of allocated space over a long period of time.
a back-end for a network API for managing virtual memory on a remote machine
A networked system should not be concerned about the memory management of a remote node. The remote node should be free to choose whether to serve the request by allocating memory on the fastest but scarcest memory or in a larger but slower storage or a hybrid of both depending on its own heuristics, other requests happening concurrently, prioritization/security policy set by the network admin, and so on.
Thinking in terms of malloc and free will limit what a remote server can do to serve the requests efficiently.
Note that strictly speaking malloc itself is only managing "theoretical"/"virtual" memory. The OS kernel can page memory allocated with malloc in and out of swap space and assign or move its physical address on the MMU. The memory "address" returned by "malloc" is actually just a virtual address which doesn't correspond to actual physical address.
Unlike networked system, when managing hardware you do usually want to dictate very closely what the hardware should do on each step, because, presumably, a hardware is only connected to a single system, so any contention is only local contention, and considerations regarding performance throughput usually trumps everything else. The nature of managing remote resource on a networked system and local hardware is similar but quite different.
If you're interested in designs for networked shared memory, you might want to checkout in-memory databases (e.g. Redis) or networked file systems. A networked shared memory can usually afford to use a user-friendly string keys to point to the memory slabs; hardware shared memory usually want to keep it simple and fast by using simple numeric handles.
Both networked and hardware need to deal with splitting a single resource for multiple independent processes securely. In many popular operating system, the code responsible for managing hardware allocation is usually the kernel. It's quite rare for userspace program to be the one responsible for managing hardware on monolithic OSes. Maybe you should be writing a device driver? Many GPGPU device now have an MMU.
Certainly, there's nothing preventing you from writing a gpu_malloc()/gpu_free(). It theoretically should be possible to mmap the remote system's address space into the process's virtual memory address space so that programs could just use the remote memory just as any other memory.
I'm going to be using this for allocating between tens and several thousand buffers per second, with different sizes; some as small as several bytes, some as large as gigabytes.
Generally, you don't want having thousands of small allocations. System calls that relates to memory management can be expensive if it involves remote systems, so it might be necessary to have the client program to allocate large slabs and do its own suballocation (malloc does this, small mallocs usually causes one system call to allocate larger memory than is requested and malloc then suballocates the slab).
I have a .NET/native C++ application. Currently, the C++ code allocates memory on the default heap which persists for the life of the application. Basically, functions/commands are executed in the C++ which results in allocation/modification of the current persistent memory. I am investigating an approach for cancelling one of these functions/commands mid-execution. We have hundreds of these commands, and many are very complicated (legacy) code.
The brute-force approach that I am trying to avoid is modifying each and every command/function to check for the cancellation and do all the appropriate clean-up (freeing heap memory). I am investigating a multi-threaded approach in which an additional thread receives the cancellation request and terminates the command-execution thread. I would want all dynamic memory to be allocated on a "private heap" using HeapCreate() (Win32). This way, the private heap could be destroyed by the thread handling the cancellation request. However, if the command runs to completion, I need the dynamic memory to persist. In this case, I would like to do the logical equivalent of "moving" the private heap memory to the default/process heap without incurring the cost of an actual copy. Is this in any way possible? Does this even make sense?
Alternatively, I recognize that I could just have a new private heap for every command/function execution (each will be a new thread). The private heap could be destroyed if the command is cancelled, or it would survive if the command completes. Is there any problem with the number of heaps growing indefinitely? I know there is some overhead involved with each heap. What limitations might I run into?
I am running on Windows 7 64-bit with 8GB RAM (consider this the target platform). The application I am working with is about 1 million SLOC (half C++, half C#). I am looking for any experience/suggestions with private heap management, or just alternatives to my solution.
You might be better off with separate processes instead of separate threads:
use memory mapped files (ie not a file at all - just cross-processed shared memory)
killing a process is 'cleaner' than killing a thread
I think you can have the shared memory 'survive' the killing without a move - you map/unmap instead of move
although you might need to do some memory management on your own.
Anyhow, worth looking into. I was looking into using inter-process memory for a few other things, and it had some unusual properties (can recall all of it clearly, it was a while ago), and you might be able to take advantage of it.
Just an idea!
From MSDN's Heap Functions page:
"Memory allocated by HeapAlloc is not movable. The address returned by HeapAlloc is valid until the memory block is freed or reallocated; the memory block does not need to be locked."
Can you re-link the legacy apps against your own malloc() implementation? If so, you should be able to manage without modifying the rest of the code. Your custom malloc library can track allocated blocks by thread, and have a "FreeAllByThreadId() function which you call after killing the legacy function's thread. You could use private heaps inside the library.
An alternative to private heaps might be doing your own allocation from memory-mapped files. See "Creating Named Shared Memory." You create the shared memory while initializing the alloc library for the legacy thread. On success, map it into the main thread so your c# can access it; on termination, close it and it is released to the system.
Heap is a sort of big chunk of memory. It is a user-level memory manager. A heap is created by lower-level system memory calls (e.g., sbrk in Linux and VirtualAlloc in Windows). In a a heap, then you can request or return a small chunk of memory by malloc/new/free/delete. By default, a process has a single heap (unlike stack, all threads share a heap). But, you can have many heaps.
Is it possible to combine two heaps w/o copying? A heap is essentially a data structure that maintains a list of used and freed memory chunks. So, a heap should have a sort of bookkeeping data called meta data. Of course, this meta data is per heap. AFAIK, no heap manager supports a merge operation of two heaps. I had reviewed entire source code of malloc implementation in Linux glibc (Doug Lea's implementation), but no such operation. Windows Heap* functions are also implemented in a similar way. So, it is currently impossible to move or merge two separate heaps.
Is it possible to have many heaps? I don't think there should be a big problem to have many heaps. As I said before, a heap is just a data structure that keeps used/freed memory chunks. So, there should be some amount of overhead. But, it's not that severe. When you look at one of malloc implementation, there is malloc_state, which is a basic data structure for each heap. For example, you can create another heap by create_mspace (in Windows, it is HeapCreate), then you will get a new malloc state. It's not that big. So, if this tread-off (some heap overhead vs. implementation easiness) is fine, then you may go on.
If I were you, I'll try the way you describe. It makes sense to me. Having a lot of heap objects would not make a big overhead.
Also, it should be noted that technically moving memory regions is impossible. Pointers that pointed the moved memory region will result in dangling pointers.
p.s. Your problem seems like a transaction, especially Software Transactional Memory. A typical implementation of STM buffers pending memory writes, and then commits to the real system memory it the transaction had no conflict.
No. Memory cannot be moved between heaps.
I am running my c++ application on an intel Xscale device. The problem is, when I run my application offtarget (Ubuntu) with Valgrind, it does not show any memory leaks.
But when I run it on the target system, it starts with 50K free memory, and reduces to 2K overnight. How to catch this kind of leakage, which is not being shown by Valgrind?
A common culprit with these small embedded deviecs is memory fragmentation. You might have free memory in your application between 2 objects. A common solution to this is the use of a dedicated allocator (operator new in C++) for the most common classes. Memory pools used purely for objects of size N don't fragment - the space between two objects will always be a multiple of N.
It might not be an actual memory leak, but maybe a situation of increasing memory usage. For example it could be allocating a continually increasing string:
string s;
for (i=0; i<n; i++)
s += "a";
50k isn't that much, maybe you should go over your source by hand and see what might be causing the issue.
This may be not a leak, but just the runtime heap not releasing memory to the operating system. This can also be fragmentation.
Possible ways to overcome this:
Split into two applications. The master application will have the simple logic with little or no dynamic memory usage. It will start the worker application to actually do work in such chunks that the worker application will not run out of memory and will restart that application periodically. This way memory is periodically returned to the operating system.
Write your own memory allocator. For example you can allocate a dedicated heap and only allocate memory from there, then free the dedicated heap entirely. This requires the operating system to support multiple heaps.
Also note that it's possible that your program runs differently on Ubuntu and on the target system and therefore different execution paths are taken and the code resulting in memory leaks is executed on the target system, but not on Ubuntu.
This does sounds like fragmentation. Fragmentation is caused by you allocating objects on the stack, say:
object1
object2
object3
object4
And then deleting some objects
object1
object3
object4
You now have a hole in the memory that is unused. If you allocate another object that's too big for the hole, the hole will remain wasted. Eventually with enough memory churn, you can end up with so many holes that they waste you memory.
The way around this is to try and decide your memory requirements up front. If you've got particular objects that you know you are creating many of, try and ensure they're the same size.
You can use a pool to make the allocations more efficient for a particular class... or at least let you track it better so you can understand what's going on and come up with a good solution.
One way of doing this is to create a single static:
struct Slot
{
Slot() : free(true) {}
bool free;
BYTE data[20]; // you'll need to tune the value 20 to what your program needs
};
Slot pool[500]; // you'll need to pick a good pool size too.
Create the pool up front when your program starts and pre-allocate it so that it is as big as the maximum requirements for your program. You may want to HeapAlloc it (or the equivalent in your OS so that you can control when it appears from somewhere in you application startup).
Then override the new and delete operators for a suspect class so that they return slots from this vector. So, your objects will be stored in this vector.
You can override new and delete for classes of the same size to be put in this vector.
Create pools of different sizes for different objects.
Just go for the worst offenders at first.
I've done something like this before and it solved my problem on an embedded device. I also was using a lot of STL, so I created a custom allocator (google for stl custom allocator - there are loads of links). This was useful for records stored in a mini-database my program used.
If your memory usage goes down, i don't think it can be defined as a memory leak.
Where are you getting reports of memory usage ? The system might just have put most of your program's memory use in virtual memory.
All i can add is that Valgrind is known to be pretty efficient at finding memory leaks !
Also, are you sure when you profiled your code, the code-coverage was enough to cover all the code-paths which might be executed on target platform?
Valgrind for sure does not lie. As has been pointed out, this might indeed be the runtime heap not releasing the memory, but i would think otherwise.
Are you using any sophisticated technique to track the scope of object..?
if yes, than valgrind is not smart enough, Though you can try by setting xscale related option with valgrind
Most applications show a pattern of memory use like this:
they use very little when they start
as they create data structures they use more and more
as they start deleting old data structures or reusing existing ones, they reach a steady state where memory use stays roughly constant
If your app is continuosly increasing in size, you may have aleak. If it increases in sizze over aperiod and then reaches arelatively steady state, you probably don't.
You can use the massif tool from Valgrind, which will show you where the most memory is allocated and how it evolves over time.