I have been researching switching my allocation method from simpling overloading new to using multiple allocators through the code base. However, how can I efficiently use multiple allocators? The only way I could devise through my research was having the allocators be globals. Although, this seemed to have issues since it is typically a "bad idea" to have the use of many globals.
I am looking to find out how to use multiple allocators efficiently. For example, I may have one allocator use only for a particular subsystem, and a different allocator for a different subsystem. I am not sure if the only way to do this is through using multiple global allocators, so I am hoping for a better insight and design.
In C++2003 the allocator model is broken and there isn't really a proper solution. For C++2011 the allocator model was fixed and you can have per instance allocators which are propagated down to contained objects (unless, of course, you choose to replace them). Generally, for this to be useful you probably want to use a dynamically polymorphic allocator type which the default std::allocator<T> is not required to be (and generally I would expect it not to be dynamically polymorphic although this may be the better implementation choice). However, [nearly] all classes in the standard C++ library which do memory allocation are templates which take the allocator type as template argument (e.g. the IOStreams are an exception but generally they don't allocate any interesting amount of memory to warrant adding allocator support).
In several of your comments you are insisting that allocators effectively need to be global: that is definitely not correct. Each allocator-aware type stores a copy of the allocator given (at least, if it has any instance level data; if it doesn't there isn't anything to store as is e.g. the case with the default allocator using operator new() and operator delete()). This effectively means that the allocation mechanism given to an object needs to stick around as long as there is any active allocator using it. This can be done using a global object but it can also be done using e.g. reference counting or associating the allocator with an object containing all objects to which it is given. For example, if each "document" (think XML, Excel, Pages, whatever structure file) passes an allocator to its members, the allocator can live as member of the document and get destroyed when the document is destroyed after all its content is destroyed. This part of the allocator model should work with pre-C++2011 classes, as long as they take an allocator argument, as well. However, in pre-C++2011 classes the allocator won't be passed to contained objects. For example, if you give an allocator to a std::vector<std::string> the C++2011 version will create the std::strings using the allocator given to the std::vector<std::string> appropriately converted to deal with std::strings. This won't happen with pre-C++2011 allocators.
To actually use allocators in a subsystem you will effectively need to pass them around, either explicitly as an argument to your functions and/or classes or implicitly by way of allocator-aware objects which serve as a context. For example, if you use any of the standard containers as [part of] the context passed around, you can obtain the used allocator using its get_allocator() method.
You can use new placement. This can be used either to specify a memory region, or to overload the type's static void* operator new(ARGS). Globals are not required, and really a bad idea here, if efficiency is important and your problems are demanding. You would need to hold on to one or more allocators, of course.
The best thing you can do is understand your problems and create strategies for your allocators based on the patterns in your program and on actual usage. The general purpose malloc is very good at what it does, so always use that as one baseline to measure against. If you don't know your usage patterns, your allocator will likely be slower than malloc.
Also keep in mind that these types you use will lose compatability with standard containers, unless you use a global or thread local and custom allocator for standard containers -- which quickly defeats the purpose in many contexts. The alternative is to also write your own allocators and containers.
Some uses for multiple allocators include reduced CPU usage, reduced fragmentation, and fewer cache misses. So the solution really depends on what type and where your allocation bottleneck is.
CPU usage will be improved by having lockless heaps for active threads, eliminating synchronization. This can be done in your memory allocator with thread local storage.
Fragmentation will be improved by having allocations with different lifespans be allocated from different heaps -- allocating background IO in a separate heap from the users active task will ensure the two do not confound one another. This is likely done by having a stack for your heaps, and push/popping when you're in different functional scopes.
Cache misses will be improved by keeping allocations within a system together. Having Quadtree/Octree allocations come from their own heap will guarantee there is locality in view frustrum queries. This is best done by overloading operator new and operator delete for the specific classes (OctreeNode).
Related
I want to write a custom memory manager/allocator for learning. I'm tempted to have a master allocator that requests n bytes of ram from the heap (via new). This would be followed by several allocator... Adaptors? Each would interface with the master, requesting a block of memory to manage, these would be stack, linear, pool, slab allocators etc each managing allocations from their slice of the master pool allocator.
The problem I have is whether I should write custom allocator_traits to interface with these for the various STL containers; or if I should just ignore the adaptor idea and simply overload new and delete to use the custom pool allocator/manager, the master one.
What I'm interested in understanding is what tangible benefit I would gain from having separate allocators for STL containers? It seems like the default std::allocator calls new and delete as needed so if I overload those to instead request from my big custom memory pool, I'd get all the benefit without the kruft of custom std::allocator code.
Or is this a matter where certain types of allocator models, like using a stack allocator for a std::deque would work better than the default allocator? And if so, wouldn't the normal stl implementation already specialise the default allocator for the various container types, or otherwise be optimised in the calls to the default allocator?
If it matters at all, I'm using C++20 via GCC 10+
If you want to replace the global allocator, including in every library you are using, you don't have to use std::allocator.
std allocators let you do things like create temporary allocation pools. Suppose you have some data structures you can guarantee will not outlive a certain scope, and you know that (whatever is allocated) 90%+ will remain allocated to the end of the scope.
A relatively simple std allocator could hand outmemory, never recycle it, and clean it up at the end of the scope much faster than any global new or delete operator could.
Whenever you have special knowledge of the contents and lifetime patterns of a container, you could hand-tune an allocator for that specific container. The standard allocator cannot. Sometimes when you are willing to make compromises that the std containers are not, you can patch their behavior with a custom allocator.
std::deque cannot efficiently use a stack allocator, because it cannot presume you'll mainly use it as a stack. You might use it mainly a queue. A stack allocator when you use it mainly as a queue would be a disaster; but if you used it 90%+ as a stack, a stack allocator could be much faster at the cost of modest memory overhead (and if 99%+, a stack allocator that handles the exceptional case and cleans up the non-stack based operations).
Finally, allocators can permit you to distinguish between kinds of containers. You might want the memory for your document (persistent) state to be allocated in one region of memory, and your "scratch" non-persistent data to be allocated elsewhere.
And yes, using a std allocator is something you should consider not doing. Optimization is fungible, and tweaking low level memory allocation is something you can work on after you have made the rest of the system more efficient and functional. Only when you have something that works, isn't fast enough, and you have identified new/delete as a fundamental bottle neck you can't design around should you say "ok, time to replace allocation!"
Use Case: Security Software needs to shred memory on delete, 'cause it cannot afford to let sensitive data remain somewhere in the physical RAM, optionally accessible by later instantiated processes. The delete operators of standard run-times won't do this expensive operation. Overwriting the heap operators might lead to linker problems with libraries depending on the runtime versions of those.
Answering the two questions in-order:
Should I write custom allocator_traits to interface my allocators for the various STL containers?
Yes, for easy manipulations. Pretty soon in the implementation, situations such as controlling memory overlaps would arise. For example, while stress-testing the implementation at full capacity of individual allocators and figuring out an algorithm for re-allocation. In this regard, you would need to specialize the allocator_traits class for the allocators rather than implement its member types from scratch using new and delete operators.
The reason allocator_traits is used is because it facilitates easy handling of certain rules that need to be respected. Such rules occur all across memory management. [Refer here for three such rules during allocator construction.]
What tangible benefit I would gain from having separate allocators for STL containers?
Absolute control of how the master allocator assigns, re-assigns, copies, moves, and destructs memory (with added controls over quantifying/enhancing performance). Pretty cool, isn't it! If the default std allocator is used, you would loose this control and rely on a (albeit very good) default implementation of memory management.
I was looking into how custom containers are created, such as eastl's container and several other models and I see that they all use an "allocator", much like std::vector does with std::allocator. Which got me thinking, why do new implementations of a vector container use an allocator when they typically have an underlying memory management override for new and delete?
Being able to replace operator new() and operator delete() (and their array versions) at program level may be sufficient for small program. If you have programs consisting of many millions lines of code, running many different threads this isn't at all suitable. You often want or even need better control. To make the use of custom allocators effective, you also need to be able to allocate subobjects using the same objects as the outer allocator.
For example, consider the use of memory arena to be used when answering a request in some sort of a server which is probably running multiple threads. Getting memory from operator new() is probably fairly expensive because it involves allocating a lock and finding a suitable chunk of memory in a heap which is getting more and more fragmented. To avoid this, you just want to allocate a few chunks of memory (ideally just one but you may not know the needed size in advance) and put all objects there. An allocator can do this. To do so, you need to inform all entities allocating memory about this chunk of memory, i.e. you need to pass the allocator to everything possibly allocating memory. If you allocate e.g. a std::vector<std::string, A> the std::string objects should know about the allocator: just telling the std::vector<std::string, A> where and how to allocate memory isn't enough to avoid most memory allocations: you also need to tell it to the std::string (well, actually the std::basic_string<char, std::char_traits<char>, B> for a suitable allocator type B which is related to A).
That is, if you really mean to take control of your memory allocations, you definitely want to pass allocators to everything which allocates memory. Using replaced versions of the global memory management facilities may help you but it is fairly constrained. If you just want to write a custom container and memory allocation isn't much of your concern you don't necessarily need to bother. In big systems which are running for extensive periods of time memory allocation is one of the many concerns, however.
Allocators are classes that define memory models to be used by Standard Library containers.
Every Standard Library container has its own default allocator, However the users of the container can provide their own allocators over the default.
This is for additional flexibility.
It ensures that users can provide their own allocator which provides an alternate form of memory management(eg: Memory Pools) apart from the regular heap.
If you want to produce a standard-compatible container then the answer is of course yes... allocators are described in the standard so they are required.
In my personal experience however allocators are not that useful... therefore if you are developing a container for a specific use to overcome some structural limitation of the standard containers then I'd suggest to forget about allocators unless you really see a reason for using them.
If instead you are developing a container just because you think you can do better than the standard vector then my guess is that you are wasting your time. I don't like the allocator idea design (dropping on the type something that shouldn't be there) but luckily enough they can be just ignored. The only annoyance with allocators when you don't need them (i.e. always) is probably some more confusion in error messages.. that however are a mess anyway.
According to C++ Primer 4th edition, page 755, there is a note saying:
Modern C++ programs ordinarily ought to use the allocator class
to allocate memory. It is safer and more flexible.
I don't quite understand this statement.
So far all the materials I read teach using new to allocate memory in C++.
An example of how vector class utilize allocator is shown in the book.
However, I cannot think of other scenarios.
Can anyone help to clarify this statement? and give me more examples?
When should I use allocator and when to use new? Thanks!
For general programming, yes you should use new and delete.
However, if you are writing a library, you should not!
I don't have your textbook, but I imagine it is discussing allocators in the context of writing library code.
Users of a library may want control over exactly what gets allocated from where. If all of the library's allocations went through new and delete, the user would have no way to have that fine-grained level of control.
All STL containers take an optional allocator template argument. The container will then use that allocator for its internal memory needs. By default, if you omit the allocator, it will use std::allocator which uses new and delete (specifically, ::operator new(size_t) and ::operator delete(void*)).
This way, the user of that container can control where memory gets allocated from if they desire.
Example of implementing a custom allocator for use with STL, and explanation: Improving Performance with Custom Pool Allocators for STL
Side Note: The STL approach to allocators is non-optimal in several ways. I recommend reading Towards a Better Allocator Model for a discussion of some of those issues.
Edit in 2019: The situation in C++ has improved since this answer was written. Stateful allocators are supported in C++11, and that support was improved in C++17. Some of the people involved in the "Towards a Better Allocator Model" were involved in those changes (eg: N2387), so that's nice (:
The two are not contradictory. Allocators are a PolicyPattern or StrategyPattern used by the STL libraries' container adapters to allocate chunks of memory for use with objects.
These allocators frequently optimize memory allocation by allowing
* ranges of elements to be allocated at once, and then initialized using a placement new
* items to be selected from secondary, specialized heaps depending on blocksize
One way or another, the end result will (almost always) be that the objects are allocated with new (placement or default)
Another vivid example would be how e.g. boost library implements smartpointers. Because smartpointers are very small (with little overhead) the allocation overhead might become a burden. It would make sense for the implementation to define a specialized allocator to do the allocations, so one may have efficient std::set<> of smartpointers, std::map<..., smartpointer> etc.
(Now I'm almost sure that boost actually optimizes storage for most smartpointers by avoiding any virtuals, therefore the vft, making the class a POD structure, with only the raw pointer as storage; some of the example will not apply. But then again, extrapolate to other kinds of smartpointer (refcounting smartpointers, pointers to member functions, pointers to member functions with instance reference etc. etc.))
I am working on a plugin for an application, where the memory should be allocated by the Application and keep track of it. Hence, memory handles should be obtained from the host application in the form of buffers and later on give them back to the application. Now, I am planning on using STL Vectors and I am wondering what sort of memory allocation does it use internally.
Does it use 'new' and 'delete' functions internally? If so, can I just overload 'new' and 'delete' with my own functions? Or should I create my own template allocator which looks like a difficult job for me since I am not that experienced in creating custom templates.
Any suggestions/sample code are welcome. Memory handles can be obtained from the application like this
void* bufferH = NULL;
bufferH = MemReg()->New_Mem_Handle(size_of_buffer);
MemReg()->Dispose_Mem_Handle(bufferH); //Dispose it
vector uses std::allocator by default, and std::allocator is required to use global operator new (that is, ::operator new(size_t)) to obtain the memory (20.4.1.1). However, it isn't required to call it exactly once per call to allocator::allocate.
So yes, if you replace global operator new then vector will use it, although not necessarily in a way that really allows your implementation to manage memory "efficiently". Any special tricks you want to use could, in principle, be made completely irrelevant by std::allocator grabbing memory in 10MB chunks and sub-allocating.
If you have a particular implementation in mind, you can look at how its vector behaves, which is probably good enough if your planned allocation strategy is inherently platform-specific.
STL containers use an allocator they are given at construction time, with a default allocator that uses operator new and operator delete.
If you find the default is not working for you, you can provide a custom allocator that conforms to the container's requirements. There are some real-world examples cited here.
I would measure performance using the default first, and optimize only if you really need to. The allocator abstraction offers you a relatively clean way to fine-tune here without major redesign. How you use the vector could have far more performance impact than the underlying allocator (reserve() in advance, avoid insert and removal in the middle of the range of elements, handle copy construction of elements efficiently - the standard caveats).
std::vector uses the unitialized_* functions to construct its elements from raw memory (using placement new). It allocates storage using whatever allocator it was created with, and by default, that allocator uses ::operator new(size_t) and ::operator delete(void *p) directly (i.e., not a type specific operator new).
From this article, "The concept of allocators was originally introduced to provide an abstraction for different memory models to handle the problem of having different pointer types on certain 16-bit operating systems (such as near, far, and so forth)" ...
"The standard provides an allocator that internally uses the global operators 'new' and 'delete'"
The author also points out the alocator interface isn't that scary. As Neil Buchanan would say, "try it yourself!"
The actual std::allocator has been optimized for a rather large extent of size objects. It isn't the best when it comes to allocating many small objects nor is it the best for many large objects. That being said, it also wasn't written for multi-threaded applications.
May I suggest, before attempting to write your own you check out the Hoard allocator if you're going the multi-threaded route. (Or you can check out the equally appealing Intel TBB page too.)
Looking at vector, I realized that I have never used the second argument when creating vectors.
std::vector<int> myInts; // this is what I usually do
std::vector<int, ???> myOtherInts; // but is there a second argument there?
Looking at the link above it says that it is for:
Allocator object to be used instead of constructing a new one.
or, as for this one:
Allocator: Type of the allocator object used to define the storage allocation model. By default, the allocator class template for type T is used, which defines the simplest memory allocation model and is value-independent.
I guess it has to do with something with memory management. However, I am not sure how to use that.
Any pointers regarding this?
The default allocator, std::allocator<>, will handle all allocations made by std::vector<> (and others). It will make new allocations from the heap each time a new allocation is needed.
By providing a custom allocator, you can for instance allocate a big chunk of memory up front and then slice it up and hand out smaller pieces when separate allocations are needed. This will increase the allocation speed dramatically, which is good for example in games, at the cost of increased complexity as compared to the default allocator.
Some std type implementations have internal stack-based storage for small amounts of data. For instance, std::basic_string<> might use what is called a small string optimization, where only strings longer than some fixed length, say 16 characters (just an example!), gets an allocation from the allocator, otherwise an internal array is used.
Custom allocators are rarely used in general case. Some examples of where they can be useful:
Optimization for a specific pattern of allocations. For example, a concurrent program can pre-allocate a large chunk of memory via standard means at the beginning of task execution and then shave off pieces off it without blocking on the global heap mutex. When task is completed, entire memory block can be disposed of. To use this technique with STL containers, a custom allocator can be employed.
Embedded software, where a device has several ranges of memory with different properties (cached/noncached, fast/slow, volatile/persistent etc). A custom allocator can be used to place objects stored in an STL container in a specific memory region.
Maybe this will help: http://www.codeguru.com/cpp/cpp/cpp_mfc/stl/article.php/c4079
You may try google for: stl allocator.
Allocators (STL) help you to manage memory for your objects in vector class. you may use the custom allocator for different memory model( etc).
Hi you can find example of custom allocator http://www.codeproject.com/KB/cpp/allocator.aspx