GIVEN that you have a fixed area of memory already allocated that you would like to use, what C or C++ libraries will allow you to store a dynamic structure (e.g. a hash) in that memory?
i.e. the hash library must not contain any calls to malloc or new, but must take a parameter that tells it the location and size of the memory it is permitted to use.
(bonus if the library uses offsets rather than pointers internally, in case the shared memory is mapped to different address spaces in each process that uses it)
You can write your own custom allocators for STL containers.
Dr.Dobb's: What Are Allocators Good For?
SO: Compelling examples of custom C++ STL allocators?
It's trivial to adapt a simple linear probing hash table to use a block of memory - just set its table(s) to point at the allocated memory when you create it, and don't implement anything to allocate more memory to let the table grow.
Related
According to the documentation of boost::sort (https://www.boost.org/doc/libs/1_75_0/libs/sort/doc/html/index.html) all algorithms uses "additional memory".
I couldn't find any information on how this memory is allocated in the documentation.
Does anyone know if it is dynamically heap-allocated using new during each sort operation, is it stack-space, or does boost::sort utilize some dynamic internal temporary memory buffer (like std::stable_sort used to utilize std::get_temporary_buffer in C++11)?
If heap-allocated, is it possible to pass a pre-allocated buffer to the sort operations in order to gain control over memory allocations?
I want to create a custom allocator for a multimap that will allocate the elements in shared memory.I came across boost.interprocess but found it quite complicated to implement.Is there any other workaround ?
I will not give here any implementation, rather to give you some directions.
If your shared memory abstraction or region, for example start at adress void* shMemAddr and if you decide that your stl container to use shared memory,
what needs to be done is to make container allocate memory starting at shMemAddr and further, until there is available memory to allocate in your shared pool. You can implement that using any allocation strategy, for example using malloc or placement new. Further, to be available for your container to use your allocator you need to provide your allocator as template argument, for multimap it would be multimap::allocator_type
class Alloc = allocator > as fourth template argument, after less as compare function,and, for example, if you store in your multimap pairs of int,double as key,value pairs, it would likely be something like this
multimap<int,double,less<int>,CustomAlloc<pair<int,double>>>
Now, your CustomAlloc allocator need to satisfies concept of Allocators which encapsulate specific lowlevel memory management, especially, if shared memory is resource to be allocate in, you need to arrange proper allocation of memory in a multithreaded enviornment. That means that, first, you need some structure for evidence of used memory. It can be some chained data structure, for example, and implementations like that is pretty common, so you need to keep invariants of that structure consistent. What that means is if your structure for book keeping of used(or free) memory need to be updated after succesfull allocation or deallocation it needs to be done atomicaly, so thread which possibly try to allocate memory see only structure in states before CustomAllocator allocation job is started or after allocation job is finished. For example, your first choice to do that could be using mutex to protect data, avoid races and keep invariants. This is just directions, and considering write your own allocators is not very hard, I hope this will help as good starting point.
i need to place a STL map in shared memory. Also have multiple process access that map. Any pointers to how it is done ?
I have checked this link. But need a more simpler method.
Map in Shared memory
For this to work you need to use a custom allocator that will allocate from the shared memory region, so that the map nodes are all in shared memory, and so that the pointer type of the allocator is not just a raw pointer but can refer to the shared memory region when it is mapped to different addresses in different processes.
You also need your std::map implementation to correctly use the allocator's pointer type everywhere it needs to use a pointer, and this isn't guaranteed by the standard.
The simplest way to do this currently is to use Boost.Interprocess which provides a nice API for shared memory and also provides allocators and containers that work correctly with it.
I have lately been trying to create custom containers that are similar to some of the library containers (i.e vector, list). and while I was using an allocator to allocate dynamic memory I noticed that the idea behind allocators and built in arrays are the same. allocators reserve a certain amount of raw, unconstructed dynamic memory and return a pointer to the first free location in that pool of memory. and built in arrays pretty much do the same thing. so if we have an std::allocator for strings called alloc
this codealloc.allocate(7) and this code string* array = new string[7] should have the same effect. and if we want to construct the raw memory we can call std::allocator::construct passing it the pointer returned from the allocate function, or we can have something like array[0] = string("something") to do the same thing. correct?
so what is there a difference between how an allocator work and how a built in array work?
You're right that they're fundamentally related, but not in that way. new string[7] could indeed be decomposed into allocate and construct (with a few extra bits for EH and other details).
Separating them out in the allocator interface allows much more fine-grained control for containers so that they can, for example, have memory with non-constructed objects in them, which is often vital for correct performance guarantees or semantics.
Additionally, The allocator interface is, of course, an interface with many possible implementations, such as memory arenas or object pools, which new string[7] really doesn't offer.
Finally, new T[] is shit and don't ever use it. The allocator interface is designed to be used only by fairly experienced programmers in quite limited ways- as a component of a better library component. new T[] is a language feature that everybody can just use, with terrible results.
An array is collinear container of slots for items in memory. The array is a range.
An allocator is an function object (or function) that reserves memory. The allocator can designate space from an array, stack, heap, or other areas of memory. The allocator can also be used to allocate space outside of the memory area, such as a hard drive or other device (maybe a server, cloud, etc.)
The space allocated for an array is usually determined by the compiler during the build phase.
An allocator is used for dynamic (during run-time) allocation of objects.
I have a (potentially dumb) question about the C++ STL. When I make a container (vector, set, map, etc), is it allocated on the stack or on the heap? If I make a set and put 5 million strings, will I have to worry about a stack overflow?
STL classes by default allocate their internal buffers from the heap, although these classes also allow custom allocators that allow a user to specify an alternate location to allocate from - e.g. a shared memory pool.
The default allocator for STL containers uses operator new and delete, so it's whatever those route to for the type being contained. (In general, it comes from the heap unless you do something to override that.)
You will not get a stack overflow from allocating 5 million strings. Even if you made a stack based allocator, it would probably overflow before you even inserted one string.
The container itself is allocated where you decide (it can be the stack, the heap, an object's member, etc) but the memory it uses is, by default, as others described, taken on the Free Store (managed through new and delete) which is not the same as the heap (managed through malloc/free).
Don't mix the two!