I am evaluating to use the basic_string template to implement a string like object to be allocated using an external memory manager.
That memory manager, keeps the size of the maximum size of the memory allocated and the current size (allowing for that current size to increase up to the maximum one).
In order to avoid redundance I would like to make use of that data for the string.
Does any body knows if it is possible and where to look for detailed instructions ?
I already know that it is possible to give a propierary allocator, but nothing more.
It is possible, for sure. Just provide a custom allocator which implements the std::allocator (http://www.cplusplus.com/reference/memory/allocator/) interface.
Then:
typedef std::basic_string<
char, std::char_traits<char>, custom_allocator<char> >
custom_string;
But be warned, this string is incompatible to std::string and you might have to implement the conversions custom_string <-> std::string.
It can look like that:
template<class T>
struct CustomAllocator: std::allocator<T> {
template<class U>
struct rebind {
typedef CustomAllocator<U> other;
};
};
typedef std::basic_string<char, std::char_traits<char>, CustomAllocator<char>> CustomString;
Now CustomString uses your CustomAllocator instead of std::allocator.
To change the way the memory is actually allocated, your can define custom allocate and deallocate(and probably some other)methods inside the CustomAllocator class(and add all the required logic for it).
Related
I trying to implement std::list (MSVC). And one thing I cannot understand:
template <class _Value_type, class _Voidptr> // voidptr? For what?
struct _List_node { // list node
using value_type = _Value_type;
using _Nodeptr = _Rebind_pointer_t<_Voidptr, _List_node>; // what is the purpose of such rebind?
...
}
I understand the reason of allocator rebind, but pointer? Why should I use it and where?
UPD: I understand, what rebind is. I mean, why not just _Nodeptr*? Why do I need rebind? (Thanks to Evg)
The answer to this question comes from allocators, too. Let's take a look at how _Rebind_pointer_t is defined:
template <class _Ptr, class _Ty>
using _Rebind_pointer_t = typename pointer_traits<_Ptr>::template rebind<_Ty>;
That is, we have
template <class _Value_type, class _Voidptr>
struct _List_node {
using _Nodeptr = typename pointer_traits<_Voidptr>::template rebind<_List_node>;
// ...
}
Now let's take a look at how _List_node is used:
using _Node = _List_node<_Ty, typename _Alty_traits::void_pointer>;
Effectively, we rebind allocator's void_pointer to _List_node pointer. This trick is needed to support allocators that use fancy pointers internally.
One such example can be found in Boost.Interprocess library. It has boost::interprocess::allocator:
An STL compatible allocator that uses a segment manager as memory source. The internal pointer type will of the same type (raw, smart) as typename SegmentManager::void_pointer type. This allows placing the allocator in shared memory, memory mapped-files, etc...
For example, we can write
namespace bi = boost::interprocess;
using Allocator = bi::allocator<int, bi::managed_shared_memory::segment_manager>;
std::list<int, Allocator> list(/* allocator object */);
Now std::allocator_traits<decltype(list)::allocator_type>::void_pointer will be not void* as with default allocator, but boost::interprocess::offset_ptr<void, ...>. As a result, _Nodeptr will be not _Nodeptr*, but boost::interprocess::offset_ptr<_Nodeptr, ...>.
The user instantiates the list with the value_type.
For example for list<int> the value_type would be int.
Also the list allocator (which can also be provided) allocates memory for objects of value_type's.
But the value_type is not what the list internally holds.
The list holds internally the **Nodes** for which the value_type is a member of.
So to be able to convert allocation and pointer's from value_type to Node (which holds value_type and pointer to the next node at least) the rebind is used.
In contrary, this would not be need for a vector<int> for example.
That's because the internal representation of vector will normaly hold internally the pointer to array of value_type's objects and that's int in this case. So no rebind needed here.
Particularly, why do we have
template<typename T, typename A = allocator<T>>
class vector
{
A alloc;
//...
};
instead of
template<typename T>
class vector
{
allocator<T> alloc;
//...
};
I saw this in a C++ manual and it confused me quite a bit. What other kinds of allocators could one possibly want/need?
Because this would work with only one allocator - standard one. But what if you want to allocate memory differently? For example, you might want to use shared memory, or file-backed memory, or anything else.
This is the whole point of having allocators - to allow user to customize the way memory is going to be allocated and freed.
Some container A has a template parameter Alloc (that is a template too) representing an allocator type. A specifies Alloc for the type A::Node.
template <template <T> Alloc>
class A {
struct Node {
};
Alloc<Node> allocator_; // the allocator object
};
Please excuse me for possibly wrong C++ code above.
So, allocator_.allocate(1) will allocate sizeof(A::Node) bytes. But during operation, container A needs a memory for some object of other than A::Node type, say a temporary string (of chars).
From technical point of view, I could use existing allocator in such a dirty way:
size_t string_len = 500;
// how much objects spanned in memory is enough to fit our string?
size_t equal_size = (string_len / sizeof(Node)) + 1;
auto mem = allocator_.allocate(equal_size);
char *p = (char*)mem; // reinterpret cast
// ... use p to store the string ... memcpy(p, str_src, str_len); //
// Now p is not needed, so return memory to allocator:
allocator_.deallocate(mem, equal_size);
Is there a less dirty approach, considering I need no more than 1 allocator and I wish to put all the memory management to it?
All this comes from those needs:
to have a single allocator that could be killed to free all (possibly leaked) memory that A is allocated for any its purposes during operation
to have not more than 1 allocator (including the default ::new, ::delete)
std::allocator has a member type rebind for exactly that purpose:
std::allocator<Node> alloc;
std::allocator<Node>::rebind<char>::other char_alloc;
char * mem = char_alloc.allocate(string_len);
You can use an allocator's rebind for this. From this documentation:
A structure that enables an allocator for objects of one type to allocate storage for objects of another type.
it is built exactly for your case - taking an allocator type oriented to one type, and building the corresponding one oriented to some other type.
In your case, it might look like
typename Alloc<T>::rebind<Node>::other allocator_;
You should probably use Alloc::rebind member template to get an allocator for that another object.
However, that does mean that you do have 2 allocators. The advantage of rebind is to allow the user of your template to specify the allocator type only for a single allocated type.
Also note that rebind is optional, so if you must support such allocators, you'll need to pass the other allocator as an argument, but you can still use the rebound allocator as a default value.
I would like to create my own allocator class, and I wonder a few things :
I have to define the following types : typedef size_t size_type;and typedef ptrdiff_t difference_type;. Can i use something different than size_t ? If I want to use the STL container in a class that uses uint32_t for lengths and positions instead of size_t, can I use uint32_t instead, or does it have to be size_t ? Can it create overflows or other unpleasant things if i use a different type, or even a signed type ?
How can I passed a parameter to my allocator ? Do I have to add a constructor with a parameter to it, and construct it manually before passing it to a constructor of the STL container to use it with ?
For the function deallocate defined below, can i just ignore the number of items to deallocate ?
void deallocate( pointer p, size_type nNum )
{
(void) nNum;
delete[] p;
}
What is the "rebinding" stuff all about ?
..
template< class U >
struct rebind
{
typedef Allocator< U > other;
};
template< class U >
Allocator( const Allocator< U > & oAlloc )
Thank you. :)
This is your allocator, so you can use the type you want.
But, note, that if you expect to use it with STL containers, you will have to use a type that STL containers expect.
size_t seems appropriate here.
You can provide parameters to your allocator like if it was a normal class.
Constructors, initialization methods or setters are fine.
In fact, you can provide all functionalities you want to your allocator, but you have to respect the allocate, deallocate signature.
You can ignore the size of your deallocate function. I saw a lot of standard allocator implementations that did not use this parameter. This information could be usefull sometimes, for example if your allocator switch to different allocation strategies depending on size, this parameter could be helpfull in deallocate method to switch on the good deallocate implementation.
It seems to be the month of C++ templates for me...
I have a SecureString. SecureString looks just like a std::string, except it uses a custom allocator which zeroizes on destruction:
class SecureString
{
public:
typedef std::basic_string< char, std::char_traits<char>, zallocator<char> > SecureStringBase;
typedef zallocator<char>::size_type size_type;
static const size_type npos = static_cast<size_type>(-1);
....
private:
SecureStringBase m_base;
};
The full code for SecureString can be found at http://code.google.com/p/owasp-esapi-cplusplus/source/browse/trunk/esapi/util/SecureString.h; and the code for the allocator can be found at http://code.google.com/p/owasp-esapi-cplusplus/source/browse/trunk/esapi/util/zAllocator.h.
Currently, we have a swap defined that takes a std::string as an argument:
void SecureString::swap(std::string& str)
{
SecureStringBase temp(str.data(), str.size());
m_base.swap(temp);
str = std::string(temp.data(), temp.size());
}
I feel like I'm missing an opportunity in swap because the underlying types only differ by allocators. Can anyone see a way to avoid the temporary? Is it possible to use rebind to make this run faster?
EDIT: SecureString::swap(std::string& str) is now gone. Reference to the function in this thread has been left in place for posterity.
Jeff
Unfortunately... no.
This is not what rebind is for. rebind is used because an allocator is meant to allocate objects of one type, and one type only (std::allocator<T>) in the STL.
However, there is a trick. When you instantiate std::list<T, std::allocator<T>> for example, then the allocator does not have to allocate Ts, it has to allocate some internal structure instead like __list_node<T>, and that is when rebind is put to use, it creates a new allocator, sibling of the precedent (they differ only by the template parameter and likely share the same memory pool under the covers).
In your case however, your allocator and the std::string allocator are different, and thus they cannot exchange memory. So you have to do a copy.
You can optimize the void swap(SecureString&, SecureString&) operation, but not this one.
One question: why not typedef std::string<char, SecureAllocator<char>> SecureString; ?