I trying to implement std::list (MSVC). And one thing I cannot understand:
template <class _Value_type, class _Voidptr> // voidptr? For what?
struct _List_node { // list node
using value_type = _Value_type;
using _Nodeptr = _Rebind_pointer_t<_Voidptr, _List_node>; // what is the purpose of such rebind?
...
}
I understand the reason of allocator rebind, but pointer? Why should I use it and where?
UPD: I understand, what rebind is. I mean, why not just _Nodeptr*? Why do I need rebind? (Thanks to Evg)
The answer to this question comes from allocators, too. Let's take a look at how _Rebind_pointer_t is defined:
template <class _Ptr, class _Ty>
using _Rebind_pointer_t = typename pointer_traits<_Ptr>::template rebind<_Ty>;
That is, we have
template <class _Value_type, class _Voidptr>
struct _List_node {
using _Nodeptr = typename pointer_traits<_Voidptr>::template rebind<_List_node>;
// ...
}
Now let's take a look at how _List_node is used:
using _Node = _List_node<_Ty, typename _Alty_traits::void_pointer>;
Effectively, we rebind allocator's void_pointer to _List_node pointer. This trick is needed to support allocators that use fancy pointers internally.
One such example can be found in Boost.Interprocess library. It has boost::interprocess::allocator:
An STL compatible allocator that uses a segment manager as memory source. The internal pointer type will of the same type (raw, smart) as typename SegmentManager::void_pointer type. This allows placing the allocator in shared memory, memory mapped-files, etc...
For example, we can write
namespace bi = boost::interprocess;
using Allocator = bi::allocator<int, bi::managed_shared_memory::segment_manager>;
std::list<int, Allocator> list(/* allocator object */);
Now std::allocator_traits<decltype(list)::allocator_type>::void_pointer will be not void* as with default allocator, but boost::interprocess::offset_ptr<void, ...>. As a result, _Nodeptr will be not _Nodeptr*, but boost::interprocess::offset_ptr<_Nodeptr, ...>.
The user instantiates the list with the value_type.
For example for list<int> the value_type would be int.
Also the list allocator (which can also be provided) allocates memory for objects of value_type's.
But the value_type is not what the list internally holds.
The list holds internally the **Nodes** for which the value_type is a member of.
So to be able to convert allocation and pointer's from value_type to Node (which holds value_type and pointer to the next node at least) the rebind is used.
In contrary, this would not be need for a vector<int> for example.
That's because the internal representation of vector will normaly hold internally the pointer to array of value_type's objects and that's int in this case. So no rebind needed here.
Related
I cannot figure out how std::set and std::map, for example allocate the memory for their nodes if they have allocators of type
std::allocator<Key>
and
std::allocator<std::pair<const Key, T> >
respectively. As far as I can guess, there should be a code like this, in std::set, for example:
std::pair<iterator, bool> insert(const value_type& value)
{
...
Node * node = new Node();
node->value = value;
...
return InsertNode(node);
}
or
std::pair<iterator, bool> insert(const value_type& value)
{
...
Node * node = new Node();
node->p_value = a1.allocate(1);
*(node->p_value) = value;
...
return InsertNode(node);
}
where Node is some internal structure like red-black tree node, for example.
So the question is how is this Node memory allocated?
Allocators in C++ (for some reason) are expected to be typed. That is, a specific allocator class instance allocates objects of a specific type.
However, because the actual type a container may need to allocate can be different from the container's value type (the type the container logically contains), allocators are (mostly) required to be able to be converted into an alternate allocator instance that allocates objects of any other type. This process is called "rebinding", and it is initiated by invoking allocator.rebind<U>, where U is the type the system wants to actually allocate.
A new allocator instance which is a rebound form of the original allocator must allocate from the same memory pool as the original allocator. Rebinding is therefore treated as a type-based change, not a truly distinct object.
Implementations of standard library containers are not permitted to use new or delete; all of their dynamic de/allocation and object creation/destruction within allocated memory must be performed through the allocator.
When std::set<T> goes to insert an item, it will rebind your allocator to some node-type which internally contains either a T itself or enough properly aligned storage to create a T within. It will then use the allocator's interface to create that node type and initialize the T it contains. std::map is a bit more complicated, but essentially its node type must store a Key and a Value.
The allocator that you provide is rebound to the type of the node structure, and then used to allocate the nodes.
std::allocator (the default) will call operator new, which then will do something implementation-specific (usually calling malloc).
Some container A has a template parameter Alloc (that is a template too) representing an allocator type. A specifies Alloc for the type A::Node.
template <template <T> Alloc>
class A {
struct Node {
};
Alloc<Node> allocator_; // the allocator object
};
Please excuse me for possibly wrong C++ code above.
So, allocator_.allocate(1) will allocate sizeof(A::Node) bytes. But during operation, container A needs a memory for some object of other than A::Node type, say a temporary string (of chars).
From technical point of view, I could use existing allocator in such a dirty way:
size_t string_len = 500;
// how much objects spanned in memory is enough to fit our string?
size_t equal_size = (string_len / sizeof(Node)) + 1;
auto mem = allocator_.allocate(equal_size);
char *p = (char*)mem; // reinterpret cast
// ... use p to store the string ... memcpy(p, str_src, str_len); //
// Now p is not needed, so return memory to allocator:
allocator_.deallocate(mem, equal_size);
Is there a less dirty approach, considering I need no more than 1 allocator and I wish to put all the memory management to it?
All this comes from those needs:
to have a single allocator that could be killed to free all (possibly leaked) memory that A is allocated for any its purposes during operation
to have not more than 1 allocator (including the default ::new, ::delete)
std::allocator has a member type rebind for exactly that purpose:
std::allocator<Node> alloc;
std::allocator<Node>::rebind<char>::other char_alloc;
char * mem = char_alloc.allocate(string_len);
You can use an allocator's rebind for this. From this documentation:
A structure that enables an allocator for objects of one type to allocate storage for objects of another type.
it is built exactly for your case - taking an allocator type oriented to one type, and building the corresponding one oriented to some other type.
In your case, it might look like
typename Alloc<T>::rebind<Node>::other allocator_;
You should probably use Alloc::rebind member template to get an allocator for that another object.
However, that does mean that you do have 2 allocators. The advantage of rebind is to allow the user of your template to specify the allocator type only for a single allocated type.
Also note that rebind is optional, so if you must support such allocators, you'll need to pass the other allocator as an argument, but you can still use the rebound allocator as a default value.
I read the book [The C++ Standard Library Second Edition] and found the section below:
namespace std {
template <typename T, typename D>
class unique_ptr<T[], D>
{
public:
typedef ... pointer; // may be D::pointer
typedef T element_type;
typedef D deleter_type;
...
};
}
The element type T might be void so that the unique pointer owns an object with an unspecified
type, like void* does. Note also that a type pointer is defined, which is not necessarily defined
as T*. If the deleter D has a pointer typedef, this type will be used instead. In such a case,
the template parameter T has only the effect of a type tag, because there is no member as part of
class unique_ptr<> that depends on T; everything depends on pointer. The advantage is that a
unique_ptr can thus hold other smart pointers.
I still cant not understand the purpose of "everything depends on pointer" after i reading this section.Is there anyone could provide some samples kindly?Thanks.
LWG issue 673 added pointer to the unique_ptr specification. It contains this bullet point for motivation:
Efforts have been made to better support containers and smart pointers in shared memory contexts. One of the key hurdles in such
support is not assuming that a pointer type is actually a T*. This
can easily be accomplished for unique_ptr by having the deleter
define the pointer type: D::pointer. Furthermore this type can
easily be defaulted to T* should the deleter D choose not to
define a pointer type (example implementation here[broken link]). This
change has no run time overhead. It has no interface overhead on
authors of custom delter types. It simply allows (but not requires)
authors of custom deleter types to define a smart pointer for the
storage type of unique_ptr if they find such functionality useful.
std::default_delete is an example of a deleter which defaults
pointer to T* by simply ignoring this issue and not including a
pointer typedef.
See boost::offset_ptr for an example smart pointer that can reference shared memory.
unique_ptr is designed in such a way that, if you don't provide a deleter in the declaration, thanks to default template parameters it uses a default_deleter(std::default_delete<T>) that just "delete" your pointer. It uses SFINAE to look for the definition of "pointer" in your Deleter Type. If your unique_ptr holds another smart_ptr(which have the "pointer" defined), then in the destructor, it calls the deleter of the unique_ptr that you're holding.
Example:
class BufferClass
{
public:
BufferClass()
{
// init
}
~BufferClass()
{
delete [] buffer;
}
int *buffer;
};
// a short declaration for unique_ptr<BufferClass>
typedef unique_ptr<BufferClass> BufferPointer;
BufferPointer myBuffer(new BufferClass(10)); // myBuffer holds BufferClass.
// when myBuffer goes out of scope, u'll have BufferClass::~BufferClass called.
//let's say you want to store a unique_ptr in another unique_ptr.
unique_ptr<BufferPointer> myBufferPointerPointer(&myBuffer);
// so when myBufferPointerPointer goes out of scope, BufferClass::~BufferClass will be called eventually
It seems to be the month of C++ templates for me...
I have a SecureString. SecureString looks just like a std::string, except it uses a custom allocator which zeroizes on destruction:
class SecureString
{
public:
typedef std::basic_string< char, std::char_traits<char>, zallocator<char> > SecureStringBase;
typedef zallocator<char>::size_type size_type;
static const size_type npos = static_cast<size_type>(-1);
....
private:
SecureStringBase m_base;
};
The full code for SecureString can be found at http://code.google.com/p/owasp-esapi-cplusplus/source/browse/trunk/esapi/util/SecureString.h; and the code for the allocator can be found at http://code.google.com/p/owasp-esapi-cplusplus/source/browse/trunk/esapi/util/zAllocator.h.
Currently, we have a swap defined that takes a std::string as an argument:
void SecureString::swap(std::string& str)
{
SecureStringBase temp(str.data(), str.size());
m_base.swap(temp);
str = std::string(temp.data(), temp.size());
}
I feel like I'm missing an opportunity in swap because the underlying types only differ by allocators. Can anyone see a way to avoid the temporary? Is it possible to use rebind to make this run faster?
EDIT: SecureString::swap(std::string& str) is now gone. Reference to the function in this thread has been left in place for posterity.
Jeff
Unfortunately... no.
This is not what rebind is for. rebind is used because an allocator is meant to allocate objects of one type, and one type only (std::allocator<T>) in the STL.
However, there is a trick. When you instantiate std::list<T, std::allocator<T>> for example, then the allocator does not have to allocate Ts, it has to allocate some internal structure instead like __list_node<T>, and that is when rebind is put to use, it creates a new allocator, sibling of the precedent (they differ only by the template parameter and likely share the same memory pool under the covers).
In your case however, your allocator and the std::string allocator are different, and thus they cannot exchange memory. So you have to do a copy.
You can optimize the void swap(SecureString&, SecureString&) operation, but not this one.
One question: why not typedef std::string<char, SecureAllocator<char>> SecureString; ?
This is a followup from stl allocator, copy constructor of other type, rebind
I am using std::map and want a custom allocator that can reuse the storage for the internal nodes. The items being stored are pointers, so I'm not talking about reusing them, just the internal allocations for the map.
The main requirements is that different instances of the map cannot share an object pool, it must be unique per instance.
I don't need a complete solution, I'd just like an idea of how to cope with the required copy constructor that takes allocators of a different type. I don't know how to manage the internal memory in that case.
As you point out in the other question, the allocators shouldn't have any state. Use thread-local storage or a pointer in each allocator object to the memory pool: the allocators merely become a type-specific interface to that pool.
struct MemoryPool {
// none of this depends on the type of objects being allocated
};
template<class T>
struct MyAllocator {
template<class U> struct rebind { typedef MyAllocator<U> other; };
MemoryPool *_pool; // copied to any allocator constructed
template<class U>
MyAllocator(MyAllocator const &other) : _pool(other._pool) {}
// allocate, deallocate use _pool
// construct, destruct deal with T
};