It seems to be the month of C++ templates for me...
I have a SecureString. SecureString looks just like a std::string, except it uses a custom allocator which zeroizes on destruction:
class SecureString
{
public:
typedef std::basic_string< char, std::char_traits<char>, zallocator<char> > SecureStringBase;
typedef zallocator<char>::size_type size_type;
static const size_type npos = static_cast<size_type>(-1);
....
private:
SecureStringBase m_base;
};
The full code for SecureString can be found at http://code.google.com/p/owasp-esapi-cplusplus/source/browse/trunk/esapi/util/SecureString.h; and the code for the allocator can be found at http://code.google.com/p/owasp-esapi-cplusplus/source/browse/trunk/esapi/util/zAllocator.h.
Currently, we have a swap defined that takes a std::string as an argument:
void SecureString::swap(std::string& str)
{
SecureStringBase temp(str.data(), str.size());
m_base.swap(temp);
str = std::string(temp.data(), temp.size());
}
I feel like I'm missing an opportunity in swap because the underlying types only differ by allocators. Can anyone see a way to avoid the temporary? Is it possible to use rebind to make this run faster?
EDIT: SecureString::swap(std::string& str) is now gone. Reference to the function in this thread has been left in place for posterity.
Jeff
Unfortunately... no.
This is not what rebind is for. rebind is used because an allocator is meant to allocate objects of one type, and one type only (std::allocator<T>) in the STL.
However, there is a trick. When you instantiate std::list<T, std::allocator<T>> for example, then the allocator does not have to allocate Ts, it has to allocate some internal structure instead like __list_node<T>, and that is when rebind is put to use, it creates a new allocator, sibling of the precedent (they differ only by the template parameter and likely share the same memory pool under the covers).
In your case however, your allocator and the std::string allocator are different, and thus they cannot exchange memory. So you have to do a copy.
You can optimize the void swap(SecureString&, SecureString&) operation, but not this one.
One question: why not typedef std::string<char, SecureAllocator<char>> SecureString; ?
Related
I trying to implement std::list (MSVC). And one thing I cannot understand:
template <class _Value_type, class _Voidptr> // voidptr? For what?
struct _List_node { // list node
using value_type = _Value_type;
using _Nodeptr = _Rebind_pointer_t<_Voidptr, _List_node>; // what is the purpose of such rebind?
...
}
I understand the reason of allocator rebind, but pointer? Why should I use it and where?
UPD: I understand, what rebind is. I mean, why not just _Nodeptr*? Why do I need rebind? (Thanks to Evg)
The answer to this question comes from allocators, too. Let's take a look at how _Rebind_pointer_t is defined:
template <class _Ptr, class _Ty>
using _Rebind_pointer_t = typename pointer_traits<_Ptr>::template rebind<_Ty>;
That is, we have
template <class _Value_type, class _Voidptr>
struct _List_node {
using _Nodeptr = typename pointer_traits<_Voidptr>::template rebind<_List_node>;
// ...
}
Now let's take a look at how _List_node is used:
using _Node = _List_node<_Ty, typename _Alty_traits::void_pointer>;
Effectively, we rebind allocator's void_pointer to _List_node pointer. This trick is needed to support allocators that use fancy pointers internally.
One such example can be found in Boost.Interprocess library. It has boost::interprocess::allocator:
An STL compatible allocator that uses a segment manager as memory source. The internal pointer type will of the same type (raw, smart) as typename SegmentManager::void_pointer type. This allows placing the allocator in shared memory, memory mapped-files, etc...
For example, we can write
namespace bi = boost::interprocess;
using Allocator = bi::allocator<int, bi::managed_shared_memory::segment_manager>;
std::list<int, Allocator> list(/* allocator object */);
Now std::allocator_traits<decltype(list)::allocator_type>::void_pointer will be not void* as with default allocator, but boost::interprocess::offset_ptr<void, ...>. As a result, _Nodeptr will be not _Nodeptr*, but boost::interprocess::offset_ptr<_Nodeptr, ...>.
The user instantiates the list with the value_type.
For example for list<int> the value_type would be int.
Also the list allocator (which can also be provided) allocates memory for objects of value_type's.
But the value_type is not what the list internally holds.
The list holds internally the **Nodes** for which the value_type is a member of.
So to be able to convert allocation and pointer's from value_type to Node (which holds value_type and pointer to the next node at least) the rebind is used.
In contrary, this would not be need for a vector<int> for example.
That's because the internal representation of vector will normaly hold internally the pointer to array of value_type's objects and that's int in this case. So no rebind needed here.
Some container A has a template parameter Alloc (that is a template too) representing an allocator type. A specifies Alloc for the type A::Node.
template <template <T> Alloc>
class A {
struct Node {
};
Alloc<Node> allocator_; // the allocator object
};
Please excuse me for possibly wrong C++ code above.
So, allocator_.allocate(1) will allocate sizeof(A::Node) bytes. But during operation, container A needs a memory for some object of other than A::Node type, say a temporary string (of chars).
From technical point of view, I could use existing allocator in such a dirty way:
size_t string_len = 500;
// how much objects spanned in memory is enough to fit our string?
size_t equal_size = (string_len / sizeof(Node)) + 1;
auto mem = allocator_.allocate(equal_size);
char *p = (char*)mem; // reinterpret cast
// ... use p to store the string ... memcpy(p, str_src, str_len); //
// Now p is not needed, so return memory to allocator:
allocator_.deallocate(mem, equal_size);
Is there a less dirty approach, considering I need no more than 1 allocator and I wish to put all the memory management to it?
All this comes from those needs:
to have a single allocator that could be killed to free all (possibly leaked) memory that A is allocated for any its purposes during operation
to have not more than 1 allocator (including the default ::new, ::delete)
std::allocator has a member type rebind for exactly that purpose:
std::allocator<Node> alloc;
std::allocator<Node>::rebind<char>::other char_alloc;
char * mem = char_alloc.allocate(string_len);
You can use an allocator's rebind for this. From this documentation:
A structure that enables an allocator for objects of one type to allocate storage for objects of another type.
it is built exactly for your case - taking an allocator type oriented to one type, and building the corresponding one oriented to some other type.
In your case, it might look like
typename Alloc<T>::rebind<Node>::other allocator_;
You should probably use Alloc::rebind member template to get an allocator for that another object.
However, that does mean that you do have 2 allocators. The advantage of rebind is to allow the user of your template to specify the allocator type only for a single allocated type.
Also note that rebind is optional, so if you must support such allocators, you'll need to pass the other allocator as an argument, but you can still use the rebound allocator as a default value.
I have created a little class and am trying to use it as the data in an STL hash_map. Lots of searching has got me as far as knowing I need to add stuff as the Allocator, but so far I have seen no examples of how to use the default allocators in STL in my class. Could someone help and show me please? I've put a reduced version of my class below.
class cwVariable {
public:
cwVariable();
template <typename T>
cwVariable(T value)
{
stash = new byte[sizeof(T)];
memcpy(stash, &value, sizeof(T));
}
template <typename T>
void set(T value)
{
if (stash != 0) {
memcpy(stash, &value, sizeof(T));
}
}
template <typename T>
T get()
{
T out;
memcpy(&out, stash, sizeof(T));
return out;
}
~cwVariable();
private:
void * stash = 0;
};
cwVariable::~cwVariable() {
delete stash;
}
For completeness: I'm using the Arduino 1.6.4 toolchain, the STL from a port for Arduino and targetting an Arduino Mega 2560. Using the Visual Micro add-in for Visual Studio 2012.
Added later:
My real code handles more than above, but just C type string pointers, not String. The trail started when I added
std::hash_map<const char*, cwVariable> months;
to my code and got the output
cwVar.cpp.o:In function `std::hash_map<char const*, cwVariable, std::hash<char const*>,
std::equal_to<char const*>, std::allocator<cwVariable> >::operator[](char const* const&)'
stl_hash_map.h:cwVariable()'
collect2.exe*:error: ld returned 1 exit status as the error.
This made me research more and got me worried about allocators.
You don't need an allocator. Those are for when you are doing your own memory management (e.g., a memory pool). What you do need, and what your class does not have, is a working deep copy constructor. Your class instead has the freebie shallow copy constructor. This will copy the pointer rather than the contents of the pointer.
As is, your class doesn't contain the necessarily information to know how to copy construct an instance. One way to solve this would be to store the size of the allocated type as a data member. The copy constructor will allocate a stash of the right size and memcpy the source stash to the new stash of the new instance.
A much better approach would be to make the entire class a class template, and this goes after an underlying problem with your design. Your use of void* and memcpy are a recipe for future problems. As is, your class works for POD classes, but not much else.
If you templatize the entire class and change void* stash to T stash, the freebie copy constructor is exactly what you want. If you change the void* stash to T* stash, you will have to provide a copy constructor, but it's rather simple. In both cases, your get and set member functions are trivial, and your code is much more generic than is your current void* stash / memcpy implementation.
Simply using your class as a data type in hash_map does not require you to have an allocator. You only need allocator if you want to implement specific allocation policy for your map.
I would like to create my own allocator class, and I wonder a few things :
I have to define the following types : typedef size_t size_type;and typedef ptrdiff_t difference_type;. Can i use something different than size_t ? If I want to use the STL container in a class that uses uint32_t for lengths and positions instead of size_t, can I use uint32_t instead, or does it have to be size_t ? Can it create overflows or other unpleasant things if i use a different type, or even a signed type ?
How can I passed a parameter to my allocator ? Do I have to add a constructor with a parameter to it, and construct it manually before passing it to a constructor of the STL container to use it with ?
For the function deallocate defined below, can i just ignore the number of items to deallocate ?
void deallocate( pointer p, size_type nNum )
{
(void) nNum;
delete[] p;
}
What is the "rebinding" stuff all about ?
..
template< class U >
struct rebind
{
typedef Allocator< U > other;
};
template< class U >
Allocator( const Allocator< U > & oAlloc )
Thank you. :)
This is your allocator, so you can use the type you want.
But, note, that if you expect to use it with STL containers, you will have to use a type that STL containers expect.
size_t seems appropriate here.
You can provide parameters to your allocator like if it was a normal class.
Constructors, initialization methods or setters are fine.
In fact, you can provide all functionalities you want to your allocator, but you have to respect the allocate, deallocate signature.
You can ignore the size of your deallocate function. I saw a lot of standard allocator implementations that did not use this parameter. This information could be usefull sometimes, for example if your allocator switch to different allocation strategies depending on size, this parameter could be helpfull in deallocate method to switch on the good deallocate implementation.
I am evaluating to use the basic_string template to implement a string like object to be allocated using an external memory manager.
That memory manager, keeps the size of the maximum size of the memory allocated and the current size (allowing for that current size to increase up to the maximum one).
In order to avoid redundance I would like to make use of that data for the string.
Does any body knows if it is possible and where to look for detailed instructions ?
I already know that it is possible to give a propierary allocator, but nothing more.
It is possible, for sure. Just provide a custom allocator which implements the std::allocator (http://www.cplusplus.com/reference/memory/allocator/) interface.
Then:
typedef std::basic_string<
char, std::char_traits<char>, custom_allocator<char> >
custom_string;
But be warned, this string is incompatible to std::string and you might have to implement the conversions custom_string <-> std::string.
It can look like that:
template<class T>
struct CustomAllocator: std::allocator<T> {
template<class U>
struct rebind {
typedef CustomAllocator<U> other;
};
};
typedef std::basic_string<char, std::char_traits<char>, CustomAllocator<char>> CustomString;
Now CustomString uses your CustomAllocator instead of std::allocator.
To change the way the memory is actually allocated, your can define custom allocate and deallocate(and probably some other)methods inside the CustomAllocator class(and add all the required logic for it).