Reallocate array with memcpy and memset - c++

I've taken over some code, and came across a weird reallocation of an array. This is a function from within an Array class (used by the JsonValue)
void reserve( uint32_t newCapacity ) {
if ( newCapacity > length + additionalCapacity ) {
newCapacity = std::min( newCapacity, length + std::numeric_limits<decltype( additionalCapacity )>::max() );
JsonValue *newPtr = new JsonValue[newCapacity];
if ( length > 0 ) {
memcpy( newPtr, values, length * sizeof( JsonValue ) );
memset( values, 0, length * sizeof( JsonValue ) );
}
delete[] values;
values = newPtr;
additionalCapacity = uint16_t( newCapacity - length );
}
}
I get the point of this; it is just allocating a new array, and doing a copy of the memory contents from the old array into the new array, then zero-ing out the old array's contents. I also know this was done in order to prevent calling destructors, and moves.
The JsonValue is a class with functions, and some data which is stored in a union (string, array, number, etc.).
My concern is whether this is actually defined behaviour or not. I know it works, and has not had a problem since we began using it a few months ago; but if its undefined then it doesn't mean it is going to keep working.
EDIT:
JsonValue looks something like this:
struct JsonValue {
// …
~JsonValue() {
switch ( details.type ) {
case Type::Array:
case Type::Object:
array.destroy();
break;
case Type::String:
delete[] string.buffer;
break;
default: break;
}
}
private:
struct Details {
Key key = Key::Unknown;
Type type = Type::Null; // (0)
};
union {
Array array;
String string;
EmbedString embedString;
Number number;
Details details;
};
};
Where Array is a wrapper around an array of JsonValues, String is a char*, EmbedString is char[14], Number is a union of int, unsigned int, and double, Details contains the type of value it holds. All values have 16-bits of unused data at the beginning, which is used for Details. Example:
struct EmbedString {
uint16_t : 16;
char buffer[14] = { 0 };
};

Whether this code has well-defined behavior basically depends on two things: 1) is JsonValue trivially-copyable and, 2) if so, are a bunch of all-zero Bytes a valid object representation for a JsonValue.
If JsonValue is trivially-copyable, then the memcpy from one array of JsonValues to another will indeed be equivalent to copying all the elements over [basic.types]/3. If all-zeroes is a valid object representation for a JsonValue, then the memset should be ok (I believe this actually falls into a bit of a grey-area with the current wording of the standard, but I believe at least the intention would be that this is fine).
I'm not sure why you'd need to "prevent calling destructors and moves", but overwriting objects with zeroes does not prevent destructors from running. delete[] values will call the destructurs of the array members. And moving the elements of an array of trivially-copyable type should compile down to just copying over the bytes anyways.
Furthermore, I would suggest to get rid of these String and EmbedString classes and simply use std::string. At least, it would seem to me that the sole purpose of EmbedString is to manually perform small string optimization. Any std::string implementation worth its salt is already going to do exactly that under the hood. Note that std::string is not guaranteed (and will often not be) trivially-copyable. Thus, you cannot simply replace String and EmbedString with std::string while keeping the rest of this current implementation.
If you can use C++17, I would suggest to simply use std::variant instead of or at least inside this custom JsonValue implementation as that seems to be exactly what it's trying to do. If you need some common information stored in front of whatever the variant value may be, just have a suitable member holding that information in front of the member that holds the variant value rather than relying on every member of the union starting with the same couple of members (which would only be well-defined if all union members are standard-layout types that keep this information in their common initial sequence [class.mem]/23).
The sole purpose of Array would seem to be to serve as a vector that zeroes memory before deallocating it for security reasons. If this is the case, I would suggest to just use an std::vector with an allocator that zeros memory before deallocating instead. For example:
template <typename T>
struct ZeroingAllocator
{
using value_type = T;
T* allocate(std::size_t N)
{
return reinterpret_cast<T*>(new unsigned char[N * sizeof(T)]);
}
void deallocate(T* buffer, std::size_t N) noexcept
{
auto ptr = reinterpret_cast<volatile unsigned char*>(buffer);
std::fill(ptr, ptr + N, 0);
delete[] reinterpret_cast<unsigned char*>(buffer);
}
};
template <typename A, typename B>
bool operator ==(const ZeroingAllocator<A>&, const ZeroingAllocator<B>&) noexcept { return true; }
template <typename A, typename B>
bool operator !=(const ZeroingAllocator<A>&, const ZeroingAllocator<B>&) noexcept { return false; }
and then
using Array = std::vector<JsonValue, ZeroingAllocator<JsonValue>>;
Note: I fill the memory via volatile unsigned char* to prevent the compiler from optimizing away the zeroing. If you need to support overaligned types, you can replace the new[] and delete[] with direct calls to ::operator new and ::operator delete (doing this will prevent the compiler from optimizing away allocations). Pre C++17, you will have to allocate a sufficiently large buffer and then manually align the pointer, e.g., using std::align…

Related

Is it undefined behaviour to read a different member than was written in a Union?

union test{
char a; // 1 byte
int b; // 4 bytes
};
int main(){
test t;
t.a = 5;
return t.b;
}
This link says: https://en.cppreference.com/w/cpp/language/union
It's undefined behavior to read from the member of the union that wasn't most recently written.
According to this, does my sample code above have UB?
If so then what's the point of a Union then? I thought the whole point it to read/write
different value types form the same memory location.
If I need to access the most recently written value then I will just use a
regular variable and not a Union.
Yes the behaviour is undefined in C++.
When you write a value to a member of union, think of that member becoming the active member.
The behaviour of reading any member of a union that is not the active member is undefined.
in C++, a union is often coupled with another variable that serves as a means of identifying the active member.
Your implication that having unions without the possibility of reading their inactive members makes them useless is wrong. Consider the following simplified implementation of a string class:
class string {
char* data_;
size_t size_;
union {
size_t capacity_;
char buffer_[16];
};
string(const char* str) : size_(strlen(str)) {
if (size_ < 16)
data_ = buffer_; // short string, buffer_ will be active
else {
capacity_ = size_; // long string, capacity_ is active
data_ = new char[capacity_ + 1];
}
memcpy(data_, str, size_ + 1);
}
bool is_short() const { return data_ == buffer_; }
...
public:
size_t capacity() const { return is_short() ? 15 : capacity_; }
const char* data() const { return data_; }
...
};
Here, if the stored string has less then 16 characters, it is stored in buffer_ and data_ points to it. Otherwise, data_ points to a dynamically-allocated buffer.
Consequently, you can distinguish between both cases (short/long string) by comparing data_ with buffer_. When the string is short, buffer_ is active and you don't need to read capacity_, since you know it is 15. When the string is long, capacity_ is active and you don't need to read buffer_, since it is unused.
Exactly this approach is used in libstdc++. It is a bit more complicated there since std::string is just a specialization of std::basic_string class template, but the idea is the same. Source code from include/bits/basic_string.h:
enum { _S_local_capacity = 15 / sizeof(_CharT) };
union
{
_CharT _M_local_buf[_S_local_capacity + 1];
size_type _M_allocated_capacity;
};
It can save a lot of space if your program works with a lot of strings at once (consider, e.g., databases). Without union, each string objects would take 8 more bytes in memory.

Portable emulation of flexible array member in C++?

I'm writing a skip list.
What I have:
template<typename T>
struct SkipListNode
{
T data;
SkipListNode* next[32];
};
The problem with this code is that it wastes space - it requires all nodes to contain 32 pointers. Especially considering that in typical list, half of the nodes will only need one pointer.
The C language has a neat feature called flexible array member that could solve that problem. Had it existed in C++ (even for trivial classes), I could write code like this:
template<typename T>
struct SkipListNode
{
alignas(T) char buffer[sizeof(T)];
SkipListNode* next[];
};
and then manually create nodes with a factory function and destroying them when deleting elements.
Which brings the question - how can I emulate such functionality portably, without undefined behaviour in C++?
I considered mallocing the buffer and then manipulating the offsets appropriately by hand - but it's too easy to violate the alignment requirements - if you malloc(sizeof(char) + sizeof(void*)*5), the pointers are unaligned. Also, I'm not even sure if such hand-created buffers are portable to C++.
Note that I don't require the exact syntax, or even ease of use - this is a node class, internal to the skip list class, which won't be a part of the interface at all.
This is the implementation I wrote, based on R. Martinho Fernandes's idea - it constructs a buffer that happens to have a correct size and alignment in specific places (the AlignmentExtractor is used extract the offset of the pointer array, which ensures that the pointers in the buffer have correct alignment). Then, placement-new is used to construct the type in the buffer.
T isn't used directly in AlignmentExtractor because offsetof requires standard layout type.
#include <cstdlib>
#include <cstddef>
#include <utility>
template<typename T>
struct ErasedNodePointer
{
void* ptr;
};
void* allocate(std::size_t size)
{
return ::operator new(size);
}
void deallocate(void* ptr)
{
return ::operator delete(ptr);
}
template<typename T>
struct AlignmentExtractor
{
static_assert(alignof(T) <= alignof(std::max_align_t), "extended alignment types not supported");
alignas(T) char data[sizeof(T)];
ErasedNodePointer<T> next[1];
};
template<typename T>
T& get_data(ErasedNodePointer<T> node)
{
return *reinterpret_cast<T*>(node.ptr);
}
template<typename T>
void destroy_node(ErasedNodePointer<T> node)
{
get_data(node).~T();
deallocate(node.ptr);
}
template<typename T>
ErasedNodePointer<T>& get_pointer(ErasedNodePointer<T> node, int pos)
{
auto next = reinterpret_cast<ErasedNodePointer<T>*>(reinterpret_cast<char*>(node.ptr) + offsetof(AlignmentExtractor<T>, next));
next += pos;
return *next;
}
template<typename T, typename... Args>
ErasedNodePointer<T> create_node(std::size_t height, Args&& ...args)
{
ErasedNodePointer<T> p = { nullptr };
try
{
p.ptr = allocate(sizeof(AlignmentExtractor<T>) + sizeof(ErasedNodePointer<T>)*(height-1));
::new (p.ptr) T(std::forward<T>(args)...);
for(std::size_t i = 0; i < height; ++i)
get_pointer(p, i).ptr = nullptr;
return p;
}
catch(...)
{
deallocate(p.ptr);
throw;
}
}
#include <iostream>
#include <string>
int main()
{
auto p = create_node<std::string>(5, "Hello world");
auto q = create_node<std::string>(2, "A");
auto r = create_node<std::string>(2, "B");
auto s = create_node<std::string>(1, "C");
get_pointer(p, 0) = q;
get_pointer(p, 1) = r;
get_pointer(r, 0) = s;
std::cout << get_data(p) << "\n";
std::cout << get_data(get_pointer(p, 0)) << "\n";
std::cout << get_data(get_pointer(p, 1)) << "\n";
std::cout << get_data(get_pointer(get_pointer(p, 1), 0)) << "\n";
destroy_node(s);
destroy_node(r);
destroy_node(q);
destroy_node(p);
}
Output:
Hello world
A
B
C
Longer explanation:
The point of this code is to create a node dynamically, without using types directly (type erasure). This node stores an object, and N pointers, with N variable at runtime.
You can use any memory as if it had a specific type, provided that:
size is correct
alignment is correct
(only non-triviably constructible types) you manually call the constructor before using
(only non-triviably destructible types) you manually call the destructor after using
In fact, you rely on this every time you call malloc:
// 1. Allocating a block
int* p = (int*)malloc(5 * sizeof *p);
p[2] = 42;
free(p);
Here, we treat the chunk of memory returned by malloc as if it was an array of ints. This must work because of these guarantees:
malloc returns a pointer guaranteed to be properly aligned for any object type.
If your pointer p points to aligned memory, (int*)((char*)p + sizeof(int)) (or p + 1, which is equivalent) also does.
The dynamically created node must have enough size to contain N ErasedNodePointers (which are used as handles here) and one object of size T. This is satisfied by allocating enough memory in create_node function - it will allocate sizeof(T) + sizeof(ErasedNodePointer<T>)*N bytes or more, but not less.
That was the first step. The second is now we extract the required position relative to the beginning of a block. That's where AlignmentExtractor<T> comes in.
AlignmentExtractor<T> is a dummy struct I use to ensure correct alignment:
// 2. Finding position
AlignmentExtractor<T>* p = (AlignmentExtractor<T>*)malloc(sizeof *p);
p->next[0].ptr = nullptr;
// or
void* q = (char*)p + offsetof(AlignmentExtractor<T>, next);
(ErasedTypePointer<T>*)q->ptr = nullptr;
It doesn't matter how I got the position of the pointer, as long as I obey the rules of pointer arithmetic.
The assumptions here are:
I can cast any pointer to void* and back.
I can cast any pointer to char* and back.
I can operate on a struct as if it was a char array of size equal to the size of the struct.
I can use pointer arithmetic to point at any element of an array.
These all are guaranteed by C++ standard.
Now, after I have allocated the block of enough size, I calculate the offset with offsetof(AlignmentExtractor<T>, next) and add it to the pointer pointing to the block. We "pretend" (the same way the code "1. Allocating a block" pretends it has an array of ints) the result pointer points to beginning of the array. This pointer is aligned correctly, because otherwise the code "2. Finding position" couldn't access the next array due to misaligned access.
If you have a struct of standard layout type, the pointer to the struct has the same address as the first member of the struct. AlignmentExtractor<T> is standard layout.
That's not all though - requirements 1. and 2. are satisfied, but we need to satisfy requirements 3. and 4. - the data in the node doesn't have to be trivially constructible or destructible. That's why we use placement-new to construct the data - the create_node uses variadic templates and perfect forwarding to forward arguments to the constructor. And the data is destroyed in the destroy_node function by calling the destructor.

Is there *any* way to get the length of a C-style array in C++/G++?

I've been trying to implement a lengthof (T* v) function for quite a while, so far without any success.
There are the two basic, well-known solutions for T v[n] arrays, both of which are useless or even dangerous once the array has been decayed into a T* v pointer.
#define SIZE(v) (sizeof(v) / sizeof(v[0]))
template <class T, size_t n>
size_t lengthof (T (&) [n])
{
return n;
}
There are workarounds involving wrapper classes and containers like STLSoft's array_proxy, boost::array, std::vector, etc. All of them have drawbacks, and lack the simplicity, syntactic sugar and widespread usage of arrays.
There are myths about solutions involving compiler-specific calls that are normally used by the compiler when delete [] needs to know the length of the array. According to the C++ FAQ Lite 16.14, there are two techniques used by compilers to know how much memory to deallocate: over-allocation and associative arrays. At over-allocation it allocates one wordsize more, and puts the length of the array before the first object. The other method obviously stores the lengths in an associative array. Is it possible to know which method G++ uses, and to extract the appropriate array length? What about overheads and paddings? Any hope for non-compiler-specific code? Or even non-platform-specific G++ builtins?
There are also solutions involving overloading operator new [] and operator delete [], which I implemented:
std::map<void*, size_t> arrayLengthMap;
inline void* operator new [] (size_t n)
throw (std::bad_alloc)
{
void* ptr = GC_malloc(n);
arrayLengthMap[ptr] = n;
return ptr;
}
inline void operator delete [] (void* ptr)
throw ()
{
arrayLengthMap.erase(ptr);
GC_free(ptr);
}
template <class T>
inline size_t lengthof (T* ptr)
{
std::map<void*, size_t>::const_iterator it = arrayLengthMap.find(ptr);
if( it == arrayLengthMap.end() ){
throw std::bad_alloc();
}
return it->second / sizeof(T);
}
It was working nicely until I got a strange error: lengthof couldn't find an array. As it turned out, G++ allocated 8 more bytes at the start of this specific array than it should have. Though operator new [] should have returned the start of the entire array, call it ptr, the calling code got ptr+8 instead, so lengthof(ptr+8) obviously failed with the exception (even if it did not, it could have potentially returned a wrong array size). Are those 8 bytes some kind of overhead or padding? Can not be the previously mentioned over-allocation, the function worked correctly for many arrays. What is it and how to disable or work around it, assuming it is possible to use G++ specific calls or trickery?
Edit:
Due to the numerous ways it is possible to allocate C-style arrays, it is not generally possible to tell the length of an arbitrary array by its pointer, just as Oli Charlesworth suggested. But it is possible for non-decayed static arrays (see the template function above), and arrays allocated with a custom operator new [] (size_t, size_t), based on an idea by Ben Voigt:
#include <gc/gc.h>
#include <gc/gc_cpp.h>
#include <iostream>
#include <map>
typedef std::map<void*, std::pair<size_t, size_t> > ArrayLengthMap;
ArrayLengthMap arrayLengthMap;
inline void* operator new [] (size_t size, size_t count)
throw (std::bad_alloc)
{
void* ptr = GC_malloc(size);
arrayLengthMap[ptr] = std::pair<size_t, size_t>(size, count);
return ptr;
}
inline void operator delete [] (void* ptr)
throw ()
{
ArrayLengthMap::const_iterator it = arrayLengthMap.upper_bound(ptr);
it--;
if( it->first <= ptr and ptr < it->first + it->second.first ){
arrayLengthMap.erase(it->first);
}
GC_free(ptr);
}
inline size_t lengthof (void* ptr)
{
ArrayLengthMap::const_iterator it = arrayLengthMap.upper_bound(ptr);
it--;
if( it->first <= ptr and ptr < it->first + it->second.first ){
return it->second.second;
}
throw std::bad_alloc();
}
int main (int argc, char* argv[])
{
int* v = new (112) int[112];
std::cout << lengthof(v) << std::endl;
}
Unfortunately due to arbitrary overheads and paddings by the compiler, there is no reliable way so far to determine the length of a dynamic array in a custom operator new [] (size_t), unless we assume that the padding is smaller than the size of one of the elements of the array.
However there are other kinds of arrays as well for which length calculation might be possible, as Ben Voigt suggested, thus it should be possible and desirable to construct a wrapper class that can accept several kinds of arrays (and their lengths) in its constructors, and is implicitly or explicitly convertible to other wrapper classes and array types. Different lifetimes of different kinds of arrays might be a problem, but it could be solved with garbage collection.
To answer this:
Any hope for non-compiler-specific code?
No.
More generally, if you find yourself needing to do this, then you probably need to reconsider your design. Use a std::vector, for instance.
Your analysis is mostly correct, however I think you've ignored the fact that types with trivial destructors don't need to store the length, and so overallocation can be different for different types.
The standard allows operator new[] to steal a few bytes for its own use, so you'll have to do a range check on the pointer instead of an exact match. std::map probably won't be efficient for this, but a sorted vector should be (can be binary searched). A balanced tree should also work really well.
Some time ago, I used a similar thing to monitor memory leaks:
When asked to allocate size bytes of data, I would alloc size + 4 bytes and store the length of the allocation in the first 4 bytes:
static unsigned int total_still_alloced = 0;
void *sys_malloc(UINT size)
{
#if ENABLED( MEMLEAK_CHECK )
void *result = malloc(size+sizeof(UINT ));
if(result)
{
memset(result,0,size+sizeof(UINT ));
*(UINT *)result = size;
total_still_alloced += size;
return (void*)((UINT*)result+sizeof(UINT));
}
else
{
return result;
}
#else
void *result = malloc(size);
if(result) memset(result,0,size);
return result;
#endif
}
void sys_free(void *p)
{
if(p != NULL)
{
#if ENABLED( MEMLEAK_CHECK )
UINT * real_address = (UINT *)(p)-sizeof(UINT);
total_still_alloced-= *((UINT *)real_address);
free((void*)real_address);
#else
free(p);
#endif
}
}
In your case, retrieving the allocated size is a matter of shifting the provided address by 4 and read the value.
Note that if you have memory corruption somewhere... you'll get invalid results.
Note also that it is often how malloc works internally: putting the size of the allocation on a hidden field before the adress returned. On some architectures, I don't even have to allocate more, using the system malloc is sufficient.
That's an invasive way of doing it... but it works (provided you allocate everything with these modified allocation routines, AND that you know the starting address of your array).

How to initialize an array that is part of a struct typedef?

If I have a typedef of a struct
typedef struct
{
char SmType;
char SRes;
float SParm;
float EParm;
WORD Count;
char Flags;
char unused;
GPOINT2 Nodes[];
} GPATH2;
and it contains an uninitialized array, how can I create an instance of this type so that is will hold, say, 4 values in Nodes[]?
Edit: This belongs to an API for a program written in Assembler. I guess as long as the underlying data in memory is the same, an answer changing the struct definition would work, but not if the underlying memory is different. The Assembly Language application is not using this definition .... but .... a C program using it can create GPATH2 elements that the Assembly Language application can "read".
Can I ever resize Nodes[] once I have created an instance of GPATH2?
Note: I would have placed this with a straight C tag, but there is only a C++ tag.
You could use a bastard mix of C and C++ if you really want to:
#include <new>
#include <cstdlib>
#include "definition_of_GPATH2.h"
using namespace std;
int main(void)
{
int i;
/* Allocate raw memory buffer */
void * raw_buffer = calloc(1, sizeof(GPATH2) + 4 * sizeof(GPOINT2));
/* Initialize struct with placement-new */
GPATH2 * path = new (raw_buffer) GPATH2;
path->Count = 4;
for ( i = 0 ; i < 4 ; i++ )
{
path->Nodes[i].x = rand();
path->Nodes[i].y = rand();
}
/* Resize raw buffer */
raw_buffer = realloc(raw_buffer, sizeof(GPATH2) + 8 * sizeof(GPOINT2));
/* 'path' still points to the old buffer that might have been free'd
* by realloc, so it has to be re-initialized
* realloc copies old memory contents, so I am not certain this would
* work with a proper object that actaully does something in the
* constructor
*/
path = new (raw_buffer) GPATH2;
/* now we can write more elements of array */
path->Count = 5;
path->Nodes[4].x = rand();
path->Nodes[4].y = rand();
/* Because this is allocated with malloc/realloc, free it with free
* rather than delete.
* If 'path' was a proper object rather than a struct, you should
* call the destructor manually first.
*/
free(raw_buffer);
return 0;
}
Granted, it's not idiomatic C++ as others have observed, but if the struct is part of legacy code it might be the most straightforward option.
Correctness of the above sample program has only been checked with valgrind using dummy definitions of the structs, your mileage may vary.
If it is fixed size write:
typedef struct
{
char SmType;
char SRes;
float SParm;
float EParm;
WORD Count;
char Flags;
char unused;
GPOINT2 Nodes[4];
} GPATH2;
if not fixed then change declaration to
GPOINT2* Nodes;
after creation or in constructor do
Nodes = new GPOINT2[size];
if you want to resize it you should use vector<GPOINT2>, because you can't resize array, only create new one. If you decide to do it, don't forget to delete previous one.
also typedef is not needed in c++, you can write
struct GPATH2
{
char SmType;
char SRes;
float SParm;
float EParm;
WORD Count;
char Flags;
char unused;
GPOINT2 Nodes[4];
};
This appears to be a C99 idiom known as the "struct hack". You cannot (in standard C99; some compilers have an extension that allows it) declare a variable with this type, but you can declare pointers to it. You have to allocate objects of this type with malloc, providing extra space for the appropriate number of array elements. If nothing holds a pointer to an array element, you can resize the array with realloc.
Code that needs to be backward compatible with C89 needs to use
GPOINT2 Nodes[1];
as the last member, and take note of this when allocating.
This is very much not idiomatic C++ -- note for instance that you would have to jump through several extra hoops to make new and delete usable -- although I have seen it done. Idiomatic C++ would use vector<GPOINT2> as the last member of the struct.
Arrays of unknown size are not valid as C++ data members. They are valid in C99, and your compiler may be mixing C99 support with C++.
What you can do in C++ is 1) give it a size, 2) use a vector or another container, or 3) ditch both automatic (local variable) and normal dynamic storage in order to control allocation explicitly. The third is particularly cumbersome in C++, especially with non-POD, but possible; example:
struct A {
int const size;
char data[1];
~A() {
// if data was of non-POD type, we'd destruct data[1] to data[size-1] here
}
static auto_ptr<A> create(int size) {
// because new is used, auto_ptr's use of delete is fine
// consider another smart pointer type that allows specifying a deleter
A *p = ::operator new(sizeof(A) + (size - 1) * sizeof(char));
try { // not necessary in our case, but is if A's ctor can throw
new(p) A(size);
}
catch (...) {
::operator delete(p);
throw;
}
return auto_ptr<A>(p);
}
private:
A(int size) : size (size) {
// if data was of non-POD type, we'd construct here, being very careful
// of exception safety
}
A(A const &other); // be careful if you define these,
A& operator=(A const &other); // but it likely makes sense to forbid them
void* operator new(size_t size); // doesn't prevent all erroneous uses,
void* operator new[](size_t size); // but this is a start
};
Note you cannot trust sizeof(A) any where else in the code, and using an array of size 1 guarantees alignment (matters when the type isn't char).
This type of structure is not trivially useable on the stack, you'll have to malloc it. the significant thing to know is that sizeof(GPATH2) doesn't include the trailing array. so to create one, you'd do something like this:
GPATH2 *somePath;
size_t numPoints;
numPoints = 4;
somePath = malloc(sizeof(GPATH2) + numPoints*sizeof(GPOINT2));
I'm guessing GPATH2.Count is the number of elements in the Nodes array, so if it's up to you to initialize that, be sure and set somePath->Count = numPoints; at some point. If I'm mistaken, and the convention used is to null terminate the array, then you would do things just a little different:
somePath = malloc(sizeof(GPATH2) + (numPoints+1)*sizeof(GPOINT2));
somePath->Nodes[numPoints] = Some_Sentinel_Value;
make darn sure you know which convention the library uses.
As other folks have mentioned, realloc() can be used to resize the struct, but it will invalidate old pointers to the struct, so make sure you aren't keeping extra copies of it (like passing it to the library).

placement new + array +alignment

SomeObj<unsigned int>* Buffer;
char* BufferPtr = MemoryManager::giveMeSomeBytes(resX*resY*sizeof(SomeObj<unsigned int>));
Buffer = new(BufferPtr) SomeObj<unsigned int>[resX*resY];
when I step past these lines with the debugger, it shows me the values for the variables Buffer and BufferPtr:
BufferPtr: 0x0d7f004c
Buffer: 0x0d7f0050
I don't really understand why those values differ. The way I understand it, placement new should use the memory starting at address 'BufferPtr' to initialize the array elements using theyr default constructors on the allocated memory and return a pointer to the first byte of the first element in the array, which should be exactly the same byte as passed to the placement new operator.
Did I understand something wrong or can someone tell me why the values differ?
thanks!
//edit: ok - i investigated the issue further and got more confusing results:
int size = sizeof(matth_ptr<int>);
char* testPtr1 = (char*)malloc(a_resX*a_resY*sizeof(int));
int* test1 = new(testPtr1) int[a_resX*a_resY];
char* testPtr2 = mmgr::requestMemory(a_resX*a_resY*sizeof(int));
int* test2 = new(testPtr2) int[a_resX*a_resY];
char* testPtr3 = (char*)malloc(a_resX*a_resY*sizeof(matth_ptr<int>));
matth_ptr<int>* test3 = new(testPtr3)matth_ptr<int>[a_resX*a_resY];
char* testPtr4 = mmgr::requestMemory(a_resX*a_resY*sizeof(matth_ptr<int>));
matth_ptr<int>* test4 = new(testPtr4)matth_ptr<int>[a_resX*a_resY];
the debugger returns me the following values for my variables:
size: 4
testPtr1:0x05100418
test1: 0x05100418
testPtr2:0x0da80050
test2: 0x0da80050
testPtr3:0x05101458
test3: 0x0510145c
testPtr4:0x0da81050
test4: 0x0da81054
so it clearly must have something to do with my generic smartpointer class matth_ptr so here it is:
template <class X> class matth_ptr
{
public:
typedef X element_type;
matth_ptr(){
memoryOfst = 0xFFFFFFFF;
}
matth_ptr(X* p)
{
unsigned char idx = mmgr::getCurrentChunkIdx();
memoryOfst = (int)p-(int)mmgr::getBaseAddress(idx);
assert(memoryOfst<=0x00FFFFFF || p==0);//NULL pointer is not yet handled
chunkIdx = idx;
}
~matth_ptr() {}
X& operator*() {return *((X*)(mmgr::getBaseAddress(chunkIdx)+(memoryOfst&0x00FFFFFF)));}
X* operator->() {return ((X*)(mmgr::getBaseAddress(chunkIdx)+(memoryOfst&0x00FFFFFF)));}
X* get() {return ((X*)(mmgr::getBaseAddress(chunkIdx)+(memoryOfst&0x00FFFFFF)));}
template<typename T>
matth_ptr(const matth_ptr<T>& other) {memoryOfst=other.memoryOfst;}//put these two operators into the private part in order to prevent copying of the smartpointers
template<typename T>
matth_ptr& operator=(const matth_ptr<T>& other) {memoryOfst = other.memoryOfst; return *this;}
template<typename T>
friend class matth_ptr;
private:
union //4GB adressable in chunks of 16 MB
{
struct{
unsigned char padding[3]; //3 bytes padding
unsigned char chunkIdx; //8 bit chunk index
};
unsigned int memoryOfst; //24bit address ofst
};
};
can anyone explain me what's going on? thanks!
Be careful with placement new on arrays. In the current standard look to section 5.3.4.12, you'll find this:
new(2,f) T[5] results in a call of operator new[](sizeof(T)*5+y,2,f)
It is clear that it will expect the placement new operator to allocate it additional space beyond what the array contents need. "y" is specified only as a non-negative integral value. It will then offset the result of the new function by this amount.
Also look to 18.4.1.3.4 where it says the placement new operator simply returns the provided pointer. This is obviously the expected part.
Based on 5.3.4.12, since that offset may be different for every invocation of the array, the standard basically means there is no way to allocate the exact amount of size needed. In practice that value is probably constant and you could just add it to the allocation, but his amount may change per platform, and again, per invocation as the standard says.
You're using the array version of the new operator which in your implementation is storing information about the array size in the first few bytes of the memory allocation.
#Mat, This is actually a great question. When I've used placement new[], I've had trouble deleting the storage. Even if I call my own symmetrical placement delete[], the pointer address is not the same as was returned by my own placement new[]. This makes placement new[] completely useless, as you've suggested in the comments.
The only solution I've found was suggested by Jonathan#: Instead of placement new[], use placement new (non-array) on each of the elements of the array. This is fine for me as I store the size myself. The problem is that I have to worry about pointer alignments for elements, which new[] is supposed to do for me.
As others have said, this is due to your C++ implementation storing the size of the array at the start of the buffer you pass to array placement new.
An easy fix for this is to simply assign your array pointer to the buffer, then loop over the array and use regular (non-array) placement new to construct each object in the buffer.