SomeObj<unsigned int>* Buffer;
char* BufferPtr = MemoryManager::giveMeSomeBytes(resX*resY*sizeof(SomeObj<unsigned int>));
Buffer = new(BufferPtr) SomeObj<unsigned int>[resX*resY];
when I step past these lines with the debugger, it shows me the values for the variables Buffer and BufferPtr:
BufferPtr: 0x0d7f004c
Buffer: 0x0d7f0050
I don't really understand why those values differ. The way I understand it, placement new should use the memory starting at address 'BufferPtr' to initialize the array elements using theyr default constructors on the allocated memory and return a pointer to the first byte of the first element in the array, which should be exactly the same byte as passed to the placement new operator.
Did I understand something wrong or can someone tell me why the values differ?
thanks!
//edit: ok - i investigated the issue further and got more confusing results:
int size = sizeof(matth_ptr<int>);
char* testPtr1 = (char*)malloc(a_resX*a_resY*sizeof(int));
int* test1 = new(testPtr1) int[a_resX*a_resY];
char* testPtr2 = mmgr::requestMemory(a_resX*a_resY*sizeof(int));
int* test2 = new(testPtr2) int[a_resX*a_resY];
char* testPtr3 = (char*)malloc(a_resX*a_resY*sizeof(matth_ptr<int>));
matth_ptr<int>* test3 = new(testPtr3)matth_ptr<int>[a_resX*a_resY];
char* testPtr4 = mmgr::requestMemory(a_resX*a_resY*sizeof(matth_ptr<int>));
matth_ptr<int>* test4 = new(testPtr4)matth_ptr<int>[a_resX*a_resY];
the debugger returns me the following values for my variables:
size: 4
testPtr1:0x05100418
test1: 0x05100418
testPtr2:0x0da80050
test2: 0x0da80050
testPtr3:0x05101458
test3: 0x0510145c
testPtr4:0x0da81050
test4: 0x0da81054
so it clearly must have something to do with my generic smartpointer class matth_ptr so here it is:
template <class X> class matth_ptr
{
public:
typedef X element_type;
matth_ptr(){
memoryOfst = 0xFFFFFFFF;
}
matth_ptr(X* p)
{
unsigned char idx = mmgr::getCurrentChunkIdx();
memoryOfst = (int)p-(int)mmgr::getBaseAddress(idx);
assert(memoryOfst<=0x00FFFFFF || p==0);//NULL pointer is not yet handled
chunkIdx = idx;
}
~matth_ptr() {}
X& operator*() {return *((X*)(mmgr::getBaseAddress(chunkIdx)+(memoryOfst&0x00FFFFFF)));}
X* operator->() {return ((X*)(mmgr::getBaseAddress(chunkIdx)+(memoryOfst&0x00FFFFFF)));}
X* get() {return ((X*)(mmgr::getBaseAddress(chunkIdx)+(memoryOfst&0x00FFFFFF)));}
template<typename T>
matth_ptr(const matth_ptr<T>& other) {memoryOfst=other.memoryOfst;}//put these two operators into the private part in order to prevent copying of the smartpointers
template<typename T>
matth_ptr& operator=(const matth_ptr<T>& other) {memoryOfst = other.memoryOfst; return *this;}
template<typename T>
friend class matth_ptr;
private:
union //4GB adressable in chunks of 16 MB
{
struct{
unsigned char padding[3]; //3 bytes padding
unsigned char chunkIdx; //8 bit chunk index
};
unsigned int memoryOfst; //24bit address ofst
};
};
can anyone explain me what's going on? thanks!
Be careful with placement new on arrays. In the current standard look to section 5.3.4.12, you'll find this:
new(2,f) T[5] results in a call of operator new[](sizeof(T)*5+y,2,f)
It is clear that it will expect the placement new operator to allocate it additional space beyond what the array contents need. "y" is specified only as a non-negative integral value. It will then offset the result of the new function by this amount.
Also look to 18.4.1.3.4 where it says the placement new operator simply returns the provided pointer. This is obviously the expected part.
Based on 5.3.4.12, since that offset may be different for every invocation of the array, the standard basically means there is no way to allocate the exact amount of size needed. In practice that value is probably constant and you could just add it to the allocation, but his amount may change per platform, and again, per invocation as the standard says.
You're using the array version of the new operator which in your implementation is storing information about the array size in the first few bytes of the memory allocation.
#Mat, This is actually a great question. When I've used placement new[], I've had trouble deleting the storage. Even if I call my own symmetrical placement delete[], the pointer address is not the same as was returned by my own placement new[]. This makes placement new[] completely useless, as you've suggested in the comments.
The only solution I've found was suggested by Jonathan#: Instead of placement new[], use placement new (non-array) on each of the elements of the array. This is fine for me as I store the size myself. The problem is that I have to worry about pointer alignments for elements, which new[] is supposed to do for me.
As others have said, this is due to your C++ implementation storing the size of the array at the start of the buffer you pass to array placement new.
An easy fix for this is to simply assign your array pointer to the buffer, then loop over the array and use regular (non-array) placement new to construct each object in the buffer.
Related
I have a function which gets as a parameter a pointer to array,
e.g. int** segs.
I need to allocate (in the function-body) memory for the array, which has size 100 for example.
My attempt was:
*segs=(int*)(malloc(100));
So far so good.
I put a value into *segs[0], and still everything is great.
But... when I try to reach *segs[1], I get an "invalid write of size 4" error from valgrind, which leads to seg-fault.
I have no idea why does that happen.
I tried to reach to *segs[2], but then I get even something weirder-
Error of uninitialised value of size 8.
Due to operator precedence, *segs[N] is treated as *(segs[N]), which is ok when N is equal to 0. However, when you do the same thing using the index 1, things break since nothing has been allocated for segs[1]
For any index other than zero, you need to use (*segs)[N].
It will be easier to use a temporary pointer in the function.
int* ptr = (int*)(malloc(100));
*segs = ptr;
// and then
ptr[0] = ...; // Good
ptr[1] = ...; // Good
...
ptr[99] = ...; // Still good
Upgrading to C++ Way
Pass the pointer by reference.
void foo(int*& segs) { ... }
Use new instead of malloc to allocate memory.
segs = new int[100];
Better yet, use std::vector insteady raw arrays.
void foo(std::vector<int>& segs) { ... }
I really need to this specifically as I am using SWIG and need to make a cast to match the function definition.
The function definition accepts
SomeType const * const variable_name[]
Also, another question would be-
How to allocate memory to
SomeType * variable[] = <??? using new or malloc >
for x entries?
Edit:
I have searched quite a lot, but I keep stumbling into post which allocate memory to SomeType** using new SomeType*[x] i.e.
SomeType** variable = new SomeType*[x];
Can you please tell me a way to do this?
The function wants an array of pointers.
The statement:
SomeType * variable[];
Is not valid syntax.
You will need:
SomeType * * variable;
Declares a pointer to a pointer of SomeType.
You will need to perform memory allocation in two steps.
First, allocate the array of pointers:
variable = new SomeType * [/* some quantity */];
Remember, the above statement only allocates room for the pointers. The memory contents is still not initialized.
Secondly, allocate pointers to the objects.
for (unsigned int i = 0; i < some_quantity; ++i)
{
variable[i] = new SomeType;
}
When deleting, delete the contents of the array before the array:
for (unsigned int i = 0; i < some_quantity; ++i)
{
delete variable[i];
}
delete[] variable;
The function definition accepts
SomeType const * const variable_name[]
I'm no expert in C++, but if the declaration of arrays in function parameters is the same as in C then this is a synonym for the following type:
SomeType const * const * variable_name
That is, an array in a function parameter is really a pointer. Your book should have explained this early on.
I have searched quite a lot, but I keep stumbling into post which allocate memory to SomeType** using new SomeType*[x] i.e.
SomeType** variable = new SomeType*[x];
Can you please tell me a way to do this?
You could indeed allocate a SomeType const * const * using similar code to that, so you've answered your own question. I assume you could also use an std::vector<SomeType const * const> like so:
std::vector<SomeType const * const> *foo = new std::vector<SomeType const * const>();
/* XXX: Add some items to the vector */
SomeType const * const *bar = &foo[0];
This would be useful if you're not sure how many items foo should contain, and you expect it to grow.
I don't understand why people don't read books anymore. It's the fastest and cheapest (if you consider the cost of man hours) way to learn correctly from a reputable figure.
I have the following structure in C++ :
struct wrapper
{
// Param constructor
wrapper(unsigned int _id, const char* _string1, unsigned int _year,
unsigned int _value, unsigned int _usage, const char* _string2)
:
id(_id), year(_year), value(_value), usage(_usage)
{
int len = strlen(_string1);
string1 = new char[len + 1]();
strncpy(string1, _string1, len);
len = strlen(_string2);
string2 = new char[len + 1]();
strncpy(string2, _string2, len);
};
// Destructor
~wrapper()
{
if(string1 != NULL)
delete [] string1;
if(string2 != NULL)
delete [] string2;
}
// Elements
unsigned int id;
unsigned int year;
unsigned int value;
unsigned int usage;
char* string1;
char* string2;
};
In main.cpp let's say I allocate memory for one object of this structure :
wrapper* testObj = new wrapper(125600, "Hello", 2013, 300, 0, "bye bye");
Can I now delete the entire object using pointer arithmetic and a pointer that points to one of the structure elements ?
Something like this :
void* ptr = &(testObj->string2);
ptr -= 0x14;
delete (wrapper*)ptr;
I've tested myself and apparently it works but I'm not 100% sure that is equivalent to delete testObj.
Thanks.
Technically, the code like this would work (ignoring the fact that wrapper testObj should be wrapper* testObj and that the offset is not necessarily 0x14, e.g. debug builds sometimes pad the structures, and maybe some other detail I missed), but it is a horrible, horrible idea. I can't stress hard enough how horrible it is.
Instead of 0x14 you could use offsetof macro.
If you like spending nights in the company of the debugger, sure, feel free to do so.
I will assume that the reason for the question is sheer curiosity about whether it is possible to use pointer arithmetic to navigate from members to parent, and not that you would like to really do it in production code. Please tell me I am right.
Can I now delete the entire object using pointer arithmetic and a pointer that points to one of the structure elements ?
Theoretically, yes.
The pointer that you give to delete needs to have the correct value, and it doesn't really matter whether that value comes from an existing pointer variable, or by "adjusting" one in this manner.
You also need to consider the type of the pointer; if nothing else, you should cast to char* before performing your arithmetic so that you are moving in steps of single bytes. Your current code will not compile because ISO C++ forbids incrementing a pointer of type 'void*' (how big is a void?).
However, I recommend not doing this at all. Your magic number 0x14 is unreliable, given alignment and padding and the potential of your structure to change shape.
Instead, store a pointer to the actual object. Also stop with all the horrid memory mess, and use std::string. At present, your lack of copy constructor is presenting a nasty bug.
You can do this sort of thing with pointer arithmetic. Whether you should is an entirely different story. Consider this macro (I know... I know...) that will give you the base address of a structure given its type, the name of a structure member and a pointer to that member:
#define ADDRESS_FROM_MEMBER(T, member, ptr) reinterpret_cast<T*>( \
reinterpret_cast<unsigned char *>(ptr) - (ptrdiff_t)(&(reinterpret_cast<T*>(0))->member))
If I have a typedef of a struct
typedef struct
{
char SmType;
char SRes;
float SParm;
float EParm;
WORD Count;
char Flags;
char unused;
GPOINT2 Nodes[];
} GPATH2;
and it contains an uninitialized array, how can I create an instance of this type so that is will hold, say, 4 values in Nodes[]?
Edit: This belongs to an API for a program written in Assembler. I guess as long as the underlying data in memory is the same, an answer changing the struct definition would work, but not if the underlying memory is different. The Assembly Language application is not using this definition .... but .... a C program using it can create GPATH2 elements that the Assembly Language application can "read".
Can I ever resize Nodes[] once I have created an instance of GPATH2?
Note: I would have placed this with a straight C tag, but there is only a C++ tag.
You could use a bastard mix of C and C++ if you really want to:
#include <new>
#include <cstdlib>
#include "definition_of_GPATH2.h"
using namespace std;
int main(void)
{
int i;
/* Allocate raw memory buffer */
void * raw_buffer = calloc(1, sizeof(GPATH2) + 4 * sizeof(GPOINT2));
/* Initialize struct with placement-new */
GPATH2 * path = new (raw_buffer) GPATH2;
path->Count = 4;
for ( i = 0 ; i < 4 ; i++ )
{
path->Nodes[i].x = rand();
path->Nodes[i].y = rand();
}
/* Resize raw buffer */
raw_buffer = realloc(raw_buffer, sizeof(GPATH2) + 8 * sizeof(GPOINT2));
/* 'path' still points to the old buffer that might have been free'd
* by realloc, so it has to be re-initialized
* realloc copies old memory contents, so I am not certain this would
* work with a proper object that actaully does something in the
* constructor
*/
path = new (raw_buffer) GPATH2;
/* now we can write more elements of array */
path->Count = 5;
path->Nodes[4].x = rand();
path->Nodes[4].y = rand();
/* Because this is allocated with malloc/realloc, free it with free
* rather than delete.
* If 'path' was a proper object rather than a struct, you should
* call the destructor manually first.
*/
free(raw_buffer);
return 0;
}
Granted, it's not idiomatic C++ as others have observed, but if the struct is part of legacy code it might be the most straightforward option.
Correctness of the above sample program has only been checked with valgrind using dummy definitions of the structs, your mileage may vary.
If it is fixed size write:
typedef struct
{
char SmType;
char SRes;
float SParm;
float EParm;
WORD Count;
char Flags;
char unused;
GPOINT2 Nodes[4];
} GPATH2;
if not fixed then change declaration to
GPOINT2* Nodes;
after creation or in constructor do
Nodes = new GPOINT2[size];
if you want to resize it you should use vector<GPOINT2>, because you can't resize array, only create new one. If you decide to do it, don't forget to delete previous one.
also typedef is not needed in c++, you can write
struct GPATH2
{
char SmType;
char SRes;
float SParm;
float EParm;
WORD Count;
char Flags;
char unused;
GPOINT2 Nodes[4];
};
This appears to be a C99 idiom known as the "struct hack". You cannot (in standard C99; some compilers have an extension that allows it) declare a variable with this type, but you can declare pointers to it. You have to allocate objects of this type with malloc, providing extra space for the appropriate number of array elements. If nothing holds a pointer to an array element, you can resize the array with realloc.
Code that needs to be backward compatible with C89 needs to use
GPOINT2 Nodes[1];
as the last member, and take note of this when allocating.
This is very much not idiomatic C++ -- note for instance that you would have to jump through several extra hoops to make new and delete usable -- although I have seen it done. Idiomatic C++ would use vector<GPOINT2> as the last member of the struct.
Arrays of unknown size are not valid as C++ data members. They are valid in C99, and your compiler may be mixing C99 support with C++.
What you can do in C++ is 1) give it a size, 2) use a vector or another container, or 3) ditch both automatic (local variable) and normal dynamic storage in order to control allocation explicitly. The third is particularly cumbersome in C++, especially with non-POD, but possible; example:
struct A {
int const size;
char data[1];
~A() {
// if data was of non-POD type, we'd destruct data[1] to data[size-1] here
}
static auto_ptr<A> create(int size) {
// because new is used, auto_ptr's use of delete is fine
// consider another smart pointer type that allows specifying a deleter
A *p = ::operator new(sizeof(A) + (size - 1) * sizeof(char));
try { // not necessary in our case, but is if A's ctor can throw
new(p) A(size);
}
catch (...) {
::operator delete(p);
throw;
}
return auto_ptr<A>(p);
}
private:
A(int size) : size (size) {
// if data was of non-POD type, we'd construct here, being very careful
// of exception safety
}
A(A const &other); // be careful if you define these,
A& operator=(A const &other); // but it likely makes sense to forbid them
void* operator new(size_t size); // doesn't prevent all erroneous uses,
void* operator new[](size_t size); // but this is a start
};
Note you cannot trust sizeof(A) any where else in the code, and using an array of size 1 guarantees alignment (matters when the type isn't char).
This type of structure is not trivially useable on the stack, you'll have to malloc it. the significant thing to know is that sizeof(GPATH2) doesn't include the trailing array. so to create one, you'd do something like this:
GPATH2 *somePath;
size_t numPoints;
numPoints = 4;
somePath = malloc(sizeof(GPATH2) + numPoints*sizeof(GPOINT2));
I'm guessing GPATH2.Count is the number of elements in the Nodes array, so if it's up to you to initialize that, be sure and set somePath->Count = numPoints; at some point. If I'm mistaken, and the convention used is to null terminate the array, then you would do things just a little different:
somePath = malloc(sizeof(GPATH2) + (numPoints+1)*sizeof(GPOINT2));
somePath->Nodes[numPoints] = Some_Sentinel_Value;
make darn sure you know which convention the library uses.
As other folks have mentioned, realloc() can be used to resize the struct, but it will invalidate old pointers to the struct, so make sure you aren't keeping extra copies of it (like passing it to the library).
I am currently writing a small custom memory Allocator in C++, and want to use it together with operator overloading of new/ delete. Anyways, my memory Allocator basically checks if the requested memory is over a certain threshold, and if so uses malloc to allocate the requested memory chunk. Otherwise the memory will be provided by some fixedPool allocators. that generally works, but for my deallocation function looks like this:
void MemoryManager::deallocate(void * _ptr, size_t _size){
if(_size > heapThreshold)
deallocHeap(_ptr);
else
deallocFixedPool(_ptr, _size);
}
So I need to provide the size of the chunk pointed to, to deallocate from the right place.
Now the problem is that the delete keyword does not provide any hint on the size of the deleted chunk, so I would need something like this:
void operator delete(void * _ptr, size_t _size){
MemoryManager::deallocate(_ptr, _size);
}
But as far as I can see, there is no way to determine the size inside the delete operator.- If I want to keep things the way it is right now, would I have to save the size of the memory chunks myself?
allocate more memory than neccessary and store the size information there. That's what your system allocator probably does already. Something like this (demonstrate with malloc for simplicity):
void *allocate(size_t size) {
size_t *p = malloc(size + sizeof(size_t));
p[0] = size; // store the size in the first few bytes
return (void*)(&p[1]); // return the memory just after the size we stored
}
void deallocate(void *ptr) {
size_t *p = (size_t*)ptr; // make the pointer the right type
size_t size = p[-1]; // get the data we stored at the beginning of this block
// do what you need with size here...
void *p2 = (void*)(&p[-1]); // get a pointer to the memory we originally really allocated
free(p2); // free it
}
You could keep a map of memory address to size for your pool-allocated memory. When you delete, check if the pointer is in the map, if it is delete that size, if it isn't call regular delete.
For class type, C++ already supports it directly. For nonclass types, you need to store the size manually like the other solution shows.
struct MyClass {
void operator delete(void *p, size_t size) {
MemoryManager::deallocate(p, size);
}
};
As of C++14 the Standard supports the second size parameter in the global delete allocation function. So want you want to do is possible natively now.
http://en.cppreference.com/w/cpp/memory/new/operator_delete