cpp custom std allocator - c++

First timer with allocator.
I am really struggling with a custom allocator.
I would say its obvious I don't know what I'm doing here.
My goal is to put all of the memory for the object in the process's heap.
I want the allocation to come from the system so its not language / runtime dependent.
I don't really want a struct, I just want:
std::list< WCHAR[SIZE], TestAllocator<WCHAR[SIZE]>> list;
or, I'd allocate the memory outside of the list and then just store a pointer to it
std::list< void*, TestAllocator<void*>> list;
But I don't see how to do it with non struct / classes. So I have a struct.
This thing crashes all over the place.
The code below that instantiates the list crashes, it doesn't even have to get to a list item creation / insert.
The allocator is used for the list creation before I attempt to add an item. (which I don't understand.) It seems to me that the allocator should only be used for the struct type I'm trying to add to the list. If the allocator is used to create the list and the items it holds how do I create the memory for the 2 different types, the list type and the list entry type?
Some other observations / questions.
When my list is created, it is placed at the memory address created:
OSAllocator::allocate count:[1] bytes each:[8][0x8] bytes total:[8][0x8]addr:[4305640][0x41B2E8]
OSAllocator::construct placement new:[4305640][0x41B2E8] sizeof(val):[8][0x8]
However, when my entry is created, it is placed 2 bytes past the address created:
OSAllocator::allocate count:[1] bytes each:[8200][0x2008] bytes total:[8200][0x2008] addr:[8837144][0x86D818]
OSAllocator::construct placement new:[8837152][0x86D820] sizeof(val):[8192][0x2000]
Is there an explanation for that?
Please help point out what I'm missing.
Thanks.
// allocator...
pointer allocate(size_type size, TestAllocator<void>::const_pointer hint = 0)
{
pointer p = (pointer)::HeapAlloc( ::GetProcessHeap(),
HEAP_ZERO_MEMORY | HEAP_NO_SERIALIZE,
size*sizeof(T) );
gCountAllocate++;
if( OSAllocatorConsole )
wprintf( L"OSAllocator::allocate: [%u][0x%X]\n", p, p );
return p;
}
size_type max_size() const throw() {return size_t(-1) / sizeof(value_type);}
void construct(pointer p, const T& val)
{
gCountConstructPlaceNew++;
if( OSAllocatorConsole )
wprintf( L"OSAllocator::construct placement new:[%u][0x%X]\n", p, p );
::CopyMemory( p, &val, sizeof(val) );
}
typedef struct _LogLine
{
WCHAR _logLine[4*1024];
} LOGLINE, *PLOGLINE;
DWORD TestAllocatorChar(int argc, WCHAR* argv[])
{
WCHAR buf[32] = {0};
DWORD total = 1;
std::list< LOGLINE, TestAllocator<LOGLINE>> list;
PLOGLINE pll = NULL;
LOGLINE ll = {0};
for(int i=0; i<total; i++ )
{
::StringCchPrintfW( ll._logLine, 4*1024, L"count:[%u]", i );
list.push_back( ll );
}
for( int i=0; i<total; i++ )
{
//pll = list.front();
wprintf( L"%s\n", list.front()._logLine );
wprintf( L"HeapFree item:[%u]\n", i );
//::HeapFree( ::GetProcessHeap(), HEAP_NO_SERIALIZE, p );
list.pop_front();
}
return 0;
}

Related

How can i make this algorithm more memory efficient?

I have been working on this piece of code for a couple of days and it seems
to work just fine for search trees of relative small size, but working with higher ones gives segmentation fault.
I have tried to find the specific place where such error comes from by using printing flags, and it seems to break at the is_goal() function. But the code of that function can be assumed as correct.
So the problems seems to be a memory problem, that's why I have the queue in the heap, and also the Nodes that I am creating. But it still crashes.
Are there any more memory saving tricks that can be taking into account that I am not using? Or maybe there is a better way to save such nodes?
It is important to note that I need to use BFS (I can't use A* to make the search tree smaller).
Also if you need to know, the map is a hash table where I save the coloring of the nodes so I don't have duplicates when exploring (This is because the hash saves depending on the state information and not the Node information).
EDIT: i have been told to indicate the goal to accomplish, such is to find the goal state in the search tree, the goal is to be be able to iterate over
big problems like rubik 3x3x3, n-puzzle 4x4 5x5, tower of hanoi and such.To do so, the memory used has to be minimal since the search tree of such problems are really big, the final objective is to compare this algorithm with others like a*, dfid, ida*, and so on. I hope this is enough information.
Node* BFS(state_t start ){
state_t state, child;
int d, ruleid;
state_map_t *map = new_state_map();
ruleid_iterator_t iter;
std::queue<Node*> *open = new std::queue<Node*>();
state_map_add( map, &start, 1);
Node* n = new Node(start);
open->push(n);
while( !open->empty() ) {
n = open->front();
state = n->get_state();
open->pop();
if (is_goal(&state) == 1) return n;
init_fwd_iter( &iter, &state );
while( ( ruleid = next_ruleid( &iter ) ) >= 0 ) {
apply_fwd_rule( ruleid, &state, &child );
const int *old_child_c = state_map_get( map, &child );
if ( old_child_c == NULL ) {
state_map_add( map, &child, 1 );
Node* nchild = new Node(child);
open->push(nchild);
}
}
}
return NULL;
}
I see a number of memory leaks.
open is never deleted but maybe could allocated in the stack instead of in the heap.
std::queue<Node*> open;
More important none of the node you push in the queue are deleted this is probably the origin of very big memory consumption.
Delete the nodes that you remove from the queue and that you don't plan to reuse. Delete the nodes of the queue before your get rid of the queue.
Node* BFS(state_t start ){
state_t state, child;
int d, ruleid;
state_map_t *map = new_state_map();
ruleid_iterator_t iter;
std::queue<Node*> open;
state_map_add( map, &start, 1);
Node* n = new Node(start);
open.push(n);
while( !open.empty() ) {
n = open.front();
state = n->get_state();
open.pop();
if (is_goal(&state) == 1) {
for (std::queue<Node*>::iterator it = open.begin(); it != open.end(); ++it)
delete *it;
return n;
}
else {
delete n;
}
init_fwd_iter( &iter, &state );
while( ( ruleid = next_ruleid( &iter ) ) >= 0 ) {
apply_fwd_rule( ruleid, &state, &child );
const int *old_child_c = state_map_get( map, &child );
if ( old_child_c == NULL ) {
state_map_add( map, &child, 1 );
Node* nchild = new Node(child);
open.push(nchild);
}
}
}
return NULL;
}

Allocating an Array in Memory Manager

I want to successfully allocate an Array in my Memory Manager. I am having a hard time getting the data setup successfully in my Heap. I don't know how to instantiate the elements of the array, and then set the pointer that is passed in to that Array. Any help would be greatly appreciated. =)
Basically to sum it up, I want to write my own new[#] function using my own Heap block instead of the normal heap. Don't even want to think about what would be required for a dynamic array. o.O
// Parameter 1: Pointer that you want to pointer to the Array.
// Parameter 2: Amount of Array Elements requested.
// Return: true if Allocation was successful, false if it failed.
template <typename T>
bool AllocateArray(T*& data, unsigned int count)
{
if((m_Heap.m_Pool == nullptr) || count <= 0)
return false;
unsigned int allocSize = sizeof(T)*count;
// If we have an array, pad an extra 16 bytes so that it will start the data on a 16 byte boundary and have room to store
// the number of items allocated within this pad space, and the size of the original data type so in a delete call we can move
// the pointer by the appropriate size and call a destructor(potentially a base class destructor) on each element in the array
allocSize += 16;
unsigned int* mem = (unsigned int*)(m_Heap.Allocate(allocSize));
if(!mem)
{
return false;
}
mem[2] = count;
mem[3] = sizeof(T);
T* iter = (T*)(&(mem[4]));
data = iter;
iter++;
for(unsigned int i = 0; i < count; ++i,++iter)
{
// I have tried a bunch of stuff, not sure what to do. :(
}
return true;
}
Heap Allocate function:
void* Heap::Allocate(unsigned int allocSize)
{
Header* HeadPtr = FindBlock(allocSize);
Footer* FootPtr = (Footer*)HeadPtr;
FootPtr = (Footer*)((char*)FootPtr + (HeadPtr->size + sizeof(Header)));
// Right Split Free Memory if there is enough to make another block.
if((HeadPtr->size - allocSize) >= MINBLOCKSIZE)
{
// Create the Header for the Allocated Block and Update it's Footer
Header* NewHead = (Header*)FootPtr;
NewHead = (Header*)((char*)NewHead - (allocSize + sizeof(Header)));
NewHead->size = allocSize;
NewHead->next = NewHead;
NewHead->prev = NewHead;
FootPtr->size = NewHead->size;
// Create the Footer for the remaining Free Block and update it's size
Footer* NewFoot = (Footer*)NewHead;
NewFoot = (Footer*)((char*)NewFoot - sizeof(Footer));
HeadPtr->size -= (allocSize + HEADANDFOOTSIZE);
NewFoot->size = HeadPtr->size;
// Turn new Header and Old Footer High Bits On
(NewHead->size |= (1 << 31));
(FootPtr->size |= (1 << 31));
// Return actual allocated memory's location
void* MemAddress = NewHead;
MemAddress = ((char*)MemAddress + sizeof(Header));
m_PoolSizeTotal = HeadPtr->size;
return MemAddress;
}
else
{
// Updating descriptors
HeadPtr->prev->next = HeadPtr->next;
HeadPtr->next->prev = HeadPtr->prev;
HeadPtr->next = NULL;
HeadPtr->prev = NULL;
// Turning Header and Footer High Bits On
(HeadPtr->size |= (1 << 31));
(FootPtr->size |= (1 << 31));
// Return actual allocated memory's location
void* MemAddress = HeadPtr;
MemAddress = ((char*)MemAddress + sizeof(Header));
m_PoolSizeTotal = HeadPtr->size;
return MemAddress;
}
}
Main.cpp
int* TestArray;
MemoryManager::GetInstance()->CreateHeap(1); // Allocates 1MB
MemoryManager::GetInstance()->AllocateArray(TestArray, 3);
MemoryManager::GetInstance()->DeallocateArray(TestArray);
MemoryManager::GetInstance()->DestroyHeap();
As far as these two specific points:
Instantiate the elements of the array
Set the pointer that is passed in to that Array.
For (1): there is no definitive notion of "initializing" the elements of the array in C++. There are at least two reasonable behaviors, this depends on the semantics you want. The first is to simply zero the array (see memset). The other would be to call the default constructor for each element of the array -- I would not recommend this option as the default (zero argument) constructor may not exist.
EDIT: Example initialization using inplace-new
for (i = 0; i < len; i++)
new (&arr[i]) T();
For (2): It is not exactly clear what you mean by "and then set the pointer that is passed in to that Array." You could "set" the memory returned as data = static_cast<T*>(&mem[4]), which you already do.
A few other words of cautioning (having written my own memory managers), be very careful about byte alignment (reinterpret_cast(mem) % 16); you'll want to ensure you are returning points that are word (or even 16 byte) aligned. Also, I would recommend using inttypes.h to explicitly use uint64_t to be explicit about sizing -- current it looks like this allocator will break for >4GB allocations.
EDIT:
Speaking from experiment -- writing a memory allocator is a very difficult thing to do, and it is even more painful to debug. As commenters have stated, a memory allocator is specific to the kernel -- so information about your platform would be very helpful.

C++ Checking if param dynamic array. Is it necessary?

void Example1( char* ArrayA, unsigned int Length )
{
if( ArrayA == 0 )
{
ArrayA = new char[ Length + 1 ];
// Fill it with 2 - whatever
::memset( ArrayA, 0x02, sizeof( char ) * Length );
ArrayA[ Length ] = '0\n';
}
// Do whatever with ArrayA
// Clean-Up
// Error occurs
delete [ ] ArrayA;
};
void Example2( char* ArrayB, unsigned int Length )
{
bool IsDynamic = false;
if( ArrayB == 0 )
{
ArrayB = new char[ Length + 1 ];
// Fill it with 2 - whatever
::memset( ArrayB, 0x02, sizeof( char ) * Length );
ArrayB[ Length ] = '0\n';
IsDynamic = true;
}
// Do whatever with ArrayA
// Clean-Up
// Have to check...
if( IsDynamic )
delete [ ] ArrayB;
};
int main( void )
{
Example1( "\x01\x02\0x03", 3 ); // Example1 WILL NOT* declare ArrayA as a dynamic array - ERROR (caused by deleting non dynamic array)
Example2( 0, 3 ); // ArrayB will be a dynamic array - OK
Example1( 0, 3 ); // OK
Example2( "\x04\x05\0x06", 3 ); // ArrayB isn't a dynamic array - OK
return ( 0 );
};
The problem occurs when attempting to delete char* ArrayA in function Example1 because ArrayA is not a dynamic array. It will only be a dynamic array if it is equal to zero/null. So, to resolve that I created a similar function - Example2. The only difference is that Example2 has a boolean that checks to see if char* ArrayB is a dynamic array or not.
I know what I am doing is either incorrect or "noobish". So please help me. I will learn from my mistake.
How would you do it?
void Example3( char* ArrayC, unsigned int Length );
Maybe you could use this:
void Example2( char* ArrayB, unsigned int Length )
{
std::vector< char > internalArray;
if ( ArrayB != 0 )
{
internalArray.assign( ArrayB, ArrayB + Length );
}
else
{
internalArray.resize( Length, 0x2 );
}
// Do whatever with internalArray !!! <-------
// No (!!!) clenup need
};
I know what I am doing is either incorrect or "noobish". So please help me.
My overall recommendation would be to move from using C arrays to using std::vector instead.
Your example1 is definitely bad, since it tries to free an array that isn't dynamically allocated - that is NEVER right. As explained elsewhere, if you call across a DLL boundary, you may also have different allocators, so if the memory was not allocated where it is being deleted, things will go wrong. Let whoever allocated it delete it. Preferrably by using already existing standard functionality, such as std::vector
Your example2 only uses delete on something that was created within the function, which is perfectly fine. It doesn't try to delete something that it doesn't know is allocated in the same allocator. Yet, a std::vector would certainly be easier to handle.

Passing pointer array as void pointer to new thread in c++

I am currently working on a project where I have to build a shell for a C++ dll, so a new C# GUI can use its functions.
However I got the following problem, in the C++ portion I have to create a new thread for specific reasons, and I want to pass an int array to the new thread.
Note that the values assigned to the array in the function in which this happens are gained from the C# portion of the code.
__declspec( dllexport ) void CreateReportPane(int &id, int &what)
{
DWORD threadId;
int iArray[2] = { id, what};
HANDLE hThread = CreateThread( NULL, 0, CreateReportPaneThread, iArray, 0, &threadId);
if (hThread == NULL)
{
ExitProcess(3);
}
}
The problem arises in the new thread, I can reliably fetch the first value out of the array, but the 2nd value seems to be released, here is the code on the other side.
DWORD WINAPI CreateReportPaneThread(LPVOID lparam)
{
int id, what;
id = *(( int * )lparam);
what = *(((int *)lparam)+1) ;
CreateReportPaneOriginal(id, what);
return 0;
}
Is there any way to prevent the values in the array from getting released while not holding the original thread captive?
A big thank you in advance
int iArray[2] = { id, what};
HANDLE hThread = CreateThread(...,CreateReportPaneThread, iArray, ...);
The problem is that iArray is a local array which means this gets destroyed when the function CreateReportPane() returns. So what CreateReportPaneThread() refers to doesn't exist. You get the first value just by chance. There is no such guarantee that you would get even the first value.
Use dynamic array:
int * iArray = new int[2];
iArray[0] = id;
iArray[1] = what;
HANDLE hThread = CreateThread(...,CreateReportPaneThread, iArray, ...);
Remember to write deallocate the array once you're done with it in CreateReportPaneThread:
DWORD WINAPI CreateReportPaneThread(PVOID *data)
{
int *array = static_cast<int*>(data);
int id = array[0], what = array[1];
delete []array; //MUST DO IT to avoid memory leak!
//rest of your code
}
Dynamically allocate the array to prevent the array going out of scope when CreateReportPane() exits:
int* iArray = new int[2];
iArray[0] = id;
iArray[1] = what;
otherwise the thread is accessing an array that is no longer valid, which is undefined behaviour. The thread routine CreateReportPaneThread() must then delete[] the array when it is no longer required (note the use of delete[] and not delete).

Got segfault accessing array of pointers to struct when using pointers instead of references

The function below takes a pointer to an array of pointers to API_AddParType structs and lists some of the fields of each array element. Everything works just like expected when accessing each element with (*ppParams)[ii]:
void ParamsListWrapper::ListParams2(API_AddParType** ppParams)
{
unsigned long ii, nParams;
nParams = BMGetHandleSize( (GSHandle)ppParams ) / sizeof(API_AddParType);
// list params
for ( ii = 0; ii < nParams; ii++ ) {
WriteReport( "Param name = \"%s\", double = %f", (*ppParams)[ii].name, (*ppParams)[ii].value.real );
}
}
Well, let's use some reference variable like API_AddParType& param:
void ParamsListWrapper::ListParams1(API_AddParType** ppParams)
{
unsigned long ii, nParams;
nParams = BMGetHandleSize( (GSHandle)ppParams ) / sizeof(API_AddParType);
// list params
for ( ii = 0; ii < nParams; ii++ ) {
API_AddParType& param = (*ppParams)[ii];
WriteReport( "Param name = \"%s\", double = %f", param.name, param.value.real );
}
}
Now trying the same code with pointer to API_AddParType:
void ParamsListWrapper::ListParams3(API_AddParType** ppParams)
{
unsigned long ii, nParams;
nParams = BMGetHandleSize( (GSHandle)ppParams ) / sizeof(API_AddParType);
// list params
for ( ii = 0; ii < nParams; ii++ ) {
API_AddParType* pParam = ppParams[ii];
if (pParam != NULL) {
WriteReport( "Param name = \"%s\", double = %f", pParam->name, pParam->value.real );
} else {
WriteReport( "Param is NULL" );
}
}
}
Got segmentation fault. How could it be? Why it fails when accessing data by pointers while references work without any problems?
The function below takes a pointer to an array of pointers to API_AddParType structs
void ParamsListWrapper::ListParams2(API_AddParType** ppParams)
The function's argument is a pointer-to-pointer-to-API_AddParType, that could be a pointer to the first element of an array of pointers to API_AddParType,
ppParams
|
v
pParam0|pParam1|pParam2|...
| | | ...
v v v
xyz abc mno ...
or it could be a pointer to (a pointer to the first element of an array of API_AddParType),
ppParams
|
v
pParams
|
v
param0|param1|param2|...
where I named the anonymous memory location ppParams points to pParams (pointer to params), and the reason the function receives a pointer to pParams instead of pParams itself is probably that the pointer pParams itself may be changed by the function.
Now, in the first case, (*ppParams)[ii] = pParam0[ii] = the object ii units after the start of xyz, so you never look at what pParam1, pParam2 etc. point to (unless these pointers are set up to point inside the array whose first element pParam1 points to).
In the second case, (*ppParams)[ii] = pParams[ii] = the object ii units after the start of param0, and you look at all the neatly arranged paramN in sequence. Looks good.
With
API_AddParType* pParam = ppParams[ii]
in the first situation, you'd look at xyz, abc, mno, ... in sequence, that looks good.
But in the second situation, pParam would become the pointer ii units after pParams, but that is not the first element of an array of pointers to API_AddParType of size larger than one, so accessing ppParams[ii] is undefined behaviour for ii > 0, and since it is unlikely that there happen to be valid pointers to API_AddParType just behind pParams, the access to pParam->name resp. pParam->value.real probably causes a segfault.
The fact that you got a segmentation fault with ppParams[ii], but not with (*ppParams)[ii] strongly hints that the situation is actually the second, and not the first.
Try changing API_AddParType* pParam = ppParams[ii]; to API_AddParType* pParam = *ppParams; and accessing pParam->name like this pParam[ii].name.
By accessing (*ppParams)[ii] you are manipulating a single API_AddParType, but accessing pParam[ii] you are manipulating an array of API_AddParType.
Try changing the line
API_AddParType* pParam = ppParams[ii];
to
API_AddParType* pParam = (*ppParams) + ii;
In the last case, you're treating ppParams as an array of pointers, not as a pointer to an array.
The ii:th element of the array is (*ppParams)[ii] in the pointer case as well as in the other cases.
You're looking for its address, which is
API_AddParType* pParam = &(*ppParams)[ii];
or
API_AddParType* pParam = (*ppParams) + ii;