#define ALIGNBUF(Length) Length % ALIGNSIZE ? \
Length + ALIGNSIZE - (Length % ALIGNSIZE) : Length
short NumCols;
long * ColLenArray, * OffsetArray;
ColLenArray = new long(NumCols * sizeof(long));
OffsetArray = new long(NumCols * sizeof(long));
// THIS CODE SHOULD NOT BE NEEDED TO UNDERSTAND THE PROBLEM
// BUT I HAVE INCLUDED IT JUST IN CASE
////////////////////////////////////////////////////////////
SQLColAttribute(hstmt, ((SQLUSMALLINT) i)+1, SQL_DESC_OCTET_LENGTH, NULL, 0, NULL, &ColLenArray[i]);
ColLenArray[i] = ALIGNBUF(ColLenArray[i]);
if (i)
OffsetArray[i] = OffsetArray[i-1]+ColLenArray[i-1]+ALIGNBUF(sizeof(SQLINTEGER));
////////////////////////////////////////////////////////////
void **DataPtr = new void*[OffsetArray[NumCols - 1] + ColLenArray[NumCols - 1] + ALIGNBUF(sizeof(long))];
delete []DataPtr;
Don't think it can be done, have tried every way imaginable.
This code works, as in the program runs, I just can't deallocate the memory. Every time this code is called(not all code included as it isn't relevant) the memory gets bigger. I think that deletion is not happening properly and that the void * keeps growing.
I have also changed some of the code above based on recommendations here, but as this code is, the memory keeps growing.
You can't invoke delete on a void *.
The solution is to not cast a pointer-to-void** (which is what new void*[...] will give you) to void*. I don't really know what your code is supposed to be doing, but have you tried changing the type of DataPtr to void **?
More generally, avoid void* as far as possible in C++. There are better solutions. If you edit your question to describe what you're trying to achieve, then we may be able to help suggest something.
You should try avoid mixing void* and new. In C++ actually, new is meant to automatically determine the type of the pointer; then why should not use it. At least you can use char*, if you are simply dealing with raw bytes.
Other point is new void*[SIZE] allocates void**. So you should change the declaration to void **DataPtr. Remove typecasting ahead of new. You can now delete[] DataPtr;.
Edit:
The code have some problems, the variables should be declared like below:
ColLenArray = new long[NumCols * sizeof(long)]; // declare as long[] (not long())
OffsetArray = new long[NumCols * sizeof(long)];
when you declare those variables as, new long(); it will simply initialize the value and assign a pointer to single long.
The memory corruption happens because, you are using ColLenArray[i], which is accessing wrong memory. Since you are going to use above variables as arrays, it should be new long[]. Then memory corruption will not happen. After usage, you should delete[] them.
You just want a block of memory that you can pass to some database library routines? Allocate thus:
char * buffer = new char[ len ];
len is the length of the buffer in bytes. To delete, simply do:
delete [] buffer;
You want a void* to pass to a function?
void * DataPtr = static_cast< void* >( buffer );
For extra merit points, use boost to manage deletion:
boost::scoped_array< char > buffer( new char[ len ] );
... then you don't have to worry about deletion. To get the buffer here, you need:
void * DataPtr = static_cast< void* >( buffer.get() );
Related
I am changing c code to c++ code and this is how how i have converted malloc to new
frame.p_data = (uint8_t*)malloc(xres * yres * 4);
free(frame.p_data);
frame.p_data = (uint8_t*)operator new(xres * yres * 4);
delete(frame.p_data);
Is this the correct way to change from malloc to new and would the delete release all the data.
No, you should write
frame.pdata = new uint8_t[xres * yres * 4];
delete [] frame.pdata;
It is very rare to call the allocation function operator new explicitly, but if you do, you need to match it with the deallocation function, operator delete, otherwise the behaviour is undefined:
frame.p_data = (uint8_t*)operator new(xres * yres * 4);
operator delete(frame.p_data);
It's also not a problem to use malloc with primitive types, as long as you remember to deallocate with free, but it's easier to stick to new/delete and not have to remember.
This answer shows how to use new/delete, but this recommendation just does not seem right/complete.
The correct way (that will pass a mainstream code-review), is not replace it with new/delete in the first place, but go directly to containers/smart pointers. std::vector (or std::array if static size) has all the tools needed, since std::vector is the dynamic array in c++. If the size tracking overhead is too much, then std::unique_ptr or std::array.
simplified example:
std::vector<uint8_t> data;
data.resize(xres * yres * 4);
uint8_t* pdata = data.data(); // <- access to raw buffer
A few guidelines regarding this:
https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines#Rr-newdelete
https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines#Ri-raw
I'm trying to free the memory of the allocated array inside struct _Stack, but the program keeps crashing
typedef struct _Stack
{
int top;
unsigned int capacity;
int* arr;
}_Stack;
_Stack* createStack(int capacity)
{
_Stack* stack = (_Stack*) malloc(sizeof(_Stack));
stack->capacity = capacity;
stack->top = -1;
stack->arr = (int*) malloc(sizeof(stack->capacity * sizeof(int)));
return stack;
}
I'm using this function to free the memory, but the program crashes here.
// I have a problem here.
void stack_free(_Stack* stack)
{
free(stack->arr);
free(stack);
}
Change this:
stack->arr = (int*) malloc(sizeof(stack->capacity * sizeof(int)));
to this:
stack->arr = (int*) malloc(stack->capacity * sizeof(int));
since you want the size of the array to be equal to stack->capacity * sizeof(int), and not equal to the size of that expression.
Your program must have invoked Undefined Behavior somewhere in code not shown in the question (because of the wrong size malloc'ed), that's why it crashes later.
PS: Since you use C++, consider using new instead (and delete, instead of free()).
sizeof(stack->capacity * sizeof(int)) in your call to malloc is wrong. Instead of the size of the array, it gives the size of the number used to represent the size of the array. You probably want stack->capacity * sizeof(int).
Another possible problem is that in C you should not cast the return value of malloc, since it can hide other mistakes and cause a crash. See Do I cast the result of malloc?
In C++ you will have to do it, because of the stricter type checking in C++, but it can still hide problems.
These are the problems I see with the code you have shown. However, remember that errors in malloc and free are not necessarily caused by the actual line where they are detected. If some part of your program damages the malloc system's internal data structures, for example by a buffer overrun, the problem can manifest in a much later call to malloc or free, in a completely different part of the program.
I'm working on an Allocator system for my game engine and I'm wonder if it is possible to reallocate memory (multiple of PageSize) without invalidating the pointers that point to locations within it. Can this be accomplished using a virtual memory interface? I'm aware virtual memory paging doesn't work for DMA/pinned memory, and that its not available on consoles.
Is realloc able to do this(but not guarantee it)? I'm looking for a POSIX, Linux, or Windows api that can do this just so I have a place to start.
Also, I would appreciate any further readings relevant to creating a memory management system, thanks.
The virtuality of the address space is not all that relevant to your proposed solution. The reason a larger block may not be possible is because there is data in the address space that you want your array to grow into. This is the same problem on physical and virtual memory spaces.
I can see two alternative approaches.
There are two easy approaches. First, reserve enough memory in a std::vector. This does should do nothing but aquire virtual memory space on efficient systems. Grow it as needed, and those virtual memory pages will be assigned physical pages. This is more practical on 64 bit systems, as you have a huge memory space.
Another approach would be to create your own segmented like array. Have a vector with unique pointers to page sized chunks, and a wrapper that hides that detail. This adds another layer of indirection on iterator access. Implementing fast-blit for trivially copiable types and other operations optional.
If there's a particular allocation that you're especially keen to keep in-place, it may make sense to malloc much more memory than initially needed. On a modern OS, that will reserve virtual address space - which will be effectively unconstrained for any 64-bit app - and actual physical backing memory will only be sought as pages are first accessed, so an excessive initial allocation is extremely "low cost" but may prevent a need to realloc beyond that size. That's much easier for you than fiddling with or replacing the allocation routines....
If you have a great many allocations, and/or a 32-bit app, you may want to pursue the ability to:
ask realloc if it can operate in-place (e.g. adding an extra function argument to prevent moving to another address and returning nullptr when it fails); this approach reduces pointer adjustments without necessarily being more aggressive in allocating virtual address space, but you'll still need to handle having to move sometimes; and/or
intercept malloc and realloc calls and allocate them much more virtual address space than requested to reduce (or - given application specific knowledge - eliminate) the risk of needing to move the content during a later realloc.
The Standard-mandated malloc/free/realloc interface has no hooks or options for reallocating only if in-place (and letting you know when it "fails"), so you will need to write your own routines or edit a malloc-et-al library.
Without writing your own allocator, or adopting a third-party one, there is no way to grow a malloc'ed block and guarantee that it doesn't move. Realloc may move it and there's no way to force it to fail if it can't grow in place.
You might want to look into std::deque. It is like a std::vector in many respects (a growable memory array), but when elements are added, it does not ever reallocate storage for old elements or move the existing objects in memory. It does have some downsides--there's no guarantee that all the objects will be contiguous in memory, and accessing an element of the deque is slightly slower than accessing a vector element, as it has to first calculate which block the object is in, then calculate the location of the element within the block. Generally this should not make a noticeable difference unless it is extremely performance critical.
Well, since you are writing it in c++:
overload malloc()/realloc()/free() with specialized versions
instead of using bare pointers, use a special class, say class myptr which
When dereferenced doesn't do anything extra, just dereferences
When assigned, adds the variable to a list which realloc() looks through to repoint any pointers needing adjustment.
The weakness of this is that unless are very few types of objects being pointed to, it might be tedious to get the types to simply work. Of course, that is a suitable task for templates...
Here is some code that does what you asked for. If realloc() moves the block of memory, then it finds and fixes the pointers into and within the block. This was inspired by the BDWGC project, but they are not to blame. The danger with this code is it will find a word in memory that looks like a pointer but is not a pointer.
struct dbbExprAllocFixup {
typedef dbbExpr * dbbExprPtr;
dbbExprAllocFixup( void *, size_t, void * );
void CheckExpr( dbbExprPtr & );
void CheckOne( void *, char * tag );
void CheckRange();
char * oldBase; // old realloc() ptr
char * oldEnd;
char * newBase; // new realloc() ptr
char * newEnd;
ptrdiff_t delta;
};
dbbExprAllocFixup::dbbExprAllocFixup( void * ob, size_t ns, void * nb )
{
oldBase = (char *) ob;
oldEnd = oldBase + ns;
newBase = (char *) nb;
newEnd = newBase + ns;
delta = newBase - oldBase;
dbbTrace::Output( "Expr realloc: (%p to %p) moved to %p, delta = %p\n",
oldBase, oldEnd, newBase, delta );
}
void
dbbExprAllocFixup::CheckOne( void * p, char * tag )
{
char * * scan = (char * *) p;
char * value = * scan;
if( value >= oldBase && value <= oldEnd ) {
// This value needs fixing
* scan = * scan + delta;
dbbTrace::Output( " Expr realloc: %s old value %p new value %p\n",
tag, value, * scan );
}
}
void
dbbExprAllocFixup::CheckExpr( dbbExprPtr & p )
{
if( p != 0 ) {
CheckOne( & p, "e" );
}
}
void
dbbExprAllocFixup::CheckRange()
{
char * * scan;
for( scan = (char * *) newBase; scan < (char * *) newEnd; scan++ ) {
CheckOne( scan, "r" );
} // for
}
The code that calls realloc() and dbbExprAllocFixup if needed.
void * p = ::realloc( m_pHead, bytes );
if( p == m_pHead ) {
// The memory did not move, do nothing
} else {
// The memory moved
dbbExprAllocFixup f( m_pHead, bytes, p );
// Fix pointer in the block to locations in the block
f.CheckRange();
// Fix pointers outside the block that point into the block
f.CheckExpr( pExpr->m_pRoot );
for( int i = 0; i < pExpr->m_MLRootSize; i++ ) {
f.CheckExpr( pExpr->m_pMLRoot[i] );
}
m_pHead = (dbbExprAllocChunk *) p;
} // if
This code is not in use so could easily have bugs. One bug I noticed now is it uses the new size for both the old and new realloc() memory blocks.
Consider the following code
struct foo
{
const int txt_len;
const int num_len;
char * txt;
int * num;
foo(int tl, int nl): txt_len(tl), num_len(nl)
{
char * tmp = new char[txt_len * sizeof(char) + num_len * sizeof(int)];
txt = new (tmp) char [txt_len * sizeof(char)];
num = new (tmp + txt_len * sizeof(char)) int[num_len * sizeof(int)];
// is this the same as above?
// txt = tmp;
// num = (int *) (tmp + txt_len * sizeof(char));
}
~foo()
{
delete[] txt; // is this the right way to free the memory?
}
};
I want *txt and *num to be contiguous, is that the best way to do it?
also is there any difference between placement new and pointer arithmetic? which one should I use?
If you want a contiguous block of memory, you have to allocate it whole with a single call to operator new[] or malloc() or similar. Multiple calls to these functions do not guarantee any contiguity of allocated blocks whatsoever. You may allocate a big block and then carve parts from it as needed.
And you should delete and free() all blocks previously allocated with new and malloc(), otherwise you'll leak memory and probably make your program unstable (it will fail to allocate more memory at some point) and exert unnecessary pressure on memory in the OS, possibly slowing down other programs or making them unstable as well.
Placement new, however, does not actually allocate any memory. It simply constructs an object at the specified location and so you don't need to free that memory twice.
One problem that I see in your code is that it doesn't align ints. On some platforms reading or writing integers bigger than 1 byte from/to the memory must be aligned and if it's not, you can either read/write values from/to wrong locations or get CPU exceptions leading to termination of your program. The x86 is very permissive in this regard and won't mind, though may tax you with degraded performance.
You'll need to put the int data first, due to the alignment issues. But we can't then do delete num[] as the type is wrong - it must be cast to a char* before deleting.
char * tmp = new char[num_len * sizeof(int) + txt_len * sizeof(char)];
num = new (tmp) int[num_len];
txt = new (tmp + num_len * sizeof(int)) char [txt_len];
(This makes liberal use of the fact that sizeof(char)==1)
You might be tempted to do delete[] num, but num is of type int*, and it was new'ed as a char*. So you need to do;
delete[] (char*) num;
This is the same as long as you use POD types. And your delete is fine.
However, as David's comment states, you need to consider alignment problems.
Placement new is mostly use when you want to call constructor of class/struct on some preallocated memory blocks.
But for native types it makes no different to use placement new & pointer arithmetic.
Please correct me if I was wrong.
If txt and num always point to int and char, other built in types or other types not requiring construction, then no. You don't need placement new.
If on the other hand you were to change one of them to a class which requires construction, i.e. changes txt to type std::string, then using placement new is necessary.
Placement new allows you to call the constructor, building, if you like, the object at that address. Built in types have default constructors that do nothing if your not initializing.
In both cases you need to do pointer arithmetic, just one way you store the answer in a pointer, the other you pass the answer to placement new which gives it back to you for storage in the pointer, and then calls the constructor.
I have a program in C++ that has a BYTE array that stores some values. I need to find the length of that array i.e. number of bytes in that array. Please help me in this regard.
This is the code:
BYTE *res;
res = (BYTE *)realloc(res, (byte_len(res)+2));
byte_len is a fictitious function that returns the length of the BYTE array and I would like to know how to implement it.
Given your code:
BYTE *res;
res = (BYTE *)realloc(res, (byte_len(res)+2));
res is a pointer to type BYTE. The fact that it points to a contiguous sequence of n BYTES is due to the fact that you did so. The information about the length is not a part of the pointer. In other words, res points to only one BYTE, and if you point it to the right location, where you have access to, you can use it to get BYTE values before or after it.
BYTE data[10];
BYTE *res = data[2];
/* Now you can access res[-2] to res[7] */
So, to answer your question: you definitely know how many BYTEs you allocated when you called malloc() or realloc(), so you should keep track of the number.
Finally, your use of realloc() is wrong, because if realloc() fails, you leak memory. The standard way to use realloc() is to use a temporary:
BYTE *tmp;
tmp = (BYTE *)realloc(res, n*2);
if (tmp == NULL) {
/* realloc failed, res is still valid */
} else {
/* You can't use res now, but tmp is valid. Reassign */
res = tmp;
}
If the array is a fixed size array, such as:
BYTE Data[200];
You can find the length (in elements) with the commonly used macro:
#define ARRAY_LENGTH(array) (sizeof(array)/sizeof((array)[0]))
However, in C++ I prefer to use the following where possible:
template<typename T, size_t N>
inline size_t array_length(T data[N])
{
return N;
};
Because it prevents this from occurring:
// array is now dynamically allocated
BYTE* data = new BYTE[200];
// oops! this is now 4 (or 8 on 64bit)!
size_t length = ARRAY_LENGTH(data);
// this on the other hand becomes a compile error
length = array_length(data);
If the array is not a fixed size array:
In C++, raw pointers (like byte*) are not bounded. If you need the length, which you always do when working with arrays, you have to keep track of the length separately. Classes like std::vector help with this because they store the length of the array along with the data.
In the C way of doing things (which is also relevant to C++) you generally need to keep a record of how long your array is:
BYTE *res;
int len = 100
res = (BYTE *)realloc(res, (byte_len(len)));
len += 2;
res = (BYTE *)realloc(res, (byte_len(len)));
An alternative in the C++ way of doing things s to use the std::vector container class; a vector has the ability to manage the length of the array by itself, and also deals with the issues of memory management..
EDIT: as others have pointed out the use of realloc here is incorrect as it will lead to memory leaks, this just deals with keeping track of the length. You should probably accept one of the other replies as the best answer
Given the information you seem to have available, there is no way to do what you want. When you are working with arrays allocated on the heap, you need to save the size somewhere if you need to work with it again. Neither new nor malloc will do this for you.
Now, if you have the number of items in the array saved somewhere, you can do this to get the total size in characters, which is the unit that realloc works with. The code would look like this:
size_t array_memsize = elems_in_array * sizeof(BYTE);
If you are really working with C++ and not C I would strongly suggest that you use the vector template for this instead of going to malloc and realloc. The vector template is fast and not anywhere near as error prone as rolling your own memory management. In addition, it tracks the size for you.
When you allocate the pointer initially you also need to keep track of the length:
size_t bufSize = 100;
BYTE* buf = malloc(sizeof(BYTE ) * bufSize);
When you re-allocate you should be carefull with the re-alloc:
BYTE* temp = realloc(buf,sizeof(BYTE ) * (bufSize+2));
if (temp != NULL)
{
bufSize += 2;
buf = temp;
}
If it is a local variable allocated on the stack you can calculate it like this:
BYTE array[] = { 10, 20, 30, ... };
size_t lenght = sizeof(array) / sizeof(BYTE);
If you receive a pointer to the beginning of the array or you allocate it dynamically(on the heap), you need to keep the length as well as the pointer.
EDIT: I also advise you use STL vector for such needs because it already implements dynamic array semantics.