Consider the following code
struct foo
{
const int txt_len;
const int num_len;
char * txt;
int * num;
foo(int tl, int nl): txt_len(tl), num_len(nl)
{
char * tmp = new char[txt_len * sizeof(char) + num_len * sizeof(int)];
txt = new (tmp) char [txt_len * sizeof(char)];
num = new (tmp + txt_len * sizeof(char)) int[num_len * sizeof(int)];
// is this the same as above?
// txt = tmp;
// num = (int *) (tmp + txt_len * sizeof(char));
}
~foo()
{
delete[] txt; // is this the right way to free the memory?
}
};
I want *txt and *num to be contiguous, is that the best way to do it?
also is there any difference between placement new and pointer arithmetic? which one should I use?
If you want a contiguous block of memory, you have to allocate it whole with a single call to operator new[] or malloc() or similar. Multiple calls to these functions do not guarantee any contiguity of allocated blocks whatsoever. You may allocate a big block and then carve parts from it as needed.
And you should delete and free() all blocks previously allocated with new and malloc(), otherwise you'll leak memory and probably make your program unstable (it will fail to allocate more memory at some point) and exert unnecessary pressure on memory in the OS, possibly slowing down other programs or making them unstable as well.
Placement new, however, does not actually allocate any memory. It simply constructs an object at the specified location and so you don't need to free that memory twice.
One problem that I see in your code is that it doesn't align ints. On some platforms reading or writing integers bigger than 1 byte from/to the memory must be aligned and if it's not, you can either read/write values from/to wrong locations or get CPU exceptions leading to termination of your program. The x86 is very permissive in this regard and won't mind, though may tax you with degraded performance.
You'll need to put the int data first, due to the alignment issues. But we can't then do delete num[] as the type is wrong - it must be cast to a char* before deleting.
char * tmp = new char[num_len * sizeof(int) + txt_len * sizeof(char)];
num = new (tmp) int[num_len];
txt = new (tmp + num_len * sizeof(int)) char [txt_len];
(This makes liberal use of the fact that sizeof(char)==1)
You might be tempted to do delete[] num, but num is of type int*, and it was new'ed as a char*. So you need to do;
delete[] (char*) num;
This is the same as long as you use POD types. And your delete is fine.
However, as David's comment states, you need to consider alignment problems.
Placement new is mostly use when you want to call constructor of class/struct on some preallocated memory blocks.
But for native types it makes no different to use placement new & pointer arithmetic.
Please correct me if I was wrong.
If txt and num always point to int and char, other built in types or other types not requiring construction, then no. You don't need placement new.
If on the other hand you were to change one of them to a class which requires construction, i.e. changes txt to type std::string, then using placement new is necessary.
Placement new allows you to call the constructor, building, if you like, the object at that address. Built in types have default constructors that do nothing if your not initializing.
In both cases you need to do pointer arithmetic, just one way you store the answer in a pointer, the other you pass the answer to placement new which gives it back to you for storage in the pointer, and then calls the constructor.
Related
Because segmentation fault related to malloc/free happens, I would like to convert malloc/free to new/delete.
Error occurred when malloc/free is converted to below.
Let me know how to solve it.
(original)
char *only_valid_data = static_cast<char*> (malloc (data_size));
(converted)
char *only_valid_data = new static_cast<char*> [data_size];
Just do
char* only_valid_data = new char[data_size];
to allocate.
To free you do
delete[] only_valid_data;
Important note: When you allocate memory with new it will allocate data_size elements, not data_size bytes (like malloc does). The size of an element is the size of the non-pointer base type, in your case e.g. sizeof(*only_valid_data). In this case the element size and the byte size is the same (as sizeof(char) is specified to always be 1), but if the base type is something else it will make a big difference.
For example, if you do
int* int_ptr = new int[10];
then ten integers will be allocated, not ten bytes. This is equivalent to
int* int_ptr = reinterpret_cast<int*>(malloc(10 * sizeof(*int_ptr)));
Also note that for complex types, like structures and classes, allocating with new (or new[]) does more than just allocating memory, it will also make sure that the allocated object(s) constructor is called. The malloc function only allocates memory, it doesn't call the constructor.
Final note: The problem you have with the segmentation fault is probably not caused by your allocation, no matter how you allocate the memory. The problem is more likely because of something else, something you do not show in your question, like writing out of bounds of the allocated memory or dereferencing a null-pointer.
You need run your program in a debugger to catch the crash in action, it will allow you to examine the function call stack, and if the crash doesn't happen in your code then you walk up the call stack until you reach your code. There you can examine the values of variables, to help you understand why the problem occurred.
The malloc family (malloc, realloc, calloc, free) is used almost always in C code, as C++ provides the new and delete operators which are a lot more reliable to use.
A problem with malloc for allocation is that you must specify the size of the type in bytes that you want to allocate. For example:
int* ptr = malloc(5);
Will not allocate space for 5 integers in memory; it will allocate 5 bytes of memory (the size of an integer is 4 bytes, so this would obviously cause problems when assigning).
To do it properly, it must be written as
int* ptr = malloc(5 * sizeof(int));
So that 20 bytes are allocated.
However, there are some exceptions to the case. char, for example only requires one byte of memory, so doing
char* ptr = malloc(5);
Will allocate enough memory to hold 5 characters, and in a way is more valid that writing:
char* ptr = malloc(5 * sizeof(char)); //5 * sizeof(char) == 5 * 1 == 5
However, the free function does not need to know the size of the pointer to be deallocated; a void* is only needed.
Note that in C++, the return of malloc must be cast properly to the type wanted; malloc returns a void* type, but C++ does not allow any pointer to assign a void* to any pointer type like C does:
int* ptr = malloc(5 * sizeof(int)); //valid C code, invalid C++
int* ptr2 = (int*)malloc(5 * sizeof(int)); //valid C code, valid C++
In C++, the new[] operator resolves the issue of remembering to add the sizeof operator.
int* ptr = new int[5];//allocates 5 integers
int* ptr2 = new int(5);//be careful: this allocates a single integer with value of 5
Note that if the new[] operator has been used, the delete[] operator must be used. Otherwise the delete operator must be used:
int* ptr = new int[5];//allocates 5 integers
delete[] ptr;//deallocate the 5 integers
int* ptr2 = new int(5);//be careful: this allocates a single integer with value of 5
delete ptr;//deallocate the integer
The problem with your code is that it does not fit the syntax of the new[] operator
The syntax could be described as:
T* p = new T[size];
Thus your code:
char *only_valid_data = new static_cast<char*> [data_size];
Should be corrected to:
char *only_valid_data = new char[data_size];
As static_cast<char*> is not a type.
Hope this helps :)
I'm working on an Allocator system for my game engine and I'm wonder if it is possible to reallocate memory (multiple of PageSize) without invalidating the pointers that point to locations within it. Can this be accomplished using a virtual memory interface? I'm aware virtual memory paging doesn't work for DMA/pinned memory, and that its not available on consoles.
Is realloc able to do this(but not guarantee it)? I'm looking for a POSIX, Linux, or Windows api that can do this just so I have a place to start.
Also, I would appreciate any further readings relevant to creating a memory management system, thanks.
The virtuality of the address space is not all that relevant to your proposed solution. The reason a larger block may not be possible is because there is data in the address space that you want your array to grow into. This is the same problem on physical and virtual memory spaces.
I can see two alternative approaches.
There are two easy approaches. First, reserve enough memory in a std::vector. This does should do nothing but aquire virtual memory space on efficient systems. Grow it as needed, and those virtual memory pages will be assigned physical pages. This is more practical on 64 bit systems, as you have a huge memory space.
Another approach would be to create your own segmented like array. Have a vector with unique pointers to page sized chunks, and a wrapper that hides that detail. This adds another layer of indirection on iterator access. Implementing fast-blit for trivially copiable types and other operations optional.
If there's a particular allocation that you're especially keen to keep in-place, it may make sense to malloc much more memory than initially needed. On a modern OS, that will reserve virtual address space - which will be effectively unconstrained for any 64-bit app - and actual physical backing memory will only be sought as pages are first accessed, so an excessive initial allocation is extremely "low cost" but may prevent a need to realloc beyond that size. That's much easier for you than fiddling with or replacing the allocation routines....
If you have a great many allocations, and/or a 32-bit app, you may want to pursue the ability to:
ask realloc if it can operate in-place (e.g. adding an extra function argument to prevent moving to another address and returning nullptr when it fails); this approach reduces pointer adjustments without necessarily being more aggressive in allocating virtual address space, but you'll still need to handle having to move sometimes; and/or
intercept malloc and realloc calls and allocate them much more virtual address space than requested to reduce (or - given application specific knowledge - eliminate) the risk of needing to move the content during a later realloc.
The Standard-mandated malloc/free/realloc interface has no hooks or options for reallocating only if in-place (and letting you know when it "fails"), so you will need to write your own routines or edit a malloc-et-al library.
Without writing your own allocator, or adopting a third-party one, there is no way to grow a malloc'ed block and guarantee that it doesn't move. Realloc may move it and there's no way to force it to fail if it can't grow in place.
You might want to look into std::deque. It is like a std::vector in many respects (a growable memory array), but when elements are added, it does not ever reallocate storage for old elements or move the existing objects in memory. It does have some downsides--there's no guarantee that all the objects will be contiguous in memory, and accessing an element of the deque is slightly slower than accessing a vector element, as it has to first calculate which block the object is in, then calculate the location of the element within the block. Generally this should not make a noticeable difference unless it is extremely performance critical.
Well, since you are writing it in c++:
overload malloc()/realloc()/free() with specialized versions
instead of using bare pointers, use a special class, say class myptr which
When dereferenced doesn't do anything extra, just dereferences
When assigned, adds the variable to a list which realloc() looks through to repoint any pointers needing adjustment.
The weakness of this is that unless are very few types of objects being pointed to, it might be tedious to get the types to simply work. Of course, that is a suitable task for templates...
Here is some code that does what you asked for. If realloc() moves the block of memory, then it finds and fixes the pointers into and within the block. This was inspired by the BDWGC project, but they are not to blame. The danger with this code is it will find a word in memory that looks like a pointer but is not a pointer.
struct dbbExprAllocFixup {
typedef dbbExpr * dbbExprPtr;
dbbExprAllocFixup( void *, size_t, void * );
void CheckExpr( dbbExprPtr & );
void CheckOne( void *, char * tag );
void CheckRange();
char * oldBase; // old realloc() ptr
char * oldEnd;
char * newBase; // new realloc() ptr
char * newEnd;
ptrdiff_t delta;
};
dbbExprAllocFixup::dbbExprAllocFixup( void * ob, size_t ns, void * nb )
{
oldBase = (char *) ob;
oldEnd = oldBase + ns;
newBase = (char *) nb;
newEnd = newBase + ns;
delta = newBase - oldBase;
dbbTrace::Output( "Expr realloc: (%p to %p) moved to %p, delta = %p\n",
oldBase, oldEnd, newBase, delta );
}
void
dbbExprAllocFixup::CheckOne( void * p, char * tag )
{
char * * scan = (char * *) p;
char * value = * scan;
if( value >= oldBase && value <= oldEnd ) {
// This value needs fixing
* scan = * scan + delta;
dbbTrace::Output( " Expr realloc: %s old value %p new value %p\n",
tag, value, * scan );
}
}
void
dbbExprAllocFixup::CheckExpr( dbbExprPtr & p )
{
if( p != 0 ) {
CheckOne( & p, "e" );
}
}
void
dbbExprAllocFixup::CheckRange()
{
char * * scan;
for( scan = (char * *) newBase; scan < (char * *) newEnd; scan++ ) {
CheckOne( scan, "r" );
} // for
}
The code that calls realloc() and dbbExprAllocFixup if needed.
void * p = ::realloc( m_pHead, bytes );
if( p == m_pHead ) {
// The memory did not move, do nothing
} else {
// The memory moved
dbbExprAllocFixup f( m_pHead, bytes, p );
// Fix pointer in the block to locations in the block
f.CheckRange();
// Fix pointers outside the block that point into the block
f.CheckExpr( pExpr->m_pRoot );
for( int i = 0; i < pExpr->m_MLRootSize; i++ ) {
f.CheckExpr( pExpr->m_pMLRoot[i] );
}
m_pHead = (dbbExprAllocChunk *) p;
} // if
This code is not in use so could easily have bugs. One bug I noticed now is it uses the new size for both the old and new realloc() memory blocks.
I am new to programming and I am trying to understand the difference between
A = (char * ) malloc(sizeof(char)*n);
and
A = (char * ) malloc(sizeof(char));
or
A = new char [n];
and
A = new char;
What is the default memory that a compiler is allocating to this pointer, when I do not specify the number of objects of particular data type.
Also when I declare
A = new char [n];
cout << A[n+1];
it does not give me a segmentation fault.
Should It not give segmentation fault because I am trying to access memory beyond what has been allocated for the Array.
Memory is not "allocated to this pointer", it's allocated and then you get a pointer to the memory.
This:
char *a = malloc(sizeof(char) * n);
is the same as
char *a = malloc(n);
since sizeof(char) is always 1. They both allocate space for n characters worth of data, and return a pointer to the location where the first character can be accessed (or NULL on failure).
Also, the casts are not needed in C, you should not have any.
Since sizeof(char) is 1, the second call is equivalent to:
char *a = malloc(1);
which means it allocates a memory block of size 1. This is of course distinct from the pointer to that memory block (the value that gets stored in the pointer variable a). The pointer is most likely larger than 1 char, but that doesn't affect the size of the block.
The argument to malloc() specifies how many chars to allocate space for.
I ignored the new usage, since that is C++ and the question is tagged C.
A = (char * ) malloc(sizeof(char)*n);
This allocates space for n characters.
A = (char * ) malloc(sizeof(char));
This allocates memory for 1 character.
Every call to malloc allocates memory in the heap.
The other code is C++, and it's exactly the same, except that it will use stack memory if A is a local variable. Accessing A[n+1] may or may not yield a segfault. A[n+1] can reference a memory address that you are allowed to use. Segfault happens when you go out of the region of memory you can access, and the way it works is that there is a "red zone" from which it is considered you accessed invalid memory. It may be the case that A[n+1] just isn't "invalid enough" to trigger a segfault.
allocate space for N characters (N should be some positive integer value here)
char *ma = (char * ) malloc(N);
char *na = new char [N];
don't forget to release this memory ...
delete [] na;
free(ma);
allocate space for a single character
char *mc = (char * ) malloc(sizeof(char));
char *nc = new char;
Now, as the others have pointed out, you tagged this C, but half your code is C++. If you were writing C, you couldn't use new/delete, and wouldn't need to cast the result of malloc.
Oh, and the reason you don't get a segmentation fault when you read off the end of your array is that this is undefined behaviour. It certainly could cause a SEGV, but it isn't required to check, so may appear to work, at least some of the time, or fail in a completely different way.
Well, the compiler doesn't allocate memory for the data. Only the pointer which is either 4 or 8 bytes depending on your architecture.
There is no difference between the first two and the last two in terms on functionality. Most C++ libraries I've seen use malloc internally for new.
When you run the code to allocate n characters and you print out the n + 1th character, you aren't getting a segmentation fault most likely because n isn't a multiple of some number, usually 8 or 16. Here's some code on how it might do that:
void* malloc(size_t size) {
if (size & 0x7 != size)
size = size & 0x7 + 1;
return _malloc(size);
}
So, if you requested, say, 5 bytes, malloc would actually allocate, with that code, 8 bytes. So, if you request the 6th byte (n + 1), you would get garbage, but it is still valid memory that your program can access.
I am trying to resize a dynamically allocated string array; here's the code!
void resize_array() {
size_t newSize = hash_array_length + 100;
string* newArr = new string[newSize];
fill_n(hash_array,newSize,"0"); //fills arrays with zeros
memcpy( newArr, hash_array, hash_array_length * sizeof(string) );
hash_array_length = newSize;
delete [] hash_array;
hash_array = newArr;
}
unfortunately it isn't working and gives a segmentation fault. any idea why? this is basically a linear probing hash table where the element gets inserted wherever there is a 0 hence I use fill_n to fill the newly created array with 0's. any help please?
memcpy( newArr, hash_array, hash_array_length * sizeof(string) );
This line is extremely dangerous, std::string is not a plain old data type,
you can't make sure that memcpy could initialize it correctly, it may cause
undefined behavior, one of the most nasty behavior of c++(or programming).
Besides, there are a better and safer(in most of the times) solution to create
a dynamic string array in c++, just use vector
//create a dynamic string array with newSize and initialize them with "0"
//in your case, I don't think you need to initialize it with "0"
std::vector<std::string> newArr(newSize, "0");
if the hash_array has the same type as newArr(std::vector)
The way of copy it is very easy.
c++98
std::copy(hash_array.begin(), hash_array.end(), newArr.begin());
c++11
std::copy(std::begin(hash_array), std::end(hash_array), std::begin(newArr));
Better treat c++ as a new language, it has too many things are different from c.
Besides, there are a lot of decent free IDE, like code::blocks and QtCreator
devc++ is a almost dead project.
If you are new to c++, c++ primer 5 is a good book to start.
If string is actually an std::string (and probably even if it isn't) then this will crash. You are creating a new array of strings, copying the old string classes over the top, and then freeing the old strings. But if the string class contains internal pointers to allocated memory this will result in a double free because all you are doing is copying the internal pointers - not making new memory allocations.
Think about it like this; imagine you had the following class:
class foo
{
char* bar;
foo() { bar = malloc(100); }
~foo() { free(bar);
};
foo* ptr1 = new foo;
foo* ptr2 = new foo;
memcpy(ptr2, ptr1, sizeof(foo*));
delete ptr1;
At this point, ptr2->bar points to the same memory that ptr1->bar did, but ptr1 and the memory it held has been freed
The best solution would be to use a std::vector because this handles the resizing automatically and you don't need to worry about copying arrays at all. But if you want to persist with your current approach, you need to change the memcpy call to the following:
for (int i = 0; i < hash_array_length; ++i)
{
newArr[i] = hash_array[i];
}
Rather than just copying the memory this will call the class's copy constructor and make a proper copy of its contents.
I suspect the culprit is memcpy call. string is complicated type which manages the char array by pointers (just as you are doing right now). Normally copying string is done using assignment operator, which for string also copies its own array. But memcpy simply copies byte-per-byte the pointer, and delete[] also deletes the array managed by string. Now the other string uses deleted string array, which is BAAAD.
You can use std::copy instead of memcpy, or even better yet, use std::vector, which is remedy to most of your dynamic memory handling problems ever.
#define ALIGNBUF(Length) Length % ALIGNSIZE ? \
Length + ALIGNSIZE - (Length % ALIGNSIZE) : Length
short NumCols;
long * ColLenArray, * OffsetArray;
ColLenArray = new long(NumCols * sizeof(long));
OffsetArray = new long(NumCols * sizeof(long));
// THIS CODE SHOULD NOT BE NEEDED TO UNDERSTAND THE PROBLEM
// BUT I HAVE INCLUDED IT JUST IN CASE
////////////////////////////////////////////////////////////
SQLColAttribute(hstmt, ((SQLUSMALLINT) i)+1, SQL_DESC_OCTET_LENGTH, NULL, 0, NULL, &ColLenArray[i]);
ColLenArray[i] = ALIGNBUF(ColLenArray[i]);
if (i)
OffsetArray[i] = OffsetArray[i-1]+ColLenArray[i-1]+ALIGNBUF(sizeof(SQLINTEGER));
////////////////////////////////////////////////////////////
void **DataPtr = new void*[OffsetArray[NumCols - 1] + ColLenArray[NumCols - 1] + ALIGNBUF(sizeof(long))];
delete []DataPtr;
Don't think it can be done, have tried every way imaginable.
This code works, as in the program runs, I just can't deallocate the memory. Every time this code is called(not all code included as it isn't relevant) the memory gets bigger. I think that deletion is not happening properly and that the void * keeps growing.
I have also changed some of the code above based on recommendations here, but as this code is, the memory keeps growing.
You can't invoke delete on a void *.
The solution is to not cast a pointer-to-void** (which is what new void*[...] will give you) to void*. I don't really know what your code is supposed to be doing, but have you tried changing the type of DataPtr to void **?
More generally, avoid void* as far as possible in C++. There are better solutions. If you edit your question to describe what you're trying to achieve, then we may be able to help suggest something.
You should try avoid mixing void* and new. In C++ actually, new is meant to automatically determine the type of the pointer; then why should not use it. At least you can use char*, if you are simply dealing with raw bytes.
Other point is new void*[SIZE] allocates void**. So you should change the declaration to void **DataPtr. Remove typecasting ahead of new. You can now delete[] DataPtr;.
Edit:
The code have some problems, the variables should be declared like below:
ColLenArray = new long[NumCols * sizeof(long)]; // declare as long[] (not long())
OffsetArray = new long[NumCols * sizeof(long)];
when you declare those variables as, new long(); it will simply initialize the value and assign a pointer to single long.
The memory corruption happens because, you are using ColLenArray[i], which is accessing wrong memory. Since you are going to use above variables as arrays, it should be new long[]. Then memory corruption will not happen. After usage, you should delete[] them.
You just want a block of memory that you can pass to some database library routines? Allocate thus:
char * buffer = new char[ len ];
len is the length of the buffer in bytes. To delete, simply do:
delete [] buffer;
You want a void* to pass to a function?
void * DataPtr = static_cast< void* >( buffer );
For extra merit points, use boost to manage deletion:
boost::scoped_array< char > buffer( new char[ len ] );
... then you don't have to worry about deletion. To get the buffer here, you need:
void * DataPtr = static_cast< void* >( buffer.get() );