How can realloc not work but malloc can? - c++

I have reached a point where realloc stops returning a pointer - I assume that there is a lack of space for the array to expand or be moved. The only problem is I really need that memory to exist or the application can't run as expected, so I decided to try malloc - expecting it not work since realloc would no work - but it did. Why?
Then I memcpy the array of pointers into the new allocated array, but found it broke it, pointers like 0x10 and 0x2b was put in the array. There are real pointers, but if I replace the memcpy with a for loop, that fixes it. Why did memcpy do that? Should I not be using memcpy in my code?
Code:
float * resizeArray_by(float *array, uint size)
{
float *tmpArray = NULL;
if (!array)
{
tmpArray = (float *)malloc(size);
}
else
{
tmpArray = (float *)realloc((void *)array, size);
}
if (!tmpArray)
{
tmpArray = (float *)malloc(size);
if (tmpArray)
{
//memcpy(tmpArray, array, size - 1);
for (int k = 0; k < size - 1; k++)
{
((float**)tmpArray)[k] = ((float **)array)[k];
}
free(array);
}
}
return tmpArray;
}
void incrementArray_andPosition(float **& array, uint &total, uint &position)
{
uint prevTotal = total;
float *tmpArray = NULL;
position++;
if (position >= total)
{
total = position;
float *tmpArray = resizeArray_by((float *)array, total);
if (tmpArray)
{
array = (float **)tmpArray;
array[position - 1] = NULL;
}
else
{
position--;
total = prevTotal;
}
}
}
void addArray_toArray_atPosition(float *add, uint size, float **& array, uint &total, uint &position)
{
uint prevPosition = position;
incrementArray_andPosition(array, total, position);
if (position != prevPosition)
{
float *tmpArray = NULL;
if (!array[position - 1] || mHasLengthChanged)
{
tmpArray = resizeArray_by(array[position - 1], size);
}
if (tmpArray)
{
memcpy(tmpArray, add, size);
array[position - 1] = tmpArray;
}
}
}
After all my fixes, the code inits probably. The interesting thing here, is after sorting out the arrays, I allocate with malloc a huge array, so to reorder the arrays into one array to be used as an GL_ARRAY_BUFFER. If realloc is no allocating because of a lack of space, then why isn't allocating?
Finally, this results it crashing in the end anyway. After going through the render function once it crashes. If I removed all my fixes and just caught when realloc doesn't allocate it would work fine. Which begs the question, what is wrong with mallocing my array instead of reallocing to cause so problems further down the line?
My Array's are pointer of pointers of floats. When I grow the array it is converted into a pointer to floats and reallocated. I am building on Android, so this is why I assumed there to be a lack of memory.

Judging from all the different bits of information (realloc not finding memory, memcpy behaving unexpectedly, crashes) this sounds much like a heap corruption. Without some code samples of exactly what you're doing it's hard to say for sure but it appears that you're mis-managing the memory at some point, causing the heap to get into an invalid state.
Are you able to compile your code on an alternate platform such as Linux (you might have to stub some android specific APIs)? If so, you could see what happens on that platform and/or use valgrind to help hunt it down.
Finally, as you have this tagged C++ why are you using malloc/realloc instead of, for example, vector (or another standard container) or new?

You are confusing size and pointer types. In the memory allocation, size is the number of bytes, and you are converting the pointer type to float *, essentially creating an array of float of size size / sizeof(float). In the memcpy-equivalent code, you are treating the array as float ** and copying size of them. This will trash the heap, assuming that sizeof(float *) > 1, and is likely the source of later problems.
Moreover, if you are copying, say, a 100-size array to a 200-size array, you need to copy over 100 elements, not 200. Copying beyond the end of an array (which is what you're doing) can lead to program crashes.
A dynamically allocated array of pointers to floats will be of type float **, not float *, and certainly not a mixture of the two. The size of the array is the number of bytes to malloc and friends, and the number of elements in all array operations.
memcpy will faithfully copy bytes, assuming the source and destination blocks don't overlap (and separately allocated memory blocks don't). However, you've specified size - 1 for the number of bytes copied, when the number copied should be the exact byte size of the old array. (Where are you getting bad pointer values anyway? If it's in the expanded part of the array, you're copying garbage in there anyway.) If memcpy is giving you nonsense, it's getting nonsense to begin with, and it isn't your problem.

And btw, you don't need to test if array is NULL
You can replace
if (!array)
{
tmpArray = (float *)malloc(size);
}
else
{
tmpArray = (float *)realloc((void *)array, size);
}
by
tmpArray = realloc(array, size*sizeof (float));
realloc acts like malloc when given a NULLpointer.
Another thing, be careful that size is not 0, as realloc with 0 as size is the same as
free.
Third point, do not typecast pointers when not strictly necessary. You typecasted the return of the allocation functions, it's considered bad practice since ANSI-C. It's mandatory in C++, but as you're using the C allocation you're obviously not in C++ (in that case you should use new/delete).
Casting the array variable to (void *) is also unecessary as it could hide some warnings if your parameter was falsely declared (it could be an int or a pointer to pointer and by casting you would have suppressed the warning).

Related

Giving the size to an array with a variable

I've been trying to give an array its size through a variable, but it's not working because "must have a constant value".
int processes[] = { 1, 2, 3 };
int n = sizeof processes / sizeof processes[0];
...
findavgTime(processes, n);
-------------------------------------------------
void findavgTime(int processes[], int n, int bt[])
{
int wt[n], tat[n]; //These two vars are giving me the error
}
Am I missing something?
As you've seen, you can declare an array with a non-constant size like this. You could, however allocate it dynamically using new:
int* wt = new int[n];
int* tat = new int[n];
Just don't forget that you need to delete[] these arrays when you're done.
You can allocate it on the heap like how Mureinik said, but you can also allocate it on the stack (as you were trying to do) with alloca() like this:
int* wt = (int*)alloca(n * sizeof(int));
int* tat = (int*)alloca(n * sizeof(int));
free() should not be called as it is allocated on the stack, not the heap.
Some say using alloca() is bad practice because if the call causes a stack overflow, program behavior is undefined. This shouldn't be a problem as long as the array isn't too long.

How to convert a byte array of size 64 to a list of double values in Arduino C++?

void Manager::byteArrayToDoubleArray(byte ch[]) {
int counter = 0;
// temp array to break the byte array into size of 8 and read it
byte temp[64];
// double result values
double res[8];
int index = 0;
int size = (sizeof(ch) / sizeof(*ch));
for (int i = 0; i < size; i++) {
counter++;
temp[i] = ch[i];
if (counter % 8 == 0) {
res[index] = *reinterpret_cast<double * const>(temp);
index++;
counter = 0;
}
}
}
Here result would be a list of double values with count = 8.
Your problem is two things. You have some typos and misunderstanding. And the C++ standard is somewhat broken in this area.
I'll try to fix both.
First, a helper function called laundry_pods. It takes raw memory and "launders" it into an array of a type of your choice, so long as you pick a pod type:
template<class T, std::size_t N>
T* laundry_pods( void* ptr ) {
static_assert( std::is_pod<std::remove_cv_t<T>>{} );
char optimized_away[sizeof(T)*N];
std::memcpy( optimized_away, ptr , sizeof(T)*N );
T* r = ::new( ptr ) T[N];
assert( r == ptr );
std::memcpy( r, optimized_away, sizeof(T)*N );
return r;
}
now simply do
void Manager::byteArrayToDoubleArray(byte ch[]) {
double* pdouble = laundry_pods<double, 8>(ch);
}
and pdouble is a pointer to memory of ch interpreted as an array of 8 doubles. (It is not a copy of it, it interprets those bytes in-place).
While laundry_pods appears to copy the bytes around, both g++ and clang optimize it down into a binary noop. The seeming copying of bytes around is a way to get around aliasing restrictions and object lifetime rules in the C++ standard.
It relies on arrays of pod not having extra bookkeeping overhead (which C++ implementations are free to do; none do that I know of. That is what the non-static assert double-checks), but it returns a pointer to a real honest to goodness array of double. If you want to avoid that assumption, you could instead create each doulbe as a separate object. However, then they aren't an array, and pointer arithmetic over non-arrays is fraught as far as the standard is concerned.
The use of the term "launder" has to do with getting around aliasing and object lifetime requirements. The function does nothing at runtime, but in the C++ abstract machine it takes the memory and converts it into binary identical memory that is now a bunch of doubles.
The trick of doing this kind of "conversion" is to always cast the double* to a char* (or unsigned char or std::byte). Never the other way round.
You should be able to do something like this:
void byteArrayToDoubleArray(byte* in, std::size_t n, double* out)
{
for(auto out_bytes = (byte*) out; n--;)
*out_bytes++ = *in++;
}
// ...
byte ch[64];
// .. fill ch with double data somehow
double res[8];
byteArrayToDoubleArray(ch, 64, res);
Assuming that type byte is an alias of char or unsigned char or std::byte.
I am not completly sure what you are trying to achieve here because of the code (sizeof(ch) / sizeof(*ch)) which does not make sense for an array of undefined size.
If you have a byte-Array (POD data type; something like a typedef char byte;) then this most simple solution would be a reinterpret_cast:
double *result = reinterpret_cast<double*>(ch);
This allows you to use result[0]..result[7] as long as ch[] is valid and contains at least 64 bytes. Be aware that this construct does not generate code. It tells the compiler that result[0] corresponds to ch[0..7] and so on. An access to result[] will result in an access to ch[].
But you have to know the number of elements in ch[] to calculate the number of valid double elements in result.
If you need a copy (because - for example - the ch[] is a temporary array) you could use
std::vector<double> result(reinterpret_cast<double*>(ch), reinterpret_cast<double*>(ch) + itemsInCh * sizeof(*ch) / sizeof(double));
So if ch[] is an array with 64 items and a byte is really an 8-bit value, then
std::vector<double> result(reinterpret_cast<double*>(ch), reinterpet_cast<double*>(ch) + 8);
will provide a std::vector containing 8 double values.
There is another possible method using a union:
union ByteToDouble
{
byte b[64];
double d[8];
} byteToDouble;
the 8 double values will occupie the same memory as the 64 byte values. So you can write the byte values to byteToDouble.b[] and read the resultingdouble values from byteToDouble.d[].

Why don't my simple code work in C++?

Error in function realloc(): invalid pointer
int indZero = 0;
int *perZero=NULL;
int zero = 0;//Initialization
ProcessBit(zero,&indZero,&perZero);// Call function
void ProcessBit(int num,int *ind,int **mas)
{
mas=(int**)realloc(&mas,((*ind))*sizeof(int));// Error
mas[num-1]++;//it's correct line
}
A few problems:
The first argument to realloc is the original pointer (or NULL).
Your ProcessBit doesn't really emulate pass-by-reference correctly.
You can use a negative index.
mas is a pointer to a pointer to int, but you use it as a pointer to int.
A "fixed" version might look something like this:
void ProcessBit(int num, int *ind, int **mas)
{
int *temp = realloc(*mas, (*ind + 1) * sizeof(int));
if (temp == NULL)
{
// TODO: Handle error
// TODO: return or exit(EXIT_FAILURE)
}
*mas = temp;
(*mas)[*ind] = 0; // Initial initialization
if (num > 0)
{
(*mas)[num - 1]++;
}
++*ind; // Increase the size
}
Now, if this really was C++ (as you tagged your question) then you should be using std::vector instead, which would solve almost all your problems with much simpler code.
The parameters are wrong. Since you are trying to realloc a NULL pointer it should behave like malloc; however, the header declared in cstdlib is
void* realloc( void* ptr, std::size_t new_size );
The formal parameter mas is already the address of the pointer, so the call should be
*mas=(int*)realloc(*mas,((*ind))*sizeof(int));
(*mas)[num-1]++;
Since realloc handles and returns pointers by copy, not by reference.
You are passing the address of the memory location where the address of a memory location (NULL) is stored to your ProcessBit function, and then the address of that location to the realloc function. The function tries to reallocate memory where the variable mac is stored, on the stack. No wonder it's an invalid pointer.
By passing &mac you are simply taking a step in the wrong direction while dereferencing pointers.

Allocating an array of aligned struct

I'm trying to allocate an array of struct and I want each struct to be aligned to 64 bytes.
I tried this (it's for Windows only for now), but it doesn't work (I tried with VS2012 and VS2013):
struct __declspec(align(64)) A
{
std::vector<int> v;
A()
{
assert(sizeof(A) == 64);
assert((size_t)this % 64 == 0);
}
void* operator new[] (size_t size)
{
void* ptr = _aligned_malloc(size, 64);
assert((size_t)ptr % 64 == 0);
return ptr;
}
void operator delete[] (void* p)
{
_aligned_free(p);
}
};
int main(int argc, char* argv[])
{
A* arr = new A[200];
return 0;
}
The assert ((size_t)this % 64 == 0) breaks (the modulo returns 16). It looks like it works if the struct only contains simple types though, but breaks when it contains an std container (or some other std classes).
Am I doing something wrong? Is there a way of doing this properly? (Preferably c++03 compatible, but any solution that works in VS2012 is fine).
Edit:
As hinted by Shokwav, this works:
A* arr = (A*)new std::aligned_storage<sizeof(A), 64>::type[200];
// this works too actually:
//A* arr = (A*)_aligned_malloc(sizeof(A) * 200, 64);
for (int i=0; i<200; ++i)
new (&arr[i]) A();
So it looks like it's related to the use of new[]... I'm very curious if anybody has an explanation.
I wonder why you need such a huge alignment requirement, moreover to store a dynamic heap allocated object in the struct. But you can do this:
struct __declspec(align(64)) A
{
unsigned char ___padding[64 - sizeof(std::vector<int>)];
std::vector<int> v;
void* operator new[] (size_t size)
{
// Make sure the buffer will fit even in the worst case
unsigned char* ptr = (unsigned char*)malloc(size + 63);
// Find out the next aligned position in the buffer
unsigned char* endptr = (unsigned char*)(((intptr_t)ptr + 63) & ~63ULL);
// Also store the misalignment in the first padding of the structure
unsigned char misalign = (unsigned char)(endptr - ptr);
*endptr = misalign;
return endptr;
}
void operator delete[] (void* p)
{
unsigned char * ptr = (unsigned char*)p;
// It's required to call back with the original pointer, so subtract the misalignment offset
ptr -= *ptr;
free(ptr);
}
};
int main()
{
A * a = new A[2];
printf("%p - %p = %d\n", &a[1], &a[0], int((char*)&a[1] - (char*)&a[0]));
return 0;
}
I did not have your align_malloc and free function, so the implementation I'm providing is doing this:
It allocates larger to make sure it will fit in 64-bytes boundaries
It computes the offset from the allocation to the closest 64-bytes boundary
It stores the "offset" in the padding of the first structure (else I would have required a larger allocation space each time)
This is used to compute back the original pointer to the free()
Outputs:
0x7fff57b1ca40 - 0x7fff57b1ca00 = 64
Warning: If there is no padding in your structure, then the scheme above will corrupt data, since I'll be storing the misalignement offset in a place that'll be overwritten by the constructor of the internal members.
Remember that when you do "new X[n]", "n" has to be stored "somewhere" so when calling delete[], "n" calls to the destructors will be done. Usually, it's stored before the returned memory buffer (new will likely allocate the required size + 4 for storing the number of elements). The scheme here avoid this.
Another warning: Because C++ calls this operator with some additional padding included in the size for storing the array's number of elements, you'll might still get a "shift" in the returned pointer address for your objects. You might need to account for it. This is what the std::align does, it takes the extra space, compute the alignment like I did and return the aligned pointer. However, you can not get both done in the new[] overload, because of the "count storage" shift that happens after returning from new(). However, you can figure out the "count storage" space once by a single allocation, and adjust the offset accordingly in the new[] implementation.

Freeing memory allocated to an array of void pointers

I am declaring an array of void pointers. Each of which points to a value of arbitary type.
void **values; // Array of void pointers to each value of arbitary type
Initializing values as follows:
values = (void**)calloc(3,sizeof(void*));
//can initialize values as: values = new void* [3];
int ival = 1;
float fval = 2.0;
char* str = "word";
values[0] = (void*)new int(ival);
values[1] = (void*)new float(fval);
values[2] = (void*)str;
//Trying to Clear the memory allocated
free(*values);
//Error: *** glibc detected *** simpleSQL: free(): invalid pointer: 0x080611b4
//Core dumped
delete[] values*;
//warning: deleting 'void*' is undefined
//Similar Error.
Now how do I free/delete the memory allocated for values ( the array of void pointers)?
I suspect the issue is with the way that you allocated values: values = (void*)calloc(3,sizeof(void)). That should be sizeof(void *) rather than just sizeof(void).
sizeof(void) may be zero or something else that makes no sense, so you're not really allocating any memory to begin with... it's just dumb luck that the assignments work, and then the error pops up when you try to deallocate the memory.
EDIT: You're also asking for trouble by alternating between C++-style new/delete with C-style malloc/free. It is okay to use them both as long as you don't delete something you malloc'ed or free something you new'ed, but you're going to mix them up in your head if you go like this.
You have 3 things that are dynamically allocated that need to be freed in 2 different ways:
delete reinterpret_cast<int*>( values[0]);
delete reinterpret_cast<float*>( values[1]);
free( values); // I'm not sure why this would have failed in your example,
// but it would have leaked the 2 items that you allocated
// with new
Note that since str is not dynamically allocated it should not (actually cannot) be freed.
A couple of notes:
I'm assuming that the sizeof(void)
was meant to be sizeof(void*)
since what you have won't compile
I'm not going to say anything about
your seemingly random casting except
that it looks like code that ready
for disaster in general
This is the perfect situation for the boost::any class
Also you may want to consider using a vector rather than allocating your own memory.
std::vector<boost::any> data;
boost::any i1 = 1; // add integer
data.push_back(i1);
boost::any f1 = 1.0; // add double
data.push_back(f1);
data.push_back("PLOP"); // add a char *
std:: cout << boost::any_cast<int>(data[0]) + boost::any_cast<double>(data[1])
<< std::endl;
Going back to your original code the main problem was:
values = (void*)calloc(3,sizeof(void));
// This should have been
void** values = (void**)calloc(3,sizeof(void*));
// Freeing the members needs care as you need to cast them
// back to the correct type before you release the memory.
// now you can free the array with
free(values);
Also note: Though it is not illegal to use both new/delete and calloc/free in the same piece of code it is frowned upon. Mainly because it is easy to get things mixed up and that could potentially be fatal.
You're mixing new and *alloc(). That's a no-no, and can lead to undefined results.
I'm not sure why you are using new if you're doing things in C (referencing the tag here).
I would malloc the individual pieces of the array I need and then free them when I'm done I suppose. You can't free something you didn't first malloc. You also can't delete a void pointer.
Note that you're also not deleting values[0] and values[1] which is a memory leak, Yet by your design you can't free values[2] since its a pointer into you .data section.
You'd have to keep track of how many void* were originally calloc'd, and iterate over them, free-ing each one, then free the original values variable.
darn formatting... (the preview is working fine).
int ct = 3;
values = (void*)calloc(ct,sizeof(void));
//can initialize values as: values = new void* [3];
int ival = 1;
float fval = 2.0;
char* str = "word";
values[0] = (void*)new int(ival);
values[1] = (void*)new float(fval);
values[2] = (void*)str;
for ( int i = 0; i < ct; i++ ) [
delete( values[i] );
}
free( values );