MSVC Access Violation when setting array elements - c++

Question

I have been struggling in finding an explanation to an error I get in the following code:
#include <stdlib.h>
int main() {
int m=65536;
int n=65536;
float *a;
a = (float *)malloc(m*n*sizeof(float));
for (int i = 0; i < m; i++){
for (int j = 0; j < n; j++){
a[i*n + j] = 0;
}
}
return 0;
}
Why do I get an "Access Violation" Error when executing this program?
The memory allocation is succesful, the problem is in the nested for loops at some iteration count. I tried with a smaller value of m&n and the program works.
Does this mean I ran out of memory?

Answer 1

The problem is that m*n*sizeof(float) is likely an overflow, resulting in a relatively small value. Thus the malloc works, but it does not allocate as much memory as you're expecting and so you run off the end of the buffer.
Specifically, if your ints are 32 bits wide (which is common), then 65336 * 65336 is already an overflow, because you would need at least 33 bits to represent it. Signed integer overflows in C++ (and I believe in C) result in undefined behavior, but a common result is that the most significant bits are lopped off, and you're left with the lower ones. In your case, that gives 0. That's then multiplied by sizeof(float), but zero times anything is still zero.
So you've tried to allocate 0 bytes. It turns out that malloc will let you do that, and it will give back a valid pointer rather than a null pointer (which is what you'd get if the allocation failed). (See Edit below.)
So you have a valid pointer, but it's not valid to dereference it. That fact that you are able to dereference it at all is a side-effect of the implementation: In order to generate a unique address that doesn't get reused, which is what malloc is required to do when you ask for 0 bytes, malloc probably allocated a small-but-non-zero number of bytes. When you try to reference far enough beyond those, you'll typically get an access violation.
EDIT:
It turns out that what malloc does when requesting 0 bytes may depend on whether you're using C or C++. In the old days, the C standard required a malloc of 0 bytes to return a unique pointer as a way of generating "special" pointer values. In modern C++, a malloc of 0 bytes is undefined (see Footnote 35 in Section 3.7.4.1 of the C++11 standard). I hadn't realized malloc's API had changed in this way when I originally wrote the answer. (I love it when a newbie question causes me to learn something new.) VC++2013 appears to preserve the older behavior (returning a unique pointer for an allocation of 0 bytes), even when compiling for C++.

Answer 2

You are victim of 2 problems.
First the size calculation:
As some people have pointned out, you are exceeding the range of size_t. You can verify the size that you are trying to allocate with this code:
cout << "Max size_t is: " << SIZE_MAX<<endl;
cout << "Max int is : " << INT_MAX<<endl;
long long lsz = static_cast<long long>(m)*n*sizeof(float); // long long to see theoretical result
size_t sz = m*n*sizeof(float); // real result with overflow as will be used by malloc
cout << "Expected size: " << lsz << endl;
cout << "Requested size_t:" << sz << endl;
You'll be surprised but with MSVC13, you are asking 0 bytes because of the overflow (!!). You might get another number with a different compiler (resulting in a lower than expected size).
Second, malloc() might return a problem pointer:
The call for malloc() could appear as successfull because it does not return nullptr. The allocated memory could be smaller than expected. And even requesting 0 bytes might appear as successfull, as documented here: If size is zero, the return value depends on the particular library implementation (it may or may not be a null pointer), but the returned pointer shall not be dereferenced.
float *a = reinterpret_cast<float*>(malloc(m*n*sizeof(float))); // prefer casts in future
if (a == nullptr)
cout << "Big trouble !"; // will not be called
Alternatives
If you absolutely want to use C, prefer calloc(), you'll get at least a null pointer, because the function notices that you'll have an overflow:
float *b = reinterpret_cast<float*>(calloc(m,n*sizeof(float)));
But a better approach would be to use the operator new[]:
float *c = new (std::nothrow) float[m*n]; // this is the C++ way to do it
if (c == nullptr)
cout << "new Big trouble !";
else {
cout << "\nnew Array: " << c << endl;
c[n*m-1] = 3.0; // check that last elements are accessible
}
Edit:
It's also subject to the size_t limit.
Edit 2:
new[] throws bad_alloc exceptions when there is a problem, or even bad_array_new_length. You could try/catch these if you want. But if you prefer to get nullptr when there's not enough memory, you have to use (std::nothrow) as pointed out in the comments by Beat.
The best approach for your case, if you really need these huge number of floats, would be to go for vectors. As they are also subject to size_t limitation, but as you have in fact a 2D array, you could use vectors of vectors (if you have enough memory):
vector <vector<float>> v (n, vector<float>(m));