While solving a DP related problem, I observed that first works but the second seg faults .
What is the actual reason and what is the memory limit for just using int ?
int main(){
static int a[3160][3160];
return 0;
}
int main(){
int a[3160][3160];
return 0;
}
Because you probably don't have enough stack memory to store that big array.
Second Example creates an array on stack, while the First example creates an array which is not located on stack but somewhere in the data/Bss segment, since you explicitly specify the storage criteria using static qualifier.
Note that c++ standard does not specify stack or heap or data segment or Bss segment these are all implementation defined details. The standard only specify's behavior expected of variables declared with different storage criteria. So, where the variables are actually created is implementation defined but one thing for sure is, both your examples will create the arrays in different memory regions and the second one crashes because there is not enough memory in that region.
Also, probably if you are creating an array of such huge dimensions in actual implementation your design seems flawed and you might want to consider revisiting it.
You might also want to consider using std::array or std::vector, instead of the traditional c-style arrays.
A stack allocation that large is not safe (unless you were to fulfill that guarantee).
The stack size varies by platform/hardware. Therefore, the 'memory limit' varies dramatically. If you use huge stack arrays like this, be prepared to see this error often when your program is run on an processor other than the one you use for development. If you absolutely need a stack that large, you must create your own threads with explicit stack sizes.
However, that measure is not needed because you should just use a dynamic allocation here.
static is not a good choice if you need it to be reentrant.
As Als noted (+1) - the reason for the runtime error is very likely the stack size.
Related
I am using Dev C++ to write a simulation program. For it, I need to declare a single dimensional array with the data type double. It contains 4200000 elements - like double n[4200000].
The compiler shows no error, but the program exits on execution. I have checked, and the program executes just fine for an array having 5000 elements.
Now, I know that declaring such a large array on the stack is not recommended. However, the thing is that the simulation requires me to call specific elements from the array multiple times - for example, I might need the value of n[234] or n[46664] for a given calculation. Therefore, I need an array in which it is easier to sift through elements.
Is there a way I can declare this array on the stack?
No there is no(we'll say "reasonable") way to declare this array on the stack. You can however declare the pointer on the stack, and set aside a bit of memory on the heap.
double *n = new double[4200000];
accessing n[234] of this, should be no quicker than accessing n[234] of an array that you declared like this:
double n[500];
Or even better, you could use vectors
std::vector<int> someElements(4200000);
someElements[234];//Is equally fast as our n[234] from other examples, if you optimize (-O3) and the difference on small programs is negligible if you don't(+5%)
Which if you optimize with -O3, is just as fast as an array, and much safer. As with the
double *n = new double[4200000];
solution you will leak memory unless you do this:
delete[] n;
And with exceptions and various things, this is a very unsafe way of doing things.
You can increase your stack size. Try adding these options to your link flags:
-Wl,--stack,36000000
It might be too large though (I'm not sure if Windows places an upper limit on stack size.) In reality though, you shouldn't do that even if it works. Use dynamic memory allocation, as pointed out in the other answers.
(Weird, writing an answer and hoping it won't get accepted... :-P)
Yes, you can declare this array on the stack (with a little extra work), but it is not wise.
There is no justifiable reason why the array has to live on the stack.
The overhead of dynamically allocating a single array once is neglegible (you could say "zero"), and a smart pointer will safely take care of not leaking memory, if that is your concern.
Stack allocated memory is not in any way different from heap allocated memory (apart from some caching effects for small objects, but these do not apply here).
Insofar, just don't do it.
If you insist that you must allocate the array on the stack, you will need to reserve 32 megabytes of stack space first (preferrably a bit more). For that, using Dev-C++ (which presumes Windows+MingW) you will either need to set the reserved stack size for your executable using compiler flags such as -Wl,--stack,34000000 (this reserves somewhat more than 32MiB), or create a thread (which lets you specify a reserved stack size for that thread).
But really, again, just don't do that. There's nothing wrong with allocating a huge array dynamically.
Are there any reasons you want this on the stack specifically?
I'm asking because the following will give you a construct that can be used in a similar way (especially accessing values using array[index]), but it is a lot less limited in size (total max size depending on 32bit/64bit memory model and available memory (RAM and swap memory)) because it is allocated from the heap.
int arraysize= 4200000;
int *heaparray= new int[arraysize];
...
k= heaparray[456];
...
delete [] heaparray;
return;
The following code is generating a stack overflow error for me
int main(int argc, char* argv[])
{
int sieve[2000000];
return 0;
}
How do I get around this? I am using Turbo C++ but would like to keep my code in C
EDIT:
Thanks for the advice. The code above was only for example, I actually declare the array in a function and not in sub main. Also, I needed the array to be initialized to zeros, so when I googled malloc, I discovered that calloc was perfect for my purposes.
Malloc/calloc also has the advantage over allocating on the stack of allowing me to declare the size using a variable.
Your array is way too big to fit into the stack, consider using the heap:
int *sieve = malloc(2000000 * sizeof(*sieve));
If you really want to change the stack size, take a look at this document.
Tip: - Don't forget to free your dynamically allocated memory when it's no-longer needed.
There are 3 ways:
Allocate array on heap - use malloc(), as other posters suggested. Do not forget to free() it (although for main() it is not that important - OS will clean up memory for you on program termination).
Declare the array on unit level - it will be allocated in data segment and visible for everybody (adding static to declaration will limit the visibility to unit).
Declare your array as static - in this case it will be allocated in data segment, but visible only in main().
That's about 7MB of stack space. In visual studio you would use /STACK:###,### to reflect the size you want. If you truely want a huge stack (could be a good reason, using LISP or something :), even the heap is limited to small'sh allocations before forcing you to use VirtualAlloc), you may also want to set your PE to build with /LARGEADDRESSAAWARE (Visual Studio's linker again), but this configure's your PE header to allow your compiled binary to address the full 4GB of 32'bit address space (if running in a WOW64). If building truely massive binaries, you would also typically need to configure /bigobj as an additional linker paramerter.
And if you still need more space, you can radically violate convention by using something simular to (again MSVC's link) /merge:, which will allow you to pack one section into another, so you can use every single byte for a single shared code/data section. Naturally you would also need to configure the SECTIONS permissions in a def file or with #pgrama.
Use malloc. All check the return type is not null, if it is null then your system simply doesn't have enought memory to fit that many values.
You would be better off allocating it on the heap, not the stack. something like
int main(int argc, char* argv[])
{
int * sieve;
sieve = malloc(20000);
return 0;
}
Your array is huge.
It's possible that your machine or OS don't have or want to allocate so much memory.
If you absolutely need an enormous array, you can try to allocate it dynamically (using malloc(...)), but then you're at risk of leaking memory. Don't forget to free the memory.
The advantage of malloc is that it tries to allocate memory on the heap instead of the stack (therefore you won't get a stack overflow).
You can check the value that malloc returns to see if the allocation succeeded or failed.
If it fails, just try to malloc a smaller array.
Another option would be to use a different data structure that can be resized on the fly (like a linked list). Wether this option is good depends on what you are going to do with the data.
Yet another option would be to store things in a file, streaming data on the fly. This approach is the slowest.
If you go for storage on the hard drive, you might as well use an existing library (for databases)
As Turbo C/C++ is 16 bit compiler int datatype consumes about 2 bytes.
2bytes*2000000=40,00,000 bytes=3.8147MB space.
The auto variables of a function is stored in stack and it caused the overflow of the stack memory. Instead use the data memory [using static or global variable] or the dynamic heap memory [using the malloc/calloc] for creating the required memory as per the availability of the processor memory mapping.
Is there some reason why you can't use alloca() to allocate the space you need on the stack frame based on how big the object really needs to be?
If you do that, and still bust the stack, put it in allocated heap. I highly recommend NOT declaring it as static in main() and putting it in the data segment.
If it really has to be that big and your program can't allocate it on the heap, your program really has no business running on that type of machine to begin with.
What (exactly) are you trying to accomplish?
The following code is generating a stack overflow error for me
int main(int argc, char* argv[])
{
int sieve[2000000];
return 0;
}
How do I get around this? I am using Turbo C++ but would like to keep my code in C
EDIT:
Thanks for the advice. The code above was only for example, I actually declare the array in a function and not in sub main. Also, I needed the array to be initialized to zeros, so when I googled malloc, I discovered that calloc was perfect for my purposes.
Malloc/calloc also has the advantage over allocating on the stack of allowing me to declare the size using a variable.
Your array is way too big to fit into the stack, consider using the heap:
int *sieve = malloc(2000000 * sizeof(*sieve));
If you really want to change the stack size, take a look at this document.
Tip: - Don't forget to free your dynamically allocated memory when it's no-longer needed.
There are 3 ways:
Allocate array on heap - use malloc(), as other posters suggested. Do not forget to free() it (although for main() it is not that important - OS will clean up memory for you on program termination).
Declare the array on unit level - it will be allocated in data segment and visible for everybody (adding static to declaration will limit the visibility to unit).
Declare your array as static - in this case it will be allocated in data segment, but visible only in main().
That's about 7MB of stack space. In visual studio you would use /STACK:###,### to reflect the size you want. If you truely want a huge stack (could be a good reason, using LISP or something :), even the heap is limited to small'sh allocations before forcing you to use VirtualAlloc), you may also want to set your PE to build with /LARGEADDRESSAAWARE (Visual Studio's linker again), but this configure's your PE header to allow your compiled binary to address the full 4GB of 32'bit address space (if running in a WOW64). If building truely massive binaries, you would also typically need to configure /bigobj as an additional linker paramerter.
And if you still need more space, you can radically violate convention by using something simular to (again MSVC's link) /merge:, which will allow you to pack one section into another, so you can use every single byte for a single shared code/data section. Naturally you would also need to configure the SECTIONS permissions in a def file or with #pgrama.
Use malloc. All check the return type is not null, if it is null then your system simply doesn't have enought memory to fit that many values.
You would be better off allocating it on the heap, not the stack. something like
int main(int argc, char* argv[])
{
int * sieve;
sieve = malloc(20000);
return 0;
}
Your array is huge.
It's possible that your machine or OS don't have or want to allocate so much memory.
If you absolutely need an enormous array, you can try to allocate it dynamically (using malloc(...)), but then you're at risk of leaking memory. Don't forget to free the memory.
The advantage of malloc is that it tries to allocate memory on the heap instead of the stack (therefore you won't get a stack overflow).
You can check the value that malloc returns to see if the allocation succeeded or failed.
If it fails, just try to malloc a smaller array.
Another option would be to use a different data structure that can be resized on the fly (like a linked list). Wether this option is good depends on what you are going to do with the data.
Yet another option would be to store things in a file, streaming data on the fly. This approach is the slowest.
If you go for storage on the hard drive, you might as well use an existing library (for databases)
As Turbo C/C++ is 16 bit compiler int datatype consumes about 2 bytes.
2bytes*2000000=40,00,000 bytes=3.8147MB space.
The auto variables of a function is stored in stack and it caused the overflow of the stack memory. Instead use the data memory [using static or global variable] or the dynamic heap memory [using the malloc/calloc] for creating the required memory as per the availability of the processor memory mapping.
Is there some reason why you can't use alloca() to allocate the space you need on the stack frame based on how big the object really needs to be?
If you do that, and still bust the stack, put it in allocated heap. I highly recommend NOT declaring it as static in main() and putting it in the data segment.
If it really has to be that big and your program can't allocate it on the heap, your program really has no business running on that type of machine to begin with.
What (exactly) are you trying to accomplish?
So I can fix this manually so it isn't an urgent question but I thought it was really strange:
Here is the entirety of my code before the weird thing that happens:
int main(int argc, char** arg) {
int memory[100];
int loadCounter = 0;
bool getInput = true;
print_memory(memory);
and then some other unrelated stuff.
The print memory just prints the array which should've initialized to all zero's but instead the first few numbers are:
+1606636544 +32767 +1606418432 +32767 +1856227894 +1212071026 +1790564758 +813168429 +0000 +0000
(the plus and the filler zeros are just for formatting since all the numbers are supposed to be from 0-1000 once the array is filled. The rest of the list is zeros)
It also isn't memory leaking because I tried initializing a different array variable and on the first run it also gave me a ton of weird numbers. Why is this happening?
Since you asked "What do C++ arrays init to?", the answer is they init to whatever happens to be in the memory they have been allocated at the time they come into scope.
I.e. they are not initialized.
Do note that some compilers will initialize stack variables to zero in debug builds; this can lead to nasty, randomly occurring issues once you start doing release builds.
The array you are using is stack allocated:
int memory[100];
When the particular function scope exits (In this case main) or returns, the memory will be reclaimed and it will not leak. This is how stack allocated memory works. In this case you allocated 100 integers (32 bits each on my compiler) on the stack as opposed to on the heap. A heap allocation is just somewhere else in memory hopefully far far away from the stack. Anyways, heap allocated memory has a chance for leaking. Low level Plain Old Data allocated on the stack (like you wrote in your code) won't leak.
The reason you got random values in your function was probably because you didn't initialize the data in the 'memory' array of integers. In release mode the application or the C runtime (in windows at least) will not take care of initializing that memory to a known base value. So the memory that is in the array is memory left over from last time the stack was using that memory. It could be a few milli-seconds old (most likely) to a few seconds old (less likely) to a few minutes old (way less likely). Anyways, it's considered garbage memory and it's to be avoided at all costs.
The problem is we don't know what is in your function called print_memory. But if that function doesn't alter the memory in any ways, than that would explain why you are getting seemingly random values. You need to initialize those values to something first before using them. I like to declare my stack based buffers like this:
int memory[100] = {0};
That's a shortcut for the compiler to fill the entire array with zero's.
It works for strings and any other basic data type too:
char MyName[100] = {0};
float NoMoney[100] = {0};
Not sure what compiler you are using, but if you are using a microsoft compiler with visual studio you should be just fine.
In addition to other answers, consider this: What is an array?
In managed languages, such as Java or C#, you work with high-level abstractions. C and C++ don't provide abstractions (I mean hardware abstractions, not language abstractions like OO features). They are dessigned to work close to metal that is, the language uses the hardware directly (Memory in this case) without abstractions.
That means when you declare a local variable, int a for example, what the compiler does is to say "Ok, im going to interpret the chunk of memory [A,A + sizeof(int)] as an integer, which I call 'a'" (Where A is the offset between the beginning of that chunk and the start address of function's stack frame).
As you can see, the compiler only "assigns" memory-segments to variables. It does not do any "magic", like "creating" variables. You have to understand that your code is executed in a machine, and the machine has only a memory and a CPU. There is no magic.
So what is the value of a variable when the function execution starts? The value represented with the data which the chunk of memory of the variable has. Commonly, that data has no sense from our current point of view (Could be part of the data used previously by a string, for example), so when you access that variable you get extrange values. Thats what we call "garbage": Data previously written which has no sense in our context.
The same applies to an array: An array is only a bigger chunk of memory, with enough space to fit all the values of the array: [A,A + (length of the array)*sizeof(type of array elements)]. So as in the variable case, the memory contains garbage.
Commonly you want to initialize an array with a set of values during its declaration. You could achieve that using an initialiser list:
int array[] = {1,2,3,4};
In that case, the compiler adds code to the function to initialize the memory-chunk which the array is with that values.
Sidenote: Non-POD types and static storage
The things explained above only applies to POD types such as basic types and arrays of basic types. With non-POD types like classes the compiler adds calls to the constructor of the variables, which are designed to initialise the values (attributes) of a class instance.
In addition, even if you use POD types, if variables have static storage specification, the compiler initializes its memory with a default value, because static variables are allocated at program start.
the local variable on stack is not initialized in c/c++. c/c++ is designed to be fast so it doesn't zero stack on function calls.
Before main() runs, the language runtime sets up the environment. Exactly what it's doing you'd have to discover by breaking at the load module's entry point and watching the stack pointer, but at any rate your stack space on entering main is not guaranteed clean.
Anything that needs clean stack or malloc or new space gets to clean it itself. Plenty of things don't. C[++] isn't in the business of doing unnecessary things. In C++ a class object can have non-trivial constructors that run implicitly, those guarantee the object's set up for use, but arrays and plain scalars don't have constructors, if you want an inital value you have to declare an initializer.
I am using Dev C++ to write a simulation program. For it, I need to declare a single dimensional array with the data type double. It contains 4200000 elements - like double n[4200000].
The compiler shows no error, but the program exits on execution. I have checked, and the program executes just fine for an array having 5000 elements.
Now, I know that declaring such a large array on the stack is not recommended. However, the thing is that the simulation requires me to call specific elements from the array multiple times - for example, I might need the value of n[234] or n[46664] for a given calculation. Therefore, I need an array in which it is easier to sift through elements.
Is there a way I can declare this array on the stack?
No there is no(we'll say "reasonable") way to declare this array on the stack. You can however declare the pointer on the stack, and set aside a bit of memory on the heap.
double *n = new double[4200000];
accessing n[234] of this, should be no quicker than accessing n[234] of an array that you declared like this:
double n[500];
Or even better, you could use vectors
std::vector<int> someElements(4200000);
someElements[234];//Is equally fast as our n[234] from other examples, if you optimize (-O3) and the difference on small programs is negligible if you don't(+5%)
Which if you optimize with -O3, is just as fast as an array, and much safer. As with the
double *n = new double[4200000];
solution you will leak memory unless you do this:
delete[] n;
And with exceptions and various things, this is a very unsafe way of doing things.
You can increase your stack size. Try adding these options to your link flags:
-Wl,--stack,36000000
It might be too large though (I'm not sure if Windows places an upper limit on stack size.) In reality though, you shouldn't do that even if it works. Use dynamic memory allocation, as pointed out in the other answers.
(Weird, writing an answer and hoping it won't get accepted... :-P)
Yes, you can declare this array on the stack (with a little extra work), but it is not wise.
There is no justifiable reason why the array has to live on the stack.
The overhead of dynamically allocating a single array once is neglegible (you could say "zero"), and a smart pointer will safely take care of not leaking memory, if that is your concern.
Stack allocated memory is not in any way different from heap allocated memory (apart from some caching effects for small objects, but these do not apply here).
Insofar, just don't do it.
If you insist that you must allocate the array on the stack, you will need to reserve 32 megabytes of stack space first (preferrably a bit more). For that, using Dev-C++ (which presumes Windows+MingW) you will either need to set the reserved stack size for your executable using compiler flags such as -Wl,--stack,34000000 (this reserves somewhat more than 32MiB), or create a thread (which lets you specify a reserved stack size for that thread).
But really, again, just don't do that. There's nothing wrong with allocating a huge array dynamically.
Are there any reasons you want this on the stack specifically?
I'm asking because the following will give you a construct that can be used in a similar way (especially accessing values using array[index]), but it is a lot less limited in size (total max size depending on 32bit/64bit memory model and available memory (RAM and swap memory)) because it is allocated from the heap.
int arraysize= 4200000;
int *heaparray= new int[arraysize];
...
k= heaparray[456];
...
delete [] heaparray;
return;