This question already has answers here:
Segmentation fault on large array sizes
(7 answers)
Closed 9 years ago.
I am developing a code in which there is need for declaring array of double of size 150000 and when one array is declared code is running successfully.if we declare two arrays then while execution it terminates throwing exception.
Code is :
double a[150000];
double b[150000];
if we declare a only then it executes perfectly.if declare both a and b then it terminates.
Can anyone suggest how to resolve this?
The two arrays are overflowing the stack (assuming they are local variables). Dynamically allocate memory for the arrays instead, using a std::vector to manage the memory for you:
std::vector<double> a(150000);
std::vector<double> b(150000);
Even though the std::vector instances are on the stack, the std::vector dynamically allocates memory internally for the data which is on the heap, avoiding the stack overflow.
Okay! You have Stack Overflow in your app!
Fixing examples:
don't use stack - use dynamic memory allocation (heap):
double* a = new double[150000];
use STL container, for example, vector - internally it allocates things on heap
std::vector<double> a(150000);
increase stack size (bad idea, but if you really need it read you compiler docs, and look here)
redesign you code somehow
There is one solution to this problem, but it leads to (at least) three different follow-on solutions. The solution is "don't use large arrays as local variables, because it blows up the stack".
The solution clearly means changing the code in some way. There are a few different ways to do that.
The obvious and straight-forwards solution is to use std::vector<double> instead.
Another solution is to use `
unique_ptr<double[]> a = std::unique_ptr<double[]>(new double[150000]);
The third, and SOMETIMES a good solutions, is to make a and b global variables.
There are several other variants, but they are generally variations on the same theme, just with slight variations. What is best in your case really depends on what the rest of your code is doing. I'd start with std::vector<double>, but other alternatives do exist, should that be an unsuitable solution for some reason.
Related
I am using Dev C++ to write a simulation program. For it, I need to declare a single dimensional array with the data type double. It contains 4200000 elements - like double n[4200000].
The compiler shows no error, but the program exits on execution. I have checked, and the program executes just fine for an array having 5000 elements.
Now, I know that declaring such a large array on the stack is not recommended. However, the thing is that the simulation requires me to call specific elements from the array multiple times - for example, I might need the value of n[234] or n[46664] for a given calculation. Therefore, I need an array in which it is easier to sift through elements.
Is there a way I can declare this array on the stack?
No there is no(we'll say "reasonable") way to declare this array on the stack. You can however declare the pointer on the stack, and set aside a bit of memory on the heap.
double *n = new double[4200000];
accessing n[234] of this, should be no quicker than accessing n[234] of an array that you declared like this:
double n[500];
Or even better, you could use vectors
std::vector<int> someElements(4200000);
someElements[234];//Is equally fast as our n[234] from other examples, if you optimize (-O3) and the difference on small programs is negligible if you don't(+5%)
Which if you optimize with -O3, is just as fast as an array, and much safer. As with the
double *n = new double[4200000];
solution you will leak memory unless you do this:
delete[] n;
And with exceptions and various things, this is a very unsafe way of doing things.
You can increase your stack size. Try adding these options to your link flags:
-Wl,--stack,36000000
It might be too large though (I'm not sure if Windows places an upper limit on stack size.) In reality though, you shouldn't do that even if it works. Use dynamic memory allocation, as pointed out in the other answers.
(Weird, writing an answer and hoping it won't get accepted... :-P)
Yes, you can declare this array on the stack (with a little extra work), but it is not wise.
There is no justifiable reason why the array has to live on the stack.
The overhead of dynamically allocating a single array once is neglegible (you could say "zero"), and a smart pointer will safely take care of not leaking memory, if that is your concern.
Stack allocated memory is not in any way different from heap allocated memory (apart from some caching effects for small objects, but these do not apply here).
Insofar, just don't do it.
If you insist that you must allocate the array on the stack, you will need to reserve 32 megabytes of stack space first (preferrably a bit more). For that, using Dev-C++ (which presumes Windows+MingW) you will either need to set the reserved stack size for your executable using compiler flags such as -Wl,--stack,34000000 (this reserves somewhat more than 32MiB), or create a thread (which lets you specify a reserved stack size for that thread).
But really, again, just don't do that. There's nothing wrong with allocating a huge array dynamically.
Are there any reasons you want this on the stack specifically?
I'm asking because the following will give you a construct that can be used in a similar way (especially accessing values using array[index]), but it is a lot less limited in size (total max size depending on 32bit/64bit memory model and available memory (RAM and swap memory)) because it is allocated from the heap.
int arraysize= 4200000;
int *heaparray= new int[arraysize];
...
k= heaparray[456];
...
delete [] heaparray;
return;
This question already has answers here:
Closed 12 years ago.
Possible Duplicate:
Segmentation fault on large array sizes
Hi all
I am trying to create a very big array in the VS 2010 using C++.
When I try to create a array like below
int dp[4501][4501]
or
int dp[1000][1000]
It threw an exception "Stack Overflow"
Then I change it to:
int dp[100][100]
everything is fine.
So if I want to create a big array like above, what should I do?
Best Regards,
Use dynamic allocation or the STL. There was a recent thread about a very similar question. See this.
You should use dynamic allocation:
typedef std::vector<int> int_vector;
int_vector dp(10000);
A double array can be simulated by nesting arrays:
typedef std::vector<int_vector> int_double_vector;
int_double_vector dp(4501, int_vector(4501));
Put it on the heap.
If you want to avoid new[], or avoid using std::vector, make the array global. This will put the array on heap and stack overflow will not occur.
Text from Parashift faq : Why should I use container classes rather than simple arrays?
EDIT:
Take a look at stackoverflow threads:
When would you use an array rather than a vector/string?
Why use iterators instead of array indices?
Your stack has overflowed with too many bits. You must drain them. Preferably onto a heap of other bits. I suggest /F67108864. The /F stands for "F'ing hell why is the stack so small compared to the heap?". The 67108863 is arbitrary.
Your declaration looks a bit as if dp will be used as a matrix. In that case, a dedicated (dense) matrix class such as boost::numeric::ublas::matrix is the simplest solution, easier and more local than a vector of vectors. If the matrix is sparsely populated, use a sparse matrix class instead.
So if I want to create a big array
like above, what should I do?
Avoid using the stack for these cases (in other words, avoid creating arrays like these which aren't heap-allocated when working inside a function). Just to give you an idea, my thread-local stack is only 16 kilobytes large. 4501 * 4501 * 4 (assuming 4 bytes per int) = ~81 megabytes.
Consider something like this instead:
typedef vector<int> Row;
typedef vector<Row> Matrix;
Matrix dp(4501, Row(4501) );
If you want to create a 10x50 matrix:
Matrix dp(10, Row(50) );
You can use this just like your normal dp array had it not overflowed the stack. This one will be allocated and automatically deallocated to/from the heap so that you don't have to worry about stack overflow when using it.
dp[5][10] = 123;
Good luck!
[Edit] There are also matrix solutions in boost worth looking into but suggesting boost might be a bit premature given the nature of the topic.
I am using Dev C++ to write a simulation program. For it, I need to declare a single dimensional array with the data type double. It contains 4200000 elements - like double n[4200000].
The compiler shows no error, but the program exits on execution. I have checked, and the program executes just fine for an array having 5000 elements.
Now, I know that declaring such a large array on the stack is not recommended. However, the thing is that the simulation requires me to call specific elements from the array multiple times - for example, I might need the value of n[234] or n[46664] for a given calculation. Therefore, I need an array in which it is easier to sift through elements.
Is there a way I can declare this array on the stack?
No there is no(we'll say "reasonable") way to declare this array on the stack. You can however declare the pointer on the stack, and set aside a bit of memory on the heap.
double *n = new double[4200000];
accessing n[234] of this, should be no quicker than accessing n[234] of an array that you declared like this:
double n[500];
Or even better, you could use vectors
std::vector<int> someElements(4200000);
someElements[234];//Is equally fast as our n[234] from other examples, if you optimize (-O3) and the difference on small programs is negligible if you don't(+5%)
Which if you optimize with -O3, is just as fast as an array, and much safer. As with the
double *n = new double[4200000];
solution you will leak memory unless you do this:
delete[] n;
And with exceptions and various things, this is a very unsafe way of doing things.
You can increase your stack size. Try adding these options to your link flags:
-Wl,--stack,36000000
It might be too large though (I'm not sure if Windows places an upper limit on stack size.) In reality though, you shouldn't do that even if it works. Use dynamic memory allocation, as pointed out in the other answers.
(Weird, writing an answer and hoping it won't get accepted... :-P)
Yes, you can declare this array on the stack (with a little extra work), but it is not wise.
There is no justifiable reason why the array has to live on the stack.
The overhead of dynamically allocating a single array once is neglegible (you could say "zero"), and a smart pointer will safely take care of not leaking memory, if that is your concern.
Stack allocated memory is not in any way different from heap allocated memory (apart from some caching effects for small objects, but these do not apply here).
Insofar, just don't do it.
If you insist that you must allocate the array on the stack, you will need to reserve 32 megabytes of stack space first (preferrably a bit more). For that, using Dev-C++ (which presumes Windows+MingW) you will either need to set the reserved stack size for your executable using compiler flags such as -Wl,--stack,34000000 (this reserves somewhat more than 32MiB), or create a thread (which lets you specify a reserved stack size for that thread).
But really, again, just don't do that. There's nothing wrong with allocating a huge array dynamically.
Are there any reasons you want this on the stack specifically?
I'm asking because the following will give you a construct that can be used in a similar way (especially accessing values using array[index]), but it is a lot less limited in size (total max size depending on 32bit/64bit memory model and available memory (RAM and swap memory)) because it is allocated from the heap.
int arraysize= 4200000;
int *heaparray= new int[arraysize];
...
k= heaparray[456];
...
delete [] heaparray;
return;
While solving a DP related problem, I observed that first works but the second seg faults .
What is the actual reason and what is the memory limit for just using int ?
int main(){
static int a[3160][3160];
return 0;
}
int main(){
int a[3160][3160];
return 0;
}
Because you probably don't have enough stack memory to store that big array.
Second Example creates an array on stack, while the First example creates an array which is not located on stack but somewhere in the data/Bss segment, since you explicitly specify the storage criteria using static qualifier.
Note that c++ standard does not specify stack or heap or data segment or Bss segment these are all implementation defined details. The standard only specify's behavior expected of variables declared with different storage criteria. So, where the variables are actually created is implementation defined but one thing for sure is, both your examples will create the arrays in different memory regions and the second one crashes because there is not enough memory in that region.
Also, probably if you are creating an array of such huge dimensions in actual implementation your design seems flawed and you might want to consider revisiting it.
You might also want to consider using std::array or std::vector, instead of the traditional c-style arrays.
A stack allocation that large is not safe (unless you were to fulfill that guarantee).
The stack size varies by platform/hardware. Therefore, the 'memory limit' varies dramatically. If you use huge stack arrays like this, be prepared to see this error often when your program is run on an processor other than the one you use for development. If you absolutely need a stack that large, you must create your own threads with explicit stack sizes.
However, that measure is not needed because you should just use a dynamic allocation here.
static is not a good choice if you need it to be reentrant.
As Als noted (+1) - the reason for the runtime error is very likely the stack size.
This question already has answers here:
Closed 12 years ago.
Possible Duplicate:
Segmentation fault on large array sizes
Hi all
I am trying to create a very big array in the VS 2010 using C++.
When I try to create a array like below
int dp[4501][4501]
or
int dp[1000][1000]
It threw an exception "Stack Overflow"
Then I change it to:
int dp[100][100]
everything is fine.
So if I want to create a big array like above, what should I do?
Best Regards,
Use dynamic allocation or the STL. There was a recent thread about a very similar question. See this.
You should use dynamic allocation:
typedef std::vector<int> int_vector;
int_vector dp(10000);
A double array can be simulated by nesting arrays:
typedef std::vector<int_vector> int_double_vector;
int_double_vector dp(4501, int_vector(4501));
Put it on the heap.
If you want to avoid new[], or avoid using std::vector, make the array global. This will put the array on heap and stack overflow will not occur.
Text from Parashift faq : Why should I use container classes rather than simple arrays?
EDIT:
Take a look at stackoverflow threads:
When would you use an array rather than a vector/string?
Why use iterators instead of array indices?
Your stack has overflowed with too many bits. You must drain them. Preferably onto a heap of other bits. I suggest /F67108864. The /F stands for "F'ing hell why is the stack so small compared to the heap?". The 67108863 is arbitrary.
Your declaration looks a bit as if dp will be used as a matrix. In that case, a dedicated (dense) matrix class such as boost::numeric::ublas::matrix is the simplest solution, easier and more local than a vector of vectors. If the matrix is sparsely populated, use a sparse matrix class instead.
So if I want to create a big array
like above, what should I do?
Avoid using the stack for these cases (in other words, avoid creating arrays like these which aren't heap-allocated when working inside a function). Just to give you an idea, my thread-local stack is only 16 kilobytes large. 4501 * 4501 * 4 (assuming 4 bytes per int) = ~81 megabytes.
Consider something like this instead:
typedef vector<int> Row;
typedef vector<Row> Matrix;
Matrix dp(4501, Row(4501) );
If you want to create a 10x50 matrix:
Matrix dp(10, Row(50) );
You can use this just like your normal dp array had it not overflowed the stack. This one will be allocated and automatically deallocated to/from the heap so that you don't have to worry about stack overflow when using it.
dp[5][10] = 123;
Good luck!
[Edit] There are also matrix solutions in boost worth looking into but suggesting boost might be a bit premature given the nature of the topic.