Related
This question already has answers here:
At what exact moment is a local variable allocated storage?
(5 answers)
Closed 6 years ago.
Even in C (not just C++) you can declare variables at the start of a code block, which is enclosed in curly braces.
Example:
#include <stdio.h>
void use_stack(int cnt)
{
if (cnt<=16) {
int a[16];
int i;
a[0]=3;
a[1]=5;
for (i=2;i<cnt;i++) {
a[i]=a[i-1]+a[i-2];
}
printf("a[%d] == %d\n",cnt-1,a[cnt-1]);
}
else {
printf("cnt is too big\n");
}
}
Now I know that variables like the array a[16] are allocated on the stack in this case.
I was wondering if the space for this array is allocated at the start of the function (first opening curly brace) or at the start of the block where it is declared (opening curly brace after if).
From examining the assembler code it seems the compiler allocates the space for a[16] directly at the entry of the function.
I actually expected that the stack would be allocated (stack pointer decreased) at the declaration of a[16] and that the stack would be de-allocated (stack pointer increased) at the end of the corresponding if code block.
But this does not happen it seems (stack for a[16] is allocated directly at function entry, even if a[16] is not used in the else branch).
Has anyone an explanation why this is the case ?
So is there any part of the C language standard, which explains this behavior, or does it have to do with things like "longjmp" or signal handling, which maybe require that the stack pointer is "constant" inside a function ?
Note: The reason why I assumed the stack would be allocated/deallocated at the start/end of the code block is, because in C++ the constructor/destructor of objects allocated on the stack will be called at the start/end of the code block. So if you examine the assembler code of a C++ program you will notice that the stack is still allocated at the function entry; just the constructor/destructor call will be done at the start/end of the code block.
I am explicitly interested why stack is not allocated/deallocated at the start/end of a code block using curly braces.
The question: At what exact moment is a local variable allocated storage? is only about a local variable allocated at the start of a function. I am surprised that stack allocation for variables allocated later inside a code block is also done at the function entry.
So far the answers have been:
Something to do with optimization
Might different for C, C++
Stack is not even mentioned in the C language specification
So: I am interested in the answer for C... (and I strongly believe that the answer will apply to C++ also, but I am not asking about C++ :-)).
Optimization: Here is an example which will directly demonstrate why I am so surprised and why I am quite sure that this is not an optimization:
#include <stdio.h>
static char *stackA;
static char *stackB;
static void calc(int c,int *array)
{
int result;
if (c<=0) {
// base case c<=0:
stackB=(char *)(&result);
printf("stack ptr calc() = %p\n",stackB);
if (array==NULL) {
printf("a[0] == 1\n");
} else {
array[0]=1;
}
return;
}
// here: c>0
if (array==NULL) {
// no array allocated, so allocate it now
int i;
int a[2500];
// calculate array entries recursively
calc(c-1,a);
// calculate current array entry a[c]
a[c]=a[c-1]+3;
// print full array
for(i=0;i<=c;i++) {
printf("a[%d] = %d\n",i,a[i]);
}
} else {
// array already allocated
calc(c-1,array);
// calculate current array entry a[c]
array[c]=array[c-1]+3;
}
}
int main()
{
int a;
stackA=(char *)(&a);
printf("stack ptr main() = %p\n",stackA);
calc(9,NULL);
printf("used stack = %d\n",(int)(stackA-stackB));
}
I am aware that this is an ugly program :-).
The function calc calculates n*3 + 1 for all 0<=n<=c in a recursive fashion.
If you look at the code for calc you notice that the array a[2500] is only declared when the input parameter array to the function is NULL.
Now this only happens in the call to calc which is done in main.
The stackA and stackB pointers are used to calculate a rough estimate how much stack is used by this program.
Now: int a[2500] should consume around 10000 bytes (4 bytes per integer, 2500 entries). So you could expect that the whole program consumes around 10000 bytes of stack + something additional (for overhead when calc is called recursively).
But: It turns out this program consumes around 100000 bytes of stack (10 times as much as expected). The reason is, that for each call of calc the array a[2500] is allocated, even if it is only used in the first call. There are 10 calls to calc (0<=c<=9) and so you consume 100000 bytes of stack.
It does not matter if you compile the program with or without optimization
GCC-4.8.4 and clang for x64, MS Visual C++ 2010, Windriver for DIAB (for PowerPC) all exhibit this behavior
Even weirder: C99 introduces Variable Length Arrays. If I replace int a[2500]; in the above code with int a[2500+c]; then the program uses less stack space (around 90000 bytes less).
Note: If I only change the call to calc in main to calc(1000,NULL); the program will crash (stack overflow == segmentation fault). If I additionally change to int a[2500+c]; the program works and uses less than 100KB stack. I still would like to see an answer, which explains why a Variable Length Array does not lead to a stack overflow whereas a fixed length array does lead to a stack overflow, in particular since this fixed length array is out of scope (except for the first invocation of calc).
So what's the reason for this behavior in C ?
I do not believe that GCC/clang both simply are not able to do better; I strongly believe there has to be a technical reason for this. Any ideas ?
Answer by Google
After more googling: I strongly believe this has something to do with "setjmp/longjmp" behavior. Google for "Variable Length Array longjmp" and see for yourself. It seems longjmp is hard to implement if you do not allocate all arrays at function entry.
The language rules for automatic storage only guarantees that the last allocated is the first deallocated.
A compiler can implement this logical stack any way it sees fit.
If it can prove that a function isn't recursive it can even allocated the storage at program start-up.
I believe that the above applies to C as well as C++, but I'm no C expert.
Please, when you ask about the details of a programming language, limit the question to one language at a time.
There is no technical reason for this other than choices that compiler makers made. It's less generated code and faster executing code to always reserve all the stack space we'll need at the beginning of the function. So all the compilers made the same reasonable performance tradeoff.
Try using a variable length array and you'll see that the compiler is fully capable of generating code that "allocates" stack just for a block. Something like this:
void
foo(int sz, int x)
{
extern void bar(char *);
if (x) {
char a[sz];
bar(a);
} else {
char a[10];
bar(a);
}
}
My compiler generates code that always reserves stack space for the x is false part, but the space for the true part is only reserved if x is true.
How this is done isn't regulated by any standard. The C and C++ standards don't mention the stack at all, in theory those languages could be used even on computers that don't have a stack.
On computers that do have a stack, how this is done is specified by the ABI of the given system. Often, stack space is reserved at the point when the program enters the function. But compilers are free to optimize the code so that the stack space is only reserved when a certain variable is used.
At any rate, the point where you declare the variable has no relation to when it gets allocated. In your example, int a[16] is either allocated when the function is entered, or it is allocated just before the first place where it is used. It doesn't matter if a is declared inside the if statement or at the top of the function.
In C++ however, there is the concept of constructors. If your variable is an object with a constructor, then that constructor will get executed at the point where the variable is declared. Meaning that the variable must be allocated before that happens.
Alf has explained the limitations and freedoms that the standard specifies (or rather, what it doesn't specify).
I'll suggest an answer to the question
why stack is not allocated/deallocated at the start/end of a code block using curly braces.
The compilers that you tested chose to do so (I'm actually just guessing, I didn't write any of them), because of better run-time performance and simplicity (and of course, because it is allowed).
Allocating 96 bytes (arbitrary example) once takes about half as much time as allocating 48 bytes twice. And third as much times as allocating 32 bytes thrice.
Consider a loop as an extreme case:
for(int i = 0; i < 1000000; i++) {
int j;
}
If j is allocated at the start of the function, there is one allocation. If j is allocated within the loop body, then there will be a million allocations. Less allocations is better (faster).
Note: The reason why I assumed the stack would be allocated/deallocated at the start/end of the code block is, because in C++ the constructor/destructor of objects allocated on the stack will be called at the start/end of the code block.
Now you know that you were wrong to have assumed so. As written in a fine answer in the linked question, the allocation/dealocation need not coincide with the construction/destruction.
Imagine I would use a[100000]
That is approaching a very significant fraction of total stack space. You should allocate large blocks of memory like that dynamically.
I have a working function implementation in c that requires a large locally allocated chunk of memory as a working space. This function gets called a lot in succession where it is guaranteed that the required amount of working space does not change. To optimize the function I have refactored it to allocate a static single continuous piece of memory the first time it is called, that is only released when it is asked to. It looks something like this
void worker(struct* ptr, size_t m) {
static double *stack;
static size_t sz_stack;
static double *alpha;
static double *delta;
if (!ptr) {
if (stack) {
free(stack);
}
stack = NULL;
sz_stack = 0;
return;
}
if (!stack) {
sz_stack = 2*m;
stack = calloc(sz_stack, sizeof(*stack));
if (stack==NULL)
// Error and cleanup
alpha = stack;
delta = alpha + m;
}
// Do work using alpha and delta as arrays
return;
}
The caller can call this function successively where ptr will hold the final result as long as the problem size given by m does not change. When the caller is done with the function, or the problem size changes, he calls worker(NULL, 0); and the allocated memory will be freed.
I am now working on rewriting this codebase to c++ and as the best practices tell me I have used individual std::vector for alpha and delta instead of the contiguous stack. However, profiling revealed that there is a huge bottleneck as the std::vector containers are allocated and free'd each and every function call.
My question now is:
What is the modern c++ way to maintain a contiguous piece of working space for a function in between calls?
If it is guaranteed that the required working space will not be changing during contiguous function calls (as you mentioned), to me it seems the simplest solution would be to use a static array (somewhat similar to your C code, but using 'new' and 'delete[]' instead of 'calloc' and 'free').
The C++ way would be to have a class with a private array that you can then manage.
It seems to me that the way you are handling static memory is very analogous to a constructor and destructor. I would have the array as the sole private member and then your worker function as a public member function.
Also, in terms of performance, STL can do some strange things and each implementation can be more or less strange than the next. If you really want speed (as you've seen), sometimes you have to handle things yourself.
static is a dreadful thing because it plays really badly with thread safety and is wholly unnecessary.
The modern way is one of the following:
Declare the memory further up on the stack. vector<> or array<> or even malloc if you like. Pass a pointer (or, equivalently, reference) to this memory into your function.
int main()
{
vector<double> storage;
while (1)
{
worker(&storage,0,0);
}
return 0;
}
Or, write your function as a member of a class. Declare the memory as a member of your class. Create one instance of your class, then call the function repeatedly:
struct oo_hack
{
void worker (struct* ptr, size_t m)
{
// Do some things using storage
}
vector<double> storage;
}
int main()
{
oo_hack myhack; // This is on the stack, but has a vector inside
while (1)
{
myhack.worker(0,0);
}
return 0;
} // memory gets freed here
I'd suggest declaring the memory further up on the stack and passing it into the functions, but you may prefer the second.
I have a quick question regarding the scope of dynamic arrays, which I assume is causing a bug in a program I'm writing. This snippet checks a function parameter and branches to either the first or the second, depending on what the user passes.
When I run the program, however, I get a scope related error:
error: ‘Array’ was not declared in this scope
Unless my knowledge of C++ fails me, I know that variables created within a conditional fall out of scope when when the branch is finished. However, I dynamically allocated these arrays, so I cannot understand why I can't manipulate the arrays later in the program, since the pointer should remain.
//Prepare to store integers
if (flag == 1) {
int *Array;
Array = new int[input.length()];
}
//Prepare to store chars
else if (flag == 2) {
char *Array;
Array = new char[input.length()];
}
Can anyone shed some light on this?
Declare Array before if. And you can't declare array of different types as one variable, so I think you should use to pointers.
int *char_array = nullptr;
int *int_array = nullptr;
//Prepare to store integers
if (flag == 1) {
int_array = new int[input.length()];
}
//Prepare to store chars
else if (flag == 2) {
char_array = new char[input.length()];
}
if (char_array)
{
//do something with char_array
}
else if (int_array)
{
//do something with int_array
}
Also as j_random_hacker points, you might want to change you program design to avoid lot's of if
While you are right that since you dynamically allocated them on the heap, the memory won't be released to the system until you explicitly delete it (or the program ends), the pointer to the memory falls out of scope when the block it was declared in exits. Therefore, your pointer(s) need to exist at a wider scope if they will be used after the block.
The memory remains allocated (i.e. taking up valuable space), there's just no way to access it after the closing }, because at that point the program loses the ability to address it. To avoid this, you need to assign the pointer returned by new[] to a pointer variable declared in an outer scope.
As a separate issue, it looks as though you're trying to allocate memory of one of 2 different types. If you want to do this portably, you're obliged to either use a void * to hold the pointer, or (less commonly done) a union type containing a pointer of each type. Either way, you will need to maintain state information that lets the program know which kind of allocation has been made. Usually, wanting to do this is an indication of poor design, because every single access will require switching on this state information.
If I understand your intend correctly what you are trying to do is: depending on some logic allocate memory to store n elements of either int or char and then later in your function access that array as either int or char without the need for a single if statement.
If the above understanding is correct than the simple answer is: "C++ is a strong-typed language and what you want is not possible".
However... C++ is also an extremely powerful and flexible language, so here's what can be done:
Casting. Something like the following:
void * Array;
if(flag1) Array = new int[len]
else Array = new char[len];
// ... later in the function
if(flag) // access as int array
int i = ((int*)Array)[0];
Yes, this is ugly and you'll have to have those ifs sprinkled around the function. So here's an alternative: template
template<class T> T foo(size_t _len)
{
T* Array = new T[_len];
T element = Array[0];
return element;
}
Yet another, even more obscure way of doing things, could be the use of unions:
union int_or_char {int i; char c;};
int_or_char *Array = new int_or_char[len];
if(flag) // access as int
int element = Array[0].i;
But one way or the other (or the third) there's no way around the fact that the compiler has to know how to deal with the data you are trying to work with.
Turix's answer is right. You need to keep in mind that two things are being allocated here, The memory for the array and the memory when the location of the array is stored.
So even though the memory from the array is allocated from the heap and will be available to the code where ever required, the memory where the location of the array is stored (the Array variable) is allocated in the stack and will be lost as soon as it goes out of scope. Which in this case is when the if block end. You can't even use it in the else part of the same if.
Another different code suggestion from Andrew I would give is :
void *Array = nullptr;
if (flag == 1) {
Array = new int[input.length()];
} else if (flag == 2) {
Array = new char[input.length()];
}
Then you can directly use if as you intended.
This part I am not sure : In case you want to know if its an int or char you can use the typeid literal. Doesn't work, at least I can't get it to work.
Alternatively you can use your flag variable to guess what type it is.
Okay so you have and array A[]... that is passed to you in some function say with the following function prototype:
void foo(int A[]);
Okay, as you know it's kind of hard to find the size of that array without knowing some sort of ending variable or knowing the size already...
Well here is the deal though. I have seem some people figure it out on a challenge problem, and I don't understand how they did it. I wasn't able to see their source code of course, that is why I am here asking.
Does anyone know how it would even be remotely possible to find the size of that array?? Maybe something like what the free() function does in C??
What do you think of this??
template<typename E, int size>
int ArrLength(E(&)[size]){return size;}
void main()
{
int arr[17];
int sizeofArray = ArrLength(arr);
}
The signature of that function is not that of a function taking an array, but rather a pointer to int. You cannot obtain the size of the array within the function, and will have to pass it as an extra argument to the function.
If you are allowed to change the signature of the function there are different alternatives:
C/C++ (simple):
void f( int *data, int size ); // function
f( array, sizeof array/sizeof array[0] ); // caller code
C++:
template <int N>
void f( int (&array)[N] ); // Inside f, size N embedded in type
f( array ); // caller code
C++ (though a dispatch):
template <int N>
void f( int (&array)[N] ) { // Dispatcher
f( array, N );
}
void f( int *array, int size ); // Actual function, as per option 1
f( array ); // Compiler processes the type as per 2
You cannot do that. Either you have a convention to signal the end of the array (e.g. that it is made of non-zero integers followed by a 0), or you transmit the size of the array (usually as an additional argument).
If you use the Boehm garbage collector (which has a lot of benefit, in particular you allocate with GC_malloc and friends but you don't care about free-ing memory explicitly), you could use the GC_size function to give you the size of a GC_malloc-ed memory zone, but standard malloc don't have this feature.
You're asking what we think of the following code:
template<typename E, int size>
int ArrLength(E(&)[size]){return size;}
void main()
{
int arr[17];
int sizeofArray = ArrLength(arr);
}
Well, void main has never been standard, neither in C nor in C++.
It's int main.
Regarding the ArrLength function, a proper implementation does not work for local types in C++98. It does work for local types by C++11 rules. But in C++11 you can write just end(a) - begin(a).
The implementation you show is not proper: it should absolutely not have int template argument. Make that a ptrdiff_t. For example, in 64-bit Windows the type int is still 32-bit.
Finally, as general advice:
Use std::vector and std::array.
One relevant benefit of this approach is that it avoid throwing away the size information, i.e. it avoids creating the problem you're asking about. There are also many other advantages. So, try it.
The first element could be a count, or the last element could be a sentinel. That's about all I can think of that could work portably.
In new code, for container-agnostic code prefer passing two iterators (or pointers in C) as a much better solution than just passing a raw array. For container-specific code use the C++ containers like vector.
No you can't. Your prototype is equivalent to
void foo(int * A);
there is obviously no size information. Also implementation dependent tricks can't help:
the array variable can be allocated on the stack or be static, so there is no information provided by malloc or friends
if allocated on the heap, a user of that function is not forced to call it with the first element of an allocation.
e.g the following are valid
int B[22];
foo(B);
int * A = new int[33];
foo(A + 25);
This is something that I would not suggest doing, however if you know the address of the beginning of the array and the address of the next variable/structure defined, you could subtract the address. Probably not a good idea though.
Probably an array allocated at compile time has information on its size in the debug information of the executable. Moreover one could search in the code for all the address corresponding to compile time allocated variables and assume the size of the array is minus the difference between its starting address and the next closest starting address of any variable.
For a dinamically allocated variable it should be possible to get its size from the heap data structures.
It is hacky and system dependant, but it is still a possible solution.
One estimate is as follows: if you have for instance an array of ints but know that they are between (stupid example) 0..80000, the first array element that's either negative or larger than 80000 is potentially right past the end of the array.
This can sometimes work because the memory right past the end of the array (I'm assuming it was dynamically allocated) won't have been initialized by the program (and thus might contain garbage values), but might still be part of the allocated pages, depending on the size of the array. In other cases it will crash or fail to provide meaningful output.
All of the other answers are probably better, i.e. you either have to pass the length of the array or terminate it with a special byte sequence.
The following method is not portable, but it works for me in VS2005:
int getSizeOfArray( int* ptr )
{
int size = 0;
void* ptrToStruct = ptr;
long adr = (long)ptrToStruct;
adr = adr - 0x10;
void* ptrToSize = (void*)adr;
size = *(int*)ptrToSize;
size /= sizeof(int);
return size;
}
This is entirely dependent of the memory model of your compiler and system so, again, it is not portable. I bet there are equivalent methods for other platforms. I would never use this in a production environment, merely stating this as an alternative.
You can use this: int n = sizeof(A) / sizeof(A[0]);
void pushSynonyms (string synline, char matrizSinonimos [1024][1024]){
stringstream synstream(synline);
vector<int> synsAux;
int num;
while (synstream >> num) {synsAux.push_back(num);}
int index=0;
while (index<(synsAux.size()-1)){
int primerSinonimo=synsAux[index];
int segundoSinonimo=synsAux[++index];
matrizSinonimos[primerSinonimo][segundoSinonimo]='S';
matrizSinonimos [segundoSinonimo][primerSinonimo]='S';
}
}
and the call..
char matrizSinonimos[1024][1024];
pushSynonyms("1 7", matrizSinonimos)
It's important for me to pass matrizSinonimos by reference.
Edit: took away the & from &matrizSinonimos.
Edit: the runtime error is:
An unhandled win32 exception occurred in program.exe [2488]![alt text][1]
What's wrong with it
The code as you have it there - i can't find a bug. The only problem i spot is that if you provide no number at all, then this part will cause harm:
(synsAux.size()-1)
It will subtract one from 0u . That will wrap around, because size() returns an unsigned integer type. You will end up with a very big value, somewhere around 2^16 or 2^32. You should change the whole while condition to
while ((index+1) < synsAux.size())
You can try looking for a bug around the call side. Often it happens there is a buffer overflow or heap corruption somewhere before that, and the program crashes at a later point in the program as a result of that.
The argument and parameter stuff in it
Concerning the array and how it's passed, i think you do it alright. Although, you still pass the array by value. Maybe you already know it, but i will repeat it. You really pass a pointer to the first element of this array:
char matrizSinonimos[1024][1024];
A 2d array really is an array of arrays. The first lement of that array is an array, and a pointer to it is a pointer to an array. In that case, it is
char (*)[1024]
Even though in the parameter list you said that you accept an array of arrays, the compiler, as always, adjusts that and make it a pointer to the first element of such an array. So in reality, your function has the prototype, after the adjustments of the argument types by the compiler are done:
void pushSynonyms (string synline, char (*matrizSinonimos)[1024]);
Although often suggested, You cannot pass that array as a char**, because the called function needs the size of the inner dimension, to correctly address sub-dimensions at the right offsets. Working with a char** in the called function, and then writing something like matrizSinonimos[0][1], it will try to interpret the first sizeof(char**) characters of that array as a pointer, and will try to dereference a random memory location, then doing that a second time, if it didn't crash in between. Don't do that. It's also not relevant which size you had written in the outer dimension of that array. It rationalized away. Now, it's not really important to pass the array by reference. But if you want to, you have to change the whole thingn to
void pushSynonyms (string synline, char (&matrizSinonimos)[1024][1024]);
Passing by reference does not pass a pointer to the first element: All sizes of all dimensions are preserved, and the array object itself, rather than a value, is passed.
Arrays are passed as pointers - there's no need to do a pass-by-reference to them. If you declare your function to be:
void pushSynonyms(string synline, char matrizSinonimos[][1024]);
Your changes to the array will persist - arrays are never passed by value.
The exception is probably 0xC00000FD, or a stack overflow!
The problem is that you are creating a 1 MB array on the stack, which probably is too big.
try declaring it as:
void pushSynonyms (const string & synline, char *matrizSinonimos[1024] )
I believe that will do what you want to do. The way you have it, as others have said, creates a 1MB array on the stack. Also, changing synline from string to const string & eliminates pushing a full string copy onto the stack.
Also, I'd use some sort of class to encapsulate matrizSinonimos. Something like:
class ms
{
char m_martix[1024][1024];
public:
pushSynonyms( const string & synline );
}
then you don't have to pass it at all.
I'm at a loss for what's wrong with the code above, but if you can't get the array syntax to work, you can always do this:
void pushSynonyms (string synline, char *matrizSinonimos, int rowsize, int colsize )
{
// the code below is equivalent to
// char c = matrizSinonimos[a][b];
char c = matrizSinonimos( a*rowsize + b );
// you could also Assert( a < rowsize && b < colsize );
}
pushSynonyms( "1 7", matrizSinonimos, 1024, 1024 );
You could also replace rowsize and colsize with a #define SYNONYM_ARRAY_DIMENSION 1024 if it's known at compile time, which will make the multiplication step faster.
(edit 1) I forgot to answer your actual question. Well: after you've corrected the code to pass the array in the correct way (no incorrect indirection anymore), it seems most probable to me that you did not check you inputs correctly. You read from a stream, save it into a vector, but you never checked whether all the numbers you get there are actually in the correct range. (end edit 1)
First:
Using raw arrays may not be what you actually want. There are std::vector, or boost::array. The latter one is compile-time fixed-size array like a raw-array, but provides the C++ collection type-defs and methods, which is practical for generic (read: templatized) code.
And, using those classes there may be less confusion about type-safety, pass by reference, by value, or passing a pointer.
Second:
Arrays are passed as pointers, the pointer itself is passed by value.
Third:
You should allocate such big objects on the heap. The overhead of the heap-allocation is in such a case insignificant, and it will reduce the chance of running out of stack-space.
Fourth:
void someFunction(int array[10][10]);
really is:
(edit 2) Thanks to the comments:
void someFunction(int** array);
void someFunction(int (*array)[10]);
Hopefully I didn't screw up elsewhere....
(end edit 2)
The type-information to be a 10x10 array is lost. To get what you've probably meant, you need to write:
void someFunction(int (&array)[10][10]);
This way the compiler can check that on the caller side the array is actually a 10x10 array. You can then call the function like this:
int main() {
int array[10][10] = { 0 };
someFunction(array);
return 0;
}