What happens to EBP when calling a function - gdb

This is the program:
#include <stdio.h>
void test_function(int a, int b, int c, int d){
int flag;
flag = 31337;
}
int main(){
test_function(1,2,3,4);
}
From GDB:
Breakpoint 1, main () at stack_example.c:14
14 test_function(1,2,3,4);
(gdb) i r esp ebp eip
esp 0xffffcf88 0xffffcf88
ebp 0xffffcf98 0xffffcf98
eip 0x8048402 0x8048402 <main+6>
(gdb) cont
Continuing.
Breakpoint 2, test_function (a=1, b=2, c=3, d=4) at stack_example.c:8
8 flag = 31337;
(gdb) i r esp ebp eip
esp 0xffffcf70 0xffffcf70
ebp 0xffffcf80 0xffffcf80
eip 0x80483f3 0x80483f3 <test_function+6>
(gdb)
Dump of assembler code for function test_function:
0x080483ed <+0>: push ebp
x080483ee <+1>: mov ebp,esp
0x080483f0 <+3>: sub esp,0x10
=> 0x080483f3 <+6>: mov DWORD PTR [ebp-0x4],0x7a69
0x080483fa <+13>: leave
0x080483fb <+14>: ret
From GDB,
Why does EBP not have the same value as what ESP was in main()? Shouldn't mov ebp, esp
make EBP == 0xffffcf88? I thought this sets EBP to what ESP is.
EDIT:
I think I may have answered my own question. Please correct me.
ESP is moved when the return address and saved frame pointer are pushed onto the stack.
The ESP value was 0xffffcf88 before the two values(both 4 bytes) were pushed onto the stack. Afterwards, it's value is 0xffffcf88 - 0x8 == 0xffffcf80. This is what EBP's current value is. Then ESP -= 0x10.
How is the value of ESP modified? Is it something like mov ESP, ESP - 0x8 ?

I hope I understand your question.
First, you are correct about new value of frame pointer. Here you need to know that "break test_function" actually stops gdb after the function prologue has been executed, so you are already past the point where ebp is explicitly stored to stack.
See detailed explanation.
ESP is decreased by four for every push (or call). It's also decreased (some immediate value subtracted) by the amount of space required by the local variables, etc.
Different "calling conventions" are being used on x86, so these details may vary a bit.
Some parts of this are also compiler specific. For example GCC by default keeps stack 4-word aligned on x86 (for one reason to keep SSE unit happy). See this SO thread.
BTW: frame pointer is actually redundant as long as compiler knows how to unwind stack from any point in function and it is typically omitted with optimized code.

Related

Decoding assembly from MSVC 32-bit release (homework). What does no-op do?

Hi heads up this is a homework. I'm given an assembly generated by MSVC 32-bit Release with optimizations on, and I'm supposed to decode it back into C++. I've included the top of the function to the line I'm having problems with. The comments are mine, which I'm wrote while trying to understand this.
Note: Code is supposedly generated from C++. Not traditional ASM.
Note 2: There is one area of undefined behavior in the code.
Here are the lines I'm stuck with
TheFunction: ; TheFunction(int* a, int s);
0F2D4670 push ebp ; Push/clear/save ebp
0F2D4671 mov ebp,esp ; ebp now points to top of stack
0F2D4673 push ecx ; Push/clear/save ecx
0F2D4674 push ebx ; Push/clear/save ebs
0F2D4675 push esi ; Push/clear/save esi
0F2D4676 mov ebx,edx ; ebx = int s
0F2D4678 mov esi,1 ; esi = 1
0F2D467D push edi ; calling convention ; Push/clear/save edi
0F2D467E mov edi,dword ptr [a (0F2D95E8h)] ; edi = a[0]
0F2D4684 cmp ebx,esi ; if(s < 1)
0F2D4686 jl SomeFunction+3Ch (0F2D46ACh) ; Jump to return
0F2D4688 nop dword ptr [eax+eax] ; !! <-- No op involving dereferencing? What does this do?
0F2D4690 mov eax,dword ptr [edi+esi*4-4] ; !! <-- edi is *a, while esi is 1. There is no address
here!
..... More code but I've figured these out ....
I've more or less got the gist of the function. Its a function that takes a pointer to an int, with an underlying array, and a size. It then goes through each element in the array from last to first, adding to each subsequent one and printing it out. However, I still haven't got the details down and need help
Two questions, both at the end of the code snippet. What does no op on a dereference pointer do, and am I reading the last line in that its attempting to dereference something not in memory?
The nop dword ptr [eax+eax] instruciton does nothing. It doesn't even access the memory location given by the operand. It literally performs no operation.
It's just there so the next instruction is aligned to a 16-byte boundary. You'll notice that next instruction address is 0F2D4690 which ends with 0 which means it's 16-byte aligned. This can improve the performance of loops. Somewhere there will be an instruction that jumps back to 0F2D4690 as part of a loop. This particular form of a NOP instruction is used because it encodes a single NOP instruction in 8 bytes.
There is no corresponding C++ code for this instruction. You shouldn't try to represent it in your C++ code, just ignore it.
Also note that your comment for mov edi,dword ptr [a (0F2D95E8h)] is incorrect. Instead of being edi = a[0] it's simply edi = a. The variable a isn't a parameter at all, instead it's a global (or file level static) variable located at memory location 0F2D95E8h. This instruction just loads the value from memory.

Defining variables in c++ inline assembly

I am searching for a way to define variables in c++ inline assembly. I found an interesting way to do it. But it confuses me, how this can work.
__asm
{
push ebp
mov ebp, esp
add esp, 4
mov [ebp - 4], 2
mov esp, ebp
pop ebp
}
I see this code as - Push base pointer address to the stack, move stack pointer address into base pointer's (Stack logically should collapse here, because this is common epilogue function of cleaning the stack). Then we move 4 to the esp address (Not even the value) And then remove that 4 from esp. So we get back to the same esp address. Strange fact for me is that, it even compiles, and it works. But when I try to test it by outputting the value
uint32_t output;
__asm
{
push ebp
mov ebp, esp
add esp, 4
mov [ebp - 4], 2
mov output,[ebp-4]
mov esp, ebp
pop ebp
}
std::cout << output;
It does not compile, showing "Operand size conflict", which seems weird to me, because I use 32 bit integer and register is also 32 bit. When using [ebp-4] without [], it gives garbage values, as expected.
So, maybe someone could explain how this works without giving error :)
And one additional question, why does db does not work in inline assembly?
It doesn't work, that doesn't define a C++ variable.
It just messes with the stack to reserve some new storage below the stack frame created by the compiler. And you modify EBP so compiler-generated addressing modes that use EBP will be broken.1
If you want to define or declare a C++ variable, do it with C++ syntax like int tmp.
asm doesn't really have variables. It has registers and memory. Keep track of where values are using comments. If you want to use some extra stack space from MSVC inline asm, I think that's safe, but don't modify EBP if you also want to reference C++ local variables.
Footnote 1:
That would be the case if your code assembled at all, which it won't because mov output,[ebp-4] has 2 explicit memory operands. MSVC inline asm can't allocate C++ variables in register.
Also mov [ebp - 4], 2 has ambiguous operand-size: neither operand has a size associated with it because neither is a register. Maybe you want mov dword ptr [ebp - 4], 2

Suspicious usage of below-stack-pointer memory area for local variables [duplicate]

That's what I understood by reading some memory segmentation documents: when a function is called, there are a few instructions (called function prologue) that save the frame pointer on the stack, copy the value of the stack pointer into the base pointer and save some memory for local variables.
Here's a trivial code I am trying to debug using GDB:
void test_function(int a, int b, int c, int d) {
int flag;
char buffer[10];
flag = 31337;
buffer[0] = 'A';
}
int main() {
test_function(1, 2, 3, 4);
}
The purpose of debugging this code was to understand what happens in the stack when a function is called: so I had to examine the memory at various step of the execution of the program (before calling the function and during its execution). Although I managed to see things like the return address and the saved frame pointer by examining the base pointer, I really can't understand what I'm going to write after the disassembled code.
Disassembling:
(gdb) disassemble main
Dump of assembler code for function main:
0x0000000000400509 <+0>: push rbp
0x000000000040050a <+1>: mov rbp,rsp
0x000000000040050d <+4>: mov ecx,0x4
0x0000000000400512 <+9>: mov edx,0x3
0x0000000000400517 <+14>: mov esi,0x2
0x000000000040051c <+19>: mov edi,0x1
0x0000000000400521 <+24>: call 0x4004ec <test_function>
0x0000000000400526 <+29>: pop rbp
0x0000000000400527 <+30>: ret
End of assembler dump.
(gdb) disassemble test_function
Dump of assembler code for function test_function:
0x00000000004004ec <+0>: push rbp
0x00000000004004ed <+1>: mov rbp,rsp
0x00000000004004f0 <+4>: mov DWORD PTR [rbp-0x14],edi
0x00000000004004f3 <+7>: mov DWORD PTR [rbp-0x18],esi
0x00000000004004f6 <+10>: mov DWORD PTR [rbp-0x1c],edx
0x00000000004004f9 <+13>: mov DWORD PTR [rbp-0x20],ecx
0x00000000004004fc <+16>: mov DWORD PTR [rbp-0x4],0x7a69
0x0000000000400503 <+23>: mov BYTE PTR [rbp-0x10],0x41
0x0000000000400507 <+27>: pop rbp
0x0000000000400508 <+28>: ret
End of assembler dump.
I understand that "saving the frame pointer on the stack" is done by " push rbp", "copying the value of the stack pointer into the base pointer" is done by "mov rbp, rsp" but what is getting me confused is the lack of a "sub rsp $n_bytes" for "saving some memory for local variables". I've seen that in a lot of exhibits (even in some topics here on stackoverflow).
I also read that arguments should have a positive offset from the base pointer (after it's filled with the stack pointer value), since if they are located in the caller function and the stack grows toward lower addresses it makes perfect sense that when the base pointer is updated with the stack pointer value the compiler goes back in the stack by adding some positive numbers. But my code seems to store them in a negative offset, just like local variables.. I also can't understand why they are put in those registers (in the main).. shouldn't they be saved directly in the rsp "offsetted"?
Maybe these differences are due to the fact that I'm using a 64 bit system, but my researches didn't lead me to anything that would explain what I am facing.
The System V ABI for x86-64 specifies a red zone of 128 bytes below %rsp. These 128 bytes belong to the function as long as it doesn't call any other function (it is a leaf function).
Signal handlers (and functions called by a debugger) need to respect the red zone, since they are effectively involuntary function calls. All of the local variables of your test_function, which is a leaf function, fit into the red zone, thus no adjustment of %rsp is needed. (Also, the function has no visible side-effects and would be optimized out on any reasonable optimization setting).
You can compile with -mno-red-zone to stop the compiler from using space below the stack pointer. Kernel code has to do this because hardware interrupts don't implement a red-zone.
But my code seems to store them in a negative offset, just like local variables
The first x86_64 arguments are passed on registers, not on the stack. So when rbp is set to rsp, they are not on the stack, and cannot be on a positive offset.
They are being pushed only to:
save register state for a second function call.
In this case, this is not required since it is a leaf function.
make register allocation easier.
But an optimized allocator could do a better job without memory spill here.
The situation would be different if you had:
x86_64 function with lots of arguments. Those that don't fit on registers go on the stack.
IA-32, where every argument goes on the stack.
the lack of a "sub rsp $n_bytes" for "saving some memory for local variables".
The missing sub rsp on red zone of leaf function part of the question had already been asked at: Why does the x86-64 GCC function prologue allocate less stack than the local variables?

Stack must be clean before function epilogue confusion

I'm studying assembly language from the book "Assembly Language Step-by-Step: Programming with Linux" by Jeff Dunteman, and have come across an interesting paragraph in the book which I'm most likely misunderstanding, hence would appreciate some clarification on:
"The stack must be clean before we destroy the stack frame and return control. This simply means that any temporary values that we may have pushed onto the stack during the program’s run must be gone. All that is left on the stack should be the caller’s EBP, EBX, ESI, and EDI values.
...
Once the stack is clean, to destroy the stack frame we must first pop the caller’s register values back into their registers, ensuring that the pops are in the correct order.
...
We restore the caller’s ESP by moving the value from EBP into ESP, and finally pop the caller’s EBP value off the stack."
Consider the following code generated from Visual Studio 2008:
int myFuncSum( int a, int b)
{
001B1020 push ebp
001B1021 mov ebp,esp
001B1023 push ecx <------------------
int c;
c = a + b;
001B1024 mov eax,dword ptr [ebp+8]
001B1027 add eax,dword ptr [ebp+0Ch]
001B102A mov dword ptr [ebp-4],eax
return c;
001B102D mov eax,dword ptr [ebp-4]
}
001B1030 mov esp,ebp
001B1032 pop ebp
001B1033 ret
The value of ecx (indicated), pushed to make space on the stack for my variable c, is, as far as I can see, only gone from the stack when we reset ESP; however, as quoted, the book states that the stack must be clean before we reset ESP. Can someone please clarify whether or not I am missing something?
The example from Visual Studio 2008 doesn't contradict the book. The book is covering the most elaborate case of a call. See the x86-32 Calling Convention as a cross-reference which spells it out with pictures.
In your example, there were no caller registers saved on the stack, so there are no pop instructions to be performed. This is part of the "clean up" that must occur before mov esp, ebp that the book is referring to. So more specifically, let's suppose the callee is saving si and di for the caller, then the prelude and postlude for the function might look like this:
push ebp ; save base pointer
mov ebp, esp ; setup stack frame in base pointer
sub esp, 4 ; reserve 4 bytes of local data
push si ; save caller's registers
push di
; do some stuff, reference 32-bit local variable -4(%ebp), etc
; Use si and di for our own purposes...
; clean up
pop di ; do the stack clean up
pop si ; restoring the caller's values
mov esp, ebp ; restore the stack pointer
pop ebp
ret
In your simple example, there were no saved caller registers, so no final pop instructions needed at the end.
Perhaps because it's simpler or faster, the compiler elected to do the following instruction in place of sub esp, 4:
push ecx
But the effect is the same: reserve 4 bytes for a local variable.
Notice the instruction:
push ebp
mov ebp,esp ; <<<<=== saves the stack base pointer
and the instruction:
mov esp,ebp ; <<<<<== restore the stack base pointer
pop ebp
So after this sequence the stack is clean again

Can anyone help me interpret this MSVC debug-mode disassembly from a simple Hello World?

I got the following simple C++ code:
#include <stdio.h>
int main(void)
{
::printf("\nHello,debugger!\n");
}
And from WinDbg, I got the following disassembly code:
SimpleDemo!main:
01111380 55 push ebp
01111381 8bec mov ebp,esp
01111383 81ecc0000000 sub esp,0C0h
01111389 53 push ebx
0111138a 56 push esi
0111138b 57 push edi
0111138c 8dbd40ffffff lea edi,[ebp-0C0h]
01111392 b930000000 mov ecx,30h
01111397 b8cccccccc mov eax,0CCCCCCCCh
0111139c f3ab rep stos dword ptr es:[edi]
0111139e 8bf4 mov esi,esp
011113a0 683c571101 push offset SimpleDemo!`string' (0111573c)
011113a5 ff15b0821101 call dword ptr [SimpleDemo!_imp__printf (011182b0)]
011113ab 83c404 add esp,4
011113ae 3bf4 cmp esi,esp
011113b0 e877fdffff call SimpleDemo!ILT+295(__RTC_CheckEsp) (0111112c)
011113b5 33c0 xor eax,eax
011113b7 5f pop edi
011113b8 5e pop esi
011113b9 5b pop ebx
011113ba 81c4c0000000 add esp,0C0h
011113c0 3bec cmp ebp,esp
011113c2 e865fdffff call SimpleDemo!ILT+295(__RTC_CheckEsp) (0111112c)
011113c7 8be5 mov esp,ebp
011113c9 5d pop ebp
011113ca c3 ret
I have some difficulties to fully understand it. What is the SimpleDemo!ILT things doing here?
What's the point of the instruction comparing ebp and esp at 011113c0?
Since I don't have any local variables in main() function, why there's still a sub esp,0C0h at the loacation of 01111383?
Many thanks.
Update 1
Though I still don't know what ILT means, but the __RTC_CheckESP is for runtime checks. These code can be elimiated by placing the following pragma before the main() function.
#pragma runtime_checks( "su", off )
Reference:
http://msdn.microsoft.com/en-us/library/8wtf2dfz.aspx
http://msdn.microsoft.com/en-us/library/6kasb93x.aspx
Update 2
The sub esp,0C0h instruction allocate another 0C0h bytes extra space on the stack. Then EAX is filled with 0xCCCCCCCC, this is 4 bytes, since ECX=30h, 4*30h=0C0h, so the instruction rep stos dword ptr es:[edi] fill exactly the extra spaces with 0xCC. But what is this extra space on stack for? Is this some kind of safe belt? Also I notice that if I turn off the runtime check as Update 1 shows, there's still such extra space on stack, though much smaller. And this space is not filled with 0xCC.
The assembly code without runtime check is like below:
SimpleDemo!main:
00231250 55 push ebp
00231251 8bec mov ebp,esp
00231253 83ec40 sub esp,40h <-- Still extra space allocated from stack, but smaller
00231256 53 push ebx
00231257 56 push esi
00231258 57 push edi
00231259 683c472300 push offset SimpleDemo!`string' (0023473c)
0023125e ff1538722300 call dword ptr [SimpleDemo!_imp__printf (00237238)]
00231264 83c404 add esp,4
00231267 33c0 xor eax,eax
00231269 5f pop edi
0023126a 5e pop esi
0023126b 5b pop ebx
0023126c 8be5 mov esp,ebp
0023126e 5d pop ebp
0023126f c3 ret
Most of the instructions are part of MSVC runtime checking, enabled by default for debug builds. Just calling printf and returning 0 in an optimized build takes much less code. (Godbolt compiler explorer). Other compilers (like GCC and clang) don't do as much stuff like stack-pointer comparison after calls, or poisoning stack memory with a recognizable 0xCC pattern to detect use-uninitialized, so their debug builds are like MSVC debug mode without its extra runtime checks.
I've annotated the assembler, hopefully that will help you a bit. Lines starting 'd' are debug code lines, lines starting 'r' are run time check code lines. I've also put in what I think a debug with no runtime checks version and release version would look like.
; The ebp register is used to access local variables that are stored on the stack,
; this is known as a stack frame. Before we start doing anything, we need to save
; the stack frame of the calling function so it can be restored when we finish.
push ebp
; These two instructions create our stack frame, in this case, 192 bytes
; This space, although not used in this case, is useful for edit-and-continue. If you
; break the program and add code which requires a local variable, the space is
; available for it. This is much simpler than trying to relocate stack variables,
; especially if you have pointers to stack variables.
mov ebp,esp
d sub esp,0C0h
; C/C++ functions shouldn't alter these three registers in 32-bit calling conventions,
; so save them. These are stored below our stack frame (the stack moves down in memory)
r push ebx
r push esi
r push edi
; This puts the address of the stack frame bottom (lowest address) into edi...
d lea edi,[ebp-0C0h]
; ...and then fill the stack frame with the uninitialised data value (ecx = number of
; dwords, eax = value to store)
d mov ecx,30h
d mov eax,0CCCCCCCCh
d rep stos dword ptr es:[edi]
; Stack checking code: the stack pointer is stored in esi
r mov esi,esp
; This is the first parameter to printf. Parameters are pushed onto the stack
; in reverse order (i.e. last parameter pushed first) before calling the function.
push offset SimpleDemo!`string'
; This is the call to printf. Note the call is indirect, the target address is
; specified in the memory address SimpleDemo!_imp__printf, which is filled in when
; the executable is loaded into RAM.
call dword ptr [SimpleDemo!_imp__printf]
; In C/C++, the caller is responsible for removing the parameters. This is because
; the caller is the only code that knows how many parameters were put on the stack
; (thanks to the '...' parameter type)
add esp,4
; More stack checking code - this sets the zero flag if the stack pointer is pointing
; where we expect it to be pointing.
r cmp esi,esp
; ILT - Import Lookup Table? This is a statically linked function which throws an
; exception/error if the zero flag is cleared (i.e. the stack pointer is pointing
; somewhere unexpected)
r call SimpleDemo!ILT+295(__RTC_CheckEsp))
; The return value is stored in eax by convention
xor eax,eax
; Restore the values we shouldn't have altered
r pop edi
r pop esi
r pop ebx
; Destroy the stack frame
r add esp,0C0h
; More stack checking code - this sets the zero flag if the stack pointer is pointing
; where we expect it to be pointing.
r cmp ebp,esp
; see above
r call SimpleDemo!ILT+295(__RTC_CheckEsp)
; This is the usual way to destroy the stack frame, but here it's not really necessary
; since ebp==esp
mov esp,ebp
; Restore the caller's stack frame
pop ebp
; And exit
ret
; Debug only, no runtime checks
push ebp
mov ebp,esp
d sub esp,0C0h
d lea edi,[ebp-0C0h]
d mov ecx,30h
d mov eax,0CCCCCCCCh
d rep stos dword ptr es:[edi]
push offset SimpleDemo!`string'
call dword ptr [SimpleDemo!_imp__printf]
add esp,4
xor eax,eax
mov esp,ebp
pop ebp
ret
; Release mode (The optimiser is clever enough to drop the frame pointer setup with no VLAs or other complications)
push offset SimpleDemo!`string'
call dword ptr [SimpleDemo!_imp__printf]
add esp,4
xor eax,eax
ret
Number one your code's main() is improperly formed. It doesn't return the int you promised it would return. Correcting this defect, we get:
#include
int main(int argc, char *argv[])
{
::printf("\nHello,debugger!\n");
return 0;
}
Additionally, any more, it is very strange to see #include <stdio.h> in a C++ program. I believe you want #include <cstdio>
In all cases, space must be made on the stack for arguments and for return values. main()'s return value requires stack space. main()s context to be saved during the call to printf() requires stack space. printf()'s arguments require stack space. printf()'s return value requires stack space. That's what the 0c0h byte stack frame is doing.
The first thing that happens is the incoming bas pointer is copied to the top of the stack. Then the new stack pointer is copied into the base pointer. We'll be checking later to be sure that the stack winds up back where it started from (because you have runtime checking turned on). Then we build the (0C0h bytes long) stack frame to hold our context and printf()'s arguments during the call to printf(). We jump to printf(). When we get back, we hop over the return value which you didn't check in your code (the only thing left on its frame) and make sure the stack after the call is in the same place it was before the call. We pop our context back off the stack. We then check that the final stack pointer matches the value we saved way up at the front. Then we pop the prior value of the base pointer off the very top of the stack and return.
That is code that is inserted by the compiler when you build with runtime checking (/RTC). Disable those options and it should be clearer. /GZ could also be causing this depending on your VS version.
For the record, I suspect that ILT means "Incremental Linking Thunk".
The way incremental linking (and Edit&Continue) works is the following: the linker adds a layer of indirection for every call via thunks which are grouped at the beginning of executable, and adds a huge reserved space after them. This way, when you're relinking the updated executable it can just put any new/changed code into the reserved area and patch only the affected thunks, without changing the rest of the code.
The 40 bytes is the worst case stack allocation for any called or subsequently called function. This is explained in glorious detail here.
What is this space reserved on the top of the stack for? First, space is created for any local variables. In this case, FunctionWith6Params() has two. However, those two local variables only account for 0x10 bytes. What’s the deal with the rest of the space created on the top of the stack?
On the x64 platform, when code prepares the stack for calling another function, it does not use push instructions to put the parameters on the stack as is commonly the case in x86 code. Instead, the stack pointer typically remains fixed for a particular function. The compiler looks at all of the functions the code in the current function calls, it finds the one with the maximum number of parameters, and then creates enough space on the stack to accommodate those parameters. In this example, FunctionWith6Params() calls printf() passing it 8 parameters. Since that is the called function with the maximum number of parameters, the compiler creates 8 slots on the stack. The top four slots on the stack will then be the home space used by any functions FunctionWith6Params() calls.