What is the lea instruction before a method call doing? - c++

In looking at my disassembled code I see a lot of the following:
00B442E9 push 4
00B442EB push 3
00B442ED lea ecx,[ebp-24h]
00B442F0 call Foo::Bar (0B41127h)
I understand pushing the parameters before the call, but what's the lea doing here?

In the thiscall calling convention used by Visual C++ for x86, the this pointer is passed in the ecx register. This lea instruction copies the this pointer into the ecx register before calling the member function.
You can read all about the lea instruction in the Stack Overflow question "What's the purpose of the LEA instruction?"

I think it's just an optimized form of
mov ecx, ebp
sub ecx, 24h

Related

Understanding a function call that uses EAX before and after for the return value

I have been trying to hook a function which is mostly optimized by the compiler. It initializes EAX before the call and its return value is stored in EAX.
Here is some code:
mov eax,dword ptr ds:[0xA6DD08]
push 0x3DC
add eax,0x800
call 0x48A2B4
mov esi,eax
At first, 0xA6DD08 is a pointer to some data in memory but once adding 0x800, EAX just points to a value of zero but the next few DWORD(s) stores pointer of pointers or data array. The function's purpose itself is to lookup and return a specific object that has a DWORD variable equal to the given value which is 0x3DC.
When using __asm to call the function from my dll, it works perfectly but I am trying to write it in c++, something like
Class1* pClass = reinterpret_cast<Class1*(__stdcall*)(DWORD)>(0x48A2B4)(988);
I believe from what I read that only __stdcall uses EAX to store its return value and that's why I choose __stdcall calling convention. What I do not understand is the purpose of EAX before calling the function.
add eax,0x800 right before a call wouldn't make sense unless EAX is an input to the called function.
Passing 1 arg in EAX and another on the stack looks to me like GCC's regparm=1 calling convention. Or if other regs are set before this, regparm=3 passes in EAX, EDX, and ECX (in that order).
32-bit x86 builds of the Linux kernel are typically built with -mregparm=3, but user-space GNU/Linux code typically follows the clunky old i386 System V convention which passes all args on the stack.
According to https://en.wikipedia.org/wiki/X86_calling_conventions#List_of_x86_calling_conventions, a couple other obscure calling conventions also pass a first arg in EAX:
Delphi and Free Pascal register: EAX, EDX, ECX (Left-to-right Pascal style arg passing, right-most arg in EAX I guess? Unlike GCC regparm)
Watcom compiler: EAX, EDX, EBX, ECX. Unless you left out some setting of EDX, EBX, and ECX before pushing a stack arg, we can rule that out.
only __stdcall uses EAX to store it's return value
Actually, all x86 calling conventions do that for integer args, across the board. Also both x86-64 conventions. See Agner Fog's calling convention guide.

Instruction overwrites previous Result

I am currently analyzing a binary and came across the following three instructions:
movzx ecx, byte [rax+r9]
movzx edx, byte [rbx+r9]
lea ecx, [rcx+rdx]
The meaning of each of these instructions is clear to me, but the first and third in combination do not make any sense. The first movzx copies the value at [rax+r9] into ecx and afterwards ecx is overwritten again by the lea instruction? Why do we need the first movzx here?
I guess I am just missing something and this is a nasty compiler trick, so I appreciate any help.

Purpose of rep stos assembly command in this code from Visual Studio [duplicate]

This question already has answers here:
Can anyone help me interpret this MSVC debug-mode disassembly from a simple Hello World?
(5 answers)
Closed 8 years ago.
Take a look at the following code:
void f()
{
}
I compiled this in Visual Studio 2013, debug, 32-bit mode and looked at the dissassembly.
void f()
{
00304CB0 push ebp
00304CB1 mov ebp,esp
00304CB3 sub esp,0C0h
00304CB9 push ebx
00304CBA push esi
00304CBB push edi
00304CBC lea edi,[ebp-0C0h]
00304CC2 mov ecx,30h
00304CC7 mov eax,0CCCCCCCCh
00304CCC rep stos dword ptr es:[edi]
}
00304CCE pop edi
00304CCF pop esi
00304CD0 pop ebx
00304CD1 mov esp,ebp
00304CD3 pop ebp
00304CD4 ret
What is the purpose of the rep stos instruction?
I'm just curious.
The rep stos instruction writes the value in eax starting at the address pointed to by edi (your local stack in this case), ecx (0x30) times. The value in eax is 0xcccccccc which is a magic number chosen by microsoft to indicate uninitialized memory. The debugger will catch you if you try and dereference a pointer from this memory. This extra diagnostic checking is enabled by the /RTCu option.
Now you might ask why, with an empty function body, any memory would be reserved on the local stack. This is because you have edit and continue turned on with the /ZI option. The compiler is just setting aside some space in case you decide to use it in a debug session.

fastcall how to use for more than 4 parameters

I was trying to build a function in assebmly(FASM) that used more than 4 parameters. in x86 it works fine but I know in x64 with fastcall you have to spill the parameters into the shadow space in the order of rcx,rdx,r8,r9 I read that for 5 and etc you have to pass them onto the stack, but I don't know how to do this. this is what I tried but it keeps saying invalid operand. I know that the first 4 parameters I am doing right because I have made x64 functions before but it is the last 3 I don't know how to spill
proc substr,inputstring,outputstring,buffer1,buffer2,buffer3,startposition,length
;spill
mov [inputstring],rcx
mov [outputstring],rdx
mov [buffer1],r8
mov [buffer2],r9
mov [buffer3],[rsp+8*4]
mov [startposition],[rsp+8*5]
mov [length],[rsp+8*6]
if I try
mov [buffer3],rsp+8*4
it says extra characters on the line.
I also saw that somepeople use rsp+20h, rsp+28h etc but that does not work either.
how do I call more than 4 parameters using fastcall on x64?
also do I have to make room on the stack? I saw some people have to put add rsp,20h right before their spill code. I tried that and it did not help the invlaid operand.
thanks
update
after playing around with it for a little bit I found that the only way it seems to work is if I spill the first 4 parameters and then ignore the rest 5-infinity
proc substr,inputstring,outputstring,buffer1,buffer2,buffer3,startposition,length
;spill
mov [inputstring],rcx
mov [outputstring],rdx
mov [buffer1],r8
mov [buffer2],r9
;start the regular code. ignore spilling buffer3,startposition and length
On x86/x64-CPUs this following instructions does not exist:
mov [buffer3],[rsp+8*4]
mov [startposition],[rsp+8*5]
mov [length],[rsp+8*6]
Workaround with using the rax-register for to read and for to write a values from and to a memory loaction:
mov rax,[rsp+8*4]
mov [buffer3],rax
mov rax,[rsp+8*5]
mov [startposition],rax
mov rax,[rsp+8*6]
mov [length],rax

Help: Application crashes on accessing source code

Here is a simple asm code I have inserted in VC++ project. addr_curr_ebp is the current address of EBP pointer. It is pointing to the old EBP value inside the stack frame. 4 bytes after this is the return address inside the application function. I extract a single byte from the code section. I run my code along with other applications like gtalk, vlc etc. The application always crashes when I include ProbStat 1 and 2 in my code. When I remove these statements everything works fine. What do you think this is?
__asm{
push eax
push ebx
push cx
mov ebx, addr_curr_ebp
mov eax, [ebx + 4]
mov cl, BYTE PTR [eax - 5] //ProbStat 1
mov ret_5, cl // ProbStat 2
pop cx
pop ebx
pop eax
}
Your code snippet isn't good enough to see where "ret_5" is located. You'll get an automatic crash if it is a member of a class. The ecx register stores the "this" pointer, you're messing it up.
Not sure what this does, sound to me like you need to use the _ReturnAddress intrinsic. It returns the address of the instruction after the call instruction that called this code. Assign it to an unsigned char*, no need for assembly this way.