I was trying to build a function in assebmly(FASM) that used more than 4 parameters. in x86 it works fine but I know in x64 with fastcall you have to spill the parameters into the shadow space in the order of rcx,rdx,r8,r9 I read that for 5 and etc you have to pass them onto the stack, but I don't know how to do this. this is what I tried but it keeps saying invalid operand. I know that the first 4 parameters I am doing right because I have made x64 functions before but it is the last 3 I don't know how to spill
proc substr,inputstring,outputstring,buffer1,buffer2,buffer3,startposition,length
;spill
mov [inputstring],rcx
mov [outputstring],rdx
mov [buffer1],r8
mov [buffer2],r9
mov [buffer3],[rsp+8*4]
mov [startposition],[rsp+8*5]
mov [length],[rsp+8*6]
if I try
mov [buffer3],rsp+8*4
it says extra characters on the line.
I also saw that somepeople use rsp+20h, rsp+28h etc but that does not work either.
how do I call more than 4 parameters using fastcall on x64?
also do I have to make room on the stack? I saw some people have to put add rsp,20h right before their spill code. I tried that and it did not help the invlaid operand.
thanks
update
after playing around with it for a little bit I found that the only way it seems to work is if I spill the first 4 parameters and then ignore the rest 5-infinity
proc substr,inputstring,outputstring,buffer1,buffer2,buffer3,startposition,length
;spill
mov [inputstring],rcx
mov [outputstring],rdx
mov [buffer1],r8
mov [buffer2],r9
;start the regular code. ignore spilling buffer3,startposition and length
On x86/x64-CPUs this following instructions does not exist:
mov [buffer3],[rsp+8*4]
mov [startposition],[rsp+8*5]
mov [length],[rsp+8*6]
Workaround with using the rax-register for to read and for to write a values from and to a memory loaction:
mov rax,[rsp+8*4]
mov [buffer3],rax
mov rax,[rsp+8*5]
mov [startposition],rax
mov rax,[rsp+8*6]
mov [length],rax
Related
I am currently analyzing a binary and came across the following three instructions:
movzx ecx, byte [rax+r9]
movzx edx, byte [rbx+r9]
lea ecx, [rcx+rdx]
The meaning of each of these instructions is clear to me, but the first and third in combination do not make any sense. The first movzx copies the value at [rax+r9] into ecx and afterwards ecx is overwritten again by the lea instruction? Why do we need the first movzx here?
I guess I am just missing something and this is a nasty compiler trick, so I appreciate any help.
Here is the walkthrough I'm using: https://i.imgur.com/LIImg.jpg
From what I'm seeing is that to call a windows function, you put the arguments in specific registers. But where is it listed what registers to use and what order?
Look at the code section of that image, it just seems to use r8d, r9, edx and ecx? Does that mean it uses edx, ecx, r8d, r9d, r10d, etc? What happens when you run out of registers for a function with many parameters?
Also why does it have to subtract from the stack? And why 0x28?
I've been trying to convert this code to C++ without any inlining and I cannot figure it out..
Say you got this line
sub edx, (offset loc_42C1F5+5)
My hex-rays gives me
edx -= (uint)((char*)loc_42C1F5 + 5))
But how would it really look like without the loc_42C1F5.
I would think it would be
edx -= 0x42C1FA;
But is that correct? (can't really step this code in any assembler-level debugger.. as it's damaged well protected)
loc_42C1F5 is a label actually..
seg000:0042C1F5 loc_42C1F5: ; DATA XREF: sub_4464A0+2B5o
seg000:0042C1F5 mov edx, [esi+4D98h]
seg000:0042C1FB lea ebx, [esi+4D78h]
seg000:0042C201 xor eax, eax
seg000:0042C203 xor ecx, ecx
seg000:0042C205 mov [ebx], eax
loc_42C1F5 is a symbol. Given the information you've provided, I cannot say what its offset is. It may be 0x42C1F5 or it may be something else entirely.
If it is 0x42C1F5, then your translation should be correct.
IDA has incorrectly identified 0x42C1FA as an offset, and Hex-Rays used that interpretation. Just convert it to plain number (press O) and all will be well. That's why it's called Interactive Disassembler :)
In looking at my disassembled code I see a lot of the following:
00B442E9 push 4
00B442EB push 3
00B442ED lea ecx,[ebp-24h]
00B442F0 call Foo::Bar (0B41127h)
I understand pushing the parameters before the call, but what's the lea doing here?
In the thiscall calling convention used by Visual C++ for x86, the this pointer is passed in the ecx register. This lea instruction copies the this pointer into the ecx register before calling the member function.
You can read all about the lea instruction in the Stack Overflow question "What's the purpose of the LEA instruction?"
I think it's just an optimized form of
mov ecx, ebp
sub ecx, 24h
Here is a simple asm code I have inserted in VC++ project. addr_curr_ebp is the current address of EBP pointer. It is pointing to the old EBP value inside the stack frame. 4 bytes after this is the return address inside the application function. I extract a single byte from the code section. I run my code along with other applications like gtalk, vlc etc. The application always crashes when I include ProbStat 1 and 2 in my code. When I remove these statements everything works fine. What do you think this is?
__asm{
push eax
push ebx
push cx
mov ebx, addr_curr_ebp
mov eax, [ebx + 4]
mov cl, BYTE PTR [eax - 5] //ProbStat 1
mov ret_5, cl // ProbStat 2
pop cx
pop ebx
pop eax
}
Your code snippet isn't good enough to see where "ret_5" is located. You'll get an automatic crash if it is a member of a class. The ecx register stores the "this" pointer, you're messing it up.
Not sure what this does, sound to me like you need to use the _ReturnAddress intrinsic. It returns the address of the instruction after the call instruction that called this code. Assign it to an unsigned char*, no need for assembly this way.