Replacing a function in a decompiled DLL - c++

I have decompiled a dll and I want to replace a call to a function in the DLL with call a custom function created by me (with the same signature) instead.
I have managed to find where the function is called in assembly. Could anybody explain what I need to do now?
Can my custom function be located in a separate dll or does it need to be included in the same dll?
How can I call replace the function call with my new one?
Thanks

Might be easier to just patch the executable module. Add your new code to the end of the text segment and place a jump to your added code at the beginning to the old function. This way you don't have to deal with problems the decompiled version of the DLL may cause.
For example here is the beginning of the existing function you want to replace:
000D0880 push ebp
000D0881 mov ebp,esp
000D0883 sub esp,0E0h
000D0889 push ebx
000D088A push esi
000D088B push edi
000D088C lea edi,[ebp-0E0h]
000D0892 mov ecx,38h
000D0897 mov eax,0CCCCCCCCh
To redirect the call just add a JMP instruction at D0880. We add NOP instructions after the jump just to make the disassembled output cleaner during debug sessions. It's not necessary but it does come in handy.
000D0880 jmp NewFunction
000D0885 nop
000D0886 nop
000D0887 nop
000D0888 nop
000D0889 push ebx
000D088A push esi
000D088B push edi
Now you append your new code to the end of the text segment and calculate the address based on the offset where the segment is loaded. This makes it much easier to debug and manage since you know beforehand exactly where in the text segment your function actually resides.
To call either function from another in-process executable module can also be done. Since you already know the offsets of both the original function and the next function you can calculate their exact location based on the load address of the module.
typedef void (*EXTPROC)(int a, int b);
// Windows loads the dll at 0xC0000
HMODULE hMod = LoadLibrary("some.dll");
// The function is at offset 0x10880 (from start of text segment)
EXTPROC proc = CalculateAddress(hMod, 0x00010880);
// Call proc at 0xd0880
proc(0, 1);
The function CalculateAddress retrieves the base address of the DLL and calculates the actual address of the function and returns pointer to it. The result is the same as in the earlier example - 0xD0880. Now that you have the address you can call it through the function pointer. You can apply the same technique to call the new function you are adding as you know the offset of that function as well.
If you wanted to go the extra mile you could even update the export table with new entries pointing to those offsets and use GetProcAddress to retrieve the address without having to calculate it yourself. This adds a bit more complexity to the patching process since you have to update an additional section in the DLL.
To call a function that exists in a different DLL is a bit more complex as you would need to do a runtime patch of the original function to make it call the address of the external one. For this I recommend looking into the suggestion made by Rich and create a proxy DLL as described in this article.

Related

How does one pass on parameters in assembly?

im working on a hook in C++ and ASM and currently i have just made an easy inline hook that places a jump in the first instruction of the target function which in this case is OutputDebugString just for testing purposes.
the thing is that my hook fianlly works after about 3 days of research and figuring out the bits and peaces of how things work, but there is one problem i have no idea how to change the parameters that come in to my "dummy" function before jumping on to the rest of the original function.
as u can see in my code i have tried to change the parameter simply in C++ but of course this does not work as im poping all the registers afterwards :/
anyways here is my dummy function which is what the hooked function jumps to:
static void __declspec(naked) MyDebugString(LPCTSTR lpOutputString) {
__asm {
PUSHAD
}
//Where i suppose i could run my code, but not be able to interfere with parameters :/
lpOutputString = L"new message!";
__asm {
POPAD
MOV EDI, EDI
PUSH EBP
MOV EBP, ESP
JMP Addr
}
original_DebugString(lpOutputString);
}
i understand why the code is not working as i said, i just can't see a proper solution to this, any help is greatly appreciated.
Every compiler has a protocol for calling functions using assembly language. The protocol may be stated deep in their manuals.
A faster method to find the function protocols is to have the compiler generate an assembly language listing for your function.
The best method for writing inline assembly is to:
First write the function in C++ source code
Next print out the assembly listing of the function.
Review and understand how the compiler generated assembly works.
Lastly, modify the internal assembly to suite your needs.
My preference is to write the C++ code as efficient as I can (or to help the compiler use optimal assembly language). I then review the assembly listing. I only change the inline assembly to invoke processor special features (such as block move instructions).

Hooking call function in c++?

Hello people,
I'm kinda newbie with c++ but i have managed to create my own dll and injecting it to my gameserver.exe
Well i have tried for days to hook a user call function but i always fail since it belongs to assembly :(
I would like you guys to show me how i write a proper lines to hook this function:
0048C1AF |. 8B9B 4C010000 MOV EBX,DWORD PTR DS:[EBX+14C]
0048C1B5 |. 8B13 MOV EDX,DWORD PTR DS:[EBX]
0048C1B7 |. 8B82 EC000000 MOV EAX,DWORD PTR DS:[EDX+EC]
0048C1BD |. 8BCB MOV ECX,EBX
0048C1BF |. FFD0 CALL EAX
0048C1C1 |. 8BF8 MOV EDI,EAX
0048C1C3 |. E8 789EF8FF CALL SR_GameS.00444040
0048C1C8 |. 8B7C24 1C MOV EDI,DWORD PTR SS:[ESP+1C]
0048C1CC |. 8BF0 MOV ESI,EAX
0048C1CE |. E8 6D9EF8FF CALL SR_GameS.00444040
What i have written on c++ so far is:
void __cdecl Global()
{
__asm
{
mov msg, edi; //msg
push ebx;
mov ebx, dword ptr[esp+1C]; //playername
mov playername, ebx;
pop ebx;
}
printf("Global [%s] -> %s\n", playername, msg);
//then calling func entry
CALL((DWORD)0x00444040);
}
when ever 0048C1CE got called, i get it into my c++ and move it's parameters into Global()
until here everything goes fine but inside Global() i can't call back the parameters successful into x00444040 even it show a strange values in console window and sometimes show a part of player message.
P.S. If it's possible an explanation about how things goes with assembly lines.
Sorry for my English, Thanks in advance.
I'll leave aside the question as to why you would want to do this. It's probably someone else's software and they probably didn't give you permission. You may be in breach of a licence somewhere.
Your description is pretty tangled. The lines of assembler are not a function, they are code with 3 function calls. I'll guess that what you meant to say is that you want to intercept the call to function 0x00444040 in order to execute your own code. You haven't shown how you do that.
The C++ code needs to do roughly three things.
On entry, it must conform to the calling sequence expected by its caller. It appears there are two arguments, in ESI and EDI.
If you want to call C++ library functions then you must save all registers that might be affected by making those calls and restore them afterwards.
When you exit, you should restore the stack and registers exactly how they were on entry, and branch (JMP not CALL) to the hooked function, so that it can return to the original caller, not to your hooking code.
At the debugger level, just make sure that every register (including the stack pointer) is the same as it was on entry, just before you branch to the hooked function.
Morality and legality aside, I am just going to focus on the technical aspects of your question - but I do feel you should give sincere thought to the points the david.pfx raised.
Having written a few projects that do similar things to what you described, for personal knowledge only, I would recommend a general purpose hooking library. I worked with the source engine (from Half-life 2 fame), and used a library called SourceHook. SourceHook is part of the AlliedModder's metamod project, which is used inside of SourceMod.
When I tried writing general purpose hooks outside of source-engine projects, I found SourceHook still useful, but also explored other options. I was pleased using mHook, another general purpose hooking library.
Its important the know the calling convention of the methods you are hooking, as restoring the registers correctly is critical to safe execution of your hooks

C++ inline assembly (Intel compiler): LEA and MOV behaving differently in Windows and Linux

I am converting a huge Windows dll to work on both Windows and Linux. The dll has a lot of assembly (and SS2 instructions) for video manipulation.
The code now compiles fine on both Windows and Linux using Intel compiler included in Intel ComposerXE-2011 on Windows and Intel ComposerXE-2013 SP1 on Linux.
The execution, however, crashes in Linux when trying to call a function pointer. I traced the code in gdb and indeed the function pointer doesn't point to the required function (whereas in Windows in does). Almost everything else works fine.
This is the sequence of code:
...
mov rdi, this
lea rdx, [rdi].m_sSomeStruct
...
lea rax, FUNCTION_NAME # if replaced by 'mov', works in Linux but crashes in Windows
mov [rdx].m_pfnFunction, rax
...
call [rdx].m_pfnFunction # crash in Linux
where:
1) 'this' has a struct member m_sSomeStruct.
2) m_sSomeStruct has a member m_pfnFunction, which is a pointer to a function.
3) FUNCTION_NAME is a free function in the same compilation unit.
4) All those pure assembly functions are declared as naked.
5) 64-bit environment.
What is confusing me the most is that if I replace the 'lea' instruction that is supposed to load the function's address into rax with a 'mov' instruction, it works fine on Linux but crashes on Windows. I traced the code in both Visual Studio and gdb and apparently in Windows 'lea' gives the correct function address, whereas in Linux 'mov' does.
I tried looking into the Intel assembly reference but didn't find much to help me there (unless I wasn't looking in the right place).
Any help is appreciated. Thanks!
Edit More details:
1) I tried using square brackets
lea rax, [FUNCTION_NAME]
but that didn't change the behaviour in Windows nor in Linux.
2) I looked at the disassembler in gdb and Windows, seem to both give the same instructions that I actually wrote. What's even worse is that I tried putting both lea/mov one after the other, and when I look at them in disassembly in gdb, the address printed after the instruction after a # sign (which I'm assuming is the address that's going to be stored in the register) is actually the same, and is NOT the correct address of the function.
It looked like this in gdb disassembler
lea 0xOffset1(%rip), %rax # 0xSomeAddress
mov 0xOffset2(%rip), %rax # 0xSomeAddress
where both (SomeAddress) were identical and both offsets were off by the same amount of difference between lea and mov instructions,
But somehow, the when I check the contents of the registers after each execution, mov seem to put in the correct value!!!!
3) The member variable m_pfnFunction is of type LOAD_FUNCTION which is defined as
typedef void (*LOAD_FUNCTION)(const void*, void*);
4) The function FUNCTION_NAME is declared in the .h (within a namespace) as
void FUNCTION_NAME(const void* , void*);
and implemented in .cpp as
__declspec(naked) void namespace_name::FUNCTION_NAME(const void* , void*)
{
...
}
5) I tried turning off optimizations by adding
#pragma optimize("", off)
but I still have the same issue
Off hand, I suspect that the way linking to DLLs works in the latter case is that FUNCTION_NAME is a memory location that actually will be set to the loaded address of the function. That is, it's a reference (or pointer) to the function, not the entry point.
I'm familiar with Win (not the other), and I've seen how calling a function might either
(1) generate a CALL to that address, which is filled in at link time. Normal enough for functions in the same module, but if it's discovered at link time that it's in a different DLL, then the Import Library is a stub that the linker treats the same as any normal function, but is nothing more than JMP [????]. The table of addresses to imported functions is arranged to have bytes that code a JMP instruction just before the field that will hold the address. The table is populated at DLL Load time.
(2) If the compiler knows that the function will be in a different DLL, it can generate more efficient code: It codes an indirect CALL to the address located in the import table. The stub function shown in (1) has a symbol name associated with it, and the actual field containing the address has a symbol name too. They both are named for the function, but with different "decorations". In general, a program might contain fixup references to both.
So, I conjecture that the symbol name you used matches the stub function on one compiler, and (that it works in a similar way) matches the pointer on the other platform. Maybe the assembler assigns the unmangled name to one or the other depending on whether it is declared as imported, and the options are different on the two toolchains.
Hope that helps. I suppose you could look at run-time in a debugger and see if the above helps you interpret the address and the stuff around it.
After reading the difference between mov and lea here What's the purpose of the LEA instruction? it looks to me like on Linux there is one additional level of indirection added into the function pointer. The mov instruction causes that extra level of indirection to be passed through, while on Windows without that extra indirection you would use lea.
Are you by any chance compiling with PIC on Linux? I could see that adding the extra indirection layer.

COM method call accidentally corrupts the stack

I have a bit of code which is calling a method from a COM object (IDirect3D9), but every call causes a run-time check failure #0. The failure is caused by ESP not being properly preserved across the call, so some kind of stack issue (as COM methods are all __stdcall). The unusual part is the simplicity of the method signature and the circumstances.
The code is built in 32-bit mode only, with MSVC 10 (VS 2010 SP1), using the DirectX SDK (June 2010) headers and libs. I've reinstalled the SDK to make sure the headers weren't corrupt, without luck.
I've run the code with both VS' debugger and WinDBG attached, as well as multiple times after reboots/updated drivers. The problem occurs every time, and is identical. Enabling heap validation (and most other options) in gflags doesn't seem to provide any more information, nor does running with Application Verifier. Both simply report the same error as the popup, or the segfault caused shortly after.
Without the call (returning a constant value instead), the program runs as expected. I'm out of ideas on what could be going wrong here.
The function in question is IDirect3D9::GetAdapterModeCount, called from a D3D8-to-9 wrapper (part of a graphics upgrade project for old games). For more general info, the full file is here.
I've tried all the following forms of the call:
UINT r = m_Object->GetAdapterModeCount(D3DADAPTER_DEFAULT, D3DFMT_X8R8G8B8);
UINT r = m_Object->GetAdapterModeCount(0, (D3DFORMAT)22);
UINT adapter = D3DADAPTER_DEFAULT;
D3DFORMAT format = D3DFMT_X8R8G8B8; // and other values
UINT r = m_Object->GetAdapterModecount(adapter, format);
All of which cause the check failure. m_Object is a valid IDirect3D9, and is used previously for a variety of other calls, specifically:
201, 80194887, Voodoo3D8, CVoodoo3D8::GetAdapterCount() == 3
201, 80195309, Voodoo3D8, CVoodoo3D8::GetAdapterIdentifier(0, 2, 0939CBAC) == 0
201, 80195309, Voodoo3D8, CVoodoo3D8::GetAdapterDisplayMode(0, 0018F5B4) == 0
201, 80196541, Voodoo3D8, CVoodoo3D8::GetAdapterModeCount(0, D3DFMT_X8R8G8B8) == 80
The sequence is logged by debug trace code, and appears to be correct and returning the expected values (3 monitors and so forth). The first 3 calls, by the same object on my part (a single instance of CVoodoo3D8), all succeed with no stack warnings. The fourth does not.
If I reorder the calls, to cause GetAdapterModeCount to be called immediately before any of the others in the same object, the same run-time check failure appears. From testing, this seems to rule out an immediately-previous call breaking the stack; the 4 methods calling those 4 functions all occur at different places, and calling GetAdapterModeCount anywhere from within this file causes the issue.
Which brings us to the unusual part. A different class (CVoodoo3D9) also calls the same sequence of IDirect3D9 methods, with similar parameters, but does not fail (it is the equivalent wrapper class for D3D9). The objects are not used at the same time (the code picks on or the other depending on the render process I need), but both give the same behavior every time. The code for the other class is held in another file, which led me to suspect preprocessor issues (more on that shortly).
After that didn't provide any information, I examined the calling conventions of my code and parameters. Again, nothing came to light. The codebase compiles with /w4 /wX and has for some time, with SAL on most functions and all PREfast rules enabled (and passing).
In particular, the call fails when called within this class, whether the call to my method comes from my code or another program using the object. It fails regardless of where it is called, but only within this file.
The full method is:
UINT STDMETHODCALLTYPE CVoodoo3D8::GetAdapterModeCount(UINT Adapter)
{
UINT r = m_Object->GetAdapterModeCount(D3DADAPTER_DEFAULT, D3DFMT_X8R8G8B8);
gpVoodooLogger->LogMessage(LL_Debug, VOODOO_D3D_NAME, Format("CVoodoo3D8::GetAdapterModeCount(%d, D3DFMT_X8R8G8B8) == %d") << Adapter << r);
return r;
}
The check failure occurs immediately after the call to GetAdapterModeCount and again as my method returns, if allowed to execute to that point.
The preprocessor output, as given by the preprocess-to-file option, has the method declaration (from d3d9.h) correctly as:
virtual __declspec(nothrow) UINT __stdcall GetAdapterModeCount( UINT Adapter,D3DFORMAT Format) = 0;
The declaration of my method is essentially identical:
virtual __declspec(nothrow) UINT __stdcall GetAdapterModeCount(UINT Adapter);
My method hardly expands, becoming:
UINT __stdcall CVoodoo3D8::GetAdapterModeCount(UINT Adapter)
{
UINT r = m_Object->GetAdapterModeCount(D3DADAPTER_DEFAULT, D3DFMT_X8R8G8B8);
gpVoodooLogger->LogMessage(LL_Debug, L"Voodoo3D8", Format("CVoodoo3D8::GetAdapterModeCount(%d, D3DFMT_X8R8G8B8) == %d") << Adapter << r);
return r;
}
The preprocessor output seems correct for both methods, in the declaration and definition.
The assembly listing up to the point of failure is:
UINT STDMETHODCALLTYPE CVoodoo3D8::GetAdapterModeCount(UINT Adapter)
{
642385E0 push ebp
642385E1 mov ebp,esp
642385E3 sub esp,1Ch
642385E6 push ebx
642385E7 push esi
642385E8 push edi
642385E9 mov eax,0CCCCCCCCh
642385EE mov dword ptr [ebp-1Ch],eax
642385F1 mov dword ptr [ebp-18h],eax
642385F4 mov dword ptr [ebp-14h],eax
642385F7 mov dword ptr [ebp-10h],eax
642385FA mov dword ptr [ebp-0Ch],eax
642385FD mov dword ptr [ebp-8],eax
64238600 mov dword ptr [ebp-4],eax
UINT r = m_Object->GetAdapterModeCount(D3DADAPTER_DEFAULT, D3DFMT_X8R8G8B8);
64238603 mov esi,esp
64238605 push 16h
64238607 push 0
64238609 mov eax,dword ptr [this]
6423860C mov ecx,dword ptr [eax+8]
6423860F mov edx,dword ptr [this]
64238612 mov eax,dword ptr [edx+8]
64238615 mov ecx,dword ptr [ecx]
64238617 push eax
64238618 mov edx,dword ptr [ecx+18h]
6423861B call edx
6423861D cmp esi,esp
6423861F call _RTC_CheckEsp (6424B520h)
64238624 mov dword ptr [r],eax
For clarification, the error comes at 6423861F (the call to _RTC_CheckEsp), suggesting that the call or preparation broke the stack. I am working with the assumption that since the same call works in other places, it is not something within the call breaking things.
To my untrained eye, the only unusual part is the pair of mov register, dword ptr [register+8]. As it is a 32-bit system, I'm not sure if +8 could be incrementing too far, or how it could be getting into the build if so.
Shortly after my method returns, apparently due to the call breaking ESP, the program segfaults. If I don't call GetAdapterModeCount and simply return a value, the program executes as expected.
Additionally, a release build (no RTC) segfaults at a similar point, with the stack:
d3d8.dll!CEnum::EnumAdapterModes() + 0x13b bytes
Voodoo_DX89.dll!ClassCreate() + 0x963 bytes
Although I'm not sure of the implications of the address. It is not, so far as I can tell, the same place that segfaults in debug builds; those are within the program after my methods returns, this appears to be during one of my methods which retrieves data from D3D8. Edit: The segfault occurs in a later call, which I'm currently debugging.
At this point, I'm at a complete loss as to what is going wrong or how, and am out of things to check.
I don't see anything wrong with what you're doing or with your generated assembly code.
I can answer your one concern, though.
ecx,dword ptr [eax+8]
What this is doing is moving the address of m_Object into the ecx register. The +8 is the offset within your class to m_Object, which is probably correct.
Something to look at. Step through the assembly code until you reach this point:
6423861B call edx
6423861D cmp esi,esp
At that point check the esi and esp registers (in VS just hover your mouse over the register names).
Before the call is executed, ESI should be 12 higher than ESP. After the call, they should be equal. If they are not, post what they are.
Update:
So what is catching my eye is that of the 4 methods you are showing that you are calling, only GetAdapterModeCount has a different signature between D3D8 and D3D9, and that signature is different by 4 bytes, which is the difference in your stack.
How is m_Object obtained? Since this is some kind of adapter between D3D8 and D3D9, is it possible that your m_Object is actually an IDirect3D8 object that is being cast as IDirect3D9 at some point? That would explain the error, and why it works in another context, if you are obtaining the D3D object in a different way.

how to create a trampoline function using DetourAttachEx? (with MS detours)

I have a dll and i wish to create a detour to one of its exported functions,
The dll is not part of windows.
I need to be able to call the real function after my detour (call the real function from a detoured one)
I know the exact signature of the function.
I already have been able to detour the function, but right now i can't call the real one.
I realize i need to use a trampoline function, I've seen examples online.
the problem is: all those examples show how to detour a windows API function, i need to do the same for a function i get thorough a dll import.
any help would be welcomed
--edit
just to clarify, I have attempted to call the original function by its pointer, but that does not work.
also tried using the method from this stack overflow article
that doesn't even crash but it looks like it goes into to an infinite loop (i assume because in the original function there is a jump to the detoured one)
edit -- solved!
not sure what solved it,
used this as reference.
stopped using getProcadder and instead started using DetourFindFunction instead
cleaned up the code (pretty sure i cleaned out whatever caused the issue)
works,
thanks anyway
I don't use detours(I actually detest it!), but detouring any non hot-patchable function can be done in a generic manner, like so:
Sstep 1:
insert a JMP <your code> at the start of the function, takes 5 bytes, probably a little more to align to the nearest instruction. as an example
the start of the function to hook:
SUB ESP,3C
PUSH EDI
PUSH ESI
//more code
would become:
JMP MyFunction
//more code
one would do this by writing 0xE9 at the first byte then writing the value (function_addr - patch_addr + sizeof(INT_PTR)) in the following DWORD. writing should be done using WriteProcessMemory after setting Read/write/execute permissions with VirtualProtectEx
Step 2:
next, we create an assembly interface:
void __declspec(naked) MyFunc()
{
__asm
{
call Check ;call out filter func
test eax,eax ; test if we let the call through
je _EXIT
sub esp,3c ; its gone through, so we replicate what we overwrote
push edi
push esi
jmp NextExecutionAddress ; now we jump back to the location just after our jump
_EXIT:
retn ; note, this must have the correct stack cleanup
}
}
NextExecutionAddress will need to be filled at run time using ModuleBase + RVA.
To be honest, its way easier, and better(!) to just EAT (Export Address Table) hook the export table of the dll, or IAT (Import Address Table) hook the import tables of whats calling the funcs you want to filter. Detours should have functions for these type of hooks, if not, there are other freely available libs to do it.
The other way would be to use detour to hook every call in the apps using the dll to reroute them to a proxy function in your own code, this has the advantage of allowing one to filter only certain calls, and not everything across a binary(it is possible to do the same using _ReturnAddress, but thats more work), the disadvantage though is capturing the locations to patch(I use ollydbg + a custom patching engine) and it won't work on non-regular calling convention functions(like those made with #pragma aux in Watcom or the optimized calls generated by VC7+).
One important thing to note: if your hooking a multithreaded app, your patches need to be done with the app suspended, or be done attomically use InterlockedExchange, InterlockExchange64 and InterlockedExchangePointer(I use the latter for all IAT/EAT hooks, especially when hooking from a 'third party process')
Looking at the post you link to, the method there is horrible in my opinion, mainly due to the assmebly :P but, how are you calling this pointer you obtain, and how is it obtained?