Prevent recursive hooking when using detours - c++

I wanted to build some sort of API monitor by hooking all ntdll functions using Detours API. Each hooked function will call the original implementation and than
add notice for this call inside std base data structure.
However, I encountered a scenario of recursive hooking where my hooked function is indirectly calling itself.
Therefore, I tried to use the Tls memory to set a bit per thread that indicate that current function is called from hooked function, so avoid calling the hook again (execute the original function only).
But my recursive hook guard also uses some calls for memory allocation function like NtAllocateVirtualMemory, and therefore I'm currently avoid hooking those functions.
Perhaps anybody has encountered a similar issue and implemented hook reentrancy guard in a way that doesn't triggered any memory allocation function (which might be impossible since even if you call a new function and your stack
memory is insufficient, it should allocate more memory).
thanks

Related

How to hook CopyTextureRegion with Minhook

Im trying to hook the ID3D12GraphicsCommandList::CopyTextureRegion method with MinHook but the problem is that the method is inside ID3D12GraphicsCommandList which makes it hard to hook.
Here is my current approach which unfortunatley is 0
// not working, CopyTextureRegionHook is 0
auto CopyTextureRegionHook = GetProcAddress(GetModuleHandleA("d3d12.dll"), "CopyTextureRegion");
MH_CreateHook(reinterpret_cast<void**>(CopyTextureRegionHook), &HK_CopyTextureRegion, reinterpret_cast<void**>(&oCopyTextureRegion));
MH_EnableHook(CopyTextureRegionHook);
Hooking methods like D3D12CreateDevice, D3D12CoreRegisterLayers.. work because they are not in a interface like ID3D12GraphicsCommandList
How would I properly hook CopyTextureRegion?
Because you're dealing with an interface, you won't be able to hook the function directly in memory since it's dependent on the pointer to the interface. In which case you have to utilize a technique called VMT(Virtual Method Table) hooking. In order to achieve this, you'll need to be able to retrieve the pointer to the interface in memory, then locate the virtual method table index(offset) of the function pointer. Which then you'll have to overwrite with your own function address. Your hooked function will have to comply with the calling convention of the original.

How to call functions in a DLL that is currently load in another process? [duplicate]

I have a DLL that I inject into another process but I want to be able to call the exports on that DLL from my application. I've read elsewhere that you have to the SendMessage API but I have no idea what to do. Is there any example code on how this is done?
While it is not possible to directly call a function in another process, you can do it indirectly pretty easily with a few steps and the Windows API.
Get the addresses of LoadLibrary and GetProcAddress from your own process. kernel32.dll should be loaded at the same address in every process, so you can rely on them being present in the process into which you are injecting
Create a struct that will hold all the arguments you want to pass to your function that will call the functions in the other process (because CreateRemoteThread can only pass one argument to a function, so we'll use it to pass a pointer to the structure) which at least contains member function pointers to hold the addresses of LoadLibrary and GetProcAddress
Allocate enough memory for a struct in the remote process via VirtualAllocEx, then fill it with the correct information with WriteProcessMemory
Write a function, taking a pointer to the struct you wrote, that uses LoadLibrary/GetProcAddress to call the function you want. Remember to use the pointers to those functions in the struct you are passing the function, not the names.
Allocate enough memory in the remote process to hold the function with VirtualAllocEx, making sure to pass VAX the PAGE_EXECUTE_READWRITE flag so that it can hold executable code
Read and write the function's code from your process to the other process via Read/WriteProcessMemory
Call the function in the remote process (which is at the address returned by the VirtualAllocEx) by using CreateRemoteThread.
Make sure that all the data you pass to the function is either stored inside the struct and/or resides in the remote process's address space (get it there with VirtualAllocEx/WriteProcessMemory.
It may look a little involved, but it's not really that complicated. If you need some help with it, feel free to ask in a comment.
You can't directly call functions in another process, in general. There are, however, some workarounds you can use.
First, if you know the address of the export (which isn't the case a lot of the time), and the function you call uses the __stdcall calling convention, takes a pointer-sized integer as an argument, and returns a DWORD, you can use CreateRemoteThread to execute it in a thread in the remote process. This is often used to run LoadLibrary to inject a DLL into a target process, since LoadLibrary is loaded in the same address on all processes on a given computer.
Otherwise, the DLL you inject will need to do some sort of RPC with the process that called it. For example, you could have your injected DLL spawn a thread in its DLL_PROCESS_ATTACH handler, which in turn connects to a named pipe, or connects over COM or something to the master process.
SendMessage would need a window handle (hidden or visible), and a message-pump associated with it, that can handle the custom message. As with UAC/Windows-7, the integrity levels of applications may prevent other applications to send/post messages from other processes having low integrity.
It is better to have another thread that waits for these custom messages. For this, you may use pipes (named or unnamed), sockets, mail-slots, shared memory (along with mutex/event for triggering). The another processes would send the message using same protocol.
But before implementing this custom messaging/protocol/IPC mechanism, I suggest you to first determine the exact need.

Call external function(from one exe to another)

Lets say the process 1 is the main process and the process 2 is the target process(i can't edit it by the way), i want to be able to call a function from the process 2 in the process 1, anyone have a nice way to do that?I was thinking in inject a dll with exports that calls that function and use GetProcAddress externally...Is that possible?Is that the best way to do it?
Thanks for the time.
The title and body of your question ask two subtly different questions.
Having one executable call a function that's contained in another executable is quite easy, at least if the name of the function in question has been exported. You can use LoadLibrary to load an executable just like you would a DLL, then use GetProcAddress to get the address of the function you want to call, and call it normally. Keep in mind, however, that the function may not work correctly without other initialization that happens before it's called inside its own executable.
Calling a function in the context of another process (not just in another executable) is considerably more work. The basic idea is to have a function that makes the call and (for example) writes a result to some memory shared with the process making the call. You then use CreateRemoteThread to have that function execute in the context of the process containing the function you need to call.
If the target process has been written to support it there are other methods such as COM that are intended to support this type of capability much more cleanly. They're generally preferable if available.

Calling functions in a DLL loaded by another process

I have a DLL that I inject into another process but I want to be able to call the exports on that DLL from my application. I've read elsewhere that you have to the SendMessage API but I have no idea what to do. Is there any example code on how this is done?
While it is not possible to directly call a function in another process, you can do it indirectly pretty easily with a few steps and the Windows API.
Get the addresses of LoadLibrary and GetProcAddress from your own process. kernel32.dll should be loaded at the same address in every process, so you can rely on them being present in the process into which you are injecting
Create a struct that will hold all the arguments you want to pass to your function that will call the functions in the other process (because CreateRemoteThread can only pass one argument to a function, so we'll use it to pass a pointer to the structure) which at least contains member function pointers to hold the addresses of LoadLibrary and GetProcAddress
Allocate enough memory for a struct in the remote process via VirtualAllocEx, then fill it with the correct information with WriteProcessMemory
Write a function, taking a pointer to the struct you wrote, that uses LoadLibrary/GetProcAddress to call the function you want. Remember to use the pointers to those functions in the struct you are passing the function, not the names.
Allocate enough memory in the remote process to hold the function with VirtualAllocEx, making sure to pass VAX the PAGE_EXECUTE_READWRITE flag so that it can hold executable code
Read and write the function's code from your process to the other process via Read/WriteProcessMemory
Call the function in the remote process (which is at the address returned by the VirtualAllocEx) by using CreateRemoteThread.
Make sure that all the data you pass to the function is either stored inside the struct and/or resides in the remote process's address space (get it there with VirtualAllocEx/WriteProcessMemory.
It may look a little involved, but it's not really that complicated. If you need some help with it, feel free to ask in a comment.
You can't directly call functions in another process, in general. There are, however, some workarounds you can use.
First, if you know the address of the export (which isn't the case a lot of the time), and the function you call uses the __stdcall calling convention, takes a pointer-sized integer as an argument, and returns a DWORD, you can use CreateRemoteThread to execute it in a thread in the remote process. This is often used to run LoadLibrary to inject a DLL into a target process, since LoadLibrary is loaded in the same address on all processes on a given computer.
Otherwise, the DLL you inject will need to do some sort of RPC with the process that called it. For example, you could have your injected DLL spawn a thread in its DLL_PROCESS_ATTACH handler, which in turn connects to a named pipe, or connects over COM or something to the master process.
SendMessage would need a window handle (hidden or visible), and a message-pump associated with it, that can handle the custom message. As with UAC/Windows-7, the integrity levels of applications may prevent other applications to send/post messages from other processes having low integrity.
It is better to have another thread that waits for these custom messages. For this, you may use pipes (named or unnamed), sockets, mail-slots, shared memory (along with mutex/event for triggering). The another processes would send the message using same protocol.
But before implementing this custom messaging/protocol/IPC mechanism, I suggest you to first determine the exact need.

Stepping inside the drag and drop process (or any other system call)

I have an application that does not use the standard libraries. Instead, it uses stubs to call LoadLibrary/GetProcAddress, then calls functions via the resultant function pointers.
Using the above method my application is calling DoDragDrop in ole32.dll. The call is returning E_UNEXPECTED, which apparently means 'catastophic failure'.
What I want to ask is does anyone know of a way to discover what is happening inside the call to DoDragDrop? Specifically, can I discover what function or variable it is looking for but cannot find? (I am guessing that it wants access to some function or variable that is not loaded at the point I call DoDragDrop, and if I can explicitly load it the call will succeed.) Obviously I can't step into the call - attempting to just runs the function and steps to the line in my code following the DoDragDrop call.
:)
(I'm using VS2005 in XP.)
I recall having something like this occur when I had forgotten to call OleInitialize(Ex) / CoInitialize(Ex) for the thread making the call. Note the apartment must be STA.