Windows DLL Backwards Compatibility - c++

I am using MS detours 3.0 Express to create a DLL that detours a function of an application.
I have used StudPE to enter the dll API and hook it to the application.
Everything works fine except for it won't work on windows XP.
Windows 7 works fine though. And I'm running out of idea's as to why it just won't work on windows XP.
I compiled it on a Windows 7 x64 machine with Microsoft Visual Studio 2012.
I'm calling the DllMain My code is: (just the relevant code - incomplete)
extern "C" __declspec(dllexport) INT APIENTRY DllMain(HMODULE hDLL, DWORD Reason, LPVOID Reserved)
{
switch(Reason) {
case DLL_PROCESS_ATTACH: //Do standard detouring
DisableThreadLibraryCalls(hDLL);
//AllocConsole();
//
DetourTransactionBegin();
DetourUpdateThread(GetCurrentThread());
DetourAttach(&(PVOID&)pSend, MySend);
if(DetourTransactionCommit() == NO_ERROR) {
cout << "[" << MySend << "] successfully detoured." << endl;
}
break;
case DLL_PROCESS_DETACH:
DetourTransactionBegin(); //Detach
DetourUpdateThread(GetCurrentThread());
DetourDetach(&(PVOID&)pSend, MySend);
DetourTransactionCommit();
break;
}
return TRUE;
}
On WinXP nothing happens when I try to run the hooked application.

Related

detours vs MinHook for DLL injection

I have succefully hooked a gog game (DRM free) with this code:
bool WINAPI DllMain(HMODULE hModule, DWORD dwReason, LPVOID lpReserved)
{
static HMODULE libogg_ori = nullptr;
int FFBInitDelayInMillliseconds;
char* DLLFileName;
std::string DLLFile = "libogg_ori.dll";
switch (dwReason)
{
case DLL_PROCESS_ATTACH:
// Load dll
// Initialize config File
Local_InitConfigComplete = Init_Config();
// Delay for timer that initialize the Force Feedback
FFBInitDelayInMillliseconds = Config.FFBDelayBeforeInit * 1000;
DLLFileName = R"(libogg_ori.dll)";
if (FS::File_Exist(DLLFileName))
{
LF::Log_Update("File DLL exit");
// Load the System DLL
d3d9dll = LoadLibraryA(DLLFileName);
if (d3d9dll == NULL)
LF::Log_Update("Load DLL error");
}
else
{
LF::Log_Update("File DLL not exist");
}
// Initilize Detour
if (Local_InitConfigComplete)
Local_InitConfigComplete = Init_Detour();
break;
case DLL_PROCESS_DETACH:
// Close the DLL
LF::Log_Update("Closing session ...");
LF::Log_Update("Release hook in progress ...");
// unhook
DetourTransactionBegin();
DetourUpdateThread(GetCurrentThread());
// this will hook the sound function
DetourDetach(&(LPVOID&)AddressOfHookSoundFunction, &HookSoundFileSub);
// this will hook the damage function (if present in configuration)
if (AddressOfHookDamageFunction>0)
DetourDetach(&(LPVOID&)AddressOfHookDamageFunction, &HookDamageSub);
if (DetourTransactionCommit() == NO_ERROR)
LF::Log_Update(LOG_FLAG_DONE);
else
LF::Log_Update(LOG_FLAG_ERROR, "Warning Hooking not relased.");
LF::Log_Update("Session closed.");
break;
}
return true;
}
bool Init_Detour()
{
// Initialize Detours
// we will find the function/s to hook with IDA pro.
try
{
LF::Log_Update("Initialize Hooking ...");
AddressOfHookSoundFunction = std::strtoul(Config.GameFileSoundOffsetPos.c_str() ,NULL,16); // BattleZoneRedux 0x43AA30
if (Config.GameFileDamageOffsetPos.size() > 1)
AddressOfHookDamageFunction = std::strtoul(Config.GameFileDamageOffsetPos.c_str(), NULL, 16); // BattleZoneRedux 0x49B430
DetourTransactionBegin();
DetourUpdateThread(GetCurrentThread());
// this will hook the sound function
DetourAttach(&(LPVOID&)AddressOfHookSoundFunction, &HookSoundFileSub);
// If is present a offset for hook the damage
if (AddressOfHookDamageFunction!=0)
DetourAttach(&(LPVOID&)AddressOfHookDamageFunction, &HookDamageSub);
if (DetourTransactionCommit() == NO_ERROR)
{
LF::Log_Update(LOG_FLAG_DONE);
return true;
}
else
{
LF::Log_Update(LOG_FLAG_ERROR, "Hooking not initilizing.");
return false;
}
}
catch (const std::exception&)
{
LF::Log_Update(LOG_FLAG_ERROR, "Error while initialize Detour.");
return false;
}
}
to do it I have found a custom version of libogg.dll and d3d9.dll (DLL injection). Both work perfectly.
But when I used the some code to another GOG game with DRM free, to inject it with libogg.dll the game crash always on this row:
DetourDetach(&(LPVOID&)AddressOfHookSoundFunction, &HookSoundFileSub);
In all my test with the game that work, never crash in this row even if AddressOfHookSoundFunction is relative to a non-existent address.
In short independently by AddressOfHookSoundFunction and HookSoundFileSub that second game crash always.
Here the difference between the two games:
The first was made in 2017 and the second was made in 2018 by the some software house.
If I disassambly with IDA the first game I see always sub_xxxxxx , but in the second game sometimes I see the real name of the functions and not the address. So I suppose it was done by a more recent visual c++ version, and for this reason it may not work.
On many forums the people suggest to use MinHook becouse is better than detours, but I don't known why and if this is true for all cases.
Can someone have the some experience in situations like this ?
If the solution is use minhook, can someone help me to re-write this code:
AddressOfHookSoundFunction = std::strtoul(Config.GameFileSoundOffsetPos.c_str() ,NULL,16); // BattleZoneRedux 0x43AA30
if (Config.GameFileDamageOffsetPos.size() > 1)
AddressOfHookDamageFunction = std::strtoul(Config.GameFileDamageOffsetPos.c_str(), NULL, 16); // BattleZoneRedux 0x49B430
DetourTransactionBegin();
DetourUpdateThread(GetCurrentThread());
// this will hook the sound function
DetourAttach(&(LPVOID&)AddressOfHookSoundFunction, &HookSoundFileSub);
// If is present a offset for hook the damage
if (AddressOfHookDamageFunction!=0)
DetourAttach(&(LPVOID&)AddressOfHookDamageFunction, &HookDamageSub);
if (DetourTransactionCommit() == NO_ERROR)
{
LF::Log_Update(LOG_FLAG_DONE);
return true;
}
else
{
LF::Log_Update(LOG_FLAG_ERROR, "Hooking not initilizing.");
return false;
}
with MinHook ?
Thank you !

MS Detours detoured function not being called

I'm working with MS detours 4.0, compiled in VS2005 on Windows 10 (Updating an older application).
I'm trying to detour the WriteProfileStringA function normally found in the kernel32.dll with a DLL injection.
The DLL injects just fine, I can debug the target process and can step through the detouring functions.
DetourTransactionBegin, DetourUpdateThread, DetourAttach and DetourTransactionCommit all pass with flying colours returning 0 each time.
typedef BOOL (__stdcall * WriteProfileStringAType)(CHAR *,LPCSTR,LPCSTR);
static WriteProfileStringAType OriginalWriteProfileStringA = NULL;
BOOL WINAPI DetouredWriteProfileStringA(CHAR * lpAppName,
LPCSTR lpKeyName,
LPCSTR lpString)
{
FILE* file = fopen("c:\\temp\\testdetour.txt","w");
fprintf(file,"DetouredWriteProfileStringA");
fclose(file);
return OriginalWriteProfileStringA(lpAppName,lpKeyName, lpString);
}
...
BOOL WINAPI DllMain(HINSTANCE hinst, DWORD dwReason, LPVOID reserved)
{
if (DetourIsHelperProcess()) {
return TRUE;
}
if (dwReason == DLL_PROCESS_ATTACH) {
...
DetourTransactionBegin();
long update = DetourUpdateThread(GetCurrentThread());
OriginalWriteProfileStringA = (WriteProfileStringAType)GetProcAddress(GetModuleHandle("kernel32"),
"WriteProfileStringA");
long attach = DetourAttach(&(PVOID&)OriginalWriteProfileStringA, DetouredWriteProfileStringA);
long commit = DetourTransactionCommit();
...
}
else if (dwReason == DLL_PROCESS_DETACH) {
DetourTransactionBegin();
DetourUpdateThread(GetCurrentThread());
DetourDetach(&(PVOID&)OriginalWriteProfileStringA, DetouredWriteProfileStringA);
DetourTransactionCommit();
}
return TRUE;
}
BUT when the target application calls the WriteProfileStringA function, the registry is updated but the detour function is never called, break points are ignored and the testdetour.txt isn't created in the c:\temp directory(which does exist)...
Anybody have any ideas?

Can't load custom DLL

I created a simple DLL that open cmd.exe.
I did it with these options:
In the default dlllmain.cpp I added a code that creates a new cmd.exe:
// dllmain.cpp : Defines the entry point for the DLL application.
#include "stdafx.h"
#include <Windows.h>
BOOL APIENTRY DllMain( HMODULE hModule,
DWORD ul_reason_for_call,
LPVOID lpReserved
)
{
switch (ul_reason_for_call)
{
case DLL_PROCESS_ATTACH:
{
STARTUPINFO info = { sizeof(info) };
PROCESS_INFORMATION processInfo;
BOOL h = CreateProcessW(L"C:\\Windows\\System32\\cmd.exe", L"", NULL, NULL, TRUE, 0, NULL, NULL, &info, &processInfo);
}
case DLL_THREAD_ATTACH:
case DLL_THREAD_DETACH:
case DLL_PROCESS_DETACH:
break;
}
return TRUE;
}
These three lines below the DLL_PROCESS_ATTACH worked for me when I tested it with a console application.
I am expecting that every process that will load this DLL will open cmd.exe.
I tried to load the DLL with PowerShell:
Add-Type -TypeDefinition #"
using System;
using System.Diagnostics;
using System.Runtime.InteropServices;
public static class Kernel32
{
[DllImport("kernel32", SetLastError=true, CharSet = CharSet.Ansi)]
public static extern IntPtr LoadLibrary(
[MarshalAs(UnmanagedType.LPStr)]string lpFileName);
}
"#
$LibHandle = [Kernel32]::LoadLibrary("C:\tmp\myDll.dll")
But nothing happens, the value of the $LibHandle was 0.
What I am doing wrong ?
I found what was the problem.
My system is 64 bit and the file was compiled in 32 bit.
I needed to specify in Visual Studio that I am compiling it in x64 bit.
I didn't check it in the beginning because I thought that when I am compiling it on "Any CPU" mode, it compile the file in 64 bit automatically as the OS architecture.
Now it works fine.

Application Crashes when hooked with MS Detours and Injected with Withdll.exe

I am hooking FindNextFile() using MS Detours. I have configured the Detours library successfully and wrote a dll named "Detuors.dll" and an application named "FNFSend.exe". The following is the code:
DLL:
#include <cstdio>
#include <stdio.h>
#include <windows.h>
#include "detours.h"
#pragma comment (lib,"detours.lib")
//Prototypes
extern "C" __declspec(dllexport) BOOL (WINAPI *pFNF)(HANDLE hFindFile, LPWIN32_FIND_DATA lpFindFileData) = FindNextFile;
extern "C" __declspec(dllexport) BOOL WINAPI MyFNF(HANDLE hFindFile, LPWIN32_FIND_DATA lpFindFileData);
//Log File
FILE* pFNFLogFile;
int counter = 0;
INT APIENTRY DllMain(HMODULE hDLL, DWORD Reason, LPVOID Reserved)
{
switch(Reason)
{
case DLL_PROCESS_ATTACH:
DisableThreadLibraryCalls(hDLL);
DetourTransactionBegin();
DetourUpdateThread(GetCurrentThread());
DetourAttach(&(PVOID&)pFNF, MyFNF);
if(DetourTransactionCommit() == NO_ERROR)
OutputDebugString("FNF() detoured successfully");
else
OutputDebugString("FNF() not detoured");
break;
case DLL_PROCESS_DETACH:
DetourTransactionBegin(); //Detach
DetourUpdateThread(GetCurrentThread());
DetourDetach(&(PVOID&)pFNF, MyFNF);
DetourTransactionCommit();
break;
case DLL_THREAD_ATTACH:
DisableThreadLibraryCalls(hDLL);
DetourTransactionBegin();
DetourUpdateThread(GetCurrentThread());
DetourAttach(&(PVOID&)pFNF, MyFNF);
if(DetourTransactionCommit() == NO_ERROR)
OutputDebugString("FNF() detoured successfully");
else
OutputDebugString("FNF() not detoured");
break;
case DLL_THREAD_DETACH:
break;
}
return TRUE;
}
//Open file, write contents, close it
extern "C" __declspec(dllexport) int WINAPI MyFNF(HANDLE hFindFile, LPWIN32_FIND_DATA lpFindFileData)
{
counter ++;
fopen_s(&pFNFLogFile, "C:\\FNFLog.txt", "a+");
fprintf(pFNFLogFile, "%s\n", counter);
fclose(pFNFLogFile);
return pFNF(hFindFile, lpFindFileData);
}
Both the codes compiled successfully with no errors. The application calls FindNextFile() recursively and the dll hooks it and write the counter to a file.
I then used the tool named "withdll.exe" that is provided by detours library itself to create a process with a dll injected in it. So I injected my dll into the application using command:
withdll /d:Detuors.dll "C:\FNFSend.exe"
After injection, the function is hooked successfully, i.e. the file is made in the dirctory but suddenly the application crashes. After debugging in visual studio, I saw the exception in "output.c" as follows:
Unhandled exception at 0x6265984f (msvcr90d.dll) in FNFSend.exe: 0xC0000005:
Access violation reading location 0x00000001.
Kindly help in rectifying the problem.
%s is not a valid format string for printing out a number. Use %d instead.
By specifying %s you're telling fprintf to read the memory at address counter as a string. The first value you try calling fprintf with is 1 which is why there is an access violation at address 0x00000001.

MS Detours 2.1 - Popping out of stack

I wont to detour PlaySoundW function inside Minesweeper.
Game is crashing as soon as it calls PlaySoundW function.
If I uncomment Beep inside my code, game beeps and than crashes.
Now code is calling original function from hooked function so it should't do anything. But it is crashing anyway.
Can you tell me what is wrong?
After debugging app in Olly I found that when detour is active not all rubbish is popped out of stack.
How to fix it?
This is my code:
#include <Windows.h>
#include <tchar.h>
#include <detours.h>
namespace Hooks
{
BOOL(__stdcall *OrgPlaySoundW)(LPCTSTR pszSound, HMODULE hmod, DWORD fdwSound) = &PlaySoundW;
BOOL HookPlaySoundW(LPCTSTR pszSound, HMODULE hmod, DWORD fdwSound)
{
//Beep(1000, 250);
//return TRUE;
return OrgPlaySoundW(pszSound, hmod, fdwSound);
}
void DetourPlaySoundW(BOOL disable)
{
if(!disable)
{
DetourTransactionBegin();
DetourUpdateThread(GetCurrentThread());
DetourAttach(&(PVOID&)OrgPlaySoundW, &HookPlaySoundW);
DetourTransactionCommit();
} else
{
DetourTransactionBegin();
DetourUpdateThread(GetCurrentThread());
DetourDetach(&(PVOID&)OrgPlaySoundW, &HookPlaySoundW);
DetourTransactionCommit();
}
}
}
BOOL WINAPI DllMain(HINSTANCE hinstDLL, DWORD fdwReason, LPVOID lpvReserved)
{
switch(fdwReason)
{
case DLL_PROCESS_ATTACH:
Hooks::DetourPlaySoundW(FALSE);
break;
case DLL_PROCESS_DETACH:
Hooks::DetourPlaySoundW(TRUE);
break;
}
return TRUE;
}
Try setting the calling convention of HookPlaySoundW to __stdcall (because the CC of PlaySoundW is also __stdcall (from Windows.h): WINMMAPI BOOL WINAPI PlaySoundW( __in_opt LPCWSTR pszSound, __in_opt HMODULE hmod, __in DWORD fdwSound);).
I have worked with detours before and after a casual glance everything looks correct except what I mentioned above. If this doesn't resolve your problem I'd be glad to do some further investigation.
The default setting for Visual C++ is __cdecl in which the call*er* cleans up the stack, but in __stdcall the call*ee* cleans up the stack. This is probably (i.e. might possibly be) the reason for all the "rubbish being popped off the stack".