Hooking CreateFile in notepad.exe does not catch API calls - c++

Question

My ultimate goal is to track file operations done by explorer.exe via hooking file api in kernel32.dll, however I have yet to get that working (either explorer.exe is not calling the APIs, or something is wrong on my end). In order to figure out what is going on, I've set a goal to track whenever notepad.exe creates a file, however this has failed for some reason as well!
I have 3 Visual Studio 2012 C++ projects: my DLL, a DLL injector that will force any executable to load my dll (although it may fail if Unicode/Multibyte and 32/64bit settings don't match up), and a test program that calls API. I have a batch file that will start my test program, and then use the injector program to load my DLL into the test program. The weird part is that the outputs do show that the APIs are being tracked on the test program (spawns the console and prints everything!) but nothing happens for notepad.exe (no console = no operations caught).
Here is the DLL that hooks the API (uses the mhook library). The format and concepts are taken from this guide. The important things to note are that I hook CreateFile(A/W), and spawn a console to print file I/O logs when the first operation occurs.
#include "stdafx.h"
#include "mhook/mhook-lib/mhook.h"
//////////////////////////////////////////////////////////////////////////
// Defines and typedefs
typedef HANDLE (WINAPI *CreateFileWFP)(
_In_ LPCWSTR lpFileName,
_In_ DWORD dwDesiredAccess,
_In_ DWORD dwShareMode,
_In_opt_ LPSECURITY_ATTRIBUTES lpSecurityAttributes,
_In_ DWORD dwCreationDisposition,
_In_ DWORD dwFlagsAndAttributes,
_In_opt_ HANDLE hTemplateFile
);
typedef HANDLE (WINAPI *CreateFileAFP)(
_In_ LPCSTR lpFileName,
_In_ DWORD dwDesiredAccess,
_In_ DWORD dwShareMode,
_In_opt_ LPSECURITY_ATTRIBUTES lpSecurityAttributes,
_In_ DWORD dwCreationDisposition,
_In_ DWORD dwFlagsAndAttributes,
_In_opt_ HANDLE hTemplateFile
);
//////////////////////////////////////////////////////////////////////////
// Original function
CreateFileWFP OriginalCreateFileW = (CreateFileWFP)::GetProcAddress(::GetModuleHandle(TEXT("kernel32")), "CreateFileW");
CreateFileAFP OriginalCreateFileA = (CreateFileAFP)::GetProcAddress(::GetModuleHandle(TEXT("kernel32")), "CreateFileA");
//////////////////////////////////////////////////////////////////////////
// Some Helper Stuff
struct Console{
HANDLE handle;
Console(){ handle = INVALID_HANDLE_VALUE; }
void write(LPCWSTR text){
if(handle==INVALID_HANDLE_VALUE){
AllocConsole();
handle = GetStdHandle(STD_OUTPUT_HANDLE);
}
DWORD numCharsWritten = 0;
WriteConsoleW(handle, text, (DWORD)wcslen(text), &numCharsWritten,NULL);
}
void write(LPCSTR text){
if(handle==INVALID_HANDLE_VALUE){
AllocConsole();
handle = GetStdHandle(STD_OUTPUT_HANDLE);
}
DWORD numCharsWritten = 0;
WriteConsoleA(handle, text, (DWORD)strlen(text), &numCharsWritten,NULL);
}
} console;
void operationPrint(LPCWSTR left, LPCWSTR middle, LPCWSTR right){
console.write(left);
console.write(middle);
console.write(right);
console.write(L"\n");
}
void operationPrint(LPCSTR left, LPCSTR middle, LPCSTR right){
console.write(left);
console.write(middle);
console.write(right);
console.write(L"\n");
}
//////////////////////////////////////////////////////////////////////////
// Hooked function
HANDLE HookedCreateFileW(
_In_ LPCWSTR lpFileName,
_In_ DWORD dwDesiredAccess,
_In_ DWORD dwShareMode,
_In_opt_ LPSECURITY_ATTRIBUTES lpSecurityAttributes,
_In_ DWORD dwCreationDisposition,
_In_ DWORD dwFlagsAndAttributes,
_In_opt_ HANDLE hTemplateFile
){
HANDLE out = OriginalCreateFileW(
lpFileName,
dwDesiredAccess,
dwShareMode,
lpSecurityAttributes,
dwCreationDisposition,
dwFlagsAndAttributes,
hTemplateFile);
if(out == INVALID_HANDLE_VALUE) return out; //ignore failiures
operationPrint(L"CreatedW file",L" at ",lpFileName);
return out;
}
HANDLE HookedCreateFileA(
_In_ LPCSTR lpFileName,
_In_ DWORD dwDesiredAccess,
_In_ DWORD dwShareMode,
_In_opt_ LPSECURITY_ATTRIBUTES lpSecurityAttributes,
_In_ DWORD dwCreationDisposition,
_In_ DWORD dwFlagsAndAttributes,
_In_opt_ HANDLE hTemplateFile
){
HANDLE out = OriginalCreateFileA(
lpFileName,
dwDesiredAccess,
dwShareMode,
lpSecurityAttributes,
dwCreationDisposition,
dwFlagsAndAttributes,
hTemplateFile);
if(out == INVALID_HANDLE_VALUE) return out; //ignore failiures
operationPrint("CreatedA file"," at ",lpFileName);
return out;
}
//////////////////////////////////////////////////////////////////////////
// Entry point
BOOL WINAPI DllMain(
__in HINSTANCE hInstance,
__in DWORD Reason,
__in LPVOID Reserved
)
{
switch (Reason)
{
case DLL_PROCESS_ATTACH:
Mhook_SetHook((PVOID*)&OriginalCreateFileW, HookedCreateFileW);
Mhook_SetHook((PVOID*)&OriginalCreateFileA, HookedCreateFileA);
break;
case DLL_PROCESS_DETACH:
FreeConsole();
Mhook_Unhook((PVOID*)&OriginalCreateFileW);
Mhook_Unhook((PVOID*)&OriginalCreateFileA);
break;
}
return TRUE;
}
I can't find exactly where I found the injector program, but it is nearly identical to this guide. Some of the comments are even the same, so I'm pretty sure one took from the other (not sure which from whom though). In any case, I just made minor alterations so that it works when compiled either under Unicode or Multibyte. I won't post the entire code here unless requested because I think it's a waste of space, but here is the important part.
#include "Injector.h"
#include <windows.h>
#include <tlhelp32.h>
#include <shlwapi.h>
#include <conio.h>
#include <stdio.h>
#include "DebugPrint.h"
#include <atlbase.h>
using namespace std;
Injector::Injector(void)
{
}
Injector::~Injector(void)
{
}
#define CREATE_THREAD_ACCESS (PROCESS_CREATE_THREAD | PROCESS_QUERY_INFORMATION | PROCESS_VM_OPERATION | PROCESS_VM_WRITE | PROCESS_VM_READ)
bool Injector::Inject(string procName, string dllName){
DWORD pID = GetTargetThreadIDFromProcName(procName.c_str());
return Inject(pID,dllName);
}
bool Injector::Inject(DWORD pID, string dllName){
const char* DLL_NAME = dllName.c_str();
HANDLE Proc = 0;
HMODULE hLib = 0;
LPVOID RemoteString, LoadLibAddy;
if(!pID)
return false;
Proc = OpenProcess(PROCESS_ALL_ACCESS, FALSE, pID);
if(!Proc)
{
DEBUG_PRINT("OpenProcess() failed: %d", GetLastError());
return false;
}
LoadLibAddy = (LPVOID)GetProcAddress(GetModuleHandle(TEXT("kernel32.dll")), "LoadLibraryA");
// Allocate space in the process for our <strong class="highlight">DLL</strong>
RemoteString = (LPVOID)VirtualAllocEx(Proc, NULL, strlen(DLL_NAME), MEM_RESERVE | MEM_COMMIT, PAGE_READWRITE);
// Write the string name of our <strong class="highlight">DLL</strong> in the memory allocated
WriteProcessMemory(Proc, (LPVOID)RemoteString, DLL_NAME, strlen(DLL_NAME), NULL);
// Load our <strong class="highlight">DLL</strong>
CreateRemoteThread(Proc, NULL, NULL, (LPTHREAD_START_ROUTINE)LoadLibAddy, (LPVOID)RemoteString, NULL, NULL);
CloseHandle(Proc);
return true;
}
DWORD Injector::GetTargetThreadIDFromProcName(const char* ProcName)
{
PROCESSENTRY32 pe;
HANDLE thSnapShot;
BOOL retval, ProcFound = false;
thSnapShot = CreateToolhelp32Snapshot(TH32CS_SNAPPROCESS, 0);
if(thSnapShot == INVALID_HANDLE_VALUE)
{
//MessageBox(NULL, "Error: Unable to create toolhelp snapshot!", "2MLoader", MB_OK);
DEBUG_PRINT("Error: Unable to create toolhelp snapshot!");
return false;
}
pe.dwSize = sizeof(PROCESSENTRY32);
retval = Process32First(thSnapShot, &pe);
while(retval)
{
#ifdef _UNICODE
char peSzExeFile[MAX_PATH];
wcstombs_s(NULL,peSzExeFile,MAX_PATH,pe.szExeFile,MAX_PATH);
#else
const char* peSzExeFile = pe.szExeFile;
#endif
DEBUG_PRINT("\nSearching for process: %s ",peSzExeFile);
if(!strcmp(peSzExeFile, ProcName))
{
DEBUG_PRINT(" Found!\n\n");
return pe.th32ProcessID;
}
retval = Process32Next(thSnapShot, &pe);
}
return 0;
}
Finally, my test program just calls some file APIs to see if the injected DLL can catch them. As mentioned before, it does catch the calls quite successfully.
#include <windows.h>
#include <tchar.h>
using namespace std;
#ifdef _UNICODE
#define printf(X,...) wprintf(TEXT(X),__VA_ARGS__); //I don't want to have to keep replacing printf whenever I switch to Unicode or Multibyte
#endif
#ifdef _DEBUG
int _tmain(int argc, TCHAR* argv[]){ //makes a console. printf() will have a place to go in this case
#else
int APIENTRY WinMain(HINSTANCE hInstance, HINSTANCE hPrevInstance, LPSTR lpCmdLine, int nCmdShow){ //no console
#endif
Sleep(2000); //let DLL finish loading
LPCTSTR fileSrc = TEXT("C:\\Users\\jajoach\\Desktop\\hi.txt");
LPCTSTR fileDst = TEXT("C:\\Users\\jajoach\\Desktop\\hi\\hi.txt");
printf("Moving file from %s to %s\n",fileSrc,fileDst);
MoveFile(fileSrc,fileDst);
Sleep(1000);
printf("Moving file from %s to %s\n",fileSrc,fileDst);
MoveFile(fileDst,fileSrc);
Sleep(1000);
printf("Copying file from %s to %s\n",fileSrc,fileDst);
CopyFile(fileSrc,fileDst,true);
Sleep(1000);
printf("Deleting file %s\n",fileDst);
DeleteFile(fileDst);
Sleep(1000);
printf("Creating file %s\n",fileDst);
HANDLE h=CreateFile(fileDst,0,0,NULL,CREATE_NEW,FILE_ATTRIBUTE_NORMAL,NULL);
Sleep(1000);
printf("Deleting file %s\n",fileDst);
CloseHandle(h);
DeleteFile(fileDst);
Sleep(5000);
return 0;
}
Here is the confirmed output for Unicode (as noted by the 'W's) and Release mode, and what I was expecting (but didn't get) from both notepad.exe and explorer.exe. For the record, the test also works for Multibyte, but instead gives 'A's as expected.
I had thought that maybe explorer.exe and notepad.exe don't use these API for their file I/O, but my research says otherwise. This post hooks CreateFile in notepad.exe using Detours and reports success for that application. Additionally, ProgramMonitor clearly shows notepad.exe calling CreateFile during a Saveas operation (after many failed requests with different parameters...):
Nevermind about explorer.exe for now; why isn't my hook working for notepad.exe when I do Saveas?
EDIT: I forgot to mention that I also hook MoveFile(A/W) and CopyFile(A/W) with the test program, but I removed that code from the DLL on this post for brevity.

Answer 1

As noted in the comments of the OP, it seems that that notepad.exe uses ASLR and your test program does not. With that the address of LoadLibraryA would be different in each process, and your injection code fails.
The situation is that you are getting the addres of LoadLibraryA in the injector address space and assume that it is the same that in the target process. That would be usually right, but ASLR is designed specifically to make that assumption fail. And so it does... the thread you create get a -most likely- invalid address, and fails.

Answer 2

A couple of reasons:
Bitness of your hook code must match the target.
CreateFileA/W is quite low, but not low enough to catch em all!
To solve 2. you must hook Zw/NtCreateFile from Ntdll.dll which is what you're seeing in procmon. There is nothing lower than these API's in user land.