I have a DLL I was handed down by a device manufacturer.
I can effortlessly call all its functions from a MS Console application (not from Qt or C++ Builder) but not from an MFC/WinAPI application? In an effort to work around the problem, I'm exploring the LoadLibrary() route. I am experimenting with this code, in VS2019 :
HINSTANCE hInstance = LoadLibrary(L"DEVICE.dll");
int(*fnDLLFuncAddress)(void) = (int(*)(void))GetProcAddress(hInstance2, "Device_RegisterDeviceEventHandler");
I have the impression that the code is working since this is the fnDLLFunAddress content :
fnDLLFuncAddress = DEVICE.dll!0x00007ffa66adda20 (load symbols for additional information)
Two questions, if I may. First, where would I look for "load symbols for additional information"? And the second, less obvious, is, once I have the Function address, in the form of a pointer, do I simply use the explicitely linked function name in lieu of the original function call to the DLL?
To start the program, I need to launch the following Event Handler, defined this way :
DEVICE_ERROR_TYPES __stdcall FPtr_Device_RegisterDeviceEventHandler(const FPtr_DeviceEventHandler inHandler);
In main(), the way I'm expected to launch the DeviceHandler is with this call :
deviceError = fnDLLFuncAddress(EventNotice);
Where "deviceError" is defined in DEVICE_ERROR_TYPES enum.
The EventNotice function is defined this way :
void EventNotice(EventCode outEventCode, uint32 outRAWDataCount, DEVICE_ERROR_TYPES outError);
When I try to execute the call, the compiler complains with E0140 "Too many arguments in function call" and E0513 "a value of type "int" cannot be assigned to an entity of type "DEVICE_ERROR_TYPES".
My intuition tells me I should try supplying the "EventNotice" function address in the function call? Is it simply a matter of "dereferencing" the function name like "*EventNotice"?
If this was to work, I guess I'd still have to resolve the "incompatible" return type of the RegisterDeviceHandler call?
Any help is appreciated.
=== additional note ===================
The reason I'm having difficulty, I think, is because I am unable to replicate the original "RegisterDeviceHandler" function prototype?
With GetProAddress(), I'm able to get a pointer to the function address in the DLL but that does not give me the corresponding "function declaration"?
So recently I wanted to add an imgui interface to an example window using DirectX, so I watched on a video I had to hook the EndScene function using DirectX9sdk to be able to add my custom imgui interface.
However I have some questions:
Where can I find any documentation for the DirectX9 functions and types,( if there is any, because I do not understand why we specifically have to hook the EndScene function) or where could I find any article explaining more in depth how directX works?
I've seen two version so far of EndScene hooks one with a patternScanning function which scans a signature in the shaderapi dll and another which creates a DirectXDevice and then accesses the vtable from there; are there any sources online, or is it something we have to do ourselves?
Here is the version I have:
while (!DirectXDevice) // loops until it finds the device
DirectXDevice = **(DWORD**)(FindPattern("shaderapidx9.dll", "A1 ?? ?? ?? ?? 50 8B 08 FF 51 0C") + 0x1);
void** pVTable = *reinterpret_cast<void***>(DirectXDevice); // getting the vtable array
oEndScene = (f_EndScene)DetourFunction((PBYTE)pVTable[42], (PBYTE)Hooked_EndScene)//getting the 42th virtual function and detouring it to our own
I don't really understand what __stdcall does here, I do know it is used to call WINAPI functions but what for here?
HRESULT __stdcall Hooked_EndScene(IDirect3DDevice9* pDevice){//some code}
Note: thats the function I hook to the original endscene.
Thank you really much, I'm sorry if there are alot of questions but I really can't wrap my head around this.
How do you know which functions you need to hook?
To put it bluntly, you have to be an experienced DirectX graphics programmer to find that out. Don't expect being able to hook into a framework that you don't understand. It just so happens that EndScene will always be called after all the other draw calls on the render target.
There are tons of D3D9 programming resources available, online and in paper form. Most of them are not free. I'm afraid this is not the answer you were hoping for.
What is the deal with pattern scanning, or creating a temporary D3D9 device?
Microsoft did not put any explicit effort into making EndScene hookable. It just happens to be hookable because every normal function is hookable. You need a way to find the function in memory, because the function will not always be at the same address.
One approach is to scan for known instructions that appear inside the function. Someone needs to be the first person to find out that pattern that you can scan for. You are far from the first person to hook EndScene, so many have reverse-engineered the function before and shared searchable patterns.
NOTE: The pattern does not necessarily need to be directly inside the target function. It might also lead you to something else first, in your case, the ID3D9Device instance. The important thing is that you can find your way to the EndScene function somehow.
Another approach is to get a pointer to the function. If it was a regular C function, that would be easy. It's hard here because OOP tends to make these things hard - you have to fight your way through various interfaces to get the correct vtable.
Both methods have advantages and disadvantages -- creating a D3D9 device is safer, but also more intrusive, because the target process might not expect someone to just randomly create new devices.
Why does the hook function need __stdcall?
Since you replace the original function with your hooked version, the calling convention of the hooked function must be the same as the calling convention of the original function. The caller of EndScene expects (and was compiled with) a __stdcall convention, so the new function must also behave the same way, otherwise the stack will be corrupted. Your act of replacing the function does not change the way the caller calls it.
if kernel32.dll is guaranteed to loaded into a process virtual memory,why couldn't i call function such as Sleep without including windows.h?
the below is an excerpt quoting from vividmachine.com
5. So, what about windows? How do I find the addresses of my needed DLL functions? Don't these addresses change with every service pack upgrade?
There are multitudes of ways to find the addresses of the functions that you need to use in your shellcode. There are two methods for addressing functions; you can find the desired function at runtime or use hard coded addresses. This tutorial will mostly discuss the hard coded method. The only DLL that is guaranteed to be mapped into the shellcode's address space is kernel32.dll. This DLL will hold LoadLibrary and GetProcAddress, the two functions needed to obtain any functions address that can be mapped into the exploits process space. There is a problem with this method though, the address offsets will change with every new release of Windows (service packs, patches etc.). So, if you use this method your shellcode will ONLY work for a specific version of Windows. Further dynamic addressing will be referenced at the end of the paper in the Further Reading section.
The article you quoted focuses on getting the address of the function. You still need the function prototype of the function (which doesn't change across versions), in order to generate the code for calling the function - with appropriate handling of input and output arguments, register values, and stack.
The windows.h header provides the function prototype that you wish to call to the C/C++ compiler, so that the code for calling the function (the passing of arguments via register or stack, and getting the function's return value) can be generated.
After knowing the function prototype by reading windows.h, a skillful assembly programmer may also be able to write the assembly code to call the Sleep function. Together with the function's address, these are all you need to make the function call.
With some black magic you can ;). there have been many custom implementations of GetProcAddress, which would allow you to get away with not needing windows.h, this however isn't be all and end all and could probably end up with problems due to internal windows changes. Another method is using toolhlp to enumerate the modules in the process to get kernel.dll's base, then spelunks its PE for the EAT and grab the address of GetProcAddress. from there you just need function pointer prototypes to call the addresses correctly(and any structure defs needed), which isn't too hard meerly labour intensive(if you have many functions), infact under windows xp this is required to disable DEP due to service pack differencing, ofc you need windows.h as a reference to get this, you just don't need to include it.
You'd still need to declare the function in order to call it, and you'd need to link with kernel32.lib. The header file isn't anything magic, it's basically just a lot of function declarations.
I can do it with 1 line of assembly and then some helper functions to walk the PEB
file by hard coding the correct offsets to different members.
You'll have to start here:
static void*
JMIM_ASM_GetBaseAddr_PEB_x64()
{
void* base_address = 0;
unsigned long long var_out = 0;
__asm__(
" movq %%gs:0x60, %[sym_out] ; \n\t"
:[sym_out] "=r" (var_out) //:OUTPUTS
);
//: printf("[var_out]:%d\n", (int)var_out);
base_address=(void*)var_out;
return( base_address );
}
Then use windbg on an executable file to inspect the data structures on your machine.
A lot of the values you'll be needing are hard to find and only really documented by random hackers. You'll find yourself on a lot of malware writing sites digging for answers.
dt nt!_PEB -r #$peb
Was pretty useful in windbg to get information on the PEB file.
There is a full working implementation of this in my game engine.
Just look in: /DEP/PEB2020 for the code.
https://github.com/KanjiCoder/AAC2020
I don't include <windows.h> in my game engine. Yet I use "GetProcAddress"
and "LoadLibraryA". Might be in-advisable to do this. But my thought was the more
moving parts, the more that can go wrong. So figured I'd take the "#define WIN32_LEAN_AND_MEAN" to it's absurd conclusion and not include <windows.h> at all.
We recently attempted to break apart some of our Visual Studio projects into libraries, and everything seemed to compile and build fine in a test project with one of the library projects as a dependency. However, attempting to run the application gave us the following nasty run-time error message:
Run-Time Check Failure #0 - The value of ESP was not properly saved across a function call. This is usually a result of calling a function pointer declared with a different calling convention.
We have never even specified calling conventions (__cdecl etc.) for our functions, leaving all the compiler switches on the default. I checked and the project settings are consistent for calling convention across the library and test projects.
Update: One of our devs changed the "Basic Runtime Checks" project setting from "Both (/RTC1, equiv. to /RTCsu)" to "Default" and the run-time vanished, leaving the program running apparently correctly. I do not trust this at all. Was this a proper solution, or a dangerous hack?
This debug error means that the stack pointer register is not returned to its original value after the function call, i.e. that the number of pushes before the function call were not followed by the equal number of pops after the call.
There are 2 reasons for this that I know (both with dynamically loaded libraries). #1 is what VC++ is describing in the error message, but I don't think this is the most often cause of the error (see #2).
1) Mismatched calling conventions:
The caller and the callee do not have a proper agreement on who is going to do what. For example, if you're calling a DLL function that is _stdcall, but you for some reason have it declared as a _cdecl (default in VC++) in your call. This would happen a lot if you're using different languages in different modules etc.
You would have to inspect the declaration of the offending function, and make sure it is not declared twice, and differently.
2) Mismatched types:
The caller and the callee are not compiled with the same types. For example, a common header defines the types in the API and has recently changed, and one module was recompiled, but the other was not--i.e. some types may have a different size in the caller and in the callee.
In that case, the caller pushes the arguments of one size, but the callee (if you're using _stdcall where the callee cleans the stack) pops the different size. The ESP is not, thus, returned to the correct value.
(Of course, these arguments, and others below them, would seem garbled in the called function, but sometimes you can survive that without a visible crash.)
If you have access to all the code, simply recompile it.
I read this in other forum
I was having the same problem, but I just FIXED it. I was getting the same error from the following code:
HMODULE hPowerFunctions = LoadLibrary("Powrprof.dll");
typedef bool (*tSetSuspendStateSig)(BOOL, BOOL, BOOL);
tSetSuspendState SetSuspendState = (tSuspendStateSig)GetProcAddress(hPowerfunctions, "SetSuspendState");
result = SetSuspendState(false, false, false); <---- This line was where the error popped up.
After some investigation, I changed one of the lines to:
typedef bool (WINAPI*tSetSuspendStateSig)(BOOL, BOOL, BOOL);
which solved the problem. If you take a look in the header file where SetSuspendState is found (powrprof.h, part of the SDK), you will see the function prototype is defined as:
BOOLEAN WINAPI SetSuspendState(BOOLEAN, BOOLEAN, BOOLEAN);
So you guys are having a similar problem. When you are calling a given function from a .dll, its signature is probably off. (In my case it was the missing WINAPI keyword).
Hope that helps any future people! :-)
Cheers.
Silencing the check is not the right solution. You have to figure out what is messed up with your calling conventions.
There are quite a few ways to change the calling convetion of a function without explicitly specifying it. extern "C" will do it, STDMETHODIMP/IFACEMETHODIMP will also do it, other macros might do it as well.
I believe if run your program under WinDBG (http://www.microsoft.com/whdc/devtools/debugging/default.mspx), the runtime should break at the point where you hit that problem. You can look at the call stack and figure out which function has the problem and then look at its definition and the declaration that the caller uses.
I saw this error when the code tried to call a function on an object that was not of the expected type.
So, class hierarchy: Parent with children: Child1 and Child2
Child1* pMyChild = 0;
...
pMyChild = pSomeClass->GetTheObj();// This call actually returned a Child2 object
pMyChild->SomeFunction(); // "...value of ESP..." error occurs here
I was getting similar error for AutoIt APIs which i was calling from VC++ program.
typedef long (*AU3_RunFn)(LPCWSTR, LPCWSTR);
However, when I changed the declaration which includes WINAPI, as suggested earlier in the thread, problem vanished.
Code without any error looks like:
typedef long (WINAPI *AU3_RunFn)(LPCWSTR, LPCWSTR);
AU3_RunFn _AU3_RunFn;
HINSTANCE hInstLibrary = LoadLibrary("AutoItX3.dll");
if (hInstLibrary)
{
_AU3_RunFn = (AU3_RunFn)GetProcAddress(hInstLibrary, "AU3_WinActivate");
if (_AU3_RunFn)
_AU3_RunFn(L"Untitled - Notepad",L"");
FreeLibrary(hInstLibrary);
}
It's worth pointing out that this can also be a Visual Studio bug.
I got this issue on VS2017, Win10 x64. At first it made sense, since I was doing weird things casting this to a derived type and wrapping it in a lambda. However, I reverted the code to a previous commit and still got the error, even though it wasn't there before.
I tried restarting and then rebuilding the project, and then the error went away.
I was getting this error calling a function in a DLL which was compiled with a pre-2005 version of Visual C++ from a newer version of VC (2008).
The function had this signature:
LONG WINAPI myFunc( time_t, SYSTEMTIME*, BOOL* );
The problem was that time_t's size is 32 bits in pre-2005 version, but 64 bits since VS2005 (is defined as _time64_t). The call of the function expects a 32 bit variable but gets a 64 bit variable when called from VC >= 2005. As parameters of functions are passed via the stack when using WINAPI calling convention, this corrupts the stack and generates the above mentioned error message ("Run-Time Check Failure #0 ...").
To fix this, it is possible to
#define _USE_32BIT_TIME_T
before including the header file of the DLL or -- better -- change the signature of the function in the header file depending on the VS version (pre-2005 versions don't know _time32_t!):
#if _MSC_VER >= 1400
LONG WINAPI myFunc( _time32_t, SYSTEMTIME*, BOOL* );
#else
LONG WINAPI myFunc( time_t, SYSTEMTIME*, BOOL* );
#endif
Note that you need to use _time32_t instead of time_t in the calling program, of course.
I was having this exact same error after moving functions to a dll and dynamically loading the dll with LoadLibrary and GetProcAddress. I had declared extern "C" for the function in the dll because of the decoration. So that changed calling convention to __cdecl as well. I was declaring function pointers to be __stdcall in the loading code. Once I changed the function pointer from __stdcall to__cdecl in the loading code the runtime error went away.
Are you creating static libs or DLLs? If DLLs, how are the exports defined; how are the import libraries created?
Are the prototypes for the functions in the libs exactly the same as the function declarations where the functions are defined?
do you have any typedef'd function prototypes (eg int (*fn)(int a, int b) )
if you dom you might be have gotten the prototype wrong.
ESP is an error on the calling of a function (can you tell which one in the debugger?) that has a mismatch in the parameters - ie the stack has restored back to the state it started in when you called the function.
You can also get this if you're loading C++ functions that need to be declared extern C - C uses cdecl, C++ uses stdcall calling convention by default (IIRC). Put some extern C wrappers around the imported function prototypes and you may fix it.
If you can run it in the debugger, you'll see the function immediatey. If not, you can set DrWtsn32 to create a minidump that you can load into windbg to see the callstack at the time of the error (you'll need symbols or a mapfile to see the function names though).
Another case where esp can get messed up is with an inadvertent buffer overflow, usually through mistaken use of pointers to work past the boundary of an array. Say you have some C function that looks like
int a, b[2];
Writing to b[3] will probably change a, and anywhere past that is likely to hose the saved esp on the stack.
You would get this error if the function is invoked with a calling convention other than the one it is compiled to.
Visual Studio uses a default calling convention setting thats decalred in the project's options. Check if this value is the same in the orignal project settings and in the new libraries. An over ambitious dev could have set this to _stdcall/pascal in the original since it reduces the code size compared to the default cdecl. So the base process would be using this setting and the new libraries get the default cdecl which causes the problem
Since you have said that you do not use any special calling conventions this seems to be a good probability.
Also do a diff on the headers to see if the declarations / files that the process sees are the same ones that the libraries are compiled with .
ps : Making the warning go away is BAAAD. the underlying error still persists.
This happened to me when accessing a COM object (Visual Studio 2010). I passed the GUID for another interface A for in my call to QueryInterface, but then I cast the retrieved pointer as interface B. This resulted in making a function call to one with an entirely signature, which accounts for the stack (and ESP) being messed up.
Passing the GUID for interface B fixed the problem.
In my MFC C++ app I am experiencing the same problem as reported in Weird MSC 8.0 error: “The value of ESP was not properly saved across a function call…”. The posting has over 42K views and 16 answers/comments none of which blamed the compiler as the problem. At least in my case I can show that the VS2015 compiler is at fault.
My dev and test setup is the following: I have 3 PCs all of which run Win10 version 10.0.10586. All are compiling with VS2015, but here is the difference. Two of the VS2015s have Update 2 while the other has Update 3 applied. The PC with Update 3 works, but the other two with Update 2 fail with the same error as reported in the posting above. My MFC C++ app code is exactly the same on all three PCs.
Conclusion: at least in my case for my app the compiler version (Update 2) contained a bug that broke my code. My app makes heavy use of std::packaged_task so I expect the problem was in that fairly new compiler code.
ESP is the stack pointer. So according to the compiler, your stack pointer is getting messed up. It is hard to say how (or if) this could be happening without seeing some code.
What is the smallest code segment you can get to reproduce this?
If you're using any callback functions with the Windows API, they must be declared using CALLBACK and/or WINAPI. That will apply appropriate decorations to make the compiler generate code that cleans the stack correctly. For example, on Microsoft's compiler it adds __stdcall.
Windows has always used the __stdcall convention as it leads to (slightly) smaller code, with the cleanup happening in the called function rather than at every call site. It's not compatible with varargs functions, though (because only the caller knows how many arguments they pushed).
Here's a stripped down C++ program that produces that error. Compiled using (Microsoft Visual Studio 2003) produces the above mentioned error.
#include "stdafx.h"
char* blah(char *a){
char p[1];
strcat(p, a);
return (char*)p;
}
int main(){
std::cout << blah("a");
std::cin.get();
}
ERROR:
"Run-Time Check Failure #0 - The value of ESP was not properly saved across a function call. This is usually a result of calling a function declared with one calling convention with a function pointer declared with a different calling convention."
I had this same problem here at work. I was updating some very old code that was calling a FARPROC function pointer. If you don't know, FARPROC's are function pointers with ZERO type safety. It's the C equivalent of a typdef'd function pointer, without the compiler type checking.
So for instance, say you have a function that takes 3 parameters. You point a FARPROC to it, and then call it with 4 parameters instead of 3. The extra parameter pushed extra garbage onto the stack, and when it pops off, ESP is now different than when it started. So I solved it by removing the extra parameter to the invocation of the FARPROC function call.
Not the best answer but I just recompiled my code from scratch (rebuild in VS) and then the problem went away.