Calling a Visual Basic DLL in C++ - c++

I have acquired a DLL that was created in Visual Basic from a third party vendor(Sensor DLL.dll). This DLL contains functions for talking to a sensor, and I need to call these functions from a Visual C++ program I am writing. The vendor will not provide a header file, and I do not know Visual Basic. If I had a header file this would be a 15 minute project... instead I am still struggling with it a week later. Please Help!
I am told one function (Get_Data) in the DLL is of the form:
Public Function Get_Data(ByVal Handle As String) As String
I have tried several methods for calling this Get_Data function with no success:
Method 1) the DllImport attribute
#using <mscorlib.dll>
using namespace System::Runtime::InteropServices;
namespace Sensor
{
[DllImport("Sensor DLL.dll", EntryPoint = "Get_Data", CharSet = System::Runtime::InteropServices::CharSet::Unicode)]
BSTR Get_Data(BSTR Handle);
}
//then I call the function
Sensor::Get_Data(Handle);
This method seems to be the closest I have gotten to a sloution. It compiles, but gives the following error when it runs:
An unhandled exception of type 'System.EntryPointNotFoundException' occurred
Additional information: Unable to find an entry point named 'Get_Data' in DLL 'Sensor DLL.dll'.
I have tried various datatype combinations/permutations besides BSTR including BSTR*, wchar_t, int, etc. It is possible that I missed one, but each datatype returns the same error.
Method 2) dllimport storage-class attribute
__declspec(dllimport) BSTR Get_Data(BSTR Handle);
//then I call the function
Get_Data(Handle);
This method is confusing to me because I don't specify the DLL I want to import from. I have copied the DLL to the project folder and I have added it to the project manually, so hopefully that means it can be found. When I compile the linker returns the following errors:
error LNK2028: unresolved token (0A00034F) "wchar_t * __cdecl Get_Data(wchar_t *)" (?Get_Data##$$FYAPA_WPA_W#Z) referenced in function "int __cdecl main(void)" (?main##$$HYAHXZ)
error LNK2019: unresolved external symbol "wchar_t * __cdecl Get_Data(wchar_t *)" (?Get_Data##$$FYAPA_WPA_W#Z) referenced in function "int __cdecl main(void)" (?main##$$HYAHXZ)
I suspected maybe this meant I should be using wchar_t or wchar_t* instead of BSTR, but changing to either datatype results in the same error.
Method 3) GetProcAddress
typedef BSTR (*Get_Data_Ptr)(BSTR Handle);
HINSTANCE LoadMe;
LoadMe = LoadLibraryA("Sensor DLL.dll");
if (!LoadMe)
std::cout << "\nDLL failed to load!\n";
Get_Data_Ptr LibMainGet_Data;
LibMainGet_Data = (Get_Data_Ptr)GetProcAddress(LoadMe,"Get_Data");
//then I call the function
LibMainGet_Data(Handle);
This will compile, but gives the following error when run:
An unhandled exception of type 'System.AccessViolationException' occurred
Additional information: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
When I mouse over the various parts of this code in debug mode it seems that, like the first method, it was also unable to find the 'Get_Data' entry point in the DLL.
Has anyone called functions from a VB DLL using C++ when you haven't made the DLL yourself and you don't have .idl files, etc? Does anyone have a working example like this you could share?
Thanks!

A VB6 DLL is normally a COM server. You do in fact have the equivalent of a .h file, it has a type library embedded in it. Start this off with Project + Properties, Common Properties, Framework and References. Add New Reference button, Browse tab, select the DLL.
Next, View + Object Browser. You should see the generated Interop library in the list. Open the node to see what is there. You write normal managed code, like gcnew, to create the COM object and call the interface methods. You do need some minimum documentation on the available methods to have a guess at how they should be called.

I believe the missing piece is the calling convention. C++ has its own function calling convention different than VB6 (I assume VB6 since you haven't stated VB.NET explicitly). VB6 uses STDCALL convention whereas C++, depending on the vendor, uses a different calling convention termed __cdecl, which is why you see the __cdecl in the compiler error line for method #2. It assumes your external function is using that calling convention by default. Calling convention is a set of rules describing how functions call one another; specifically about how registers are used, what order parameters are delivered in, how by-value / by-reference is determined, etc.
I would suggest sticking with method #3 since method #1 is for Managed C++ which is not standard C++, and method #2 is unfamiliar to me and looks a bit ambiguous. What you want to try is declaring the function pointer typedef to use STDCALL.
typedef BSTR (__stdcall *Get_Data_Ptr)(BSTR Handle);

In the OLE/COM viewer, in order to view the COM type library in a dll/exe/... you have to open it by using "File->View TypeLib" instead of "File->Bind to File"

It sounds like the DLL isn't actually exporting a function named Get_Data. Open up a command prompt and use dumpbin to get the list of exports of the DLL, e.g.:
dumpbin /exports "Sensor DLL.dll"
(dumpbin.exe is located in VC\bin within your Visual Studio install folder, which istypically something like C:\Program Files\Microsoft Visual Studio 10.0).
Then, replace Get_Data with the actual entry point and see if you have any better luck.

A Visual basic program normally needs a runtime to execute.
If you have a COM object (implemented in VB) use the COM API to communicate with it from C++. You will have to register the COM first. Here is a thread that explains hot to do that: http://forums.devx.com/archive/index.php/t-87059.html
If you use a .NET language, use the method of Hans Passant with a reference that will create an interop dll for you. This is far much easier.
Method 1: Do not do that, if you have a COM object that you want to use from a .NET environment, reference it.
Method 2: You get errors because you lack the the .lib file to properly link to the DLL (statically dynamically linking)
Method 3: Would be a pure dynamic solution but you have to know the exact names of the methods in the DLL. These may vary according to the parameters and calling convention used. This is very similar (actually identical, I would say) to the issue you face with your Method 1 solution. The name of the method is for yure not "Get_Data" but something else. With a tool like the dependency viewer you can have a look at the exported names.
Even Method 3 with the right names is likely to fail because if it is a COM object you will need some environment called Appartment to use the COM objects. You "enter" this appartment by calling CoInitialize. This creates some magical stuff in the TLS (Thread Local Storage) to perform the COM magic. I hope this explains why your tries will be pointless if the DLL you have is happening to be a COM component, what is quite likely according to the ATL like naming we can see.
EDIT:
I forgot to say that you can also easily see what is inside the dll if it is a COM with the OLE/COM Viewer (normally if you have a compiler you will have such a tool around).

Related

Creating a process in memory C++

I've been working on this code for hours, and it is driving me crazy!
The entire source is here http://pastebin.com/Urxh68W4 but I'm pretty sure I know the problem.
extern "C" NTSYSAPI LONG NTAPI ZwUnmapViewOfSection(HANDLE, PVOID);
When I run it I get the following error:
Error 1 error LNK2019: unresolved external symbol __imp__ZwUnmapViewOfSection#8 referenced in function _wWinMain#16
I'm guessing that there is some dll or library I should be including so I added Ntoskrnl.lib into my project because it contains the ZwUnmapViewOfSection function.
I have absolutely no idea what to do. Should I have used the Ntdll.dll? If so, how do I even link a dll? I thought you could only use the libraries in Visual Studio 2010.
Also, what exactly is NTSYSAPI and NTAPI? There is hardly any information on the net.
This looks like user-mode code, so you'll likely not want to link against ntoskrnl.lib. You would rather link against ntdll.
The way I would probably do this is to use dynamic linking and call GetProcAddress passing in a HANDLE to ntdll.dll and ZwUnmapViewOfSection.
Example code:
typedef LONG (NTAPI *pfnZwUnmapViewOfSection)(HANDLE, PVOID);
HMODULE hMod = GetModuleHandle("ntdll.dll");
pfnZwUnmapViewOfSection pZwUnmapViewOfSection= (pfnZwUnmapViewOfSection)GetProcAddress(hMod, "ZwUnmapViewOfSection");
I haven't compiled this, but it should look something like that (maybe add some error checking, etc).
With regard to your other questions: NTAPI is a macro that defines the calling-convention, in this case __stdcall. The calling convention has to do with how the arguments to the function are passed, and who will be cleaning up those arguments.
For example, __stdcall requires the arguments to be pushed on the stack in reverse order and the callee will clean-up the stack.
Similarly, NTSYSAPI is a macro that just resolves to __declspec(dllimport) if I recall correctly.
Also, I should point out that calling functions exported by NtDll in user-mode is generally frowned upon. And, the code that you're writing will also have additional problems along the way (even after it appears to be working).
In case you're looking for another example of code that performs a very similar task to the one you're writing, you might check here. It was a technique used by the Duqu malware. Good luck!

exporting C++ dll to Delphi program

I have a dll with exporting function
extern "C" __declspec(dllexport) IDriver * __stdcall GetDriver()
There is a programm which is written on Delphi. It can't see the function GetDriver().
It's double awful because I can not get and modify sources of this program.
What may be the reason of successful loading my dll (according log file) and failed call exporting function? Thank you.
Window 7 x64, Visual Studio 2010, C++ project for x86 target
The most likely explanation is that the function will have been exported with a decorated name. I'd expect it to have been exported with the name GetDriver#0. So you could import it like this:
function GetDriver: IDriver; stdcall; external 'DllName.dll' name 'GetDriver#0';
Use a tool like Dependency Walker to check the exact name used to export the function.
If you cannot modify the Delphi code, then you'll need to make your C++ DLL match. Do that by using a .def file which allows you control over the exported name.
The other problem you will face is that Delphi's ABI for return values differs from that used by most other tools on the Windows platform. Specifically a return value is semantically a var parameter. On the other hand, your C++ compiler will regard the return value as an out parameter. My question on Delphi WideString return values covers exactly this issue.
Because of this, I'd expect the function declaration above to lead to access violations. Instead you should declare the return value to be a Pointer and cast it to an interface reference in your Delphi code. You'll need to double check and make sure that the reference counting is handled appropriately.
Again, if you cannot modify the Delphi code, you need to make the C++ code match. A Delphi interface return value is implemented as an additional var parameter following the other parameters. So, to make your C++ function match, declare it like this:
void __stdcall GetDriver(IDriver* &retVal);

Win32 DLL importing issues (DllMain)

I have a native DLL that is a plug-in to a different application (one that I have essentially zero control of). Everything works just great until I link with an additional .lib file (links my DLL to another DLL named ABQSMABasCoreUtils.dll). This file contains some additional API from the parent application that I would like to utilize. I haven't even written any code to use any of the functions exported but just linking in this new DLL is causing problems. Specifically, I get the following error when I attempt to run the program:
The application failed to initialize properly (0xc0000025). Click on OK to terminate the application.
I believe I have read somewhere that this is typically due to a DllMain function returning FALSE. Also, the following message is written to the standard output:
ERROR: Memory allocation attempted before component initialization
I am almost 100% sure this error message is coming from the application and is not some type of Windows error.
Looking into this a little more (aka flailing around and flipping every switch I know of) I linked with /MAP turned on and found this in the resulting .map file:
0001:000af220 ??3#YAXPEAX#Z 00000001800b0220 f ABQSMABasCoreUtils_import:ABQSMABasCoreUtils.dll
0001:000af226 ??2#YAPEAX_K#Z 00000001800b0226 f ABQSMABasCoreUtils_import:ABQSMABasCoreUtils.dll
0001:000af22c ??_U#YAPEAX_K#Z 00000001800b022c f ABQSMABasCoreUtils_import:ABQSMABasCoreUtils.dll
0001:000af232 ??_V#YAXPEAX#Z 00000001800b0232 f ABQSMABasCoreUtils_import:ABQSMABasCoreUtils.dll
If I undecorate those names using "undname" they give the following (same order):
void __cdecl operator delete(void * __ptr64)
void * __ptr64 __cdecl operator new(unsigned __int64)
void * __ptr64 __cdecl operator new[](unsigned __int64)
void __cdecl operator delete[](void * __ptr64)
I am not sure I understand how anything from ABQSMABasCoreUtils.dll can exist within this .map file or why my DLL is even attempting to load ABQSMABasCoreUtils.dll if I don't have any code that references this DLL. Can anyone help me put this information together and find out why this isn't working? For what it's worth I have confirmed via "dumpbin" that the parent application imports ABQSMABasCoreUtils.dll, so it is being loaded no matter what. I have also tried delay loading this DLL in my DLL but that did not change the results.
EDIT
I have double checked and all files involved are 64 bit.
I just had exactly the same problem. This is an issue with the Abaqus API rather than with the loading of DLLS.
I think it is because the Abaqus API overrides the new and delete functions (as you seem to have noticed). If you call new or delete in your program before initializing the Abaqus API, such as by calling odb_initializeAPI(); then you get the
ERROR: Memory allocation attempted before component initialization
error message and the program crashes.
In my program, calling odb_initializeAPI(); before the first new resolved the problem.
Well, sure you'll reference the imports of that library. Hard to write a C++ program without using the new or delete operator. Dealing with 3rd party software that thinks it needs to override the CRT version of those operators is hard enough, impossible when it won't allow you to call them until it thinks the time is right. Abandon all hope or seek help from the vendor.
One of the possible reason of an error during loading of ABQSMABasCoreUtils.dll is that some dependency module (inclusive delayed load DLLs) could not be found. Use Dependency Walker (see http://www.dependencywalker.com/) to examine all dependencies of ABQSMABasCoreUtils.dll.
I have two suggestions:
Verify that you can load ABQSMABasCoreUtils.dll with respect of LoadLibrary. You don't need call any function from ABQSMABasCoreUtils.dll. Usage of LoadLibrary I don't see as the end solution. It' s only a diagnostic test. With the test you can verify either you have some general problem of loading ABQSMABasCoreUtils.dll in your program or you have some kind of process initialization problem.
If loading of ABQSMABasCoreUtils.dll with respect of LoadLibrary will failed, then use profiling feature of Dependency Walker to protocol of all calls done during loading of ABQSMABasCoreUtils.dll. One other way would be usage of Process Monitor (see http://technet.microsoft.com/en-us/sysinternals/bb896645.aspx) to trace what file and registry operations will be done during loading of ABQSMABasCoreUtils.dll.
If LoadLibrary is not failed, then you have really an initialization problem of DLLs. Typically the problem exist if a DLL inside of DllMain try use a function from another DLL which is not yet initialized (not yet returns from DllMain). Before one start diagnostic of this problem, we should try to exclude a more simple problems with LoadLibrary.
The ABQSMABasCoreUtils.dll looks like it's importing 64-bit functions. Is your dll also 64-bit? If not, then that's the problem - you cannot mix DLLs compiled for different architectures in the same process.

Call function in c++ dll without header

I would like to call a method from an dll, but i don't have the source neither the header file. I tried to use the dumpbin /exports to see the name of the method, but i can found the methods signature?
Is there any way to call this method?
Thanks,
If the function is a C++ one, you may be able to derive the function signature from the mangled name. Dependency Walker is one tool that will do this for you. However, if the DLL was created with C linkage (Dependency Walker will tell you this), then you are out of luck.
The C++ language does not know anything about dlls.
Is this on Windows? One way would be to:
open the dll up in depends.exe shipped with (Visual Studio)
verify the signature of the function you want to call
use LoadLibrary() to get load this dll (be careful about the path)
use GetProcAddress() to get a pointer to the function you want to call
use this pointer-to-function to make a call with valid arguments
use FreeLibrary() to release the handle
BTW: This method is also commonly referred to as runtime dynamic linking as opposed to compile-time dynamic linking where you compile your sources with the associated lib file.
There exists some similar mechanism for *nixes with dlopen, but my memory starts to fail after that. Something called objdump or nm should get you started with inspecting the function(s).
As you have found, the exports list in a DLL only stores names, not signatures. If your DLL exports C functions, you will probably have to disassemble and reverse engineer the functions to determine method signatures. However, C++ encodes the method signature in the export name. This process of combining the method name and signature is called "name mangling". This Stackoverflow question has a reference for determining the method signature from the mangled export name.
Try the free "Dependency Walker" (a.k.a. "depends") utility. The "Undecorate C++ Functions" option should determine the signature of a C++ method.
It is possible to figure out a C function signature by analysing beginnig of its disassembly. The function arguments will be on the stack and the function will do some "pops" to read them in reverse order. You will not find the argument names, but you should be able to find out their number and the types. Things may get more difficult with return value - it may be via 'eax' register or via a special pointer passed to the function as the last pseudo-argument (on the top of the stack).
If you indeed know or strongly suspect the function is there, you can dynamically load the DLL with loadLibrary and get a pointer to the function with getProcAddress. See MSDN
Note that this is a manual, dynamic way to load the library; you'll still have to know the correct function signature to map to the function pointer in order to use it. AFAIK there is no way to use the dll in a load-time capability and use the functions without a header file.
Calling non-external functions is a great way to have your program break whenever the 3rd party DLL is updated.
That said, the undname utility may also be helpful.

Weird MSC 8.0 error: "The value of ESP was not properly saved across a function call..."

We recently attempted to break apart some of our Visual Studio projects into libraries, and everything seemed to compile and build fine in a test project with one of the library projects as a dependency. However, attempting to run the application gave us the following nasty run-time error message:
Run-Time Check Failure #0 - The value of ESP was not properly saved across a function call. This is usually a result of calling a function pointer declared with a different calling convention.
We have never even specified calling conventions (__cdecl etc.) for our functions, leaving all the compiler switches on the default. I checked and the project settings are consistent for calling convention across the library and test projects.
Update: One of our devs changed the "Basic Runtime Checks" project setting from "Both (/RTC1, equiv. to /RTCsu)" to "Default" and the run-time vanished, leaving the program running apparently correctly. I do not trust this at all. Was this a proper solution, or a dangerous hack?
This debug error means that the stack pointer register is not returned to its original value after the function call, i.e. that the number of pushes before the function call were not followed by the equal number of pops after the call.
There are 2 reasons for this that I know (both with dynamically loaded libraries). #1 is what VC++ is describing in the error message, but I don't think this is the most often cause of the error (see #2).
1) Mismatched calling conventions:
The caller and the callee do not have a proper agreement on who is going to do what. For example, if you're calling a DLL function that is _stdcall, but you for some reason have it declared as a _cdecl (default in VC++) in your call. This would happen a lot if you're using different languages in different modules etc.
You would have to inspect the declaration of the offending function, and make sure it is not declared twice, and differently.
2) Mismatched types:
The caller and the callee are not compiled with the same types. For example, a common header defines the types in the API and has recently changed, and one module was recompiled, but the other was not--i.e. some types may have a different size in the caller and in the callee.
In that case, the caller pushes the arguments of one size, but the callee (if you're using _stdcall where the callee cleans the stack) pops the different size. The ESP is not, thus, returned to the correct value.
(Of course, these arguments, and others below them, would seem garbled in the called function, but sometimes you can survive that without a visible crash.)
If you have access to all the code, simply recompile it.
I read this in other forum
I was having the same problem, but I just FIXED it. I was getting the same error from the following code:
HMODULE hPowerFunctions = LoadLibrary("Powrprof.dll");
typedef bool (*tSetSuspendStateSig)(BOOL, BOOL, BOOL);
tSetSuspendState SetSuspendState = (tSuspendStateSig)GetProcAddress(hPowerfunctions, "SetSuspendState");
result = SetSuspendState(false, false, false); <---- This line was where the error popped up.
After some investigation, I changed one of the lines to:
typedef bool (WINAPI*tSetSuspendStateSig)(BOOL, BOOL, BOOL);
which solved the problem. If you take a look in the header file where SetSuspendState is found (powrprof.h, part of the SDK), you will see the function prototype is defined as:
BOOLEAN WINAPI SetSuspendState(BOOLEAN, BOOLEAN, BOOLEAN);
So you guys are having a similar problem. When you are calling a given function from a .dll, its signature is probably off. (In my case it was the missing WINAPI keyword).
Hope that helps any future people! :-)
Cheers.
Silencing the check is not the right solution. You have to figure out what is messed up with your calling conventions.
There are quite a few ways to change the calling convetion of a function without explicitly specifying it. extern "C" will do it, STDMETHODIMP/IFACEMETHODIMP will also do it, other macros might do it as well.
I believe if run your program under WinDBG (http://www.microsoft.com/whdc/devtools/debugging/default.mspx), the runtime should break at the point where you hit that problem. You can look at the call stack and figure out which function has the problem and then look at its definition and the declaration that the caller uses.
I saw this error when the code tried to call a function on an object that was not of the expected type.
So, class hierarchy: Parent with children: Child1 and Child2
Child1* pMyChild = 0;
...
pMyChild = pSomeClass->GetTheObj();// This call actually returned a Child2 object
pMyChild->SomeFunction(); // "...value of ESP..." error occurs here
I was getting similar error for AutoIt APIs which i was calling from VC++ program.
typedef long (*AU3_RunFn)(LPCWSTR, LPCWSTR);
However, when I changed the declaration which includes WINAPI, as suggested earlier in the thread, problem vanished.
Code without any error looks like:
typedef long (WINAPI *AU3_RunFn)(LPCWSTR, LPCWSTR);
AU3_RunFn _AU3_RunFn;
HINSTANCE hInstLibrary = LoadLibrary("AutoItX3.dll");
if (hInstLibrary)
{
_AU3_RunFn = (AU3_RunFn)GetProcAddress(hInstLibrary, "AU3_WinActivate");
if (_AU3_RunFn)
_AU3_RunFn(L"Untitled - Notepad",L"");
FreeLibrary(hInstLibrary);
}
It's worth pointing out that this can also be a Visual Studio bug.
I got this issue on VS2017, Win10 x64. At first it made sense, since I was doing weird things casting this to a derived type and wrapping it in a lambda. However, I reverted the code to a previous commit and still got the error, even though it wasn't there before.
I tried restarting and then rebuilding the project, and then the error went away.
I was getting this error calling a function in a DLL which was compiled with a pre-2005 version of Visual C++ from a newer version of VC (2008).
The function had this signature:
LONG WINAPI myFunc( time_t, SYSTEMTIME*, BOOL* );
The problem was that time_t's size is 32 bits in pre-2005 version, but 64 bits since VS2005 (is defined as _time64_t). The call of the function expects a 32 bit variable but gets a 64 bit variable when called from VC >= 2005. As parameters of functions are passed via the stack when using WINAPI calling convention, this corrupts the stack and generates the above mentioned error message ("Run-Time Check Failure #0 ...").
To fix this, it is possible to
#define _USE_32BIT_TIME_T
before including the header file of the DLL or -- better -- change the signature of the function in the header file depending on the VS version (pre-2005 versions don't know _time32_t!):
#if _MSC_VER >= 1400
LONG WINAPI myFunc( _time32_t, SYSTEMTIME*, BOOL* );
#else
LONG WINAPI myFunc( time_t, SYSTEMTIME*, BOOL* );
#endif
Note that you need to use _time32_t instead of time_t in the calling program, of course.
I was having this exact same error after moving functions to a dll and dynamically loading the dll with LoadLibrary and GetProcAddress. I had declared extern "C" for the function in the dll because of the decoration. So that changed calling convention to __cdecl as well. I was declaring function pointers to be __stdcall in the loading code. Once I changed the function pointer from __stdcall to__cdecl in the loading code the runtime error went away.
Are you creating static libs or DLLs? If DLLs, how are the exports defined; how are the import libraries created?
Are the prototypes for the functions in the libs exactly the same as the function declarations where the functions are defined?
do you have any typedef'd function prototypes (eg int (*fn)(int a, int b) )
if you dom you might be have gotten the prototype wrong.
ESP is an error on the calling of a function (can you tell which one in the debugger?) that has a mismatch in the parameters - ie the stack has restored back to the state it started in when you called the function.
You can also get this if you're loading C++ functions that need to be declared extern C - C uses cdecl, C++ uses stdcall calling convention by default (IIRC). Put some extern C wrappers around the imported function prototypes and you may fix it.
If you can run it in the debugger, you'll see the function immediatey. If not, you can set DrWtsn32 to create a minidump that you can load into windbg to see the callstack at the time of the error (you'll need symbols or a mapfile to see the function names though).
Another case where esp can get messed up is with an inadvertent buffer overflow, usually through mistaken use of pointers to work past the boundary of an array. Say you have some C function that looks like
int a, b[2];
Writing to b[3] will probably change a, and anywhere past that is likely to hose the saved esp on the stack.
You would get this error if the function is invoked with a calling convention other than the one it is compiled to.
Visual Studio uses a default calling convention setting thats decalred in the project's options. Check if this value is the same in the orignal project settings and in the new libraries. An over ambitious dev could have set this to _stdcall/pascal in the original since it reduces the code size compared to the default cdecl. So the base process would be using this setting and the new libraries get the default cdecl which causes the problem
Since you have said that you do not use any special calling conventions this seems to be a good probability.
Also do a diff on the headers to see if the declarations / files that the process sees are the same ones that the libraries are compiled with .
ps : Making the warning go away is BAAAD. the underlying error still persists.
This happened to me when accessing a COM object (Visual Studio 2010). I passed the GUID for another interface A for in my call to QueryInterface, but then I cast the retrieved pointer as interface B. This resulted in making a function call to one with an entirely signature, which accounts for the stack (and ESP) being messed up.
Passing the GUID for interface B fixed the problem.
In my MFC C++ app I am experiencing the same problem as reported in Weird MSC 8.0 error: “The value of ESP was not properly saved across a function call…”. The posting has over 42K views and 16 answers/comments none of which blamed the compiler as the problem. At least in my case I can show that the VS2015 compiler is at fault.
My dev and test setup is the following: I have 3 PCs all of which run Win10 version 10.0.10586. All are compiling with VS2015, but here is the difference. Two of the VS2015s have Update 2 while the other has Update 3 applied. The PC with Update 3 works, but the other two with Update 2 fail with the same error as reported in the posting above. My MFC C++ app code is exactly the same on all three PCs.
Conclusion: at least in my case for my app the compiler version (Update 2) contained a bug that broke my code. My app makes heavy use of std::packaged_task so I expect the problem was in that fairly new compiler code.
ESP is the stack pointer. So according to the compiler, your stack pointer is getting messed up. It is hard to say how (or if) this could be happening without seeing some code.
What is the smallest code segment you can get to reproduce this?
If you're using any callback functions with the Windows API, they must be declared using CALLBACK and/or WINAPI. That will apply appropriate decorations to make the compiler generate code that cleans the stack correctly. For example, on Microsoft's compiler it adds __stdcall.
Windows has always used the __stdcall convention as it leads to (slightly) smaller code, with the cleanup happening in the called function rather than at every call site. It's not compatible with varargs functions, though (because only the caller knows how many arguments they pushed).
Here's a stripped down C++ program that produces that error. Compiled using (Microsoft Visual Studio 2003) produces the above mentioned error.
#include "stdafx.h"
char* blah(char *a){
char p[1];
strcat(p, a);
return (char*)p;
}
int main(){
std::cout << blah("a");
std::cin.get();
}
ERROR:
"Run-Time Check Failure #0 - The value of ESP was not properly saved across a function call. This is usually a result of calling a function declared with one calling convention with a function pointer declared with a different calling convention."
I had this same problem here at work. I was updating some very old code that was calling a FARPROC function pointer. If you don't know, FARPROC's are function pointers with ZERO type safety. It's the C equivalent of a typdef'd function pointer, without the compiler type checking.
So for instance, say you have a function that takes 3 parameters. You point a FARPROC to it, and then call it with 4 parameters instead of 3. The extra parameter pushed extra garbage onto the stack, and when it pops off, ESP is now different than when it started. So I solved it by removing the extra parameter to the invocation of the FARPROC function call.
Not the best answer but I just recompiled my code from scratch (rebuild in VS) and then the problem went away.