VB6 declaration for C++ function gives "Bad DLL calling convention" - c++

What is the correct VB6 declaration for this C++ function?
LPCWSTR* MW_ListReaders(_ULONG Context, int* NumberOfReaders);
The following gave me "Bad DLL calling convention":
Private Declare Function ListReaders Lib "MyDLL.dll" (ByVal Context As Long, _
ByRef NumberOfReaders As Integer) As Long

There is no calling convention specified in that C++ declaration. Most C/C++ compilers default to __cdecl. If the function does actually use __cdecl then you will not be able to call it in VB6:
How To Call C Functions That Use the _cdecl Calling Convention
It is not possible to directly call a C function in a DLL if that function uses the _cdecl calling convention. This is because Visual Basic uses the _stdcall calling convention for calling functions. This is a problem because if _cdecl is used, the calling function is responsible for cleaning up the stack. However, if _stdcall is used, the called function is responsible for cleaning up the stack.
NOTE: An .EXE file created in Visual Basic will allow you to call a DLL function that has been declared with the _cdecl calling convention without an error. It is only when you try to call such a function when running a program from the Visual Basic IDE, that Visual Basic generates the following error:
Run-time Error '49':
Bad DLL Calling Convention
The fact that the EXE version allows you to call such functions has been confirmed to be a bug by Microsoft. You should not rely on this behavior as this might change in future versions of Visual Basic.

In addition to Remy's answer, you have also got the Vb declaration slightly wrong:
Private Declare Function ListReaders Lib "MyDLL.dll" (ByVal Context As Long, ByRef NumberOfReaders As Long) As Long
"Integer" is a 2 byte integer in vb.

Related

“The value of ESP was not properly saved across a function call.”

I have a DLL written in Delphi 7 that I need to use in Visual C++ 2008.
From documentation that came with DLL, I can see that function is declared as (Delphi 7):
function ReadInfo(pCOM, pBuf, pErr: Pointer):boolean;
where pCom is pointer to data structure:
TCOM = record
dwBaudRate: Longword;
nCom,
nErr,
nLang : Byte;
end;
pBuf is pointer to "array of byte" (as it is written in DLL's documentation).
pErr - not used.
So now in c++ (after successfully loading DLL with LoadLibrary), I call:
myFunc = (MY_FUNC_POINTER)GetProcAddress(dllHandle, "ReadInfo");
which also doesn't return any errors.
MY_FUNC_POINTER is defined as:
typedef bool (*MY_FUNC_POINTER)(TCOM*, BYTE*, void*);
where TCOM is:
struct TCOM
{
unsigned long dwBaudRate;
BYTE nComm;
BYTE nError;
BYTE nLanguage;
};
I defined:
TCOM MyCom;
BYTE *myRes;
myRes = new BYTE[1024*1024];
But after calling
myFunc(&MyCom, myRes, NULL)
I get “The value of ESP was not properly saved across a function call.” error.
There would appear to be a calling convention mismatch. On the face of it, the function declares no calling convention in the Delphi, so the default Borland register convention is used. Your C++ code does not declare a calling convention for the import so the default of cdecl is used. But it is plausible that the Delphi code and documentation are not aligned and the Delphi code actually uses a different calling convention. Check the Delphi code, or contact the vendor. No matter what, the error message that you report does indicate a binary mis-match across the boundary between your module and the other module.
If the function really does use the Borland register calling convention (but see below for more), then you cannot readily call the function from languages other than Delphi. In that case you'd need a bridge to adapt that to a standard calling convention such as stdcall. By that I mean a Delphi DLL that can call the original DLL and expose it's functionality a way suited to interop. A better solution would be to fix the root problem and build the DLL again using standard calling conventions.
In fact, I now suspect that all the other commentators are correct. I suspect that the Delphi documentation does not match the Delphi code. I suspect that the function really is stdcall. So you can, probably, solve your problem by changing the function pointer typedef to be as follows:
typedef bool (__stdcall *MY_FUNC_POINTER)(TCOM*, BYTE*, void*);
My reasoning for this is that in stdcall the callee is responsible for cleaning the stack. That's not the case for cdecl, and since all the parameters, and the return value, fit in registers, it's not the case for Delphi register calling convention, for this function. Since there is a stack pointer mis-match, it follows that the most likely explanation is that the Delphi function is stdcall.
All the same, it's not comfortable to be working out calling conventions this way. If you cannot get any help from the DLL vendor then I'd be inclined to dig a little deeper by looking at the DLL function's code under a disassembler.

What is better to declare for calling convention of Windows program?

I have read an article about Calling convention (__stdcall, __fastcall, pascal, cdecl etc...)
And I wonder: Why is often __stdcall (WinAPI) declared for WinMain() function?
As I read, __stdcall operates with the stack, __fastcall operates on registers and not using stack at all.
So , I've tried to declare WinMain() with the __fastcall. Compiler (Visual C++) has given me an error.
error C2373: 'WinMain' : redefinition; different type modifiers
c:\program files\microsoft sdks\windows\v6.0a\include\winbase.h(2560) : see declaration of 'WinMain'
Why can't I use __fastcall for WinMain() and is there any possibility to use it for it?
Because to work on registers without using stack is miles faster, isn't it?
PS
My suggestion is, that there are some method contracts that disallow me to use __fastcall, but it's only my suggestion.
You can only specify the calling convention on functions that you write and/or you have the source code of.
You cannot change calling conventions of function that are in a library (static/dynamic) since those are already compiled/linked.
Important is that the declaration and definition have the same convention.
BTW : you wouldn't gain anything by having (win-)main having the fastcall convention since it's only called once!
You would consider fastcall on functions with many small parameters (that fit in registers) that are called very very often during long periodes of time.
The (buildin) startup routine for windows programs will call either WinMain or main (depending on GUI or Console app) with a specific convention.
If you write a WinMain or main with a different convention then the linker will complain.
WinMain must be __stdcall. It's called by the CRT start-up code, which is already built to pass parameters in a way defined by __stdcall convention.

What's is the VB6 equivalent of a C++ char array?

I'm trying to call a DLL written in C++ from a VB6 application.
Here's the C++ example code for calling the DLL.
char firmware[32];
int maxUnits = InitPowerDevice(firmware);
However, when I try to call it from VB6 I get the error bad DLL calling convention.
Public Declare Function InitPowerDevice Lib "PwrDeviceDll.dll" (ByRef firmware() As Byte) As Long
Dim firmware(32) As Byte
InitPowerDevice(firmware)
Edit: The C++ Prototype:
Name: InitPowerDevice
Parameters: firmware: returns firmware version in ?.? format in a character string (major revision and minor revision)
Return: >0 if successful. Returns number of Power devices connected
CLASS_DECLSPEC int InitPowerDevice(char firmware[]);
Been a long time, but I think you also need to change your C function to be stdcall.
// In the C code when compiling to build the dll
CLASS_DECLSPEC int __stdcall InitPowerDevice(char firmware[]);
' VB Declaration
Public Declare Function InitPowerDevice Lib "PwrDeviceDll.dll" _
(ByVal firmware As String) As Long
' VB Call
Dim fmware as String
Dim r as Long
fmware = Space(32)
r = InitPowerDevice(fmware)
I don't think VB6 supports calling cdecl functions in any normal way - there may be hacks for doing it. May be you can write a wrapper dll which wraps the cdecl function with a stdcall function and just forwards the call.
These are some hacks - but I haven't tried it.
http://planet-source-code.com/vb/scripts/ShowCode.asp?txtCodeId=49776&lngWId=1
http://planet-source-code.com/vb/scripts/ShowCode.asp?txtCodeId=62014&lngWId=1
You need to pass a pointer to the beginning of the array contents, not a pointer to the SAFEARRAY.
Perhaps what you need is either:
Public Declare Function InitPowerDevice Lib "PwrDeviceDll.dll" ( _
ByRef firmware As Byte) As Long
Dim firmware(31) As Byte
InitPowerDevice firmware(0)
or
Public Declare Function InitPowerDevice CDecl Lib "PwrDeviceDll.dll" ( _
ByRef firmware As Byte) As Long
Dim firmware(31) As Byte
InitPowerDevice firmware(0)
The CDecl keyword only works in a VB6 program compiled to native code. It never works in the IDE or in p-code EXEs.
Since your error is "bad calling convention", you should try changing the calling convention. C code use __cdecl by default, and IIRC VB6 had a Cdecl keyword you could use with Declare Function.
Otherwise you can change the C code to use __stdcall, or create a type library (.tlb) with the type information and calling convention. That might be better than Declare Function because you use the C datatypes when defining the type library, but VB6 recognizes them just fine.
As far as the argument type is concerned, firmware() As Byte with ByVal (not ByRef) should be fine.

exporting C++ dll to Delphi program

I have a dll with exporting function
extern "C" __declspec(dllexport) IDriver * __stdcall GetDriver()
There is a programm which is written on Delphi. It can't see the function GetDriver().
It's double awful because I can not get and modify sources of this program.
What may be the reason of successful loading my dll (according log file) and failed call exporting function? Thank you.
Window 7 x64, Visual Studio 2010, C++ project for x86 target
The most likely explanation is that the function will have been exported with a decorated name. I'd expect it to have been exported with the name GetDriver#0. So you could import it like this:
function GetDriver: IDriver; stdcall; external 'DllName.dll' name 'GetDriver#0';
Use a tool like Dependency Walker to check the exact name used to export the function.
If you cannot modify the Delphi code, then you'll need to make your C++ DLL match. Do that by using a .def file which allows you control over the exported name.
The other problem you will face is that Delphi's ABI for return values differs from that used by most other tools on the Windows platform. Specifically a return value is semantically a var parameter. On the other hand, your C++ compiler will regard the return value as an out parameter. My question on Delphi WideString return values covers exactly this issue.
Because of this, I'd expect the function declaration above to lead to access violations. Instead you should declare the return value to be a Pointer and cast it to an interface reference in your Delphi code. You'll need to double check and make sure that the reference counting is handled appropriately.
Again, if you cannot modify the Delphi code, you need to make the C++ code match. A Delphi interface return value is implemented as an additional var parameter following the other parameters. So, to make your C++ function match, declare it like this:
void __stdcall GetDriver(IDriver* &retVal);

MSVS 2010 C++ Compiler and Stack alignment issue?

My problem is MSVS 2010 C++ compiler is generating code in a way after returning from a function call resolved in runtime(GetProcAddress+GetModuleHandle) from another dll the compiler then tries to align stack this way:
CALL DWORD PTR DS:[2000367C] ; apiresolvedinruntime.dll
ADD ESP,12 ; <- this is the stack alignment
This is of course overwriting the return address and my program crashes, can someone explain me why compiler aligning the stack when it really shouldn't do it?
You didn't call the runtime loaded function using the correct calling convention. Calling convention specifies the default handling of what happens to the stack. Most likely, the DLL was compiled using the __stdcall calling convention (which is what e.g. the Windows DLLs use), which specifies that the called function is supposed to clean up the stack, but the calling code was declared with a function pointer using the __cdecl calling convention (which is the default). Under __cdecl, functions support variadic arguments, so the caller needs to do the cleanup of the stack, because the called function does not know how many arguments are passed.
You need to verify that the DLL and the calling code are compiled using the same calling conventions.