I declared some C++ functions prototyped as follows:
extern "C" void __stdcall function();
I also have some third-party dll with exported function() - no name decorations at all.
I'm unable to build my exe or dll due to undefined reference to function#..., because of MinGW's stdcall #-suffix. How can I obtain object file without #... just plain function names?
It sounds like you're trying to use MinGW to compile a program that uses external C functions from a third-party dll. There's a way to export those external functions into a proper import library that MinGW's gnu ld linker can use but it involves creating a .def definition file. The advantage with this is once you create a proper import library you won't have to fiddle around with switches like --add-stdcall-alias or --kill-at because the import library will contain the symbols expected by the compiler and linker.
Here's a rough outline of the procedure for doing this:
You'll need a tool call dlltool.exe which should be included in the same MinGW/bin directory as the compiler.
You'll need to create a definition file (*.def) listing all the external functions you're interested in importing.
Create the import file stub (*.a) by running dlltool passing in the .def file you created as input.
Pass the newly created import file *.a to the linker when building your project so the symbols can be resolved properly.
Here's how the definition file looks like:
;Run the dlltool like this:
;dlltool -k -d third_party.def -l libthird_party.a
LIBRARY third_party.dll
EXPORTS
dll_function1#0
dll_function2#8
dll_function3#16
; ...
dll_function_n#24
Couple of important things to note. The EXPORTS section has to list the exported symbols/functions in the same name-decorated format as expected by the toolchain. In this case, MinGW compiler and ld linker expects __stdcall C functions to be appended with an '#' follow by the number of bytes in the arguments. The second important thing to note is that dlltool -k will remove the '#', which does the same thing as the --kill-at option you've already seen. The end result of this is you have an import library with the correct internal name-decoration, so things resolve properly, and that internal name will map to the exported visible name found in your 3rd party dll.
One final thing that needs mentioning. Throughout the entire example, we were assuming the undecorated names in the dll used the __stdcall which isn't necessarily true. The following chart(taken from here) shows how different compilers decorate __cdecl vs __stdcall differently:
MSVC DLL
Call Convention | (dllexport) | DMC DLL | MinGW DLL | BCC DLL
----------------------------------------------------------------------------
__stdcall | _Function#n | _Function#n | Function#n | Function
__cdecl | Function | Function | Function | _Function
It's up to you to make sure that the call conventions match-up properly or risk stack corruption and mysterious program crashes.
man ld gives you the --enable-stdcall-fixup option for linking against non-#-decorated libraries. I'm not sure if the leading underscores will be a problem, you'll have to try.
If you want to produce object files or DLLs with mingw and want export names without the "#", --kill-at is your friend.
Related
The C++ modules TS provides an excellent facility for eliminating the preprocessor, improving compile times, and generally supporting much more robust, modular, code development in C++, for non-template code at least.
The underlying machinery provides control over import and export of symbols in ordinary programs.
However, there is a major problem developing libraries for two kinds of dynamic loading: startup time loading, and run time loading. This problem involves the exporting of symbols from the library, which is often discussed in terms of visibility.
Generally, not all the extern symbols of the translation units used to construct a dynamic link library should be made visible to the user. In addition, with run time loading, especially with a plugin concept, the same symbol must be exported from many concurrently loaded libraries.
On Windows the use of language extensions
__declspec(dllexport)
__declspec(dllimport)
attached in source code as attributes of symbols, and more recently on gcc and clang systems on unix platforms, the use of
__attribute__((visibility("default")))
__attribute__((visibility("hidden")))
are intended to support the provision and use of symbols intended to be made public by the library. Using these is complicated and messy: on Windows macros must be used to export the symbols whilst compiling the library, but import them when using it. On the unix platforms, the visibility must be set to default to both export and import the symbols, the compiler deciding itself, based on whether a definition is found or not: the compiler must be invoked with
-fvisibility=hidden
switch. The export/import attributes are not required for static linkage, and probably should be macro'd out to an empty string. Making code and fiddling the build system so that this all works, especially considering that #includes must have the correct symbol visibility set during compilation of library translation units is very hard, the file structure required in repositories is a mess, the source code is littered with macros, and in general .. the whole thing is a disaster. Almost all open source repositories FAIL to correctly export symbols for dynamic linkage, and most programmers have no idea that dynamic library code structure (using two level namespaces) is quite different to static linkage.
An example of how to do it (hopefully correctly) can be seen here:
https://github.com/calccrypto/uint256_t
This repository used to have 2 headers and 2 implementation files, the user of the built library would see 2 headers. There are now 7 headers and 2 implementation files and the user of the built library will see 5 header files (3 with extension include to indicate they're not to be directly included).
So after that long winded explanation, the question is: will the final C++ modules specification help to solve problems with export and import of symbols for dynamic linkage? Can we expect to be able to develop for shared libraries without polluting our code with vendor specific extensions and macros?
Modules don't help you with symbol visibility across DLL boundaries. We can check this with a quick experiment.
// A.ixx
export module A;
export int f() { return 1; }
Here we have a simple module interface file exporting one symbol f in the module interface of module A (happens to share the same name as the file base name, but this isn't necessary). Let's compile this like so:
cl /c /std:c++20 /interface /TP A.ixx
The /c flag avoids invoking the linker (happens automatically by default), c++20 or later is required for module syntax to work, and the /interface flag lets the compiler know we are compiling a module interface unit. The /TP arg says "treat the source input as a C++ input" and is needed when /interface is specified. Finally, we have our input file.
Running the above produces an interface file A.ifc and an object file A.obj. Note that there is no import lib file or exp file you would expect if you were compiling a DLL.
Next, let's write a file that consumes this.
// main.cpp
import A;
int main() { return f(); }
To compile this into an executable, we can use the following command:
cl /std:c++20 main.cpp A.obj
The presence of the A.obj input there is not optional. Without it, we get a classic linker error of f being an unresolved symbol. If we run this, we'll get a main.exe which statically links the code in A.obj.
What happens if we try and compile A.ixx into a DLL? That is, what if we try to produce a DLL with the linker from A.obj? The answer you get a DLL but no import lib or exp. If you try running link /noentry /dll A.obj /out:A.dll you will get an A.dll with the expected /disasm section (visible via dump bin), but no export table.
Dump of file A.dll
File Type: DLL
0000000180001000: B8 01 00 00 00 mov eax,1
0000000180001005: C3 ret
Summary
1000 .rdata
1000 .text
That's the disassembly in A.dll which we expect, but checking the exports section with dumpbin /export A.dll reveals nothing. The reason is of course, we didn't export the symbol!
If we change the source code of A.ixx to the following:
// A.ixx
export module A;
export __declspec(export) int f() { return 1; }
... we can repeat the steps (compile A.obj, link A.dll) to find that this time, the linker produces an import lib and exp file as we'd expect. Invoking dumpbin /exports A.lib on the import lib generated should show the ?f##YAHXZ symbol present.
Now, we can link main.cpp again A.lib (as opposed to A.obj) via cl /std:c++20 main.cpp A.lib to produce a valid executable, this time relying on A.dll for the code instead of having f statically embedded.
We can check that this is in fact happening as expected in WinDbg.
Note on the lower left module pane the presence of A.dll. Note also that in the disassembly view in the center, we are about to call main!f. Uh oh, not good. While this does properly resolve to the !A module, it does so via an extra indirection in the import address table as seen here:
This is the classic problem that happens when you forget to decorate a function or symbol with the __declspec(dllimport) directive. When the compiler encounters the symbol without the dllimport directive that it doesn't recognize, it emits a relocation entry which is expected to be resolved at link time. Along with that entry, it emits a jmp and an unresolved address. This is a classic problem that I won't get into here, but the upshot is that we have an extra unnecessary indirection because the symbol recognized as exported from the module A was expected to be statically linked.
It turns out, we can't fix this easily. If we try to add another declaration of f to main.cpp or some other translation unit, the linker will complain that it sees f with "inconsistent dll linkage." The only way to resolve this is to compile a second version of the A module interface with dllimport decorations (much like how headers typically have macros that expand to dllexport or dllimport depending on the TU using the header).
The moral of the story is that DLL linkage and module linkage, while not completely at odds, aren't particularly compatible either. The module export does not include exported symbols in the export table, needed to resolve symbols across DLL boundaries. Furthermore, putting these symbols in the export table still leaves you the trouble of an extra indirection after the implicit dynamic link is done via the import address table.
I cannot get a DLL to link, that needs a class exported from the app this DLL will be used with. Visual Studio 2008, Windows 7.
I had a small sample DLL compiling (the default MSFT DLL project actually), and the app can use LoadLibrary() and GetProcAddress() on it correctly. I'm experienced with dllimport/dllexport. Using dumpbin /exports was critical to discover the mangled names to use with GetProcAddress() (MSFT's default empty DLL includes a sample function without extern "C") but this is a solved problem.
The next step was to have the app export a class that the DLL will need to subclass. I do the dllimport/dllexport #define's in the reverse sense from usual: a special symbol when the app is compiled tags the class with dllexport and non-app code (such as the DLL) using that header, without the special symbol, gets a dllimport spec. dumpbin /exports on the .exe file shows exactly this class's (mangled) symbols being exported, and no others, as expected.
The next step was to have the DLL include the header and create an object of the exported object's type (as a baby-step towards actually subclassing it). Compiles fine, but linker shows error:
DynTest.obj : error LNK2019: unresolved external symbol "public: __thiscall Test::Test(class Toast*,double)" (??0Test##QAE#PAVToast##N#Z) referenced in function "public: __thiscall CDynTest::CDynTest(void)" (??0CDynTest##QAE#XZ)
OK, that didn't surprise me, as I know on WIN32 I need to supply DLL's at link time to make a DLL, unlike my usual Unix. Since Windows seems to treat DLLs and executables a bit similarly, I tried adding the .exe at Properties->Configuration Properties->Linker->Input->Additional Dependencies. That gets an error that looks like LINK.EXE didn't auto-detect that it was being given an .exe:
T:\mypath\MyBinary.exe : fatal error LNK1107: invalid or corrupt file: cannot read at 0x348
I then tried adding instead the object file that defines this class... That seems to be understood by the linker, and is probably successfully satisfying the DLL's need to link, but now shows a myriad of other symbols this file depends upon.
So I've considered refactoring the app such that most of it is in a DLL or LIB, just so I can supply that as an "Additional Dependencies" to the DLL I'm actually worried about. But this seems to be draconian. Is that my only option?
Invoking dumpbin /exports gives you a list of mangled names of exe exports. You need to create a module definition file (.def) containing these names:
EXPORTS
#d3d_some_fancy_mangedled_method_1
#d3d_some_fancy_mangedled_method_2
...
Notice that it is basically dumpbin output with first columns removed.
Then you use lib tool to generate export library from module definition file:
LIB /DEF:prog.def /NAME:prog.exe /MACHINE:x86
Finally you link generated export library prog.lib into your application. /MACHINE option should match to your executable. Notice that you don't need to link or anyhow use program executable to link it, only export library is used.
I'd like to call a function in DLL (lets say foo function defined in a X project ). So DLL never knows the exact implementation of this foo function. But later I want to use this DLL in the X project (that contains the foo function).
When I try to compile the DLL, linker gives the "Unresolved external" error. How can I tell the compiler that foo function is defined in another project.
For example;
foo.cpp in DLL project
extern void foo();
void DLLFunction()
{
foo();
}
impl.cpp in X project
void foo()
{
//Do something
}
And if I can compile foo.cpp in the DLL project, I will use this DLL in the X Project.
If I understand the question correctly, you want to have a DLL, which imports a function from the executable it is linked against.
In general it is possible, but tricky, so first thing you should consider is if you really need to do that or if you can change the DLL function to e.g. accept a function pointer (which then does not require to import a symbol from executable into the DLL).
It is still possible to use symbol from executable in the DLL. It is possible to export a symbol (function) from the executable and use in the DLL (normally the same way you export from DLL, e.g. "__declspec(dllexport)" on Windows). However the tricky part is, that by default you need the dependency to link the particular object, i.e. in this case you'd need the DLL to link the executable and the executable to link the DLL, which is a cyclic dependency.
So you need to break the cyclic dependency. There are some possibilities:
Loading the DLL dynamically as a plugin - using dlopen() on Posix and LoadLibrary() under Windows. Then you do not need the DLL to build the executable, and you can link the EXE providing the symbol with the DLL.
The other way around - on Windows, you can dynamically import the exported symbol from the executable into the DLL (GetModuleHandle(), GetProcAddress()).
Allowing the DLL to be build with unresolved symbols. Then you build the DLL with the symbol unresolved, and build the EXE against the DLL. When executed, the executable will load the DLL and will provide the symbol.
To allow building the DLL with unresolved symbols:
In Linux GCC this is the default behaviour (shared objects do not enforce symbols to be resolved), unless you build with "-Wl,--no-undefined".
In Windows MSVC, you can ignore the unresolved symbols when building the DLL by providing the "/FORCE:UNRESOLVED" parameter to the linker. Note that on Windows/MSVC, after you build the EXE .lib file, you then need to rebuild the DLL again against that file to import the symbol.
What you can do is to instead of writing extern, you can load the DLL dynamically
e.g.
auto hd = LoadLibrary("YourDll.DLL");
typedef void(*userfunc)() ;
userfunc p = GetProcAddress(hd, "foo");
p();
Alt. link with the .lib file created when you built the DLL
discalimer: code for cleanup, error checks etc omitted.
If you are using a recent version of the Visual Studio IDE, you can simply reference it - Visual Studio will take care of the rest. To do that, right click on the "References" entry in your Solution Explorer for the project you want to link against your DLL. From there, select the other DLL project in your solution:
This will take care of all the linking required and saves you the trouble of changing your project settings if you rename your projects or change the name of the output-files.
If you don't have the other project in your solution you will have to stick with the regular static linking procedure:
Project Settings -> Linker -> General -> Additional Library Directories
Project Settings -> Linker -> Input -> Additional Dependencies
If you want to dynamically link against the DLL at runtime, you can use the method described by Andreas K.
I wanted to create a C++ dll (to be used in a dot net application). Some functionality I needed was already implemented in another C++ dll.
The dll I was referencing was set up like this (including the comment):
extern "C"
{
__declspec(dllexport) BOOL SomeFunctionToBeUsedExternally();
}
// internal functions
BOOL OtherFunctions();
I need to use one of the OtherFunctions in my code.
So, I added the proper include in my own code, added dependencies on the lib created by the dll above, and used the method I needed. As a result, of course, I got another __declspec(dllexport)... function.
It refused to link though, I got an error about the OtherFunction.
I verified everything, looked online - nothing seems to solve my problem.
Then, I added a __declspec(dllexport) in front of the function I needed, and it works.
I don't understand though. I thought, the dllexport marked functions will be exported to the dll, but aren't all functions sent to the lib ?
Why do I have to export functions to the dll, if I am not linking against the dll but against the lib ?
No, the linker does not automatically export all identifiers. The dllexport attribute tells the linker which identifiers are exported. Without this you would be forced to either export every identifier in the DLL or specify which identifiers should not be exported. When the linker creates the DLL it also creates an import library and includes information about which identifiers are exported based on that attribute.
When you want to use a DLL you need link with the appropriate .lib file for good reason. The .lib file tells the linker which identifiers are exported, the name of the DLL they are in and other information. It is also possible to export identifiers from the DLL by ordinal instead of by name. In this case the linker still needs to match the identifier with the appropriate ordinal. This is only possible by having an accompanying library file that contains that information since it is not present in DLL's export table.
No, only exported functions end up in the .lib. As you can tell.
It is not a static link library, it the import library for the DLL. It is a very simple and very small file since it contains no code at all. Just a list of the exported functions. The linker needs it to resolve the external in the client code, it needs to know the name of the DLL and the actual exported function name or ordinal (could be different) so it can add the entry to client's import table. Import libraries having the same filename extension as static libraries was perhaps a bit unfortunate.
So, I have an interesting issue. I am working with a proprietary set of dlls that I ,obviously, don't have the source for. The goal is to write an intermediate dll that groups together a large series of funnction calls from the proprietary dlls. The problem I am having, when compiling with g++, is that I get errors for the original dlls along the lines of:
cannot export libname_NULL_THUNK_DATA. Symbol not found.
If I add a main and just compile to an executable everything works as expected. I'm using mingw for compilation. Thanks for any help.
In response to the first reply: Either I'm confused about what you're saying or I didn't word my question very well. I'm not explicitly trying to export anything from my wrapper I am just calling functions from their dlls. The problem is that I get errors that it can't export these specific symbols from the dll to my wrapper. The issue is that I'm not even entirely sure what these _NULL_THUNK_DATA symbols are for. I did a search and read somewhere that they shouldn't be exported because they're internal symbols that windows uses. I have tried using the --exclude-symbols directive to the linker but it didn't seem to do anything. I apologize if I'm completely misunderstanding what you're trying to say.
So, I think my issue was related to this. When just compiling a standard executable that uses a dll I was able to include the headers and directly call the functions for example:
#include :3rdparty.h
int main(){
dostuff(); // a function in the 3rdparty.dll
}
this would compile and run fine. I just needed to link the libraries in the g++ command.
When linking with the -shared flag I would get these errors (with main removed of course). I think it has something to do with the fact that by default g++ attempts to import all symbols from the dll. What I didn't understand is why this happens in the dll vs in an executable. I will try doing it using GetProcAddress(). Thank you!
it should be as easy as you think it should be.
eg:
your dll code needs:
void doStuff()
{
3rdparty.login();
3rdparty.dostuff();
3rdparty.logoff();
};
so far - so good, you've included the right headers .... (if you have them, if you don't then you need to import the library using LoadLibrary(), then create a function pointer to each exported dll entrypoint using GetProcAddress() and then call that function pointer)
You then link with the 3rd party lib and that's it. Occasionally you will have to wrap the definitions with 'extern "C"' in order to get the linkage name mangling correct.
As you say you're using g++, you can't be getting confused with __declspec(dllimport) which is a MS VC extension.
"Compiling" tells me that you're approaching this from the wrong end. Your DLL should not export its own wrapper functions, but directly refer to exports from other DLLs.
E.g. in a Windows Kernel32.DEF file, the following forward exists:
EXPORTS
...
HeapAlloc = NTDLL.RtlAllocHeap
There's no code for the HeapAlloc function.