I have two Visual Studio C++ projects, that generate the following outputs:
L.lib
E.exe
I would like to add a dependency between E.exe and L.lib, without actually calling any methods exported from L.lib. L.lib exports a single function:
__declspec(dllexport) unsigned foo();
Then in E.exe, I have the following compilation unit:
// L.cpp
#pragma comment(lib, "L.lib")
extern unsigned foo();
The issue is that since foo is unused by E.exe, the dependency on L.lib is optimized away. What is the best way to force the dependency on L.lib, without actually executing any code? The best that I could come up with is the following (appended to L.cpp from above):
__declspec(dllexport) decltype(&foo) bar{ foo };
However, the above results in generating additional files (E.lib, among others) during the linking process, and adds public methods to the EXPORTS table of the resulting E.exe executable. Can anyone think of a better way to force an entry into the EXPORTS table?
I know that I could disable the linker optimization to remove unused code, but that would affect the entirety of E.exe, which I don't want to do.
MORE INFORMATION
The issue that I am seeing is that E.exe has a transitive dependency on a third-party L.lib (and associated L.dll) which is non-relocatable (e.g., linked with the /FIXED option). By the time L.dll is loaded, the fixed memory address it is required to be loaded at has been taken by other DLL's loaded by E.exe. I need a way to ensure that L.dll is loaded as early as possible, to prevent another DLL from taking the memory addresses that L.dll requires in order to function properly.
Just take the address of foo in a way that can't be optimized out: auto volatile pFoo = &foo;. It's not actually called, but the compiler can't prove it's not called so it has to play it safe.
[edit]
Visual Studio is apparently more aggressive than I thought, with LTCG. Adding pFoo=pFoo; fixes it. Linker map: 0003:0000037c ?pFoo##3R6AXXZA 0040337c
While MSalters answer works in 90% of cases, it failed to always add a reference to L.lib from E.exe. In addition, his edited suggestion requires modifying the body of a function that is used by the executable. I was looking for a solution that only required including an additional source file (compilation unit) sent to the linker.
After further investigation, I determined that the following code does exactly that:
// L.cpp
#pragma comment(lib, "L.lib")
extern "C" unsigned foo();
#pragma comment(linker, "/include:_foo")
This forces you to know the mangled name of foo, but it my case that was easy to determine due to the extern "C" declaration.
Related
I've downloaded the latest copy from Latest Open EXR repo and compiled most everything without any problems. However, when I get to PyILMBase it fails.
There are two distinct issues with my attempts:
Error C2491 'PyImath::FixedArray::name': definition of dllimport function not allowed PyImath (...)\src\pyilmbase-2.2.1\PyImath\PyImath.cpp 41
and an import to unistd.h that kind of gets addressed Here
my question being, has anybody successfully compiled this for windows? It seems odd that I can't find any specific info about that compiling error. Am I missing something in the way that C++ works?
This project makes extensive use of whatever it is that causes the error C2491, so going in and changing it would be a little complicated
IlmBase is compiled as a shared library. A shared library must provide their function declarations in order your program can find them.
It's perform by declaring them as dllimport or dllexport depending on the situation.
__declspec(dllimport): imports the implementation from a DLL so your application can use it.
__declspec(dllexport): tells the linker that you want this object to be made available for other DLL's to import. It is used when creating a DLL that others can link to.
These keywords are generally handled by a single macro. In your case PYIMATH_EXPORT.
These lines are from PyImathTask.h
#if defined(PYIMATH_EXPORTS) // create library
#define PYIMATH_EXPORT __declspec(dllexport)
#else // use library
#define PYIMATH_EXPORT __declspec(dllimport)
#endif
So, because you want to make these functions available from your application, you must add PYIMATH_EXPORTS as a preprocessor definition.
Actually, PyIMath's properties mispells it - they've write it as PyImath_EXPORTS - ...
To fix that, in vs2015, go to the PyImath's properties > Configuration Properties > C/C++ > Preprocessor > Replace PyImath_EXPORTS by PYIMATH_EXPORTS.
It's probably the same thing for PyIex...
I've always found it a bit of a pain that you have to use LoadLibrary() and define function pointers when wanting to make use of a DLL, it just feels a bit archaic when so much is automated now days. So when I stumbled across a forum post explaining that the VS auto-generated lib file that always (in my experience) gets compiled alonside your DLL can be used to save yourself the hassle, as a sort of auto-import mechanism, I figured I'd put it to the test.
I created two very minimal projects to test this, but it doesn't appear to be working.
The code in its entrirety for the DLL:
extern "C"
{
__declspec(dllexport) int get10()
{
return 10;
}
}
And for the exe (an otherwise empty Win32 console app):
#include <Windows.h>
#pragma comment(lib, "Kernel32.lib")
#pragma comment(lib, "dll1.lib")
__declspec(dllimport) int get10();
int main(int, char**)
{
int x = 0;
if (LoadLibrary(L"dll1.dll") != NULL)
{
x = get10();
}
return 0;
}
dll1.dll and dll1.lib are local and visible to the project, but when I go to link it get10() is unresolved. If I remove the call to this it builds and runs and the LoadLibrary() call succeeds.
I also tried copying across the dll1.exp, just clutching at straws really, but that didn't help either.
Is what I'm trying to do actually valid?
By the way I appreciate you'd usually just use a lib file proper, it just grabbed my curiosity for circumstances where you might have these dll and "reduced" lib files but not have the source to compile a regular lib file.
You either link as static, or as dynamic.
So, in essence, you need __declspec(dllexport) only if you're loading the function at runtime by name. Read some of this, for example.
As mentioned here, regarding __declspec(dllexport):
This convenience is most apparent when trying to export decorated C++ function names. Because there is no standard specification for name decoration, the name of an exported function might change between compiler versions. If you use __declspec(dllexport), recompiling the DLL and dependent .exe files is necessary only to account for any naming convention changes.
I have created several C++ libraries that currently are header-only. Both the interface and the implementation of my classes are written in the same .hpp file.
I've recently started thinking that this kind of design is not very good:
If the user wants to compile the library and link it dynamically, he/she can't.
Changing a single line of code requires full recompilation of existing projects that depend on the library.
I really enjoy the aspects of header-only libraries though: all functions get potentially inlined and they're very very easy to include in your projects - no need to compile/link anything, just a simple #include directive.
Is it possible to get the best of both worlds? I mean - allowing the user to choose how he/she wants to use the library. It would also speed up development, as I'd work on the library in "dynamically-linking mode" to avoid absurd compilation times, and release my finished products in "header-only mode" to maximize performance.
The first logical step is dividing interface and implementation in .hpp and .inl files.
I'm not sure how to go forward, though. I've seen many libraries prepend LIBRARY_API macros to their function/class declarations - maybe something similar would be needed to allow the user to choose?
All of my library functions are prefixed with the inline keyword, to avoid "multiple definition of..." errors. I assume the keyword would be replaced by a LIBRARY_INLINE macro in the .inl files? The macro would resolve to inline for "header-only mode", and to nothing for the "dynamically-linking mode".
Preliminary note: I am assuming a Windows environment, but this should be easily transferable to other environments.
Your library has to be prepared for four situations:
Used as header-only library
Used as static library
Used as dynamic library (functions are imported)
Built as dynamic library (functions are exported)
So let's make up four preprocessor defines for those cases: INLINE_LIBRARY, STATIC_LIBRARY, IMPORT_LIBRARY, and EXPORT_LIBRARY (it is just an example; you may want to use some sophisticated naming scheme).
The user has to define one of them, depending on what he/she wants.
Then you can write your headers like this:
// foo.hpp
#if defined(INLINE_LIBRARY)
#define LIBRARY_API inline
#elif defined(STATIC_LIBRARY)
#define LIBRARY_API
#elif defined(EXPORT_LIBRARY)
#define LIBRARY_API __declspec(dllexport)
#elif defined(IMPORT_LIBRARY)
#define LIBRARY_API __declspec(dllimport)
#endif
LIBRARY_API void foo();
#ifdef INLINE_LIBRARY
#include "foo.cpp"
#endif
Your implementation file looks just like usual:
// foo.cpp
#include "foo.hpp"
#include <iostream>
void foo()
{
std::cout << "foo";
}
If INLINE_LIBRARY is defined, the functions are declared inline and the implementation gets included like a .inl file.
If STATIC_LIBRARY is defined, the functions are declared without any specifier, and the user has to include the .cpp file into his/her build process.
If IMPORT_LIBRARY is defined, the functions are imported, and there isn't a need for any implementation.
If EXPORT_LIBRARY is defined, the functions are exported and the user has to compile those .cpp files.
Switching between static / import / export is a really common thing, but I'm not sure if adding header-only to the equation is a good thing. Normally, there are good reasons for defining something inline or not to do so.
Personally, I like to put everything into .cpp files unless it really has to be inlined (like templates) or it makes sense performance-wise (very small functions, usually one-liners). This reduces both compile time and - way more important - dependencies.
But if I choose to define something inline, I always put it in separate .inl files, just to keep the header files clean and easy to understand.
It is operating system and compiler specific. On Linux with a very recent GCC compiler (version 4.9) you might produce a static library using interprocedural linktime optimization.
This means that you build your library with g++ -O2 -flto both at compile and at library link time, and that you use your library with g++ -O2 -flto both at compile and link time of the invoking program.
This is to complement #Horstling's answer.
You can either create a static or a dynamic library. When you create statically-linked libraries, compiled code for all functions/objects will be saved to a file (with .lib extension in Windows). At main project (the project using the library) 's link time, these codes will be linked into your final executable together with the main project codes. So the final executable wouldn't have any runtime dependency.
Dynamically linked libraries will be merged into the main project at run time (and not link time). When you compile the library you get a .dll file (which contains actual compiled code) and a .lib file (which contains enough data for the compiler/runtime to find functions/objects in the .dll file). At link time, the executable will be configured to load the .dll and use compiled code from that .dll as needed. You will need to distribute the .dll file with your executable to be able to run it.
There is no need to choose between static or dynamic linking (or header-only) when designing your library, you create multiple project/makefiles, one to create a static .lib, another to create a .lib/.dll pair, and distribute both versions, for the user to choose between. (You'll need to use preprocessor macros like the ones #Horstling suggested).
You cannot put any templates in a pre-compiled library, unless you use a technique called Explicit Instantiation, which limits template parameters.
Also note that modern compiler/linkers usually do not respect the inline modifier. They may inline a function even if it's not designated as inline, or may dynamically call another that has inline modifier, as they see fit. (Regardless, I'll advise explicitly putting inline where applicable for maximum compatibility). So, there won't be any runtime performance penalty if you use a statically linked library instead of a header-only library (and enable compiler/linker optimizations, of course). As others have suggested, for really small functions that are sure to benefit from being called inline, it is best practice to put them in header files, so dynamically linked libraries will also not suffer any significant performance loss. (In any case, inlining functions will only affect performance for functions that are being called very often, inside loops that are going to be called thousands/millions of times).
Instead of putting inline functions in header files (with an #include "foo.cpp" in your header), you can change makefile/project settings and add foo.cpp to the list of source files to be compiled. This way, if you change any function implementation there will be no need to re-compile the whole project and only foo.cpp will be re-compiled. As I mentioned earlier, your small functions will still be inlined by the optimizing compiler, and you don't need to worry about that.
If you use/design a pre-compiled library, you should consider the case where the library is compiled with a different version of compiler to the main project. Each different compiler version (even different configurations, like Debug or Release) uses a different C runtime (things like memcpy, printf, fopen, ...) and C++ standard library runtime (things like std::vector<>, std::string, ...). These different library implementations may complicate linking, or even create runtime errors.
As a general rule, always avoid sharing compiler runtime objects (data structures that are not defined by standards, like FILE*) across libraries, because incompatible data structures will lead to runtime errors.
When linking your project, C/C++ runtime functions must be linked into your library .lib or .lib/.dll, or your executable .exe. C/C++ runtime itself can be linked as static or dynamic library (you can set this in makefile/project settings).
You will find that dynamically linking to C/C++ runtime in both the library and the main project (even when you compile the library itself as a static library) avoids most linking problems (with duplicate function implementations in multiple runtime versions). Of course you would need to distribute runtime DLLs for all used versions with your executable and library.
There are scenarios that statically linking to C/C++ runtime is needed, and the best approach in these cases would be to compile the library with the same compiler setting as the main project to avoid linking problems.
Rationale
Put as little as necessary in header files and as much as possible in library modules, because of the very reasons that you mentioned: compile-time dependency and long compilation time. The only good reasons for header-only modules are:
generic templates for user-defined template parameter;
very short convenience functions when inlining gives significant
performance.
In case 1, it is often possible to hide some functionality that does not depend on the user-defined type in a .cpp file.
Conclusion
If you stick to this rationale, then there is no choice: templated functionality that must allow user-defined types cannot be pre-compiled, but requires a header-only implementation. Other functionality should be hidden from the user in a library to avoid exposing them to the implementation details.
Rather than a dynamic library, you could have a precompiled static library and thin header file. In an interactive quick build, you get the benefit of not having to recompile the world if implementation details changes. But a fully optimized release build can do global optimization and still figure out it can inline functions. Basically, with "link-time code generation" the toolset does the trick you were thinking about.
I'm familiar with Microsoft's compiler, which I know for sure does this as of Visual Studio 2010 (if not earlier).
Templated code will necessarily be header-only: for instantiating this code, the type parameters must be known at compilation time. There is no way to embed template code in shared libraries. Only .NET and Java support JIT instantiation from byte-code.
Re: non-template code, for short one-liners I suggest keeping it header-only. Inline functions give the compiler a lot more opportunities to optimize the final code.
To avoid "insane compilation time", Microsoft Visual C++ has a "precompiled headers" feature. I do not think GCC has a similar feature.
Long functions should not be inlined in any case.
I had one project which had header-only bits, compiled library bits and some bits I could not decide where belonged. I ended up having .inc files, conditionally included in either .hpp or .cxx depending on #ifdef. Truth to be told, the project was always compiled in "max inline" mode, so after a while I got rid of the .inc files and simply moved the contents to .hpp files.
Is it possible to get the best of both worlds?
In terms; limitations arise because tools aren't smart enough. This answer gives the current best effort that is still portable enough to be used effectively.
I've recently started thinking that this kind of design is not very good.
It ought to be. Header-only libraries are ideal because they simplify deployment: makes the reusing mechanism of the language similar to almost all others', which is just the sane thing to do. But this is C++. Current C++ tools still rely on half-a-century-old linking models that remove important degrees of flexibility, such as choosing which entry points to import or export on an individual level without being forced to change the library's original source code. Also, C++ lacks a proper module system and still relies on glorified copy-paste operations to work (although this is just a side factor to the problem in question).
In fact, MSVC is a little better in this regard. It is the only major implementation trying to achieve some degree of modularity in C++ (by attempting e.g. C++ modules). And it is the only compiler that actually allows e.g. the following:
//// Module.c++
#pragma once
inline void Func() { /* ... */ }
//// Program1.c++
#include <Module.c++>
// Inlines or "vague" links Func(), whatever is better.
int main() { Func(); }
//// Program2.c++
// This forces Func() to be imported.
// The declaration must come *BEFORE* the definition.
__declspec(dllimport) __declspec(noinline) void Func();
#include <Module.c++>
int main() { Func(); }
//// Program3.c++
// This forces Func() to be exported.
__declspec(dllexport) __declspec(noinline) void Func();
#include <Module.c++>
Note that this can be used to selectively import and export individual symbols from the library, although still cumbersomely.
GCC also accepts this (but the order of the declarations must be changed) and Clang does not have any way to achieve the same effect without changing the library's source.
I have a dll, which accesses some classes outside of its project (I'm using Visual Studio, so I have two projects). The thing is, in the header that the dll includes, which is outside of the dll's project, there are only bodies of functions, like this:
x.h
class x
{
void myFunc();
}
And in another cpp file, outside of the dll file:
#include "x.h"
x::myFunc()
{
//.....
}
The dll is only getting the bodies of the functions, so when I compile, I get lots of unresolved external symbols (I'm quite sure that this is the issue, because I tested with another class fully built in a .h file, in another project, and no errors). So how can I solve this mystery?
It is normal for the import headers to only have function signatures; the actual function bodies are already compiled into the DLL binary and are resolved at link time by linking into the actual DLL.
The first thing to try is to make sure you are actually linking to the said DLL. It isn't enough to just include the header, you also need to link to the binary. So in your project configuration, you need to add a link to (for example) the .lib file that gets created along-side the DLL when the DLL is compiled (if in MSVC). This lib file lets the linker know how to connect the function signatures you included via the import header to the actual implementations contained in the DLL. If you're on a different platform, the mechanics might be a little different, but the concepts will be similar.
Edits:
The next step is to make sure the binary is actually exporting the symbols you're trying to link against. Make sure that all interface signatures are being exported via __declspec(dll_export) prefixes. Normally this is wrapped up in an IFDEF so that the header is declared export while the DLL is being compiled, but not when that header is included in a client project. Next, you could use dumpbin to check the mangled export names, and see if there is anything unexpected.
Here's a modified version of your example that illustrates this style of export (note, I haven't tested if this compiles, apologies for any typos):
#ifdef BUILDING_MYDLL
#define MYDLL_API __declspec(dllexport)
#else
#define MYDLL_API __declspec(dllimport)
#endif
class MYDLL_API x
{
void myFunc();
}
You would then set your configuration to define BUILDING_MYDLL when compiling the dll, but not when compiling the executable. This way the functions are only marked export when compiling the library dll. Now you can mark your public API functions with MYDLL_API and they should get exported during build.
Please note that dll_export, dll_import, and declspec are all very MSVC-specific constructs. Other compilers/runtimes handle symbol export in different ways.
There's multiple ways to link DLL into your app on Windows, check out this existing question/answer:
https://stackoverflow.com/a/2060508/1701823
I'm building a DLL from a group of static libraries and I'm having a problem where only parts of classes are exported.
What I'm doing is declaring all symbols I want to export with a preprocessor definition like:
#if defined(MYPROJ_BUILD_DLL)
//Build as a DLL
# define MY_API __declspec(dllexport)
#elif defined(MYPROJ_USE_DLL)
//Use as a DLL
# define MY_API __declspec(dllimport)
#else
//Build or use as a static lib
# define MY_API
#endif
For example:
class MY_API Foo{
...
}
I then build static library with MYPROJ_BUILD_DLL & MYPROJ_USE_DLL undefined causing a static library to be built.
In another build I create a DLL from these static libraries. So I define MYPROJ_BUILD_DLL causing all symbols I want to export to be attributed with __declspec(dllexport) (this is done by including all static library headers in the DLL-project source file).
Edit:
Note on unrefenced symbols: Linker options Keep Unreferenced Data (/OPT:NOREF) and Do Not Remove Redundant COMDATs (/OPT:NOICF) is set so that no unreferenced symbols will be removed.
Ok, so now to the problem. When I use this new DLL I get unresolved externals because not all symbols of a class is exported. For example in a class like this:
class MY_API Foo{
public:
Foo(char const* );
int bar();
private:
Foo( char const*, char const* );
};
Only Foo::Foo( char const*, char const*); and int Foo::bar(); is exported. How can that be? I can understand if the entire class was missing, due to e.g. I forgot to include the header in the DLL-build. But it's only partial missing.
Also, say if Foo::Foo( char const*) was not implemented; then the DLL build would have unresolved external errors. But the build is fine (I also double checked for declarations without implementation).
Note: The combined size of the static libraries I'm combining is in the region of 30MB, and the resulting DLL is 1.2MB.
I'm using Visual Studio 9.0 (2008) to build everything. And Depends to check for exported symbols.
Edit:
For the ones who wonder why I don't just build a DLL from each of the static libraries: I can't because they cross-reference each other (that's why I need to group them together in one a single DLL). I know, it's horrible I can't really understand the logic behind it.
Remember that when you link against a static LIB, by default the linker is only pulling in the functions, classes, and data that the client (which in this case is your DLL) actually references.
So what happens is this:
You build your static LIB, and that's fine. This LIB is 100% valid.
You now build your static DLL around the original binary LIB. It pulls in only the stuff that it actually references. It doesn't pull in the entire class definition which is what it needs to do. This DLL, though it builds, is invalid.
A client then uses the DLL, and expects to see a complete binary definition for the exported class, because that's what the client sees in the accompanying header file.
However the class has only been partially imported, and this is why you're getting those link errors.
To fix:
If you're able to, don't build your DLL off your static LIB. Build your DLL off the original source code. And build your static LIB off the original source code.
Otherwise you can possibly fiddle with linker settings but I strongly recommend option 1 above. Note that OPT:NOREF won't work here as OPT:NOREF has no effect on static LIBs.
What wont't fix it:
High-level linker tweaks like OPT:NOREF, anything involving COMDATs, etc. If you want those functions present, you have to make sure they're referenced, either by referencing them, or by telling the linker explicitly, "hey, this Symbol X is referenced".
As a side note:
Anytime I build a DLL or LIB, I build both a DLL and a LIB, using exactly the technique you're using. Having a single code base that can generate either a DLL or a LIB by toggling a setting is ideal. But building a DLL off of a (binary) static LIB when you own the source code for both... I'm having a hard time imagining when such a scenario would be necessary.
The problem is surely that you use the already-built static .lib in your DLL project. That cannot work, you have to rebuild the .lib so that the functions get the __declspec(dllexport) declarator and will be exported by the linker.
At that point, it just isn't all that useful anymore to create the DLL compatible version of the .lib in the first place. Just create two projects, one that creates the static .lib, another that creates the DLL. Technically it is possible to still use the static .lib in your DLL project but you'll have to export the functions with a .def file. That can be high maintenance if the library has a lot of exports.
This is probably years too late, but OP's complaint was that a private method (the destructor) was not exported from a dllexport'd class. Isn't that normal? No external user is allowed to call a private method.