I'm writing this Editor.exe program that loads a game.dll, gets the address of a function inside the dll, and pass a pointer to a Core object.
gameInitFuncPtr init =
(gameInitFuncPtr) GetProcAddress(LoadLibraryA("game.dll"),"gameInit");
init(&core); // core is already instanced somewhere, maybe on the stack
The game.dll includes the core.h where the Core class is defined.
The Core class is implemented and compiled into Editor.exe.
On the dll side, calling functions from the passed object pointer, results in an Unresolved external symbol
An example of a call the game.dll would do with the given object pointer would be:
void gameInit(ldk::Core* core)
{
_core->renderer.drawText("initializing...");
}
How can I compile the dll so that it does not try to find, for example, the drawText() implementation within the dll module ?
1 - Please, note that this is NOT a question about how to declare pointers to member functions.
2 - I know it could easily be fixed if i passed a struct with only pointer to the methods, but I'm really curious about this.
3 - I'm using Microsoft's cl compiler 18.00, the one that ships with Visual studio 2013
It is not clear where you initiallize _core. At first glance gameInit should do it.
Declare interface class Core, i.e. it should be abstract. Implement it in a successor class for example CoreImpl in exe. This will fix Unresolved external symbols.
Looks like I was overthinking it.
Wen compiling the editor.exe the Core sould be declared just link any class.
struct Core
{
struct Renderer
{
void drawText(const char* text);
}
...
}
But, since the editor and the game.dll share the same Core.h, I used a macro to modify the declaration of Core.h member functions to be pure virtual, for example:
struct Core
{
struct Renderer
{
virtual void drawText(const char* text) = 0;
}
...
}
So the Unresolved external symbol linking error is gone
BUT: I does not work as expected in RUNTIME! :(
I had a similar problem as you with almost the same setting - Game as dll and Engine as exe. Here are some notes on how to tackle this problem.
Call only virtual methods. As you pointed out, if the method you call is not declared virtual, the linker tries to find an implementation for it and fails (if it's not in the header - a thing we try to avoid). The method does not need to be abstract, virtual is enough. Also, note that in your struct Renderer you can have methods that are not virtual, as long as you don't call them from the dll (if you do, the linker complains). It is probably not advisable to have such an interface, it would be much better to have some sort of API class which has only virtual public methods so users of this class cannot make a mistake.
All classes used from the dll need to be shared or header only. What I mean by this is, that as far as I know, there is no magic way, to have classes declared in header, implemented in cpp which is compiled to the exe and then use these classes from the dll. E.g., if you have a custom string class, it needs to be in a shared library. If it's just in the exe you will not be able to instantiate it in the dll (return it from functions etc.). A solution to this is to use header-only classes. E.g., your string may be implemented in a header in the Editor project and this header may be included by your Game project. This way you essentially compile the same code to both exe and dll.
To see a small working example see my repository with VS 2017 solution which demonstrates this exact problem and nothing else. repo link.
Much larger working example of this problem can be seen in idTech4 engine - DOOM 3 version here. It also uses a game as a dll and an engine as an exe. And also needs to exchange pointers to the engine's systems which are used from the game. The project is big, but if you take a look at project Game-d3xp class Game.h all the way down, they have the game's API with a single function GetGameAPI_t which expects to get gameImport_t struct with pointers to engine systems and returns gameExport_t with game informations. The loading then happens in the Common.cpp
As you can see they use shared library idLib in the respective project for things such as idString. All engine classes used from the dll are usually very small and implemented in headers only (they are mostly structs).
Note that id themselves are moving away from this architecture and even their latest version of DOOM 3 - DOOM 3 BFG edition compiles to a single exe and the modules are static libraries instead of dlls.
Related
I am working on an API that in wraps C++ behaviors with C calling convention functions. The API is composed of a collection of shared libraries (dll) that may be used in a variety of languages. In instances where a C++ class object is pass across the dll boundary, a C opaque pointer or "Handle" is used to refer to the underlying C++ object, similar to the Win32 API. An example header file prototype of such a wrapper function is
extern "C" { __declspec(dllexport) int __cdecl MyObjConfig(MyObj_t* handle); }
Many of the API functions / classes interface with hardware peripherals. In many cases it's not practical to be able to test on the representative hardware system. I would like to find a way to mock the lower level components so that higher level libraries or exectuables using those libraries can be tested in a simulated environment. However, I'm loathe to include phrases in the underlying source code such as
if(is_test) { return 0; }
For example, I would like to mock the behavior of a function float GetSensorData() so that I can test an executable that links to GetSensorData's parent dll and calls get sensor data, by returning a reasonable imitation of the sensor's normal data, without setting up the sensor explicitly. Also, I would like to avoid having to alter the source of the executable beyond making sure it is linking to an imitation of GetSensorData's dll.
A key part of my interest in an automated framework for creating the dll mocks is that I don't want to have to maintain two seperate versions of each library: a test version and an actual version. Rather I would like to work on the actual and have the "mock" compilation generated programmatically.
Can anyone suggest a good way to do this? I've looked at Gtest and CMock / Unity. Both seem fine for testing the dlls themselves but don't seem well equipped to accomodate the
extern "C" { __declspec(dllexport) int __cdecl
function prototype declarations.
Thanks!
If you have a function that you wish to mock that is located in a library you can either do function pointer substitution or link-time substitution. I wrote about it more in-depth in this answer: https://stackoverflow.com/a/65814339/4441211
I personally recommend the function pointer substitution since it is much more simple and straightforward.
I am developing a static library A with a mix of Objective C and C++ in Xcode, I run into the need to "weak link" another static library and Let's call it B. A calls some methods defined in B. The idea is that if B is not linked/provided, A will not throw any undefined symbols error.
I know that in Objective C you can do that and in code I can rely on run time methods such as NSClassFromString or [someObject Class] to check if a certain function is present/available from another static library, but I don't know if I can achieve that in one of my .cpp source file. Please advise and thank you!
I created a very simple sample project for illustration purpose:
Library A:
Core_ObjC.h, this is the header that will be exposed
#import <Foundation/Foundation.h>
#interface Core_Objc : NSObject
-(int) calculate;
#end
Core_ObjC.mm
#import "Core_ObjC.h"
#include "Core_CPP.h"
#implementation Core_Objc
-(int) calculate{
return calculate_Core();//Call into cpp here
}
#end
Core_CPP.cpp
#include "Core_CPP.h"
#include "NonCore_CPP.h"
int calculate_Core(){
return calculate_NonCore();//Call into another cpp here but it's defined in Library B
}
Library B:
NonCore_CPP.cpp
#include "NonCore_CPP.h"
int calculate_NonCore(){
return 100;
}
If I link both libraries in a sample app, the app will compile fine. However, when I link only A from the sample app, I will encounter error like:
Undefined symbols for architecture arm64:
"calculate_NonCore()", referenced from:
calculate_Core() in CoreFramework(Core_CPP.o)
The error does make sense to me because B will have the missing definition, but I am just looking for a solution that the compilation won't complain when there is only A.
So, weak linking works at the C function level as well. For details, see Apple's documentation but basically, the symbol needs to be declared using __attribute__((weak_import)), as follows
extern int MyFunction() __attribute__((weak_import));
extern int MyVariable __attribute__((weak_import));
and then you can check if its address is zero to check if it was found during link time, using e.g. if (MyFunction != NULL) or if (&MyVariable != NULL).
Your title mentions static libraries. Note that static libraries are just collections of object files, so you either link the library or you don't; there's no weak linking a static library, because by definition, it's present at build time. Or not. If you want runtime behaviour, use a dynamic library. (dylib)
The above should actually cover the example you have given: you'll just need to provide alternative code for when calculate_NonCore is unavailable, as you can't call it without crashing the program. (Makes sense, it's missing)
You mention C++ too, although your code doesn't make any obvious use of it. C++ makes this a little more complicated because of its mangled names; additionally, classes themselves don't have any linkage, so you can't test for presence of a class. The compiler seems to allow applying the weak_import attribute to member functions, static member variables, etc. but I'm not sure to what extent the NULL checks will work. Free functions should certainly not be an issue - treat them like C functions - and I'd guess that static members are probably a good candidate because you can easily take their address.
I'd probably try to avoid calling into any code that makes use of any part of a class that might not be there, and set things up such that you can simply check for the presence of a "canary" static member variable or function, which, if present, you can conclude that the whole class is present.
Let's say I have this lib
//// testlib.h
#pragma once
#include <iostream>
void __declspec(dllexport) test();
int __declspec(dllexport) a();
If I omit the definition for a() and test() from my testlib.cpp, the library still compiles, because the interface [1] is still valid. (If I use them from a client app then they won't link, obviously)
Is there a way I can ensure that when the obj is created (which I gather is the compiler's job) it actually looks for the definitions of the functions that I explicitly exported, and fails if doesn't ?
This is not related to any real world issue. Just curious.
[1] MSVC docs
No, it's not possible.
Partially because a dllexport declaration might legally not even be implemented in the same DLL, let alone library, but be a mere forward declaration for something provided by yet another DLL.
Especially it's impossible to decide on the object level. It's just another forward declaration like any other.
You can dump the exported symbols once the DLL has been linked, but there is no common tool for checking completeness.
Ultimately, you can't do without a client test application which attempts to load all exported interfaces. You can't check that on compile time yet. Even just successfully linking the test application isn't enough, you have to actually run it.
It gets even worse if there are delay-loaded DLLs (and yes, there usually are), because now you can't even check for completeness unless you actually call at least one symbol from each involved DLL.
Tools like Dependency Walker etc. exist for this very reason.
You asked
"Is there a way I can ensure that when the obj is created (which I gather is the compiler's job) it actually looks for the definitions of the functions that I explicitly exported, and fails if doesn't ?"
When you load a DLL then the actual function binding is happenning at runtime(late binding of functions) so it is not possible for the compiler to know if the function definition are available in the DLL or not. Hope this answer to your query.
How do I (programmatically) ensure that a dll contains definitions for all exported functions?
In general, in standard C++11 (read the document n3337 and see this C++ reference), you cannot
Because C++11 does not know about DLL or dynamic linking. It might sometimes make sense to dynamically load code which does not define a function which you pragmatically know will never be called (e.g. an incomplete DLL for drawing shapes, but you happen to know that circles would never be drawn in your particular usage, so the loaded DLL might not define any class Circle related code. In standard C++11, every called function should be defined somewhere (in some other translation unit).
Look also into Qt vision of plugins.
Read Levine's Linkers and loaders book. Notice that on Linux, plugins loaded with dlopen(3) have a different semantics than Windows DLLs. The evil is in the details
In practice, you might consider using some recent variant of the GCC compiler and develop your GCC plugin to check that. This could require several weeks of work.
Alternatively, adapt the Clang static analyzer for your needs. Again, budget several weeks of work.
See also this draft report and think about C++ cross compilers (e.g. compiling on Windows a plugin for a RaspBerry Pi)
Consider also runtime code generation frameworks like asmjit or libgccjit. You might think of generating at runtime the missing stubs or functions (and fill appropriately function pointers with them, or even vtables). C++ exceptions could also be an additional issue.
If your DLL contains calls to the functions, the linker will fail if a definition isn't provided for those functions. It doesn't matter if the calls are never executed, only that they exist.
//// testlib.h
#pragma once
#include <iostream>
#ifndef DLLEXPORT
#define DLLEXPORT(TYPE) TYPE __declspec(dllexport)
#endif
DLLEXPORT(void) test();
DLLEXPORT(int) a();
//// testlib_verify.c
#define DLLEXPORT(TYPE)
void DummyFunc()
{
#include testlib.h
}
This macro-based solution only works for functions with no parameters, but it should be easy to extend.
I'm new to C++ and I'm having a hard time getting my dll references to work. I've been trying to get it to work for a couple of days, but the few explainations I've found often refer to doing x or y, but don't tell me how to do x or y. Since I'm not a C++ veteran, I need someone to walk me through it. What I want to do is the following:
MySolution
MyExe (Win32 .exe)
Main.h
Main.cpp
(constructs ImplementationB calls the methods as defined by InterfaceA, then deletes the instances)
(calls/fills HelperC.Foobar)
MyInterfaces (dll)
InterfaceA.h
~InterfaceA();
virtual void DoSomething();
MyUtils (dll)
HelperC.h
static float Foobar;
HelperD.cpp
float HelperC::Foobar = 1.0f;
MyImplementations (dll)
ImplementationB : InterfaceA
(uses the value from HelperC.Foobar)
The MyExe and MyImplementations projects contain most of the executing code. But, I need an interface, so I need an interface project (MyInterfaces). I need some helper classes that need to be accessible from both MyExe and MyImplementations, hence MyUtils. I would like this helper class to be statically available, though it is not mandatory.
I had a compiling version before I started adding MyUtils with the HelperC class. I had to mark the interface destructor with __declspec(dllexport), along with the DoSomething method. I also had to mark the constructor of ImplementationB in order to instantiate it from MyExe, which makes sense. However, when I tried to mark the entire class (both the implementation and the interface) with __declspec(dllexport), the example wouldn't compile (which does not make sense).
From what I've read, having static fields in a dll and using them from external code doesn't really work all too well. So, as an alternative, I made Foobar non-static and passed a HelperC instance to the method as described by InterfaceA. Since I had already gotten simple classes to work, I figured that should work as well. However, now the compiler is throwing errors on the constructor of ImplementationB (LNK2019).
In short: I'm getting link errors all over the place in sections that have nothing to do with my changes, and there's little documentation describing the specific steps I need to perform in order to get a simple dll reference to work.
Can someone point out what I need to add and where I need to add it in order to make it compile? Also, some do's and don't's about C++ dll references would help a lot (e.g. don't use statics across projects).
After much digging, I found out that the culprit was a magical project setting. It is called Ignore Import Library and is located at Project Properties->Linker->General, and is set to Yes by default, while it should be set to No in most cases. The setting tells the executable project to use the dll's lib file during compilation. This still sounds strange to me (sounds like duplicate build output), but as far as I understand it, the lib file describes how to link to the dll. If your dll produces a lib during build, you probably want to set the setting to No.
I also learned that to be able to use the HelperC class as a statically accessible helper, I needed to use dllimport in combination with the macro trick, as described by #drescherjm. The dllimport declaration is only ever needed to be able to use data members across libraries (static class fields or globally defined variables). It may be applied to functions as well, though it is not required, in which case it provides a slight performance boost during library linking.
For completeness, my project structure after getting it to work:
MySolution
MyExe (Win32 .exe, Debugger Type=Mixed)
Main.h
Main.cpp
(constructs ImplementationB calls the methods as defined by InterfaceA, then deletes the instances)
(calls/fills HelperC::Foobar)
MyInterfaces (dll, Ignore Import Library=Yes, because there is no .lib after building)
InterfaceA.h
class __declspec(dllexport) InterfaceA
~InterfaceA() {};
virtual void DoSomething() = 0;
MyUtils (dll, Ignore Import Library=No)
HelperC.h
class __declspec(dllimport/dllexport) HelperC // (see macro trick)
static float Foobar;
HelperD.cpp
float HelperC::Foobar = 1.0f;
MyImplementations (dll, Ignore Import Library=No)
ImplementationB.h
class __declspec(dllexport) ImplementationB : public InterfaceA
ImplementationB();
~ImplementationB();
void DoSomething();
ImplementationB.cpp
ImplementationB::ImplementationB() {};
ImplementationB::~ImplementationB() {};
ImplementationB::DoSomething() { /* Omitted */ };
(uses HelperC::Foobar in implementation)
On a side note: if you added a default C++ class library project in Visual Studio, you may need to flip the Project Properties->Debugging->Debugger Type setting to Mixed before you will be able to set/use breakpoints in the dll code. See this.
I hope this helps others who are wrestling with dll's in C++ (and Visual Studio).
I am trying to handle a project with multiple dynamic libraries, in the libraries some global objects are created and registered in a list so that the executable can use them to build other things. In the executable there's no reference to any function of any library, it just needs the objects, and then it knows what to do. These libraries are intended to work like plugins that are chosen at link time.
Ideally when compiling ExecutableProject I link LibraryA, an object of type A gets created (by some code that's compiled in the library) and ExecutableProject does things because of it, using functions in LibraryA. If instead I link LibraryB then an object of type B gets created and other things happen.
The problem is, since ExecutableProject does not directly use any function or class in the library, the library is not loaded, and objects are never created.
I have found some workarounds:
I can use /INCLUDE:symbol at link time when compiling ExecutableProject, where symbol is any symbol exported in LibraryA or LibraryB. I don't like this because I have to know the decorated name of a function in the library, which is not always available. Also, it's not elegant.
I can use a dummy variable. I could put __declspec(dllexport) int force_link_A_or_B; in LibraryA and LibraryB and __declspec(dllimport) extern int force_link_A_or_B; in ExecutableProject. I don't like this because if I had more libraries I'd have to add a variable for each of them. Basically although the code in ExecutableProject does not know which libraries will be linked, it still needs to know how many of them can be linked together.
On Linux, using gcc and ld, it's very easy to solve this problem, all it takes is a simple linker flag -Wl,--no-as-needed. Is there any way to do this using Visual Studio? I would be happy if there was something that worked like /INCLUDE but used entire libraries instead of symbols and did not need decorated names.
Thanks
EDIT: I have been asked to clarify how this design is intended to work. Ideally, when I compile ExecutableProject I link LibraryA or B and an object is created.
`ClassA A;`
This is placed outside all functions in a source file that is compiled in LibraryA. Same thing happens in LibraryB. The key here is that ClassA and ClassB inherit from a BaseClass that ExecutableProject knows well. When the object is created, the BaseClass constructor is run, and inside it a BaseClass* pointer pointing to this (therefore the derived object) is saved, and ExecutableProject uses that pointer. It does not need to know anything about the derived classes, because it uses only a BaseClass pointer, but when it calls virtual functions from it, different functions are executed depending on the actual type of the pointed object. If there were more objects instead that just two alternatives, I would save BaseClass pointers in a list or a map, and let ExecutableProject access it.
Don't try this. The design is fundamentally flawed.
In particular, since you are not making any explicit calls to the DLL, you would rely only on object A being created via an implicit call to DllMain.
But under Windows rules, you are barred from doing anything interesting in or from DllMain, so object A cannot be very useful.