GCC has the ability to make a symbol link weakly via __attribute__((weak)). I want to use the a weak symbol in a static library that users can override in their application. A GCC style weak symbol would let me do that, but I don't know if it can be done with visual studio.
Does Visual Studio offer a similar feature?
You can do it, here is an example in C:
/*
* pWeakValue MUST be an extern const variable, which will be aliased to
* pDefaultWeakValue if no real user definition is present, thanks to the
* alternatename directive.
*/
extern const char * pWeakValue;
extern const char * pDefaultWeakValue = NULL;
#pragma comment(linker, "/alternatename:_pWeakValue=_pDefaultWeakValue")
MSVC++ has __declspec(selectany) which covers part of the functionality of weak symbols: it allows you to define multiple identical symbols with external linkage, directing the compiler to choose any one of several available. However, I don't think MSVC++ has anything that would cover the other part of weak symbol functionality: the possibility to provide "replaceable" definitions in a library.
This, BTW, makes one wonder how the support for standard replaceable ::operator new and ::operator delete functions works in MSVC++.
MSVC used to behave such that if a symbol is defined in a .obj file and a .lib it would use the one on the .obj file without warning. I recall that it would also handle the situation where the symbol is defined in multiple libs it would use the one in the library named first in the list.
I can't say I've tried this in a while, but I'd be surprised if they changed this behavior (especially that .obj defined symbols override symbols in .lib files).
The only way i know. Place each symbol in a separate library. User objects with overrides also must be combined to library. Then link all together to an application. User library must be specified as input file, your lib's must be transfered to linker using /DEFAULTLIB: option.
From here:
... if a needed symbol can be satisfied without consulting a library,
then the OBJ in the library will not be used. This lets you override a
symbol in a library by explicitly placing it an OBJ. You can also
override a symbol in a library to putting it in another library that
gets searched ahead of the one you want to override. But you can’t
override a symbol in an explicit OBJ, because those are part of the
initial conditions.
This behavior results from the algorithm adopted by the linker.
So for short, to override a function,
With GCC, you must use the __attribute__((weak)). You cannot rely on the input order of object files into the linker to decide which function implementation is used.
With VS toolchain, you can rely on the order of the object/lib files and you must place your implementation of the function before the LIB that implements the original function. And you can put your implementation as an OBJ or LIB.
There isn't an MS-VC equivalent to this attribute. See http://connect.microsoft.com/VisualStudio/feedback/details/505028/add-weak-function-references-for-visual-c-c. I'm going to suggest something horrible: reading the purpose of it here: http://www.kolpackov.net/pipermail/notes/2004-March/000006.html it is essentially to define functions that, if their symbols exist, are used, otherwise, are not, so...
Why not use pre-processor for this purpose, with the huge caveat of "if you need to do this at all"? (I'm not a fan of recommending pre-processor).
Example:
#ifdef USE_MY_FUNCTION
extern void function();
#endif
then call appropriately in the application logic, surrounded by #ifdef statements. If your static library is linked in, as part of the linking in process, tweak the defines to define USE_MY_FUNCTION.
Not quite a direct equivalent and very ugly but it's the best I can think of.
Related
Let's say I have this lib
//// testlib.h
#pragma once
#include <iostream>
void __declspec(dllexport) test();
int __declspec(dllexport) a();
If I omit the definition for a() and test() from my testlib.cpp, the library still compiles, because the interface [1] is still valid. (If I use them from a client app then they won't link, obviously)
Is there a way I can ensure that when the obj is created (which I gather is the compiler's job) it actually looks for the definitions of the functions that I explicitly exported, and fails if doesn't ?
This is not related to any real world issue. Just curious.
[1] MSVC docs
No, it's not possible.
Partially because a dllexport declaration might legally not even be implemented in the same DLL, let alone library, but be a mere forward declaration for something provided by yet another DLL.
Especially it's impossible to decide on the object level. It's just another forward declaration like any other.
You can dump the exported symbols once the DLL has been linked, but there is no common tool for checking completeness.
Ultimately, you can't do without a client test application which attempts to load all exported interfaces. You can't check that on compile time yet. Even just successfully linking the test application isn't enough, you have to actually run it.
It gets even worse if there are delay-loaded DLLs (and yes, there usually are), because now you can't even check for completeness unless you actually call at least one symbol from each involved DLL.
Tools like Dependency Walker etc. exist for this very reason.
You asked
"Is there a way I can ensure that when the obj is created (which I gather is the compiler's job) it actually looks for the definitions of the functions that I explicitly exported, and fails if doesn't ?"
When you load a DLL then the actual function binding is happenning at runtime(late binding of functions) so it is not possible for the compiler to know if the function definition are available in the DLL or not. Hope this answer to your query.
How do I (programmatically) ensure that a dll contains definitions for all exported functions?
In general, in standard C++11 (read the document n3337 and see this C++ reference), you cannot
Because C++11 does not know about DLL or dynamic linking. It might sometimes make sense to dynamically load code which does not define a function which you pragmatically know will never be called (e.g. an incomplete DLL for drawing shapes, but you happen to know that circles would never be drawn in your particular usage, so the loaded DLL might not define any class Circle related code. In standard C++11, every called function should be defined somewhere (in some other translation unit).
Look also into Qt vision of plugins.
Read Levine's Linkers and loaders book. Notice that on Linux, plugins loaded with dlopen(3) have a different semantics than Windows DLLs. The evil is in the details
In practice, you might consider using some recent variant of the GCC compiler and develop your GCC plugin to check that. This could require several weeks of work.
Alternatively, adapt the Clang static analyzer for your needs. Again, budget several weeks of work.
See also this draft report and think about C++ cross compilers (e.g. compiling on Windows a plugin for a RaspBerry Pi)
Consider also runtime code generation frameworks like asmjit or libgccjit. You might think of generating at runtime the missing stubs or functions (and fill appropriately function pointers with them, or even vtables). C++ exceptions could also be an additional issue.
If your DLL contains calls to the functions, the linker will fail if a definition isn't provided for those functions. It doesn't matter if the calls are never executed, only that they exist.
//// testlib.h
#pragma once
#include <iostream>
#ifndef DLLEXPORT
#define DLLEXPORT(TYPE) TYPE __declspec(dllexport)
#endif
DLLEXPORT(void) test();
DLLEXPORT(int) a();
//// testlib_verify.c
#define DLLEXPORT(TYPE)
void DummyFunc()
{
#include testlib.h
}
This macro-based solution only works for functions with no parameters, but it should be easy to extend.
I have a project in pure C - st usb library and I need to migrate it to c++ and change same structures into classes. I removed all c++ "protections" like:
#ifdef __cplusplus
extern "C" {
#endif
#ifdef __cplusplus
}
#endif
I changed all files extensions from .c to .cpp (except HAL library).
I realized that c++ .hex is 7kB smaller then c .hex. When I looked into .map file I saw that many functions are missing. I thought that staticfunctions caused that, but removing static key word didn't help. Does anyone have idea what could cause that some functions weren't compiled. When extensions are .c everything is fine.
I can think of two main reasons:
Inlining. The compiler can decide there is no need to emit the function as a standalone function if all usages can be inlined.
Unused code. The compiler can see that a function isn't used anywhere in your code and decide to eliminate it from the final result.
If the result is to be used as sort of library, that your environment calls specific functions without having an explicit call in your own code, I think the best method is to compile and link your code as a library (probably dynamic lib) and exporting those functions as library interface (visibility in gcc, dllexport in MSVC) will force the compiler/linker to include them in the produced binary even if they don't see why they are needed right now.
(Of course, this is a wild guess about your target environment.)
Another option is to turn off the specific compiler/linker optimizations for removing dead code and forcing emitting standalone function instance for inlined functions, but I think it's very indirect approach, have a wider effect and can complicate maintenance later.
C++ functions are given different signatures than C functions. Since you lost functionality and the code is much smaller, it's likely that a function that requires C linkage is being compiled as C++ and the signature mismatch is preventing proper linking.
A likely place for this to happen is in the interrupt vector table. If the handler function is compiled with C++ linkage, the handler address won't make it into a table that's compiled with C.
Double check the interrupt vectors and make sure that they reference the correct functions. If they are correct, check any other code compiled with C that might reference an external symbol compiled with C++.
Background
First of all, I think this question goes beyond the C++ standard. The standard deals with multiple translation units (instantiation units) and thus multiple object modules, but does not seem to acknowledge the possibility of having multiple independently compiled and linked binary modules (i.e., .so files on Linux and .dll files on Windows). After all, the latter more of less enters into the world of application binary interface (ABI) that the standard leaves to implementations to consider at present.
When only a single binary module is involved, the following code snippet illustrates an elegant and portable (standard-compliant) solution to make singletons.
inline T& get() {
static T var{};
return var;
}
There are two things to note about this solution. First, the inline specifier makes the function a candidate to be included in multiple translation units, which is very convenient. Note that, the standard guarantees there is only a single instance of get() and the local static variable var in the final binary module (see here).
The second thing to note is that since C++11, initialization of static local variables is properly synchronized (see the Static local variables section here). So, concurrent invocations of get() is fine.
Now, I try to extend this solution to the case when multiple binary modules are involved. I find the following variant works with VC++ on Windows.
// dllexport is used in building the library module, and
// dllimport is used in using the library in an application module.
// Usually controlled by a macro switch.
__declspec(dllexport/dllimport) inline T& get() {
static T var{};
return var;
}
Note for non-Windows users: __declspec(dllexport) specifies that an entity (i.e., a function, a class or an object) is implemented (defined) in this module and is to be referenced by other modules. __declspec(dllimport), on the other hand, specifies that an entity is not implemented in this module and is to be found in some other module.
Since VC++ supports exporting and importing template instantiations (see here), the above solution can even be templated. For example:
template <typename T> inline
T& get() {
static T var{};
return var;
}
// EXTERN is defined to be empty in building the library module, and
// to `extern` in using the library module in an application module.
// Again, this is usually controlled by a macro switch.
EXTERN template __declspec(dllexport/dllimport) int& get<int>();
As a side note, the inline specifier is not mandatory here. See this S.O. question.
The Question
Since there is no __declspec(dllexport/import) equivalents in GCC and clang, is there a way to make a variant of the above solution that works on these two compilers?
Also, in Boost.Log, I noticed the BOOST_LOG_INLINE_GLOBAL_LOGGER_DEFAULT macro (see the Global logger objects section here). It is claimed to create singletons even if the application consists of multiple modules. If someone knows about the inner workings of this macro, explanations are welcome here.
Finally, if you know about any better solutions for making singletons, feel free to post it as an answer.
Since there is no __declspec(dllexport/import) equivalents in GCC and clang, is there a way to make a variant of the above solution that works on these two compilers?
First, this is not as much a compiler-related question but rather an underlying operating system one. GCC (and supposedly clang) do support __declspec(dllexport/import) on Windows and basically do the same as what MSVC does with the functions and object marked this way. Basically, the marked symbol is placed in a table of exported symbols from the dll (an export table). This table can be used, for instance, when you query for a symbol in a dll in run time (see GetProcAddress).
Along with the dll there comes an associated lib file that contains auxiliary data for linking your application with the dll. When you link your application with the library, the linker uses the lib file to resolve references to the dll symbols and compose the import table in your application binary. When the application starts, the OS (or rather the runtime loader component of the OS) uses the import table to find out what dlls your application depends on and what symbols it imports from those dlls. It then uses export tables in the dlls to resolve addresses of the referenced symbols in the dlls and complete the linking process.
The important side effect of this process is that only imported symbols are dynamically resolved and every symbol you dynamically link to is associated with a particular dll. You can have same-named symbols in multiple dlls and the application itself, and these symbols will refer to distinct entities as long as they are not exported. If they are exported, linking process will fail because of ambiguity. This makes process-wide singletons difficult on Windows. This also breaks some C/C++ language rules, because taking address of an object or function with external linkage (in the language terms) can produce different addresses in different parts of the program. On the other hand, the dlls are more self-contained and depend on the loading context in a lesser degree.
Things are significantly different on Linux and other POSIX-like OSs. When linked, for each shared object (which can be an so library or the application executable) a table of symbols is compiled. It lists both the symbols this shared object implements and the symbols it is missing. Additionally, the linker may embed into the shared object a list of other shared objects (optionally, with search paths) that could be used to resolve the missing symbols. The runtime loader includes a linker which loads the shared objects sequentially and constructs a global table of symbols comprising symbols from all shared objects. As that table is constructed, the duplicate symbols from multiple shared objects are resolved to a single implementation (as all implementations are considered equivalent, the first shared object in the load list that implements the symbol is used). Any missing symbols are also resolved as the subsequent shared objects in the link order are loaded.
The effect of this process is that each symbol with external linkage resolve to a single implementation in one of the shared objects, even if multiple shared objects implement it. This is more in line with the C/C++ language rules and makes it simpler to implement process-wide singletons. A simple function-local static variable, not marked in any special way, is enough.
Now, there are ways to influence the linking process, and in particular there are ways to limit the symbols that are exported from a shared object. The most common ways to do that are using symbol visibility and linker scripts. With these tools it is possible to achieve linking behavior very close to Windows, with all its pros and cons. Note that when you limit symbol visibility you do have to mark the symbols you intend to export from the shared object with the visibility attribute or pragma. There's no need to mark symbols for import though.
Also, in Boost.Log, I noticed the BOOST_LOG_INLINE_GLOBAL_LOGGER_DEFAULT macro (see the Global logger objects section here). It is claimed to create singletons even if the application consists of multiple modules. If someone knows about the inner workings of this macro, explanations are welcome here.
Boost.Log requires to be built as a shared library when it is used from a multi-module application. This makes it possible for it to have a process-wide storage of references to global loggers declared throughout the application (the storage is implemented within the Boost.Log dll/so). When you obtain a logger declared with the BOOST_LOG_INLINE_GLOBAL_LOGGER_DEFAULT or similar macro, the storage is first looked up for the reference to the logger. If it is not found, the logger is created and a reference to it is stored back to the internal storage. Otherwise the existing reference is used. Along with reference caching, this provides performance very close to a function-local static variable.
Finally, if you know about any better solutions for making singletons, feel free to post it as an answer.
Although this is not really an answer, you should generally avoid singletons. They are difficult to implement correctly and in a way that does not hamper performance. If you really do have to implement one then the solution similar to Boost.Log looks generic enough. Note however that with this solution it is generally not known which module created (and as such, 'owns') the singleton, so you can't unload any modules dynamically. There may be simpler ways that are case-specific, like exporting a function returning a reference to the local static object. If you want portability and support non-default symbol visibility by default, always explicitly export your symbols.
GCC has the ability to make a symbol link weakly via __attribute__((weak)). I want to use the a weak symbol in a static library that users can override in their application. A GCC style weak symbol would let me do that, but I don't know if it can be done with visual studio.
Does Visual Studio offer a similar feature?
You can do it, here is an example in C:
/*
* pWeakValue MUST be an extern const variable, which will be aliased to
* pDefaultWeakValue if no real user definition is present, thanks to the
* alternatename directive.
*/
extern const char * pWeakValue;
extern const char * pDefaultWeakValue = NULL;
#pragma comment(linker, "/alternatename:_pWeakValue=_pDefaultWeakValue")
MSVC++ has __declspec(selectany) which covers part of the functionality of weak symbols: it allows you to define multiple identical symbols with external linkage, directing the compiler to choose any one of several available. However, I don't think MSVC++ has anything that would cover the other part of weak symbol functionality: the possibility to provide "replaceable" definitions in a library.
This, BTW, makes one wonder how the support for standard replaceable ::operator new and ::operator delete functions works in MSVC++.
MSVC used to behave such that if a symbol is defined in a .obj file and a .lib it would use the one on the .obj file without warning. I recall that it would also handle the situation where the symbol is defined in multiple libs it would use the one in the library named first in the list.
I can't say I've tried this in a while, but I'd be surprised if they changed this behavior (especially that .obj defined symbols override symbols in .lib files).
The only way i know. Place each symbol in a separate library. User objects with overrides also must be combined to library. Then link all together to an application. User library must be specified as input file, your lib's must be transfered to linker using /DEFAULTLIB: option.
From here:
... if a needed symbol can be satisfied without consulting a library,
then the OBJ in the library will not be used. This lets you override a
symbol in a library by explicitly placing it an OBJ. You can also
override a symbol in a library to putting it in another library that
gets searched ahead of the one you want to override. But you can’t
override a symbol in an explicit OBJ, because those are part of the
initial conditions.
This behavior results from the algorithm adopted by the linker.
So for short, to override a function,
With GCC, you must use the __attribute__((weak)). You cannot rely on the input order of object files into the linker to decide which function implementation is used.
With VS toolchain, you can rely on the order of the object/lib files and you must place your implementation of the function before the LIB that implements the original function. And you can put your implementation as an OBJ or LIB.
There isn't an MS-VC equivalent to this attribute. See http://connect.microsoft.com/VisualStudio/feedback/details/505028/add-weak-function-references-for-visual-c-c. I'm going to suggest something horrible: reading the purpose of it here: http://www.kolpackov.net/pipermail/notes/2004-March/000006.html it is essentially to define functions that, if their symbols exist, are used, otherwise, are not, so...
Why not use pre-processor for this purpose, with the huge caveat of "if you need to do this at all"? (I'm not a fan of recommending pre-processor).
Example:
#ifdef USE_MY_FUNCTION
extern void function();
#endif
then call appropriately in the application logic, surrounded by #ifdef statements. If your static library is linked in, as part of the linking in process, tweak the defines to define USE_MY_FUNCTION.
Not quite a direct equivalent and very ugly but it's the best I can think of.
Iam looking at a piece of code that creates global class variables. The constructors of these classes calls a symbol table singleton and adds the this pointers in it..
In a Keywords.cpp file
class A : class KeyWord
{
A() { add(); }
} A def;
similarly for keywords B,C etc
void KeyWord::add()
{
CSymbolCtrl& c = CSymbolCtrl::GetInstance();
c.addToTable(this);
}
These translation units are compiled to form a library. When i "dumpbin" the library, i see the dynamic initializers for ADef, BDef etc.
No in the exe, when i call the CSymbolCtrl instance, i didnt find the ADef, BDef.. stored in its map. When i set a breakpoint in add(), its not getting hit. Is there a way that the linker is ignoring ADef, BDef because they are not referenced anywhere?
}
From Standard docs 1.9 Program execution,
4) This provision is sometimes called the “as-if” rule, because an implementation is free to disregard any requirement of this International Standard
as long as the result is as if the requirement had been obeyed, as far as can be determined from the observable behavior of the program. For instance,
an actual implementation need not evaluate part of an expression if it can deduce that its value is not used and that no side effects affecting the
observable behavior of the program are produced.
So, it might, yes.
The short answer is yes. A pretty common way to force registration is to do something like:
static bool foo = register_type();
It's not too clear from your question, but are you actually
including the compiled object files in your link or not? Just
putting a file in a library doesn't cause it to be included in
the final program. By definition, a file from a library will
only be included in the executable if it resolves an unresolved
external symbol. If you want an object file to be part of the
final executable, and it doesn't contain any globals which would
resolve an undefined external, then you have several choices:
-- Link the object file directly, rather than putting it in
a library. (This is the "standard" or "canonical" way of
doing it.)
-- Use a DLL. Dispite the name, DLL's are not libraries, but
object files, and are linked in an all or nothing way.
-- Create a dummy global symbol, and reference it somewhere.
(This can often be automated, and might be the preferred
solution if you're delivering a library as a third party
supplier.)