GCC has the ability to make a symbol link weakly via __attribute__((weak)). I want to use the a weak symbol in a static library that users can override in their application. A GCC style weak symbol would let me do that, but I don't know if it can be done with visual studio.
Does Visual Studio offer a similar feature?
You can do it, here is an example in C:
/*
* pWeakValue MUST be an extern const variable, which will be aliased to
* pDefaultWeakValue if no real user definition is present, thanks to the
* alternatename directive.
*/
extern const char * pWeakValue;
extern const char * pDefaultWeakValue = NULL;
#pragma comment(linker, "/alternatename:_pWeakValue=_pDefaultWeakValue")
MSVC++ has __declspec(selectany) which covers part of the functionality of weak symbols: it allows you to define multiple identical symbols with external linkage, directing the compiler to choose any one of several available. However, I don't think MSVC++ has anything that would cover the other part of weak symbol functionality: the possibility to provide "replaceable" definitions in a library.
This, BTW, makes one wonder how the support for standard replaceable ::operator new and ::operator delete functions works in MSVC++.
MSVC used to behave such that if a symbol is defined in a .obj file and a .lib it would use the one on the .obj file without warning. I recall that it would also handle the situation where the symbol is defined in multiple libs it would use the one in the library named first in the list.
I can't say I've tried this in a while, but I'd be surprised if they changed this behavior (especially that .obj defined symbols override symbols in .lib files).
The only way i know. Place each symbol in a separate library. User objects with overrides also must be combined to library. Then link all together to an application. User library must be specified as input file, your lib's must be transfered to linker using /DEFAULTLIB: option.
From here:
... if a needed symbol can be satisfied without consulting a library,
then the OBJ in the library will not be used. This lets you override a
symbol in a library by explicitly placing it an OBJ. You can also
override a symbol in a library to putting it in another library that
gets searched ahead of the one you want to override. But you can’t
override a symbol in an explicit OBJ, because those are part of the
initial conditions.
This behavior results from the algorithm adopted by the linker.
So for short, to override a function,
With GCC, you must use the __attribute__((weak)). You cannot rely on the input order of object files into the linker to decide which function implementation is used.
With VS toolchain, you can rely on the order of the object/lib files and you must place your implementation of the function before the LIB that implements the original function. And you can put your implementation as an OBJ or LIB.
There isn't an MS-VC equivalent to this attribute. See http://connect.microsoft.com/VisualStudio/feedback/details/505028/add-weak-function-references-for-visual-c-c. I'm going to suggest something horrible: reading the purpose of it here: http://www.kolpackov.net/pipermail/notes/2004-March/000006.html it is essentially to define functions that, if their symbols exist, are used, otherwise, are not, so...
Why not use pre-processor for this purpose, with the huge caveat of "if you need to do this at all"? (I'm not a fan of recommending pre-processor).
Example:
#ifdef USE_MY_FUNCTION
extern void function();
#endif
then call appropriately in the application logic, surrounded by #ifdef statements. If your static library is linked in, as part of the linking in process, tweak the defines to define USE_MY_FUNCTION.
Not quite a direct equivalent and very ugly but it's the best I can think of.
Related
I have a project in pure C - st usb library and I need to migrate it to c++ and change same structures into classes. I removed all c++ "protections" like:
#ifdef __cplusplus
extern "C" {
#endif
#ifdef __cplusplus
}
#endif
I changed all files extensions from .c to .cpp (except HAL library).
I realized that c++ .hex is 7kB smaller then c .hex. When I looked into .map file I saw that many functions are missing. I thought that staticfunctions caused that, but removing static key word didn't help. Does anyone have idea what could cause that some functions weren't compiled. When extensions are .c everything is fine.
I can think of two main reasons:
Inlining. The compiler can decide there is no need to emit the function as a standalone function if all usages can be inlined.
Unused code. The compiler can see that a function isn't used anywhere in your code and decide to eliminate it from the final result.
If the result is to be used as sort of library, that your environment calls specific functions without having an explicit call in your own code, I think the best method is to compile and link your code as a library (probably dynamic lib) and exporting those functions as library interface (visibility in gcc, dllexport in MSVC) will force the compiler/linker to include them in the produced binary even if they don't see why they are needed right now.
(Of course, this is a wild guess about your target environment.)
Another option is to turn off the specific compiler/linker optimizations for removing dead code and forcing emitting standalone function instance for inlined functions, but I think it's very indirect approach, have a wider effect and can complicate maintenance later.
C++ functions are given different signatures than C functions. Since you lost functionality and the code is much smaller, it's likely that a function that requires C linkage is being compiled as C++ and the signature mismatch is preventing proper linking.
A likely place for this to happen is in the interrupt vector table. If the handler function is compiled with C++ linkage, the handler address won't make it into a table that's compiled with C.
Double check the interrupt vectors and make sure that they reference the correct functions. If they are correct, check any other code compiled with C that might reference an external symbol compiled with C++.
Background
First of all, I think this question goes beyond the C++ standard. The standard deals with multiple translation units (instantiation units) and thus multiple object modules, but does not seem to acknowledge the possibility of having multiple independently compiled and linked binary modules (i.e., .so files on Linux and .dll files on Windows). After all, the latter more of less enters into the world of application binary interface (ABI) that the standard leaves to implementations to consider at present.
When only a single binary module is involved, the following code snippet illustrates an elegant and portable (standard-compliant) solution to make singletons.
inline T& get() {
static T var{};
return var;
}
There are two things to note about this solution. First, the inline specifier makes the function a candidate to be included in multiple translation units, which is very convenient. Note that, the standard guarantees there is only a single instance of get() and the local static variable var in the final binary module (see here).
The second thing to note is that since C++11, initialization of static local variables is properly synchronized (see the Static local variables section here). So, concurrent invocations of get() is fine.
Now, I try to extend this solution to the case when multiple binary modules are involved. I find the following variant works with VC++ on Windows.
// dllexport is used in building the library module, and
// dllimport is used in using the library in an application module.
// Usually controlled by a macro switch.
__declspec(dllexport/dllimport) inline T& get() {
static T var{};
return var;
}
Note for non-Windows users: __declspec(dllexport) specifies that an entity (i.e., a function, a class or an object) is implemented (defined) in this module and is to be referenced by other modules. __declspec(dllimport), on the other hand, specifies that an entity is not implemented in this module and is to be found in some other module.
Since VC++ supports exporting and importing template instantiations (see here), the above solution can even be templated. For example:
template <typename T> inline
T& get() {
static T var{};
return var;
}
// EXTERN is defined to be empty in building the library module, and
// to `extern` in using the library module in an application module.
// Again, this is usually controlled by a macro switch.
EXTERN template __declspec(dllexport/dllimport) int& get<int>();
As a side note, the inline specifier is not mandatory here. See this S.O. question.
The Question
Since there is no __declspec(dllexport/import) equivalents in GCC and clang, is there a way to make a variant of the above solution that works on these two compilers?
Also, in Boost.Log, I noticed the BOOST_LOG_INLINE_GLOBAL_LOGGER_DEFAULT macro (see the Global logger objects section here). It is claimed to create singletons even if the application consists of multiple modules. If someone knows about the inner workings of this macro, explanations are welcome here.
Finally, if you know about any better solutions for making singletons, feel free to post it as an answer.
Since there is no __declspec(dllexport/import) equivalents in GCC and clang, is there a way to make a variant of the above solution that works on these two compilers?
First, this is not as much a compiler-related question but rather an underlying operating system one. GCC (and supposedly clang) do support __declspec(dllexport/import) on Windows and basically do the same as what MSVC does with the functions and object marked this way. Basically, the marked symbol is placed in a table of exported symbols from the dll (an export table). This table can be used, for instance, when you query for a symbol in a dll in run time (see GetProcAddress).
Along with the dll there comes an associated lib file that contains auxiliary data for linking your application with the dll. When you link your application with the library, the linker uses the lib file to resolve references to the dll symbols and compose the import table in your application binary. When the application starts, the OS (or rather the runtime loader component of the OS) uses the import table to find out what dlls your application depends on and what symbols it imports from those dlls. It then uses export tables in the dlls to resolve addresses of the referenced symbols in the dlls and complete the linking process.
The important side effect of this process is that only imported symbols are dynamically resolved and every symbol you dynamically link to is associated with a particular dll. You can have same-named symbols in multiple dlls and the application itself, and these symbols will refer to distinct entities as long as they are not exported. If they are exported, linking process will fail because of ambiguity. This makes process-wide singletons difficult on Windows. This also breaks some C/C++ language rules, because taking address of an object or function with external linkage (in the language terms) can produce different addresses in different parts of the program. On the other hand, the dlls are more self-contained and depend on the loading context in a lesser degree.
Things are significantly different on Linux and other POSIX-like OSs. When linked, for each shared object (which can be an so library or the application executable) a table of symbols is compiled. It lists both the symbols this shared object implements and the symbols it is missing. Additionally, the linker may embed into the shared object a list of other shared objects (optionally, with search paths) that could be used to resolve the missing symbols. The runtime loader includes a linker which loads the shared objects sequentially and constructs a global table of symbols comprising symbols from all shared objects. As that table is constructed, the duplicate symbols from multiple shared objects are resolved to a single implementation (as all implementations are considered equivalent, the first shared object in the load list that implements the symbol is used). Any missing symbols are also resolved as the subsequent shared objects in the link order are loaded.
The effect of this process is that each symbol with external linkage resolve to a single implementation in one of the shared objects, even if multiple shared objects implement it. This is more in line with the C/C++ language rules and makes it simpler to implement process-wide singletons. A simple function-local static variable, not marked in any special way, is enough.
Now, there are ways to influence the linking process, and in particular there are ways to limit the symbols that are exported from a shared object. The most common ways to do that are using symbol visibility and linker scripts. With these tools it is possible to achieve linking behavior very close to Windows, with all its pros and cons. Note that when you limit symbol visibility you do have to mark the symbols you intend to export from the shared object with the visibility attribute or pragma. There's no need to mark symbols for import though.
Also, in Boost.Log, I noticed the BOOST_LOG_INLINE_GLOBAL_LOGGER_DEFAULT macro (see the Global logger objects section here). It is claimed to create singletons even if the application consists of multiple modules. If someone knows about the inner workings of this macro, explanations are welcome here.
Boost.Log requires to be built as a shared library when it is used from a multi-module application. This makes it possible for it to have a process-wide storage of references to global loggers declared throughout the application (the storage is implemented within the Boost.Log dll/so). When you obtain a logger declared with the BOOST_LOG_INLINE_GLOBAL_LOGGER_DEFAULT or similar macro, the storage is first looked up for the reference to the logger. If it is not found, the logger is created and a reference to it is stored back to the internal storage. Otherwise the existing reference is used. Along with reference caching, this provides performance very close to a function-local static variable.
Finally, if you know about any better solutions for making singletons, feel free to post it as an answer.
Although this is not really an answer, you should generally avoid singletons. They are difficult to implement correctly and in a way that does not hamper performance. If you really do have to implement one then the solution similar to Boost.Log looks generic enough. Note however that with this solution it is generally not known which module created (and as such, 'owns') the singleton, so you can't unload any modules dynamically. There may be simpler ways that are case-specific, like exporting a function returning a reference to the local static object. If you want portability and support non-default symbol visibility by default, always explicitly export your symbols.
I'm trying to use a third party C++ library that isn't using namespaces and is causing symbol conflicts. The conflicting symbols are for classes my code isn't utilizing, so I was considering creating custom header files for the third party library where the class declarations only include the public members my code is using, leaving out any members that use the conflicting classes. Basically creating an interface.
I have three questions:
If the compilation to .obj files works, will this technique still cause symbol conflicts when I get to linking?
If that isn't a problem, will the varying class declarations cause problems when linking? For example, does the linker verify that the declaration of a class used by each .obj file has the same number of members?
If neither of those are a problem and I'm able to link the .obj files, will it cause problems when invoking methods? I don't know exactly how C++ works under the hood, but if it uses indexes to point to class methods, and those indexes were different from one .obj file to another, I'm guessing this approach would blow up at runtime.
In theory, you need identical declarations for this to work.
In practice, you will definitely need to make sure your declarations contain:
All the methods you use
All the virtual methods, used or not.
All the data members
You need all these in the right order of declaration too.
You might get away with faking the data members, but would need to make sure you put in stubs that had the same size.
If you do not do all this, you will not get the same object layout and even if a link works it will fail badly and quickly at run-time.
If you do this, it still seems risky to me and as a worst case may appear to work but have odd run time failures.
"if it uses indexes ": To some extent exactly how virtual functions work is implementation defined, but typically it does use an index into a virtual function table.
What you might be able to do is to:
Take the original headers
Keep the full declarations for the classes you use
Stub out the classes and declarations you do not use but are referenced by the ones you do.
Remove all the types not referenced at all.
For explanatory purposes a simplified explaination follows.
c++ allows you to use functions you declare. what you do is putting multiple definitions to a single declaration across multiple translation units. if you expose the class declaration in a header file your compiler sees this in each translation unit, that includes the header file.
Therefore your own class functions have to be defined exactly as they have been declared (same function names same arguments).
if the function is not called you are allowed not to define it, because the compiler doesn't know whether it might be defined in another translation unit.
Compilation causes label creation for each defined function(symbol) in the object code. On the other hand a unresolved label is created for each symbol that is referenced to (a call site, a variable use).
So if you follow this rules you should get to the point where your code compiles but fails to link. The linker is the tool that maps defined symbols from each translation-unit to symbol references.
If the object files that are linked together have multiple definitions to the same functions the linker is unable to create an exact match and therefore fails to link.
In practice you most likely want to provide a library and enjoy using your own classes without bothering what your user might define. In spite of the programmer taking extra care to put things into a namespace two users might still choose the same name for a namespace. This will lead to link failures, because the compiler exposed the symbols and is supposed to link them.
gcc has added an attribute to explicitly mark symbols, that should not be exposed to the linker. (called attribute hidden (see this SO question))
This makes it possible to have multiple definitions of a class with the same name.
In order for this to work across compilation units, you have to make sure class declarations are not exposed in an interface header as it could cause multiple unmatching declarations.
I recommend using a wrapper to encapsulate the third party library.
Wrapper.h
#ifndef WRAPPER_H_
#define WRAPPER_H_
#include <memory>
class third_party;
class Wrapper
{
public:
void wrappedFunction();
Wrapper();
private:
// A better choice would be a unique_ptr but g++ and clang++ failed to
// compile due to "incomplete type" which is the whole point
std::shared_ptr<third_party> wrapped;
};
#endif
Wrapper.cpp
#include "Wrapper.h"
#include <third_party.h>
void Wrapper::wrappedFunction()
{
wrapped->command();
}
Wrapper::Wrapper():wrapped{std::make_shared<third_party>()}
{
}
The reason why a unique_ptr doesn't work is explained here: std::unique_ptr with an incomplete type won't compile
You can move the entire library into a namespace by using a clever trick to do with imports. All the import directive does is copy the relevant code into the current "translation unit" (a fancy name for the current code). You can take advantage of this as so
I've borrowed heavily from another answer by user JohnB which was later deleted by him.
// my_thirdparty.h
namespace ThirdParty {
#include "thirdparty.h"
//... Include all the headers here that you need to use for thirdparty.
}
// my_thirdparty.cpp / .cc
namespace ThirdParty {
#include "thirdparty.cpp"
//... Put all .cpp files in here that are currently in your project
}
Finally, remove all the .cpp files in the third party library from your project. Only compile my_thirdparty.cpp.
Warning: If you include many library files from the single my_thirdparty.cpp this might introduce compiler issues due to interaction between the individual .cpp files. Things such as include namespace or bad define / include directives can cause this. Either resolve or create multiple my_thirdparty.cpp files, splitting the library between them.
Iam looking at a piece of code that creates global class variables. The constructors of these classes calls a symbol table singleton and adds the this pointers in it..
In a Keywords.cpp file
class A : class KeyWord
{
A() { add(); }
} A def;
similarly for keywords B,C etc
void KeyWord::add()
{
CSymbolCtrl& c = CSymbolCtrl::GetInstance();
c.addToTable(this);
}
These translation units are compiled to form a library. When i "dumpbin" the library, i see the dynamic initializers for ADef, BDef etc.
No in the exe, when i call the CSymbolCtrl instance, i didnt find the ADef, BDef.. stored in its map. When i set a breakpoint in add(), its not getting hit. Is there a way that the linker is ignoring ADef, BDef because they are not referenced anywhere?
}
From Standard docs 1.9 Program execution,
4) This provision is sometimes called the “as-if” rule, because an implementation is free to disregard any requirement of this International Standard
as long as the result is as if the requirement had been obeyed, as far as can be determined from the observable behavior of the program. For instance,
an actual implementation need not evaluate part of an expression if it can deduce that its value is not used and that no side effects affecting the
observable behavior of the program are produced.
So, it might, yes.
The short answer is yes. A pretty common way to force registration is to do something like:
static bool foo = register_type();
It's not too clear from your question, but are you actually
including the compiled object files in your link or not? Just
putting a file in a library doesn't cause it to be included in
the final program. By definition, a file from a library will
only be included in the executable if it resolves an unresolved
external symbol. If you want an object file to be part of the
final executable, and it doesn't contain any globals which would
resolve an undefined external, then you have several choices:
-- Link the object file directly, rather than putting it in
a library. (This is the "standard" or "canonical" way of
doing it.)
-- Use a DLL. Dispite the name, DLL's are not libraries, but
object files, and are linked in an all or nothing way.
-- Create a dummy global symbol, and reference it somewhere.
(This can often be automated, and might be the preferred
solution if you're delivering a library as a third party
supplier.)
GCC has the ability to make a symbol link weakly via __attribute__((weak)). I want to use the a weak symbol in a static library that users can override in their application. A GCC style weak symbol would let me do that, but I don't know if it can be done with visual studio.
Does Visual Studio offer a similar feature?
You can do it, here is an example in C:
/*
* pWeakValue MUST be an extern const variable, which will be aliased to
* pDefaultWeakValue if no real user definition is present, thanks to the
* alternatename directive.
*/
extern const char * pWeakValue;
extern const char * pDefaultWeakValue = NULL;
#pragma comment(linker, "/alternatename:_pWeakValue=_pDefaultWeakValue")
MSVC++ has __declspec(selectany) which covers part of the functionality of weak symbols: it allows you to define multiple identical symbols with external linkage, directing the compiler to choose any one of several available. However, I don't think MSVC++ has anything that would cover the other part of weak symbol functionality: the possibility to provide "replaceable" definitions in a library.
This, BTW, makes one wonder how the support for standard replaceable ::operator new and ::operator delete functions works in MSVC++.
MSVC used to behave such that if a symbol is defined in a .obj file and a .lib it would use the one on the .obj file without warning. I recall that it would also handle the situation where the symbol is defined in multiple libs it would use the one in the library named first in the list.
I can't say I've tried this in a while, but I'd be surprised if they changed this behavior (especially that .obj defined symbols override symbols in .lib files).
The only way i know. Place each symbol in a separate library. User objects with overrides also must be combined to library. Then link all together to an application. User library must be specified as input file, your lib's must be transfered to linker using /DEFAULTLIB: option.
From here:
... if a needed symbol can be satisfied without consulting a library,
then the OBJ in the library will not be used. This lets you override a
symbol in a library by explicitly placing it an OBJ. You can also
override a symbol in a library to putting it in another library that
gets searched ahead of the one you want to override. But you can’t
override a symbol in an explicit OBJ, because those are part of the
initial conditions.
This behavior results from the algorithm adopted by the linker.
So for short, to override a function,
With GCC, you must use the __attribute__((weak)). You cannot rely on the input order of object files into the linker to decide which function implementation is used.
With VS toolchain, you can rely on the order of the object/lib files and you must place your implementation of the function before the LIB that implements the original function. And you can put your implementation as an OBJ or LIB.
There isn't an MS-VC equivalent to this attribute. See http://connect.microsoft.com/VisualStudio/feedback/details/505028/add-weak-function-references-for-visual-c-c. I'm going to suggest something horrible: reading the purpose of it here: http://www.kolpackov.net/pipermail/notes/2004-March/000006.html it is essentially to define functions that, if their symbols exist, are used, otherwise, are not, so...
Why not use pre-processor for this purpose, with the huge caveat of "if you need to do this at all"? (I'm not a fan of recommending pre-processor).
Example:
#ifdef USE_MY_FUNCTION
extern void function();
#endif
then call appropriately in the application logic, surrounded by #ifdef statements. If your static library is linked in, as part of the linking in process, tweak the defines to define USE_MY_FUNCTION.
Not quite a direct equivalent and very ugly but it's the best I can think of.