C++ template symbol visibility differences between macOS and Linux? - c++

What are the rules and rule differences between macOS and linux, for the visibility
of internal/external symbols in shared libraries? In particular, the external
visibility of C++ template instantiations. It looks like gcc on linux allows linking to 'unexported' symbols....
Example:
If I have an templated class with a virtual function "interface.h":
template <int T>
class SomeTemplate {
public:
virtual void test_fun(void) const;
};
Whose function is defined in a private source file inside a library somelib.so
"somelib.cpp" (compiled with g++ -fPIC -shared -o libsomelib.so somelib.cpp),
where the library also uses the instantiated template in other functions/class methods:
#include <iostream>
#include "interface.h"
template <int T>
void SomeTemplate<T>::test_fun(void) const
{
std::cout << "Test function " << T << std::endl;
}
void something(void)
{
SomeTemplate<1>().test_fun();
}
// Explicit template instantiation required for macOS - see below
// template class SomeTemplate<1>;
When compiled on linux the template symbols end up in the "W" section, as
determined by nm -C libsomelib.so | grep SomeTemplate:
0000000000000bc4 W SomeTemplate<1>::SomeTemplate()
0000000000000bc4 W SomeTemplate<1>::SomeTemplate()
0000000000000be0 W SomeTemplate<1>::test_fun() const
0000000000201060 V typeinfo for SomeTemplate<1>
0000000000000c90 V typeinfo name for SomeTemplate<1>
0000000000201040 V vtable for SomeTemplate<1>
And using this in a program/library that links to this somelib works fine, e.g. "program.cpp" (g++ -o testlib libsomelib.so program.cpp):
#include "interface.h"
int main(int argc, char const *argv[])
{
SomeTemplate<1>().test_fun();
return 0;
}
However, compiling on macOS (and manually demangling) puts the symbols required
by the program in what man nm calls the "local (non-external) text section":
0000000000000f70 t SomeTemplate<1>::SomeTemplate()
0000000000000fe0 t SomeTemplate<1>::SomeTemplate()
0000000000000f90 t SomeTemplate<1>::test_fun() const
00000000000020d8 D typeinfo for SomeTemplate<1>
0000000000001f50 S typeinfo name for SomeTemplate<1>
00000000000020c0 d vtable for SomeTemplate<1>
And so it seems that being a private symbol, the linking of the program fails with undefined symbols:
Undefined symbols for architecture x86_64:
"SomeTemplate<1>::test_fun() const"
It seems that the usage of the templated class inside the library isn't
enough - I need to explicitly declare the template instantiation e.g.
the commented line in somelib.cpp, then the symbol ends up in "T", visible, and the test program compiles and runs.
What's causing this difference in behaviour, what terminology do I have to use to understand this, and is there a way to get the platforms to match behaviour? Is the non-explicit instantiation just considered bad practice that was bound to break eventually?
(this is a large project I've joined where the upstream library developers aren't necessarily amenable to changes, and my predecessor evidently gave up and forced everything to static linking when compiling on macOS).

Related

Is it possible to explicitly specialize template to match lambda?

Suppose I have a header wrapper.h:
template <typename Func> void wrapper(const Func func);
and a file wrapper.cpp containing:
#include "wrapper.h"
template <typename Func>
void wrapper(const Func func)
{
func();
}
And a file main.cpp containing:
#include "wrapper.h"
#include <iostream>
int main()
{
wrapper( [](){std::cout<<"hello."<<std::endl;} );
}
If I compile these together (e.g., cat wrapper.cpp main.cpp | g++ -std=c++11 -o main -x c++ -), I get no linker errors.
But if I compile them separately (e.g., g++ -std=c++11 -o wrapper.o -c wrapper.cpp && g++ -std=c++11 -o main main.cpp wrapper.o), I --- of course --- get a linker error:
Undefined symbols for architecture x86_64:
"void wrapper<main::$_0>(main::$_0)", referenced from:
_main in main-5f3a90.o
Normally, I could explicitly specialize wrapper and add something like this to wrapper.cpp:
template void wrapper<void(*)()>(void(*)())
But this particular template specialization doesn't work.
Is it possible to specialize a template on a lambda?
First, I assume you know about Why can templates only be implemented in the header file?
To your question:
Is it possible to specialize a template on a lambda?
Unfortunately No, template specializations work with exact match, and a lambda is a unique unnamed type. The problem is specializing for that type which you do not know.
Your best bet is to use std::function; or as you have done, then additionally force the lambda to be converted into a function pointer by adding +
int main()
{
wrapper(+[](){std::cout<<"hello."<<std::endl;} );
}
Full example:
#include <iostream>
template <typename Func>
void wrapper(const Func func)
{
std::cout << "PRIMARY\n";
func();
}
template <>
void wrapper<void(*)()>(void(*func)())
{
std::cout << "SPECIALIZATION\n";
func();
}
int main()
{
wrapper([](){std::cout<<"hello\n"<<std::endl;} );
wrapper(+[](){std::cout<<"world."<<std::endl;} );
}
This will print
PRIMARY
hello
SPECIALIZATION
world
Also, decltype facility wouldn't help, if it does, it will take away the flexibility of your need for lambda
Unfortunately you can't.
Your problem is that lambda types are randomly generated inside each compilation unit.
You can use functions across units because you can declare them in headers; then the linker will find the one with the correct name and type in the compilation unit which defines it. Same if you declare a variable, though less common. If the declaration resulted in a different type in each unit, this would fail and no interaction across units would be possible.
So if you try to make that lambda the "same" object in the two units (i.e. defining in header) the linking will fail because you cannot define the same object twice. And if you make them "different" objects (i.e. adding inline, or defining in source) they will have different types, so linking will fail to unify them as you would need to. Can't win.
Listen to : Why can templates only be implemented in the header file?
So when template definition is moved from file wrapper.cpp into headerfile wrapper.h, the wrapper() can be called by the suggested ways in main.cpp:
int main()
{
wrapper( [](){std::cout<<"hello"<<std::endl;} );
wrapper(+[](){std::cout<<"world"<<std::endl;} );
wrapper(std::function<void()>( [](){std::cout<<"best"<<std::endl;} ));
}

Template inheritance in C++ and undefined symbols on Xcode

I have seen many related questions to this problem, but after carefully following advice from members, my problem still persists. The code is quite simple. I only have the following header file ("instrument.h"), which contains the base class and the template class:
#include <stdio.h>
#include <string>
using namespace std;
class Instrument
{
public:
Instrument();
virtual void print() const = 0;
};
template <class parameter> class Equity : public Instrument
{
public:
Equity();
virtual void print() const;
};
Now, in my main function on main.cpp I only do the following:
#include "instrument.h"
#include <iostream>
int main() {
Equity<double> pb;
return 0;
}
Well, I get the very well-known error:
Undefined symbols for architecture x86_64:
"Equity<double>::Equity()", referenced from:
_main in main.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
I have already changed in Build Settings the C++ standard library to libstdc++, also to default compiler, and so on. Do I have a problem with my project settings? Is perhaps the template wrongly implemented? I was thinking I should also have a instrument.cpp file, but then again definitions for templates must be kept in the header file so that would probably crash too.
Thanks in advance
You declared the default constructors for both Instrument and Equity but defined them nowhere.
Alter their definitions appropriately:
public:
Equity() = default; // Or {} in pre-C++11
// ^^^^^^^^^
(And equivalently for Instrument)
You can also completely omit the declarations of any default constructors for now since you didn't declare any other constructors in both Equity and Instrument and the default constructors will be generated automatically.

Template class with static members across multiple DLL/DSO

There were many questions about C++ template classes which contain static member variables, as well as about exporting them from dynamic libraries or shared objects. But this one is a bit deeper: what to do if there are multiple shared objects, each of them having its own set of instantiations but possibly using instantiations from another shared object?
Consider the following example code:
/* file: common.h */
#include <stdio.h>
#define PRINT fprintf (stderr, "(template %d) %d -> %d\n", parameter, data, new_data)
template <int parameter>
class SharedClass
{
static int data;
public:
static void Set(int new_data) { PRINT; data = new_data; }
};
template <int parameter>
int SharedClass<parameter>::data = parameter;
/* file: library1.h */
extern template class SharedClass<1>;
void Library1Function();
/* file: library1.cpp */
#include "common.h"
#include "library1.h"
#include "library2.h"
template class SharedClass<1>;
void Library1Function()
{
SharedClass<1>::Set (100);
SharedClass<2>::Set (200);
}
/* file: library2.h */
extern template class SharedClass<2>;
void Library2Function();
/* file: library2.cpp */
#include "common.h"
#include "library1.h"
#include "library2.h"
template class SharedClass<2>;
void Library2Function()
{
SharedClass<1>::Set (1000);
SharedClass<2>::Set (2000);
}
/* file: main.cpp */
#include "common.h"
#include "library1.h"
#include "library2.h"
int main()
{
Library1Function();
Library2Function();
SharedClass<1>::Set (-1);
SharedClass<2>::Set (-2);
}
Let's then assume we build the two libraries and an application using GCC:
$ g++ -fPIC -fvisibility=default -shared library1.cpp -o lib1.so
$ g++ -fPIC -fvisibility=default -shared library2.cpp -o lib2.so
$ g++ -fvisibility=default main.cpp -o main -Wl,-rpath=. -L. -l1 -l2
And then run the executable, we'll get the following result:
$ ./main
(template 1) 1 -> 100
(template 2) 2 -> 200
(template 1) 100 -> 1000
(template 2) 200 -> 2000
(template 1) 1000 -> -1
(template 2) 2000 -> -2
Which means that both libraries and the executable access the same per-template static storage.
If we run "nm -C" on the binaries, we'll see that each static member is defined only once and in the corresponding library:
$ nm -C -A *.so main | grep ::data
lib1.so:0000000000001c30 u SharedClass<1>::data
lib2.so:0000000000001c30 u SharedClass<2>::data
But I've got some questions.
Why, if we remove the extern template class ... from both headers, we'll see that the static members are present in each binary, but the test application will continue to work properly?
$ nm -C -A *.so main | grep ::data
lib1.so:0000000000001c90 u SharedClass<1>::data
lib1.so:0000000000001c94 u SharedClass<2>::data
lib2.so:0000000000001c94 u SharedClass<1>::data
lib2.so:0000000000001c90 u SharedClass<2>::data
main:0000000000401e48 u SharedClass<1>::data
main:0000000000401e4c u SharedClass<2>::data
Is it possible to build this under MSVC?
Or, more specifically, how to deal with __declspec(dllexport) and __declspec(dllimport) to make some instantiations exported, and some - imported?
And, finally: is this an example of undefined behavior?
To answer point 1: When the dynamic linker resolves symbols, it uses a list of modules to link against. The first module loaded is checked first, then the second, and so on.
IIRC, when the data member is used in main, lib1.so, and lib2.so, this is still treated as a dynamic symbol reference, even though the member is declared in the same module. So when the linker goes to resolve the symbols when you run the program, all three modules wind up using the data member implementation in just one of the three modules: whichever was loaded first. The other two pairs are still loaded into memory, but are unused.
(Try std::cout << &(SharedClass<n>::data) << std::endl in all three modules; the address printed should be the same for all six cases.)
To answer point 3, I don't believe this behavior is undefined at all. What happens exactly depends on your system's dynamic linker, but I don't know of any linker that wouldn't handle this situation in exactly the same way.
I can't speak to point 2 since I don't have a whole lot of experience with MSVC.

Separate compiling with MinGW

Using this tutorial Makefile I want to build a simple program with a separate compiling, The main problem is that the IDE Eclpse Indigo C\C++ (prespective) or MinGW I cannot compile the files. The error which I get is :
undefined reference to double getAverage<int, 85u>(int (&) [85u])'
undefined reference to int getMax<int, 85u>(int (&) [85u])'
undefined reference to int getMin<int, 85u>(int (&) [85u])'
undefined reference to void print<int, 85u>(int (&) [85u])'
undefined reference to void sort<int, 85u>(int (&) [85u])'
undefined reference to void print<int, 85u>(int (&) [85u])'
The main.cpp file is this :
#include "Tools.h"
#include <iostream>
using namespace std;
int main(int argc,char* argv[])
{
int numbers[] = {1,-2,7,14,5,6,16,8,-2,7,14,5,6,16,8,-2,7,14,5,6,16,8,-2,7,14,5,6,16,8,-2,7,14,5,6,16,8,-2,7,14,5,6,16,8,-2,7,14,5,6,16,8,-2,7,14,5,6,16,8,-2,7,14,5,6,16,8,-2,7,14,5,6,16,8,-2,7,14,5,6,16,8,-2,7,14,5,6,16,8};
cout <<"Average = "<< getAverage(numbers) << endl;
cout <<"Max element = "<< getMax(numbers) << endl;
cout <<"Minimal element = "<< getMin(numbers) << endl;
print(numbers);
sort(numbers);
print(numbers);
return 0;
}
and I have a Tools.h file :
#ifndef TOOLS_H_
#define TOOLS_H_
#include <iostream>
int getBigger(int numberOne,int numberTwo);
template <typename T,size_t N> double getAverage(T (&numbers)[N]);
template <typename T,size_t N> T getMax(T (&numbers)[N]);
template <typename T,size_t N> T getMin(T (&numbers)[N]);
/**
* Implementing a simple sort method of big arrays
*/
template <typename T,size_t N> void sort(T (&numbers)[N]);
/**
* Implementing a method for printing arrays
*/
template <typename T,size_t N> void print(T (&numbers)[N]);
#endif
When you compile Tools.cpp your compiler has no idea about the template parameters that you have used in main.cpp. Therefore it compiles nothing related to this templates.
You need to include theses template definitions from the compilation unit that uses them. The file Tools.cpp is often renamed to something like Tools.inl to indicate that it's neither a header file nor a separate compilation unit.
The compilation unit "main.cpp" could look like this:
#include "tools.h"
#include "tools.inl"
main()
{
int number[] = {1,2,3};
getaverage(numbers);
}
Since the compiler identifies the required specialization it can generate the code from the implementation file.
For most cases, harper's answer is appropriate. But for completeness' sake, explicit template instantiation should also be mentioned.
When you include the implementation in every compilation unit, your template classes and functions will be instantiated and compiled in all of them. Sometimes, this is not desirable. It is mostly due to compile-time memory restrictions and compilation time, if your template classes and functions are very complicated. This becomes a very real issue when you, or the libraries you use rely heavily on template metaprogramming. Another situation could be that your template function implementations might be used in many compilation units, and when you change the implementation, you will be forced to re-compile all those compilation units.
So, the solution in these situations is to include a header file like your tools.h, and have a tools.cpp, implementing the templates. The catch is that, you should explicitly instantiate your templates for all the template arguments that will be used throughout your program. This is accomplished via adding the following to tools.cpp:
template double getAverage<int,85>(int (&numbers)[85]);
Note: You obviously have to do something about that "85", such as defining it in a header file and using it across tools.cpp and main.cpp
I've found this article which is useful : templates and header files
I declared the function in the Tools.h file and include there the file Tool.hpp and after this I defined them in the Tools.hpp file.
I haven't tried to compile .cpp and .c files together but maybe my example will help.
I had similar problem compiling two separate assembly files .s on mingw with standard gcc
compiler and i achieved it as follows:
gcc -m32 -o test test.s hello.s
-m32 means i'm compiling 32bit code
-o is the output file ( which in my example is the "test" file )
test.s and hello.s are my source files. test.s is the main file and hello.s has the helper function. (Oh, to mention is the fact that both files are in the same directory)

C++ Shared Library with Templates: Undefined symbols error

I'm trying to link to a shared library with a template class, but it is giving me "undefined symbols" errors. I've condensed the problem to about 20 lines of code.
shared.h
template <class Type> class myclass {
Type x;
public:
myclass() { x=0; }
void setx(Type y);
Type getx();
};
shared.cpp
#include "shared.h"
template <class Type> void myclass<Type>::setx(Type y) { x = y; }
template <class Type> Type myclass<Type>::getx() { return x; }
main.cpp
#include <iostream>
#include "shared.h"
using namespace std;
int main(int argc, char *argv[]) {
myclass<int> m;
cout << m.getx() << endl;
m.setx(10);
cout << m.getx() << endl;
return 0;
}
This is how I compile the library:
g++ -fPIC -c shared.cpp -o shared.o
g++ -dynamiclib -Wl,-dylib_install_name -Wl,libshared.dylib -o libshared.dylib shared.o
And the main program:
g++ -c main.cpp
g++ -o main main.o -L. -lshared
Only to get the following errors:
Undefined symbols:
"myclass<int>::getx()", referenced from:
_main in main.o
_main in main.o
"myclass<int>::setx(int)", referenced from:
_main in main.o
If I remove the 'template' stuff in shared.h/cpp, and replace them with just 'int', everything works fine. Also, if I just copy&paste the template class code right into main.cpp, and don't link to the shared library, everything works as well.
How can I get a template class like this to work through a shared library?
I'm using MacOS 10.5 with GCC 4.0.1.
In addition to the other answers, you can explicitly instantiate template classes. This is only useful if you know beforehand what types the template parameters may assume. You instantiate the template with all these types in the library.
For your example to compile, just add the following to the end of shared.cpp:
// Instantiate myclass for the supported template type parameters
template class myclass<int>;
template class myclass<long>;
This instantiates the template with Type=int and places the instantiated code in the shared library. Add as many explicit instantiations as you need, for all the types you need.
Again, if you want to be able to instantiate the template with any arbitrary Type parameter, then you must add the definitions to the header file, so that the compiler knows the source code of the template when instantiating it in other compilation units.
Template function definitions must reside in header files. Move the definitions from shared.cpp to shared.h.
So, you can't compile this to a shared library and then link to it. It just doesn't work like that.
You need to include the implementation of the template classes in the header files as well. This is a constraint of templates in C++. So either include shared.cpp from main (#include ) or just move the code from shared.cpp in shared.h
The compiler has to see all the code for a template, so it can generate the appropriate code for the actual type you want to use.
So you should place all the code in your .h. file.