Preventing template instantiations in a static library - c++

I'm attempting to build a static library, containing definitions of DDS topics obtained from various IDL files. I'm using OpenDDS as my middleware.
When I create an IDL file containing a sequence<long>, compile it into my static library, and then link the static library against my application, I get linker errors involving multiple definition of symbols:
Error LNK2005 "public: void __cdecl TAO::unbounded_value_sequence<int>::length(unsigned int)" (?length#?$unbounded_value_sequence#H#TAO##QEAAXI#Z) already defined in TAO.lib(TAO.dll)
I believe this is because my static library contains a template instantiation of unbounded_value_sequence, and my application also contains an instantiation. It seems to be coming from within ACE TAO, which is used by OpenDDS.
I'm looking for a way to avoid instantiating the template in my static library altogether, so that it can just use the definition within the application when they are linked together. I tried adding the following:
extern template class TAO::unbounded_value_sequence<int>;
This produced the following error:
Error C2961 'TAO::unbounded_value_sequence<CORBA::Long>': inconsistent explicit instantiations, a previous explicit instantiation did not specify '__declspec(dllimport)'
I've tried to locate that instantiation, but its not in my code. It may be within ACE itself.
The problem does not occur if I build everything in the one project, but that is not an ideal solution.

What you have to do to use extern templates is a bit different. Indeed, declaring the extern template will prevent it's instanciation. But you will need an instanciation somewhere. That somewhere is usually in a cpp with the name of the template you want to compile.
unbounded_value_sequence.h:
// template struct here
extern template class TAO::unbounded_value_sequence<int>;
extern template class TAO::unbounded_value_sequence<long>;
// and every other instantiation you want to be in your static library
unbounded_value_sequence.cpp:
#include "unbounded_value_sequence.h"
// Here you compile them one time.
template class TAO::unbounded_value_sequence<int>;
template class TAO::unbounded_value_sequence<long>;
// and every other instantiation you want to be in your static library
That will make your template to be instantiated one time only, inside your library. The compiler will generate a unbounded_value_sequence object file that contains your template instantiations. They will exist only there.
Don't forget that you still need to make your template implementation visible in the header if you want users of your library use your template class with theirs.

Related

The linker attempts to include all not used functions and template instantiations even with function level linkage

I have a library which uses templates very much to avoid rewriting the same code many times . But as known this leads to horrible compile times and in visual studio also leads to heavy and slow intellisense which makes coding on a computer with 4 GB of ram very noisy .
However most classes are designed so that they only can be instantiated with particular types and templates are there to avoid rewriting for each type , this made me move most of the implementations to .cpp files and use explicit template instantiation there .
For example I have :
// socket.h
template<class Protocol>
class socket
{
};
// socket.cpp
template class socket<tcp>;
template class socket<udp>;
template class socket<local::tcp>;
template class socket<local::udp>;
// sslstream.h
template<class NextLayer, class Engine>
class sslstream
{
};
// sslstream.cpp
template class sslstream<tcp::socket, openssl::engine>;
template class sslstream<tcp::socket, wolfssl::engine>;
template class sslstream<udp::socket, openssl::engine>;
template class sslstream<udp::socket, wolfssl::engine>;
// and so on
// websocket.h
template<class NextLayer>
class wsstream
{
};
// websocket.cpp
template class wsstream<tcp::socket>;
template class wsstream<sslstream<tcp::socket, openssl::engine>>;
// and so on for all possible layers
Problems begin when I try to use something like wsstream<tcp::socket> in any source that links to the library . The linker attempts to include all non-static functions and all template instantiations along the path and will emit errors for unresolved functions from openssl and wolfssl libraries if I don't pass them to the linker . This increases the program size and includes tons of unused code from openssl and wolfssl even if I don't use any ssl functionality at all ! And tens of template instantiations will be included and never referenced only because they are instantiated in the same translation unit !
I read that the linker in ordinary circumstances can't remove unused functions from an object file (either no to include it or include the whole object) because functions lie in the same code segmentation and the linker can't separate them easily .
Then I read about function level linking which is introduced to solve such problems (I think) and puts each function in a separated section so the linker can throw unused sections away
This is what I did :
Enabled Function-Level Linking in the library project /Gy
Enable Link Time Code Generation /LTCG in the program project and also /Gy
The lib file resulted was so huge : more than 60 MB ! Before it was about 8 MB .
I expected now the linker will be able no to include the unused code but the result exe didn't reduce in size and also had much code and strings from openssl and wolfssl !
Is this the expected result ? Or I'm doing something wrong ? and if this is intended , is there any method to separate the implementation from the template interfaces and not to be forced to include the whole instantiations ?
For anyone who comes to this question and looks for a solution to similar problem, this is what I ended with :
Instead of moving the implementation to a translation unit .cpp and instantiating all templates there I put the implementation in an internal header file and removed the templates instantiations from the header . For each template instantiation I made a .cpp file which includes the implementation header plus the underlying type interface and instantiate an instant of the class with this type .
Now each instantiation resides in its own file and the linker won't pull unused instants.
A drawback of this approach is the increased implementation files which are there only to go around this problem.

Force template instantiation in another header

I have a templated "collection" class. I don't wan't the code be recompiled in every source file. But this collection should be used with DataTypes (class SpecialDataMember : DataMember) not defined on definition of the the template class.
This is why I can't force the instantiation in the template source file (throug moving the definitions to the source file and add template LinkCollection<DataMember>;), because the definitions of the template must be accessible in other source files ("SpecialDataMember.cpp").
Is it possible to make "SpecialDataMember.o" hold the code for LinkCollection<SpecialDataMember>. Every includer of SpecialDataMember.h should know I leave LinkCollection<SpecialDataMember> to the linker.
I see two options but I don't know if they work:
Make two template headers (with header guard) one with the definitions one without.
SpecialDataMember.h
// ..
include "LinkCollection.h"
// ..
SpecialDataMember.cpp
// ..
include "LinkCollectionImpl.h"
// ..
include "SpecialDataMember.h"
// ..
template LinkCollection<SpecialDataMember>;
All includers of "SpecialDataMember.h" will not know the definitions of the template so they will let the linker do his work.And the linker will find the instantiation of LinkCollection<SpecialDataMember> in SpecialDataMember.o. Is that correct?
But I would have to maintain two header Files. Is there a better way to gain this effect?
Use template derived classes
If I create a special class SpecialDataMemberLinkCollection : LinkCollection<SpecialDataMemberLink> in the "SpecialDataMember" header and source file, I could reference this special class instead and so the compiler knows there is an instantiation of that class and the base class template and leaves the work to the linker. Will this work as expect?
Is it possible to make "SpecialDataMember.o" hold the code for LinkCollection<SpecialDataMember>. Every includer of SpecialDataMember.h should know I leave LinkCollection<SpecialDataMember> to the linker.
Simple, just put:
extern template class LinkCollection<SpecialDataMember>;
in the header, which tells the compiler not to instantiate that template and leave it to another file.
Then in one SpecialDataMember.cpp file provide that instantiation:
template class LinkCollection<SpecialDataMember>;

C++ templated member function linker errors in client application

I tried the recommendation here. I have the following class with a templated member function defined in the implementation as follows:
Header (with compiler DLL import directive evaluated as __declspec(dllimport) in client application):
class __declspec(dllimport) A {
...
template<typename T> bool func(T&) const;
}
Implementation:
template<typename T> bool A::func(T&) {...}
Due to some constraints, I can't define it the header. In the standalone library I get no build errors. But in the client application, built on the same platform using same compiler (x64 vs100), the linker error I get is:
error LNK2019: unresolved external symbol bool "public: bool __cdecl A::func(...)"
Appreciate any ideas on this. Thanks!
Your template code will not be generated until you actually instantiate it using some data type. For every data type different code is generated by the compiler.
Therefore if you want to use templated code that other code parts shall be able to use with any data type then the template implementations needs to be provided inline (i.e. in a header file of some sort). Only this way the template can be instantiated for any unknown data types in other compilation units.
If you only need a small number of variants of the template, say for int and float then you can declare explicit template instantiations. These declarations cause the compiler to actually output the template code for the given data types whether they're needed or not in a compilation unit. Then you also need to add dllimport/dllexport statements (in case of Windows Visual Studio compilers) for ex-/importing that code in/from a DLL.
More about this can surely be found in other discussions and documentation.
Make sure that your implemetation is in the same header (or it is taken in with some other #include statement) as the definition of the class. In this case it should work.

C++ linking for an overloaded << function (logger class)

I'm currently writing a logger for my C++ project (I need it to have near to no dependences, so I don't want to take a prewritten one). It contains this function:
template <typename T>
Logger& Logger::operator<<(T f) {
m_file<<f;
return *this;
}
The logger.cpp code compiles, but when I call the log functions in my main.cpp, I have this compiler error:
/home/tuxer/prog/cpp/PRay/server/src/main.cpp:110: undefined reference to `Logger& Logger::operator<< <int>(int)'
for this line of code :
log<<lul; (lul being a int variable equals to 2)
The logger.o file is correctly included, as the Logger::init() function works properly and doesn't raise any linking error.
Thanks :)
Since you have non-inline templates, you need to force instantiation. See for example How do I force a particular instance of a C++ template to instantiate?.
The simple thing to do is to put the Logger::operator<< template in the header file. The compiler will automatically instantiate the versions it needs, and the linker will remove the duplicates (well, at least the ones that weren't inlined).
You shouldn't need to force instantiation unless your linker is old (e.g. gcc 2.7 or earlier).
This is the general rule with template code: put the definitions in the header file unless you have a good reason not to.
See Why can templates only be implemented in the header file? as well.

template class, implementation code causing linking issues

I currently have a program where my main code is in a file main.cpp.
Main.cpp includes a header file "class.h" that declares a class that is used within main.cpp.
Also in main.cpp I have function declarations that declare the functions I use within main.cpp.
The code for these functions is in a separate .cpp file fucntions.cpp.
Like main.cpp, functions.cpp also includes class.h as the class type is used within the functions.
class.h contains the class declaration only.
The implementation code for class.h is in a separate .cpp file classimplementation.cpp.
It all works fine until I try to make the class in class.h a template class.
Then I get linking problems. Research and testing has shown me that this is because the definition of the template class functions needs to reside in class.h with the declaration.
I therefore took the required code out of classimplementations.cpp and put it into class.h.
This did solve my original linking issues but instead I get more linking errors that seem to be telling me I am trying to redefine the functions that I moved to into class.h.
This I think is because class.h is being called by main.cpp and again by functions.cpp.
Therefore the functions in class.h are being defined twice:
Error 41 error LNK2005: "public: __thiscall RecordPocket::RecordPocket(int)" (??0?$RecordPocket#VT####QAE#H#Z) already defined in classimplementation.obj functions.obj
I know that class implementation code should really be kept out of include files but due to the template class limitation of having to keep the class functions local I appear (in my novice mind) to have no choice.
Has anyone been in this scenario and can offer any advice.
I have tried surrounding the functions I moved from classimplementation.cpp to class.h with the standard ifndef CLASSIMP, #define CLASSIMP code and PRAGMA ONCE but neither make any difference.
If all else fails I will move the functions from functions.cpp into main.cpp so that class.h gets called just the once but I’d rather find out what I’m doing wrong as I’m sure it will happen again.
You could keep the template functions inside the template<> class what{/HERE/};
template<typename T>
class MyTempClass{
void myFunctions{
// code here
}
}
EDITED: I removed the code corrected by Glen
I think your problem is revolves around these issues. As you have implied any template function definition (i.e. template function of member function of a template class) needs to be fully expressed in the .h file because when the compiler finds a specific instance of the template it needs to build the function.
You figured this out and moved some implementation into your class.h file. Now if the linker find a MyFunction() in more than one module then is just discards one of them a no linker error is reported.
However you can't define the same non-template function in two different modules as this generates the error you are getting above. So I suspect you also moved some non-template functionality into the .h file; thus including it in two separate obj files and generating the linker error. This theory is support by your quoted error message as I note __thiscall RecordPocket::RecordPocket(int) does not appear to be template.