I have unmanaged, template class for thread safe queue:
template<class T>
public class TSQueue {...}
And Producer class:
public ref class Producer
{
Producer(TSQueue<int>* Q) {...}
};
Producer is used in following way:
Producer^ p = gcnew Producer(new TSQueue<int>());
Both are defined in one C++/CLI DLL Producer.DLL.
When Producer is instantiated and provided with TSQueue< int >* inside this DLL, there are no problems with compilation and execution.
But, when I try to instantiate Producer with TSQueue< int >* in another, C++/CLI dll I am having the following compiler error:
Error 23 error C2664: 'Producer(TSQueue<int>*)' : cannot convert parameter 1 from 'TSQueue<T> *' to 'TSQueue<int> *'
As if compiler cannot determine the type of Q I provide to Producer constructor.
I have added reference to Producer.DLL.
Does anyone know how to overcome this problem.
IMO, adding reference to the DLL isn't enough since we are dealing here w/ "native" templates and not w/ generics.
Please make sure that your other DLL has a #include directive to the header of TSQueue
You need to include the complete definition of TSQueue in both libraries. Don't make it part of the interface use it only internally. If it needs to be a part of the API, follow the advice from Hans Passant: make it a generic, or better, use the proper framework implementation.
As Hans Passant sad:
Fairly sure you have a type identity problem here, induced by having a native C++ template cross a module boundary. That's in general a notorious problem in C++, templates don't have external linkage. It is easy to solve in C++/CLI by using generics, you should have a drop-in replacement for TSQueue from the .NET Queue<> class, ConcurrentQueue<> if it needs to be thread-safe.
But the generics cannot be used for unmanaged types, so I am using type specific unmanaged container class.
Related
I'm trying to export classes from a DLL that contain objects such as std::vectors and std::strings - the whole class is declared as DLL export through:
class DLL_EXPORT FontManager {
The problem is that for members of the complex types I get this warning:
warning C4251: 'FontManager::m__fonts' : class 'std::map<_Kty,_Ty>' needs to have dll-interface to be used by clients of class 'FontManager'
with
[
_Kty=std::string,
_Ty=tFontInfoRef
]
I'm able to remove some of the warnings by putting the following forward class declaration before them even though I'm not changing the type of the member variables themselves:
template class DLL_EXPORT std::allocator<tCharGlyphProviderRef>;
template class DLL_EXPORT std::vector<tCharGlyphProviderRef,std::allocator<tCharGlyphProviderRef> >;
std::vector<tCharGlyphProviderRef> m_glyphProviders;
Looks like the forward declaration "injects" the DLL_EXPORT for when the member is compiled but is it safe?
Does it really change anything when the client compiles this header and uses the std:: container on his side?
Will it make all future uses of such a container DLL_EXPORT (and possibly not inline)?
And does it really solve the problem that the warning tries to warn about?
Is this warning anything I should be worried about or would it be best to disable it in the scope of these constructs?
The clients and the DLL will always be built using the same set of libraries and compilers and those are header only classes...
I'm using Visual Studio 2003 with the standard STD library.
Update
I'd like to target you more though as I see the answers are general and here we're talking about std containers and types (such as std::string) - maybe the question really is:
Can we disable the warning for standard containers and types available to both the client and the DLL through the same library headers and treat them just as we'd treat an int or any other built-in type? (It does seem to work correctly on my side)
If so would should be the conditions under which we can do this?
Or should maybe using such containers be prohibited or at least ultra care taken to make sure no assignment operators, copy constructors etc will get inlined into the DLL client?
In general I'd like to know if you feel designing a DLL interface having such objects (and for example using them to return stuff to the client as return value types) is a good idea or not and why, I'd like to have a "high level" interface to this functionality...
Maybe the best solution is what Neil Butterworth suggested - creating a static library?
When you touch a member in your class from the client, you need to provide a DLL-interface.
A DLL-interface means, that the compiler creates the function in the DLL itself and makes it importable.
Because the compiler doesn't know which methods are used by the clients of a DLL_EXPORTED class it must enforce that all methods are dll-exported.
It must enforce that all members which can be accessed by clients must dll-export their functions too. This happens when the compiler is warning you of methods not exported and the linker of the client sending errors.
Not every member must be marked with with dll-export, e.g. private members not touchable by clients. Here you can ignore/disable the warnings (beware of compiler generated dtor/ctors).
Otherwise the members must export their methods.
Forward declaring them with DLL_EXPORT does not export the methods of these classes. You have to mark the according classes in their compilation-unit as DLL_EXPORT.
What it boils down to ... (for not dll-exportable members)
If you have members which aren't/can't be used by clients, switch off the warning.
If you have members which must be used by clients, create a dll-export wrapper or create indirection methods.
To cut down the count of externally visible members, use approaches such as the PIMPL idiom.
template class DLL_EXPORT std::allocator<tCharGlyphProviderRef>;
This does create an instantiation of the template specialization in the current compilation unit. So this creates the methods of std::allocator in the dll and exports the corresponding methods. This does not work for concrete classes as this is only an instantiation of template classes.
That warning is telling you that users of your DLL will not have access to your container member variables across the DLL boundary. Explicitly exporting them makes them available, but is it a good idea?
In general, I'd avoid exporting std containers from your DLL. If you can absolutely guarantee your DLL will be used with the same runtime and compiler version you'd be safe. You must ensure memory allocated in your DLL is deallocated using the same memory manager. To do otherwise will, at best, assert at runtime.
So, don't expose containers directly across DLL boundaries. If you need to expose container elements, do so via accessor methods. In the case you provided, separate the interface from the implementation and expose the inteface at the DLL level. Your use of std containers is an implementation detail that the client of your DLL shouldn't need to access.
Alternatively, do what Neil suggest and create a static library instead of a DLL. You lose the ability to load the library at runtime, and consumers of your library must relink anytime you change your library. If these are compromises you can live with, a static library would at least get you past this problem. I'll still argue you're exposing implementation details unnecessarily but it might make sense for your particular library.
There are other issues.
Some STL containers are "safe" to export (such as vector), and some aren't (e.g. map).
Map for instance is unsafe because it (in the MS STL distribution anyway) contains a static member called _Nil, the value of which is compared in iteration to test for the end. Every module compiled with STL has a different value for _Nil, and so a map created in one module will not be iterable from another module (it never detects the end and blows up).
This would apply even if you statically link to a lib, since you can never guarantee what the value of _Nil will be (it's uninitialised).
I believe STLPort doesn't do this.
One alternative that few people seem to consider is not to use a DLL at all but to link statically against a static .LIB library. If you do that, all the issues of exporting/importing go away (though you will still have name-mangling issues if you use different compilers). You do of course lose the features of the DLL architecture, such as run-time loading of functions, but this can be a small price to pay in many cases.
The best way I found to handle this scenario is:
create your library, naming it with the compiler and stl versions included in the library name, exactly like boost libraries do.
examples:
- FontManager-msvc10-mt.dll for dll version, specific for MSVC10 compiler, with the default stl.
- FontManager-msvc10_stlport-mt.dll for dll version, specific for MSVC10 compiler, with the stl port.
- FontManager-msvc9-mt.dll for dll version, specific for MSVC 2008 compiler, with the default stl
- libFontManager-msvc10-mt.lib for static lib version, specific for MSVC10 compiler, with the default stl.
following this pattern, you will avoid problems related with different stl implementations. remember, the stl implementation in vc2008 differs from the stl implementation in the vc2010.
See your example using boost::config library:
#include <boost/config.hpp>
#ifdef BOOST_MSVC
# pragma warning( push )
# pragma warning( disable: 4251 )
#endif
class DLL_EXPORT FontManager
{
public:
std::map<int, std::string> int2string_map;
}
#ifdef BOOST_MSVC
# pragma warning( pop )
#endif
Found this article. In short Aaron has the 'real' answer above; Don't expose standard containers across library boundaries.
Though this thread is pretty old, I found a problem recently, which made me think again about having templates in my exported classes:
I wrote a class which had a private member of type std::map. Everything worked quite well untill it got compiled in release mode, Even when used in a build system, which ensures that all compiler settings are the same for all targets. The map was completely hidden and nothing was directly exposed to the clients.
As a result the code was just crashing in release mode. I gues, because different binary std::map instances were created for implementation and client code.
I guess the C++ Standard is not saying anaything about how this shall be handled for exported classes as this is pretty much compiler specific. So I guess the biggest portability rule is to just expose Interfaces and use the PIMPL idiom as much as possible.
Thanks for any enlightenment
In such cases, consider the uses of the pimpl idiom. Hide all the complex types behind a single void*. Compiler typically fails to notice that your members are private and all methods included in the DLL.
none of the workarounds above are acceptable with MSVC because of the static data members inside template classes like stl containers
each module (dll/exe) gets its own copy of each static definition...wow! this will lead to terrible things if you somehow 'export' such data (as 'pointed' above)...so don't try this at home
see http://support.microsoft.com/kb/172396/en-us
Exporting classes containing std:: objects (vector, map, etc) from a dll
Also see Microsoft's KB 168958 article How to export an instantiation of a Standard Template Library (STL) class and a class that contains a data member that is an STL object. From the article:
To Export an STL Class
In both the DLL and the .exe file, link with the same DLL version of the C run time. Either link both with Msvcrt.lib (release build) or
link both with Msvcrtd.lib (debug build).
In the DLL, provide the __declspec specifier in the template instantiation declaration to export the STL class instantiation from
the DLL.
In the .exe file, provide the extern and __declspec specifiers in the template instantiation declaration to import the class from the
DLL. This results in a warning C4231 "nonstandard extension used :
'extern' before template explicit instantiation." You can ignore this
warning.
And:
To Export a Class Containing a Data Member that Is an STL Object
In both the DLL and the .exe file, link with the same DLL version of the C run time. Either link both with Msvcrt.lib (release build) or
link both with Msvcrtd.lib (debug build).
In the DLL, provide the __declspec specifier in the template instantiation declaration to export the STL class instantiation from
the DLL. NOTE: You cannot skip step 2. You must export the
instantiation of the STL class that you use to create the data member.
In the DLL, provide the __declspec specifier in the declaration of the class to export the class from the DLL.
In the .exe file, provide the __declspec specifier in the declaration of the class to import the class from the DLL. If the
class you are exporting has one or more base classes, then you must
export the base classes as well. If the class you are exporting
contains data members that are of class type, then you must export the
classes of the data members as well.
If you use a DLL make initialization of all objects at event "DLL PROCESS ATTACH" and export a pointer to its classes/objects.
You may provide specific functions to create and destroy objects and functions to obtain the pointer of the objects created, so you can encapsulate these calls in a wrapper class of access at include file.
The best approach to use in such scenarios is to use the PIMPL design pattern.
I would like to implement a simple class in a DLL, something like:
class MY_EXPORT_IMPORT MyClass
{
public:
//std::string anyPublicStr; //see point 3
protected:
std::string anyStr;
};
The problem is that Visual C++ compiler (2013 in this case) throws the following warning:
C:...MyClass.hpp:X: warning: C4251: 'MyClass::postfix' : class
'std::basic_string,std::allocator>'
needs to have dll-interface to be used by clients of struct 'MyClass'
I read several forums about why this warning is shown and how to solve it. But it is still not clear to me:
Most forums speak about exporting the templates, which make sense for templates, but std::string is already a specific type.
An other page said "Exporting std::string from a DLL is a VERY bad idea, for several reasons." Which make sense.
Others indicate as solution to encapsulate the parameter in a non-inline function. Well, in our case, the std::string is already protected, and switching to private does not solve the warning.
Personally I do not understand why this class could not just statically link the required STL libraries and make it to work without exporting anything. Or if the STL is dynamically linked, it should be also automatically linked in the dll.
Why is this Warning? How to solve it?
std::string is just a typedef:
typedef basic_string<char, char_traits<char>, allocator<char> >
string;
Regarding using STL on the dll interface boundary, generally it is simpler if you avoid that and I personally prefer it if possible. But the compiler is just giving you a warning and it doesn't mean that you will end in a problem. Have a look at: DLLs and STLs and static data (oh my!)
I followed the guide at CodeProject and built a DLL with an abstract interface and exported the factory functions using the extern "C" command along with __declspec(dllexport) and __cdecl and by doing this the article claims that the DLL becomes compiler independent with a clean interface. However when using the DLL on two different versions of g++ the non-standard ABI created the standard c++ problems with DLL calling. In addition to the CodeProject article I also used an article from MinGW in order to be able to create multiple instances of the class.
How is it then possible to make the DLL compiler independent? and if this is not possible, is it then possible to instantiate a class within a DLL and make the functions pure C but calling the C++ functions related to the instantiated class?
Have you tried to reproduce exactly the example that was given there? And what was the failure? You have to consider that not only your class, but any other class like std::string can't be used directly. That said, I wouldn't try to reinvent the wheel but use some kind of component framework like COM.
To answer your second part, consider this:
// internal class
struct X
{
void jump();
};
// compiler-independent C interface
struct X;
struct X* x_alloc(void);
void x_jump(struct X* x, float height);
void x_free(struct X* x);
This works. I have no idea what exactly you tried and what didn't work for you, there's just not enough info to tell.
Here is what I am trying to do:
I am developing a cross-platform IDE (Linux and Windows) that supports plug-ins. I need to support extensibility using an adapter framework similar to the one that Eclipse provides. See here for more details, but basically I need the following:
Let Adaptee and Adapted be completely unrelated classes which already exist and which we are not allowed to change in any way. I want to create an AdapterManager class which has a method
template <class Adaptee, class Adapted> Adapted* adapt( Adaptee* object);
which will create an instance of Adapted given an instance of Adaptee. How exactly the instance is created depends on an adapter function which will have to be registered with AdapterManager. Each new plug-in should be able to contribute adapter functions for arbitrary types.
Here are my thoughts about a possible solution and why it does not work:
C++11's RTTI functions and the type_info class provide a hash_code() method which returns a unique integer for each type in the program. See here. Thus AdapterManager could simply contain a map that given the hash codes for the Adaptee and Adapter classes returns a function pointer to the adapter function. This makes the implementation of the adapt() function above trivial:
template <class Adaptee, class Adapted> Adapted* AdapterManager::adapt( Adaptee* object)
{
AdapterMapKey mk( typeid(Adapted).hash_code(), typeid(Adaptee).hash_code());
AdapterFunction af = adapterMap.get(mk);
if (!af) return nullptr;
return (Adapted*) af(object);
}
Any plug-in can easily extend the framework by simply inserting an additional function into the map. Also note that any plug-in can try to adapt any class to any other class and succeed if there exists a corresponding adapter function registered with AdapterManager regardless of who registered it.
A problem with this is the combination of templates and plug-ins (shared objects / DLLs). Since two plug-ins can instantiate a template class with the same parameters, this could potentially lead to two separate instances of the corresponding type_info structures and potentially different hash_code() results, which will break the mechanism above. Adapter functions registered from one plug-in might not always work in another plug-in.
In Linux, the dynamic linker seems to be able to deal with multiple declarations of types in different shared libraries under some conditions according to this (point 4.2). However the real problem is in Windows, where it seems that each DLL will get its own version of a template instantiation regardless of whether it is also defined in other loaded DLLs or the main executable. The dynamic linker seems quite inflexible compared to the one used in Linux.
I have considered using explicit template instantiations which seems to reduce the problem, but still does not solve it as two different plug-ins might still instantiate the same template in the same way.
Questions:
Does anyone know of a way to achieve this in Windows? If you were allowed to modify existing classes, would this help?
Do you know of another approach to achieve this functionality in C++, while still preserving all the desired properties: no change to existing classes, works with templates, supports plug-ins and is cross-platform?
Update 1:
This project uses the Qt framework for many things including the plug-in infrastructure. Qt really helps with cross platform development. If you know of a Qt specific solution to the problem, that's also welcome.
Update 2:
n.m.'s comment made me realize that I only know about the problem in theory and have not actually tested it. So I did some testing in both Windows and Linux using the following definition:
template <class T>
class TypeIdTest {
public:
virtual ~TypeIdTest() {};
static int data;
};
template <class T> int TypeIdTest<T>::data;
This class is instantiated in two different shared libraries/DLLs with T=int. Both libraries are explicitly loaded at run-time. Here is what I found:
In Linux everything just works:
The two instantiations used the same vtable.
The object returned by typeid was at the same address.
Even the static data member was the same.
So the fact that the template was instantiated in multiple dynamically loaded shared libraries made absolutely no difference. The linker seems to simply use the first loaded instantiation and ignore the rest.
In Windows the two instantiations are 'somewhat' distinct:
The typeid for the different instances returns type_info objects at different addresses. These objects however are equal when tested with ==. The corresponding hash codes are also equal. It seems like on Windows equality between types is established using the type's name - which makes sense. So far so good.
However the vtables for the two instances were different. I'm not sure how much of a problem this is. In my tests I was able to use dynamic_cast to downcast an instance of TypeIdTest to a derived type across shared library boundaries.
What's also a problem is that each instantiation used its own copy of the static field data. That can cause a lot of problems and basically disallows static fields in template classes.
Overall, it seems that even in Windows things are not as bad as I thought, but I'm still reluctant to use this approach given that template instantiations still use distinct vtables and static storage. Does anyone know how to avoid this problem? I did not find any solution.
I think Boost Extension deals with exactly this problem domain:
http://boost-extension.redshoelace.com/docs/boost/extension/index.html
(in preparation for this library's submission to Boost for review)
In particular you'd be interested in what the author wrote in this blog post: "Resource Management Across DLL Boundaries:
RTTI does not always function as expected across DLL boundaries. Check out the type_info classes to see how I deal with that.
I'm not sure whether his solution is actually robust, but he sure gave this thought, before. In fact, there are some samples using Boost Extensions that you can give a go, you might want to use it.
I am in the process of designing a C++ static library.
I want to make the classes generic/configuarable so that they can support a number of data types(and I don't want to write any data type specific code in my library).
So I have templatized the classes.
But since the C++ "export" template feature is not supported by the compiler I am currently using, I am forced to provide the implementation of the classes in the header file.
I dont want to expose the implementation details of my Classes to the client code which is going to use my library.
Can you please provide me with some design alternatives to the above problem??
Prior to templates, type-agnostic C++ code had to be written using runtime polymorphism. But with templates as well, you can combine the two techniques.
For example, suppose you wanted to store values of any type, for later retrieval. Without templates, you'd have to do this:
struct PrintableThing
{
// declare abstract operations needed on the type
virtual void print(std::ostream &os) = 0;
// polymorphic base class needs virtual destructor
virtual ~PrintableThing() {}
};
class PrintableContainer
{
PrintableThing *printableThing;
public:
// various other secret stuff
void store(PrintableThing *p);
};
The user of this library would have to write their own derived version of PrintableThing by hand to wrap around their own data and implement the print function on it.
But you can wrap a template-based layer around such a system:
template <T>
struct PrintableType : PrintableThing
{
T instance;
virtual void print(std::ostream &os)
{ os << instance; }
PrintableType(const T &i)
: instance(i) {}
};
And also add a method in the header of the library, in the declaration of the PrintableContainer class:
template <class T>
void store(const T &p)
{
store(new PrintableType(p));
}
This acts as the bridge between templates and runtime polymorphism, compile-time binding to the << operator to implement print, and to the copy-constructor also (and of course also forwarding to the nested instance's destructor).
In this way, you can write a library entirely based on runtime polymorphism, with the implementation capable of being hidden away in the source of the library, but with a little bit of template "sugar" added to make it convenient to use.
Whether this is worth the trouble will depend on your needs. It has a purely technical benefit in that runtime polymorphism is sometimes exactly what you need, in itself. On the downside, you will undoubtedly reduce the compiler's ability to inline effectively. On the upside, your compile times and binary code bloat may go down.
Examples are std::tr1::function and boost::any, which have a very clean, modern C++ template-based front end but work behind the scenes as runtime polymorphic containers.
I've got some news for you, buddy. Even with export, you'd still have to release all of your template code -- export just makes it that you don't have to put the definitions in a header file. You're totally stuck. The only technique you can use is split off some functions that are non-templates and put them into a different class. But that's ugly, and usually involves void* and placement new and delete. That's just the nature of the beast.
You can try to obfuscate your code - but you have little choice in C++03 asides from including template code in header files.
Vandevoorde does describe another technique in his book: Explicit instantiation - but that entails having to explicitly instantiate all possible useful combinations.
But for the most comprehensive review of this topic read chapter 6 from C++ Templates: The Complete Guide.
Edit (in response to your comment): You have two options for writing generic code without using templates:
1) Preprocessor - still requires header files
2) using void* - yuk - incredibly unsafe
So no, i do not recommend not using templates for solving problems that templates were specifically designed (albeit somewhat flawed) for.
One problem with templates is that they require compiled code. You never know how the end-user will specialize/instantiate your templates, so your dll-file would have to contain all possible template specializations in compiled form. This means that to export pair<X,Y> template you would have to force the compilication of pair<int,float>, pair<int,string>, pair<string,HWND> and so on... to infinity..
I guess more practical solution for you would be to un-template private/hidden code. You can create special internal functions that would only be compiled for single template specialization. In the following example internal_foo-function is never called from MyClass where A is not int.
template<class A>
class MyClass
{
int a;
float b;
A c;
int foo(string param1);
{
((MyClass<int>*)this)->internal_foo(param1);
}
int internal_foo(string param1); // only called on MyClass<int> instances
};
template<>
__declspec(dllexport) int MyClass<int>::internal_foo(string param1)
{
... secret code ...
}
This of course is a hack. When using it you should be extra careful not to use member variable "c", because it's not always integer (even though internal_foo thinks that it is). And you can't even guard yourself with assertions. C++ allows you to shoot yourself in the foot, and gives you no indications about it until it's too late.
PS. I haven't tested this code so it might require some fine tuning. Not sure for example if __declspec(dllimport) is needed in order for compiler to find internal_foo function from dll-file...
With templates you cannot avoid shipping the code (unless your code only works with a fixed set of types, in which case you can explicitly instantiate). Where I work we have a library that must work on POD types (CORBA/DDS/HLA data definitions), so at the end we ship templates.
The templates delegate most of the code to non-templated code that is shipped in binary form. In some cases, work must be done directly in the types that were passed to the template, and cannot thus be delegated to non-templated code, so it is not a perfect solution, but it hides enough part of the code to make our CEO happy (the people in charge of the project would gladly provide all the code in templates).
As Neil points in a comment to the question, in the vast majority of cases there is nothing magical in the code that could not be rewritten by others.