Our project uses boost::serialization to serialize many things.
But some types are not correctly registered and when serializing them we get an "unregistered class" error
I have narrowed the problem to the BOOST_CLASS_EXPORT_KEY, which, for some types are not generating code.
What BOOST_CLASS_EXPORT_KEY does is :
namespace boost {
namespace serialization {
template<>
struct guid_defined< T > : boost::mpl::true_ {};
template<>
inline const char * guid< T >(){
return K;
}
} /* serialization */
} /* boost */
All objects that are serialized inherit from a base class called Serializable.
Serialization is always done via a pointer to the base class.
This works fine except for one case:
There is a class template SerializableList which is a Serializable which holds a list of T
template< typename T>
class SerializableList
{
...
std::vector<T> m_list;
template<class Archive>
void serialize( Archive & ar, const unsigned int /*version*/ )
{
ar & boost::serialization::base_object<businessObjects::Serializable>(*this);
ar & mList;
}
};
in a dedicated cpp and hpp files we then declare each instantiation of this template to boost serialization like this:
hpp:
BOOST_CLASS_EXPORT_KEY( SerializableList<SomeT*> );
BOOST_CLASS_EXPORT_KEY( SerializableList<SomeOtherT*> );
BOOST_CLASS_EXPORT_KEY( SerializableList<AThirdT*> );
cpp:
BOOST_CLASS_EXPORT_IMPLEMENT( SerializableList<SomeT*> );
BOOST_CLASS_EXPORT_IMPLEMENT( SerializableList<SomeOtherT*> );
BOOST_CLASS_EXPORT_IMPLEMENT( SerializableList<AThirdT*> );
But half of these lines do not produce executable code in the final executable! if we put a breakpoint on each of those lines and run, half the breakpoints disappear, those who stay are on the working types (those we can serialize).
For instance the breakpoints would stay on SerializableList<SomeT*> and SerializableList<AThirdT*> but not SerializableList<SomeOtherT*>.
Btw, we have also tried to call directly boost::serialization::guid<T>(), and while it works fine for say:
boost::serialization::guid<SerializableList<SomeT*> >() which returns the key,
it doesn't for
boost::serialization::guid<SerializableList<SomeOtherT*> >() which calls the default implementation ...
So is there a compiler bug (we use Visual C++ 2010 SP1), or some good reason for the compiler to ignore some of those specializations?
I forgot to mention, all this code lies in a library, which is linked against the exe project. I've tried with different exe projects and sometimes it works sometimes it doesn't ... the compilation options are the same... I really have no clue what's going on :'(
We found the solution,
One (serializable) class had several SerializableList members, and did not include the file with all the "BOOST_CLASS_EXPORT_KEY" lines.
the other projects which were working didn't use that particular class ...
Related
I have a project with quite a lot of function templates that I wrote back in Visual Studio 2017, and it always worked fine. Now I have to build it in VS2019, because I need to include the lib in another project that is written in VS2019, and the thing won't build.
There is one function template it seems to take issue with, although it doesn't actually complain about the function itself. The compiler just says "identifier not found" when I invoke it in the code. The thing is there in the namespace, however, even InteliSense sees it and links to it without complaining. Just the compiler won't.
Here's the code in question:
// declaration
namespace Oparse
{
// lots of other functions, many of them templates
template <typename T> OpModel<T> *_ModelPtr(T *receiver) { return new OpModel<T>(receiver); };
}
// Invocation
namespace Oparse
{
template <class T, class U>
class OpModelFactory
: public OpNestable
{
public:
OpModelFactory<T, U>(vector<U*> &receiver) : OpNestable(OP_MODELFACTORY), receiver(receiver) {};
// other stuff
void Serialize(string key, stringstream &stream, unsigned int indents)
{
for (unsigned int i = 0; i < receiver.size(); ++i)
{
// check if the instances are really of the type of this OpModel, otherwise there may be duplicates between polymorphic factories populating the same receiver.
T *currentModel = dynamic_cast<T*>(receiver[i]);
if (currentModel != NULL)
{
OpModel<T> *parser = _ModelPtr<T>(currentModel); // <-- identifier not found
parser->Serialize(key, stream, indents);
delete parser;
}
}
};
private:
vector<U*> &receiver;
}
}
If I comment that invocation, the project builds, despite there being a whole lot more function templates declared right where this one is. I have no clue what to do to make the linker find it. Are there any Visual Studio wizzards who could give me a hint? I must honestly confess that I haven't used the IDE in years, and it's my first time in Visual Studio 2019...
Here's the complete output of the error. There's a second message to it, but I found it perfectly unhelpful:
1>D:\Orbiter_installs\Orbiter2016\Orbitersdk\Oparse\include\OpModel.h(138,27): error C3861: '_ModelPtr': identifier not found
1>D:\Orbiter_installs\Orbiter2016\Orbitersdk\Oparse\include\OpModel.h(152): message : see reference to class template instantiation 'Oparse::OpModelFactory<T,U>' being compiled
And no, there's not further message attached. I have seen similar messages that usually go on with "with ... $further information", but this is all I get.
There is a circular dependency problem.
In Oparse.h, you first include OpModel.h that requires_ModelPtr in implementation of Serialize , but_ModelPtr is only defined later in the header.
You need to forward declare the template method.
In OpModel.h, write this instead:
namespace Oparse
{
template<typename T> class OpModel;
template <typename T> OpModel<T>* _ModelPtr(T* receiver);
// Remaining of OpModel.h...
typedef map<string, pair<Oparse::OpValue*, vector<Oparse::OpValidator*>>> OpModelDef;
typedef vector<pair<Oparse::OpValue*, vector<Oparse::OpValidator*>>> OpValues;
...
I'm trying to optimize my build time using extern templates because I have a lot of generated headers that contain typedefs to a template class.
The template class
// TypeID.h
template <typename T>
class TypeID
{
public:
TypeID(/* <some parameters> */);
bool isNull() const;
// ... many other methods
};
template <typename T>
TypeID<T>::TypeID(/* <some parameters> */)
{
// impl
}
template <typename T>
bool TypeID<T>::isNull() const
{
// impl
}
// same for the rest of the methods
Example of generated header
// NamedID.h
#include "TypeID.h"
typedef TypeID</* some type */> NamedID;
There are many (~2k) headers like NamedID with different types and they're included throughout the project.
I changed the code generator to add this line above the typedef:
extern template class TypeID</* some type */>;
and in addition to the header files, it now also generates a cpp where all the extern templates have a corresponding
template class TypeID</* some type */>;
Due to the number of headers and how many times they're used in the project I expected a big difference in compile time (at least something noticeable) but there's no difference.
I ran several runs of the build with and without this change and all of them take 2h 30m +/-2m.
Did I implement this wrong ? Am I expecting too much ?
My environment:
RHEL 7.7
GCC 8.3.1
CMake + ninja, no ccache, no icecream/distcc
I'm trying to serialize my abstract class according to those questions:
Get private data members for non intrusive boost serialization C++
Error serializing an abstract class with boost
Error serializing an abstract class with boost
My neuron.h looks like this:
class Neuron {
public:
struct access;
API virtual ~Neuron();
API virtual double activate( double x, double b ) = 0;
};
I have to keep all the Boost related members in neuron.cpp to prevent including Boost headers when using neuron.h in some other codes.
My neuron.cpp looks like this:
#include "Neuron.h"
#include <boost/archive/text_oarchive.hpp>
#include <boost/archive/text_iarchive.hpp>
BOOST_SERIALIZATION_ASSUME_ABSTRACT(Neuron);
struct Neuron :: access {
template <class Archive>
static void serialize(Archive &ar, Neuron& n, const unsigned int version) {}
};
namespace boost {
namespace serialization {
template<class Archive>
void serialize(Archive & ar, Neuron& n, const unsigned int version)
{
Neuron::access::serialize(ar, n, version);
}
} // namespace serialization
} // namespace boost
Neuron::~Neuron() {
}
The problem is, that when I'm using its inherited classes elsewhere, I'm getting the error
***/boost/boost/serialization/access.hpp:116:11: error: ‘class Neuron’ has no member named ‘serialize’
What am I doing wrong here?
I think the key here is "when I'm using its inherited classes elsewhere". Correct me (and your question, please) if I'm wrong, but this suggests that you are getting the compile error while compiling a source file other than neuron.cpp.
This makes sense, given what the compiler has to work with. You might have noticed that changes to one source file tend to not require re-compiling other source files. So adding something -- like an overload of serialize() -- to neuron.cpp does not change how other translation units are compiled. (It can change how everything is linked together in the end, but we're not there yet.) If another translation unit tries to serialize Neuron, the stuff in neuron.cpp does not matter. The compiler is not aware of an appropriate overload of serialize(), so serializing Neuron in another source file results in intrusive serialization. That is, the compiler will look for a member function of Neuron called serialize().
In order for your overload of serialize() to affect how other translation units are compiled, it needs to be declared in a header file.
Since you cannot put Boost stuff in neuron.h, you might have to create a new header file, say neuron_boost.h. This file would #include "neuron.h" then provide the declarations needed for Boost serialization. Source files that serialize descendants of Neuron would include neuron_boost.h while other source files could continue to include the original neuron.h.
I cannot seem to get boost::serialization to work well. I have sprinkled the class .cpp files with BOOST_CLASS_EXPORT_GUID macros, after including the class and archive headers, but I still get unregistered_class exceptions.
I have looked around and it seems that whatever I can find is 1. either outdated, dealing with old versions of the library, or 2. works only for a simple one-file solution in which all the serializable classes are defined one after the other. Nothing I've found helps.
My solution at present consists of a project, compiled into a static library, that contains the core functionality with the basic archivable classes, and another test project that will eventually be fleshed out into a more concrete logic layer. Getting everything to work with boost::serialization is proving a nightmare. I'm almost tempted to write it myself.
Anyway, the class in question which raises the exception is defined in a header, which looks something like this:
#include <boost/serialization/assume_abstract.hpp>
#include <boost/serialization/serialization.hpp>
// Other includes...
namespace GameCore { class Component; }
// Forward declare some boost::serialization functions that appear at the bottom.
// ...
BOOST_SERIALIZATION_ASSUME_ABSTRACT(GameCore::Component);
namespace GameCore
{
// Some forward declares..
//////////////////////////////////////////////////////////////////////////
// Base component type.
//////////////////////////////////////////////////////////////////////////
class Component : public Updatable, public Object
{
friend class boost::serialization::access;
protected:
template <typename Archive>
friend void boost::serialization::serialize(Archive& archive, Component& object, const unsigned int version);
template <typename Archive> friend void boost::serialization::load_construct_data(Archive& archive, Component* t, const unsigned int version);
template <typename Archive> friend void boost::serialization::save_construct_data(Archive& archive, const Component* t, const unsigned int version);
public:
Component(GameObject& owner);
virtual ~Component() = 0;
// Irrelevant stuff..
GameObject& gameObject;
Transform* transform;
};
}
// The component includes have to be placed here because it would otherwise create a cyclic inclusion when trying to compile the
// individual component classes, say, Transform, which would end up including itself.
#include "Transform.h"
namespace boost
{
namespace serialization
{
template<class Archive>
inline void save_construct_data(Archive& archive, const GameCore::Component* t, const unsigned int version)
{
archive << t->gameObject;
}
template<class Archive>
inline void load_construct_data(Archive& archive, GameCore::Component* t, const unsigned int version)
{
// Retrieve data from archive required to construct new instance.
GameCore::GameObject owner;
archive >> owner;
// Invoke inplace constructor to initialize instance of class.
::new(t)GameCore::Component(owner);
}
//////////////////////////////////////////////////////////////////////////
// Serialization function for save/load.
//////////////////////////////////////////////////////////////////////////
template <typename Archive>
void serialize(Archive& archive, GameCore::Component& t, const unsigned int version)
{
archive & boost::serialization::base_object<GameCore::Object>(t);
archive & boost::serialization::base_object<GameCore::Updatable>(t);
archive & t.gameObject;
archive & t.transform;
}
}
}
That is one header file. Sorry for the verbosity. Its .cpp file starts like this:
#include "Component.h"
#include <boost/serialization/export.hpp>
#include <boost/archive/text_iarchive.hpp>
#include <boost/archive/text_oarchive.hpp>
// Other includes...
BOOST_CLASS_EXPORT_GUID(GameCore::Component, "GameCore::Component");
// Class method definitions here.
The exception is raised when I try to archive an instance object that derives from Component, which itself is an abstract class. I'm archiving through a simple method defined in a different manager class:
std::ofstream outputFile(fileName);
boost::archive::text_oarchive outputArchive(outputFile);
outputArchive << objects;
where objects is a std::list of Objects. Object is the base class from which all things, including Components, derive.
I apologize if this sounds convoluted, but there's only three layers of inheritance, and I believe I had a neat and effective architecture before thoughts of serialization crept in.
If you could help me get rid of the irrational unregistered_class exceptions I'll light a candle for your souls!
Update: Funny thing is, the exception isn't raised for all derived classes of Component.
After frying my neurones looking for an answer, I stumbled upon this line in the documentation:
Static Libraries and Serialization
Code for serialization of data types can be saved in libraries just as
it can for the rest of the type implementation. This works well, and
can save huge amount of compilation time. Only compile serialization
definitions in the library. Explicitly instantiate serialization code
for ALL archive classes you intend to use in the library. For exported
types, only use BOOST_CLASS_EXPORT_KEY in headers. For exported types,
only use BOOST_CLASS_EXPORT_IMPLEMENT in definitions compiled in the
library. For any particular type, there should be only one file which
contains BOOST_CLASS_EXPORT_IMPLEMENT for that type. This ensures that
only one copy of serialization code will exist within the program. It
avoids wasted space and the possibility of having different versions
of the serialization code in the same program. Including
BOOST_CLASS_EXPORT_IMPLEMENT in multiple files could result in a
failure to link due to duplicated symbols or the throwing of a runtime
exception.
Splitting BOOST_CLASS_EXPORT into BOOST_CLASS_EXPORT_KEY and BOOST_CLASS_EXPORT_IMPLEMENT seems to work.
I have an odd issue in MSVS 2010. I have a class with a function that is templitized and contains an parameter with a default value.
In my header file:
typedef unsinged int data32
class myClass
{
private:
...
public:
...
template <typename T>
T* myF(data32);
}
...
template<typename T>
T* myClass::myF(data32 size = 1)
{
...
}
Ok, now in my main i have something like this:
int main()
{
myClass A;
data32* myInt = A.myF<data32>(100); // no complaints from pre-compiler
data32* myInt2 = A.myF<data32>(); // pre-compiler complains "Error: no instance of the function template "myClass::myF" matches the argument list"
}
I understand why it is unhappy as i do not have a function prototype defined for 'myF()' in the class, but shouldn't it know better? I thought the point of defaults were to make the parameters optional in the call. The code DOES compile and run just fine even thought the pre-compiler is unhappy and flags this as a problem.
Any thoughts??
Thanks!
There are bugs (false alarms) in the intellisense analyzer in VS 2010. And this seems like one of them. The analyzer used for intellisense is different from the actual parser used in compiler.