Serializing a map of objects to xml using boost::serialization - c++

The serialization example below is from the boost mailing list which is pretty much the same as what I would like to do. However, I have changed the archive so that it will serialize to XML. The compile does not fail if I serialize to binary, but it fails when serializing to xml. The compile fails in basic_xml_oarchive.hpp in the following method:
// boost code where compile fails
template<class T>
void save_override(T & t, BOOST_PFTO int)
{
// If your program fails to compile here, its most likely due to
// not specifying an nvp wrapper around the variable to
// be serialized.
BOOST_MPL_ASSERT((serialization::is_wrapper<T>));
this->detail_common_oarchive::save_override(t, 0);
}
It seems I haven't done enough to allow the std::map<int, CSomeData> object to be serialized, any ideas on how to fix this?
My serialization implementation:
#include <boost/archive/xml_oarchive.hpp>
#include <boost/archive/xml_iarchive.hpp>
#include <boost/serialization/map.hpp>
#include <fstream>
#include <string>
#include <map>
using namespace std;
// This is a test class to use as the map data.
class CSomeData {
public:
CSomeData(){};
CSomeData(float f0, string str0)
{
m_f0 = f0;
m_str0 = str0;
}
float m_f0;
string m_str0;
private:
friend class boost::serialization::access;
template<class Archive>
void serialize(Archive &ar, const unsigned int version)
{
ar & m_f0;
ar & m_str0;
}
};
// This is the class we really want to try serializing.
class CTest {
public:
CTest(){};
CTest(int nNumber)
{
m_nNumber = nNumber;
// Fill with some dummy data.
m_mTst.insert(make_pair(0, CSomeData(0.23f, "hi hi hi")));
m_mTst.insert(make_pair(1, CSomeData(7.65f, "second one")));
m_mTst.insert(make_pair(2, CSomeData(9.23f, "third one")));
m_mTst.insert(make_pair(3, CSomeData(5.6766, "chosen one")));
}
~CTest(){};
save()
{
std::ofstream ofs("filename");
// Write class instance to archive. Writing seems to work ok.
boost::archive::xml_oarchive oa(ofs);
oa << BOOST_SERIALIZATION_NVP(*this);
}
int m_nNumber;
private:
map<int, CSomeData> m_mTst;
friend class boost::serialization::access;
template<class Archive>
void serialize(Archive &ar, const unsigned int version)
{
ar & m_nNumber;
ar & m_mTst;
}
};

I believe you need to tag the members with a name for XML serialisation. This specifies the element name to use in the XML. I.e. use something like:
ar & BOOST_SERIALIZATION_NVP(m_f0);
or (better in this case):
ar & make_nvp("field0", my_f0);
The tags will be ignored for binary serialisation. More details here:
http://www.boost.org/doc/libs/1_43_0/libs/serialization/doc/wrappers.html

Related

I want to serialize a vector of structs containing ints and it does not work (cereal library)

I am using the cereal library to serialize stuff. I am trying to serialize a class member of type std::vector with struct{ some unsingned short ints }.
This fails with the compiler message /usr/include/cereal/cereal.hpp:543: error: static assertion failed: cereal could not find any output serialization functions for the provided type and archive combination.
Replacing the data to be archived with an std::vector of unsigned short ints directly works as expected. Can somebody tell what I am doing wrong or if cereal is even capable of doing this with structs?
I don't get it, because int's of any kind are evidently supported and vectors are too after adding the appropriate include. Just wrapping the ints in a struct does not work?
Simplified data class with stuff to be serialized: database.h
#include <utils/x_precompiled_headers.h>
#include <utils/serialize.h>
#include <database/datamodel.h>
class Database : public QObject
{
Q_OBJECT
private:
std::vector<Datamodel::model> models_;
// std::vector<unsigned short int> test = {1,2,3};
void Database::SaveToDisk(){
Serialize::ExportData(*this, "database");
}
void Database::LoadFromDisk(){
Serialize::ImportData(*this, "database");
}
// serialization
friend class cereal::access;
template<class Archive>
void save(Archive &ar) const {
// ar(test); // this does not complain
ar(models_); //this gives the compiler error
}
};
Definition header of the struct that I want to serialze: database/datamodel.h
namespace Datamodel
{
typedef struct{
unsigned short int number1;
unsigned short int number2;
template<class Archive>
void save(Archive &ar) const{
ar(number1, number2);
}
template<class Archive>
void load(Archive &ar) const{
ar(number1, number2);
}
} model;
}
Serialization class: utils/serialize.h
class Serialize
{
public:
template<typename T>
static void ExportData(T &object, const std::string &filename)
{
std::string path = std::filesystem::current_path() /= filename;
std::ofstream ofs(path);
if(ofs.is_open()){
cereal::BinaryOutputArchive oarchive(ofs);
oarchive(object);
ofs.close();
}
}
template <typename T>
static void ImportData(T &object, const std::string &filename)
{
std::string path = std::filesystem::current_path() /= filename;
if(!std::filesystem::exists(path))
return;
std::ifstream ifs(path);
if(ifs.is_open())
{
cereal::BinaryInputArchive iarchive(ifs);
iarchive(object);
ifs.close();
}
}
};
Precompiled headers file, containing relevant includes: x_precompiled_headers.h
#include <cereal/access.hpp>
#include <cereal/archives/binary.hpp>
#include <cereal/types/string.hpp>
#include <cereal/types/vector.hpp>
Update 1: Okay, I have now updated the code and provided my struct with serialization methods. I have updated the code in th OP to reflect the changes. I have also included the save and load functions that I use in the database.h in the OP. Sadly it sill gives me this compiler error (/usr/include/cereal/cereal.hpp:822: error: no matching function for call to ‘cereal::BinaryInputArchive::processImpl(const std::vectorDatamodel::model&)’).
Cereal knows how to serialize standard types such as vectors and ints out of the box, but not how to serialize Datamodel::model.
It also does not know out of the box how to serialize Database, but you told it how to with save
// serialization
friend class cereal::access;
template<class Archive>
void save(Archive &ar) const {
// ar(test); // this does not complain
ar(models_); //this gives the compiler error
}
You need to provide a similar implementation for Datamodel::model.
// serialization
friend class cereal::access;
template<class Archive>
void save(Archive &ar) const {
ar(number1);
ar(number2);
}
Solution: the load function (when you split save and loads) cannot be const.
When splitting save and load templated functions, this is the correct scheme, also specified in the serialization function specification on cereal's website (https://uscilab.github.io/cereal/serialization_functions.html):
template<class Archive>
void save(Archive &ar) const{
ar(models_);
}
template<class Archive>
void load(Archive &ar){
ar(models_);
}

How to boost::serialize an std/boost::optional?

How can I serialize a class (with boost::serialization) that contains a boost::optional?
I.e. the following code will give an error when instantiated.
error C2039: 'serialize' : is not a member of 'boost::optional' C:\boost\boost_1_55_0\boost\serialization\access.hpp 118
#include <boost/archive/text_oarchive.hpp>
#include <boost/archive/text_iarchive.hpp>
class MyClass {
private:
friend class boost::serialization::access;
template<class Archive>
void serialize(Archive & ar, const unsigned int version)
{
ar & my_member;
}
boost::optional<int> my_member;
};
int main() {
std::ofstream ofs("filename.txt");
const MyClass g;
boost::archive::text_oarchive oa(ofs);
oa << g;
return 0;
}
I understand there's probably a deeper question involved (what should you write to the file when the value is not present?), but there must be some standard solution for it. I am looking for the most simple way to solve this.
For boost::optional you just need to add #include <boost/serialization/optional.hpp>
It implements a non-member serialize function that will allow you to serialize boost::optional without worrying about the details.
Under the hood it first saves/loads the boolean value of t.is_initialized() and depending on its value decides if to save/load the rest.
You can see the source code here: http://www.boost.org/doc/libs/1_56_0/boost/serialization/optional.hpp

Serializing Vector of Objects which contains Vectors of Pointers

I have 3 classes ("Leader", "Researchers", "Workers") which all derive from a base-class "Team".
I have a class "Project" which contains a vector of pointers to different Teams.
I use all of the following headers, in this order, in all of my class declarations:
#include <sstream>
#include <boost/archive/text_oarchive.hpp>
#include <boost/archive/text_iarchive.hpp>
#include <boost/serialization/serialization.hpp>
#include <boost/serialization/export.hpp>
#include <boost/serialization/vector.hpp>
#include <boost/serialization/version.hpp>
#include <boost/serialization/split_member.hpp>
To (de)serialize the Team object I use:
private:
friend class boost::serialization::access ;
template <typename Archive>
void serialize(Archive& ar, const unsigned int /*version*/)
{
ar & teamname ;
}
To (de)serialize the Leader, Researchers, Workers objects I use:
typedef Team _super;
friend class boost::serialization::access ;
template <typename Archive>
void serialize(Archive& ar, const unsigned int /*version*/)
{
ar & boost::serialization::base_object<_super>(*this) ;
ar & contactTelephoneNumber ;
}
The Project holds a std::vector of pointers to different teams and a string using:
std::vector<Team *> teams ;
std::string note ;
I use the following code in the Project class for serialization:
private:
friend class boost::serialization::access ;
template <typename Archive>
void serialize(Archive& ar, const unsigned int /*version*/)
{
//ar & BOOST_SERIALIZATION_NVP(teams) ; //ERROR OCCURS HERE
ar & teams;
ar & note ;
}
And to serialize the vector of Project objects in the main I use:
{
std::ostringstream archiveStream ;
boost::archive::text_oarchive archive(archiveStream) ;
archive << BOOST_SERIALIZATION_NVP(projects) ;
//Get serialized info as string
archivedProjects = archiveStream.str() ;
}
This all compiles fine. The issue is at Run-Time. When the above section of the code is reached, I get the following error:
terminate called after throwing an instance of 'boost::archive::archive_exception'
what():
unregistered class - derevided class not registered or exported"
The program goes as far as:
ar & teams;
In the Questionnaire class's serialization attempt.
As in n.m.'s link: you need to register the classes with Boost so that it knows what classes are what when serialising.
You need to add the following line for each class where "Project" serializes:
ar.template register_type<ClassName>() ; //ClassName = Team etc

Boost deserialize a derived class to base class pointer

Please help me deserialize a derived class to base-class pointer. I attach the complete source code example.
request.hpp (no pair cpp file)
#ifndef REQUEST_HPP
#define REQUEST_HPP
#include <memory>
#include <string>
#include <boost/archive/xml_oarchive.hpp>
#include <boost/archive/xml_iarchive.hpp>
namespace demo {
namespace common {
class request {
public:
static const int INVALID_ID = -42;
request()
: id_(INVALID_ID), timestamp_(0), source_ip_("unknown") {};
request(int id, long timestamp, const std::string& source_ip)
: id_(id), timestamp_(timestamp), source_ip_(source_ip) {};
virtual ~request() {};
int id() const { return id_; }
long timestamp() const { return timestamp_; }
std::string source_ip() const { return source_ip_; }
protected:
int id_;
long timestamp_;
std::string source_ip_;
private:
friend class boost::serialization::access;
template<class Archive>
void serialize(Archive& ar, const unsigned version) {
ar & BOOST_SERIALIZATION_NVP(id_);
ar & BOOST_SERIALIZATION_NVP(timestamp_);
ar & BOOST_SERIALIZATION_NVP(source_ip_);
}
};
typedef std::shared_ptr<request> request_ptr;
}
};
#endif
command.hpp (derived class)
#ifndef COMMAND_HPP
#define COMMAND_HPP
#include <memory>
#include <string>
#include <boost/serialization/export.hpp>
#include <demo/common/request.hpp>
namespace demo {
namespace common {
class command : public request {
public:
command(): name_("untitled") {};
explicit command(const std::string& name) : name_(name) {};
virtual ~command() {};
virtual void execute();
std::string name() const { return name_; }
protected:
std::string name_;
private:
friend class boost::serialization::access;
template<class Archive>
void serialize(Archive& ar, const unsigned version) {
ar & BOOST_SERIALIZATION_BASE_OBJECT_NVP(request);
ar & BOOST_SERIALIZATION_NVP(name_);
}
};
typedef std::shared_ptr<command> command_ptr;
}
};
BOOST_CLASS_EXPORT_KEY(demo::common::command)
#endif
command.cpp
#include "command.hpp"
#include <iostream>
BOOST_CLASS_EXPORT_IMPLEMENT(demo::common::command)
namespace demo {
namespace common {
void command::execute() {
std::cout << " I am '" + name_ +"' and I am executing..." << std::endl;
}
}
};
serializer.hpp
#ifndef SERIALIZER_HPP
#define SERIALIZER_HPP
#include <sstream>
#include <string>
/* classes to serialize */
#include <demo/common/request.hpp>
#include <demo/common/command.hpp>
namespace demo {
namespace common {
class serializer {
public:
serializer() : {};
template<typename T>
std::string serialize(const T& t){
std::stringstream stream;
boost::archive::xml_oarchive archive(stream);
archive << BOOST_SERIALIZATION_NVP(t);
std::string serialized = stream.str();
return serialized;
}
template<typename T>
void deserialize(const std::string& serialized, T& t) {
std::stringstream stream(serialized);
boost::archive::xml_iarchive archive(stream);
archive >> BOOST_SERIALIZATION_NVP(t);
}
};
}
}
#endif
sample usage
#include <iostream>
#include <demo/common/serializer.hpp>
#include <demo/common/command.hpp>
using namespace std;
using namespace demo::common;
int main(){
serializer serializer_;
command r("123"); // <-- (1) my desired way of declaring
//request* r = new command("123"); <-- (2) replacing with this makes all work!
//command* r = new command("123"); <-- (3) replacing with this crashes the app, like (1)
std::string s = serializer_.serialize(r);
std::cout << s << std::endl;
request* rr = nullptr;
serializer_.deserialize(s, rr); //this throws an exception
command* rrr = dynamic_cast<command*>(rr);
rrr->execute();
}
I thought I did everything that needs to be done, archives included before any class export, all default constructors initialize members..
Note that the serializable classes and the serializer are compiled to a lib file. Then that lib is used in two sub-projects that have access to the headers and have that lib linked. They use those classes to communicate with each other, they send serialized objects over network.
Why can't I deserialize a derived class to a base class pointer?
I am using Boost 1.51 and VC11.
Problems:
The two major things I found finicky and not documented enough about Boost::serialization that caused me issues are as follows:
Serialization / deserialization of objects on the stack mixed with objects on the heap. For example if you serialize from a object on the stack then attempt to deserialize to a pointer (e.g. invoke your load_construct_data<>) an exception may occur. Same with the reverse scenario.
Not having your exports linked in properly. If you create serialization templates/classes and place them in a .lib for example, it seems the exports may not be properly linked in / exposed. This goes for linking in and then using from a shared object/DLL.
Solutions:
For #1, I've found it easiest to make a rule of always serializing/deserializing to/from pointers. Even objects on the stack can use a temporary pointer when serializing to allow for this rule. For example:
// serialize
MyObject myobj;
std::ostringstream oss;
boost::archive::text_oarchive oa(oss);
MyObject* myObjPtr = &myObj;
oa << myObjPtr; // this is different than oa << myObj!!
std::string serialized = oss.str();
// deserialize
MyObject* myNewObjPtr;
std::stringstream iss(serialized);
boost::archive::text_iarchive ia(iss);
ia >> myNewObjPtr; // invokes new, don't forget to delete (or use smart ptrs!!!)
For #2, simply create a .cpp file that contains all of your exports. Link this CPP into your module(s) directly. In other words, you'll have a .cpp with a bunch of BOOST_CLASS_EXPORT_IMPLEMENT():
BOOST_CLASS_EXPORT_IMPLEMENT(MyObject);
// ...
More Complete Example:
Below is a more complete example showing some of the serialization tricks using non-intrusive templates. Intrusive member methods will be very similar:
MyObject.h
// Can be broken into MyObject.h, MyObject.cpp, MyObjectSerialization.h for example as well.
// This stuff can live in your .lib
#include <boost/serialization/export.hpp>
#include <boost/archive/text_oarchive.hpp>
#include <boost/archive/text_iarchive.hpp>
// assume this class contains GetSomeMember() returning SomeMemberType
class MyObject { /* ... */ };
BOOST_CLASS_EXPORT_KEY(MyObject);
namespace boost { namespace serialization {
template<class Archive>
void serialize(Archive& ar, MyObject& myObj, const unsigned int version)
{
ar & myObj.m_someMember;
}
template<class Archive>
inline void save_construct_data(Archive& ar, const MyObject* myObj, const unsigned int version)
{
ar & boost::serialization::make_nvp("SomeMemberType", static_cast<const SomeMemberType&>(myObj->GetSomeMember()));
}
template<class Archive>
inline void load_construct_data(Archive& ar, MyObject* myObj, const unsigned int version)
{
SomeMemberType t;
ar & boost::serialization::make_nvp("SomeMemberType", t);
::new(myObj)MyObject(t);
}
} } // end boost::serialization ns
MyObjectExports.cpp
// This file must be explicitly linked into your module(s) that use serialization.
// This means your executable or shared module/DLLs
#include <boost/serialization/export.hpp>
#include <boost/archive/text_oarchive.hpp>
#include <boost/archive/text_iarchive.hpp>
#include "MyObject.h"
BOOST_CLASS_EXPORT_IMPLEMENT(MyObject);
You're probably getting an input_stream_error in your demo and unregistered_class exception when using your library. This is caused by the way boost is registering the classes, in your case, automatically.
It appears that the automatic registration process gets confused when you serialize a derived object and deserialize to its base, despite the use of the BOOST_CLASS_EXPORT* macros.
However, you can register the classes explicitly before you perform any i/o operation on the archive:
// ...
boost::archive::xml_iarchive archive(stream);
// register the class(es) with the archive
archive.template register_type<command>();
archive >> BOOST_SERIALIZATION_NVP(t);
// ...
Use the same order of registration when serializing. This makes the export macros superfluous.

boost serialization polymorphic issues

Boost serialization of polymorphic classes seems not working ( 1.40+ boost), e.g. with the following code, I believe I followed the rule: of exporting the class and
I tried on both gcc4.4 (ubuntu) and windows VS2010(with boost 1.48):
in following program, I expect both 10 and 100 are printed, but it only print 10, that means it only serialized the base class;
I mostly copied the example from boost's document, yet it still doesn't work; anybody has any idea?
thanks a lot
LS
#include <iostream>
#include <sstream>
#include <boost/serialization/base_object.hpp>
#include <boost/serialization/serialization.hpp>
#include <boost/archive/text_oarchive.hpp>
#include <boost/archive/text_iarchive.hpp>
#include <boost/serialization/export.hpp>
#define NVP(X) X
class base {
public:
friend class boost::serialization::access;
base (){ v1 = 10;}
int v1;
template<class Archive>
void serialize(Archive & ar, const unsigned int file_version)
{
ar & NVP(v1);
}
};
class derived : public base {
public:
friend class boost::serialization::access;
int v2 ;
derived() { v2 = 100;}
template<class Archive>
void serialize(Archive & ar, const unsigned int file_version){
boost::serialization::base_object<base>(* this);
ar & NVP(v2);
}
};
BOOST_CLASS_EXPORT(base);
BOOST_CLASS_EXPORT_GUID(derived, "derived");
int main ( )
{
std::stringstream ss;
boost::archive::text_oarchive ar(ss);
base *b = new derived();
ar << NVP(b);
std::cout << ss.str();
}
You forgot
virtual ~base() {}
which is not only required for the polymorphic seriialization to work (without it your class is not polymorphic), but I believe omitting it is a misdemeanor in 48 states. IANAL, so YMMV.
Oh, and it should be ar & boost::serialization::base_object<...>.