How can I serialize a class (with boost::serialization) that contains a boost::optional?
I.e. the following code will give an error when instantiated.
error C2039: 'serialize' : is not a member of 'boost::optional' C:\boost\boost_1_55_0\boost\serialization\access.hpp 118
#include <boost/archive/text_oarchive.hpp>
#include <boost/archive/text_iarchive.hpp>
class MyClass {
private:
friend class boost::serialization::access;
template<class Archive>
void serialize(Archive & ar, const unsigned int version)
{
ar & my_member;
}
boost::optional<int> my_member;
};
int main() {
std::ofstream ofs("filename.txt");
const MyClass g;
boost::archive::text_oarchive oa(ofs);
oa << g;
return 0;
}
I understand there's probably a deeper question involved (what should you write to the file when the value is not present?), but there must be some standard solution for it. I am looking for the most simple way to solve this.
For boost::optional you just need to add #include <boost/serialization/optional.hpp>
It implements a non-member serialize function that will allow you to serialize boost::optional without worrying about the details.
Under the hood it first saves/loads the boolean value of t.is_initialized() and depending on its value decides if to save/load the rest.
You can see the source code here: http://www.boost.org/doc/libs/1_56_0/boost/serialization/optional.hpp
Related
I am using the cereal library to serialize stuff. I am trying to serialize a class member of type std::vector with struct{ some unsingned short ints }.
This fails with the compiler message /usr/include/cereal/cereal.hpp:543: error: static assertion failed: cereal could not find any output serialization functions for the provided type and archive combination.
Replacing the data to be archived with an std::vector of unsigned short ints directly works as expected. Can somebody tell what I am doing wrong or if cereal is even capable of doing this with structs?
I don't get it, because int's of any kind are evidently supported and vectors are too after adding the appropriate include. Just wrapping the ints in a struct does not work?
Simplified data class with stuff to be serialized: database.h
#include <utils/x_precompiled_headers.h>
#include <utils/serialize.h>
#include <database/datamodel.h>
class Database : public QObject
{
Q_OBJECT
private:
std::vector<Datamodel::model> models_;
// std::vector<unsigned short int> test = {1,2,3};
void Database::SaveToDisk(){
Serialize::ExportData(*this, "database");
}
void Database::LoadFromDisk(){
Serialize::ImportData(*this, "database");
}
// serialization
friend class cereal::access;
template<class Archive>
void save(Archive &ar) const {
// ar(test); // this does not complain
ar(models_); //this gives the compiler error
}
};
Definition header of the struct that I want to serialze: database/datamodel.h
namespace Datamodel
{
typedef struct{
unsigned short int number1;
unsigned short int number2;
template<class Archive>
void save(Archive &ar) const{
ar(number1, number2);
}
template<class Archive>
void load(Archive &ar) const{
ar(number1, number2);
}
} model;
}
Serialization class: utils/serialize.h
class Serialize
{
public:
template<typename T>
static void ExportData(T &object, const std::string &filename)
{
std::string path = std::filesystem::current_path() /= filename;
std::ofstream ofs(path);
if(ofs.is_open()){
cereal::BinaryOutputArchive oarchive(ofs);
oarchive(object);
ofs.close();
}
}
template <typename T>
static void ImportData(T &object, const std::string &filename)
{
std::string path = std::filesystem::current_path() /= filename;
if(!std::filesystem::exists(path))
return;
std::ifstream ifs(path);
if(ifs.is_open())
{
cereal::BinaryInputArchive iarchive(ifs);
iarchive(object);
ifs.close();
}
}
};
Precompiled headers file, containing relevant includes: x_precompiled_headers.h
#include <cereal/access.hpp>
#include <cereal/archives/binary.hpp>
#include <cereal/types/string.hpp>
#include <cereal/types/vector.hpp>
Update 1: Okay, I have now updated the code and provided my struct with serialization methods. I have updated the code in th OP to reflect the changes. I have also included the save and load functions that I use in the database.h in the OP. Sadly it sill gives me this compiler error (/usr/include/cereal/cereal.hpp:822: error: no matching function for call to ‘cereal::BinaryInputArchive::processImpl(const std::vectorDatamodel::model&)’).
Cereal knows how to serialize standard types such as vectors and ints out of the box, but not how to serialize Datamodel::model.
It also does not know out of the box how to serialize Database, but you told it how to with save
// serialization
friend class cereal::access;
template<class Archive>
void save(Archive &ar) const {
// ar(test); // this does not complain
ar(models_); //this gives the compiler error
}
You need to provide a similar implementation for Datamodel::model.
// serialization
friend class cereal::access;
template<class Archive>
void save(Archive &ar) const {
ar(number1);
ar(number2);
}
Solution: the load function (when you split save and loads) cannot be const.
When splitting save and load templated functions, this is the correct scheme, also specified in the serialization function specification on cereal's website (https://uscilab.github.io/cereal/serialization_functions.html):
template<class Archive>
void save(Archive &ar) const{
ar(models_);
}
template<class Archive>
void load(Archive &ar){
ar(models_);
}
I'm using SFML and cereal to serialize/deserialize data and I want to do that for sf::vector2 and sf::vector3 class:
Data.h
#include <SFML\System.hpp>
#include <fstream>
#include <iostream>
#include "cereal-1.2.2\include\cereal\archives\xml.hpp"
#include "cereal-1.2.2\include\cereal\types\map.hpp"
struct DataInfo {
map<string, sf::Vector2f> vector2FloatData;
map<string, sf::Vector3f> vector3FloatData;
map<string, sf::Vector2i> vector2IntData;
map<string, sf::Vector3i> vector3IntData;
template <class Archive>
void serialize(Archive & ar)
{
ar(vector2FloatData, vector3FloatData, vector2IntData, vector3IntData);
}
};
main.cpp
int Main()
{
std::ofstream file("Test.xml");
cereal::XMLOutputArchive archive(file);
DataInfo data;
archive(data);
return 0;
}
But cereal doesn't know what are sf::vectors and i get the following error:
Error C2338 cereal could not find any output serialization functions for the provided type and archive combination.
I know that exist CEREAL_REGISTER_TYPE() but i don't know how to make it work.
Adding to Data.h:
#include "cereal-1.2.2\include\cereal\types\polymorphic.hpp"
CEREAL_REGISTER_TYPE(sf::Vector2f)
CEREAL_REGISTER_TYPE(sf::Vector3f)
CEREAL_REGISTER_TYPE(sf::Vector2i)
CEREAL_REGISTER_TYPE(sf::Vector3i)
I get this error:
Error C2338 Attempting to register non polymorphic type.
Any idea?
Thanks.
I can't verify if it works, i guess you need to add the serialization for sfml types manually such as:
namespace /*cereal*/ sf
{
template<class Archive>
void serialize(Archive & archive, sf::Vector2i & v)
{
archive( v.x, v.y);
}
}
Not sure again, yet it may work with templated types too:
namespace /*cereal*/ sf
{
template<class Archive, class Type>
void serialize(Archive & archive, sf::Vector2<Type> & v)
{
archive( v.x, v.y);
}
}
They are not polymorphic types, so you don't need to register anything.
--EDIT--
Add the function under sf namespace.
I am trying to serialise/deserialise a simple object.
I am able to serialise it:
#include <vector>
#include <sstream>
#include <boost/archive/text_oarchive.hpp>
#include <boost/archive/text_iarchive.hpp>
#include <boost/serialization/vector.hpp>
class DataClass{
public:
std::vector<std::string> data;
DataClass(){}
~DataClass(){}
friend class boost::serialization::access;
template<typename Archive>
void serialize(Archive & ar, const unsigned int version) const{
ar & data;
}
};
int main(){
using std::cout;
using std::endl;
using std::string;
DataClass data_obj;
data_obj.data.push_back("some data 1");
data_obj.data.push_back("some data 2");
std::ostringstream archive_stream;
boost::archive::text_oarchive archive(archive_stream);
archive << data_obj;
string str_data_to_send = archive_stream.str();
const char* data_to_send = archive_stream.str().c_str();
std::istringstream archive_stream2(data_to_send);
boost::archive::text_iarchive archive2(archive_stream2);
DataClass received_data_obj;
//archive2 >> received_data_obj;
}
I receive an error if I uncomment the last statement
archive2 >> received_data_obj;
In file included from /opt/local/include/boost/archive/text_oarchive.hpp:31:
In file included from /opt/local/include/boost/archive/basic_text_oarchive.hpp:32:
In file included from /opt/local/include/boost/archive/detail/common_oarchive.hpp:22:
In file included from /opt/local/include/boost/archive/detail/interface_oarchive.hpp:23:
In file included from /opt/local/include/boost/archive/detail/oserializer.hpp:67:
/opt/local/include/boost/archive/detail/check.hpp:162:5: error: static_assert failed "typex::value"
BOOST_STATIC_ASSERT(typex::value);
^ ~~~~~~~~~~~~
I can not post the whole error message because my post will be "mostly code".
Go to the source code, where static assert occurred, and you'll see the comments that explain the issue:
// cannot load data into a "const" object unless it's a
// wrapper around some other non-const object.
This happens because you defined serialization member function as const, so data member is also const when being accessed within serialization function.
Please help me deserialize a derived class to base-class pointer. I attach the complete source code example.
request.hpp (no pair cpp file)
#ifndef REQUEST_HPP
#define REQUEST_HPP
#include <memory>
#include <string>
#include <boost/archive/xml_oarchive.hpp>
#include <boost/archive/xml_iarchive.hpp>
namespace demo {
namespace common {
class request {
public:
static const int INVALID_ID = -42;
request()
: id_(INVALID_ID), timestamp_(0), source_ip_("unknown") {};
request(int id, long timestamp, const std::string& source_ip)
: id_(id), timestamp_(timestamp), source_ip_(source_ip) {};
virtual ~request() {};
int id() const { return id_; }
long timestamp() const { return timestamp_; }
std::string source_ip() const { return source_ip_; }
protected:
int id_;
long timestamp_;
std::string source_ip_;
private:
friend class boost::serialization::access;
template<class Archive>
void serialize(Archive& ar, const unsigned version) {
ar & BOOST_SERIALIZATION_NVP(id_);
ar & BOOST_SERIALIZATION_NVP(timestamp_);
ar & BOOST_SERIALIZATION_NVP(source_ip_);
}
};
typedef std::shared_ptr<request> request_ptr;
}
};
#endif
command.hpp (derived class)
#ifndef COMMAND_HPP
#define COMMAND_HPP
#include <memory>
#include <string>
#include <boost/serialization/export.hpp>
#include <demo/common/request.hpp>
namespace demo {
namespace common {
class command : public request {
public:
command(): name_("untitled") {};
explicit command(const std::string& name) : name_(name) {};
virtual ~command() {};
virtual void execute();
std::string name() const { return name_; }
protected:
std::string name_;
private:
friend class boost::serialization::access;
template<class Archive>
void serialize(Archive& ar, const unsigned version) {
ar & BOOST_SERIALIZATION_BASE_OBJECT_NVP(request);
ar & BOOST_SERIALIZATION_NVP(name_);
}
};
typedef std::shared_ptr<command> command_ptr;
}
};
BOOST_CLASS_EXPORT_KEY(demo::common::command)
#endif
command.cpp
#include "command.hpp"
#include <iostream>
BOOST_CLASS_EXPORT_IMPLEMENT(demo::common::command)
namespace demo {
namespace common {
void command::execute() {
std::cout << " I am '" + name_ +"' and I am executing..." << std::endl;
}
}
};
serializer.hpp
#ifndef SERIALIZER_HPP
#define SERIALIZER_HPP
#include <sstream>
#include <string>
/* classes to serialize */
#include <demo/common/request.hpp>
#include <demo/common/command.hpp>
namespace demo {
namespace common {
class serializer {
public:
serializer() : {};
template<typename T>
std::string serialize(const T& t){
std::stringstream stream;
boost::archive::xml_oarchive archive(stream);
archive << BOOST_SERIALIZATION_NVP(t);
std::string serialized = stream.str();
return serialized;
}
template<typename T>
void deserialize(const std::string& serialized, T& t) {
std::stringstream stream(serialized);
boost::archive::xml_iarchive archive(stream);
archive >> BOOST_SERIALIZATION_NVP(t);
}
};
}
}
#endif
sample usage
#include <iostream>
#include <demo/common/serializer.hpp>
#include <demo/common/command.hpp>
using namespace std;
using namespace demo::common;
int main(){
serializer serializer_;
command r("123"); // <-- (1) my desired way of declaring
//request* r = new command("123"); <-- (2) replacing with this makes all work!
//command* r = new command("123"); <-- (3) replacing with this crashes the app, like (1)
std::string s = serializer_.serialize(r);
std::cout << s << std::endl;
request* rr = nullptr;
serializer_.deserialize(s, rr); //this throws an exception
command* rrr = dynamic_cast<command*>(rr);
rrr->execute();
}
I thought I did everything that needs to be done, archives included before any class export, all default constructors initialize members..
Note that the serializable classes and the serializer are compiled to a lib file. Then that lib is used in two sub-projects that have access to the headers and have that lib linked. They use those classes to communicate with each other, they send serialized objects over network.
Why can't I deserialize a derived class to a base class pointer?
I am using Boost 1.51 and VC11.
Problems:
The two major things I found finicky and not documented enough about Boost::serialization that caused me issues are as follows:
Serialization / deserialization of objects on the stack mixed with objects on the heap. For example if you serialize from a object on the stack then attempt to deserialize to a pointer (e.g. invoke your load_construct_data<>) an exception may occur. Same with the reverse scenario.
Not having your exports linked in properly. If you create serialization templates/classes and place them in a .lib for example, it seems the exports may not be properly linked in / exposed. This goes for linking in and then using from a shared object/DLL.
Solutions:
For #1, I've found it easiest to make a rule of always serializing/deserializing to/from pointers. Even objects on the stack can use a temporary pointer when serializing to allow for this rule. For example:
// serialize
MyObject myobj;
std::ostringstream oss;
boost::archive::text_oarchive oa(oss);
MyObject* myObjPtr = &myObj;
oa << myObjPtr; // this is different than oa << myObj!!
std::string serialized = oss.str();
// deserialize
MyObject* myNewObjPtr;
std::stringstream iss(serialized);
boost::archive::text_iarchive ia(iss);
ia >> myNewObjPtr; // invokes new, don't forget to delete (or use smart ptrs!!!)
For #2, simply create a .cpp file that contains all of your exports. Link this CPP into your module(s) directly. In other words, you'll have a .cpp with a bunch of BOOST_CLASS_EXPORT_IMPLEMENT():
BOOST_CLASS_EXPORT_IMPLEMENT(MyObject);
// ...
More Complete Example:
Below is a more complete example showing some of the serialization tricks using non-intrusive templates. Intrusive member methods will be very similar:
MyObject.h
// Can be broken into MyObject.h, MyObject.cpp, MyObjectSerialization.h for example as well.
// This stuff can live in your .lib
#include <boost/serialization/export.hpp>
#include <boost/archive/text_oarchive.hpp>
#include <boost/archive/text_iarchive.hpp>
// assume this class contains GetSomeMember() returning SomeMemberType
class MyObject { /* ... */ };
BOOST_CLASS_EXPORT_KEY(MyObject);
namespace boost { namespace serialization {
template<class Archive>
void serialize(Archive& ar, MyObject& myObj, const unsigned int version)
{
ar & myObj.m_someMember;
}
template<class Archive>
inline void save_construct_data(Archive& ar, const MyObject* myObj, const unsigned int version)
{
ar & boost::serialization::make_nvp("SomeMemberType", static_cast<const SomeMemberType&>(myObj->GetSomeMember()));
}
template<class Archive>
inline void load_construct_data(Archive& ar, MyObject* myObj, const unsigned int version)
{
SomeMemberType t;
ar & boost::serialization::make_nvp("SomeMemberType", t);
::new(myObj)MyObject(t);
}
} } // end boost::serialization ns
MyObjectExports.cpp
// This file must be explicitly linked into your module(s) that use serialization.
// This means your executable or shared module/DLLs
#include <boost/serialization/export.hpp>
#include <boost/archive/text_oarchive.hpp>
#include <boost/archive/text_iarchive.hpp>
#include "MyObject.h"
BOOST_CLASS_EXPORT_IMPLEMENT(MyObject);
You're probably getting an input_stream_error in your demo and unregistered_class exception when using your library. This is caused by the way boost is registering the classes, in your case, automatically.
It appears that the automatic registration process gets confused when you serialize a derived object and deserialize to its base, despite the use of the BOOST_CLASS_EXPORT* macros.
However, you can register the classes explicitly before you perform any i/o operation on the archive:
// ...
boost::archive::xml_iarchive archive(stream);
// register the class(es) with the archive
archive.template register_type<command>();
archive >> BOOST_SERIALIZATION_NVP(t);
// ...
Use the same order of registration when serializing. This makes the export macros superfluous.
The serialization example below is from the boost mailing list which is pretty much the same as what I would like to do. However, I have changed the archive so that it will serialize to XML. The compile does not fail if I serialize to binary, but it fails when serializing to xml. The compile fails in basic_xml_oarchive.hpp in the following method:
// boost code where compile fails
template<class T>
void save_override(T & t, BOOST_PFTO int)
{
// If your program fails to compile here, its most likely due to
// not specifying an nvp wrapper around the variable to
// be serialized.
BOOST_MPL_ASSERT((serialization::is_wrapper<T>));
this->detail_common_oarchive::save_override(t, 0);
}
It seems I haven't done enough to allow the std::map<int, CSomeData> object to be serialized, any ideas on how to fix this?
My serialization implementation:
#include <boost/archive/xml_oarchive.hpp>
#include <boost/archive/xml_iarchive.hpp>
#include <boost/serialization/map.hpp>
#include <fstream>
#include <string>
#include <map>
using namespace std;
// This is a test class to use as the map data.
class CSomeData {
public:
CSomeData(){};
CSomeData(float f0, string str0)
{
m_f0 = f0;
m_str0 = str0;
}
float m_f0;
string m_str0;
private:
friend class boost::serialization::access;
template<class Archive>
void serialize(Archive &ar, const unsigned int version)
{
ar & m_f0;
ar & m_str0;
}
};
// This is the class we really want to try serializing.
class CTest {
public:
CTest(){};
CTest(int nNumber)
{
m_nNumber = nNumber;
// Fill with some dummy data.
m_mTst.insert(make_pair(0, CSomeData(0.23f, "hi hi hi")));
m_mTst.insert(make_pair(1, CSomeData(7.65f, "second one")));
m_mTst.insert(make_pair(2, CSomeData(9.23f, "third one")));
m_mTst.insert(make_pair(3, CSomeData(5.6766, "chosen one")));
}
~CTest(){};
save()
{
std::ofstream ofs("filename");
// Write class instance to archive. Writing seems to work ok.
boost::archive::xml_oarchive oa(ofs);
oa << BOOST_SERIALIZATION_NVP(*this);
}
int m_nNumber;
private:
map<int, CSomeData> m_mTst;
friend class boost::serialization::access;
template<class Archive>
void serialize(Archive &ar, const unsigned int version)
{
ar & m_nNumber;
ar & m_mTst;
}
};
I believe you need to tag the members with a name for XML serialisation. This specifies the element name to use in the XML. I.e. use something like:
ar & BOOST_SERIALIZATION_NVP(m_f0);
or (better in this case):
ar & make_nvp("field0", my_f0);
The tags will be ignored for binary serialisation. More details here:
http://www.boost.org/doc/libs/1_43_0/libs/serialization/doc/wrappers.html