How to serialize a boost::uuid with cereal - c++

Trying to serialize this simple class:
class Data
{
public:
Data();
Data(boost::uuids::uuid id);
Data(const Data&) = delete;
Data& operator=(const Data&) = delete;
inline boost::uuids::uuid getGuid() { return guid; }
template <class Archive>
void serialize(Archive & ar)
{
ar(guid);
}
private:
boost::uuids::uuid guid;
};
But I get this error message
error C2338: Trying to serialize an unserializable type with an output archive.
Poiting to the uuid. The boost serialization way to enable this would be to add
#include <boost/uuid/uuid_serialize.hpp>
but this doesn't work for cereal out of the box. Cereal documentation says
cereal archives operate on either an std::ostream or std::istream object.
so I tried adding the header where there are defined but no luck
#include <boost/uuid/uuid_io.hpp>

This worked with cereal JSON archives. I also included in comment a way to do it for a binary archive.
#ifndef CEREAL_TYPES_BOOST_UUID_
#define CEREAL_TYPES_BOOST_UUID_
#include <boost/uuid/uuid.hpp>
#include <boost/uuid/uuid_io.hpp>
#include <boost/lexical_cast.hpp>
namespace cereal
{
template <class Archive> inline
void save(Archive& ar, boost::uuids::uuid const& uuid)
{
std::string val = boost::lexical_cast<std::string>(uuid);
ar(val);
// Other approach, probably better for binary
//ar(make_size_tag(static_cast<size_type>(uuid.size())));
//for (auto it = uuid.begin(), end = uuid.end(); it != end; ++it)
// ar(*it);
}
template <class Archive> inline
void load(Archive& ar, boost::uuids::uuid& uuid)
{
std::string val;
ar(val);
uuid = boost::lexical_cast<boost::uuids::uuid>(val);
// Other approach, probably better for binary
//size_type size;
//ar(make_size_tag(size));
//for (auto it = uuid.begin(), end = uuid.end(); it != end; ++it) {
// uint8_t val;
// ar(val);
// *it = val;
//}
}
} // namespace cereal
#endif // CEREAL_TYPES_BOOST_UUID_

The fact that
cereal archives operate on either an std::ostream or std::istream object.
doesn't (at all) imply that it using the IO streaming operators (>>, <<). That's just the archive implementation.
You'll have to implement the free function serialize to let Cereal know about your type. You should be able to reuse the implementation shown in uuid_serialize.hpp. Chances are you should simply
move those definitions into the cereal namespace (preferrably) or into the boost::uuids namespace (might conflict in the future) for ADL to find them
best to treat the UUIDs as simple arrays of bytes (AFAIR boost::uuids::uuid is a POD data type)

Related

I want to serialize a vector of structs containing ints and it does not work (cereal library)

I am using the cereal library to serialize stuff. I am trying to serialize a class member of type std::vector with struct{ some unsingned short ints }.
This fails with the compiler message /usr/include/cereal/cereal.hpp:543: error: static assertion failed: cereal could not find any output serialization functions for the provided type and archive combination.
Replacing the data to be archived with an std::vector of unsigned short ints directly works as expected. Can somebody tell what I am doing wrong or if cereal is even capable of doing this with structs?
I don't get it, because int's of any kind are evidently supported and vectors are too after adding the appropriate include. Just wrapping the ints in a struct does not work?
Simplified data class with stuff to be serialized: database.h
#include <utils/x_precompiled_headers.h>
#include <utils/serialize.h>
#include <database/datamodel.h>
class Database : public QObject
{
Q_OBJECT
private:
std::vector<Datamodel::model> models_;
// std::vector<unsigned short int> test = {1,2,3};
void Database::SaveToDisk(){
Serialize::ExportData(*this, "database");
}
void Database::LoadFromDisk(){
Serialize::ImportData(*this, "database");
}
// serialization
friend class cereal::access;
template<class Archive>
void save(Archive &ar) const {
// ar(test); // this does not complain
ar(models_); //this gives the compiler error
}
};
Definition header of the struct that I want to serialze: database/datamodel.h
namespace Datamodel
{
typedef struct{
unsigned short int number1;
unsigned short int number2;
template<class Archive>
void save(Archive &ar) const{
ar(number1, number2);
}
template<class Archive>
void load(Archive &ar) const{
ar(number1, number2);
}
} model;
}
Serialization class: utils/serialize.h
class Serialize
{
public:
template<typename T>
static void ExportData(T &object, const std::string &filename)
{
std::string path = std::filesystem::current_path() /= filename;
std::ofstream ofs(path);
if(ofs.is_open()){
cereal::BinaryOutputArchive oarchive(ofs);
oarchive(object);
ofs.close();
}
}
template <typename T>
static void ImportData(T &object, const std::string &filename)
{
std::string path = std::filesystem::current_path() /= filename;
if(!std::filesystem::exists(path))
return;
std::ifstream ifs(path);
if(ifs.is_open())
{
cereal::BinaryInputArchive iarchive(ifs);
iarchive(object);
ifs.close();
}
}
};
Precompiled headers file, containing relevant includes: x_precompiled_headers.h
#include <cereal/access.hpp>
#include <cereal/archives/binary.hpp>
#include <cereal/types/string.hpp>
#include <cereal/types/vector.hpp>
Update 1: Okay, I have now updated the code and provided my struct with serialization methods. I have updated the code in th OP to reflect the changes. I have also included the save and load functions that I use in the database.h in the OP. Sadly it sill gives me this compiler error (/usr/include/cereal/cereal.hpp:822: error: no matching function for call to ‘cereal::BinaryInputArchive::processImpl(const std::vectorDatamodel::model&)’).
Cereal knows how to serialize standard types such as vectors and ints out of the box, but not how to serialize Datamodel::model.
It also does not know out of the box how to serialize Database, but you told it how to with save
// serialization
friend class cereal::access;
template<class Archive>
void save(Archive &ar) const {
// ar(test); // this does not complain
ar(models_); //this gives the compiler error
}
You need to provide a similar implementation for Datamodel::model.
// serialization
friend class cereal::access;
template<class Archive>
void save(Archive &ar) const {
ar(number1);
ar(number2);
}
Solution: the load function (when you split save and loads) cannot be const.
When splitting save and load templated functions, this is the correct scheme, also specified in the serialization function specification on cereal's website (https://uscilab.github.io/cereal/serialization_functions.html):
template<class Archive>
void save(Archive &ar) const{
ar(models_);
}
template<class Archive>
void load(Archive &ar){
ar(models_);
}

Serialize/deserialize SFML Vectors class using cereal

I'm using SFML and cereal to serialize/deserialize data and I want to do that for sf::vector2 and sf::vector3 class:
Data.h
#include <SFML\System.hpp>
#include <fstream>
#include <iostream>
#include "cereal-1.2.2\include\cereal\archives\xml.hpp"
#include "cereal-1.2.2\include\cereal\types\map.hpp"
struct DataInfo {
map<string, sf::Vector2f> vector2FloatData;
map<string, sf::Vector3f> vector3FloatData;
map<string, sf::Vector2i> vector2IntData;
map<string, sf::Vector3i> vector3IntData;
template <class Archive>
void serialize(Archive & ar)
{
ar(vector2FloatData, vector3FloatData, vector2IntData, vector3IntData);
}
};
main.cpp
int Main()
{
std::ofstream file("Test.xml");
cereal::XMLOutputArchive archive(file);
DataInfo data;
archive(data);
return 0;
}
But cereal doesn't know what are sf::vectors and i get the following error:
Error C2338 cereal could not find any output serialization functions for the provided type and archive combination.
I know that exist CEREAL_REGISTER_TYPE() but i don't know how to make it work.
Adding to Data.h:
#include "cereal-1.2.2\include\cereal\types\polymorphic.hpp"
CEREAL_REGISTER_TYPE(sf::Vector2f)
CEREAL_REGISTER_TYPE(sf::Vector3f)
CEREAL_REGISTER_TYPE(sf::Vector2i)
CEREAL_REGISTER_TYPE(sf::Vector3i)
I get this error:
Error C2338 Attempting to register non polymorphic type.
Any idea?
Thanks.
I can't verify if it works, i guess you need to add the serialization for sfml types manually such as:
namespace /*cereal*/ sf
{
template<class Archive>
void serialize(Archive & archive, sf::Vector2i & v)
{
archive( v.x, v.y);
}
}
Not sure again, yet it may work with templated types too:
namespace /*cereal*/ sf
{
template<class Archive, class Type>
void serialize(Archive & archive, sf::Vector2<Type> & v)
{
archive( v.x, v.y);
}
}
They are not polymorphic types, so you don't need to register anything.
--EDIT--
Add the function under sf namespace.

Boost deserialize a derived class to base class pointer

Please help me deserialize a derived class to base-class pointer. I attach the complete source code example.
request.hpp (no pair cpp file)
#ifndef REQUEST_HPP
#define REQUEST_HPP
#include <memory>
#include <string>
#include <boost/archive/xml_oarchive.hpp>
#include <boost/archive/xml_iarchive.hpp>
namespace demo {
namespace common {
class request {
public:
static const int INVALID_ID = -42;
request()
: id_(INVALID_ID), timestamp_(0), source_ip_("unknown") {};
request(int id, long timestamp, const std::string& source_ip)
: id_(id), timestamp_(timestamp), source_ip_(source_ip) {};
virtual ~request() {};
int id() const { return id_; }
long timestamp() const { return timestamp_; }
std::string source_ip() const { return source_ip_; }
protected:
int id_;
long timestamp_;
std::string source_ip_;
private:
friend class boost::serialization::access;
template<class Archive>
void serialize(Archive& ar, const unsigned version) {
ar & BOOST_SERIALIZATION_NVP(id_);
ar & BOOST_SERIALIZATION_NVP(timestamp_);
ar & BOOST_SERIALIZATION_NVP(source_ip_);
}
};
typedef std::shared_ptr<request> request_ptr;
}
};
#endif
command.hpp (derived class)
#ifndef COMMAND_HPP
#define COMMAND_HPP
#include <memory>
#include <string>
#include <boost/serialization/export.hpp>
#include <demo/common/request.hpp>
namespace demo {
namespace common {
class command : public request {
public:
command(): name_("untitled") {};
explicit command(const std::string& name) : name_(name) {};
virtual ~command() {};
virtual void execute();
std::string name() const { return name_; }
protected:
std::string name_;
private:
friend class boost::serialization::access;
template<class Archive>
void serialize(Archive& ar, const unsigned version) {
ar & BOOST_SERIALIZATION_BASE_OBJECT_NVP(request);
ar & BOOST_SERIALIZATION_NVP(name_);
}
};
typedef std::shared_ptr<command> command_ptr;
}
};
BOOST_CLASS_EXPORT_KEY(demo::common::command)
#endif
command.cpp
#include "command.hpp"
#include <iostream>
BOOST_CLASS_EXPORT_IMPLEMENT(demo::common::command)
namespace demo {
namespace common {
void command::execute() {
std::cout << " I am '" + name_ +"' and I am executing..." << std::endl;
}
}
};
serializer.hpp
#ifndef SERIALIZER_HPP
#define SERIALIZER_HPP
#include <sstream>
#include <string>
/* classes to serialize */
#include <demo/common/request.hpp>
#include <demo/common/command.hpp>
namespace demo {
namespace common {
class serializer {
public:
serializer() : {};
template<typename T>
std::string serialize(const T& t){
std::stringstream stream;
boost::archive::xml_oarchive archive(stream);
archive << BOOST_SERIALIZATION_NVP(t);
std::string serialized = stream.str();
return serialized;
}
template<typename T>
void deserialize(const std::string& serialized, T& t) {
std::stringstream stream(serialized);
boost::archive::xml_iarchive archive(stream);
archive >> BOOST_SERIALIZATION_NVP(t);
}
};
}
}
#endif
sample usage
#include <iostream>
#include <demo/common/serializer.hpp>
#include <demo/common/command.hpp>
using namespace std;
using namespace demo::common;
int main(){
serializer serializer_;
command r("123"); // <-- (1) my desired way of declaring
//request* r = new command("123"); <-- (2) replacing with this makes all work!
//command* r = new command("123"); <-- (3) replacing with this crashes the app, like (1)
std::string s = serializer_.serialize(r);
std::cout << s << std::endl;
request* rr = nullptr;
serializer_.deserialize(s, rr); //this throws an exception
command* rrr = dynamic_cast<command*>(rr);
rrr->execute();
}
I thought I did everything that needs to be done, archives included before any class export, all default constructors initialize members..
Note that the serializable classes and the serializer are compiled to a lib file. Then that lib is used in two sub-projects that have access to the headers and have that lib linked. They use those classes to communicate with each other, they send serialized objects over network.
Why can't I deserialize a derived class to a base class pointer?
I am using Boost 1.51 and VC11.
Problems:
The two major things I found finicky and not documented enough about Boost::serialization that caused me issues are as follows:
Serialization / deserialization of objects on the stack mixed with objects on the heap. For example if you serialize from a object on the stack then attempt to deserialize to a pointer (e.g. invoke your load_construct_data<>) an exception may occur. Same with the reverse scenario.
Not having your exports linked in properly. If you create serialization templates/classes and place them in a .lib for example, it seems the exports may not be properly linked in / exposed. This goes for linking in and then using from a shared object/DLL.
Solutions:
For #1, I've found it easiest to make a rule of always serializing/deserializing to/from pointers. Even objects on the stack can use a temporary pointer when serializing to allow for this rule. For example:
// serialize
MyObject myobj;
std::ostringstream oss;
boost::archive::text_oarchive oa(oss);
MyObject* myObjPtr = &myObj;
oa << myObjPtr; // this is different than oa << myObj!!
std::string serialized = oss.str();
// deserialize
MyObject* myNewObjPtr;
std::stringstream iss(serialized);
boost::archive::text_iarchive ia(iss);
ia >> myNewObjPtr; // invokes new, don't forget to delete (or use smart ptrs!!!)
For #2, simply create a .cpp file that contains all of your exports. Link this CPP into your module(s) directly. In other words, you'll have a .cpp with a bunch of BOOST_CLASS_EXPORT_IMPLEMENT():
BOOST_CLASS_EXPORT_IMPLEMENT(MyObject);
// ...
More Complete Example:
Below is a more complete example showing some of the serialization tricks using non-intrusive templates. Intrusive member methods will be very similar:
MyObject.h
// Can be broken into MyObject.h, MyObject.cpp, MyObjectSerialization.h for example as well.
// This stuff can live in your .lib
#include <boost/serialization/export.hpp>
#include <boost/archive/text_oarchive.hpp>
#include <boost/archive/text_iarchive.hpp>
// assume this class contains GetSomeMember() returning SomeMemberType
class MyObject { /* ... */ };
BOOST_CLASS_EXPORT_KEY(MyObject);
namespace boost { namespace serialization {
template<class Archive>
void serialize(Archive& ar, MyObject& myObj, const unsigned int version)
{
ar & myObj.m_someMember;
}
template<class Archive>
inline void save_construct_data(Archive& ar, const MyObject* myObj, const unsigned int version)
{
ar & boost::serialization::make_nvp("SomeMemberType", static_cast<const SomeMemberType&>(myObj->GetSomeMember()));
}
template<class Archive>
inline void load_construct_data(Archive& ar, MyObject* myObj, const unsigned int version)
{
SomeMemberType t;
ar & boost::serialization::make_nvp("SomeMemberType", t);
::new(myObj)MyObject(t);
}
} } // end boost::serialization ns
MyObjectExports.cpp
// This file must be explicitly linked into your module(s) that use serialization.
// This means your executable or shared module/DLLs
#include <boost/serialization/export.hpp>
#include <boost/archive/text_oarchive.hpp>
#include <boost/archive/text_iarchive.hpp>
#include "MyObject.h"
BOOST_CLASS_EXPORT_IMPLEMENT(MyObject);
You're probably getting an input_stream_error in your demo and unregistered_class exception when using your library. This is caused by the way boost is registering the classes, in your case, automatically.
It appears that the automatic registration process gets confused when you serialize a derived object and deserialize to its base, despite the use of the BOOST_CLASS_EXPORT* macros.
However, you can register the classes explicitly before you perform any i/o operation on the archive:
// ...
boost::archive::xml_iarchive archive(stream);
// register the class(es) with the archive
archive.template register_type<command>();
archive >> BOOST_SERIALIZATION_NVP(t);
// ...
Use the same order of registration when serializing. This makes the export macros superfluous.

How can I serialize and send a std::list over the network?

I need to send a dynamically sized list of data stored inside a std::list over a network connection. I would like to do this in one pass using serialization, rather than sending each element individually. Any suggestions?
boost::serialization makes this fairly easy to do. It provides all of the mechanics you need for std::list for free, all you will need to do is add support to the type your list holds. (If it's a "standard" type this will already exist too)
Complete example (adapted from this example):
#include <list>
#include <sstream>
#include <boost/archive/binary_oarchive.hpp>
#include <boost/archive/binary_iarchive.hpp>
// Provide an implementation of serialize for std::list
#include <boost/serialization/list.hpp>
class foo
{
private:
friend class boost::serialization::access;
template<class Archive>
void serialize(Archive & ar, const unsigned int /*version*/)
{
// This is the only thing you have to implement to serialize a std::list<foo>
ar & value;
// if we had more members here just & each of them with ar
}
public:
int value;
};
int main() {
std::stringstream out;
// setup a list
std::list<foo> list;
{
const foo f = {-1};
list.push_back(f);
}
// serialize into the stream
{
boost::archive::binary_oarchive oa(out);
oa << list;
}
// read the stream into a newlist
std::list<foo> newlist;
{
boost::archive::binary_iarchive ia(out);
ia >> newlist;
}
std::cout << newlist.front().value << std::endl;
}
This "sends" and "receives" via a std::stringstream, but it should be fairly trivial to adapt this to send and receive via the network API of your choice.
Use xerces-C++ to convert to/from XML and send/receive it.

Serializing a map of objects to xml using boost::serialization

The serialization example below is from the boost mailing list which is pretty much the same as what I would like to do. However, I have changed the archive so that it will serialize to XML. The compile does not fail if I serialize to binary, but it fails when serializing to xml. The compile fails in basic_xml_oarchive.hpp in the following method:
// boost code where compile fails
template<class T>
void save_override(T & t, BOOST_PFTO int)
{
// If your program fails to compile here, its most likely due to
// not specifying an nvp wrapper around the variable to
// be serialized.
BOOST_MPL_ASSERT((serialization::is_wrapper<T>));
this->detail_common_oarchive::save_override(t, 0);
}
It seems I haven't done enough to allow the std::map<int, CSomeData> object to be serialized, any ideas on how to fix this?
My serialization implementation:
#include <boost/archive/xml_oarchive.hpp>
#include <boost/archive/xml_iarchive.hpp>
#include <boost/serialization/map.hpp>
#include <fstream>
#include <string>
#include <map>
using namespace std;
// This is a test class to use as the map data.
class CSomeData {
public:
CSomeData(){};
CSomeData(float f0, string str0)
{
m_f0 = f0;
m_str0 = str0;
}
float m_f0;
string m_str0;
private:
friend class boost::serialization::access;
template<class Archive>
void serialize(Archive &ar, const unsigned int version)
{
ar & m_f0;
ar & m_str0;
}
};
// This is the class we really want to try serializing.
class CTest {
public:
CTest(){};
CTest(int nNumber)
{
m_nNumber = nNumber;
// Fill with some dummy data.
m_mTst.insert(make_pair(0, CSomeData(0.23f, "hi hi hi")));
m_mTst.insert(make_pair(1, CSomeData(7.65f, "second one")));
m_mTst.insert(make_pair(2, CSomeData(9.23f, "third one")));
m_mTst.insert(make_pair(3, CSomeData(5.6766, "chosen one")));
}
~CTest(){};
save()
{
std::ofstream ofs("filename");
// Write class instance to archive. Writing seems to work ok.
boost::archive::xml_oarchive oa(ofs);
oa << BOOST_SERIALIZATION_NVP(*this);
}
int m_nNumber;
private:
map<int, CSomeData> m_mTst;
friend class boost::serialization::access;
template<class Archive>
void serialize(Archive &ar, const unsigned int version)
{
ar & m_nNumber;
ar & m_mTst;
}
};
I believe you need to tag the members with a name for XML serialisation. This specifies the element name to use in the XML. I.e. use something like:
ar & BOOST_SERIALIZATION_NVP(m_f0);
or (better in this case):
ar & make_nvp("field0", my_f0);
The tags will be ignored for binary serialisation. More details here:
http://www.boost.org/doc/libs/1_43_0/libs/serialization/doc/wrappers.html