Non-intruisive Boost serialization of labelled enums C++ - c++

I would like to serialize struct A where I could save the tag name of the enum expressions instead of its integer in a non intruisive way (without having to change struct A).
enum e_fruit {
apple,
banana,
coconut
};
struct A {
e_fruit fruit;
int num;
};
namespace boost { namespace serialization {
template<class Archive>
void serialize(Archive & ar, A &a, const unsigned int version)
{
ar & boost::serialization::make_nvp("FruitType", a.fruit); // this will store an integer
ar & boost::serialization::make_nvp("Number", a.num);
}
}}
I have tried introducing a lookup table locally to the serialize function :
template<class Archive>
void serialize(Archive & ar, A &a, const unsigned int version)
{
static const char* const labels[] = { "Apple", "Banana", "Coconut" };
ar & boost::serialization::make_nvp("FruitType", labels[a.fruit]); // this won't work
ar & boost::serialization::make_nvp("Number", a.num);
}
Unfortunately I get the error :
Error 78 error C2228: left of '.serialize' must have
class/struct/union
since the prototype of make_nvp is
nvp< T > make_nvp(const char * name, T & t){
return nvp< T >(name, t);
}
so T should be a deduced template argument. I then thought about making a structure which would contains these labels but then I would have to add this in struct A which is what I want to avoid...
So how can we achieve this in the most non-intruisive way ?

I think you need to separate loading from saving. This compiles, I wonder it it is worth the game...
struct fruit_serializer
{
e_fruit &a_;
fruit_serializer(e_fruit &a) : a_(a) {}
template<class Archive>
void save(Archive & ar, const unsigned int version) const
{
std::string label = labels[static_cast<int>(a_)];
ar & boost::serialization::make_nvp("label", label);
}
template<class Archive>
void load(Archive & ar, const unsigned int version)
{
std::string label ;
ar & boost::serialization::make_nvp("label", label);
a_ = static_cast<e_fruit>(std::find(labels.begin(), labels.end(), label) - labels.begin());
}
BOOST_SERIALIZATION_SPLIT_MEMBER();
static std::vector<std::string> labels ;
};
std::vector<std::string> fruit_serializer::labels({ "Apple", "Banana", "Coconut" });
template<class Archive>
void serialize(Archive & ar, A &a, const unsigned int version)
{
fruit_serializer a1(a.fruit);
ar & boost::serialization::make_nvp("FruitType", a1);
}

As much as I hate to resurrect an old question, I wanted to do the same thing, but needed to make the enums string-serializable without any decorators. Since I didn't find much else regarding the topic, I'm posting my hacky solution as an option for anyone else who needs to serialize their enums as strings.
First off, some sample types to (eventually) serialize:
#include <iostream>
#include <sstream>
#include <boost/serialization/nvp.hpp>
#include <boost/archive/xml_oarchive.hpp>
// A few dummy enum types to test the solution
enum MyEnum00 {
Barb, Sarah, Lucy,
};
enum MyEnum01 {
Corey, Trevor, Jacob = 99,
};
const char* const to_cstring(const MyEnum01 e) {
switch (e) {
case Corey: return "Corey";
case Trevor: return "Trevor";
case Jacob: return "Jacob";
default: return "UNKNOWN";
}
}
inline std::ostream& operator<<(std::ostream& o, const MyEnum01 e) { return o << to_cstring(e); }
enum class MyEnumClass00 {
Ricky, Julian, Bubbles
};
enum class MyEnumClass01 {
Jim, Randy, Cyrus
};
const char* const to_cstring(const MyEnumClass01 e) {
switch (e) {
case MyEnumClass01::Jim: return "I let the liquor do the thinking, bud!";
case MyEnumClass01::Randy: return "Got any cheeeeeseburgers?";
case MyEnumClass01::Cyrus: return "I got work to do";
default: return "UNKNOWN";
}
}
inline std::ostream& operator<<(std::ostream& o, const MyEnumClass01 e) { return o << to_cstring(e); }
From boost_1_63_0/boost/archive/detail/oserializer.hpp, the function save_enum_type::invoke() is where enums get clobbered into ints.
Boost uses a somewhat clunky combination of templated structs with separately templated members, so it's difficult to make our change apply any time our desired enum types are used. As a workaround, we can specialize boost::archive::detail::save_enum_type for the archive type we're using. Then, we can overload its invoke() function to keep it from clobbering the enum types we need to have archived as strings.
save_enum_type::invoke is called roughly in the middle of boost's callstack, which eventually makes its way down into the basic_text_oprimitive class. There, an insertion operator is finally used to save the value to the ostream underlying the destination archive. We can take advantage of that implementation detail to get our enum types archived as strings by specializing the save_enum_type and impelmenting insertion operators for the target types.
namespace boost {
namespace archive {
namespace detail {
using xml_oarchive_ = boost::archive::xml_oarchive;
using save_non_pointer_type_ = detail::save_non_pointer_type<xml_oarchive_>;
template<>
struct save_enum_type<xml_oarchive_>
{
// This is boost's stock function that converts enums to ints before serializing them.
// We've added a copy to our specialized version of save_enum_type to maintain the exisitng behavior for any
// enum types we don't care to have archived in string form
template<class T>
static void invoke(xml_oarchive_& ar, const T& t) {
const int i = static_cast<int>(t);
ar << boost::serialization::make_nvp(NULL, i);
}
///// specialized enum types /////
// You could probably reduce all the repeated code with some type-trait magic...
static void invoke(xml_oarchive_& ar, const MyEnum00 &e) {
save_non_pointer_type_::invoke(ar, e);
}
static void invoke(xml_oarchive_& ar, const MyEnum01 &e) {
save_non_pointer_type_::invoke(ar, e);
}
// Won't work -- MyEnumClass00 doesn't have an insertion operator, so the underlying ostream won't know
// how to handle it
//static void invoke(xml_oarchive_& ar, const MyEnumClass00 &e) {
// save_non_pointer_type_::invoke(ar, e);
//}
static void invoke(xml_oarchive_& ar, const MyEnumClass01 &e) {
save_non_pointer_type_::invoke(ar, e);
}
};
} // namespace detail
} // namespace archive
} // namespace boost
And Finally some code to test everything:
int main()
{
std::stringstream outstream;
boost::archive::xml_oarchive ar(outstream);
MyEnum00 e00_0 = Barb;
MyEnum00 e00_1 = Sarah;
MyEnum00 e00_2 = Lucy;
MyEnum01 e01_0 = Corey;
MyEnum01 e01_1 = Trevor;
MyEnum01 e01_2 = Jacob;
MyEnumClass00 ec00_0 = MyEnumClass00::Ricky;
MyEnumClass00 ec00_1 = MyEnumClass00::Julian;
MyEnumClass00 ec00_2 = MyEnumClass00::Bubbles;
MyEnumClass01 ec01_0 = MyEnumClass01::Jim;
MyEnumClass01 ec01_1 = MyEnumClass01::Randy;
MyEnumClass01 ec01_2 = MyEnumClass01::Cyrus;
ar
// regular enums get passed down as int even if you don't explicitly convert them
<< BOOST_SERIALIZATION_NVP(e00_0)
<< BOOST_SERIALIZATION_NVP(e00_1)
<< BOOST_SERIALIZATION_NVP(e00_2)
// regular enums can also get special treatment
<< BOOST_SERIALIZATION_NVP(e01_0)
<< BOOST_SERIALIZATION_NVP(e01_1)
<< BOOST_SERIALIZATION_NVP(e01_2)
// enum classes that aren't specialized pass down as ints
<< BOOST_SERIALIZATION_NVP(ec00_0)
<< BOOST_SERIALIZATION_NVP(ec00_1)
<< BOOST_SERIALIZATION_NVP(ec00_2)
// enum classes can also get special treatment
<< BOOST_SERIALIZATION_NVP(ec01_0)
<< BOOST_SERIALIZATION_NVP(ec01_1)
<< BOOST_SERIALIZATION_NVP(ec01_2)
;
std::cout << outstream.str() << std::endl;
return 0;
}

Related

Boost serialization of a std::vector of pointers. Error: "unregistered class - derived class not registered or exported"

I am using boost to (de)serialize some classes. All is well except for a class which has astd::vector<Artefact*> member (see below for definitions). I have problems with vectors of pointers.
I can (de)serialize single instances of class Artefact perfectly.
I cannot serialize a class which has a vector of pointers to class Artefact.
The boost error when I try to do so is:
"unregistered class - derived class not registered or exported"
I am getting some output. Archive file:
22 serialization::archive 19 1 0
0 1 0
1 1 0
2 0 0 66 0
Class Definitions
#include <boost/serialization/vector.hpp>
#include <boost/serialization/map.hpp>
...
class Tag {
public:
bool tag = false;
};
class Artefact : public Tag {
public:
std::vector<enjyn_Vertex> vertices;
std::vector<unsigned int> indices;
std::vector<enjyn_Texture*> textures;
glm::vec3 position{ 0.0f };
glm::vec3 scale{ 1.0f };
glm::vec3 rotation{ 0.0f, 1.0f, 0.0f };
float rotation_degrees{ 0.0f };
glm::vec4 colour{ 1.0f, 1.0f, 1.0f, 1.0f };
...
std::vector<Artefact> artefacts; // note, no problem serializing std::vector<Artefact> here
...
};
//Mainly static data and function members
class State {
public:
...
static std::map<std::string, bool> state;
...
}
class Event {
public:
...
virtual void OnMouseFocus(); //Lots of virtual data members like this one here
..
}
//Large class with lots of data and function members.
class Enjyn : public Event, public State {
public:
...
std::vector<Artefact*> artefacts; // I believe this is the problem member
...
}
Free Non-Intrusive (De)Serialization Functions
I have non-intrusive free serialize functions for all classes I want to archive including
serialize functions for all base classes and non built in data members).
The serialize function below (de)serializes perfectly for single instances of Artefact:
#include <boost/serialization/export.hpp>
#include <boost/archive/text_oarchive.hpp>
#include <boost/archive/text_iarchive.hpp>
...
void serialize(Archive& ar, Artefact& a, const unsigned int version) {
//ar.template register_type<Artefact>(); ; commented as classes exported see below
ar& boost::serialization::base_object<enjyn_Tag>(a);
ar& a.vertices;
ar& a.indices;
ar& a.textures;
ar& a.position;
ar& a.scale;
ar& a.rotation;
ar& a.rotation_degrees;
ar& a.colour;
ar& a.aabb;
ar& a.bsphere;
ar& a.tex_offset_x;
ar& a.tex_offset_y;
ar& a.tex_scale_x;
ar& a.tex_scale_y;
ar& a.name;
ar& a.artefacts; //works fine for vector of non-pointer vector of Artefact.
}
However when I try to archive an object with a std::vector<Artefact*>, I get:
"unregistered class - derived class not registered or exported"
void serialize(Archive& ar, Enjyn& e, const unsigned int version)
{
//ar.template register_type<enjyn>(); commented as classes exported see below:
ar& boost::serialization::base_object<Event>(e); // not sure whether this is necessary, no tracking requried of base classes.
ar& boost::serialization::base_object<State>(e);
ar.template register_type<Artefact>();
...
ar& e.artefacts; //This is a std::vector<Artefact*> which I cant archive
//if i remove this and add include any 'normal' non-pointer
//built in types the serialization appears to work perfectly.
...
}
Serializing Program Functions
I am serializing single instances of Artefact objects perfectly like this:
if (e.primary_artefact) { //primary_artefact is a pointer to an instance of Artefact
try {
std::string filepath = "test_filename";
std::ofstream o(filepath);
{
boost::archive::text_oarchive oa(o);
oa << *e.primary_artefact;
}
}
catch (std::ifstream::failure e) {
std::cerr << "ERROR:IFSTREAM: " << e.what() << std::endl;
}
catch (boost::archive::archive_exception e) {
std::cerr << "ERROR:BOOST SERIALISATION: " << e.what() << std::endl;
}
}
Attempting to serialize class Enjyn using this function creates error if I include Enjyn.artefacts (std::vector<Artefact*>) in the serialize function:
void serialize_enjyn(enjyn& e, std::string& name)
try {
std::string filepath = name;
std::ofstream o(filepath);
{
boost::archive::text_oarchive oa(o);
oa << e;
}
}
catch (std::ifstream::failure e) {
std::cerr << "ERROR:IFSTREAM: " << e.what() << std::endl;
}
catch (boost::archive::archive_exception e) {
std::cerr << "ERROR:BOOST SERIALISATION: " << e.what() << std::endl;
}
}
I went mad and exported GUIDS for all types in my main program file so all relevant types could be tracked within the archive. Not that this is particularly necessary. Only tracking of '''class Artefact''' is required.
#include <boost/serialization/export.hpp>
BOOST_CLASS_EXPORT_GUID(glm::vec2, "vec2")
BOOST_CLASS_EXPORT_GUID(glm::vec3, "vec3")
BOOST_CLASS_EXPORT_GUID(glm::vec4, "vec4")
BOOST_CLASS_EXPORT_GUID(Artefact, "enjyn_Artefact")
BOOST_CLASS_EXPORT_GUID(Vertex, "enjyn_Vertex")
BOOST_CLASS_EXPORT_GUID(Texture, "enjyn_Texture")
BOOST_CLASS_EXPORT_GUID(Tag, "enjyn_Tag")
BOOST_CLASS_EXPORT_GUID(State, "enjyn_State")
BOOST_CLASS_EXPORT_GUID(Event, "enjyn_event")
BOOST_CLASS_EXPORT_GUID(Enjyn, "enjyn")
int main(int argc, char* argv[]){
Discussion
Note, the Artefact object is used as a base for other classes. I have not registered those classes or written serialization functions for them as Artefact has no virtual functions, I am trying to serialize Artefact directly (though indirectly also via pointers), and the documentation for registration states:
It turns out that the kind of object serialized depends upon whether the base class (base in this case) is polymophic or not. If base is not polymorphic, that is if it has no virtual functions, then an object of the type base will be serialized. Information in any derived classes will be lost. If this is what is desired (it usually isn't) then no other effort is required.
The problem appears to be with the member std::vector<Artefact*> artefact in Enjyn. When I remove this from the serialize function for Enjyn, Enjyn serializes perfectly. I am obviously missing something fundemental to do with:
pointers, std::vector, and derived types and base classes from the boost documentation.
Please help!
If you actually took the time to make that a correct, self-contained example, you may find that there is no problem:
Live On Coliru
#include <boost/serialization/map.hpp>
#include <boost/serialization/vector.hpp>
#include <boost/serialization/array.hpp>
#include <iostream>
// mocking stuff
namespace glm {
struct vec3 {
std::array<float, 3> a = {0,0,0};
bool operator==(vec3 const& rhs) const = default;
};
struct vec4 {
std::array<float, 4> a = {0,0,0,0};
bool operator==(vec4 const& rhs) const =default;
};
} // namespace glm
struct enjyn_Vertex {
bool operator==(enjyn_Vertex const&) const = default;
};
struct enjyn_Texture {
bool operator==(enjyn_Texture const&) const = default;
};
struct enjyn_Tag {
bool tag = false;
bool operator==(enjyn_Tag const&) const = default;
};
class Artefact : public enjyn_Tag {
public:
std::vector<enjyn_Vertex> vertices;
std::vector<unsigned int> indices;
std::vector<enjyn_Texture*> textures; // not owned, so leaked?
glm::vec3 position{0.0f};
glm::vec3 scale{1.0f};
glm::vec3 rotation{0.0f, 1.0f, 0.0f};
float rotation_degrees{0.0f};
glm::vec4 colour{1.0f, 1.0f, 1.0f, 1.0f};
std::vector<Artefact> artefacts;
bool operator==(Artefact const& rhs) const {
if (!enjyn_Tag::operator==(rhs))
return false;
auto tie = [](Artefact const& a) {
return std::tie(a.vertices, a.indices, /*a.textures,*/ a.position,
a.scale, a.rotation, a.rotation_degrees, a.colour,
a.artefacts);
};
if (tie(*this) != tie(rhs))
return false;
// compare textures indirectly
if (textures.size() != rhs.textures.size())
return false;
for (size_t i = 0; i<textures.size(); ++i)
if (*textures[i] != *rhs.textures[i])
return false;
return true;
}
private:
};
//Mainly static data and function members
class State {
public:
static std::map<std::string, bool> state;
bool operator==(State const&) const = default;
};
/*static*/ std::map<std::string, bool> State::state;
class Event {
public:
virtual void OnMouseFocus() {
} // Lots of virtual data members like this one here
bool operator==(Event const&) const = default;
};
//Large class with lots of data and function members.
class Enjyn
: public Event
, public State {
public:
// Rule Of Three...
~Enjyn() { for (auto a : artefacts) delete a; }
Enjyn() = default;
Enjyn(Enjyn const&) = delete;
Enjyn(Enjyn&&) = default;
Enjyn& operator=(Enjyn const&) = delete;
std::vector<Artefact*> artefacts;
bool operator==(Enjyn const& rhs) const {
if (! Event::operator==(rhs)) return false;
if (! State::operator==(rhs)) return false;
// compare artefacts indirectly
if (artefacts.size() != rhs.artefacts.size())
return false;
for (size_t i = 0; i<artefacts.size(); ++i)
if (*artefacts[i] != *rhs.artefacts[i])
return false;
return true;
}
};
#include <boost/serialization/export.hpp>
#include <boost/archive/text_oarchive.hpp>
#include <boost/archive/text_iarchive.hpp>
namespace boost::serialization {
template <typename Archive> void serialize(Archive& ar, enjyn_Vertex&, unsigned) { }
template <typename Archive> void serialize(Archive& ar, enjyn_Texture&, unsigned) { }
template <typename Archive> void serialize(Archive& ar, enjyn_Tag& t, unsigned) { ar& t.tag; }
template <typename Archive> void serialize(Archive& ar, glm::vec3&v, unsigned) { ar& v.a; }
template <typename Archive> void serialize(Archive& ar, glm::vec4&v, unsigned) { ar& v.a; }
template <typename Archive> void serialize(Archive& ar, Artefact& a, unsigned)
{
//ar.template register_type<Artefact>(); ; commented as classes exported see below
ar& boost::serialization::base_object<enjyn_Tag>(a);
ar& a.vertices;
ar& a.indices;
ar& a.textures;
ar& a.position;
ar& a.scale;
ar& a.rotation;
ar& a.rotation_degrees;
ar& a.colour;
//ar& a.aabb;
//ar& a.bsphere;
//ar& a.tex_offset_x;
//ar& a.tex_offset_y;
//ar& a.tex_scale_x;
//ar& a.tex_scale_y;
//ar& a.name;
ar& a.artefacts; // works fine for vector of non-pointer vector of Artefact.
}
template <typename Archive> void serialize(Archive& ar, Event& e, unsigned)
{
}
template <typename Archive> void serialize(Archive& ar, Enjyn& e, unsigned)
{
ar& e.state; // warning: serializing statics may not do what you want
ar& e.artefacts;
}
} // namespace boost
Enjyn bake_me_one_with_everything();
int main() {
Enjyn const original = bake_me_one_with_everything();
std::stringstream ss;
{
boost::archive::text_oarchive oa(ss);
oa << original;
}
std::cout << ss.str() << "\n";
{
boost::archive::text_iarchive ia(ss);
Enjyn e;
ia >> e;
std::cout << "Roundtrip equal: " << std::boolalpha << (e == original)
<< std::endl;
}
}
#include <random>
Enjyn bake_me_one_with_everything()
{
Enjyn e;
std::mt19937 urbg{std::random_device{}()};
auto frand =
std::bind(std::uniform_real_distribution<float>{}, std::ref(urbg));
auto r10 =
std::bind(std::uniform_int_distribution<>{1, 10}, std::ref(urbg));
for (int i = 0; i < r10(); ++i) { // very sloppy loop
auto& a = *e.artefacts.emplace_back(new Artefact());
a.artefacts.resize(r10());
a.colour = {frand(), frand(), frand()};
a.position = {{frand(), frand(), frand()}};
a.rotation = {{frand(), frand(), frand()}};
a.rotation_degrees = frand() * 360;
a.scale = {{frand(), frand(), frand()}};
a.tag = r10() % 2;
std::generate_n(back_inserter(a.indices), r10(), r10);
a.vertices.assign(r10(), enjyn_Vertex{});
a.textures.assign(r10(), new enjyn_Texture{});
}
return e;
}
Prints things like
Your princess is in another castle.
Imaging The Flaws
I notice a lot of "gratuitous" use of inheritance (e.g. to inherit static data members), but specifically you inherit Enjyn from Event, which is a polymorphic type.
So my best guess is that this contains your actual issue.
Otherwise, you should be able to go from this live sample.
Caveats
Your code presentation shows some ordering that indicates you may be separating code across source files. In that case, don't fall into the trap of registering your types in a file that doesn't include all the relevant archive types BEFORE the export defitions: docs
BOOST_CLASS_EXPORT in the same source module that includes any of the archive class headers will instantiate code required to serialize polymorphic pointers of the indicated type to the all those archive classes. If no archive class headers are included, then no code will be instantiated.

Is there a way to serialize iterators using boost serialization?

In my project I have a class containing and std::list and in another class I maintain an iterator pointing to a place in the middle of that list.
I can successfully serialize the list, but the iterator member variable is causing problems. Here is a program to reproduce:
#include <boost/serialization/list.hpp>
#include <boost/archive/text_oarchive.hpp>
#include <boost/archive/text_iarchive.hpp>
#include <iostream>
#include <fstream>
class A
{
public:
A(){}
A(const std::list<int> & _i) : i(_i) {}
virtual ~A(){}
std::list<int> i;
void display() {
std::cout << "i:";
for (const int j : i)
std::cout << " " << j;
std::cout << std::endl;
}
private:
friend class boost::serialization::access;
//friend std::ostream & operator<<(std::ostream &os, const A &a);
template<class Archive>
void serialize(Archive &ar, const unsigned int version)
{
ar & i;
}
};
class Stepper
{
public:
Stepper() {}
Stepper(const A& a)
: p(a.i.size()>0 ? a.i.begin() : a.i.end()) {}
std::list<int>::const_iterator p;
void display() {
std::cout << "p: " << *p << std::endl;
}
void step() { p++; }
private:
friend class boost::serialization::access;
//friend std::ostream & operator<<(std::ostream &os, const A &a);
template<class Archive>
void serialize(Archive &ar, const unsigned int version)
{
ar & p;
}
};
int main()
{
{
A a({5,6,7});
Stepper sa(a);
a.display();
sa.display();
sa.step();
sa.display();
std::ofstream ofs( "a.txt" );
boost::archive::text_oarchive ar(ofs);
ar & a;
ar & sa;
}
A b;
Stepper sb;
{
std::ifstream ifs( "a.txt" );
boost::archive::text_iarchive ar(ifs);
ar & b;
ar & sb;
}
b.display();
sb.display();
return 0;
}
In this program, the class A can be serialized without problems. (Remove the ar&sa stuff..) But unfortunately when trying to serialize the class containing the iterator (the exact code above), I get the following compilation errors:
[..snip..]
testser.cpp:72:10: required from here /usr/include/boost/serialization/access.hpp:116:11:
error: ‘struct std::_List_const_iterator<int>’ has no member named ‘serialize’
t.serialize(ar, file_version);
~~^~~~~~~~~
[..snip..]
testser.cpp:81:10: required from here /usr/include/boost/serialization/access.hpp:116:11:
error: ‘struct std::_List_const_iterator<int>’ has no member named ‘serialize’
So, it seems that boost/serialization/list.hpp does not support iterators. And yet, as far as I can tell, it's totally legitimate to keep an iterator to a list item somewhere, as they cannot be invalidated unless erased. Is there a way to serialize this iterator using boost? Do I need to write a custom function? Do I have to return a custom iterator from my std::list? (That sounds particularly ugly..)
Thanks for any insight.
Okay it seems the only way to do this is to split the serialization into save and load, and calculate the iterator's position in the list. This works as long as the iterator is valid. Unfortunately it means needing to add a pointer to the list to the structure, which I didn't want, but actually in my application I can access this so it is not a problem for me.
class Stepper
{
public:
Stepper() {}
Stepper(const A& _a)
: a(&_a), p(a->i.size()>0 ? a->i.begin() : a->i.end()) {}
const A* a;
std::list<int>::const_iterator p;
void display() {
std::cout << "p: " << *p << std::endl;
}
void step() { p++; }
private:
friend class boost::serialization::access;
template<class Archive>
void save(Archive &ar, const unsigned int version) const
{
int d = std::distance(a->i.begin(), p);
ar & a;
ar & d;
}
template<class Archive>
void load(Archive &ar, const unsigned int version)
{
int d;
ar & a;
ar & d;
p = a->i.begin();
for (; d>0; --d)
p++;
}
BOOST_SERIALIZATION_SPLIT_MEMBER()
};

Boost serialization : read varying type of data

I have a C++ / CLI project that uses boost serialization to serialize three different classes. I would like to know if it is possible to parse the first line of the boost serialization archive in order to know what class was serialized in this archive, and then create an object of the appropriate class and deserialize the archive into the object. That line would contain an ID (maybe a int or value of an enum class) to identify which class was serialized.
The file format is already handled by your choice of Archive implementation.
In practice that would be boost::archive::text_oarchive, boost::archive::binary_oarchive, boost::archive::xml_oarchive.
As long as your archive type itself doesn't vary, you can very easily use a Boost Variant to distinguish your payloads. In other words, make the serialization framework do the work for you, instead of "duct taping" around it:
Here's a demo that serializes 3 different (compound) payloads and roundtrips just fine without external knowledge of the payload actually there:
Live On Coliru
#include <boost/archive/text_oarchive.hpp>
#include <boost/archive/text_iarchive.hpp>
#include <boost/serialization/variant.hpp>
#include <boost/serialization/vector.hpp>
#include <boost/serialization/string.hpp>
#include <boost/serialization/access.hpp>
struct A {
int simple;
private:
friend class boost::serialization::access;
template <typename Ar> void serialize(Ar& ar, unsigned) {
ar & simple;
}
};
struct B {
std::string text;
private:
friend class boost::serialization::access;
template <typename Ar> void serialize(Ar& ar, unsigned) {
ar & text;
}
};
struct C {
A composed_a;
B composed_b;
private:
friend class boost::serialization::access;
template <typename Ar> void serialize(Ar& ar, unsigned) {
ar & composed_a & composed_b;
}
};
struct FileContents { // conventions...
boost::variant<A, B, C> payload;
private:
friend class boost::serialization::access;
template <typename Ar> void serialize(Ar& ar, unsigned) {
ar & payload;
}
};
#include <sstream>
#include <boost/lexical_cast.hpp>
//////////////////////////////////////////////////////////////////////////////////////////////////////////////
// For our roundtrip test, implement streaming as well so we can independently check equivalence
inline static std::ostream& operator<<(std::ostream& os, A const& v) {
return os << "A{" << v.simple << "}";
}
inline static std::ostream& operator<<(std::ostream& os, B const& v) {
return os << "B{" << v.text << "}";
}
inline static std::ostream& operator<<(std::ostream& os, C const& v) {
return os << "C{" << v.composed_a << ", " << v.composed_b << "}";
}
void roundtrip_test(FileContents const& original) {
std::stringstream ss;
{
boost::archive::text_oarchive oa(ss);
oa << original;
}
{
boost::archive::text_iarchive ia(ss);
FileContents clone;
ia >> clone;
std::string const before = boost::lexical_cast<std::string>(original.payload);
std::string const after = boost::lexical_cast<std::string>(clone.payload);
std::cout << "Roundtrip '" << before << "': " << std::boolalpha << (before == after) << "\n";
}
}
int main() {
roundtrip_test({ A { 42 } });
roundtrip_test({ B { "Life The Universe And Everything" } });
roundtrip_test({ C { {42}, { "Life The Universe And Everything" } } });
}
The output being:
Roundtrip 'A{42}': true
Roundtrip 'B{Life The Universe And Everything}': true
Roundtrip 'C{A{42}, B{Life The Universe And Everything}}': true

binary reading and writing complicated stucts in c++

I need the ability to save/read my data structures in my project, but all the data is in the form of quite complicated and distinct structures themselves which I typically implement through other structs and vectors. I have wrapped all of them up into a single struct so that I have something like
struct master{
std::vector<apprentice_type1> a;
std::vector<apprentice_type2> b; //etc.
std::string label;
};
with other ones defined like
struct apprentice_type1{
vec location;
int point_label;
std::vector<int> relational_data;
};
struct vec{
double x,y,z;
};
So it gets pretty complicated! I was desperately hoping something nice, quick and naive like
master obj;
//write to obj....
std::ofstream ofs("data.dat", std::ios::binary);
ofs.write((char *)&obj, sizeof(obj));
would work, but at present it doesn't seem to. Before I get lost in the debugging rabbit hole is this actually possible the way I'm approaching it or do I need to rethink? If so, how?
Thanks.
If you want an alternative to Boost serialization and have access to a C++11 compiler, you can also check out cereal. It works in a near identical fashion to Boost serialize but is a header only library so there is nothing to link against.
[...] or do I need to rethink? If so, how?
You will probably need to provide a full implementation (i.e. explore the "rabbit-hole").
This is a known problem (stream serialization) and there is no single best-approach solution to it, because most implementations need to solve different needs.
You can do one of the following:
implement std::i/ostream serialization; This means you will go over your classes and implement operator>>(std::istream*, your_type&) and it's inverse operator<<(std::ostream*, your_type&).
implement serialization based on a stream library (like boost.archive).
use a JSON or XML library.
use google protocol buffers (or something else)
roll your own implementation, depending on your needs.
If you go for boost::serialization, here is a little sample:
#include <fstream>
#include <vector>
#include <boost/archive/binary_iarchive.hpp>
#include <boost/archive/binary_oarchive.hpp>
#include <boost/serialization/vector.hpp>
template <typename T>
inline const boost::serialization::nvp<T> archive_value(const char* name, T& value) {
return boost::serialization::make_nvp(name, value);
}
const unsigned Version = 0;
class Point{
public:
double x,y,z;
private:
template <typename P, typename Archive>
static void serialize(P& p, Archive& ar, const unsigned int version) {
std::cout << "Point\n";
ar & archive_value("X", p.x);
ar & archive_value("Y", p.y);
ar & archive_value("Z", p.z);
}
public:
template <typename Archive>
void serialize(Archive& ar, const unsigned int version) const {
serialize(*this, ar, version);
}
template <typename Archive>
void serialize(Archive& ar, const unsigned int version) {
serialize(*this, ar, version);
}
};
BOOST_CLASS_VERSION(Point, Version)
struct Scene{
std::vector<Point> points;
private:
template <typename S, typename Archive>
static void serialize(S& s, Archive& ar, const unsigned int version) {
std::cout << "Scene\n";
ar & archive_value("Points", s.points);
}
public:
template <typename Archive>
void serialize(Archive& ar, const unsigned int version) const {
serialize(*this, ar, version);
}
template <typename Archive>
void serialize(Archive& ar, const unsigned int version) {
serialize(*this, ar, version);
}
};
BOOST_CLASS_VERSION(Scene, Version)
template <typename Archive>
void register_types(Archive& ar) {
ar.template register_type<Point>();
ar.template register_type<Scene>();
}
int main() {
Scene scene;
scene.points = { { 0, 0, 0 }, { 1, 1, 1 }, { 2, 2, 2 } };
// Output
{
std::ofstream out("test.dat", std::ios_base::binary);
boost::archive::binary_oarchive output(out);
// First the version!
output & archive_value("Version", Version);
// Next the types!
register_types(output);
// Finally the data
output & archive_value("Scene", scene);
}
scene.points = {};
// Input
{
int version;
std::ifstream in("test.dat", std::ios_base::binary);
boost::archive::binary_iarchive input(in);
// First the version!
input & archive_value("Version", Version);
// Next the types!
register_types(input);
// Finally the data
input & archive_value("Scene", scene);
}
for(const auto& p : scene.points)
std::cout << p.x << '\n';
}
Note: The file format may evolve and serialization functions (input and/or output) may get adjustment depending on the file version.

Boost.Serialization: Error on Calling Class Method during serialization process

I have the following problem when I try to call method in "TestSerialize" class during serialization process.
Here is my code:
class TestSerialize
{
public:
std::string GetVal() { return Val + "abc"; }
void SetVal(std::string tVal) { Val = tVal.substr(0, 2); }
protected:
std::string Val;
friend class boost::serialization::access;
template<class Archive> void save(Archive & ar, const unsigned int version) const
{
using boost::serialization::make_nvp;
std::string tVal = GetVal(); // Error here
ar & make_nvp("SC", tVal);
}
template<class Archive> void load(Archive & ar, const unsigned int version)
{
using boost::serialization::make_nvp;
std::string tVal;
ar & make_nvp("SC", tVal);
SetVal(tVal);
}
BOOST_SERIALIZATION_SPLIT_MEMBER();
};
int main()
{
TestSerialize tS;
std::ofstream ofs("test.xml");
boost::archive::xml_oarchive oa(ofs, boost::archive::no_header);
oa << BOOST_SERIALIZATION_NVP(tS);
ofs.close();
return 0;
}
The error that I encountered is:
'TestSerialize::GetVal' : cannot convert 'this' pointer from 'const TestSerialize' to 'TestSerialize &'
This error only happens on "save" but not "load"
I wonder why I get this error. I would like to know what Boost.Serialization do such that we have these two different behaviors.
I use Boost Library 1.47.0
save is a const function and can only call other const functions. GetVal isn't. Change it:
std::string GetVal() const { ... }