binary reading and writing complicated stucts in c++ - c++

I need the ability to save/read my data structures in my project, but all the data is in the form of quite complicated and distinct structures themselves which I typically implement through other structs and vectors. I have wrapped all of them up into a single struct so that I have something like
struct master{
std::vector<apprentice_type1> a;
std::vector<apprentice_type2> b; //etc.
std::string label;
};
with other ones defined like
struct apprentice_type1{
vec location;
int point_label;
std::vector<int> relational_data;
};
struct vec{
double x,y,z;
};
So it gets pretty complicated! I was desperately hoping something nice, quick and naive like
master obj;
//write to obj....
std::ofstream ofs("data.dat", std::ios::binary);
ofs.write((char *)&obj, sizeof(obj));
would work, but at present it doesn't seem to. Before I get lost in the debugging rabbit hole is this actually possible the way I'm approaching it or do I need to rethink? If so, how?
Thanks.

If you want an alternative to Boost serialization and have access to a C++11 compiler, you can also check out cereal. It works in a near identical fashion to Boost serialize but is a header only library so there is nothing to link against.

[...] or do I need to rethink? If so, how?
You will probably need to provide a full implementation (i.e. explore the "rabbit-hole").
This is a known problem (stream serialization) and there is no single best-approach solution to it, because most implementations need to solve different needs.
You can do one of the following:
implement std::i/ostream serialization; This means you will go over your classes and implement operator>>(std::istream*, your_type&) and it's inverse operator<<(std::ostream*, your_type&).
implement serialization based on a stream library (like boost.archive).
use a JSON or XML library.
use google protocol buffers (or something else)
roll your own implementation, depending on your needs.

If you go for boost::serialization, here is a little sample:
#include <fstream>
#include <vector>
#include <boost/archive/binary_iarchive.hpp>
#include <boost/archive/binary_oarchive.hpp>
#include <boost/serialization/vector.hpp>
template <typename T>
inline const boost::serialization::nvp<T> archive_value(const char* name, T& value) {
return boost::serialization::make_nvp(name, value);
}
const unsigned Version = 0;
class Point{
public:
double x,y,z;
private:
template <typename P, typename Archive>
static void serialize(P& p, Archive& ar, const unsigned int version) {
std::cout << "Point\n";
ar & archive_value("X", p.x);
ar & archive_value("Y", p.y);
ar & archive_value("Z", p.z);
}
public:
template <typename Archive>
void serialize(Archive& ar, const unsigned int version) const {
serialize(*this, ar, version);
}
template <typename Archive>
void serialize(Archive& ar, const unsigned int version) {
serialize(*this, ar, version);
}
};
BOOST_CLASS_VERSION(Point, Version)
struct Scene{
std::vector<Point> points;
private:
template <typename S, typename Archive>
static void serialize(S& s, Archive& ar, const unsigned int version) {
std::cout << "Scene\n";
ar & archive_value("Points", s.points);
}
public:
template <typename Archive>
void serialize(Archive& ar, const unsigned int version) const {
serialize(*this, ar, version);
}
template <typename Archive>
void serialize(Archive& ar, const unsigned int version) {
serialize(*this, ar, version);
}
};
BOOST_CLASS_VERSION(Scene, Version)
template <typename Archive>
void register_types(Archive& ar) {
ar.template register_type<Point>();
ar.template register_type<Scene>();
}
int main() {
Scene scene;
scene.points = { { 0, 0, 0 }, { 1, 1, 1 }, { 2, 2, 2 } };
// Output
{
std::ofstream out("test.dat", std::ios_base::binary);
boost::archive::binary_oarchive output(out);
// First the version!
output & archive_value("Version", Version);
// Next the types!
register_types(output);
// Finally the data
output & archive_value("Scene", scene);
}
scene.points = {};
// Input
{
int version;
std::ifstream in("test.dat", std::ios_base::binary);
boost::archive::binary_iarchive input(in);
// First the version!
input & archive_value("Version", Version);
// Next the types!
register_types(input);
// Finally the data
input & archive_value("Scene", scene);
}
for(const auto& p : scene.points)
std::cout << p.x << '\n';
}
Note: The file format may evolve and serialization functions (input and/or output) may get adjustment depending on the file version.

Related

How to use boost::serialization with nested structs and minimal code changes?

Currently we use POD which is stored in nested structs. Example:
#define MaxNum1 100;
#define MaxNum2 50;
struct A
{
int Value[MaxNum1];
char SomeChar = 'a';
};
struct B
{
A data[MaxNum2];
float SomeFloat = 0.1f;
};
int main()
{
B StructBObject = {};
}
We want to enhance our data structures using std::vector just like this:
struct NewA
{
std::vector<int> Value;
char SomeChar = 'a';
};
struct NewB
{
std::vector<NewA> data;
float SomeFloat = 0.1f;
};
int main()
{
NewB StructNewBObject = {};
}
The only argument against this modification is that NewA and NewB are no POD anymore and this makes reading/writing to a file more complicated.
How is it possible to read/write NewA and NewB to a file using boost::serialization with minimal
code changes to NewA and NewB? Minimal code changes are important because we use for example big structs which have up to 7 nested levels.
You can serialize using boost serialization¹:
template <typename Ar> void serialize(Ar& ar, A& a, unsigned) {
ar & a.Value & a.SomeChar;
}
template <typename Ar> void serialize(Ar& ar, B& b, unsigned) {
ar & b.data & b.SomeFloat;
}
Using these, you will already have the correct behaviour out of the box with both the C-array and std::vector approaches.
If you want to keep using fixed-size trivially-copyable types², you can use something like Boost Container's static_vector: it will keep track of the current size, but the data is statically allocated right inside the structures.
TRIPLE DEMO
Here's a triple demo program with three implementations depending on the IMPL variable.
As you can see the bulk of the code is kept invariant. However, for "best comparison" I've made sure that all the containers are at half capacity (50/25) before serialization.
The main program also deserializes.
Live On Coliru
#include <boost/iostreams/device/back_inserter.hpp>
#include <boost/iostreams/device/array.hpp>
#include <boost/iostreams/stream.hpp>
#include <boost/archive/binary_oarchive.hpp>
#include <boost/archive/binary_iarchive.hpp>
#include <boost/serialization/access.hpp>
#include <boost/serialization/is_bitwise_serializable.hpp>
#include <boost/serialization/binary_object.hpp>
#include <iostream>
#if (IMPL==0) // C arrays
struct A {
int Value[100];
char SomeChar = 'a';
};
struct B {
A data[50];
float SomeFloat = 0.1f;
};
template <typename Ar> void serialize(Ar& ar, A& a, unsigned) {
ar & a.Value & a.SomeChar;
}
template <typename Ar> void serialize(Ar& ar, B& b, unsigned) {
ar & b.data & b.SomeFloat;
}
#elif (IMPL==1) // std::vector
#include <boost/serialization/vector.hpp>
struct A {
std::vector<int> Value;
char SomeChar = 'a';
};
struct B {
std::vector<A> data;
float SomeFloat = 0.1f;
};
template <typename Ar> void serialize(Ar& ar, A& a, unsigned) {
ar & a.Value & a.SomeChar;
}
template <typename Ar> void serialize(Ar& ar, B& b, unsigned) {
ar & b.data & b.SomeFloat;
}
#elif (IMPL==2) // static_vector
#include <boost/serialization/vector.hpp>
#include <boost/container/static_vector.hpp>
struct A {
boost::container::static_vector<int, 100> Value;
char SomeChar = 'a';
};
struct B {
boost::container::static_vector<A, 50> data;
float SomeFloat = 0.1f;
};
template <typename Ar> void serialize(Ar& ar, A& a, unsigned) {
ar & boost::serialization::make_array(a.Value.data(), a.Value.size()) & a.SomeChar;
}
template <typename Ar> void serialize(Ar& ar, B& b, unsigned) {
ar & boost::serialization::make_array(b.data.data(), b.data.size()) & b.SomeFloat;
}
#endif
namespace bio = boost::iostreams;
static constexpr auto flags = boost::archive::archive_flags::no_header;
using BinaryData = std::vector</*unsigned*/ char>;
int main() {
char const* impls[] = {"C style arrays", "std::vector", "static_vector"};
std::cout << "Using " << impls[IMPL] << " implementation: ";
BinaryData serialized_data;
{
B object = {};
#if IMPL>0
{
// makes sure all containers half-full
A element;
element.Value.resize(50);
object.data.assign(25, element);
}
#endif
bio::stream<bio::back_insert_device<BinaryData>> os { serialized_data };
boost::archive::binary_oarchive oa(os, flags);
oa << object;
}
std::cout << "Size: " << serialized_data.size() << "\n";
{
bio::array_source as { serialized_data.data(), serialized_data.size() };
bio::stream<bio::array_source> os { as };
boost::archive::binary_iarchive ia(os, flags);
B object;
ia >> object;
}
}
Printing
Using C style arrays implementation: Size: 20472
Using std::vector implementation: Size: 5256
Using static_vector implementation: Size: 5039
Final Thoughts
See also:
Boost serialization bitwise serializability
https://www.boost.org/doc/libs/1_72_0/libs/serialization/doc/wrappers.html#binaryobjects
¹ (but keep in mind portability, as you probably already are aware with the POD approach, see C++ Boost::serialization : How do I archive an object in one program and restore it in another?)
² not POD, as with the NSMI your types weren't POD

How to serialize unique_ptr<char[]> in struct

I haven't been able to find any docs on how to serialize a unique_ptr to an array. Any help would be great.
struct Counter{
int index;
unique_ptr<char []> name;
template<class Archive>
void serialize(Archive & archive){
archive(index, name ); // serialize things by passing them to the archive
}
};
How it is assigned.
auto buffer = std::unique_ptr<char[]>(new char[BUFFER_SIZE]);
instance.name = std::move(buffer);
You can do it, but it needs some extra work. It differs depending on the type of the archive.
For text-based archives (e.g., XMLOutputArchive/XMLInputArchive and JSONOutputArchive/JSONInputArchive) you can use the saveBinaryValue() / loadBinaryValue() (http://uscilab.github.io/cereal/assets/doxygen/classcereal_1_1JSONOutputArchive.html).
Here's a complete example:
#include <iostream>
#include <memory>
#include <cereal/archives/xml.hpp>
#include <cereal/cereal.hpp>
struct Counter
{
static constexpr std::size_t BUFFER_SIZE = 12;
int index{};
std::unique_ptr<char[]> name;
Counter() = default;
Counter(int i)
: index{i},
name{new char[BUFFER_SIZE]}
{}
template<class Archive>
void load(Archive& archive)
{
archive(index);
name.reset(new char[BUFFER_SIZE]);
archive.loadBinaryValue(name.get(), BUFFER_SIZE * sizeof(decltype(name[0])));
}
template<class Archive>
void save(Archive& archive) const
{
archive(index);
archive.saveBinaryValue(name.get(), BUFFER_SIZE * sizeof(decltype(name[0])));
}
};
int main()
{
cereal::XMLOutputArchive archive(std::cout);
Counter c(42);
archive(CEREAL_NVP(c));
}
If you are using BinaryOutputArchive / BinaryInputArchive or PortableBinaryOutputArchive / PortableBinaryInputArchive the functions become saveBinary() and loadBinary() (http://uscilab.github.io/cereal/assets/doxygen/classcereal_1_1PortableBinaryOutputArchive.html).
For those, you can also wrap your array using binary_data():
template<class Archive>
void save(Archive& archive) const
{
archive(index, cereal::binary_data(name.get(), BUFFER_SIZE));
}

boost::serialization with immutable abstract base and virtual inheritance

The code below is my current thinking to permit boost::serialization of an immutable abstract base with virtual inheritance. I hope I missed something and there is a simpler solution...?
As it stands, it raises a few questions:
Is the comment in IObject::serialize valid?
The comment in bs::save_construct_data for House seems to indicate a bug in boost::serialization. Is that correct, or is there a better way to do this?
Is there a more elegant way to deserialise Building then the Deserialise function combined with a protected constructor?
The result of (3) is that another Building implementation will necessitate a chunk of duplicated code. I suspect this will require a bit of CRTP to mitigate - any alternatives?
How does one make this work if the virtual base contains data members? I suspect this is similar (or identical) to the Building::Deserialise.
#include <fstream>
#include <boost/archive/xml_oarchive.hpp>
#include <boost/archive/xml_iarchive.hpp>
#include <boost/serialization/export.hpp>
namespace bs = boost::serialization;
// IBase comes from an external library, and we are not interested in
// serialising an IBase*.
class IBase {
public:
virtual ~IBase(){};
};
class IObject : public virtual IBase {
private:
friend class bs::access;
template <class Archive>
void serialize(Archive &ar, const unsigned int version) {
std::cout << "Called IObject's serialize\n";
// IBase contains no members so there is no need to serialise it to/from the
// archive. However, the inheritance relationship must be registered in
// boost::serialization. We cannot use base_object to do this: It will try
// to static_cast *this to IBase, but we might not have created the instance
// yet, in which case there is no virtual table and a structured exception
// will be generated.
bs::void_cast_register<IObject, IBase>(static_cast<IObject *>(nullptr),
static_cast<IBase *>(nullptr));
}
public:
virtual ~IObject() {}
};
class IBuilding : public virtual IBase {
private:
friend class bs::access;
template <class Archive>
void serialize(Archive &ar, const unsigned int version) {
std::cout << "Called IBuilding's serialize\n";
bs::void_cast_register<IBuilding, IBase>(static_cast<IBuilding *>(nullptr),
static_cast<IBase *>(nullptr));
}
public:
virtual ~IBuilding() {}
};
/* Tedious forward declarations to permit later friending. */
class Building;
class House;
namespace boost {
namespace serialization {
template <class Archive>
inline void save_construct_data(Archive &ar, const Building *t,
const unsigned int version);
template <class Archive>
inline void save_construct_data(Archive &ar, const House *t,
const unsigned int version);
template <class Archive>
inline void load_construct_data(Archive &ar, House *t,
const unsigned int version);
}
}
/* Tedious forward declarations end. */
class Building : public IBuilding, public IObject {
private:
friend class bs::access;
template <class Archive>
void serialize(Archive &ar, const unsigned int version) {
std::cout << "Called Building's serialize\n";
// We can use base_object here because although the instance might not be
// created, the memory has been allocated. Since there is no virtual
// inheritance, the static_cast can succeed.
ar &bs::make_nvp("IObject", bs::base_object<IObject>(*this));
ar &bs::make_nvp("IBuilding", bs::base_object<IBuilding>(*this));
}
template <class Archive>
inline friend void bs::save_construct_data(Archive &ar, const Building *t,
const unsigned int version);
const double weight_;
protected:
const double height_;
// The Members, associated constructor, and Deserialise facilitate recreating
// this immutable base.
struct Members {
double weight_;
double height_;
};
Building(const Members &members)
: weight_(members.weight_), height_(members.height_) {}
template <class Archive> const Members Deserialise(Archive &ar) const {
double weight;
double height;
ar >> bs::make_nvp("weight_", weight) >> bs::make_nvp("height_", height);
return {weight, height};
}
public:
bool operator==(const Building &other) const {
return weight_ == other.weight_ && height_ == other.height_;
}
virtual double Height() const = 0;
};
class House : public Building {
private:
template <class Archive>
inline friend void bs::save_construct_data(Archive &ar, const House *t,
const unsigned int version);
template <class Archive>
inline friend void bs::load_construct_data(Archive &ar, House *t,
const unsigned int version);
template <class Archive>
explicit House(Archive &ar) : Building(Deserialise(ar)) {}
public:
House(double weight, double height) : Building({weight, height}) {}
virtual double Height() const { return height_; }
};
BOOST_CLASS_EXPORT(House);
namespace boost {
namespace serialization {
template <class Archive>
inline void save_construct_data(Archive &ar, const Building *t,
const unsigned int version) {
std::cout << "Called Building's save_construct_data\n";
ar << make_nvp("weight_", t->weight_) << make_nvp("height_", t->height_);
}
template <class Archive>
inline void bs::save_construct_data(Archive &ar, const House *t,
const unsigned int version) {
std::cout << "Called House's save_construct_data\n";
const auto &base = base_object<const Building>(*t);
ar << make_nvp("Building", base);
// ar << make_nvp("Building", &base); doesn't seem to work.
// Serialising out a reference calls Building's serialize method, the
// save_construct_data is only called for a pointer. This means we
// have to call it explicitly.
save_construct_data(ar, &base, version);
}
template <class Archive>
inline void bs::load_construct_data(Archive &ar, House *t,
const unsigned int version) {
std::cout << "Called House's load_construct_data\n";
ar >> make_nvp("Building", base_object<Building>(*t));
::new (t) House{ar};
}
}
}
int main() {
const char *file_name = "house.ser";
const bool save_first = true;
const House house(45367, 2.43);
std::cout << house.Height() << "\n";
const IObject *iHouse = &house;
if (save_first) {
std::ofstream ofs(file_name);
boost::archive::xml_oarchive oa(ofs);
oa << BOOST_SERIALIZATION_NVP(iHouse);
}
IBuilding *iHouse2;
{
std::ifstream ifs(file_name);
boost::archive::xml_iarchive ia(ifs);
ia >> BOOST_SERIALIZATION_NVP(iHouse2);
}
if (dynamic_cast<const Building &>(*iHouse) ==
dynamic_cast<const Building &>(*iHouse2))
std::cout << "ok\n";
else
std::cout << "uh oh\n";
return 0;
}
I don't believe that the comment is correct. I believe that the void_cast_register is un-necessary since IBase is known to be a virtual base class of IObject.
Futhermore, if you don't add the serialization_support free functions for IBase, you won't be able to serialise/deserialise an IBase*, only an IObject* (although I think that's fine unless you are managing ownership through the IBase rather than the IObject).

Non-intruisive Boost serialization of labelled enums C++

I would like to serialize struct A where I could save the tag name of the enum expressions instead of its integer in a non intruisive way (without having to change struct A).
enum e_fruit {
apple,
banana,
coconut
};
struct A {
e_fruit fruit;
int num;
};
namespace boost { namespace serialization {
template<class Archive>
void serialize(Archive & ar, A &a, const unsigned int version)
{
ar & boost::serialization::make_nvp("FruitType", a.fruit); // this will store an integer
ar & boost::serialization::make_nvp("Number", a.num);
}
}}
I have tried introducing a lookup table locally to the serialize function :
template<class Archive>
void serialize(Archive & ar, A &a, const unsigned int version)
{
static const char* const labels[] = { "Apple", "Banana", "Coconut" };
ar & boost::serialization::make_nvp("FruitType", labels[a.fruit]); // this won't work
ar & boost::serialization::make_nvp("Number", a.num);
}
Unfortunately I get the error :
Error 78 error C2228: left of '.serialize' must have
class/struct/union
since the prototype of make_nvp is
nvp< T > make_nvp(const char * name, T & t){
return nvp< T >(name, t);
}
so T should be a deduced template argument. I then thought about making a structure which would contains these labels but then I would have to add this in struct A which is what I want to avoid...
So how can we achieve this in the most non-intruisive way ?
I think you need to separate loading from saving. This compiles, I wonder it it is worth the game...
struct fruit_serializer
{
e_fruit &a_;
fruit_serializer(e_fruit &a) : a_(a) {}
template<class Archive>
void save(Archive & ar, const unsigned int version) const
{
std::string label = labels[static_cast<int>(a_)];
ar & boost::serialization::make_nvp("label", label);
}
template<class Archive>
void load(Archive & ar, const unsigned int version)
{
std::string label ;
ar & boost::serialization::make_nvp("label", label);
a_ = static_cast<e_fruit>(std::find(labels.begin(), labels.end(), label) - labels.begin());
}
BOOST_SERIALIZATION_SPLIT_MEMBER();
static std::vector<std::string> labels ;
};
std::vector<std::string> fruit_serializer::labels({ "Apple", "Banana", "Coconut" });
template<class Archive>
void serialize(Archive & ar, A &a, const unsigned int version)
{
fruit_serializer a1(a.fruit);
ar & boost::serialization::make_nvp("FruitType", a1);
}
As much as I hate to resurrect an old question, I wanted to do the same thing, but needed to make the enums string-serializable without any decorators. Since I didn't find much else regarding the topic, I'm posting my hacky solution as an option for anyone else who needs to serialize their enums as strings.
First off, some sample types to (eventually) serialize:
#include <iostream>
#include <sstream>
#include <boost/serialization/nvp.hpp>
#include <boost/archive/xml_oarchive.hpp>
// A few dummy enum types to test the solution
enum MyEnum00 {
Barb, Sarah, Lucy,
};
enum MyEnum01 {
Corey, Trevor, Jacob = 99,
};
const char* const to_cstring(const MyEnum01 e) {
switch (e) {
case Corey: return "Corey";
case Trevor: return "Trevor";
case Jacob: return "Jacob";
default: return "UNKNOWN";
}
}
inline std::ostream& operator<<(std::ostream& o, const MyEnum01 e) { return o << to_cstring(e); }
enum class MyEnumClass00 {
Ricky, Julian, Bubbles
};
enum class MyEnumClass01 {
Jim, Randy, Cyrus
};
const char* const to_cstring(const MyEnumClass01 e) {
switch (e) {
case MyEnumClass01::Jim: return "I let the liquor do the thinking, bud!";
case MyEnumClass01::Randy: return "Got any cheeeeeseburgers?";
case MyEnumClass01::Cyrus: return "I got work to do";
default: return "UNKNOWN";
}
}
inline std::ostream& operator<<(std::ostream& o, const MyEnumClass01 e) { return o << to_cstring(e); }
From boost_1_63_0/boost/archive/detail/oserializer.hpp, the function save_enum_type::invoke() is where enums get clobbered into ints.
Boost uses a somewhat clunky combination of templated structs with separately templated members, so it's difficult to make our change apply any time our desired enum types are used. As a workaround, we can specialize boost::archive::detail::save_enum_type for the archive type we're using. Then, we can overload its invoke() function to keep it from clobbering the enum types we need to have archived as strings.
save_enum_type::invoke is called roughly in the middle of boost's callstack, which eventually makes its way down into the basic_text_oprimitive class. There, an insertion operator is finally used to save the value to the ostream underlying the destination archive. We can take advantage of that implementation detail to get our enum types archived as strings by specializing the save_enum_type and impelmenting insertion operators for the target types.
namespace boost {
namespace archive {
namespace detail {
using xml_oarchive_ = boost::archive::xml_oarchive;
using save_non_pointer_type_ = detail::save_non_pointer_type<xml_oarchive_>;
template<>
struct save_enum_type<xml_oarchive_>
{
// This is boost's stock function that converts enums to ints before serializing them.
// We've added a copy to our specialized version of save_enum_type to maintain the exisitng behavior for any
// enum types we don't care to have archived in string form
template<class T>
static void invoke(xml_oarchive_& ar, const T& t) {
const int i = static_cast<int>(t);
ar << boost::serialization::make_nvp(NULL, i);
}
///// specialized enum types /////
// You could probably reduce all the repeated code with some type-trait magic...
static void invoke(xml_oarchive_& ar, const MyEnum00 &e) {
save_non_pointer_type_::invoke(ar, e);
}
static void invoke(xml_oarchive_& ar, const MyEnum01 &e) {
save_non_pointer_type_::invoke(ar, e);
}
// Won't work -- MyEnumClass00 doesn't have an insertion operator, so the underlying ostream won't know
// how to handle it
//static void invoke(xml_oarchive_& ar, const MyEnumClass00 &e) {
// save_non_pointer_type_::invoke(ar, e);
//}
static void invoke(xml_oarchive_& ar, const MyEnumClass01 &e) {
save_non_pointer_type_::invoke(ar, e);
}
};
} // namespace detail
} // namespace archive
} // namespace boost
And Finally some code to test everything:
int main()
{
std::stringstream outstream;
boost::archive::xml_oarchive ar(outstream);
MyEnum00 e00_0 = Barb;
MyEnum00 e00_1 = Sarah;
MyEnum00 e00_2 = Lucy;
MyEnum01 e01_0 = Corey;
MyEnum01 e01_1 = Trevor;
MyEnum01 e01_2 = Jacob;
MyEnumClass00 ec00_0 = MyEnumClass00::Ricky;
MyEnumClass00 ec00_1 = MyEnumClass00::Julian;
MyEnumClass00 ec00_2 = MyEnumClass00::Bubbles;
MyEnumClass01 ec01_0 = MyEnumClass01::Jim;
MyEnumClass01 ec01_1 = MyEnumClass01::Randy;
MyEnumClass01 ec01_2 = MyEnumClass01::Cyrus;
ar
// regular enums get passed down as int even if you don't explicitly convert them
<< BOOST_SERIALIZATION_NVP(e00_0)
<< BOOST_SERIALIZATION_NVP(e00_1)
<< BOOST_SERIALIZATION_NVP(e00_2)
// regular enums can also get special treatment
<< BOOST_SERIALIZATION_NVP(e01_0)
<< BOOST_SERIALIZATION_NVP(e01_1)
<< BOOST_SERIALIZATION_NVP(e01_2)
// enum classes that aren't specialized pass down as ints
<< BOOST_SERIALIZATION_NVP(ec00_0)
<< BOOST_SERIALIZATION_NVP(ec00_1)
<< BOOST_SERIALIZATION_NVP(ec00_2)
// enum classes can also get special treatment
<< BOOST_SERIALIZATION_NVP(ec01_0)
<< BOOST_SERIALIZATION_NVP(ec01_1)
<< BOOST_SERIALIZATION_NVP(ec01_2)
;
std::cout << outstream.str() << std::endl;
return 0;
}

boost:serialization reconstruction (loading)

I'm using boost:serialization to save data structures into a file. The actual data is a pointer vector of classes and subclasses.
However the constructor of the class which is serialized takes as parameter another instantiated class Agent which is an object that controls communication with a simulation API (webots).
I see that in boost::serialization examples, the serializable objects need an empty constructor class() {}; to be used for the reconstruction. However this is impractical in my case. How can I use reconstruction but include the object which communicates with the API ?
One of the serializable classes has this constructor:
State(Agent &A, ACTION_MODE const& m);
and I've seen from examples in boost docs that I need something like this:
State() {};
Yet Agent &A must be passed as parameter.
Should I find a way around this (using an extern, a singleton, a global object) or is there a way to modify this behavior when reconstructing ? I'm sure I'm missing something here.
Thank you
EDIT:
Maybe I didn't explain this clearly enough. I get an error message when trying to "load" by reconstructing the serialized data.
error: no matching function to call State::State()
Which is what made me look into boost::serialize code, and think that it is calling a constructor or copy operator.
How do I make it use a specific constructor to serialize the data and take as an argument the Agent reference &a ?
EDIT#2:
template <class S, class P, class A> void Task<S,P,A>::save(const char* file)
{
std::ofstream ofs(file);
assert(ofs.good());
boost::archive::text_oarchive oa(ofs);
oa << states;
ofs.close();
}
template <class S, class P, class A> void Task<S,P,A>::load(const char* file)
{
std::ifstream ifs(file);
boost::archive::text_iarchive ia(ifs);
ia >> states;
ifs.close();
}
States is friend to boost::serialization::access and has a function serialize.
Saving works fine, loading is the problem.
states is: boost::ptr_vector<S> states; where S is a type of State polymorphic class.
State is the base class and has "serialize"
template <class Archive>
void State::serialize(Archive& ar, const unsigned int version)
{
ar & accel.Xaxis & accel.Yaxis & accel.Zaxis;
ar & gyro.Xaxis & gyro.Yaxis & gyro.Zaxis;
ar & gps.Yaxis;
ar & positions;
ar & reward & value & hash_value;
}
guState inherits from State.
template <class Archive>
void guState::serialize(Archive& ar, const unsigned int version)
{
ar & boost::serialization::base_object<State>(*this);
ar & accel.Xaxis & accel.Yaxis & accel.Zaxis;
ar & gyro.Xaxis & gyro.Yaxis & gyro.Zaxis;
ar & gps.Yaxis;
ar & positions;
ar & reward & value & hash_value;
}
accel, gyro, gps are simple structures with 3 double variables. They get serialized above^^.
Positions is an std::map<std::string,float> positions;
Looking at the serialized text file, everything appears Ok.
I cannot understand why it calls a constructor when trying to load the file.
EDIT#3:
Base Constructor is:
State(Agent &A, ACTION_MODE const& m);
Derived Constuctor is:
guState::guState(Agent& A, ACTION_MODE const& m) :
State(A, m)
{
...
}
Agent reference &A, kept in each State (or derived State) refers to an object obtained from the simulation API. It controls a robot. I cannot serialize it, and it doesn't make sense serializing it.
When I use:
namespace boost { namespace serialization {
template <class Archive>
void save_construct_data(Archive & ar,const guState* d,const unsigned int file_version)
{
ar << guState::caller;
ar << guState::mode;
}
template <class Archive>
void load_construct_data(Archive & ar, guState* d,const unsigned int file_version)
{
Agent &a;
ACTION_MODE &m;
ar >> a;
ar >> m;
::new(d) guState(a,m);
}
}
}
I get the following errors:
invalid use of non-static data member State::caller
invalid use of non-static data member State::mode
referring to the references used from the constructor.
And:
error: 'a' declared as reference but not initialized
error: 'm' declared as reference but not initialized
As you can see, it makes no sense to try and save the reference to Agent, because that reference (even if it could be saved or serialized) will probably be different every time the application is started.
And in loading the construct data, apart from me probably using the wrong syntax, it makes no sense to construct from a serialized reference to agent.
What I belive i need, is a way to tell the load_construct_data how to obtain a reference to an Agent (after initializing an agent object) and use that reference to construct the data.
Does that make any sense ? Do you think this is doable ?
EDIT#4
namespace boost { namespace serialization {
template <class Archive>
void save_construct_data(Archive & ar,const guState* d,const unsigned int file_version)
{
ar << guState::caller;
}
template <class Archive>
void load_construct_data(Archive & ar, guState* d,const unsigned int file_version)
{
Agent * a;
ACTION_MODE mode = RAND_SING;
ar >> a;
::new(d) guState(*a,mode);
}
}
}
It will not allow to serialize guState::caller
I have also made class Agent serializable, and overloaded the load_construct_data and save_construct_data of Agent in order to request from the simulation app a new instance of Agent to control the API.
EDIT:
There is a part of the manual that I think we both missed: The section non-default constructors here. To make it work you need to a save_construct_data and a load_construct_data function. There is a slight technicallity in terms of where these are friended explored here.
Also, you said you had this problem when trying to load only, but that you could save fine. This makes me think that you might have omitted
BOOST_CLASS_EXPORT_GUID(state, "state")
This omission might lead to segmentation faults once you get the load compiling (see the export section of the manual)
To make sure I was not mistaken, I made a compiling example, which I add in case its useful.
#include <boost/archive/text_oarchive.hpp>
#include <boost/archive/text_iarchive.hpp>
#include <boost/serialization/export.hpp>
#include <iostream>
#include <fstream>
//base class
struct base
{
base(double d) : m_d(d) {}
virtual double run() = 0;
private:
friend class boost::serialization::access;
double m_d;
template <class Archive>
void serialize(Archive& ar, const unsigned int version)
{
ar & m_d;
}
};
//forward declare the save construct data before friending it
// (something about friend being in a different namespace)
class derived;
namespace boost { namespace serialization {
template<class Archive>
inline void save_construct_data(Archive & ar, const derived * t, const unsigned int file_version);
}}
//derived class with non-default constructor
struct derived : public base
{
derived(double a , double b) :
base(a+b),
m_a(a),m_b(b),m_c(a*b)
{}
//some checks
double get_a() const {return m_a;}
double get_b() const {return m_b;}
double get_c() const {return m_c;}
double run(){return 1.0;}
private:
friend class boost::serialization::access;
template<class Archive>
friend void boost::serialization::save_construct_data(Archive & ar, const derived * t, const unsigned int file_version);
template <class Archive>
void serialize(Archive& ar, const unsigned int version)
{
ar & boost::serialization::base_object<base>(*this);
//only need to return c, a and b already done for constructor
ar & m_c;
}
double m_a, m_b, m_c;
};
//Save and load the data required for the constructor.
namespace boost { namespace serialization {
template <class Archive>
inline void save_construct_data(
Archive & ar,const derived* d,const unsigned int file_version
)
{
// save data required to construct instance
ar << d->m_a;
ar << d->m_b;
}
template <class Archive>
inline void load_construct_data(
Archive & ar, derived* d,const unsigned int file_version
)
{
double a,b;
ar >> a;
ar >> b;
// invoke inplace constructor to initialize instance of my_class
::new(d) derived(a,b);
}
}
}
//register the derived class with boost.
BOOST_CLASS_EXPORT_GUID(derived, "derived")
int
main (int ac, char **av)
{
std::ofstream ofs("filename");
base* p = new derived(2,3);
// save data to archive
{
boost::archive::text_oarchive oa(ofs);
oa << p;
}
// ... some time later restore the class instance to its orginal state
base* p2;
{
std::ifstream ifs("filename");
boost::archive::text_iarchive ia(ifs);
ia >> p2;
}
derived* d = static_cast<derived*>(p2);
std::cout<<"p2 vals are: "<<d->get_a()<<" "<<d->get_b()<<" "<<d->get_c()<<std::endl;
}
OLD RESPONSE:
Not sure I entirely understood your problem (a fuller example would help me) -
The constructor doesn't usually come into it when you serialize an object: you serialize the raw data?
Do you mean that you don't want to serialize all the raw data of the object, but just want to reconstruct it again when you deserialize the object (using the constructor)? If so then you can do this by seriazing the data that you need for the reconstruction and splitting the save and load operations.
struct my_class
{
my_class(Agent& A, ACTION_MODE const & m)
: m_state(A,M)
{}
private:
State m_state;
friend class boost::serialization::access;
void save(Archive & ar, const unsigned int version) const
{
// note, version is always the latest when saving
Agent tmp_A = m_state.get_A();
ACTION_MODE tmp_m = m_state.get_m();
ar & tmp_A;
ar & tmp_m;
}
template<class Archive>
void load(Archive & ar, const unsigned int version)
{
Agent tmp_A;
ACTION_MODE tmp_m
ar & tmp_A;
ar & tmp_m;
m_state = State(tmp_A,tmp_m);
}
BOOST_SERIALIZATION_SPLIT_MEMBER()
}
Does that help, or have I missed the point?