How to use boost::serialization with nested structs and minimal code changes? - c++

Question

Currently we use POD which is stored in nested structs. Example:
#define MaxNum1 100;
#define MaxNum2 50;
struct A
{
int Value[MaxNum1];
char SomeChar = 'a';
};
struct B
{
A data[MaxNum2];
float SomeFloat = 0.1f;
};
int main()
{
B StructBObject = {};
}
We want to enhance our data structures using std::vector just like this:
struct NewA
{
std::vector<int> Value;
char SomeChar = 'a';
};
struct NewB
{
std::vector<NewA> data;
float SomeFloat = 0.1f;
};
int main()
{
NewB StructNewBObject = {};
}
The only argument against this modification is that NewA and NewB are no POD anymore and this makes reading/writing to a file more complicated.
How is it possible to read/write NewA and NewB to a file using boost::serialization with minimal
code changes to NewA and NewB? Minimal code changes are important because we use for example big structs which have up to 7 nested levels.

Answer 1

You can serialize using boost serialization¹:
template <typename Ar> void serialize(Ar& ar, A& a, unsigned) {
ar & a.Value & a.SomeChar;
}
template <typename Ar> void serialize(Ar& ar, B& b, unsigned) {
ar & b.data & b.SomeFloat;
}
Using these, you will already have the correct behaviour out of the box with both the C-array and std::vector approaches.
If you want to keep using fixed-size trivially-copyable types², you can use something like Boost Container's static_vector: it will keep track of the current size, but the data is statically allocated right inside the structures.
TRIPLE DEMO
Here's a triple demo program with three implementations depending on the IMPL variable.
As you can see the bulk of the code is kept invariant. However, for "best comparison" I've made sure that all the containers are at half capacity (50/25) before serialization.
The main program also deserializes.
Live On Coliru
#include <boost/iostreams/device/back_inserter.hpp>
#include <boost/iostreams/device/array.hpp>
#include <boost/iostreams/stream.hpp>
#include <boost/archive/binary_oarchive.hpp>
#include <boost/archive/binary_iarchive.hpp>
#include <boost/serialization/access.hpp>
#include <boost/serialization/is_bitwise_serializable.hpp>
#include <boost/serialization/binary_object.hpp>
#include <iostream>
#if (IMPL==0) // C arrays
struct A {
int Value[100];
char SomeChar = 'a';
};
struct B {
A data[50];
float SomeFloat = 0.1f;
};
template <typename Ar> void serialize(Ar& ar, A& a, unsigned) {
ar & a.Value & a.SomeChar;
}
template <typename Ar> void serialize(Ar& ar, B& b, unsigned) {
ar & b.data & b.SomeFloat;
}
#elif (IMPL==1) // std::vector
#include <boost/serialization/vector.hpp>
struct A {
std::vector<int> Value;
char SomeChar = 'a';
};
struct B {
std::vector<A> data;
float SomeFloat = 0.1f;
};
template <typename Ar> void serialize(Ar& ar, A& a, unsigned) {
ar & a.Value & a.SomeChar;
}
template <typename Ar> void serialize(Ar& ar, B& b, unsigned) {
ar & b.data & b.SomeFloat;
}
#elif (IMPL==2) // static_vector
#include <boost/serialization/vector.hpp>
#include <boost/container/static_vector.hpp>
struct A {
boost::container::static_vector<int, 100> Value;
char SomeChar = 'a';
};
struct B {
boost::container::static_vector<A, 50> data;
float SomeFloat = 0.1f;
};
template <typename Ar> void serialize(Ar& ar, A& a, unsigned) {
ar & boost::serialization::make_array(a.Value.data(), a.Value.size()) & a.SomeChar;
}
template <typename Ar> void serialize(Ar& ar, B& b, unsigned) {
ar & boost::serialization::make_array(b.data.data(), b.data.size()) & b.SomeFloat;
}
#endif
namespace bio = boost::iostreams;
static constexpr auto flags = boost::archive::archive_flags::no_header;
using BinaryData = std::vector</*unsigned*/ char>;
int main() {
char const* impls[] = {"C style arrays", "std::vector", "static_vector"};
std::cout << "Using " << impls[IMPL] << " implementation: ";
BinaryData serialized_data;
{
B object = {};
#if IMPL>0
{
// makes sure all containers half-full
A element;
element.Value.resize(50);
object.data.assign(25, element);
}
#endif
bio::stream<bio::back_insert_device<BinaryData>> os { serialized_data };
boost::archive::binary_oarchive oa(os, flags);
oa << object;
}
std::cout << "Size: " << serialized_data.size() << "\n";
{
bio::array_source as { serialized_data.data(), serialized_data.size() };
bio::stream<bio::array_source> os { as };
boost::archive::binary_iarchive ia(os, flags);
B object;
ia >> object;
}
}
Printing
Using C style arrays implementation: Size: 20472
Using std::vector implementation: Size: 5256
Using static_vector implementation: Size: 5039
Final Thoughts
See also:
Boost serialization bitwise serializability
https://www.boost.org/doc/libs/1_72_0/libs/serialization/doc/wrappers.html#binaryobjects
¹ (but keep in mind portability, as you probably already are aware with the POD approach, see C++ Boost::serialization : How do I archive an object in one program and restore it in another?)
² not POD, as with the NSMI your types weren't POD