boost serialization switching from binary_archive to polymorphic_archive - c++

Question

Is it possible to use a polymorphic_binary_iarchive to deserialize an object serialized with binary_oarchive?
My library has been written using binary archives for all the serialisation methods (as well as EOS portable archives). This results in overloaded serialize methods everywhere, significant code bloat and link time cost. I'd like to switch to polymorphic archive methods to reduce the number of exposed serialize methods and facilitate easy use of other archives (in particular XML). However, I would also like to maintain backwards compatibility with already serialized data.
polymorphic_binary_iarchive seems to work usually, but there is at least one case where it fails: vector.hpp changes the method of serialization depending on use_array_optimization, which for binary archives is set for any type that is_bitwise_serializable. So types that contain vector<size_t> fail to deserialize.
I'm tempted to try to specialize use_array_optimization for polymorphic_binary_iarchive (and the oarchive similarly). Is there any reason this would not work, or any other reasons why this is just A Bad Idea(TM)?
Edit
I tried that specialization:
struct use_array_optimization_in_polymorphic_archive {
template <class T>
struct apply : public boost::serialization::is_bitwise_serializable< T > {};
};
namespace boost {
namespace serialization {
template <>
struct use_array_optimization<boost::archive::polymorphic_iarchive> {
template <class ValueType>
struct apply : boost::mpl::apply1<
use_array_optimization_in_polymorphic_archive,
BOOST_DEDUCED_TYPENAME boost::remove_const<ValueType>::type>::type {};
};
}}
But it doesn't work because array.hpp then expects polymorphic_iarchive to implement load_array, which is only implemented for a basic_binary_iprimitive.
I'm also concerned that this will change behaviour for all polymorphic_iarchive implementations, not just the polymorphic_binary_iarchive. More thought required...
Edit2
Herefollows some code by way of demonstration. Toggle POLY_ON to use the polymorphic archive to deserialize; this works for doubles. Toggle VEC_ON to use vectors, demonstrating the problem. NB: I haven't yet double checked that this is exactly the same problem, but I'm reasonably sure it is. NNB: This is using Boost 1.59.
#include <fstream>
#include <boost/archive/binary_oarchive.hpp>
// #define POLY_ON
#ifdef POLY_ON
#include <boost/archive/polymorphic_binary_iarchive.hpp>
#else
#include <boost/archive/binary_iarchive.hpp>
#endif
// #define VEC_ON
#ifdef VEC_ON
#include <vector>
#include <boost/serialization/vector.hpp>
#endif
class bank_balance {
private:
friend class boost::serialization::access;
template <class archive>
void serialize(archive& ar, const unsigned int version) {
ar & date_;
ar & rate_;
}
#ifdef VEC_ON
std::vector<double> date_;
std::vector<double> rate_;
#else
double date_;
double rate_;
#endif
public:
bank_balance() : date_(0) {}
bank_balance(
#ifdef VEC_ON
std::vector<double> date, std::vector<double> rate
#else
double date, double rate
#endif
)
: date_(date), rate_(rate)
{}
bool operator==(const bank_balance& other) const {
return date_ == other.date_ && rate_ == other.rate_;
}
};
int main() {
std::ofstream ofs("bank_balance.ser");
#ifdef VEC_ON
const bank_balance balance({45367, 45369}, {5.6, 2.43});
#else
const bank_balance balance(45367, 5.6);
#endif
{
boost::archive::binary_oarchive oa(ofs);
oa << balance;
}
bank_balance balance2;
{
std::ifstream ifs("bank_balance.ser");
#ifdef POLY_ON
boost::archive::polymorphic_binary_iarchive ia(ifs);
#else
boost::archive::binary_iarchive ia(ifs);
#endif
ia >> balance2;
}
if (balance == balance2) std::cout << "ok\n";
else std::cout << "dammit\n";
return 0;
}

Answer 1

Is it possible to use a polymorphic_binary_iarchive to deserialize an object serialized with binary_oarchive?
Short answer: yes.
The only difference is the call-site interface here.
EDIT
Perhaps, unintentionally, this "promise" was broken when they introduced optimized serialization for POD containers.
Here's my analysis, with yours.cpp from your question, and mine.cpp as edited below:
{
std::ofstream ofs("bank_balance.ser");
#ifdef POLY_ON
boost::archive::polymorphic_binary_oarchive oa(ofs);
#else
boost::archive::binary_oarchive oa(ofs);
#endif
oa << balance;
}
I compile all flavours with the following command line:
for src in yours mine; do for a in {,-DPOLY_ON}\ {,-DVEC_ON}; do time g++ $a -O2 -std=c++11 $src.cpp -lboost_{system,serialization} -o "$src${a//[D _]/}.exe"; done; done
Which results in mine.exe, mine-POLYON.exe, mine-POLYON-VECON.exe, mine-VECON.exe, yours.exe, yours-POLYON.exe, yours-POLYON-VECON.exe and yours-VECON.exe. Running them:
(set -x; for a in ./*.exe; do $a; done)
Results in
+ ./mine.exe
ok: true
+ ./mine-POLYON.exe
ok: true
+ ./mine-POLYON-VECON.exe
ok: true
+ ./mine-VECON.exe
ok: true
+ ./yours.exe
ok
+ ./yours-POLYON.exe
ok
+ ./yours-POLYON-VECON.exe
terminate called after throwing an instance of 'std::length_error'
what(): vector::_M_default_append
+ ./yours-VECON.exe
ok
Note that all combinations are fine if you write using the same archive implementation as while reading. You're also right that sadly ./yours-POLYON-VECON.exe is the only one to break. I think this is unintentional but your hunch could be spot on:
doc
Note that the concept of polymophic archives is fundamentally incompatible with the serialization of new types that are marked "primitive" by the user with:
BOOST_CLASS_IMPLEMENTATION(my_primitive_type, boost::serialization::primitive_type)
Code to implement serialization for these types is instantiated "on the fly" in the user's program. But this conflicts with the whole purpose of the polymorphic archive. An attempt to serialize such a primitive type will result in a compilation error since the common polymorhic interface is static and cannot instantiate code for a new type.
It looks like the vector optimization path might be sharing one of these code paths.
RECOMMENDATION
I'd recommend making a conversion tool to convert old format files to the new format. You can read using the non-polymorphic iarchive and write using the polymorphic archive. That, of course, means you'll have to compile both approaches for this release, but
it doesn't have to be baked into the main executable(s) - only the conversion tools does need the "old" method
the conversion tool will not have to be versioned, it stays the same for all future releases; this means you can stop building new versions and drop the required extra code.

Answer 2

I know this is an old topic, but I've been fighting with this precise problem, and think I have found a good solution.
The core of the problem is that polymorphic archives never get dispatched to use the optimized load/save versions in boost/serialization/vector.hpp. How I fixed this was to overload those dispatch functions for the polymorphic archive types like this:
namespace boost::serialization
{
template<class U, class Allocator>
inline void load(boost::archive::polymorphic_iarchive& ar, std::vector<U, Allocator>& t, const unsigned int file_version)
{
using use_optimized = boost::has_trivial_constructor<U>::type;
load(ar, t, file_version, use_optimized());
}
template<class U, class Allocator>
inline void save(boost::archive::polymorphic_oarchive& ar, const std::vector<U, Allocator>& t, const unsigned int file_version)
{
using use_optimized = boost::has_trivial_constructor<U>::type;
save(ar, t, file_version, use_optimized());
}
}