How to serialize a class with a boost::dynamic_bitset member?
#include <boost/archive/text_oarchive.hpp>
#include <boost/archive/text_iarchive.hpp>
#include <boost/dynamic_bitset.hpp>
#include <boost/serialization/bitset.hpp>
#include <sstream>
class A
{
friend class boost::serialization::access;
boost::dynamic_bitset<> x;
template<class Archive>
void serialize(Archive & ar, const unsigned int){
ar & x;
}
};
int main()
{
A a;
std::stringstream ss;
boost::archive::text_oarchive oa(ss);
oa << a;
return 0;
}
Compiling gives an error (boost 1.57)
In file included from /usr/include/boost/serialization/extended_type_info_typeid.hpp:37:0,
from /usr/include/boost/archive/detail/oserializer.hpp:38,
from /usr/include/boost/archive/detail/interface_oarchive.hpp:23,
from /usr/include/boost/archive/detail/common_oarchive.hpp:22,
from /usr/include/boost/archive/basic_text_oarchive.hpp:32,
from /usr/include/boost/archive/text_oarchive.hpp:31,
from dynamic_bitset_setial.cpp:1:
/usr/include/boost/serialization/access.hpp: In static member function ‘static void boost::serialization::access::serialize(Archive&, T&, unsigned int) [with Archive = boost::archive::text_oarchive, T = boost::dynamic_bitset<>]’:
/usr/include/boost/serialization/serialization.hpp:69:5: instantiated from ‘void boost::serialization::serialize(Archive&, T&, unsigned int) [with Archive = boost::archive::text_oarchive, T = boost::dynamic_bitset<>]’
/usr/include/boost/serialization/serialization.hpp:128:9: instantiated from ‘void boost::serialization::serialize_adl(Archive&, T&, unsigned int) [with Archive = boost::archive::text_oarchive, T = boost::dynamic_bitset<>]’
/usr/include/boost/archive/detail/oserializer.hpp:148:5: instantiated from ‘void boost::archive::detail::oserializer<Archive, T>::save_object_data(boost::archive::detail::basic_oarchive&, const void*) const [with Archive = boost::archive::text_oarchive, T = boost::dynamic_bitset<>]’
dynamic_bitset_setial.cpp:25:1: instantiated from here
/usr/include/boost/serialization/access.hpp:118:9: error: ‘class boost::dynamic_bitset<>’ has no member named ‘serialize’
I went ahead and filed the pull request to add Serialization support to Boost Dynamic Bitset
Serialization using the public interface isn't optimal as to_block_range()/from_block_range() require copying of m_bits (and subsequent resize()).
I added a generic implementation to Boost Dynamic Bitset. The changes merge cleanly against develop or master (1_58_0).
Changes
Implementation added with
minimal intrusiveness, only a nested friend (class serialization_impl;) has been forward declared to "key-hole" the required friend access through
This class, as well as the actual ADL hook for Boost Serialization are implemented in a separate header (dynamic_bitset/serialization.hpp, similar to other boost libraries with serialization support).
This means that zero dependencies on Boost Serialization stuff exists unless boost/dynamic_bitset/serialization.hpp is actually included
Zero copy is achieved (leveraging std::vector<Block>'s builtin support in Boost Serialization)
Tests
The second commit in the pull request adds tests for this feature. I'm not sure how to add the dyn_bitset_unit_tests5.cpp to the Jamfile. I suppose something else must be done to ensure linking to Boost System and Boost Serialization. I have run the tests myself using a simple wrapper:
#include <modular-boost/libs/dynamic_bitset/dyn_bitset_unit_tests5.cpp>
int main() {
test_main(0, nullptr);
}
Which can then be compiled and run with e.g.
g++ main.cpp -lboost_system -lboost_serialization && ./a.out
No output means no errors.
dynamic_bitset<> is not serializable, as you've found out (std::bitset<N> is different type).
Not to worry, though, you can add it without too much effort:
namespace boost { namespace serialization {
template <typename Ar, typename Block, typename Alloc>
void save(Ar& ar, dynamic_bitset<Block, Alloc> const& bs, unsigned) {
size_t num_bits = bs.size();
std::vector<Block> blocks(bs.num_blocks());
to_block_range(bs, blocks.begin());
ar & num_bits & blocks;
}
template <typename Ar, typename Block, typename Alloc>
void load(Ar& ar, dynamic_bitset<Block, Alloc>& bs, unsigned) {
size_t num_bits;
std::vector<Block> blocks;
ar & num_bits & blocks;
bs.resize(num_bits);
from_block_range(blocks.begin(), blocks.end(), bs);
bs.resize(num_bits);
}
template <typename Ar, typename Block, typename Alloc>
void serialize(Ar& ar, dynamic_bitset<Block, Alloc>& bs, unsigned version) {
split_free(ar, bs, version);
}
} }
This works e.g. Live On Coliru
int main() {
A a;
for (int i=0; i<128; ++i)
a.x.resize(11*i, i%2);
std::stringstream ss;
{
boost::archive::text_oarchive oa(ss);
oa << a;
}
std::cout << ss.str();
{
boost::archive::text_iarchive ia(ss);
A b;
ia >> b;
assert(a.x == b.x);
}
}
Note that if you can't afford to copy the blocks vector, it's equally easy to add serialization directly on the m_bits level, but that requires intrusive changes (friend access required at a minimum).
Such a thing would easily be added to boost in a pull request.
Update added that pull request
Related
I wrote the following code into a file named main.cpp.
It involves the curiously recurring template pattern (CRTP) with the standard type std::variant.
#include <string>
#include <variant>
#include <vector>
template<typename T>
struct either {
std::vector<T> arg;
};
template<typename T>
struct maybe_either: std::variant<T, either<maybe_either<T>>> {
template<typename U>
maybe_either(U&& v):
std::variant<T, either<maybe_either<T>>>(std::forward<U>(v)) {
}
};
struct var {
std::string name;
};
int main(int, char**) {
auto expression = maybe_either<var>(either<maybe_either<var>>{});
std::visit([&](auto&& v) {
using T = std::decay_t<decltype (v)>;
if constexpr (std::is_same_v<T, var>) {
// ...
} else if constexpr (std::is_same_v<T, either<maybe_either<var>>>) {
// ...
}
}, expression);
return 0;
}
When compiling it with the following command line, I get the error message below:
$ g++ -c -std=c++17 main.cpp
In file included from main.cpp:2:0:
/usr/include/c++/7/variant: In instantiation of ‘constexpr const size_t std::variant_size_v<maybe_either<var> >’:
/usr/include/c++/7/variant:702:10: required from ‘struct std::__detail::__variant::__gen_vtable<void, main(int, char**)::<lambda(auto:1&&)>&&, maybe_either<var>&>’
/usr/include/c++/7/variant:1255:23: required from ‘constexpr decltype(auto) std::visit(_Visitor&&, _Variants&& ...) [with _Visitor = main(int, char**)::<lambda(auto:1&&)>; _Variants = {maybe_either<var>&}]’
main.cpp:32:18: required from here
/usr/include/c++/7/variant:97:29: error: incomplete type ‘std::variant_size<maybe_either<var> >’ used in nested name specifier
inline constexpr size_t variant_size_v = variant_size<_Variant>::value;
^~~~~~~~~~~~~~
/usr/include/c++/7/variant: In instantiation of ‘constexpr const auto std::__detail::__variant::__gen_vtable<void, main(int, char**)::<lambda(auto:1&&)>&&, maybe_either<var>&>::_S_vtable’:
/usr/include/c++/7/variant:711:29: required from ‘struct std::__detail::__variant::__gen_vtable<void, main(int, char**)::<lambda(auto:1&&)>&&, maybe_either<var>&>’
/usr/include/c++/7/variant:1255:23: required from ‘constexpr decltype(auto) std::visit(_Visitor&&, _Variants&& ...) [with _Visitor = main(int, char**)::<lambda(auto:1&&)>; _Variants = {maybe_either<var>&}]’
main.cpp:32:18: required from here
/usr/include/c++/7/variant:711:49: error: ‘_S_apply’ was not declared in this scope
static constexpr auto _S_vtable = _S_apply();
~~~~~~~~^~
My class maybe_either derived from std::variant<...> can be used normally in other contexts, but when I call std::visit(...) on it, it fails to compile. What is wrong?
This is basically LWG3052 which I'm trying to address in P2162.
maybe_either<T> isn't a specialization of std::variant - it inherits from one. And std::visit is currently underspecified. It's not at all clear what kinds of "variants" are allowed to be visited.
libstdc++ implements the original suggested direction in that library issue, which is only specializations of std::variant (of which you are not). libc++, on the other hand, allows types that inherit from std::variant to be visited, so it accepts your example.
The intent is that the example as-is will become well-formed eventually. But until then, you'll have to ensure that the visit you do is directly on a std::variant. You can do so by adding your own member or non-member visit that does this cast for you, so the callers don't have to do it themselves.
For example, this:
template<typename T>
struct maybe_either: std::variant<T, either<maybe_either<T>>> {
using base = typename maybe_either::variant;
template<typename U>
maybe_either(U&& v):
std::variant<T, either<maybe_either<T>>>(std::forward<U>(v)) {
}
template <typename F>
decltype(auto) visit(F&& f) & {
return std::visit(std::forward<F>(f), static_cast<base&>(*this));
}
};
allows this to work:
int main(int, char**) {
auto expression = maybe_either<var>(either<maybe_either<var>>{});
expression.visit([&](auto&& v) {
using T = std::decay_t<decltype (v)>;
if constexpr (std::is_same_v<T, var>) {
// ...
} else if constexpr (std::is_same_v<T, either<maybe_either<var>>>) {
// ...
}
});
return 0;
}
The following code is a minimum working (or perhaps non-working) example.
What it does is basically encapsulates a bunch of std::map structures as private members in a base class. To avoid writing a lot of setters and getters, they are implemented as template functions.
// test.cpp
#include <map>
#include <iostream>
enum class E0
{
F0, F1, F2,
};
The declaration of the base class.
using std::map;
class P_base
{
private:
map<E0, int> m_imap;
// ...
// ... Other std::map members with different key types and value types.
public:
map<E0, int> & imap;
// ...
// ... Other std::map references.
P_base() : imap(m_imap) {}
template<typename map_type, typename key_type, typename val_type>
void set(map_type & m, const key_type & k, const val_type & v)
{
m[k] = v;
}
template<typename map_type, typename key_type>
auto access_to_map(const map_type & m, const key_type & k) -> decltype(m.at(k))
{
return m.at(k);
}
};
class P : private P_base
{
public:
decltype(P_base::imap) & imap;
P() : P_base(), imap(P_base::imap) {}
template<typename map_type, typename key_type, typename val_type>
void set(map_type & m, const key_type & k, const val_type & v)
{
P_base::set(m, k, v);
}
template<typename map_type, typename key_type>
auto access_to_map(const map_type & m, const key_type & k) -> decltype(P_base::access_to_map(m, k))
{
return P_base::access_to_map(m, k);
}
};
main
int main(int argc, const char * argv[])
{
using std::cout;
using std::endl;
P op;
op.set(op.imap, E0::F0, 100);
op.set(op.imap, E0::F1, 101);
op.set(op.imap, E0::F2, 102);
cout << op.access_to_map(op.imap, E0::F1) << endl;
}
$ clang++ -std=c++11 test.cpp && ./a.out
101
But if I compile it with intel compiler (icpc version 15.0.3 (gcc version 5.1.0 compatibility)), the compiler gives me this error message (which I don't undertand at all, especially when clang will compile the code):
$ icpc -std=c++ test.cpp && ./a.out
test.cpp(67): error: no instance of function template "P::access_to_map" matches the argument list
argument types are: (std::__1::map<E0, int, std::__1::less<E0>, std::__1::allocator<std::__1::pair<const E0, int>>>, E0)
object type is: P
cout << op.access_to_map(op.imap, E0::F1) << endl;
And it also confuses me by not complaining about the set function.
Does anyone have any idea what is going on here?
Note: My answer applies to g++ - hopefully it's the same as icc.
Here is a smaller test case:
struct Base
{
int func(int t) { return t; }
};
struct Der : Base
{
template<typename T>
auto f(T t) -> decltype(Base::func(t))
{
return t;
}
};
int main(){ Der d; d.f(5); }
The error is:
mcv.cc: In function 'int main()':
mcv.cc:16:25: error: no matching function for call to 'Der::f(int)'
int main(){ Der d; d.f(5); }
^
mcv.cc:16:25: note: candidate is:
mcv.cc:9:7: note: template<class T> decltype (t->Base::func()) Der::f(T)
auto f(T t) -> decltype(Base::func(t))
^
mcv.cc:9:7: note: template argument deduction/substitution failed:
mcv.cc: In substitution of 'template<class T> decltype (t->Base::func()) Der::f(T) [with T = int]':
mcv.cc:16:25: required from here
mcv.cc:9:38: error: cannot call member function 'int Base::func(int)' without object
auto f(T t) -> decltype(Base::func(t))
This can be fixed by changing decltype(Base::func(t)) to decltype(this->Base::func(t)). A corresponding fix fixes your code sample, for me.
Apparently, the compiler doesn't consider that Base::func(t) should be called with *this as hidden argument. I don't know if this is a g++ bug, or if clang is going beyond the call of duty.
Note that in C++14, since the function has a single return statement, the trailing return type can be omitted entirely:
template<typename T>
auto f(T t)
{
return t;
}
Consider the following code:
#include <boost/serialization/nvp.hpp>
#include <boost/archive/xml_iarchive.hpp>
#include <boost/archive/xml_oarchive.hpp>
class Foo{
friend class boost::serialization::access;
template<class Archive>
void serialize(Archive & ar, const unsigned int)
{
ar & BOOST_SERIALIZATION_NVP(i);
}
int i;
Foo():i(0){}
public:
Foo(int k):i(k){}
};
int main(int argc, char *argv[])
{
std::vector< Foo> f;
f.push_back(Foo(12));
std::ofstream os("path");
boost::archive::xml_oarchive oa(os);
oa << boost::serialization::make_nvp("f", f);
os.close();
std::vector<Foo> g;
std::ifstream is("path");
boost::archive::xml_iarchive ia(is);
ia >> boost::serialization::make_nvp("f", g);
}
Which works fine when serializing a vector of Foos. However, if I try to serialize a map of Foos, it fails on the private default constructor:
std::map<std::string, Foo> f;
f.insert(std::make_pair("hello", Foo(12)));
std::ofstream os("path");
boost::archive::xml_oarchive oa(os);
oa << boost::serialization::make_nvp("f", f);
os.close();
std::map<std::string, Foo> g;
std::ifstream is("path");
boost::archive::xml_iarchive ia(is);
ia >> boost::serialization::make_nvp("f", g);
fails with
In file included from main.cpp:2:
In file included from /usr/local/include/boost/serialization/nvp.hpp:19:
In file included from /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/utility:70:
/usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/bits/stl_pair.h:109:18: error: field of type 'Foo' has private default constructor
: first(), second() { }
^
/usr/local/include/boost/serialization/access.hpp:132:17: note: in instantiation of member function 'std::pair<const std::basic_string<char>, Foo>::pair' requested here
::new(t)T;
^
/usr/local/include/boost/serialization/serialization.hpp:93:13: note: in instantiation of function template specialization 'boost::serialization::access::construct<std::pair<const std::basic_string<char>, Foo> >' requested here
access::construct(t);
^
/usr/local/include/boost/serialization/serialization.hpp:158:9: note: in instantiation of function template specialization 'boost::serialization::load_construct_data<boost::archive::xml_iarchive, std::pair<const std::basic_string<char>, Foo> >' requested here
load_construct_data(ar, t, v);
^
/usr/local/include/boost/serialization/detail/stack_constructor.hpp:58:31: note: in instantiation of function template specialization 'boost::serialization::load_construct_data_adl<boost::archive::xml_iarchive, std::pair<const std::basic_string<char>, Foo> >' requested here
boost::serialization::load_construct_data_adl(
^
/usr/local/include/boost/serialization/collections_load_imp.hpp:83:48: note: in instantiation of member function 'boost::serialization::detail::stack_construct<boost::archive::xml_iarchive, std::pair<const std::basic_string<char>, Foo> >::stack_construct' requested here
detail::stack_construct<Archive, type> t(ar, v);
^
/usr/local/include/boost/serialization/collections_load_imp.hpp:158:16: note: (skipping 12 contexts in backtrace; use -ftemplate-backtrace-limit=0 to see all)
hint = ifunc(ar, s, item_version, hint);
^
/usr/local/include/boost/archive/detail/common_iarchive.hpp:66:18: note: in instantiation of function template specialization 'boost::archive::load<boost::archive::xml_iarchive, std::map<std::basic_string<char>, Foo, std::less<std::basic_string<char> >, std::allocator<std::pair<const std::basic_string<char>, Foo> > > >' requested here
archive::load(* this->This(), t);
^
/usr/local/include/boost/archive/basic_xml_iarchive.hpp:86:39: note: in instantiation of function template specialization 'boost::archive::detail::common_iarchive<boost::archive::xml_iarchive>::load_override<std::map<std::basic_string<char>, Foo, std::less<std::basic_string<char> >, std::allocator<std::pair<const std::basic_string<char>, Foo> > > >' requested here
this->detail_common_iarchive::load_override(t.value(), 0);
^
/usr/local/include/boost/archive/xml_iarchive.hpp:93:38: note: in instantiation of function template specialization 'boost::archive::basic_xml_iarchive<boost::archive::xml_iarchive>::load_override<std::map<std::basic_string<char>, Foo, std::less<std::basic_string<char> >, std::allocator<std::pair<const std::basic_string<char>, Foo> > > >' requested here
basic_xml_iarchive<Archive>::load_override(t, 0);
^
/usr/local/include/boost/archive/detail/interface_iarchive.hpp:60:23: note: in instantiation of function template specialization 'boost::archive::xml_iarchive_impl<boost::archive::xml_iarchive>::load_override<const boost::serialization::nvp<std::map<std::basic_string<char>, Foo, std::less<std::basic_string<char> >, std::allocator<std::pair<const std::basic_string<char>, Foo> > > > >' requested here
this->This()->load_override(t, 0);
^
main.cpp:50:8: note: in instantiation of function template specialization 'boost::archive::detail::interface_iarchive<boost::archive::xml_iarchive>::operator>><const boost::serialization::nvp<std::map<std::basic_string<char>, Foo, std::less<std::basic_string<char> >, std::allocator<std::pair<const std::basic_string<char>, Foo> > > > >' requested here
ia >> boost::serialization::make_nvp("f", g);
^
main.cpp:34:5: note: implicitly declared private here
Foo():i(0){}
^
I'm using clang
Ubuntu clang version 3.4-1ubuntu3 (tags/RELEASE_34/final) (based on LLVM 3.4)
and boost version 1.55 as shipped with Ubuntu 14.04LTS.
I have tried providing the load_construct_data() function as follows:
namespace boost
{
namespace serialization
{
template<class Archive>
inline void load_construct_data(Archive &archive, Foo*a, unsigned int
file_version)
{
::new(a)Foo(0);
}
}
}
but I'm still getting the same error, because it requires the constructor when instantiating a std::pair
Oh. Aha.
I just used Boost 1.57.0 to compare the situation with map<string, Foo>.
Well, you're in luck. You've found another library version dependency (likely a bug).
Not using that, but providing the private default constructor, GCC 4.8.2 compiles it just fine: Live On Coliru [1]
GCC 4.9.0 fails to compile it though (it uses a newer version of the standard library too). The std::pair<> default constructor fails to compile there, since Foo is not default constructible: Live On Coliru
Solution
Luckily the solution with save_construct_data/load_construct_data saves the day, again.
However, you need to cater for the fact that the element type is actually not Foo, but std::pair<T const, Foo>.
template <class Archive, typename K> inline friend void save_construct_data(Archive& ar, std::pair<K, Foo> const* v, const unsigned int) {
std::cerr << __PRETTY_FUNCTION__ << "\n";
ar & boost::serialization::make_nvp("first", v->first);
ar & boost::serialization::make_nvp("second", v->second.i);
}
template <class Archive, typename K> inline friend void load_construct_data(Archive& ar, std::pair<K, Foo>* v, const unsigned int) {
std::cerr << __PRETTY_FUNCTION__ << "\n";
typename std::remove_cv<K>::type first;
ar & boost::serialization::make_nvp("first", first);
int tmp;
ar & boost::serialization::make_nvp("second", tmp);
new(v) std::pair<K, Foo>(first, tmp);
}
Now it all works:
Live On Coliru
#include <boost/serialization/nvp.hpp>
#include <boost/serialization/serialization.hpp>
#include <boost/archive/xml_iarchive.hpp>
#include <boost/archive/xml_oarchive.hpp>
#include <boost/serialization/access.hpp>
#include <boost/serialization/vector.hpp>
#include <boost/serialization/map.hpp>
#include <boost/version.hpp>
#include <fstream>
#include <iostream>
class Foo {
friend class boost::serialization::access;
template <class Archive> void serialize(Archive &, const unsigned int) {
std::cerr << __PRETTY_FUNCTION__ << "\n";
}
template <class Archive, typename K> inline friend void save_construct_data(Archive& ar, std::pair<K, Foo> const* v, const unsigned int) {
std::cerr << __PRETTY_FUNCTION__ << "\n";
ar & boost::serialization::make_nvp("first", v->first);
ar & boost::serialization::make_nvp("second", v->second.i);
}
template <class Archive, typename K> inline friend void load_construct_data(Archive& ar, std::pair<K, Foo>* v, const unsigned int) {
std::cerr << __PRETTY_FUNCTION__ << "\n";
typename std::remove_cv<K>::type first;
ar & boost::serialization::make_nvp("first", first);
int tmp;
ar & boost::serialization::make_nvp("second", tmp);
new(v) std::pair<K, Foo>(first, tmp);
}
int i;
public:
Foo(int k) : i(k) {}
friend std::ostream& operator<<(std::ostream& os, Foo const& foo) {
return os << "Foo { " << foo.i << " }";
}
};
namespace boost { namespace serialization {
} }
int main() {
using Data = std::map<std::string, Foo>;
std::cout << "Boost version: " << BOOST_VERSION << "\n";
{
auto f = Data { {"a", 12 }, {"b", 42} };
//for (auto& e : f) std::cout << e.first << ", " << e.second << "\n";
std::ofstream os("path");
boost::archive::xml_oarchive oa(os);
oa << boost::serialization::make_nvp("f", f);
}
{
Data g;
std::ifstream is("path");
boost::archive::xml_iarchive ia(is);
ia >> boost::serialization::make_nvp("f", g);
for (auto& e : g)
std::cout << e.first << ", " << e.second << "\n";
}
}
Which prints:
Boost version: 105700
void save_construct_data(Archive&, const std::pair<K, Foo>*, unsigned int) [with Archive = boost::archive::xml_oarchive; K = const std::__cxx11::basic_string<char>]
void Foo::serialize(Archive&, unsigned int) [with Archive = boost::archive::xml_oarchive]
void save_construct_data(Archive&, const std::pair<K, Foo>*, unsigned int) [with Archive = boost::archive::xml_oarchive; K = const std::__cxx11::basic_string<char>]
void Foo::serialize(Archive&, unsigned int) [with Archive = boost::archive::xml_oarchive]
void load_construct_data(Archive&, std::pair<K, Foo>*, unsigned int) [with Archive = boost::archive::xml_iarchive; K = const std::__cxx11::basic_string<char>]
void Foo::serialize(Archive&, unsigned int) [with Archive = boost::archive::xml_iarchive]
void load_construct_data(Archive&, std::pair<K, Foo>*, unsigned int) [with Archive = boost::archive::xml_iarchive; K = const std::__cxx11::basic_string<char>]
void Foo::serialize(Archive&, unsigned int) [with Archive = boost::archive::xml_iarchive]
a, Foo { 12 }
b, Foo { 42 }
[1] (I can't link it on Coliru because the boost libraries there have been recompiled to the GCC 5.0 ABI)
Notes
A better, general, solution would be to do the load/save_construct_data trick generically for non-default-constructible types in the boost::serialization namespace. That way, people won't have to "know" about the std::pair<> implementation detail. They could just implement load/save_construct_data for their own user-types and it would JustWork™ whether they put it in a vector or a map.
Implementing that generically is less-than-trivial though, and might interfere with some other machineries internal to the Boost Serialization framework.
I'll prefer to get some help of the Boost Serialization maintainers to do that in a reliable way. So, it seems I'll be submitting two tickets today.
It doesn't necessarily work for vector either. Vector deserialization resizes the vector to the required size first. This requires the elements to be default-constructible.
Note how this is only a problem because
construction is not accessed via the serialization::access friend "token"
the class is default constructible
The Documented Solution
The documentation tells you to use save_construct_data and load_construct_data for types that are not default-constructible.
Specifically they promise it will work for STL containers of these as well:
In addition to the deserialization of pointers, these overrides are used in the deserialization of STL containers whose element type has no default constructor.
In practice, this works well in v1.57.0:
Live On Coliru
But in 1.58.0 it isn't true...
A Bug In Boost 1.58.0
Version 1.58.0 seems to have broken this:
The code seems to do required check (from serialization/vector.hpp in the unoptimized version of load(...)):
if(detail::is_default_constructible<U>()){
t.resize(count);
// ... snip ...
}
else{
t.reserve(count);
// ... snip ...
}
However, this does the check at runtime. The method will statically refuse to compile. Oops.
The Fix
Instead of having the branches in the same flow code, it should be dispatched so that only the applicable branch is instantiated. I tested with this simplistic approach:
namespace sehe_bugfix {
template<class Archive, class U, class Allocator>
inline void load_elements(
Archive & ar,
std::vector<U, Allocator> &t,
const unsigned int /* file_version */,
collection_size_type count,
mpl::true_
){
const boost::archive::library_version_type library_version(
ar.get_library_version()
);
item_version_type item_version(0);
if(boost::archive::library_version_type(3) < library_version){
ar >> BOOST_SERIALIZATION_NVP(item_version);
}
t.resize(count);
typename std::vector<U, Allocator>::iterator hint;
hint = t.begin();
while(count-- > 0){
ar >> boost::serialization::make_nvp("item", *hint++);
}
}
template<class Archive, class U, class Allocator>
inline void load_elements(
Archive & ar,
std::vector<U, Allocator> &t,
const unsigned int /* file_version */,
collection_size_type count,
mpl::false_
){
const boost::archive::library_version_type library_version(
ar.get_library_version()
);
item_version_type item_version(0);
if(boost::archive::library_version_type(3) < library_version){
ar >> BOOST_SERIALIZATION_NVP(item_version);
}
t.reserve(count);
while(count-- > 0){
detail::stack_construct<Archive, U> u(ar, item_version);
ar >> boost::serialization::make_nvp("item", u.reference());
t.push_back(u.reference());
ar.reset_object_address(& t.back() , & u.reference());
}
}
}
template<class Archive, class U, class Allocator>
inline void load(
Archive & ar,
std::vector<U, Allocator> &t,
const unsigned int file_version,
mpl::false_
){
const boost::archive::library_version_type library_version(
ar.get_library_version()
);
// retrieve number of elements
item_version_type item_version(0);
collection_size_type count;
ar >> BOOST_SERIALIZATION_NVP(count);
sehe_bugfix::load_elements(ar, t, file_version, count, detail::is_default_constructible<U>());
}
And it works.
SUMMARY
Sadly I don't have time now to investigate on the map<> situation. But I suspect that things are similar. The documented solution should still work. And it might still be broken.
I'll report the above issue later today to the issue tracker of boost.
I hope the answer helps you find the/a solution none-the-less
I am compiling a c++ code on a linux ubuntu with g++4.8.1 and boost 1.55.0.
My program uses a class A, which has a member table which is an unsigned long long array. The same class has other members which are simple int. I am using boost to serialize my data.
My code work and compile all fine, if I serialize all but the table in A.
However it does not compile if I try to serialize table. I get the following error:
/usr/local/include/boost/serialization/access.hpp: In instantiation of ‘static void boost::serialization::access::serialize(Archive&, T&, unsigned int) [with Archive = boost::archive::text_oarchive; T = long long unsigned int]’:
/usr/local/include/boost/serialization/access.hpp: In instantiation of ‘static void boost::serialization::access::serialize(Archive&, T&, unsigned int) [with Archive = boost::archive::text_oarchive; T = long long unsigned int]’:
/usr/local/include/boost/serialization/serialization.hpp:69:69: required from ‘void boost::serialization::serialize(Archive&, T&, unsigned int) [with Archive = boost::archive::text_oarchive; T = long long unsigned int]’
/usr/local/include/boost/serialization/serialization.hpp:128:27: required from ‘void boost::serialization::serialize_adl(Archive&, T&, unsigned int) [with Archive = boost::archive::text_oarchive; T = long long unsigned int]’
/usr/local/include/boost/archive/detail/oserializer.hpp:152:5: required from ‘void boost::archive::detail::oserializer<Archive, T>::save_object_data(boost::archive::detail::basic_oarchive&, const void*) const [with Archive = boost::archive::text_oarchive; T = long long unsigned int]’
/usr/local/include/boost/archive/detail/oserializer.hpp:101:1: required from ‘class boost::archive::detail::oserializer<boost::archive::text_oarchive, long long unsigned int>’
/usr/local/include/boost/archive/detail/oserializer.hpp:214:5: required from ‘boost::archive::detail::pointer_oserializer<Archive, T>::pointer_oserializer() [with Archive = boost::archive::text_oarchive; T = long long unsigned int]’
/usr/local/include/boost/serialization/singleton.hpp:106:7: [ skipping 95 instantiation contexts, use -ftemplate-backtrace-limit=0 to disable ]
/usr/local/include/boost/archive/detail/oserializer.hpp:314:44: required from ‘static void boost::archive::detail::save_non_pointer_type<Archive>::invoke(Archive&, T&) [with T = Metapop; Archive = boost::archive::text_oarchive]’
/usr/local/include/boost/archive/detail/oserializer.hpp:525:24: required from ‘void boost::archive::save(Archive&, T&) [with Archive = boost::archive::text_oarchive; T = Metapop]’
/usr/local/include/boost/archive/detail/common_oarchive.hpp:69:40: required from ‘void boost::archive::detail::common_oarchive<Archive>::save_override(T&, int) [with T = Metapop; Archive = boost::archive::text_oarchive]’
/usr/local/include/boost/archive/basic_text_oarchive.hpp:80:9: required from ‘void boost::archive::basic_text_oarchive<Archive>::save_override(T&, int) [with T = Metapop; Archive = boost::archive::text_oarchive]’
/usr/local/include/boost/archive/detail/interface_oarchive.hpp:63:9: required from ‘Archive& boost::archive::detail::interface_oarchive<Archive>::operator<<(T&) [with T = Metapop; Archive = boost::archive::text_oarchive]’
simulation.cpp:1403:9: required from here
/usr/local/include/boost/serialization/access.hpp:118:9: error: request for member ‘serialize’ in ‘t’, which is of non-class type ‘long long unsigned int’
t.serialize(ar, file_version);
^
I have read aroud that if I used vectors or/and an other data type it would work. However it is critical for me (for speed) to use a raw array of unsigned long long. Any idea ?
Thnaks a lot for you help !
Serializing unsigned long long arrays works for me using gcc 4.7.2 with boost 1.49, gcc 4.2.1 with boost 1.55, and clang 3.4 with boost 1.55:
#include <sstream>
#include <boost/archive/text_iarchive.hpp>
#include <boost/archive/text_oarchive.hpp>
#include <boost/serialization/access.hpp>
#include <boost/version.hpp>
struct Foo {
unsigned long long bar[3];
template<class Archive>
void serialize(Archive& ar, const unsigned int /*version*/) {
ar & bar;
}
};
std::ostream& operator<<(std::ostream& os, const Foo& foo) {
return os << foo.bar[0] << ' ' << foo.bar[1] << ' ' << foo.bar[2];
}
int main() {
std::cout << "Boost version " << BOOST_LIB_VERSION << '\n';
Foo before;
before.bar[0] = 0;
before.bar[1] = 1;
before.bar[2] = 2;
std::cout << "before: " << before << '\n';
std::ostringstream os;
{
boost::archive::text_oarchive oa(os);
oa << before;
}
Foo after;
{
std::istringstream is(os.str());
boost::archive::text_iarchive ia(is);
ia >> after;
}
std::cout << "after: " << after << '\n';
return 0;
}
Here's gcc 4.8 with boost 1.55 on Coliru, also works.
If you are using a pointer to an allocated array, then I think that is your problem. I don't believe you can serialize a bare pointer to a primitive, and I'm sure you can't serialize a bare pointer to an array of primitives because there is no way for serialization to know how many elements a pointer points to.
I would use a std::vector over an allocated array because there is no speed disadvantage in doing so. However, if you really want to allocate your own array then you can serialize it with the boost::serialization::make_array() wrapper like this:
#include <iostream>
#include <boost/archive/text_oarchive.hpp>
#include <boost/serialization/access.hpp>
#include <boost/serialization/array.hpp>
struct Foo {
size_t dataSize;
unsigned long long *data;
Foo()
: dataSize(3)
, data(new unsigned long long[dataSize]) {
}
~Foo() {
delete[] data;
}
// TODO: Production code should disallow default copy constructor
// and assignment operator.
template<class Archive>
void serialize(Archive& ar, const unsigned int /*version*/) {
ar & dataSize;
ar & boost::serialization::make_array(data, dataSize);
}
};
int main() {
Foo foo;
foo.data[0] = 0;
foo.data[1] = 1;
foo.data[2] = 2;
boost::archive::text_oarchive oa(std::cout);
oa << foo;
return 0;
}
It turns out that this question was not about unsigned long long at all, but is essentially a duplicate of boost serialization, deserialization of raw C arrays.
I am using boost serialization, mostly the intrusive version. For a template Matrix class I would like to have the non-intrusive version which works on Visual Studio with the following code:
namespace boost
{
namespace serialization
{
template<class Archive, int R, int C, class ElementType>
void serialize(Archive & ar, Matrix<R, C, ElementType> & m, const unsigned int version)
{
ar & ...
}
}
}
int R, int C are the row and columns, ElementType is double or float.
However, this does not work on GCC 4.3.2 with the error
error: 'class Matrix<1u, 3u, double>' has no member named 'serialize'
If I use a special form like
namespace boost
{
namespace serialization
{
template<class Archive>
void serialize(Archive & ar, Matrix<3,1,double> & m, const unsigned int version)
{
ar & ...
}
}
}
it compiles on GCC, but of course only for a special set of template arguments.
What can I do to make it work on both compilers for all R, C and ElementType?
EDIT: These are the lines causing the error:
/[myfolder]/lib/BOOST/1_44_0/include/boost/serialization/access.hpp: In static member function 'static void boost::serialization::access::serialize(Archive&, T&, unsigned int) [with Archive = boost::archive::binary_iarchive, T = Matrix<3u, 1u, double>]':
/[myfolder]/lib/BOOST/1_44_0/include/boost/serialization/serialization.hpp:70: instantiated from 'void boost::serialization::serialize(Archive&, T&, unsigned int) [with Archive = boost::archive::binary_iarchive, T = Matrix<3u, 1u, double>]'
/[myfolder]/lib/BOOST/1_44_0/include/boost/serialization/serialization.hpp:129: instantiated from 'void boost::serialization::serialize_adl(Archive&, T&, unsigned int) [with Archive = boost::archive::binary_iarchive, T = Matrix<3u, 1u, double>]'
/[myfolder]/lib/BOOST/1_44_0/include/boost/archive/detail/iserializer.hpp:182: instantiated from 'void boost::archive::detail::iserializer<Archive, T>::load_object_data(boost::archive::detail::basic_iarchive&, void*, unsigned int) const [with Archive = boost::archive::binary_iarchive, T = Matrix<3u, 1u, double>]'
It looks like a signed/unsigned mismatch to me. Your template function is declared with ints but the error indicates that the object which it's trying to match with the template has parameters 1u and 3u. When you instantiate the object that you're trying to serialize, are you using unsigned values for the dimensions? Try changing your serialize template function to take unsigneds or instantiating your Matrix with ints.