Unsigned long long serialization in boost - c++

I am compiling a c++ code on a linux ubuntu with g++4.8.1 and boost 1.55.0.
My program uses a class A, which has a member table which is an unsigned long long array. The same class has other members which are simple int. I am using boost to serialize my data.
My code work and compile all fine, if I serialize all but the table in A.
However it does not compile if I try to serialize table. I get the following error:
/usr/local/include/boost/serialization/access.hpp: In instantiation of ‘static void boost::serialization::access::serialize(Archive&, T&, unsigned int) [with Archive = boost::archive::text_oarchive; T = long long unsigned int]’:
/usr/local/include/boost/serialization/access.hpp: In instantiation of ‘static void boost::serialization::access::serialize(Archive&, T&, unsigned int) [with Archive = boost::archive::text_oarchive; T = long long unsigned int]’:
/usr/local/include/boost/serialization/serialization.hpp:69:69: required from ‘void boost::serialization::serialize(Archive&, T&, unsigned int) [with Archive = boost::archive::text_oarchive; T = long long unsigned int]’
/usr/local/include/boost/serialization/serialization.hpp:128:27: required from ‘void boost::serialization::serialize_adl(Archive&, T&, unsigned int) [with Archive = boost::archive::text_oarchive; T = long long unsigned int]’
/usr/local/include/boost/archive/detail/oserializer.hpp:152:5: required from ‘void boost::archive::detail::oserializer<Archive, T>::save_object_data(boost::archive::detail::basic_oarchive&, const void*) const [with Archive = boost::archive::text_oarchive; T = long long unsigned int]’
/usr/local/include/boost/archive/detail/oserializer.hpp:101:1: required from ‘class boost::archive::detail::oserializer<boost::archive::text_oarchive, long long unsigned int>’
/usr/local/include/boost/archive/detail/oserializer.hpp:214:5: required from ‘boost::archive::detail::pointer_oserializer<Archive, T>::pointer_oserializer() [with Archive = boost::archive::text_oarchive; T = long long unsigned int]’
/usr/local/include/boost/serialization/singleton.hpp:106:7: [ skipping 95 instantiation contexts, use -ftemplate-backtrace-limit=0 to disable ]
/usr/local/include/boost/archive/detail/oserializer.hpp:314:44: required from ‘static void boost::archive::detail::save_non_pointer_type<Archive>::invoke(Archive&, T&) [with T = Metapop; Archive = boost::archive::text_oarchive]’
/usr/local/include/boost/archive/detail/oserializer.hpp:525:24: required from ‘void boost::archive::save(Archive&, T&) [with Archive = boost::archive::text_oarchive; T = Metapop]’
/usr/local/include/boost/archive/detail/common_oarchive.hpp:69:40: required from ‘void boost::archive::detail::common_oarchive<Archive>::save_override(T&, int) [with T = Metapop; Archive = boost::archive::text_oarchive]’
/usr/local/include/boost/archive/basic_text_oarchive.hpp:80:9: required from ‘void boost::archive::basic_text_oarchive<Archive>::save_override(T&, int) [with T = Metapop; Archive = boost::archive::text_oarchive]’
/usr/local/include/boost/archive/detail/interface_oarchive.hpp:63:9: required from ‘Archive& boost::archive::detail::interface_oarchive<Archive>::operator<<(T&) [with T = Metapop; Archive = boost::archive::text_oarchive]’
simulation.cpp:1403:9: required from here
/usr/local/include/boost/serialization/access.hpp:118:9: error: request for member ‘serialize’ in ‘t’, which is of non-class type ‘long long unsigned int’
t.serialize(ar, file_version);
^
I have read aroud that if I used vectors or/and an other data type it would work. However it is critical for me (for speed) to use a raw array of unsigned long long. Any idea ?
Thnaks a lot for you help !

Serializing unsigned long long arrays works for me using gcc 4.7.2 with boost 1.49, gcc 4.2.1 with boost 1.55, and clang 3.4 with boost 1.55:
#include <sstream>
#include <boost/archive/text_iarchive.hpp>
#include <boost/archive/text_oarchive.hpp>
#include <boost/serialization/access.hpp>
#include <boost/version.hpp>
struct Foo {
unsigned long long bar[3];
template<class Archive>
void serialize(Archive& ar, const unsigned int /*version*/) {
ar & bar;
}
};
std::ostream& operator<<(std::ostream& os, const Foo& foo) {
return os << foo.bar[0] << ' ' << foo.bar[1] << ' ' << foo.bar[2];
}
int main() {
std::cout << "Boost version " << BOOST_LIB_VERSION << '\n';
Foo before;
before.bar[0] = 0;
before.bar[1] = 1;
before.bar[2] = 2;
std::cout << "before: " << before << '\n';
std::ostringstream os;
{
boost::archive::text_oarchive oa(os);
oa << before;
}
Foo after;
{
std::istringstream is(os.str());
boost::archive::text_iarchive ia(is);
ia >> after;
}
std::cout << "after: " << after << '\n';
return 0;
}
Here's gcc 4.8 with boost 1.55 on Coliru, also works.
If you are using a pointer to an allocated array, then I think that is your problem. I don't believe you can serialize a bare pointer to a primitive, and I'm sure you can't serialize a bare pointer to an array of primitives because there is no way for serialization to know how many elements a pointer points to.
I would use a std::vector over an allocated array because there is no speed disadvantage in doing so. However, if you really want to allocate your own array then you can serialize it with the boost::serialization::make_array() wrapper like this:
#include <iostream>
#include <boost/archive/text_oarchive.hpp>
#include <boost/serialization/access.hpp>
#include <boost/serialization/array.hpp>
struct Foo {
size_t dataSize;
unsigned long long *data;
Foo()
: dataSize(3)
, data(new unsigned long long[dataSize]) {
}
~Foo() {
delete[] data;
}
// TODO: Production code should disallow default copy constructor
// and assignment operator.
template<class Archive>
void serialize(Archive& ar, const unsigned int /*version*/) {
ar & dataSize;
ar & boost::serialization::make_array(data, dataSize);
}
};
int main() {
Foo foo;
foo.data[0] = 0;
foo.data[1] = 1;
foo.data[2] = 2;
boost::archive::text_oarchive oa(std::cout);
oa << foo;
return 0;
}
It turns out that this question was not about unsigned long long at all, but is essentially a duplicate of boost serialization, deserialization of raw C arrays.

Related

I'm getting an error when trying to compile this constructor

I have a class with a constructor but the compilation fails with the error:
error: use of deleted function 'std::atomic<long unsigned int>::atomic(const std::atomic<long unsigned int>&)'
I'm struggling to understand what this error means. I've been trying to update some code from using tbb::atomic to using std::atomic, and this error wasn't appearing before with tbb::atomic.
The bit of code that upsets the compiler is:
#include <boost/aligned_storage.hpp>
template <typename T>
class Holder<T,true>
{
public:
//! Construct a value (there mustn't be one already)
template <typename ARG>
void Construct(const ARG &arg) { new(Raw()) T(arg); }
private:
typename boost::aligned_storage<sizeof(T), boost::alignment_of<T>::value>::type impl;
void *Raw() { return &impl; }
};
Specifically the line:
void Construct(const ARG &arg) { new(Raw()) T(arg); }
I'd be very grateful if anyone could help me understand what is going on.
EDIT:
Full error message:
holder.hpp:75:34: error: use of deleted function 'std::atomic<long unsigned int>::atomic(const std::atomic<long unsigned int>&)'
75 | void Construct(const T &arg) { new(Raw()) T(arg); }
| ^~~~~~~~~~~~~~~~~
In file included from /include/TPtools/boost/smart_ptr/detail/sp_counted_base_std_atomic.hpp:19,
from /include/TPtools/boost/smart_ptr/detail/sp_counted_base.hpp:48,
from /include/TPtools/boost/smart_ptr/detail/shared_count.hpp:29,
from /include/TPtools/boost/smart_ptr/shared_ptr.hpp:28,
from /include/TPtools/boost/shared_ptr.hpp:17,
from movegate.cpp:14:
/pkgsData/gcc-v9.3.0p1/Linux/RHEL6.0-2013-x86_64/include/c++/9.3.0/atomic:824:7: note: declared here
824 | atomic(const atomic&) = delete;
| ^~~~~~
holder.hpp: In instantiation of 'void Holder<T, true>::Construct(const T&) [with T = std::atomic<long unsigned int>; T = std::atomic<long unsigned int>]':
element.hpp:185:9: required from 'void Element<PRESENT, ABSENT>::Accessor<PRESENCE>::MakePresentImpl(const ARG&) const [with ARG = std::atomic<long unsigned int>; PRESENCE = Cluster<Element<std::atomic<long unsigned int>, Link>, true>::NonLockingPresence; PRESENT = std::atomic<long unsigned int>; ABSENT = Link]'
element.hpp:119:7: required from 'Element<PRESENT, ABSENT>::PresentResult Element<PRESENT, ABSENT>::Accessor<PRESENCE>::MakePresent(const ARG&) const [with ARG = std::atomic<long unsigned int>; PRESENCE = Cluster<Element<std::atomic<long unsigned int>, Link>, true>::NonLockingPresence; PRESENT = std::atomic<long unsigned int>; ABSENT = Link; Element<PRESENT, ABSENT>::PresentResult = std::atomic<long unsigned int>&]'
dispenser.hpp:78:5: required from 'Index Dispenser<T>::Insert(const T&) [with T = std::atomic<long unsigned int>]'

C++17 Cannot use std::bind to produce a std::function

I have a Register function which takes a std::function<void(const uint8_t* data, size_t len)> as a parameter. I want to use the member function of an object as the target.
I found this question according to which the answer is to use std::bind to bind the first first argument (the implicit this pointer) to the actual object pointer and then use it as the std::function argument.
This however doesn't work anymore in neither C++11, C++14 nor C++17?
Consider the following test program.
#include <iostream>
#include <cstdint>
#include <functional>
void Register(std::function<void(const uint8_t* data, size_t len)> func) {
//Dummy - directly call into function
func(nullptr, 0);
}
class TestClass {
public:
void TestRegister() {
Register(
std::bind(&TestClass::TestTarget, this, std::placeholders::_1)
);
}
void TestTarget(const uint8_t* data, size_t len) {
(void) data;
(void) len;
std::cout << "Hello there" << std::endl;
}
};
int main() {
TestClass testObj;
testObj.TestRegister();
return 0;
}
When compiling for -std=c++17 this throws a rather cryptic error message (I have no idea what it's trying to say here with Wrong number of arguments for pointer-to-member):
In file included from /home/max/Documents/TestingFunctions/main.cpp:3:0:
/usr/include/c++/7/functional: In instantiation of ‘struct std::_Bind_check_arity<void (TestClass::*)(const unsigned char*, long unsigned int), TestClass*, const std::_Placeholder<1>&>’:
/usr/include/c++/7/functional:854:12: required from ‘struct std::_Bind_helper<false, void (TestClass::*)(const unsigned char*, long unsigned int), TestClass*, const std::_Placeholder<1>&>’
/usr/include/c++/7/functional:875:5: required by substitution of ‘template<class _Func, class ... _BoundArgs> typename std::_Bind_helper<std::__is_socketlike<_Func>::value, _Func, _BoundArgs ...>::type std::bind(_Func&&, _BoundArgs&& ...) [with _Func = void (TestClass::*)(const unsigned char*, long unsigned int); _BoundArgs = {TestClass*, const std::_Placeholder<1>&}]’
/home/max/Documents/TestingFunctions/main.cpp:14:78: required from here
/usr/include/c++/7/functional:841:7: error: static assertion failed: Wrong number of arguments for pointer-to-member
static_assert(_Varargs::value
^~~~~~~~~~~~~
/home/max/Documents/TestingFunctions/main.cpp: In member function ‘void TestClass::TestRegister()’:
/home/max/Documents/TestingFunctions/main.cpp:14:26: error: could not convert ‘std::bind(_Func&&, _BoundArgs&& ...) [with _Func = void (TestClass::*)(const unsigned char*, long unsigned int); _BoundArgs = {TestClass*, const std::_Placeholder<1>&}; typename std::_Bind_helper<std::__is_socketlike<_Func>::value, _Func, _BoundArgs ...>::type = std::_Bind<void (TestClass::*(TestClass*, std::_Placeholder<1>))(const unsigned char*, long unsigned int)>](((TestClass*)this), std::placeholders::_1)’ from ‘std::_Bind_helper<false, void (TestClass::*)(const unsigned char*, long unsigned int), TestClass*, const std::_Placeholder<1>&>::type {aka std::_Bind<void (TestClass::*(TestClass*, std::_Placeholder<1>))(const unsigned char*, long unsigned int)>}’ to ‘std::function<void(const unsigned char*, long unsigned int)>’
std::bind(&TestClass::TestTarget, this, std::placeholders::_1)
~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Replacing the TestRegister function with one that does the exact same thing in a lambda expression compiles and runs without problems.
void TestRegister() {
Register(
[this](const uint8_t* data, size_t len) {
TestTarget(data, len);
}
);
}
Question: Why does the std::bind approach from the linked question not work? Was this feature removed or do I have an error in my code?
Your function Register expects a function with two parameters, but you try to pass to it a function with one placeholded parameter.
void TestRegister() {
Register(
std::bind(&TestClass::TestTarget, this, std::placeholders::_1, std::placeholders::_2)
);
}
Your function takes two parameters, while you are only passing one placeholder.
std::bind(&TestClass::TestTarget, this, std::placeholders::_1, std::placeholders::_2);

How to serialize boost::dynamic_bitset?

How to serialize a class with a boost::dynamic_bitset member?
#include <boost/archive/text_oarchive.hpp>
#include <boost/archive/text_iarchive.hpp>
#include <boost/dynamic_bitset.hpp>
#include <boost/serialization/bitset.hpp>
#include <sstream>
class A
{
friend class boost::serialization::access;
boost::dynamic_bitset<> x;
template<class Archive>
void serialize(Archive & ar, const unsigned int){
ar & x;
}
};
int main()
{
A a;
std::stringstream ss;
boost::archive::text_oarchive oa(ss);
oa << a;
return 0;
}
Compiling gives an error (boost 1.57)
In file included from /usr/include/boost/serialization/extended_type_info_typeid.hpp:37:0,
from /usr/include/boost/archive/detail/oserializer.hpp:38,
from /usr/include/boost/archive/detail/interface_oarchive.hpp:23,
from /usr/include/boost/archive/detail/common_oarchive.hpp:22,
from /usr/include/boost/archive/basic_text_oarchive.hpp:32,
from /usr/include/boost/archive/text_oarchive.hpp:31,
from dynamic_bitset_setial.cpp:1:
/usr/include/boost/serialization/access.hpp: In static member function ‘static void boost::serialization::access::serialize(Archive&, T&, unsigned int) [with Archive = boost::archive::text_oarchive, T = boost::dynamic_bitset<>]’:
/usr/include/boost/serialization/serialization.hpp:69:5: instantiated from ‘void boost::serialization::serialize(Archive&, T&, unsigned int) [with Archive = boost::archive::text_oarchive, T = boost::dynamic_bitset<>]’
/usr/include/boost/serialization/serialization.hpp:128:9: instantiated from ‘void boost::serialization::serialize_adl(Archive&, T&, unsigned int) [with Archive = boost::archive::text_oarchive, T = boost::dynamic_bitset<>]’
/usr/include/boost/archive/detail/oserializer.hpp:148:5: instantiated from ‘void boost::archive::detail::oserializer<Archive, T>::save_object_data(boost::archive::detail::basic_oarchive&, const void*) const [with Archive = boost::archive::text_oarchive, T = boost::dynamic_bitset<>]’
dynamic_bitset_setial.cpp:25:1: instantiated from here
/usr/include/boost/serialization/access.hpp:118:9: error: ‘class boost::dynamic_bitset<>’ has no member named ‘serialize’
I went ahead and filed the pull request to add Serialization support to Boost Dynamic Bitset
Serialization using the public interface isn't optimal as to_block_range()/from_block_range() require copying of m_bits (and subsequent resize()).
I added a generic implementation to Boost Dynamic Bitset. The changes merge cleanly against develop or master (1_58_0).
Changes
Implementation added with
minimal intrusiveness, only a nested friend (class serialization_impl;) has been forward declared to "key-hole" the required friend access through
This class, as well as the actual ADL hook for Boost Serialization are implemented in a separate header (dynamic_bitset/serialization.hpp, similar to other boost libraries with serialization support).
This means that zero dependencies on Boost Serialization stuff exists unless boost/dynamic_bitset/serialization.hpp is actually included
Zero copy is achieved (leveraging std::vector<Block>'s builtin support in Boost Serialization)
Tests
The second commit in the pull request adds tests for this feature. I'm not sure how to add the dyn_bitset_unit_tests5.cpp to the Jamfile. I suppose something else must be done to ensure linking to Boost System and Boost Serialization. I have run the tests myself using a simple wrapper:
#include <modular-boost/libs/dynamic_bitset/dyn_bitset_unit_tests5.cpp>
int main() {
test_main(0, nullptr);
}
Which can then be compiled and run with e.g.
g++ main.cpp -lboost_system -lboost_serialization && ./a.out
No output means no errors.
dynamic_bitset<> is not serializable, as you've found out (std::bitset<N> is different type).
Not to worry, though, you can add it without too much effort:
namespace boost { namespace serialization {
template <typename Ar, typename Block, typename Alloc>
void save(Ar& ar, dynamic_bitset<Block, Alloc> const& bs, unsigned) {
size_t num_bits = bs.size();
std::vector<Block> blocks(bs.num_blocks());
to_block_range(bs, blocks.begin());
ar & num_bits & blocks;
}
template <typename Ar, typename Block, typename Alloc>
void load(Ar& ar, dynamic_bitset<Block, Alloc>& bs, unsigned) {
size_t num_bits;
std::vector<Block> blocks;
ar & num_bits & blocks;
bs.resize(num_bits);
from_block_range(blocks.begin(), blocks.end(), bs);
bs.resize(num_bits);
}
template <typename Ar, typename Block, typename Alloc>
void serialize(Ar& ar, dynamic_bitset<Block, Alloc>& bs, unsigned version) {
split_free(ar, bs, version);
}
} }
This works e.g. Live On Coliru
int main() {
A a;
for (int i=0; i<128; ++i)
a.x.resize(11*i, i%2);
std::stringstream ss;
{
boost::archive::text_oarchive oa(ss);
oa << a;
}
std::cout << ss.str();
{
boost::archive::text_iarchive ia(ss);
A b;
ia >> b;
assert(a.x == b.x);
}
}
Note that if you can't afford to copy the blocks vector, it's equally easy to add serialization directly on the m_bits level, but that requires intrusive changes (friend access required at a minimum).
Such a thing would easily be added to boost in a pull request.
Update added that pull request

boost::serialization: object with private default constructor works in a vector, but not in a map

Consider the following code:
#include <boost/serialization/nvp.hpp>
#include <boost/archive/xml_iarchive.hpp>
#include <boost/archive/xml_oarchive.hpp>
class Foo{
friend class boost::serialization::access;
template<class Archive>
void serialize(Archive & ar, const unsigned int)
{
ar & BOOST_SERIALIZATION_NVP(i);
}
int i;
Foo():i(0){}
public:
Foo(int k):i(k){}
};
int main(int argc, char *argv[])
{
std::vector< Foo> f;
f.push_back(Foo(12));
std::ofstream os("path");
boost::archive::xml_oarchive oa(os);
oa << boost::serialization::make_nvp("f", f);
os.close();
std::vector<Foo> g;
std::ifstream is("path");
boost::archive::xml_iarchive ia(is);
ia >> boost::serialization::make_nvp("f", g);
}
Which works fine when serializing a vector of Foos. However, if I try to serialize a map of Foos, it fails on the private default constructor:
std::map<std::string, Foo> f;
f.insert(std::make_pair("hello", Foo(12)));
std::ofstream os("path");
boost::archive::xml_oarchive oa(os);
oa << boost::serialization::make_nvp("f", f);
os.close();
std::map<std::string, Foo> g;
std::ifstream is("path");
boost::archive::xml_iarchive ia(is);
ia >> boost::serialization::make_nvp("f", g);
fails with
In file included from main.cpp:2:
In file included from /usr/local/include/boost/serialization/nvp.hpp:19:
In file included from /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/utility:70:
/usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/bits/stl_pair.h:109:18: error: field of type 'Foo' has private default constructor
: first(), second() { }
^
/usr/local/include/boost/serialization/access.hpp:132:17: note: in instantiation of member function 'std::pair<const std::basic_string<char>, Foo>::pair' requested here
::new(t)T;
^
/usr/local/include/boost/serialization/serialization.hpp:93:13: note: in instantiation of function template specialization 'boost::serialization::access::construct<std::pair<const std::basic_string<char>, Foo> >' requested here
access::construct(t);
^
/usr/local/include/boost/serialization/serialization.hpp:158:9: note: in instantiation of function template specialization 'boost::serialization::load_construct_data<boost::archive::xml_iarchive, std::pair<const std::basic_string<char>, Foo> >' requested here
load_construct_data(ar, t, v);
^
/usr/local/include/boost/serialization/detail/stack_constructor.hpp:58:31: note: in instantiation of function template specialization 'boost::serialization::load_construct_data_adl<boost::archive::xml_iarchive, std::pair<const std::basic_string<char>, Foo> >' requested here
boost::serialization::load_construct_data_adl(
^
/usr/local/include/boost/serialization/collections_load_imp.hpp:83:48: note: in instantiation of member function 'boost::serialization::detail::stack_construct<boost::archive::xml_iarchive, std::pair<const std::basic_string<char>, Foo> >::stack_construct' requested here
detail::stack_construct<Archive, type> t(ar, v);
^
/usr/local/include/boost/serialization/collections_load_imp.hpp:158:16: note: (skipping 12 contexts in backtrace; use -ftemplate-backtrace-limit=0 to see all)
hint = ifunc(ar, s, item_version, hint);
^
/usr/local/include/boost/archive/detail/common_iarchive.hpp:66:18: note: in instantiation of function template specialization 'boost::archive::load<boost::archive::xml_iarchive, std::map<std::basic_string<char>, Foo, std::less<std::basic_string<char> >, std::allocator<std::pair<const std::basic_string<char>, Foo> > > >' requested here
archive::load(* this->This(), t);
^
/usr/local/include/boost/archive/basic_xml_iarchive.hpp:86:39: note: in instantiation of function template specialization 'boost::archive::detail::common_iarchive<boost::archive::xml_iarchive>::load_override<std::map<std::basic_string<char>, Foo, std::less<std::basic_string<char> >, std::allocator<std::pair<const std::basic_string<char>, Foo> > > >' requested here
this->detail_common_iarchive::load_override(t.value(), 0);
^
/usr/local/include/boost/archive/xml_iarchive.hpp:93:38: note: in instantiation of function template specialization 'boost::archive::basic_xml_iarchive<boost::archive::xml_iarchive>::load_override<std::map<std::basic_string<char>, Foo, std::less<std::basic_string<char> >, std::allocator<std::pair<const std::basic_string<char>, Foo> > > >' requested here
basic_xml_iarchive<Archive>::load_override(t, 0);
^
/usr/local/include/boost/archive/detail/interface_iarchive.hpp:60:23: note: in instantiation of function template specialization 'boost::archive::xml_iarchive_impl<boost::archive::xml_iarchive>::load_override<const boost::serialization::nvp<std::map<std::basic_string<char>, Foo, std::less<std::basic_string<char> >, std::allocator<std::pair<const std::basic_string<char>, Foo> > > > >' requested here
this->This()->load_override(t, 0);
^
main.cpp:50:8: note: in instantiation of function template specialization 'boost::archive::detail::interface_iarchive<boost::archive::xml_iarchive>::operator>><const boost::serialization::nvp<std::map<std::basic_string<char>, Foo, std::less<std::basic_string<char> >, std::allocator<std::pair<const std::basic_string<char>, Foo> > > > >' requested here
ia >> boost::serialization::make_nvp("f", g);
^
main.cpp:34:5: note: implicitly declared private here
Foo():i(0){}
^
I'm using clang
Ubuntu clang version 3.4-1ubuntu3 (tags/RELEASE_34/final) (based on LLVM 3.4)
and boost version 1.55 as shipped with Ubuntu 14.04LTS.
I have tried providing the load_construct_data() function as follows:
namespace boost
{
namespace serialization
{
template<class Archive>
inline void load_construct_data(Archive &archive, Foo*a, unsigned int
file_version)
{
::new(a)Foo(0);
}
}
}
but I'm still getting the same error, because it requires the constructor when instantiating a std::pair
Oh. Aha.
I just used Boost 1.57.0 to compare the situation with map<string, Foo>.
Well, you're in luck. You've found another library version dependency (likely a bug).
Not using that, but providing the private default constructor, GCC 4.8.2 compiles it just fine: Live On Coliru [1]
GCC 4.9.0 fails to compile it though (it uses a newer version of the standard library too). The std::pair<> default constructor fails to compile there, since Foo is not default constructible: Live On Coliru
Solution
Luckily the solution with save_construct_data/load_construct_data saves the day, again.
However, you need to cater for the fact that the element type is actually not Foo, but std::pair<T const, Foo>.
template <class Archive, typename K> inline friend void save_construct_data(Archive& ar, std::pair<K, Foo> const* v, const unsigned int) {
std::cerr << __PRETTY_FUNCTION__ << "\n";
ar & boost::serialization::make_nvp("first", v->first);
ar & boost::serialization::make_nvp("second", v->second.i);
}
template <class Archive, typename K> inline friend void load_construct_data(Archive& ar, std::pair<K, Foo>* v, const unsigned int) {
std::cerr << __PRETTY_FUNCTION__ << "\n";
typename std::remove_cv<K>::type first;
ar & boost::serialization::make_nvp("first", first);
int tmp;
ar & boost::serialization::make_nvp("second", tmp);
new(v) std::pair<K, Foo>(first, tmp);
}
Now it all works:
Live On Coliru
#include <boost/serialization/nvp.hpp>
#include <boost/serialization/serialization.hpp>
#include <boost/archive/xml_iarchive.hpp>
#include <boost/archive/xml_oarchive.hpp>
#include <boost/serialization/access.hpp>
#include <boost/serialization/vector.hpp>
#include <boost/serialization/map.hpp>
#include <boost/version.hpp>
#include <fstream>
#include <iostream>
class Foo {
friend class boost::serialization::access;
template <class Archive> void serialize(Archive &, const unsigned int) {
std::cerr << __PRETTY_FUNCTION__ << "\n";
}
template <class Archive, typename K> inline friend void save_construct_data(Archive& ar, std::pair<K, Foo> const* v, const unsigned int) {
std::cerr << __PRETTY_FUNCTION__ << "\n";
ar & boost::serialization::make_nvp("first", v->first);
ar & boost::serialization::make_nvp("second", v->second.i);
}
template <class Archive, typename K> inline friend void load_construct_data(Archive& ar, std::pair<K, Foo>* v, const unsigned int) {
std::cerr << __PRETTY_FUNCTION__ << "\n";
typename std::remove_cv<K>::type first;
ar & boost::serialization::make_nvp("first", first);
int tmp;
ar & boost::serialization::make_nvp("second", tmp);
new(v) std::pair<K, Foo>(first, tmp);
}
int i;
public:
Foo(int k) : i(k) {}
friend std::ostream& operator<<(std::ostream& os, Foo const& foo) {
return os << "Foo { " << foo.i << " }";
}
};
namespace boost { namespace serialization {
} }
int main() {
using Data = std::map<std::string, Foo>;
std::cout << "Boost version: " << BOOST_VERSION << "\n";
{
auto f = Data { {"a", 12 }, {"b", 42} };
//for (auto& e : f) std::cout << e.first << ", " << e.second << "\n";
std::ofstream os("path");
boost::archive::xml_oarchive oa(os);
oa << boost::serialization::make_nvp("f", f);
}
{
Data g;
std::ifstream is("path");
boost::archive::xml_iarchive ia(is);
ia >> boost::serialization::make_nvp("f", g);
for (auto& e : g)
std::cout << e.first << ", " << e.second << "\n";
}
}
Which prints:
Boost version: 105700
void save_construct_data(Archive&, const std::pair<K, Foo>*, unsigned int) [with Archive = boost::archive::xml_oarchive; K = const std::__cxx11::basic_string<char>]
void Foo::serialize(Archive&, unsigned int) [with Archive = boost::archive::xml_oarchive]
void save_construct_data(Archive&, const std::pair<K, Foo>*, unsigned int) [with Archive = boost::archive::xml_oarchive; K = const std::__cxx11::basic_string<char>]
void Foo::serialize(Archive&, unsigned int) [with Archive = boost::archive::xml_oarchive]
void load_construct_data(Archive&, std::pair<K, Foo>*, unsigned int) [with Archive = boost::archive::xml_iarchive; K = const std::__cxx11::basic_string<char>]
void Foo::serialize(Archive&, unsigned int) [with Archive = boost::archive::xml_iarchive]
void load_construct_data(Archive&, std::pair<K, Foo>*, unsigned int) [with Archive = boost::archive::xml_iarchive; K = const std::__cxx11::basic_string<char>]
void Foo::serialize(Archive&, unsigned int) [with Archive = boost::archive::xml_iarchive]
a, Foo { 12 }
b, Foo { 42 }
[1] (I can't link it on Coliru because the boost libraries there have been recompiled to the GCC 5.0 ABI)
Notes
A better, general, solution would be to do the load/save_construct_data trick generically for non-default-constructible types in the boost::serialization namespace. That way, people won't have to "know" about the std::pair<> implementation detail. They could just implement load/save_construct_data for their own user-types and it would JustWork™ whether they put it in a vector or a map.
Implementing that generically is less-than-trivial though, and might interfere with some other machineries internal to the Boost Serialization framework.
I'll prefer to get some help of the Boost Serialization maintainers to do that in a reliable way. So, it seems I'll be submitting two tickets today.
It doesn't necessarily work for vector either. Vector deserialization resizes the vector to the required size first. This requires the elements to be default-constructible.
Note how this is only a problem because
construction is not accessed via the serialization::access friend "token"
the class is default constructible
The Documented Solution
The documentation tells you to use save_construct_data and load_construct_data for types that are not default-constructible.
Specifically they promise it will work for STL containers of these as well:
In addition to the deserialization of pointers, these overrides are used in the deserialization of STL containers whose element type has no default constructor.
In practice, this works well in v1.57.0:
Live On Coliru
But in 1.58.0 it isn't true...
A Bug In Boost 1.58.0
Version 1.58.0 seems to have broken this:
The code seems to do required check (from serialization/vector.hpp in the unoptimized version of load(...)):
if(detail::is_default_constructible<U>()){
t.resize(count);
// ... snip ...
}
else{
t.reserve(count);
// ... snip ...
}
However, this does the check at runtime. The method will statically refuse to compile. Oops.
The Fix
Instead of having the branches in the same flow code, it should be dispatched so that only the applicable branch is instantiated. I tested with this simplistic approach:
namespace sehe_bugfix {
template<class Archive, class U, class Allocator>
inline void load_elements(
Archive & ar,
std::vector<U, Allocator> &t,
const unsigned int /* file_version */,
collection_size_type count,
mpl::true_
){
const boost::archive::library_version_type library_version(
ar.get_library_version()
);
item_version_type item_version(0);
if(boost::archive::library_version_type(3) < library_version){
ar >> BOOST_SERIALIZATION_NVP(item_version);
}
t.resize(count);
typename std::vector<U, Allocator>::iterator hint;
hint = t.begin();
while(count-- > 0){
ar >> boost::serialization::make_nvp("item", *hint++);
}
}
template<class Archive, class U, class Allocator>
inline void load_elements(
Archive & ar,
std::vector<U, Allocator> &t,
const unsigned int /* file_version */,
collection_size_type count,
mpl::false_
){
const boost::archive::library_version_type library_version(
ar.get_library_version()
);
item_version_type item_version(0);
if(boost::archive::library_version_type(3) < library_version){
ar >> BOOST_SERIALIZATION_NVP(item_version);
}
t.reserve(count);
while(count-- > 0){
detail::stack_construct<Archive, U> u(ar, item_version);
ar >> boost::serialization::make_nvp("item", u.reference());
t.push_back(u.reference());
ar.reset_object_address(& t.back() , & u.reference());
}
}
}
template<class Archive, class U, class Allocator>
inline void load(
Archive & ar,
std::vector<U, Allocator> &t,
const unsigned int file_version,
mpl::false_
){
const boost::archive::library_version_type library_version(
ar.get_library_version()
);
// retrieve number of elements
item_version_type item_version(0);
collection_size_type count;
ar >> BOOST_SERIALIZATION_NVP(count);
sehe_bugfix::load_elements(ar, t, file_version, count, detail::is_default_constructible<U>());
}
And it works.
SUMMARY
Sadly I don't have time now to investigate on the map<> situation. But I suspect that things are similar. The documented solution should still work. And it might still be broken.
I'll report the above issue later today to the issue tracker of boost.
I hope the answer helps you find the/a solution none-the-less

Non-intrusive serialize method for template class

I am using boost serialization, mostly the intrusive version. For a template Matrix class I would like to have the non-intrusive version which works on Visual Studio with the following code:
namespace boost
{
namespace serialization
{
template<class Archive, int R, int C, class ElementType>
void serialize(Archive & ar, Matrix<R, C, ElementType> & m, const unsigned int version)
{
ar & ...
}
}
}
int R, int C are the row and columns, ElementType is double or float.
However, this does not work on GCC 4.3.2 with the error
error: 'class Matrix<1u, 3u, double>' has no member named 'serialize'
If I use a special form like
namespace boost
{
namespace serialization
{
template<class Archive>
void serialize(Archive & ar, Matrix<3,1,double> & m, const unsigned int version)
{
ar & ...
}
}
}
it compiles on GCC, but of course only for a special set of template arguments.
What can I do to make it work on both compilers for all R, C and ElementType?
EDIT: These are the lines causing the error:
/[myfolder]/lib/BOOST/1_44_0/include/boost/serialization/access.hpp: In static member function 'static void boost::serialization::access::serialize(Archive&, T&, unsigned int) [with Archive = boost::archive::binary_iarchive, T = Matrix<3u, 1u, double>]':
/[myfolder]/lib/BOOST/1_44_0/include/boost/serialization/serialization.hpp:70: instantiated from 'void boost::serialization::serialize(Archive&, T&, unsigned int) [with Archive = boost::archive::binary_iarchive, T = Matrix<3u, 1u, double>]'
/[myfolder]/lib/BOOST/1_44_0/include/boost/serialization/serialization.hpp:129: instantiated from 'void boost::serialization::serialize_adl(Archive&, T&, unsigned int) [with Archive = boost::archive::binary_iarchive, T = Matrix<3u, 1u, double>]'
/[myfolder]/lib/BOOST/1_44_0/include/boost/archive/detail/iserializer.hpp:182: instantiated from 'void boost::archive::detail::iserializer<Archive, T>::load_object_data(boost::archive::detail::basic_iarchive&, void*, unsigned int) const [with Archive = boost::archive::binary_iarchive, T = Matrix<3u, 1u, double>]'
It looks like a signed/unsigned mismatch to me. Your template function is declared with ints but the error indicates that the object which it's trying to match with the template has parameters 1u and 3u. When you instantiate the object that you're trying to serialize, are you using unsigned values for the dimensions? Try changing your serialize template function to take unsigneds or instantiating your Matrix with ints.