Serialization of non default constructible and std::reference wrapper alternatives - c++

I tried serialising my (neural) network and am currently stuck-ish.
The issue seems to be that you can't serialize a std::reference_wrapper. I am unsure whether I should either change the way the references to the upper nodes are stored or come up with a way to serialize those.
Are there alternatives to reference_wrappers, which I neglected and still avoid c style pointers? (which are to be avoided as far as i know)
#include <iostream>
#include <fstream>
#include <list>
#include <vector>
#include <functional>
#include <boost/archive/tmpdir.hpp>
#include <boost/archive/text_iarchive.hpp>
#include <boost/archive/text_oarchive.hpp>
#include <boost/serialization/base_object.hpp>
#include <boost/serialization/utility.hpp>
#include <boost/serialization/list.hpp>
#include <boost/serialization/assume_abstract.hpp>
#include <boost/serialization/shared_ptr.hpp>
#include <boost/serialization/vector.hpp>
typedef float nodeValueType;
typedef std::pair<std::vector<nodeValueType>, std::vector<nodeValueType>> Example;
typedef std::list<Example> ExampleList;
class Node;
class Link
{
public:
Link() = delete;
Link(Node& upperNode, Node& lowerNode)
: Link(upperNode, lowerNode, 1.0e-3f * (std::rand() / (nodeValueType) RAND_MAX))
{
}
Link(Node& upperNode, Node& lowerNode, nodeValueType weight)
: weight_(weight), upperNode_(upperNode), lowerNode_(lowerNode)
{
}
Link(const Link&) = delete;
Link& operator=(const Link&) = delete;
nodeValueType weight_;
std::reference_wrapper<Node> upperNode_;
std::reference_wrapper<Node> lowerNode_;
friend class boost::serialization::access;
template<class Archive>
void serialize(Archive& ar, const unsigned int version){
ar & this->weight_;
ar & this->upperNode_;
ar & this->lowerNode_;
}
};
class Node
{
public:
Node();
// making it hard to copy us since we really never want to move.
// we are referred in loads of pointers
// therefore moving us invalidates all of them TODO invalidation scheme?
Node(const Node&) = delete;
Node& operator=(const Node&) = delete;
void linkTo(Node& other)
{
assert(this->lowerLinks_.max_size() > (this->lowerLinks_.size() + 1) * 2);
// Link creation
this->lowerLinks_.push_back(std::shared_ptr<Link>(new Link(*this, other)));
other.upperLinks_.push_back(std::shared_ptr<Link>(this->lowerLinks_.back()));
}
std::vector<std::shared_ptr<Link>> lowerLinks_;
std::vector<std::shared_ptr<Link>> upperLinks_;
// serialization
friend class boost::serialization::access;
template<class Archive>
void serialize(Archive& ar, const unsigned int version){
ar & this->lowerLinks_;
}
};
int main()
{
Node n1;
Node n2;
n2.linkTo(n1);
std::string filename(boost::archive::tmpdir());
filename += "/demofile.txt";
std::ofstream ofs(filename);
boost::archive::text_oarchive oa(ofs);
oa << n1 << n2;
Node n3,n4;
// open the archive
std::ifstream ifs(filename);
boost::archive::text_iarchive ia(ifs);
// restore the schedule from the archive
ia >> n3 >> n4;
return 0;
}

There are 2 problems to overcome.
The first is that std::reference_wrapper cannot be null, so you might want to consider a 'nullable reference' type:
template<class T>
struct NullableReference
{
constexpr NullableReference() : ptr_(nullptr) {}
constexpr NullableReference(T& ref) : ptr_(std::addressof(ref)) {}
constexpr auto operator*() const -> T& { assert(!empty()); return *ptr_; }
constexpr operator T&() const { assert(!empty()); return *ptr_; }
constexpr bool empty() const { return !ptr_; }
template<class Archive>
void serialize(Archive& ar, unsigned int version)
{
ar & ptr_;
}
private:
T* ptr_;
};
The other is that Link does not have a default constructor (no doubt because of the non-nullable reference wrappers).
For this reason you may want to consider custom handling of the constructor when deserialising a Link (this is covered in the boost docs IIRC).
Of course, now that you're using a NullableReference, you can allow the default constructor for Link:
...
Link() : weight_(), upperNode_(), lowerNode_() {};
nodeValueType weight_;
NullableReference<Node> upperNode_;
NullableReference<Node> lowerNode_;
....

Related

why my boost serialization failed with input stream error?

I have a complex class, which need to serialization.
I use boost.serialization.
The class is a singleton class, with several function. singleton is not the vital part, because i have tried some simple code with simple singleton class, it worked well.
// static_player.h
#pragma once
#include <map>
#include <unordered_set>
#include <vector>
#include <memory>
#include <boost/archive/text_oarchive.hpp>
#include <boost/archive/text_iarchive.hpp>
#include <boost/serialization/vector.hpp>
#include <boost/serialization/unordered_map.hpp>
#include <boost/serialization/unordered_set.hpp>
#include <boost/serialization/map.hpp>
#include "util/file_util.h"
#include "struct/struct_util.hpp"
namespace util:trade_util {
typedef std::unordered_map<std::string, std::unordered_map<std::string, std::unordered_map<std::string, double>>> NTKD; // map of name_ticker_key_value
typedef std::unordered_map<std::string, std::unordered_map<std::string, std::unordered_map<std::string, std::string>>> NTKS; // map of name_ticker_key_value
typedef std::unordered_map<std::string, std::map<int32_t, std::unordered_map<std::string, std::unordered_map<std::string, double>> > > NDTKD;
typedef std::unordered_map<std::string, std::map<int32_t, std::unordered_map<std::string, std::unordered_map<std::string, std::string>> > > NDTKS;
class StaticPlayer {
private:
StaticPlayer();
StaticPlayer(const StaticPlayer&) = delete;
StaticPlayer(const StaticPlayer&&) = delete;
virtual ~StaticPlayer();
class DM {
public:
DM() {}
DM(NTKD * numeric_data, NTKS * string_data) : numeric_data_(numeric_data), string_data_(string_data) {}
virtual ~DM() = default;
inline double get_numeric(const std::string & name, const std::string& ticker, const std::string& key) const;
inline std::string get_string(const std::string & name, const std::string& ticker, const std::string& key) const;
private:
NTKD* numeric_data_;
NTKS* string_data_;
friend class boost::serialization::access;
template<class Archive>
void serialize(Archive & ar, const unsigned int version) {
ar & numeric_data_;
ar & string_data_;
}
};
public:
static StaticPlayer& Inst() { static StaticPlayer um; return um; }
double query_numeric(const std::string&name, const std::string&ticker, int32_t date, const std::string& key);
std::string query_string(const std::string&name, const std::string&ticker, int32_t date, const std::string& key);
void operator=(const StaticPlayer&) = delete;
DM* operator[](int32_t date);
void load_config(const std::set<std::string> & fset);
void load_config(bool simple = false);
private:
double special_query(const std::string&ticker, int32_t date, const std::string& key);
private:
bool loaded_ = false;
std::map<int32_t, DM*> umap_;
// std::mutex mut_;
NDTKD numeric_data_; // name : date : ticker : key : value
NDTKS string_data_; // name : date : ticker : key : value
std::unordered_map<std::string, std::string> fill_na_method_; // name : fill_na
std::map<int32_t, std::unordered_map<std::string, std::unordered_map<int32_t, std::string> >> rank_map_;
std::unordered_set<std::string> tickers_, chains_;
friend class boost::serialization::access;
template<class Archive>
void serialize(Archive & ar, const unsigned int version) {
ar & loaded_; ar & umap_; ar & numeric_data_;
ar & string_data_; ar & fill_na_method_;
ar & rank_map_; ar & tickers_; ar & chains_;
// ar & mut_;
}
};
}
the main.cpp is:
#include "./static_player.h"
void write_univ() {
auto & u = util::trade_util::StaticPlayer::Inst();
u.load_config();
std::ofstream ofs("a");
boost::archive::text_oarchive oa(ofs);
oa << u;
std::ifstream ifs("a");
boost::archive::text_iarchive ia(ifs);
util::trade_util::StaticPlayer& u2 = (util::trade_util::StaticPlayer::Inst());
ia >> u2;
for (const auto & ticker : u2.get_tickers()) cout << ticker << endl;
}
int main() {
write_univ();
}
the error message is:
terminate called after throwing an instance of 'boost::archive::archive_exception'
what(): input stream error
[1] 20341 abort (core dumped) ./a.out
I have read all i can find on the Internet, but still can't figure out why.
Could you help on this?
Oof, serializing from/into a singleton is probably not a good idea to start with. Why not make load return an instance, that you can then choose to replace the singleton with if you must?
The real problem is a classic pitfall with serialization: if you put serialization/deserialization in the same scope, make sure your archive is completed (and written to disk) before trying to load it back...:
void write_univ()
{
{
std::ofstream ofs("a");
boost::archive::text_oarchive oa(ofs);
auto& u = util::trade_util::StaticPlayer::Inst();
u.load_config();
oa << u;
}
{
std::ifstream ifs("a");
boost::archive::text_iarchive ia(ifs);
util::trade_util::StaticPlayer& u2 =
util::trade_util::StaticPlayer::Inst();
ia >> u2;
}
}
That's all.
Side Notes
Also, maybe you should try to tidy up the type "spaghetti" surrounding the maps:
template <typename T> using Dict = std::unordered_map<std::string, T>;
template <typename T> using NTK = std::unordered_map<std::string, Dict<T>>;
template <typename T> using NDTK = std::unordered_map<std::string, std::map<int32_t, Dict<T>>>;
using NTKD = NTK<double>;
using NTKS = NTK<std::string>;
using NDTKD = NDTK<double>;
using NDTKS = NDTK<std::string>;
And this then tells me you're basically having straight records like
struct Record {
std::string name;
int32_t id;
variant<double, string> value;
};
Why don't you consider using a single Multi-Index container with two indexes - or perhaps even just the one composed-key index by (name,id) that you can query with a partial key (id unspecified) if you prefer?
See e.g. this similar example Using boost multi index like relational DB
Self-Contained Test:
LiveUndead On Coliru
// static_player.h
//#pragma once
#include <boost/archive/text_iarchive.hpp>
#include <boost/archive/text_oarchive.hpp>
#include <boost/serialization/map.hpp>
#include <boost/serialization/unordered_map.hpp>
#include <boost/serialization/unordered_set.hpp>
#include <boost/serialization/vector.hpp>
#include <fstream>
#include <memory>
#include <set>
#include <vector>
//#include "util/file_util.h"
//#include "struct/struct_util.hpp"
namespace util::trade_util {
template <typename T> using Dict = std::unordered_map<std::string, T>;
template <typename T> using NTK = std::unordered_map<std::string, Dict<T>>;
template <typename T> using NDTK = std::unordered_map<std::string, std::map<int32_t, Dict<T>>>;
using NTKD = NTK<double>;
using NTKS = NTK<std::string>;
using NDTKD = NDTK<double>;
using NDTKS = NDTK<std::string>;
class StaticPlayer {
private:
StaticPlayer() = default;
StaticPlayer(const StaticPlayer&) = delete;
StaticPlayer(const StaticPlayer&&) = delete;
virtual ~StaticPlayer() = default;
class DM {
public:
DM() = default;
DM(NTKD* numeric_data, NTKS* string_data)
: numeric_data_(numeric_data)
, string_data_(string_data)
{
}
virtual ~DM() = default;
[[nodiscard]] inline double
get_numeric(const std::string& name, const std::string& ticker,
const std::string& key) const;
[[nodiscard]] inline std::string
get_string(const std::string& name, const std::string& ticker,
const std::string& key) const;
private:
NTKD* numeric_data_ = nullptr;
NTKS* string_data_ = nullptr;
friend class boost::serialization::access;
template <class Archive>
void serialize(Archive& ar, const unsigned int version)
{
ar& numeric_data_;
ar& string_data_;
}
};
public:
static StaticPlayer& Inst()
{
static StaticPlayer um;
return um;
}
double query_numeric(const std::string& name, const std::string& ticker,
int32_t date, const std::string& key);
std::string query_string(const std::string& name,
const std::string& ticker, int32_t date,
const std::string& key);
void operator=(const StaticPlayer&) = delete;
DM* operator[](int32_t date);
void load_config(const std::set<std::string>& fset);
void load_config(bool simple = false) {}
private:
double special_query(const std::string& ticker, int32_t date,
const std::string& key);
bool loaded_ = false;
std::map<int32_t, DM*> umap_;
// std::mutex mut_;
NDTKD numeric_data_; // name : date : ticker : key : value
NDTKS string_data_; // name : date : ticker : key : value
std::unordered_map<std::string, std::string>
fill_na_method_; // name : fill_na
std::map<int32_t,
std::unordered_map<std::string,
std::unordered_map<int32_t, std::string>>>
rank_map_;
std::unordered_set<std::string> tickers_, chains_;
friend class boost::serialization::access;
template <class Archive>
void serialize(Archive& ar, const unsigned int version)
{
ar& loaded_;
ar& umap_;
ar& numeric_data_;
ar& string_data_;
ar& fill_na_method_;
ar& rank_map_;
ar& tickers_;
ar& chains_;
// ar & mut_;
}
};
} // namespace util::trade_util
//#include "./static_player.h"
void write_univ()
{
{
std::ofstream ofs("a");
boost::archive::text_oarchive oa(ofs);
auto& u = util::trade_util::StaticPlayer::Inst();
u.load_config();
oa << u;
}
{
std::ifstream ifs("a");
boost::archive::text_iarchive ia(ifs);
util::trade_util::StaticPlayer& u2 =
util::trade_util::StaticPlayer::Inst();
ia >> u2;
}
// for (const auto& ticker : u2.get_tickers())
// std::cout << ticker << std::endl;
}
int main() { write_univ(); }
Which works as expected. On my machine creates an archive a containing:
22 serialization::archive 19 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 1 0

How can boost::serialization be used with std::unique_ptr from C++

I am trying to use boost::serialization library to serialize a class (the class A in following code) which is including a std::unique_ptr member.
Environment:
OS: Windows 10 1909
IDE: Microsoft Visual Studio Community 2019 Version 16.4.4
Library: boost-serialization 1.72.0 (installed with vcpkg tool)
#include <boost/archive/binary_oarchive.hpp>
#include <boost/archive/binary_iarchive.hpp>
#include <boost/archive/text_oarchive.hpp>
#include <boost/archive/text_iarchive.hpp>
#include <boost/serialization/export.hpp>
#include <boost/serialization/list.hpp>
#include <boost/serialization/nvp.hpp>
#include <boost/serialization/split_free.hpp>
#include <boost/serialization/unique_ptr.hpp>
#include <boost/serialization/vector.hpp>
#include <fstream>
#include <iostream>
#include <memory>
using namespace std;
using namespace boost;
class A
{
public:
A()
{
}
A(int input_size, int input_value) // Constructor
{
this->data = std::make_unique<int[]>(input_size);
this->size = input_size;
for (int loop_number = 0; loop_number < size; loop_number++) {
data[loop_number] = input_value;
}
}
std::unique_ptr<int[]> get_data()
{
// deep copy
auto return_data = std::make_unique<int[]>(size);
for (int loop_number = 0; loop_number < size; loop_number++) {
return_data[loop_number] = data[loop_number];
}
return return_data;
}
int get_size()
{
return this->size;
}
~A()
{
}
private:
std::unique_ptr<int[]> data;
int size;
friend class boost::serialization::access;
template<class Archive>
void serialize(Archive& ar, const unsigned int version)
{
ar& data;
ar& size;
}
};
int main()
{
// create and open a character archive for output
std::ofstream ofs("filename");
// create class instance
const A a_object(10, 5);
// save data to archive
{
boost::archive::text_oarchive oa(ofs);
// write class instance to archive
oa << a_object;
// archive and stream closed when destructors are called
}
// ... some time later restore the class instance to its orginal state
A load_from_file;
{
// create and open an archive for input
std::ifstream ifs("filename");
boost::archive::text_iarchive ia(ifs);
// read class state from archive
ia >> load_from_file;
// archive and stream closed when destructors are called
}
return 0;
}
The compiler throws C2440 error and the error message is 'initializing': cannot convert from 'int *' to 'const T (*const )'. It refers to %includePath%\boost\serialization\unique_ptr.hpp
template<class Archive, class T>
inline void save(
Archive & ar,
const std::unique_ptr< T > &t,
const unsigned int /*file_version*/
){
// only the raw pointer has to be saved
// the ref count is rebuilt automatically on load
const T * const tx = t.get();
ar << BOOST_SERIALIZATION_NVP(tx);
}
It seems that std::unique_ptr still serialize unsuccessfully in boost-serialization library (version 1.72.0)?
Moreover, is there any solution to serialize a class with std::unique_ptr members?
Based on the fact that std::unique_ptr doesn't stored size, ar& data; in the "serialize" template function which is used for saving/loading data members can't be serialized due to the count of data is unknown in this line.
template<class Archive>
void serialize(Archive& ar, const unsigned int version)
{
ar& data; // <== the count of data is unknown
ar& size;
}
The iterating process (as shown as the following code) of smart pointer "data" is needed in order to access all valid data.
for (int i = 0; i < size; ++i)
ar& data[i];
Therefore, the save part is like below.
template<class Archive>
void save(Archive& ar, const unsigned int version) const
{
ar& size;
for (int i = 0; i < size; ++i)
ar& data[i];
}
When it comes to load part, it is important to allocate space of smart pointer before loading data to avoid Access Violation Error
template<class Archive>
void load(Archive& ar, const unsigned int version)
{
ar& size;
data = std::make_unique<int[]>(size); // <== allocate space of smart pointer first
for (int i = 0; i < size; ++i)
ar& data[i];
}
Note: The macro BOOST_SERIALIZATION_SPLIT_MEMBER() need to be put because save/load function is split.
Finally, the serialization part in class A could be done as the following code.
friend class boost::serialization::access;
template<class Archive>
void save(Archive& ar, const unsigned int version) const
{
ar& size;
for (int i = 0; i < size; ++i)
ar& data[i];
}
template<class Archive>
void load(Archive& ar, const unsigned int version)
{
ar& size;
data = std::make_unique<int[]>(size);
for (int i = 0; i < size; ++i)
ar& data[i];
}
BOOST_SERIALIZATION_SPLIT_MEMBER()
Reference: Boost serialization tutorial
Depending of the version of Boost you are using, you may need to implement your own non-intrusive adapter, something like this:
namespace boost {
namespace serialization {
template<class Archive, class T>
inline void save( Archive & ar, const std::unique_ptr< T > &t, const unsigned int /*file_version*/){
// only the raw pointer has to be saved
const T * const base_pointer = t.get();
ar & BOOST_SERIALIZATION_NVP(base_pointer);
}
template<class Archive, class T>
inline void load(Archive & ar,std::unique_ptr< T > &t, const unsigned int /*file_version*/){
T *base_pointer;
ar & BOOST_SERIALIZATION_NVP(base_pointer);
t.reset(base_pointer);
}
template<class Archive, class T>
inline void serialize(Archive & ar,std::unique_ptr< T > &t, const unsigned int file_version){
boost::serialization::split_free(ar, t, file_version);
}
} // namespace serialization
} // namespace boost
See this for a complete example.

How to implement Boost::Serialize for Boost::Nested_Container

(Followup of another question.)
Boost::Serialize often delivers an exception on oarchive, complaining that re-creating a particular object would result in duplicate objects. Some archives save and re-load successfully, but many result in the error above. I have not been able yet to determine the exact conditions under which the error occurs, but I have proven that none of the content used to populate the nested_container and the flat object list contains duplicate object IDs. I am using text archive, not binary. Here is how I have modified the code for nested_container and also for another, separate flat object list in order to do Boost::Serialize:
struct obj
{
int id;
const obj * parent = nullptr;
obj()
:id(-1)
{ }
obj(int object)
:id(object)
{ }
int getObjId() const
{
return id;
}
bool operator==(obj obj2)
{
if (this->getObjId() == obj2.getObjId())
return true;
else
return false;
}
#if 1
private:
friend class boost::serialization::access;
friend std::ostream & operator<<(std::ostream &os, const obj &obj);
template<class Archive>
void serialize(Archive &ar, const unsigned int file_version)
{
ar & id & parent;
}
#endif
};
struct subtree_obj
{
const obj & obj_;
subtree_obj(const obj & ob)
:obj_(ob)
{ }
#if 1
private:
friend class boost::serialization::access;
friend std::ostream & operator<<(std::ostream &os, const subtree_obj &obj);
template<class Archive>
void serialize(Archive &ar, const unsigned int file_version)
{
ar & obj_;
}
#endif
};
struct path
{
int id;
const path *next = nullptr;
path(int ID, const path *nex)
:id(ID), next(nex)
{ }
path(int ID)
:id(ID)
{ }
#if 1
private:
friend class boost::serialization::access;
friend std::ostream & operator<<(std::ostream &os, const path &pathe);
template<class Archive>
void serialize(Archive &ar, const unsigned int file_version)
{
ar & id & next;
}
#endif
};
struct subtree_path
{
const path & path_;
subtree_path(const path & path)
:path_(path)
{ }
#if 1
private:
friend class boost::serialization::access;
friend std::ostream & operator<<(std::ostream &os, const subtree_path &pathe);
template<class Archive>
void serialize(Archive &ar, const unsigned int file_version)
{
ar & path_;
}
#endif
};
//
// My flattened object list
//
struct HMIObj
{
int objId;
std::string objType;
HMIObj()
:objId(-1), objType("")
{ }
bool operator==(HMIObj obj2)
{
if (this->getObjId() == obj2.getObjId())
&& this->getObjType() == obj2.getObjType())
return true;
else
return false;
}
int getObjId() const
{
return objId;
}
std::string getObjType() const
{
return objType;
}
#if 1
private:
friend class boost::serialization::access;
friend std::ostream & operator<<(std::ostream &os, const HMIObj &obj);
template<class Archive>
void serialize(Archive &ar, const unsigned int file_version)
{
ar & objId & objType;
}
#endif
};
The problem you're experiencing is most likely due to, again, the particular order in which elements are traversed in index #0 (the hashed one). For instance, if we populate the container like this:
nested_container c;
c.insert({54});
auto it=c.insert({0}).first;
insert_under(c,it,{1});
Then the elements are listed in index #0 as (1, 54, 0). The crucial problem here is that 1 is a child of 0: when loading elements in the same order as they were saved, the first one is then 1, but this needs 0 to be loaded before in order to properly point to it. This is what Boost.Serialization very smartly detects and complains about. Such child-before-parent situations depend on the very unpredictable way elements are sorted in a hashed index, which is why you see the problem just sometimes.
You have two simple solutions:
Swap indices #0 and #1 in the definition of your nested container: as index #1 sorting order is the tree preorder, it is guaranteed that parents get processed before their children.
Override the serialization code of the nested container so as to go through index #1:
template<class Archive>
void serialize(Archive& ar,nested_container& c,unsigned int)
{
if constexpr(Archive::is_saving::value){
boost::serialization::stl::save_collection(ar,c.get<1>());
}
else{
boost::serialization::load_set_collection(ar,c.get<1>());
}
}
Complete demo code for solution #2 follows:
Live On Coliru
#include <boost/multi_index_container.hpp>
#include <boost/multi_index/hashed_index.hpp>
#include <boost/multi_index/ordered_index.hpp>
#include <boost/multi_index/identity.hpp>
#include <boost/multi_index/member.hpp>
#include <iterator>
struct obj
{
int id;
const obj* parent=nullptr;
};
namespace boost{
namespace serialization{
template<class Archive>
void serialize(Archive& ar,obj& x,unsigned int)
{
ar&x.id&x.parent;
}
}} /* namespace boost::serialization */
struct subtree_obj
{
const obj& obj_;
};
struct path
{
int id;
const path* next=nullptr;
};
struct subtree_path
{
const path& path_;
};
inline bool operator<(const path& x,const path& y)
{
if(x.id<y.id)return true;
else if(y.id<x.id)return false;
else if(!x.next) return y.next;
else if(!y.next) return false;
else return *(x.next)<*(y.next);
}
inline bool operator<(const subtree_path& sx,const path& y)
{
const path& x=sx.path_;
if(x.id<y.id)return true;
else if(y.id<x.id)return false;
else if(!x.next) return false;
else if(!y.next) return false;
else return subtree_path{*(x.next)}<*(y.next);
}
inline bool operator<(const path& x,const subtree_path& sy)
{
return x<sy.path_;
}
struct obj_less
{
private:
template<typename F>
static auto apply_to_path(const obj& x,F f)
{
return apply_to_path(x.parent,path{x.id},f);
}
template<typename F>
static auto apply_to_path(const obj* px,const path& x,F f)
->decltype(f(x))
{
return !px?f(x):apply_to_path(px->parent,{px->id,&x},f);
}
public:
bool operator()(const obj& x,const obj& y)const
{
return apply_to_path(x,[&](const path& x){
return apply_to_path(y,[&](const path& y){
return x<y;
});
});
}
bool operator()(const subtree_obj& x,const obj& y)const
{
return apply_to_path(x.obj_,[&](const path& x){
return apply_to_path(y,[&](const path& y){
return subtree_path{x}<y;
});
});
}
bool operator()(const obj& x,const subtree_obj& y)const
{
return apply_to_path(x,[&](const path& x){
return apply_to_path(y.obj_,[&](const path& y){
return x<subtree_path{y};
});
});
}
};
using namespace boost::multi_index;
using nested_container=multi_index_container<
obj,
indexed_by<
hashed_unique<member<obj,int,&obj::id>>,
ordered_unique<identity<obj>,obj_less>
>
>;
#if 1 /* set to 0 to trigger pointer conflict exception */
#include <boost/serialization/set.hpp>
namespace boost{
namespace serialization{
template<class Archive>
void serialize(Archive& ar,nested_container& c,unsigned int)
{
if constexpr(Archive::is_saving::value){
boost::serialization::stl::save_collection(ar,c.get<1>());
}
else{
boost::serialization::load_set_collection(ar,c.get<1>());
}
}
}} /* namespace boost::serialization */
#endif
template<typename Iterator>
inline auto insert_under(nested_container& c,Iterator it,obj x)
{
x.parent=&*it;
return c.insert(std::move(x));
}
#include <boost/archive/text_iarchive.hpp>
#include <boost/archive/text_oarchive.hpp>
#include <iostream>
#include <sstream>
void print(const nested_container& c)
{
for(const obj& x:c){
std::cout<<"("<<x.id;
if(x.parent)std::cout<<"->"<<x.parent->id;
std::cout<<")";
}
std::cout<<"\n";
}
int main()
{
nested_container c;
c.insert({54});
auto it=c.insert({0}).first;
insert_under(c,it,{1});
print(c);
std::ostringstream oss;
boost::archive::text_oarchive oa(oss);
oa<<c;
nested_container c2;
std::istringstream iss(oss.str());
boost::archive::text_iarchive ia(iss);
ia>>c2;
print(c2);
}
By the way, why are you providing serialize functions for subtree_obj, path and subtree_path? You don't need that to serialize nested_containers.

How to serialize unique_ptr<char[]> in struct

I haven't been able to find any docs on how to serialize a unique_ptr to an array. Any help would be great.
struct Counter{
int index;
unique_ptr<char []> name;
template<class Archive>
void serialize(Archive & archive){
archive(index, name ); // serialize things by passing them to the archive
}
};
How it is assigned.
auto buffer = std::unique_ptr<char[]>(new char[BUFFER_SIZE]);
instance.name = std::move(buffer);
You can do it, but it needs some extra work. It differs depending on the type of the archive.
For text-based archives (e.g., XMLOutputArchive/XMLInputArchive and JSONOutputArchive/JSONInputArchive) you can use the saveBinaryValue() / loadBinaryValue() (http://uscilab.github.io/cereal/assets/doxygen/classcereal_1_1JSONOutputArchive.html).
Here's a complete example:
#include <iostream>
#include <memory>
#include <cereal/archives/xml.hpp>
#include <cereal/cereal.hpp>
struct Counter
{
static constexpr std::size_t BUFFER_SIZE = 12;
int index{};
std::unique_ptr<char[]> name;
Counter() = default;
Counter(int i)
: index{i},
name{new char[BUFFER_SIZE]}
{}
template<class Archive>
void load(Archive& archive)
{
archive(index);
name.reset(new char[BUFFER_SIZE]);
archive.loadBinaryValue(name.get(), BUFFER_SIZE * sizeof(decltype(name[0])));
}
template<class Archive>
void save(Archive& archive) const
{
archive(index);
archive.saveBinaryValue(name.get(), BUFFER_SIZE * sizeof(decltype(name[0])));
}
};
int main()
{
cereal::XMLOutputArchive archive(std::cout);
Counter c(42);
archive(CEREAL_NVP(c));
}
If you are using BinaryOutputArchive / BinaryInputArchive or PortableBinaryOutputArchive / PortableBinaryInputArchive the functions become saveBinary() and loadBinary() (http://uscilab.github.io/cereal/assets/doxygen/classcereal_1_1PortableBinaryOutputArchive.html).
For those, you can also wrap your array using binary_data():
template<class Archive>
void save(Archive& archive) const
{
archive(index, cereal::binary_data(name.get(), BUFFER_SIZE));
}

Is there a way to serialize iterators using boost serialization?

In my project I have a class containing and std::list and in another class I maintain an iterator pointing to a place in the middle of that list.
I can successfully serialize the list, but the iterator member variable is causing problems. Here is a program to reproduce:
#include <boost/serialization/list.hpp>
#include <boost/archive/text_oarchive.hpp>
#include <boost/archive/text_iarchive.hpp>
#include <iostream>
#include <fstream>
class A
{
public:
A(){}
A(const std::list<int> & _i) : i(_i) {}
virtual ~A(){}
std::list<int> i;
void display() {
std::cout << "i:";
for (const int j : i)
std::cout << " " << j;
std::cout << std::endl;
}
private:
friend class boost::serialization::access;
//friend std::ostream & operator<<(std::ostream &os, const A &a);
template<class Archive>
void serialize(Archive &ar, const unsigned int version)
{
ar & i;
}
};
class Stepper
{
public:
Stepper() {}
Stepper(const A& a)
: p(a.i.size()>0 ? a.i.begin() : a.i.end()) {}
std::list<int>::const_iterator p;
void display() {
std::cout << "p: " << *p << std::endl;
}
void step() { p++; }
private:
friend class boost::serialization::access;
//friend std::ostream & operator<<(std::ostream &os, const A &a);
template<class Archive>
void serialize(Archive &ar, const unsigned int version)
{
ar & p;
}
};
int main()
{
{
A a({5,6,7});
Stepper sa(a);
a.display();
sa.display();
sa.step();
sa.display();
std::ofstream ofs( "a.txt" );
boost::archive::text_oarchive ar(ofs);
ar & a;
ar & sa;
}
A b;
Stepper sb;
{
std::ifstream ifs( "a.txt" );
boost::archive::text_iarchive ar(ifs);
ar & b;
ar & sb;
}
b.display();
sb.display();
return 0;
}
In this program, the class A can be serialized without problems. (Remove the ar&sa stuff..) But unfortunately when trying to serialize the class containing the iterator (the exact code above), I get the following compilation errors:
[..snip..]
testser.cpp:72:10: required from here /usr/include/boost/serialization/access.hpp:116:11:
error: ‘struct std::_List_const_iterator<int>’ has no member named ‘serialize’
t.serialize(ar, file_version);
~~^~~~~~~~~
[..snip..]
testser.cpp:81:10: required from here /usr/include/boost/serialization/access.hpp:116:11:
error: ‘struct std::_List_const_iterator<int>’ has no member named ‘serialize’
So, it seems that boost/serialization/list.hpp does not support iterators. And yet, as far as I can tell, it's totally legitimate to keep an iterator to a list item somewhere, as they cannot be invalidated unless erased. Is there a way to serialize this iterator using boost? Do I need to write a custom function? Do I have to return a custom iterator from my std::list? (That sounds particularly ugly..)
Thanks for any insight.
Okay it seems the only way to do this is to split the serialization into save and load, and calculate the iterator's position in the list. This works as long as the iterator is valid. Unfortunately it means needing to add a pointer to the list to the structure, which I didn't want, but actually in my application I can access this so it is not a problem for me.
class Stepper
{
public:
Stepper() {}
Stepper(const A& _a)
: a(&_a), p(a->i.size()>0 ? a->i.begin() : a->i.end()) {}
const A* a;
std::list<int>::const_iterator p;
void display() {
std::cout << "p: " << *p << std::endl;
}
void step() { p++; }
private:
friend class boost::serialization::access;
template<class Archive>
void save(Archive &ar, const unsigned int version) const
{
int d = std::distance(a->i.begin(), p);
ar & a;
ar & d;
}
template<class Archive>
void load(Archive &ar, const unsigned int version)
{
int d;
ar & a;
ar & d;
p = a->i.begin();
for (; d>0; --d)
p++;
}
BOOST_SERIALIZATION_SPLIT_MEMBER()
};