I have a complex network of objects connected with QSharedPointers and QWeakPointers. Is there a simple way to save/load them with Boost.Serialization? So far I have this:
namespace boost {
namespace serialization {
template<class Archive, class T>
void save(Archive& ar, QSharedPointer<T> const& ptr, unsigned version) {
T* sharedPointer = ptr.data();
ar & BOOST_SERIALIZATION_NVP(sharedPointer);
}
template<class Archive, class T>
void load(Archive& ar, QSharedPointer<T>& ptr, unsigned version) {
T* sharedPointer = 0;
ar & BOOST_SERIALIZATION_NVP(sharedPointer);
ptr = QSharedPointer<T>(sharedPointer);
}
template<class Archive, class T>
void save(Archive& ar, QWeakPointer<T> const& ptr, unsigned version) {
T* weakPointer = ptr.toStrongRef().data();
ar & BOOST_SERIALIZATION_NVP(weakPointer);
}
template<class Archive, class T>
void load(Archive& ar, QWeakPointer<T>& ptr, unsigned version) {
T* weakPointer = 0;
ar & BOOST_SERIALIZATION_NVP(weakPointer);
ptr = QSharedPointer<T>(weakPointer);
}
}
}
This is not working because the shared pointers are always constructed from raw pointers so they all think the reference count is 1. It also immediately frees weak pointers.
With some effort I can convert the classes to use boost::shared_ptr and boost::weak_ptr. Will that do any good?
The question is what do you really want to achieve by serializing pointers? What is your expected output? Note that pointers point to a place in memory -- several may point to the same place in memory.
Serializing the address won't work. You can't just write down the exact memory address, because there's no way to guarantee that objects on the next run will be able to take the same space (another program may have already reserved that place).
Serializing the pointed object in each place where we have a pointer wont work:
This would be a lot more data that we'd need to serialize
If we had weak pointers creating a circular dependency we wouldn't be able to stop and retrieve that connection later.
There is no way to merge the same objects into one when deserializing
Now that you think about it, you can try the official answer of boost::serialization:
boost::serialization on shared_ptr
boost::serialization on shared_ptr 2
Related
Okay, since it's my first question, let me try again. I edited post and changed code to make it easier for you all to understand.
template <typename T>
class Structure
{
protected:
DWORD64 _instanceAddress;
T _structPointer;
public:
Structure<T>(DWORD64 instanceAddress);
~Structure();
T* operator->();
};
template <typename T>
T* Structure<T>::operator->()
{
Driver::ReadMemory( _instanceAddress, sizeof(T), &_structPointer );
return static_cast<T*>( &_structPointer );
}
struct C
{
int some_value;
}
struct B
{
C *pointer_to_C;
}
struct A
{
B *pointer_to_B;
}
//CODE
Structure<A> myStructure(DWORD64 AddressInMemory);
myStructure->pointer_to_B->pointer_to_C->some_value;
I'm reading other process memory through driver. Address of structure I'm trying to reach is static, so there is nothing to worry about. Is there any way that I can change my memory reading class to make it work like I've specified in bottom line of code? The ideal situation would be to do some changes in operator->() overload to do some checks regarding type user is trying to reach.
It should allow to define only lowest Structure in pointer chain and then just dereference it like its being done normally on mother-process raw memory.
So I'm imagining it would be something like that:
template <typename T>
T* Structure<T>::operator->(type_of_desired_variable var_name)
check if var_name is int or float
if not then define new Structure<type_of_desired_variable> and store it somewhere, then call its operator->()
How to serialize/deserialize object with boost (c++11) when object contains
vector<map<u_int64_t, Student*>*>* students;
like:
class YearContainer{
public:
vector<map<u_int64_t, Student*>*>* students;
};
where class Student has serialize method like
template<class Archive>
void serialize(Archive & ar, const unsigned int version)
{
ar & name;
// ....
}
How to deserialize and create all on heap ?
This looks like it uses the Boost.Serialization library. I don't really see a more specific question here.
I have been trying to make this work for a long time now.
In my project there are 6 classes that are being serialized using the exact tutorial from boost, by implementing the template function serialize.
Those classes are: State, guState, Policy, Action, Param, Vec3D.
When I serialize and save them, it works fine. I get indeed a text file, with various numbers and strings in it.
No complains, no warnings, no exceptions thrown. The only case is that if I try to serialize a pointer member of a class, the hole process becomes a zombie. So I do not try doing so, and saving works.
When however I try loading, I get:
terminate called after throwing an instance of
'boost::archive::archive_exception' what(): stream error
Now the interesting part is that I serialize two boost::ptr_vectors, one which consists of State pointers and one which consists of Policy pointers.
The state vector, I have saved and loaded without a problem.
The policy vector, I can save, but when I try loading i get the exception.
Furthermore, after reading the boost tutorials, I was under the impression that In order to load I did not need anything else other than the serialize function.
However when I tried loading, boost serialization would complain for not finding a default constructor such as State(), Policy(), etc (I implement my own constructors in each class ).
After reading this tutorial here I implemented a default constructor which does nothing, so that boost-serialization would work. Indeed it did compile, and I got the results mentioned above.
I have tried going down a much complicated road seen in my old question here where I tried seperating and implementing save_construct_data and load_construct_data, but I found this too instrusive, again I got the exact error as above.
Can someone please help me, explain how the loading works, what is the deal with the default constructors ? Or at least point me to a link that might be helpfull. I have gone through the manuals in boost, and they do not explain much about reconstruction.
Thank you.
Edit (Added a few snippets)
class State
{
friend class boost::serialization::access;
template<class Archive>
void serialize(Archive & ar, const unsigned int version);
protected:
std::map<std::string,float> positions;
float reward;
std::size_t hash_value;
bool exists(const Action* A);
bool operator== (State const& S);
std::size_t hash();
void clean_noise();
State(){}; // NOTE: This is used only by serializer, without it, code won't compile
public:
enum position { standing, on_chest, on_back, on_left, on_right, getting_up };
position current_position;
Policy *my_policy;
Vec3D gps;
boost::ptr_vector<Action> actions;
State(ACTION_MODE &m);
~State();
bool operator== (State const* S);
bool operator< (State const* S) const ;
const float& getR() const;
bool addAction(Action *A);
Action* findAction(const Action *A);
boost::ptr_vector<Action>& getAllActions();
void printState();
virtual bool isTerm();
};
template <class Archive>
void State::serialize(Archive& ar, const unsigned int version)
{
ar & BOOST_SERIALIZATION_NVP(positions);
ar & BOOST_SERIALIZATION_NVP(gps);
ar & BOOST_SERIALIZATION_NVP(current_position);
ar & BOOST_SERIALIZATION_NVP(reward);
ar & BOOST_SERIALIZATION_NVP(hash_value);
ar & BOOST_SERIALIZATION_NVP(actions);
ar & BOOST_SERIALIZATION_NVP(my_policy);
}
Other Classes inheriting from State, have also their serialize functions, using:
ar & boost::serialization::base_object<State>(*this);
Class Policy:
class Policy
{
friend class boost::serialization::access;
template<class Archive>
void serialize(Archive & ar, const unsigned int version);
Policy() {}; // NOTE: Again same as with state, used only by serialize load
protected:
float QValue;
State *state;
public:
//! Base class constructor
Policy(State *s);
...
};
template <class Archive>
void Policy::serialize(Archive& ar, const unsigned int version)
{
ar & BOOST_SERIALIZATION_NVP(action);
ar & BOOST_SERIALIZATION_NVP(state);
ar & BOOST_SERIALIZATION_NVP(QValue);
ar & BOOST_SERIALIZATION_NVP(r);
}
As you can see those are the two main classes, Other classes are also serialized because of reconstruction dependencies ( class Action, class Param, etc )
The master Class:
template <class S, class P> class Task
{
protected:
...
//! Container of states of type S (template parameter)
boost::ptr_vector<S> states;
//! Container of policies of type P (template parameter)
boost::ptr_vector<P> policies;
...
public:
Task(Agent &a, ACTION_MODE &m);
...
void save_to_file();
void load_from_file(std::string filename);
};
template <class S, class P>
void Task<S,P>::save_to_file()
{
std::string output = ramdisk+"serialized";
char *file = (char*)output.c_str();
std::ofstream ofs(file);
assert(ofs.good());
boost::archive::text_oarchive oa(ofs);
oa << states;
oa << policies;
ofs.close();
}
template <class S, class P>
void Task<S,P>::load_from_file(std::string filename)
{
char *file = (char*)output.c_str();
std::cout << file << std::endl;
std::ifstream ifs(file);
boost::archive::text_iarchive ia(ifs);
ia >> states;
ia >> policies;
ifs.close();
}
Effectively contains the two boost::ptr_vectors which hold the States and Policies.
States are saved and loaded without a problem.
The problem arises when loading policies. Saving them does not seem to create a problem (but then again I might be wrong).
Having tested save/load without policies, and with, The issue appears to be with policy reconstruction.
Note the default constructors used only by serialization, without which the code will not compile.
EDIT#2: After running application using valgrind and memcheck, it reports that there is a pointer memory leak. However since I am not good at debugging with valgrind I can't really tell where the leak is occuring or if it is relevant to my serialization (I think it is).
The problem is that you are serializing states and policies while Policy also holds references to the same instances of State. You can only serialize classes which don't have such cross-references. Boost should throw a pointer-conflict exception when writing to the file. In my tests it dependet on the order of writes if the exception was thrown or not - which is unfortunate because loading fails even if writing succeeded.
A workaround would be to delete the line oa << states when saving and loading and fix up the pointers by hand in a post-loading step.
About the constructors: it's mostly something the boost-api needs to do it's template magic. However when using versioning it's important to specify default-values for your member-variables so they are not left uninitialized when loading files with an older version-number.
As far as I understand, there is no serialization (boost::serialization, actually) support for boost::any placeholder.
Does someone know if there is a way to serialize a custom boost::any entity?
The problem here is obvious: boost::any uses template-based placeholders to store objects and typeid to check if boost::any_cast is appropriate.
So, there is a custom abstract superclass placeholder and custom template-based derived classes, which are created the following way:
template <T> custom_placeholder : public placeholder {
virtual std::type_info type() const { return typeid(T); }
virtual ...
};
Obviously, this brings some troubles when even thinking about serializing this stuff. Maybe someone knows some trick to make such kind of serialization (and of course, proper deserialization)?
Thank you
If you want to stick with boost::any i am not sure but you can write your own "boost::any". I'm using this code for proxy methods to pass the parameters.
#include <iostream>
#include <boost\smart_ptr\scoped_ptr.hpp>
#include <boost/shared_ptr.hpp>
#include <boost/serialization/access.hpp>
#include <boost/serialization/shared_ptr.hpp>
#include <boost/archive/text_oarchive.hpp>
#include <boost/archive/text_iarchive.hpp>
#include <boost/serialization/export.hpp>
#include <sstream>
class my_placeholder
{
public:
virtual ~my_placeholder(){}
my_placeholder(){}
private:
friend class boost::serialization::access;
template<class Archive>
void serialize(Archive & ar, const unsigned int version)
{
// serialize base class information
//ar & boost::serialization::base_object<bus_stop>(*this);
//ar & m_placeholder;
}
};
template<typename T>
class my_derivedplaceholder:
public my_placeholder
{
public:
my_derivedplaceholder()
{
}
my_derivedplaceholder(T &value)
{
m_value=value;
}
T m_value;
private:
friend class boost::serialization::access;
template<class Archive>
void serialize(Archive & ar, const unsigned int version)
{
// serialize base class information
ar & boost::serialization::base_object<my_placeholder>(*this);
ar & m_value;
}
};
BOOST_CLASS_EXPORT_GUID(my_derivedplaceholder<int>, "p<int>");
class my_any
{
public:
my_any()
{
}
template<typename T>
my_any(const T &value)
{
m_placeholder.reset(new my_derivedplaceholder<T>(const_cast<T&>(value)));
}
template<typename T>
void operator=(const T &value)
{
m_placeholder.reset(new my_derivedplaceholder<T>(const_cast<T&>(value)));
}
protected:
friend class boost::serialization::access;
template<class Archive>
void serialize(Archive & ar, const unsigned int version)
{
// serialize base class information
//ar & boost::serialization::base_object<bus_stop>(*this);
ar & m_placeholder;
}
template<typename T>
friend T my_anycast(my_any &val);
boost::shared_ptr<my_placeholder> m_placeholder;
};
template<typename T>
T my_anycast(my_any &val)
{
boost::shared_ptr<my_derivedplaceholder<T>> concrete=boost::dynamic_pointer_cast<my_derivedplaceholder<T>>(val.m_placeholder);
if (concrete.get()==NULL)
throw std::invalid_argument("Not convertible");
return concrete->m_value;
}
void main()
{
my_any m=10;
int a=my_anycast<int>(m);
std::cout << a << std::endl;
std::stringstream ss,ss2;
boost::archive::text_oarchive oa(ss);
oa << m;
boost::archive::text_iarchive ia(ss);
my_any m2;
ia >> m2;
std::cout << my_anycast<int>(m2) << std::endl;
}
It is not possible at all, at least for arbitrary types. Note that maybe you could serialize using some tricky code (like finding the size of the elements contained in the any), but the any code relies on the compiler statically putting the any type_code and the proper types inside the placeholder. You surely cannot do that in deserialization in C++, as the type that you'd get from the deserialization is not known at compile time (as required by the newly formed boost::any).
The best solution is to build some kind of specialized any type for the exact types of elements you're going to serialize. Then, you can have special cases for the actual type of element being deserialized, but note that each element type serialization/deserialization has to be phisically written as static C++ code.
PD. Some others suggested using boost::variant as a representation of this specialized type holding the exact types you're going to serialize. You need a way of discerning the exact type on deserialization, though (maybe assigning identifiers to types in the variant).
Assuming you have to use boost::any and you cannot switch to variant, a map<type_info const*, string(*)(any)> based solution could get you done.
You have to initialize at runtime such a map with all the types you plan to use. Of course, you can use something along the lines of
template <typename T>
struct any_serializer
{
static string perform(any a)
{
T const& x = any_cast<T const&>(a);
stringstream out;
out << x;
return out.str();
}
};
and populate the map with addresses of any_serializer<T>::perform under the key &typeid(T). You can specialize the class any_serializer and use some (ugly) macros to populate the map.
More difficult is of course the deserialization. I haven't had a look at boost::lexical_cast for a while, perhaps it can provide some help. I am afraid that this is totally problem-dependant. However, you only need one function, which takes a string and returns one any. You may also want to prepend your output string with a custom type identifier.
There is no need to create new class. Try to use xany https://sourceforge.net/projects/extendableany/?source=directory
xany class allows to add new methods to any's existing functionality. By the way there is a example in documentation which does exactly what you want.
Using boost::serialization, what's the "best" way to serialize an object that contains cached, derived values in mutable members?
class Example
{
public:
Example(float n) :
num(n),
sqrt_num(-1.0)
{}
// compute and cache sqrt on first read
float get_sqrt() const
{
if(sqrt_num < 0)
sqrt_num = sqrt(num);
return sqrt_num;
}
template <class Archive>
void serialize(Archive& ar, unsigned int version)
{ ... }
private:
float num;
mutable float sqrt_num;
};
I'd like to avoid splitting serialize() into separate save() and load() methods, for maintenance reasons.
One suboptimal implementation of serialize:
template <class Archive>
void serialize(Archive& ar, unsigned int version)
{
ar & num;
sqrt_num = -1.0;
}
This handles the deserialization case, but in the serialization case, the cached value is killed and must be recomputed.
What is the best practice in this case?
Splitting your saving and loading methods doesn't mean you have to maintain two copies of your serialization code. You can split them and then join them back again with a common function.
private:
friend class boost::serialization::access;
BOOST_SERIALIZATION_SPLIT_MEMBER()
template <class Archive>
void save(Archive& ar, const unsigned int version) const {
const_cast<Example*>(this)->common_serialize(ar, version);
}
template <class Archive>
void load(Archive& ar, const unsigned int version) {
common_serialize(ar, version);
sqrt_num = -1;
}
template <class Archive>
void common_serialize(Archive& ar, const unsigned int version) {
ar & num;
}
You probably noticed the const_cast. That's an unfortunate caveat to this idea. Although the serialize member function is non-const for saving operations, the save member function needs to be const. As long as the object you're serializing wasn't originally declared const, though, it's safe to cast it away as shown above. The documentation briefly mentions the need to cast for const members; this is similar.
With the changes above, your code will correctly print "2" for both ex1 and ex2, and you only have to maintain one copy of the serialization code. The load code only contains code specific to re-initializing the object's internal cache; the save function doesn't touch the cache.
You can check the Archive::is_loading field, and load cached values if it's true.
template <class Archive>
void serialize(Archive& ar, unsigned int version)
{
ar & num;
if(Archive::is_loading::value == true)
sqrt_num = -1.0;
}