I have Players class and it has some variables
class Players
{
private:
friend class boost::serialization::access;
template <class Ar>
void serialize(Ar &ar, unsigned)
{
ar &username &password;
}
public:
std::string username = "", password = "", email = "";
};
I just want to serialize username and password. I dont want to add email too. I'm getting error after trying to deserialize it.
Your code seems correct.
Remember that you cannot deserialize an archive that you created with an old version of the serialization routines.
If that is the case, as a very primitive workaround you can do this below, or learn how to deal with archive versions.
template <class Ar>
void serialize(Ar &ar, unsigned)
{
std::string dummy_email = "";
ar &username &password & dummy_email;
}
Related
I'm trying to load parameters from a JSON file into my own parameter class using cereal (v1.12) on VS2013.
Compilation and loading into a POD works fine, but loading into my own class throws cereal::RapidJSONException.
Here is my code:
#include <iostream>
#include "cereal/archives/json.hpp"
struct CMyStruct {
CMyStruct(int f_value) : m_value(f_value) {}
int m_value;
};
template<class Archive> void load(Archive& ar, CMyStruct& parameter) {
int input;
ar(input);
parameter.m_value = input;
};
void test_cereal() {
int l_scalar_i(42);
CMyStruct l_scalar(42);
std::ifstream l_jsonFile("example_minimal.json");
cereal::JSONInputArchive l_ar(l_jsonFile);
l_ar( cereal::make_nvp("scalar", l_scalar_i) ); // WORKS
l_ar( cereal::make_nvp("scalar", l_scalar) ); // throws cereal::RapidJSONException
return;
}
The JSON file:
{
"bool": false,
"scalar": 3
}
Following the call stack shows that document.h:244 inside RapidJSON throws the exception:
ConstMemberIterator MemberEnd() const
{ RAPIDJSON_ASSERT(IsObject()); return data_.o.members + data_.o.size; }
However, I have no idea what to make of it. How can I make my code work?
Thanks a lot!
UPDATE WITH SOLUTION:
As written by ChrisBFX, it is necessary to implement load_minimal/save_minimal in order to deserialize an C++ object to a JSON scalar.
load/save on the other hand serializes a C++ object from/to a JSON object.
For my code, the load() function has to be removed and replaced by this:
template<class Archive>
int save_minimal(const Archive& ar,
const CMyStruct& parameter)
{
return parameter.m_value;
};
template<class Archive>
void load_minimal(const Archive& ar,
CMyStruct& parameter,
const int & value_i)
{
parameter.m_value = value_i;
};
Now it works like a charm! :-)
Your CMStruct is not a scalar but an object. So cereal expects
{
"bool": false,
"scalar": {
"value": 3
}
}
If you want to serialize your struct as a scalar you have to provide a minimal_load function for it. See http://uscilab.github.io/cereal/serialization_functions.html "minimal split serialization".
How to serialize/deserialize object with boost (c++11) when object contains
vector<map<u_int64_t, Student*>*>* students;
like:
class YearContainer{
public:
vector<map<u_int64_t, Student*>*>* students;
};
where class Student has serialize method like
template<class Archive>
void serialize(Archive & ar, const unsigned int version)
{
ar & name;
// ....
}
How to deserialize and create all on heap ?
This looks like it uses the Boost.Serialization library. I don't really see a more specific question here.
I want to be able to serialize a Windows HANDLE:
typedef void *HANDLE
If I try to compile using following:
struct Foo
{
HANDLE file;
protected:
friend class boost::serialization::access;
template<class Archive>
void serialize(Archive & ar, const unsigned int /*version*/)
{
ar & file;
}
};
I get compile errors:
c:\projects\3rdparty\src\boost\include\boost/mpl/print.hpp(51) : warning C4308: negative integral constant converted to unsigned type
c:\projects\3rdparty\src\boost\include\boost/serialization/static_warning.hpp(92) : see reference to class template instantiation 'boost::mpl::print<T>' being compiled
with
[
T=boost::serialization::BOOST_SERIALIZATION_STATIC_WARNING_LINE<98>
]
c:\projects\3rdparty\src\boost\include\boost/archive/detail/check.hpp(98) : see reference to class template instantiation 'boost::serialization::static_warning_test<B,L>' being compiled
with
[
B=false,
L=98
]
c:\projects\3rdparty\src\boost\include\boost/archive/detail/oserializer.hpp(313) : see reference to
function template instantiation 'void boost::archive::detail::check_object_tracking<T>(void)' being compiled
with
[
T=Foo
]
c:\projects\3rdparty\src\boost\include\boost/archive/detail/oserializer.hpp(525) : see reference to
function template instantiation 'void boost::archive::detail::save_non_pointer_type<Archive>::invoke<T>(Archive &,T &)' being compiled
with
[
Archive=boost::archive::text_oarchive,
T=Foo
]
But if I change file to an int, everything is fine.
How do I tell boost to serialize HANDLEs as ints?
Thanks
HANDLE is a Windows-API-specific data type defined in winnt.h. According to the MSDN,
A handle to an object.
This type is declared in WinNT.h as follows:
typedef PVOID HANDLE;
So, now we see that HANDLE is really just void * -- representing a handle to an object. Think about what it is that you're trying to do; does it make sense to serialize a pointer to some object in the Windows API?
Instead, try to serialize what it takes to retrieve an equivalent HANDLE; judging by the name of the member, I'm going to guess you used CreateFile -- so, you'll need to know...
The file name
The desired access (e.g. GENERIC_READ | GENERIC_WRITE)
The share mode (e.g. FILE_SHARE_DELETE)
Optionally, the security attributes
The creation disposition (i.e. CREATE_NEW, TRUNCATE_EXISTING, etc.)
The file or device flags and attributes
Optionally, a template file -- for copying its attributes when creating a file
Now, if you really don't want to be doing that -- you're positive you want the pointer value -- maybe try serializing it after casting via reinterpret_cast to std::intptr_t or std::uintptr_t (as might be defined in cstdint as of C++11).
ar & reinterpret_cast<std::intptr_t>(file);
... then you should couple this with something as follows (when deserializing):
std::intptr_t _file;
ar & _file;
HANDLE file = std::reinterpret_cast<HANDLE>(_file);
Ended up solving the problem by splitting the serialization. Seems like a hack though:
struct Foo
{
HANDLE file;
protected:
friend class boost::serialization::access;
template<class Archive>
void save(Archive & ar, const unsigned int /*version*/) const
{
unsigned int _file = reinterpret_cast<unsigned int>(file);
ar & _file;
}
template<class Archive>
void load(Archive & ar, const unsigned int /*version*/)
{
unsigned int _file;
ar & _file;
file = reinterpret_cast<HANDLE>(_file);
}
BOOST_SERIALIZATION_SPLIT_MEMBER()
};
I have been trying to make this work for a long time now.
In my project there are 6 classes that are being serialized using the exact tutorial from boost, by implementing the template function serialize.
Those classes are: State, guState, Policy, Action, Param, Vec3D.
When I serialize and save them, it works fine. I get indeed a text file, with various numbers and strings in it.
No complains, no warnings, no exceptions thrown. The only case is that if I try to serialize a pointer member of a class, the hole process becomes a zombie. So I do not try doing so, and saving works.
When however I try loading, I get:
terminate called after throwing an instance of
'boost::archive::archive_exception' what(): stream error
Now the interesting part is that I serialize two boost::ptr_vectors, one which consists of State pointers and one which consists of Policy pointers.
The state vector, I have saved and loaded without a problem.
The policy vector, I can save, but when I try loading i get the exception.
Furthermore, after reading the boost tutorials, I was under the impression that In order to load I did not need anything else other than the serialize function.
However when I tried loading, boost serialization would complain for not finding a default constructor such as State(), Policy(), etc (I implement my own constructors in each class ).
After reading this tutorial here I implemented a default constructor which does nothing, so that boost-serialization would work. Indeed it did compile, and I got the results mentioned above.
I have tried going down a much complicated road seen in my old question here where I tried seperating and implementing save_construct_data and load_construct_data, but I found this too instrusive, again I got the exact error as above.
Can someone please help me, explain how the loading works, what is the deal with the default constructors ? Or at least point me to a link that might be helpfull. I have gone through the manuals in boost, and they do not explain much about reconstruction.
Thank you.
Edit (Added a few snippets)
class State
{
friend class boost::serialization::access;
template<class Archive>
void serialize(Archive & ar, const unsigned int version);
protected:
std::map<std::string,float> positions;
float reward;
std::size_t hash_value;
bool exists(const Action* A);
bool operator== (State const& S);
std::size_t hash();
void clean_noise();
State(){}; // NOTE: This is used only by serializer, without it, code won't compile
public:
enum position { standing, on_chest, on_back, on_left, on_right, getting_up };
position current_position;
Policy *my_policy;
Vec3D gps;
boost::ptr_vector<Action> actions;
State(ACTION_MODE &m);
~State();
bool operator== (State const* S);
bool operator< (State const* S) const ;
const float& getR() const;
bool addAction(Action *A);
Action* findAction(const Action *A);
boost::ptr_vector<Action>& getAllActions();
void printState();
virtual bool isTerm();
};
template <class Archive>
void State::serialize(Archive& ar, const unsigned int version)
{
ar & BOOST_SERIALIZATION_NVP(positions);
ar & BOOST_SERIALIZATION_NVP(gps);
ar & BOOST_SERIALIZATION_NVP(current_position);
ar & BOOST_SERIALIZATION_NVP(reward);
ar & BOOST_SERIALIZATION_NVP(hash_value);
ar & BOOST_SERIALIZATION_NVP(actions);
ar & BOOST_SERIALIZATION_NVP(my_policy);
}
Other Classes inheriting from State, have also their serialize functions, using:
ar & boost::serialization::base_object<State>(*this);
Class Policy:
class Policy
{
friend class boost::serialization::access;
template<class Archive>
void serialize(Archive & ar, const unsigned int version);
Policy() {}; // NOTE: Again same as with state, used only by serialize load
protected:
float QValue;
State *state;
public:
//! Base class constructor
Policy(State *s);
...
};
template <class Archive>
void Policy::serialize(Archive& ar, const unsigned int version)
{
ar & BOOST_SERIALIZATION_NVP(action);
ar & BOOST_SERIALIZATION_NVP(state);
ar & BOOST_SERIALIZATION_NVP(QValue);
ar & BOOST_SERIALIZATION_NVP(r);
}
As you can see those are the two main classes, Other classes are also serialized because of reconstruction dependencies ( class Action, class Param, etc )
The master Class:
template <class S, class P> class Task
{
protected:
...
//! Container of states of type S (template parameter)
boost::ptr_vector<S> states;
//! Container of policies of type P (template parameter)
boost::ptr_vector<P> policies;
...
public:
Task(Agent &a, ACTION_MODE &m);
...
void save_to_file();
void load_from_file(std::string filename);
};
template <class S, class P>
void Task<S,P>::save_to_file()
{
std::string output = ramdisk+"serialized";
char *file = (char*)output.c_str();
std::ofstream ofs(file);
assert(ofs.good());
boost::archive::text_oarchive oa(ofs);
oa << states;
oa << policies;
ofs.close();
}
template <class S, class P>
void Task<S,P>::load_from_file(std::string filename)
{
char *file = (char*)output.c_str();
std::cout << file << std::endl;
std::ifstream ifs(file);
boost::archive::text_iarchive ia(ifs);
ia >> states;
ia >> policies;
ifs.close();
}
Effectively contains the two boost::ptr_vectors which hold the States and Policies.
States are saved and loaded without a problem.
The problem arises when loading policies. Saving them does not seem to create a problem (but then again I might be wrong).
Having tested save/load without policies, and with, The issue appears to be with policy reconstruction.
Note the default constructors used only by serialization, without which the code will not compile.
EDIT#2: After running application using valgrind and memcheck, it reports that there is a pointer memory leak. However since I am not good at debugging with valgrind I can't really tell where the leak is occuring or if it is relevant to my serialization (I think it is).
The problem is that you are serializing states and policies while Policy also holds references to the same instances of State. You can only serialize classes which don't have such cross-references. Boost should throw a pointer-conflict exception when writing to the file. In my tests it dependet on the order of writes if the exception was thrown or not - which is unfortunate because loading fails even if writing succeeded.
A workaround would be to delete the line oa << states when saving and loading and fix up the pointers by hand in a post-loading step.
About the constructors: it's mostly something the boost-api needs to do it's template magic. However when using versioning it's important to specify default-values for your member-variables so they are not left uninitialized when loading files with an older version-number.
Using boost::serialization, what's the "best" way to serialize an object that contains cached, derived values in mutable members?
class Example
{
public:
Example(float n) :
num(n),
sqrt_num(-1.0)
{}
// compute and cache sqrt on first read
float get_sqrt() const
{
if(sqrt_num < 0)
sqrt_num = sqrt(num);
return sqrt_num;
}
template <class Archive>
void serialize(Archive& ar, unsigned int version)
{ ... }
private:
float num;
mutable float sqrt_num;
};
I'd like to avoid splitting serialize() into separate save() and load() methods, for maintenance reasons.
One suboptimal implementation of serialize:
template <class Archive>
void serialize(Archive& ar, unsigned int version)
{
ar & num;
sqrt_num = -1.0;
}
This handles the deserialization case, but in the serialization case, the cached value is killed and must be recomputed.
What is the best practice in this case?
Splitting your saving and loading methods doesn't mean you have to maintain two copies of your serialization code. You can split them and then join them back again with a common function.
private:
friend class boost::serialization::access;
BOOST_SERIALIZATION_SPLIT_MEMBER()
template <class Archive>
void save(Archive& ar, const unsigned int version) const {
const_cast<Example*>(this)->common_serialize(ar, version);
}
template <class Archive>
void load(Archive& ar, const unsigned int version) {
common_serialize(ar, version);
sqrt_num = -1;
}
template <class Archive>
void common_serialize(Archive& ar, const unsigned int version) {
ar & num;
}
You probably noticed the const_cast. That's an unfortunate caveat to this idea. Although the serialize member function is non-const for saving operations, the save member function needs to be const. As long as the object you're serializing wasn't originally declared const, though, it's safe to cast it away as shown above. The documentation briefly mentions the need to cast for const members; this is similar.
With the changes above, your code will correctly print "2" for both ex1 and ex2, and you only have to maintain one copy of the serialization code. The load code only contains code specific to re-initializing the object's internal cache; the save function doesn't touch the cache.
You can check the Archive::is_loading field, and load cached values if it's true.
template <class Archive>
void serialize(Archive& ar, unsigned int version)
{
ar & num;
if(Archive::is_loading::value == true)
sqrt_num = -1.0;
}