I created an output text_archive and I restored it using binary archive and obviously, got some issues.
Can I know somehow the kind of archive, so that I could possibly use the appropriate code for binary/xml/text archive.
class Info
{
private:
// Allow serialization to access non-public data members.
friend class boost::serialization::access;
// Serialize the std::vector member of Info
template<class Archive>
void serialize(Archive & ar, const unsigned int version)
{
ar & filenames;
}
std::vector<std::string> filenames;
};
int main(int argc, char** argv)
{
Info info;
// Save filename data contained in Info object
{
// Create an output archive
std::ofstream ofs( "store.dat" );
**boost::archive::text_oarchive ar(ofs);**
ar & info;
}
// Restore from saved data and print to verify contents
Info restored_info;
{
// Create and input archive
std::ifstream ifs( "store.dat" );
**boost::archive::binary_iarchive ar(ifs);**
// Load the data
ar & restored_info;
}
return 0;
}
You can use first two bytes to specific type, let's say
00 for binary
01 for xml
10 for text
and the reset of the content is for data itself
Related
I am using Cereal library for serialization of structured data, to send them over MPI and later on process them on GPU using CUDA. Due to the latter, I cannot use std::vector, because CUDA has problems with dynamic objects. Therefore I am using static arrays of structures.
GOAL
To do binary serialization & deserialization on objects with nested static array of objects.
OBJECTS TO BE SERIALIZED
//==========================================================//
// ELEMENT DEFINITION //
//==========================================================//
class element{
public:
//==========================//
// CONTENT //
//==========================//
int grid_id;
int NN_id;
int debug;
//==========================//
// SERIALIZATION //
//==========================//
// function required by cereal library
template<class Archive>
void serialize(Archive & ar){
ar( grid_id );
ar( NN_id );
ar( debug );
}
};
//==========================================================//
// CONTAINER DEFINITION //
//==========================================================//
class GPU_in_grid : public Managed{
public:
//==========================//
// CONTENT //
//==========================//
int gpu_id;
element element[GRID_SIZE]; <------ static array of structures
int debug;
//==========================//
// SERIALIZATION //
//==========================//
template<class Archive>
void serialize(Archive & ar){
ar( gpu_id );
ar( debug );
ar( element );
}
};
SERIALIZATION FUNCTIONS
// cereal serialization
#include <cereal/archives/binary.hpp>
#include <cereal/archives/portable_binary.hpp>
//#include <cereal/archives/xml.hpp>
template<typename data_type>
int Process::serialize(data_type * data, char *serial_data, int *size){
// serialize the data
std::ostringstream oss(std::ios::binary);
cereal::BinaryOutputArchive ar(oss);
//cereal::XMLOutputArchive ar(std::cout);
ar(*data);
std::string s=oss.str();
*size = s.length();
strncpy(serial_data, s.c_str(), s.length());
std::cout << "buffer["<< s.length() << "] >> " << s << std::endl;
s.clear();
return _SUCCESS_;
};
template<typename data_type>
int Process::deserialize(data_type * data, char *serial_data, int size){
// create temporary buffer to store the received data
char * buf=new char[size];
strncpy(buf, serial_data, size);
std::istringstream iss(std::string(serial_data, size), std::ios::binary);
cereal::BinaryInputArchive arin(iss);
arin(*data);
// clean buffer
delete[] buf;
return _SUCCESS_;
};
DEBUGGING OUTPUT
cpu_mem BEFORE serialization
cpu_mem.debug = -1
cpu_mem.gpu_id = -5
cpu_mem.element[0].NN_id = 1
cpu_mem.element[0].debug = 2
buffer[248] >> �������� <------ binary format
cpu_mem AFTER deserialization
cpu_mem.debug = -1 <----- CORRECT
cpu_mem.gpu_id = -5 <----- CORRECT
cpu_mem.element[0].NN_id = 0 <----- INCORRECT
cpu_mem.element[0].debug = 0 <----- INCORRECT
If I play around a bit with the streams, I can print out the serialized object as XML, to check whether the serialization is successful.
<?xml version="1.0" encoding="utf-8"?>
<cereal>
<value0>
<gpu_id>-5</gpu_id> <----- CORRECT
<debug>-1</debug> <----- CORRECT
<element>
<value0>
<grid_id>0</grid_id>
<NN_id>1</NN_id> <----- CORRECT
<debug>2</debug> <----- CORRECT
</value0>
<value1>
<grid_id>32522</grid_id>
<NN_id>2</NN_id>
<debug>4612412</debug>
</value1>
</element>
</value0>
</cereal>
PROBLEM
The above outputs shows, that deserialization correctly recognizes variables in the container (class GPU_in_grid) but cannot deserialize the lower levels of my structure, namely static array of structures -> element[].
For some unknown reason to me, the strncpy() malfunctions. The content of the string differs from the content of the array, EVEN if I keep eye on '\0', which need to be appended at the end of the char array, shown here http://www.cplusplus.com/reference/string/string/copy/ .
I have ended up using std::string.copy() instead, which performs as expected, as long as you add '\0' at the end of the char array.
the above code works then just fine.
How can I use boost serialization to save and get multiple objects (number of objects varies)?
For example I have class Contact, and I input contact and save it to file. Another time I input another contact, and it should also be saved in file.
I think the save function should be like this:
void save_contact(const Contact &s, const char * filename){
std::ofstream ofs(filename, std::ios::app);
boost::archive::text_oarchive oa(ofs);
oa << s;
ofs << "\n";
}
And to retrieve the contacts I should keep track of contacts number, am I right?
To retrieve single contact I use the following code:
void retrieve_contact(Contact &s, const char * filename)
{
std::ifstream ifs(filename);
boost::archive::text_iarchive ia(ifs);
ia >> s;
}
Here is how serialization function inside Contact class looks like:
template<class Archive>
void serialize(Archive & ar, const unsigned int version)
{
ar & m_CompanyName;
ar & m_FirstName;
ar & m_LastName;
ar & m_PhoneNumbers;
}
(m_CompanyName, m_FirstName and m_LastName are std::string, m_PhoneNumbers is std::vector)
So is there way somehow to retrieve contacts without keeping track of number of contacts? Or can you suggest me another way to save and retrieve contacts, saved at different time? Also how can I edit the saved file to modify Contact?
Yeah, use the predefined serialization for standard containers. So, e.g.
#include <boost/serialization/vector.hpp>
And then
void retrieve_contact(std::vector<Contact>& s, const char * filename)
{
std::ifstream ifs(filename);
boost::archive::text_iarchive ia(ifs);
ia >> s;
}
I am trying to work with boost::serialisation for saving and loading some objects.
So far from the boost::tutorial I have managed to do things work for all the different stl stuff (vectors, pairs, lists etc), for derived classes, for boost multi-arrays and other things, but I am stuck trying to work around how to serialize a boost::any vector.
Let me say in advance that I found and checked in the forum some posts for boost::varian serialization and even one for boost::any (which even has an almost identical title) but yet again I didn't manage to solve my problems.
So let me go with a small example:
Say I have this class:
class class_2
{
private:
template<class Archive>
void serialize(Archive & ar, const unsigned int version)
{
ar & degrees;
ar & minutes;
ar & seconds;
for( std::vector<boost::any>::iterator it = v_any.begin() ; it != v_any.end() ; ++it ) {
//Trying to cast all the possible types that may enter the
//boost any vector, for the sake of this example let's just
//say that we will only pass a second class called name class_1
//to the boost::any vector and we only have to cast this class
if (it->type() == typeid(class_1)){
class_1 lan = boost::any_cast< class_1 > (*it );
ar & (lan);
}
}// for boost::any*/
} //serialization
public:
int degrees;
int minutes;
float seconds;
class_2(){};
class_2(int d, int m, float s) :
degrees(d), minutes(m), seconds(s)
{}
std::vector<boost::any> v_any;
};
And to be more precise the class_1 that we expect for this small example to exist inside the boost::any vector is the following class:
class class_1
{
private:
friend class boost::serialization::access;
template<class Archive>
void serialize(Archive & ar, const unsigned int version)
{
ar & a;
}
public:
class_1(){};
int a;
};
So when I compile the above code with a main function where I save and load an object of class_2 that containts inside the boost::any vector an object of class_1 everything compiles and even runs:
int main() {
class_1 alpha;
class_2 beta;
alpha.a=5;
beta.v_any.push_back(alpha);
std::ofstream ofs("file");
// save data to archive
{
boost::archive::text_oarchive oa(ofs);
// write class instance to archive
oa << beta;
// archive and stream closed when destructors are called
}
// ... some time later restore the class instance to its orginal state
class_2 newclass;
{
// create and open an archive for input
std::ifstream ifs("file");
boost::archive::text_iarchive ia(ifs);
// read class state from archive
ia >> newclass;
// archive and stream closed when destructors are called
}
return 0;
}
Yet again the loaded newclass object has an empty boost::any vector with nothing saved inside.
So my question is what am I doing wrong in the above code and what should I change in order to serialize the boost::any vector correctly..?
Any help/suggestion would be very appreciated.
I have two classes that will represent two very simple databases, and each has a "Save" function which will write what's in the class to a file. Since the code within the "Save" function is very similar, I was wondering if I could factor it out.
One of my colleagues said this might be possible with inheritance and/or metadata, so I tried looking into it myself with Google. However, I couldn't find anything that was helpful and am still unsure if what I want to do is even possible.
If it's possible to factor out, then I think I'd need to have another class or function know about each class's types and iterate through them somehow (metadata?). It would check the type of every data, and depending on what the type is, it would make sure that it's correctly output to the text file.
(I know data like name, age, etc. should be private, but to keep this simple I just had everything be public)
class A
{
public:
A() : name(""), age(0) {};
void Save(void)
{
std::string filename = "A.txt";
std::string data;
data += name + "\n";
data += std::to_string(age) + "\n";
std::ofstream outfile(filename);
outfile.write(data.c_str(), data.size());
outfile.close();
}
std::string name;
int age;
};
class B
{
public:
B() : ID(0), points(0) {};
void Save(void)
{
std::string filename = "B.txt";
std::string data;
data += std::to_string(ID) + "\n";
data += std::to_string(points) + "\n";
std::ofstream outfile(filename);
outfile.write(data.c_str(), data.size());
outfile.close();
}
int ID;
int points;
};
int main(void)
{
A a;
B b;
a.name = "Bob"; a.age = 20;
b.ID = 4; b.points = 95;
a.Save();
b.Save();
return 0;
}
A possible solution could be to use metaprogramming (not sure what you mean by metadata), i.e. templates to reuse the common parts
template<typename T1, typename T2>
void TSave(const std::string fname, const T1& p1, const T2& p2) {
std::string filename = fname;
std::stringstream data;
data << p1 << "\n";
data << p2 << "\n";
std::ofstream outfile(filename);
outfile.write(data.str().c_str(), data.str().size());
outfile.close();
}
class A {
...
void Save(void) {
TSave("A.txt", name, age);
}
std::string name;
int age;
};
class B {
...
void Save(void) {
TSave("B.txt", ID, points);
}
int ID;
int points;
};
Live Example
What you are looking for is serialization: saving objects to a file (and one day or another, restore the objects).
Of course, you could write your own serialization framework, and Marco's answer is an interesting start in that direction. But alternatively, you could consider existing libraries, such as boost::serialization :
#include <boost/archive/text_oarchive.hpp>
#include <boost/archive/text_iarchive.hpp>
class A {
private:
friend class boost::serialization::access;
template<class Archive>
void serialize(Archive & ar, const unsigned int version)
{
ar & name;
ar & age;
}
...
};
class B {
private:
friend class boost::serialization::access;
template<class Archive>
void serialize(Archive & ar, const unsigned int version)
{
ar & ID;
ar & points;
}
...
};
main() {
A a;
B b;
...
{
std::ofstream ofs("myfile");
boost::archive::text_oarchive arch(ofs);
arch << a << b;
}
}
As you see, it's still needed to say what's to be written to the file. However, the code is simplified : you don't have to worry about file management and transformation of data. And it works also with standard containers.
You won't find a C++ trick that automatically determines for a class what's to be saved. Two reasons for that:
C++ allows metaprogramming, but it is not reflexive: there are no standard process to find out at execution time which members compose a class.
In an object, some data can be transient, i.e. it means only something at the time of the execution and depends on the context. For example pointers: you could save the value of a pointer to a file, but it will mean nothing when you reload it later (the pointer is only valid until you free the object). The proper way would be to save the object that is pointed to (but where, when, how?).
I have a tree-like structure that needs to be serialized. Typical structure, with each node having parent members and children vectors. parent is a raw pointer-to-class, and children are vectors of shared_ptrs. Now it seems that serialization works fine, but de-serialization leaves the parent members uninitialized (pointers to 0xcccccccc or 0x00000000).
The parent members are being loaded when the actual parent object has not yet finished deserializing, i.e. the child's parent member is loaded through the deserialization request of the parent's children. Since this is cyclic I was wondering whether I need to take special measures for it to work.
Thanks for the help.
Update: This is how my serializing function looks like:
template <typename Archive>
void serialize(Archive& archive, GameCore::GameObject& t, const unsigned int version)
{
archive & boost::serialization::base_object<GameCore::Object>(t);
archive & boost::serialization::base_object<GameCore::Updatable>(t);
archive & t.parent;
archive & t.transform;
archive & t.components;
archive & t.children;
}
If I comment out archive & t.children, parent gets populated correctly.
Update 2: Ok, I've managed to turn this into a minimal sample that exhibits the problem. The following should compile:
#include <boost\archive\binary_oarchive.hpp>
#include <boost\archive\binary_iarchive.hpp>
#include <fstream>
#include <memory>
#include <vector>
class A
{
public:
A() {}
A(const A& rhs) = delete;
int someInt = 0;
A* parent = nullptr;
std::vector<A*> children;
template <class Archive>
void serialize(Archive& archive, const unsigned int version)
{
archive & someInt;
archive & parent;
int count = children.size();
archive & count;
children.resize(count);
for (int i = 0; i < count; ++i)
{
A* ptr = children[i];
archive & ptr;
children[i] = ptr;
}
}
};
int main()
{
A* newA = new A();
newA->someInt = 0;
A* newPtr = new A();
newPtr->someInt = 5;
newPtr->parent = newA;
newA->children.push_back(newPtr);
// Save.
std::ofstream outputFile("test", std::fstream::out | std::fstream::binary);
if (outputFile.is_open())
{
boost::archive::binary_oarchive outputArchive(outputFile);
// Serialize objects.
outputArchive << newA;
outputFile.close();
}
delete newA;
delete newPtr;
A* loadedPtr = nullptr;
// Load.
std::ifstream inputFile("test", std::fstream::binary | std::fstream::in);
if (inputFile && inputFile.good() && inputFile.is_open())
{
boost::archive::binary_iarchive inputArchive(inputFile);
// Load objects.
inputArchive >> loadedPtr;
inputFile.close();
}
return 0;
}
Step through the code. The child's parent stays null, always.
Alright, apparently I fell prey to another unlucky bug. Boost 1.55 doesn't yet have a working serialization library for VS2013, according to the latest Boost release page.
Talk about wasted time...
Known Bugs with Visual Studio 2013/Visual C++ 12
Visual Studio 2013
was released quite late in the release process, so there exist several
unresolved issues. These include:
Serialization can't compile because of a missing include.
I had the same problem in past, and I did not find solid solution from out-of-the-box.. but the following small hack works fine - you could specify serialization and de-serialization functions separately (not using default template and &-operator):
//! Serialization
void A::serialize(xml_oarchive& ar, const unsigned int version)
{
ar << value;
}
//! De-serialization
void A::serialize(xml_iarchive& ar, const unsigned int version)
{
ar >> value;
}
And after that you could specify restoring of a pointer to parent object in deserialization method, like the following:
//! Serialization
void A::serialize(xml_oarchive& ar, const unsigned int version)
{
ar << children;
}
//! De-serialization
void A::serialize(xml_iarchive& ar, const unsigned int version)
{
ar >> children;
for(var child: children)
child->parent = this;
}
This has been resolved on the development branch.
See. https://svn.boost.org/trac/boost/ticket/9601
If you use Boost Serialization everything should work out-of-box. You class serialize can look like
#include <boost/serialization/vector.hpp>
void serialize (Archive&ar, unsigned int version)
{
//declare possible derived types of nodes
ar & parent;
ar & children;
}
Archive will hash pointers so it will not create each element more than once.
One important detail: If your nodes can be of some derived type you need to teach archive what types to expect at // place,
see details in boost documentation on serializing derived types through pointers to base type.
If you are sure your tree structure is correct and self-contained (root has NULL as a parent, all other nodes are "children" of their respective "parent", etc.) than you can organize code a little more efficiently:
#include <boost/serialization/vector.hpp>
void serialize (Archive&ar, unsigned int version)
{
//declare possible derived types of nodes (either here or at archive creation point)
ar & children;
if (typename Archive::is_loading())
{
for (auto child : children)
child->parent = this;
}
}
(I assume you set "parent" to NULL in constructor).
I did not test this code with VS 2013.