C++ serialization of static array of struct using Cereal - c++

I am using Cereal library for serialization of structured data, to send them over MPI and later on process them on GPU using CUDA. Due to the latter, I cannot use std::vector, because CUDA has problems with dynamic objects. Therefore I am using static arrays of structures.
GOAL
To do binary serialization & deserialization on objects with nested static array of objects.
OBJECTS TO BE SERIALIZED
//==========================================================//
// ELEMENT DEFINITION //
//==========================================================//
class element{
public:
//==========================//
// CONTENT //
//==========================//
int grid_id;
int NN_id;
int debug;
//==========================//
// SERIALIZATION //
//==========================//
// function required by cereal library
template<class Archive>
void serialize(Archive & ar){
ar( grid_id );
ar( NN_id );
ar( debug );
}
};
//==========================================================//
// CONTAINER DEFINITION //
//==========================================================//
class GPU_in_grid : public Managed{
public:
//==========================//
// CONTENT //
//==========================//
int gpu_id;
element element[GRID_SIZE]; <------ static array of structures
int debug;
//==========================//
// SERIALIZATION //
//==========================//
template<class Archive>
void serialize(Archive & ar){
ar( gpu_id );
ar( debug );
ar( element );
}
};
SERIALIZATION FUNCTIONS
// cereal serialization
#include <cereal/archives/binary.hpp>
#include <cereal/archives/portable_binary.hpp>
//#include <cereal/archives/xml.hpp>
template<typename data_type>
int Process::serialize(data_type * data, char *serial_data, int *size){
// serialize the data
std::ostringstream oss(std::ios::binary);
cereal::BinaryOutputArchive ar(oss);
//cereal::XMLOutputArchive ar(std::cout);
ar(*data);
std::string s=oss.str();
*size = s.length();
strncpy(serial_data, s.c_str(), s.length());
std::cout << "buffer["<< s.length() << "] >> " << s << std::endl;
s.clear();
return _SUCCESS_;
};
template<typename data_type>
int Process::deserialize(data_type * data, char *serial_data, int size){
// create temporary buffer to store the received data
char * buf=new char[size];
strncpy(buf, serial_data, size);
std::istringstream iss(std::string(serial_data, size), std::ios::binary);
cereal::BinaryInputArchive arin(iss);
arin(*data);
// clean buffer
delete[] buf;
return _SUCCESS_;
};
DEBUGGING OUTPUT
cpu_mem BEFORE serialization
cpu_mem.debug = -1
cpu_mem.gpu_id = -5
cpu_mem.element[0].NN_id = 1
cpu_mem.element[0].debug = 2
buffer[248] >> �������� <------ binary format
cpu_mem AFTER deserialization
cpu_mem.debug = -1 <----- CORRECT
cpu_mem.gpu_id = -5 <----- CORRECT
cpu_mem.element[0].NN_id = 0 <----- INCORRECT
cpu_mem.element[0].debug = 0 <----- INCORRECT
If I play around a bit with the streams, I can print out the serialized object as XML, to check whether the serialization is successful.
<?xml version="1.0" encoding="utf-8"?>
<cereal>
<value0>
<gpu_id>-5</gpu_id> <----- CORRECT
<debug>-1</debug> <----- CORRECT
<element>
<value0>
<grid_id>0</grid_id>
<NN_id>1</NN_id> <----- CORRECT
<debug>2</debug> <----- CORRECT
</value0>
<value1>
<grid_id>32522</grid_id>
<NN_id>2</NN_id>
<debug>4612412</debug>
</value1>
</element>
</value0>
</cereal>
PROBLEM
The above outputs shows, that deserialization correctly recognizes variables in the container (class GPU_in_grid) but cannot deserialize the lower levels of my structure, namely static array of structures -> element[].

For some unknown reason to me, the strncpy() malfunctions. The content of the string differs from the content of the array, EVEN if I keep eye on '\0', which need to be appended at the end of the char array, shown here http://www.cplusplus.com/reference/string/string/copy/ .
I have ended up using std::string.copy() instead, which performs as expected, as long as you add '\0' at the end of the char array.
the above code works then just fine.

Related

Why is this binary output code causing a memory leak

I am new to C++ and I am trying to output a wave file in. I have had success with binary files in both C# and Java but I am not yet comfortable with C++ yet. I know about that arrays and objects should generally be created on the heap.
It is fine with the strings and the first getter
Whenever it gets to the second getter for the base class it runs out of memory.
This class is called waveWriter and it extends a class called WaveFormat that contains the getters
WaveWriter header:
class WaveWriter : WaveFormat {
private:
std::string fileName;
public:
WaveWriter(uint16_t channels, uint32_t sampleRate,
uint16_t bitsPerSample, double* lSampleData,
uint32_t lSampleLength, double* rSampleData,
uint32_t rSampleLength, bool isFloat, std::string outputName);
~WaveWriter();
void writeWave() {
std::ofstream myFile;
myFile = std::ofstream(fileName, std::ios::out | std::ios::binary);
// write the header
// sGroupID
myFile << S_GROUP_ID_HEADER;
// dwfilelength
myFile.write(reinterpret_cast<const char *> (GetDwFileLength()),
sizeof (GetDwFileLength()));
// sRiffType
myFile << S_RIFF_TYPE;
// write the format
// sGroupID
myFile << S_GROUP_ID_FORMAT;
// dwChunkSize
myFile.write(reinterpret_cast<const char *> (GetDwFormatChunkSize()),
sizeof (GetDwFormatChunkSize()));
// wFormatTag
........ blah blah blah
// close file
myFile.close();
return;
}
};
cpp for this class:
WaveWriter::WaveWriter(uint16_t channels, uint32_t sampleRate,
uint16_t bitsPerSample, double* lSampleData,
uint32_t lSampleLength, double* rSampleData,
uint32_t rSampleLength, bool isFloat, std::string outputName) :
WaveFormat(channels, sampleRate, bitsPerSample,
lSampleData, lSampleLength, rSampleData,
rSampleLength, isFloat) {
outputName.append(".wav");
this->fileName = outputName;
}
WaveWriter::~WaveWriter() {
this->~WaveFormat();
}
Header for WaveFormat contains private variables a constructor and getters like these to access the private variables:
public:
uint16_t GetCbSize() {
return cbSize;
}
uint32_t GetDwAvgBytesPerSec() {
return dwAvgBytesPerSec;
}
uint32_t GetDwChannelMask() {
return dwChannelMask;
}......
This is speculation based on your functions name, but I think this code:
myFile.write(reinterpret_cast<const char *> (GetDwFileLength()),sizeof (GetDwFileLength()));
is incorrect. Assuming GetDwFileLength() return size as value, it is incorrect to cast it to const char *. You need to save it in another argument and post its address to cast. Something like this:
auto val = GetDwFileLength();
myFile.write(reinterpret_cast<const char *> (&val), sizeof (val));
I see similar mistake several times in your code. This mistake can make invalid memory access.
In addition you should use virtual base destructor rather than calling base destructor from derived class. Never call base destructor in derived class.

Check an archive whether it is a binary/text/xml

I created an output text_archive and I restored it using binary archive and obviously, got some issues.
Can I know somehow the kind of archive, so that I could possibly use the appropriate code for binary/xml/text archive.
class Info
{
private:
// Allow serialization to access non-public data members.
friend class boost::serialization::access;
// Serialize the std::vector member of Info
template<class Archive>
void serialize(Archive & ar, const unsigned int version)
{
ar & filenames;
}
std::vector<std::string> filenames;
};
int main(int argc, char** argv)
{
Info info;
// Save filename data contained in Info object
{
// Create an output archive
std::ofstream ofs( "store.dat" );
**boost::archive::text_oarchive ar(ofs);**
ar & info;
}
// Restore from saved data and print to verify contents
Info restored_info;
{
// Create and input archive
std::ifstream ifs( "store.dat" );
**boost::archive::binary_iarchive ar(ifs);**
// Load the data
ar & restored_info;
}
return 0;
}
You can use first two bytes to specific type, let's say
00 for binary
01 for xml
10 for text
and the reset of the content is for data itself

How does boost::serialization allocate memory when deserializing through a pointer?

In short, I'd like to know how boost::serialization allocates memory for an object when deserializing through a pointer. Below, you'll find an example of my question, clearly illustrated alongside companion code. This code should be fully functional and compile fine, there are no errors, per se, just a question on how the code actually works.
#include <cstddef> // NULL
#include <iomanip>
#include <iostream>
#include <fstream>
#include <string>
#include <boost/archive/text_iarchive.hpp>
#include <boost/archive/text_oarchive.hpp>
class non_default_constructor; // Forward declaration for boost serialization namespacing below
// In order to "teach" boost how to save and load your class with a non-default-constructor, you must override these functions
// in the boost::serialization namespace. Prototype them here.
namespace boost { namespace serialization {
template<class Archive>
inline void save_construct_data(Archive& ar, const non_default_constructor* ndc, const unsigned int version);
template<class Archive>
inline void load_construct_data(Archive& ar, non_default_constructor* ndc, const unsigned int version);
}}
// Here is the actual class definition with no default constructor
class non_default_constructor
{
public:
explicit non_default_constructor(std::string initial)
: some_initial_value{initial}, state{0}
{
}
std::string get_initial_value() const { return some_initial_value; } // For save_construct_data
private:
std::string some_initial_value;
int state;
// Notice that we only serialize state here, not the
// some_initial_value passed into the ctor
friend class boost::serialization::access;
template<class Archive>
void serialize(Archive& ar, const unsigned int version)
{
std::cout << "serialize called" << std::endl;
ar & state;
}
};
// Define the save and load overides here.
namespace boost { namespace serialization {
template<class Archive>
inline void save_construct_data(Archive& ar, const non_default_constructor* ndc, const unsigned int version)
{
std::cout << "save_construct_data called." << std::endl;
ar << ndc->get_initial_value();
}
template<class Archive>
inline void load_construct_data(Archive& ar, non_default_constructor* ndc, const unsigned int version)
{
std::cout << "load_construct_data called." << std::endl;
std::string some_initial_value;
ar >> some_initial_value;
// Use placement new to construct a non_default_constructor class at the address of ndc
::new(ndc)non_default_constructor(some_initial_value);
}
}}
int main(int argc, char *argv[])
{
// Now lets say that we want to save and load a non_default_constructor class through a pointer.
non_default_constructor* my_non_default_constructor = new non_default_constructor{"initial value"};
std::ofstream outputStream("non_default_constructor.dat");
boost::archive::text_oarchive outputArchive(outputStream);
outputArchive << my_non_default_constructor;
outputStream.close();
// The above is all fine and dandy. We've serialized an object through a pointer.
// non_default_constructor will call save_construct_data then will call serialize()
// The output archive file will look exactly like this:
/*
22 serialization::archive 17 0 1 0
0 13 initial value 0
*/
/*If I want to load that class back into an object at a later time
I'd declare a pointer to a non_default_constructor */
non_default_constructor* load_from_archive;
// Notice load_from_archive was not initialized with any value. It doesn't make
// sense to intialize it with a value, because we're trying to load from
// a file, not create a whole new object with "new".
std::ifstream inputStream("non_default_constructor.dat");
boost::archive::text_iarchive inputArchive(inputStream);
// <><><> HERE IS WHERE I'M CONFUSED <><><>
inputArchive >> load_from_archive;
// The above should call load_construct_data which will attempt to
// construct a non_default_constructor object at the address of
// load_from_archive, but HOW DOES IT KNOW HOW MUCH MEMORY A NON_DEFAULT_CONSTRUCTOR
// class uses?? Placement new just constructs at the address, assuming
// memory at the passed address has been allocated for construction.
// So my question is this:
// I want to verify that *something* is (or isn't) allocating memory for a non_default_constructor
// class to be constructed at the address of load_from_archive.
std::cout << load_from_archive->get_initial_value() << std::endl; // This works.
return 0;
}
Per the boost::serialization documentation when a class with a non-default constructor is to be (de)serialized, the load/save_construct_data is used, but I'm not actually seeing a place where memory is being allocated for the object to be loaded into, just where placement new is constructing an object at a memory address. But what allocated the memory at that address?
It's probably a misunderstanding with how this line works:
::new(ndc)non_default_constructor(some_initial_value);
but I'd like to know where my misunderstanding lies. This is my first question, so I apologize if I've made some sort of mistake on how I've asked my question. Thanks for your time.
That's one excellent example program, with very apt comments. Let's dig in.
// In order to "teach" boost how to save and load your class with a
// non-default-constructor, you must override these functions in the
// boost::serialization namespace. Prototype them here.
You don't have to. Any overload (not override) accessible via ADL suffices, apart from the in-class option.
Skipping to the meat of it:
// So my question is this: I want to verify that *something* is (or isn't)
// allocating memory for a non_default_constructor
// class to be constructed at the address of load_from_archive.
Yes. The documentation states this. But it's a little bit trickier, because it's conditional. The reason is object tracking. Say, we serialize multiple pointers to the same object, they will get serialized once.
On deserialization, the objects will be represented in the archive stream with the object tracking-id. Only the first instance will lead to allocation.
See documentation.
Here's a simplified counter-example:
demonstrating ADL
demonstrating Object Tracking
removing all forward declarations (they're unnecessary due to template POI)
It serializes a vector with 10 copies of the pointer. I used unique_ptr to avoid leaking the instances (both the one manually created in main, as well as the one created by the deserialization).
Live On Coliru
#include <iomanip>
#include <iostream>
#include <fstream>
#include <boost/archive/text_iarchive.hpp>
#include <boost/archive/text_oarchive.hpp>
#include <boost/serialization/vector.hpp>
namespace mylib {
// Here is the actual class definition with no default constructor
class non_default_constructor {
public:
explicit non_default_constructor(std::string initial)
: some_initial_value{ initial }, state{ 0 } {}
std::string get_initial_value() const {
return some_initial_value;
} // For save_construct_data
private:
std::string some_initial_value;
int state;
// Notice that we only serialize state here, not the some_initial_value
// passed into the ctor
friend class boost::serialization::access;
template <class Archive> void serialize(Archive& ar, unsigned) {
std::cout << "serialize called" << std::endl;
ar& state;
}
};
// Define the save and load overides here.
template<class Archive>
inline void save_construct_data(Archive& ar, const non_default_constructor* ndc, unsigned)
{
std::cout << "save_construct_data called." << std::endl;
ar << ndc->get_initial_value();
}
template<class Archive>
inline void load_construct_data(Archive& ar, non_default_constructor* ndc, unsigned)
{
std::cout << "load_construct_data called." << std::endl;
std::string some_initial_value;
ar >> some_initial_value;
// Use placement new to construct a non_default_constructor class at the address of ndc
::new(ndc)non_default_constructor(some_initial_value);
}
}
int main() {
using NDC = mylib::non_default_constructor;
auto owned = std::make_unique<NDC>("initial value");
{
std::ofstream outputStream("vector.dat");
boost::archive::text_oarchive outputArchive(outputStream);
// serialize 10 copues, for fun
std::vector v(10, owned.get());
outputArchive << v;
}
/*
22 serialization::archive 17 0 0 10 0 1 1 0
0 13 initial value 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0
*/
std::vector<NDC*> restore;
{
std::ifstream inputStream("vector.dat");
boost::archive::text_iarchive inputArchive(inputStream);
inputArchive >> restore;
}
std::unique_ptr<NDC> take_ownership(restore.front());
for (auto& el : restore) {
assert(el == take_ownership.get());
}
std::cout << "restored: " << restore.size() << " copies with " <<
std::quoted(take_ownership->get_initial_value()) << "\n";
}
Prints
save_construct_data called.
serialize called
load_construct_data called.
serialize called
restored: 10 copies with "initial value"
The vector.dat file contains:
22 serialization::archive 17 0 0 10 0 1 1 0
0 13 initial value 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0
The Library Internals
You shouldn't really care, but you can of course read the source code. Predictably, it's way more involved than you'd naively expect, after all: this is C++.
The library deals with types that have overloaded operator new. In that case it calls T::operator new instead of the globale operator new. It always passes sizeof(T) as you correctly surmised.
The code lives in the exception-safe wrapper: detail/iserializer.hpp
struct heap_allocation {
explicit heap_allocation() { m_p = invoke_new(); }
~heap_allocation() {
if (0 != m_p)
invoke_delete(m_p);
}
T* get() const { return m_p; }
T* release() {
T* p = m_p;
m_p = 0;
return p;
}
private:
T* m_p;
};
Yes, this code be simplified a lot with C++11 or later. Also, the NULL-guard in the destructor is redunant for compliant implementations of operator delete.
Now of course, invoke_new and invoke_delete are where it's at. Presenting condensed:
static T* invoke_new() {
typedef typename mpl::eval_if<boost::has_new_operator<T>,
mpl::identity<has_new_operator>,
mpl::identity<doesnt_have_new_operator>>::type typex;
return typex::invoke_new();
}
static void invoke_delete(T* t) {
typedef typename mpl::eval_if<boost::has_new_operator<T>,
mpl::identity<has_new_operator>,
mpl::identity<doesnt_have_new_operator>>::type typex;
typex::invoke_delete(t);
}
struct has_new_operator {
static T* invoke_new() { return static_cast<T*>((T::operator new)(sizeof(T))); }
static void invoke_delete(T* t) { (operator delete)(t); }
};
struct doesnt_have_new_operator {
static T* invoke_new() { return static_cast<T*>(operator new(sizeof(T))); }
static void invoke_delete(T* t) { (operator delete)(t); }
};
There's some conditional compilation and verbose comments, so per-use the source code if you want the full picture.

c++ Read/Write class from/to binary file

I need to write a class to a binary file, and then I need to read it back.
I have Triangle and BinaryFile classes, and some other classes. I am not sure if I am writing incorrectly or reading incorrectly. An error occurs when reading. After debugging, I think that it gets inappropriate data for my private variables. I will be very glad if someone can give me some advice on how to make it work properly.
I wasn't sure if I should paste the whole code or not, so I will give you a short snippet of code. Just in case, here is a download link for my source code:
https://my.pcloud.com/publink/show?code=XZJ7CYZbsLWLglqV5p83csijcEUTFqqpM3k
I am a newbie in programming and I don't speak English very well, so I apologize in advance for my mistakes.
class Point
{
private:
int x;
int y;
};
class Figure
{
private:
string name;
string type;
};
class Triangle: public Figure
{
private:
Point p1, p2, p3;
};
class BinaryFile
{
private:
string FileName;
fstream File;
public:
//...
void AddNewFigure(istream& stream)
{
File.open(this->FileName, ios::binary | ios::app);
if(!this->File)
{
cerr<<"File error <"<<this->FileName<<">\n";
exit(1);
}
Triangle fig;
fig.MakeNewFigure(stream);
File.write((char*)&fig, sizeof(Triangle));
File.close();
}
Triangle GetTriangle()
{
Triangle trig;
Point p;
string str(""); int x(0);
File.open(this->FileName, ios::binary | ios::in);
if(!this->File)
{
cerr<<"File error <"<<this->FileName<<">\n";
exit(1);
}
File.read((char*)&trig, sizeof(Triangle));
File.close();
return trig;
}
};
The answer depends on whether you are just doing this to learn how files work or whether saving to the file is just incidental and you don't care how it works.
If you just want to get the stuff to save and restore and you don't care how it works then use a third party library. There are many many of them.
If you want to learn how to read and write things to files then you will need to make your own read and write functions. I have made a sample program that will explain how it works:
#include <string>
#include <fstream>
#include <iostream>
class Point
{
private:
int x;
int y;
public:
Point():x(0),y(0){}
Point(int x,int y):x(x),y(y){}
void write(std::ostream& f)
{
// We can just write out the bytes for x and y because
// they are primitive types stored in the class
f.write( (char*)&x, sizeof(x) );
f.write( (char*)&y, sizeof(y) );
}
void read(std::istream& f)
{
// We can just read the bytes directly into x and y because
// they are primitive types stored in the class
f.read( (char*)&x, sizeof(x) );
f.read( (char*)&y, sizeof(y) );
}
};
class Figure
{
private:
std::string name;
std::string type;
public:
Figure(){}
Figure(std::string name,std::string type):name(name),type(type){}
void write(std::ostream& f)
{
size_t size;
// we need to store the data from the string along with the size
// because to restore it we need to temporarily read it somewhere
// before storing it in the std::string (istream::read() doesn't
// read directly to std::string)
size = name.size();
f.write( (char*)&size, sizeof(size_t) );
f.write( (char*)name.c_str(), size );
size = type.size();
f.write( (char*)&size, sizeof(size_t) );
f.write( (char*)type.c_str(), size );
}
void read(std::istream& f)
{
size_t size;
char *data;
// when we read the string data we need somewhere to store it
// because we std::string isn't a primitive type. So we read
// the size, allocate an array, read the data into the array,
// load the std::string, and delete the array
f.read( (char*)&size, sizeof(size) );
data = new char[size+1];
f.read( data, size );
data[size]='\0';
name = data;
delete data;
f.read( (char*)&size, sizeof(size) );
data = new char[size+1];
f.read( data, size );
data[size]='\0';
type = data;
delete data;
}
};
class Triangle: public Figure
{
private:
Point p1, p2, p3;
public:
Triangle(){}
Triangle(Point x,Point y,Point z,Figure f):p1(x),p2(y),p3(z),Figure(f){}
void write(std::ostream& f)
{
// First write the base class then write the members of this class
Figure::write(f);
p1.write(f);
p2.write(f);
p3.write(f);
}
void read(std::istream& f)
{
// First read the base class then read the members of this class
Figure::read(f);
p1.read(f);
p2.read(f);
p3.read(f);
}
};
class BinaryFile
{
private:
std::string FileName;
std::fstream File;
public:
BinaryFile(std::string FileName) : FileName(FileName) {};
void WriteTriangle()
{
File.open(FileName, std::ios::binary | std::ios::out);
if(!File)
{
std::cerr<<"File error <"<<FileName<<">\n";
exit(1);
}
Triangle trig({1,2},{3,4},{5,6},{"name","type"}); // something new
trig.write(File);
File.close();
}
Triangle ReadTriangle()
{
File.open(FileName, std::ios::binary | std::ios::in);
if(!File)
{
std::cerr<<"File error <"<<FileName<<">\n";
exit(1);
}
Triangle trig; // default values
trig.read(File);
File.close();
return trig;
}
};
main()
{
BinaryFile bin("file.bin");
bin.WriteTriangle();
Triangle trig = bin.ReadTriangle();
// at this point trig has the values we stored
return 0;
}
It's not easy to reproduce the error, due to your large source code and missing data file. But a quick inspection shows that you read and write the binary data using bloc operations:
Triangle trig;
...
File.read((char*)&trig, sizeof(Triangle));
Unfortunately this kind of approach only works if the object you want to save/load is of a class that is trivially copyable, as the following code will demonstrate:
if (is_trivially_copyable<Triangle>::value)
cout << "Triangle is trivially copyable" << endl;
else cout << "Triangle is not trivially copyable" << endl;
So you'll have to serialize the object content writing field by field instead of using a bloc operation. This FAQ on serialization should help you to consider the alternatives.
What you are looking for is to serialize your classes/data that should be saved to file. There are several libraries that has been optimized regarding time and memory consumption for this. Would you mind using a 3rd party library?
If not, have a look at for example boost serialization, cereal or maybe even Google's ProtoBuf. I recon Cereal is a good start if you are using C++11.
If you'd like to write your own serialization you'd have to consider that for every object that has a dynamic size (such as a string), you will also need to save the object's size to the file. For more info please have a look here:
https://stackoverflow.com/a/11003590/5874704

Derived class serialization without class tracking in Boost (C++)

I have some problems with boost serialization when serializing derived class through base class pointer. I need a system which serializes some objects as they are being received in the system, so I need to serialize over time. This is not really a problem since I can open a boost::archive::binary_oarchive and serialize objects when required. Rapidly I noticed that boost was performing object tracking by memory address, so the first problem was that different objects in time that share the same memory address were saved as the same object. This can be fixed by using the following macro in the required derived class:
BOOST_CLASS_TRACKING(className, boost::serialization::track_never)
This works fine, but again, when the base class is not abstract, the base class is not serialized properly. In the following example, the base class serialization method is only called once with the first object. In the following, boost assumes that this object has been serialized before although the object has different type.
#include <iostream>
#include <fstream>
#include <boost/serialization/export.hpp>
#include <boost/serialization/base_object.hpp>
#include <boost/serialization/list.hpp>
#include <boost/serialization/map.hpp>
#include <boost/serialization/vector.hpp>
#include <boost/serialization/shared_ptr.hpp>
#include <boost/archive/archive_exception.hpp>
#include <boost/archive/binary_oarchive.hpp>
#include <boost/archive/binary_iarchive.hpp>
using namespace std;
class AClass{
public:
AClass(){}
virtual ~AClass(){}
private:
double a;
double b;
//virtual void virtualMethod() = 0;
private:
friend class boost::serialization::access;
template<class Archive>
void serialize(Archive & ar, const unsigned int version)
{
ar & a;
ar & b;
cout << "A" << endl;
}
};
//BOOST_SERIALIZATION_ASSUME_ABSTRACT(Aclass)
//BOOST_CLASS_TRACKING(AClass, boost::serialization::track_never)
class BClass : public AClass{
public:
BClass(){}
virtual ~BClass(){}
private:
double c;
double d;
virtual void virtualMethod(){};
private:
friend class boost::serialization::access;
template<class Archive>
void serialize(Archive & ar, const unsigned int version)
{
ar & boost::serialization::base_object<AClass>(*this);
ar & c;
ar & d;
cout << "B" << endl;
}
};
// define export to be able to serialize through base class pointer
BOOST_CLASS_EXPORT(BClass)
BOOST_CLASS_TRACKING(BClass, boost::serialization::track_never)
class CClass : public AClass{
public:
CClass(){}
virtual ~CClass(){}
private:
double c;
double d;
virtual void virtualMethod(){};
private:
friend class boost::serialization::access;
template<class Archive>
void serialize(Archive & ar, const unsigned int version)
{
ar & boost::serialization::base_object<AClass>(*this);
ar & c;
ar & d;
cout << "C" << endl;
}
};
// define export to be able to serialize through base class pointer
BOOST_CLASS_EXPORT(CClass)
BOOST_CLASS_TRACKING(CClass, boost::serialization::track_never)
int main() {
cout << "Serializing...." << endl;
{
ofstream ofs("serialization.dat");
boost::archive::binary_oarchive oa(ofs);
for(int i=0;i<5;i++)
{
AClass* baseClassPointer = new BClass();
// serialize object through base pointer
oa << baseClassPointer;
// free the pointer so next allocation can reuse memory address
delete baseClassPointer;
}
for(int i=0;i<5;i++)
{
AClass* baseClassPointer = new CClass();
// serialize object through base pointer
oa << baseClassPointer;
// free the pointer so next allocation can reuse memory address
delete baseClassPointer;
}
}
getchar();
cout << "Deserializing..." << endl;
{
ifstream ifs("serialization.dat");
boost::archive::binary_iarchive ia(ifs);
try{
while(true){
AClass* a;
ia >> a;
delete a;
}
}catch(boost::archive::archive_exception const& e)
{
}
}
return 0;
}
When executing this piece of code, the result is as follow:
Serializing....
A
B
B
B
B
B
C
C
C
C
C
Deserializing...
A
B
B
B
B
B
C
C
C
C
C
So the base class is only being serialized once, although the derived class has explicitly the track_never flag. There are two different workarounds to fix this behaviour. The first one is to make the base class abstract with a pure virtual method and calling the macro BOOST_SERIALIZATION_ASSUME_ABSTRACT(Aclass), and the second one is to put the track_never flag also in the base class (commented in code).
None of these solutions meets my requirements, since I want to do in the future punctual serializations of the system state, which would require tracking features for a given DClass extending A (not B or C), and also the AClass should not be abstract.
Any hints? Is there any way to call explicitly the base class serialization method avoiding the tracking feature in the base class (that already has been disabled in the derived class)?
After having a little closer look to boost::serialization I'm also convinced there is no straightforward solution for you request.
As you already mentioned the tracking behavior for the serialization is declared on a class by class base with BOOST_CLASS_TRACKING.
This const global information is than interpret in the virtual method tracking from class oserializer.
virtual bool tracking(const unsigned int /* flags */)
Because this is a template class you can explicitly instantiate this method for your classes.
namespace boost {
namespace archive {
namespace detail {
template<>
virtual bool oserializer<class binary_oarchive, class AClass >::tracking(const unsigned int f /* flags */) const {
return do_your_own_tracking_decision();
}
}}}
Now you can try to e.g have something like a global variable and change the tracking behavior from time to time. (E.g depending on which derivate class is written to the archive.)
This seems to wok for “Serializing“ but the “Deserializing“ than throw an exception.
The reason for this is, that the state of “tracking” for each class is only written ones to the archive. Therefore the deserialize does always expect the data for AClass if BClass or CClass is read (at leased if the first write attempt for AClass was with tracking disabled).
One possible solution could be to use the flags parameter in tracking() method.
This parameter represent the flags the archive is created with, default “0”.
binary_oarchive(std::ostream & os, unsigned int flags = 0)
The archive flags are declared in basic_archive.hpp
enum archive_flags {
no_header = 1, // suppress archive header info
no_codecvt = 2, // suppress alteration of codecvt facet
no_xml_tag_checking = 4, // suppress checking of xml tags
no_tracking = 8, // suppress ALL tracking
flags_last = 8
};
no_tracking seems currently not to be supported, but you can now add this behavior to tracking.
template<>
virtual bool oserializer<class binary_oarchive, class AClass >::tracking(const unsigned int f /* flags */) const {
return !(f & no_tracking);
}
Now you can at leased decide for different archives whether AClass should be tracked or not.
boost::archive::binary_oarchive oa_nt(ofs, boost::archive::archive_flags::no_tracking);
And this are the changes in your example.
int main() {
cout << "Serializing...." << endl;
{
ofstream ofs("serialization1.dat");
boost::archive::binary_oarchive oa_nt(ofs, boost::archive::archive_flags::no_tracking);
//boost::archive::binary_oarchive oa(ofs);
for(int i=0;i<5;i++)
{
AClass* baseClassPointer = new BClass();
// serialize object through base pointer
oa_nt << baseClassPointer;
// free the pointer so next allocation can reuse memory address
delete baseClassPointer;
}
ofstream ofs2("serialization2.dat");
boost::archive::binary_oarchive oa(ofs2);
//boost::archive::binary_oarchive oa(ofs);
for(int i=0;i<5;i++)
{
AClass* baseClassPointer = new CClass();
// serialize object through base pointer
oa << baseClassPointer;
// free the pointer so next allocation can reuse memory address
delete baseClassPointer;
}
}
getchar();
cout << "Deserializing..." << endl;
{
ifstream ifs("serialization1.dat");
boost::archive::binary_iarchive ia(ifs);
try{
while(true){
AClass* a;
ia >> a;
delete a;
}
}catch(boost::archive::archive_exception const& e)
{
}
ifstream ifs2("serialization2.dat");
boost::archive::binary_iarchive ia2(ifs2);
try{
while(true){
AClass* a;
ia2 >> a;
delete a;
}
}catch(boost::archive::archive_exception const& e)
{
}
}
return 0;
}
namespace boost {
namespace archive {
namespace detail {
template<>
virtual bool oserializer<class binary_oarchive, class AClass >::tracking(const unsigned int f /* flags */) const {
return !(f & no_tracking);
}
}}}
This still may not be what you are looking for. There are lot more methods which could be adapted with an own implementation. Or your have to derivate your own archive class.
Ultimately the problem seems to be that a boost::serialization archive represents state at a single point in time, and you want your archive to contain state that has changed, i.e. pointers that have been reused. I don't think there is a simple boost::serialization flag that induces the behavior you want.
However, I think there are other workarounds that might be sufficient. You can encapsulate the serialization for a class into its own archive, and then archive the encapsulation. That is, you can implement the serialization for B like this (note that you have to split serialize() into save() and load()):
// #include <boost/serialization/split_member.hpp>
// #include <boost/serialization/string.hpp>
// Replace serialize() member function with this.
template<class Archive>
void save(Archive& ar, const unsigned int version) const {
// Serialize instance to a string (or other container).
// std::stringstream used here for simplicity. You can avoid
// some buffer copying with alternative stream classes that
// directly access an external container or iterator range.
std::ostringstream os;
boost::archive::binary_oarchive oa(os);
oa << boost::serialization::base_object<AClass>(*this);
oa << c;
oa << d;
// Archive string to top level.
const std::string s = os.str();
ar & s;
cout << "B" << endl;
}
template<class Archive>
void load(Archive& ar, const unsigned int version) {
// Unarchive string from top level.
std::string s;
ar & s;
// Deserialize instance from string.
std::istringstream is(s);
boost::archive::binary_iarchive ia(is);
ia >> boost::serialization::base_object<AClass>(*this);
ia >> c;
ia >> d;
cout << "B" << endl;
}
BOOST_SERIALIZATION_SPLIT_MEMBER()
Because each instance of B is serialized into its own archive, A is effectively not tracked because there is only one reference per archive of B. This produces:
Serializing....
A
B
A
B
A
B
A
B
A
B
A
C
C
C
C
C
Deserializing...
A
B
A
B
A
B
A
B
A
B
A
C
C
C
C
C
A potential objection to this technique is the storage overhead of encapsulation. The result of the original test program are 319 bytes while the modified test program produces 664 bytes. However, if gzip is applied to both output files then the sizes are 113 bytes for the original and 116 bytes for the modification. If space is a concern then I would recommend adding compression to the outer serialization, which can be easily done with boost::iostreams.
Another possible workaround is to extend the life of instances to the lifespan of the archive so pointers are not reused. You could do this by associating a container of shared_ptr instances to your archive, or by allocating instances from a memory pool.