Serialize/ Deserialize C++ classes - c++

I'm looking for a way to send C++ class between 2 clients aptication.
I was looking for a way doing so and all i can find is that I need to create for each Class Serialize/ Deserialize (to JSON for example) functions and send it over TCP/IP.
The main problem I'm faceing is that I have ~600 classes (some are classes including instances of others) that I need to pass which mean I need to spent the next writing Serialize/ Deserialize functions.
Is there any generic way writing Serialize/Deserialize functions ?
Is there any other way sending C++ classes ?
Thanks,
Guy Ergas.

Are you using a Framework at all? Qt and MFC for example have built in Serialization that would make your task easier. Otherwise I would guess that you'd need to spend at least some effort on each of the 600 classes.
As recommended above Boost Serialization is probably a good way to go, you can send the serialized class over Tcp using Boost Asio too:
http://www.boost.org/doc/libs/1_54_0/doc/html/boost_asio.html
Alternatively, there is a C++ API for Google Protocol Buffers (protobuf):
https://developers.google.com/protocol-buffers/docs/reference/cpp/

Boost Serialization
Although I haven't used it my self, it is very popular around my peers at work.
More info about it can be found in "Boost (1.54.00) Serialization"
Thrift
Thrift have a very limited serialize functionality which I don't think fits your requirements. But it can help you "move" the data from one client to anther even if they are using different languages.
More info about it can be found in "Thrift: The Missing Guide"

try s11n or nosjob
s11n (an abbreviation for serialization) is an Open Source project
focused on the generic serialization of objects (i.e., object
persistence) in the C++ programming language.
nosjob, a C++ library for generating and consuming JSON data.

You may be interested in ASN.1. It's not necessarily the easiest to use and tools/libraries are a little hard to come by (Objective Systems at http://www.obj-sys.com/index.php is worth a look, though not free).
However the big advantage is that it is very heavily standardised (so no trouble with library version incompatibilities) and most languages are supported one way or another. Handy if you need support across multiple platforms. It also does binary encodings, so its way less bloaty than XML (which it also supports). I chose it for these reasons and didn't regret it.

If you are at linux platform , You can directly use json.h library for serialization.
Here is sample code i have come across :)
Json Serializer

Related

Best way to "mangle" (represent) the memory

I would like to know what would be the best way to map/represent the memory. I mean, how to describe, for example, a structure with all its field to be serialized.
I am creating a RPC library that will create the client and server using the dwarf debug data, so i need to create a function wrapper to serialize and deserialize the functions´s parameters.
Now, i am using the gcc mangling types to identify all the fields, but the compiler sometimes creates holes to optimize the memory access time;
Any idea ?
I use the "cereal" library for serialization (http://uscilab.github.io/cereal/)
Alternatives include Google's Protocol Buffers, although I found it too difficult to integrate for my comparably simple serialization tasks.
For communication between processes, and languages, I've had a good experience with ZeroC's ICE library (https://zeroc.com/products/ice). You specific the structure as an external compilation step similar to Google's Protocol Buffers. The nice part is that the network connect was also taken care off.

Google Protocol Buffers in C++: Creating a message from an existing struct

I'm considering Google protocol buffers as a solution to my problem of communication between C++ and C# using named pipes. But I have one concern: all I've been able to find on protobuf is how to create a message from a prototype using protobuf compiler. This is neat, but I would also need to be able to serialize existing structs. I can't seem to find any info (but maybe I'm overlooking it). Do you know if it is possible to serialize a struct in C++ using protobufs, so it can be read in .NET, without modifying said existing struct?
Yes and no.
It's possible. In fact, I have done it. Not the .NET loading part, but the serialization to protobuf and the generation of a prototype from a C++ class. However, doing so requires a number of things and is not that easy.
First of all, protobufs are quite limited in their ability to represent data. They are basically only capable of representing POD-types (in the C++ sense), and very little else. I personally had to add a few basic things to the format to make it into a proper full-featured serialization format. But if you restrict yourself to POD-types, then the plain protobuf format will work fine.
The second thing is that you'll need a serialization library of some kind, and that will require that you add some code for each struct / class to perform the serialization / de-serialization (not necessarily "intrusively", meaning that you might not have to change the classes, just add some code on the side). You can look at Boost.Serialization, that's the basic template for how to create a serialization library in C++. Boost.Serialization is not particularly flexible for this purpose, and so, you might have to change a few things (like I had to do).
The third thing is that you will need quite a bit of wizardry under-the-hood to make this happen. In particular, you are going to need a reliable and feature-rich run-time type identification system (RTTI) in order to able to have useful type names, and you might need to clever meta-programming or some intrusive class hierarchy to be able to detect user-defined types for which you need to generate a prototype.
So, that's why my answer is "yes and no" because it is possible, but not without quite a bit of work and a good framework to rely on.
N.B.: Writing code to encode/decode data into the proto-buf format (with those small-ints, and all that) is really the easy part, proto-buf format is so simple, it's almost laughable. Writing the serialization framework that will allow you to do fancy things like generating prototypes, that's the hard part.

Working with protobuf and POCOs in C++

I would like to use protobuf with a C++ project I'm working on.
However, I don't like to work with the auto-generated classes protoc creates and prefer to stick with the POCOs I already have. This is because the POCOs are already in use in other parts of the code and I want to be able to switch the serialization mechanism with ease later on. But manually writing converters between POCOs and protobuf message classes seems tedious and wrong.
I want to know if there's a way to use protobuf to create a serializer - an auto-generated class that will be able to serialize and deserialize my POCOs, without bugging me with internals.
Thanks.
First, you may like Cap'n Proto better, it was created by one of Google's former Google Protocol Buffer maintainers. Worth looking into, anyway.
But otherwise, you really need to consider why you're using Google Protocol Buffers.
If you want to achieve the forward and backward compatibility, and to be able to open, then edit, then save an object that possibly a different person created, with a different version of your protocol buffer declaration, and then sent along to yet another person with an even different version of the declaration... then you need to just bite the bullet and use the generated C++ from the Google Protocol Buffer Compiler.
It's really not just a serialization format. It's specifically designed to make it easy living with different versions of your serialization, over time.
If you don't need that flexibility, and you don't like the generated code, you may want to consider a different serialization tool.

Thrift or Protocol buffer as cross-language serialization solution?

I've already chosen to use thrift as RPC framework in a project. This project has a lot of serialization / deserialization operations (e.g., store the data to disks). And the serialized format should be accessible for at least C++/Java/Python. It seems that thrift's serialization solution is more complicated than Protobuf (e.g., it needs to create a protocol before serializing an object).
So my question is: is it worth to use Protobuf for the serialization / deserialization part even if thrift is capable of this task?
I would agree that Thrift is a better choice for cross language RPC than Protobuf RPC ( see http://pjklauser.wordpress.com/2013/02/27/why-googles-protobuf-rpc-will-not-reach-widespread-adoption/ ). If you're using thrift already it's difficult to justify using a different "library" for serialization to file/storage. You'll need to write endless mapping code. Both libraries will have different maintenance cycles which you need to maintain independently which will give extra future effort. The cost of writing a line or two more code, or save one or two bytes of space, or save a microsecond of CPU time will be nothing compared to your additional efforts.

Best XML serialization library for a MFC C++ app

I have an application, written in C++ using MFC and Stingray libraries. The application works with a wide variety of large data types, which are all currently serialized based on MFC Document/View serialize derived functionality. I have also added options for XML serialization based on the Stingray libraries, which implements DOM via the Microsoft XML SDK. While easy to implement the performance is terrible, to the extent that it is unusable on anything other than very small documents.
What other XML serialization tools would you folks recommend for this scenario. I don't want DOM, as it seems to be a memory hog, and I'm already dealing with large in memory data. Ideally, i'd like a streaming parser that is fast, and easy to use with MFC. My current front runner is expat which is fast and simple, but would require a lot of class by class serialization code to be added. Any other efficient and easier to implement alternatives out there that people would recommend?
The Boost Serialization library supports XML. This library basically consists in:
Start from the principles of MFC serialization and take all the good things it provides.
Solve every single issue of MFC serialization!
Among the improvements compared to MFC is support for XML.
Note that you don't necessarily control the XML schema of this serialization. It uses its own schema.
This is an age old problem. I was the team lead of the development team with the most critical path dependencies on the largest software project in the world during 1999 and 2000 and this very issue was the focus of my work during that time. I am convinced that the wheel was invented by multiple engineers who were unaware that others had already invented it. The same is true of XML Data binding in C++. I invented it too, and I've been perfecting it for over 10 years on various projects. I have a solution that addresses the issues noted here and some additional issues that repeatedly arise:
XML Updates. This is the ability to re-apply a subset of XML into an existing object model. In many cases the XML is bound to indexed objects and we cannot afford to re-index for each update.
COM and CORBA interface management. In the same respect that the XML Data Binding can be automated through object oriented practices - so can the instances of interface objects that provide that data to the application layer.
State Tracking. The application often needs to distinguish between an empty value vs. a missing value - both create an empty string. This provides the validation along with Data Binding.
The source code uses the least restrictive license - less so that GPL. The project is supported and managed from here:
http://www.codeproject.com/KB/XML/XMLFoundation.aspx
Now that it's the year 2010, I believe that nobody else will attempt to reinvent the wheel because there are a few to choose from. IMHO - this wheel is the most polished and well rounded implementation available.
Enjoy.
A good solution would be libxml. It provides lightweight SAX parsing and data structures for XML processing. There are several DOM libraries which are built on top of libxml.
Unfortunatly it is a C library, but C++ wrappers are available.
A few years ago I switched from MSXML to libxml because of the performance issues you mentioned.
If you decide to use libxml, you should also take a look at libxslt.
We use Xerces-C++. It was easy to setup and performance is good enough so we don't need to think about changing. However we aren't XML heavy.
I did listen to a podcast by Scott Hanselman (from Hansel Minutes) where they discuss the XML performance of MSXML and XSLT.
what about RapidXML, I am using it in an MFC app with some modification to support UTF-16 with std::string. I am quite satisfied with it so far.
The gSOAP toolkit auto-serializes native C and C++ data to/from XML and supports the full XML schema specification through XML data bindings:
gSOAP SourceForge Project
It has evolved since 1999 to a significant code base with code generation tools and libraries. It supports many databinding and customization features, which is especially critical for mapping XML schema types to/from the C and C++ types. It can serialize any C/C++ type and also STL containers, container templates, and cyclic data structures. It has been used in the W3C Schema Patterns for Databinding working group (with 100% schema pattern coverage success since years). There is an active open source user base and the gSOAP development functionality has been used in many industrial projects and Fortune 100 companies to develop SOAP/XML infrastructures.
This is late in the game, I just want to mention that we also use LIBXML. It's robust and reliable, and has worked well. A little bit too low-level, you'll want to build some wrappers on top of its functions.
For instance, you'll get a different sequence of function returns depending on whether you have this:
<tag attribute="value"/>
or this:
<tag attribute="value"> </tag>
Sometimes you may want that, sometimes you don't care.
We use TinyXML for all our XML needs be it MFC or straight C++.
http://sourceforge.net/projects/tinyxml