What I am trying to do is to convert a mapimessage object (LPMESSAGE) into a transferable format let's say to serialize it into bytes ( I prefer this approach), or to xml format. What is the best practice in this case? And how to do it? Is there a library to do so?
NOTE: I can convert lpmessage into mime and back, but I noticed it loses much of its properties when I use the iconversionsession.
If you have access to these classes' internals (and I belive you do), I would advise you to go through this FAQ on serialization: https://isocpp.org/wiki/faq/serialization. There's everything you would want to know on this topic. (hint: if you don't then you can always derive from them and extend their interface with additional, serialization method.)
If you are not interested in implementing your own solution, you could try 3rd party, like boost serialization lib: http://www.boost.org/doc/libs/1_60_0/libs/serialization/doc/index.html
You can convert the IMessage to a .MSG file. While not a perfect process, most properties are preserved, most of the time, and Outlook can open these serialized messages so they're easier to use (and verify). Search for code samples for OpenIMsgOnIStg.
If you don't want to use MSG as your serialization format, then you'll have to roll your own. IMAPIProp objects are just property bags with numeric ID's, but all of the different types of property values will have to be persisted differently.
Related
I have to create an application layer protocol for a C++ application, but I have some doubts about how I can do it, especially about the serialization:
My idea is to create a class for describing the header, something like this:
class Header {
int type;
int length;
char[] message;
}
Now, in order to serialize it and to pass it through a socket, I'm thinking about using Boost Serialization. But my question is: is it "cross-platform"? In the sense that if I want to receive the data into a Python/Ruby/any-other-language server with its own class, can I do it or not (since I've serialized a C++ class)?
If not, is useful to serialize the class data into a JSON/XML file and transmit it?
If I want to serialize an object into a string, Does I have to pay attention to the big/little-endian and/or the string encoding and/or other details?
Since, not all the machines using the same number of bytes to define, for example, the primitive data types, is it necessary to use something like uint32_t data types to force the system to use a certain amount of bytes?
Thank you very much!
I think you should check this answer first.
Serializing into a C-style string is fine as far as big/little endian woes go, but is not as good performance-wise.
Using C-style string serialization will mostly solve this problem as well. On read side you should make sure that when parsing numbers they don't exceed local data size.
If this is a serious project then maybe consider using JSON or XML.
I wonder how could I serialize an object of a given class (e.g. Person) with its attributes (e.g. name, age) to a JSON string using POCO C++ libraries.
Maybe I should create my models using Poco::Dynamic and Poco::Dynamic::Var in order to use POCO::JSON::Stringifier? I can't imagine how to do this...
Thanks in advance!
Unlike Java or C#, C++ doesn't have an introspection/reflection feature outside of Run-time type information (RTTI), which has a different focus and is limited to polymorphic objects. That means outside of a non-standard pre-compiler, you'll have to tell the serialisation framework one way or another how your object is structured and how you would eventually like to map it to a hierarchy of int, std::string and other basic data types. I usually differentiate between three different approaches to do so: pre-compiler, inline specification, property conversion.
Pre-compiler: A good example of the pre-compiler approach is Google Protocol Buffers: https://developers.google.com/protocol-buffers/docs/cpptutorial. You define your entities in a separate .proto file, which is transformed using a proprietary compiler to .c and .h entity classes. These classes can be used like regular POCO entities and can be serialised using Protocol Buffers.
Inline specification: Boost serialization (https://www.boost.org/doc/libs/1_67_0/libs/serialization/doc/index.html), s11n (www.s11n.net) and restc-cpp (https://github.com/jgaa/restc-cpp) are examples of explicitly specifying the structure of your POCOs for the framework inside your own code. The API to do so may be more or less sophisticated, but the principle behind it is always the same: You provide the framework serialise/deserialise implementations for your classes or you register metadata information which allows the framework to generate them. The example below is from restc-cpp:
struct Post {
int userId = 0;
int id = 0;
string title;
string body;
};
BOOST_FUSION_ADAPT_STRUCT(
Post,
(int, userId)
(int, id)
(string, title)
(string, body)
)
Property conversion: The last kind of serialisation that I don't want to miss mentioning is the explicit conversion to a framework-provided intermediate data type. Boost property tree (https://www.boost.org/doc/libs/1_67_0/doc/html/property_tree.html) and JsonCpp (http://open-source-parsers.github.io/jsoncpp-docs/doxygen/index.html) are good examples of this approach. You are responsible for implementing a conversion from your own types to ptree, which Boost can serialise to and from any format you like (XML, JSON).
Having had my share of experience with all three approaches in C++, I would recommend option 3 as your default. It seems to map nicely to POCO C++'s Parser and Var model for JSON. One option is to have all your entity POCO classes implement a to_var or from_var function, or you can keep these serialisation functions in a different namespace for each POCO class, so that you only have to include them when necessary.
If you are working on projects with a significant number of objects to serialise (e.g. messages in communication libraries), the pre-compiler option may be worth the initial setup effort and additional build complexity, but that depends, as always, on the specific project you're dealing with.
Is there any OCaml library that would serialize my data along with type information? For example: serialize [1;2;3] may give me <int list: (1,2,3)> or similar?
I want to store the type information with the data. Is it possible?
I tried looking at Sexplib, but I cannot figure out if they support this.
Thanks!
If all that you need is to store typenames, then you can use typerep library, to retrieve them, and then adapt sexp converters to handle this information.
Another option is to store data actually as a valid OCaml string, and to load it using compiler libs, i.e. actually parse and evaluate. In that case you can store types or whatever you want. You may find odn library useful for dumping data as OCaml.
I think sexplib or bin_prot are the current best bet. Sexplib does not attach type names to the data and I guess bin_prot does not either, but type safety is assured at the reader side.
If you really want "type information" attached to the serialized data, you can write your own serializer using ppx_deriving. It is rather new but much easier than writing CamlP4 based serializer like sexplib, bin_prot and meta_conv.
I'm asking here because i have already tried to search but i have no idea if this things even exist and what their names are.
I start explaining that with custom pattern i mean this: suppose that i need to serialize objects or data of type foo, bar and boo, usually the library handle this for the user in a very simple way, what comes first goes in first in the serialization process, so if i serialize all the foo first they are written "at the top" of the file and all the bar and boo are after the foo.
Now I would like to keep order in my file and organize things based on a custom pattern, it's this possible with Boost ? What section provides this feature ?
Second thing, that is strictly related to the first one, I also would like to access my serialized binary files in a way that I'm not forced to parse and read all the previous values to extract only the one that I'm interested in, kinda like the RAM that works based on memory address and offers a random access without forcing you to parse all the others addresses.
Thanks.
On the first issue: the Boost serialization library is agnostic as to what happens after it turns an object into its serialized form. It does this by using input and output streams. Files are just that - fostream/fistream. For other types of streams however, the order/pattern that you speak of doesn't make sense. Imagine you're sending serialized objects over the network - the library can't know that it'll have to rearrange the order of objects and, in fact, it can't do that once they've been sent. For this reason, it does not support what you're looking for.
What you can do is create a wrapper that either just caches serialized versions of the objects and arranges them in memory before you tell it to write them out to a file, or that knows that since you're working with files, it can later tellg to the appropriate place in the file and append (this approach would require you to store the locations of the objects you wrote to the file).
As for the second thing - random access file reading. You will have to know exactly where the object is in memory. If you know that the structure of your file won't change, you can seekg on the file stream before handing it to boost for deserialization. If the file structure will change however, you still need to know the location of objects in the file. If you don't want to parse the file to find it, you'll have to store it somewhere during serialization. For example - you can maintain a sort of registry of objects at the top of the file. You will still have to parse it, but it should be just a simple [Object identifier]-[location in file] sort of thing.
I want to compare two Messages or (two sub parameters) of Google protocol buffers.
I don't find an API to achieve it.
Any ideas?
You can use the class google::protobuf::util::MessageDifferencer for this. I think it's only available since v3.0.2:
Introduced new utility functions/classes in the google/protobuf/util
directory:
MessageDifferencer: compare two proto messages and report their differences.
#include <google/protobuf/util/message_differencer.h>
MessageDifferencer::Equals(msg1, msg2);
You can rely on the fact that all of your protobuf messages inherit from the google::protobuf::MesageLite type, which in turn has everything you need to compare any two protobuf messages, regardless of if they are even of the same derived type:
bool operator==(const google::protobuf::MessageLite& msg_a,
const google::protobuf::MessageLite& msg_b) {
return (msg_a.GetTypeName() == msg_b.GetTypeName()) &&
(msg_a.SerializeAsString() == msg_b.SerializeAsString());
}
EDIT
As was pointed out in the comments below, and especially for map fields, this answer is incorrect. map elements have non-deterministic ordering. Use MessageDifferencer if map fields might be present in your messages.
Instead of using message.DebugString you could also do
std::string strMsg;
message.SerializeToString(&strMsg);
with both messages and then compare the two (binary) strings. I didn't test the performance but I assume that it is faster than comparing the human readable message strings returned by .DebugString(). +You can do that with the protobuf-lite library (while for message.DebugString you need the full version).
Well, a protocol buffer is just a serialization format for some object type. Why not use the protocol buffer to reconstruct the original objects, and then allow those objects to compare themselves, using whatever comparison logic you've built into the class?
This might not be the ideal solution, but I think it could be done by:
messageA.DebugString() == messageB.DebugString();
Other than that, I think the only solution would be to create your own Message child class and implement a bool operator==(const Message&).
You can compare the descriptor's pointer (super fast):
if (mMessages[i]->body()->GetDescriptor() == T::descriptor())
mMessages it's a pool of network messages with header and crypto which creates a packet with the protobuf body(google::protobuf::Message*).
so, to get the right kind of message i compare the descriptors constant pointer which is the same for every single type of message (not %100 sure but i haven't got any problem so far).
That would be the fastest way to compare a protobuf Message wthout having to use string comparasion, which by the way you gan get the type name from the descriptor. :-)