I need to model a JSon in C++. Firstly I thought about:
boost::property_tree
But unfortunately - it does not care about JSon types. Integers are represented as strings - and it is not an option.
The second idea would be:
boost::variant with recursive_wrapper
Which looks quite promising.
What kind of approach could you recommend? Do you know better approaches? It looks like a common problem, so there have to be a lot of well tested solutions.
I'm unfamiliar with C++ JSon libraries. If you could recommend any - I would be grateful as well.
For JSon you typically want to take a library. You could ofcourse write your own implementation (parser etc.) but I don't see why you would do that considering there are so many good ones freely available.
Some examples:
https://github.com/kazuho/picojson (Header only)
https://github.com/miloyip/rapidjson
Related
I'm writing an embedded C++ program, and need to add serialization/deserialization. The format should be human readable and writeable, and I would much prefer to use (a subset of) a standard format like YAML. I also prefer YAML to JSON since it is more concise.
While yaml-cpp has the exact functionality I'd like, the source code is almost 300K and would almost double my code size, which seems excessive to me just in order to add human readable serialization/deserialization.
Before I start writing my own reader/writer for a subset of YAML, I'd like to first check whether this already exists? I have not been able to find one, but would much prefer to use existing code rather than rolling my own. Are there any C or C++ YAML readers/writers out there of, say, 50K code or less? I only need functionality for the basic data structures (scalar, array, hash), not any advanced stuff.
With many thanks in advance.
The Oops library is doing what you are looking for. It is written for serialization using reflection and supports YAML format as well.
https://bitbucket.org/barczpe/oops
This question already has answers here:
Is there a C++ library to read JSON documents into C++ objects? [duplicate]
(4 answers)
Closed 8 years ago.
I can't figure out whether or not to use XML, YAML, or JSON for a C++ 2D RPG.
Here are my thoughts:
I need something which is simple to save not just player data, but environment data, such as object (x, y) coordinates; load times; dates; graphics configurations, etc.
I need something flexible, easy to use, and definitely light weight, but powerful to handle the above.
Which is the best choice? I have experience with JSON in JavaScript, but not C++. Are there any good references for parsing JSON in C++ if this is the route to go?
Edit
Honestly, if a text file seems like the simplest and most effective solution for something like this (especially if I can just write it to binary), then I'm all ears.
Edit 2
Feel free to provide other suggestions as well.
I would use the simplest thing that satisfies your requirements.
If you don't need hierarchical storage, then flat tabular files are so much easier to deal with than anything else. All you have to do is read lines off disk and split on tab.
If you are looking at more of key/value pair type storage (as opposed to lists of things), then INI files can be reasonable. This format has a lot of flexibility, though reasoning about it can less approachable when you start doing more complicated things than it was designed for.
If you need hierarchical, it's possible that JSON would be simpler. There are JSON libraries in wide range of languages, and it sounds like you already familiar.
https://stackoverflow.com/questions/245973/whats-the-best-c-json-parser
sqlite may be another option. There be dragons in SQL, but with a nice C++ wrapper around sqlite, it can be manageable. The primary benefit would be ACID, in my opinion.
The YAML spec looks somewhat lengthy, so I can guess that it has more kitchen sinks. Just skimming the libyaml docs, the API looks somewhat like SAX interfaces that I've used in the past. I have no a posteriori knowledge of it, but I would be reticent to start using it without a good reason.
XML sucks to deal with, don't opt in to it. There lots of reasons for this. I think the most relevant one in my mind is that it's prone to make the code that uses it more complicated than it would be otherwise. Any system I've seen designed with XML, reasoning about the XML is more complicated than the design interests that its trying to support. There are valid uses for it, though it's rare that another storage system wouldn't have been just as adequate.
Regardless of which one you choose, write as little code as you can managing it. You really want to write the classes your engine will use first. Then worry about serializing them. If you let your serialization influence your class design, you'll probably regret it. :)
What would be the most effective way to take an XML string of type std::string and convert it to an in-memory XML object, the object structure is of no importance, what I'm after is whether I'd need to go through the string char by char and pick out all the pieces or if there is some easier way?
The easiest way is probably to use a library to do that.
If you want to do that yourself, you'll need to parse the string containing XML code. There are many ways to do that; easiest is probably a recursive-descent parser.
Take a look at Arabica: http://www.jezuk.co.uk/cgi-bin/view/arabica Of all XML libraries/wrappers I am aware of it is most std friendly.
Try Expat, compact, user-friendly and free:
http://expat.sourceforge.net/
You could use MSXML for this, it will take the string and produce a DOM (Document Object Model) representation. Documentation is OK but this is not the easiest library to use. WIndows only of course.
Pros - reliable, widely used.
Cons - you have to learn COM programming model to a degree. Not the most intuitive to use.
A simpler option would be xerces. Sample file parse code here, there are other samples as well. I've used both this and MSXML in different jobs.
Use libxml2. Not a highly complicated library and easy to use. Portable, written in C but bindings to other languages available, and loads of examples to use and learn from.
Does anyone have any references for building a full Object/Class reflection system in C++ ?
Ive seen some crazy macro / template solutions however ive never found a system which solves everything to a level im comfortable with.
Thanks!
Using templates and macros to automatically, or semi-automatically, define everything is pretty much the only option in C++. C++ has very weak reflection/introspection abilities. However, if what you want to do is mainly serialization and storage, this has already been implemented in the Boost Serialization libraries. You can do this by either implementing a serializer method on the class, or have an external function if you don't want to modify the class.
This doesn't seem to be what you were asking though. I'm guessing you want something like automatic serialization which requires no extra effort on the part of the class implementer. They have this in Python, and Java, and many other languages, but not C++. In order to get what you want, you would need to implement your own object system like, perhaps, the meta-object system that IgKh mentioned in his answer.
If you want to do that, I'd suggest looking at how JavaScript implements objects. JavaScript uses a prototype based object system, which is reasonably simple, yet fairly powerful. I recommend this because it seems to me like it would be easier to implement if you had to do it yourself. If you are in the mood for reading a VERY long-winded explanation on the benefits and elegance of prototypes, you can find an essay on the subject at Steve Yegge's blog. He is a very experienced programmer, so I give his opinions some credence, but I have never done this myself so I can only point to what others have said.
If you wanted to remain with the more C++ style of classes and instances instead of the less familiar prototypes, look at how Python objects and serialization work. Python also use a "properties" approach to implementing its objects, but the properties are used to implement classes and inheritance instead of a prototype based system, so it may be a little more familiar.
Sorry that I don't have a simpler answer to your question! But hopefully this will help.
I'm not entirely sure that I understood you intention, however the Qt framework contains a powerful meta object system that lets you do most operation expected from a reflection a system: Getting the class name as string, checking if a object is a instance of a given type, listing and invoking methods, etc.
I've used ROOT's Reflex library with good results. Rather than using crazy macro / template solutions like you described, it processes your C++ header files at build time to create reflection dictionaries then operates off of those.
I'm building a distributed C++ application that needs to do lots of serialization and deserialization of simple data structures that's being passed between different processes and computers.
I'm not interested in serializing complex class hierarchies, but more of sending structures with a few simple members such as number, strings and data vectors. The data vectors can often be many megabytes large.
I'm worried that text/xml-based ways of doing it is too slow and I really don't want to write this myself since problems like string encoding and number endianess can make it way more complicated than it looks on the surface.
I've been looking a bit at protocol buffers and boost.serialize. According to the documents protocol buffers seems to care much about performance.
Boost seems somewhat more lightweight in the sense that you don't have an external language for specifying the data format which I find quite convenient for this particular project.
So my question comes down to this: does anyone know if the boost serialization is fast for the typical use case I described above?
Also if there are other libraries that might be right for this, I'd be happy to hear about them.
I would strongly suggest protocol buffers. They're incredibly simple to use, offer great performance, and take care of issues like endianness and backwards compatibility. To make it even more attractive, serialized data is language-independent thanks to numerous language implementations.
ACE and ACE TAO come to mind, but you might not like the size and scope of it.
http://www.cs.wustl.edu/~schmidt/ACE.html
Regarding your query about "fast" and boost. That is a subjective term and without knowing your requirements (throughput, etc) it is difficult to answer that for you. Not that I have any benchmarks for the boost stuff myself...
There are messaging layers you can use, but those are probably slower than boost. I'd say that you identified a good solution in boost, but I've only used ACE and other proprietary communications/messaging products.
My guess is that boost is fast enough. I have used it in previous projects to serialize data to and from disk, and its performance never even came up as an issue.
My answer here talks about serialization in general, which may be helpful to you beyond which serialization library you choose to use.
Having said that, it looks like you know most of the main trouble spots with serialization (endianess string encoding). You did leave out versioning and forwards/backwards compatibility. If time is not critical I recommend writing your own serialization code. It is an enlightening experience, and the lessons you learn are invaluable. Though I will warn you it will tend to make you hate XML based protocols for their bloatedness. :)
Whichever path you choose good luck with your project.
Also check out ONC-RPC (old SUN-RPC)
boost.serialization doesn't care about string encodings or endianness. You'll be similarly well off not using it if that matters to you.
You might want to look into ICE from ZeroC: http://www.zeroc.com/
It works similar to CORBA, except that it's entirely specced and defined by the company. The upside is that the implementations work as intended, since there aren't all that many. The downside is that if you're using a language they don't support, you're out of luck.
If you are only sending well defined defined data structures, then perhaps you should be looking at ASN.1 as an encoding methodology ?
There's also Thrift, which looks like an alpha project but is used and developed by Facebook, so it has a few users of it.
Or good old DCE, which was the standard MS decided to use for COM. Its now open-source, 20 years too late, but better than never.
Don't pre-emptively optimize. Measure first and optimize second.