I have an application, written in C++ using MFC and Stingray libraries. The application works with a wide variety of large data types, which are all currently serialized based on MFC Document/View serialize derived functionality. I have also added options for XML serialization based on the Stingray libraries, which implements DOM via the Microsoft XML SDK. While easy to implement the performance is terrible, to the extent that it is unusable on anything other than very small documents.
What other XML serialization tools would you folks recommend for this scenario. I don't want DOM, as it seems to be a memory hog, and I'm already dealing with large in memory data. Ideally, i'd like a streaming parser that is fast, and easy to use with MFC. My current front runner is expat which is fast and simple, but would require a lot of class by class serialization code to be added. Any other efficient and easier to implement alternatives out there that people would recommend?
The Boost Serialization library supports XML. This library basically consists in:
Start from the principles of MFC serialization and take all the good things it provides.
Solve every single issue of MFC serialization!
Among the improvements compared to MFC is support for XML.
Note that you don't necessarily control the XML schema of this serialization. It uses its own schema.
This is an age old problem. I was the team lead of the development team with the most critical path dependencies on the largest software project in the world during 1999 and 2000 and this very issue was the focus of my work during that time. I am convinced that the wheel was invented by multiple engineers who were unaware that others had already invented it. The same is true of XML Data binding in C++. I invented it too, and I've been perfecting it for over 10 years on various projects. I have a solution that addresses the issues noted here and some additional issues that repeatedly arise:
XML Updates. This is the ability to re-apply a subset of XML into an existing object model. In many cases the XML is bound to indexed objects and we cannot afford to re-index for each update.
COM and CORBA interface management. In the same respect that the XML Data Binding can be automated through object oriented practices - so can the instances of interface objects that provide that data to the application layer.
State Tracking. The application often needs to distinguish between an empty value vs. a missing value - both create an empty string. This provides the validation along with Data Binding.
The source code uses the least restrictive license - less so that GPL. The project is supported and managed from here:
http://www.codeproject.com/KB/XML/XMLFoundation.aspx
Now that it's the year 2010, I believe that nobody else will attempt to reinvent the wheel because there are a few to choose from. IMHO - this wheel is the most polished and well rounded implementation available.
Enjoy.
A good solution would be libxml. It provides lightweight SAX parsing and data structures for XML processing. There are several DOM libraries which are built on top of libxml.
Unfortunatly it is a C library, but C++ wrappers are available.
A few years ago I switched from MSXML to libxml because of the performance issues you mentioned.
If you decide to use libxml, you should also take a look at libxslt.
We use Xerces-C++. It was easy to setup and performance is good enough so we don't need to think about changing. However we aren't XML heavy.
I did listen to a podcast by Scott Hanselman (from Hansel Minutes) where they discuss the XML performance of MSXML and XSLT.
what about RapidXML, I am using it in an MFC app with some modification to support UTF-16 with std::string. I am quite satisfied with it so far.
The gSOAP toolkit auto-serializes native C and C++ data to/from XML and supports the full XML schema specification through XML data bindings:
gSOAP SourceForge Project
It has evolved since 1999 to a significant code base with code generation tools and libraries. It supports many databinding and customization features, which is especially critical for mapping XML schema types to/from the C and C++ types. It can serialize any C/C++ type and also STL containers, container templates, and cyclic data structures. It has been used in the W3C Schema Patterns for Databinding working group (with 100% schema pattern coverage success since years). There is an active open source user base and the gSOAP development functionality has been used in many industrial projects and Fortune 100 companies to develop SOAP/XML infrastructures.
This is late in the game, I just want to mention that we also use LIBXML. It's robust and reliable, and has worked well. A little bit too low-level, you'll want to build some wrappers on top of its functions.
For instance, you'll get a different sequence of function returns depending on whether you have this:
<tag attribute="value"/>
or this:
<tag attribute="value"> </tag>
Sometimes you may want that, sometimes you don't care.
We use TinyXML for all our XML needs be it MFC or straight C++.
http://sourceforge.net/projects/tinyxml
Related
I'd doing some research to see if it's possible to convert an MFC-based Windows app to an HTML5 browser-based app. However, I'd like to keep some non-visual classes in a separate library that can be called from the browser or used by other applications, as required. Mostly this non-visual code is CObject-derived data classes that use strings, collections, etc. I'm not really familiar with the C++ Standard Library, would it be possible to post most of my existing of code to it without too many problems? Also, if I wanted to port the serialization code could I use something like the Boost serialization library as an extension to the standard library?
Are there any tools out there that would help with this type of conversion, and identify potential problems?
It is certainly possible to replace a UI layer of an application with a different UI technology. Exactly how difficult this will be will depend on how well separated the layers are in the existing application architecture (i.e. it's easier to replace the 'V' of an MVC app then an app where the model is mixed with the presentation logic)
Keeping non-visual MFC code in separate libraries (typically DLLs) is common place, but this usually implies that the application will continue to be an MFC application.
There are a few unusual things mentioned in your post:
You are talking about replacing desktop UI (MFC) with web UI ("HTML5"). This is still somewhat unusual, unless you are also transitioning to a web application, and placing the non-visual code on the server. That said, if you search on "using html5 for desktop apps" you'll find a lot of tools and discussions. For some apps, coding a UI within the constraints of a browser is still too limiting (e.g. limits on top-level window management and placement, etc.). If your goal is to have a more portable desktop application, you may want to investigate desktop UI libraries (e.g. Qt, wxWindows, etc.)
The C++ standard library and MFC string and container classes are significantly different, aside from offering some of the same general data structures. AFAIK there are no automated tools to convert from MFC to C++ standard library. Again, if your goal is portability, this is a sensible step, and you can write some wrapper classes around the C++ implementation to help adapt your code (although this can be a good opportunity to re-implement classes, or even entire libraries.)
A not-so-unusual thing is to consider using Boost libraries in addition to the standard library - they are high-quality libraries, and can save you a lot of time over rolling your own.
I know of at least three light weight C++ XML parsers: RapidXML, TinyXML and PugiXML. However, all three use a DOM based interface (ie, they build their own in-memory representation of the XML document and then provide an interface to traverse and manipulate it). For most situations that I have to deal with, I much prefer the SAX interface (where the parser just spits out a stream of events like start-of-tag, and the application code is responsible for doing whatever it wants based on those events).
Can anyone recommend a light weight C++ XML library with a SAX interface?
Edit: I should also note the Microsoft XmlLite library, which does use a SAX interface (well, actually a "pull" interface which is possibly even better). Unfortunately, it's ruled out for me at the moment since as far as I know it's closed source and Windows only (please correct me if I'm wrong on this).
I've used expat when I needed to parse XML. It's very light-weight (well, it used to be; it's a while since I've done XML stuff) and does the job.
you can try https://github.com/thinlizzy/die-xml . it seems to be very small and easy to use
this is a recently made C++0x XML SAX parser open source and the author is willing feedbacks
it parses an input stream and generates events on callbacks compatible to std::function
the stack machine uses finite automata as a backend and some events (start tag and text nodes) use iterators in order to minimize buffering, making it pretty lightweight
PugiXML and RapidXML do not have DOM conforming interfaces.. those API came with severe limitations on functionalities and conformance. You might want to investigate VTD-XML that is signifiantly more advanced than either DOM or SAX/Pull
I'm looking for a simple non-validating XML parser in either C or C++.
Several years back I found one that was just a single file solution but I can't find
it anymore.
I'm after some links and suggested ones that are very small and lightweight
ideally suited for an embedded platform.
Expat
You can work with or without validation and in "streaming mode". It is very lightweight.
What about something like pugixml. From their site...
pugixml is a light-weight C++ XML
processing library. It features:
DOM-like interface with rich traversal/modification capabilities
Extremely fast non-validating XML parser which constructs the DOM
tree from an XML file/buffer
XPath 1.0 implementation for complex data-driven tree queries
Full Unicode support with Unicode interface variants and
automatic encoding conversions
The library is extremely portable and
easy to integrate and use.
pugixml is developed and maintained
since 2006 and has many users. All
code is distributed under the MIT
license, making it completely free to
use in both open-source and
proprietary applications.
Also, this answer has more info.
There is also tinyxml and RapidXml.
There is definitely a pure C, tiny xml parser available. It was cited in an earlier answer on SO, but I can't find it right now. If I remember right, it's just a few hundred lines of code.
Update: Here's the question/answer that references it:
Is there a good tiny XML parser for an embedded C project?
And the actual code:
http://mercurial.intuxication.org/hg/cstuff/file/tip/tinyxml
RapidXML is a single-header (multiple headers if you want extra functionality) ultra-lightweight, ultra-fast implementation. It can operate in "destructive" mode, that means by setting pointers right into the XML and possibly overwriting some, avoiding all extra memory allocations and data copies.
tinyxml is not precisely single-header, but it is still fairly lightweight compared to other parsers. I've used it for half a decade without ever encountering an issue. The author has recently started with "tinyxml-2", which is supposedly better and even more lightweight, but I've not had occasion to actually try that one yet.
http://mercurial.intuxication.org/hg/cstuff/file/tip/tinyxml
can this parser work with nested XML like
<CServiceType>
<serno>61</serno>
<caption1 />
<caption2>Satelite</caption2>
<caption3 />
</CServiceType>
I'm not sure what exactly the right term is, kind of like ORM using XML as the data store. Are there any decent tools which will autogenerate C++ classes (including data and serialization/deserialization) based on an XML schema? Or will create XML-sync code and schema based on a C++ class definition?
TinyXML is great but it's so old-school to spend all that time writing code to load/save XML data to classes. I've seen similar tools focused on SOAP/WSDL, but they generated all kinds of other code on top of the basics.
Any good open-source libraries out there?
The only thing I've seen that attempts to do this is CodeSynthesisXSD.
If you are looking for an open source and commercial licensed tool to auto-generate C++ classes, including data and serialization/deserialization, based on an XML schema, then I strongly recommend GSOAP. It is easy to use, compliant to industry standards, and actively maintained.
See also http://www.rpbourret.com/xml/XMLDataBinding.htm
I was disappointed with many other C++ XML tools that promise full data bindings but will fail to process more extensive sets of WSDLs and schemas such as ONVIF. Having to retool an entire project was a pain. I know that GSOAP will do the job. A winner IMHO.
Not open source, but won't XML Thunder work for you?
I know that a lot of c/c++ XML library questions have been asked already (I tried to read through all of them before getting to this).
Here are the things I'm going to need in my own project:
Excellent performance
SAX2
Validation
Open source
Cross platform
I was going to use Xerces-C, but I see that a simple SAX2 setup with nothing going on in the filter is taking 5 seconds to run. (Perhaps I'm doing something wrong here?)
I would like to use libxml++, but as I tried to get it set up on my MacBook, there were some crazy dependencies that took me all the way back to gtk-doc, at which point I sort of tabled the idea.
So now I'm at libxml2. Is this the way to go? Have I missed an important option, bearing in mind the five requirements above? I don't mind using a (good) c-library like libxml2, but a c++ interface would be nice. (I don't like Xerces-C's API very much.)
I am willing to bend on the SAX2 requirement if comparable functionality is available.
Having spent a goodly amount of time on this same problem, it was my conclusion that libxml2 is the best option available under your guidelines. The C interface is not too difficult to use and it's very fast.
There are some other good options for commercial libraries, but most of the other comparable open-source options are either painfully slow or are mired in a deep, annoying vat of dependency soup.
You say you need these things in your project, but don't give any idea of the pipeline. For example, we had a whole load of static XML files which needed to be loaded quickly, but only validated rarely. So validated using a separate process in batch (using RelaxNG as it was human writable markup ) and loaded the XML using expat. The system also used XMPP, so checked streaming input, but that didn't require validating against a schema (partly because it was streamed, and mostly because most of the possible errors were not expressible in a schema).
If you need a whole host of other facilities, you can consider Qt, which has good XML support. Be warned though, it's WAY more than an XML processing library; it's a full blown application framework with support for GUIs, networking and a whole host of other things.
Qt
You can also try Poco. It's another application framework, but not as huge as Qt (i.e. no GUI-related things etc.)
Poco
Lastly, if you don't mind a C library, you can use Expat. It's not SAX per se, but writing code using Expat is somewhat like SAX. It has C++ wrappers, but they're not officially part of the project IIRC, and may not be as well-maintained or designed. I'm not too sure though.
Expat
Hope this helps!
EDIT: I misread your original post: not too sure about the validation features of these libraries, I've never used them before.