Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 8 years ago.
Improve this question
I am looking for an IDL-like (or whatever) translator which turns a DOM- or JSON-like document definition into classes which
are accessible from both C++ and Python, within the same application
expose document properties as ints, floats, strings, binary blobs and compounds: array, string dict (both nestable) (basically the JSON type feature set)
allow changes to be tracked to refresh views of an editing UI
provide a change history to enable undo/redo operations
can be serialized to and from JSON (can also be some kind of binary format)
allow to keep large data chunks on disk, with parts only loaded on demand
provide non-blocking thread-safe read/write access to exchange data with realtime threads
allow multiple editors in different processes (or even on different machines) to view and modify the document
The thing that comes closest so far is the Blender 2.5 DNA/RNA system, but it's not available as a separate library, and badly documented.
I'm most of all trying to make sure that such a lib does not exist yet, so I know my time is not wasted when I start to design and write such a thing. It's supposed to provide a great foundation to write editing UI components.
ICE is the closest product I could think of. I don't know if you can do serialization to disk with ICE, but I can't think of a reason why it wouldn't. Problem is it costs $$$. I haven't personally negotiated a license with them, but ICE is the biggest player I know of in this domain.
Then you have Pyro for python which is Distributed Objects only.
Distributed Objects in Objective-C (N/A for iPhone/iPad Dev, which sucks IMHO)
There are some C++ distributed objects libraries but they're mostly dead and unusable (CORBA comes to mind).
I can tell you that there would be a lot of demand for this type of technology. I've been delving into some serialization and remote object stuff since off-the-shelf solutions can be very expensive.
As for open-source frameworks to help you develop in-house, I recommend boost::asio's strands for async thread-safe read/write and boost::serialization for serialization. I'm not terribly well-read in JSON tech but this looks like an interesting read.
I wish something freely available already existed for this networking/serialization glue that so many projects could benefit from.
SWIG doesn't meet all your requirements, but does make interfacing c++ <-> python a lot easier.
Related
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 7 years ago.
Improve this question
Normally, to analyse big C projects, I prefer grep/GNU command line tools, lint, simple Python scripts. Saying "to analyse" C project I mean to collect code statistics, to understand project's structure, its data structures and flow of execution - what function calls what, entry points in different modules, static members, threads, etc. But it works not so good with an object-oriented code.
Whenever I have a big C++ (or Objective-C) project, containing large number of source files and several directories, I would like to see it's class diagram, data fields, methods, messages, instances, etc.
I am looking for a most Unix way solution. Can you help me?
Doxygen is the closest i could find, when i was searching last time. It is not unix way, but it is available free for linux/windows/mac. It generated descent graphs for me. Hope it helps.
http://www.doxygen.nl/
http://en.wikipedia.org/wiki/Doxygen
With message passing and dynamic dispatch going around you are pretty much screwed. It doesn't even depend on language, message is as well used in C++ world. There is no tool that can analyze the code and tell what the application flow will look like. In those cases, the whole data/execution flow may depend on configuration files, how you hook up producers/consumers together etc.. and change significantly. If you are lucky, there would be some high-level documentation, maybe with pictures and description of overall ideas etc. Otherwise, the only option here is to run it under debugger for any given configuration and see what is going on in there, step by step. Isn't that a true UNIX way?
Your request is for a variety of views, some text-based, some structure based.
You might consider Understand for C++ which does a mixture of these. Don't know if it does ObjectiveC.
Our Source Code Search Engine (SCSE) is rather more limited, but provides a much faster way to "grep" than grep does. For large code bases this matters. It will handle multiple languages and dialects. We don't have an Objective C dialect, but I think our C or C++ front ends would actually work pretty well for this, since Objective C uses pretty much the same lexical syntax.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 5 years ago.
Improve this question
I'm looking for a solution for serializing of c++ packets to a network stream.
I have seen many posts here refering people to:
ACE
Google Protocol Buffers
Boost::Serialization
Qt ::QDataStream
My Requirements/ Constraints:
The solution must be unaware of LitteEndian/BigEndian. Machine Architecture x86/x64 and platform independant.
The foot print (RAM & ROM) of the first 3 solution are too big for my platform,and the fourth is conflicting with the next requirement.
The solution won't require a lot of boilerplate code (there will be 200+ packet to be serialized).
Thanks,
Koby Meir
If you find that Google Protocol Buffers are to heavy (I can agree with that because compiled library could take more than 1 MB), you could try the lite version of protobuf which is a few times smaller.
It can be enabled in *.proto files by inserting the following line
option optimize_for = LITE_RUNTIME;
But if you need a protobuf solution with minimal overhead I would go with protobuf-c,
a C implementation of protobuf.
It will be a little harder to use, but binary code size overhead should be minimal (30-50 KB). I know that this C implementation is used for example by umurmur - a voice server that runs very well on embedded Linux ARM and MIPS routers.
Another thought: Serialize as text, parsing back on the other side.
We do this a lot (TCP, UDP, serial, other protocols). There is tremendous precedence for this approach, like in robotic control systems, Lab Information Management Systems, and any other place where "hook-up" among many vendors is helpful: Everybody just sends plain text (ASCII or UTF-8), and it's easy to read, easy to debug, easy to reverse-engineer, and easy to repair/hook-into. (If you want it opaque, you can encrypt your payload, like with public/private keys.)
This fits your requirement of agnostic-to-endien-ness/different platform requirements. We've used XML, which works fine and is fairly "standard" with some reference ontology for what you want as (possibly nested) "Key=Value" values, but we tend to prefer an INI-style format, using "named sections" as it gets more complicated. If you "nest" lots of stuff, you might look at JSON-like implementations (it's really easy).
Strong vote for ASCII, IMHO.
Wow, ACE, boost serialization ... these frameworks have serialization built in but looking at them for serialization alone is like buying a car because you need CD player. SUN/DEC RPC uses a format called XDR - very well described in Steven's "UNIX Network programming" - essentially it is a 1000 LOC header/c file. that you can use alone. CORBA also uses XDR underneath. Just look for "xdr.h" in Google code - plenty of OSS implementations.
If you still need something sophisticated, I would find ASN.1 to be most comprehensive solution which is granted little more complicated than needed for most applications, however ASN.1 compilers generate compact code. It is mostly used by telecom stacks, in cll phones, GSM messaging etc. ASN.1 is used to encode RSA keys.
I think you'll have to roll your own final solution, but there are a few building blocks you can use:
Boost::Spirit (Karma to serialize, Qi to deserialize)
that might still be too big for you, in which case you'll have to roll your own for serialization/deserialization, though you could use the Barton-Nackman idiom to keep the code readable and still use a simply serialize function
a light-weight communications layer, such as Arachnida (which you could use just for communication, without using its HTTP facilities).Arachnida builds on OpenSSL. If that's too heavy for you too, you really should roll your own.
Ignorance/knowledge of little/big endianness would pretty much be in the hands of your serialization code..
Good luck
I implemented something like this using the same technique microsoft used in MFC. Basically you implement methods in each serializable class to serialize and deserialize
whatever you want saved from that class. It's very lightweight and fast.
Instead of the 'archive' class Microsoft used I used a binary stream class.
My requirements didn't need endian awareness but it would be
trivial to implement if you implement a base class to serialize POD variables.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 8 years ago.
Improve this question
I am looking for a lightweight open source paging B+ tree implementation that uses a disk file for storing the tree.
So far I have found only memory-based implementations, or something that has dependency on QT (?!) and does not even compile.
Modern C++ is preferred, but C will do too.
I prefer to avoid full embeddable DBMS solution, because: 1) for my needs bare bone index that can use the simplest possible disk file organization is enough, no need for concurrency, atomicity and everything else. 2) I am using this to prototype my own index, and most likely will change some of the algorithms and storage layout. I want to do that with a minimum of effort. It's not going to be production code.
http://people.csail.mit.edu/jaffer/WB.
You can also consider re-using the B-Tree implementations from an open source embeddable database. (BDB, SQLite etc)
My own implementation is under http://www.die-schoens.de/prg License is Apache. Its disk-based, maps to shared memory where it also can do locking (i. e. multiuser), file format protects against crash etc. All of the above can be easily switched off (compile or runtime if you like). So bare bone would be almost ANSI-C, basically caching in ones own memory and not locking at all. Test program is included. Currently, it only deals with fixed-size fields, but I am working on that...
I second the suggestion for Berkeley DB. I used it before it was bought by Oracle. It is not a full relational database, but just stores key-value pairs. We switched to that after writing our own paging B-Tree implementation. It was a good learning experience, but we kept adding features until is was just a (poorly) implemented version of BDB.
If you want to do it yourself, here is an outline of what we did. We used mmap to map pages into memory. The structure of each page was index based, so with the page start address you could access any element on the page. Then we mapped and unmapped pages as necessary. We were indexing multi GB text files, back when 1 GB of main memory was considered a lot.
Faircom's C-Tree Plus has been available commercially for over 20 years. Don't work for them etc... FairCom
There is also Berkley DB which was bought by Oracle but is still free from their site.
I' pretty sure it's not the solution you're looking but why don't you store the tree in a file yourself? All you need is an approach for serialization and an if/ofstream.
Basically you could serialize it like that: go to root, write '0' in your file a divider like '|', the number of elements in root and then all root elements. Repeat with '1' for level 1 and so on. As long as you don't change the level keep the level index, empty leafs could look like 2|0.
You could look at Berkeley DB, its supported ny Oracle but it is open source and can be found here.
RogueWave, the software company, have a nice implementation of BTreeOnDisk as part of their Tools++ product. I've been using it since late 90's. The nice thing about it is that you can have multiple trees in one file. But you do need a commercial license.
In their code they do make a reference to a book by a guy called 'Ammeraal' (see http://home.planet.nl/~ammeraal/algds.html , Ammeraal, L. (1996) Algorithms and Data Structures in C++). He seems to have an imlementation of a BTree on disk, and the source code seems to be accessible online. I have never used it though.
I currently work on projects for which I'd like to distribute the source code so I need to find an open source replacement for the Rogue Wave classes. Unfortunately I don't want to rely on GPL type licenses, otherwise a solution would be simply to use 'libdb' or equivalent. I need a BSD type license, and for a long time I couldn't find anything suitable. But I will have a look at some the links in earlier posts.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 5 years ago.
Improve this question
Are there any good existing C++ serialization libraries that support partial serialization?
By "partial serialization" I mean that I might want to save the values of 3 specific members, and later be able to apply that saved copy to a different instance. I'd only update those 3 members and leave the others intact.
This would be useful for synchronizing data over a network. Say I have some object on a client and a server, and when a member changes on the server I want to send the client a message containing the updated value for that member and that member only. I don't want to send a copy of the whole object over the wire.
boost::serialization at a glance looks like it only supports all or nothing.
Edit: 3 years after originally writing this I look back at it and say to myself, 'wut?' boost::serialization lets you define what members you want saved or not, so it would support 'partial serialization' as I seem to have described it. Further, since C++ lacks reflection serialization libraries require you to explicitly specify each member you're saving anyway unless they come with some sort of external tooling to parse the source files or have a separate input file format that is used to generate C++ code (e.g. what Protocol Buffers does). I think I must have been conceptually confused when I wrote this.
You're clearly not looking for serialization here.
Serialization is about saving an object and then recreating it from the stream of bytes. Think video games saves or the session context for a webserver.
Here what you need is messaging. Google's FlatBuffers is nice for that. Specify a message that will contain every single field as optional, upon reception of the message, update your object with the fields that do exist and leave the others untouched.
The great thing with FlatBuffers is that it handles forward and backward compatibility nicely, as well as text and binary encoding (text being great for debugging and binary being better for pure performance), on top of a zero-cost parsing step.
And you can even decode the messages with another language (say python or ruby) if you save them somewhere and want to throw together a html gui to inspect it!
Although I'm not familiar with them, you could also check out Google's Protocol Buffers
.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 7 months ago.
Improve this question
Currently I'm working on a C/C++ cross-platform client/server software. I'm very experienced developer when it comes to low level socket development.
The problem with Berkley sockets/Winsock, is that you always have to make some kind of parser to get things right on the receiver side. I mean, you have to interpret data, and concatenate packets in order to transmit correctly. (packets often get sliced)
Have in mind that the communication is going to be bidirectional. Is pure socket the best way to transmit data nowadays? Would you recommend SOAP, Webservices or another kind of encapsulation to this application?
I can highly recommend Google Protocol Buffers.
These days many people use Web Services and SOAP. There are C++ packages available for this purpose. These will use sockets for you and handle all the data wrangling. If you are on Unix/Linux, give or take System V.4 network handles, your data will eventually travel over sockets.
On Windows there are other choices if you only want to talk to other Windows boxes.
You could also look into CORBA, but it's not the common practice.
In any data transfer, there is going to be a need to serialize and deserialize the objects.
The first question you want to ask is whether you want a binary or text format for the transfer. Binary data formats have the distinct advantage that they are totally easy to parse (provided they are simple POD structures - you can just cast them into a struct).
Text based transfers should be easier to debug since you can just read the text. You are still going to have to parse them.
SOAP based web services are simple XML based packets sent normally over HTTP. Something will have to parse the HTTP and the XML. The ease of use is not intrinsic but rather dependent of the tools at your disposal. If you have good tools, the by all means, but the same applies to any form of data exchange.
You can take a look at the Boost Serialization Library. It is a fairly complex library and does require you to write code indicating what members need to be serialized. IT does have good support for both text (including xml) and binary serialization. It is also cross platform.
I have used ZMQ with grate success. I highly recommend it as it is a middle-level library witch takes care the socket related overhead. It also supports binary packets/messages.