Using ZODB directly from C++. Examples and design hints - c++

I'd like to use ZODB directly from C++ and don't want to write Python code for that. Have you had any experience doing so? If I were to use C++ for GUI and quering/writing data from/to ZODB, how the design should be?

seems like you have 2 choices
a) work out how to call ZODB python module from c++
google shows boost has a library, and I am sure python.org will tell you too
b) work out the file format and write the equivalent code in c++
Probably not impossible for reading, harder for writing. However you will eventually end up with the impedance mismatch of python->dynamic, c++->static
I dont know ZODB but I will guess it is tightly matched to the dynamic nature of python's objects and so having a general purpose equivalent for c++ wont work. You would be able to create a particular object schema implementation though. I mean you could have a zodb with Customer, Order, Product and you can create a layer that maps the ZODB data to equivalent C++ objects

Related

How interoperability works

I know that many large-scale applications such as video games are created using multiple langages. For example, it's likely the game/physics engines are written in C++ while gameplay tasks, GUI are written in something like Python or Lua.
I understand why this division of roles is done; use lower-level languages for tasks that require extreme optimization, tweaking, efficiency and speed, while using higher-level languages to speed up production time, reduce nasty bugs ect.
Recently, I've decided to undertake a larger personal project and would like to divy-up parts of the project similar to above. At this point in time, I'm really confused about how this interoperability between languages (especially compiled vs interpreted) works.
I'm quite familiar with the details of going from ANSCII code test to loading an executable, when written in something like C/C++. I'm very curious at how something like a video game, built from many different languages, works. This is a large/broad question, but specifically I'm interested in:
How does the code-level logic work? I.e. how can I call Python code from a C++ program? Especially since they don't support the same built-in types?
What does the program image look like? From what I can tell, a video game is running in a single process, so what does the runtime image look like when running a C/C++ program that calls a Python function?
If calling code from an interpreted language from a compiled program, what are the sequence of events that occur? I.e If I'm inside my compiled executable, and for some reason have a call to an interpreted language inside a loop, do I have to wait for the interpreter every iteration?
I'm actually finding a hard time finding information on what happening at the machine-level, so any help would be appreciated. Although I'm curious in general about interoperation of software, I'm specifically interested in C++ and Python interaction.
Thank you very much for any insight, even if it's just pointing me to where I can find more information.
In the specific case of python, you have basically three options (and this generally applies across the board):
Host python in C++: From the perspective of the C++ programme, the python interpreter is a C library. On the python side, you may or may not need to use something like ctypes to expose the C(++) api.
Python uses C++ code as DLLs/SOs - C++ code likely knows nothing of python, python definitely has to use a foreign function interface.
Interprocess communication - basically, two separate processes run, and they talk over a socket. These days you'd likely use some kind of web services architecture to accomplish this.
Depending on what you want to do:
Have a look at SWIG: http://www.swig.org/ It's a tool that aims to connect C/C++ code with Python, Tcl, Perl, Ruby, etc. The common use case is a Python interface (graphical or not) that will call the C/C++ code. SWIG will parse the C/C++ code in order to generate the interfaces.
Libpython: it's a lib that allows you to embed Python code. You have some examples here: http://docs.python.org/3.0/extending/embedding.html

Calling a C++ Functions through a Python Script

I have a scenario where I have some functions in C++ classes and I want to be able to call them using a python script. Let's say I have a function
void greet(_msg);
std::cout >> _msg >> std::endl;
I want to be able to call it trough a custom Python call and pass arguments to it, for example using
saySomething("Hello")
As a .py file I want it to call the greet function and pass "Hello" as an argument.
I know it's a subject that has been throughly discussed, and I've done a share of research on embedding python in C++, I've managed to read values from a python script using the standard Python/C API and run a function in Python from C++ and pass argument to it, but I can't seem to get my head around how to achieve this specific outcome.
I've had a look at ctypes and various wrappin libraries such as boost:python or swig, but I can't seem to understand to which degree they could help me achieve want I want.
Depending on which version of Python you are interested in, 2.x or 3.x,
read through the Extending and Embedding the Python Interpreter chapter for 2.x or 3.x. You are interested only in extending Python, so section the 1. Extending Python with C or C++ will provide you with complete explanation how to implement what you need in order to be able to call your functions implemented in C++ from Python script.
Certainly, there are numerous libraries and generators which allow you to wrap C/C++ APIs for Python (e.g. Boost.Python or SWIG), but your case sounds simple enough, that for the purpose of learning it is IMO better to get familiar with Python C API. Even if you use these tools, you will frequently have to get down to Python C API anyway or at least understand it.
I recently needed to do this very thing. Boost.Python does what we're looking for (and more) but personally (as much as I love Boost) I find it a little overkill to have to drag in half the Boost library to get one feature. SWIG also wasn't really an option for me as code generation always becomes a pain to maintain while class structures change (Don't get me wrong, these are BRILLIANT solutions!, just not what I was looking for).
So, the only thing left for me was to implement it from first principles (Python/C API). Hense, "ECS:Python" was born. ECS:Python (Embedded C++ Scripting with Python) is a simple C++ Python wrapper library I designed specifically for C++ developers. It allows you to expose objects from a C++ application to an embedded Python interpreter for interactive scripting, and it's very light-weight and easy to use.
Its free (BSD) and open source. If you're interested here it is:
http://sourceforge.net/projects/ecspython
You can use the weave.inline() function, which is part of the scipy package, to compile and execute C/C++ files and get their output from within your python script.

Testing a C++ library with Python

I have a libfoo.so library built from C++ code (compiled with gcc), and I would like to quickly test some of its exported classes (basically, instantiating a class then calling its methods to check the output).
While I could do that in C/C++ with a main file that links to the library in question and build my tests, but I think it would be much easier if it was possible to simply call Python from the command line and call the methods from there.
I know I can use CDLL from ctypes to load C-style libraries, but is there a similar functionality for C++ libraries and objects?
EDIT : Ideally, I do not want to modify the C++ code, I would need to use it as-is.
Honestly C++ is a bit messy. You could do something like create a pure C function which wraps the C++ functionality (which you then call from python) but at that point you might as well write your tests in C++. Unfortunately the only tool out there for this (that I know of) is SWIG.
It's sad that it's called the "simplified" wrapper and interface generator, because there's nothing simple about it. If you have VERY primitive data types in your signatures (like, JUST ints or maybe char*) it'll be a pretty quick job. Otherwise you have to tell swig how to marshal your data types between languages, and that gets REALLY ugly very quickly. Furthermore, after short time you realize you have to learn the CPython API in order to write your marshalling code.
And by that point you may as well write your own CPython wrapper without involving SWIG. You suddenly realize you've spent a good month learning a new API and feel horribly frustrated. If you're going to be doing this a lot, it's definitely worth your time. However if this is a one-time thing, just write your tests in C / C++.
(I'm speaking from experience here)
I agree with Chris answer. However, I want to point out that Cython supports C++ (with some limitations).

Efforts or plans to port gson to C++

Does anyone know if there have been any efforts or plans to create a C++ version of gson? The library is very helpful and one the easiest to use and it would be great if there was something similar for C++/
You mean the google project to convert json into java objects and back?
There are c++ json libraries but from what I understand, the advantage of gson is the ability to automatically serialize objects.
The problem that makes this difficult, and impossible to do as well as in other languages is that C++ doesn't support reflection, and has basically zero run-time type info. This makes something like gson about impossible to do in a similar manner.
Take this with a grain of salt because while I do have a whole lot of C++ experience. I've never actually used gson, and thus don't really know what it is you desire.

Scripting language for C++ [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 6 years ago.
Improve this question
I'm getting a little rusty in scripting languages, provided they're popping like mushrooms lately :)
Today I thought that it would be nice to have a scripting language that talks seamlessly to C++, that is, could use C++ classes, and, the most important for me, could be compiled into C++ or some DLL/.SO (plus its .h) so that I could link it into my C++ program and make use of the classes the script defines or implements.
I know I could embed any popular scripting language such as lua, ruby, python... but the interface usually includes some kind of "eval" function that evaluates the provided scripting code. Depending on the tool used to couple C++ and the scripting language, the integration for callbacks of the script into C++ could be more or less easy to write, but I haven't seen any scripting language that actually allows me to write independent modules that are exposed as a .h and .so/dll to my program (maybe along the lines of a scripting language that generates C++ code).
Do you know any such tool/scripting language?
Thanks in advance.
PD. I've been thinking along the lines of Vala or Haskell's GHC. They generate C, but not C++...
UPDATE 2020: Today I would probably go with Lua + Sol2/3 except if I really want to avoid Lua as a language. Chaiscript becomes a good candidate in this case though it is not optimal performance-wise compared to Lua+Sol2/3 (though it was greatly improved through years so it is still good enough in many cases).
Falcon have been dead for some years, RIP.
The following ones are more C++ integration oriented than language bindings :
ChaiScript - trying at the moment in a little project, interesting, this one is MADE with C++ in mind and works by just including a header! Not sure if it's good for a big project yet but will see, try it to have some taste!
(not maintained anymore) Falcon - trying on a big project, excellent; it's not a "one include embed" as ChaiScript but it's because it's really flexible, and totally thought to be used in C++ (only C++ code in libs) - I've decided to stick with it for my biggest project that require a lot of scripting flexibility (comparable to ruby/python )
AngelScript - didn't try yet
GameMonkey - didn't try yet
Io - didn't try yet
For you, if you really want to write your scripting module in C++ and easily expose it to the scripting language, I would recommand going with Falcon. It's totally MADE in C++, all the modules/libraries are written that way.
The question usually asked in this context is: how do I expose my C++ classes so they can be instantiated from script? And the answer is often something like http://www.swig.org/
You're asking the opposite question and it sounds like you're complicating matters a bit. A scripting engine that produced .h files and .so files wouldn't really be a scripting engine - it would be a compiler! In which case you could use C++.
Scripting engines don't work like that. You pass them a script and some callbacks that provide a set of functions that can be called from the script, and the engine interprets the script.
Try lua: http://www.lua.org/
For using C++ classes in lua you can use:
To generate binding use tolua++: http://www.codenix.com/~tolua/
It takes a cleaned up header as input and outputs a c file that does the hard work. Easy, nice and a pleasure to work with.
For using Lua objects in C++ I'd take the approach of writing a generic Proxy object with methods like (field, setField, callMethod, methods, fields).
If you want a dll you could have the .lua as a resource (in Windows, I don't know what could be a suitable equivalent for Linux) and on your DllMain initialize your proxy object with the lua code.
The c++ code can then use the proxy object to call the lua code, with maybe a few introspection methods in the proxy to make this task easier.
You could just reuse the proxy object for every lua library you want to write, just changing the lua code provided to it.
This is slightly outside my area of expertise, but I'm willing to risk the downvotes. :-)
Boost::Python seems to be what you're looking for. It uses a bit of macro magic to do its stuff, but it does expose Python classes to C++ rather cleanly.
I'm the author of LikeMagic, a C++ binding library for the Io language. (I am not the author of Io.)
http://github.com/dennisferron/LikeMagic
One of my explicit goals with LikeMagic is complete and total C++ interoperability, in both directions. LikeMagic will marshal native Io types as C++ types (including converting between STL containers and Io's native List type) and it will represent C++ classes, methods, fields, and arrays within Io. You can even pass a block of Io code out of the Io environment and use it in C++ as a functor!!
Wrapping C++ types up for consumption in Io script is simple, quick and easy. Accessing script objects from C++ does require an "eval" function like you described, but the template based type conversion and marshaling makes it easy to access the result of executing a script string. And there is the aforementioned ability to turn Io block() objects into C++ functors.
Right now the project is still in the early stages, although it is fully operational. I still need to do things like document its build steps and dependencies, and it can only be built with gcc 4.4.1+ (not Microsoft Visual C++) because it uses C++0x features not yet supported in MSVC. However, it does fully support Linux and Windows, and a Mac port is planned.
Now the bad news: Making the scripts produce .h files and .so or .dll files callable from C++ would not only require a compiler (of a sort) but it would also have to be a JIT compiler. That's because (in many scripting languages, but most especially in Io) an object's methods and fields are not known until runtime - and in Io, methods can even be added and removed from live objects! At first I was going to say that the very fact that you're asking for this makes me wonder if perhaps you don't really understand what a dynamic language is. But I do believe in a way of design in which you first try to imagine the ideal or easiest possible way of doing something, and then work backwards from there to what is actually possible. And so I'll admit from an ease-of-use standpoint, what you describe sounds easier to use.
But while it's ideal, and just barely possible (using a script language with JIT compilation), it isn't very practical, so I'm still unsure if what you're asking for is what you really want. If the .h and .so/.dll files are JITted from the script, and the script changes, you'd need to recompile your C++ program to take advantage of the change! Doesn't that violate the main benefit of using script in the first place?
The only way it is practical would be if the interfaces defined the scripts do not change, and you just are making C++ wrappers for script functions. You'd end up having a lot of C++ functions like:
int get_foo() { return script.eval("get_foo()"); }
int get_bar() { return script.eval("get_bar()"); }
I will admit that's cleaner looking code from the point of view of the callers of the wrapper function. But if that's what you want, why not just use reflection in the scripting language and generate a .h file off of the method lists stored in the script objects? This kind of reflection can be easily done in Io. At some point I plan to integrate the OpenC++ source-to-source translator as a callable library from LikeMagic, which means you could even use a robust C++ code generator instead of writing out strings.
You can do this with Lua, but if you have a lot of classes you'll want a tool like SWIG or toLua++ to generate some of the glue code for you.
None of these tools will handle the unusual part of your problem, which is to have a .h file behind which is hidden a scripting language, and to have your C++ code call scripts without knowing that that are scripts. To accomplish this, you will have to do the following:
Write the glue code yourself. (For Lua, this is relatively easy, until you get into classes, whereupon it's not so easy, which is why tools like SWIG and toLua++ exist.)
Hide behind the interface some kind of global state of the scripting interpreter.
Supposing you have multiple .h files that each are implemented using scripts, you have to decide which ones share state in the scripting language and which ones use separate scripting states. (What you basically have is a VM for the scripting language, and the extremes are (a) all .h files use the same VM in common and (b) each .h file has its own separate, isolated VM. Other choices are more complicated.)
If you decide to do this yourself, writing the glue code to turn Lua tables into C++ classes (so that Lua code looks like C++ to the rest of the program) is fairly straightforward. Going in the other direction, where you wrap your C++ in Lua (so that C++ objects look to the scripts like Lua values) is a big pain in the ass.
No matter what you do, you have some work ahead of you.
Google's V8 engine is written in C++, I expect you might be able to integrate it into a project. They talk about doing that in this article.
Good question, I have often thought about this myself, but alas there is no easy solution to this kind of thing. If you are on Windows (I guess not), then you could achieve something like this by creating COM components in C++ and VB (considering that as a scripting language). The talking happens through COM interfaces, which is a nice way to interop between disparate languages. Same holds for .NET based languages which can interop between themselves.
I too am eager to know if something like this exists for C++, preferably open source.
You might check into embedding Guile (a scheme interpreter) or V8 (Google's javascript interpreter - used in Chrome - which is written in C++).
Try the Ring programming language
http://ring-lang.net
(1) Extension using the C/C++ languages
https://en.wikibooks.org/wiki/Ring/Lessons/Extension_using_the_C/C%2B%2B_languages
(2) Embedding Ring Interpreter in C/C++ Programs
https://en.wikibooks.org/wiki/Ring/Lessons/Embedding_Ring_Interpreter_in_C/C%2B%2B_Programs
(3) Code Generator for wrapping C/C++ Libraries
https://en.wikibooks.org/wiki/Ring/Lessons/Code_Generator_for_wrapping_C/C%2B%2B_Libraries