I need to extend my c++ app with python scripts but I'm unsure which interface library I should use. The basic communication is: c++ app registers some class methods to the script (so that they could be loaded as module in the script), then calls a specific function in the python script. The script should then perform its task while being able to call the c++ methods. When the script has finished, the c++ app should be able to use the return value of the script (e.g. as a std::list).
From what I've read boost.python is pretty powerful and considering the fact that I already use boost, it seems the way to go. But in the documentation, I have not seen a way to expose class methods (static or not) to the python script.
Is this possible? How can I do that?
Edit:
Ok, I think I should have been a little more specific, sorry. I already know about how to import python libraries etc. into my c++ code and I've taken a look into the boost documentation regarding embedding but didn't got what I was looking for.
For clarity, this is the workflow in my scenario:
C++ app starts running
At some point, a C++ class needs support from a specific python script which is located somewhere at a defined plugin-location on the hard disk.
Before calling the script, the C++ class wants to expose some of its methods to the script for callbacks (the script should only be able to call this methods this time, i.e. when the script has finished execution and gets invoked from another place in the c++ app, it should not be able to call the class methods anymore. Also, I don't want to expose the whole class to the script - only the specific methods and the script should not be able to create instances of the class. Furthermore, I don't want that other python scripts are able to call the exposed methods, only the one that I'm going to call.)
After "registering" the class methods to the script, the c++ app calls the needed function in the script and gets the return value as a c++ type (e.g. a std::list)
I think the third point is the tricky part here and I've no idea how I can achieve this...
Is this possible?
Absolutely, yes.
How can I do that?
In the most general sense,
Import python libraries with:
boost::python::object obj = boost::python::import("module_name");
Access library objects and subobjects and functions with:
obj.attr("name");
That should be enough to get you going - as you get further, the details are in the documentation.
(Example:)
auto calendar = boost::python::import( "calendar" );
auto leap = calendar.attr("isleap")("2015");
Related
Before April 2019, it was possible for a Lua script to reflect the methods and properties of a LuaBridge class using string keys __parent, __class, __propget, and __propset. This was an incredibly useful tool for creating test scripts and development tools to maintain a large class framework exported into Lua.
From looking through the release notes of LuaBridge, it seems the string keys were removed for security reasons. But I find myself placing more priority on the ability to reflect classes from scripts than on whether scripts can muck around in the object model. Especially in dev mode.
The LuaBridge Reference Manual implies there is a way for a C/C++ program to expose the metatables, but I haven't been able to figure out how to do it. I am quite new to interfacing between C and Lua (or LuaBridge), so I'm not surprised that I am perplexed. If anyone could share an example of how to do this, I would appreciate it.
Since no one seems to have a good suggestion, I realized I could roll my own reflection of the names of properties, methods, and constants. For my purposes I don't really need to execute or access them.
So rather than muck around inside LuaBridge I added __class, __propget, and __propset static properties to every class in my framework. These return a table implemented with std::map which LuaBridge supports. The tables simple have the names as keys and a dummy value.
One caveat I discovered is that you have to add static properties to derived classes when creating the derived class with deriveClass. If you try to add them later (using beginClass), LuaBridge throws an assert.
The DYNAMIC FACTORY pattern describes how to create a factory that
allows the creation of unanticipated products derived from the same
abstraction by storing the information about their concrete type in
external metadata
from : http://www.wirfs-brock.com/PDFs/TheDynamicFactoryPattern.pdf
The PDF says:
Configurability
. We can change the behavior of an application by just changing its configuration
information. This can be done without the need to change any source code (just change the descriptive information about the type in the metadata repository) or to restart the application (if caching is not used – if caching is used the cache will need to be flushed).
It is not possible to introduce new types to a running C++ program without modifying source code. At the very least, you'd need to write a shared library containing a factory to generate instances of the new type: but doing so is expressly rules out by the PDF:
Extensibility / Evolvability
. New product types should be easily
added without requiring neither a
new factory class nor modifying
any existing one.
This is not practical in C++.
Still, the functionality can be achieved by using metadata to guide some code writing function, then invoking the compiler (whether as a subprocess or a library) to create a shared library. This is pretty much what the languages mentioned in the PDF are doing when they use reflection and metadata to ask the virtual machine to create new class instances: it's just more normal in those language environments to need bits of the compiler/interpreter hanging around in memory, so it doesn't seem such a big step.
Yes...
Look at the Factories classes in the Qtilities Qt library.
#TonyD regarding
We can change the behavior of an application by just changing its configuration information.
It is 100% possible if you interpret the sentence in another way. What I read and understand is you change a configuration file (xml in the doc) that gets loaded to change the behaviour of the application. So perhaps your application has 2 loggers, one to file and one to a GUI. So the config file can be edited to choose one or both to be used. Thus no change of the application but the behaviour is changed. The requirement is that anything that you can configure in the file is available in the code, so to say log using network will not work since it is not implemented.
New product types should be easily added without requiring neither a new factory class nor modifying any existing one.
Yes that sounds a bit impossible. I will accept the ability to add ones without having to change the original application. Thus one should be able to add using plugins or another method and leave the application/factory/existing classes in tact and unchanged.
All of the above is supported by the example provided. Although Qtilities is a Qt library, the factories are not Qt specific.
I need to provide my users the ability to write mathematical computations into the program. I plan to have a simple text interface with a few buttons including those to validate the script grammar, save etc.
Here's where it gets interesting. These functions the user is writing need to execute at multi-megabyte line speeds in a communications application. So I need the speed of a compiled language, but the usage of a script. A fully interpreted language just won't cut it.
My idea is to precompile the saved user modules into objects at initialization of the C++ application. I could then use these objects to execute the code when called upon. Here are the workflows I have in mind:
1) Testing(initial writing) of script: Write code in editor, save, compile into object (testing grammar), run with test I/O, Edit Code
2) Use of Code (Normal operation of application): Load script from file, compile script into object, Run object code, Run object code, Run object code, etc.
I've looked into several off the shelf interpreters, but can't find what I'm looking for. I considered JAVA, as it is pretty fast, but I would need to load the JAVA virtual machine, which means passing objects between C and the virtual machine... The interface is the bottleneck here. I really need to create a native C++ object running C++ code if possible. I also need to be able to run the code on multiple processors effectively in a controlled manner.
I'm not looking for the whole explanation on how to pull this off, as I can do my own research. I've been stalled for a couple days here now, however, and I really need a place to start looking.
As a last resort, I will create my own scripting language to fulfill the need, but that seems a waste with all the great interpreters out there. I've also considered taking an existing open source complier and slicing it up for the functionality I need... just not saving the compiled results to disk... I don't know. I would prefer to use a mainline language if possible... but that's not required.
Any help would be appreciated. I know this is not your run of the mill idea I have here, but someone has to have done it before.
Thanks!
P.S.
One thought that just occurred to me while writing this was this: what about using a true C compiler to create object code, save it to disk as a dll library, then reload and run it inside "my" code? Can you do that with MS Visual Studio? I need to look at the licensing of the compiler... how to reload the library dynamically while the main application continues to run... hmmmmm I could then just group the "functions" created by the user into library groups. Ok that's enough of this particular brain dump...
A possible solution could be use gcc (MingW since you are on windows) and build a DLL out of your user defined code. The DLL should export just one function. You can use the win32 API to handle the DLL (LoadLibrary/GetProcAddress etc.) At the end of this job you have a C style function pointer. The problem now are arguments. If your computation has just one parameter you can fo a cast to double (*funct)(double), but if you have many parameters you need to match them.
I think I've found a way to do this using standard C.
1) Standard C needs to be used because when it is compiled into a dll, the resulting interface is cross compatible with multiple compilers. I plan to do my primary development with MS Visual Studio and compile objects in my application using gcc (windows version)
2) I will expose certain variables to the user (inputs and outputs) and standardize them across units. This allows multiple units to be developed with the same interface.
3) The user will only create the inside of the function using standard C syntax and grammar. I will then wrap that function with text to fully define the function and it's environment (remember those variables I intend to expose?) I can also group multiple functions under a single executable unit (dll) using name parameters.
4) When the user wishes to test their function, I dump the dll from memory, compile their code with my wrappers in gcc, and then reload the dll into memory and run it. I would let them define inputs and outputs for testing.
5) Once the test/create step was complete, I have a compiled library created which can be loaded at run time and handled via pointers. The inputs and outputs would be standardized, so I would always know what my I/O was.
6) The only problem with standardized I/O is that some of the inputs and outputs are likely to not be used. I need to see if I can put default values in or something.
So, to sum up:
Think of an app with a text box and a few buttons. You are told that your inputs are named A, B, and C and that your outputs are X, Y, and Z of specified types. You then write a function using standard C code, and with functions from the specified libraries (I'm thinking math etc.)
So now your done... you see a few boxes below to define your input. You fill them in and hit the TEST button. This would wrap your code in a function context, dump the existing dll from memory (if it exists) and compile your code along with any other functions in the same group (another parameter you could define, basically just a name to the user.) It then runs the function using a functional pointer, using the inputs defined in the UI. The outputs are sent to the user so they can determine if their function works. If there are any compilation errors, that would also be outputted to the user.
Now it's time to run for real. Of course I kept track of what functions are where, so I dynamically open the dll, and load all the functions into memory with functional pointers. I start shoving data into one side and the functions give me the answers I need. There would be some overhead to track I/O and to make sure the functions are called in the right order, but the execution would be at compiled machine code speeds... which is my primary requirement.
Now... I have explained what I think will work in two different ways. Can you think of anything that would keep this from working, or perhaps any advice/gotchas/lessons learned that would help me out? Anything from the type of interface to tips on dynamically loading dll's in this manner to using the gcc compiler this way... etc would be most helpful.
Thanks!
I have looked at multiple sources and I just do not understand them. Mostly all of them are either using a library like luaBind (Which I do not want to use because it relies on Boost) or they are not already instanced objects in C++ but rather created in Lua directly.
I have a Player class, which is wrapped in another class to handle Lua calling. I have created a std::vector list of both these objects in my initialization of the application.
So basically I do not want the Lua script to be creating these player objects I would just like to create a function getPlayer() which then returns the Lua wrapped object. I have no idea where to start with defining the Lua wrapped object for Lua to call nor do I know how to return an instance of the object to Lua so I can use calls from it.
Here's an example of what I would like my Lua script to look like:
player = getPlayer(1) -- Returns the Lua wrapped object from C++ where 1 is the object's index in the std::vector list
print("Player's name: ", player:name()) -- Print's the player's name
Could someone point me to a decent tutorial that explains how to do this. I am pretty sure that this isn't some sort of "Never been done and why would you want to" case so surely there must be at least one tutorial out there. I have been searching for about 2 days now with no positive results.
This isn't a Never been done task, but companies which they use Lua have been already done lots of changes and improvements on the raw source of Lua to make it fit in the place that they want. Since all of us programmers are not that cool to do such stuffs on third-parties sources, we have to use whatever exists out there. You think Crytek is using the exact same version of Lua which is resided on its website's repository? No sir.
You can make this happen by using userdata. Or you can use meta tables. There is also another piece of code named Luna which it helps you to achieve faster.
Otherwise, it's all Luabind, and believe me you're going to end up using it one way or another. Besides, its use of Boost is not that broad and there is going to be just 200kb added to the final executable. Check this address if you want to use Luabind sometime, it's really a good article on the web. http://blog.nuclex-games.com/tutorials/cxx/luabind-introduction/
If you're too bored with Lua, I may suggest you to use squirrel as an alternative to Lua. Personally, I prefer squirrel more than Lua because of its C like syntax and its abilities which they are the ones that we want. Besides, you can even declare constants and use real classes in its context which Lua is pretty failed at this. The funny thing about squirrel is that the designer is a guy that he has been working on FarCry on developing Lua for its engine.
I'm trying to get this thing with using different ClassLoaders to work with no success and I'm getting kind of desperate. What I'm trying to do is to launch 2 different instances of a 3rd party program that was created with several static attributes and therefore can't just be instantiated twice in my code. I've been advised to use different ClassLoaders to load the .jar file, though I don't really know how to do this. Could I maybe get some pointers on where I could start?
I also need to pass an object of my own program to this program instance and I also have no idea of how I could do this...
Thanks in advance for the help,
Andre
PS: This whole discussion started in the java forums, so there's a more detailed description of what I need to do and why I need to do this. The advice I'm referring to in this post was from malcommmc:
Another approach might be to run the
whole system in one JVM, but create a
new classloader for each instance of
the 3rd party code and make sure that
the classes containing the offending
statics are only loadable by these new
class loaders. Under these conditions
the JVM can support multiple copies of
the same class, providing the FQN is
unique within a class loader.
In this case run each instance as a
separate Thread, and set the thread's
context class loader to point to the
relevant class loader.
Presumably the object you are
proposing to pass is specified with
some interface or abstract class known
to the 3rd party app. You need that
interface to be known to the system
class loader, i.e. on the class path,
but the principal classes of the 3rd
party app must not be on the
classpath, but be accessed as a jar
referenced by a URLClassLoader.
This was already solved in the Java forums - link provided in the question