I would like to embed a scripting language(js, python, perl, even php, anything that's easy to use) in an OpenGL C++ application. I'd like to do this in order to be able to do things like:
print values of various C++ class members at runtime
cause interrupts that would wake up gdb at runtime
after I find a bug I'd like to write a small script oneliner to replicate it
I'm pretty sure this won't be easy. I want to ask whether this is a good/bad idea, and if it's worth the effort.
Example usecase
Let's suppose I rotate a line until it collides something and my collision detection has some SIGSEGV which occurs upon collision. I print out all the angles, find out which one was the one before the SIGSEGV and I write a small python thingie which displays some values so I can figure out what went wrong etc.
I guess basically What I'm trying to do is to avoid gdb and uhm.. I'd like if the program blows to have a way to check things in Python instead.
It's not that I don't like gdb, it's that I don't like the limited commands it has..
UPDATE: GDB can now be extended with Python out of the box. That solves a lot of the limitations of canned sequences of commands.
I don't think that for the purpose of debugging embedding a scripting language is a good idea. It's definitely possible, but for everything that you'd want to be able to access from the scripting language, you would have to provide some interface, since there's -- to the best of my knowledge at least -- no way to directly call C++ or read C++ data structures from a scripting language. I'd suggest you learn gdb or look for gdb frontends if you don't want to use gdb directly. I've used ddd and found it quite useful. The gdb frontend of Eclipse CDT is usable, too. Using a debugger gives you more flexibility, since the debugger knows about C++ and its data structures and thus allows you to inspect anything at runtime, without having to manually write much support code for that.
First of all: You can print all those out using GDB quite easily. Once in GDB you might want to try what "help data" shows.
Which IDE do you use? You might want to try the cross-platform IDE Code::Blocks, which interfaces GDB quite neatly.
If you want to interface with another language, you might want to have a look on "Lua". It is very easy to learn.
See http://en.wikipedia.org/wiki/Lua_%28programming_language%29
Lua is intended to be embedded into other applications, and accordingly it provides a robust, easy-to-use C API. The API is divided into two parts: the Lua core and the Lua auxiliary library.
This works well with C++, too, of course.
i can´t help with the questions if it´s a good or bad idea.
but you could take a look at Chaiscript that´s really easy to use with c++
Python is a very good choice as application scripting language. Its not a strong argument but I know that lua is widely used in game programming (I d'ont know why), it may be useful since you are using OpenGL.
Use lua - it's simple stack based interpreter. With very clean API.
E.g.: Simple code that add dt_format|dt_convert functions to lua
Related
I know that many large-scale applications such as video games are created using multiple langages. For example, it's likely the game/physics engines are written in C++ while gameplay tasks, GUI are written in something like Python or Lua.
I understand why this division of roles is done; use lower-level languages for tasks that require extreme optimization, tweaking, efficiency and speed, while using higher-level languages to speed up production time, reduce nasty bugs ect.
Recently, I've decided to undertake a larger personal project and would like to divy-up parts of the project similar to above. At this point in time, I'm really confused about how this interoperability between languages (especially compiled vs interpreted) works.
I'm quite familiar with the details of going from ANSCII code test to loading an executable, when written in something like C/C++. I'm very curious at how something like a video game, built from many different languages, works. This is a large/broad question, but specifically I'm interested in:
How does the code-level logic work? I.e. how can I call Python code from a C++ program? Especially since they don't support the same built-in types?
What does the program image look like? From what I can tell, a video game is running in a single process, so what does the runtime image look like when running a C/C++ program that calls a Python function?
If calling code from an interpreted language from a compiled program, what are the sequence of events that occur? I.e If I'm inside my compiled executable, and for some reason have a call to an interpreted language inside a loop, do I have to wait for the interpreter every iteration?
I'm actually finding a hard time finding information on what happening at the machine-level, so any help would be appreciated. Although I'm curious in general about interoperation of software, I'm specifically interested in C++ and Python interaction.
Thank you very much for any insight, even if it's just pointing me to where I can find more information.
In the specific case of python, you have basically three options (and this generally applies across the board):
Host python in C++: From the perspective of the C++ programme, the python interpreter is a C library. On the python side, you may or may not need to use something like ctypes to expose the C(++) api.
Python uses C++ code as DLLs/SOs - C++ code likely knows nothing of python, python definitely has to use a foreign function interface.
Interprocess communication - basically, two separate processes run, and they talk over a socket. These days you'd likely use some kind of web services architecture to accomplish this.
Depending on what you want to do:
Have a look at SWIG: http://www.swig.org/ It's a tool that aims to connect C/C++ code with Python, Tcl, Perl, Ruby, etc. The common use case is a Python interface (graphical or not) that will call the C/C++ code. SWIG will parse the C/C++ code in order to generate the interfaces.
Libpython: it's a lib that allows you to embed Python code. You have some examples here: http://docs.python.org/3.0/extending/embedding.html
My question is a little bit stupid but I decided to ask advanced programmers like some of you. So I want to make a "dynamic" C++ program. My idea is to compile it and after compilation (maybe with scripting language like python) to change some how the code of the program. I know you will tell me that after the compilation I can not change the code but is there a way of doing that. Thank you!
You could design a modular architecture using plugins in external libraries. For example you could use the Command Pattern. That way you can dynamically load code that was generated after you main program. You would have to fix an interface though. Functions like GetProcAddress in the Windows api might be a good point to start.
For dynamic coding and rapid prototyping I recommend to have a look at Lua. The engine is very small and easy to integrate in your c++ program.
The only way to do that in C++ is to unload the DLL with the code to be
modified, modify the sources, invoke the compiler to regenerate the DLL,
and reload the DLL. It's very, very heavy weight, and it only works if
the compiler is present on the machines where the code is to be run.
(Usually the case under Unix, rarely the case with Windows.)
Interpreted languages like Python are considerably more dynamic; Python
has a built-in function to execute a string as Python code, for example.
If you need dynamically modifiable code, I'd suggest embedding Python in
your application, and using it for the dynamic parts.
Personally I have never played with re-compiling C++ during runtime, and I do not intend too. However I have been doing a lot of embedding of scripting languages lately.
Someone else mentioned the obvious first choice for embedding: Lua. Lua is a pretty popular language to embed and you will find a bunch of documentation about how to do it. Integrating it into the C++ will allow you to define behavior at runtime like you want.
What I am using is a wonderful langauge called Squirrel. It is a lot like Lua but with native object(class) support and C++-like syntax. I have managed to embed it into a C++ application, and through using the sqrat binding library both languages can share information easily.
I have squirrel building and initializing my UI. As a result, 0 compiling is required in order to craft and structure my UI for my game. I intend to take this a step further and use this gameplay-side in order to create similar behavior you are looking for(changing behavior at runtime through dynamic code usage)
I recommend Checking out squirrel here: http://www.squirrel-lang.org/
I plan on writing tutorials on how to embed squirrel and install the binding library, but I have not started on them yet. If I can remember, I will edit this post in the future (could be a few months) once I have completed them. In the meantime give it a shot yourself. You may find it to your liking.
I recently asked this question https://softwareengineering.stackexchange.com/questions/129076/go-instead-of-c-c-with-cgo and got some very interesting input. However there's a mistake in my question: I assumed cgo could also be used to access c++ code but that's not possible. Instead you need to use SWIG.
The go faq says "The cgo program provides the mechanism for a “foreign function interface” to allow safe calling of C libraries from Go code. SWIG extends this capability to C++ libraries. "
my question:
Is it possible to access high-level c++ frameworks such as QT with SWIG + Go and get productive? I'd like to use Go as a "scripting language" to utilize c++ libraries.
Have you any experience with go and swig? Are there pitfalls I have to be aware of?
Update/Answer: I've asked this over IRC too and I think the question is solved:
SWIG is a rather clean way of interfacing c++ code from other languages. Sadly matching the types of c++ to something like go can be very complex and in most cases you have to specify the mapping yourself. That means that SWIG is a good way to leverage an existing codebase to reuse already written algorithms. However mapping a library like Qt to go will take you ages. Mind it's surely possible but you don't want to do it.
Those of you that came here for gui programming with go might want try go-gtk or the go version of wxWidgets.
Is it possible? Yes.
Can it be done in a reasonably short period of time? No.
If you go back and look at other projects that have taken large frameworks and tried to put an abstraction layer on it, you'll find most are "incomplete". You can probably make a fairly good start and get some initial wrappers in place, but generally even the work to get the simple cases solved takes time when there is a lot of underlying code to wrap, even with automated tools (which help, but are never a complete solution). And then... you get to the nasty remaining 10% that will take you forever (ok, a really really long time at least). And then think about how it's a changing target in the first place. Qt, for example, is about to release the next major rewrite.
Generally, it's safest to stick to the framework language that the framework was designed for. Though many have language extensions within the project itself. For example, for Qt you should check out QML, which provides (among many other things) a javascript binding to Qt. Sort of. But it might meet your "scripting" requirement.
A relevant update on this issue: it is now possible to interact with C++ using cgo with this CL, which is merged for Go 1.2. It is limited, however, to C-like functions calls, and classes, methods and C++ goodies are not supported (yet, I hope).
There are plenty of logging/trace systems for letting your program output data or strings or state as it runs. Most of these let you print arbitrary strings which you can view live or after your program runs.
I noticed an ad here on SO for Smartinspect which seems to take this to a higher level, giving stack traces for each log, fancier options like plotting graphs and data values which change over time, and a lot of polish to the basic idea of a simple list of output text strings.
Since I use C++, Smartinspect won't even work for me.
A little googling finds tons of logging frameworks, but nothing that seems to do anything more than text dumps. Are their fancier tools (similar to Smartinspect?) that do more? Commercial or open source is fine, and multiplatform is a big plus.
I know this is not the answer you are most probably looking for but I would suggest that such a framework will be very hard (if not impossible) to find for C++. Doing something like dumping the stack cannot be done in a portable way as it can in a language like Java, which not only shares a common runtime accross all platforms, but provides powerful introspection capabilities too.
I don't program in Java, but my guess is that it can provide a stack-trace in the same way as Python: the stack is probably just another object in the runtime which can be inspected and manipulated.
C++ on the other hand has none of these niceities: its meant to be a close-to-the-metal language that basically adds object-orientism to C (I'm sure others will come up with much more elanorate explanations of C++'s benefit's over C but thats another discussion).
In short, C++ is not rich enough at the level required to provide the kind of features you require in a generic way. There may be some platform-specific code that could get some of this info at defined points for you out there, but it certainly wont be standards compliant, cross-platform C++.
With regards to graphs etc, that sounds much more like post-processing, which you should either be able to find something for, or more likely, you can perhaps output your log messages in a format which can be interpreted by some of these existing tools.
Other things you could look at would be integrating with syslogd, for which again, there may be richer analysis tools for (this would provide you with a capability along the lines of the one advertised for SmartInspect - that is TCP/IP based logging).
NB: a lot of what I said here about C++ comes from previous experiences trying to find decent frameworks in C++ to do tweaky, introspective type things (such as proper mock objects etc).
I wrote a article about dumping the stack in C/C++ with Windows and Unix/Linux at DDJ some years ago. Maybe it helps you:
See http://www.ddj.com/architect/185300443
If you can restrict yourself to a certain platform you can add stack traces to your logs manually. We use e.g. the glibc functionality to get stack traces on Linux to attach stack traces into our exception class. There is similar functionalyty available on Windows, but as mentioned these infrastructures are not portable.
I want to move various parts of my app into simple scripts, to allow people that do not have a strong knowledge of c++ to be able to edit and implement various features.
Because it's a real time app, I need to have some kind of multitasking for these scripts. Ideally I want it so that the c++ app calls a script function which then continues running (under the c++ thread) until either a pause point (Wait(x)), or it returns. In the case of it waiting the state needs to be saved ready for the script to be restarted the next time the app loops after the duration has expired.
The scripts also need to be able to call c++ class methods, ideally using the c++ classes rather than plain wrapper functions around c++ classes.
I don't want to spend a massive amount of time implementing this, so using an existing scripting language is preferred to writing my own. I heard that Python and Lua can be integrated into a c++ app, but I do not know how to do this to achieve my goals.
The scripts must be able to call c++ functions
The scripts must be able to "pause" when certain functions are called (eg. Wait), and be restarted again by the c++ thread
Needs to be fast -- this is for a real time app and there could potentially be a lot of scripts running.
I can probably roll the multitasking code fairly easily, provided the scripts can be saved and restarted (possibly by a different thread to the original).
You can use either Lua or Python. Lua is more "lightweight" than python. It's got a smaller memory footprint than python does and in our experience was easier to integrate (people's mileage on this point might vary). It can support a bunch of scripts running simultaneously. Lua, at least, supports stopping/starting threads in the manner you desire.
Boost.python is nice, but in my (limited) experience, it was difficult for us to get compiling for our different environments and was pretty heavyweight. It has (in my opinion) the disadvantage of requiring Boost. For some, that might not be a problem, but if you don't need Boost (or are not using it), you are introducing a ton of code to get Boost.python working. YMMV.
We have built Lua into apps on multiple platforms (win32, Xbox360 and PS3). I believe that it will work on x64. The suggestion to use Luabind is good. We wound up writing our own interface between the two and while not too complicated, having that glue code will save you a lot of time and perhaps aggravation.
With either solution though, debugging can be a pain. We currently have no good solution for debugging Lua scripts that are embedded into our app. Since we haven't used python in our apps I can't speak to what tools might be available there, but a couple of years ago the landscape was roughly the same -- poor debugging. Having scripting to extend functionality is nice, but bugs in the scripts can cause problems and might be difficult to locate.
The Lua code itself is kind of messy to work with if you need to make changes there. We have seen bugs in the Lua codebase itself that were hard to track down. I suspect that Boost::Python might have similar problems.
And with any scripting language, it's not necessarily a solution for "non-programmers" to extend functionality. It might seem like it, but you will likely wind up spending a fair amount of time either debugging scripts or even perhaps Lua.
That all said, we've been very happy with Lua and have shipped it in two games. We currently have no plans to move away from the language. All in all, we've found it better than other alternatives that were available a couple of years ago. Python (and IronPython) are other choices, but based on experience, they seem more heavy handed than Lua. I'd love to hear about other experiences there though.
I can highly recommend that you take a look at Luabind. It makes it very simple to integrate Lua in your C++ code and vice versa. It is also possible to expose whole C++ classes to be used in Lua.
Your best bet is to embed either lua (www.lua.org) or python (www.python.org). Both are used in the game industry and both access extern "C" functions relatively easily with lua having an edge here (because data types are easier to translate between lua and C). Interfacing to C++ objects will be a bit more work on your end, but you can look up how to do this on Google, or on lua or python discussion forums.
I hope that helps!
You can definitely do what you want with Python. Here are the docs on embedding Python into an application. I'm pretty sure Lua would work too, I'm just less familiar with it.
You're describing cooperative multi-tasking, where the script needs to call a Break or Wait function periodically. Perhaps a better solution would be to run the scripting language in its own thread, and then use mutexes or lock-free queues for the interfaces between the scripting language and the rest of your program. That way a buggy script that doesn't call Break() often enough can't accidentally freeze your program.
Take a look at the Boost.Python library. It looks like it should be fairly straightforward to do what you want.
Take a look at SWIG. I've used it to interface with Python, but it supports many other languages.
One more vote for Lua. It's small, it's fast, it doesnt consume much memory (for games your best bet is to allocate big buffer at the initialization and re-direct all Lua memory allocations there). We used tolua to generate bindings, but there are other options, most of them much smaller/easier to use (IMO) than boost.python.
As for debugging Lua (if you go that route), I have been using DeCoda, and it has not been bad. It pretends to be an IDE, but sorta fails at that, but you can attach The debugging process to visual studio, and go down the call stack at break points. Very handy for Tracking down that bug.
You can also embed C/C++ scripts using Ch. I've been using it for a game project I'm working on, and it does well. Nice blend of power and adaptability.