I would like to embed a scripting language(js, python, perl, even php, anything that's easy to use) in an OpenGL C++ application. I'd like to do this in order to be able to do things like:
print values of various C++ class members at runtime
cause interrupts that would wake up gdb at runtime
after I find a bug I'd like to write a small script oneliner to replicate it
I'm pretty sure this won't be easy. I want to ask whether this is a good/bad idea, and if it's worth the effort.
Example usecase
Let's suppose I rotate a line until it collides something and my collision detection has some SIGSEGV which occurs upon collision. I print out all the angles, find out which one was the one before the SIGSEGV and I write a small python thingie which displays some values so I can figure out what went wrong etc.
I guess basically What I'm trying to do is to avoid gdb and uhm.. I'd like if the program blows to have a way to check things in Python instead.
It's not that I don't like gdb, it's that I don't like the limited commands it has..
UPDATE: GDB can now be extended with Python out of the box. That solves a lot of the limitations of canned sequences of commands.
I don't think that for the purpose of debugging embedding a scripting language is a good idea. It's definitely possible, but for everything that you'd want to be able to access from the scripting language, you would have to provide some interface, since there's -- to the best of my knowledge at least -- no way to directly call C++ or read C++ data structures from a scripting language. I'd suggest you learn gdb or look for gdb frontends if you don't want to use gdb directly. I've used ddd and found it quite useful. The gdb frontend of Eclipse CDT is usable, too. Using a debugger gives you more flexibility, since the debugger knows about C++ and its data structures and thus allows you to inspect anything at runtime, without having to manually write much support code for that.
First of all: You can print all those out using GDB quite easily. Once in GDB you might want to try what "help data" shows.
Which IDE do you use? You might want to try the cross-platform IDE Code::Blocks, which interfaces GDB quite neatly.
If you want to interface with another language, you might want to have a look on "Lua". It is very easy to learn.
See http://en.wikipedia.org/wiki/Lua_%28programming_language%29
Lua is intended to be embedded into other applications, and accordingly it provides a robust, easy-to-use C API. The API is divided into two parts: the Lua core and the Lua auxiliary library.
This works well with C++, too, of course.
i can´t help with the questions if it´s a good or bad idea.
but you could take a look at Chaiscript that´s really easy to use with c++
Python is a very good choice as application scripting language. Its not a strong argument but I know that lua is widely used in game programming (I d'ont know why), it may be useful since you are using OpenGL.
Use lua - it's simple stack based interpreter. With very clean API.
E.g.: Simple code that add dt_format|dt_convert functions to lua
I recently asked this question https://softwareengineering.stackexchange.com/questions/129076/go-instead-of-c-c-with-cgo and got some very interesting input. However there's a mistake in my question: I assumed cgo could also be used to access c++ code but that's not possible. Instead you need to use SWIG.
The go faq says "The cgo program provides the mechanism for a “foreign function interface” to allow safe calling of C libraries from Go code. SWIG extends this capability to C++ libraries. "
my question:
Is it possible to access high-level c++ frameworks such as QT with SWIG + Go and get productive? I'd like to use Go as a "scripting language" to utilize c++ libraries.
Have you any experience with go and swig? Are there pitfalls I have to be aware of?
Update/Answer: I've asked this over IRC too and I think the question is solved:
SWIG is a rather clean way of interfacing c++ code from other languages. Sadly matching the types of c++ to something like go can be very complex and in most cases you have to specify the mapping yourself. That means that SWIG is a good way to leverage an existing codebase to reuse already written algorithms. However mapping a library like Qt to go will take you ages. Mind it's surely possible but you don't want to do it.
Those of you that came here for gui programming with go might want try go-gtk or the go version of wxWidgets.
Is it possible? Yes.
Can it be done in a reasonably short period of time? No.
If you go back and look at other projects that have taken large frameworks and tried to put an abstraction layer on it, you'll find most are "incomplete". You can probably make a fairly good start and get some initial wrappers in place, but generally even the work to get the simple cases solved takes time when there is a lot of underlying code to wrap, even with automated tools (which help, but are never a complete solution). And then... you get to the nasty remaining 10% that will take you forever (ok, a really really long time at least). And then think about how it's a changing target in the first place. Qt, for example, is about to release the next major rewrite.
Generally, it's safest to stick to the framework language that the framework was designed for. Though many have language extensions within the project itself. For example, for Qt you should check out QML, which provides (among many other things) a javascript binding to Qt. Sort of. But it might meet your "scripting" requirement.
A relevant update on this issue: it is now possible to interact with C++ using cgo with this CL, which is merged for Go 1.2. It is limited, however, to C-like functions calls, and classes, methods and C++ goodies are not supported (yet, I hope).
I have been using C++ for a while now and I began to get interested in lower level system programming like drivers and stuff. Even some kind of primitive operating system could be very interesting project!
I have no clue where I could start. Are there any not-too-challenging things I could get started with and are there anything about C++ I should try to avoid like exceptions in performance critical code?
My current OS is Windows 7 if that matters much.
Writing Windows device drivers in C++ isn't impossible, there are not many CRT functions that you could use to get you into trouble. The new operator is unusable for example, you don't have to fear a std::bad_alloc. Unless you replace it, that cuts out a rather large swath of standard C++ library classes.
But that's not really the point of a device driver, it is rather important that you make it as small as possible. C++ pays off when you write complex code. You explicitly do not want to write complex code in a device driver. Debugging it is redrum.
Linus really likes C in the kernel. There's a good reason for that.
C++ doesn't provide quite all of the tools you will need to actually implement a full operating system in it. There are a few machine specific things that cannot be done in c++. These things are handling and raising interrupts, controlling the MMU, controlling access to supervisor cpu instructions, and a handful of other small odds and ends.
Fortunately, these things are few enough that they can be written in assembly language accessed from C++.
Have a look at osdev.org (lots of questions that will pop into your mind when considering developing your own OS are answered here).
I would strongly suggest you start by hacking existing open source device-drivers and kernels, which you can really only do in Linux or *BSD. The experience will also give you a good sense of whether you're cut out for this kind of programming.
I have heard the recently open sourced Symbian OS is written using C and C++. Not sure which parts of it are done with C++ as I have not read the code base. Consider looking into it.
Kerneltrap.org has some very good discussions about why the Linux kernel does not have C++ in its code base. Consider reading that as well.
Symbian OS is written in a variant of C++. Of course, there's assembly code for low-level things, but that is all wrapped up. You cannot use exceptions, and for real-time drivers you cannot do normal things like dynamic memory allocation, not even in C.
I recommend C Programming Language and assembler. I'm not sure if it's possible to low-level much with C++.
I want to move various parts of my app into simple scripts, to allow people that do not have a strong knowledge of c++ to be able to edit and implement various features.
Because it's a real time app, I need to have some kind of multitasking for these scripts. Ideally I want it so that the c++ app calls a script function which then continues running (under the c++ thread) until either a pause point (Wait(x)), or it returns. In the case of it waiting the state needs to be saved ready for the script to be restarted the next time the app loops after the duration has expired.
The scripts also need to be able to call c++ class methods, ideally using the c++ classes rather than plain wrapper functions around c++ classes.
I don't want to spend a massive amount of time implementing this, so using an existing scripting language is preferred to writing my own. I heard that Python and Lua can be integrated into a c++ app, but I do not know how to do this to achieve my goals.
The scripts must be able to call c++ functions
The scripts must be able to "pause" when certain functions are called (eg. Wait), and be restarted again by the c++ thread
Needs to be fast -- this is for a real time app and there could potentially be a lot of scripts running.
I can probably roll the multitasking code fairly easily, provided the scripts can be saved and restarted (possibly by a different thread to the original).
You can use either Lua or Python. Lua is more "lightweight" than python. It's got a smaller memory footprint than python does and in our experience was easier to integrate (people's mileage on this point might vary). It can support a bunch of scripts running simultaneously. Lua, at least, supports stopping/starting threads in the manner you desire.
Boost.python is nice, but in my (limited) experience, it was difficult for us to get compiling for our different environments and was pretty heavyweight. It has (in my opinion) the disadvantage of requiring Boost. For some, that might not be a problem, but if you don't need Boost (or are not using it), you are introducing a ton of code to get Boost.python working. YMMV.
We have built Lua into apps on multiple platforms (win32, Xbox360 and PS3). I believe that it will work on x64. The suggestion to use Luabind is good. We wound up writing our own interface between the two and while not too complicated, having that glue code will save you a lot of time and perhaps aggravation.
With either solution though, debugging can be a pain. We currently have no good solution for debugging Lua scripts that are embedded into our app. Since we haven't used python in our apps I can't speak to what tools might be available there, but a couple of years ago the landscape was roughly the same -- poor debugging. Having scripting to extend functionality is nice, but bugs in the scripts can cause problems and might be difficult to locate.
The Lua code itself is kind of messy to work with if you need to make changes there. We have seen bugs in the Lua codebase itself that were hard to track down. I suspect that Boost::Python might have similar problems.
And with any scripting language, it's not necessarily a solution for "non-programmers" to extend functionality. It might seem like it, but you will likely wind up spending a fair amount of time either debugging scripts or even perhaps Lua.
That all said, we've been very happy with Lua and have shipped it in two games. We currently have no plans to move away from the language. All in all, we've found it better than other alternatives that were available a couple of years ago. Python (and IronPython) are other choices, but based on experience, they seem more heavy handed than Lua. I'd love to hear about other experiences there though.
I can highly recommend that you take a look at Luabind. It makes it very simple to integrate Lua in your C++ code and vice versa. It is also possible to expose whole C++ classes to be used in Lua.
Your best bet is to embed either lua (www.lua.org) or python (www.python.org). Both are used in the game industry and both access extern "C" functions relatively easily with lua having an edge here (because data types are easier to translate between lua and C). Interfacing to C++ objects will be a bit more work on your end, but you can look up how to do this on Google, or on lua or python discussion forums.
I hope that helps!
You can definitely do what you want with Python. Here are the docs on embedding Python into an application. I'm pretty sure Lua would work too, I'm just less familiar with it.
You're describing cooperative multi-tasking, where the script needs to call a Break or Wait function periodically. Perhaps a better solution would be to run the scripting language in its own thread, and then use mutexes or lock-free queues for the interfaces between the scripting language and the rest of your program. That way a buggy script that doesn't call Break() often enough can't accidentally freeze your program.
Take a look at the Boost.Python library. It looks like it should be fairly straightforward to do what you want.
Take a look at SWIG. I've used it to interface with Python, but it supports many other languages.
One more vote for Lua. It's small, it's fast, it doesnt consume much memory (for games your best bet is to allocate big buffer at the initialization and re-direct all Lua memory allocations there). We used tolua to generate bindings, but there are other options, most of them much smaller/easier to use (IMO) than boost.python.
As for debugging Lua (if you go that route), I have been using DeCoda, and it has not been bad. It pretends to be an IDE, but sorta fails at that, but you can attach The debugging process to visual studio, and go down the call stack at break points. Very handy for Tracking down that bug.
You can also embed C/C++ scripts using Ch. I've been using it for a game project I'm working on, and it does well. Nice blend of power and adaptability.
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
C++ just sucks too much of my time by making me micro-manage my own memory, making me type far too much (hello std::vector<Thingy>::const_iterator it = lotsOfThingys.begin()), and boring me with long compile times. What's the single best alternative for serious real-time graphics programming? Garbage collection is a must (as is the ability to avoid its use when necessary), and speed must be competitive with C++. A reasonable story for accessing C libs is also a must.
(Full disclosure: I have my own answer to this, but I'm interested to see what others have found to be good alternatives to C++ for real-time graphics work.)
Edit: Thanks everyone for the thoughtful replies. Given that there's really no "right" answer to this question I won't be selecting any particular answer. Besides I'd just pick the language I happen to like as a C++ alternative, which wouldn't really be fair.
What about D Programming Language?
Some links requested in the comment:
Win32 Api
Derelict (Multimedia lib)
I wouldn't discard C++. In fact, I would consider adding Boost to your C++ library, which makes the language much more usable. Your example would become:
BOOST_FOREACH( Thingy& t, lostOfThingys ) {
// do something with 't'
}
Boost has tons of tools that help make C++ a better language.
C# is a nice language that fits your requirements, and it is definitely suited for graphics, thanks to the efforts of Microsoft to provide it with great tools and libraries like Visual Studio and XNA.
Real-time + garbage collection don't match very well I'm afraid.
It's a bit hard to make any real-time response guarantees if a garbage collector can kick in at any time and spend an undefined amount of processing...
I disagree with your premise. When used carefully and properly, C++ is a great language, especially for a domain like real-time graphics, where speed is of the essence.
Memory management becomes easy if you design your system well, and use stl containers and smart pointers.
std::vector::const_iterator it = lotsOfThingys.begin()) will become much shorter if you use
using namespace std;
typedef vector::const_iterator ThingyConstIter;
And you can shorten compile times significantly by breaking up your systems into reasonably self-contained modules, by using precompiled headers, or by using the PIMPL idiom.
Perhaps a hybrid approach. Python and C++ make a good combination (see, for example, PyGame).
Some variation of Lisp that compiles to machine code could be almost as fast as C++ for this kind of programming. The Naughty Dog team created a version of Lisp called Game Oriented Assembly Lisp, which they used to create several AAA titles, including the Jak and Daxter series. The two major impediments to a Lisp approach in the game industry would be the entrenched nature of C/C++ development (both tools and human assets are heavily invested in C/C++), as well as the difficulty of finding talented engineers who are stars in both the game programming domain and the Lisp language.
Many programming teams in the industry are shifting to a hybrid approach wherein the real-time code, especially graphics and physics code, is written in C or C++, but game logic is done in a higher-level scripting language, which is accessible to and editable by programmers and non-programmers alike. Lua and Python are both popular for higher-level scripting.
Let's not forget to mention the new 'auto' use:
auto it = lotsOfThingys.begin(); // Let the compiler figure it out.
auto it2 = lotsOfFoos.begin();
if (it==it2) // It's still strongly typed; a Thingy iter is not a Foo iter.
As a developer/researcher/professor of 3D VR applications for some 20 years I would suggest there is no alternative (except possibly C). The only way to reduce latency and enable real-time interaction is an optimized compiled language (eg C or C++) with access to a fast relaible 3D graphics library such as OpenGL. While I agree it is flustrating to have to code everything, this is also essential for performanc and optimization.
Sometimes, looking outside the beaten path you can find a real gem. You might want to consider PureBasic (Don't let the name mislead you). Here's some details:
PureBasic Features
Machine Code (Assembly) executables (FASM)
In-line Assembly support
No run-times needed (no DLLs needed,etc.) 1 executable file
Tiny executables (as small or smaller/as fast or faster than C++ w/out the runtime)
You can write DLLs
Multi-thread support
Full OS API support
Multi-platform support
Windows 95-2003
Linux
Mac-OS X
Amiga
2D & 3D game development
DirectX
OGRE
Generous Licensing
Inexpensive (79 Euros or about $112)
Life-time license (all future updates & versions included)
One price for all platforms
External Library support
3rd party DLLs
User Libraries
On-line Support
Responsive development team led by it's creator
On-line forum
One place for answers (don’t have to go all over the net)
Huge amount of sample code (try code out while in IE with IEtool)
Fast replies to questions
Bonus learning (alternative to learning C++)
API
Structures
Interfaces
Pointers
Visit the online forum to get a better idea of PureBasic (http://www.purebasic.fr/english/index.php) or the main site: www.purebasic.com
I completely agree with the mention of C# for graphics programming. It has the slight disadvantage of being a managed language and allowing the garbage collector free reign over your application is framerate suicide after a while but with some relatively intelligent pool allocations made early in the program's life any real issues can be avoided.
Several people have already mentioned XNA, which is incredibly friendly and well-documented and I would like to echo that recommendation as well. I'm personally using it for my hobby game projects and it has treated me very well.
XNA isn't the only alternative, though. There is also SlimDX which is under constant development as a means of providing a lean wrapper of DirectX in a similar fashion as Managed DirectX (which was, I believe, discontinued by Microsoft in favor of XNA). Both are worthy of research: http://code.google.com/p/slimdx/
There are no true alternatives for big AAA titles, especially on the consoles. For smaller titles C# should do.
C# is a good answer here - it has a fair garbage collection (although you'd have to profile it quite a bit - to change the way you handle things now that the entire memory handling is out of your hands), it is simple to use, have a lot of examples and is well documented.
In the 3D department it gives full support for shaders and effects and so - that would be my choice.
Still, C# is not as efficient as C++ and is slower due to overhead, so if it is speed and the flexibility to use any trick in the book you like (with pointers and assembly if you like to get your hands dirty) - stick to C++ and the price would be writing way more code as you mentioned, but having full control over everything including memory management.
I would say the D programming language is a good option. You can link to C object files and interface with C++ code through C libraries. D has garbage collection, inline assembly, and game developers have created bindings to SDL and OpenGL libraries, and are also actively working on new game development apis. I love D. Too bad my job doesn't demand it's use. :(
Like James (hopkin), for me, the hybrid approach is the best solution. Python and C++ is a good choice, but other style like C#/C++ works. All depends of your graphical context. For game, XNA is a good platform (limited to win32), in this case C#/C++ is the best solution. For scientific visualization, Python/C++ is accepted (like vtk's bindings in python). For mobile game JAVA/C++ can works...
If you are targeting Windows, C++/CLI (Microsoft's .NET 'managed' dialect of C++) is an interesting possibility, particularly if you want to leverage your C++ experience. You can mix native code (e.g. calls to C-style libraries) with .NET managed code quite seamlessly, and take advantage of .NET GC and libraries.
As far as concerns about GC impacting 'real time' performance, I think those tend to be overblown. The multi-generational .NET GC is very good at never taking much time to do a collection, unless you are in some kind of critical low-memory situation. I write .NET code that interacts with electronic derivatives exchanges, where time delays == lots of $$$, and we have never had a GC-related issue. A few milliseconds is a long, long time for the GC, but not for a human interacting with a piece of software, even a 'real time' game. If you really need true "real time" performance (for medical devices, process control, etc.) then you can't use Windows anyway - it's just not a real-time OS.
Lot of game engines can fit your need, I suppose. For example, using SDL or Cairo, if portability is needed. Lot of scripting languages (coming in general with easy syntax and garbage collection) have binding to these canvas.
Flash might be another alternative.
I will just point out Processing, which is an open source programming language and environment for people who want to program images, animation, and interactions.
Actually, it is a thin wrapper around Java, making it look like a scripting language: it has a (primitive) IDE when you can type a few lines of code and hit Run without even having to save the file. Actually it wraps the code around a class and adds a main() call, compiles it and run it in a window.
Lot of people use it for real-time exhibitions (VJ and similar).
It has the power and limitations of Java, but adds out of the box a number of nice wrappers (libraries) to simplify access to Java2D, OpenGL, SVG, etc.
Somehow, it has become a model of simple graphics language: there are several applications trying to mimic Processing in other languages, like Ruby, Scala or Python. One of the most impressive is a JavaScript implementation, using the canvas component implemented in Firefox, Safari, Opera, etc.
I vote c++0x. Partial support is already available in gcc-4.3+ using the -std=c++0x flag.
Would 'C' be too obvious an answer?
I have very successfully used C++ for the engine, with the application written in Lua on top. JavaScript is also very practical, now the latest generation of JIT based JS engines are around (tracemonkey, V8 etc).
I think C++ will be with us for a while yet; even Tim Sweeney hasn't actually switched to Haskell (pdf) yet, AFAIK :-)
Java and LWJGL (OpenGL wrapper) has worked well for me. If you're looking for more of a scene graph type library like Orge have a look at jMonkeyEngine which we used to create a google earth type application (see www.skapeworld.com). If you're sensible with object creation the garbage collection is a non issue.
If your target is a PC, I think you can try C#, or embed Lua in your C++ app and run scripts for 'high-level' stuff. However if your target is a console, you must manage your own memory!
Objective-C looks like a good match for your requirements (the latest version with optional GC), although it is too dynamic and Smalltalk-like for my taste.
XNA is your best bet I think. Being supported by the .NET framework you can build for a Windows or Xbox 360 platform by simply changing a setting in Game Studio. Best yet, all the tools are free!
If you decide to go with XNA you can easily get started using their quickstart guide
XNA Quickstart guide
It has been a rewarding and fun experiance for me so far, and a nice break from the memory management of C++.
Garbage collection is a must (as is
the ability to avoid its use when
necessary)
You can't disable a garbage collector temporarily. You would need a deterministic garbage collector then. But such a beast does come with a performance hit also. I think BEA JRockit is such a beast and then you should stick to Java.
Just to comment on your example; typedef is your friend...
typedef std::vector<Thingy> Thingys;
Thingys::const_iterator it = lotsOfThingys.begin()
Don't overlook independent languages in your quest. Emergence BASIC from Ionic Wind Software has a built in DirectX 9 engine, supports OOP and can easily interface with C libraries.
http://www.ionicwind.com
James.
The best enviroment for your project is the one you get your task done in the fastest way possible. This - especially for 3D-graphics - includes libraries.
Depending on the task, you may get away with some minor directx hacking. Then you could use .NET and slimdx. Managed languages tend to be faster to programm and easier to debug.
Perhaps you need a really good 3D-engine? Try Ogre3D or Irrlicht. You need commercial grade quality (one might argue that Ogre3D offers that) - go for Cryengine or Unreal. With Ogre3D and Irrlicht you might uses .NET as well, though the ports are not always up to date and plugins are not as easyly included as in the C++ versions. For Cryengine/Unrealengine you won't have a real choice I guess.
You need it more portable? OpenGL for the rescue - though you might need some wrapper (e.g. SDL).
You need a GUI as well? wxWidgets, QT might be a possiblity.
You already have a toolchain? Your libraries need to be able to handle the file formats.
You want to write a library? C / C++ might be a solution, since most of the world can use C / C++ libraries. Perhaps with the use of COM?
There are still a lot of projects/libraries I did not mention (XNA, Boost, ...) and if you want to create some program that does not only display 3D-graphics, you might have other needs as well (Input, Sound, Network, AI, Database, GUI, ...)
To sum it up: A programming language is a tool to reach a goal. It has to be seen in the context of the task at hand. The task has its own needs and these needs may chose the language for you (e.g. you need a certain library to get a feature that takes long to programm and you can only access the library with language X).
If you need the one-does-nearly-all: try C++/CLI (perhaps in combination with C# for easier syntax).
Good question.
As for the 'making me type far too much', C++0x seems to address most of it
as mentioned:
auto it = lotsOfThingys.begin()) // ... deduce type, just like in *ML
VS2010beta implements this already.
As for the memory management - for efficiency - you will have to keep good track of memory allocations, with or without garbage collection (ie, make memory-pools, re-use allocated object sometimes) anyhow, so that eventually whether your environment is garbage collected or not, matters less. You'll have to explicitly call the gc() as well, to keep the memory from fragmenting.
Having consistent ways to manage memory is important anywhere.
RAII - is a killer feature of C++
Another thing - is that memory is just one resource, you still have to keep track of other resources with a GC, so RIAA.
Anyhow, C# - is a nice alternative in many respects, I find it a very nice language, especially the ability to write functional-style code in it (the cute lambda -> syntax, map/select 'LINQ' syntax etc), thus the possibility to write parallel code; while it's still a 'standard curly-brackets', when you (or your colleagues) need it.
Have a look to Delphi/Pascal Object and some exemples :
http://www.delphigamer.com or http://glscene.cjb.net/
You can look at Ada. There is no garbage collector but this language is oftenly used for real-time system needing high reliability. That means less debuging times for your 3D applications.
And, you can also look at Haskell, if you don't know the functional paradigm this language will look weird to you, but it's worth a bit of your time. Tim Sweeney (EPIC Inc) is considering this language as a C++ alternative.