I'm coding something in C++ and I would like to offer a scripting layer within my application, during a search for a workable solution I found this FFI extensions but I can't really find proper documentation for them, the guy who invented this or any other "reliable" and technical source, the only things that is clear to me is that this technology is cross-language, in LuaJit, Ruby and Haskell there is someone talking about this stuff but I have no clue about what this "thing" is.
Is something comparable to SWIG ? It's the new kid on the block ?
FFI is a concept. It's what many languages call their glue layer that enables you to call into other languages (very often this bridge is a to a C ABI), and it's thus different for each project. (e.g. this for Erlang
libffi is often used to implement that glue layer in the language, as is SWIG.
Excuse me for a very generic answer, but your questio also seems such..
In contrast to Java, there is no "one FFI" and there's no "one reliable scripting environment". In Java you have the JS Rhino scripting language almost 'built in' into the JRE.
C/C++ have the power of "relatively easily" interfacing with all/any of other platforms. This is because all other platforms are usually implemented in C/C++. Sorry for generalizing and oversimplifying. But, the point is, that IF you had the other-platforms' C++ sourcecodes, you'd add new things to them, just by observing how the standard ones were created.
The point is, that usually all other platforms are either:
closed source
open source, but licensed in a way that prevents you from using it
open source, well-licensed, but LARGE/COMPLICATED enough, that you simply dont want to compile/change their internals
Therefore, every other platform produces on its own some way of calling C APIs. It's just so that i.e. you can use Win32API from Ruby/Python without really recompiling whole Python's runtime just because you wanted to call SendKeys etc.
So, usually, it's the "other platforms" that call into C/C++ than the other way. It's because they all already have a way to do that. Even Java has JNI and .net has P/Invoke, right?
But, now, the calling C++-from-X is possible, depending on X or Y or Z platform, it differs very much. And very often, it works but is tedious to define or just unwieldy. I.e. in Ruby on Windows you can call any C api by:
giving the actual name of the function i.e. _##2AV21moveMeByDistance
giving encoded list of parameter types i.e. ifpp (int,float,pointer,pointer)
giving the proper parameters in proper types and order
if you don't follow, it will crash badly. The above examples are artificial an incorrect by the way. But it looks similar.
So, on almost any platform, there have been many tools to simplify that. On all platforms - different ones. But all are called FFI - line nos said, because it is a concept.
The tools may i.e. require you to annotate the C++ code with some extra information like:
class RUBIZE MyClass :: BaseThing
{
PUBLIC(void,int,int) void moveMeByDistance(int x, int y) { ... }
PUBLIC(void) void .......
};
and then they can process your .h files to generate the "bindings" automatically and they then generate a "binding library" that will adjust the scripting environment so that you can now directly call MyClass::moveMyByDistance(5,6) from script.
Or, they can require you to register all the bits at runtime, ie.:
class MyClass :: ScriptObject
{
void moveMeByDistance(int x, int y) { ... }
void .......
};
int dllmain()
{
ScriptingEnv::RegisterClass<MyClass>();
ScriptingEnv::RegisterClass( "move", &MyClass::moveMeByDistance );
ScriptingEnv::RegisterClass( "eat", .... );
}
but again, all of the above examples are artificial.
As the C/C++ can interface to any other platform (thanks to the fact that any other platform usually has tools for that :P), you have to first decide which scripting language you would like to use. JS? Lua? Ruby? Python? All of them can. Then look for libraries that provide you with the most comfy way of bridging. There are many, but you will have to look at sites/forums/etc related to that very language. Or even here, but then ask about concrete languge/library.
Keywords to look for are:
modules --- i.e. "extending XYZ Script with new modules"
extensions --- i.e. "writing extensions for Ruby in C++"
bindings --- i.e. "Ruby bindings for FastFourierTransform Library"
calling C/C++ code/classes from XYZ languge
etc.
Related
I have a simulation program which repeats a set of functions in a specific order a large number of times and then outputs the result of some calculations based on those functions. Each function will mutate a Simulator object in some way and will take a variable number of arguments of different types. For example:
class Simulator
{
Simulator();
void A(int x, double y, string z);
void B(int x, int y);
void C(double x);
};
I want the user to be able to specify the order in which these functions will be called as well as the arguments, and I need to be able to store this information in some sort of script for the program to follow. For example, a script could end up being translated to the following actions for the program:
Simulator mySim;
mySim.A(5, 6.0, "a string");
mySim.B(1, 1);
mySim.A(3, 4.0, "another string");
mySim.C(10.0);
Is there an efficient way of doing this? My initial thought is to have a linked list of objects, each of which stores the name of the function, as well as the arguments. The program would then traverse the list, calling each function in turn. However, this poses a couple of problems:
(a) How to store the arguments in the list, which will be different for each function (both different in quantity as well as type).
(b) It seems rather inefficient as my program will have to execute a series of if statements on every run of the simulation, in order to determine which functions to call.
Anyone have any suggestions?
My suggestion: don't invent your own scripting language/interpreter.
If you want to quickly get started supporting user-defined scripts in a single day, check out Lua. It's arguably one of the easiest languages to embed into a C/C++ program. With luabind, it's even easier:
module(L)
[
class<Simulator>("Simulator")
.def("A", &Simulator::A)
.def("B", &Simulator::B)
.def("C", &Simulator::C)
];
Done! Now I can run lua scripts that can create Simulator and call functions on it and get all the features of the full lua language and its base libraries, computing mathematical expressions with each call, creating user-defined functions, closures, lambdas, etc. And lua is one speed demon when it comes to scripting languages.
I actually managed to turn a fairly large scale, commercial C++ program into a fully scriptable one with Lua in a single day. It's so easy to embed since that's what it was designed to do.
There's also Python which is quite similar to luabind if we use Boost.Python though a lot more painful with the C API and a bit more tedious to get started (we can't just build a statically linked library and go and there's a lot of environment variables we have to deal with like the sys.path and making sure we use the correct Python implementation for debug/release versions).
You could also choose among other languages, but those two of the most popular ones I know of which provide C++ libraries to make binding a lot easier.
You could also go with SWIG and support multiple scripting languages out of the box but SWIG takes quite a long time to learn since it's got massive documentation with a lot of details you have to pay attention to since it works for many different languages.
If you want to add a scripting language to your C++ application, you can embed LUA:
http://www.lua.org/home.html
I'm trying to adjust some mathematical code I've written to allow for arbitrary functions, but I only seem to be able to do so by pre-defining them at compile time, which seems clunky. I'm currently using function pointers, but as far as I can see the same problem would arise with functors. To provide a simplistic example, for forward-difference differentiation the code used is:
double xsquared(double x) {
return x*x;
}
double expx(double x) {
return exp(x);
}
double forward(double x, double h, double (*af)(double)) {
double answer = (af(x+h)-af(x))/h;
return answer;
}
Where either of the first two functions can be passed as the third argument. What I would like to do, however, is pass user input (in valid C++) rather than having to set up the functions beforehand. Any help would be greatly appreciated!
Historically the kind of functionality you're asking for has not been available in C++. The usual workaround is to embed an interpreter for a language other than C++ (Lua and Python for example are specifically designed for being integrated into C/C++ apps to allow scripting of them), or to create a new language specific to your application with your own parser, compiler, etc. However, that's changing.
Clang is a new open source compiler that's having its development by Apple that leverages LLVM. Clang is designed from the ground up to be usable not only as a compiler but also as a C++ library that you can embed into your applications. I haven't tried it myself, but you should be able to do what you want with Clang -- you'd link it as a library and ask it to compile code your users input into the application.
You might try checking out how the ClamAV team already did this, so that new virus definitions can be written in C.
As for other compilers, I know that GCC recently added support for plugins. It maybe possible to leverage that to bridge GCC and your app, but because GCC wasn't designed for being used as a library from the beginning it might be more difficult. I'm not aware of any other compilers that have a similar ability.
As C++ is a fully compiled language, you cannot really transform user input into code unless you write your own compiler or interpreter. But in this example, it can be possible to build a simple interpreter for a Domain Specific Language which would be mathematical formulae. All depends on what you want to do.
You could always take the user's input and run it through your compiler, then executing the resulting binary. This of course would have security risks as they could execute any arbitrary code.
Probably easier is to devise a minimalist language that lets users define simple functions, parsing them in C++ to execute the proper code.
The best solution is to use an embedded language like lua or python for this type of task. See e.g. Selecting An Embedded Language for suggestions.
You may use tiny C compiler as library (libtcc).
It allows you to compile arbitrary code in run-time and load it, but it is only works for C not C++.
Generally the only way is following:
Pass the code to compiler and create shared object or DLL
Load this Shared object or DLL
Use function from this shared object.
C++, unlike some other languages like Perl, isn't capable of doing runtime interpretation of itself.
Your only option here would be to allow the user to compile small shared libraries that could be dynamically-loaded by your application at runtime.
Well, there are two things you can do:
Take full advantage of boost/C++0x lambda's and to define functions at runtime.
If only mathematical formula's are needed, libraries like muParser are designed to turn a string into bytecode, which can be seen as defining a function at runtime.
While it seems like a blow off, there are a lot of people out there who have written equation parsers and interpreters for c++ and c, many commercial, many flawed, and all as different as faces in a crowd. One place to start is the college guys writing infix to postfix translators. Some of these systems use paranthetical grouping followed by putting the items on a stack like you would find in the old HP STL library. I spent 30 seconds and found this one:
http://www.speqmath.com/tutorials/expression_parser_cpp/index.html
possible search string:"gcc 'equation parser' infix to postfix"
I'm talking about ROOT's CINT.
I've been developing a game in c++ which uses Python for programming the AI. As much as I love Python, and how easy it makes programming the AI (generators and FP are really sexy), it makes non trivial algorithms run so slow.
Then I remembered I read somewhere about CINT, and how it can be embeddable. Now I need your help to decide if implement CINT as an alternate scripting system. With python I use Boost::Python, which makes it almost unpainful to expose classes and objects once you get used to it. Is there such ease with CINT?
I've written classes compiled against Root, and then accessed them directly in the interpreter. That's easy, though all such classes are expected to derive from TObject. What I don't know is if that is a cint requirement or a ROOT requirement. you might be best off asking on the RootTalk CINT Support forum
To address the questions in the comments:
The derivation from TObject can be second hand: your classes can be derived from something derived from TObject, it just has to be a TObject.
Root provides a tool (makecint) and some macros (ClassDef and ClassImp) to support integrating your code with the interpreted execution environment: write your clas deriving it from TObject; include the ClassDef macro in the header and the ClassImp macro in the source file; run makecint over the code to generate all the tedious integration nonesense, then compile your code and the generated code to a shared object (or, I presume, a dll on a windows box); start the interpreter; load the library with .L; and your class is fully integrated with the interpreted environment (tab completion will work and all that). The build can be automated with make (and presumable other tools). ##Again,## I don't know how much of this belongs to ROOT and how much to cint. But it is all open source, so you can snag and adapt what you need.
I have several functions in my program that look like this:
void foo(int x, int y)
Now I want my program to take a string that looks like:
foo(3, 5)
And execute the corresponding function. What's the most straightforward way to implement this?
When I say straightforward, I mean reasonably extensible and elegant, but it shouldn't take too long to code up.
Edit:
While using a real scripting language would of course solve my problem, I'd still like to know if there is a quick way to implement this in pure C++.
You can embed Python fairly simply, and that would give you a really powerful, extensible way to script your program. You can use the following to easily (more or less) expose your C++ code to Python:
Boost Python
SWIG
I personally use Boost Python and I'm happy with it, but it is slow to compile and can be difficult to debug.
You could take a look at Lua.
I'd also go for the scripting language answer.
Using pure C++, I would probably use a parser generator, which will will get the token and grammar rules, and will give me C code that exactly can parse the given function call language, and provides me with an syntax tree of that call. flex can be used to tokenize an input, and bison can be used to parse the tokens and transform them into an syntax tree. Alternatively to that approach, Boost Spirit can be used to parse the function call language too. I have never used any of these tools, but have worked on programs that use them, thus I somewhat know what I would use in case I had to solve that problem.
For very simple cases, you could change your syntax to this:
func_name arg1, arg2
Then you can use:
std::istringstream str(line);
std::string fun_name; str >> fun_name;
map[fun_name](tokenize_args(str));
The map would be a
std::map<std::string, boost::function<void(std::vector<std::string>)> > map;
Which would be populated with the functions at the start of your program. tokenize_args would just separate the arguments, and return a vector of them as strings. Of course, this is very primitive, but i think it's reasonable if all you want is some way to call a function (of course, if you want really script support, this approach won't suffice).
As Daniel said:
Script languages like Lua and Python would be the most used script languages for binding together c++ libraries.
You will have to add a script interface to your c++ application. The buildup of this interface obviously depends on what script language you chose.
CERN provides CINT, a C/C++ interpreter that can be embedded within your application to provide scripting capabilities.
If you only wish to call a function by literal name, you could use linker-specific functions.
On POSIX-compliant operating systems (like Linux), you can use dlopen() and dlsym(). You simply parse the input string and figure out the function name and arguments. Then you can ask the linker to find the function by name using dlsym().
On Windows however, these functions aren't available (unless there's some POSIX environment around, like Cygwin). But you can use the Windows API.
You can take a look here for details on these things: http://en.wikipedia.org/wiki/Dynamic_loading
C++ Reflection [2] by Fabio Lombardelli provides full re-flection for C++ through template metaprogramming tech-niques. While it is fully compliant with the C++ standards,it requires the programmer to annotate the classes in order forthem to be reflective
http://cppreflect.sourceforge.net/
otherwise you'd want a function pointer hash table i think
Does your system have to "take a string"? You could expose COM (or CORBA, or whatever) interfaces on your application and have whatever is generating these commands call into your application directly.
Most mature C++ projects seem to have an own reflection and attribute system, i.e for defining attributes which can be accessed by string and are automatically serializable. At least many C++ projects I participated in seemed to reinvent the wheel.
Do you know any good open source libraries for C++ which support reflection and attribute containers, specifically:
Defining RTTI and attributes via macros
Accessing RTTI and attributes via code
Automatic serialisation of attributes
Listening to attribute modifications (e.g. OnValueChanged)
There is a new project providing reflection in C++ using a totally different approach: CAMP.
https://github.com/tegesoft/camp
CAMP doesn't use a precompiler, the classes/properties/functions/... are declared manually using a syntax similar to boost.python or luabind. Of course, people can use a precompiler like gccxml or open-c++ to generate this declaration if they prefer.
It's based on pure C++ and boost headers only, and thanks to the power of template meta-programming it supports any kind of bindable entity (inheritance and strange constructors are not a problem, for example).
It is distributed under the MIT licence (previously LGPL).
This is what you get when C++ meets Reflection:
Whatever you choose, it'll probably have horrible macros, hard to debug code or weird build steps. I've seen one system automatically generate the serialisation code from DevStudio's PDB file.
Seriously though, for small projects, it'll be easier to write save/load functions (or use streaming operators). In fact, that might hold for big projects too - it's obvious what's going on and you'd usually need to change code anyway if the structure changes.
You could have a look at the two tools below. I've never used either of them, so I can't tell you how (im)practical they are.
XRTTI:
Xrtti is a tool and accompanying C++ library which extends the standard runtime type system of C++ to provide a much richer set of reflection information about classes and methods to manipulate these classes and their members.
OpenC++:
OpenC++ is C++ frontend library (lexer+parser+DOM/MOP) and source-to-source translator. OpenC++ enables development of C++ language tools, extensions, domain specific compiler optimizations and runtime metaobject protocols.
I looked at these things for quite a while but they tend to be very heavy-handed. They might prevent you from using inheritance, or having strange constructors etc etc. In the end they ended up being too much of a burden instead of a convenience.
This approach for exposing members that I now use is quite lightweight and lets you explore a class for serialization or setting all fields called "x" to 0, for example. It's also statically determined so is very very fast. No layers of library code or code-gen to worry about messing with the build process. It generalises to hierarchies of nested types.
Set your editor up with some macros to automate writing some of these things.
struct point
{
int x;
int y;
// add this to your classes
template <typename Visitor>
void visit(Visitor v)
{
v->visit(x, "x");
v->visit(y, "y");
}
};
/** Outputs any type to standard output in key=value format */
struct stdout_visitor
{
template <typename T>
void visit(const T& rhs)
{
rhs.visit(this);
}
template <typename Scalar>
void visit (const Scalar& s, const char* name)
{
std::cout << name << " = " << s << " ";
}
}
This is a notorious weakness of the C++ language in general because the things that would need to be standardized to make reflection implementations portable and worthwhile aren't standard. Calling conventions, object layouts, and symbol mangling come to mind, but there are others as well.
The lack of direction from the standard means that compiler implementers will do some things differently, which means that very few people have the motivation to write a portable reflection library, which means that people who need reflection re-invent the wheel, but only just enough for what they need. This happens ad infinitum, and here we are.
Looked at this for a while too. The current easiest solution seems to be BOOST_FUSION_ADAPT_STRUCT. Practically once you have a library/header you only need to add your struct fields into the BOOST_FUSION_ADAPT_STRUCT() macro, as the last segment of the code shows. Yes it has restrictions many other people have mentioned. And it does not support listeners directly.
The other promising solutions I looked into are
CAMP and XRTTI/gccxml, however both seem to be a hurdle to bring external tools dependency into your project.
Years ago I used perl c2ph/pstruct to dump the meta info from the output of gcc -gstabs, that is less intrusive but needs more work though it worked perfectly for me.
Regarding the boost/__cxa approach, once you figure out all the small details, adding/changing structs or fields is simple to maintain. we currently use it to build a custom types binding layer on top of dbus, to serialize the API and hide the transport/RPC details for a managed object service subsystem.
Not a general one but QT supports this via a meta compiler, and is GPL.
My understanding from talking to the QT people was that this isn't possible with pure C++, hence the need for the moc.
Automatic introspection/reflection toolkit. Use meta compiler like Qt's and adding meta information directly into object files. Intuitive easy to use. No external dependencies. Even allow automatically reflect std::string and then use it in scripts. Please visit IDK