This Lua code, creates a table and dynamically adds a new member. Running this I can get "hello" on the screen as expected:
foo = {}
foo.x = "hello"
print(foo.x)
But now I'm using SWIG to bind some C++ classes to Lua.
For that purpose, in a test.i (SWIG module file) I created a simple class like this:
%module test
%inline
%{
class Foo
{
public:
Foo() { X = 0; }
void SetX(int x) { X = x; }
int GetX() { return X; }
private:
int X;
};
%}
Then I wrote a test Lua code like that:
obj = test.Foo()
obj:SetX(5)
print("Number: " .. obj:GetX())
Running and getting "Number 5" as expected. The problem is that when I dynamically add a new member to my SWIG-binded object, and I try to access it, like so:
obj.Y = 7
print("Number: " .. obj.Y)
I get this error message:
"attempt to concatenate field 'Y' (a nil value)"
Is it possible to dynamically add new members on objects binded using SWIG? Is there some option without having to move to another Lua binding library?
SWIG doesn't use tables for its objects; it uses userdata. After all, those objects are C++ objects, and need to store C++ data that Lua code shouldn't be able to touch.
And I wouldn't bother looking for "another Lua binding library"; pretty much all of them use userdata, which Lua code explicitly cannot modify (in order to provide the ability to do exactly this).
However, that doesn't mean you can't cheat.
You can always wrap the object you get from C++ code into your own Lua table, which would have a metatable that forwards unknown calls to the C++ object. The code to do so would look something like this:
local function WrapObject(cppObject)
local proxy = {}
local wrapper_metatable = {}
function wrapper_metatable.__index(self, key)
local ret = rawget(self, key)
if(not ret) then
ret = cppObject[key]
if(type(ret) == "function") then
return function(self, ...)
return ret(cppObject, ...)
end
else
return ret
end
else
return ret
end
end
setmetatable(proxy, wrapper_metatable)
return proxy
end
The returned proxy object is a Lua table that can have keys and values set on it. When you get a value, such as to call a function, it will see if that value was set in the table. If not, it attempts to fetch it from the C++ object that you wrapped, which will go through its metatable.
You'll need to expand this metatable if your C++ class uses other metafunctions like __add, __sub, __tostring and so forth.
Related
I'm totally new to Lua. From what I understand it is possible to bind Lua variables and methods to ones on the C/C++ side.
But is it possible to bind a user type variable to the call of the C/C++ function which may look like properties on the Lua side ?
Example:
// C++
struct player {
int get_value(const std::string& property) const {
auto it = values.find(name);
if (it != values.end()) {
return it->second;
}
return -1;
}
std::map<std::string, int> values;
};
And on the Lua side:
-- Lua
p = player.new()
print(p.score)
So when I call p.score in Lua, it gets translated to the call to the player::get_value function on C++ side with the value of property "score" ?
SOLUTION
Thanks to #Vlad for the directions!
I came up using sol2 which is in my opinion a very nice C++ library binding to Lua!
Here is how it can be done with sol2:
struct Player {
sol::object get(const std::string& key, sol::this_state state) {
const auto& it = values.find(key);
if (it != values.cend()) {
return sol::make_object(state, it->second);
}
return sol::lua_nil;
}
std::map<std::string, int> values;
};
int main() {
Player player;
player.values.emplace("score", 123);
sol::state lua;
lua.open_libraries(sol::lib::base);
lua.new_usertype<Player>("Player", sol::meta_function::index, &Player::get);
lua.set("player", &player);
lua.script(R"(
print("Player score: ", player.score)
)");
return 0;
}
Console output
Player score: 123
The object player should have metatable set with fields __index/__newindex set to C function that will be called when Lua is trying to read or write the field that doesn't exist in the Lua object.
Normally the Lua object representing the native object (player in your case) will be a userdata, either storing the pointer to the C++ object, or hosting it within its storage.
The metamethods __index/__newindex will receive in arguments the reference to the object being queried, the key value (e.g. your score field), and the value to store in case of __newindex metamethod. So you can find your native object and desired property easily.
Some people prefer using existing binding solutions - tolua++/sol/luabind/etc, but it's very simple to implement required functionality yourself.
I own a Go library, gofileseq, for which I would like to try and made a C/C++ binding.
It is pretty straightforward to be able to export functions that use simple types (ints, strings, ...). It is even easy enough to export data from custom Go types to C by defining a C struct and translating the Go type to it, to be used in the exported functions, since you are allocating C memory to do it. But with the go 1.5 cgo rules I am finding it difficult to figure out how to export functionality from a more complex struct that stores state.
Example of a struct from gofileseq that I would like to export somehow to a C++ binding:
// package fileseq
//
type FrameSet struct {
frange string
rangePtr *ranges.InclusiveRanges
}
func NewFrameSet(frange string) (*FrameSet, error) {
// bunch of processing to set up internal state
}
func (s *FrameSet) Len() int {
return s.rangePtr.Len()
}
// package ranges
//
type InclusiveRanges struct {
blocks []*InclusiveRange
}
type InclusiveRange struct {
start int
end int
step int
cachedEnd int
isEndCached bool
cachedLen int
isLenCached bool
}
As you can see, the FrameSet type that I want to expose contains a slice of pointers to an underlying type, each of which stores state.
Ideally, I would love to be able to store a void* on a C++ class, and make it just a simple proxy for calling back into exported Go functions with the void*. But the cgo rules disallow C storing a Go pointer longer than the function call. And I am failing to see how I could use an approach of defining C++ classes that could be allocated and used to operate with my Go library.
Is it possible to wrap complex types for exposure to C/C++?
Is there a pattern that would allow a C++ client to create a Go FrameSet?
Edit
One idea I can think of would be to let C++ create objects in Go that get stored on the Go side in a static map[int]*FrameSet and then return the int id to C++. Then all the C++ operations make requests into Go with the id. Does that sound like a valid solution?
Update
For now, I am proceeding with testing a solution that uses global maps and unique ids to store objects. C++ would request a new object to be created and only get back an opaque id. Then they can call all of the methods exported as functions, using that id, including requesting for it to be destroyed when done.
If there is a better approach than this, I would love to see an answer. Once I get a fully working prototype, I will add my own answer.
Update #2
I've written a blog post about the final solution that I ended up using: http://justinfx.com/2016/05/14/cpp-bindings-for-go/
The way I ended up solving this, for lack of a better solution, was to use private global maps on the Go side (ref). These maps would associate instances of the Go objects with a random uint64 id, and the id would be returned to C++ as an "opaque handle".
type frameSetMap struct {
lock *sync.RWMutex
m map[FrameSetId]*frameSetRef
rand idMaker
}
//...
func (m *frameSetMap) Add(fset fileseq.FrameSet) FrameSetId {
// fmt.Printf("frameset Add %v as %v\n", fset.String(), id)
m.lock.Lock()
id := FrameSetId(m.rand.Uint64())
m.m[id] = &frameSetRef{fset, 1}
m.lock.Unlock()
return id
}
Then I use reference counting to determine when C++ no longer needs the object, and remove it from the map:
// Go
func (m *frameSetMap) Incref(id FrameSetId) {
m.lock.RLock()
ref, ok := m.m[id]
m.lock.RUnlock()
if !ok {
return
}
atomic.AddUint32(&ref.refs, 1)
// fmt.Printf("Incref %v to %d\n", ref, refs)
}
func (m *frameSetMap) Decref(id FrameSetId) {
m.lock.RLock()
ref, ok := m.m[id]
m.lock.RUnlock()
if !ok {
return
}
refs := atomic.AddUint32(&ref.refs, ^uint32(0))
// fmt.Printf("Decref %v to %d\n", ref, refs)
if refs != 0 {
return
}
m.lock.Lock()
if atomic.LoadUint32(&ref.refs) == 0 {
// fmt.Printf("Deleting %v\n", ref)
delete(m.m, id)
}
m.lock.Unlock()
}
//C++
FileSequence::~FileSequence() {
if (m_valid) {
// std::cout << "FileSequence destroy " << m_id << std::endl;
m_valid = false;
internal::FileSequence_Decref(m_id);
m_id = 0;
m_fsetId = 0;
}
}
And all C++ interactions with the exported Go library communicate via the opaque handle:
// C++
size_t FileSequence::length() const {
return internal::FileSequence_Len(m_id);
}
Unfortunately it does mean that in a multhreaded C++ environment, all threads would go through a mutex to the map. But it is only a write lock when objects are created and destroyed, and for all method calls on an object it is a read lock.
I have a Visual Studio 2008 C++03 application that uses Lua 5.2.1. I would like to extend Lua with a module called "foo", but when I call require("foo") in my Lua script, I get the error:
foo_test.lua:1: module 'foo' not found:
no field package.preload['process']
no file '!\lua\process.lua'
no file '!\lua\process\init.lua'
no file '!\process.lua'
no file '!\process\
My Lua script:
foo.bar()
My lua_foo.h file:
#include <lua.h>
extern "C" int luaopen_foo( lua_State* L );
My lua_foo.cpp file:
#include "lua_foo.h"
#include <lua.hpp>
static int l_bar( lua_State *L )
{
puts( "in bar()" );
return 1;
}
int luaopen_foo( lua_State *L )
{
static const luaL_Reg foo[] = {
{ "bar", l_bar },
{ NULL, NULL }
};
luaL_newlib( L, foo );
return 1;
}
These are compiled in to a static library "lua_foo.lib" which is statically linked to my main Lua executable.
Can anybody help me understand where I'm going wrong? thanks. I would prefer to avoid c++ wrappers (for now) and I do not want to package this library as a separate DLL from the main Lua engine.
EDIT
The issue was in the lua engine code. I added the luaL_requiref per #NicolBolas 's suggestion.
lua_State* L = luaL_newstate();
if( NULL != L )
{
luaL_openlibs( L );
luaL_requiref( token.get(), "foo", luaopen_foo, 1 );
luaL_dofile( L, "foo_test.lua" );
lua_close( L );
}
It's important to understand how the require machinery works and therefore why your code doesn't.
require is designed to look for Lua scripts in the file system and DLLs. Static libraries are not DLLs; indeed, as far as C/C++ is concerned, once you've finished linking, static libraries are no different than compiling those .c/.cpp files into your application directly.
When require finds a DLL with the appropriate name, it loads it and attempts to find a function named luaopen_<modname>, where <modname> is the name of the module. When it does, it will execute this function and store the value it returns in an internal database of loaded modules.
Calling require for a module will return whatever this function returned; if the module has already been loaded, then the return value is pulled from the database and returned directly.
Simply calling luaopen_foo will not do any of this. Indeed, simply calling this function is a bad idea; it is a Lua function and needs to be called as a Lua function (ie: you need to push it onto the Lua stack with lua_pushcfunction and call it with lua_call and so forth).
If you want to create a local module (one not in a Lua script or DLL, but exposed from your code), then you need to use the Lua facilities to do that. Specifically, use luaL_requiref:
luaL_requiref(L, "foo", luaopen_foo, 0);
Call this instead of calling luaopen_foo directly. This will automatically register the return value from luaopen_foo with require's internal database of loaded modules. Thus, subsequent calls to require "foo" will return this table.
One more thing: do is a keyword in Lua; you should not use keywords for Lua table key names. You can, but you always have to quote them (ie: your script must do foo["do"](...) to call it).
luaopen_foo creates a table with one function in it, but it doesn't expose it to Lua in any way. You need to assign it to something your scripts can access if you want to access it. You can do this with the package mechanism, or just assign it to a global (which is what Lua's built-in libraries do).
You have a field named do, which is problematic if you want to use foo.do syntax, because do is a keyword.
The return value of a Lua function tells Lua how many values you left on the stack. Your l_do function lies with its return value.
In the case of luaopen_foo, since you're calling it directly and ignoring it's return value, there's no need for it to return anything at all.
Change your code to this:
static int l_bar( lua_State *L )
{
puts("l_bar called.");
return 0;
}
void luaopen_foo( lua_State *L )
{
static const struct luaL_Reg foo[] = {
{ "bar", l_bar },
{ NULL, NULL }
};
luaL_newlib( L, foo ); // create table containing `bar`
lua_setglobal(L, "foo"); // assign that table to global `foo`
}
And change your script to this:
foo.bar()
I'm using luabind 0.9.1 from Ryan Pavlik's master distribution with Lua 5.1, cygwin on Win XP SP3 + latest patches x86, boost 1.48, gcc 4.3.4. Lua and boost are cygwin pre-compiled versions.
I've successfully built luabind in both static and shared versions.
Both versions pass all the tests EXCEPT for the test_object_identity.cpp test which fails in both versions.
I've tracked down the problem to the following issue:
If an entry in a table is created for NON built-in class (i.e., not int, string, etc), the value CANNOT be retrieved.
Here's a code piece that demonstrates this:
#include "test.hpp"
#include <luabind/luabind.hpp>
#include <luabind/detail/debug.hpp>
using namespace luabind;
struct test_param
{
int obj;
};
void test_main(lua_State* L)
{
using namespace luabind;
module(L)
[
class_<test_param>("test_param")
.def_readwrite("obj", &test_param::obj)
];
test_param temp_object;
object tabc = newtable(L);
tabc[1] = 10;
tabc[temp_object] = 30;
TEST_CHECK( tabc[1] == 10 ); // passes
TEST_CHECK( tabc[temp_object] == 30 ); // FAILS!!!
}
tabc[1] is indeed 10 while tabc[temp_object] is NOT 30! (actually, it seems to be nil)
However, if I use iterate to go over tabc entries, there're the two entries with the CORRECT key/value pairs.
Any ideas?
BTW, overloading the == operator like this:
#include <luabind/operator.hpp>
struct test_param
{
int obj;
bool operator==(test_param const& rhs) const
{
return obj == rhs.obj;
}
};
and
module(L)
[
class_<test_param>("test_param")
.def_readwrite("obj", &test_param::obj)
.def(const_self == const_self)
];
Doesn't change the result.
I also tried switching to settable() and gettable() from the [] operator. The result is the same. I can see with the debugger that default conversion of the key is invoked, so I guess the error arises from somewhere therein, but it's beyond me to figure out what exactly the problem is.
As the following simple test case show, there're definitely a bug in Luabind's conversion for complex types:
struct test_param : wrap_base
{
int obj;
bool operator==(test_param const& rhs) const
{ return obj == rhs.obj ; }
};
void test_main(lua_State* L)
{
using namespace luabind;
module(L)
[
class_<test_param>("test_param")
.def(constructor<>())
.def_readwrite("obj", &test_param::obj)
.def(const_self == const_self)
];
object tabc, zzk, zzv;
test_param tp, tp1;
tp.obj = 123456;
// create new table
tabc = newtable(L);
// set tabc[tp] = 5;
// o k v
settable( tabc, tp, 5);
// get access to entry through iterator() API
iterator zzi(tabc);
// get the key object
zzk = zzi.key();
// read back the value through gettable() API
// o k
zzv = gettable(tabc, zzk);
// check the entry has the same value
// irrespective of access method
TEST_CHECK ( *zzi == 5 &&
object_cast<int>(zzv) == 5 );
// convert key to its REAL type (test_param)
tp1 = object_cast<test_param>(zzk);
// check two keys are the same
TEST_CHECK( tp == tp1 );
// read the value back from table using REAL key type
zzv = gettable(tabc, tp1);
// check the value
TEST_CHECK( object_cast<int>(zzv) == 5 );
// the previous call FAILS with
// Terminated with exception: "unable to make cast"
// this is because gettable() doesn't return
// a TRUE value, but nil instead
}
Hopefully, someone smarter than me can figure this out,
Thx
I've traced the problem to the fact that Luabind creates a NEW DISTINCT object EVERY time you use a complex value as key (but it does NOT if you use a primitive one or an object).
Here's a small test case that demonstrates this:
struct test_param : wrap_base
{
int obj;
bool operator==(test_param const& rhs) const
{ return obj == rhs.obj ; }
};
void test_main(lua_State* L)
{
using namespace luabind;
module(L)
[
class_<test_param>("test_param")
.def(constructor<>())
.def_readwrite("obj", &test_param::obj)
.def(const_self == const_self)
];
object tabc, zzk, zzv;
test_param tp;
tp.obj = 123456;
tabc = newtable(L);
// o k v
settable( tabc, tp, 5);
iterator zzi(tabc), end;
std::cerr << "value = " << *zzi << "\n";
zzk = zzi.key();
// o k v
settable( tabc, tp, 6);
settable( tabc, zzk, 7);
for (zzi = iterator(tabc); zzi != end; ++zzi)
{
std::cerr << "value = " << *zzi << "\n";
}
}
Notice how tabc[tp] first has the value 5 and then is overwritten with 7 when accessed through the key object. However, when accessed AGAIN through tp, a new entry gets created. This is why gettable() fails subsequently.
Thx,
David
Disclaimer: I'm not an expert on luabind. It's entirely possible I've missed something about luabind's capabilities.
First of all, what is luabind doing when converting test_param to a Lua key? The default policy is copy. To quote the luabind documentation:
This will make a copy of the parameter. This is the default behavior when passing parameters by-value. Note that this can only be used when passing from C++ to Lua. This policy requires that the parameter type has an accessible copy constructor.
In pratice, what this means is that luabind will create a new object (called "full userdata") which is owned by the Lua garbage collector and will copy your struct into it. This is a very safe thing to do because it no longer matters what you do with the c++ object; the Lua object will stick around without really any overhead. This is a good way to do bindings for by-value sorts of objects.
Why does luabind create a new object each time you pass it to Lua? Well, what else could it do? It doesn't matter if the address of the passed object is the same, because the original c++ object could have changed or been destroyed since it was first passed to Lua. (Remember, it was copied to Lua by value, not by reference.) So, with only ==, luabind would have to maintain a list of every object of that type which had ever been passed to Lua (possibly weakly) and compare your object against each one to see if it matches. luabind doesn't do this (nor do I think should it).
Now, let's look at the Lua side. Even though luabind creates two different objects, they're still equal, right? Well, the first problem is that, besides certain built-in types, Lua can only hold objects by reference. Each of those "full userdata" that I mentioned before is actually a pointer. That means that they are not identical.
But they are equal, if we define an __eq meta operation. Unfortunately, Lua itself simply does not support this case. Userdata when used as table keys are always compared by identity, no matter what. This actually isn't special for userdata; it is also true for tables. (Note that to properly support this case, Lua would need to override the hashcode operation on the object in addition to __eq. Lua also does not support overriding the hashcode operation.) I can't speak for the authors of Lua why they did not allow this (and it has been suggested before), but there it is.
So, what are the options?
The simplest thing would be to convert test_param to an object once (explicitly), and then use that object to index the table both times. However, I suspect that while this fixes your toy example, it isn't very helpful in practice.
Another option is simply not to use such types as keys. Actually, I think this is a very good suggestion, since this kind of light-weight binding is quite useful, and the only other option is to discard it.
It looks like you can define a custom conversion on your type. In your example, it might be reasonable to convert your type to a Lua number which will behave well as a table index.
Use a different kind of binding. There will be some overhead, but if you want identity, you'll have to live with it. It sounds like luabind has some support for wrappers, which you may need to use to preserve identity:
When a pointer or reference to a registered class with a wrapper is passed to Lua, luabind will query for it's dynamic type. If the dynamic type inherits from wrap_base, object identity is preserved.
I need some advice how can I bind a C/C++ structure to Ruby. I've read some manuals and I found out how to bind class methods to a class, but I still don't understand how to bind structure fields and make them accessible in Ruby.
Here is the code I'm using:
myclass = rb_define_class("Myclass", 0);
...
typedef struct nya
{
char const* name;
int age;
} Nya;
Nya* p;
VALUE vnya;
p = (Nya*)(ALLOC(Nya));
p->name = "Masha";
p->age = 24;
vnya = Data_Wrap_Struct(myclass, 0, free, p);
rb_eval_string("def foo( a ) p a end"); // This function should print structure object
rb_funcall(0, rb_intern("foo"), 1, vnya); // Here I call the function and pass the object into it
The Ruby function seems to assume that a is a pointer. It prints the numeric value of the pointer instead of it's real content (i.e., ["Masha", 24]). Obviously the Ruby function can't recognize this object —I didn't set the object's property names and types.
How can I do this? Unfortunately I can't figure it out.
You have already wrapped your pointer in a Ruby object. Now all you have to do is define how it can be accessed from the Ruby world:
/* Feel free to convert this function to a macro */
static Nya * get_nya_from(VALUE value) {
Nya * pointer = 0;
Data_Get_Struct(value, Nya, pointer);
return pointer;
}
VALUE nya_get_name(VALUE self) {
return rb_str_new_cstr(get_nya_from(self)->name);
}
VALUE nya_set_name(VALUE self, VALUE name) {
/* StringValueCStr returns a null-terminated string. I'm not sure if
it will be freed when the name gets swept by the GC, so maybe you
should create a copy of the string and store that instead. */
get_nya_from(self)->name = StringValueCStr(name);
return name;
}
VALUE nya_get_age(VALUE self) {
return INT2FIX(get_nya_from(self)->age);
}
VALUE nya_set_age(VALUE self, VALUE age) {
get_nya_from(self)->age = FIX2INT(age);
return age;
}
void init_Myclass() {
/* Associate these functions with Ruby methods. */
rb_define_method(myclass, "name", nya_get_name, 0);
rb_define_method(myclass, "name=", nya_set_name, 1);
rb_define_method(myclass, "age", nya_get_age, 0);
rb_define_method(myclass, "age=", nya_set_age, 1);
}
Now that you can access the data your structure holds, you can simply define the high level methods in Ruby:
class Myclass
def to_a
[name, age]
end
alias to_ary to_a
def to_s
to_a.join ', '
end
def inspect
to_a.inspect
end
end
For reference: README.EXT
This is not a direct answer to your question about structures, but it is a general solution to the problem of porting C++ classes to Ruby.
You could use SWIG to wrap C/C++ classes, structs and functions. In the case of a structure, it's burning a house to fry an egg. However, if you need a tool to rapidly convert C++ classes to Ruby (and 20 other languages), SWIG might be useful to you.
In your case involving a structure, you just need to create a .i file which includes (in the simplest case) the line #include <your C++ library.h>.
P.S. Once more, it's not a direct answer to your question involving this one struct, but maybe you could make use of a more general solution, in which case this may help you.
Another option is to use RubyInline - it has limited support for converting C and Ruby types (such as int, char * and float) and it also has support for accessing C structurs - see accessor method in the API.