I'm working on a very large framework exposing many algorithms, functions and features from C/C++ to Lua. This framework also exposes some functionality that works with tables, indexers, and often is merely a proxy for default behaviour if a certain condition is not true.
My question is very basic; Is it safe to pop function arguments from a C function in i.e. a newindex implementation in C? I currently implement it by getting an object which should hold the key and value and insert it before the key and value provided to the __newindex metamethod. I then use lua_rawset to set it to the object I placed in front, popping off two of the function arguments. Is this safe?
I ran extensive tests, including checkstack tests, and monitoring the top of the Lua stack. None of these tests are conclusive enough for me though, I would like to have the opinion of other Lua developers.
I've read many parts of the Lua documentation, however there does not seem to be a clear statement about this (or I have not found it). The manual (https://www.lua.org/manual/5.3/manual.html#lua_CFunction) does state:
Any other value in the stack below the results will be properly discarded by Lua.
However this doesn't answer my question if discarding these values myself can cause problems.
// An example, is this safe to do? My guts say yes
int lm_entity__newindex(lua_State* L) {
luaL_checkany(L, 1); // udata
luaL_checkany(L, 2); // key
luaL_checkany(L, 3); // value
// does not manipulate the stack, merely casts lua_touserdata
auto entity = lm_entity(L, 1);
// get the members table
lua_rawgeti(L, LUA_REGISTRYINDEX, entity->members_ref);
// place before key and value (lua_rotate)
lua_insert(L, -3);
// store, in turn popping the two function arguments
lua_rawset(L, -3);
// pop members table
lua_pop(L, 1);
return 0;
}
I expect this to work safely, because in a function call the stack is restored afterwards based on the number of parameters passed on to the function and the number of return values pushed onto the stack. I'm not certain, though, that popping two of the arguments off the stack will cause issues in that regard, even though my testing seems to prove otherwise.
Related
I am using the Lua API for C/C++ to create functions in C/C++ and then register them for use in Lua scripts. Obviously these functions expect a certain number of arguments with specific types, which is no problem in C/C++ because of static typing. However, given that Lua is dynamic, I need to manually check the number of arguments passed and their types when I call this C-function from Lua. These arguments are pushed onto the Lua stack, so I can check the number of arguments with lua_gettop and then check their individual types with lua_type.
My question, though, is can I guarantee the number of elements on the stack, that is, the number of arguments passed to my function? That way, I could check this at the beginning of my C-function with an assertion: assert(lua_gettop(L) == x), with x just being a placeholder for the desired number of arguments and L being a lua_State*. This extends a little deeper, as I wonder if other functions that interact with the Lua stack clean up so that the stack is empty when I call this function; otherwise, if some things were left sitting on the stack from a previous function call, the assertion would fail. Below is a simple example.
C/C++ code:
int foo(lua_State* L) {
assert(lua_gettop(L) == 2);
// do stuff
return 1;
}
Calling the function in Lua. Assume that the function expects two strings as arguments; thus, the first call would succeed and the second would fail.
foo("hello", "world")
foo("hello", "world", 1)
If you have directly registered a function to Lua, and a Lua script calls that function with N arguments, then that function will have N values on the stack when it gets called. That's how it works, and no process can interfere with this (well, you could manually call that function from C with an improper lua_State, but that's your fault).
So long as you don't do any stack manipulation between the start of that function and whatever you use to check the current stack size, then your test will be accurate.
Every Lua function is well-documented in how (or if) it manipulates the stack. Whether it inserts values to the stack, removes them from the stack, and so forth.
That being said, assert is probably the wrong tool. If the user provides the wrong number of parameters, and you consider that erroneous behavior, you should invoke lua_error. And not just on debug builds.
I have objects in my C++ program that I pass to Lua as userdata, and I override the metatable for this userdata so that assignments to and from indices of the object (via __newindex and __index) result in a call to C, which converts the assignment to affect the C++ object or convert the C++ element to a Lua value (either another userdata or a base type like bool, number, string). The userdata is passed in as arguments to event-like Lua functions that are called from my C++ program.
luaL_newmetatable(L, "object");
lua_pushstring(L, "__index");
lua_pushvalue(L, -2); /* pushes the metatable */
lua_settable(L, -3); /* metatable.__index = metatable */
luaL_openlib(L, NULL, vallib_m, 0);
luaL_openlib(L, "internal", vallib_f, 0);
lua_pushstring(L, "__index");
lua_pushstring(L, "get");
lua_gettable(L, 2); /* get val.get */
lua_settable(L, 1); /* metatable.__index = val.get */
lua_pushstring(L, "__newindex");
lua_pushstring(L, "set");
lua_gettable(L, 2); /* get array.set */
lua_settable(L, 1); /* metatable.__newindex = val.set */
However, this doesn't allow me to assign the actual variable itself, only an index of the variable. There is no meta-event for directly overriding the assignment operator, so I am looking for a workaround.
In other words, I can do this: lua_userdata_object_passed_as_arg_to_event["is_it_true"]=true
and it assigns the Lua boolean to my internal C++ object, but if I do this:
lua_userdata_object_passed_as_arg_to_event = new_object()
it will change what the Lua variable references, but it won't do anything to the core object as I understand it.
One workaround I've considered is some hack requiring developers to do lua_userdata_object_passed_as_arg_to_event["__self"] = new_object()
if they want to change the object itself, but this is undesirable.
So I've found some unique solutions to overriding the assignment operator by using global variables and overriding the global metatable assignment operators, but I am looking to see if anyone can help me expound this solution. See http://lua-users.org/lists/lua-l/2012-01/msg00413.html and https://www.lua.org/pil/14.2.html.
In particular, my variables are function arguments, not globals, so how can I convert to globals through the C API so that any assignments will be captured by a custom C function that will take action if the assignment is happening to a global userdata?
By the way, my userdata is a pointer to an object to avoid duplicating large objects, if that matters.
Lua and C/C++ are different languages with different needs. In C/C++, a variable always references a specific object. You can change the contents of this object, but you can never make a variable deal with a different object. If you do a = b;, you are copying the value of b into a. You can never change what object a is talking about.
In Lua, variables do not permanently reference anything. Thus, there is a distinction between the object the variable currently holds and the value of that object.
Your kludge via global variables functions, but local variables are things that don't really exist. They're locations on the Lua stack; you can't override Lua's default behavior with regard to them.
The best way to handle this is to accept the difference between C and Lua. Write code in Lua the way you would write code in Lua. Don't try to make Lua code work like C; that way is folly. They are different languages, and you should embrace their differences, not work against them.
If you want to give Lua the ability to do the equivalent of a = b, then you should create a function in your type called assign or something that will allow you to assign to the object's value. Just like if you want to copy all of the table elements from one table to another, you have to write a function to do that.
I am sure some of you might run into this problem. I have a userdata object called matrix written in C++ with usual ways of operator overloading, such as.
CMatrix<T>& operator=(const CMatrix<T>& b);
CMatrix<T>& operator=(const T& rhs);
In C++ when I create two matrices say A and B and make A=B then A and B can be used as two independent objects. However, in Lua when I write A=B and change any property of B, then A changes as well.
It is apparent and also from the Lua manual it is said that Lua makes the assignment of userdata by reference which explains the above mentioned behaviour. However, how can I make A=B to pass by value so that when B changes A is not affected.
As a matter of fact, I want to make the assignment A=B pass by reference which is indeed very fast and which Matlab does, but when I set a property of B for the first time, I want B to be created independently which is practiced my Matlab as I could have tracked from memory usage of Matlab.
If this is possible, is it done inside C++ or somewhere in the lua wrapper codes? Any example code would be great.
EDIT 1: Here is my idea, I am not sure if it will work at all or if so fast enough
typedef struct luaelement
{
int type;
std::string name;
void* addr; //newly added field
bool isRef; //newly added
} luaelement;
glbLuaElementSet=new set<luaelement,comparenocaseforluaelement>();
int l_newindex(lua_State* L)
{
luaelement element;
const char* key=lua_tostring(L,-2);
string str=key;
element.name=key;
element.type=lua_type(L,-1);
//How can I get the address, maybe a metamethod named address
glbLuaElementSet->insert(element);
lua_rawset(L,1);
}
void l_registermetamethod(lua_State* L)
{
lua_getglobal(L,"_G");
lua_createtable(L, 0, 1);
lua_pushcfunction(L, l_newindex);
lua_setfield(L, -2, "__newindex");
lua_setmetatable(L, -2);
}
Now with the glbLuaElementSet variable and l_newindex metamethod I can track all the variables inserted at global _G table. I was planning to implement and see if any reference to already existing userdata variable is in place by checking the void* address. I am not sure if this will really work and if it is worth the effort in terms of performance.
However, how can I make A=B to pass by value so that when B changes A is not affected.
You can't.
Remember: Lua is dynamically typed. So while A and B happen to store your matrix type right now, it's perfectly fine to later go A = 1. Now, A stores an integer.
C++ and Lua are very different languages. In C++, variables are either objects or references to objects (pointers are objects of pointer type). Each variable will only ever hold the object that it starts with. The value stored in that object can be changed, but the object itself exists and has a lifetime defined by the lifetime of the variable in question.
In Lua, a variable is just a box that objects can be stored in. That box has no relationship to the object that it currently happens to store; any box can hold any object. And at any time, you can swap what's in that box with the object from any other box.
You cannot interfere in the copying of one variable into another (generally. You could do metatable gymnastics, but that would only apply to members of that table. Local variables would never be affected). This is simply how Lua works as a language. C++ variables are objects; Lua variables are storage boxes. It's best to accept that you can't write C++-style code in Lua, and instead focus on writing Lua-style code in Lua.
So if you want to create a copy of an object, you have to create a copy of that object explicitly.
Here is my idea, I am not sure if it will work at all or if so fast enough
It will not work for several reasons.
First, you're applying a metatable to the global table itself, which is typically... rude.
Second, even if your code worked, it wouldn't work for something as simple as this:
globalVar = {} --Perfectly legal to set a table inside the global table.
globalVar.value = A
globalVar.value = B --Will not alert your copying code.
The __newindex metamethod is not recursive. It can't walk up and down the hierarchy of tables. So a table stored within the global table can still have its members changed.
Stop trying to make Lua into something it isn't. Work with the language you have, not the language you want.
In C++ FAQ:
Assuming a typical C++ implementation that has registers and a stack,
the registers and parameters get written to the stack just before the
call to g(), then the parameters get read from the stack inside g()
and read again to restore the registers while g() returns to f().
regarding the nested function call
void f()
{
int x = /*...*/;
int y = /*...*/;
int z = /*...*/;
...code that uses x, y and z...
g(x, y, z);
...more code that uses x, y and z...
}
1/ Are all implementation of C++ with registers and stack? Does it mean: implementation dependent on compiler/processor/computer architecture?
2/ What is sequence of instructions (without assembly language, just the big picture) when i call f() ? I have read diverging things on this topic, and also I don't remember that registers where mentionned, but only stack.
3/ what are additional specificities/points to underline when you deal with nested functions?
thanks
For number 2 this depends on many things including the compiler and the platform. Basically the different ways of passing and returning arguments to functions are called calling conventions. the article Calling conventions on the x86 platform goes into some detail on sequence of operations and you can see how ugly and complicated it gets with just this small combination of platforms and compilers which is most likely why you have heard all sorts of different scenarios, The gen on function calling conventions. covers a wider set of scenarios including 64 bit platforms but is harder to read. It gets even more complicated because gcc may not actually push and pop the stack but directly manipulate the stack pointer, we can see an example of this, albeit in assembly here. It is hard to generalize about calling conventions, if the number of arguments is small enough many calling conventions can avoid using the stack at all and will use registers exclusively.
As to number 3, nested functions does not change anything, it will just repeat the procedure over again for the next function call.
As to number 1 As Sean pointed out .Net compiles to byte code with performs all it's operations on the stack. The Wikipedia page on Common Intermediate Language has a good example.
The x86-64 ABI document is another great document if you want to understand how one specific calling convention works in detail. Figure 3.5 and 3.6 are neat since they give a nice example of a function with many parameters and how each parameter is passed using a combination of general purpose registers, floating point registers and the stack. This sort of nice diagram is a rare find when looking at documents that cover calling conventions.
1. Although register/stack implementations are the most common underlying implementations of a C++ compiler there's nothing to stop you using a different architecture. For example, you could write a compiler to generate Java bytecode or .NET byte code, in which case you'd have a stack based C++ compiler.
2. When you call f() the typical approach is:
Push the return address on the stack and jump to f()
In f():
Allocate space for the locals x,y and z. This is normally done on the stack. Take a look at this article on call stacks.
When you get to g(x,y,z) the compiler will generate code to push the values onto the stack by accessing their values in the stack frame of f(). Note that C/C++ pushes parameters from right to left.
When you get to the end of f() the compiler inserts a return instructions. The top of the stack has the address to return to (it was pushed prior to the call to f() )
3. There's nothing special about nested functions as everything follows the same basic template:
To call a function - push parameters and call the function.
Within the function - allocate space for local variables within a stack
Now this is the general approach. Compilers will introduce their own optimizations to improve performace. For example a compiler may choose to store the first 2 parameters in registers (for example).
NOTE: Although parameter passing by the stack is by far the most common approach there are others. Take a look at this article on register windows if you're interested in finding out more.
I have C++ objects and I have Lua objects/tables. (Also have SWIG C++ bindings.)
What I need to be able to do is associate the two objects so that if I do say
CObject* o1 = getObject();
o1->Update();
it will do the equivalent Lua:
myluatable1.Update();
So far I can imagine that CObject::Update would have the following code:
void CObject::Update(){
// Acquire table.
// ???
// Do the following operations on the table.
lua_getfield(L, -1, "Update");
lua_pcall(L, 0, 0, 0);
}
How would I store/set the Lua table to be used, and what would go in the // ??? above to make the Update call work?
I cant believe nobody noticed this!
http://www.lua.org/pil/27.3.2.html
A section of the Lua API for storing references to lua objects and tables and returning references for the purposes of being stored in C structures!!
I am curious for the reasons of this "reverse SWIG"...
The objects in Lua live within the Lua contexts, so at a minimum you would need to store "L" inside your object.
The issue of passing the "table pointer" is a bit more delicate - even though Lua allows to retrieve the pointer off the Lua stack (using lua_topointer()), there is no way of putting that back. Understandingly - because otherwise one would also need to check that the pointer does point to a valid object, etc, etc.
What you might do however, is to store the references to the tables in the global table, the index being the lightuserdata being the pointer to your object. Then by having the Lua state, and the name of the global array, you can retrieve the reference to the table and push it onto the Lua stack for that context.
This is sketchy, and I haven't even touched the question of garbage-collecting with this construct.
But in any case this is not going to be the speed racer performance-wise, and looks like a lot of boilerplate C++ code to me. I'd try to reconsider the approach and push some of what you want to do into the Lua domain.
p.s. looks like it is the third question which seems almost a dupe of the previous two ones, here and here are the previous ones. If those were not answered fully, would have been better to edit them/add the bounty to accumulate the answers.