I am sure some of you might run into this problem. I have a userdata object called matrix written in C++ with usual ways of operator overloading, such as.
CMatrix<T>& operator=(const CMatrix<T>& b);
CMatrix<T>& operator=(const T& rhs);
In C++ when I create two matrices say A and B and make A=B then A and B can be used as two independent objects. However, in Lua when I write A=B and change any property of B, then A changes as well.
It is apparent and also from the Lua manual it is said that Lua makes the assignment of userdata by reference which explains the above mentioned behaviour. However, how can I make A=B to pass by value so that when B changes A is not affected.
As a matter of fact, I want to make the assignment A=B pass by reference which is indeed very fast and which Matlab does, but when I set a property of B for the first time, I want B to be created independently which is practiced my Matlab as I could have tracked from memory usage of Matlab.
If this is possible, is it done inside C++ or somewhere in the lua wrapper codes? Any example code would be great.
EDIT 1: Here is my idea, I am not sure if it will work at all or if so fast enough
typedef struct luaelement
{
int type;
std::string name;
void* addr; //newly added field
bool isRef; //newly added
} luaelement;
glbLuaElementSet=new set<luaelement,comparenocaseforluaelement>();
int l_newindex(lua_State* L)
{
luaelement element;
const char* key=lua_tostring(L,-2);
string str=key;
element.name=key;
element.type=lua_type(L,-1);
//How can I get the address, maybe a metamethod named address
glbLuaElementSet->insert(element);
lua_rawset(L,1);
}
void l_registermetamethod(lua_State* L)
{
lua_getglobal(L,"_G");
lua_createtable(L, 0, 1);
lua_pushcfunction(L, l_newindex);
lua_setfield(L, -2, "__newindex");
lua_setmetatable(L, -2);
}
Now with the glbLuaElementSet variable and l_newindex metamethod I can track all the variables inserted at global _G table. I was planning to implement and see if any reference to already existing userdata variable is in place by checking the void* address. I am not sure if this will really work and if it is worth the effort in terms of performance.
However, how can I make A=B to pass by value so that when B changes A is not affected.
You can't.
Remember: Lua is dynamically typed. So while A and B happen to store your matrix type right now, it's perfectly fine to later go A = 1. Now, A stores an integer.
C++ and Lua are very different languages. In C++, variables are either objects or references to objects (pointers are objects of pointer type). Each variable will only ever hold the object that it starts with. The value stored in that object can be changed, but the object itself exists and has a lifetime defined by the lifetime of the variable in question.
In Lua, a variable is just a box that objects can be stored in. That box has no relationship to the object that it currently happens to store; any box can hold any object. And at any time, you can swap what's in that box with the object from any other box.
You cannot interfere in the copying of one variable into another (generally. You could do metatable gymnastics, but that would only apply to members of that table. Local variables would never be affected). This is simply how Lua works as a language. C++ variables are objects; Lua variables are storage boxes. It's best to accept that you can't write C++-style code in Lua, and instead focus on writing Lua-style code in Lua.
So if you want to create a copy of an object, you have to create a copy of that object explicitly.
Here is my idea, I am not sure if it will work at all or if so fast enough
It will not work for several reasons.
First, you're applying a metatable to the global table itself, which is typically... rude.
Second, even if your code worked, it wouldn't work for something as simple as this:
globalVar = {} --Perfectly legal to set a table inside the global table.
globalVar.value = A
globalVar.value = B --Will not alert your copying code.
The __newindex metamethod is not recursive. It can't walk up and down the hierarchy of tables. So a table stored within the global table can still have its members changed.
Stop trying to make Lua into something it isn't. Work with the language you have, not the language you want.
Related
I have a function call
void moveMeToThread(UnsafeStruct *ptr)
{
// do stuff with ptr
}
Now I want to move moveMeToThread to a different thread, so I do not want anyone creating an object of UnsafeStruct on the stack and I also want memory of all UnsafeStruct objects made on the heap to be freed automatically. Anyone have an elegant way to do this?
Sounds like you'd like to make a heap-only class. There are many ways to force this:
you might make private ctors (all of them!) and create a static create() function that returns a pointer (sometimes called named ctor)
you might make dtor private
The latter technically does not save you from placement new to a suitable memory block, but otherwise protects from sensible coding mistakes and is way more compatible with algorithms and containers. E.g. you can still copy such an object via copy ctor outside the class, which is not possible if you make all ctors private (which is a requirement for the first version).
You might do this:
template<typename T>
class HeapOnly
{
public:
T t;
operator T&() { return t; }
operator const T&() const { return t; }
private:
~HeapOnly();
};
void moveMeToThread(HeapOnly<UnsafeStruct> *ptr)
{ /* ... */ }
int main()
{
HeapOnly<UnsafeStruct> *ptr =
new HeapOnly<UnsafeStruct>{/* args to UnsafeStruct */};
moveToThread(ptr);
}
Small note: there's no such thing as (call/parameter) stack in the C++ standard. It only appears in ItaniumABI (and potentially in other ABIs). Standard says ASDV (automatic storage duration variables) for what's commonly referred to as 'on the stack', but nothing prevents an implementation to allocate the memory on the stack (as long as compiler can prove that the object's lifetime cannot extend the stack unroll - this works e.g. if it's allocated before static initialization). It might be completely unimportant in your case, but in security-related codes, where buffer overflow is important, you can't strictly enforce not having objects allocated from the same stack this way (and thus it's suggested to do a runtime check) - but you can still enforce that the object is allocated via new (or the given static member function).
I do not want anyone creating an object of UnsafeStruct on the stack.
I want to forbid it because if someone creates an object on the stack and sends it on the thread, it can cause a crash(dangling pointer)
Do you also want to prevent anybody from creating int variables on the stack? Because if somebody creates an int variable on the stack, and if they allow a reference or a pointer to it to outlive the stack frame that contains the variable, then their program could crash.
Seriously.
That problem is older than C++. That problem has existed since the very first edition of the C programming language. Every C and C++ programmer has to learn not to do that. Always have. Always will.
In some languages (e.g., Java, Python), No object of any kind can be allocated anywhere else except the garbage-collected heap. Variables can only hold references to objects, and dangling references are impossible. Programmers in those languages expect an assignment a=b to copy an object reference. That is, after the assignment, a and b both refer to the same object.
That's not the C++ way. C++ programmers expect that if some type T is publicly constructable, then they expect to be allowed to declare one wherever they want. And when they see a=b, they think of that assignment operator as copying a value. They expect that after the assignment, a and b still are two different objects that both have (in some sense) the same "value."
You will find more people saying positive things about your library if you design it to work in that same way.
I have objects in my C++ program that I pass to Lua as userdata, and I override the metatable for this userdata so that assignments to and from indices of the object (via __newindex and __index) result in a call to C, which converts the assignment to affect the C++ object or convert the C++ element to a Lua value (either another userdata or a base type like bool, number, string). The userdata is passed in as arguments to event-like Lua functions that are called from my C++ program.
luaL_newmetatable(L, "object");
lua_pushstring(L, "__index");
lua_pushvalue(L, -2); /* pushes the metatable */
lua_settable(L, -3); /* metatable.__index = metatable */
luaL_openlib(L, NULL, vallib_m, 0);
luaL_openlib(L, "internal", vallib_f, 0);
lua_pushstring(L, "__index");
lua_pushstring(L, "get");
lua_gettable(L, 2); /* get val.get */
lua_settable(L, 1); /* metatable.__index = val.get */
lua_pushstring(L, "__newindex");
lua_pushstring(L, "set");
lua_gettable(L, 2); /* get array.set */
lua_settable(L, 1); /* metatable.__newindex = val.set */
However, this doesn't allow me to assign the actual variable itself, only an index of the variable. There is no meta-event for directly overriding the assignment operator, so I am looking for a workaround.
In other words, I can do this: lua_userdata_object_passed_as_arg_to_event["is_it_true"]=true
and it assigns the Lua boolean to my internal C++ object, but if I do this:
lua_userdata_object_passed_as_arg_to_event = new_object()
it will change what the Lua variable references, but it won't do anything to the core object as I understand it.
One workaround I've considered is some hack requiring developers to do lua_userdata_object_passed_as_arg_to_event["__self"] = new_object()
if they want to change the object itself, but this is undesirable.
So I've found some unique solutions to overriding the assignment operator by using global variables and overriding the global metatable assignment operators, but I am looking to see if anyone can help me expound this solution. See http://lua-users.org/lists/lua-l/2012-01/msg00413.html and https://www.lua.org/pil/14.2.html.
In particular, my variables are function arguments, not globals, so how can I convert to globals through the C API so that any assignments will be captured by a custom C function that will take action if the assignment is happening to a global userdata?
By the way, my userdata is a pointer to an object to avoid duplicating large objects, if that matters.
Lua and C/C++ are different languages with different needs. In C/C++, a variable always references a specific object. You can change the contents of this object, but you can never make a variable deal with a different object. If you do a = b;, you are copying the value of b into a. You can never change what object a is talking about.
In Lua, variables do not permanently reference anything. Thus, there is a distinction between the object the variable currently holds and the value of that object.
Your kludge via global variables functions, but local variables are things that don't really exist. They're locations on the Lua stack; you can't override Lua's default behavior with regard to them.
The best way to handle this is to accept the difference between C and Lua. Write code in Lua the way you would write code in Lua. Don't try to make Lua code work like C; that way is folly. They are different languages, and you should embrace their differences, not work against them.
If you want to give Lua the ability to do the equivalent of a = b, then you should create a function in your type called assign or something that will allow you to assign to the object's value. Just like if you want to copy all of the table elements from one table to another, you have to write a function to do that.
In C++, say I have a class that creates a binary tree like structure and I use it something like this:
CTreeRoot* root = new CTreeRoot(/* whatever */);
CNode* leftNode = root->getLeftNode();
CNode* rightNode = root->getRightNOde();
leftNode->doSomething();
rightNode->doSomething();
// etc
And assume that the left and right nodes have their own left and right nodes (hence, a binary tree). Now I want to expose this to Lua (not using luabind) so I can do the same kind of thing:
local root = treeroot.new(/* whatever */)
local left = root:getLeftNode()
local right = root:getRightNode()
left:doSomething();
right:doSomething();
I've gotten most of it to work. However, for the getLeftNode() and getRightNode() methods, I'm pretty sure I'm doing it "wrong". Here is how I'm implementing getLeftNode() in C++ for example:
int MyLua::TreeRootGetLeftNode(luaState* L)
{
CTreeRoot* root = (CTreeRoot*)luaL_checkudata(L, 1, "MyLua.treeroot");
CNode* leftNode = root->getLeftNode();
if (leftNode != NULL)
{
int size = sizeof(CNode);
// create a new copy of the CNode object inplace of the memory Lua
// allocated for us
new ((CNode*)lua_newuserdata(L,size)) CNode((const CNode&)*leftNode);
lua_setmetatable(L, "MyLua.treenode");
}
else
{
lua_pushnil(L);
}
return 1;
}
I cast the userdata back to a CTreeRoot object, call getLeftNode(), make sure it exists and then (here's the "wrong part") I create another userdata data object with a copy constructor copying the object I want to return.
Is this "standard practice" for this type of scenario? It seems like you would want to avoid creating another copy of the object since what you really want is just a reference to an already existing object.
It sounds like this would be the perfect place for lightuserdata since I don't want to create a new object, I would be happy returning the object that already exists. The problem, though, is that lightuserdata has no meta table so the objects would be useless to me once I got them back. Basically, I want to do something like:
int MyLua::TreeRootGetLeftNode(luaState* L)
{
CTreeRoot* root = (CTreeRoot*)luaL_checkudata(L, 1, "MyLua.treeroot");
CNode* leftNode = root->getLeftNode();
if (leftNode != NULL)
{
// "WRONG" CODE BUT SHOWS WHAT I WISH I COULD DO
lua_pushlightuserdata(L, (void*)leftNode);
lua_setmetatable(L, "MyLua.treenode");
}
else
{
lua_pushnil(L);
}
return 1;
}
Can someone please tell me how I can have my MyLua::TreeRootGetLeftNode method return to Lua a copy of the object that already exists in such a way that I can use that object as an 'object' in Lua?
There are two levels of memory optimization that can be performed here. In your first functional-but-inefficient solution, when the user calls getLeftNode(), it has to create a copy of the CNode in order to store it in the Lua userdata. Furthermore, each time the user calls getLeftNode() repeatedly on the same tree, it will keep creating new userdata to represent the CNode even though it has have been created previously.
In the first level of optimization, you can memoize this call so that each time the user requests for the same subtree, you can simply return the userdata that was originally created instead of copying and constructing another userdata to represent the same thing. There are 3 approaches to this though, depending on whether you want to modify the Lua interface, alter the C++ implementation, or just bite the bullet.
The MyLua.treenode userdata currently contains the actual data of a CNode object. This is unfortunate, however, because that means whenever you create a CNode object, you have to use placement new to store it into the memory allocated by Lua immediately upon creation. What is probably better is to simply store a pointer instead (CNode*) in the userdata for MyLua.treenode. This does require you to modify the Lua interface for MyLua.treenode so that it will now consider its data as a pointer to a CNode object.
If you would rather store the CNode data in the MyLua.treenode userdata directly, then you will have to make sure that when you create your CTreeRoot, it would use placement new to construct CNodes from the memory allocated by Lua each time (or perhaps you can use the allocator pattern used in the C++ standard library?). This is less elegant however as your CNode implementation now depends on the Lua runtime, and I don't recommend this.
If neither of the above solutions are appropriate, then you'll just have to make a copy whenever you return a subnode, although you can still improve the efficiency for repeated calls on the same node by keeping track of whether you have created the same userdata before (using a Lua table, for example).
In the second level of optimization, you can further save memory by making your copied subtree to be a weak reference to a fragment of the original tree. This way, whenever you copy a subtree, you are merely creating a pointer to a part of the original tree. Or, you can use a strong reference if you want your subtree to persist even after the original tree is destroyed, but then you'll have to go into the gory details of reference-counting. In either case, this optimization is purely on the C++ level and is not related to the Lua interface, and judging from your code I assume you are already using weak references (CNode*).
Note: Light userdata are probably best avoided except for use in the internal implementation. The reason being that light userdata are essentially equivalent to C pointers and thus can point to just about anything. If you expose light userdata to Lua code, you will have no idea where the light userdata may have come from or what type of data it contains, using it poses a security risk (as well as the possibility to segfault your program). An appropriate way to use light user data would be to use it as an index of a Lua lookup table stored in the Lua registry, which can be used to implement the memoization that was mentioned earlier.
I successfully integrated Lua into my C my application, giving scripting access to users. Now my problem: how can I prevent double free or bad access violations?
I already implement Init and Free functions for each struct, for example:
structaInit
structaFree
I also track every time a struct pointer is linked to another, incrementing the reference count which is present in all of my structs.
However the users could always do something like this in Lua:
a = structaInit();
b = structbInit();
structbSetA( b, a ); -- This add ++a.reference
a.reference = 0;
a = structaFree( a ); -- If a->reference == 0 then I free
-- Then struct b->a is still a valid pointer but that have been free.
Is there anyway I can prevent this from happening?
The problem has to do with ownership. Let's take your Lua scripts:
a = structaInit();
b = structbInit();
This creates C objects that Lua now owns. Lua will decide when to free the memory for these objects.
So what about this?
structbSetA( b, a ); -- This add ++a.reference
First of all, structbSetA should be a member of b, via a metatable (so it becomes b:setA(a)). But more importantly, who owns a?
Lua does. Because it must own A; Lua cannot fully relinquish ownership of an object that is still in Lua memory. This means that your internal reference count is ultimately meaningless; the only one that matters is Lua's.
If you intend to store the value of a within b, such that b can reference a so long as b is still alive, then you need to create this relationship through Lua methods. You can't just stick the C pointer to a in b and expect everything to go well.
The easiest way to do that is to, for each object you create, make a table in the Lua registry that stores the Lua object for any references it has. When an object is destroyed, you go into the Lua registry and remove this table from it, thus leading to the destruction of any referenced Lua objects. Obviously, you will need to change this value when it is modified by later calls to structbSetA.
Also, why are you exposing this to Lua:
a.reference = 0;
That's a terrible API. Lua code should never have to deal with a reference count. You should also never expose an explicit "free" method to Lua, unless you need Lua to release some resource immediately after it is finished using it. And that should only be necessary for OS-type resources like FILE handles and such. For a regular object, let the garbage collector do its job.
Do not expose C-isms to Lua code. Let Lua code look like Lua code.
In this case it comes down to programming practice you do not really want to prevent it, you actually want to let it error out because that way the person using your scripts knows that they are doing something wrong (same way in obj-c it crashes when you over-release). If you don't want this to happen you might have to keep track of all active pointers on the heap to your structs in a linked list or some structure but I don't think it's worth it.
I have C++ objects and I have Lua objects/tables. (Also have SWIG C++ bindings.)
What I need to be able to do is associate the two objects so that if I do say
CObject* o1 = getObject();
o1->Update();
it will do the equivalent Lua:
myluatable1.Update();
So far I can imagine that CObject::Update would have the following code:
void CObject::Update(){
// Acquire table.
// ???
// Do the following operations on the table.
lua_getfield(L, -1, "Update");
lua_pcall(L, 0, 0, 0);
}
How would I store/set the Lua table to be used, and what would go in the // ??? above to make the Update call work?
I cant believe nobody noticed this!
http://www.lua.org/pil/27.3.2.html
A section of the Lua API for storing references to lua objects and tables and returning references for the purposes of being stored in C structures!!
I am curious for the reasons of this "reverse SWIG"...
The objects in Lua live within the Lua contexts, so at a minimum you would need to store "L" inside your object.
The issue of passing the "table pointer" is a bit more delicate - even though Lua allows to retrieve the pointer off the Lua stack (using lua_topointer()), there is no way of putting that back. Understandingly - because otherwise one would also need to check that the pointer does point to a valid object, etc, etc.
What you might do however, is to store the references to the tables in the global table, the index being the lightuserdata being the pointer to your object. Then by having the Lua state, and the name of the global array, you can retrieve the reference to the table and push it onto the Lua stack for that context.
This is sketchy, and I haven't even touched the question of garbage-collecting with this construct.
But in any case this is not going to be the speed racer performance-wise, and looks like a lot of boilerplate C++ code to me. I'd try to reconsider the approach and push some of what you want to do into the Lua domain.
p.s. looks like it is the third question which seems almost a dupe of the previous two ones, here and here are the previous ones. If those were not answered fully, would have been better to edit them/add the bounty to accumulate the answers.