I successfully integrated Lua into my C my application, giving scripting access to users. Now my problem: how can I prevent double free or bad access violations?
I already implement Init and Free functions for each struct, for example:
structaInit
structaFree
I also track every time a struct pointer is linked to another, incrementing the reference count which is present in all of my structs.
However the users could always do something like this in Lua:
a = structaInit();
b = structbInit();
structbSetA( b, a ); -- This add ++a.reference
a.reference = 0;
a = structaFree( a ); -- If a->reference == 0 then I free
-- Then struct b->a is still a valid pointer but that have been free.
Is there anyway I can prevent this from happening?
The problem has to do with ownership. Let's take your Lua scripts:
a = structaInit();
b = structbInit();
This creates C objects that Lua now owns. Lua will decide when to free the memory for these objects.
So what about this?
structbSetA( b, a ); -- This add ++a.reference
First of all, structbSetA should be a member of b, via a metatable (so it becomes b:setA(a)). But more importantly, who owns a?
Lua does. Because it must own A; Lua cannot fully relinquish ownership of an object that is still in Lua memory. This means that your internal reference count is ultimately meaningless; the only one that matters is Lua's.
If you intend to store the value of a within b, such that b can reference a so long as b is still alive, then you need to create this relationship through Lua methods. You can't just stick the C pointer to a in b and expect everything to go well.
The easiest way to do that is to, for each object you create, make a table in the Lua registry that stores the Lua object for any references it has. When an object is destroyed, you go into the Lua registry and remove this table from it, thus leading to the destruction of any referenced Lua objects. Obviously, you will need to change this value when it is modified by later calls to structbSetA.
Also, why are you exposing this to Lua:
a.reference = 0;
That's a terrible API. Lua code should never have to deal with a reference count. You should also never expose an explicit "free" method to Lua, unless you need Lua to release some resource immediately after it is finished using it. And that should only be necessary for OS-type resources like FILE handles and such. For a regular object, let the garbage collector do its job.
Do not expose C-isms to Lua code. Let Lua code look like Lua code.
In this case it comes down to programming practice you do not really want to prevent it, you actually want to let it error out because that way the person using your scripts knows that they are doing something wrong (same way in obj-c it crashes when you over-release). If you don't want this to happen you might have to keep track of all active pointers on the heap to your structs in a linked list or some structure but I don't think it's worth it.
Related
Is it possible that luabind checks, if a member function call to an exported class (object) is for a valid object?
lets assume that i have a Class called Actor exposed using luabind to lua. Im calling a lua function from C++ with an actor object as parameter. Now before the function finishes, a script write would put the actor object in a global lua reference to be accessed later.
Later on, the actor object is deleted from the C++ site, another function is called which tries to access the invalidated actor object (any method from it) - and obviously since it has been deleted, it results in a crash (access violation)
sample:
local myObjRef = nil
function doSomethingWithActor(actor)
-- save, still valid object
actor:Say("hello")
myObjRef = actor
end
function calledAfterActorWasDeleted()
--- will crash if the c++ object has been deleted meanwhile, works fine if it still exists
myObjRef:Say("Crash...")
end
A NIL check doesnt help here, is this something that can be checked on luabinds site? The functions are executed using lua_pcall(....) and the stacktrace shows the error at luabinds call.hpp results = maybe_yield(L, lua_gettop(L) - arguments, (Policies*)0);
If not, is there another solution how to make sure somebody who writes a script cannot create these issues?
Now before the function finishes, a script write would put the actor object in a global lua reference to be accessed later.
That right there is where your problem is coming from. If you want Lua code to own the object (that is, preserve the existence of this object), then you need to use Luabind mechanics to tell Luabind that you want to do that. Otherwise, if you pass a pointer to some Lua function, Luabind will assume that the function will not be trying to gain ownership of it.
If you want ownership to be shared between Lua and Luabind, then you should wrap your objects in a boost::shared_ptr, and use Luabind's smart pointer mechanisms to do this.
You could also simply segregate your scripts better. If you have some script that operates on a particular actor, then that script and any functions it contains should be destroyed (ie: lose all references to it) along with the object. This requires proper coding discipline on the C++ side. It will also require that you use Lua environments to properly encapsulate each instance of a script, so that they can't sneak things out via globals. Lastly, you will need to have C++ maintain total control over when scripts are called and when they aren't.
Otherwise, ownership is something your scripters are simply going to have to know about and be careful of. They can't treat C++ parameters like any old Lua value.
If exercising disciplined programming practice is not possible or practical for you, then you will simply have to not pass Lua the actual C++ object. Instead, you need to pass Lua some proxy object, which is a reference to the original. boost::weak_ptr is a good example of such an object (though you wouldn't pass it exactly to Lua). The proxy would forward calls to the actual object. If the object has been deleted, the proxy would detect this and fail or do nothing or whatever.
I solved my issue the following way:
When im about to delete an object, i iterate through all lua functions from C++ (i have them in a list, they are bound to specific actor objects each). Then i inspect each upvalue (global/local vars accessable to a function) - then i compare the userdata pointer with my object im about to delete - if they match (and their classes) and NIL the upvalue. Optionally, i could just remove that offending function because it would not work well anymore anyway.
So the next the time the function is called, im just getting a soft lua error "trying to access xxx a nil value..." - no more access violations.
I know people would say "dont use lua_getupvalue/lua_setupvalue - they are only for debugging!" - but there is actually no documented or spoken side effect - and in my case its perfectly safe and works well - also there isnt the issue with left over proxy objects i could not delete.
I'm wondering if it's possible to access all of the userdata "tables" (is it called userdata tables?) and then delete them from Lua because this is my problem:
a = Object(5, 5)
a:Delete()
a:SetPosition(3,3)
As you can see first I create an object and save an pointer to a c++ class called Object which is allocated using "new" in my map class. Then I delete the object which means I delete the allocated memory for the pointer in my map class. And last I call SetPosition, if the memory still is allocated for the c++ Object class everything will run fun. But if it is deletes (as it is in this case because we called Delete() before the SetPosition(...) call) my program will crash. So what I'm wondering is following:
Is it possible to set the varaible 'a' in lua to nil by calling Delete ? I know I could do something like 'a = a:Delete()' if Delete return nil but if I forget to do the 'a =' part it fail. Also I'm wondering if it's possible to delete the userdata and check if it doesn't exist when I call SetPositon(), if it doesn't I will just return.
Also, the base code is from: http://lua-users.org/wiki/SimpleCppBinding
First, let me answer your question:
Is it possible to set the varaible 'a' in lua to nil by calling Delete ?
No. There is no means to do what you're saying. And there's a reason for that: what you're trying to do is terrible code.
Lua is a garbage collected system. Lua should not be expected to delete objects. If Lua gets a pointer to some external object, then either your code owns it or Lua owns it.
If your code owns it, Lua should not be deleting it. Lua can use it for some period of time. But it is up to your Lua code to use it for exactly and only that period of time. Once its lifetime has expired, Lua shouldn't be talking to it anymore.
This is no different from dealing with pointers to objects in C and C++. If your function is handed a naked pointer (ie: not a smart pointer), your code needs to know how long it can reasonably expect to talk to that object. Can it store it? How long can it store a pointer to that object? When will that object die, and who's responsible for destroying it?
If you pass an object to Lua such that Lua now owns the object, Lua shouldn't be explicitly deleting it either. Lua is a garbage collected system; you should attach a __gc metamethod to your type, so that Lua's garbage collector will call your code when the userdata is collected. That way, you can call destructors, free memory, etc.
When you give Lua something that now belongs to Lua, it should look like a regular Lua object. You don't call Delete methods for tables and strings you create in Lua; you let the garbage collector do its job. It is your job, as the one writing the C++-to-Lua interface, to ensure that the objects you give to Lua behave the way that Lua wants them to.
In cases where you need to do significant resource management, where you want Lua to release resources as quickly as possible (such as for file handles, etc), then you need to store a pointer to your C++ object inside of the non-light userdata. That's the pointer you NULL out. All of your interface functions on that object will check the pointer to see if it's NULL and simply do nothing or raise an error.
Lua's file handles (returned by io.open) are a good example of this. If you try to call functions on them, Lua throws a Lua error.
In the Delete method, set the metatable of the received object to nil and you'll get an error message if you later call a method on that object.
I'd rather advice using SWIG or LuaBind instead, they've already taken care of such pitfalls for you.
I am trying to embed lua in an existing C++ application and have made a standard procedure for it by inheriting from a class that does the work.
The serious problem I see is that if the exposed object gets deallocated or deleted in the C++ environment then a call from Lua will cause crashes. If the memory is being deleted by the program using 'delete' then I can maybe write a wrapper on delete to take care of deallocation in Lua as well, but if the memory was allocated by C++ and deallocated when the appropriate variable is out of scope I don't see a way on how to find that out and then take appropriate actions in the lua space, anybody has any ideas on this?
Thanks.
In general, virtually every Lua wrapper has some way to decide who owns what memory. That is, whether an object is owned by (and therefore will be deleted by) Lua or by your application.
If you have given Lua a pointer to an object that C++ owns, then you must find a way to ensure that Lua does not use this pointer past the point where C++ deletes it. There are several ways to avoid this. One way is to transfer ownership to Lua.
Another way is to use a boost/std::shared_ptr, which allows you to share ownership between C++ and Lua. If you're manually doing this, then you are creating some non-light userdata which is the size of a shared_ptr in Lua. You attach a cleanup metamethod to it that will destroy the shared_ptr, and you use placement-new to construct the shared_ptr on the Lua userdata. Luabind actually has this built-in as a feature: if you pass a shared_ptr to Lua, then they both share ownership of the memory.
You could also use a boost/std::weak_ptr. This is an object that you query to get a shared_ptr. The idea is that you're not supposed to keep the pointer around; you query it temporarily as needed, but you only store the weak_ptr permanently. If the object has lost all of its shared_ptr references, then querying the weak_ptr will return a null pointer.
You will have to use an RAII wrapper that can bind to the Lua instance using the registry and expose the values to Lua using a table- you can remove an internal pointer from it when you're done.
template<typename T> class LuaExposedValue {
T t;
lua_State* ls;
public:
LuaExposedValue(lua_State* L) {
// set registry[&t] = { &t }
ls = L;
}
~LuaExposedValue() {
// remove &t from the table
}
}
Alternatively, just ban Lua from accessing it after the variable is gone and let the scripter worry about it.
Finally, you could just allocate everything that Lua can access using the Lua GC.
Disclaimer: I wrote the library I'm about to recommend
You might want to try using this LuaWrapper Library that sounds like it'll handle what you're trying to do. It's not even really a library, it's just a single header file.
You can use luaW_push<MyType>(L, myObj); to push your objects into Lua. Lua will not own the objects you create from C++ unless you run luaW_hold<MyType> on them. In other words, unless you tell Lua to, it will not garbage collect your object.
Conversely, you can use MyType.new() in your Lua code to create an object, which Lua does own. It will be garbage collected as you would expect. If you want to pass ownership to C++ you can call luaW_release<MyType> on your object.
There's also functions like luaW_to<MyType> and luaW_check<MyType> and to a limited degree it correctly supports inheritance from base types (though at the moment it only allows for single inheritance). I find that this greatly simplifies my own attempts at using C++ and Lua together because it make managing pointer ownership very straightforward.
I'm having problems with luabind. I define a std::map to allocate objects created in lua. I publish this map as a global object in lua in this way:
luabind::globals(L)["g_SceneManager2D"] = this;
After that, this object is used into a function into lua, where many objects are created and inserted into this map. The problem comes when lua function ends and luabind returns the control to C++ side program, because automatically all contents of the map are lost.
I was looking for the error. I keep the lua context alive, so this object must exists.
Could you helpme??
Many thanks :)
I suggest use a shared_ptr<>(this) rather than raw this. boost::shared_from_this might help. Make sure your class is registered using Luabind too, and that the class_ is specified as held by a shared_ptr.
Another fun idea might be to make your Lua function just generate the "map" as a Lua table, return it, and you can iterate over it in C++ to build your std::map.
If I understand your problem correctly, it seems you are creating objects in Lua, which you then insert into the map (either through Lua or C++) and subsequently lose. Without some more code, it's hard to tell exactly what the problem is. However, I would first look to make sure that those objects are indeed being created (double check it) and then I would check to see that Lua isn't garbage collecting them. If Lua is indeed garbage collecting those objects, then you won't see them on the C++ side because they're, well, gone.
If it helps, I'm finishing up a project which does something similar. I had to create and retrieve C++ objects from Lua, but instead of creating the objects in Lua, I just called C++ functions to do it for me, sending any necessary data in the Lua call (bound by Luabind). Those (C++) functions indexed the objects by IDs into hash tables and the IDs were returned to Lua in case it needed to retrieve the object script-side for operations. This setup makes it easier (and safer) to handle memory stuff correctly and prevents Lua from garbage collecting your objects.
I'm lead dev for Bitfighter, a game primarily written in C++, but using Lua to script robot players. We're using Lunar (a variant of Luna) to glue the bits together.
I'm now wrestling with how our Lua scripts can know that an object they have a reference to has been deleted by the C++ code.
Here is some sample robot code (in Lua):
if needTarget then -- needTarget => global(?) boolean
ship = findClosest(findItems(ShipType)) -- ship => global lightUserData obj
end
if ship ~= nil then
bot:setAngleToPoint(ship:getLoc())
bot:fire()
end
Notice that ship is only set when needTarget is true, otherwise the value from a previous iteration is used. It is quite possible (likely, even, if the bot has been doing it's job :-) that the ship will have been killed (and its object deleted by C++) since the variable was last set. If so, C++ will have a fit when we call ship:getLoc(), and will usually crash.
So the question is how to most elegantly handle the situation and limit the damage if (when) a programmer makes a mistake.
I have some ideas. First, we could create some sort of Lua function that the C++ code can call when a ship or other item dies:
function itemDied(deaditem)
if deaditem == ship then
ship = nil
needTarget = true
end
end
Second, we could implement some sort of reference counting smart pointer to "magically" fix the problem. But I would have no idea where to start with this.
Third, we can have some sort of deadness detector (not sure how that would work) that bots could call like so:
if !isAlive(ship) then
needTarget = true
ship = nil -- superfluous, but here for clarity in this example
end
if needTarget then -- needTarget => global(?) boolean
ship = findClosest(findItems(ShipType)) -- ship => global lightUserData obj
end
<...as before...>
Fourth, I could retain only the ID of the ship, rather than a reference, and use that to acquire the ship object each cycle, like this:
local ship = getShip(shipID) -- shipID => global ID
if ship == nil then
needTarget = true
end
if needTarget then -- needTarget => global(?) boolean
ship = findClosest(findItems(ShipType)) -- ship => global lightUserData obj
shipID = ship:getID()
end
<...as before...>
My ideal situation would also throw errors intelligently. If I ran the getLoc() method on a dead ship, I'd like to trigger error handling code to either give the bot a chance to recover, or at least allow the system to kill the robot and log the problem, hopefully cuing me to be more careful in how I code my bot.
Those are my ideas. I'm leaning towards #1, but it feels clunky (and might involve lots of back and forth because we've got lots of short-lifecycle objects like bullets to contend with, most of which we won't be tracking). It might be easy to forget to implement the itemDied() function. #2 is appealing, because I like magic, but have no idea how it would work. #3 & #4 are very easy to understand, and I could limit my deadness detection only to the few objects that are interesting over the span of several game cycles (most likely a single ship).
This has to be a common problem. What do you think of these ideas, and are there any better ones out there?
Thanks!
Here's my current best solution:
In C++, my ship object is called Ship, whose lifecycle is controlled by C++. For each Ship, I create a proxy object, called a LuaShip, which contains a pointer to the Ship, and Ship contains a pointer to the LuaShip. In the Ship's destructor, I set the LuaShip's Ship pointer to NULL, which I use as an indicator that the ship has been destroyed.
My Lua code only has a reference to the LuaShip, and so (theoretically, at least, as this part is still not working properly) Lua will control the lifecycle of the LuaShip once the corresponding Ship object is gone. So Lua will always have a valid handle, even after the Ship object is gone, and I can write proxy methods for the Ship methods that check for Ship being NULL.
So now my task is to better understand how Luna/Lunar manages the lifecycle of pointers, and make sure that my LuaShips do not get deleted when their partner Ships get deleted if there is still some Lua code pointing at them. That should be very doable.
Actually, it turned out not to be doable (at least not by me). What did seem to work was to decouple the Ship and the LuaShip objects a little. Now, when the Lua script requests a LuaShip object, I create a new one and hand it off to Lua, and let Lua delete it when it's done with it. The LuaShip uses a smart pointer to refer to the Ship, so when the Ship dies, that pointer gets set to NULL, which the LuaShip object can detect.
It is up to the Lua coder to check that the Ship is still valid before using it. If they do not, I can trap the sitation and throw out an stern error message, rather than having the whole game crash (as was happening before).
Now Lua has total control over the lifecyle of the LuaShip, C++ can delete Ships without causing problems, and everything seems to work smoothly. The only drawback is that I'm potentially creating a lot of LuaShip objects, but it's really not that bad.
If you are interested in this topic, please see the mailing list thread I posted about a related concept, that ends in some suggestions for refining the above:
http://lua-users.org/lists/lua-l/2009-07/msg00076.html
I don't think you have a probelm on your Lua side, and you should not be solving it there.
Your C++ code is deleting objects that are still being referenced. No matter how they're referenced, that's bad.
The simple solution may be to let Lunar clean up all your objects. It already knows which objects must be kept alive because the script is using them, and it seems feasible to let it also do GC for random C++ objects (assuming smart pointers on the C++ side, of course - each smart pointer adds to Lunars reference count)
Our company went with solution number four, and it worked well for us. I recommend it. However, in the interests of completeness:
Number 1 is solid. Let the ship's destructor invoke some Lunar code (or mark that it should be invoked, at any rate), and then complain if you can't find it. Doing things this way means that you'll have to be incredibly careful, and maybe hack the Lua runtime a bit, if you ever want to run the game engine and the robots in separate threads.
Number 2 isn't as hard as you think: write or borrow a reference-counting pointer on the C++ side, and if your Lua/C++ glue is accustomed to dealing with C++ pointers it'll probably work without further intervention, unless you're generating bindings by inspecting symbol tables at runtime or something. The trouble is, it'll force a pretty profound change in your design; if you're using reference-counted pointers to refer to ships, you have to use them everywhere - the risks inherent in referring to ships with a mixture of bare pointers and smart ones should be obvious. So I wouldn't go that route, not as late in the project as you seem to be.
Number 3 is tricky. You need a way to determine whether a given ship object is alive or dead even after the memory representing it has been freed. All the solutions I can think of for that problem basically devolve into number 4: you can let dead ships leave behind some kind of token that's copied into the Lua object and can be used to detect deadness (you'd keep dead objects in a std::set or something similar), but then why not just refer to ships by their tokens?
In general, you can't detect whether a particular C++ pointer points to an object that's been deleted, so there's no easy magical way to solve your problem. Trapping the error of calling ship:getLoc() on a deleted ship is possible only if you take special action in the destructor. There's no perfect solution to this problem, so good luck.
This is an old question, but the right solution, IMO, is to have lua_newuserdata() create a shared_ptr or weak_ptr via either boost::shared_ptr/boost::weak_ptr or C++11's std::shared_ptr/std::weak_ptr. From there, you create a reference whenever you need it, or fail if the weak_ptr is unable to obtain lock() a shared_ptr. For example (using Boost's shared_ptr in this example since this is an old question where you probably do not have have C++11 support yet, though for new projects where possible I'd recommend C++11's shared_ptr):
using MyObjectPtr = boost::shared_ptr<MyObject>;
using MyObjectWeakPtr = boost::weak_ptr<MyObject>;
auto mySharedPtr = boost::make_shared<MyObject>();
auto userdata = static_cast<MyObjectWeakPtr*>(lua_newuserdata(L, sizeof(MyObjectWeakPtr)));
new(userdata) MyObjectWeakPtr(mySharedPtr);
And then when you need to get a C++ object:
auto weakObj = *static_cast<MyObjectWeakPtr*>(
luaL_checkudata(L, 1, "MyObject.Metatable"));
luaL_argcheck(L, weakObj != nullptr, 1, "'MyObjectWeakPtr' expected");
// If you're using a weak_ptr, this is required!!!! If your userdata is a
// shared_ptr, you can just act on the shared_ptr after luaL_argcheck()
if (auto obj = weakObj.lock()) {
// You have a valid shared_ptr, the C++ object is alive and you can
// dereference like a normal shared_ptr.
} else {
// The C++ object went away, you can safely garbage collect userdata
}
It's critical that you don't forget to deallocate the weak_ptr in your lua __gc metamethod:
static int
myobject_lua__gc(lua_State* L) {
auto weakObj = *static_cast<MyObjectWeakPtr*>(
luaL_checkudata(L, 1, "MyObject.Metatable"));
luaL_argcheck(L, weakObj != nullptr, 1, "'MyObjectWeakPtr' expected");
weakObj.~MyObjectWeakPtr();
}
Don't forget to make use of macros or template metaprogramming to avoid much of the code duplication re: static_cast<>, luaL_argcheck(), etc.
Use shared_ptr when you need to keep the C++ object alive for as long as the lua object also exists. Use weak_ptr when C++ may reap the object and it's okay for it to disappear out from under lua's feet. ALWAYS use either shared_ptr or weak_ptr when the life of an object is not known and needs to be managed automatically by refcount.
Tip: have your C++ class inherit from boost::enable_shared_from_this or std::enable_shared_from_this because it enables use of shared_from_this().
I agree with MSalters, I really don't think you should be freeing the memory from the C++ side. Lua userdata supports the ___gc metamethod to give you a chance to clean things up. If the gc is not agressive enough you can tweak it a bit, or run it manually with a small step size, more often. The lua gc is not deterministic, so if you need to have resources released then you will need to have a function that you can call to release those resources (which will also be called by __gc, with appropriate checks).
You might also want to look into using weak tables for your ship references so that you don't have to assign EVERY reference to nil to get it freed. Have one strong reference (say, in a list of all active ships) then all the others are weak references. When a ship is destroyed, set a flag on the ship that marks it as such, then set the reference to nil in the active ships table. Then, when the other ship wants to interact your logic is the same except you check for:
if ship==nil or ship.destroyed then
ship = findClosest(findItems(ShipType))
end