I'm currently wrapping a lua class in c++ and it's going pretty well so far. But I'm wondering if there some way to break a lua script from running(could be in the middle of the script) for another thread. So if I run my lua script on thread 1, can I break it from thread 2? Would lua_close(...) do that?
Thanks.
If this is an expected occurrence and most of the Lua script's time is spent inside Lua functions (i.e., not lengthy, blocking C calls), you could install a debug hook that checks for your "break flag" every N instructions and aborts the script. See the "debug library" section of Programming in Lua.
Then there must be some way to force the script to halt?
No, there doesn't.
Lua provides exactly one thread safety guarantee: two separate lua_States (separate being defined as different objects returned by lua_open) can be called freely from two different CPU threads. Thread A can call lua_State X, and thread B can call lua_State Y.
That is the only guarantee Lua gives you. Lua is inherently single-threaded; it only provides this guarantee because all of the global information for Lua is contained in a self-contained object: lua_State.
The rules of Lua require that only one thread talk to a lua_State at any one time. Therefore, it is impossible for Lua code to be executing and then be halted or paused by external C++ code. That would require breaking the rule: some other thread would have to call functions on this thread's lua_State.
If you want to abort a script in-progress, you will have to use debug hooks. This will dramatically slow down the performance of your script (if that matters to you). You will also need thread synchronization primitives, so that thread B can set a flag that thread A will read and halt on.
As for how you "abort" the script, that... is impossible in the most general case. And here's why.
The lua_State represents a Lua thread of execution. It's got a stack, a pointer to instructions to interpret, and some code it's currently executing. To "abort" this is not a concept that Lua has. What Lua does have is the concept of erroring. Calling lua_error from within C code being called by Lua will cause it to longjmp out of that function. This will also unwind the Lua thread, and it will return an error to the most recent function that handles errors.
Note: if you use lua_error in C++ code, you need to either have compiled Lua as C++ (thus causing errors to work via exception handling rather than setjmp/longjmp) or you need to make sure that every object with a destructor is off the stack when you call it. Calling longjmp in C++ code is not advisable without great care.
So if you issue lua_pcall to call some script, and in your debug hook, you issue lua_error, then you will get an error back in your lua_pcall statement. Lua's stack will be unwound, objects destroyed, etc. That's good.
The reason why this does not work in the general case (besides the fact that I have no idea how Lua would react to a debug hook calling lua_error) is quite simple: Lua scripts can execute pcall too. The error will go to the nearest pcall statement, whether it's in Lua or in C.
This Lua script will confound this method:
local function dostuff()
while true do end
end
while true do
pcall(dostuff)
end
So issuing lua_error does not guarantee that you'll "abort the script." Think of it like throwing exceptions; anyone could catch them, so you can't ensure that they are only caught in one place.
In general, you should never want to abort a Lua scripts execution in progress. This is not something you should be wanting to do.
Related
I wrote a shared library in C++ on Linux, which contains a function f(). This library is used from multiple programs, calling the function.
Now I would like to conduct some debugging with that function, while calling it from program A. When calling it from any other program, it should fail, as long as I debug the function. After I do not have complete control over when the other programs are run, I would like to add an exception, which stops every program except program A when calling function f during Debug sessions.
How could I solve that?
The only way I can think of is by checking the information of the currently running process. You can get the pid by calling the getpid() function. All the information on all processes on a linux system can be found in the /proc/<pid> directory. When calling function f, you can check this information and decide whether to throw an exception or not.
There isn't a way that can't be gotten around. If another process has PTRACE on yours, it can make anything at all happen in your process, so any check you perform to try to make the function not work can be disabled.
I'm designing the threading architecture for my game engine, and I have reached a point where I am stumped.
The engine is partially inspired by Grimrock's engine, where they put as much as they could into LuaJIT, with some things, including low level systems, written in C++.
This seemed like a good plan, given that LuaJIT is easy to use, and I can continue to add API functions in C++ and expand it further. Faster iteration is nice, the ability to have a custom IDE attached to the game and edit the code while it runs is an interesting option, and serializing from Lua is also easy.
But I am stumped on how to go about adding threading. I know Lua has coroutines, but that is not true threading; it's basically to keep Lua from stalling as it waits for code that takes too long.
I originally had in mind to have the main thread running in Lua and calling C++ functions which are dispatched to the scheduler, but I can't find enough information on how Lua functions. I do know that when Lua calls a C++ function it runs outside of the state, so theoretically it may be possible.
I also don't know whether, if Lua makes such a call that is not supposed to return anything, it will hang on the function until it's done.
And I'm not sure whether the task scheduler runs in the main thread, or if it is simply all worker threads pulling data from a queue.
Basically meaning that, instead of everything running at once, it waits for the game state update before doing anything.
Does anyone have any ideas, or suggestions for threading?
In general, a single lua_State * is not thread safe. It's written in pure C and meant to go very fast. It's not safe to allow exceptions go through it either. There's no locks in there and no way for it to protect itself.
If you want to run multiple lua scripts simultaneously in separate threads, the most straightforward way is to use luaL_newstate() separately in each thread, initialize each of them, and load and run scripts in each of them. They can talk to the C++ safely as long as your callbacks use locks when necessary. At least, that's how I would try to do it.
There are various things you could do to speed it up, for instance, if you are loading copies of a single script in each of the threads, you could compile it to lua bytecode before you launch any of the threads, then put the buffer into shared memory, and have the scripts load the shared byte code without changing. That's most likely an unnecessary optimization though, depending on your application.
I'm working on a C++ application which uses a library written in C by another team. The writers of the library like to call exit() when errors happen, which ends the program immediately without calling the destructors of objects on the stack in the C++ application. The application sets up some system resources which don't automatically get reclaimed by the operating system after the process ends (shared memory regions, interprocess mutexes, etc), so this is a problem.
I have complete source code for both the app and the library, but the library is very well-established and has no unit tests, so changing it would be a big deal. Is there a way to "hook" the calls to exit() so I can implement graceful shutdown for my app?
One possibility I'm considering is making one big class which is the application - meaning all cleanup would happen either in its destructor or in the destructor of one of its members - then allocating one of these big objects on the heap in main(), setting a global pointer to point to it, and using atexit() to register a handler which simply deletes the object via the global pointer. Is that likely to work?
Is there a known good way to approach this problem?
In the very worst case, you can always write your own implementation of exit and link it rather than the system's own implementation. You can handle the errors there, and optionally call _exit(2) yourself.
Since you have the library source, it's even easier - just add a -Dexit=myExit flag when building it, and then provide an implementation of myExit.
install exit handler with atexit and implement the desired behavior
If you want to make the C library more usable from C++, you could perhaps run it in a separate process. Then make sure (with an exit handler or otherwise) that when it exits, your main application process notices and throws an exception to unwind its own stack. Perhaps in some cases it could handle the error in a non-fatal way.
Of course, moving the library use into another process might not be easy or particularly efficient. You'll have some work to do to wrap the interface, and to copy inputs and outputs via the IPC mechanism of your choice.
As a workaround to use the library from your main process, though, I think the one you describe should work. The risk is that you can't identify and isolate everything that needs cleaning up, or that someone in future modifies your application (or another component you use) on the assumption that the stack will get unwound normally.
You could modify the library source to call a runtime- or compile-time-configurable function instead of calling exit(). Then compile the library with exception-handling and implement the function in C++ to throw an exception. The trouble with that is that the library itself probably leaks resources on error, so you'd have to use that exception only to unwind the stack (and maybe do some error reporting). Don't catch it and continue even if the error could be non-fatal as far as your app is concerned.
If the call exit and not assert or abort, there are a few points to get control again:
When calling exit, the destructors for objects with static lifetime (essentially: globals and objects declared with static) are still executed. This means you could set up a (few) global "resource manager" object(s) and release the resources in their destructor(s).
As you already found, you can register hooks with atexit. This is not limited to one. You can register more.
If all else fails, because you have the source of the library, you can play some macro tricks to effectively replace the calls to exit with a function of your own that could, for example, throw an exception.
I'm integrating a 3rd party C++ package to a python application using SWIG. The package connects to a proprietary API over a network and receives updates. The overall flow is that python instantiates a C++ object, calls its functions to set it up and then waits for the updates.
I implemented a callback mechanism for the updates using SWIG's directors feature, and while testing from python, or from C++ functions called by python, it works well. Namely I'm able to inherit a C++ class in Python, call its virtual functions from C++ and see the python code take priority and execute.
The problem:
When I receive the updates from the network I get:
The thread 'Win32 Thread' (0x1f78) has exited with code 0 (0x0).
Unhandled exception at 0x1e0650cb in python.exe: 0xC0000005: Access violation writing location 0x0000000c.
This exception is thrown from within python27.dll while calling the callback function.
My suspicion is this: I've violated the GIL
AFAIU the updates come from a different thread and call python's code using that thread.
At this point I'm at a loss. Is SWIG's director feature limited only to flows initiated within python (i.e. from python managed threads)?
How do I circumvent this? How do I induce updates from C++ to python? Is it even possible using SWIG?
Should I use a completely different approach?
I'm open for any suggestions on this matter...
If your SWIG-wrapped C++ code invokes the callback routine in-thread, then there're probably no GIL problems - SWIG-generated code does not perform any GIL management that I've seen, which means that when Python code calls in to your C++ code, you retain the GIL throughout the call.
However, if your C++ code defers the callback to another thread, then you very likely have violated the GIL. This is simple enough to work around: Before you invoke the callback, call PyGILState_Ensure(), and when the callback completes, invoke PyGILState_Release. Refer to http://docs.python.org/c-api/init.html, the section "Non-Python created threads". (If you're using C++ exception handling here, you might need to take extra care to ensure that you can release the GIL.)
If you haven't yet looked at the stack trace, it's worth verifying that that NULL pointer deref isn't something silly happening in your code. (You can attach to the Python process running your code with VS/GDB/WinDBG; the Python execution will remain inscrutable, but you can trace your C++ code this way.)
I have an application that allows users to write their own code in a language of our own making that's somewhat like C++. We're getting problems, however, where sometimes our users will accidentally write an infinite loop into their script. Once the script gets into the infinite loop, the only way they can get out is to shut the application down and restart, potentially losing their work. I'd like to add some means where the user, when he realizes that his code is in an infinite loop, can hit a special key, like F10 or something, and the code will break out of the loop. But I'd like to do it without implementing a ton of checks within the script runtime. Optimally, I'd like to have a separate "debugger" thread that's mostly idle, but as one of its tasks it listens for that F10 key, and when it gets the F10 key, it will cause the script runtime thread to throw an exception, so that it will stop executing the script. So my question is, is there a way to have one thread cause another thread to throw an exception? My application is written in C++.
If the script is actually interpreted by your application then you can just tell the interpreter to stop executing whenever some user event occurs.
It's possible. Detect the keystroke in a separate thread, a hidden window and WM_HOTKEY for example. Call SuspendThread() to freeze the interpreter thread. Now use GetThreadContext() to get the CPU registers of the interpreter thread. Modify CONTEXT.Eip to the address of a function and call SetThreadContext(). Have that function call RaiseException() or throw a C++ exception. ResumeThread() and boom.
A short answer - no.
If your application runs on Windows, maybe you can send a message from this "debugger" tread and have a message loop in the main one?
The problem with that solution is, to do a message sending implementation, I'd have to set up a "listener" as part of the script interpreter. Right now, the interpreter just executes the function. The message loop is implemented outside of the interpreter. If within the function there is an infinite loop, then to break out of that script, I'd have to check for a message in between execution of each instruction in the interpreter, i.e. while(more instructions){check F10, execute script instruction}. That seems like a lot of extra unneeded checks that can slow down the script execution. But if that's the only solution, then I guess that's what it has to be. I still think there's got to be a better way. Maybe the script interpreter needs to be run on a child thread, while the main thread continues its message loop, and will then kill the script interpreter thread when it gets an F10.
Whether you code it explicitly or not, you will need to check a "interrupt" variable in the message loop. If you implement this by a simple volatile int, you will have both a very simple test and very little overhead.
It is unsafe to terminate a thread, as it is probably using resources shared across the entire process.
It is less unsafe to terminate an entire process, but that's not going to help you.
A more safe way to deal with this would be to have the interpreter check for events on a regular basis and treat the stop event as a case to terminate (or at least spill out to a higher loop).
For windows, you could also queue an APC to that thread that calls RaiseException(...) or throws an exception, (although I would avoid the latter, since that crosses API boundaries), but that also implies that the thread will put itself into an alertable state. And I don't really recommend it.