Invoking Tcl proc in multithreaded mode - c++

My function func() is being called by multiple threads. (Each thread will call this function only once.)
From inside the func(), I want each thread to call a Tcl proc named tcl_proc_name (which takes no arguments).
For this, I did like this
Tcl_Eval( Tcl_CreateInterp() , "tcl_proc_name");
But it somehow this code is not able to invoke the Tcl proc.
Am I missing something?

Every Tcl interpreter (i.e., every instance of Tcl_Interp) is strongly bound by design to the thread that creates it; the implementation internally uses thread-specific data extensively so as to virtually completely avoid the need for major locks (like the Global Interpreter Lock that bedevils performant multithreaded Python code). Tcl commands are utterly bound to the interpreter that contains them. You have to either:
Send messages to a single thread to perform the action. (See the Tcl_QueueEvent() function, or use the Thread package's thread::send command from the Tcl level.)
Duplicate the command in multiple interpreters, one per thread. That might be easy or complex in your application.
For avoidance of all doubt: you cannot safely use a Tcl interpreter from multiple threads. It will not work; guaranteed. It will cause crashes.

Related

How do I associate two Tcl_Interp?

I have a code that looks like this:
myCmd::myCmd(std::string outFile) : _tclInterp(Tcl_CreateInterp()), _outFile(outFile)
{
_myParser = CC::CmdParser(_tclInterp, registerAllCommands);
}
The thread which created the Tcl_CreateInterp() might be different from what is being used in the CC::CmdParser. Internally CC::CmdParser also calls Tcl_DeleteInterp() once all the procedures are complete. It will be a problem if the thread id is different during calls to CreateInterp and DeleteInterp.
How do I associate or tell such that same thread does Tcl_CreateInterp() (in the constructor) and Tcl_DeleteInterp() (in the CmdParser).
NOTE that the CC::CmdParser is a third-party API and I don't have source code.
Tcl interpreter objects internally use thread-specific data extensively in order to avoid most global locks. This goes as far as including thread-specific memory management pools, at least for common allocations, and Tcl code is executed as part of initialising the interpreter state, so it's really not possible to use an interpreter in a different thread from the one in which it was created. The Tcl library checks that you're using it correctly in a few places, at least in some build modes, and will make the process abort() (or Windows's equivalent) if you disobey the rules; that's much friendlier than the crash that would otherwise be coming due to fundamental assumptions not being respected…
This means that you must create, use, and destroy the interpreter on the same thread.
You might be advised to initialise the Tcl interpreter lazily if you can't guarantee that the containing object is used in the same thread that it was created in. You might think that this results in handling of things that isn't really the C++ way, but the thread-bound nature of interpreters means that you've really not got any alternative. (You might want to look at Tcl_CreateExitHandler() for help with cleaning up the interpreter when the thread goes away, but that will only help if Tcl_FinalizeThread() or Tcl_ExitThread() are called; if you're being thread-promiscuous on the C++ side then there's a good chance that you'll be baking in all sorts of problems anyway.)
If the C++ part insists on using multiple threads, you'll have to have a single thread for the Tcl code and use inter-thread messaging to let the other threads tell it things. That's done with Tcl_ThreadQueueEvent(). The canonical way to use that is the thread package, and that includes the canonical example of how to do so (github.com/tcltk/thread/generic/threadCmd.c); most of the complexity in there relates to turning an essentially one-way (but “reliable”) operation into a two-way one that lets you do a procedure call into a different thread; one-way calls are far simpler.

Is using popen() in C/C++ is a bad coding practise?

I want to change the timezone for Linux system. I know there are many ways.
One way is to use tzset() function and another is to call 'timedatectl' command from 'popen()' function.
I am using second approach i.e, using "popen()".
I just want to ask is it a good programming practice to use "popen()" in your code?
Also, I am carefully calling "pclose()" for every "popen()".
There is nothing wrong about popen in general, if you really need a child process to do a specific job for you.
popen creates a pipe allowing you to either read the output (what it wrote to stdout) of the child process or write input to its stdin - but not both at the same time.
If you are not interested in either option, you possibly might prefer calling system instead (however, system will wait for the process to terminate, in contrast to popen - pclose waits for).
But why would you want to create a separate process, if you can do the same job by simply calling an ordinary function (system call or not)? You are creating a lot of overhead using a process then (process must be initialised and hooked into OS, it needs its own memory for executable code, heap and stack, ...)!
It gets a little more complicated, if the job in question requires a considerable amount of time and you cannot afford to wait for the function to complete, but need to do some other stuff. However, in such a case, I'd rather create a thread only and again call the function from there...
popen() invokes a shell to run the command which is an extra unnecessary layer of indirection. Plus there are all sorts of security pitfalls, for instance, you don't have control over the environment - or which shell actually gets invoked.
I'd say it's fine for prototypes and proofs of concept, but for production code you should use fork(), one of the execs and pipes for IO.
EDIT
If there is a function equivalent to doinf something by invoking a command, always use that first. For example, if you can achieve what you want with tzset(), always use that in preference to spawning a new process.

Load Lua script once, execute in multiple states in multiple threads

I would like to load a Lua script into my C++ program, and then invoke that script multiple times in separate threads. I'm trying to avoid loading the script in each thread (why go through that overhead).
I'm thinking something like this in the C++ program:
create lua state L
load script into L
and in n threads do:
create local lua state Si (i = 1..n, i.e., separate state per thread)
grab "compiled" script from L and invoke in the context in Si
Is there a "standard" approach to doing this? The primary goal is to avoid having each thread load the script. The script may also be executed multiple times in state Si. Note that the scripts running in separate threads are not cooperating (i.e., they know nothing about each other and we would like to keep it that way).
As I said in a comment, I dont think you can do what you want with c++ threads without data races or mutexes that block all parallel execution.
A single lua state does not seem to be designed to be used by multiple threads and lua threads dont support multithreading either and there is no way to move data from a completely separate state to another magically.
However what you can try is to "compile" the lua scripts on a state by doing loadstring and then dump and save that in a safe way so that you can access it from all threads to load the script faster from the bytecode directly with loadstring.
Otherwise you would need to keep all states separate and do all communication between them by C++ and ensure thread safety there.

Synchronization requirements for DbgHelp functions

I recently implemented a utility to write user-mode dumps using MiniDumpWriteDump (part of the Debug Help Library). The implementation is separated into its own process. A target process can launch this utility and communicate with it to trigger writing a minidump.
This is all working as expected. I am not sure, though, how to interpret the rules with respect to concurrency for MiniDumpWriteDump:
All DbgHelp functions, such as this one, are single threaded. Therefore, calls from more than one thread to this function will likely result in unexpected behavior or memory corruption. To avoid this, you must synchronize all concurrent calls from more than one thread to this function.
Does the above refer to calls from different threads to a single process? Or do concurrent calls also require synchronization, if MiniDumpWriteDump is called in different processes (e.g. when each target process launches its own utility process)?
The problem is in the library: "All DbgHelp functions, such as this one, are single threaded".
Therefore, any process invoking this library will have to make sure that only one single thread is using the DbgHelp functions simultaneously.
Just use a lock before you use DbgHelp, that is, if your program using DbgHelp is multi-threaded (otherwise there's no problem).
Another way to remove this limitation is to add a (per-thread) context to the DbgHelp library: you move all (file-scope) static and (program-scope) global variables in a C structure passed as an argument to all library function calls.

Break a lua script in c++ from another thread?

I'm currently wrapping a lua class in c++ and it's going pretty well so far. But I'm wondering if there some way to break a lua script from running(could be in the middle of the script) for another thread. So if I run my lua script on thread 1, can I break it from thread 2? Would lua_close(...) do that?
Thanks.
If this is an expected occurrence and most of the Lua script's time is spent inside Lua functions (i.e., not lengthy, blocking C calls), you could install a debug hook that checks for your "break flag" every N instructions and aborts the script. See the "debug library" section of Programming in Lua.
Then there must be some way to force the script to halt?
No, there doesn't.
Lua provides exactly one thread safety guarantee: two separate lua_States (separate being defined as different objects returned by lua_open) can be called freely from two different CPU threads. Thread A can call lua_State X, and thread B can call lua_State Y.
That is the only guarantee Lua gives you. Lua is inherently single-threaded; it only provides this guarantee because all of the global information for Lua is contained in a self-contained object: lua_State.
The rules of Lua require that only one thread talk to a lua_State at any one time. Therefore, it is impossible for Lua code to be executing and then be halted or paused by external C++ code. That would require breaking the rule: some other thread would have to call functions on this thread's lua_State.
If you want to abort a script in-progress, you will have to use debug hooks. This will dramatically slow down the performance of your script (if that matters to you). You will also need thread synchronization primitives, so that thread B can set a flag that thread A will read and halt on.
As for how you "abort" the script, that... is impossible in the most general case. And here's why.
The lua_State represents a Lua thread of execution. It's got a stack, a pointer to instructions to interpret, and some code it's currently executing. To "abort" this is not a concept that Lua has. What Lua does have is the concept of erroring. Calling lua_error from within C code being called by Lua will cause it to longjmp out of that function. This will also unwind the Lua thread, and it will return an error to the most recent function that handles errors.
Note: if you use lua_error in C++ code, you need to either have compiled Lua as C++ (thus causing errors to work via exception handling rather than setjmp/longjmp) or you need to make sure that every object with a destructor is off the stack when you call it. Calling longjmp in C++ code is not advisable without great care.
So if you issue lua_pcall to call some script, and in your debug hook, you issue lua_error, then you will get an error back in your lua_pcall statement. Lua's stack will be unwound, objects destroyed, etc. That's good.
The reason why this does not work in the general case (besides the fact that I have no idea how Lua would react to a debug hook calling lua_error) is quite simple: Lua scripts can execute pcall too. The error will go to the nearest pcall statement, whether it's in Lua or in C.
This Lua script will confound this method:
local function dostuff()
while true do end
end
while true do
pcall(dostuff)
end
So issuing lua_error does not guarantee that you'll "abort the script." Think of it like throwing exceptions; anyone could catch them, so you can't ensure that they are only caught in one place.
In general, you should never want to abort a Lua scripts execution in progress. This is not something you should be wanting to do.