I need a way to launch executable (process) from secondary thread. The wxWidgets toolkit I use give me no way to the best of my knowledge and now I have to seek platform dependent way (Linux and Windows only) way to do that.
The executable will read and write to the file. Other than knowing that the process is still running or not (and if possible exit code) there is no much this thread is going to do with process itself. After process terminate, the thread will read the written file, analyze it and send result to GUI thread for displaying. I have no idea how to do it since I have been depending on wxExecute all the times.
Do you really want separate processes, or will threads do.
To get separate processes, you need fork() for Linux and CreateProcess() for Windows. The best approach is often to make a function to encapsulate it.
bool RunProces(const std::string &sCmdLine)
{
#if defined(WIN32)
// launch with CreateProcess() ...
#else
// launch with fork() ...
#endif
return bSuccess;
}
If, however, you can just use threads, you can use the portable std::thread class.
An alternative approach is to use popen() (_popen() for Windows). This launches a new process and redirects its output (stdout) to a pipe. You can then read the pipe to retrieve the sub-process output.
A couple of notes about popen():
Redirecting the standard output makes it impractical for the sub-process to interact with the user.
There would be some overhead setting up the pipe. Probably not important unless you are spawning large numbers of sub-processes.
I've never seen popen() used to launch a non-console Windows program (with no main() function). I wouldn't be surprised if there are problems doing that.
You will have to move quite a few parts of your code from the secondary thread to the main thread. The wxWidgets documentation you refer to says it all:
When a wxProcess object is passed to wxExecute(), its OnTerminate()
virtual method is called when the process terminates. This allows the
program to be (asynchronously) notified about the process termination
and also retrieve its exit status which is unavailable from
wxExecute() in the case of asynchronous execution.
And below:
Currently wxExecute() can only be used from the main thread, calling
this function from another thread will result in an assert failure in
debug build and won't work.
So, you need to eliminate the need to call wxExecute from the secondary thread. How exactly you should do this depends on your application, but it will probably involve the secondary thread sending a custom wxWidgets event to the main thread using wxQueueEvent. The main thread then handles the event by calling wxExecute in asynchronous mode and retrieving its result later on.
Now what to do with the result?
Ideally, you rework your application logic such that the secondary thread does not need the result but instead:
the main thread handles it all by itself, -or-
another secondary thread is started and handles it.
This will minimize the amount of synchronization you have to do, and thus reduce the probability of hard-to-find concurrency programming errors.
Related
I would like to have your opinion for this general technical concept. (I am working on microsoft windows OS)
There is a Process, this process creates multiple threads for different tasks.
Main process: it is a windows service written by C# code.
There are several threads that are create inside the main process: Thread_01, Thread_02, ...
Inside Thread_01: There is a Wrapper dll written in managed C++ to consume DLL_01. (DLL_01 is a dll written by me in native C++ code, that provides some APIs: Add, Remove, Connect)
Add and Remove can run very fast, but Connect may take more than 10 seconds and blocks the caller until it finishes.
I am thinking to use std::async to do the Connect function code, and send the result through a callback to the caller (main process).
Is it a good approach? I heard we cannot create or it is better not to create any thread inside inner threads, is it true? If so, how about std::async ?
Any recommendation is appreciated.
Thanks in advance,
None of what you describe makes the use of threads inacceptable for your code.
As usual, threads have issues that need to be cared for:
Data races due to access to shared data.
Problems of ownership of resources is now not just "Who own what?" but "Who and when owns what?".
When a thread is blocked and you want to abort this operation, how do you cancel it without causing issues down the line? In your case, you must avoid calling the callback, when the receiver doesn't exist any more.
Concerning your approach of using a callback, consider std::future<> instead. This takes care of a few of the issues above, though some are only shifted to the caller instead.
In a multithreaded Linux/C++-program, I want to use fork() with a signal handler for SIGCHLD.
In the child process I use open() to create two new file descriptors, sendfile() and close(), then the child exits.
I planned to use fork() to implement the following requirements:
The threads in the parent process shall be able to
detect the normal termination of the child process, and in that case shall be able to create another fork() doing the open()/sendfile()/close() for a range of files
kill the sendfile()-child process in case of a specific event and detect the intentional termination to clean up
For requirement 1 I could just wait for the result of sendfile().
Requirement 2 is why I think I need to use fork() in the first place.
After reading the following posts
Threads and fork(). How can I deal with that?
fork in multi-threaded program
I think that my solution might not be a good one.
My questions are:
Is there any other solution to implement requirement 2 ?
Or how can I make sure that the library calls open(), close() and sendfile() will be okay?
Update:
The program will run on a Busybox Linux / ARM
I've assumed that I should use sendfile() for having the most efficient file transfer due to several posts I've read regarding this topic.
A safe way to implement my requirement could be using fork() and exec*() with cp, with the disadvantage that the file transfer might be less efficient
Update 2:
it's sufficient to fork() once in case of a specific event (instead of once per file) since I switched to exec*() with rsync in the child process. However the program needs invoke that rsync always in case of a specific event.
You can use threads, but forcefully terminating threads typically leads to memory leaks and other problems.
My linux experience is somewhat limited, but I would probably try to fork the program early, before it gets multithreaded. Now that you have two instances, the single threaded instance can be safely used to manage the starting and stopping of additional instances.
Using only standard C++ (no platform specific API), I would like to launch an external application that may complete immediately or timeout. If the application halts, my app has a timeout after which it simply terminates the app and relaunches it.
Now, down to the nitty gritty, I tried launching two threads:
first thread launches the app and waits for it to terminate
second thread waits for a few seconds and checks if the first thread terminated. If it did not, then it considers it as stalled.
Question is, how do I terminate the first thread from the second? The way I'm launching the app is using the system() function. It's synchronous so there isn't any way for me to check from that thread if I wish for termination. It has to be forced somehow through an exception, externally.
How is this done properly?7
P.S.: if this is not possible, and I suspect it isn't, then I simply do not wish to wait for that thread anymore. It can simply remain stalled in the background. How do I achieve that? (currently, I'm waiting for that thread with a join())
You cannot forcefully terminate another thread. You can only politely ask it to exit. This holds in C++ and POSIX thread models. Windows has TerminateThread, but it's so dangerous it's practically unusable. POSIX has pthread_cancel. That's cooperative termination which could fit your bill, but there's no standard C++ equivalent.
Even if you terminate a thread somehow, it does nothing to any program it might have launched via system.
To let a thread go free with no obligation to join, use thread::detach().
To answer your question about killing a thread, POSIX offers two functions:
pthread_cancel();
This will stop the thread at a cancellation point.
The other is:
pthread_kill();
This function will send a signal to the thread. Contrary to the cancellation pointer concept, this one can happen at any point in the thread. In other words, if the thread has a mutex locked at that time, you're going to lose that lock... (unless you cleanly handle the signal in that thread).
However, what you are describing is a system() call which you make in a separate thread so that way you are not blocked. I don't think that either of these functions are going to help you because there is another process running, not just a simple thread. What you need is to stop that other process.
In your case, what you need to do is find out the pid of the child (or children) and send a signal to that child process (or all children and grandchildren, etc). In that case, you use the kill() function like so:
kill(child_pid, SIGINT);
Once the child died and cleaned up, the system() call will return and your thread is ready to be joined. So in order, you do:
...
child_pid = find_child_pid(); // Note: there is no such function, you have to write it or use a library which offers such
kill(child_pid, SIGNINT);
pthread_join(thread_id);
If that child process can create children and you want them out of the picture too (like in a shell when you hit Ctrl-C) then you need to find about all the children of your child, and their children, etc. You do so by looking at the PPID (Parent PID) of each process to see if it matches one of the children. That info is available in the /proc/<pid>/stat or /proc/<pid>/status. (the first file is probably best because it's just one line, however, it is tricky to go past the process name since it can include parenthesis... so you have to make sure to search the ')' from the end of the line (otherwise you could find a ) from the program name). Once you've got that, skip the state and there is the PPID. (So ) S <ppid>).
Repeat the search until all the parent/child are found and then start sending a SIGINT or SIGTERM or SIGKILL to each on of them.
As mentioned in the other answer, you can use pthread_detach() to quit your software and leave that other thread behind. This is probably much less desirable if you want that other process to end before your main process ends. It very much depends on what you are trying to accomplish, too.
Another, probably much more complicated way, is to use fork() + execve(). That means you have to re-implement your own system() call, but the advantage is that you do not need a thread and you get the pid of the child for free (i.e. thus you can kill it without searching for the child pid). If the function you need to run is not dynamically defined with a varying set of command line arguments, it's not too complicated. If you need to change stdin, stdout, stderr, and the arguments depend on all sorts of things, it becomes much more involved...
In my code the main loop looks like the following
while ( (data = foo()) != NULL ) {
// do processing on data here
}
where foo() is written in C (it fetches the next frame in a video stream, using libavcodec, if you're curious).
My problem is that due to reasons too complicated to go in here, sometimes foo() hangs, which stops the whole program. What I want to do is to detect this condition, i.e. foo() is taking more than N seconds and if this is so take action.
I thought of creating a separate thread to run foo() to implement this by I haven't done any multithreaded programming before. Here's what I want to do:
Main thread creates a child thread and which calls foo()
When foo() is done, the child thread returns
Main thread processes data returned by foo()
If the child takes more than a specified number of time an action is taken by the main thread.
Steps 1-4 are repeated as long as foo() doesn't return null, which signals the end.
How do I go about doing this? Do I need three threads (main, to run foo() and for timing)?
Thanks!
This is exceedingly difficult to do well. The problem is what you're going to do when foo hangs. Nearly the only thing you can do at that point is abort the program (not just the thread) and start over -- killing the thread and attempting to re-start it might work, but it's dangerous at best. The OS will clean up resources when you kill a process, but not when you kill a single thread. It's essentially impossible to figure out what resources belong exclusively to that thread, and what might be shared with some other thread in the process.
That being the case, perhaps you could move the hanging-prone part to a separate process instead, and kill/restart that process when/if it hangs? You'd then send the data to the parent process via some normal form of IPC (e.g., a pipe). In this case, you could have two threads in the parent (processor and watchdog), or (if available) you could do some sort of asynchronous read with time out, and kill the child when/if the read times out (using only one thread).
How do I go about doing this?
You don't. The hard thing is that there is no reliable way to stop a thread - assuming the hang is in libavcodec, interrupting/killing a thread stuck in code you do not have control over leads to more problems than it solves(it might just be memory and file handle leaks if you're not too unlucky). The thread has to stop itself - but that's not an option if you're stuck inside libavcodec.
Many threading implementation doesn't let you kill threads either - though you might request that the thread cancels , if it's stuck in a infinite loop, it'll never cancel though as the cancel requests are processed only at certain boundary points in the OS or low level library calls.
To work around a buggy library like that in a reliable way, you need process isolation. What you do is create a separate program out of your foo() function, execute that and communicated with it using its stdin/stout streams - or some other form of IPC. Talking to an external program, you have various options for doing I/O with timeouts, and can kill the program when you determin it's hanging.
On Linux you can use pthread_timedjoin_np to make this happen with two threads really easily.
I think you can do this with two threads and use the sleep() command in the main thread for the timing part as long as you don't need to do other work there.
You'd probably be better off just fixing what ever is hanging your application.
The Windows and Solaris thread APIs both allow a thread to be created in a "suspended" state. The thread only actually starts when it is later "resumed". I'm used to POSIX threads which don't have this concept, and I'm struggling to understand the motivation for it. Can anyone suggest why it would be useful to create a "suspended" thread?
Here's a simple illustrative example. WinAPI allows me to do this:
t = CreateThread(NULL,0,func,NULL,CREATE_SUSPENDED,NULL);
// A. Thread not running, so do... something here?
ResumeThread(t);
// B. Thread running, so do something else.
The (simpler) POSIX equivalent appears to be:
// A. Thread not running, so do... something here?
pthread_create(&t,NULL,func,NULL);
// B. Thread running, so do something else.
Does anyone have any real-world examples where they've been able to do something at point A (between CreateThread & ResumeThread) which would have been difficult on POSIX?
To preallocate resources and later start the thread almost immediately.
You have a mechanism that reuses a thread (resumes it), but you don't have actually a thread to reuse and you must create one.
It can be useful to create a thread in a suspended state in many instances (I find) - you may wish to get the handle to the thread and set some of it's properties before allowing it to start using the resources you're setting up for it.
Starting is suspended is much safer than starting it and then suspending it - you have no idea how far it's got or what it's doing.
Another example might be for when you want to use a thread pool - you create the necessary threads up front, suspended, and then when a request comes in, pick one of the threads, set the thread information for the task, and then set it as schedulable.
I dare say there are ways around not having CREATE_SUSPENDED, but it certainly has its uses.
There are some example of uses in 'Windows via C/C++' (Richter/Nasarre) if you want lots of detail!
There is an implicit race condition in CreateThread: you cannot obtain the thread ID until after the thread started running. It is entirely unpredictable when the call returns, for all you know the thread might have already completed. If the thread causes any interaction in the rest of that process that requires the TID then you've got a problem.
It is not an unsolvable problem if the API doesn't support starting the thread suspended, simply have the thread block on a mutex right away and release that mutex after the CreateThread call returns.
However, there's another use for CREATE_SUSPENDED in the Windows API that is very difficult to deal with if API support is lacking. The CreateProcess() call also accepts this flag, it suspends the startup thread of the process. The mechanism is identical, the process gets loaded and you'll get a PID but no code runs until you release the startup thread. That's very useful, I've used this feature to setup a process guard that detects process failure and creates a minidump. The CREATE_SUSPEND flag allowed me to detect and deal with initialization failures, normally very hard to troubleshoot.
You might want to start a thread with some other (usually lower) priority or with a specific affinity mask. If you spawn it as usual it can run with undesired priority/affinity for some time. So you start it suspended, change the parameters you want, then resume the thread.
The threads we use are able to exchange messages, and we have arbitrarily configurable priority-inherited message queues (described in the config file) that connect those threads. Until every queue has been constructed and connected to every thread, we cannot allow the threads to execute, since they will start sending messages off to nowhere and expect responses. Until every thread was constructed, we cannot construct the queues since they need to attach to something. So, no thread can be allowed to do work until the very last one was configured. We use boost.threads, and the first thing they do is wait on a boost::barrier.
I stumbled with a similar problem once upon I time. The reasons for suspended initial state are treated in other answer.
My solution with pthread was to use a mutex and cond_wait, but I don't know if it is a good solution and if can cover all the possible needs. I don't know, moreover, if the thread can be considered suspended (at the time, I considered "blocked" in the manual as a synonim, but likely it is not so)