I'm working on a project that requires to execute some processes inside a docker container. I want to handle the case when the process doesn't terminate on time (let's say within 10 s).
I'm using this DockerClientpp library for managing the containers that basically just makes HTTP reqs to the Docker socket. Everything is fine up to this point.
To stop a container that is taking too long I'm using a separate thread. The problems is that I was able to implement it using ptheads but I cannot find a way using std::thread and lambas
Here is my working implementation with pthread
void *ContainerManager::spawnKiller(void *ref) {
ContainerManager *self = (ContainerManager *)ref;
std::unique_ptr<DockerClientpp::DockerClient> dc(new DockerClientpp::DockerClient());
std::cout << "[slave]forceStop(): Waiting " << self->timeOut << " before stopping " << self->activeId << std::endl;
std::this_thread::sleep_for(std::chrono::seconds(self->timeOut));
try {
dc->stopContainer(self->activeId);
std::cout << "[slave]forceStop(): Container will be force-stopped" << std::endl;
} catch(std::exception &e) {
// container has already been destroyed
std::cout << "[slave]forceStop(): Error => " << e.what() << std::endl;
}
pthread_exit(0);
}
void ContainerManager::execute() {
pthread_t killerId;
pthread_create(&killerId, nullptr, &(ContainerManager::spawnKiller), (void *)this);
pthread_detach(killerId);
}
And here is my std::thread and lambda implementation that fails with SEGFAULT as soon as I try to detach the thread.
void ContainerManager::execute() {
std::thread([this]() {
std::this_thread::sleep_for(std::chrono::seconds(timeOut));
try {
dc->stopContainer(activeId);
std::cout << "[slave]forceStop(): Container will be force-stopped" << std::endl;
} catch(std::exception &e) {
// container has already been destroyed
std::cout << "[slave]forceStop(): Error => " << e.what() << std::endl;
}
}).detach();
}
And this is what gdb shows
Thread 1 "test" received signal SIGSEGV, Segmentation fault.
0x0000000000000000 in ?? ()
(gdb) bt
#0 0x0000000000000000 in ?? ()
#1 0x00000000007c6801 in std::thread::detach() ()
#2 0x0000000000410785 in ContainerManager::execute (this=0x7fffffffe2a0, processName=...)
at ../container_manager.cpp:223
#3 0x0000000000412c99 in ContainerManager::executeNew (this=0x7fffffffe2a0, processName=...,
replace=false, language=#0x7fffffffe020: ContainerManager::GO) at ../container_manager.cpp:336
#4 0x00000000004094a9 in main () at test.cpp:36
I tried with a regular function instead of a lamba, I tried capturing the parameters, I also tried passing the parameters as arguments but I'm stuck.
I haven't tried allocating the thread dynamically with new thread(...) but from my understanding even if the std::thread variable goes out of scope, the thread is still alive.
Do you have any suggestion on what I'm doing wrong? I feel like I'm really missing something about std::thread and lambda.
The execute method is a method of the class ContainerManager that it's guaranteed not to go out of scope before the spawned thread has terminated, also the variables that I use (timeOut and activeId are fields of the object)
EDIT:
It really seems there is something wrong with detach()
If I run this
void ContainerManager::execute() {
int *t = new int;
*t = timeOut;
std::string *s = new std::string;
*s = activeId;
std::thread x([&t, &s]() {
std::cout << "LOL" << std::endl;
std::this_thread::sleep_for(std::chrono::seconds(*t));
std::unique_ptr<DockerClientpp::DockerClient> _dc(new DockerClientpp::DockerClient());
try {
_dc->stopContainer(*s);
std::cout << "[slave]forceStop(): Container will be force-stopped" << std::endl;
} catch(std::exception &e) {
// container has already been destroyed
std::cout << "[slave]forceStop(): Error => " << e.what() << std::endl;
}
});
std::cout << "Detaching" << std::endl;
if(x.joinable()) {
std::cout << ".. in a moment" << std::endl;
x.detach();
}
}
I get this output
Detaching
.. in a moment
Segmentation fault (core dumped)
EDIT 2
I tried running this code on my laptop and everything works fine
void ContainerManager::execute() {
// activeId and timeOut are fields of the ContainerManager object
std::thread([this]() {
std::this_thread::sleep_for(std::chrono::seconds(timeOut));
std::unique_ptr<DockerClientpp::DockerClient> dc(new DockerClientpp::DockerClient());
try {
dc->stopContainer(activeId);
std::cout << "[slave]forceStop(): Container will be force-stopped" << std::endl;
} catch(std::exception &e) {
// container has already been destroyed
std::cout << "[slave]forceStop(): Error => " << e.what() << std::endl;
}
}).detach();
}
In the thread, you are accessing references to variables int *t and std::string *s which are local to the ContainerManager::execute() method. As soon as ContainerManager::execute() finishes, accesses to the two variables cause undefined behaviour and in your case the SEGFAULT. Instead pass the two pointers per value to the lamdba (and even better: don't use new at all):
void ContainerManager::execute() {
int *t = new int;
*t = timeOut;
std::string *s = new std::string;
*s = activeId;
std::thread x([t, s]() { // <<--- Pass by value
std::cout << "LOL" << std::endl;
std::this_thread::sleep_for(std::chrono::seconds(*t));
std::unique_ptr<DockerClientpp::DockerClient> _dc(new DockerClientpp::DockerClient());
try {
_dc->stopContainer(*s);
std::cout << "[slave]forceStop(): Container will be force-stopped" << std::endl;
} catch(std::exception &e) {
// container has already been destroyed
std::cout << "[slave]forceStop(): Error => " << e.what() << std::endl;
}
});
std::cout << "Detaching" << std::endl;
if(x.joinable()) {
std::cout << ".. in a moment" << std::endl;
x.detach();
}
}
The segfault suggests, to me, that the class is going out of scope, even though you expect it not to. Another possibility is that you're getting a race condition on the variables you are accessing.
Rather than capturing this in the lambda, try passing all variables by copy to the lambda. This will remove any race conditions having to do with scope, and solve any potential lifetime issues as the lambda will be completely decoupled from any other threads. Of course, this means no pointers or references to data elsewhere, make sure you are really doing a full copy of timeOut and activeId.
Alternatively, rather than detach, I would recommend storing the thread as a data member of the class. Then, join in the destructor. If the thread finishes earlier, the join will basically be a no-op. If the thread is not finished, that will prevent the resources the thread is using from going out of scope until the thread is finished. This would address variables going out of scope, but not any race conditions. Race conditions can be solved by using std::atomic or mutexes.
Since the second solution (using join, std::atomic, and/or mutexes) is more convoluted and requires checking lifetimes and race conditions, I would recommend the first solution (using a lambda that doesn't capture anything, with all arguments passed by copy) if possible.
Related
I've come across classes whose only function is to continuously do some work in a loop and they are designed such that they define a public method that can be called to invoke this member function in a new std::thread. I'm referring to something like this:
class ThreadLooper {
public:
ThreadLooper(const std::string &thread_name)
: thread_name_{thread_name}, loopCounter_{0} {}
~ThreadLooper() {
cout << thread_name_ << ": destroyed and counter is " << loopCounter_
<< std::endl;
}
void run() {
std::thread([this]() { detachedThreadLoop(); }).detach();
}
private:
void detachedThreadLoop() {
cout << thread_name_ << ": detachedThreadLoop() started running"
<< std::endl;
while (true) {
using namespace std::literals::chrono_literals;
std::this_thread::sleep_for(2s);
++loopCounter_;
cout << thread_name_ << ": counter is " << loopCounter_ << std::endl;
}
}
std::string thread_name_;
std::atomic_uint64_t loopCounter_;
};
int main() {
cout << "In main()" << std::endl;
{
ThreadLooper threadLooper{"looper1"};
threadLooper.run();
using namespace std::literals::chrono_literals;
std::this_thread::sleep_for(20s);
cout << "main() done sleeping, exiting block scope..." << std::endl;
}
while (true) {
using namespace std::literals::chrono_literals;
std::this_thread::sleep_for(20s);
cout << "main() woke up..." << std::endl;
}
return 0;
}
It seems like because the function running in the detached thread has a pointer to the instance but can continue to run beyond the lifetime of that instance this is bad. I've seen other classes where the thread isn't detached and then in the destructor a flag is set to tell the thread loop to exit and the thread is then joined in the destructor. It seems like the latter is the correct way to do this and that the former relies on the fact that the class will only be used in situations where instances of it live for the duration of the program. Is this correct or am I missing something?
Yes, using std::thread::detach means you need to have your own method of making sure the thread terminates before all the resources it uses are destroyed.
In this case ThreadLooper will invoke undefined behaviour when the program exits the first block scope in main(). It's better to not use detach() then std::thread will call std::terminate if you've forgotten to call join() before the thread (and its containing object) are destroyed.
According to my knowledge and as far as I checked the boost::asio documentation and source code there is no way to destroy explicitly all pending handlers on a given io_context aside from destroying the context itself?
I need to be able, if possible, to stop the io_context, destroy the pending handlers on the io_context, then do some other things and finally destroy all io objects (timers, pipes, etc) associated with the given io_context and the io_context itself.
I know that I can use work_guard::reset and let all pending handlers to run and then the io_context will stop by itself, but the problem is that many of the handlers may produce (post/defer/etc) new pending handlers, etc i.e. each such handler will need to be guarded with something like 'if stopped'.
I think that the io_context::shutdown does exactly this but there is no way, aside from inheritance maybe, to call explicitly the shutdown function because it's not public.
Thanks.
Trying your suggestion using the protected shutdown results in a segfault on my system. I think it's protected for a reason :)
Anyhow, it looks like a judicious combination of restart/stop/reset might do the job. It's weird that some of the handler queue apparently stays around UNLESS one does a (empty) run/run_one. In fact even a poll_one seems to suffice. So, by all means, include that.
Here's my test bed code, you might find it useful:
Live On Coliru
#include <boost/asio.hpp>
#include <iostream>
using namespace std::chrono_literals;
struct Handler {
void operator()(boost::system::error_code ec) { std::cout << "Handler invoked: " << ec.message() << std::endl; }
struct Instance { // logging only unique instance to avoid noise of moved handlers
Instance() { std::cout << "Created handler instance" << std::endl; }
~Instance() { std::cout << "Destroyed handler instance" << std::endl; }
};
std::unique_ptr<Instance> _instance = std::make_unique<Instance>();
};
int main()
{
struct Hack : boost::asio::io_context {
using boost::asio::io_context::shutdown;
} io;
auto work = make_work_guard(io);
std::cout << " -- run" << std::endl;
auto t = std::thread([&]{ io.run(); });
{
boost::asio::high_resolution_timer tim(io, 2s);
tim.async_wait(Handler{});
work.reset(); // no longer needed
std::this_thread::sleep_for(500ms);
#if 1
io.stop();
#else
io.shutdown(); // segfaults
#endif
}
std::cout << " -- timer destructed" << std::endl;
std::cout << " -- joining" << std::endl;
t.join();
std::cout << " -- empy run to flush handler queue" << std::endl;
io.reset();
//io.run();
//io.run_one();
io.poll_one();
std::cout << " -- bye" << std::endl;
}
Prints
-- run
Created handler instance
-- timer destructed
-- joining
-- empy run to flush handler queue
Handler invoked: Operation canceled
Destroyed handler instance
-- bye
UPDATE
Here's my best suggestion (apart from, I guess, not sharing io at all):
Live On Coliru
#include <boost/asio.hpp>
#include <iostream>
using namespace std::chrono_literals;
struct Handler {
void operator()(boost::system::error_code ec) { std::cout << "Handler invoked: " << ec.message() << std::endl; }
struct Instance { // logging only unique instance to avoid noise of moved handlers
Instance() { std::cout << "Created handler instance" << std::endl; }
~Instance() { std::cout << "Destroyed handler instance" << std::endl; }
};
std::unique_ptr<Instance> _instance = std::make_unique<Instance>();
};
int main()
{
std::unique_ptr<boost::asio::io_context> io;
int i = 1;
for (auto delay : { 1500ms, 500ms }) {
std::cout << " ------------------- reinitialized -------------- \n";
io = std::make_unique<boost::asio::io_context>();
boost::asio::high_resolution_timer tim(*io, 1s);
std::cout << i << " -- run" << std::endl;
auto t = std::thread([&]{ io->run(); });
tim.async_wait(Handler{});
std::this_thread::sleep_for(delay);
std::cout << i << " -- stop" << std::endl;
io->stop();
std::cout << i << " -- joining" << std::endl;
t.join();
std::cout << " ------------------- destruct ------------------- \n";
io.reset();
}
std::cout << "Bye" << std::endl;
}
Prints
------------------- reinitialized --------------
1 -- run
Created handler instance
Handler invoked: Success
Destroyed handler instance
1 -- stop
1 -- joining
------------------- destruct -------------------
------------------- reinitialized --------------
1 -- run
Created handler instance
1 -- stop
1 -- joining
------------------- destruct -------------------
Destroyed handler instance
Bye
At a point in my code, I pass a *this to a method foo(const MyClass& arg). An exception is thrown deep inside this foo, but although a syntactically correct try-catch block exists up the stack, it gets neither handled (a message should have been emitted in that case), nor the process crashes. From the debugging logs, I can see that related thread gets stuck, although the rest of the threads keep going.
I've been through stack unwinding documentation, and somewhere I've seen that arguments to functions are also considered to be auto variables, and get destroyed during the unwinding process. That brings me to the question: what happens when I pass a const reference of this (inside which there is a corresponding catch block) to a method where an exception is thrown? Is it possible that the ref gets the caller object destroyed, and catch block is now unreachable even though stack unwinding has begun already?
Let me add some pseudoish-code:
void MyClass0::someFunc(void)
{
try
{
MyClass1 obj1;
obj1.someOtherFunc(*this);
// Some other stuff
}
catch (MyException&)
{
std::cout << "Handling exception...";
// Whatever... This message is not emitted.
}
}
void MyClass1::someOtherFunc(const MyClass0& argObj0)
{
// Some functions that eventually throw an unhandled MyException
}
Thanks in advance...
EDIT:
OK, trying to generate an executable code for reference, I believe I pretty much answered my own question.
Here's the code:
#include "sandbox.h"
#include <iostream>
MyClass0::MyClass0(void)
{
std::cout << "\nConstructing MyClass0";
}
MyClass0::~MyClass0(void)
{
std::cout << "\nDestructing MyClass0";
}
void MyClass0::trustIssues(void)
{
std::cout << "\nEntering " << __FUNCTION__;
try
{
MyClass1 myClass1;
myClass1.unwaryFunction(*this);
}
catch (MyException& exc)
{
std::cout << "\nException caught in " << __FUNCTION__;
std::cout << "\nLeaving " << __FUNCTION__ << " from inside catch block.";
return;
}
std::cout << "\nLeaving " << __FUNCTION__;
}
MyClass1::MyClass1(void)
{
std::cout << "\nConstructing MyClass1";
}
MyClass1::~MyClass1(void)
{
std::cout << "\nDestructing MyClass1";
}
void MyClass1::unwaryFunction(MyClass0& argClass0)
{
std::cout << "\nEntering " << __FUNCTION__;
suicidalFunction();
std::cout << "\nLeaving " << __FUNCTION__;
}
void suicidalFunction(void)
{
std::cout << "\nEntering " << __FUNCTION__;
MyException myException;
throw myException;
std::cout << "\nLeaving " << __FUNCTION__;
}
int main(int argc, char* argv[])
{
MyClass0 myClass0;
myClass0.trustIssues();
return 0;
}
The output has been:
Constructing MyClass0
Entering MyClass0::trustIssues
Constructing MyClass1
Entering MyClass1::unwaryFunction
Entering suicidalFunction
Destructing MyClass1
Exception caught in MyClass0::trustIssues
Leaving MyClass0::trustIssues from inside catch block.
This implies that the *this argument does not get destroyed on stack unwinding of unwaryFunction. I probably have some other bug in the actual code (as the message analogous to "Exception caught in..." does not get printed). I'll keep this question for future reference. Thanks for your concern anyway.
I'm trying to implement timer with standard environment
Here is a code I have:
bool shutdownDetected = false;
void signal_handler(const int sigid)
{
shutdownDetected = true;
}
int main(int argc, const char * argv[])
{
signal(SIGTERM, (sig_t)signal_handler);
std::async(std::launch::async, [&] () {
std::this_thread::sleep_for( std::chrono::milliseconds{5000});
std::cout << "On TIMER!" << std::endl;
} );
std::cout << "main function" << std::endl;
while (!shutdownDetected) {
}
return EXIT_SUCCESS;
}
As result I see in output after 5 seconds:
// 5 seconds left
On Timer
main function
but would like to see:
main function
// 5 seconds left
On Timer
Seems that my implementation hangs main thread as well. How to avoid this?
Your std::async command returns an std::future, which is then immediately destroyed. The problem is that destruction of a future involves 'joining' the thread you created, which means that the destructor is going to wait until the thread has ended itself and code execution in your main thread doesn't advance until that process has completed.
Simple answer is to assign the result of your std::async call to a variable, and possibly call its get() member function in your loop that tests for termination.
auto t = std::async(std::launch::async, [&] () {
std::this_thread::sleep_for( std::chrono::milliseconds{5000});
std::cout << "On TIMER!" << std::endl;
} );
std::cout << "main function" << std::endl;
t.get();
std::async(std::launch::async, [&] () {
std::this_thread::sleep_for( std::chrono::milliseconds{5000});
std::cout << "On TIMER!" << std::endl;
} );
Does not work unless you assign the std::future returned by std::async to a variable and keep it around. I did not know why this is, clearly because I couldn't be bothered to look it up. Vincent Savard did, and linked us to documentation on the destructor for std::future which says:
it may block if all of the following are true: the shared state was created by a call to std::async, the shared state is not yet ready, and this was the last reference to the shared state.
Since the returnded std::future is not assigned to anything, it is instantly destroyed and the destructor blocks until completion.
I'm going to leave out the signal handler as it's not relevant to the problem.
#include <iostream>
#include <future>
int main()
{
auto letMeLive = std::async(std::launch::async, [] () {
std::this_thread::sleep_for( std::chrono::milliseconds{5000});
std::cout << "On TIMER!" << std::endl;
} );
std::cout << "main function" << std::endl;
letMeLive.wait(); // instead of the signal handler
return EXIT_SUCCESS;
}
I have blocking task which will be performed by find_the_question() function. However, I do not want thread executing this function take more than 10 seconds. So in case it takes more than 10 seconds, I want to close that thread with cleaning all the resources.
I tried to write a code for that, but somehow I am not able to get a interrupt in find_the_question() function if thread takes more than 10 seconds. Could you please tell me what am I doing wrong?
void find_the_question(std::string value)
{
//allocate x resources
try{
//do some process on resources
sleep(14);
//clean resources
}
catch(boost::thread_interrupted const& )
{
//clean resources
std::cout << "Worker thread interrupted" << std::endl;
}
}
int main()
{
boost::posix_time::time_duration timeout = boost::posix_time::milliseconds(10000);
std::cout << "In main" << std::endl;
boost::thread t1(find_the_question, "Can you block me");
t1.interrupt();
if (t1.timed_join(timeout))
{
//finished
std::cout << "Worker thread finished" << std::endl;
}
else
{
//Not finished;
std::cout << "Worker thread not finished" << std::endl;
}
std::cout << "In main end" << std::endl;
}
Output:
If t1 takes more than 10 seconds to complete, I am getting following console output.
std::cout << "In main" << std::endl;
std::cout << "Worker thread not finished" << std::endl;
std::cout << "In main end" << std::endl;
whereas, I am expecting following output
std::cout << "In main" << std::endl;
std::cout << "Worker thread interrupted" << std::endl;
std::cout << "Worker thread not finished" << std::endl;
std::cout << "In main end" << std::endl;
Could you please tell me what am I doing wrong.
Thanks in advance
For using boost::thread::interrupt(), you have to use boost::thread::sleep() for it to work.
A running thread can be interrupted by invoking the interrupt() member
function of the corresponding boost::thread object. When the
interrupted thread next executes one of the specified interruption
points (or if it is currently blocked whilst executing one) with
interruption enabled, then a boost::thread_interrupted exception will
be thrown in the interrupted thread. If not caught, this will cause
the execution of the interrupted thread to terminate. As with any
other exception, the stack will be unwound, and destructors for
objects of automatic storage duration will be executed
Predefined interruption points:
The following functions are interruption points, which will throw
boost::thread_interrupted if interruption is enabled for the current
thread, and interruption is requested for the current thread:
boost::thread::join()
boost::thread::timed_join()
boost::condition_variable::wait()
boost::condition_variable::timed_wait()
boost::condition_variable_any::wait()
boost::condition_variable_any::timed_wait()
boost::thread::sleep()
boost::this_thread::sleep()
boost::this_thread::interruption_point()