I have encountered a strange problem with boost::asio::io_service::run. Sometimes this run function seems to eat the whole cpu(100%), and sometimes not. I am not very clear about the pattern.
the relevant code:
class Asio {
public:
Asio() :
io_service_(new boost::asio::io_service),
run_() {}
void Start() {
if (!asio_thread_.joinable()) {
run_ = true;
asio_thread_ = std::thread([=] {
Run();
});
}
}
boost::asio::io_service* io_service() { return io_service_.get(); }
protected:
void Run() {
for (;;) {
boost::system::error_code ec;
io_service()->run(ec);
if (run_) {
io_service()->reset();
std::this_thread::sleep_for(std::chrono::milliseconds(100));
} else {
break;
}
}
}
protected:
std::unique_ptr<boost::asio::io_service> io_service_;
std::thread asio_thread_;
std::atomic<bool> run_;
};
When the run function runs normally, below is the callstack
#0 0x00000035f74e9163 in epoll_wait () from /lib64/libc.so.6
#1 0x0000000000b3f6ef in boost::asio::detail::epoll_reactor::run(bool, boost::asio::detail::op_queue<boost::asio::detail::task_io_service_operation>&) ()
#2 0x0000000000b40111 in boost::asio::detail::task_io_service::do_run_one(boost::asio::detail::scoped_lock<boost::asio::detail::posix_mutex>&, boost::asio::detail::task_io_service_thread_info&, boost::system::error_code const&) ()
#3 0x0000000000b3feaf in boost::asio::detail::task_io_service::run(boost::system::error_code&) ()
#4 0x0000000000b403fd in boost::asio::io_service::run(boost::system::error_code&) ()
#5 0x0000000000b3ddc1 in Asio::Run() ()
When the run function behaves abnormally, below is the callstack:
#0 0x00000031bbee53c9 in syscall () from /lib64/libc.so.6
#1 0x00007f831d1d3d68 in std::chrono::_V2::steady_clock::now() () from /usr/local/gcc48/lib64/libstdc++.so.6
#2 0x0000000000b45b6d in boost::asio::detail::chrono_time_traits<std::chrono::_V2::steady_clock, boost::asio::wait_traits<std::chrono::_V2::steady_clock> >::now() ()
#3 0x0000000000b45608 in boost::asio::detail::timer_queue<boost::asio::detail::chrono_time_traits<std::chrono::_V2::steady_clock, boost::asio::wait_traits<std::chrono::_V2::steady_clock> > >::get_ready_timers(boost::asio::detail::op_queue<boost::asio::detail::task_io_service_operation>&) ()
#4 0x0000000000b3f5d7 in boost::asio::detail::timer_queue_set::get_ready_timers(boost::asio::detail::op_queue<boost::asio::detail::task_io_service_operation>&) ()
#5 0x0000000000b3f815 in boost::asio::detail::epoll_reactor::run(bool, boost::asio::detail::op_queue<boost::asio::detail::task_io_service_operation>&) ()
#6 0x0000000000b40111 in boost::asio::detail::task_io_service::do_run_one(boost::asio::detail::scoped_lock<boost::asio::detail::posix_mutex>&, boost::asio::detail::task_io_service_thread_info&, boost::system::error_code const&) ()
#7 0x0000000000b3feaf in boost::asio::detail::task_io_service::run(boost::system::error_code&) ()
#8 0x0000000000b403fd in boost::asio::io_service::run(boost::system::error_code&) ()
#9 0x0000000000b3ddc1 in Asio::Run() ()
In both cases, there're some pending handlers in the io_service, so the io_service::run should not return and should be wait for the event to happen.
Any advise is welcome.
I did further check, it's seems it's due to the boost::asio::steady_timer used. The usage of steady_timer involves the usage of the following pattern:
boost::asio::steady_timer timer;
timer.expires_at(some_expiry, error_code)
timer.async_wait((=)(boost::system::error_code ec) {
// some operation
timer.expires_at(new_expiry, error_code);
timer.asyn_wait(...);
});
Where the timer is wrapped in a shared pointer, and it's safe to copy into the lamda function.
Related
I have this small snippet of code:
// (...)
class Time
{
std::atomic<bool> m_running;
std::thread m_worker;
// ...
};
Time::Time()
{
// ...
m_running = true;
m_worker = std::move(std::thread(std::bind(&Time::Worker, this)));
}
bool Time::HasTimedOut() const
{
return (!m_disabled) &&
(IsPending() && (GetRunTime() >= m_maximum_timeout) && (CloseHandlesDiff() >= m_minimum_close_time));
}
Time::~Time()
{
if (m_running)
{
m_running = false;
m_worker.join();
}
}
void Time::Worker()
{
while (m_running)
{
if (time_data->HasTimedOut())
{
time_data->RunTimedOutCallback();
}
if (m_count < 0)
{
m_count = 0;
}
if (m_running)
{
std::this_thread::sleep_for(std::chrono::milliseconds(20));
}
}
}
std::shared_ptr<Time> time_data(std::make_shared<Time>());
To my surprise, I have gotten a coredump, and the backtrace command from gdb shows this:
(gdb) bt
#0 0x09438408 in monitor::Time::HasTimedOut (this=0x0)
at monitor.cxx: // return (!m_disabled) &&
#1 0x09438a84 in monitor::Time::Worker (this=0xbd96dd8)
monitor.cxx: // if(time_data->HasTimedOut()
#2 0x0943cf81 in std::__invoke_impl<void, void (monitor::Time::*&)(), monitor::Time*&> (
__f=#0xbd96e54: (void (monitor::Time::*)(monitor::Time * const)) 0x94389f0 <monitor::Time::Worker()>, __t=#0xbd96e5c: 0xbd96dd8)
A nullptr seems to be the problem (SEGFAULT):
(this=0x0)
This means that my class got destroyed, without the destructor being called.
This might be possible when the OS/watchdog for my application does a force exit / quick terminate as far as I know/suspect.
Are there any ways to deal with this? Maybe some shared_ptr atomic wrapping where I could check if the shared_ptr is a nullptr, is there some atomic if-not-null-execute-this? Then again.. this happened literally mid-execution.
I know one can add quick-exit hooks, but the quick exit is sometimes used for good reason, it would be agains the design to slow down the quick-exit. What would be the best way to handle this becoming a nullptr?
Or should I just let the SEGFALT happen because the application is being quick-exited anyway?
Here is the full stack, but that probably won't add any more useful information:
(gdb) bt
(gdb) bt
#0 0x09438408 in monitor::Time::HasTimedOut (this=0x0)
at /opt/procesleiding/vptlib/lib/oracle_monitor.cxx:111
#1 0x09438a84 in monitor::Time::Worker (this=0xbd96dd8)
at /opt/procesleiding/vptlib/lib/oracle_monitor.cxx:142
#2 0x0943cf81 in std::__invoke_impl<void, void (monitor::Time::*&)(), monitor::Time*&> (
__f=#0xbd96e54: (void (monitor::Time::*)(monitor::Time * const)) 0x94389f0 <monitor::Time::Worker()>, __t=#0xbd96e5c: 0xbd96dd8)
at /usr/include/c++/7/bits/invoke.h:73
#3 0x0943c99f in std::__invoke<void (monitor::Time::*&)(), monitor::Time*&> (
__fn=#0xbd96e54: (void (monitor::Time::*)(monitor::Time * const)) 0x94389f0 <monitor::Time::Worker()>, __args#0=#0xbd96e5c: 0xbd96dd8)
at /usr/include/c++/7/bits/invoke.h:95
#4 0x0943c70c in std::_Bind<void (monitor::Time::*(monitor::Time*))()>::__call<void, , 0u>(std::tuple<>&&, std::_Index_tuple<0u>) (
this=0xbd96e54, __args=...)
at /usr/include/c++/7/functional:467
#5 0x0943c28c in std::_Bind<void (monitor::Time::*(monitor::Time*))()>::operator()<, void>() (this=0xbd96e54)
at /usr/include/c++/7/functional:551
#6 0x0943bcaf in std::__invoke_impl<void, std::_Bind<void (monitor::Time::*(monitor::Time*))()>>(std::__invoke_other, std::_Bind<void (monitor::Time::*(monitor::Time*))()>&&) (__f=...) at /usr/include/c++/7/bits/invoke.h:60
#7 0x0943b022 in std::__invoke<std::_Bind<void (monitor::Time::*(monitor::Time*))()>>(std::_Bind<void (monitor::Time::*(monitor::Time*))()>&&) (__fn=...) at /usr/include/c++/7/bits/invoke.h:95
#8 0x0943e2a6 in std::thread::_Invoker<std::tuple<std::_Bind<void (monitor::Time::*(monitor::Time*))()> > >::_M_invoke<0u>(std::_Index_tuple<0u>) (this=0xbd96e54) at /usr/include/c++/7/thread:234
#9 0x0943e15c in std::thread::_Invoker<std::tuple<std::_Bind<void (monitor::Time::*(monitor::Time*))()> > >::operator()() (this=0xbd96e54) at /usr/include/c++/7/thread:243
#10 0x0943e067 in std::thread::_State_impl<std::thread::_Invoker<std::tuple<std::_Bind<void (monitor::Time::*(monitor::Time*))()> > > >::_M_run() (this=0xbd96e50) at /usr/include/c++/7/thread:186
Your code has a race condition.
Time::Time()
{
// ...
m_running = true;
m_worker = std::move(std::thread(std::bind(&Time::Worker, this)));
}
Here tread starts before
std::shared_ptr<Time> time_data(std::make_shared<Time>());
is completed.
Simply thread reaches monitor::Time::HasTimedOut before std::shared_ptr<Time> time_data(std::make_shared<Time>()); is completed.
Spawn thread not in constructor, but in separate method which you will invoke after time_data is assigned.
Anyway it would be better if your Timer do not use time_data global variable at all.
Code:
Listener.hpp:
template <typename SocketType>
void Listener<SocketType>::BeginAccept()
{
auto worker = SelectWorker();
auto socket = worker->CreateSocket();
m_acceptor->async_accept(socket->GetAsioSocket(),
[this, worker, socket] (const boost::system::error_code &ec)
{
this->OnAccept(worker, socket, ec);
});
}
template <typename SocketType>
void Listener<SocketType>::OnAccept(NetworkThread<SocketType> *worker, std::shared_ptr<SocketType> const& socket, const boost::system::error_code &ec)
{
// an error has occurred
if (ec)
worker->RemoveSocket(socket.get());
else
socket->Open();
BeginAccept();
}
Socket.cpp:
bool Socket::Open()
{
try
{
const_cast<std::string &>(m_address) = m_socket.remote_endpoint().address().to_string();
const_cast<std::string &>(m_remoteEndpoint) = boost::lexical_cast<std::string>(m_socket.remote_endpoint());
}
catch (boost::system::error_code& error)
{
sLog.outInfo("Socket::Open() failed to get remote address. Error: %s", error.message().c_str());
return false;
}
catch (const std::exception& error)
{
sLog.outInfo("Socket::Open() failed(with std::exception) to get remote address. Error: %s", error.what());
return false;
}
catch (...)
{
sLog.outError("Socket::Open() failed to get remote address. Other error");
return false;
}
m_outBuffer.reset(new PacketBuffer);
m_secondaryOutBuffer.reset(new PacketBuffer);
m_inBuffer.reset(new PacketBuffer);
StartAsyncRead();
return true;
}
Is this related to TCP attack?
EDIT: gdb:
(gdb) p m_socket
$1 = {<boost::asio::basic_socket<boost::asio::ip::tcp, boost::asio::stream_socket_service<boost::asio::ip::tcp> >> = {<boost::asio::basic_io_object<boost::asio::stream_socket_service<boost::asio::ip::tcp>, true>> = {
implementation = {<boost::asio::detail::reactive_socket_service_base::base_implementation_type> = {socket_ = 21, state_ = 80 'P', reactor_data_ = 0x7ffff0005120}, protocol_ = {family_ = 2}}, service_ = 0x7f2b30}, <boost::asio::socket_base> = {
static message_peek = 2, static message_out_of_band = 1, static message_do_not_route = 4, static message_end_of_record = 128, static max_connections = 128}, <No data fields>}, <No data fields>}
(gdb) p m_socket.is_open()
$2 = true
(gdb) bt
#0 MaNGOS::Socket::Open (this=0x7ffff0003c40) at /home/ubuntu/MoltenCore/MoltenCore/src/shared/Network/Socket.cpp:52
#1 0x00000000004fbff1 in MaNGOS::Listener<AuthSocket>::OnAccept (this=0x7fffffffe6c0, worker=0x8041e0, socket=std::shared_ptr (count 2, weak 1) 0x7ffff0003c40, ec=...) at /home/ubuntu/MoltenCore/MoltenCore/src/shared/Network/Listener.hpp:121
#2 0x00000000004fa030 in MaNGOS::Listener<AuthSocket>::BeginAccept()::{lambda(boost::system::error_code const&)#1}::operator()(boost::system::error_code const&) const (__closure=0x7ffff4ec8be0, ec=...)
at /home/ubuntu/MoltenCore/MoltenCore/src/shared/Network/Listener.hpp:110
#3 0x0000000000502a2d in boost::asio::detail::binder1<MaNGOS::Listener<AuthSocket>::BeginAccept()::{lambda(boost::system::error_code const&)#1}, boost::system::error_code>::operator()() (this=0x7ffff4ec8be0) at /usr/include/boost/asio/detail/bind_handler.hpp:47
#4 0x00000000005020c3 in boost::asio::asio_handler_invoke<boost::asio::detail::binder1<MaNGOS::Listener<AuthSocket>::BeginAccept()::{lambda(boost::system::error_code const&)#1}, boost::system::error_code> >(boost::asio::detail::binder1<MaNGOS::Listener<AuthSocket>::BeginAccept()::{lambda(boost::system::error_code const&)#1}, boost::system::error_code>&, ...) (function=...) at /usr/include/boost/asio/handler_invoke_hook.hpp:69
#5 0x00000000005014b8 in boost_asio_handler_invoke_helpers::invoke<boost::asio::detail::binder1<MaNGOS::Listener<AuthSocket>::BeginAccept()::{lambda(boost::system::error_code const&)#1}, boost::system::error_code>, {lambda(boost::system::error_code const&)#1}>(boost::asio::detail::binder1<MaNGOS::Listener<AuthSocket>::BeginAccept()::{lambda(boost::system::error_code const&)#1}, boost::system::error_code>&, {lambda(boost::system::error_code const&)#1}&) (function=..., context=...)
at /usr/include/boost/asio/detail/handler_invoke_helpers.hpp:37
#6 0x0000000000500886 in boost::asio::detail::reactive_socket_accept_op<boost::asio::basic_socket<boost::asio::ip::tcp, boost::asio::stream_socket_service<boost::asio::ip::tcp> >, boost::asio::ip::tcp, MaNGOS::Listener<AuthSocket>::BeginAccept()::{lambda(boost::system::error_code const&)#1}>::do_complete(boost::asio::detail::task_io_service*, boost::asio::detail::task_io_service_operation*, boost::system::error_code const&, unsigned long) (owner=0x8084d0, base=0x804910) at /usr/include/boost/asio/detail/reactive_socket_accept_op.hpp:123
#7 0x00000000004f0300 in boost::asio::detail::task_io_service_operation::complete (this=0x804910, owner=..., ec=..., bytes_transferred=0) at /usr/include/boost/asio/detail/task_io_service_operation.hpp:38
#8 0x00000000004f1d55 in boost::asio::detail::epoll_reactor::descriptor_state::do_complete (owner=0x8084d0, base=0x804140, ec=..., bytes_transferred=1) at /usr/include/boost/asio/detail/impl/epoll_reactor.ipp:651
#9 0x00000000004f0300 in boost::asio::detail::task_io_service_operation::complete (this=0x804140, owner=..., ec=..., bytes_transferred=1) at /usr/include/boost/asio/detail/task_io_service_operation.hpp:38
#10 0x00000000004f2823 in boost::asio::detail::task_io_service::do_run_one (this=0x8084d0, lock=..., this_thread=..., ec=...) at /usr/include/boost/asio/detail/impl/task_io_service.ipp:372
#11 0x00000000004f23a1 in boost::asio::detail::task_io_service::run (this=0x8084d0, ec=...) at /usr/include/boost/asio/detail/impl/task_io_service.ipp:149
#12 0x00000000004f2abc in boost::asio::io_service::run (this=0x7ef540) at /usr/include/boost/asio/impl/io_service.ipp:59
#13 0x00000000004f7c89 in MaNGOS::Listener<AuthSocket>::Listener(int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, int)::{lambda()#1}::operator()() const () at /home/ubuntu/MoltenCore/MoltenCore/src/shared/Network/Listener.hpp:84
#14 0x0000000000505624 in std::_Bind_simple<MaNGOS::Listener<AuthSocket>::Listener(int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, int)::{lambda()#1} ()>::_M_invoke<>(std::_Index_tuple<>) (this=0x8049c8)
at /usr/include/c++/5/functional:1531
#15 0x0000000000504f38 in std::_Bind_simple<MaNGOS::Listener<AuthSocket>::Listener(int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, int)::{lambda()#1} ()>::operator()() (this=0x8049c8) at /usr/include/c++/5/functional:1520
#16 0x0000000000504416 in std::thread::_Impl<std::_Bind_simple<MaNGOS::Listener<AuthSocket>::Listener(int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, int)::{lambda()#1} ()> >::_M_run() (this=0x8049b0) at /usr/include/c++/5/thread:115
#17 0x00007ffff6c98c80 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#18 0x00007ffff7bc16ba in start_thread (arg=0x7ffff4ec9700) at pthread_create.c:333
#19 0x00007ffff63fe82d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
EDIT:
NetworkThread.hpp:
template <typename SocketType>
class NetworkThread
{
private:
boost::asio::io_service m_service;
std::mutex m_socketLock;
std::unordered_set<std::shared_ptr<SocketType>> m_sockets;
// note that the work member *must* be declared after the service member for the work constructor to function correctly
std::unique_ptr<boost::asio::io_service::work> m_work;
std::thread m_serviceThread;
public:
NetworkThread() : m_work(new boost::asio::io_service::work(m_service))
{
m_serviceThread = std::thread([this] { boost::system::error_code ec; this->m_service.run(ec); });
}
~NetworkThread()
{
// Allow io_service::run() to exit.
m_work.reset();
m_service.stop();
m_serviceThread.join();
// attempt to gracefully close any open connections
for (auto i = m_sockets.begin(); i != m_sockets.end();)
{
auto const current = i;
++i;
if (!(*current)->IsClosed())
(*current)->Close();
}
}
size_t Size() const { return m_sockets.size(); }
std::shared_ptr<SocketType> CreateSocket();
void RemoveSocket(Socket *socket)
{
std::lock_guard<std::mutex> guard(m_socketLock);
m_sockets.erase(socket->shared<SocketType>());
}
};
template <typename SocketType>
std::shared_ptr<SocketType> NetworkThread<SocketType>::CreateSocket()
{
std::lock_guard<std::mutex> guard(m_socketLock);
auto const i = m_sockets.emplace(std::make_shared<SocketType>(m_service, [this] (Socket *socket) { this->RemoveSocket(socket); }));
return *i.first;
}
The problem is totally different:
In your code:
template <typename SocketType>
void Listener<SocketType>::OnAccept(
NetworkThread<SocketType> *worker,
std::shared_ptr<SocketType> const& socket,
const boost::system::error_code &ec )
{
// at this point, socket has 1 reference count, held by the lambda
// in BeginAccept, its reference count cannot increase, since it's
// constant.
// ...
BeginAccept(); // this creates a new, distinct instance of the lambda
// fuinction object in BeginAccept.
// when we exit, constrol is given back to calling lambda in BeginAccept
// its destructor happily deletes the socket, since its reference count
// is still only 1.
}
Bottom line: OnAccept() needs to receive the socket pointer by value, and it needs to transfer or store this shared pointer somehow. This is usually done
by passing the shared_ptr to BeginReceive() BY VALUE. BeginReceive() must then take the responsibility to keep the socket pointer alive until the connection dies, this, it usually does that by passing the pointer back to itself, either as a parameter, or as a lambda capture, always by value.
The easiest way to fix your code is to move the call to StartAsyncRead() to OnAccept(), since there is a convenient copy of a shared_ptr of the socket there.
I am trying to join the thread using boost and the program seems to wait indefinitely when I do that.
Thread->join();
On the console I see the following output when I break.
Program received signal SIGINT, Interrupt.
pthread_cond_wait##GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
185 ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S: No such file or directory.
Here is the GDB trace.
#0 pthread_cond_wait##GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
#1 0x00000000004900a3 in boost::condition_variable::wait(boost::unique_lock<boost::mutex>&) () at /usr/local/boost/include/boost/thread/pthread/condition_variable.hpp:73
#2 0x00007ffff724e333 in boost::thread::join_noexcept() () at libs/thread/src/pthread/thread.cpp:316
#3 0x000000000048e135 in LIN::waitForThreads() () at /usr/local/boost/include/boost/thread/detail/thread.hpp:767
#4 0x00000000004882fd in fa::testinit::test_method() () at /home/ubuntu/UnitTest01.cpp:205
#5 0x0000000000488ef1 in fa::testinit_invoker() () at /home/ubuntu/UnitTest01.cpp:173
#6 0x000000000047296b in boost::detail::function::function_obj_invoker0<boost::detail::forward, int>::invoke(boost::detail::function::function_buffer&) ()
at /usr/local/boost/include/boost/function/function_template.hpp:771
#7 0x000000000044aabd in boost::execution_monitor::catch_signals(boost::function<int ()> const&) () at /usr/local/boost/include/boost/function/function_template.hpp:771
#8 0x000000000044ab91 in boost::execution_monitor::execute(boost::function<int ()> const&) () at /usr/local/boost/include/boost/test/impl/execution_monitor.ipp:1207
#9 0x000000000044b2d5 in boost::execution_monitor::vexecute(boost::function<void ()> const&) () at /usr/local/boost/include/boost/test/impl/execution_monitor.ipp:1313
#10 0x0000000000452bb2 in boost::unit_test::unit_test_monitor_t::execute_and_translate(boost::function<void ()> const&, unsigned int) () at /usr/local/boost/include/boost/test/impl/unit_test_monitor.ipp:46
#11 0x0000000000484760 in boost::unit_test::framework::state::execute_test_tree(unsigned long, unsigned int) () at /usr/local/boost/include/boost/test/impl/framework.ipp:685
#12 0x00000000004849ac in boost::unit_test::framework::state::execute_test_tree(unsigned long, unsigned int) () at /usr/local/boost/include/boost/test/impl/framework.ipp:636
#13 0x0000000000460329 in boost::unit_test::framework::run(unsigned long, bool) () at /usr/local/boost/include/boost/test/impl/framework.ipp:636
#14 0x0000000000460957 in boost::unit_test::unit_test_main(boost::unit_test::test_suite* (*)(int, char**), int, char**) () at usr/local/boost/include/boost/test/impl/unit_test_main.ipp:228
#15 0x00007ffff6673aed in __libc_start_main (main=0x4419c0 <main>, argc=1, argv=0x7fffffffe6c8, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffffffe6b8) at libc-start.c:269
#16 0x0000000000442ec9 in _start () at ../sysdeps/x86_64/start.S:122
waitForThreads code :
{
stopthread = true;
//wait for Thread 2
boost::mutex::scoped_lock lock(thread2_lock);
lock.unlock();
m_ConditionVariable.notify_one();
m_Thread->join();
}
Thread 2 code:
while(!stopthread){
boost::mutex::scoped_lock lock(thread2_lock);
while (m_Queue.empty())
{
m_ConditionVariable.wait(lock);
}
if (!m_Queue.empty()){
struct queue_output out;
out = m_Queue.front();
m_Queue.pop_front();
boost::mutex::scoped_lock new_lock(another_lock);
if (write(fd, &out, sizeof(struct queue_output)) == -1)
close(out.fd);
}
}
code to update queue:
{
boost::mutex::scoped_lock lock(thread2_lock);
m_Queue.push_back(input);
lock.unlock();
m_ConditionVariable.notify_one();
}
In UnitTest01.cpp, testinit first I am creating the thread, then updating the queue, then calling join.
Can someone please help me understand why it is hanging while thread joining?
Thanks in advance.
Hy,
i have a gtkmm application, which does some async network-requests, to ask the server for additional properties of the gtk-widgets.
This means for example, that the application should be able to change the label of a widget.
In this example I have created a new widget based on Gtk::ToggleButton.
But I found out that sometimes the gtkmm-application crashes with a segfault. When debuging with gdb I always get the line where i set the label.
For better understanding, I have created a MWE which does the label-changes in a loop, to simulate lots of async-calls:
#include <boost/asio.hpp>
#include <boost/asio/steady_timer.hpp>
#include <iostream>
#include <thread>
#include <mutex>
#include <gtkmm/application.h>
#include <gtkmm/window.h>
#include <gtkmm/togglebutton.h>
class led_label_t : public Gtk::ToggleButton {
public:
using value_list_t = std::vector<Glib::ustring>;
using lock_t = std::lock_guard<std::mutex>;
led_label_t(Glib::ustring label = "<no data>", bool mnemonic = false)
: Gtk::ToggleButton(std::move(label), std::move(mnemonic)),
_values{"SEL1", "SEL2"} {}
protected:
virtual void on_toggled(void) override {
std::cout << "Clicked Button." << std::endl;
lock_t lock(_mtx);
value_changed(_values[get_active()]);
}
virtual void value_changed(Glib::ustring& value) {
std::string path;
if (get_active()) {
path =
"/usr/share/icons/Adwaita/16x16/emblems/emblem-important.png";
} else {
path = "/usr/share/icons/Adwaita/16x16/emblems/emblem-default.png";
}
remove(); // remove previous label
std::cout << "Changed Label of led_label: "
<< ", value: " << value << std::endl;
add_pixlabel(path, value);
}
private:
mutable std::mutex _mtx;
value_list_t _values;
};
int main(void) {
auto app = Gtk::Application::create();
Gtk::Window window;
window.set_default_size(200, 200);
led_label_t inst{};
inst.show();
window.add(inst);
auto f = [&inst, &window]() {
using namespace std::chrono_literals;
boost::asio::io_service io;
{ //wait for startup
boost::asio::steady_timer t{io, 100ms};
t.wait();
}
bool toggle = true;
for (auto i = 0; i < 2000; i++) {
std::cout << "i=" << i << std::endl;
//wait until next simulated button click
boost::asio::steady_timer t{io, 1ms};
t.wait();
inst.set_active(toggle);
toggle = !toggle;
}
};
std::thread c1(f);
std::thread w([&app, &window]() { app->run(window); });
c1.join();
window.hide();
w.join();
return EXIT_SUCCESS;
}
To compile this example, I use following command:
g++ main.cpp -o main `pkg-config --cflags --libs gtkmm-3.0` -Wall -pedantic -Wextra -Werror -Wcast-qual -Wcast-align -Wconversion -fdiagnostics-color=auto -g -O0 -std=c++14 -lboost_system -pthread
I am using GCC 4.9.2 and libgtkmm-3.14 (both standard debian jessie)
The segfault I get is the following:
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffe7fff700 (LWP 7888)]
0x00007ffff6288743 in ?? () from /usr/lib/x86_64-linux-gnu/libgtk-3.so.0
(gdb) bt
#0 0x00007ffff6288743 in ?? () from /usr/lib/x86_64-linux-gnu/libgtk-3.so.0
#1 0x00007ffff6288838 in ?? () from /usr/lib/x86_64-linux-gnu/libgtk-3.so.0
#2 0x00007ffff6267ce9 in ?? () from /usr/lib/x86_64-linux-gnu/libgtk-3.so.0
#3 0x00007ffff627241b in ?? () from /usr/lib/x86_64-linux-gnu/libgtk-3.so.0
#4 0x00007ffff63a1601 in ?? () from /usr/lib/x86_64-linux-gnu/libgtk-3.so.0
#5 0x00007ffff63a154c in ?? () from /usr/lib/x86_64-linux-gnu/libgtk-3.so.0
#6 0x00007ffff63a26b8 in ?? () from /usr/lib/x86_64-linux-gnu/libgtk-3.so.0
#7 0x00007ffff644d5ff in ?? () from /usr/lib/x86_64-linux-gnu/libgtk-3.so.0
#8 0x00007ffff644d9b7 in gtk_widget_realize ()
from /usr/lib/x86_64-linux-gnu/libgtk-3.so.0
#9 0x00007ffff644dbe8 in gtk_widget_map ()
from /usr/lib/x86_64-linux-gnu/libgtk-3.so.0
#10 0x00007ffff621c387 in ?? () from /usr/lib/x86_64-linux-gnu/libgtk-3.so.0
#11 0x00007ffff626270f in ?? () from /usr/lib/x86_64-linux-gnu/libgtk-3.so.0
#12 0x00007ffff46bf474 in ?? ()
from /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0
#13 0x00007ffff46d9087 in g_signal_emit_valist ()
from /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0
#14 0x00007ffff46d99df in g_signal_emit ()
from /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0
#15 0x00007ffff644db99 in gtk_widget_map ()
from /usr/lib/x86_64-linux-gnu/libgtk-3.so.0
#16 0x00007ffff64506d8 in gtk_widget_set_parent ()
from /usr/lib/x86_64-linux-gnu/libgtk-3.so.0
#17 0x00007ffff6217a9b in ?? () from /usr/lib/x86_64-linux-gnu/libgtk-3.so.0
#18 0x00007ffff79a44eb in Gtk::Container_Class::add_callback(_GtkContainer*, _GtkWidget*) () from /usr/lib/x86_64-linux-gnu/libgtkmm-3.0.so.1
#19 0x00007ffff46c253b in g_cclosure_marshal_VOID__OBJECTv ()
from /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0
#20 0x00007ffff46bf474 in ?? ()
from /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0
#21 0x00007ffff46d9087 in g_signal_emit_valist ()
from /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0
#22 0x00007ffff46d99df in g_signal_emit ()
from /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0
#23 0x00007ffff6261aa5 in gtk_container_add ()
from /usr/lib/x86_64-linux-gnu/libgtk-3.so.0
#24 0x000000000040b0b5 in led_label_t::value_changed (this=0x7fffffffe2a0,
value=...) at main.cpp:38
#25 0x000000000040afb1 in led_label_t::on_toggled (this=0x7fffffffe2a0)
at main.cpp:24
#26 0x00007ffff7a18af0 in Gtk::ToggleButton_Class::toggled_callback(_GtkToggleButton*) () from /usr/lib/x86_64-linux-gnu/libgtkmm-3.0.so.1
#27 0x00007ffff46bf245 in g_closure_invoke ()
from /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0
#28 0x00007ffff46d083b in ?? ()
from /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0
#29 0x00007ffff46d9778 in g_signal_emit_valist ()
from /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0
#30 0x00007ffff46d99df in g_signal_emit ()
from /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0
#31 0x00007ffff63ecb4d in ?? () from /usr/lib/x86_64-linux-gnu/libgtk-3.so.0
#32 0x00007ffff798a4a0 in Gtk::Button_Class::clicked_callback(_GtkButton*) ()
from /usr/lib/x86_64-linux-gnu/libgtkmm-3.0.so.1
#33 0x00007ffff46bf474 in ?? ()
from /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0
#34 0x00007ffff46d9087 in g_signal_emit_valist ()
from /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0
#35 0x00007ffff46d99df in g_signal_emit ()
from /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0
#36 0x00007ffff63ec936 in gtk_toggle_button_set_active ()
from /usr/lib/x86_64-linux-gnu/libgtk-3.so.0
#37 0x0000000000405e12 in <lambda()>::operator()(void) const (
__closure=0x74f4f8) at main.cpp:73
#38 0x000000000040811a in std::_Bind_simple<main()::<lambda()>()>::_M_invoke<>(std::_Index_tuple<>) (this=0x74f4f8) at /usr/include/c++/4.9/functional:1700
#39 0x0000000000407fa9 in std::_Bind_simple<main()::<lambda()>()>::operator()(void) (this=0x74f4f8) at /usr/include/c++/4.9/functional:1688
#40 0x0000000000407e9e in std::thread::_Impl<std::_Bind_simple<main()::<lambda()>()> >::_M_run(void) (this=0x74f4e0) at /usr/include/c++/4.9/thread:115
#41 0x00007ffff3f47970 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#42 0x00007ffff37650a4 in start_thread (arg=0x7fffe7fff700)
at pthread_create.c:309
#43 0x00007ffff349a04d in clone ()
at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
Maybe the Interesting line of this is
#24: 0x000000000040b0b5 in led_label_t::value_changed (this=0x7fffffffe2a0,
value=...) at main.cpp:38)
this is the line where add_pixlabel(path, value); is called.
What am I doing wrong here?
Attention:
This segfault doesn't come always, I found out, that on my desktop-machine I get the error once every 10 calls. (Intel i7-3xxx)
And on my laptop I get the error nearly every call (Intel i5-3xxx)
Now I have found a solution, based on the answer of #user4581301. He was right, that gtkmm doesn't support multithreading. (To be more precise, libsigc++ and sigc::trackable are not thread-safe)
However, care is required when writing programs based on gtkmm using
multiple threads of execution, arising from the fact that libsigc++,
and in particular sigc::trackable, are not thread-safe.
Quote from gtkmm documentation.
Therefore I have used Glib::Dispatcher, to execute the set_label() - method in the context of the gtkmm-Main-Loop of the window.
Here is the code, that did not segfault anymore on my machine(s) (even with many retries)
#include <boost/asio.hpp>
#include <boost/asio/steady_timer.hpp>
#include <cassert>
#include <iostream>
#include <thread>
#include <mutex>
#include <gtkmm/application.h>
#include <gtkmm/window.h>
#include <gtkmm/togglebutton.h>
#include <glibmm/dispatcher.h>
#define LOG() \
std::cout << (std::chrono::system_clock::now() - start).count() << " " \
<< std::this_thread::get_id() << ": "
auto start = std::chrono::system_clock::now();
class led_label_t : public Gtk::ToggleButton {
public:
using value_list_t = std::vector<Glib::ustring>;
using lock_t = std::lock_guard<std::mutex>;
using action_queue_t = std::vector<Glib::ustring>;
led_label_t(Glib::ustring label = "<no data>", bool mnemonic = false)
: Gtk::ToggleButton(std::move(label), std::move(mnemonic)),
_values{"SEL1", "SEL2"} {}
void set_dispatcher(Glib::Dispatcher* dp) {
_dp = dp;
_dp->connect([this](void) { dispatcher_task(); });
}
protected:
virtual void on_toggled(void) override {
LOG() << "Clicked Button." << std::endl;
{
lock_t lock(_action_mtx);
auto value = _values[get_active()];
_action_queue.push_back({value});
LOG() << "Added label into queue " << value << std::endl;
if (_action_queue.size() > 1) {
return;
}
}
_dp->emit();
}
void dispatcher_task(void) {
Glib::ustring label;
for (;;) {
{
lock_t lock(_action_mtx);
if (_action_queue.size() == 0) {
return;
}
label = *_action_queue.begin();
_action_queue.erase(_action_queue.begin());
}
set_label(label);
LOG() << "Set the label " << label << std::endl;
}
}
private:
mutable std::mutex _action_mtx;
action_queue_t _action_queue;
value_list_t _values;
Glib::Dispatcher* _dp;
};
int main(void) {
auto app = Gtk::Application::create();
Gtk::Window window;
window.set_default_size(200, 200);
led_label_t inst{};
inst.show();
window.add(inst);
auto f = [&inst, &window]() {
using namespace std::chrono_literals;
boost::asio::io_service io;
{ // wait for startup
boost::asio::steady_timer t{io, 100ms};
t.wait();
}
bool toggle = true;
for (auto i = 0; i < 200000; i++) {
// wait until next simulated button click
boost::asio::steady_timer t{io, 250us};
t.wait();
LOG() << "i=" << i << std::endl;
inst.set_active(toggle);
toggle = !toggle;
LOG() << "finished" << std::endl;
}
};
std::thread c1(f);
std::thread w([&app, &window, &inst]() {
Glib::Dispatcher dp;
inst.set_dispatcher(&dp);
app->run(window);
});
c1.join();
window.hide();
w.join();
return EXIT_SUCCESS;
}
Accessing and changing UI components from multiple threads is always tricky. UIs need to be fast and responsive to user input, so they can't hang around for background tasks to complete. As a result UI components are rarely protected by mutex or other synchronization. You write, it happens. Except when something else gets in the way.
If you write from two threads... Ooops.
You're half way through a write when another thread reads... Ooops.
Say for example Thread 4 is part way through writing a new string into the label when a screen refresh is triggered. If the backend for label is a c-style string, the terminating NULL may have been overwritten and the label write runs off the end into bad RAM.
All sorts of things could go wrong, and some will be survivable or, worse, look like it. You're better off having all of the UI management in one thread and have the other threads queue updates to the UI thread. Start by looking into Model View Controller and then try related patterns if needed.
I am trying to implement thread pool in C++ using pthread. I want to encapsulate logic related to threads management in one object which is taking ownership of these threads. That means whenever this object is destroyed, threads must be stopped and cleaned up.
I've been testing my code and it turns out that I get segmentation fault when I destroy WorkerThreadManager object while there is boost::function called. See the code and backtrace from GDB. I don't really understand why it happens, as far as I know boost::function is copyable, so once I get a copy of it from the queue, I can pop() it and even destroy whole queue (I prooved that in some small test) and then call the function's copy.
WorkerThreadManager.h:
#include "WorkerThreadManagerInterface.h"
#include "utils/mutex.h"
#include <queue>
#include <semaphore.h>
#include <iostream>
class WorkerThreadManager : public WorkerThreadManagerInterface
{
public:
WorkerThreadManager(unsigned threadsNumber = 5);
virtual ~WorkerThreadManager();
virtual void PushTask(thread_function_t A_threadFun, result_function_t A_resultFun);
void SignalResults();
private:
static void* WorkerThread(void* A_data);
void PushResult(int A_result, result_function_t A_resultFun);
typedef boost::function<void ()> signal_function_t;
struct worker_thread_data_t
{
worker_thread_data_t(thread_function_t A_threadFun, result_function_t A_resultFun) :
threadFun(A_threadFun), resultFun(A_resultFun) {}
worker_thread_data_t() {}
thread_function_t threadFun;
result_function_t resultFun;
};
const unsigned m_threadsNumber;
pthread_t* m_pthreads;
utils::Mutex m_tasksMutex;
sem_t m_tasksSem;
std::queue<worker_thread_data_t> m_tasks;
utils::Mutex m_resultsMutex;
std::queue<signal_function_t> m_results;
};
WorkerThreadManager.cpp:
#include "WorkerThreadManager.h"
#include "gateway_log.h"
#include <pthread.h>
/**
* #brief Creates semaphore and starts threads.
*/
WorkerThreadManager::WorkerThreadManager(unsigned threadsNumber) : m_threadsNumber(threadsNumber)
{
if ( sem_init(&m_tasksSem, 0, 0) )
{
std::stringstream ss;
ss << "Semaphore could not be initialized: " << errno << " - " << strerror(errno);
LOG_FATAL(ss);
throw std::runtime_error(ss.str());
}
m_pthreads = new pthread_t[m_threadsNumber];
for (unsigned i = 0; i < m_threadsNumber; ++i)
{
int rc = pthread_create(&m_pthreads[i], NULL, WorkerThreadManager::WorkerThread, (void*) this );
if(rc)
{
std::stringstream ss;
ss << "Pthread could not be started: " << errno << " - " << strerror(errno);
LOG_FATAL(ss.str());
if ( sem_destroy(&m_tasksSem) )
LOG_ERROR("Semaphore could not be destroyed: " << errno << " - " << strerror(errno));
delete [] m_pthreads;
throw std::runtime_error(ss.str());
}
else
{
LOG_DEBUG("Worker thread started " << m_pthreads[i]);
if(pthread_detach(m_pthreads[i]))
LOG_WARN("Failed to detach worker thread");
}
}
}
/**
* #brief Cancels all threads, destroys semaphore
*/
WorkerThreadManager::~WorkerThreadManager()
{
LOG_DEBUG("~WorkerThreadManager()");
for(unsigned i = 0; i < m_threadsNumber; ++i)
{
if ( pthread_cancel(m_pthreads[i]) )
LOG_ERROR("Worker thread cancellation failed");
}
if ( sem_destroy(&m_tasksSem) )
LOG_ERROR("Semaphore could not be destroyed: " << errno << " - " << strerror(errno));
delete [] m_pthreads;
}
/**
* #brief Adds new task to queue, so worker threads can
* #param A_threadFun function which will be executed by thread
* #param A_resultFun function which will be enqueued for calling with return value of A_threadFun as parameter
* after worker thread executes A_threadFun.
*/
void WorkerThreadManager::PushTask(thread_function_t A_threadFun, result_function_t A_resultFun)
{
utils::ScopedLock mutex(m_tasksMutex);
worker_thread_data_t data(A_threadFun, A_resultFun);
m_tasks.push( data );
sem_post(&m_tasksSem);
LOG_DEBUG("Task for worker threads has been added to queue");
}
/**
* #brief Executes result functions (if there are any) to give feedback
* to classes which requested task execution in worker thread.
*/
void WorkerThreadManager::SignalResults()
{
while(true)
{
signal_function_t signal;
{
utils::ScopedLock mutex(m_resultsMutex);
if(m_results.size())
{
signal = m_results.front();
m_results.pop();
}
else
return;
}
signal();
}
}
/**
* #brief Enqueues result of function executed in worker thread.
* #param A_result return value of function executed in worker thread
* #param A_resultFun function which will be enqueued for calling with A_result as a parameter.
*/
void WorkerThreadManager::PushResult(int A_result, result_function_t A_resultFun)
{
utils::ScopedLock mutex(m_resultsMutex);
signal_function_t signal = boost::bind(A_resultFun, A_result);
m_results.push( signal );
}
/**
* #brief worker thread body
* #param A_data pointer to WorkerThreadManager instance
*/
void* WorkerThreadManager::WorkerThread(void* A_data)
{
WorkerThreadManager* manager = reinterpret_cast<WorkerThreadManager*>(A_data);
LOG_DEBUG("Starting worker thread loop");
while (1)
{
if ( -1 == sem_wait(&manager->m_tasksSem) && errno == EINTR )
{
LOG_DEBUG("sem_wait interrupted with signal");
continue;
}
LOG_DEBUG("WorkerThread:::::: about to call lock mutex");
worker_thread_data_t data;
{
utils::ScopedLock mutex(manager->m_tasksMutex);
data = manager->m_tasks.front();
manager->m_results.pop();
}
LOG_DEBUG("WorkerThread:::::: about to call resultFun");
int result = data.threadFun();
LOG_DEBUG("WorkerThread:::::: after call resultFun");
pthread_testcancel();
manager->PushResult(result, data.resultFun);
}
return NULL;
}
main.cpp:
#include "gateway_log.h"
#include "WorkerThreadManager.h"
#include <memory>
class A {
public:
int Fun() { LOG_DEBUG("Fun before sleep"); sleep(8); LOG_DEBUG("Fun after sleep");return 0; }
void Result(int a) { LOG_DEBUG("Result: " << a); }
};
int main()
{
sd::auto_ptr<WorkerThreadManager> workerThreadManager = new WorkerThreadManager;
A a;
workerThreadManager->PushTask(boost::bind(&A::Fun, &a), boost::bind(&A::Result, &a, _1));
sleep(3);
LOG_DEBUG("deleting workerThreadManager");
workerThreadManager.reset(); // <<<--- CRASH
LOG_DEBUG("deleted workerThreadManager");
sleep(10);
LOG_DEBUG("after sleep");
return 0;
}
GDB:
(gdb) bt
#0 0xb7ad33a0 in ?? () from /lib/i386-linux-gnu/libc.so.6
#1 0x0807d3a7 in boost::function0<void>::clear (this=0x858db48) at /home/marcin/intel_build/boost_1_42_0/boost/function/function_template.hpp:856
#2 0x0807d17b in boost::function0<void>::~function0 (this=0x858db48, __in_chrg=<optimized out>) at /home/marcin/intel_build/boost_1_42_0/boost/function/function_template.hpp:752
#3 0x0807cec5 in boost::function<void()>::~function(void) (this=0x858db48, __in_chrg=<optimized out>) at /home/marcin/intel_build/boost_1_42_0/boost/function/function_template.hpp:1043
#4 0x0807ced8 in std::_Destroy<boost::function<void ()> >(boost::function<void ()>*) (__pointer=0x858db48) at /usr/include/c++/4.6/bits/stl_construct.h:94
#5 0x0807c868 in std::_Destroy_aux<false>::__destroy<boost::function<void ()>*>(boost::function<void ()>*, boost::function<void ()>*) (__first=0x858db48, __last=0x858d928) at /usr/include/c++/4.6/bits/stl_construct.h:104
#6 0x0807bd05 in std::_Destroy<boost::function<void ()>*>(boost::function<void ()>*, boost::function<void ()>*) (__first=0x858d938, __last=0x858d928) at /usr/include/c++/4.6/bits/stl_construct.h:127
#7 0x0807af23 in std::_Destroy<boost::function<void ()>*, boost::function<void ()> >(boost::function<void ()>*, boost::function<void ()>*, std::allocator<boost::function<void ()> >&) (__first=0x858d938, __last=0x858d928)
at /usr/include/c++/4.6/bits/stl_construct.h:153
#8 0x0807a037 in std::deque<boost::function<void ()>, std::allocator<boost::function<void ()> > >::_M_destroy_data_aux(std::_Deque_iterator<boost::function<void ()>, boost::function<void ()>&, boost::function<void ()>*>, std::_Deque_iterator<boost::function<void ()>, boost::function<void ()>&, boost::function<void ()>*>) (this=0x858beec, __first=..., __last=...) at /usr/include/c++/4.6/bits/deque.tcc:795
#9 0x08076153 in std::deque<boost::function<void ()>, std::allocator<boost::function<void ()> > >::_M_destroy_data(std::_Deque_iterator<boost::function<void ()>, boost::function<void ()>&, boost::function<void ()>*>, std::_Deque_iterator<boost::function<void ()>, boost::function<void ()>&, boost::function<void ()>*>, std::allocator<boost::function<void ()> > const&) (this=0x858beec, __first=..., __last=...) at /usr/include/c++/4.6/bits/stl_deque.h:1816
#10 0x08073411 in std::deque<boost::function<void()>, std::allocator<boost::function<void()> > >::~deque(void) (this=0x858beec, __in_chrg=<optimized out>) at /usr/include/c++/4.6/bits/stl_deque.h:898
#11 0x0806a355 in std::queue<boost::function<void()>, std::deque<boost::function<void()>, std::allocator<boost::function<void()> > > >::~queue(void) (this=0x858beec, __in_chrg=<optimized out>)
at /usr/include/c++/4.6/bits/stl_queue.h:92
#12 0x0815a054 in WorkerThreadManager::~WorkerThreadManager (this=0x858be98, __in_chrg=<optimized out>) at WorkerThreadManager.cpp:42
#13 0x0815a1e3 in WorkerThreadManager::~WorkerThreadManager (this=0x858be98, __in_chrg=<optimized out>) at WorkerThreadManager.cpp:56
#14 0x080c6c51 in std::auto_ptr<WorkerThreadManager>::reset (this=0x85463e4, __p=0x0) at /usr/include/c++/4.6/backward/auto_ptr.h:244
#15 0x080604a9 in main ()
I would really appreciate any help.
There is no guarantee that pthread_cancel waits for the cancellation of the target completes before it returns. When successful, it simply requests cancellation, but does not wait for it to complete. You need to use pthread_join to wait for the threads to have completed.
I suspect that as the destructor is proceeding in one thread, one of the threads wakes up (due to the sem_destroy), and erroneously attempts to read/pop the queue. I'm not sure why it's causing a crash in the main thread, but I would eliminate this potential issue first.
Finally, I would highly recommend you move some of these semaphore and thread mechanisms to their own classes, to make the code more exception safe.
I found a bug, it was trivial - shame on me :(
In function void* WorkerThreadManager::WorkerThread(void* A_data) I popped m_results queue instead of m_tasks as I had intended:
worker_thread_data_t data;
{
utils::ScopedLock mutex(manager->m_tasksMutex);
data = manager->m_tasks.front();
manager->m_results.pop();
}
Anyway I don't really understand why it caused crash so late - in destructor of the queue.