ROS1 Callback Threads Behavior: Two Callback Accessing Shared Data - c++

My question is that is the callback thread-safe? For multiple callbacks that may access and potentially modify shared data, is adding mutex locks a good practice?
I am testing the behavior of running two threads with two callback functions that access and modify shared data.
I have found a similar post here, but in my simple experiments, I am expecting something such as race condition happening, but it did not happen.
I wrote a publisher node which can publish two String messages in two threads and each message will be pubished for one second with different frequency.
#include <boost/thread.hpp>
#include <ros/ros.h>
#include <std_msgs/String.h>
int main(int argc, char **argv)
{
ros::init(argc, argv, "AB_topic_pub");
ros::NodeHandle nh;
ros::Publisher pub_a = nh.advertise<std_msgs::String>("msg_a", 200);
ros::Publisher pub_b = nh.advertise<std_msgs::String>("msg_b", 200);
int COUNT_A = 0, COUNT_B = 0;
ros::Duration(2.0).sleep();
boost::thread pub_thread_a(
[&]()
{
ROS_INFO("Publishing A for one second.");
ros::Time start_time = ros::Time::now();
ros::Rate freq(1000);
while (ros::Time::now() - start_time < ros::Duration(1.0)) {
std_msgs::String MSG_A;
MSG_A.data = "A" + std::to_string(COUNT_A);
pub_a.publish(MSG_A);
freq.sleep();
COUNT_A++;
}
}
);
boost::thread pub_thread_b(
[&]()
{
ROS_INFO("Publishing B for one second.");
ros::Time start_time = ros::Time::now();
ros::Rate freq(200);
while (ros::Time::now() - start_time < ros::Duration(1.0)) {
std_msgs::String MSG_B;
MSG_B.data = "B" + std::to_string(COUNT_B);
pub_b.publish(MSG_B);
freq.sleep();
COUNT_B++;
}
}
);
pub_thread_a.join();
pub_thread_b.join();
std::cout << "A COUNT: " << COUNT_A << std::endl;
std::cout << "B COUNT: " << COUNT_B << std::endl;
}
I wrote a subscriber node with two callback queues. In the following code, I want to count the how many times the callbacks are called in total. I defined two variables COUNT_WO_LOCK without mutex guarding and COUNT_W_LOCK with a mutex guarding.
#include <iostream>
#include <boost/function.hpp>
#include <boost/atomic/atomic.hpp>
#include <boost/thread.hpp>
#include <ros/callback_queue.h>
#include <ros/ros.h>
#include <std_msgs/String.h>
int main(int argc, char **argv)
{
ros::init(argc, argv, "callback_lock_test");
ros::NodeHandle nh_a;
ros::NodeHandle nh_b;
ros::CallbackQueue cq_a, cq_b;
nh_a.setCallbackQueue(&cq_a);
nh_b.setCallbackQueue(&cq_b);
int COUNT_WO_LOCK;
COUNT_WO_LOCK = 0;
int COUNT_W_LOCK;
COUNT_W_LOCK = 0;
boost::mutex LOCK;
boost::function<void(const std_msgs::String::ConstPtr &msg)> cb_a =
[&](const std_msgs::StringConstPtr &msg)
{
LOCK.lock();
COUNT_W_LOCK++;
LOCK.unlock();
COUNT_WO_LOCK++;
ROS_INFO("[Thread ID: %s] I am A, heard: [%s]", boost::lexical_cast<std::string>(boost::this_thread::get_id()).c_str() , msg->data.c_str());
};
boost::function<void(const std_msgs::String::ConstPtr &msg)> cb_b =
[&](const std_msgs::StringConstPtr &msg)
{
LOCK.lock();
COUNT_W_LOCK++;
LOCK.unlock();
COUNT_WO_LOCK++;
ROS_WARN("\t\t[Thread ID: %s] I am B, heard: [%s]", boost::lexical_cast<std::string>(boost::this_thread::get_id()).c_str() , msg->data.c_str());
};
// A pub for one second at 1000Hz
ros::Subscriber sub_a = nh_a.subscribe<std_msgs::String>("msg_a", 200,
cb_a, ros::VoidConstPtr(), ros::TransportHints().tcpNoDelay(true));
// B pub for one second at 200Hz
ros::Subscriber sub_b = nh_b.subscribe<std_msgs::String>("msg_b", 200,
cb_b, ros::VoidConstPtr(), ros::TransportHints().tcpNoDelay(true));
boost::thread spin_thread_a(
[&]()
{
ROS_ERROR("\t\t\t\t\t [Spinner Thead ID: %s]", boost::lexical_cast<std::string>(boost::this_thread::get_id()).c_str());
while (!sub_a.getNumPublishers()) { }
while (ros::ok())
cq_a.callAvailable(ros::WallDuration());
}
);
boost::thread spin_thread_b(
[&]()
{
ROS_ERROR("\t\t\t\t\t [Spinner Thead ID: %s]", boost::lexical_cast<std::string>(boost::this_thread::get_id()).c_str());
while (!sub_b.getNumPublishers()) { }
while (ros::ok())
cq_b.callAvailable(ros::WallDuration());
}
);
spin_thread_a.join();
spin_thread_b.join();
if (ros::isShuttingDown()) {
std::cout << "LOCKED COUNT: " << COUNT_W_LOCK << std::endl;
std::cout << "UNLOCKED COUNT: " << COUNT_WO_LOCK << std::endl;
}
}
I first launched the subscriber node and then launched the publisher node. I am expecting that COUNT_W_LOCK is 1200 and COUNT_WO_LOCK is <1200, but in fact they are all 1200.

As mentioned in a comment callbacks are thread safe in ROS. I think the issue is what you're describing vs what you're expecting might be a little different. The short answer is that for a single callback, even if you have multiple messages published at once, there will only every be one execution of that callback at a given time; i.e. 2 threads will not execute that callback at the same time. This is because on the backend ROS subscribers use a queue. So as callbacks are executed the last message is popped off the queue.
Do note that there are potential thread safety issues with accessing that data before the threads return. It's not something your code does, but issues can pop up if your main thread is editing or accessing data at the same time as the callback.

Related

How could one delay a function without the use of sleep / suspending the code?

I need to delay a function by x amount of time. The problem is that I can't use sleep nor any function that suspends the function (that's because the function is a loop that contains more function, sleeping / suspending one will sleep / suspend all)
Is there a way I could do it?
If you want to execute some specific code at a certain time interval and don't want to use threads (to be able to suspend), then you have to keep track of time and execute the specific code when the delay time was exceeded.
Example (pseudo):
timestamp = getTime();
while (true) {
if (getTime() - timestamp > delay) {
//main functionality
//reset timer
timestamp = getTime();
}
//the other functionality you mentioned
}
With this approach, you invoke a specific fuction every time interval specified by delay. The other functions will be invoked at each iteration of the loop.
In other words, it makes no difference if you delay a function or execute it at specific time intervals.
Assuming that you need to run functions with their own arguments inside of a loop with custom delay and wait for them to finish before each iteration:
#include <cstdio>
void func_to_be_delayed(const int &idx = -1, const unsigned &ms = 0)
{
printf("Delayed response[%d] by %d ms!\n", idx, ms);
}
#include <chrono>
#include <future>
template<typename T, typename ... Ta>
void delay(const unsigned &ms_delay, T &func, Ta ... args)
{
std::chrono::time_point<std::chrono::high_resolution_clock> start = std::chrono::high_resolution_clock::now();
double elapsed;
do {
std::chrono::time_point<std::chrono::high_resolution_clock> end = std::chrono::high_resolution_clock::now();
elapsed = std::chrono::duration<double, std::milli>(end - start).count();
} while(elapsed <= ms_delay);
func(args...);
}
int main()
{
func_to_be_delayed();
const short iterations = 5;
for (int i = iterations; i >= 0; --i)
{
auto i0 = std::async(std::launch::async, [i]{ delay((i+1)*1000, func_to_be_delayed, i, (i+1)*1000); } );
// Will arrive with difference from previous
auto i1 = std::async(std::launch::async, [i]{ delay(i*1000, func_to_be_delayed, i, i*1000); } );
func_to_be_delayed();
// Loop will wait for all calls
}
}
Notice: this method potentially will spawn additional thread on each call with std::launch::async type of policy.
Standard solution is to implement event loop.
If you use some library, framework, system API, then most probably there is something similar provided to solve this kind of problem.
For example Qt has QApplication which provides this loop and there is QTimer.
boost::asio has io_context which provides even loop in which timer can be run boost::asio::deadline_timer.
You can also try implement such event loop yourself.
Example wiht boost:
#include <boost/asio.hpp>
#include <boost/date_time.hpp>
#include <exception>
#include <iostream>
void printTime(const std::string& label)
{
auto timeLocal = boost::posix_time::second_clock::local_time();
boost::posix_time::time_duration durObj = timeLocal.time_of_day();
std::cout << label << " time = " << durObj << '\n';
}
int main() {
boost::asio::io_context io_context;
try {
boost::asio::deadline_timer timer{io_context};
timer.expires_from_now(boost::posix_time::seconds(5));
timer.async_wait([](const boost::system::error_code& error){
if (!error) {
printTime("boom");
} else {
std::cerr << "Error: " << error << '\n';
}
});
printTime("start");
io_context.run();
} catch (const std::exception& e) {
std::cerr << e.what() << '\n';
}
return 0;
}
https://godbolt.org/z/nEbTvMhca
C++20 introduces coroutines, this could be a good solution too.

wait() for thread made via clone?

I plan on rewriting this to assembly so I can't use c or c++ standard library. The code below runs perfectly. However I want a thread instead of a second process. If you uncomment /*CLONE_THREAD|*/ on line 25 waitpid will return -1. I would like to have a blocking function that will resume when my thread is complete. I couldn't figure out by looking at the man pages what it expects me to do
#include <sys/wait.h>
#include <sched.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/mman.h>
int globalValue=0;
static int childFunc(void*arg)
{
printf("Global value is %d\n", globalValue);
globalValue += *(int*)&arg;
return 31;
}
int main(int argc, char *argv[])
{
auto stack_size = 1024 * 1024;
auto stack = (char*)mmap(NULL, stack_size, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS | MAP_STACK, -1, 0);
if (stack == MAP_FAILED) { perror("mmap"); exit(EXIT_FAILURE); }
globalValue = 5;
auto pid = clone(childFunc, stack + stack_size, /*CLONE_THREAD|*/CLONE_VM|CLONE_SIGHAND|SIGCHLD, (void*)7);
sleep(1); //So main and child printf don't collide
if (pid == -1) { perror("clone"); exit(EXIT_FAILURE); }
printf("clone() returned %d\n", pid);
int status;
int waitVal = waitpid(-1, &status, __WALL);
printf("Expecting 12 got %d. Expecting 31 got %d. ID=%d\n", globalValue, WEXITSTATUS(status), waitVal);
return 0;
}
If you want to call functions asynchronously with threads I recommend using std::async. Example here :
#include <iostream>
#include <future>
#include <mutex>
#include <condition_variable>
int globalValue = 0; // could also have been std::atomic<int> but I choose a mutex (to also serialize output to std::cout)
std::mutex mtx; // to protect access to data in multithreaded applications you can use mutexes
int childFunc(const int value)
{
std::unique_lock<std::mutex> lock(mtx);
globalValue = value;
std::cout << "Global value set to " << globalValue << "\n";
return 31;
}
int getValue()
{
std::unique_lock<std::mutex> lock(mtx);
return globalValue;
}
int main(int argc, char* argv[])
{
// shared memory stuff is not needed for threads
// launch childFunc asynchronously
// using a lambda function : https://en.cppreference.com/w/cpp/language/lambda
// to call a function asynchronously : https://en.cppreference.com/w/cpp/thread/async
// note I didn't ues the C++ thread class, it can launch things asynchronously
// however async is both a better abstraction and you can return values (and exceptions)
// to the calling thread if you need to (which you do in this case)
std::future<int> future = std::async(std::launch::async, []
{
return childFunc(12);
});
// wait until asynchronous function call is complete
// and get its return value;
int value_from_async = future.get();
std::cout << "Expected global value 12, value = " << getValue() << "\n";
std::cout << "Expected return value from asynchronous process is 31, value = " << value_from_async << "\n";
return 0;
}

boost::interprocess::interprocess_condition::timed_wait waits forever

I've the following minimal example:
#include <iostream>
#include <boost/thread.hpp>
#include <boost/interprocess/sync/interprocess_condition.hpp>
#include <boost/interprocess/sync/interprocess_mutex.hpp>
int main(int argc, char* argv[]) {
boost::interprocess::interprocess_condition ic;
boost::interprocess::interprocess_mutex im;
bool test = false;
auto testFunction = [&ic, &im, &test]() {
boost::unique_lock lk(im);
auto tin = boost::posix_time::microsec_clock::local_time();
auto waitTime = tin + boost::posix_time::milliseconds(100);
if (!ic.timed_wait(lk, waitTime)) {
test = false;
}
else {
test = true;
}
};
auto notifyFunction = [&ic]() {
ic.notify_all();
};
boost::thread t(testFunction);
boost::this_thread::sleep_for(boost::chrono::milliseconds(2000));
boost::thread t2(notifyFunction);
t.join();
t2.join();
std::cout << "Result is: " << std::boolalpha << test << std::endl;
return 0;
}
What I was expecting is that inside the lambda function the interprocess_condition variable ic waits for 100 milliseconds, and then the function continues. Instead the condition waits for the notify called in second thread.
What I' doing wrong? How should I use in correct way timed_wait in order to wait the desired amount of time?
I'm using boost version 1.72.0.
I've found the solution. I need to use boost::posix_time::microsec_clock::universal_time rather than boost::posix_time::microsec_clock::local_time.

Boost Timer in Qt Application

I'm trying to create qt application that uses a timer from boost,
I have a problem with the following code
Compiled on osx, qt 5.2, boost 1.55
This is my code:
CCTimer::CCTimer(int interval)
: m_pingTimer(m_ioServiceTimer, boost::posix_time::seconds(interval)), m_interval(interval)
{
m_isRunning = false;
}
CCTimer::~CCTimer()
{
Stop();
}
void CCTimer::Start()
{
if(!m_isRunning)
{
m_isRunning = true;
Reset(m_interval);
m_timerThread = boost::shared_ptr<boost::thread>(new boost::thread(boost::bind(&boost::asio::io_service::run, &m_ioServiceTimer)));
}
}
void CCTimer::Reset(int durationTime)
{
m_beforeTime = boost::posix_time::microsec_clock::universal_time();
m_pingTimer.expires_from_now(boost::posix_time::seconds(durationTime));
m_pingTimer.async_wait(boost::bind(&CCTimer::Wait, this, _1));
}
void CCTimer::Wait(const boost::system::error_code& errorCode)
{
boost::posix_time::time_duration duration = boost::posix_time::microsec_clock::universal_time() - m_beforeTime;
std::cout << boost::posix_time::microsec_clock::universal_time() << std::endl;
if(duration.seconds() > m_interval) {
std::cout << "Error time " << std::endl;
}
Reset(m_interval);
}
void CCTimer::Stop()
{
if(m_isRunning)
{
m_isRunning = false;
m_pingTimer.cancel();
m_ioServiceTimer.stop();
m_timerThread->join();
m_ioServiceTimer.reset();
}
}
int main(int argc, char *argv[])
{
QGuiApplication app(argc, argv);
CCTimer timer(1);
timer.Start();
return app.exec();
}
After creation of the timer in qt application, console shows:
2014-Mar-26 22:04:30.549722
2014-Mar-26 22:04:31.550977
2014-Mar-26 22:04:32.552229
2014-Mar-26 22:04:33.553467
2014-Mar-26 22:04:34.554734
2014-Mar-26 22:04:43.684300
Error time
2014-Mar-26 22:04:54.694440
Error time
2014-Mar-26 22:05:05.694371
Error time
2014-Mar-26 22:05:11.669329
Error time
what can be wrong ?
I haven't spotted anything glaring. Though, of course, you never mention whether/when/where you class Stop() and Start().
I made the implementation marginally simpler and fixed a few potential races:
no useless use of shared pointer for the time thread
make the CCTimer non-copyable
no more m_isRunning kludge; instead, intentionally cancel the timer and detect cancellation so you can end the thread. If the thread is joinable(), it's running
fixed a race condition where you cancel the timer from the thread that calls Stop(), whereas the timer is being async-awaited by the io_service thread, byt posting the cancel to the io_service thread too:
void Stop()
{
if(m_thread.joinable())
{
m_service.post(boost::bind(&deadline_timer::cancel, &m_timer));
m_thread.join();
m_service.stop();
m_service.reset();
}
}
fixed a cosmetic thing that could have led to confusion where you use a different timestamp for printing than you do for calculating the elapsed time
Update
So I downloaded Qt5 and goaded my compiler to use it. /GREAT SUCCESS/. And indeed, I got the same behaviour (but far worse, sometimes). The strange thing was it blocked more on the terminal than the eventual output would show. This made me suspect it's maybe Qt that's intercepting/reopening the stdin/stdout streams for it's own purposes and adding some blocking/buffering behaviour in the process.
I checked this by writing the timer trace to a separate file instead, and indeed this removed the symptoms (when left running for over hald an hour). So this is likely the cause of your issue. Just don't write to stdout from your timer thread and your spurious blocking issue should be gone.
Here's the code I used to test this, with the above suggestions incorporated:
#include <boost/date_time/posix_time/posix_time_io.hpp>
#include <boost/asio.hpp>
#include <boost/thread.hpp>
#include <boost/shared_ptr.hpp>
#include <QtGui/QGuiApplication>
#include <QtCore/QCoreApplication>
#include <fstream>
using namespace boost::asio;
namespace pt = boost::posix_time;
class CCTimer : boost::noncopyable
{
io_service m_service;
deadline_timer m_timer;
pt::ptime m_starttime;
int m_interval_seconds;
boost::thread m_thread;
std::ofstream ofs;
public:
CCTimer(int interval)
: m_service(),
m_timer(m_service),
m_interval_seconds(interval)
{
assert(m_interval_seconds>0);
}
~CCTimer()
{
Stop();
}
void Start()
{
if(!m_thread.joinable())
{
ofs.open("output.log");
Reset(m_interval_seconds);
m_thread = boost::thread(boost::bind(&io_service::run, &m_service));
}
}
void Stop()
{
if(m_thread.joinable())
{
m_service.post(boost::bind(&deadline_timer::cancel, &m_timer));
m_thread.join();
ofs.close();
m_service.stop();
m_service.reset();
}
}
private:
static pt::ptime now() { return pt::microsec_clock::universal_time(); }
void Reset(int durationTime)
{
m_timer.expires_from_now(pt::seconds(durationTime));
m_starttime = now();
m_timer.async_wait(boost::bind(&CCTimer::Elapsed, this, placeholders::error));
}
void Elapsed(const boost::system::error_code& errorCode)
{
pt::ptime const event = now();
pt::time_duration const elapsed = event - m_starttime;
if (errorCode != error::operation_aborted && ofs.is_open() && ofs.good())
{
ofs << event << " (" << elapsed.total_milliseconds() << "ms) ";
if(elapsed.seconds() > m_interval_seconds) {
ofs << " Overdue!" << std::endl;
} else {
ofs << "" << std::endl;
}
Reset(m_interval_seconds);
} else
{
ofs << "Stopped (" << elapsed.total_milliseconds() << "ms)" << std::endl;
}
}
};
int main(int argc, char** argv)
{
QGuiApplication app(argc, argv);
CCTimer timer(1);
timer.Start();
return app.exec();
}
Old suggestions:
I'd have to assume you're doing something in the rest of the code that inhibits the CCTimer thread
are you running under a debugger, profiler, whatnot?
are you running under a (overwhelmed) virtualizer?
are you doing more on the same io_service/thread?
can IO be blocking? In particular do you log the timestamps to an actual console or to a file/pipe? Could the output pipe be blocking the timer thread?

Implementing an event timer using boost::asio

The sample code looks long, but actually it's not so complicated :-)
What I'm trying to do is, when a user calls EventTimer.Start(), it will execute the callback handler (which is passed into the ctor) every interval milliseconds for repeatCount times.
You just need to look at the function EventTimer::Stop()
#include <iostream>
#include <string>
#include <boost/asio.hpp>
#include <boost/bind.hpp>
#include <boost/thread.hpp>
#include <boost/function.hpp>
#include <boost/date_time/posix_time/posix_time.hpp>
#include <ctime>
#include <sys/timeb.h>
#include <Windows.h>
std::string CurrentDateTimeTimestampMilliseconds() {
double ms = 0.0; // Milliseconds
struct timeb curtime;
ftime(&curtime);
ms = (double) (curtime.millitm);
char timestamp[128];
time_t now = time(NULL);
struct tm *tp = localtime(&now);
sprintf(timestamp, "%04d%02d%02d-%02d%02d%02d.%03.0f",
tp->tm_year + 1900, tp->tm_mon + 1, tp->tm_mday, tp->tm_hour, tp->tm_min, tp->tm_sec, ms);
return std::string(timestamp);
}
class EventTimer
{
public:
static const int kDefaultInterval = 1000;
static const int kMinInterval = 1;
static const int kDefaultRepeatCount = 1;
static const int kInfiniteRepeatCount = -1;
static const int kDefaultOffset = 10;
public:
typedef boost::function<void()> Handler;
EventTimer(Handler handler = NULL)
: interval(kDefaultInterval),
repeatCount(kDefaultRepeatCount),
handler(handler),
timer(io),
exeCount(-1)
{
}
virtual ~EventTimer()
{
}
void SetInterval(int value)
{
// if (value < 1)
// throw std::exception();
interval = value;
}
void SetRepeatCount(int value)
{
// if (value < 1)
// throw std::exception();
repeatCount = value;
}
bool Running() const
{
return exeCount >= 0;
}
void Start()
{
io.reset(); // I don't know why I have to put io.reset here,
// since it's already been called in Stop()
exeCount = 0;
timer.expires_from_now(boost::posix_time::milliseconds(interval));
timer.async_wait(boost::bind(&EventTimer::EventHandler, this));
io.run();
}
void Stop()
{
if (Running())
{
// How to reset everything when stop is called???
//io.stop();
timer.cancel();
io.reset();
exeCount = -1; // Reset
}
}
private:
virtual void EventHandler()
{
// Execute the requested operation
//if (handler != NULL)
// handler();
std::cout << CurrentDateTimeTimestampMilliseconds() << ": exeCount = " << exeCount + 1 << std::endl;
// Check if one more time of handler execution is required
if (repeatCount == kInfiniteRepeatCount || ++exeCount < repeatCount)
{
timer.expires_at(timer.expires_at() + boost::posix_time::milliseconds(interval));
timer.async_wait(boost::bind(&EventTimer::EventHandler, this));
}
else
{
Stop();
std::cout << CurrentDateTimeTimestampMilliseconds() << ": Stopped" << std::endl;
}
}
private:
int interval; // Milliseconds
int repeatCount; // Number of times to trigger the EventHandler
int exeCount; // Number of executed times
boost::asio::io_service io;
boost::asio::deadline_timer timer;
Handler handler;
};
int main()
{
EventTimer etimer;
etimer.SetInterval(1000);
etimer.SetRepeatCount(1);
std::cout << CurrentDateTimeTimestampMilliseconds() << ": Started" << std::endl;
etimer.Start();
// boost::thread thrd1(boost::bind(&EventTimer::Start, &etimer));
Sleep(3000); // Keep the main thread active
etimer.SetInterval(2000);
etimer.SetRepeatCount(1);
std::cout << CurrentDateTimeTimestampMilliseconds() << ": Started again" << std::endl;
etimer.Start();
// boost::thread thrd2(boost::bind(&EventTimer::Start, &etimer));
Sleep(5000); // Keep the main thread active
}
/* Current Output:
20110520-125506.781: Started
20110520-125507.781: exeCount = 1
20110520-125507.781: Stopped
20110520-125510.781: Started again
*/
/* Expected Output (timestamp might be slightly different with some offset)
20110520-125506.781: Started
20110520-125507.781: exeCount = 1
20110520-125507.781: Stopped
20110520-125510.781: Started again
20110520-125512.781: exeCount = 1
20110520-125512.781: Stopped
*/
I don't know why that my second time of calling to EventTimer::Start() does not work at all. My questions are:
What should I do in
EventTimer::Stop() in order to reset
everything so that next time of
calling Start() will work?
Is there anything else I have to modify?
If I use another thread to start the EventTimer::Start() (see the commented code in the main function), when does the thread actually exit?
Thanks.
Peter
As Sam hinted, depending on what you're attempting to accomplish, most of the time it is considered a design error to stop an io_service. You do not need to stop()/reset() the io_service in order to reschedule a timer.
Normally you would leave a thread or thread pool running attatched to an io_service and then you would schedule whatever event you need with the io_service. With the io_service machinery in place, leave it up to the io_service to dispatch your scheduled work as requested and then you only have to work with the events or work requests that you schedule with the io_service.
It's not entirely clear to me what you are trying to accomplish, but there's a couple of things that are incorrect in the code you have posted.
io_service::reset() should only be invoked after a previous invocation of io_service::run() was stopped or ran out of work as the documentation describes.
you should not need explicit calls to Sleep(), the call to io_service::run() will block as long as it has work to do.
I figured it out, but I don't know why that I have to put io.reset() in Start(), since it's already been called in Stop().
See the updated code in the post.