I have hard time finding out how exactly does thread pool built with boost::asio::io_service behave.
The documentation says:
Multiple threads may call the run() function to set up a pool of
threads from which the io_service may execute handlers. All threads
that are waiting in the pool are equivalent and the io_service may
choose any one of them to invoke a handler.
I would imagine, that when threads executing run() are taking a handler to execute, they execute it, and then come back to wait for next handlers to execute. When executing a handler, a thread is not considered waiting, and hence no new handlers to execute are assigned to it. Is that correct? Or does io_service assign work to threads, without considering whether these are busy or not?
I am asking, because in one project that we are using (OSRM), that uses boost::asio::io_service based thread pool to handle incoming HTTP requests, I noticed that long running request, sometimes block other, fast requests, even though more threads and cores are available.
When executing a handler, a thread is not considered waiting, and hence no new handlers to execute are assigned to it. Is that correct?
Yes. It's a pull model queue.
A notable "apparent" exception is when strands are used: handlers wrapped on a on a strand do synchronize with other handlers running on that same strand.
Related
The common way to process Asio handlers is to have a thread (or several threads) either polling io_service (i.e. calling io_service::poll()) regularly to run the handlers or using io_service::run(), which blocks the thread until there's work to do, in which case the thread will run the required handlers and either return or go to sleep again.
However, I want to make a system where a thread is not only responsible for running Asio handlers, but also needs to sync up with another thread using a condition variable. Basically, I want the thread to do all of these:
Wake up when there are Asio handlers that need to be processed (i.e. if I call io_service::poll(), one or more handlers will be processed).
Wake up when there is non-Asio work to be done, indicated by my condition variable.
Sleep otherwise.
In other words, I need a way for Asio to signal me that there are handlers ready to execute, without having to busy-wait or continuously poll. Ideally, Asio will somehow signal a thread when work is available, and that thread will in turn wake up my main worker thread, which will process Asio handlers. That worker thread will also be occasionally woken up by yet another thread, and will process other, non-Asio related work.
Is this even feasible, or should I reconsider how I am designing my system?
I have a thread pool made with boost::asio::io_service from recipe like this. I wonder how to interrupt posted into it tasks (not killing threads), so that they would be replaced with next tasks in pool?
Callbacks in boost::asio in general should be reasonably fast. They should just do some work, schedule another one and finish. This will be the points, where task looses CPU and other task will be taken into execution.
There are boost::thread::interrupt() and boost::thread_group::interrupt_all(), but they could stop execution thread only at interruption points. The interruption will be seen as exception boost::thread_interrupted. This means, that you have to handle exception somehow and in your case - release current task. It is much more complicated, than just doing one step of processing and scheduling handler for the rest.
Moreover, you could play with interrupt() and interrupt_all() in threads which do execute your own routine, while this is not the case for threads running io_service::run(). One can imagine, that boost::thread_interrupted is being thrown inside run() method, instead of async handler, which could end with unexpected behavior.
Is I am looking at writing a multithreaded tcp server using boost ASIO. I have read through the tutorials and had a look at some of the examples and just want to check that my understanding is correct.
The server will accept connections then service requests from multiple clients.
My understanding is as follows:
The server uses "a single io_service and a thread pool calling io_service::run()"
All threads call io_service::run().
The calls to io_service::run() are not within a strand, ergo completion handlers can run simultaneously.
When a request arrives one of the threads is chosen, its read handler will be called
Another request may arrive,starting the read handler on a second thread
When one of the threads has finished handling the request it calls async_write, from within a strand
Another thread also finishes processing its request, it also calls async_write, from within a strand
The writes to the io_service are serialised via the strand, ergo they are thread safe.
When the write operation completes the thread calls async_read()
This call is not protected by a strand and the thread will be used for handling requests
Is my understanding correct? Is this solution vulnerable to race conditions?
As Sam miller said, your assumptions are quite correct.
However I would like to point out an issue that you may have not spotted.
It is right that strands will serialize async_write(s) and therefore there will be thread safe.
But the issue is not here, async_write is by itself thread safe if not used on the same socket. And strands will not help here since you should not interleave async_write on the same socket.
Strands will not wait the previous async_write to finish before calling the next one. you will have to create a structure that async_write only if none is already in action on the socket.
I am reading multicast input using async_receive_from. So the idea is that when I detect a gap, I will notify another helper thread to request/get the gap filling messages. While this is in the works the main thread will continue to receive and queue any incoming messages. This part I can implement. The other thread can use waitforsingleobject and I can pass it the details through shared memory and notify an event to wake it up.
But once it completes it task, how do I get the helper thread to interrupt the async_receive_from in the initiating thread? And when it comes up out of the the read it knows who interrupted so it will then know what to do next?
Why are you using shared memory between threads?
That aside, the mechanism you should use for executing something in the context of the io_service which is managing the socket is post(). You can post any arbitrary event to the io_service, and it will execute in that context. Quite easy really... Because you are calling async_receive_from, it's not blocking, i.e. the io_service can dispatch other events, which is why the post will work.
I was just going over the asio chat server example. My question is about their usage of the io_service.run() function. The documentation for the io_service.run() function says:
The run() function blocks until all work has finished and there are no
more handlers to be dispatched, or until the io_service has been
stopped. Multiple threads may call the run() function to set up a
pool of threads from which the io_service may execute handlers. All
threads that are waiting in the pool are equivalent and the io_service
may choose any one of them to invoke a handler. The run() function
may be safely called again once it has completed only after a call to
reset().
It says that the run function will return, and I'm assuming that when it does return the network thread stops until it is called again. If that is true, then why isn't the run function called in a loop, or at least given its own thread? the io_service.run() function is pretty much a mystery to me.
"until all work has finished and there are no more handlers to be dispatched, or until the io_service has been stopped"
Notice that you DO install a handler, named handle_accept, that reinstalls itself at each execution. Hence, the io_service.run will never return, at least until you quit it manually.
Basically, at the moment you run io_service.run in a thread, io_services proactor takes over program flow, using the handler's you installed. From that point on, you handle the program based on events (like the handle_accept) instead of normal procedural program flow. The loop you're mentioning is somewhere deep in the scary depths of the asio's proactor ;-).