The following schema come from boost asio documentation:
I understand that if I call io_service::run method twice (in two separate threads), I will have two threads to deque events from the completion Event Queue via Asynchronous Event Demultiplexer am I right?
More precisely, my doubt is on the parrallelization achieve by multiple call of io_service::run method. For instance when dealing with socket, if for example I have two sockets bound on the same io_service object, each socket calling socket.async_read_some method, does it involved the 2 registered callbacks (via async_read_some method) can be called concurently when calling io_service::run twice.
Your assumptions are correct. Each thread which calls io_service::run() will dequeue and execute handlers (simple function objects) in parallel. This of course only makes sense if you have more than one source of events feeding the io_service (such as two sockets, a socket and a timer, several simultaneous post() calls and so on).
Each call to a socket's async_read() will result in exactly one handler being queued in the io_service. Only one of your threads will dequeue it and execute it.
Be careful not to call async_read() more than once at a time per socket.
Related
I am looking at the Boost Asio Blocking TCP Client timeout example with a special interest on how connection timeouts are implmented. How do we know from the documentation that the callback handler and subsequent checks don't introduce a race condition?
The Asynchronous connection command
boost::asio::async_connect(socket_, iter, var(ec) = _1);
executes the var(ec) = _1 which is the handler for setting the error code once execute. Alternatively, a full and explicit lambda could be used here.
At the same time, the check_deadline function appears to be
called by the deadline_ member. The timeout appears to be enforced by having the deadline forcibly close the socket whereup we assume that perhaps that the blocking statement
do io_service_.run_one(); while (ec == boost::asio::error::would_block);
would return. At first I thought that the error code must be atomic but that doesn't appear to be the case. Instead, this page, appears to indicate that the strand model will work whenever the calls to the socket/context come from the same thread.
So we assume that each callback for the deadline (which is in Asio) and the handle for the async_connect routine will not be run concurrently. Pages such as this in the documentation hint that handlers will only execute during run() calls which will prevent the command while(ec == whatever) from behind executed during the handler currently changing its value.
How do I know this explicitly? What in the documentation that tells me explicitly that no handlers will ever execute outside these routines? If true, the page on the proactor design pattern must infer this, but never explicitly where the "Initiator" leads to the "Completion Handler".
The closes I've found is the documentation for io_context saying
Synchronous operations on I/O objects implicitly run the io_context
object for an individual operation. The io_context functions run(),
run_one(), run_for(), run_until(), poll() or poll_one() must be called
for the io_context to perform asynchronous operations on behalf of a
C++ program. Notification that an asynchronous operation has completed
is delivered by invocation of the associated handler. Handlers are
invoked only by a thread that is currently calling any overload of
run(), run_one(), run_for(), run_until(), poll() or poll_one() for the
io_context.
This implies that if I have one thread running the run_one() command then its control path will wait until a handler is available and eventually wind its way through a handler whereupon it will return and check ther ec value.
Is this correct and is "Handlers are invoked only by a thread that is currently calling any overload of run(), run_one(), run_for(), run_until(), poll() or poll_one() for the io_context." the best statement to find for understanding how the code will always function? Is there any other exposition?
The Asio library is gearing up to be standardized as NetworkingTS. This part is indeed the deal:
Handlers are invoked only by a thread that is currently calling any overload of run(), run_one(), run_for(), run_until(), poll() or poll_one() for the io_context
You are correct in concluding that the whole example is 100% single-threaded¹. There cannot be a race.
I personally feel the best resource is the Threads and Boost.Asio page:
By only calling io_context::run() from a single thread, the user's code can avoid the development complexity associated with synchronisation. For example, a library user can implement scalable servers that are single-threaded (from the user's point of view).
It also reiterates the truth from earlier:
[...] the following guarantee:
Asynchronous completion handlers will only be called from threads that are currently calling io_context::run().
¹ except potential internal service threads depending on platform/extensions, as the threads page details
I'm currently trying to get my hands on boost::asio strands. Doing so, I keep reading about "invoking strand post/dispatch inside or outside a strand". Somehow I can't figure out how inside a strand differs from through a strand, and therefore can't grasp the concept of invoking a strand function outside the strand at all.
Probably there is just a small piece missing in my puzzle. Can somebody please give an example how calls to a strand can be inside or outside it?
What I think I've understood so far is that posting something through a strand would be
m_strand.post(myfunctor);
or
m_strand.wrap(myfunctor);
io_svc.post(myfunctor);
Is the latter considered a call to dispatch outside the strand (as opposed to the other being a call to post inside it)? Is there some relation between the strand's "inside realm" and the threads the strand operates on?
If being inside a strand simply meant to invoke a strand's function, then the strand class's documentation would be pointless. It states that strand::post can be invoked outside the strand... That's precisely the part I don't understand.
Even I had some trouble in understanding this concept, but became clear once I started working on libdispatch. It helped me map things with asio better.
Now lets see how to make some sense out of strand. Consider strand as a serial queue of handlers which needs to be executed.
Now, where does these handlers get executed ? Within the worker threads.
Where did these worker threads come from ? From the io_service object you passed while creating the strand.
Something like:
asio::strand s(io_serv_obj);
Now, as you must be knowing, the io_service::run can be called by a single thread or multiple threads. The threads calling the run method of the io_serv_obj are the worker threads for that strand in our case. So, it could be either single threaded or multithreaded.
Coming back to strands, when you post a handler, that handler is always enqueued in the serial queue which we talked about. The worker threads will pick up the handler from the queue one after the other.
Now, when you do a dispatch, asio does some optimization for you:
It checks whether you are calling it from inside one of the worker thread or from some other thread (maybe of some other io_service instance). When it is called outside the current execution context of the strand, thats when it is called outside the strand. So, in the outside case, the dispatch will just enqueue the handler like post when there are other handlers waiting in the queue or will call it directly when it can guarantee that it will not be called concurrently with any other handler from that queue that may be running in one of the worker threads at that moment.
UPDATE:
As noted in the comments section, inside means called within another handler i.e for eg: I posted a handler A and inside that handler, I am doing a dispatch of another handler. Now, as would be explained in #2, if there are no other handlers waiting in the strands serial queue, the dispatch handler will be called synchronously. If this condition is not met, that means, the dispatch is called from outside.
Now, if you call dispatch from outside of the strand i.e not within the current execution context, asio checks its callstack to see if any other handler present in its serial queue is running or not. If not, then it will directly call that handler synchronously. So, there is no cost of enqueueing the handler (I think no extra allocation will be done as well, not sure though).
Lets see the documentation link now:
s.dispatch(a) happens-before s.post(b), where the former is performed
outside the strand
This means that, if dispatch was called from some outside the current run OR there are other handlers already enqueued, then it needs to enqueue the handler, it just cannot call it synchronously. Since its a serial queue, a will get executed before b.
Had there been another call s.dispatch(c) along with a and b but before a and b(in the mentioned order) enqueued, then c will get executed before a and b, but in no way b can get executed before a.
Hope this clears your doubt.
For a given strand object s, running outside s implies that s.running_in_this_thread() returns false. This returns true if the calling thread is executing a handler that was submitted to the strand via post(), dispatch(), or wrap(). Otherwise, it returns false:
io_service.post(handler); // handler will run outside of strand
strand.post(handler); // handler will run inside of strand
strand.dispatch(handler); // handler will run inside of strand
io_service.post(strand.wrap(handler)); // handler will run inside of strand
Given:
a strand object s
a function object f1 that is added to strand s via s.post(), or s.dispatch() when s.running_in_this_thread() == false
a function object f2 that is added to strand s via s.post(), or s.dispatch() when s.running_in_this_thread() == false
then the strand provides a guarantee of ordering and non-concurrency, such that f1 and f2 will not be invoked concurrently. Furthermore, if the addition of f1 happens before the addition of f2, then f1 will be invoked before f2.
io_service::run() is called by thread A. Is it safe to call async_write from thread B?
io_service::run() is called by thread A. Are async operations executed by thread A, or is thread A only guaranteed to call handlers and behind the scenes there could be additional threads that execute the operations?
io_service::run() is called by thread A. Some thread calls async_read and async_write using the same buffer. Is it safe to assume that the buffer will be accessed by at most one operation at a time? Or is it so that only handlers are called serially, but behind the scenes reads and writes can occur simultaneously?
The documentation says "The program must ensure that the stream performs no other read operations (such as async_read, the stream's async_read_some function, or any other composed operations that perform reads) until this operation completes.". Is it correct to interpret this as "You must not perform more than one read operation on a socket at a time. But you may perform 10 read operations on 10 distinct sockets."?
Having a socket that indefinitely accepts data, is it a good idea to call async_read and call it again from async_read's handler?
Does io_service::stop() stop all pending async operations or simply stops accepting new ones and executes the pending ones?
Yes, providing the io_service is tied to whatever is calling async_write. However, it should be noted that it is safe to call async_write from thread B even if the run is not called: it'll get queued in the io_service and wait until one of the run-ing calls are completed.
The callbacks posted to the io_service will run on thread A. Other async operations (such as timer operations) can happen on other threads. What is guarenteed to be on A and what is on its own thread is defined by the specific object being used, not by io_service.
Nope. Yup-ish. Depends on the class calling io_service.
Yes.
Yes, in fact this is super common, as it both ensures that only 1 async_read call is running at a time for a given socket and that there is always "work" for the io_service.
It usually finished the last callback and then stops accepting new ones and stops processing pending ones. It actually still accepts new ones but forces a reset is called before any other callbacks are called.
io_service is a message queue (basically), while a socket that posts its messages to the io_service is something else entirely.
1: Yes
4: Yes, it's okay to perform distinct operations on distinct sockets.
5: Yes, if you check the examples that's how they do it.
6: Considering the reference manual says
All invocations of its run() or run_one() member functions should return as soon as possible.
I would say it might do any.
For number 2 and 6, the source is available so the best way to answer those question is by downloading and reading it.
I have a method that launches a new std::thread for new connections so that I can read data and do other things.
The method the thread invokes runs the reads in an asynchronous way(using boost functions) and it returns once it calls async_read_some, my question is:
What thread handles the call-back? Is it the same thread that made the call to the async_read_some or did that thread die after it called it and returned and now the main thread is handling the reads?
Here's a code snippet:
connection::connection_thread = std::thread(&connection::read_header,
this);
connection::connection_thread.detach();
.
.
.
void connection::read_header() {
socket_.async_read_some(boost::asio::buffer(headbuf_),
strand_.wrap(
boost::bind(&connection::on_header_read, shared_from_this(),
boost::asio::placeholders::error,
boost::asio::placeholders::bytes_transferred)));
begin_timeout();
}
What thread handles the call-back?
The thread (or one of the threads, if there are more than one) which polls or runs the associated io_service. The handler is passed to the service to be called on completion.
Is is the same thread that made the call to the async_read_some
No, the async functions never call the handler directly; it is always called by the io_service, even if the operation completes immediately.
or did that thread die after it called it and returned and now the main thread is handling the reads?
That entirely depends on how you're managing the threads. The thread that calls async may die, if you don't need it any more; you'll need some other thread or threads (possibly the main thread, possibly others) to process the io_service and complete the asynchronous operation.
However, there's no point launching a thread to start an asynchronous operation, since that will complete immediately. Either move the call to async_read_some to where you're currently launching the thread; or use the thread to perform a synchronous operation. If you opt for a multithreaded synchronous design, then you won't need a thread to poll the io_service for asynchronous operations.
I am using (single threaded) a boost::asio:io_service to handle a lot of tcp connections. For each connection I use a deadline_timer to catch timeouts. If any of the connections times out, I can use none of the results of the other connections. Therefore I want to completely restart my io_service. I thought that calling io_service.stop() would allow "finished" handlers in the queue to be called and would call handlers in the queue with an error.
However it looks like the handlers remain in the queue and therefore calling io_service.reset() and later io_service.run() brings the old handlers back up.
Can anyone confirm that the handlers indeed remain in the queue even after io_service.stop() is called. And if so, what are the possibilities to completly reset the io_service, e.g. remove all queued handlers?
io_service::stop() and io_service::reset() only control the state of the io_service's event loop; neither affect the lifespan of handlers scheduled for deferred invocation (ready-to-run) or user-defined handler objects.
The destructor for io_service will cause all outstanding handlers to be destroyed:
Each service object associated with the io_service will have its shutdown_service() member function invoked. Per the Service type requirement, the shutdown_service() member function will destroy all copies of user-defined handler objects that are held by the service.
Uninvoked handler objects scheduled for deferred invocation are destroyed for the io_service and any of its strands.
Consider either:
Controlling the lifespan of the io_service object. One approach can be found in this answer.
Running the io_service to completion. This often requires setting state, cancelling outstanding operations, and preventing completion handlers from posting additional work into the io_service. Boost.Asio provides an official timeout example, and a timeout approach with running to the io_service to completion is also shown here.