io_service::run() is called by thread A. Is it safe to call async_write from thread B?
io_service::run() is called by thread A. Are async operations executed by thread A, or is thread A only guaranteed to call handlers and behind the scenes there could be additional threads that execute the operations?
io_service::run() is called by thread A. Some thread calls async_read and async_write using the same buffer. Is it safe to assume that the buffer will be accessed by at most one operation at a time? Or is it so that only handlers are called serially, but behind the scenes reads and writes can occur simultaneously?
The documentation says "The program must ensure that the stream performs no other read operations (such as async_read, the stream's async_read_some function, or any other composed operations that perform reads) until this operation completes.". Is it correct to interpret this as "You must not perform more than one read operation on a socket at a time. But you may perform 10 read operations on 10 distinct sockets."?
Having a socket that indefinitely accepts data, is it a good idea to call async_read and call it again from async_read's handler?
Does io_service::stop() stop all pending async operations or simply stops accepting new ones and executes the pending ones?
Yes, providing the io_service is tied to whatever is calling async_write. However, it should be noted that it is safe to call async_write from thread B even if the run is not called: it'll get queued in the io_service and wait until one of the run-ing calls are completed.
The callbacks posted to the io_service will run on thread A. Other async operations (such as timer operations) can happen on other threads. What is guarenteed to be on A and what is on its own thread is defined by the specific object being used, not by io_service.
Nope. Yup-ish. Depends on the class calling io_service.
Yes.
Yes, in fact this is super common, as it both ensures that only 1 async_read call is running at a time for a given socket and that there is always "work" for the io_service.
It usually finished the last callback and then stops accepting new ones and stops processing pending ones. It actually still accepts new ones but forces a reset is called before any other callbacks are called.
io_service is a message queue (basically), while a socket that posts its messages to the io_service is something else entirely.
1: Yes
4: Yes, it's okay to perform distinct operations on distinct sockets.
5: Yes, if you check the examples that's how they do it.
6: Considering the reference manual says
All invocations of its run() or run_one() member functions should return as soon as possible.
I would say it might do any.
For number 2 and 6, the source is available so the best way to answer those question is by downloading and reading it.
Related
I am writing an application using boost.asio. I've an object of type boost::asio::ip::tcp::socket and (of course) I've boost::asio::io_context which run's function was called from only one thread. For writing data to the socket there are a couple of ways but currently I use socket's function async_write_some, something like the code below:
void tcp_connection::write(packet_ptr packet)
{
m_socket.async_write_some(boost::asio::buffer(packet->data(), packet->size()),
std::bind(&tcp_connection::on_write, this, std::placeholders::_1, std::placeholders::_2, packet));
}
There is another function in boost::asio namespace - async_write. And the documentation of async_write says:
This operation is implemented in terms of zero or more calls to the stream's async_write_some function, and is known as a composed operation. The program must ensure that the stream performs no other write operations (such as async_write, the stream's async_write_some function, or any other composed operations that perform writes) until this operation completes.
In async_write_some's documentation there is no such kind of 'caution'.
That's a little bit confusing to me and here I've got the following questions:
Is it safe to call async_write_some without waiting for the previous call to be finished? As far as I understood from boost's documentation I shouldn't do that with async_write, but what about async_write_some?
If yes, is the order in which the data is written to the socket the same as the functions were called? I mean if I called async_write_some(packet1) and async_write_some(packet2) - are the packets going to be written to the socket in the same order?
Which function I should use? What is the difference between them?
What is the reason that it's not safe to call async_write while the previous one hasn't finished yet?
no; the reason for that is probably documented with the underlying sockets API (BSD/WinSock).
not applicable. Note that the order in which handlers are invoked is guaranteed to match the order in which they were posted, so you could solve it using an async chain of async_write_some calls where the completion handler posts the next write. This is known as an implicit strand (see https://www.boost.org/doc/libs/master/doc/html/boost_asio/overview/core/async.html and Why do I need strand per connection when using boost::asio?).
99% of the time, use the free function. The difference is that it implements composed operation to send a "unit" of information, i.e. an entire buffer, message, or until a given completion condition is met.
async_write_some is the lowest-level building block, which doesn't even guarantee to write all of the data: remarks:
The write operation may not transmit all of the data to the peer.
Consider using the async_write function if you need to ensure that all
data is written before the asynchronous operation completes.
It's not unsafe¹ in the strictest sense. It just will not lead to correct results: this is because the order in which handlers are invoked leads to data being written to the socket in mixed-up order.
¹(unless you access the shared IO objects concurrently without synchronization)
I'm currently trying to get my hands on boost::asio strands. Doing so, I keep reading about "invoking strand post/dispatch inside or outside a strand". Somehow I can't figure out how inside a strand differs from through a strand, and therefore can't grasp the concept of invoking a strand function outside the strand at all.
Probably there is just a small piece missing in my puzzle. Can somebody please give an example how calls to a strand can be inside or outside it?
What I think I've understood so far is that posting something through a strand would be
m_strand.post(myfunctor);
or
m_strand.wrap(myfunctor);
io_svc.post(myfunctor);
Is the latter considered a call to dispatch outside the strand (as opposed to the other being a call to post inside it)? Is there some relation between the strand's "inside realm" and the threads the strand operates on?
If being inside a strand simply meant to invoke a strand's function, then the strand class's documentation would be pointless. It states that strand::post can be invoked outside the strand... That's precisely the part I don't understand.
Even I had some trouble in understanding this concept, but became clear once I started working on libdispatch. It helped me map things with asio better.
Now lets see how to make some sense out of strand. Consider strand as a serial queue of handlers which needs to be executed.
Now, where does these handlers get executed ? Within the worker threads.
Where did these worker threads come from ? From the io_service object you passed while creating the strand.
Something like:
asio::strand s(io_serv_obj);
Now, as you must be knowing, the io_service::run can be called by a single thread or multiple threads. The threads calling the run method of the io_serv_obj are the worker threads for that strand in our case. So, it could be either single threaded or multithreaded.
Coming back to strands, when you post a handler, that handler is always enqueued in the serial queue which we talked about. The worker threads will pick up the handler from the queue one after the other.
Now, when you do a dispatch, asio does some optimization for you:
It checks whether you are calling it from inside one of the worker thread or from some other thread (maybe of some other io_service instance). When it is called outside the current execution context of the strand, thats when it is called outside the strand. So, in the outside case, the dispatch will just enqueue the handler like post when there are other handlers waiting in the queue or will call it directly when it can guarantee that it will not be called concurrently with any other handler from that queue that may be running in one of the worker threads at that moment.
UPDATE:
As noted in the comments section, inside means called within another handler i.e for eg: I posted a handler A and inside that handler, I am doing a dispatch of another handler. Now, as would be explained in #2, if there are no other handlers waiting in the strands serial queue, the dispatch handler will be called synchronously. If this condition is not met, that means, the dispatch is called from outside.
Now, if you call dispatch from outside of the strand i.e not within the current execution context, asio checks its callstack to see if any other handler present in its serial queue is running or not. If not, then it will directly call that handler synchronously. So, there is no cost of enqueueing the handler (I think no extra allocation will be done as well, not sure though).
Lets see the documentation link now:
s.dispatch(a) happens-before s.post(b), where the former is performed
outside the strand
This means that, if dispatch was called from some outside the current run OR there are other handlers already enqueued, then it needs to enqueue the handler, it just cannot call it synchronously. Since its a serial queue, a will get executed before b.
Had there been another call s.dispatch(c) along with a and b but before a and b(in the mentioned order) enqueued, then c will get executed before a and b, but in no way b can get executed before a.
Hope this clears your doubt.
For a given strand object s, running outside s implies that s.running_in_this_thread() returns false. This returns true if the calling thread is executing a handler that was submitted to the strand via post(), dispatch(), or wrap(). Otherwise, it returns false:
io_service.post(handler); // handler will run outside of strand
strand.post(handler); // handler will run inside of strand
strand.dispatch(handler); // handler will run inside of strand
io_service.post(strand.wrap(handler)); // handler will run inside of strand
Given:
a strand object s
a function object f1 that is added to strand s via s.post(), or s.dispatch() when s.running_in_this_thread() == false
a function object f2 that is added to strand s via s.post(), or s.dispatch() when s.running_in_this_thread() == false
then the strand provides a guarantee of ordering and non-concurrency, such that f1 and f2 will not be invoked concurrently. Furthermore, if the addition of f1 happens before the addition of f2, then f1 will be invoked before f2.
The following schema come from boost asio documentation:
I understand that if I call io_service::run method twice (in two separate threads), I will have two threads to deque events from the completion Event Queue via Asynchronous Event Demultiplexer am I right?
More precisely, my doubt is on the parrallelization achieve by multiple call of io_service::run method. For instance when dealing with socket, if for example I have two sockets bound on the same io_service object, each socket calling socket.async_read_some method, does it involved the 2 registered callbacks (via async_read_some method) can be called concurently when calling io_service::run twice.
Your assumptions are correct. Each thread which calls io_service::run() will dequeue and execute handlers (simple function objects) in parallel. This of course only makes sense if you have more than one source of events feeding the io_service (such as two sockets, a socket and a timer, several simultaneous post() calls and so on).
Each call to a socket's async_read() will result in exactly one handler being queued in the io_service. Only one of your threads will dequeue it and execute it.
Be careful not to call async_read() more than once at a time per socket.
I have a C++ program that uses overlapped IO for network communication. The main thread has a loop that calls SleepEx(5, true);. There are also two TCP sockets. I assume that the completion callbacks are called during the alertable wait. Assume also that by the time SleepEx gets called both of my TCP connections have received some data. Now the question is what happens if the first completion callback takes longer than 5ms? Does the SleepEx return after calling the first callback or does it also call the second callback? In other words does the SleepEx guarantee to call ALL of the scheduled completion callbacks? This is not clear because the documentation says it will return when at least one of the events meet...
Your code must not assume that both APCs will be called before SleepEx() returns. Conversely, it must not assume that a pending APC will not be called simply because the specified wait period has expired.
The only behaviour that you can rely upon is that if one or more APCs are pending, at least one will be executed.
Generally speaking, best practice is to wait for APCs in a loop that does nothing else, using an infinite timeout in the wait. If you need to do something periodically, you can use a waitable timer to generate an APC periodically.
Alternatively, you can use WaitForSingleObjectEx() or WaitForMultipleObjectsEx() to detect when a waitable timer or other synchronization object is triggered, while still handling APCs.
However, if you must perform some periodic action that cannot be handled in an APC or be triggered by a synchronization object, you can use nested loops: the inner loop does nothing but call the wait repeatedly (with a timeout period reduced by however long the loop has already been running) and the outer loop performs the periodic action.
If you must perform some periodic action that cannot be delayed by pending APCs, you will need to do it in a separate thread. Note that because Windows is not a real-time OS, you will still not be able to guarantee that any given action will take place within any particular timeframe, although you can reduce the risk by increasing the thread priority.
I'm writing some code with boost::asio, using asynchronous TCP connections. I've to admit that I have some doubts about it. All these regarding concurrency. Here are some:
What happens if I start two or more async_write on the same socket without waiting completion of the first one? Will the handlers (and the async_write) overlap or asio provides serialization and synchronization?
Same question of above with async_connect and async_read. In general is it safe to call these functions from different threads (I'm not talking about using different buffers, that's another problem...).
I assume from your question that you have a single instance of io_service and you want to call async_write() on it from multiple threads.
async_write() ultimately calls the post() method of io_service, which in turn takes a lock and pushes the bits to be written into a work queue, ensuring that the bits won't be written interleaved. Those bits will eventually get written out and the underlying data structure that holds them (a char array or whatever) must remain valid until you get the callback signifying that the write has completed. If you are using the exact same callback function as your completion handler, you will have no way of knowing which of the two writes resulted in that function being called and if that function does anything not thread-safe, behavior may be undefined or incorrect. A popular way to handle this situation is to have a instance of a struct that is the completion handler (just overload the call () operator): you can set the properties of the struct to denote which write it corresponds to and then consult these values when the completion handler is called.
However, absent a shared lock, you have no way of controlling which of the threads actually executes its async_write() method. In fact, even if you start up two threads and have one thread immediately call async_write() and have the other sleep for an hour and then call async_write(), you are still not assured that the OS didn't schedule your threads stupidly and execute the second thread's call first. (The example is pathological but the point is universally valid.)
The same situation applies to async_read(). You certainly can interleave calls (ie do one async_read() and then another before the completion handler is called) but there is no guarantee that the will execute in the order you intend without some external means to ensure this.
If you start two asynchronous operations on the same socket, they could occur in either order. As long as they are using different buffers it's "safe", in that it won't crash, but such behavior is almost never what you want.