TLDR: Strands serialise resources shared across completion handlers: how does that prevent the ssl::stream implementation from concurrent access of the SSL context (used internally) for concurrent read/write requests (stream::ssl is not full duplex)? Remember, strands only serialise the completion handler invocation or the original queueing of the read/write requests. [Thanks to sehe for helping me express this better]
I've spent most of a day reading about ASIO, SSL and strands; mostly on stackoverflow (which has some VERY detailed and well expressed explanations, e.g. Why do I need strand per connection when using boost::asio?), and the Boost documentation; but one point remains unclear.
Obviously strands can serialise invocation of callbacks within the same strand, and so also serialise access to resources shared by those strands.
But it seems to me that the problem with boost::asio::ssl::stream isn't in the completion handler callbacks because it's not the callbacks that are operating concurrently on the SSL context, but the ssl::stream implementation that is.
I can't be confident that use of strands in calling async_read_some and async_write_some, or that use of strands for the completion handler, will prevent the io engine from operating on the SSL context at the same time in different threads.
Clearly strand use while calling async_read_some or async_write_some will mean that the read and write can't be queued at the same instant, but I don't see how that prevents the internal implementation from performing the read and write operations at the same time on different threads if the encapsulated tcp::socket becomes ready for read and write at the same time.
Comments at the end of the last answer to this question boost asio - SSL async_read and async_write from one thread claim that concurrent writes to ssl::stream could segfault rather than merely interleave, suggesting that the implementation is not taking the necessary locks to guard against concurrent access.
Unless the actual delayed socket write is bound to the thread/strand that queued it (which I can't see being true, or it would undermine the usefulness of worker threads), how can I be confident that it is possible to queue a read and a write on the same ssl::stream, or what that way could be?
Perhaps the async_write_some processes all of the data with the SSL context immediately, to produce encrypted data, and then becomes a plain socket write, and so then can't conflict with a read completion handler on the same strand, but it doesn't mean that it can't conflict with the internal implementations socket-read-and-decrypt before the completion handler gets queued on the strand. Never mind transparent SSL session re-negotiation that might happen...
I note from: Why do I need strand per connection when using boost::asio? "Composed operations are unique in that intermediate calls to the stream are invoked within the handler's strand, if one is present, instead of the strand in which the composed operation is initiated." but I'm not sure if what I am refering to are "intermediate calls to the stream". Does it mean: "any subsequent processing within that stream implementation"? I suspect not
And finally, for why-oh-why, why doesn't the ssl::stream implementation use a futex or other lock that is cheap when there is no conflict? If the strand rules (implicit or explicit) were followed, then the cost would be almost non-existent, but it would provide safety otherwise. I ask because I've just transitioned the propaganda of Sutter, Stroustrup and the rest, that C++ makes everything better and safer, to ssl::stream where it seems easy to follow certain spells but almost impossible to know if your code is actually safe.
The answer is that the boost ssl::stream implementation uses strands internally for SSL operations.
For example, the async_read_some() function creates an instance of openssl_operation and then calls strand_.post(boost::bind(&openssl_operation::start, op)).
[http://www.boost.org/doc/libs/1_57_0/boost/asio/ssl/old/detail/openssl_stream_service.hpp]
It seems reasonable to assume that all necessary internal ssl operations are performed on this internal strand, thus serialising access to the SSL context.
Q. but I'm not sure if what I am refering to are "intermediate calls to the stream". Does it mean: "any subsequent processing within that stream implementation"? I suspect not
The docs spell it out:
This operation is implemented in terms of zero or more calls to the stream's async_read_some function, and is known as a composed operation. The program must ensure that the stream performs no other read operations (such as async_read, the stream's async_read_some function, or any other composed operations that perform reads) until this operation completes. doc
And finally, for why-oh-why, why doesn't the ssl::stream implementation use a futex or other lock that is cheap when there is no conflict?
You can't hold a futex across async operations because any thread may execute completion handlers. So, you'd still need the strand here, making the futex redundant.
Comments at the end of the last answer to this question boost asio - SSL async_read and async_write from one thread claim that concurrent writes to ssl::stream could segfault rather than merely interleave, suggesting that the implementation is not taking the necessary locks to guard against concurrent access.
See previous entry. Don't forget about multiple service threads. Data races are Undefined Behaviour
TL;DR
Long story short: async programming is different. It is different for good reasons. You will have to adapt your thinking to it though.
Strands help the implementation by abstracting sequential execution over the async scheduler.
This makes it so that you don't have to know what the scheduling is, how many service threads are running etc.
Related
While checkig the documentation of boost::asio in verion 1.66.0, I noticed that the io_context constructor provides a concurrency_hint parameter. After reading the documentation, I'm unsure if I can use BOOST_ASIO_CONCURRENCY_HINT_UNSAFE_IO.
I have following situation:
I have a single io_context to do the IO. ioc.run() is executed from a single thread.
In this thread, some network IO using async calls are executed.
Other threads call boost::asio::dispatch(ioc, ...) to execute code on the IO thread.
I'm trying to figure out what concurrency hint value is safe to use in the situation as described above:
Using no concurrency hint is ok (eg. BOOST_ASIO_CONCURRENCY_HINT_SAFE), but slower than with hints.
Using 1 is ok.
Using BOOST_ASIO_CONCURRENCY_HINT_UNSAFE is not ok because it doesn't allow async calls.
What is unclear to me is BOOST_ASIO_CONCURRENCY_HINT_UNSAFE_IO. Documentation says:
This special concurrency hint disables locking in the reactor I/O. This hint has the following restrictions:
— Care must be taken to ensure that run functions on the io_context, and all operations on the context's associated I/O objects (such as sockets and timers), occur in only one thread at a time.
I wonder if it's safe do a boost::asio::dispatch from another thread when using this concurrency hint.
Since boost::asio::dispatch¹ ends up calling io_context::dispatch I would conclude that it's not ok to use BOOST_ASIO_CONCURRENCY_HINT_UNSAFE_IO if you call it from another thread:
— Care must be taken to ensure that run functions on the io_context, and all operations on the context's associated I/O objects (such as sockets and timers), occur in only one thread at a time.
¹ same for post/defer
I've been doing a lot of reading, but I just cannot wrap my head around the difference between synchronous and asynchronous calls in Boost ASIO: what they are, how they work, and why to pick one over the other.
My model is a server which accepts connections and appends the new connection to a list. A different thread loops over the list and sends each registered connection data as it becomes available. Each write operation should be safe. It should have a timeout so that it cannot hang, it should not allocate arbitrarily large amounts of memory, or in general cause the main application to crash.
Confusion:
How does accept_async differ from regular accept? Is a new thread allocated for each connection accepted? From examples I've seen it looks like after a connection is accepted, a request handler is called. This request handler must tell the acceptor to prepare to accept again. Nothing about this seems asynchronous. If the requset handler hangs then the acceptor blocks.
In the boost mailing list the OP was told to use async_write with a timer instead of regular write. In this configureation I don't see any asynchronous behaviour or why they would be recommended. From the Boost docs async_write seems more dangerous than write because the user must not call async_write again before the first one completes.
Asynchronous calls return immediately.
That's the important bit.
Now how do you control "the next thing" that happens when the asynchronous operation has completed? You got it, you supply the completion handler.
The strength of asynchrony is so you can have an IO operation (or similar) run "in the background" without necessarily incurring any thread switch or synchronization overhead. This way you can handle many asynchronous control flows at the same time, on a single thread.
Indeed asynchronous operations can be more complicated and require more thought (e.g. about lifetime of references used in the completion handler). However, when you need it, you need it.
Boost.Asio basic overview from the official site explains it well:
http://www.boost.org/doc/libs/1_61_0/doc/html/boost_asio/overview/core/basics.html
The io_service object is what handles the multiple operations.
Calls to io_service.run() should be made carefully (that could explain the "dangerous async_write")
I know that OpenSSL, boost asio SSL implementation is based on, doesn't allow concurrent SSL_read() and SSL_write() (i.e. SSL_read() and SSL_write() executed by different threads).
Is it safe to call boost asio async_read() and async_write() on SSL socket from the same thread?
Thanks
The requirement for boost::asio::ssl:::stream is for thread safety; it does not place a requirement as to which thread may initiate operations:
Distinct objects: Safe.
Shared objects: Unsafe. The application must also ensure that all asynchronous operations are performed within the same implicit or explicit strand.
If the application only has one thread processing the io_service, and async_read() and async_write() are initiated from within that thread, then it is safe, as the operation and completion handler(s) are running within an implicit strand.
On the other hand, if multiple threads are processing the io_service, then an explicit strand is necessary. The async_read() and async_write() operations need to be initiated from within a strand, and the completion handlers need to be wrapped by the same strand.
For more details on Boost.Asio's thread safety requirements, strands, and composed operations, consider reading this answer.
It is safe to call async_read() and async_write() on SSL socket from the same thread, but in general case it is not enough to avoid concurrency issues with ssl::stream. The actual requirement is provided in ssl::stream documentation:
Thread Safety (...) Shared objects: Unsafe. The application must also ensure that all asynchronous operations are performed within the same implicit or explicit strand.
Of course, the standard boost::asio requirement to ensure that:
no other read operations are performed until async_read handler gets
called, and
no other write operations are performed until async_write handler gets called.
must also be met.
Note, that it is allowed to schedule a read operation while a write operation is underway and conversely. Simultaneous asynchronous read and write operations are possible in ssl::stream thanks to handling OpenSSL's network needs asynchronously through BIO mechanism. SSL_read() and SSL_write() manifest their needs to communicate by returning SSL_ERROR_WANT_READ and SSL_ERROR_WANT_WRITE error codes. These error codes are used by ssl::stream implementation to schedule the network operations asynchronously. A read or write operation on ssl::stream may need multiple both read and write operations on the underlying network socket and multiple calls to SSL_read() / SSL_write(), which will be performed from asynchronous network operation completion handlers (particularly not from the original async_read/async_write call), this is why it is not enough to ensure that async_read and async_write are not called simultaneously, but a strand is needed.
It is safe. But simulateous 2 or more async_write-s on same socket are unsafe and will segfault often(at least for SSL case).
A way of doing this is to construct the ssl::stream using a strand as it's execution context. You can then perform async_read and async_write operations from that strand. Both of these operations can be scheduled at the same time.
You just have to make sure that the context the stream was constructed with and the context that async_read and async_write are called from are effectively an explicit or implicit strand.
Related question: boost asio ssl async_read + async_write within a strand is failing
I have been trying to understand the logic in boost's http server 3 example. The request in this example is read inside connection.cpp, in the start() method, which calls:
socket_.async_read_some(boost::asio::buffer(buffer_),
strand_.wrap(
boost::bind(&connection::handle_read, shared_from_this(),
boost::asio::placeholders::error,
boost::asio::placeholders::bytes_transferred)));
Note that the async_read_some method is documented to return immediately. Then inside the read handler (connection::handle_read()), we may again call async_read_some if parse returns boost::indeterminate. What benefit does this provide over socket_.read_some(buffer), given that we already know we are working in a separate thread. The reason I ask is I want to change the message parsing a bit to call read_some on demand, but the method I have in mind won't work with an async read.
Also, a related question: is there any difference between
async_read_some()
and
boost::thread th([](){ ret = read_some(); handle_read(ret) });?
Boost.Asio's HTTP Server 3's example is coded in a way that it remains agnostic to the size of the thread pool. As such, there is no guarantee that work will be done in separate threads. Nevertheless, the benefit in being agnostic is that it scales better with more connections. For example, consider the C10K problem that examines 10000 clients simultaneously connected. A synchronous solution may run into various performance issues or resource limitations with 10000 clients. Additionally, the asynchronous nature helps insulate the program from behavior changes in the network. For instance, consider a synchronous program that has 3 clients and 2 threads, but 2 of the clients have high latency due to an increase in noise on the network. The 3rd client could be inadvertently affected if both of the threads are blocked waiting for data from the other clients.
If there is a low and finite number of connections, with each connection serviced by a thread, then the performance difference between a synchronous and asynchronous server may be minimal. When possible, it is often advised to avoid mixing asynchronous and synchronous programming, as it can turn a complex solution into a complicated one. Furthermore, most synchronous algorithms can be written asynchronously.
There are two major differences between an asynchronous operation and a synchronous operation (even those running within a dedicated thread):
Thread safety. As noted in the documentation:
In general, it is safe to make concurrent use of distinct objects, but unsafe to make concurrent use of a single object.
Therefore, asynchronous and synchronous operations cannot safely be initiated while a synchronous operation is in progress, even if the operation is invoked within its own thread. This may be minimal in a half duplex protocol, but should be considered with full duplex protocols.
Ability to cancel an operation. As noted in this answer, synchronous operations cannot be cancelled through the cancel() member functions Boost.Asio provides. Instead, the application may need to use lower level mechanics, such as signals.
Trying to learn asio, and I'm following the examples from the website.
Why is io_service needed and what does it do exactly? Why do I need to send it to almost every other functions while performing asynchronous operations, why can't it "create" itself after the first "binding".
Asio's io_service is the facilitator for operating on asynchronous functions. Once an async operation is ready, it uses one of io_service's running threads to call you back. If no such thread exists it uses its own internal thread to call you.
Think of it as a queue containing operations. It guarantees you that those operations, when run, will only do so on the threads that called its run() or run_once() methods, or when dealing with sockets and async IO, its internal thread.
The reason you must pass it to everyone is basically that someone has to wait for async operations to be ready, and as stated in its own documentation io_service is ASIO's link to the Operating System's I/O service so it abstracts away the platform's own async notifiers, such as kqueue, /dev/pool/, epoll, and the methods to operate on those, such as select().
Primarily I end up using io_service to demultiplex callbacks from several parts of the system, and make sure they operate on the same thread, eliminating the need for explicit locking, since the operations are serialized. It is a very powerful idiom for asynchronous applications.
You can take a look at the core documentation to get a better feeling of why io_service is needed and what it does.