Boost Asio and usage of OpenSSL in multithreaded app [duplicate] - c++

Question

Do I create one strand that all of my SSL sockets share, or one strand per SSL context (shared by any associated sockets)?
Boost.Asio SSL documentation states this, but it doesn't mention contexts. I assume that this means I must use only one strand for everything, but I think this was written before OpenSSL had multithreading support.
SSL and Threads
SSL stream objects perform no locking of their own. Therefore, it is essential that all asynchronous SSL operations are performed in an implicit or explicit strand. Note that this means that no synchronisation is required (and so no locking overhead is incurred) in single threaded programs.
I'm most likely only going to have only one SSL context, but I'm wondering if it's more proper for the strand to be owned by the SSL context, or by the global network service.
I did provide a handler to CRYPTO_set_locking_callback in case that matters.

Answer 1

I think there is some confusion on this thread because there's a few things that need to be clarified. Lets start by asserting that ::asio::ssl::context == SSL_CTX. The two are one.
Second, when using boost::asio::ssl, unless you're doing something crazy where you're bypassing the internal init objects, there's no need for you to manually set the crypto locking callbacks. This is done for you, as you can see in the sources here.
In fact, you may cause issues by doing this, because the destructor for the init object operates under the assumption that they have done this work internally. Take that last bit with a grain of salt, because I have not reviewed this in depth.
Third, I believe you're confusing SSL streams with SSL contexts. For simplicity, think of the SSL stream as the socket, think of the SSL context as separate object that the sockets can use for various SSL functions, such as handshaking with a particular negotiation key, or as a server, providing information about your server certificate to a connected client so that you can handshake with the client.
The mention of strands comes down to preventing possible simultaneous IO against one specific stream (socket), not contexts. Obviously attempting to read into a buffer and write from the same buffer at the same time on the same socket would be an issue. So, when you supply completion handlers wrapped by strands to the various ::asio::async_X methods, you're enforcing a specific ordering to prevent the aforementioned scenario. You can read more in this answer given by someone who knows much more about this than I.
Now as far as contexts go, David Schwartz points out in the comments and another answer he wrote I need to dig up, that whole purpose of contexts themselves are to provide information that facilitates the function of SSL across multiple SSL streams. He appears to imply that they essentially must be thread safe, given their intended purpose. I believe perhaps he is speaking within the context of ::asio::ssl::context, only because of the way that ::asio::ssl correctly employs the thread safety callbacks, or perhaps he's just speaking in the context of using openSSL correctly in a multithreaded program.
Regardless, beyond such comments and answers on SO, and my own practical experience, it's incredibly difficult to come across concrete evidence of this in documentation, or clearly defined boundaries between what is and isn't thread safe. boost::asio::ssl:context is, as David also points out, simply a very thin wrapper around SSL_CTX. I would further add that its meant to give a more "c++ ish" feel to working with the underlying structure(s). It was probably also designed with some intent of decoupling ::asio::ssl and underlying implementation library, but it doesn't achieve this, the two are tightly bound. David mentions again rightly that this thin wrapper is poorly documented, and one must look at the implementation to get insight.
If you start digging into the implementation, there's a rather easy way to find out what is and isn't thread safe when it comes to contexts. You can do a search for CRYPTO_LOCK_SSL_CTX within sources like ssl_lib.c.
int SSL_CTX_set_generate_session_id(SSL_CTX *ctx, GEN_SESSION_CB cb)
{
CRYPTO_w_lock(CRYPTO_LOCK_SSL_CTX);
ctx->generate_session_id = cb;
CRYPTO_w_unlock(CRYPTO_LOCK_SSL_CTX);
return 1;
}
As you can see, CRYPTO_w_lock is used, which brings us back to the official page about openSSL and threads, here, which states:
OpenSSL can safely be used in multi-threaded applications provided
that at least two callback functions are set, locking_function and
threadid_func.
Now we come full circle to the linked asio/ssl/detail/impl/openssl_init.ipp source code in the first paragraph of this answer, where we see:
do_init()
{
::SSL_library_init();
::SSL_load_error_strings();
::OpenSSL_add_all_algorithms();
mutexes_.resize(::CRYPTO_num_locks());
for (size_t i = 0; i < mutexes_.size(); ++i)
mutexes_[i].reset(new boost::asio::detail::mutex);
::CRYPTO_set_locking_callback(&do_init::openssl_locking_func);
::CRYPTO_set_id_callback(&do_init::openssl_id_func);
#if !defined(SSL_OP_NO_COMPRESSION) \
&& (OPENSSL_VERSION_NUMBER >= 0x00908000L)
null_compression_methods_ = sk_SSL_COMP_new_null();
#endif // !defined(SSL_OP_NO_COMPRESSION)
// && (OPENSSL_VERSION_NUMBER >= 0x00908000L)
}
Note of course:
CRYPTO_set_locking_callback
CRYPTO_set_id_callback
So at least in terms of ::asio::ssl::context, thread safety here has nothing to do with strands and everything to do with openSSL working as openSSL is designed to work when used correctly in a multithreaded program.
Coming to the original question, now with all of this explained, David also gave the answer very simply in the comments with:
The most common method is to use one strand per SSL connection.
Take an example of a HTTPS server that serves up content of example.com. The server has a single context, configured with information such as the certificate for example.com. A client connects, this context is used on all connected clients to perform the handshake and such. You wrap your connected client in a new session object, where you handle that client. It is within this session that you would have a single strand, either implicit or explicit, to protect the socket, not the context.
While I'm not an expert by any means and I welcome corrections to this answer, I have put everything that I know about these subjects into practice in an open source transparent filtering HTTPS proxy. It's a little over 50% of comments to code ratio with over 17K lines total, so everything I know is written down there (be it right or wrong ;)). If you'd like to see an example of this stuff in action, you can look at the TlsCapableHttpBridge.hpp source, which acts as both a client a server on a per-host, per-connection basis.
Server contexts and certificates are spoofed/generated once and shared across all clients panning multiple threads. The only manual locking done is during storage and retrieval of the contexts. There is one strand per side of the bridge, one for the real downstream client socket and one for the upstream server connection, although they technically aren't even necessary because the order of operations creates an implicit strand anyway.
Note that the project is under development as I'm rewriting many things, (dep build instructions are not present yet), but everything is functional in terms of the MITM SSL code, so you're looking at a fully functional class and associated components.

Answer 2

UPDATE
The gist of this answer is contested by David Schwarz, whose authority in this area I hold in high esteem.
There are reasons to expect that ssl contexts can be shared between threads - at least for some operations, if only to facilitate SSL session resumption.
I think David has experience with SSL context as OpenSSL uses it. Boost ASIO uses that in turn (at least on all platforms I know of). So, either David writes an answer sharing his knowledge, or you/me would have to spend some time with the OpenSSL documentation and Boost Asio source code to figure out the effective constraints that apply to Boost Asio's ssl::context usage.
Below are the constraints as currently documented.
[old answer text follows]
Thread Safety
In general, it is safe to make concurrent use of distinct objects, but unsafe to make concurrent use of a single object. However, types such as io_service provide a stronger guarantee that it is safe to use a single object concurrently.
Logically, because the documentation doesn't mention thread-safety of the ssl_context class in particular, you must conclude that it is not.
It doesn't matter that you know that the underlying SSL library supports this if you use some particular hooks (like you mention). This only tells you that it might not be hard to make ssl_context thread-aware.
But until you (work with the library devs to) provide this patch, it's not available.
Long story short, you access each ssl_context from a single strand.

Answer 3

I'd say it depends on how your protocol is like. If it's HTTP, there's no need to use an (explicit) strand as you don't read and write to your socket in parallel.
In fact, what would cause problems, is code like this:
void func()
{
async_write(...);
async_read(...);
}
because here -if your io_service() has a POOL of threads associated with it-, actual read and write could be carried out in parallel by several threads.
If you only have one thread per io_service, no need for a strand. The same is true if you're implementing HTTP for example. In HTTP, you don't read and write to the socket in parallel, due to the layout of the protocol. You read a request from the client -though this might be done in several async calls-, then you somehow process request and headers, then you (async or not) send your reply.
Pretty much the same you can also read in ASIO's strand documentation.