multiple boost::asio ssl clients running on same system - c++

Question

I have a simple Boost ASIO SSL Client which calls a web api. The client is slight modification of the Boost SSL documentation example.
//http.h
class Http {
public:
static void WebApiCall(...);
}
//http.cpp
void Http::WebApiCall(...) {
try {
// .......
boost::asio::io_service io_service;
tcp::resolver resolver(io_service);
tcp::resolver::query query(serverip, serverport);
tcp::resolver::iterator endpoint_iterator = resolver.resolve(query);
boost::asio::ssl::context ctx(io_service, boost::asio::ssl::context::tlsv1); // ERROR # 1
// ....
// Setting SSL Context Properties Here
// ....
boost::shared_ptr<boost::asio::ssl::stream<tcp::socket> > ssocket(new boost::asio::ssl::stream<tcp::socket>(io_service, ctx));
boost::asio::ip::tcp::endpoint endpoint = *endpoint_iterator;
ssocket->lowest_layer().connect(endpoint);
boost::system::error_code er;
ssocket->handshake(boost::asio::ssl::stream_base::client,er);
boost::asio::streambuf request;
std::ostream request_stream(&request);
// ....
// Set Headers & Body of HTTP Request here
// ....
size_t written = 0;
written = boost::asio::write(*ssocket, request); // ERROR # 2
// .....
// Read server response
boost::asio::streambuf response;
boost::system::error_code error;
int read_bytes = 0;
std::string TempBuf = "";
std::ostringstream responseStringstream;
std::stringstream response_stream;
while ( boost::asio::read(*ssocket,response,boost::asio::transfer_at_least(1), error)) {
read_bytes = read_bytes + response.size();
responseStringstream << &response;
}
}
// Do some stuff with server response....
// ....
} catch ( const boost::system::system_error &error ) {
// Print the exception ..
}
}
// client.cpp
Http::WebApiCall(<api_to_call>)
You can see its a simple HTTP client with one static function which implements the actual SSL enabled HTTP Client using ASIO.
Use Case:
1000 processes are running of this client on one machine. All processes are making a POST request periodically (e.g after every one minute) to one resource in approximately the same time. Machine is Ubuntu and I do not seems to be out of memory (I have around 6 GB free)
This client works perfect but in one case where I have to simulate some load on my server I have launched 1000 processes of this client, all on one machine, all calling same API to same server using same public certificates, except that every client has its own OAuth token¹. In this situation I am getting two types of exceptions:
Errors:
ERROR # 2: Some clients (NOT ALL) while writing get error (write: short read). From different forums and Boost sources it seems the server is sending SSL_Shutdown causing ASIO to throw this error, which, as per my finding is normal behavior. My question is, why server is sending SSL_Shutdown at this point? Does this have to do anything with multiple processes calling the same resource from same machine? From ASIO docs ASIO SSL is not thread safe, but in this case I am running only one thread but different processes (which I believe is perfectly safe), besides above code is itself thread safe. Is underlying openssl behaving erratically?
ERROR # 1: Sometimes get an exception while creating Boost ASIO SSL Context, simply saying "context: ssl error". Again same thoughts, why it behaves like this? Does this has something to do with multiple processes, is openssl mixing things up in this scenario?
My client is running perfectly for last one year as one process per machine and I have never seen these errors before. Any thoughts are appreciated.
¹ (Just mentioning about OAuth but I don't think this has anything to do with it)

Answer 1

Q: Some clients (NOT ALL) while writing get error (write: short read). From different forums and Boost sources it seems the server is sending SSL_Shutdown causing ASIO to throw this error, which, as per my finding is normal behavior.
Most likely cause is you're pushing the system beyond a resource limit. E.g. the client or the server may run out of file handles. E.g. on my Linux box the number of open files is limited to 1024 by default: ulimit -a outputs:
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 256878
max locked memory (kbytes, -l) unlimited
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 95
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 256878
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
Q: My question is, why server is sending SSL_Shutdown at this point?
Most likely because of the above.
Q: Does this have to do anything with multiple processes calling the same resource from same machine?
No.
Q: From ASIO docs ASIO SSL is not thread safe, but in this case I am running only one thread but different processes (which I believe is perfectly safe), besides above code is itself thread safe. Is underlying openssl behaving erratically?
Thread safety or the underlying SSL library is not the issue here.
Q: Sometimes get an exception while creating Boost ASIO SSL Context, simply saying "context: ssl error". Again same thoughts, why it behaves like this? Does this has something to do with multiple processes, is openssl mixing things up in this scenario?
It's unlikely but it's possible that each instance of ssl::context incurs overhead. You might try allocating it statically/out of the loop.
That said, it's more likely that the initialization if the SSL context simply runs into (the same) resource limit, as it will likely open some system-configuration files and/or check for existence of well known paths (e.g. the CApath etc.)