Handeling SSL Client not reading all data - c++

Question

I am trying to accomplish, that my ssl server does not break down, when a client does not collect all data. (fixed with one minor bug)
when the data is too long.
Basically what I'm trying to do is write in a non-blocking way. For that I found two different approaches:
First approach
using this code
int flags = fcntl(ret.fdsock, F_GETFL, 0);
fcntl(ret.fdsock, F_SETFL, flags | O_NONBLOCK);
and creating the ssl connection with it
Second approach:
Doing this directly after creating the SSL Object using SSL_new(ctx)
BIO *sock = BIO_new_socket(ret.fdsock, BIO_NOCLOSE);
BIO_set_nbio(sock, 1);
SSL_set_bio(client, sock, sock);
Both of which have their downsides, but neither of which helps solving the problem.
The first approach seems to read in a unblocking way just fine, but when I write more data, than the client reads, my server crashes.
The second approach does not seem to do anything, so my guess is, that I did something wrong or did not understand what a BIO actually does.
For more Information here is how the server writes to the client:
int SSLConnection::send(char* msg, const int size){
int rest_size = size;
int bytes_sent = 0;
char* begin = msg;
std::cout << "expected bytes to send: " << size << std::endl;
while(rest_size > 0) {
int tmp_bytes_sent = SSL_write(connection, begin, rest_size);
std::cout << "any error : " << ERR_get_error()<< std::endl;
std::cout << "tmp_bytes_sent: " << tmp_bytes_sent << std::endl;
if (tmp_bytes_sent < 0){
std::cout << tmp_bytes_sent << std::endl;
std::cout << "ssl error : " << SSL_get_error(this->connection, tmp_bytes_sent)<< std::endl;
} else {
bytes_sent += tmp_bytes_sent;
rest_size -= tmp_bytes_sent;
begin = msg+bytes_sent;
}
}
return bytes_sent;
}
Output:
expected bytes to send: 78888890
Betätigen Sie die <RETURN> Taste, um das Fenster zu schließen...
(means: hit <return> to close window)
EDIT: After people said, that I need to cache errors appropriate, here is my new code:
Setup:
connection = SSL_new(ctx);
if (connection){
BIO * sbio = BIO_new_socket(ret.fdsock, BIO_NOCLOSE);
if (sbio) {
BIO_set_nbio(sbio, false);
SSL_set_bio(connection, sbio, sbio);
SSL_set_accept_state(connection);
} else {
std::cout << "Bio is null" << std::endl;
}
} else {
std::cout << "client is null" << std::endl;
}
Sending:
int SSLConnection::send(char* msg, const int size){
if(connection == NULL) {
std::cout << "ERR: Connection is NULL" << std::endl;
return -1;
}
int rest_size = size;
int bytes_sent = 0;
char* begin = msg;
std::cout << "expected bytes to send: " << size << std::endl;
while(rest_size > 0) {
int tmp_bytes_sent = SSL_write(connection, begin, rest_size);
std::cout << "any error : " << ERR_get_error()<< std::endl;
std::cout << "tmp_bytes_sent: " << tmp_bytes_sent << std::endl;
if (tmp_bytes_sent < 0){
std::cout << tmp_bytes_sent << std::endl;
std::cout << "ssl error : " << SSL_get_error(this->connection, tmp_bytes_sent)<< std::endl;
break;
} else if (tmp_bytes_sent == 0){
std::cout << "tmp_bytes are 0" << std::endl;
break;
} else {
bytes_sent += tmp_bytes_sent;
rest_size -= tmp_bytes_sent;
begin = msg+bytes_sent;
}
}
return bytes_sent;
}
Using a client, that fetches 60 bytes, here is the output:
Output writing 1,000,000 Bytes:
expected bytes to send: 1000000
any error : 0
tmp_bytes_sent: 16384
any error : 0
tmp_bytes_sent: 16384
Betätigen Sie die <RETURN> Taste, um das Fenster zu schließen...
(translates to: hit <RETURN> to close window)
Output writing 1,000 bytes:
expected bytes to send: 1000
any error : 0
tmp_bytes_sent: 1000
connection closed <- expected output

Answer 1

First, a warning: non-blocking I/O over SSL is a rather baroque API, and it's difficult to use correctly. In particular, the SSL layer sometimes needs to read internal data before it can write user data (or vice versa), and the caller's code is expected to be able to handle that based on the error-codes feedback it gets from the SSL calls it makes. It can be made to work correctly, but it's not easy or obvious -- you are de facto required to implement a state machine in your code that echoes the state machine inside the SSL library.
Below is a simplified version of the logic that is required (it's extracted from the Write() method in this file which is part of this library, in case you want to see a complete, working implementation)
enum {
SSL_STATE_READ_WANTS_READABLE_SOCKET = 0x01,
SSL_STATE_READ_WANTS_WRITEABLE_SOCKET = 0x02,
SSL_STATE_WRITE_WANTS_READABLE_SOCKET = 0x04,
SSL_STATE_WRITE_WANTS_WRITEABLE_SOCKET = 0x08
};
// a bit-chord of SSL_STATE_* bits to keep track of what
// the SSL layer needs us to do next before it can make more progress
uint32_t _sslState = 0;
// Note that this method returns the number of bytes sent, or -1
// if there was a fatal error. So if this method returns 0 that just
// means that this function was not able to send any bytes at this time.
int32_t SSLSocketDataIO :: Write(const void *buffer, uint32 size)
{
int32_t bytes = SSL_write(_ssl, buffer, size);
if (bytes > 0)
{
// SSL was able to send some bytes, so clear the relevant SSL-state-flags
_sslState &= ~(SSL_STATE_WRITE_WANTS_READABLE_SOCKET | SSL_STATE_WRITE_WANTS_WRITEABLE_SOCKET);
}
else if (bytes == 0)
{
return -1; // the SSL connection was closed, so return failure
}
else
{
// The SSL layer's internal needs aren't being met, so we now have to
// ask it what its problem is, then give it what it wants. :P
int err = SSL_get_error(_ssl, bytes);
if (err == SSL_ERROR_WANT_READ)
{
// SSL can't write anything more until the socket becomes readable,
// so we need to go back to our event loop, wait until the
// socket select()'s as readable, and then call SSL_Write() again.
_sslState |= SSL_STATE_WRITE_WANTS_READABLE_SOCKET;
_sslState &= ~SSL_STATE_WRITE_WANTS_WRITEABLE_SOCKET;
bytes = 0; // Tell the caller we weren't able to send anything yet
}
else if (err == SSL_ERROR_WANT_WRITE)
{
// SSL can't write anything more until the socket becomes writable,
// so we need to go back to our event loop, wait until the
// socket select()'s as writeable, and then call SSL_Write() again.
_sslState &= ~SSL_STATE_WRITE_WANTS_READABLE_SOCKET;
_sslState |= SSL_STATE_WRITE_WANTS_WRITEABLE_SOCKET;
bytes = 0; // Tell the caller we weren't able to send anything yet
}
else
{
// SSL had some other problem I don't know how to deal with,
// so just print some debug output and then return failure.
fprintf(stderr,"SSL_write() ERROR!");
ERR_print_errors_fp(stderr);
}
}
return bytes; // Returns the number of bytes we actually sent
}

Answer 2

I think your problem is
rest_size -= bytes_sent;
You should do rest_size -= tmp_bytes_sent;
Also
if (tmp_bytes_sent < 0){
std::cout << tmp_bytes_sent << std::endl;
//its an error condition
return bytes_sent;
}
I dont know whether this will fix the issue, but the code you pasted has the above mentioned issues

Answer 3

When I write more data, than the client reads, my server crashes.
No it doesn't, unless you've violently miscoded something else that you haven't posted here. It either loops forever or it gets an error: probably ECONNRESET, which means the client has behaved as you described, and you've detected it, so you should close the connection and forget about him. Instead of which, you are just looping forever, trying to send the data to a broken connection, which can never happen.
And when you get an error, there's not much use in just printing a -1. You should print the error, with perror() or errno or strerror().
Speaking of looping forever, don't loop like this. SSL_write() can return 0, which you aren't handling at all: this will cause an infinite loop. See also David Schwartz's comments below.
NB you should definitely use the second approach. OpenSSL needs to know that the socket is in non-blocking mode.
Both of which have their downsides
Such as?
And as noted in the other answer,
rest_size -= bytes_sent;
should be
rest_size -= tmp_bytes_sent;