OpenSSL C++ DTLS client - c++

Question

I'm trying to establish DTLS connection using openssl (c++).
However whereas there is DTLSv1_listen() function for server side I can't find any client side equivalent to actually establish UDP connection to server from client. Or send something to DTLS server. Could someone help me understand how to establish "connection" to DTLS server (I know the point of UDP and datagram communication is to be connectionless but by connection I mean scheme like DNS request+response)?
I need to send single message to server and then receive 1 response message. Encrypted. With certificate verification. How such communication scheme would work in DTLS world?

Answer 1

What are you using for signalling. You actually don't have to use openssl for your signalling layer you could use a Memory Bio and read and write from it.
With DTLS You will have 2 sides here, a Client and a Server. The Client will initiate things with a Client Hello. The server hopefully receives it, writes it into the bio, reads back a Server Hello and responds via the signalling layer.
There is a lot of stuff required to really finish up your app into the real world so I wont cover everything.
If you have a SSL Context, and you have your BIO's created. To Initiate the Server:
SSL_set_accept_state(*sslContext)
and to initiate the client:
SSL_set_connect_state(*sslContext);
You then want to start a handshake on the client. There is so many factors here on how your application is going to work its hard to give advice but the client should call:
SSL_do_handshake(*sslContext);
Depending on how you have wired everything up, your program may automatically send the client hello at this point. You can use Wireshark, grab only UDP and at the top filter DTLS to see it. If not you may be required to signal manually.
I wrote the following program myself to test OpenSSL dll's to see if there were any issues after compiling them. I can't share all the code with you but it shows how in memory you can use a single console app to do the handshake (no transmission via the internet just to see how it works).
Notes:
The BIO is a Memory BIO in this case.
Handshake is a SSL_do_handshake
WriteCipherText is a call to BIO_write
ReadCipherText is a call to BIO_read
You have to open it and do the other basic setup states first. At the end both of the handshakes will return 1 for succcess. This just kind of shows you how to do a basic memory DTLS handshake using openssl.
Console.WriteLine("[Client] Open: " + Client.Open("ECDHE-ECDSA-AES128-SHA:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-SHA256:AES128-SHA:AES128-GCM-SHA256:AES128-SHA256", "SRTP_AES128_CM_SHA1_80"));
Console.WriteLine("[Server] Open: " + Server.Open("ECDHE-ECDSA-AES128-SHA:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-SHA256:AES128-SHA:AES128-GCM-SHA256:AES128-SHA256", "SRTP_AES128_CM_SHA1_80"));
Console.WriteLine("[Client] Handshake" + Client.Handshake());
Console.WriteLine("Client: [Read] Client Hello");
var clientHello = Client.ReadCipherText();
Console.WriteLine("[Server] Write Hello: " + Server.WriteCipherText(clientHello));
Console.WriteLine("[Server] Handshake" + Server.Handshake());
clientHello.Free();
Console.WriteLine("[Server] Read Server Hello");
var serverHello = Server.ReadCipherText();
Console.WriteLine("[Client] Write Server Hello: " + Client.WriteCipherText(serverHello));
Console.WriteLine("[Client] Handshake" + Client.Handshake());
serverHello.Free();
Console.WriteLine("[Client] Read Certificate");
var clientCertificate = Client.ReadCipherText();
Console.WriteLine("[Server] Write Certificate: " + Server.WriteCipherText(clientCertificate));
Console.WriteLine("[Server] Handshake: " + Server.Handshake());
clientCertificate.Free();
Console.WriteLine("[Server] Read Change Cipher Spec");
var serverChangeCipherSpec = Server.ReadCipherText();
Console.WriteLine("[Client] Write Change Cipher Spec: " + Client.WriteCipherText(serverChangeCipherSpec));
serverChangeCipherSpec.Free();
Console.WriteLine("[Client] Handshake" + Client.Handshake());
Console.WriteLine("[Server] Handshake" + Server.Handshake());
Console.ReadLine();
Lots of helpful resources too to take a look at:
https://web.archive.org/web/20150814081716/http://sctp.fh-muenster.de/dtls-samples.html This ones really good
http://chris-wood.github.io/2016/05/06/OpenSSL-DTLS.html
https://wiki.openssl.org/index.php/SSL/TLS_Client

Answer 2

On the client side you create the UDP-socket, pass it with BIO_new_dgram to the SSL context and connect it with connect() or BIO_ctrl(bio, BIO_CTRL_DGRAM_SET_CONNECTED, 0, &server_sockaddr). The connect just forces the file descriptors write to always be sent to server_sockaddr.
Now just do a regular SSL_connect and it should work.
For server side it's a little more complicated. I made a post and an example implementation