OpenSSL/C++: can't get SNI extension to appear in ClientHello - c++

I'm trying to initiate a TLS connection and it must include a SNI extension. The following program works but, despite SSL_set_tlsext_host_name being called, does not produce a SNI record in the Client Hello packet:
#include <openssl/ssl.h>
#include <openssl/bio.h>
int main()
{
SSL_CTX* ctx = SSL_CTX_new(TLS_client_method());
SSL* ssl = SSL_new(ctx);
BIO* bio = BIO_new_ssl_connect(ctx);
BIO_set_conn_hostname(bio, "demo.piesocket.com:443");
SSL_set_tlsext_host_name(ssl, "demo.piesocket.com");
BIO_do_connect(bio);
BIO_free_all(bio);
SSL_free(ssl);
SSL_CTX_free(ctx);
return 0;
}
Yet the following command does:
openssl s_client demo.piesocket.com:443
I've checked both exchanges with Wireshark and the Client Hello packets differ only in the absence/presence of SNI. Yes, I've looked long and hard at s_client.c but it does too many things and I'm new to OpenSSL and TLS. What am I missing?
(For brevity I've removed all checks from this example code but it works, the server I'm using here doesn't seem to require SNI and the TLS connection is indeed established in both cases. The TLS portion of the packet from openssl.exe looks like this:
TLSv1.2 Record Layer: Handshake Protocol: Client Hello
Content Type: Handshake (22)
Version: TLS 1.0 (0x0301)
Length: 319
Handshake Protocol: Client Hello
Handshake Type: Client Hello (1)
Length: 315
Version: TLS 1.2 (0x0303)
Random: 8772af2a36342435d6b73a0593087c229b67342030d23ae5…
Session ID Length: 32
Session ID: 49a1e1b7eb761bd9b279efcb4cac15bae2f09bb92e641a75…
Cipher Suites Length: 62
Cipher Suites (31 suites)
Compression Methods Length: 1
Compression Methods (1 method)
Extensions Length: 180
Extension: server_name (len=23)
Extension: ec_point_formats (len=4)
Extension: supported_groups (len=22)
Extension: session_ticket (len=0)
Extension: encrypt_then_mac (len=0)
Extension: extended_master_secret (len=0)
Extension: signature_algorithms (len=42)
Extension: supported_versions (len=9)
Extension: psk_key_exchange_modes (len=2)
Extension: key_share (len=38)
The packet from the program doesn't have the Extension: server_name (len=23) line. OpenSSL version 3.0.7, Windows 7, MinGW.)

You are creating an SSL BIO via BIO_new_ssl_connect. This essentially creates a BIO and inside it creates a new SSL object for the connection based on the SSL_CTX that you pass it. Entirely separate to that you are creating a different SSL object, setting the SNI hostname on it, and then you're not using it for anything - you just free it.
Don't create a separate SSL object. Instead get hold of the SSL object inside the BIO using BIO_get_ssl. Set the SNI hostname on that.
https://www.openssl.org/docs/man3.0/man3/BIO_get_ssl.html

Related

Can't connect to host using OpenSSL in C++

When I try to connect to some hosts(not all) through HTTPS using OpenSSL in C++ I gets OpenSSL error error:14094410:SSL routines:ssl3_read_bytes:sslv3 alert handshake failure. I'm using TLS_client_method SSL method. But if I use openssl test executable, it's ok - openssl s_client -connect host:443 -tls1_2.
My connect code here:
const SSL_METHOD* method = TLS_client_method();
inet->ssl_ctx = SSL_CTX_new(method);
inet->ssl = SSL_new(inet->ssl_ctx);
SSL_set_fd(inet->ssl, s);
int err = SSL_connect(inet->ssl);
Why? Maybe I need some .pem/.pm files? I don't know, but I saw it somewhere.
Your code does not use the SNI extension when connecting to the server, i.e. does not include the hostname of the server into the TLS handshake. Multi-domain sites usually require SNI and might fail or return some unrelated certificate when SNI is not provided. This is also true for CDN like Cloudflare where different domains are accessible by the same IP address but should result in different certificates.
... I tried to connect to hostiman.ru
This is for example the case with hostiman.ru. With SNI (as set by newer s_client versions by default):
$ openssl s_client -connect hostiman.ru:443 -tls1_2
CONNECTED(00000005)
...
depth=0 C = US, ST = CA, L = San Francisco, O = "Cloudflare, Inc.", CN = sni.cloudflaressl.com
...
Cipher : ECDHE-ECDSA-CHACHA20-POLY1305
Without SNI it instead looks like this:
$ openssl s_client -connect hostiman.ru:443 -tls1_2 -noservername
CONNECTED(00000005)
140420123775424:error:14094410:SSL routines:ssl3_read_bytes:sslv3 alert handshake failure:../ssl/record/rec_layer_s3.c:1528:SSL alert number 40
...
Cipher : 0000
To setup SNI in the client use SSL_set_tlsext_host_name, i.e.
inet->ssl = SSL_new(inet->ssl_ctx);
SSL_set_tlsext_host_name(inet->ssl, servername)

After Successfull TLS handshake the server closes with error SSL routines:SSL3_GET_RECORD:wrong version number

We are using openssl 1.0.2k for our TLS related functionalities.
In one of our deployment the client is able to complete the TLS handshakes using TLSv1.2 and was able to send application data towards server.After some requests the TLS connections closed from the server side with the below error
"error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong version number"
TLS handshake steps:
1. Client hello
2. Server Hello
3. Certificate,Certificate Request, Server hello done
4. Certificate,Client Key Exchange,Change Cipher spec,Encrypted handshake message
5. Change Cipher spec,Encrypted handshake message
6. Application data exchanges between client and server
7. Encrypted Alert(server to client)
8. Encrypted Alert( client to server
The error logs from server side says "error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong version number"
Can you please let us know the cause for this issue. If the ssl version is mismatching then the handshake phase should not succeed right?
But in our case handshake is successful and after some application data transfer our server is failing with this error.
If the ssl version is mismatching then the handshake phase should not succeed right?
No. Any TLS packet have header, and header has TLS version inside:
(
byte - record_type
byte[2] - version
byte[2] - length
) header
byte[length] - encrypted or raw data
Header is always in raw, it is never encrypted. Even if during handshake client sent TLS 1.2 version in all TLS packets, he can send another version after handshake is finished. Or someone in between can modify network traffic. In this case OpenSSL throws described error.
In my case, I was using OpenSSL for client functionality.
I was calling SSL_set_connect_state after SSL_connect. It should be called before.
SSL_set_connect_state (for client only) cleans up all the state!
snippet:
void SSL_set_connect_state(SSL *s)
{
s->server = 0;
s->shutdown = 0;
ossl_statem_clear(s);
s->handshake_func = s->method->ssl_connect;
clear_ciphers(s);
}
In my case:
1) Client <-> Server handshake succeeded.
2) SSL_write from client side (client sending message to server) lead to exact same error as mentioned in question (on server side)
I looked at pkt dump on server side.
read from 0x2651570 [0x2656c63] (5 bytes => 5 (0x5)) .
0000 - 16 03 01 01 e2 .....
ERROR
139688140752544:error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong >version number:s3_pkt.c:337:
1) 5 Bytes read in the above snipped is the size of SSL record. Server received data, and it attempted reading SSL record.
2) 1'st byte of the record is the SSL record type In this case ===> x16 => '22'
This itself is wrong, as far as server is concerned, handshake was successful and it was expecting application data. Instead it received data with SSL record for handshake, hence it was throwing the error.
A correct snippet of application data is as follows: 'x17' ==> 23
read from 0x2664f80 [0x2656c63] (5 bytes => 5 (0x5)) .
0000 - 17 03 03 00 1c
Since SSL_set_connect_state was called after connecting, client state was lost and SSL_write will attempt handshake if handshake wasnt performed before (client thought so as its state was lost!)
More data on these SSL records can be found here:
https://www.ibm.com/support/knowledgecenter/SSB23S_1.1.0.12/gtps7/s5rcd.html

AWS SES Not sending emails despite following guide- Windows OS

I'm following the steps listed in the Documentation and I can get a successful connection but attempting to actually send an email gets a 250 Ok response but no message id and no message is sent to my inbox.
openssl s_client -crlf -starttls smtp -connect email-smtp.us-east-1.amazonaws.com:587 0<input.txt
CONNECTED(000002F4)
depth=3 C = US, ST = Arizona, L = Scottsdale, O = "Starfield Technologies, Inc.", CN = Starfield Services Root Certificate Authority - G2
verify error:num=20:unable to get local issuer certificate
---
Certificate chain
0 s:/CN=email-smtp.us-east-1.amazonaws.com
i:/C=US/O=Amazon/OU=Server CA 1B/CN=Amazon
1 s:/C=US/O=Amazon/OU=Server CA 1B/CN=Amazon
i:/C=US/O=Amazon/CN=Amazon Root CA 1
2 s:/C=US/O=Amazon/CN=Amazon Root CA 1
i:/C=US/ST=Arizona/L=Scottsdale/O=Starfield Technologies, Inc./CN=Starfield Services Root Certificate Authority - G2
3 s:/C=US/ST=Arizona/L=Scottsdale/O=Starfield Technologies, Inc./CN=Starfield Services Root Certificate Authority - G2
i:/C=US/O=Starfield Technologies, Inc./OU=Starfield Class 2 Certification Authority
---
Server certificate
-----BEGIN CERTIFICATE-----
<super long server certificate>
-----END CERTIFICATE-----
subject=/CN=email-smtp.us-east-1.amazonaws.com
issuer=/C=US/O=Amazon/OU=Server CA 1B/CN=Amazon
---
No client certificate CA names sent
Peer signing digest: SHA512
Server Temp Key: ECDH, P-256, 256 bits
---
SSL handshake has read 5292 bytes and written 469 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES256-GCM-SHA384
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
Protocol : TLSv1.2
Cipher : ECDHE-RSA-AES256-GCM-SHA384
Session-ID: 5B1EDC39B0B80D50AEAEF28A1F7E49846B77A4B9FF5A7BA347A620F56F9D0FE8
Session-ID-ctx:
Master-Key: <Long master key>
Key-Arg : None
PSK identity: None
PSK identity hint: None
SRP username: None
Start Time: 1528749113
Timeout : 300 (sec)
Verify return code: 20 (unable to get local issuer certificate)
---
250 Ok
421 Timeout waiting for data from client.
closed
The main thing that has me worried is this: Verify return code: 20 (unable to get local issuer certificate) which isn't mentioned anywhere in the documentation and makes me wonder if I'm supposed to be sending a cert of some sort. My Input file looks like this:
EHLO domain.com
AUTH LOGIN
smtp_username
smtp_password line 1
smtp_password line 2
MAIL FROM: verified_email1#domain.com
RCPT TO: verified_email1#domain.com
DATA
From: Sender Name <verified_email1#domain.com>
To: verified_email1#domain.com
Subject: Amazon SES SMTP Test
This message was sent using the Amazon SES SMTP interface.
.
QUIT
Does anyone have any idea what I'm doing wrong? I am attempting to do this on a Windows 10 os using the openssl for Windows if that could be the problem but considering the fact that I get all the way to a 250 Ok I don't think that's the issue.
Turns out the issue is Window's version of OpenSSL. Running the exact same command on a linux machine works just fine. Leaving answer here and closing just in case someone else follows in my footsteps and wonders why their windows version isn't working.

OpenSSL C++ DTLS client

I'm trying to establish DTLS connection using openssl (c++).
However whereas there is DTLSv1_listen() function for server side I can't find any client side equivalent to actually establish UDP connection to server from client. Or send something to DTLS server. Could someone help me understand how to establish "connection" to DTLS server (I know the point of UDP and datagram communication is to be connectionless but by connection I mean scheme like DNS request+response)?
I need to send single message to server and then receive 1 response message. Encrypted. With certificate verification. How such communication scheme would work in DTLS world?
What are you using for signalling. You actually don't have to use openssl for your signalling layer you could use a Memory Bio and read and write from it.
With DTLS You will have 2 sides here, a Client and a Server. The Client will initiate things with a Client Hello. The server hopefully receives it, writes it into the bio, reads back a Server Hello and responds via the signalling layer.
There is a lot of stuff required to really finish up your app into the real world so I wont cover everything.
If you have a SSL Context, and you have your BIO's created. To Initiate the Server:
SSL_set_accept_state(*sslContext)
and to initiate the client:
SSL_set_connect_state(*sslContext);
You then want to start a handshake on the client. There is so many factors here on how your application is going to work its hard to give advice but the client should call:
SSL_do_handshake(*sslContext);
Depending on how you have wired everything up, your program may automatically send the client hello at this point. You can use Wireshark, grab only UDP and at the top filter DTLS to see it. If not you may be required to signal manually.
I wrote the following program myself to test OpenSSL dll's to see if there were any issues after compiling them. I can't share all the code with you but it shows how in memory you can use a single console app to do the handshake (no transmission via the internet just to see how it works).
Notes:
The BIO is a Memory BIO in this case.
Handshake is a SSL_do_handshake
WriteCipherText is a call to BIO_write
ReadCipherText is a call to BIO_read
You have to open it and do the other basic setup states first. At the end both of the handshakes will return 1 for succcess. This just kind of shows you how to do a basic memory DTLS handshake using openssl.
Console.WriteLine("[Client] Open: " + Client.Open("ECDHE-ECDSA-AES128-SHA:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-SHA256:AES128-SHA:AES128-GCM-SHA256:AES128-SHA256", "SRTP_AES128_CM_SHA1_80"));
Console.WriteLine("[Server] Open: " + Server.Open("ECDHE-ECDSA-AES128-SHA:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-SHA256:AES128-SHA:AES128-GCM-SHA256:AES128-SHA256", "SRTP_AES128_CM_SHA1_80"));
Console.WriteLine("[Client] Handshake" + Client.Handshake());
Console.WriteLine("Client: [Read] Client Hello");
var clientHello = Client.ReadCipherText();
Console.WriteLine("[Server] Write Hello: " + Server.WriteCipherText(clientHello));
Console.WriteLine("[Server] Handshake" + Server.Handshake());
clientHello.Free();
Console.WriteLine("[Server] Read Server Hello");
var serverHello = Server.ReadCipherText();
Console.WriteLine("[Client] Write Server Hello: " + Client.WriteCipherText(serverHello));
Console.WriteLine("[Client] Handshake" + Client.Handshake());
serverHello.Free();
Console.WriteLine("[Client] Read Certificate");
var clientCertificate = Client.ReadCipherText();
Console.WriteLine("[Server] Write Certificate: " + Server.WriteCipherText(clientCertificate));
Console.WriteLine("[Server] Handshake: " + Server.Handshake());
clientCertificate.Free();
Console.WriteLine("[Server] Read Change Cipher Spec");
var serverChangeCipherSpec = Server.ReadCipherText();
Console.WriteLine("[Client] Write Change Cipher Spec: " + Client.WriteCipherText(serverChangeCipherSpec));
serverChangeCipherSpec.Free();
Console.WriteLine("[Client] Handshake" + Client.Handshake());
Console.WriteLine("[Server] Handshake" + Server.Handshake());
Console.ReadLine();
Lots of helpful resources too to take a look at:
https://web.archive.org/web/20150814081716/http://sctp.fh-muenster.de/dtls-samples.html This ones really good
http://chris-wood.github.io/2016/05/06/OpenSSL-DTLS.html
https://wiki.openssl.org/index.php/SSL/TLS_Client
On the client side you create the UDP-socket, pass it with BIO_new_dgram to the SSL context and connect it with connect() or BIO_ctrl(bio, BIO_CTRL_DGRAM_SET_CONNECTED, 0, &server_sockaddr). The connect just forces the file descriptors write to always be sent to server_sockaddr.
Now just do a regular SSL_connect and it should work.
For server side it's a little more complicated. I made a post and an example implementation

OpenSSL client not sending client certificate

I am struggling with a client certificate problem and hope somebody here can help me. I'm developing a client/server pair using boost asio but I'll try to be unspecific. I'm on windows and using openssl 1.0.1e
Basically, I want to have client authentication by using client certificates. The server shall only accept clients that have a certificate signed by my own CA. So I have setup a self signed CA. This has issued two more certificates. One for the client and one for the server. Both signed by the CA.
I have done that quite a few times now and I am confident that I got it.
My server side also works fine. It requests client certificates and if I'm using s_client and give those certs everything works. Also if I'm using a browser and have my root CA installed as trusted and then import the client certs.
The only thing that I can't get to work is the libssl client. It always fails during the handshake and as far as I can see it will not send the client certficate:
$ openssl.exe s_server -servername localhost -bugs -CAfile myca.crt -cert server.crt
-cert2 server.crt -key private/server.key -key2 private/server.key -accept 8887 -www
-state -Verify 5
verify depth is 5, must return a certificate
Setting secondary ctx parameters
Using default temp DH parameters
Using default temp ECDH parameters
ACCEPT
SSL_accept:before/accept initialization
SSL_accept:SSLv3 read client hello A
SSL_accept:SSLv3 write server hello A
SSL_accept:SSLv3 write certificate A
SSL_accept:SSLv3 write key exchange A
SSL_accept:SSLv3 write certificate request A
SSL_accept:SSLv3 flush data
SSL3 alert read:warning:no certificate
SSL3 alert write:fatal:handshake failure
SSL_accept:error in SSLv3 read client certificate B
SSL_accept:error in SSLv3 read client certificate B
2675716:error:140890C7:SSL routines:SSL3_GET_CLIENT_CERTIFICATE:peer did not return a
certificate:s3_srvr.c:3193:
ACCEPT
I'm using this s_server as debugging tool but against my real server the same thing occurs.
s_client will work fine with the same certificates. Also, if I disable "-Verify" in the server the connection works. So it really seems just the client refusing to send it's certficate. What can be the reason for that?
Since I'm using boost asio as an SSL wrapper the code looks like this:
m_ssl_context.set_verify_mode( asio::ssl::context::verify_peer );
m_ssl_context.load_verify_file( "myca.crt" );
m_ssl_context.use_certificate_file( "testclient.crt", asio::ssl::context::pem );
m_ssl_context.use_private_key_file( "testclient.key", asio::ssl::context::pem );
I have also tried to bypass asio and access the SSL context directly by saying:
SSL_CTX *ctx = m_ssl_context.impl();
SSL *ssl = m_ssl_socket.impl()->ssl;
int res = 0;
res = SSL_CTX_use_certificate_chain_file(ctx, "myca.crt");
if (res <= 0) {
// handle error
}
res = SSL_CTX_use_certificate_file(ctx, "testclient.crt", SSL_FILETYPE_PEM);
if (res <= 0) {
// handle error
}
res = SSL_CTX_use_PrivateKey_file(ctx, "testclient.key", SSL_FILETYPE_PEM);
if (res <= 0) {
// handle error
}
I can't see any difference in behavior. It should be mentioned that I am using a very old boost 1.43 asio which I cannot update but I suppose all relevant calls go more or less directly to OpenSSL anyway and the server works fine with that version so I think I can rule that out.
If I start forcing client and server to specific versions, the error messages change but it never works and still always works with the s_client test. Currently it is set to TLSv1
If I switch it to TLSv1 for example there is more chatter between client and server and eventually I get the error:
...
SSL_accept:SSLv3 read client key exchange A
<<< TLS 1.0 ChangeCipherSpec [length 0001]
01
<<< TLS 1.0 Handshake [length 0010], Finished
14 00 00 0c f4 71 28 4d ab e3 dd f2 46 e8 8b ed
>>> TLS 1.0 Alert [length 0002], fatal unexpected_message
02 0a
SSL3 alert write:fatal:unexpected_message
SSL_accept:failed in SSLv3 read certificate verify B
2675716:error:140880AE:SSL routines:SSL3_GET_CERT_VERIFY:missing verify
message:s3_srvr.c:2951:
2675716:error:140940E5:SSL routines:SSL3_READ_BYTES:ssl handshake failure:s3_pkt.c:989:
ACCEPT
I have found an older bug entry posted on the openssl mailing list that refereed to this. Apparently a wrong CRLF in the handshake that has been fixed two yrs ago. Or has it?
I have been debugging this for almost a week now and I'm really stuck. Does anyone have a suggestion on what to try? I'm out of ideas...
Cheers,
Stephan
PS: Here is what the above s_server debug out would be with s_client and the same certficate:
$ openssl s_client -CAfile ca.crt -cert testclient.crt -key private/testclient.key -verify 2 -connect myhost:8887
ACCEPT
SSL_accept:before/accept initialization
SSL_accept:SSLv3 read client hello A
SSL_accept:SSLv3 write server hello A
SSL_accept:SSLv3 write certificate A
SSL_accept:SSLv3 write key exchange A
SSL_accept:SSLv3 write certificate request A
SSL_accept:SSLv3 flush data
depth=1 C = DE, // further info
verify return:1
depth=0 C = DE, // further info
verify return:1
SSL_accept:SSLv3 read client certificate A
SSL_accept:SSLv3 read client key exchange A
SSL_accept:SSLv3 read certificate verify A
SSL_accept:SSLv3 read finished A
SSL_accept:SSLv3 write session ticket A
SSL_accept:SSLv3 write change cipher spec A
SSL_accept:SSLv3 write finished A
SSL_accept:SSLv3 flush data
ACCEPT
... handshake completes and data is transferred.
All right, after much suffering, the answer has been found by Dave Thompson of OpenSSL.
The reason was that my ssl code called all those functions on the OpenSSL context after the socket object (SSL*) was created from it. Which means all those functions did practically nothing or the wrong thing.
All I had to do was either:
1. Call SSL_use_certificate_file
res = SSL_use_certificate_file(ssl, "testclient.crt", SSL_FILETYPE_PEM);
if (res <= 0) {
// handle error
}
res = SSL_use_PrivateKey_file(ssl, "testclient.key", SSL_FILETYPE_PEM);
if (res <= 0) {
// handle error
}
(notice the missing CTX)
2. Call the CTX functions
Call the CTX functions upon the context before the socket was created. As asio seemingly encourages to create the context and socket right afterwards (as I did in the initializer list) the calls were all but useless.
The SSL context (in lib OpenSSL or asio alike) encapsulates the SSL usage and each socket created from it will share it's properties.
Thank you guys for your suggestions.
You should not use both SSL_CTX_use_certificate_chain_file() and SSL_CTX_use_certificate_file(), as SSL_CTX_use_certificate_chain_file() tries to load a chain including the client certificate, not just the CA chain. From SSL_CTX_use_certificate(3):
SSL_CTX_use_certificate_chain_file() loads a certificate chain from file into ctx. The certificates must be in PEM format and must be sorted starting with the subject's certificate (actual client or server certificate), followed by intermediate CA certificates if applicable, and ending at the highest level (root) CA.
I think you should be fine using only SSL_CTX_use_certificate_file() and SSL_CTX_use_PrivateKey_file(), as the client does not care much for the CA chain anyway.
I think you need to call SSL_CTX_set_client_CA_list on the server side. This sets a list of certificate authorities to be sent together with the client certificate request.
The client will not send its certificate, even if one was requested, if the certificate does not match that CA list sent by the server.