When I try to connect to some hosts(not all) through HTTPS using OpenSSL in C++ I gets OpenSSL error error:14094410:SSL routines:ssl3_read_bytes:sslv3 alert handshake failure. I'm using TLS_client_method SSL method. But if I use openssl test executable, it's ok - openssl s_client -connect host:443 -tls1_2.
My connect code here:
const SSL_METHOD* method = TLS_client_method();
inet->ssl_ctx = SSL_CTX_new(method);
inet->ssl = SSL_new(inet->ssl_ctx);
SSL_set_fd(inet->ssl, s);
int err = SSL_connect(inet->ssl);
Why? Maybe I need some .pem/.pm files? I don't know, but I saw it somewhere.
Your code does not use the SNI extension when connecting to the server, i.e. does not include the hostname of the server into the TLS handshake. Multi-domain sites usually require SNI and might fail or return some unrelated certificate when SNI is not provided. This is also true for CDN like Cloudflare where different domains are accessible by the same IP address but should result in different certificates.
... I tried to connect to hostiman.ru
This is for example the case with hostiman.ru. With SNI (as set by newer s_client versions by default):
$ openssl s_client -connect hostiman.ru:443 -tls1_2
CONNECTED(00000005)
...
depth=0 C = US, ST = CA, L = San Francisco, O = "Cloudflare, Inc.", CN = sni.cloudflaressl.com
...
Cipher : ECDHE-ECDSA-CHACHA20-POLY1305
Without SNI it instead looks like this:
$ openssl s_client -connect hostiman.ru:443 -tls1_2 -noservername
CONNECTED(00000005)
140420123775424:error:14094410:SSL routines:ssl3_read_bytes:sslv3 alert handshake failure:../ssl/record/rec_layer_s3.c:1528:SSL alert number 40
...
Cipher : 0000
To setup SNI in the client use SSL_set_tlsext_host_name, i.e.
inet->ssl = SSL_new(inet->ssl_ctx);
SSL_set_tlsext_host_name(inet->ssl, servername)
Related
I'm trying to initiate a TLS connection and it must include a SNI extension. The following program works but, despite SSL_set_tlsext_host_name being called, does not produce a SNI record in the Client Hello packet:
#include <openssl/ssl.h>
#include <openssl/bio.h>
int main()
{
SSL_CTX* ctx = SSL_CTX_new(TLS_client_method());
SSL* ssl = SSL_new(ctx);
BIO* bio = BIO_new_ssl_connect(ctx);
BIO_set_conn_hostname(bio, "demo.piesocket.com:443");
SSL_set_tlsext_host_name(ssl, "demo.piesocket.com");
BIO_do_connect(bio);
BIO_free_all(bio);
SSL_free(ssl);
SSL_CTX_free(ctx);
return 0;
}
Yet the following command does:
openssl s_client demo.piesocket.com:443
I've checked both exchanges with Wireshark and the Client Hello packets differ only in the absence/presence of SNI. Yes, I've looked long and hard at s_client.c but it does too many things and I'm new to OpenSSL and TLS. What am I missing?
(For brevity I've removed all checks from this example code but it works, the server I'm using here doesn't seem to require SNI and the TLS connection is indeed established in both cases. The TLS portion of the packet from openssl.exe looks like this:
TLSv1.2 Record Layer: Handshake Protocol: Client Hello
Content Type: Handshake (22)
Version: TLS 1.0 (0x0301)
Length: 319
Handshake Protocol: Client Hello
Handshake Type: Client Hello (1)
Length: 315
Version: TLS 1.2 (0x0303)
Random: 8772af2a36342435d6b73a0593087c229b67342030d23ae5…
Session ID Length: 32
Session ID: 49a1e1b7eb761bd9b279efcb4cac15bae2f09bb92e641a75…
Cipher Suites Length: 62
Cipher Suites (31 suites)
Compression Methods Length: 1
Compression Methods (1 method)
Extensions Length: 180
Extension: server_name (len=23)
Extension: ec_point_formats (len=4)
Extension: supported_groups (len=22)
Extension: session_ticket (len=0)
Extension: encrypt_then_mac (len=0)
Extension: extended_master_secret (len=0)
Extension: signature_algorithms (len=42)
Extension: supported_versions (len=9)
Extension: psk_key_exchange_modes (len=2)
Extension: key_share (len=38)
The packet from the program doesn't have the Extension: server_name (len=23) line. OpenSSL version 3.0.7, Windows 7, MinGW.)
You are creating an SSL BIO via BIO_new_ssl_connect. This essentially creates a BIO and inside it creates a new SSL object for the connection based on the SSL_CTX that you pass it. Entirely separate to that you are creating a different SSL object, setting the SNI hostname on it, and then you're not using it for anything - you just free it.
Don't create a separate SSL object. Instead get hold of the SSL object inside the BIO using BIO_get_ssl. Set the SNI hostname on that.
https://www.openssl.org/docs/man3.0/man3/BIO_get_ssl.html
I'm following the steps listed in the Documentation and I can get a successful connection but attempting to actually send an email gets a 250 Ok response but no message id and no message is sent to my inbox.
openssl s_client -crlf -starttls smtp -connect email-smtp.us-east-1.amazonaws.com:587 0<input.txt
CONNECTED(000002F4)
depth=3 C = US, ST = Arizona, L = Scottsdale, O = "Starfield Technologies, Inc.", CN = Starfield Services Root Certificate Authority - G2
verify error:num=20:unable to get local issuer certificate
---
Certificate chain
0 s:/CN=email-smtp.us-east-1.amazonaws.com
i:/C=US/O=Amazon/OU=Server CA 1B/CN=Amazon
1 s:/C=US/O=Amazon/OU=Server CA 1B/CN=Amazon
i:/C=US/O=Amazon/CN=Amazon Root CA 1
2 s:/C=US/O=Amazon/CN=Amazon Root CA 1
i:/C=US/ST=Arizona/L=Scottsdale/O=Starfield Technologies, Inc./CN=Starfield Services Root Certificate Authority - G2
3 s:/C=US/ST=Arizona/L=Scottsdale/O=Starfield Technologies, Inc./CN=Starfield Services Root Certificate Authority - G2
i:/C=US/O=Starfield Technologies, Inc./OU=Starfield Class 2 Certification Authority
---
Server certificate
-----BEGIN CERTIFICATE-----
<super long server certificate>
-----END CERTIFICATE-----
subject=/CN=email-smtp.us-east-1.amazonaws.com
issuer=/C=US/O=Amazon/OU=Server CA 1B/CN=Amazon
---
No client certificate CA names sent
Peer signing digest: SHA512
Server Temp Key: ECDH, P-256, 256 bits
---
SSL handshake has read 5292 bytes and written 469 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES256-GCM-SHA384
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
Protocol : TLSv1.2
Cipher : ECDHE-RSA-AES256-GCM-SHA384
Session-ID: 5B1EDC39B0B80D50AEAEF28A1F7E49846B77A4B9FF5A7BA347A620F56F9D0FE8
Session-ID-ctx:
Master-Key: <Long master key>
Key-Arg : None
PSK identity: None
PSK identity hint: None
SRP username: None
Start Time: 1528749113
Timeout : 300 (sec)
Verify return code: 20 (unable to get local issuer certificate)
---
250 Ok
421 Timeout waiting for data from client.
closed
The main thing that has me worried is this: Verify return code: 20 (unable to get local issuer certificate) which isn't mentioned anywhere in the documentation and makes me wonder if I'm supposed to be sending a cert of some sort. My Input file looks like this:
EHLO domain.com
AUTH LOGIN
smtp_username
smtp_password line 1
smtp_password line 2
MAIL FROM: verified_email1#domain.com
RCPT TO: verified_email1#domain.com
DATA
From: Sender Name <verified_email1#domain.com>
To: verified_email1#domain.com
Subject: Amazon SES SMTP Test
This message was sent using the Amazon SES SMTP interface.
.
QUIT
Does anyone have any idea what I'm doing wrong? I am attempting to do this on a Windows 10 os using the openssl for Windows if that could be the problem but considering the fact that I get all the way to a 250 Ok I don't think that's the issue.
Turns out the issue is Window's version of OpenSSL. Running the exact same command on a linux machine works just fine. Leaving answer here and closing just in case someone else follows in my footsteps and wonders why their windows version isn't working.
I am new to python and still learning it so my question can be little naive. Please bear with it ;)
The problem is client will be sending CSR and I want to sign it with my CA root certificate and return the signed certificate back to client.
I have been using this command to do it using command line
openssl x509 -req -in device.csr -CA root.pem -CAkey root.key -CAcreateserial -out device.crt -days 500
same thing I want achieve using python. I have come across python library for openssl pyopenssl
is it possible using this library ? How ? or shoudl I go for M2Crypto ?
You can indeed go with pyOpenSSL. As you are saying you already have CA root certificate and a private key, and CSR will be sent by a client then you can use functions of crypto to read all those ( CA cert, private key and Device CSR ) from file or manage to have them in buffer.
Use below functions to start with. Check dir(crypto) and crypto.function_name.__doc__on python interpreter for more info :) You need to import crypto from pyOpenSSL
crypto.load_certificate_request() - to get device CSR obj
crypto.load_privatekey() - to get private key obj for CA private key
crypto.load_certificate() - to get CA root certificate
then you can write simple funcation to return certificate
def create_cert():
cert = crypto.X509()
cert.set_serial_number(serial_no)
cert.gmtime_adj_notBefore(notBeforeVal)
cert.gmtime_adj_notAfter(notAfterVal)
cert.set_issuer(caCert.get_subject())
cert.set_subject(deviceCsr.get_subject())
cert.set_pubkey(deviceCsr.get_pubkey())
cert.sign(CAprivatekey, digest)
return cert
where caCert , deviceCsr and CAprivatekey are values from above three funcations.
Now that you have certificate with you, you can write this to a file using crypto.dump_certificate(crypto.FILETYPE_PEM, cert) with file name of your choice.
You can modify this function as per your requirement. After this you can verify generated device certificate with CA root certificate with openssl command e.g. openssl verify -CApath <CA cert path> <name of device cert file>
You can also go through few examples from github.
M2Crypto Example , pyOpenSSL example
Hope this gives you idea about the implementation
The maintainer of pyOpenSSL recommends to use cryptography module for X509 manipulation (see note on top of the documentation page: https://www.pyopenssl.org/en/stable/api/crypto.html).
Here is the code to create a certificate from a CSR signed by a CA:
def sign_certificate_request(csr_cert, ca_cert, private_ca_key):
cert = x509.CertificateBuilder().subject_name(
csr_cert.subject
).issuer_name(
ca_cert.subject
).public_key(
csr_cert.public_key()
).serial_number(
x509.random_serial_number()
).not_valid_before(
datetime.utcnow()
).not_valid_after(
# Our certificate will be valid for 10 days
datetime.utcnow() + timedelta(days=10)
# Sign our certificate with our private key
).sign(private_ca_key, hashes.SHA256())
# return DER certificate
return cert.public_bytes(serialization.Encoding.DER)
csr_cert is the cryptography CSR certificate object - can be loaded from a file with x509.load_der_x509_csr()
ca_cert is the cryptography certificate object - can be loaded from a file with x509.load_pem_x509_certificate()
private_ca_key is the cryptography private key object - can be loaded from a file with serialization.load_pem_private_key()
I am struggling with a client certificate problem and hope somebody here can help me. I'm developing a client/server pair using boost asio but I'll try to be unspecific. I'm on windows and using openssl 1.0.1e
Basically, I want to have client authentication by using client certificates. The server shall only accept clients that have a certificate signed by my own CA. So I have setup a self signed CA. This has issued two more certificates. One for the client and one for the server. Both signed by the CA.
I have done that quite a few times now and I am confident that I got it.
My server side also works fine. It requests client certificates and if I'm using s_client and give those certs everything works. Also if I'm using a browser and have my root CA installed as trusted and then import the client certs.
The only thing that I can't get to work is the libssl client. It always fails during the handshake and as far as I can see it will not send the client certficate:
$ openssl.exe s_server -servername localhost -bugs -CAfile myca.crt -cert server.crt
-cert2 server.crt -key private/server.key -key2 private/server.key -accept 8887 -www
-state -Verify 5
verify depth is 5, must return a certificate
Setting secondary ctx parameters
Using default temp DH parameters
Using default temp ECDH parameters
ACCEPT
SSL_accept:before/accept initialization
SSL_accept:SSLv3 read client hello A
SSL_accept:SSLv3 write server hello A
SSL_accept:SSLv3 write certificate A
SSL_accept:SSLv3 write key exchange A
SSL_accept:SSLv3 write certificate request A
SSL_accept:SSLv3 flush data
SSL3 alert read:warning:no certificate
SSL3 alert write:fatal:handshake failure
SSL_accept:error in SSLv3 read client certificate B
SSL_accept:error in SSLv3 read client certificate B
2675716:error:140890C7:SSL routines:SSL3_GET_CLIENT_CERTIFICATE:peer did not return a
certificate:s3_srvr.c:3193:
ACCEPT
I'm using this s_server as debugging tool but against my real server the same thing occurs.
s_client will work fine with the same certificates. Also, if I disable "-Verify" in the server the connection works. So it really seems just the client refusing to send it's certficate. What can be the reason for that?
Since I'm using boost asio as an SSL wrapper the code looks like this:
m_ssl_context.set_verify_mode( asio::ssl::context::verify_peer );
m_ssl_context.load_verify_file( "myca.crt" );
m_ssl_context.use_certificate_file( "testclient.crt", asio::ssl::context::pem );
m_ssl_context.use_private_key_file( "testclient.key", asio::ssl::context::pem );
I have also tried to bypass asio and access the SSL context directly by saying:
SSL_CTX *ctx = m_ssl_context.impl();
SSL *ssl = m_ssl_socket.impl()->ssl;
int res = 0;
res = SSL_CTX_use_certificate_chain_file(ctx, "myca.crt");
if (res <= 0) {
// handle error
}
res = SSL_CTX_use_certificate_file(ctx, "testclient.crt", SSL_FILETYPE_PEM);
if (res <= 0) {
// handle error
}
res = SSL_CTX_use_PrivateKey_file(ctx, "testclient.key", SSL_FILETYPE_PEM);
if (res <= 0) {
// handle error
}
I can't see any difference in behavior. It should be mentioned that I am using a very old boost 1.43 asio which I cannot update but I suppose all relevant calls go more or less directly to OpenSSL anyway and the server works fine with that version so I think I can rule that out.
If I start forcing client and server to specific versions, the error messages change but it never works and still always works with the s_client test. Currently it is set to TLSv1
If I switch it to TLSv1 for example there is more chatter between client and server and eventually I get the error:
...
SSL_accept:SSLv3 read client key exchange A
<<< TLS 1.0 ChangeCipherSpec [length 0001]
01
<<< TLS 1.0 Handshake [length 0010], Finished
14 00 00 0c f4 71 28 4d ab e3 dd f2 46 e8 8b ed
>>> TLS 1.0 Alert [length 0002], fatal unexpected_message
02 0a
SSL3 alert write:fatal:unexpected_message
SSL_accept:failed in SSLv3 read certificate verify B
2675716:error:140880AE:SSL routines:SSL3_GET_CERT_VERIFY:missing verify
message:s3_srvr.c:2951:
2675716:error:140940E5:SSL routines:SSL3_READ_BYTES:ssl handshake failure:s3_pkt.c:989:
ACCEPT
I have found an older bug entry posted on the openssl mailing list that refereed to this. Apparently a wrong CRLF in the handshake that has been fixed two yrs ago. Or has it?
I have been debugging this for almost a week now and I'm really stuck. Does anyone have a suggestion on what to try? I'm out of ideas...
Cheers,
Stephan
PS: Here is what the above s_server debug out would be with s_client and the same certficate:
$ openssl s_client -CAfile ca.crt -cert testclient.crt -key private/testclient.key -verify 2 -connect myhost:8887
ACCEPT
SSL_accept:before/accept initialization
SSL_accept:SSLv3 read client hello A
SSL_accept:SSLv3 write server hello A
SSL_accept:SSLv3 write certificate A
SSL_accept:SSLv3 write key exchange A
SSL_accept:SSLv3 write certificate request A
SSL_accept:SSLv3 flush data
depth=1 C = DE, // further info
verify return:1
depth=0 C = DE, // further info
verify return:1
SSL_accept:SSLv3 read client certificate A
SSL_accept:SSLv3 read client key exchange A
SSL_accept:SSLv3 read certificate verify A
SSL_accept:SSLv3 read finished A
SSL_accept:SSLv3 write session ticket A
SSL_accept:SSLv3 write change cipher spec A
SSL_accept:SSLv3 write finished A
SSL_accept:SSLv3 flush data
ACCEPT
... handshake completes and data is transferred.
All right, after much suffering, the answer has been found by Dave Thompson of OpenSSL.
The reason was that my ssl code called all those functions on the OpenSSL context after the socket object (SSL*) was created from it. Which means all those functions did practically nothing or the wrong thing.
All I had to do was either:
1. Call SSL_use_certificate_file
res = SSL_use_certificate_file(ssl, "testclient.crt", SSL_FILETYPE_PEM);
if (res <= 0) {
// handle error
}
res = SSL_use_PrivateKey_file(ssl, "testclient.key", SSL_FILETYPE_PEM);
if (res <= 0) {
// handle error
}
(notice the missing CTX)
2. Call the CTX functions
Call the CTX functions upon the context before the socket was created. As asio seemingly encourages to create the context and socket right afterwards (as I did in the initializer list) the calls were all but useless.
The SSL context (in lib OpenSSL or asio alike) encapsulates the SSL usage and each socket created from it will share it's properties.
Thank you guys for your suggestions.
You should not use both SSL_CTX_use_certificate_chain_file() and SSL_CTX_use_certificate_file(), as SSL_CTX_use_certificate_chain_file() tries to load a chain including the client certificate, not just the CA chain. From SSL_CTX_use_certificate(3):
SSL_CTX_use_certificate_chain_file() loads a certificate chain from file into ctx. The certificates must be in PEM format and must be sorted starting with the subject's certificate (actual client or server certificate), followed by intermediate CA certificates if applicable, and ending at the highest level (root) CA.
I think you should be fine using only SSL_CTX_use_certificate_file() and SSL_CTX_use_PrivateKey_file(), as the client does not care much for the CA chain anyway.
I think you need to call SSL_CTX_set_client_CA_list on the server side. This sets a list of certificate authorities to be sent together with the client certificate request.
The client will not send its certificate, even if one was requested, if the certificate does not match that CA list sent by the server.
I'm trying to use SSLSniff's tool, and I have some technical issues... I've been looking for any similar problems, but the only results are from Twitter feeds, with no public useful answer. So, here it is:
(My version of SSLSniff is 0.8) I'm launching sslsniff with args:
sslsniff -a -c cert_and_key.pem -s 12345 -w out.log
where: cert_and_key.pem file is my authority's certificate concatenate with my unencrypted private key (in PEM format of course), and 12345 is the port where I redirect traffic with my iptables rule.
So sslsniff is correctly running:
INFO sslsniff : Certificate ready: [...]
[And anytime I connect with a client, there are these 2 following lines:]
DEBUG sslsniff : SSL Accept Failed!
DEBUG sslsniff : Got exception: Error with SSL connection.
On my client' side, I've register my AC as a trusted CA (with FF). Then when I connect through SSL I'm having the error:
Secure Connection Failed.
Error code: ssl_error_bad_cert_domain
What is super strange (moreover the fact that the certificate is not automatically accepted since it should be signed by my trusted CA) is that I cannot accept the forged certificate by clicking on "Add exception..." : I am always returning to the error page asking me to add an(other) exception...
Moreover, when I try to connect to for example: https://www.google.com, SSLSniff's log is completed with a new line :
DEBUG sslsniff : Encoded Length: 7064 too big for session cache, skipping...
Does anyone know what I'm doing wrong?
-- Edit to summer up the different answers --
The problem is that SSLSniff is not taking care of alternive names when it forges certificates. Apparently, Firefox refuses any certificate as soon as the Common Name doesn't match exactly the domain name.
For example, for Google.com : CN = www.google.com and there is no alternative name. So when you connect to https://www.google.com, it's working fine.
But for Google.fr : CN = *.google.fr, with these alternative names: *.google.fr and google.fr. So when you connect to https://www.google.fr, FF is looking for alternative names and, since it obviously doesn't find any, refuses the malformed certificate.
... So a solution would be to patch/commit... I don't know if Moxie Marlinspike has intentionally forgot this functionnality because it was too complicated, or if he was just not aware of this issue. Anyway, I'll try to have a look at the code.
The session encoded length error message: When caching the SSL session fails, it means that SSL session resumption on subsequent connections will fail, resulting in degraded performance, because a full SSL handshake needs to be done on every request. However, despite using the CPU more heavily, sslsniff will still work fine. The caching fails because the serialized representation of the OpenSSL session object (SSL_SESSION) was larger than the maximum size supported by sslsniff's session cache.
As for your real problem, note that sslsniff does not support X.509v3 subjectAltNames, so if you are connecting to a site whose hostname does not match the subject common name of the certificate, but instead matches only a subjectAltName, then sslsniff will generate a forged certificate without subjectAltNames, which will cause a hostname verification mismatch on the connecting client.
If your problem happens only for some specific sites, let us know the site so we can examine the server certificate using e.g. openssl s_client -connect host:port -showcerts and openssl x509 -in servercert.pem -text. If it happens for all sites, then the above is not the explanation.
Try a straight MITM with a cert you fully control , and make sure you don't have some OCSP/Perspectives/Convergance stuff meddling with things. Other than that, maybe add the cert to the OS trusted roots. I think FF on windows uses the windows cert store (start->run->certmgr.msc). It may also be worth trying with something like Burp to see if the error is localized to SSLSniff or all MITM attempts.