SSL_connect() produces certificate verify failure - c++

I'm currently rewriting some existing technologies that were once using RSA Security's libraries into OpenSSL, but I'm starting to run into a few issues. Currently, all of the certificate verification code appears to be running without a hitch, until that is, I call SSL_connect().
Before, the call to SSL_connect() would produce SSL_ERROR_WANT_READ.
An answer to this issue on another forum suggested to me that SSL_connect() should be called until it stops producing SSL_ERROR_WANT_READ errors. Unforunately, this only produces something more confusing:
error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
even though SSL_CTX_load_verify_locations() succeeds. Does anyone have any idea as to why a verification error wouldn't register with certificate methods and wait until SSL_connect() is triggered?

Usually this error means that the server certificate your client received in response to SSL_connect() couldn't be verified.
This can happen for different reasons:
If the server certificate is self-signed, you'll have to authorize that on your SSL_CONTEXT.
If the server certificate was signed by a certificate authority that is not in the list of trusted CA certificates
If the server certificate is not valid yet or not valid anymore
Actually, you should set a callback for certificate verification and make it accept any certificate, so you can focus on the connection part. Once it works, just tweak your callback or check your certificates to be valid.
At any time you get a failure, you can call some SSL_get_error() function that will indicate you why the certificate was rejected.
(Unfortunately, I can't access my code base right now, so I cannot give concrete examples)
Here are some code samples from my own SSL Socket wrapper class, adapted for you:
// ctx is a SSL_CONTEXT
// internalCertificateVerificationCallback is a callback static method (or function)
ctx->setVerify(SSL_VERIFY_PEER | SSL_VERIFY_FAIL_IF_NO_PEER_CERT, internalCertificateVerificationCallback);
Here is the definition for internalCertificateVerificationCallback:
int SecureSocket::internalCertificateVerificationCallback(int preverify_ok, X509_STORE_CTX* x509_ctx)
{
//preverify_ok contains 1 if the pre-verification succeeded, 0 otherwise.
return 1; // This accepts every certificate
}

Related

OpenSSL how to request client certificate, but don't verify it

I want to setup openssl c/c++ server request certificate from client but don't verify it.
I already use this piece of code to query certificate from client:
/** Force the client-side have a certificate **/
SSL_CTX_set_verify(ctx, SSL_VERIFY_PEER | SSL_VERIFY_FAIL_IF_NO_PEER_CERT, nullptr);
SSL_CTX_set_verify_depth(ctx, 4);
Can anybody give me example of such server code?
You should read the manual. It states:
void SSL_CTX_set_verify(SSL_CTX *ctx, int mode, SSL_verify_cb verify_callback);
[...]
The return value of verify_callback controls the strategy of the further verification process. If verify_callback returns 0, the verification process is immediately stopped with "verification failed" state. [...] If verify_callback always returns 1, the TLS/SSL handshake will not be terminated with respect to verification failures and the connection will be established. [...]
Writing a function that always returns 1 and passing it as verify_callback should help you solve the problem.

How to do "Client Authentification" if I'm not able to provide private key

I want to set up my local server to communicate with my client. They build TLS connection using Openssl. I try to implement double side authentication, like server would verify client and client also needs to verify server.
When I use certificates generated by my self, everything works fine. The code is as following. It's C++ code in client. I set up client cert, private key and intermediate cert. In server side I saved a CA cert.
The relationship is: CA signs intermediate cert, intermediate cert signs client cert.
As we know, the reason that we need to provide client private key is the client will signature a "challenge" then send to server. Server could get client public key by certificate chain and use it to decode the encrypt "challenge" to see if they matched. You could see this link for detailed process:
https://en.wikipedia.org/wiki/Transport_Layer_Security#TLS_handshake
However in my scenario, I have no permission to get the private key. I only have an API to call, which takes the digest or anything we want to encode as input and return a string encoded by client private key.
Therefore I'm not able to pass any "ClientPrivateKeyFileTest" to TLS.
I searched openssl source code but all handshakes were done in this function: SSL_do_handshake() and I'm not allowed to modify this function.
// load client-side cert and key
SSL_CTX_use_certificate_file(m_ctx, ClientCertificateFileTest, SSL_FILETYPE_PEM);
SSL_CTX_use_PrivateKey_file(m_ctx, ClientPrivateKeyFileTest, SSL_FILETYPE_PEM);
// load intermediate cert
X509* chaincert = X509_new();
BIO* bio_cert = BIO_new_file(SignerCertificateFileTest, "rb");
PEM_read_bio_X509(bio_cert, &chaincert, NULL, NULL);
SSL_CTX_add1_chain_cert(m_ctx, chaincert)
m_ssl = SSL_new(m_ctx);
// get_seocket is my own API
m_sock = get_socket();
SSL_set_fd(m_ssl, m_sock)
// doing handshake and build connection
auto r = SSL_connect(m_ssl);
I think all handshake processes would be done after I call SSL_connect(). So I wonder is there other way I can do to complete the client-authentication?
For example, I could skip adding private key step but set up a callback function somewhere which can handle all cases when SSL needs to use private key to calculate something.
PS: The API is a black box in the client machine.
One more thing, these days I found that openssl engine may help this problem. But does anybody know what kind of engine is useful for this problem? The EC sign, verification or others?
Final update: I implemented a OpenSSL engine to reload EC_KEY_METHOD so that I'm able to use my own sign function.
Thanks a lot!

Boost, asio, https, and host/certificate verifcation

I'm looking at Boost's SSL Client. There's a reference to OpenSSL in the comments (sorry, no line numbers):
// The verify callback can be used to check whether the certificate that is
// being presented is valid for the peer. For example, RFC 2818 describes
// the steps involved in doing this for HTTPS. Consult the OpenSSL
// documentation for more details. Note that the callback is called once
// for each certificate in the certificate chain, starting from the root
// certificate authority.
Proper OpenSSL use and verification can be tricky. From experience, I know I have to perform the following to use the library correctly:
Disable SSLv2, SSLv3, and Compression on the Context object
Provide the proper root certificate for chain building and checking
Call SSL_get_peer_certificate and verify the certificate is non-NULL
Call SSL_get_verify_result and verify the result is X509_V_OK
Perform name matching (CN or SAN must match requested host)
OpenSSL 1.1.0 will provide name checking, but its only in HEAD at this point in time. From the OpenSSL Change Log:
Integrate hostname, email address and IP address checking with certificate
verification. New verify options supporting checking in opensl utility.
And:
New functions to check a hostname email or IP address against a
certificate. Add options x509 utility to print results of checks against
a certificate.
I don't see where Boost is performing any of the configurations or checks in the client code.
What precisely is Boost configuring, and what is it checking or verifying in its asio library component when using SSL?
Short answer: The Boost callback function, from the link you cited, doesn't verify anything. It returns whatever preliminary verification result was supplied to it by OpenSSL (via bool preverified). If there is any fine grained verification required (like the CN match, etc.), it has to be done explicitly by the callback.
Long answer: By the time the OpenSSL (or the Boost wrapper for OpenSSL) calls the verification function, in this case, bool verify_certificate(bool preverified, boost::asio::ssl::verify_context& ctx), a set of preliminary (or mandatory) verification is already done by OpenSSL. This is explained in the documentation.
The certificate chain is checked starting with the deepest nesting level (the root CA certificate) and worked upward to the peer's certificate. At each level signatures and issuer attributes are checked. Whenever a verification error is found, the error number is stored in x509_ctx and verify_callback is called with preverify_ok=0. By applying X509_CTX_store_* functions verify_callback can locate the certificate in question and perform additional steps (see EXAMPLES). If no error is found for a certificate, verify_callback is called with preverify_ok=1 before advancing to the next level.
The documentation also cites an example of how a more fine-grained verification callback could be written. You can draw inspiration from that depending on what your needs are.
EDIT: To be sure that Boost's internal callback function doesn't do anything special other than calling the application callback function, I took a look at engine.ipp, the C++ module that invokes OpenSSL's SSL_set_verify to set up callback functions. Take a look at how verify_callback_function is implemented. It simply invokes the application callback.

OpenSSL client not sending client certificate

I am struggling with a client certificate problem and hope somebody here can help me. I'm developing a client/server pair using boost asio but I'll try to be unspecific. I'm on windows and using openssl 1.0.1e
Basically, I want to have client authentication by using client certificates. The server shall only accept clients that have a certificate signed by my own CA. So I have setup a self signed CA. This has issued two more certificates. One for the client and one for the server. Both signed by the CA.
I have done that quite a few times now and I am confident that I got it.
My server side also works fine. It requests client certificates and if I'm using s_client and give those certs everything works. Also if I'm using a browser and have my root CA installed as trusted and then import the client certs.
The only thing that I can't get to work is the libssl client. It always fails during the handshake and as far as I can see it will not send the client certficate:
$ openssl.exe s_server -servername localhost -bugs -CAfile myca.crt -cert server.crt
-cert2 server.crt -key private/server.key -key2 private/server.key -accept 8887 -www
-state -Verify 5
verify depth is 5, must return a certificate
Setting secondary ctx parameters
Using default temp DH parameters
Using default temp ECDH parameters
ACCEPT
SSL_accept:before/accept initialization
SSL_accept:SSLv3 read client hello A
SSL_accept:SSLv3 write server hello A
SSL_accept:SSLv3 write certificate A
SSL_accept:SSLv3 write key exchange A
SSL_accept:SSLv3 write certificate request A
SSL_accept:SSLv3 flush data
SSL3 alert read:warning:no certificate
SSL3 alert write:fatal:handshake failure
SSL_accept:error in SSLv3 read client certificate B
SSL_accept:error in SSLv3 read client certificate B
2675716:error:140890C7:SSL routines:SSL3_GET_CLIENT_CERTIFICATE:peer did not return a
certificate:s3_srvr.c:3193:
ACCEPT
I'm using this s_server as debugging tool but against my real server the same thing occurs.
s_client will work fine with the same certificates. Also, if I disable "-Verify" in the server the connection works. So it really seems just the client refusing to send it's certficate. What can be the reason for that?
Since I'm using boost asio as an SSL wrapper the code looks like this:
m_ssl_context.set_verify_mode( asio::ssl::context::verify_peer );
m_ssl_context.load_verify_file( "myca.crt" );
m_ssl_context.use_certificate_file( "testclient.crt", asio::ssl::context::pem );
m_ssl_context.use_private_key_file( "testclient.key", asio::ssl::context::pem );
I have also tried to bypass asio and access the SSL context directly by saying:
SSL_CTX *ctx = m_ssl_context.impl();
SSL *ssl = m_ssl_socket.impl()->ssl;
int res = 0;
res = SSL_CTX_use_certificate_chain_file(ctx, "myca.crt");
if (res <= 0) {
// handle error
}
res = SSL_CTX_use_certificate_file(ctx, "testclient.crt", SSL_FILETYPE_PEM);
if (res <= 0) {
// handle error
}
res = SSL_CTX_use_PrivateKey_file(ctx, "testclient.key", SSL_FILETYPE_PEM);
if (res <= 0) {
// handle error
}
I can't see any difference in behavior. It should be mentioned that I am using a very old boost 1.43 asio which I cannot update but I suppose all relevant calls go more or less directly to OpenSSL anyway and the server works fine with that version so I think I can rule that out.
If I start forcing client and server to specific versions, the error messages change but it never works and still always works with the s_client test. Currently it is set to TLSv1
If I switch it to TLSv1 for example there is more chatter between client and server and eventually I get the error:
...
SSL_accept:SSLv3 read client key exchange A
<<< TLS 1.0 ChangeCipherSpec [length 0001]
01
<<< TLS 1.0 Handshake [length 0010], Finished
14 00 00 0c f4 71 28 4d ab e3 dd f2 46 e8 8b ed
>>> TLS 1.0 Alert [length 0002], fatal unexpected_message
02 0a
SSL3 alert write:fatal:unexpected_message
SSL_accept:failed in SSLv3 read certificate verify B
2675716:error:140880AE:SSL routines:SSL3_GET_CERT_VERIFY:missing verify
message:s3_srvr.c:2951:
2675716:error:140940E5:SSL routines:SSL3_READ_BYTES:ssl handshake failure:s3_pkt.c:989:
ACCEPT
I have found an older bug entry posted on the openssl mailing list that refereed to this. Apparently a wrong CRLF in the handshake that has been fixed two yrs ago. Or has it?
I have been debugging this for almost a week now and I'm really stuck. Does anyone have a suggestion on what to try? I'm out of ideas...
Cheers,
Stephan
PS: Here is what the above s_server debug out would be with s_client and the same certficate:
$ openssl s_client -CAfile ca.crt -cert testclient.crt -key private/testclient.key -verify 2 -connect myhost:8887
ACCEPT
SSL_accept:before/accept initialization
SSL_accept:SSLv3 read client hello A
SSL_accept:SSLv3 write server hello A
SSL_accept:SSLv3 write certificate A
SSL_accept:SSLv3 write key exchange A
SSL_accept:SSLv3 write certificate request A
SSL_accept:SSLv3 flush data
depth=1 C = DE, // further info
verify return:1
depth=0 C = DE, // further info
verify return:1
SSL_accept:SSLv3 read client certificate A
SSL_accept:SSLv3 read client key exchange A
SSL_accept:SSLv3 read certificate verify A
SSL_accept:SSLv3 read finished A
SSL_accept:SSLv3 write session ticket A
SSL_accept:SSLv3 write change cipher spec A
SSL_accept:SSLv3 write finished A
SSL_accept:SSLv3 flush data
ACCEPT
... handshake completes and data is transferred.
All right, after much suffering, the answer has been found by Dave Thompson of OpenSSL.
The reason was that my ssl code called all those functions on the OpenSSL context after the socket object (SSL*) was created from it. Which means all those functions did practically nothing or the wrong thing.
All I had to do was either:
1. Call SSL_use_certificate_file
res = SSL_use_certificate_file(ssl, "testclient.crt", SSL_FILETYPE_PEM);
if (res <= 0) {
// handle error
}
res = SSL_use_PrivateKey_file(ssl, "testclient.key", SSL_FILETYPE_PEM);
if (res <= 0) {
// handle error
}
(notice the missing CTX)
2. Call the CTX functions
Call the CTX functions upon the context before the socket was created. As asio seemingly encourages to create the context and socket right afterwards (as I did in the initializer list) the calls were all but useless.
The SSL context (in lib OpenSSL or asio alike) encapsulates the SSL usage and each socket created from it will share it's properties.
Thank you guys for your suggestions.
You should not use both SSL_CTX_use_certificate_chain_file() and SSL_CTX_use_certificate_file(), as SSL_CTX_use_certificate_chain_file() tries to load a chain including the client certificate, not just the CA chain. From SSL_CTX_use_certificate(3):
SSL_CTX_use_certificate_chain_file() loads a certificate chain from file into ctx. The certificates must be in PEM format and must be sorted starting with the subject's certificate (actual client or server certificate), followed by intermediate CA certificates if applicable, and ending at the highest level (root) CA.
I think you should be fine using only SSL_CTX_use_certificate_file() and SSL_CTX_use_PrivateKey_file(), as the client does not care much for the CA chain anyway.
I think you need to call SSL_CTX_set_client_CA_list on the server side. This sets a list of certificate authorities to be sent together with the client certificate request.
The client will not send its certificate, even if one was requested, if the certificate does not match that CA list sent by the server.

SSLSniff error: "SSL Accept Failed"

I'm trying to use SSLSniff's tool, and I have some technical issues... I've been looking for any similar problems, but the only results are from Twitter feeds, with no public useful answer. So, here it is:
(My version of SSLSniff is 0.8) I'm launching sslsniff with args:
sslsniff -a -c cert_and_key.pem -s 12345 -w out.log
where: cert_and_key.pem file is my authority's certificate concatenate with my unencrypted private key (in PEM format of course), and 12345 is the port where I redirect traffic with my iptables rule.
So sslsniff is correctly running:
INFO sslsniff : Certificate ready: [...]
[And anytime I connect with a client, there are these 2 following lines:]
DEBUG sslsniff : SSL Accept Failed!
DEBUG sslsniff : Got exception: Error with SSL connection.
On my client' side, I've register my AC as a trusted CA (with FF). Then when I connect through SSL I'm having the error:
Secure Connection Failed.
Error code: ssl_error_bad_cert_domain
What is super strange (moreover the fact that the certificate is not automatically accepted since it should be signed by my trusted CA) is that I cannot accept the forged certificate by clicking on "Add exception..." : I am always returning to the error page asking me to add an(other) exception...
Moreover, when I try to connect to for example: https://www.google.com, SSLSniff's log is completed with a new line :
DEBUG sslsniff : Encoded Length: 7064 too big for session cache, skipping...
Does anyone know what I'm doing wrong?
-- Edit to summer up the different answers --
The problem is that SSLSniff is not taking care of alternive names when it forges certificates. Apparently, Firefox refuses any certificate as soon as the Common Name doesn't match exactly the domain name.
For example, for Google.com : CN = www.google.com and there is no alternative name. So when you connect to https://www.google.com, it's working fine.
But for Google.fr : CN = *.google.fr, with these alternative names: *.google.fr and google.fr. So when you connect to https://www.google.fr, FF is looking for alternative names and, since it obviously doesn't find any, refuses the malformed certificate.
... So a solution would be to patch/commit... I don't know if Moxie Marlinspike has intentionally forgot this functionnality because it was too complicated, or if he was just not aware of this issue. Anyway, I'll try to have a look at the code.
The session encoded length error message: When caching the SSL session fails, it means that SSL session resumption on subsequent connections will fail, resulting in degraded performance, because a full SSL handshake needs to be done on every request. However, despite using the CPU more heavily, sslsniff will still work fine. The caching fails because the serialized representation of the OpenSSL session object (SSL_SESSION) was larger than the maximum size supported by sslsniff's session cache.
As for your real problem, note that sslsniff does not support X.509v3 subjectAltNames, so if you are connecting to a site whose hostname does not match the subject common name of the certificate, but instead matches only a subjectAltName, then sslsniff will generate a forged certificate without subjectAltNames, which will cause a hostname verification mismatch on the connecting client.
If your problem happens only for some specific sites, let us know the site so we can examine the server certificate using e.g. openssl s_client -connect host:port -showcerts and openssl x509 -in servercert.pem -text. If it happens for all sites, then the above is not the explanation.
Try a straight MITM with a cert you fully control , and make sure you don't have some OCSP/Perspectives/Convergance stuff meddling with things. Other than that, maybe add the cert to the OS trusted roots. I think FF on windows uses the windows cert store (start->run->certmgr.msc). It may also be worth trying with something like Burp to see if the error is localized to SSLSniff or all MITM attempts.