Boost, asio, https, and host/certificate verifcation - c++

I'm looking at Boost's SSL Client. There's a reference to OpenSSL in the comments (sorry, no line numbers):
// The verify callback can be used to check whether the certificate that is
// being presented is valid for the peer. For example, RFC 2818 describes
// the steps involved in doing this for HTTPS. Consult the OpenSSL
// documentation for more details. Note that the callback is called once
// for each certificate in the certificate chain, starting from the root
// certificate authority.
Proper OpenSSL use and verification can be tricky. From experience, I know I have to perform the following to use the library correctly:
Disable SSLv2, SSLv3, and Compression on the Context object
Provide the proper root certificate for chain building and checking
Call SSL_get_peer_certificate and verify the certificate is non-NULL
Call SSL_get_verify_result and verify the result is X509_V_OK
Perform name matching (CN or SAN must match requested host)
OpenSSL 1.1.0 will provide name checking, but its only in HEAD at this point in time. From the OpenSSL Change Log:
Integrate hostname, email address and IP address checking with certificate
verification. New verify options supporting checking in opensl utility.
And:
New functions to check a hostname email or IP address against a
certificate. Add options x509 utility to print results of checks against
a certificate.
I don't see where Boost is performing any of the configurations or checks in the client code.
What precisely is Boost configuring, and what is it checking or verifying in its asio library component when using SSL?

Short answer: The Boost callback function, from the link you cited, doesn't verify anything. It returns whatever preliminary verification result was supplied to it by OpenSSL (via bool preverified). If there is any fine grained verification required (like the CN match, etc.), it has to be done explicitly by the callback.
Long answer: By the time the OpenSSL (or the Boost wrapper for OpenSSL) calls the verification function, in this case, bool verify_certificate(bool preverified, boost::asio::ssl::verify_context& ctx), a set of preliminary (or mandatory) verification is already done by OpenSSL. This is explained in the documentation.
The certificate chain is checked starting with the deepest nesting level (the root CA certificate) and worked upward to the peer's certificate. At each level signatures and issuer attributes are checked. Whenever a verification error is found, the error number is stored in x509_ctx and verify_callback is called with preverify_ok=0. By applying X509_CTX_store_* functions verify_callback can locate the certificate in question and perform additional steps (see EXAMPLES). If no error is found for a certificate, verify_callback is called with preverify_ok=1 before advancing to the next level.
The documentation also cites an example of how a more fine-grained verification callback could be written. You can draw inspiration from that depending on what your needs are.
EDIT: To be sure that Boost's internal callback function doesn't do anything special other than calling the application callback function, I took a look at engine.ipp, the C++ module that invokes OpenSSL's SSL_set_verify to set up callback functions. Take a look at how verify_callback_function is implemented. It simply invokes the application callback.

Related

Look up leaf certificate from issuer’s thumbprint?

I have a leaf certificate installed on my machine, which was issued by a Certificate Authority (CA); this CA is not present on the system.
If I am given the thumbprint (i.e. the SHA-1 hash) of the CA, can I look up and thereby retrieve the installed leaf signers issued by this CA? If I am able to, then what are the required Windows function calls for accomplishing this?
I have been examining a leaf certificate and I only see the standard string representation name of the CA and not a thumbprint. This string name is not unique, hence why I am utilizing the thumbprint (SHA-1’s poor security here is not a problem as it is only used for looking up a proper signer). Microsoft has the CertGetIssuerCertificateFromStore function, but this requires having the CA in memory.
No, it is not possible to look up an issued leaf signer using only the issuer's CA's thumbprint. You first have to install the CA on the machine and then use the entirety of the CA to find its issued leaf signers. The CA's thumbprint is not stored in the leaf signers it issues from what it appears.
All in all, the first operation that needs to be done is installing the CA on the machine you want to find its issued leaf signers on.

How to do "Client Authentification" if I'm not able to provide private key

I want to set up my local server to communicate with my client. They build TLS connection using Openssl. I try to implement double side authentication, like server would verify client and client also needs to verify server.
When I use certificates generated by my self, everything works fine. The code is as following. It's C++ code in client. I set up client cert, private key and intermediate cert. In server side I saved a CA cert.
The relationship is: CA signs intermediate cert, intermediate cert signs client cert.
As we know, the reason that we need to provide client private key is the client will signature a "challenge" then send to server. Server could get client public key by certificate chain and use it to decode the encrypt "challenge" to see if they matched. You could see this link for detailed process:
https://en.wikipedia.org/wiki/Transport_Layer_Security#TLS_handshake
However in my scenario, I have no permission to get the private key. I only have an API to call, which takes the digest or anything we want to encode as input and return a string encoded by client private key.
Therefore I'm not able to pass any "ClientPrivateKeyFileTest" to TLS.
I searched openssl source code but all handshakes were done in this function: SSL_do_handshake() and I'm not allowed to modify this function.
// load client-side cert and key
SSL_CTX_use_certificate_file(m_ctx, ClientCertificateFileTest, SSL_FILETYPE_PEM);
SSL_CTX_use_PrivateKey_file(m_ctx, ClientPrivateKeyFileTest, SSL_FILETYPE_PEM);
// load intermediate cert
X509* chaincert = X509_new();
BIO* bio_cert = BIO_new_file(SignerCertificateFileTest, "rb");
PEM_read_bio_X509(bio_cert, &chaincert, NULL, NULL);
SSL_CTX_add1_chain_cert(m_ctx, chaincert)
m_ssl = SSL_new(m_ctx);
// get_seocket is my own API
m_sock = get_socket();
SSL_set_fd(m_ssl, m_sock)
// doing handshake and build connection
auto r = SSL_connect(m_ssl);
I think all handshake processes would be done after I call SSL_connect(). So I wonder is there other way I can do to complete the client-authentication?
For example, I could skip adding private key step but set up a callback function somewhere which can handle all cases when SSL needs to use private key to calculate something.
PS: The API is a black box in the client machine.
One more thing, these days I found that openssl engine may help this problem. But does anybody know what kind of engine is useful for this problem? The EC sign, verification or others?
Final update: I implemented a OpenSSL engine to reload EC_KEY_METHOD so that I'm able to use my own sign function.
Thanks a lot!

aws root pinning error CURLE_SSL_PINNEDPUBKEYNOTMATCH

I am using libcurl and shifting cert pinning to AWS root as per this document https://www.amazontrust.com/repository/
I used the SHA-256 Hash of Subject Public Key Information data from that website, formed a string:
static string PUBLIC_KEY = "sha256//fbe3018031f9586bcbf41727e417b7d1c45c2f47f93be372a17b96b50757d5a2;sha256//7f4296fc5b6a4e3b35d3c369623e364ab1af381d8fa7121533c9d6c633ea2461;sha256//36abc32656acfc645c61b71613c4bf21c787f5cabbee48348d58597803d7abc9;sha256//f7ecded5c66047d28ed6466b543c40e0743abe81d109254dcf845d4c2c7853c5;sha256//2b071c59a0a0ae76b0eadb2bad23bad4580b69c3601b630c2eaf0613afa83f92";
and set the string to curl
curl_easy_setopt(handle, CURLOPT_PINNEDPUBLICKEY, PUBLIC_KEY.c_str());
The curl error I get is CURLE_SSL_PINNEDPUBKEYNOTMATCH
Google does not have any insight into why, as far as I searched. If anyone has any input on how to fix this and still pin to the root, it would be super useful. Thanks.
Found the reason. Root pinning is not supported yet.
13.11 Support intermediate & root pinning for PINNEDPUBLICKEY
CURLOPT_PINNEDPUBLICKEY does not consider the hashes of intermediate & root certificates when comparing the pinned keys. Therefore it is not compatible with "HTTP Public Key Pinning" as there also intermediate and root certificates can be pinned. This is very useful as it prevents webadmins from "locking themself out of their servers".
Adding this feature would make curls pinning 100% compatible to HPKP and allow more flexible pinning.
https://curl.haxx.se/docs/todo.html

SSLSniff error: "SSL Accept Failed"

I'm trying to use SSLSniff's tool, and I have some technical issues... I've been looking for any similar problems, but the only results are from Twitter feeds, with no public useful answer. So, here it is:
(My version of SSLSniff is 0.8) I'm launching sslsniff with args:
sslsniff -a -c cert_and_key.pem -s 12345 -w out.log
where: cert_and_key.pem file is my authority's certificate concatenate with my unencrypted private key (in PEM format of course), and 12345 is the port where I redirect traffic with my iptables rule.
So sslsniff is correctly running:
INFO sslsniff : Certificate ready: [...]
[And anytime I connect with a client, there are these 2 following lines:]
DEBUG sslsniff : SSL Accept Failed!
DEBUG sslsniff : Got exception: Error with SSL connection.
On my client' side, I've register my AC as a trusted CA (with FF). Then when I connect through SSL I'm having the error:
Secure Connection Failed.
Error code: ssl_error_bad_cert_domain
What is super strange (moreover the fact that the certificate is not automatically accepted since it should be signed by my trusted CA) is that I cannot accept the forged certificate by clicking on "Add exception..." : I am always returning to the error page asking me to add an(other) exception...
Moreover, when I try to connect to for example: https://www.google.com, SSLSniff's log is completed with a new line :
DEBUG sslsniff : Encoded Length: 7064 too big for session cache, skipping...
Does anyone know what I'm doing wrong?
-- Edit to summer up the different answers --
The problem is that SSLSniff is not taking care of alternive names when it forges certificates. Apparently, Firefox refuses any certificate as soon as the Common Name doesn't match exactly the domain name.
For example, for Google.com : CN = www.google.com and there is no alternative name. So when you connect to https://www.google.com, it's working fine.
But for Google.fr : CN = *.google.fr, with these alternative names: *.google.fr and google.fr. So when you connect to https://www.google.fr, FF is looking for alternative names and, since it obviously doesn't find any, refuses the malformed certificate.
... So a solution would be to patch/commit... I don't know if Moxie Marlinspike has intentionally forgot this functionnality because it was too complicated, or if he was just not aware of this issue. Anyway, I'll try to have a look at the code.
The session encoded length error message: When caching the SSL session fails, it means that SSL session resumption on subsequent connections will fail, resulting in degraded performance, because a full SSL handshake needs to be done on every request. However, despite using the CPU more heavily, sslsniff will still work fine. The caching fails because the serialized representation of the OpenSSL session object (SSL_SESSION) was larger than the maximum size supported by sslsniff's session cache.
As for your real problem, note that sslsniff does not support X.509v3 subjectAltNames, so if you are connecting to a site whose hostname does not match the subject common name of the certificate, but instead matches only a subjectAltName, then sslsniff will generate a forged certificate without subjectAltNames, which will cause a hostname verification mismatch on the connecting client.
If your problem happens only for some specific sites, let us know the site so we can examine the server certificate using e.g. openssl s_client -connect host:port -showcerts and openssl x509 -in servercert.pem -text. If it happens for all sites, then the above is not the explanation.
Try a straight MITM with a cert you fully control , and make sure you don't have some OCSP/Perspectives/Convergance stuff meddling with things. Other than that, maybe add the cert to the OS trusted roots. I think FF on windows uses the windows cert store (start->run->certmgr.msc). It may also be worth trying with something like Burp to see if the error is localized to SSLSniff or all MITM attempts.

SSL_connect() produces certificate verify failure

I'm currently rewriting some existing technologies that were once using RSA Security's libraries into OpenSSL, but I'm starting to run into a few issues. Currently, all of the certificate verification code appears to be running without a hitch, until that is, I call SSL_connect().
Before, the call to SSL_connect() would produce SSL_ERROR_WANT_READ.
An answer to this issue on another forum suggested to me that SSL_connect() should be called until it stops producing SSL_ERROR_WANT_READ errors. Unforunately, this only produces something more confusing:
error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
even though SSL_CTX_load_verify_locations() succeeds. Does anyone have any idea as to why a verification error wouldn't register with certificate methods and wait until SSL_connect() is triggered?
Usually this error means that the server certificate your client received in response to SSL_connect() couldn't be verified.
This can happen for different reasons:
If the server certificate is self-signed, you'll have to authorize that on your SSL_CONTEXT.
If the server certificate was signed by a certificate authority that is not in the list of trusted CA certificates
If the server certificate is not valid yet or not valid anymore
Actually, you should set a callback for certificate verification and make it accept any certificate, so you can focus on the connection part. Once it works, just tweak your callback or check your certificates to be valid.
At any time you get a failure, you can call some SSL_get_error() function that will indicate you why the certificate was rejected.
(Unfortunately, I can't access my code base right now, so I cannot give concrete examples)
Here are some code samples from my own SSL Socket wrapper class, adapted for you:
// ctx is a SSL_CONTEXT
// internalCertificateVerificationCallback is a callback static method (or function)
ctx->setVerify(SSL_VERIFY_PEER | SSL_VERIFY_FAIL_IF_NO_PEER_CERT, internalCertificateVerificationCallback);
Here is the definition for internalCertificateVerificationCallback:
int SecureSocket::internalCertificateVerificationCallback(int preverify_ok, X509_STORE_CTX* x509_ctx)
{
//preverify_ok contains 1 if the pre-verification succeeded, 0 otherwise.
return 1; // This accepts every certificate
}