I'm using QNAM and QNetworkRequest to make a post request to our server. On most machines it works fine, but on some it fails. All machines are running windows and connecting to the same Ubuntu server. Between two windows 7 machines, one works and one fails. Both machines should have the same ssleay32.dll and libeay32.dll (I include them in my installation package). After installing chrome on to the "broken" machine, it can now properly perform the SSL handshake. If I remove all certificates (intermediate and trusted) relating to our CA (Thawte) the SSL handshake fails again.
manager = new QNetworkAccessManager( this );
connect( manager, SIGNAL( finished( QNetworkReply* ) ),
this, SLOT( licenseServerReply( QNetworkReply* ) ) );
connect( manager, SIGNAL(sslErrors(QNetworkReply*,QList<QSslError>)),
this, SLOT( sslErrorOccured(QNetworkReply*,QList<QSslError>))
);
request.setUrl( "https://www.myURL.com/postFromMachine/");
postData.append( "computer-name=" );
postData.append( hostInfo.hostName() );
postData.append( "&" );
manager->post( request, postData );
I've connected a slot to the sslErrors() signal of the QNetworkRequest object and I get the following when the SSL handshake fails:
Debug: "The issuer certificate of a locally looked up certificate could not be found"
Debug: "The root CA certificate is not trusted for this purpose"
In an attempt to fix the missing cert I added all certificates (intermediate and trusted) on the working machine concerning our Root Authority "Thawte" and related certificates "Thawte consulting" etc... to an QSSLSocket object and passed that object to the QNAM via QSSLCOnfiguration. There were 9 of them total and it didn't seem to fix the issue. I added the following code before manager->post().
QSslSocket *socket = new QSslSocket( this );
socket->addCaCertificates( ":/Certs/thawte1.cer" );
socket->addCaCertificates( ":/Certs/thawte2.cer" );
// Several more certs ...
socket->addCaCertificates( ":/Certs/thawte9.cer" );
QSslConfiguration conf;
conf.setCaCertificates( socket->caCertificates() );
request.setSslConfiguration( conf );
After discussion on an IRC chat I was pointed to the following command on the server to ensure it has the proper certificate chain:
openssl s_client -verify 5 -CApath /etc/ssl/certs -connect www.myurl.com:443 -showcerts
It replied with [ok] and no errors. The issuer is always the subject of the next cert in the chain, and the chain goes all the way to the root CA. Unless there is more to check with that command, it seems good to me.
I also checked the chain of certificates on the following site, and it seems like everything was good (expect possibly a weak certificate for path #2)
https://www.ssllabs.com/ssltest/analyze.html
Am at a loss of what to do, how can I prove that the server has the right certificate chain? If it does have the right certificate chain, why don't all machines perform the handshake properly (i.e. what certificates or settings do I need to add/change in my application to get them to work)? I don't want to simply ignore the SSLErrors as I want to be 100% sure that I have an encrypted connection to the proper host.
Thanks in advance for all the help!
I modified the socket->addCaCertificate to addDefaultCaCertificate and switched them from .cer to .pem ( I got the .pem files directly from our server ). Now everything works great. Note, don't copy your private key from your server, ensure it is the CA public keys.
Related
I want to set up my local server to communicate with my client. They build TLS connection using Openssl. I try to implement double side authentication, like server would verify client and client also needs to verify server.
When I use certificates generated by my self, everything works fine. The code is as following. It's C++ code in client. I set up client cert, private key and intermediate cert. In server side I saved a CA cert.
The relationship is: CA signs intermediate cert, intermediate cert signs client cert.
As we know, the reason that we need to provide client private key is the client will signature a "challenge" then send to server. Server could get client public key by certificate chain and use it to decode the encrypt "challenge" to see if they matched. You could see this link for detailed process:
https://en.wikipedia.org/wiki/Transport_Layer_Security#TLS_handshake
However in my scenario, I have no permission to get the private key. I only have an API to call, which takes the digest or anything we want to encode as input and return a string encoded by client private key.
Therefore I'm not able to pass any "ClientPrivateKeyFileTest" to TLS.
I searched openssl source code but all handshakes were done in this function: SSL_do_handshake() and I'm not allowed to modify this function.
// load client-side cert and key
SSL_CTX_use_certificate_file(m_ctx, ClientCertificateFileTest, SSL_FILETYPE_PEM);
SSL_CTX_use_PrivateKey_file(m_ctx, ClientPrivateKeyFileTest, SSL_FILETYPE_PEM);
// load intermediate cert
X509* chaincert = X509_new();
BIO* bio_cert = BIO_new_file(SignerCertificateFileTest, "rb");
PEM_read_bio_X509(bio_cert, &chaincert, NULL, NULL);
SSL_CTX_add1_chain_cert(m_ctx, chaincert)
m_ssl = SSL_new(m_ctx);
// get_seocket is my own API
m_sock = get_socket();
SSL_set_fd(m_ssl, m_sock)
// doing handshake and build connection
auto r = SSL_connect(m_ssl);
I think all handshake processes would be done after I call SSL_connect(). So I wonder is there other way I can do to complete the client-authentication?
For example, I could skip adding private key step but set up a callback function somewhere which can handle all cases when SSL needs to use private key to calculate something.
PS: The API is a black box in the client machine.
One more thing, these days I found that openssl engine may help this problem. But does anybody know what kind of engine is useful for this problem? The EC sign, verification or others?
Final update: I implemented a OpenSSL engine to reload EC_KEY_METHOD so that I'm able to use my own sign function.
Thanks a lot!
When doing a HTTPS request using Qt, I try to obtain the peer's certificate after the SSL handshake, in order to track future changes in the certificate.
QNetworkAccessManager nam;
nam.get(QNetworkRequest(QUrl("https://google.com/"))); // example URL
QObject::connect(&nam, &QNetworkAccessManager::encrypted, [](QNetworkReply *reply){
qDebug() << reply->sslConfiguration().peerCertificate();
});
According to the documentation of QNetworkAccessManager::encrypted, the above code should get access to the server's certificate:
This signal is emitted when an SSL/TLS session has successfully completed the initial handshake. At this point, no user data has been transmitted. The signal can be used to perform additional checks on the certificate chain, for example to notify users when the certificate for a website has changed. If the reply does not match the expected criteria then it should be aborted by calling QNetworkReply::abort() by a slot connected to this signal. The SSL configuration in use can be inspected using the QNetworkReply::sslConfiguration() method.
Also, from the documentation of QSslConfiguration::peerCertificate():
Because the peer certificate is set during the handshake phase, it is safe to access the peer certificate from a slot connected to the QSslSocket::sslErrors() signal, QNetworkReply::sslErrors() signal, or the QSslSocket::encrypted() signal.
However, the certificate is always empty. The debug output of the above code (after entering the application's event loop) is:
QSslCertificate( "" , "" , "1B2M2Y8AsgTpgAmY7PhCfg==" , () , () , QMap() , QDateTime(" Qt::LocalTime") , QDateTime(" Qt::LocalTime") )
On the other hand, if SSL errors where encountered, and if I connect to sslErrors, I do get the certificate. For example, for a default certificate under Ubuntu / Apache, which isn't accepted by Qt because of a missing host name in the certificate, I get for "https://localhost" the following:
QSslCertificate( "3" , "95:b0:93:f2:16:bb:22:cb" , "cXB6WctE7oZsrvZLU2BWUw==" , () , () , QMap() , QDateTime("2014-07-10 23:04:06.000 UTC Qt::UTC") , QDateTime("2024-07-07 23:04:06.000 UTC Qt::UTC") )
How can I get the certificate when the SSL handshake was successful?
I tested with the QNetworkAccessManager's signals, as well as with the QNetworkReply's signals; the results are the same.
MCVE can be found at https://bitbucket.org/leemes/ssltest, feel free to clone and fiddle:
git clone https://bitbucket.org/leemes/ssltest.git
I tested with Qt 5.4.0 and with Qt 5.3.1; results are the same.
It was a bug in Qt up to 5.4.0. It has been fixed in Qt 5.4.1.
https://bugreports.qt.io/browse/QTBUG-40401
I am struggling with a client certificate problem and hope somebody here can help me. I'm developing a client/server pair using boost asio but I'll try to be unspecific. I'm on windows and using openssl 1.0.1e
Basically, I want to have client authentication by using client certificates. The server shall only accept clients that have a certificate signed by my own CA. So I have setup a self signed CA. This has issued two more certificates. One for the client and one for the server. Both signed by the CA.
I have done that quite a few times now and I am confident that I got it.
My server side also works fine. It requests client certificates and if I'm using s_client and give those certs everything works. Also if I'm using a browser and have my root CA installed as trusted and then import the client certs.
The only thing that I can't get to work is the libssl client. It always fails during the handshake and as far as I can see it will not send the client certficate:
$ openssl.exe s_server -servername localhost -bugs -CAfile myca.crt -cert server.crt
-cert2 server.crt -key private/server.key -key2 private/server.key -accept 8887 -www
-state -Verify 5
verify depth is 5, must return a certificate
Setting secondary ctx parameters
Using default temp DH parameters
Using default temp ECDH parameters
ACCEPT
SSL_accept:before/accept initialization
SSL_accept:SSLv3 read client hello A
SSL_accept:SSLv3 write server hello A
SSL_accept:SSLv3 write certificate A
SSL_accept:SSLv3 write key exchange A
SSL_accept:SSLv3 write certificate request A
SSL_accept:SSLv3 flush data
SSL3 alert read:warning:no certificate
SSL3 alert write:fatal:handshake failure
SSL_accept:error in SSLv3 read client certificate B
SSL_accept:error in SSLv3 read client certificate B
2675716:error:140890C7:SSL routines:SSL3_GET_CLIENT_CERTIFICATE:peer did not return a
certificate:s3_srvr.c:3193:
ACCEPT
I'm using this s_server as debugging tool but against my real server the same thing occurs.
s_client will work fine with the same certificates. Also, if I disable "-Verify" in the server the connection works. So it really seems just the client refusing to send it's certficate. What can be the reason for that?
Since I'm using boost asio as an SSL wrapper the code looks like this:
m_ssl_context.set_verify_mode( asio::ssl::context::verify_peer );
m_ssl_context.load_verify_file( "myca.crt" );
m_ssl_context.use_certificate_file( "testclient.crt", asio::ssl::context::pem );
m_ssl_context.use_private_key_file( "testclient.key", asio::ssl::context::pem );
I have also tried to bypass asio and access the SSL context directly by saying:
SSL_CTX *ctx = m_ssl_context.impl();
SSL *ssl = m_ssl_socket.impl()->ssl;
int res = 0;
res = SSL_CTX_use_certificate_chain_file(ctx, "myca.crt");
if (res <= 0) {
// handle error
}
res = SSL_CTX_use_certificate_file(ctx, "testclient.crt", SSL_FILETYPE_PEM);
if (res <= 0) {
// handle error
}
res = SSL_CTX_use_PrivateKey_file(ctx, "testclient.key", SSL_FILETYPE_PEM);
if (res <= 0) {
// handle error
}
I can't see any difference in behavior. It should be mentioned that I am using a very old boost 1.43 asio which I cannot update but I suppose all relevant calls go more or less directly to OpenSSL anyway and the server works fine with that version so I think I can rule that out.
If I start forcing client and server to specific versions, the error messages change but it never works and still always works with the s_client test. Currently it is set to TLSv1
If I switch it to TLSv1 for example there is more chatter between client and server and eventually I get the error:
...
SSL_accept:SSLv3 read client key exchange A
<<< TLS 1.0 ChangeCipherSpec [length 0001]
01
<<< TLS 1.0 Handshake [length 0010], Finished
14 00 00 0c f4 71 28 4d ab e3 dd f2 46 e8 8b ed
>>> TLS 1.0 Alert [length 0002], fatal unexpected_message
02 0a
SSL3 alert write:fatal:unexpected_message
SSL_accept:failed in SSLv3 read certificate verify B
2675716:error:140880AE:SSL routines:SSL3_GET_CERT_VERIFY:missing verify
message:s3_srvr.c:2951:
2675716:error:140940E5:SSL routines:SSL3_READ_BYTES:ssl handshake failure:s3_pkt.c:989:
ACCEPT
I have found an older bug entry posted on the openssl mailing list that refereed to this. Apparently a wrong CRLF in the handshake that has been fixed two yrs ago. Or has it?
I have been debugging this for almost a week now and I'm really stuck. Does anyone have a suggestion on what to try? I'm out of ideas...
Cheers,
Stephan
PS: Here is what the above s_server debug out would be with s_client and the same certficate:
$ openssl s_client -CAfile ca.crt -cert testclient.crt -key private/testclient.key -verify 2 -connect myhost:8887
ACCEPT
SSL_accept:before/accept initialization
SSL_accept:SSLv3 read client hello A
SSL_accept:SSLv3 write server hello A
SSL_accept:SSLv3 write certificate A
SSL_accept:SSLv3 write key exchange A
SSL_accept:SSLv3 write certificate request A
SSL_accept:SSLv3 flush data
depth=1 C = DE, // further info
verify return:1
depth=0 C = DE, // further info
verify return:1
SSL_accept:SSLv3 read client certificate A
SSL_accept:SSLv3 read client key exchange A
SSL_accept:SSLv3 read certificate verify A
SSL_accept:SSLv3 read finished A
SSL_accept:SSLv3 write session ticket A
SSL_accept:SSLv3 write change cipher spec A
SSL_accept:SSLv3 write finished A
SSL_accept:SSLv3 flush data
ACCEPT
... handshake completes and data is transferred.
All right, after much suffering, the answer has been found by Dave Thompson of OpenSSL.
The reason was that my ssl code called all those functions on the OpenSSL context after the socket object (SSL*) was created from it. Which means all those functions did practically nothing or the wrong thing.
All I had to do was either:
1. Call SSL_use_certificate_file
res = SSL_use_certificate_file(ssl, "testclient.crt", SSL_FILETYPE_PEM);
if (res <= 0) {
// handle error
}
res = SSL_use_PrivateKey_file(ssl, "testclient.key", SSL_FILETYPE_PEM);
if (res <= 0) {
// handle error
}
(notice the missing CTX)
2. Call the CTX functions
Call the CTX functions upon the context before the socket was created. As asio seemingly encourages to create the context and socket right afterwards (as I did in the initializer list) the calls were all but useless.
The SSL context (in lib OpenSSL or asio alike) encapsulates the SSL usage and each socket created from it will share it's properties.
Thank you guys for your suggestions.
You should not use both SSL_CTX_use_certificate_chain_file() and SSL_CTX_use_certificate_file(), as SSL_CTX_use_certificate_chain_file() tries to load a chain including the client certificate, not just the CA chain. From SSL_CTX_use_certificate(3):
SSL_CTX_use_certificate_chain_file() loads a certificate chain from file into ctx. The certificates must be in PEM format and must be sorted starting with the subject's certificate (actual client or server certificate), followed by intermediate CA certificates if applicable, and ending at the highest level (root) CA.
I think you should be fine using only SSL_CTX_use_certificate_file() and SSL_CTX_use_PrivateKey_file(), as the client does not care much for the CA chain anyway.
I think you need to call SSL_CTX_set_client_CA_list on the server side. This sets a list of certificate authorities to be sent together with the client certificate request.
The client will not send its certificate, even if one was requested, if the certificate does not match that CA list sent by the server.
I'm trying to use SSLSniff's tool, and I have some technical issues... I've been looking for any similar problems, but the only results are from Twitter feeds, with no public useful answer. So, here it is:
(My version of SSLSniff is 0.8) I'm launching sslsniff with args:
sslsniff -a -c cert_and_key.pem -s 12345 -w out.log
where: cert_and_key.pem file is my authority's certificate concatenate with my unencrypted private key (in PEM format of course), and 12345 is the port where I redirect traffic with my iptables rule.
So sslsniff is correctly running:
INFO sslsniff : Certificate ready: [...]
[And anytime I connect with a client, there are these 2 following lines:]
DEBUG sslsniff : SSL Accept Failed!
DEBUG sslsniff : Got exception: Error with SSL connection.
On my client' side, I've register my AC as a trusted CA (with FF). Then when I connect through SSL I'm having the error:
Secure Connection Failed.
Error code: ssl_error_bad_cert_domain
What is super strange (moreover the fact that the certificate is not automatically accepted since it should be signed by my trusted CA) is that I cannot accept the forged certificate by clicking on "Add exception..." : I am always returning to the error page asking me to add an(other) exception...
Moreover, when I try to connect to for example: https://www.google.com, SSLSniff's log is completed with a new line :
DEBUG sslsniff : Encoded Length: 7064 too big for session cache, skipping...
Does anyone know what I'm doing wrong?
-- Edit to summer up the different answers --
The problem is that SSLSniff is not taking care of alternive names when it forges certificates. Apparently, Firefox refuses any certificate as soon as the Common Name doesn't match exactly the domain name.
For example, for Google.com : CN = www.google.com and there is no alternative name. So when you connect to https://www.google.com, it's working fine.
But for Google.fr : CN = *.google.fr, with these alternative names: *.google.fr and google.fr. So when you connect to https://www.google.fr, FF is looking for alternative names and, since it obviously doesn't find any, refuses the malformed certificate.
... So a solution would be to patch/commit... I don't know if Moxie Marlinspike has intentionally forgot this functionnality because it was too complicated, or if he was just not aware of this issue. Anyway, I'll try to have a look at the code.
The session encoded length error message: When caching the SSL session fails, it means that SSL session resumption on subsequent connections will fail, resulting in degraded performance, because a full SSL handshake needs to be done on every request. However, despite using the CPU more heavily, sslsniff will still work fine. The caching fails because the serialized representation of the OpenSSL session object (SSL_SESSION) was larger than the maximum size supported by sslsniff's session cache.
As for your real problem, note that sslsniff does not support X.509v3 subjectAltNames, so if you are connecting to a site whose hostname does not match the subject common name of the certificate, but instead matches only a subjectAltName, then sslsniff will generate a forged certificate without subjectAltNames, which will cause a hostname verification mismatch on the connecting client.
If your problem happens only for some specific sites, let us know the site so we can examine the server certificate using e.g. openssl s_client -connect host:port -showcerts and openssl x509 -in servercert.pem -text. If it happens for all sites, then the above is not the explanation.
Try a straight MITM with a cert you fully control , and make sure you don't have some OCSP/Perspectives/Convergance stuff meddling with things. Other than that, maybe add the cert to the OS trusted roots. I think FF on windows uses the windows cert store (start->run->certmgr.msc). It may also be worth trying with something like Burp to see if the error is localized to SSLSniff or all MITM attempts.
I'm currently rewriting some existing technologies that were once using RSA Security's libraries into OpenSSL, but I'm starting to run into a few issues. Currently, all of the certificate verification code appears to be running without a hitch, until that is, I call SSL_connect().
Before, the call to SSL_connect() would produce SSL_ERROR_WANT_READ.
An answer to this issue on another forum suggested to me that SSL_connect() should be called until it stops producing SSL_ERROR_WANT_READ errors. Unforunately, this only produces something more confusing:
error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
even though SSL_CTX_load_verify_locations() succeeds. Does anyone have any idea as to why a verification error wouldn't register with certificate methods and wait until SSL_connect() is triggered?
Usually this error means that the server certificate your client received in response to SSL_connect() couldn't be verified.
This can happen for different reasons:
If the server certificate is self-signed, you'll have to authorize that on your SSL_CONTEXT.
If the server certificate was signed by a certificate authority that is not in the list of trusted CA certificates
If the server certificate is not valid yet or not valid anymore
Actually, you should set a callback for certificate verification and make it accept any certificate, so you can focus on the connection part. Once it works, just tweak your callback or check your certificates to be valid.
At any time you get a failure, you can call some SSL_get_error() function that will indicate you why the certificate was rejected.
(Unfortunately, I can't access my code base right now, so I cannot give concrete examples)
Here are some code samples from my own SSL Socket wrapper class, adapted for you:
// ctx is a SSL_CONTEXT
// internalCertificateVerificationCallback is a callback static method (or function)
ctx->setVerify(SSL_VERIFY_PEER | SSL_VERIFY_FAIL_IF_NO_PEER_CERT, internalCertificateVerificationCallback);
Here is the definition for internalCertificateVerificationCallback:
int SecureSocket::internalCertificateVerificationCallback(int preverify_ok, X509_STORE_CTX* x509_ctx)
{
//preverify_ok contains 1 if the pre-verification succeeded, 0 otherwise.
return 1; // This accepts every certificate
}