The TL;DR version
I'd like to know:
Where does the specification for the use of ECDHE get defined (in a cert parameter or a server configuration of SSL contexts, or elsewhere)?
In a non-home-rolled certificate setup, who's responsibility is it to define the ECDHE public and private information (the end user or cert provider)?
Can an existing Certificate which does not appear to use ECDHE be made to without causing issues with the Certificate?
Are there any examples of someone using SSL in Boost::ASIO with an ECDHE setup?
The Longer Version
We've been building an application which is using a proper-paid-for certificate from an external Cert Authority. The application uses a home-rolled server setup based off of Boost ASIO and Boost Beast, and we only recently noticed it doesn't play nice with iOS - ASIO says there is no shared cipher.
Reading into how TLS works has led me to the fact that some part of our server was preventing us from serving TLS using the ECDHE-* suite of ciphers (which iOS seems to want) - but I'm having difficulty in figuring out how to wrangle ASIO and our current cert/key into serving ECDHE.
What I've tried:
Using the same cert and key, adding in the results of openssl dhparam into ASIO using set_tmp_dh, then specifying ciphers. Curl reports that this allows a connection using DHE but not ECDHE. Specifying ciphers that only use ECDHE causes errors when connecting.
Trying to pass the output of openssl ecparam to ASIO using a similar method to the above. I've not been able to format something that ASIO accepts.
Trying to see if there is a way you can use the output of openssl ecparam with another combining function to modify the original cert into one that uses ECDHE. I clued onto this one from the OpenSSL wiki suggesting that if the cert does not contain the line ASN1 OID: prime256v1 (or a similar named curve), then it is not suitable for ECDHE usage.
At this point I'm unsure as to where the issue truly lies (in ASIO, in the certificates or in how I'm putting it all together) and most of the information on the internet I can find relates to home-rolling everything from scratch, rather than working with existing certs.
Update 11/05/19
https://github.com/chriskohlhoff/asio/pull/117 pulled in changes for ASIO with ECDHE. Will need to wait a while to see which Boost lib version it makes it into.
Original Answer
I seem to have found an answer for any googlers - ASIO does not appear to support ECDHE natively at the time of writing. This issue from the main repo suggests that ECDHE is on the cards for support but is not yet implemented.
Here is a link to the ECDHE implementation that's been waiting to be merged since 2016: https://github.com/chriskohlhoff/asio/pull/117.
+1 to get the attention of the Boost ASIO maintainer; he's been pretty slow with it.
Related
I am trying to implement secure communication between a server and client in c++. The limitation is that both the client and server must run on windows and have to be in c++. This is for a research project I am working on at my university.
So far I have found that SChannel is the best option, but the documentation is extremely confusing and I can not find any guides/tutorials on how to use it. I have already looked at this link https://learn.microsoft.com/en-us/windows/desktop/secauthn/creating-a-secure-connection-using-schannel but still do not understand how to get it working. Could someone guide me through this if this is the best way?
I also looked into use SSLStream using the CLR to have .net run inside of a c++ application. However I can not use this because the client application is threaded and threads can't be used with CLR.
I already have a dummy client and server set up with communication between the two, I am just trying to secure and encrypt that communication.
Any help is greatly appreciated!
Whichever SSL library you choose to use there are a few things you need to know as a beginner in this field:
The server and client implementations will end up looking quite different in places.
Your server is absolutely going to need a certificate with a private key. During development you clearly don't want to get one from Verisign or something so you need to create a self-signed certificate. You can do this with openssl or other tools.
The certificate consists of a private part and a public part. The public part needs to go to the client, and will be used to validate the connection. When you are using something like SChannel the certificates (private and public) will need to be installed in the certificate stores of the server and client respectively.
SChannel does not send or receive data for you. So the core of your implementation is going to be: when the network has data: read ciphertext from socket and write to SChannel. Read clear text from SChannel (if any) and pass to application. When the application has data to send, get clear text from Application and pass to SChannel. Get the resulting ciphertext buffers from SChannel and write to the socket.
buffers from the internet may be partial, and negotiations and re-negotiations means there's no 1:1 mapping of passing data into SChannel and getting data out.
You therefore can't get away with a naive implementation that calls SChannel once to pass data in, and once again to get un/encrypted data. There will potentially be nothing available, or a whole lot of packets to send between the client and the server, before you'll get any application bytes. i.e. You will need some kind of state machine to keeptrack of this.
Obviously, don't write both the client and server at the same time: Start with your client against an https server.
That's the general outline of the process - the things that confused me when I first encountered SSL and why none of the samples were nearly as simple as I had hoped them to be.
Novice to Qt and developing a cross platform app, which requires SSL authentication from the server as well as client sides The .pem based encryption is working on Linux, Android, Windows. However there are problems with Mac OSX. Our code looks like below:
QFile privateKeyFile(":/Certificate.pem"); // --> has certificate + key
privateKeyFile.open(QIODevice::ReadOnly | QIODevice::Text);
setLocalCertificateChain(QSslCertificate::fromPath(":/Certificate.pem", QSsl::Pem));
setPrivateKey(QSslKey(privateKeyFile.readAll(), QSsl::Rsa));
In above code privateKey().isNull() returns true for Mac. When we referred this post, it says that Mac doesn't support .pem based encryption.
The Secure Transport back-end to curl only supports client IDs that are in PKCS#12 (P12) format; it does not support client IDs in PEM format because Apple does not allow us to create a security identity from an identity file in PEM format without using a private API. And we can't use the private API, because apps that use private API are not allowed in any of Apple's app stores.
With my limited understanding, I interpreted that .pem is not a good idea for SSL communication with the server. Please stop me if it's wrong!
Hence, we decided to move to .pfx for all the platforms. We already had a .pfx file with a passphrase. We converted above code to be compatible with .pfx (i.e. "Certificate.pfx", we had this old file along with "Certificate.pem"). Instead of QSsl::Pem, we tried QSsl::Der. But as expected, it didn't work. However, there was no encryption error either, but we are sure that we are doing something wrong. :-)
We referred this post and try to regenerate a .pfx from .pem, but that also didn't help.
QSslCertificate::importPkcs12 fails to parse PFX file
In above case, the QSslCertificate::importPkcs12() returns false for the original .pfx file. Even if we generate a new .pfx from the command line, that also fails for the above function.
Question: Can someone help with exact way of performing the .pfx encryption with the server?
.pem authentication is also fine.
Note:
Server supports both .pfx & .pem. That we confirmed with regular C OpenSSL libraries. But we want to achieve it using Qt.
We are open to formats other than .pfx, should they work in all the platforms
DISCLAIMER: I am writing this from the top of my mind, since I don't personally own a Mac and cannot verify it anymore.
We had this exact problem about a year or two ago at my last job.
It all boils down to Apple dropping support for OpenSSL.
Because of that, Qt switched from OpenSSL backend to Secure Transport backend on Mac with Qt5.6. Now the Secure Transport implementation is lacking some features. For example we were not able to load private key pem-files. I think switching from PKCS#8 to PKCS#1 helped, which can both be stored in .pem files and look almost identical, so that took a while to figure out.
We also noticed that a successfully loaded private key will be stored inside the Mac's key store and could be viewed and exported from there by the user, which we also did not want.
We finally went with re-compiling the QtNetwork module to use OpenSSL instead of Secure Transport. You will need to provide OpenSSL for that, since OSX does not include the headers anymore. A homebrew installation was sufficient I think. Other than that the compilation was surprisingly painless and fast, since you just have to compile one small module, not the whole Qt.
The easiest way to do this is:
download the source distribution of the Qt version you are running
./configure it to use OpenSSL (the -openssl switch I believe)
cdinto the network folder
make
copy the generated QtNetwork.framework inside your Qt-Installation and replace the existing one.
With that everything worked as expected.
I'm developing application in C++ (cross-platform; Windows, Mac and Linux) that needs to communicate securely with servers using https protocol with libcurl (built with winssl/darwinssl/openssl on Windows/Mac/Linux respectively). I've changed a curl option, CURLOPT_SSL_VERIFYPEER from 0 to 1 which should help prevent MitM issues.
This has caused issues that an initial search points to turning that option off, but after digging deeper I found:
Get a CA certificate that can verify the remote server and use the proper option to point out this CA cert for verification when connecting. For libcurl hackers: curl_easy_setopt(curl, CURLOPT_CAPATH, capath); from curl docs
and
Get a better/different/newer CA cert bundle! One option is to extract the one a recent Firefox browser uses by running 'make ca-bundle' in the curl build tree root, or possibly download a version that was generated this way for you.
from curl docs
I actually use CURLOPT_CAINFO to the bundle as I had seen some word of issues using CURLOPT_CAPATH on Windows; curl docs. I have downloaded and installed this bundle along with the application on Windows and Mac and I'd like to know if this is the correct way to do it or if there is a better practice.
Initially this caused issues for users of the application running behind some corporate networks or proxy which seemed to get fixed by building libcurl against winssl instead of openssl on Windows; though potentially disguising itself as a firewall issue, still unclear although it seems likely.
Sorry for the length.
Is anything silly about installing the ca-cert-bundle.crt along with the application, and is there anything that should be done differently to communicate securely with the server from this installed application?
A slightly separate, but still very related, issue I have is CURLOPT_CAINFO on Linux giving the error:
error setting certificate verify locations:
CAfile: ../share/my_application/curl-ca-bundle.crt
CApath: none
Though attempting to open the file for reading from within the application does work successfully. Edit: This issue I solved by NOT setting the CURLOPT_CAINFO field on Linux (leaving it blank) and adding the dependency package ca-certificates to the application package. The default path is correctly /etc/ssl/certs/ca-certificates.crt and seems to be working. To me this feels a bit better than installing the bundle with the application.
Edit2: Although solved it appears the ca-certificates package sometimes doesn't install ca-certificates.crt and instead ca-bundle.crt and the locations vary on different distros as this source, happyassassin.net shows that different Linux systems store the CA bundles in different locations. It did not seem to have a clear answer as to HOW to handle this. Should I be using a value in the configuration file that the user can then modify, or any other thoughts on the subject?
Edit3: Some users have pointed out that my name exists in one of the paths curl looks for, I'm not entirely sure how that is possible as the only thing I've specified for curl is where I built openssl/cares libraries...
I realize this is a loaded/multipart question but it is all on the same subject as the title states, I'd appreciate any help.
Thanks.
In my opinion, it is better to use system certificate then package certificates with application (if you are not using some special certs). For the linux it should be easy according to https://serverfault.com/questions/394815/how-to-update-curl-ca-bundle-on-redhat And for windows you can either use winssl or create the file from system https://superuser.com/questions/442793/why-cant-curl-properly-verify-a-certificate-on-windows Configure cURL to use default system cert store
A default libcurl build is setup to attempt to use the "right" CA bundle.
Linux
A libcurl built on Linux will scan and check where the CA store is located on your system and use that. If you install libcurl on a regular Linux distro, it should've been built to use the distro's "typical" CA store.
macOS
If you build libcurl for mac and tell it to use the Secure Transport backend, it will automatically use the macOS CA store. So will the default-installed curl and libcurls that come shipped bundled with macOS from Apple.
Windows
If you build libcurl for Windows to use Schannel (the windows TLS system) it will by default use the Windows CA store.
Other setups
If you deviate from these setups, you basically opt to not use the CA store that comes bundled in the operating system you're using. Then you need to handle and update the CA store yourself.
I am quite new to secure networking and I am trying to make a simple networking program with boost asio ssl and I have read all of the documentation available and also lots of questions and answers, but no one has yet asked:
Can I make a simple encrypted network using boost asio ssl without all of the certificate and verifying stuff. Just a public/private key pair on both ends, then some sort of public key exchange (handshake?) and then secure communication?
If I am wrong or mistaken about something, please do correct me. Simple examples would be much appreciated.
See here for a minimal demo client/server using Boost Asio:
Boost Asio SSL Certification on iOS
This answer adds instructions on how to create a certificate and use it:
Some OpenSSL source that mysteriously doesn't work
Can I use boost asio for HTTPS requests? I can make GET and POST HTTP requests, but what about HTTPS? How can I handle it? Can somebody provide me a code snippet?
Yes you can.
http://www.boost.org/doc/libs/1_47_0/doc/html/boost_asio/example/ssl/client.cpp
Simply integrate it to your HTTP request.
Asio offers basic SSL support through OpenSSL. A code example is available as part of the documentation
In general, HTTPS is quite similar to HTTP, except for the fact that you have to perform an SSL handshake to initialize the connection. Asio offers an implementation for this.
The actual communication is quite easy, as you simply encrypt your HTTP stream, the actual communication patterns are the same.
Therefore, if the functionality offered by Asio is not flexible enough, you can also write your own encryption layer on top of Asio using OpenSSL (although I would not recommend this unless you already have a fair deal of experience with encryption).