Unable to derive Diffie Hellman shared secret with OpenSSL in C++ - c++

Question

I'm trying to familiarise myself with the OpenSSL Diffie Hellman features and, in doing so, I've tried to create a simple programme which will generate two sets of Diffie Hellman private and public keys and then derive the shared secret. I have followed the Diffie Hellman tutorial on the OpenSSL wiki and I am able to generate the keys, however I am unable to derive the shared secret. My C++ (Linux) code is as follows:
#include <iostream>
#include <openssl/dh.h>
#include <openssl/engine.h>
#include <openssl/err.h>
#include <openssl/evp.h>
#include <openssl/rsa.h>
int main(int argc, char** argv)
{
EVP_PKEY* params;
EVP_PKEY_CTX *kctx1, *kctx2, *dctx1, *dctx2;
unsigned char *skey1, *skey2;
size_t skeylen1, skeylen2;
EVP_PKEY *dh_1, *dh_2;
BIO* bio_out = NULL;
int result = 0;
ENGINE* eng;
BIO* fp = BIO_new_fp(stdout, BIO_NOCLOSE);
// Initalise Diffie Hellman PKEY for client 1
if(NULL == (dh_1 = EVP_PKEY_new())) {
std::cout << "error 1" << std::endl;
}
// Initalise Diffie Hellman PKEY for client 2
if(NULL == (dh_2 = EVP_PKEY_new())) {
std::cout << "error 2" << std::endl;
}
// Initalise Diffie Hellman parameter PKEY
if(NULL == (params = EVP_PKEY_new())) {
std::cout << "error 3" << std::endl;
}
// Set Diffie Hellman paramerers
if(1 != EVP_PKEY_set1_DH(params, DH_get_2048_256())) {
std::cout << "error 4" << std::endl;
}
// Initalise client 1 PKEY Context
if(!(kctx1 = EVP_PKEY_CTX_new(params, NULL))) {
std::cout << "error 5" << std::endl;
}
// Initalise client 2 PKEY Context
if(!(kctx2 = EVP_PKEY_CTX_new(params, NULL))) {
std::cout << "error 6" << std::endl;
}
if(!kctx1) {
std::cout << "error 7" << std::endl;
}
if(!kctx2) {
std::cout << "error 8" << std::endl;
}
// Initalise both contexts key generators
if(1 != EVP_PKEY_keygen_init(kctx1)) {
std::cout << "error 9" << std::endl;
}
if(1 != EVP_PKEY_keygen_init(kctx2)) {
std::cout << "error 10" << std::endl;
}
// Generate DH public and private keys for client 1
if(1 != EVP_PKEY_keygen(kctx1, &dh_1)) {
std::cout << "error 11" << std::endl;
}
// Generate DH public and private keys for client 2
if(1 != EVP_PKEY_keygen(kctx2, &dh_2)) {
std::cout << "error 12" << std::endl;
}
// EVP_PKEY_print_public(fp, dh_1, 3, NULL);
// EVP_PKEY_print_public(fp, dh_2, 3, NULL);
// Create key derivation context
if(NULL == (dctx1 = EVP_PKEY_CTX_new(dh_1, NULL))) {
std::cout << "error 13" << std::endl;
}
if(!dctx1) {
std::cout << "error 14" << std::endl;
}
// Initalise first key derivation context
if(1 != EVP_PKEY_derive_init(dctx1)) {
std::cout << "error 15" << std::endl;
}
if(1 != EVP_PKEY_check(dctx1)) {
std::cout << "error 16" << std::endl;
}
if(1 != EVP_PKEY_param_check(dctx1)) {
std::cout << "error 17" << std::endl;
}
// Set first key derivation context peer key to the second DH PKEY
if(1 != EVP_PKEY_derive_set_peer(dctx1, dh_2)) {
std::cout << "error 18" << std::endl;
}
if(1 != EVP_PKEY_public_check(dctx1)) {
std::cout << "error 19" << std::endl;
}
/* Determine buffer length */
if(EVP_PKEY_derive(dctx1, NULL, &skeylen1) <= 0) {
}
// Assign memory for shared key variable
skey1 = (unsigned char*)OPENSSL_malloc(skeylen1);
if(result = EVP_PKEY_derive(dctx1, skey1, &skeylen1) <= 0) {
std::cout << "Key: " << skey1 << std::endl;
for(int i = 0; i < skeylen1; i++) {
std::cout << std::hex << (unsigned int)skey1[i];
}
}
ERR_print_errors_fp(stdout);
std::cout << result << std::endl;
}
My main concerns are:
What do I need to do to derive the shared secret?
Do I need a new context for each key pair and key derivation?
Thanks!
EDIT:
I get the following errors and no shared key:
140690271102784:error:05079079:Diffie-Hellman routines:DH_check_ex:unable to check generator:../crypto/dh/dh_check.c:92:
140690271102784:error:05079076:Diffie-Hellman routines:DH_check_ex:check p not safe prime:../crypto/dh/dh_check.c:96:
140690271102784:error:0507B07B:Diffie-Hellman routines:DH_check_pub_key_ex:check pubkey too large:../crypto/dh/dh_check.c:190:
140690271102784:error:0507B07A:Diffie-Hellman routines:DH_check_pub_key_ex:check pubkey invalid:../crypto/dh/dh_check.c:192:
when I use
if(NULL == (dctx1 = EVP_PKEY_CTX_new(dh_1, NULL))) {
std::cout << "error 13" << std::endl;
}
and I get the following errors
error 16
error 19
Key: �ȭ��U
c0c8add7c4550000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
139939742943040:error:060BA096:digital envelope routines:EVP_PKEY_check:operation not supported for this keytype:../crypto/evp/pmeth_gn.c:188:
139939742943040:error:0507C07D:Diffie-Hellman routines:dh_pkey_public_check:missing pubkey:../crypto/dh/dh_ameth.c:517:
139939742943040:error:05066064:Diffie-Hellman routines:compute_key:no private value:../crypto/dh/dh_key.c:183:
1
when I change "EVP_PKEY_CTX_new(dh_1, NULL)" to "EVP_PKEY_CTX_new(params, NULL)", like so:
if(NULL == (dctx1 = EVP_PKEY_CTX_new(params, NULL))) {
std::cout << "error 13" << std::endl;
}
The difference between these two variables is that "dh_1" stores a DH keypair, whereas "params" contains the DH parameters. It appears that the derivation context initalises to some extent with the latter, though it does not have a private and public key associated with it.

Answer 1

With the version of the code that you posted I got slightly different errors to you. I get the following output from your code (linked against OpenSSL 1.1.1):
error 16
error 17
140673674348352:error:060BA096:digital envelope routines:EVP_PKEY_check:operation not supported for this keytype:crypto/evp/pmeth_gn.c:187:
140673674348352:error:05079076:Diffie-Hellman routines:DH_check_ex:check p not safe prime:crypto/dh/dh_check.c:93:
0
The first "error 16" output is simply because OpenSSL does not currently support the EVP_PKEY_check() call for Diffie-Hellman keys. So this can be safely ignored.
The second "error 17" output (which is associated with the "check p not safe prime" error in the error queue) is more of an issue. This is because Diffie-Hellman keys come in two different varieties. By default OpenSSL uses PKCS#3 Diffie-Hellman keys. However it also supports X9.42 Diffie-Hellman keys. The function DH_get_2048_256() gives you a built-in set of parameters for X9.42 keys. However the function EVP_PKEY_set1_DH() expects the provided DH object to be a PKCS#3 key.
This actually looks like a bug in OpenSSL to me (EVP_PKEY_set1_DH() should really detect what type of key it is and do the right thing), so I have raised the following OpenSSL issue for this:
https://github.com/openssl/openssl/issues/10592
You can work around this by replacing the EVP_PKEY_set1_DH() call with one to EVP_PKEY_assign() instead and specify the type as EVP_PKEY_DHX as follows:
if(1 != EVP_PKEY_assign(params, EVP_PKEY_DHX, DH_get_2048_256())) {
std::cout << "error 4" << std::endl;
}
Alternatively you might just use PKCS#3 parameters instead.
Finally, the reason you get no shared secret derived is because you are only printing it if the EVP_PKEY_derive() call fails! It is actually succeeding!
I made the following changes to your code and it works for me:
--- derive.cpp 2019-12-09 11:11:15.493349734 +0000
+++ derive-new.cpp 2019-12-09 11:14:59.348715074 +0000
## -37,7 +37,7 ##
}
// Set Diffie Hellman paramerers
- if(1 != EVP_PKEY_set1_DH(params, DH_get_2048_256())) {
+ if(1 != EVP_PKEY_assign(params, EVP_PKEY_DHX, DH_get_2048_256())) {
std::cout << "error 4" << std::endl;
}
## -96,9 +96,11 ##
std::cout << "error 15" << std::endl;
}
+#if 0
if(1 != EVP_PKEY_check(dctx1)) {
std::cout << "error 16" << std::endl;
}
+#endif
if(1 != EVP_PKEY_param_check(dctx1)) {
std::cout << "error 17" << std::endl;
## -120,7 +122,7 ##
// Assign memory for shared key variable
skey1 = (unsigned char*)OPENSSL_malloc(skeylen1);
- if(result = EVP_PKEY_derive(dctx1, skey1, &skeylen1) <= 0) {
+ if((result = EVP_PKEY_derive(dctx1, skey1, &skeylen1)) > 0) {
std::cout << "Key: " << skey1 << std::endl;
for(int i = 0; i < skeylen1; i++) {

Answer 2

Above is Eliptic Curve Diffie and Hellman. A working example with OpenSSL 3 (none eliptic curve)
https://gist.github.com/digitalhuman/2a2b85d61672e4bf83596d41351723ba