OpenSSL: EVP_DigestSign() does't work on the Raspberry pi - c++

Question

I have trouble creating signatures with OpenSSL on a Raspberry Pi. RSA, ECDSA and EdDSA fail. Here is a small example:
#pragma once
#include <vector>
#include <string>
#include <openssl/evp.h>
#include <openssl/rsa.h>
#include <openssl/err.h>
void main()
{
// message to be sign
std::string msg_str = "my messsage";
std::vector<uint8_t> msg(msg_str.begin(), msg_str.end());
// generate a key pair
EVP_PKEY* key = EVP_PKEY_Q_keygen(nullptr, nullptr, "RSA", 2048);
// signature
std::vector<uint8_t> sig_out(1024); // reserve some memory
// signing process
EVP_MD_CTX* md_ctx(EVP_MD_CTX_new());
size_t out_len = 0; // final signature length (calculated by openssl)
if (EVP_DigestSignInit(md_ctx, nullptr, EVP_sha1(), nullptr, key) == 0)
{
std::cout << "EVP_DigestSignInit error: " << ERR_error_string(ERR_get_error(), NULL) << std::endl;
}
if (EVP_DigestSign(md_ctx, sig_out.data(), &out_len, msg.data(), msg.size()) == 0)
{
std::cout << "EVP_DigestSign error: " << ERR_error_string(ERR_get_error(), NULL) << std::endl;
}
else
{
sig_out.resize(out_len);
std::cout << "signature length: " << sig_out .size() << " data: " << std::endl;
for (size_t i = 0; i < out_len; i++)
{
printf("%.2X ", sig_out[i]);
}
}
EVP_PKEY_free(key);
EVP_MD_CTX_free(md_ctx);
return;
}
On my desktop PC all signatures work but not on the Raspberry Pi. Here is the screen output:
Windows 10, AMD CPU, x64 system, OpenSSL 3.0.0:
signature length: 256 data:
59 A9 45 F5 2B 97 51 F5 53 A8 AE 17 16 7A 26 28
F5 68 3F 1F 3D B2 05 4F 0E 28 AF F2 F5 0E DA FF
37 71 50 DD DA E1 DE F0 91 05 0A 07 79 30 00 03
A4 1E F5 60 F5 7E 47 97 EF 88 9C 27 70 CE 64 63
0B 6C 2E 50 7B D7 89 48 B6 73 44 AD 7A 02 EA 49
BC D3 95 67 B8 E6 D9 E4 A1 4F 2B E8 F4 5C F8 73
B5 53 B0 A5 FB BB 7A 81 1C 25 23 6F 30 D8 8F D8
EC 9E 02 00 C2 0D 7C 9C 23 66 D7 44 62 FF 51 1A
94 3F 6F FB D7 B2 C5 2B A4 03 09 E5 10 44 D4 AE
A2 69 F3 EB 31 1B CB 2A 14 1D 76 CD 11 09 B9 76
99 59 42 5A 74 3D 14 98 B7 87 FD 98 16 17 AC 9E
DA 55 82 0B 93 3D 24 28 4F 09 EB EA AE 82 77 47
B2 E2 C8 1E 62 FF E4 90 E6 18 E8 88 94 B4 F0 AF
DF A2 2B D7 79 32 BD C5 0F B1 03 36 B6 D8 44 9A
FA DB 02 EB 7D FE D5 D7 15 34 77 72 4D 4E 44 A8
E7 DA D9 2B 49 80 43 58 1F AA F4 1D 27 80 1C EE
Raspberry Pi, ARM CPU, Debian (bullseye), x64 system, OpenSSL 3.1.0:
EVP_DigestSign error: error:1C8000B3:Provider routines::invalid signature size
does anyone have any idea what the problem may be? I am now a bit desperate.
best regards,
SBond

Answer 1

solved!
this error occurs (only on ARM) when out_len is smaller than the final signature length. I need to change the code as follows:
from
if (EVP_DigestSign(md_ctx, sig_out.data(), &out_len, msg.data(), msg.size()) == 0)
{
std::cout << "EVP_DigestSign error: " << ERR_error_string(ERR_get_error(), NULL) << std::endl;
}
to
EVP_DigestSign(md_ctx, nullptr, &out_len, msg.data(), msg.size()); // calculates the final length (does not perform signature; therfore a fast process)
sig_out.resize(out_len);
if (EVP_DigestSign(md_ctx, sig_out.data(), &out_len, msg.data(), msg.size()) == 0)
{
std::cout << "EVP_DigestSign error: " << ERR_error_string(ERR_get_error(), NULL) << std::endl;
}
now I'm happy :)