Why is my OpenSSL/libcrypto signature incorrect? - c++

I need to sign a string, publish the public key as a string, and then somewhere else use the public key to verify the signed message. This is the part where the message is signed:
// RSA keypair generation
EVP_PKEY *keypair = NULL;
EVP_PKEY_CTX *ctx = EVP_PKEY_CTX_new_id(EVP_PKEY_RSA, NULL);
if (1 != EVP_PKEY_keygen_init(ctx)) {
initFail = true;
}
if (1 != EVP_PKEY_CTX_set_rsa_keygen_bits(ctx, 2048)) {
initFail = true;
}
if (1 != EVP_PKEY_keygen(ctx, &keypair)) {
initFail = true;
}
EVP_PKEY_CTX_free(ctx);
// Create public key string.
BIO* bo = BIO_new(BIO_s_mem());
PEM_write_bio_PUBKEY(bo, keypair);
char buff[1000];
BIO_read(bo, &buff[0], 1000);
BIO_free(bo);
pubkey = buff;
// Create signature
size_t *slen = new size_t;
unsigned char *sig = NULL;
std::string msg;
msg = stuffThatCreatesMessage();
EVP_MD_CTX *mdctx = EVP_MD_CTX_create();
if (1 != EVP_DigestSignInit(mdctx, NULL, EVP_sha256(),
NULL, keypair)) {
initFail = true;
}
if (1 != EVP_DigestSignUpdate(mdctx, msg.c_str(), msg.length())) {
initFail = true;
}
if (1 != EVP_DigestSignFinal(mdctx, NULL, slen)) {
initFail = true;
}
sig = (unsigned char *) OPENSSL_malloc(
sizeof(unsigned char) * (*slen));
if (1 != EVP_DigestSignFinal(mdctx, sig, slen)) {
initFail = true;
}
signature = *sig;
OPENSSL_free(sig);
bool isSuccess = verifySignature(signature, pubkey, msg);
Here's the code that takes the string message and key and actually verifies the signature:
bool verifySignature(std::string sig, std::string key_str, std::string msg) {
BIO* bo = BIO_new(BIO_s_mem());
BIO_write(bo, key_str.c_str(), key_str.length());
EVP_PKEY *key = EVP_PKEY_new();
PEM_read_bio_PUBKEY(bo, &key, 0, 0);
BIO_free(bo);
unsigned char *unsigned_sig = new unsigned char[sig.length()+1];
strcpy((char *) unsigned_sig, sig.c_str());
EVP_MD_CTX *mdctx = EVP_MD_CTX_create();
if (1 != EVP_DigestVerifyInit(mdctx, NULL, EVP_sha256(), NULL, key)) {
return false;
}
if (1 != EVP_DigestVerifyUpdate(mdctx, msg.c_str(), msg.length())) {
return false;
}
bool retval = (1 == EVP_DigestVerifyFinal(mdctx, unsigned_sig,
sig.length()));
delete unsigned_sig;
return retval;
}
Every time I do this, the latter function tells me the signature is invalid, with the isSuccess variable equal to 0. It's because the EVP_DigestSignFinal is returning 0, indicating that the signature is incorrect. Where am I going wrong?

The problem, pointed out by #jww, was the line signature = *sig. I was ending up with only the first byte as my signature string. The correct way to do it is:
std::string temp_sig = reinterpret_cast<char*>(sig);
signature = temp_sig.substr(0, *slen);

Related

EVP_KEY_sign gives signature length 0 unexpectedly

I'm trying to sign a message with a private key in C++ with openssl. I have followed an example but for some reason I can't get EVP_PKEY_sign to work because it sets the signature length to 0. Is there anything wrong with my code?
function to sign message
void sign_buff(const string& priv_key_file_path, const unsigned char* buff, int buff_len, string& sig) {
FILE* f = fopen(priv_key_file_path.c_str(), "r");
EC_KEY* ec_key = PEM_read_ECPrivateKey(f, NULL, NULL, NULL);
fclose(f);
assert(1 == EC_KEY_check_key(ec_key));
EVP_PKEY* key = EVP_PKEY_new();
assert(1 == EVP_PKEY_assign_EC_KEY(key, ec_key));
EVP_PKEY_CTX* key_ctx = EVP_PKEY_CTX_new(key, NULL);
assert(1 == EVP_PKEY_sign_init(key_ctx));
assert(1 == EVP_PKEY_CTX_set_signature_md(key_ctx, EVP_sha256()));
size_t sig_len = 0;
assert(1 == EVP_PKEY_sign(key_ctx, NULL, &sig_len, buff, buff_len));
cout << sig_len; //gives 0
sig.assign(sig_len, 0);
assert(1 == EVP_PKEY_sign(key_ctx, (unsigned char*)&sig[0], &sig_len, buff, buff_len));
EVP_PKEY_CTX_free(key_ctx);
EVP_PKEY_free(key);
}
I call the function with
string priv="priv.pem";
string sig;
unsigned char buff[] = "Random message to be signed";
sign_buff(priv, &buff[0], sizeof(buff), sig);
where a private key is stored in priv.pem

RSA_Verify fails returns Bad signature error

I can't get the message verified after signing it.
I'm writing a project to simulate signing and verifying data from files with OpenSSL.
the signing process seems to work great, but on the verification process RSA_Verify always return 0. which is not verified.
I can't get it. what am I missing here?
#include <string>
#include <fstream>
#include <openssl/rsa.h>
#include <openssl/pem.h>
#include <openssl/applink.c>
#include <openssl/err.h>
#define PRIKEY_FILENAME "private"
#define PUBKEY_FILENAME "public"
using namespace std;
typedef struct _INFO
{
unsigned char *sig;
unsigned int nLen;
}INFO, *pINFO;
bool ReadFileContent(string fileName, char** out, size_t &nLen)
{
if (NULL == out)
return false;
ifstream file(fileName, ios::in | ios::binary | ios::ate);
if (file.is_open())
{
nLen = (int)file.tellg();
*out = new char[nLen];
file.seekg(0, ios::beg);
file.read(*out, nLen);
file.close();
}
else
{
cout << "Unable to open file \"" << fileName << " \"\n";
return false;
}
return true;
}
bool WriteFileContent(const char* data, int nLen, string fileName)
{
if (NULL == data)
return false;
ofstream file(fileName, ios::out | ios::binary | ios::ate);
if (file.is_open())
{
file.write(data, nLen);
file.close();
}
else
{
cout << "Unable to open file \"" << fileName << " \"\n";
return false;
}
return true;
}
bool GenerateKeyPairs()
{
int ret = 0;
RSA *r = NULL;
BIGNUM *bne = NULL;
BIO *bp_public = NULL, *bp_private = NULL;
int bits = 2048;
unsigned long e = RSA_F4;
// 1. generate rsa key
bne = BN_new();
ret = BN_set_word(bne, e);
if (ret != 1)
{
goto free_all;
}
r = RSA_new();
ret = RSA_generate_key_ex(r, bits, bne, NULL);
if (ret != 1)
{
goto free_all;
}
// 2. save public key
bp_public = BIO_new_file(PUBKEY_FILENAME, "w+");
ret = PEM_write_bio_RSAPublicKey(bp_public, r);
if (ret != 1)
{
goto free_all;
}
// 3. save private key
bp_private = BIO_new_file(PRIKEY_FILENAME, "w+");
ret = PEM_write_bio_RSAPrivateKey(bp_private, r, NULL, NULL, 0, NULL, NULL);
// 4. free
free_all:
BIO_free_all(bp_public);
BIO_free_all(bp_private);
RSA_free(r);
BN_free(bne);
return (ret == 1);
}
bool DoSign(string priKeyFile, pINFO pInfo, string fileName)
{
int ret;
unsigned char* data = NULL;
unsigned char* encodedData = NULL;
size_t nFileSize = 0;
unsigned int nEncodedDataLen = 0;
RSA* priKey = NULL;
FILE* fp = NULL;
if (!ReadFileContent(fileName, (char**)&data, nFileSize))
{
return false;
}
if (data == NULL || nFileSize <= 0)
return false;
//SHA512(data, nFileSize, pInfo->sig);
fp = fopen(PRIKEY_FILENAME, "r");
priKey = PEM_read_RSAPrivateKey(fp, &priKey, NULL, NULL);
pInfo->sig = (unsigned char*)malloc(RSA_size(priKey));
/* Sign */
ret = RSA_sign(NID_sha512, data, nFileSize, pInfo->sig, &pInfo->nLen, priKey);
WriteFileContent((char*)pInfo->sig, pInfo->nLen, fileName + ".sign");
return true;
}
bool DoVerify(string pubKeyFile, pINFO pInfo, string fileName)
{
int ret = 0;
unsigned char* data = NULL;
unsigned char* encodedData = NULL;
size_t nFileSize = 0;
FILE* fp = NULL;
RSA* pubkey = NULL;
unsigned int nEncodedDataLen = 0;
if (!ReadFileContent(fileName, (char**)&data, nFileSize))
{
return false;
}
if (data == NULL || nFileSize <= 0)
return false;
fp = fopen(PUBKEY_FILENAME, "r");
pubkey = PEM_read_RSAPublicKey(fp, NULL, NULL, NULL);
ret = RSA_verify(NID_sha512, data, nFileSize, pInfo->sig, pInfo->nLen, pubkey);
char buf[512];
ERR_error_string(ERR_get_error(), buf);
//Error here says bad signature
return true;
}
int main()
{
INFO info = { 0 };
GenerateKeyPairs();
DoSign(PRIKEY_FILENAME, &info, "Hello.txt");
DoVerify(PUBKEY_FILENAME, &info, "Hello.txt.sign");
return 0;
}
It should return 1 in RSA_Sign.
DoVerify(PUBKEY_FILENAME, &info, "Hello.txt.sign");
This should be:
DoVerify(PUBKEY_FILENAME, &info, "Hello.txt");
The signature itself is being passed through the &info parameter. The last parameter is supposed to be the actual thing signed, not the signature.

How to get privatekey in openssl ecc with the format hex

code:
EC_KEY *key;
if (NULL == (key = EC_KEY_new_by_curve_name(NID_secp224r1)))
handleErrors();
if (1 > EC_KEY_generate_key(key)) handleErrors();
const BIGNUM *prv = EC_KEY_get0_private_key(key);
const EC_GROUP *group = EC_KEY_get0_group(key);
const EC_POINT *pub = EC_KEY_get0_public_key(key);
char* hexPubkey = EC_POINT_point2hex(group, pub, EC_KEY_get_conv_form(key), NULL);
char* hexPrikey=BN_bn2hex(prv);
my result:
hexPubkey:04EAD3AF4BA89F513B2D89FC749C43CC7B95523F1BD40A5713C5228F91B5F928D43B396C64A3293053550065C02E9A06B4FB078C4944BD0933
length:57
hexPrikey:5D5356F3551602A89710DA40CC24FB6CDBB851FE612C977C9AB0F393
length:114
correct result:
hexPubkey:04381987517AEEDB6F83FAA0EB60EFF4C1B7A78D66FF17CB04D92B37588826A64B78E0B3A965C72438860D4B3897893BB31397D1625EEA0E41
length:57
hexPrikey:30820144020101041C20F21D6B7FE0D5E1D21C6B57AFE6FAD8A5A5AC31AE0596B972E01877A081E23081DF020101302806072A8648CE3D0101021D00FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF0000000000000000000000013053041CFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFEFFFFFFFFFFFFFFFFFFFFFFFE041CB4050A850C04B3ABF54132565044B0B7D7BFD8BA270B39432355FFB4031500BD71344799D5C7FCDC45B59FA3B9AB8F6A948BC5043904B70E0CBD6BB4BF7F321390B94A03C1D356C21122343280D6115C1D21BD376388B5F723FB4C22DFE6CD4375A05A07476444D5819985007E34021D00FFFFFFFFFFFFFFFFFFFFFFFFFFFF16A2E0B8F03E13DD29455C5C2A3D020101A13C033A0004381987517AEEDB6F83FAA0EB60EFF4C1B7A78D66FF17CB04D92B37588826A64B78E0B3A965C72438860D4B3897893BB31397D1625EEA0E41
length:328
From the length of the private key, my private key is obviously wrong, isn't it my writing method wrong?
BIO *bio = BIO_new(BIO_s_mem());
PEM_write_bio_ECPrivateKey(bio, key, NULL, NULL, 0, NULL, NULL);
int keylen = BIO_pending(bio);
char *pem_key = (char *) calloc(keylen + 1, 1);
BIO_read(bio, pem_key, keylen);
cout << pem_key << endl;
BIO_free_all(bio);
EC_KEY_free(key);
free(pem_key);

C++ openssl - Generate RSA Keypair and read

I am trying to generate RSA keypair using openssl library and then read the same keys later. However, it fails. Sometimes it gives me this error:
error:0906D06C:PEM routines:PEM_read_bio:no start line
And sometimes, it gives me this error:
error:0906D06C:lib(9):func(109):reason(108)
What is the correct way to generate the keypair and later be able to read it ? Here is my code. If you run it, you will find that it correctly generates the RSA key pair but not able read them later.
#include <stdio.h>
#include <iostream>
#include <openssl/rsa.h>
#include <openssl/pem.h>
#include <openssl/err.h>
#include <exception>
bool generate_key() {
size_t pri_len; // Length of private key
size_t pub_len; // Length of public key
char *pri_key; // Private key in PEM
char *pub_key; // Public key in PEM
int ret = 0;
RSA *r = NULL;
BIGNUM *bne = NULL;
BIO *bp_public = NULL, *bp_private = NULL;
int bits = 2048;
unsigned long e = RSA_F4;
EVP_PKEY *evp_pbkey = NULL;
EVP_PKEY *evp_pkey = NULL;
BIO *pbkeybio = NULL;
BIO *pkeybio = NULL;
// 1. generate rsa key
bne = BN_new();
ret = BN_set_word(bne, e);
if (ret != 1) {
goto free_all;
}
r = RSA_new();
ret = RSA_generate_key_ex(r, bits, bne, NULL);
if (ret != 1) {
goto free_all;
}
// 2. save public key
//bp_public = BIO_new_file("public.pem", "w+");
bp_public = BIO_new(BIO_s_mem());
ret = PEM_write_bio_RSAPublicKey(bp_public, r);
if (ret != 1) {
goto free_all;
}
// 3. save private key
//bp_private = BIO_new_file("private.pem", "w+");
bp_private = BIO_new(BIO_s_mem());
ret = PEM_write_bio_RSAPrivateKey(bp_private, r, NULL, NULL, 0, NULL, NULL);
//4. Get the keys are PEM formatted strings
pri_len = BIO_pending(bp_private);
pub_len = BIO_pending(bp_public);
pri_key = (char*) malloc(pri_len + 1);
pub_key = (char*) malloc(pub_len + 1);
BIO_read(bp_private, pri_key, pri_len);
BIO_read(bp_public, pub_key, pub_len);
pri_key[pri_len] = '\0';
pub_key[pub_len] = '\0';
printf("\n%s\n%s\n", pri_key, pub_key);
//verify if you are able to re-construct the keys
pbkeybio = BIO_new_mem_buf((void*) pub_key, -1);
if (pbkeybio == NULL) {
return -1;
}
evp_pbkey = PEM_read_bio_PUBKEY(pbkeybio, &evp_pbkey, NULL, NULL);
if (evp_pbkey == NULL) {
char buffer[120];
ERR_error_string(ERR_get_error(), buffer);
printf("Error reading public key:%s\n", buffer);
}
pkeybio = BIO_new_mem_buf((void*) pri_key, -1);
if (pkeybio == NULL) {
return -1;
}
evp_pkey = PEM_read_bio_PrivateKey(pkeybio, &evp_pkey, NULL, NULL);
if (evp_pbkey == NULL) {
char buffer[120];
ERR_error_string(ERR_get_error(), buffer);
printf("Error reading private key:%s\n", buffer);
}
BIO_free(pbkeybio);
BIO_free(pkeybio);
// 4. free
free_all:
BIO_free_all(bp_public);
BIO_free_all(bp_private);
RSA_free(r);
BN_free(bne);
return (ret == 1);
}
int main(int argc, char* argv[]) {
generate_key();
return 0;
}
Looks good to me. Except on reloading; I would have used PEM_read_bio_RSAPublicKey in stead of PEM_read_bio_PUBKEY.
I am not sure it is the best way to do it though.
--- /tmp/stack_openssl.cpp.back 2018-05-25 12:53:12.366488025 +0000
+++ /tmp/stack_openssl.cpp 2018-05-25 13:57:20.614066828 +0000
## -18,6 +18,8 ##
int bits = 2048;
unsigned long e = RSA_F4;
+ RSA *pb_rsa = NULL;
+ RSA *p_rsa = NULL;
EVP_PKEY *evp_pbkey = NULL;
EVP_PKEY *evp_pkey = NULL;
## -66,27 +68,32 ##
printf("\n%s\n%s\n", pri_key, pub_key);
//verify if you are able to re-construct the keys
- pbkeybio = BIO_new_mem_buf((void*) pub_key, -1);
+ pbkeybio = BIO_new_mem_buf((void*) pub_key, pub_len);
if (pbkeybio == NULL) {
return -1;
}
- evp_pbkey = PEM_read_bio_PUBKEY(pbkeybio, &evp_pbkey, NULL, NULL);
- if (evp_pbkey == NULL) {
+ pb_rsa = PEM_read_bio_RSAPublicKey(pbkeybio, &pb_rsa, NULL, NULL);
+ if (pb_rsa == NULL) {
char buffer[120];
ERR_error_string(ERR_get_error(), buffer);
printf("Error reading public key:%s\n", buffer);
}
+ evp_pbkey = EVP_PKEY_new();
+ EVP_PKEY_assign_RSA(evp_pbkey, pb_rsa);
- pkeybio = BIO_new_mem_buf((void*) pri_key, -1);
+ pkeybio = BIO_new_mem_buf((void*) pri_key, pri_len);
if (pkeybio == NULL) {
return -1;
}
- evp_pkey = PEM_read_bio_PrivateKey(pkeybio, &evp_pkey, NULL, NULL);
- if (evp_pbkey == NULL) {
+ p_rsa = PEM_read_bio_RSAPrivateKey(pkeybio, &p_rsa, NULL, NULL);
+ if (p_rsa == NULL) {
char buffer[120];
ERR_error_string(ERR_get_error(), buffer);
printf("Error reading private key:%s\n", buffer);
}
+ evp_pkey = EVP_PKEY_new();
+ EVP_PKEY_assign_RSA(evp_pkey, p_rsa);
BIO_free(pbkeybio);
BIO_free(pkeybio);

error:FFFFFFFFFFFFFFFF:lib(255):func(4095):reason(4095) during RSA decyption

I'm trying to create a hybrid cryptography tool in C++ with Qt Gui.
(The data will be encrypted with AES 256-CBC, the AES Key RSA encrypted and saved then.)
But the RSA part of this tool doesn't work.
I wrote the sourcecode several times but I always get the same error on decrypt.
error:FFFFFFFFFFFFFFFF:lib(255):func(4095):reason(4095)
I hope someone could help me get a working RSA encrypt + decrypt implementation.
You can see the sourcecode here or download a testing Qt Project from my dropbox..
Dropbox Download: https://db.tt/6HKsYRTa
Sourcecode 1. Implementation:
void MainWindow::rsaEncrypt()
{
EVP_PKEY *pk = NULL;
EVP_PKEY_CTX *ctx = NULL;
QByteArray encrypted = QByteArray();
//------------------------------------------------
//--- READ PUBLIC KEY ----------------------------
FILE *pkFile = fopen(ui->publicKeyPath->text().toStdString().c_str(), "r");
if(pkFile == NULL) throw NULL;
pk = PEM_read_PUBKEY(pkFile, NULL, NULL, NULL);
if(pk == NULL) throw NULL;
fclose(pkFile);
//------------------------------------------------
ctx = EVP_PKEY_CTX_new(pk, NULL);
//------------------------------------------------
//--- ENCRYPT DATA -------------------------------
int err;
err = EVP_PKEY_encrypt_init(ctx);
if(err <= 0) throw NULL;
err = EVP_PKEY_CTX_set_rsa_padding(ctx, RSA_PADDING);
if(err <= 0) throw NULL;
size_t outLen = 0;
err = EVP_PKEY_encrypt(
ctx,
NULL,
&outLen,
(uchar*) ui->plainTextEdit->document()->toPlainText().toStdString().c_str(),
ui->plainTextEdit->document()->toPlainText().size()
);
if(err <= 0) throw NULL;
encrypted.resize(outLen);
err = EVP_PKEY_encrypt(
ctx,
(uchar*) encrypted.data(),
&outLen,
(uchar*) ui->plainTextEdit->document()->toPlainText().toStdString().c_str(),
ui->plainTextEdit->document()->toPlainText().size()
);
//------------------------------------------------
EVP_PKEY_CTX_free(ctx);
EVP_PKEY_free(pk);
if(err > 0) ui->encryptedTextEdit->document()->setPlainText(QString(encrypted));
else {
QByteArray errStr = QByteArray();
errStr.resize(256);
ERR_load_ERR_strings();
ERR_error_string(err, errStr.data());
ui->encryptedTextEdit->document()->setPlainText( QString(errStr) );
}
}
void MainWindow::rsaDecrypt()
{
EVP_PKEY *pk = NULL;
EVP_PKEY_CTX *ctx = NULL;
QByteArray decrypted = QByteArray();
//------------------------------------------------
//--- READ PRIVATE KEY ---------------------------
FILE *pkFile = fopen(ui->privateKeyPath->text().toStdString().c_str(), "r");
if(pkFile == NULL) throw NULL;
pk = PEM_read_PrivateKey(pkFile, NULL, NULL, NULL);
if(pk == NULL) throw NULL;
fclose(pkFile);
//------------------------------------------------
ctx = EVP_PKEY_CTX_new(pk, NULL);
//------------------------------------------------
//--- DECRYPT DATA -------------------------------
int err;
err = EVP_PKEY_decrypt_init(ctx);
if(err <= 0) throw NULL;
err = EVP_PKEY_CTX_set_rsa_padding(ctx, RSA_PADDING);
if(err <= 0) throw NULL;
size_t outLen = 0;
err = EVP_PKEY_decrypt(
ctx,
NULL,
&outLen,
(uchar*) ui->encryptedTextEdit->document()->toPlainText().toStdString().c_str(),
ui->encryptedTextEdit->document()->toPlainText().size()
);
if(err <= 0) throw NULL;
decrypted.resize(outLen);
err = EVP_PKEY_decrypt(
ctx,
(uchar*) decrypted.data(),
&outLen,
(uchar*) ui->encryptedTextEdit->document()->toPlainText().toStdString().c_str(),
ui->encryptedTextEdit->document()->toPlainText().size()
);
//------------------------------------------------
EVP_PKEY_CTX_free(ctx);
EVP_PKEY_free(pk);
if(err > 0) ui->decryptedTextEdit->document()->setPlainText(QString(decrypted));
else {
QByteArray errStr = QByteArray();
errStr.resize(256);
ERR_load_ERR_strings();
ERR_error_string(err, errStr.data());
ui->decryptedTextEdit->document()->setPlainText( QString(errStr) );
}
}
Sourcecode 2. Implementation:
void MainWindow::rsaEncrypt()
{
RSA *rsa = createRSAFromFile(ui->publicKeyPath->text().toStdString().c_str(), 1);
QByteArray encrypted = QByteArray();
encrypted.resize(2048);
int err = RSA_public_encrypt(
ui->plainTextEdit->document()->toPlainText().size(),
(uchar*) ui->plainTextEdit->document()->toPlainText().toStdString().c_str(),
(uchar*) encrypted.data(),
rsa,
RSA_PADDING
);
RSA_free(rsa);
if(err > 0) ui->encryptedTextEdit->document()->setPlainText( QString(encrypted) );
else {
QByteArray errStr = QByteArray();
errStr.resize(256);
ERR_load_ERR_strings();
ERR_error_string(err, errStr.data());
ui->encryptedTextEdit->document()->setPlainText( QString(errStr) );
}
}
void MainWindow::rsaDecrypt()
{
RSA *rsa = createRSAFromFile(ui->privateKeyPath->text().toStdString().c_str(), 0);
QByteArray decrypted = QByteArray();
decrypted.resize(2048);
int err = RSA_private_decrypt(
ui->encryptedTextEdit->document()->toPlainText().size(),
(uchar*) ui->encryptedTextEdit->document()->toPlainText().toStdString().c_str(),
(uchar*) decrypted.data(),
rsa,
RSA_PADDING
);
RSA_free(rsa);
if(err > 0) ui->decryptedTextEdit->document()->setPlainText( QString(decrypted) );
else {
QByteArray errStr = QByteArray();
errStr.resize(256);
ERR_load_ERR_strings();
ERR_error_string(err, errStr.data());
ui->decryptedTextEdit->document()->setPlainText( QString(errStr) );
}
}
RSA *MainWindow::createRSAFromFile(const char *keyPath, int pub)
{
FILE *keyFile = fopen(keyPath, "rb");
if(keyFile==NULL)
{
return 0;
}
RSA *rsa = RSA_new();
if(pub)
{
rsa = PEM_read_RSA_PUBKEY(keyFile, &rsa, NULL, NULL);
}
else
{
rsa = PEM_read_RSAPrivateKey(keyFile, &rsa, NULL, NULL);
}
fclose(keyFile);
return rsa;
}
Includes and defines for both implementations:
#include "mainwindow.h"
#include "ui_mainwindow.h"
#include <QDebug>
#include <QByteArray>
#include <openssl/evp.h>
#include <openssl/rsa.h>
#include <openssl/pem.h>
#include <openssl/err.h>
#define RSA_PADDING RSA_PKCS1_OAEP_PADDING
Try to use this Code for creating a RSA-object. It definetly works. You should read the .pem file before and then call this function:
RSA *CryptClassRSA::createRSA(unsigned char *key, int isPublic){
RSA *rsa = NULL;
BIO *keybio;
keybio = BIO_new_mem_buf(key, -1);
if (keybio==NULL){
printf( "Failed to create key BIO");
return NULL;
}
if(isPublic){
rsa = PEM_read_bio_RSA_PUBKEY(keybio, &rsa,NULL, NULL);
}
else{
rsa = PEM_read_bio_RSAPrivateKey(keybio, &rsa,NULL, NULL);
}
if(rsa == NULL){
printf( "Failed to create RSA");
}
return rsa;
}
C++ RSA decrypt error:FFFFFFFFFFFFFFFF:lib(255):func(4095):reason(4095)
And
int err;
...
err = EVP_PKEY_CTX_set_rsa_padding(ctx, RSA_PADDING);
if(err <= 0) throw NULL;
...
Its not clear to me where the error is happening or how you are getting the errstr output of error:FFFFFFFFFFFFFFFF:lib(255):func(4095):reason(4095). I picked EVP_PKEY_CTX_set_rsa_padding as the snippet because I think the return value is -2 (and it has significance as explained below).
err is just a return code. To get the actual error, you need to call ERR_get_error. Maybe something like:
int rc;
unsigned long err;
...
rc = EVP_PKEY_CTX_set_rsa_padding(ctx, RSA_PADDING);
err = ERR_get_error();
if(rc <= 0)
{
// err is valid
}
You should also visit the EVP_PKEY_CTX_set_rsa_padding and ERR_get_error man pages.
Often, OpenSSL will return 0 for success, so I'm not sure about err <= 0 in some places. Also, because your error is 0xffff...4095 (and not 4096), I think you are getting the -2 return value discussed in the man page:
EVP_PKEY_CTX_ctrl() and its macros return a positive value for success and 0 or a negative value for failure. In particular a return value of -2 indicates the operation is not supported by the public key algorithm.
Also note... If you gather your error and print it in hex:
rc = EVP_PKEY_CTX_set_rsa_padding(ctx, RSA_PADDING);
err = ERR_get_error();
if(rc <= 0)
{
// err is valid
std::cerr << std::hex << err << std::endl;
}
Then you can use the openssl errstr 0xNNNN to print it.