I am trying to decrypt AES-256-CBC encrypted data.
I can decrypt the entire file, but I get additional characters at the end. I also get an error from the OpenSSL libraries that says this...
EVP_DecryptFinal_ex:wrong final block length:evp_enc.c:532
Here is my decryption function which is right out of the wiki:
int decrypt(unsigned char *ciphertext, int ciphertext_len, unsigned char *key, unsigned char *iv, unsigned char *plaintext)
{
int plaintext_len = 0;
EVP_CIPHER_CTX *ctx;
int len = 0;
plaintext_len = 0;
int error = 0;
ctx = EVP_CIPHER_CTX_new();
if(ctx == NULL)
{
handleErrors(0,1);
}
error = EVP_DecryptInit_ex(ctx, EVP_aes_256_cbc(), NULL, key, iv);
handleErrors(error,2);
error = EVP_DecryptUpdate(ctx,plaintext, &len, ciphertext, ciphertext_len);
handleErrors(error, 3);
plaintext_len = len;
error = EVP_DecryptFinal_ex(ctx,plaintext + len, &len);
handleErrors(error, 4);
plaintext_len += len;
EVP_CIPHER_CTX_free(ctx);
return plaintext_len;
}
Note that I am passing in raw bytes using the uint8_t type, but converted it for this function. The plain text that is working contains the file, plus some additional characters at the end.
Here is the encrypted data in hex:
aa371af640b8481203c6bdc21bfccf109d047f0ad38daf0f03d1d7650a25616c019cdf9b80cda8980aa99809c2d9346fd1b501ef3b2f3548479840f897c85592ae90f6d5e9c87e428b00d36631963ca311b4486285495d50296a3d370aa0b82322ad0891adf43b262611cc2238769180f2ec8f7fba7dcb48f6efe9fc0f8725d18a7d29c52fc7d40193b7bed18f549f4e23c5def8f49fbf775e41e9c26d6a6ff3685d2f1a5050e660a9903305500dd946b81f49381b8db215b290b8e526eb463bb4cb9af1fdc3df14e6dfcc14eb6824400b6de24ee9e87170554caa825a2306b9d2f75432532e89871e0a4e64b43b22b69ede8127887408877f50e00368aad3c29a370ecbe3533b2e3b6a3de05ad836fc84eb2fd76c7563f24f215697bfc354c4a6e809c7e25f5d2de6df4ddae6dbb05bcd07217125268aab811183359e714df3e696eca1df2783c8855a048e0fa252c80930cf37be2c1e5eac6ac83e819a639dae9972363d7d89dcf2a2b1761888edd6c263b84cf5c6118e7544656f724156d396f8d8b3b6a49123ef633ced82bc9b65fff6f1d39d5e9e2fa32a3c58fc79459a94d237a5361d2a2db1f7b862cb0f081e6328076339392e59734a006718136d6784a12dfe93384e3e48941870bae48c563cb2474fd926963051c273bf26ce4a9d29e00d628719564cbf8769efb9726157c79b29f10644ea5656df99e20f84e0867b559b682453567c970df8ade768b6cd3498001a30160c83dd3efa3fc58c13ecac7e53041dd617905f1a9f02cc249b8c3523c92eace60a0ad9eba29ec973678859643d0d3765d7b9e2e5b58d639c7d2f6109da979981003c2e41e55d270f276b9edcfe0294d0aa34ade2cfb7140b4fbfb5d202b3079af5f7f2ec5c84a34a8d94cf3698a17fa1ee25767e00a8337f0ed6c2af9fc2dfd6dedfec45a23d376e8d1d
Here is the AES encryption key in hex
174126874a7c7f9a44da4f559cfd628586894e86f7e2eb0561a0809b7a294fd4
Here is the IV
603da7b9f9c365219a8121e528e7dddc
The result is
echo Y | format H: /q /x /fs:NTFS
#echo off
echo.
echo.Stopping the Backup Exec services...
echo.
net stop "BackupExecAgentBrowser"
net stop "BackupExecJobEngine"
net stop "BackupExecManagementService"
net stop "BackupExecRPCService"
net stop "BackupExecDeviceMediaService"
net stop "BackupExecAgentAccelerator"
net stop "bedbg"
echo.
echo.Starting the Backup Exec services...
echo.
net start "bedbg"
net start "BackupExecAgentAccelerator"
net start "BackupExecDeviceMediaService"
net start "BackupExecRPCService"
net start "BackupExecManagementService"
net start "BackupExecJobEngine"
net start "BackupExecAgentBrowser"
pause
exitC��Q�H
Notice the odd few characters at the end.
Also note, I've tried the same thing in Python and am getting the same results. I believe it has to do with the padding. But I don't know if I am supposed to pad the encrypted data, and if so, with what data, or what...
Related
I need to generate (in C/C++) a secure random string (32byte) to use as Key and another (16byte) to use as IV for AES-256-CBC encryption using WinApi. The problem is that I need to save the generated string in a text file in order to manually test the decryption using OpenSSL in terminal.
What can I use to generate the secure random string other than CryptGenRandom? The problem of CryptGenRandom is that it generates a random byte sequence that I can't save/use as OpenSSL input because it's not an ASCII text:
openssl aes-256-cbc -d -K "..." -iv ".." -in encrypted.txt
Is there an alternative?
This is my working code:
// handles for csp and key
HCRYPTPROV hProv = NULL;
HCRYPTKEY hKey = NULL;
BYTE *szKey = (BYTE*)calloc(DEFAULT_AES_KEY_SIZE + 1, sizeof(BYTE));
BYTE *szIV = (BYTE*)calloc(DEFAULT_IV_SIZE + 1, sizeof(BYTE));
char* ciphertext= 0;
DWORD dwPlainSize = lstrlenA(*plaintext), dwBufSize = 0;
AES256KEYBLOB AESBlob;
memset(&AESBlob, 0, sizeof(AESBlob));
// initalize key and plaintext
StrCpyA((LPSTR)szKey, "00112233445566778899001122334455");
StrCpyA((LPSTR)szIV, "4455667788990011");
// generate key & IV
//if (!CryptGenRandom(hProv, DEFAULT_AES_KEY_SIZE, szKey)) {goto error;}
//if (!CryptGenRandom(hProv, DEFAULT_IV_SIZE, szIV)) {goto error;}
// blob data for CryptImportKey() function (include key and version and so on...)
AESBlob.bhHdr.bType = PLAINTEXTKEYBLOB;
AESBlob.bhHdr.bVersion = CUR_BLOB_VERSION;
AESBlob.bhHdr.reserved = 0;
AESBlob.bhHdr.aiKeyAlg = CALG_AES_256;
AESBlob.dwKeySize = DEFAULT_AES_KEY_SIZE;
StrCpyA((LPSTR)AESBlob.szBytes, (LPCSTR)szKey);
// create a cryptographic service provider (CSP)
if (!CryptAcquireContextA(&hProv, NULL, MS_ENH_RSA_AES_PROV_A, PROV_RSA_AES, CRYPT_VERIFYCONTEXT)) {goto error;}
// populate crypto provider (CSP)
if (!CryptImportKey(hProv, (BYTE*)&AESBlob, sizeof(AES256KEYBLOB), NULL, CRYPT_EXPORTABLE, &hKey)) { goto error; }
if (!CryptSetKeyParam(hKey, KP_IV, szIV, 0)) { goto error; }
// ciphertext allocation
dwBufSize = BUFFER_FOR_PLAINTEXT + dwPlainSize;
ciphertext = (char*)calloc(dwBufSize, sizeof(char));
memcpy_s(ciphertext, dwBufSize, *plaintext, dwPlainSize);
// encryption
if (!CryptEncrypt(hKey, NULL, TRUE, 0, (BYTE*)ciphertext, &dwPlainSize, dwBufSize) ) { goto error; }
The key and IV are given to OpenSSL as hexadecimal. There's no need to generate visible ASCII characters only. You could have figured that out yourself just by pretending that the generated key and IV was "Hello". You get an error message that's pretty helpful.
$> openssl aes-256-cbc -d -K Hello -iv Hello
hex string is too short, padding with zero bytes to length
non-hex digit
invalid hex iv value
And hey, look at your code,
// initalize key and plaintext
StrCpyA((LPSTR)szKey, "00112233445566778899001122334455");
StrCpyA((LPSTR)szIV, "4455667788990011");
These string are not only ASCII text, they happen to be hex as well. It looks to as if you just need to convert hex to bytes in your code or convert bytes to a hex string if you have the bytes in code and want to generate the command line.
Background
I am trying to verify signature of a given binary file using openssl. Actual signing of binary hash is done by a 3rd party. Both 3rd party and I have the same exact certificate - they sent me the certificate.
I have verified health of my certificate by running openssl x509 -noout -text -inform DER -in CERT_PATH. This displays contents of cert correctly.
Following is my code so far - I based it on openssl wiki example here:
static std::vector<char> ReadAllBytes(char const* filename){
std::ifstream ifs(filename, std::ios::binary|std::ios::ate);
std::ifstream::pos_type pos = ifs.tellg();
std::vector<char> result(pos);
ifs.seekg(0, std::ios::beg);
ifs.read(result.data(), pos);
return result;
}
int main(int ac, const char * av[]) {
OpenSSL_add_all_algorithms();
ERR_load_crypto_strings();
// most of error check omitted for brevity
auto foundBinBytes = ReadAllBytes("BINARY_PATH");
auto foundSgnBytes = ReadAllBytes("SIGNATURE_PATH");
auto foundCertBytes = ReadAllBytes("CERT_PATH");
ERR_clear_error();
BIO *b = NULL;
X509 *c;
b = BIO_new_mem_buf(reinterpret_cast<const unsigned char *>(foundCertBytes.data()), foundCertBytes.size());
c = d2i_X509_bio(b, NULL);
EVP_MD_CTX* ctx = NULL;
ctx = EVP_MD_CTX_create();
const EVP_MD* md = EVP_get_digestbyname("SHA256");
int rc = EVP_DigestInit_ex(ctx, md, NULL);
EVP_PKEY *k = NULL;
k = X509_get_pubkey(c);
rc = EVP_DigestVerifyInit(ctx, NULL, md, NULL, k);
rc = EVP_DigestVerifyUpdate(ctx, reinterpret_cast<const unsigned char *>(foundBinBytes.data()), foundBinBytes.size());
ERR_clear_error();
rc = EVP_DigestVerifyFinal(ctx, reinterpret_cast<const unsigned char *>(foundSgnBytes.data()), foundSgnBytes.size());
ERR_print_errors_fp( stdout );
// openssl free functions omitted
if(ctx) {
EVP_MD_CTX_destroy(ctx);
ctx = NULL;
}
return 0;
}
Issue
Running this code produces following errors:
4511950444:error:0D07207B:asn1 encoding routines:ASN1_get_object:header too long:/.../crypto/asn1/asn1_lib.c:152:
4511950444:error:0D068066:asn1 encoding routines:ASN1_CHECK_TLEN:bad object header:/.../crypto/asn1/tasn_dec.c:1152:
4511950444:error:0D07803A:asn1 encoding routines:ASN1_ITEM_EX_D2I:nested asn1 error:/.../crypto/asn1/tasn_dec.c:314:Type=X509_SIG
Question
What is wrong with my setup/code? Did I miss something along the way?
You never check the errors when reading the files. You might have errors there (does the file "CERT_PATH" exist? Do you have read permissions? ...).
If "CERT_PATH" cannot be read, then foundCertBytes.data() is an empty byte array, and this explains the subsequent errors.
If you get these errors in a mosquitto secured server's log, check the config file.
My mosquitto.conf was containing :
require_certificate true
my meross device doesn't send certificate. Turning this to "false" solved my problem after restart
I'm using this decryption function to get the plain text value of a cipher which was encrypted using EVP AES 265 GCM; I can see data in rawOut but ret = EVP_DecryptFinal_ex(ctx, rawOut, &len); returns 0; can you provide any insight as to why? I've also seen sources which do rawOut + len in the EVP_DecryptFinal_ex code, I'm not sure why this would be needed as it would move the pointer to the end of the buffer.
unsigned char* keyDecrypter(unsigned char* pszMasterKey)
{
ERR_load_crypto_strings();
int ret, len;
EVP_CIPHER_CTX* ctx;
unsigned char* rawOut = new unsigned char[48]; // ToDo Remove Hardcoded Value
Info info = m_header.processKeyInfo();
if (NULL == info.nonce)
return NULL;
if (!(ctx = EVP_CIPHER_CTX_new()))
return NULL;
if (!EVP_DecryptInit_ex(ctx, EVP_aes_256_gcm(), NULL, pszMasterKey, info.nonce))
return NULL;
if (!EVP_DecryptUpdate(ctx, NULL, &len, m_header.aad, m_header.aad_len))
return NULL;
if (!EVP_DecryptUpdate(ctx, rawOut, &len, m_header.encryptedValue, m_header.encryptedValueLen))
return NULL;
// Finalise the decryption. A positive return value indicates success,
// anything else is a failure - the plain text is not trustworthy.
ret = EVP_DecryptFinal_ex(ctx, rawOut, &len);
ERR_print_errors_fp(stderr);
EVP_CIPHER_CTX_free(ctx);
if (ret > 0)
{
return rawOut;
}
else
{
return NULL;
}
}
From the OpenSSL doc:
"EVP_DecryptFinal() will return an error code if padding is enabled and the final block is not correctly formatted."
Apparently, the padding scheme between encryption and decryption do not match, or perhaps the size of ciphertext fed into the decryption engine did not exactly match the size of the ciphertext that was output from the encryption engine. Note that the ciphertext must include the result of a corresponding call to EVP_EncryptFinal_ex.
It is unfortunate that the original poster did not provide sufficient information to make an exact determination.
You need to pass rawOut + len to EVP_DecryptFinal_ex. See in the example at the end of the documentation:
/* Buffer passed to EVP_EncryptFinal() must be after data just
* encrypted to avoid overwriting it.
*/
if(!EVP_EncryptFinal_ex(ctx, outbuf + outlen, &tmplen))
{
/* Error */
return 0;
}
outlen += tmplen;
Also note that rawOut must have enough room for (m_header.aad_len + cipher_block_size) bytes. You can get the block size with EVP_CIPHER_block_size().
I'm trying to use the OpenSSL library for AES encryption. Everything compiles and seems to work fine. However, when I use BIO_dump_fp(stdout, (char*)ciphertext, ciphertext_len) valgrind ends up reporting hundreds of errors, mostly "conditional jump of move depends on uninitialized value(s)" errors, like this one:
Conditional jump or move depends on uninitialised value(s)
at 0x579A9C3: fwrite (iofwrite.c:49)
by 0x4F187B0: ??? (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
by 0x4F18AC4: BIO_dump_indent_cb (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
by 0x401748: main (in /home/van/Desktop/aes-test/temp/test)
Can these errors be safely ignored (i.e. are these false positives)? If it matters, I'm using Ubuntu 14.04, g++ version 4.8.2, valgrind 3.10.
UPDATE: my full source code is as follows:
#include <stdio.h>
#include "QAesHelper.h"
int main(int argc, char *argv[])
{
unsigned char iv[] = "1234567812345678";
unsigned char key[] = "Testing Testing...";
printf("Size of key: %d\n", (int)sizeof(key));
unsigned char plaintext[] = "The quick brown fox jumps over the lazy dog";
int plaintext_len = sizeof(plaintext);
printf("Size of plaintext: %d\n", plaintext_len);
unsigned char *ciphertext = (unsigned char*)malloc(plaintext_len + 32);
unsigned char *decryptedtext = (unsigned char*)malloc(plaintext_len + 2);
QAesHelper *aesHelper = new QAesHelper(key, sizeof(key));
int ciphertext_len = aesHelper->encrypt(plaintext, plaintext_len, iv, sizeof(iv), &ciphertext);
int decryptedtext_len = aesHelper->decrypt(ciphertext, ciphertext_len + 1, iv, sizeof(iv), &decryptedtext);
// If I remove the following line (BIO_dump_fp...), then
// valgrind reports no errors. With this line left in, there
// are over 900 errors reported.
BIO_dump_fp(stdout, (char*)ciphertext, ciphertext_len);
delete aesHelper;
free(ciphertext);
free(decryptedtext);
return 0;
}
And QAesHelper::encrypt() is:
int QAesHelper::encrypt(unsigned char *plaintext, int plaintext_len, unsigned char *iv, int iv_len, unsigned char **ciphertext)
{
EVP_CIPHER_CTX *ctx;
int len;
int ciphertext_len;
if(!(ctx = EVP_CIPHER_CTX_new())) handleErrors();
if(1 != EVP_EncryptInit_ex(ctx, EVP_aes_256_cbc(), NULL, key, iv))
handleErrors();
if(1 != EVP_EncryptUpdate(ctx, *ciphertext, &len, plaintext, plaintext_len))
handleErrors();
ciphertext_len = len;
if(1 != EVP_EncryptFinal_ex(ctx, *ciphertext + len, &len)) handleErrors();
ciphertext_len += len;
EVP_CIPHER_CTX_free(ctx);
return ciphertext_len;
}
Its not an error. OpenSSL heavily uses uninitialized memory segments.
Valgrind treats such usage as error and warns about it. Its normal behaviour which can be beautified in some extent:
write and use valgrind suppression file valgrind --gen-suppressions=no|yes|all
compile openssl with PURIFY macro enabled in cflags
push to valgrind --error-limit=no and ignore warnings from libssl/libcrypto
I am working on a task to encrypt large files with AES CCM mode (256-bit key length). Other parameters for encryption are:
tag size: 8 bytes
iv size: 12 bytes
Since we already use OpenSSL 1.0.1c I wanted to use it for this task as well.
The size of the files is not known in advance and they can be very large. That's why I wanted to read them by blocks and encrypt each blocks individually with EVP_EncryptUpdate up to the file size.
Unfortunately the encryption works for me only if the whole file is encrypted at once. I get errors from EVP_EncryptUpdate or strange crashes if I attempt to call it multiple times. I tested the encryption on Windows 7 and Ubuntu Linux with gcc 4.7.2.
I was not able to find and information on OpenSSL site that encrypting the data block by block is not possible (or possible).
Additional references:
http://www.fredriks.se/?p=23
http://incog-izick.blogspot.in/2011/08/using-openssl-aes-gcm.html
Please see the code below that demonstrates what I attempted to achieve. Unfortunately it is failing where indicated in the for loop.
#include <QByteArray>
#include <openssl/evp.h>
// Key in HEX representation
static const char keyHex[] = "d896d105b05aaec8305d5442166d5232e672f8d5c6dfef6f5bf67f056c4cf420";
static const char ivHex[] = "71d90ebb12037f90062d4fdb";
// Test patterns
static const char orig1[] = "Very secret message.";
const int c_tagBytes = 8;
const int c_keyBytes = 256 / 8;
const int c_ivBytes = 12;
bool Encrypt()
{
EVP_CIPHER_CTX *ctx;
ctx = EVP_CIPHER_CTX_new();
EVP_CIPHER_CTX_init(ctx);
QByteArray keyArr = QByteArray::fromHex(keyHex);
QByteArray ivArr = QByteArray::fromHex(ivHex);
auto key = reinterpret_cast<const unsigned char*>(keyArr.constData());
auto iv = reinterpret_cast<const unsigned char*>(ivArr.constData());
// Initialize the context with the alg only
bool success = EVP_EncryptInit(ctx, EVP_aes_256_ccm(), nullptr, nullptr);
if (!success) {
printf("EVP_EncryptInit failed.\n");
return success;
}
success = EVP_CIPHER_CTX_ctrl(ctx, EVP_CTRL_CCM_SET_IVLEN, c_ivBytes, nullptr);
if (!success) {
printf("EVP_CIPHER_CTX_ctrl(EVP_CTRL_CCM_SET_IVLEN) failed.\n");
return success;
}
success = EVP_CIPHER_CTX_ctrl(ctx, EVP_CTRL_CCM_SET_TAG, c_tagBytes, nullptr);
if (!success) {
printf("EVP_CIPHER_CTX_ctrl(EVP_CTRL_CCM_SET_TAG) failed.\n");
return success;
}
success = EVP_EncryptInit(ctx, nullptr, key, iv);
if (!success) {
printf("EVP_EncryptInit failed.\n");
return success;
}
const int bsize = 16;
const int loops = 5;
const int finsize = sizeof(orig1)-1; // Don't encrypt '\0'
// Tell the alg we will encrypt size bytes
// http://www.fredriks.se/?p=23
int outl = 0;
success = EVP_EncryptUpdate(ctx, nullptr, &outl, nullptr, loops*bsize + finsize);
if (!success) {
printf("EVP_EncryptUpdate for size failed.\n");
return success;
}
printf("Set input size. outl: %d\n", outl);
// Additional authentication data (AAD) is not used, but 0 must still be
// passed to the function call:
// http://incog-izick.blogspot.in/2011/08/using-openssl-aes-gcm.html
static const unsigned char aadDummy[] = "dummyaad";
success = EVP_EncryptUpdate(ctx, nullptr, &outl, aadDummy, 0);
if (!success) {
printf("EVP_EncryptUpdate for AAD failed.\n");
return success;
}
printf("Set dummy AAD. outl: %d\n", outl);
const unsigned char *in = reinterpret_cast<const unsigned char*>(orig1);
unsigned char out[1000];
int len;
// Simulate multiple input data blocks (for example reading from file)
for (int i = 0; i < loops; ++i) {
// ** This function fails ***
if (!EVP_EncryptUpdate(ctx, out+outl, &len, in, bsize)) {
printf("DHAesDevice: EVP_EncryptUpdate failed.\n");
return false;
}
outl += len;
}
if (!EVP_EncryptUpdate(ctx, out+outl, &len, in, finsize)) {
printf("DHAesDevice: EVP_EncryptUpdate failed.\n");
return false;
}
outl += len;
int finlen;
// Finish with encryption
if (!EVP_EncryptFinal(ctx, out + outl, &finlen)) {
printf("DHAesDevice: EVP_EncryptFinal failed.\n");
return false;
}
outl += finlen;
// Append the tag to the end of the encrypted output
if (!EVP_CIPHER_CTX_ctrl(ctx, EVP_CTRL_CCM_GET_TAG, c_tagBytes, out + outl)) {
printf("DHAesDevice: EVP_CIPHER_CTX_ctrl failed.\n");
return false;
};
outl += c_tagBytes;
out[outl] = '\0';
EVP_CIPHER_CTX_cleanup(ctx);
EVP_CIPHER_CTX_free(ctx);
QByteArray enc(reinterpret_cast<const char*>(out));
printf("Plain text size: %d\n", loops*bsize + finsize);
printf("Encrypted data size: %d\n", outl);
printf("Encrypted data: %s\n", enc.toBase64().data());
return true;
}
EDIT (Wrong Solution)
The feedback that I received made me think in a different direction and I discovered that EVP_EncryptUpdate for size must be called for each block that it being encrypted, not for the total size of the file. I moved it just before the block is encrypted: like this:
for (int i = 0; i < loops; ++i) {
int buflen;
(void)EVP_EncryptUpdate(m_ctx, nullptr, &buflen, nullptr, bsize);
// Resize the output buffer to buflen here
// ...
// Encrypt into target buffer
(void)EVP_EncryptUpdate(m_ctx, out, &len, in, buflen);
outl += len;
}
AES CCM encryption block by block works this way, but not correctly, because each block is treated as independent message.
EDIT 2
OpenSSL's implementation works properly only if the complete message is encrypted at once.
http://marc.info/?t=136256200100001&r=1&w=1
I decided to use Crypto++ instead.
For AEAD-CCM mode you cannot encrypt data after associated data was feed to the context.
Encrypt all the data, and only after it pass the associated data.
I found some mis-conceptions here
first of all
EVP_EncryptUpdate(ctx, nullptr, &outl
calling this way is to know how much output buffer is needed so you can allocate buffer and second time give the second argument as valid big enough buffer to hold the data.
You are also passing wrong (over written by previous call) values when you actually add the encrypted output.