I am extracting some data from a UNIVERSE system and want to encrypt it for transfer via email.
I am no UNIVERSE expert so am using bits and pieces we have found from around the internet and it "looks" like it is working BUT I just can't seem to decrypt the data.
Below is the script I have used based on code found on the web:
RESULT=''
ALGORITHM="rc2-cbc" ; * 128 bit rc2 algorithm in CBC mode
MYKEY="23232323" ; * HEX - Actual Key
IV= "12121212" ; * HEX - Initialization Vector
DATALOC=1 ; * Data in String
KEYLOC=1 ; * Key in String
ACTION=5 ; * Base64 encode after encryption
KEYACTION=1 ; * KEY_ACTUAL_OPENSSL
SALT='' ; * SALT not used
RESULTLOC=1 ; * Result in String RESULT
OPSTRING = ''
RETURN.CODE=ENCRYPT(ALGORITHM,ACTION,DATASTRING,DATALOC,MYKEY,KEYLOC,KEYACTION,SALT,IV,OPSTRING,RESULTLOC)
RETURN.CODE = OPSTRING
Below are a few data strings I have processed through this script and the resulting string:
INPUT 05KI
OUTPUT iaYoHzxYlmM=
INPUT 05FOAA
OUTPUT e0XB/jyE9ZM=
When I try to decode and decrypt the resulting OUTPUT with an online decrypter, I still get no results: https://www.tools4noobs.com/online_tools/decrypt/
I'm thinking it might be a character encoding issue or perhaps the encryption is not working but I have no idea how to resolve - we have been working on this for a few weeks and cannot get any data that is decryptable...
All setups and fields have been set based on this: https://www.dropbox.com/s/ban1zntdy0q27z3/Encrypt%20Function.pdf?dl=0
If I feed the base-64 encrypted string from your code back into the Unidata DECRYPYT function with the same parameters it decrypts just fine.
I suspect something funny is happening with the key. This page mentions something like that: https://u2devzone.rocketsoftware.com/accelerate/articles/data-encryption/data-encryption.html "Generating a suitable key is one of the thornier problems associated with encryption. Keys should be generated as random binary strings, making them obviously difficult to remember. Accordingly, it is probably more common for applications to supply a pass phrase to the ENCRYPT function and have the function internally generate the actual encryption key."
One option to remove the Universe ENCRYPT function from the picture is to use openSSL directly. It looks like the ENCRYPT/DECRYPT functions are just thin wrappers around the openSSL library, so you can execute that to get the result. I'm having problems with the php page you're using for verification, but if I feed the base-64 encrypted string to an openSSL decrypt command on a different machine, it decrypts fine.
MYKEY="A long secret key"
DATASTRING="data to be encrypted data here"
EXECUTE '!echo "':DATASTRING:'"| openssl enc -base64 -e -rc2-cbc -nosalt -k "':MYKEY:'"' CAPTURING RESULT
Related
I am implementing Azure DPS (device provisioning service) for my ESP32-based firmware.
The bash script I use so far is as follows (where KEY is the primary key of the DPS enrolment group and REG_ID is the registration device Id for the given ESP it runs on):
#!/bin/sh
KEY=KKKKKKKKK
REG_ID=RRRRRRRRRRR
keybytes=$(echo $KEY | base64 --decode | xxd -p -u -c 1000)
echo -n $REG_ID | openssl sha256 -mac HMAC -macopt hexkey:$keybytes -binary | base64
I use the Arduino platform in platformIO.
How to translate the script in C/C++?
[UPDATE] The reason why I can't run openSSL: I need to generate the symmetric key from the actual device MAC address in order to obtain the credential from DPS and then be granted to connect to IoT Hub - I run on an EPS32-based custom PCB. No shell. No OS.
I manage to do it by using bed library (which is available from both ESP32/Arduino platforms).
Here is my implementation for the Arduino platform:
#include <mbedtls/md.h> // mbed tls lib used to sign SHA-256
#include <base64.hpp> // Densaugeo Base64 version 1.2.0 or 1.2.1
/// Returns the SHA-256 signature of [dataToSign] with the key [enrollmentPrimaryKey]
/// params[in]: dataToSign The data to sign (for our purpose, it is the registration ID (or the device ID if it is different)
/// params[in]: enrollmentPrimaryKey The group enrollment primary key.
/// returns The SHA-256 base-64 signature to present to DPS.
/// Note: I use mbed to SHA-256 sign.
String Sha256Sign(String dataToSign, String enrollmentPrimaryKey){
/// Length of the dataToSign string
const unsigned dataToSignLength = dataToSign.length();
/// Buffer to hold the dataToSign as a char[] buffer from String.
char dataToSignChar[dataToSignLength + 1];
/// String to c-style string (char[])
dataToSign.toCharArray(dataToSignChar, dataToSignLength + 1);
/// The binary decoded key (from the base 64 definition)
unsigned char decodedPSK[32];
/// Encrypted binary signature
unsigned char encryptedSignature[32];
/// Base 64 encoded signature
unsigned char encodedSignature[100];
Serial.printf("Sha256Sign(): Registration Id to sign is: (%d bytes) %s\n", dataToSignLength, dataToSignChar);
Serial.printf("Sha256Sign(): DPS group enrollment primary key is: (%d bytes) %s\n", enrollmentPrimaryKey.length(), enrollmentPrimaryKey.c_str());
// Need to base64 decode the Preshared key and the length
const unsigned base64DecodedDeviceLength = decode_base64((unsigned char*)enrollmentPrimaryKey.c_str(), decodedPSK);
Serial.printf("Sha256Sign(): Decoded primary key is: (%d bytes) ", base64DecodedDeviceLength);
for(int i= 0; i<base64DecodedDeviceLength; i++) {
Serial.printf("%02x ", (int)decodedPSK[i]);
}
Serial.println();
// Use mbed to sign
mbedtls_md_type_t mdType = MBEDTLS_MD_SHA256;
mbedtls_md_context_t hmacKeyContext;
mbedtls_md_init(&hmacKeyContext);
mbedtls_md_setup(&hmacKeyContext, mbedtls_md_info_from_type(mdType), 1);
mbedtls_md_hmac_starts(&hmacKeyContext, (const unsigned char *) decodedPSK, base64DecodedDeviceLength);
mbedtls_md_hmac_update(&hmacKeyContext, (const unsigned char *) dataToSignChar, dataToSignLength);
mbedtls_md_hmac_finish(&hmacKeyContext, encryptedSignature);
mbedtls_md_free(&hmacKeyContext);
Serial.print("Sha256Sign(): Computed hash is: ");
for(int i= 0; i<sizeof(encryptedSignature); i++) {
Serial.printf("%02x ", (int)encryptedSignature[i]);
}
Serial.println();
// base64 decode the HMAC to a char
encode_base64(encryptedSignature, sizeof(encryptedSignature), encodedSignature);
Serial.printf("Sha256Sign(): Computed hash as base64: %s\n", encodedSignature);
// creating the real SAS Token
return String((char*)encodedSignature);
}
You have a very interesting question from mathematical/algorithmical point of view. So just for fun decided to implement ALL sub-algorithms of it from scratch, without almost NO dependacy on standard C++ library.
All algorithms of me are based on Wikipedia and described well in its articles SHA-256, HMAC, Base64 (and StackOverflow), Hex.
I made whole my code specifically from scratch and without almost NO dependency on std C++ library. Only two headers used right now <cstdint> for implementing all sized integers u8, u16, u32, i32, u64, i64.
And <string> is used only to implement Heap allocations. Also you can easily implement this heap allocations inside my HeapMem class, or by removing using String = std::string; (and #include <string>) on first lines of my code and using built-in heap-allocated String of Arduino if it has built-in one.
Header <iostream> is used only in few last lines of code snippet, only to output result to console, so that StackOverflow visitors my run program without external dependencies. This console output may be removed of course.
Besides main algorithms I had to implement my own classes Array, Vector, Str, Tuple, HeapMem to re-implement basic concepts of standard C++ library. Also standard library function like MemSet(), MemCpy(), MemCmp(), StrLen(), Move() had to be implemented.
You may notice too that I never used exceptions in code, specifically if you have disabled/non-supporting them. Instead of exceptions I implemented special Result<T> template that resembles Result from Rust language. This template is used to return/check correct and error results from whole stack of functions.
All algorithms (Sha256, Hmac, Base64) are tested by simple test cases with reference vectors taken from internet. Final SignSha256() function that you desired is also tested by several test cases against your reference bash OpenSSL script.
Important!. Don't use this code snippet directly inside production code, because it is not very well tested and might contain some errors. Use it Only for educational purposes or test it thourughly before using.
Code snippet is very large, around 32 KiB, bigger that limit of StackOverflow post size (which is 30 000 symbols), so I'm sharing code snippet through two external services - GodBolt (click Try it online! link), where you can also test it online, and GitHub Gist service for download/view only.
SOURCE CODE HERE
Try it online on GodBolt!
GitHub Gist
I am developing rest api system running on ec2 service.
My requirement is
encrypt/decrypt on plain text with AES256
generate key with pbkdf2
Store key on aws cloud system
Java
I tried to used KMS & Crypto sdk but it’s not working. (ex) result of encryption value is changed every time When I called method with same plaint text.
Do you have any ideas?
public String encrypt(String text) {
String plaintext = text;
try {
ByteBuffer byteBuffer = getByteBuffer(plaintext);
EncryptRequest encryptRequest = new EncryptRequest().withKeyId(key_arn).withPlaintext(byteBuffer);
EncryptResult encryptResult = client.encrypt(encryptRequest);
String ciphertext = getString(java.util.Base64.getEncoder().encode(encryptResult.getCiphertextBlob()));
plaintext = ciphertext;
} catch (Exception e) {
e.printStackTrace();
}
return plaintext;
}
public String decrypt(String text) {
String bb = null;
try {
byte[] ciphertextBytes = text.getBytes();
DecryptRequest request = new DecryptRequest();
request.setCiphertextBlob(ByteBuffer.wrap(ciphertextBytes));
DecryptResult result = client.decrypt(request);
// Convert to byte array
byte[] plaintext = new byte[result.getPlaintext().remaining()];
ByteBuffer a = result.getPlaintext().get(plaintext);
bb = getString(a);
} catch (Exception e) {
e.printStackTrace();
}
return bb;
}
This is a hard question to answer succinctly because there are a lot of details and background knowledge that go into crypto and using the AWS crypto capabilities correctly.
First to your specific question - different values from the same plaintext is ok and expected. Encrypt returns different values for the same plain text because it attaches additional data to the plain text, such as an Initialization Vector (IV). This is a way to include non-deterministic data in the plain text precisely so that you don't end up with the exact same cipher text from the same plain text when using the same key.
More importantly, though, note that Encrypt and Decrypt are not general-purpose tools - they are designed to handle small payloads (< 8KB), specifically Data Keys. So where you go from here will depend on what kind of data you are encrypting. If you just need to decrypt a small value like a password, you can continue with Encrypt/Decrypt, and don't worry that two Encrypt operations produce different cipher texts. If you need to encrypt files or other arbitrary chunks of data, read on.
AWS promotes the idea of Envelope Encryption, which is the notion that the key used to actually en/decrypt is stored alongside data it protects, and is itself en/decrypted via a separate master key. In the AWS case, that's Customer Master Key (CMK), which never leaves KMS.
So you can use Encrypt to encrypt an encryption key that you generate, or you could use the AWS method GenerateDataKey to 1) create a key and 2) encrypt it for you. That will return both a plaintext (base64) and cipher text version of the key. Use the plaintext in memory, store the cipher text to disk.
A more typical workflow would be something like:
Call GenerateDataKey - you need to specify the KeyId and a KeySpec (AES_128 or AES_256). There are also options to generate asymmetric keys.
The result includes both a plain text and encrypted version of the generated key. Store the encrypted version.
At some later time, call Decrypt when you need to use the key.
Use the plaintext from that Decrypt method as the key in your local crypto code.
AWS actually provides a separate library to do all that for you - the AWS Encryption SDK, with support for a range of languages including Java. I've not used it extensively, but it provides the framework to do envelope encryption via best practices (which algorithms, master vs. data keys, etc). Take a look (link below).
Hope this helps; encryption is tough to get right.
For more info:
https://docs.aws.amazon.com/kms/latest/developerguide/concepts.html#enveloping
https://docs.aws.amazon.com/encryption-sdk/latest/developer-guide/introduction.html
https://en.wikipedia.org/wiki/Initialization_vector
I need to create a internal AES/Base64 encrypt system for my application.
I mean , I don't want to have any side dll for my exe like openssl library.
I found this threads :
Problems with CryptoPP C++ AES-256+Base64
Crypto++ encrypt and decrypt in two different c++ programs
But they wasn't what I need , I searched and I found nothing.
What I need exactly is like this site : https://aesencryption.net/
First an AES encryption with a key at 256bit mode
Next convert it to Base64 ( a Result of encryption in base64 in aesencryption.net )
And a decrypt system to get the first string from encrypted result.
In this thread the key has to be 32 lenght but in the aesencryption.net it can be everything .
Can you please guide me how can I create a system like aesencryption.net site ?
I'm trying to do this homework https://www.root-me.org/en/Challenges/Cryptanalysis/File-PKZIP When I write a function to crack it.
import subprocess from time import sleep
file = open('/home/begood/Downloads/SecLists-master/Passwords/'
'rockyou-75.txt', 'r') lines = file.readlines() file.close() for line in lines:
command = 'unzip -P ' + line.strip() + ' /home/begood/Downloads/ch5.zip'
print command
p = subprocess.Popen(
command,
stdout=subprocess.PIPE, shell=True).communicate()[0]
if 'replace' in p:
print 'y\n'
sleep(1)
It stop in password = scooter:
unzip -P scooter /home/begood/Downloads/ch5.zip replace readme.txt? [y]es, [n]o, [A]ll, [N]one, [r]ename:
but when I use it to unzip it said:
inflating: /home/begood/readme.txt
error: invalid compressed data to inflate
And it real password is : 14535. Why pkzip accept two password?
I presume that the encryption being used is the old, very weak, encryption that was part of the original PKZIP format.
That encryption method has a 12-byte salt header before the compressed data. From the PKWare specification:
After the header is decrypted, the last 1 or 2 bytes in Buffer
should be the high-order word/byte of the CRC for the file being
decrypted, stored in Intel low-byte/high-byte order. Versions of
PKZIP prior to 2.0 used a 2 byte CRC check; a 1 byte CRC check is
used on versions after 2.0. This can be used to test if the password
supplied is correct or not.
It was originally two bytes in the 1.0 specification, but in the 2.0 specification, and in the associated version of PKZIP, the check value was changed to one byte in order to make password searches like what you are doing more difficult. The result is that about one out of every 256 random passwords will result in passing that first check, and then proceeding to try to decompress the incorrectly decrypted compressed data, only then running into an error.
So it's far, far more than two passwords that will be "accepted". However it won't take very many bytes of decompressed data to detect that the password was nevertheless incorrect.
I've generated a random 256 bit symmetric key, in a file, to use for encrypting some data using the OpenSSL command line which I need to decrypt later programmatically using the OpenSSL library. I'm not having success, and I think the problem might be in the initialization vector I'm using (or not using).
I encrypt the data using this command:
/usr/bin/openssl enc -aes-256-cbc -salt -in input_filename -out output_filename -pass file:keyfile
I'm using the following call to initialize the decrypting of the data:
EVP_DecryptInit_ex(ctx, EVP_aes_256_cbc(), nullptr, keyfile.data(), nullptr))
keyfile is a vector<unsigned char> that holds the 32 bytes of the key. My question is regarding that last parameter. It's supposed to be an initialization vector to the cipher algorithm. I didn't specify an IV when encrypting, so some default must have been used.
Does passing nullptr for that parameter mean "use the default"? Is the default null, and nothing is added to the first cipher block?
I should mention that I'm able to decrypt from the command line without supplying an IV.
What is the default IV when encrypting with EVP_aes_256_cbc() [sic] cipher...
Does passing nullptr for that parameter mean "use the default"? Is the default null, and nothing is added to the first cipher block?
There is none. You have to supply it. For completeness, the IV should be non-predictable.
Non-Predictable is slightly different than both Unique and Random. For example, SSLv3 used to use the last block of ciphertext for the next block's IV. It was Unique, but it was neither Random nor Non-Predictable, and it made SSLv3 vulnerable to chosen plaintext attacks.
Other libraries do clever things like provide a null vector (a string of 0's). Their attackers thank them for it. Also see Why is using a Non-Random IV with CBC Mode a vulnerability? on Stack Overflow and Is AES in CBC mode secure if a known and/or fixed IV is used? on Crypto.SE.
/usr/bin/openssl enc -aes-256-cbc...
I should mention that I'm able to decrypt from the command line without supplying an IV.
OpenSSL uses an internal mashup/key derivation function which takes the password, and derives a key and iv. Its called EVP_BytesToKey, and you can read about it in the man pages. The man pages also say:
If the total key and IV length is less than the digest length and MD5 is used then the derivation algorithm is compatible with PKCS#5 v1.5 otherwise a non standard extension is used to derive the extra data.
There are plenty of examples of EVP_BytesToKey once you know what to look for. Openssl password to key is one in C. How to decrypt file in Java encrypted with openssl command using AES in one in Java.
EVP_DecryptInit_ex(ctx, EVP_aes_256_cbc(), nullptr, keyfile.data(), nullptr))
I didn't specify an IV when encrypting, so some default must have been used.
Check your return values. A call should have failed somewhere along the path. Maybe not at EVP_DecryptInit_ex, but surely before EVP_DecryptFinal.
If its not failing, then please file a bug report.
EVP_DecryptInit_ex is an interface to the AES decryption primitive. That is just one piece of what you need to decrypt the OpenSSL encryption format. The OpenSSL encryption format is not well documented, but you can work it backwards from the code and some of the docs. The key and IV computation is explained in the EVP_BytesToKey documentation:
The key and IV is derived by concatenating D_1, D_2, etc until enough
data is available for the key and IV. D_i is defined as:
D_i = HASH^count(D_(i-1) || data || salt)
where || denotes concatentaion, D_0 is empty, HASH is the digest
algorithm in use, HASH^1(data) is simply HASH(data), HASH^2(data) is
HASH(HASH(data)) and so on.
The initial bytes are used for the key and the subsequent bytes for the
IV.
"HASH" here is MD5. In practice, this means you compute hashes like this:
Hash0 = ''
Hash1 = MD5(Hash0 + Password + Salt)
Hash2 = MD5(Hash1 + Password + Salt)
Hash3 = MD5(Hash2 + Password + Salt)
...
Then you pull of the bytes you need for the key, and then pull the bytes you need for the IV. For AES-128 that means Hash1 is the key and Hash2 is the IV. For AES-256, the key is Hash1+Hash2 (concatenated, not added) and Hash3 is the IV.
You need to strip off the leading Salted___ header, then use the salt to compute the key and IV. Then you'll have the pieces to feed into EVP_DecryptInit_ex.
Since you're doing this in C++, though, you can probably just dig through the enc code and reuse it (after verifying its license is compatible with your use).
Note that the OpenSSL IV is randomly generated, since it's the output of a hashing process involving a random salt. The security of the first block doesn't depend on the IV being random per se; it just requires that a particular IV+Key pair never be repeated. The OpenSSL process ensures that as long as the random salt is never repeated.
It is possible that using MD5 this way entangles the key and IV in a way that leaks information, but I've never seen an analysis that claims that. If you have to use the OpenSSL format, I wouldn't have any hesitations over its IV generation. The big problems with the OpenSSL format is that it's fast to brute force (4 rounds of MD5 is not enough stretching) and it lacks any authentication.