Issues with SHA 512 HMAC message authentication using openssl - c++

Question

I need to authenticate to a websocket endpoint to subscribe to private data, the authentication steps are as follows:
Hash the challenge with the SHA-256 algorithm
Base64-decode your api_secret
Use the result of step 2 to hash the result of step 1 with the HMAC-SHA-512 algorithm
Base64-encode the result of step 3
I am using openssl in my C++ program for all the crypto and I am using some base64 encoding and decoding algorithms I found on stackoverflow, however I am unable to follow the authentication procedure and produce the correct result.
I am confident that the base64 decoder is correct as it produces the correct binary when I decode the secret furthermore the openssl sha256 algorithm is also correct as it produces the correct hash for the challenge, (I used cryptiis online base64 decoder and sha256 to verify this), something must be wrong with the way I am using the openssl HMAC or the base64 encoder for the final step.
#include <openssl/sha.h>
#include <cstdio>
#include <cstring>
void sha256(const char *string, char outputBuffer[65])
{
unsigned char hash[SHA256_DIGEST_LENGTH];
SHA256_CTX sha256;
SHA256_Init(&sha256);
SHA256_Update(&sha256, string, strlen(string));
SHA256_Final(hash, &sha256);
for (int i = 0; i < SHA256_DIGEST_LENGTH; ++i) {
sprintf(outputBuffer + (i * 2), "%02x", hash[i]);
}
outputBuffer[64] = 0;
}
#include <openssl/bio.h>
#include <openssl/evp.h>
#include <cstring>
#include <memory>
#include <string>
#include <vector>
namespace {
struct BIOFreeAll { void operator()(BIO* p) { BIO_free_all(p); } };
}
auto Base64Encode(const std::vector<unsigned char>& binary)
{
std::unique_ptr<BIO,BIOFreeAll> b64(BIO_new(BIO_f_base64()));
BIO_set_flags(b64.get(), BIO_FLAGS_BASE64_NO_NL);
BIO* sink = BIO_new(BIO_s_mem());
BIO_push(b64.get(), sink);
BIO_write(b64.get(), binary.data(), binary.size());
BIO_flush(b64.get());
const unsigned char* encoded;
const unsigned long len = BIO_get_mem_data(sink, &encoded);
return std::basic_string<unsigned char>{encoded, len};
}
// Assumes no newlines or extra characters in encoded string
std::vector<unsigned char> Base64Decode(const char* encoded)
{
std::unique_ptr<BIO,BIOFreeAll> b64(BIO_new(BIO_f_base64()));
BIO_set_flags(b64.get(), BIO_FLAGS_BASE64_NO_NL);
BIO* source = BIO_new_mem_buf(encoded, -1); // read-only source
BIO_push(b64.get(), source);
const int maxlen = strlen(encoded) / 4 * 3 + 1;
std::vector<unsigned char> decoded(maxlen);
const int len = BIO_read(b64.get(), decoded.data(), maxlen);
decoded.resize(len);
return decoded;
}
#include <openssl/hmac.h>
int main(int argc, const char * argv[])
{
const char* challenge = "c100b894-1729-464d-ace1-52dbce11db42";
static char buffer[65];
sha256(challenge, buffer);
printf("%s\n", buffer);
const char* encoded = "7zxMEF5p/Z8l2p2U7Ghv6x14Af+Fx+92tPgUdVQ748FOIrEoT9bgT+bTRfXc5pz8na+hL/QdrCVG7bh9KpT0eMTm";
std::cout << "encoded = " << encoded << std::endl;
const std::vector<unsigned char> decoded = Base64Decode(encoded);
std::cout << "decoded = " << decoded.data() << '\n';
// The data that we're going to hash using HMAC
std::basic_string<unsigned char> data = {decoded.data(), decoded.size()};
unsigned char* digest;
// Using sha512 hash engine here.
// You may use other hash engines. e.g EVP_md5(), EVP_sha224, EVP_sha512, etc
digest = HMAC(EVP_sha512(), data.c_str(), data.size(), reinterpret_cast<unsigned char*>(buffer), strlen(buffer), NULL, NULL);
// Be careful of the length of string with the choosen hash engine. SHA1 produces a 20-byte hash value which rendered as 40 characters.
// Change the length accordingly with your choosen hash engine
char mdString[128];
for(int i = 0; i < 64; ++i)
sprintf(&mdString[i*2], "%02x", (unsigned int)digest[i]);
printf("HMAC digest: %s\n", mdString);
const std::vector<unsigned char> binary{&digest[0], &digest[127] + 1};
const std::basic_string<unsigned char> encoded_result = Base64Encode(binary);
for (unsigned i = 0; i < 64; ++i)
{
std::cout << std::hex << std::setw(2) << (unsigned int)encoded_result[i];
}
std::cout << '\n';
return 0;
}
The code may not compile first time around as I have pulled the snippets from a larger repository, however if all put into one file it should compile (or require minor effort to successfully compile).
When the value of the initial challenge is
"c100b894-1729-464d-ace1-52dbce11db42"
and the api secret is
"7zxMEF5p/Z8l2p2U7Ghv6x14Af+Fx+92tPgUdVQ748FOIrEoT9bgT+bTRfXc5pz8na+hL/QdrCVG7bh9KpT0eMTm"
The line following HMAC digest should be the signed output, I am expecting it to be
"4JEpF3ix66GA2B+ooK128Ift4XQVtc137N9yeg4Kqsn9PI0Kpzbysl9M1IeCEdjg0zl00wkVqcsnG4bm
nlMb3A=="
whereas it is actually
"336e394b567a55634d46376478344b594354767267636d39456f584f51326c376f334f2f3348796f6939647a7a516a456e41786c3551537541453930422f424b".
What is more troublesome is that I am able to replicate the correct result using python and C# quite simply using the library functions, I am quite unsure as to where I am going wrong here.

Answer 1

You appear to be overly fond of hex encoding your data!
First of all, in your sha256 function you correctly hash the data to get the 32 byte digest, but then you hex encode this to get 64 hex characters (plus the null terminator) which you later use as the input to the HMAC. You need to use those original 32 bytes.
Then later, after you calculate the HMAC and base 64 encode the result, you hex encode that before printing it out. There’s no need to do that, base 64 already consists of printable characters.
Take out those two loops where you do the hex encoding (and change sha256 so you return the correct buffer) and it should work correctly.