I am new to C++ so go easy :)
I have the following code snippet, however I am using OpenSSL 3, and the compiler is giving various errors: 'HMAC_CTX_new': Since OpenSSL 3.0. I understand that some of the APIs used below are now deprecated and need updating.
Can someone show me to update the below snippet to work with the updated APIs for OpenSSL 3?
const std::string secretKey = "secret";
std::vector<unsigned char> secretKeyByteArray(secretKey.begin(), secretKey.end());
std::vector<unsigned char> signature(signatureRawData.begin(), signatureRawData.end());
std::vector<unsigned char> signatureBytes(HMAC_MAX_MD_CBLOCK);
unsigned int signatureLength = 0;
HMAC_CTX* hmacCtx = HMAC_CTX_new();
HMAC_Init_ex(hmacCtx, secretKeyByteArray.data(), secretKeyByteArray.size(), EVP_sha256(), nullptr);
HMAC_Update(hmacCtx, signature.data(), signature.size());
HMAC_Final(hmacCtx, signatureBytes.data(), &signatureLength);
HMAC_CTX_free(hmacCtx);
std::ostringstream signatureHex;
for (const auto& c : signatureBytes)
signatureHex << std::hex << std::setw(2) << std::setfill('0') << static_cast<int>(c);
std::string requestSignatureHexString = signatureHex.str();
Related
I have been using mostly C so I am pretty new into c++. I want to convert a int vector (std::vector) into hexadecimal representation and then store that into a string. I found something I would use in C in the following thread: Converting hex into a string using 'sprintf'
The code proposed by user411313 is the following:
static unsigned char digest[16];
static unsigned char hex_tmp[16];
for (i = 0; i < 16; i++) {
printf("%02x",digest[i]);
sprintf(&hex_tmp[i], "%02x", digest[i]);
}
One of my concerns is that this could go out of index at some point since sprintf may try to add a 0 after the content. Also, I was wondering if there is any way to do it with native C++, perhaps any built function could be used instead of C functions. Is this preferable in c++ over c functions? Thank you very much for your assistance!
if there is any way to do it with native C++, perhaps any built function could be used instead of C functions. Is this preferable in c++ over c functions?
Sure there is a way, and yes it's preferable:
static std::array<unsigned char,16> digest;
static std::string hex_tmp;
for (auto x : digest) {
std::ostringstream oss;
oss << std::hex << std::setw(2) << std::setfill('0') << (unsigned)x;
hex_tmp += oss.str();
}
One of my concerns is that this could go out of index at some point since sprintf may try to add a 0 after the content.
That's a valid concern. The classes used in my code snippet above will overcome all these issues, and you don't need to care about.
You can use std::stringstream
std::string hex_representation(const std::vector<int>& v) {
std::stringstream stream;
for (const auto num : v) {
stream << "0x" << std::hex << std::setw(2) << std::setfill('0') << num
<< ' ';
}
return stream.str();
}
Obviously you can remove the "0x" prefix if you don't need it
Here a live demo.
I want to keep a bunch of simple structures (just 3 ints per structure at the moment) in a file, and be able to read back just one of those structures at any given time.
As a first step, I'm trying to output them to a file, then read them back using boost::serialization. Currently I'm doing this, which crashes:
std::array<Patch, 3> outPatches;
outPatches[0].ZOrigin = 0;
outPatches[0].XOrigin = 0;
outPatches[0].Resolution = 64;
outPatches[1].ZOrigin = 1;
outPatches[1].XOrigin = 5;
outPatches[1].Resolution = 3;
outPatches[2].ZOrigin = 123;
outPatches[2].XOrigin = 546;
outPatches[2].Resolution = 6;
std::ofstream ofs("testing.sss", std::ios::binary);
for (auto const& patch : outPatches)
{
std::cout << "start archive: " << ofs.tellp() << std::endl;
{
boost::archive::binary_oarchive oa(ofs);
std::cout << "start patch: " << ofs.tellp() << std::endl;
oa << patch;
}
}
ofs.close();
std::array<Patch, 3> inPatches;
std::ifstream ifs("testing.sss", std::ios::binary);
for (auto& patch : inPatches)
{
std::cout << "start archive: " << ifs.tellg() << std::endl;
{
boost::archive::binary_iarchive ia(ifs); // <-- crash here on second patch
std::cout << "start patch: " << ifs.tellg() << std::endl;
ia >> patch;
}
}
ifs.close();
for (int i = 0; i != 3; ++i)
std::cout << "check: " << (inPatches[i] == outPatches[i]) << std::endl;
I was planning on using tell to make an index of where each structure is, and seek to skip to that structure on load. Is this a reasonable approach to take? I don't know much about streams beyond the basics.
I've tried putting all the patches in one o/iarchive instead, which works fine for reading everything sequentially. However, seeking on the stream didn't work.
I've found this, which might be what I want, but I have no idea what it's doing or how to use it, or whether it would work with boost::serialization: read part of a file with iostreams
I'd probably be willing to switch to another serialization method if necessary, since I've not got very far with this.
Edit 3: Moved edits 1 and 2 to an answer.
I once had a similar case (with boost / serialization). What I did back then (and it was quite efficient, if I remember) was to map the file into a virtual address, write a streamer that operates on memory buffers instead of files and for each part that I wanted to read assign appropriate offset to the streamer as buffer start / length and initialize the iarchive with the streamer so the serialization library treated it as if each object was in a separate file.
Of course, adding to the file required a re-map. Now that I look back at this, it seems a bit weird, but it was efficient, afair.
Boost serialization
It doesn't seem possible to skip around inside a boost serialization archive. The best I've got so far is to use multiple archives on one stream:
static const int numPatches = 5000;
std::vector<int> indices(numPatches, 0);
std::iota(indices.begin(), indices.end(), 0);
std::vector<Patch> outPatches(numPatches, Patch());
std::for_each(outPatches.begin(), outPatches.end(),
[] (Patch& p)
{
p.ZOrigin = rand();
p.XOrigin = rand();
p.Resolution = rand();
});
std::vector<int64_t> offsets(numPatches, 0);
std::ofstream ofs("testing.sss", std::ios::binary);
for (auto i : indices)
{
offsets[i] = ofs.tellp();
boost::archive::binary_oarchive oa(ofs,
boost::archive::no_header | boost::archive::no_tracking);
oa << outPatches[i];
}
ofs.close();
std::random_shuffle(indices.begin(), indices.end());
std::vector<Patch> inPatches(numPatches, Patch());
std::ifstream ifs("testing.sss", std::ios::binary);
for (auto i : indices)
{
ifs.seekg(offsets[i]);
boost::archive::binary_iarchive ia(ifs,
boost::archive::no_header | boost::archive::no_tracking);
ia >> inPatches[i];
ifs.clear();
}
std::cout << std::all_of(indices.begin(), indices.end(),
[&] (int i) { return inPatches[i] == outPatches[i]; }) << std::endl;
Unfortunately, this is very slow, so I don't think I can use it. Next up is testing protobuf.
google::protobuf
I've got something working with protobuf. It required a bit of fiddling around (apparently I have to use the LimitingInputStream type, and store the size of each object), but it's a lot faster than the boost::serialization version:
static const int numPatches = 500;
std::vector<int> indices(numPatches, 0);
std::iota(indices.begin(), indices.end(), 0);
std::vector<Patch> outPatches(numPatches, Patch());
std::for_each(outPatches.begin(), outPatches.end(),
[] (Patch& p)
{
p.ZOrigin = rand();
p.XOrigin = rand();
p.Resolution = 64;
});
std::vector<int64_t> streamOffset(numPatches, 0);
std::vector<int64_t> streamSize(numPatches, 0);
std::ofstream ofs("testing.sss", std::ios::binary);
PatchBuffer buffer;
for (auto i : indices)
{
buffer.Clear();
WriteToPatchBuffer(buffer, outPatches[i]);
streamOffset[i] = ofs.tellp();
streamSize[i] = buffer.ByteSize();
buffer.SerializeToOstream(&ofs);
}
ofs.close();
std::random_shuffle(indices.begin(), indices.end());
std::vector<Patch> inPatches(numPatches, Patch());
std::ifstream ifs("testing.sss", std::ios::binary);
for (auto i : indices)
{
ifs.seekg(streamOffset[i]);
buffer.Clear();
google::protobuf::io::IstreamInputStream iis(&ifs);
google::protobuf::io::LimitingInputStream lis(&iis, streamSize[i]);
buffer.ParseFromZeroCopyStream(&lis);
ReadFromPatchBuffer(inPatches[i], buffer);
ifs.clear();
}
std::cout << std::all_of(indices.begin(), indices.end(),
[&] (int i) { return inPatches[i] == outPatches[i]; }) << std::endl;
I need to get all the contents from a stream, without actually extracting them (just like stringstream::str()). I've tried basic_stringbuf::str(), but it behaves incorrectly when the stream is empty. To avoid that case, I had a go at basic_stringbuf::in_avail(), but that hasn't worked out very well either.
In the following test case, in_avail() doesn't return the number of available elements on the stream, and str() returns more elements than what is currently there:
#include <iostream>
#include <iterator>
#include <vector>
#include <sstream>
// extracts everything from the stream
std::vector<unsigned char> stream2vector(std::basic_istream<unsigned char>& stream)
{
std::vector<unsigned char> retreivedData;
std::istreambuf_iterator<unsigned char> it(stream);
const std::istreambuf_iterator<unsigned char> endOfStream;
retreivedData.insert(retreivedData.begin(), it, endOfStream);
return retreivedData;
}
int main() {
std::basic_stringbuf<unsigned char> buf;
std::basic_iostream<unsigned char> stream(&buf);
unsigned char array[5] = { 1, 2, 3, 4, 5 };
stream.write(array, 5);
std::cout << "rdbuf()->in_avail(): " << buf.in_avail() << "\n";
std::vector<unsigned char> d1 = stream2vector(stream);
std::cout << "d1.size(): " << d1.size() << "\n";
std::cout << "\n";
// d2 should be empty
std::vector<unsigned char> d2 = stream2vector(stream);
std::cout << "d2.size(): " << d2.size() << "\n";
std::basic_string<unsigned char> s = buf.str();
std::cout << "buf.str().size(): " << buf.str().size() << "\n";
}
Compiling on g++ 4.4, the output is:
rdbuf()->in_avail(): 1 // expected: 5
d1.size(): 5 // as expected
d2.size(): 0 // as expected
buf.str().size(): 5 // expected: 0
What am I doing wrong? What's the best way to do what I'm trying?
Thanks a lot.
in_avail is the number of characters ready to be read from the buffer, not the size of the buffer itself. It's really allowed to return any nonzero value here.
However, I can't answer what the best way is of what you're doing, because I don't know what you're doing. If you already have things as an unsigned char array, then you're going to want to do:
std::vector<unsigned char> data(array, array + sizeof(array)/sizeof(unsigned char));
If you're just trying to read a whole stream into a vector, then I would do exactly what you're doing; I'd just replace you're stream2vector function with this, equivalent, simpler one:
// extracts everything from the stream
std::vector<unsigned char> stream2vector(std::basic_istream<unsigned char>& stream)
{
std::istreambuf_iterator<unsigned char> it(stream);
const std::istreambuf_iterator<unsigned char> endOfStream;
return std::vector<unsigned char>(it, endOfStream);
}
I'm not entirely sure why you're specializing every operation here for unsigned char -- I would just use the default char versions, because unsigned char is allowed to be the same size as a short, which is probably not what you want (but I am not aware of any implementation that does this).
I have read that boost iostreams supposedly supports 64 bit access to large files semi-portable way. Their FAQ mentions 64 bit offset functions, but there is no examples on how to use them. Has anyone used this library for handling large files? A simple example of opening two files, seeking to their middles, and copying one to the other would be very helpful.
Thanks.
Short answer
Just include
#include <boost/iostreams/seek.hpp>
and use the seek function as in
boost::iostreams::seek(device, offset, whence);
where
device is a file, stream, streambuf or any object convertible to seekable;
offset is a 64-bit offset of type stream_offset;
whence is BOOST_IOS::beg, BOOST_IOS::cur or BOOST_IOS::end.
The return value of seek is of type std::streampos, and it can be converted to a stream_offset using the position_to_offset function.
Example
Here is an long, tedious and repetitive example, which shows how to open two files, seek to offstets >4GB, and copying data between them.
WARNING: This code will create very large files (several GB). Try this example on an OS/file system which supports sparse files. Linux is ok; I did not test it on other systems, such as Windows.
/*
* WARNING: This creates very large files (several GB)
* unless your OS/file system supports sparse files.
*/
#include <boost/iostreams/device/file.hpp>
#include <boost/iostreams/positioning.hpp>
#include <cstring>
#include <iostream>
using boost::iostreams::file_sink;
using boost::iostreams::file_source;
using boost::iostreams::position_to_offset;
using boost::iostreams::seek;
using boost::iostreams::stream_offset;
static const stream_offset GB = 1000*1000*1000;
void setup()
{
file_sink out("file1", BOOST_IOS::binary);
const char *greetings[] = {"Hello", "Boost", "World"};
for (int i = 0; i < 3; i++) {
out.write(greetings[i], 5);
seek(out, 7*GB, BOOST_IOS::cur);
}
}
void copy_file1_to_file2()
{
file_source in("file1", BOOST_IOS::binary);
file_sink out("file2", BOOST_IOS::binary);
stream_offset off;
off = position_to_offset(seek(in, -5, BOOST_IOS::end));
std::cout << "in: seek " << off << std::endl;
for (int i = 0; i < 3; i++) {
char buf[6];
std::memset(buf, '\0', sizeof buf);
std::streamsize nr = in.read(buf, 5);
std::streamsize nw = out.write(buf, 5);
std::cout << "read: \"" << buf << "\"(" << nr << "), "
<< "written: (" << nw << ")" << std::endl;
off = position_to_offset(seek(in, -(7*GB + 10), BOOST_IOS::cur));
std::cout << "in: seek " << off << std::endl;
off = position_to_offset(seek(out, 7*GB, BOOST_IOS::cur));
std::cout << "out: seek " << off << std::endl;
}
}
int main()
{
setup();
copy_file1_to_file2();
}
I need to create a GUID in an unmanaged windows C++ project. I'm used to C#, where I'd use Guid.NewGuid(). What's the (unmanaged windows) C++ version?
I think CoCreateGuid is what you're after. Example:
GUID gidReference;
HRESULT hCreateGuid = CoCreateGuid( &gidReference );
UuidCreate() in Win32 API has exactly the same effect. However you need to pass an address of the variable that will receive the generated value:
UUID newId;
UuidCreate( &newId );
I believe Guid.NewGuid() simply maps onto it inside the .NET runtime.
Here's a snippet of code to get the resulting string value of the generated GUID:
// For UUID
#include <Rpc.h>
#pragma comment(lib, "Rpcrt4.lib")
int _tmain(int argc, _TCHAR* argv[])
{
// Create a new uuid
UUID uuid;
RPC_STATUS ret_val = ::UuidCreate(&uuid);
if (ret_val == RPC_S_OK)
{
// convert UUID to LPWSTR
WCHAR* wszUuid = NULL;
::UuidToStringW(&uuid, (RPC_WSTR*)&wszUuid);
if (wszUuid != NULL)
{
//TODO: do something with wszUuid
// free up the allocated string
::RpcStringFreeW((RPC_WSTR*)&wszUuid);
wszUuid = NULL;
}
else
{
//TODO: uh oh, couldn't convert the GUID to string (a result of not enough free memory)
}
}
else
{
//TODO: uh oh, couldn't create the GUID, handle this however you need to
}
return 0;
}
API reference:
UuidCreate
UuidToString
RpcStringFree
The documentation for Guid.NewGuid points out, how it is implemented:
This is a convenient static method that you can call to get a new Guid. The method wraps a call to the Windows CoCreateGuid function.
So the native equivalent to Guid.NewGuid() is CoCreateGuid().
CoCreateGuid calls UuidCreate, to generate a GUID. Both API calls are slightly different, though: While UuidCreate returns a UUID, that is guaranteed to be unique to the computer that created it, CoCreateGuid produces a GUID that is absolutely unique.
If you need to decide, which API to use, here are the relevant sections from the documentation.
UuidCreate:
For security reasons, it is often desirable to keep ethernet addresses on networks from becoming available outside a company or organization. The UuidCreate function generates a UUID that cannot be traced to the ethernet address of the computer on which it was generated. It also cannot be associated with other UUIDs created on the same computer.
CoCreateGuid:
The CoCreateGuid function calls the RPC function UuidCreate, which creates a GUID, a globally unique 128-bit integer. Use CoCreateGuid when you need an absolutely unique number that you will use as a persistent identifier in a distributed environment.
To generate a new guid in windows and get the resulting value as a string.
#include <string>
#include <sstream>
#include <iostream>
#include <windows.h>
#include <iomanip>
int main()
{
GUID guid;
CoCreateGuid(&guid);
std::ostringstream os;
os << std::hex << std::setw(8) << std::setfill('0') << guid.Data1;
os << '-';
os << std::hex << std::setw(4) << std::setfill('0') << guid.Data2;
os << '-';
os << std::hex << std::setw(4) << std::setfill('0') << guid.Data3;
os << '-';
os << std::hex << std::setw(2) << std::setfill('0') << static_cast<short>(guid.Data4[0]);
os << std::hex << std::setw(2) << std::setfill('0') << static_cast<short>(guid.Data4[1]);
os << '-';
os << std::hex << std::setw(2) << std::setfill('0') << static_cast<short>(guid.Data4[2]);
os << std::hex << std::setw(2) << std::setfill('0') << static_cast<short>(guid.Data4[3]);
os << std::hex << std::setw(2) << std::setfill('0') << static_cast<short>(guid.Data4[4]);
os << std::hex << std::setw(2) << std::setfill('0') << static_cast<short>(guid.Data4[5]);
os << std::hex << std::setw(2) << std::setfill('0') << static_cast<short>(guid.Data4[6]);
os << std::hex << std::setw(2) << std::setfill('0') << static_cast<short>(guid.Data4[7]);
std::string s(os.str());
std::cout << s << std::endl;
}
Alternatively, you could use sprintf_s for the string conversion
GUID guid;
CoCreateGuid(&guid);
char guidStr[37];
sprintf_s(
guidStr,
"%08lX-%04hX-%04hX-%02hhX%02hhX-%02hhX%02hhX%02hhX%02hhX%02hhX%02hhX",
guid.Data1, guid.Data2, guid.Data3,
guid.Data4[0], guid.Data4[1], guid.Data4[2], guid.Data4[3],
guid.Data4[4], guid.Data4[5], guid.Data4[6], guid.Data4[7]);
std::string s(guidStr);
To get in guid in std::string on Windows
#include <Windows.h>
#include <array>
std::string getGUID()
{
std::string result{};
GUID guid;
if (S_OK == CoCreateGuid(&guid))
{
std::array<char, 36> buffer{}; //32 characters of guid + 4 '-' in-between
snprintf(buffer.data(), buffer.size(), "{%08x-%04x-%04x-%02x%02x-%02x%02x%02x%02x%02x%02x}",
guid.Data1, guid.Data2, guid.Data3, guid.Data4[0], guid.Data4[1], guid.Data4[2], guid.Data4[3], guid.Data4[4], guid.Data4[5], guid.Data4[6], guid.Data4[7]);
result = std::string(buffer.data());
}
return result;
}
In the new WinRT api you can use winrt::Windows::Foundation::GuidHelper::CreateNewGuid() which returns a struct of winrt::guid type. You can later pass it to winrt::to_hstring to get struct of winrt::hstring type, then pass it to winrt::to_string to get std::string type.