HEX string to UTF-8(UNICODE) string - c++

Question

I have a HEX string that has Unicode characters in it. and I need to convert that UTF-8(Unicode) and store in a string variable.
I am new in Unicode and I don't have much idea to try anything.
std::string HEX_string= "0635 0628 0627 062d 0020 0627 0644 062e 064a 0631";
std:string unicode_string=getUnicodeString(HEX_string);
I expect صباح الخير value in unicode_string variable.

Answer 1

Since that hex string is a bunch of space-separated base-16 encoded Unicode codepoints, it's easy to convert using just standard functions, in particular std::c32rtomb():
#include <iostream>
#include <string>
#include <sstream>
#include <cstdlib>
#include <clocale>
#include <cuchar>
#include <climits>
std::string
getUnicodeString(const std::string &hex)
{
std::istringstream codepoints{hex};
std::string cp;
std::string out;
std::mbstate_t state;
char u8[MB_LEN_MAX];
while (codepoints >> cp) {
char32_t c = std::stoul(cp, nullptr, 16);
auto len = std::c32rtomb(u8, c, &state);
if (len == std::size_t(-1)) {
std::cerr << "Unable to convert " << cp << " to UTF-8 codepoint!\n";
std::exit(EXIT_FAILURE);
} else if (len > 0) {
out.append(u8, len);
}
}
return out;
}
int main() {
// Make sure that c32rtomb() works with UTF-32 code units
static_assert(__STDC_UTF_32__);
// Requires a UTF-8 locale to get a UTF-8 string.
std::setlocale(LC_ALL, "");
std::string HEX_string = "0635 0628 0627 062d 0020 0627 0644 062e 064a 0631";
std::string unicode_string = getUnicodeString(HEX_string);
std::cout << unicode_string << '\n';
return 0;
}
After compiling it, running it produces:
$ echo $LANG
en_US.utf8
$ ./a.out
صباح الخير
You don't have any codepoints outside the BMP in that sample to be sure if your input is encoded in UTF-16 or UTF-32. The above code assumes UTF-32, but if it's UTF-16 you can change c32rtomb() to c16rtomb() and char32_t to char16_t and it'll handle UTF-16 surrogate pairs correctly.