Compiling Unicode with Visual Studio 2019 - c++

Question

I try to compile this C++17 code on VS2019:
int main() {
if(!testCodepointEncode(U'\u221A', '\xFB') ||
!testCodepointEncode(U'\u0040', '\x40') ||
!testCodepointEncode(U'\u03A3', '\xE4') ||
!testCodepointEncode(U'𠲖', '\xFE')) {
return 1;
}
// Test 1 byte
if(!testEncode("\u0040", "\x40")) {
return 2;
}
// Test 2 byte
if(!testEncode("\u03A3", "\xE4")) {
return 3;
}
// Test 3 byte
if(!testEncode("\u2502", "\xB3")) {
return 4;
}
// Test 4 byte
if(!testEncode("𠲖", "\xFE")) {
return 5;
}
if(!testArray("F ⌠ Z", "\x46\x20\xF4\x20\x5A")) {
return 6;
}
if(!testView("F ⌠ Z", "\x46\x20\xF4\x20\x5A")) {
return 7;
}
return 0;
}
It compiles and works fine with gcc and clang on Linux, but MSVC complains:
UNICODE_TEST.CPP(65,27): error C2015: too many characters in constant
UNICODE_TEST.CPP(75,18): warning C4566: character represented by universal-character-name '\u03A3' cannot be represented in the current code page (1252)
UNICODE_TEST.CPP(80,18): warning C4566: character represented by universal-character-name '\u2502' cannot be represented in the current code page (1252)
I tried setting the current codepage to UTF-8, but the errors persisted.
How is one supposed to compile this code on Windows?

Answer 1

Look carefully at what you are doing on this line:
if(!testEncode("\u03A3", "\xE4")) {
References the string literal:
"\u03a3"
You are trying to express a UTF-16 character inside an 8-bit (char*) string literal. That just won't work. That's kind of equivalent to doing this:
char sz[2] = {0};
sz[0] = (char)(0x03a3);
And expecting sz[0] to hold the original UTF-16 character. That's what the compiler is warning you about.
If you want to express a 16-bit unicode character inside a string literal, use a wide string. Like follows with the L prefix:
L"\u03a3"
The above is a string literal which holds a signal wide-char character: L"Σ"
And if we really want to be pendantic, we could say this to portably express a UTF-16 character string, use the u prefix:
u"\u03a3"
But on Windows wchar_t is 16-bit, so it doesn't really matter.
You'll probably need to fix your testEncode functions to expect a const wchar_t* instead of a const char* parameter. (I'm honestly not sure what your test* functions are doing, but some of your parameters look suspcious if the goal is to confirm UTF8 to UTF16 conversions)
If you want to express a UTF-8 string in code, you could say this:
"\xCE\xA3"
The above is the UTF-8 representation for a sigma Σ character as UTF-8 string