I am attempting to write the following character out in windows command prompt: ｭ (U+FF6D).
I am able to see the character get written out using WriteConsoleW. I am also able to see the character if i use WideCharToMultiByte using the CP_ACP code page (chcp returns 932: Japanese). However, when I attempt to use regular wcout on the same string which WriteConsoleW successfully prints, it chokes.
When I execute setlocale(LC_ALL, "") it prints English_UnitedStates.1252 (the default code page that I had when I installed).
Why is wcout failing when the others are succeeding?
Note: I rebooted the machine to change its system locale to Japan Japanese
The default locale for C++ iostreams is always the "C" locale. From the C++03 standard, §27.4.2.3/4:
locale getloc() const;
If no locale has been imbued, a copy of the global C++ locale, locale(), in effect at the time of construction.
From §22.1.1.2/1-2:
locale() throw();
Default constructor: a snapshot of the current global locale.
Constructs a copy of the argument last passed to locale::global(locale&), if it has been called; else, the resulting facets have virtual function semantics identical to those of locale::classic().
From §22.1.1.5/4-6:
static const locale& classic();
The "C" locale.
Returns: A locale that implements the classic "C" locale semantics, equivalent to the value locale("C").
Notes: This locale, its facets, and their member functions, do not change with time.
As std::cout and std::wcout have static storage duration, they are guaranteed to be initialized before main is called, and consequently will always have the "C" locale at application startup; i.e., there is no point early enough in execution that one can call locale::global and change the default locale for std::cout and std::wcout. Thus, you must always imbue the global streams yourself if you want to use a non-default code page.
wcout is created before any code in main executes. By the time you call setlocale, wcout is already there, ready to do its thing. It makes no attempt at tracking subsequent changes you might make with setlocale, so it continues to use the default instead of whatever you set with setlocale.
Related
This is happening inside a big project so I cannot really post a minimal reproducible example but I'll try asking anyway. I'm building a list of benchmarks application integrated with a framework we're working on and on one of them the convertion we require to make (float -> string) with to_string appears to reproduce a comma separated result.
| Monitored values:
| [ my_time_monitor.average = 61720,000000 ]
This is the function responsible:
std::string operating_point_parser::operator()(const int32_t num_threads, const float exec_time_ms) const {
return "{\"compute\":[{\"knobs\":{\"num_threads\":" + std::to_string(num_threads) + "},\"metrics\":{\"exec_time_ms\":[" + std::to_string(exec_time_ms) + ",0]}}]}";
}
Since as I said the same exact function is being called by other applications which don't show this unexpected behavior, my guess is that some internal compilation flags are messing around.
set( CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++17 -DENABLE_THREADS -DHAVE_RANDOM -DHAVE_UNISTD_H -DHAVE_SYS_FILE_H -DHAVE_SYS_MMAN_H -DHAVE_CONFIG_H -DVIPSDATASET_PATH=\"\\\"${CMAKE_CURRENT_SOURCE_DIR}/dataset/orion_18000x18000.v\\\"\"" )
set( CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -DHAVE_RANDOM -DHAVE_UNISTD_H -DIM_PREFIX=\"\\\"${CMAKE_CURRENT_SOURCE_DIR}/dataset\\\"\" -DIM_EXEEXT=\"\\\"\\\"\" -DIM_LIBDIR=\"\\\"${CMAKE_INSTALL_PREFIX}/lib\\\"\" -DGETTEXT_PACKAGE=\"\\\"vips7\"\\\" -DHAVE_SYS_FILE_H -DHAVE_SYS_MMAN_H -DHAVE_CONFIG_H")
If you want to take a look at the full application code here's the link. The operating_point_parse::operator() is called inside margot::compute::push_custom_monitor_values().
As stated in the Notes of https://en.cppreference.com/w/cpp/string/basic_string/to_string, to_string relies on the current locale for formatting purposes:
std::to_string relies on the current locale for formatting purposes,
and therefore concurrent calls to std::to_string from multiple threads
may result in partial serialization of calls. C++17 provides
std::to_chars as a higher-performance locale-independent alternative.
So, if you want to have dots instead of commans, you have to adjust the current locale.
Or instead of changing the global locale with std::locale::global(...) you could use a stringstream and imbue() the locale on that stream only, for ex.
stringstream ss;
ss.imbue( locale you want )
ss << ... write what you need
ss.str(); // get formatted string
std::to_string uses the currently active locale for formatting.
You can set the active locale using a C locale name using:
const char* locale = "C";
std::locale::global(std::locale(locale));
Meaning of locale name is specified in the C standard (quote from C11 draft):
7.11.1.1 The setlocale function
A value of "C" for locale specifies the minimal environment for C translation; a value of "" for locale specifies the locale-specific native environment. Other implementation-defined strings may be passed as the second argument to setlocale.
That value is going to be formatted inside a JSON string
In this case, and more generally when ever you wish to format using style that shouldn't depend on the global locale, should avoid std::to_string.
What would you recommend since you advised on avoiding it?
Anything that doesn't use locale, or lets you specify the locale to use instead of using the global locale. For example:
std::format("{}", 0.42); // doesn't use locale
std::format(std::locale("C"), "{}", 0.42); // use specific locale
Another example is a stringstream with imbued locale as suggested in the other answer.
I saw code that used a locale but didn't provide a name for it. I saw this several times actually and I'm not sure how it works.
auto& f = std::use_facet<std::ctype<wchar_t>>(std::locale());
I was wondering what it means to just use an empty locale constructor. I thought all locale names had to be specified.
For some context, here is where I found the code: http://en.cppreference.com/w/cpp/locale/ctype/toupper
http://www.cplusplus.com/reference/locale/locale/locale/
The default constructor constructs a copy of the current global locale, which is the locale set by a previous call to locale::global, or locale::classic if locale::global has not been called.
What is locale::classic?
http://www.cplusplus.com/reference/locale/locale/classic/
Returns the classic locale, which corresponds in semantics to the "C" locale (the default locale).
What is the "C" locale?
http://www.cplusplus.com/reference/clocale/
The "C" locale is the minimal locale. It is a rather neutral locale which has the same settings across all systems and compilers, and therefore the exact results of a program using this locale are predictable. This is the locale used by default on all C programs.
std::locale has a constructor that takes no parameter. It creates a locale that's a snapshot of the current global locale.
In a project I am currently working on I link to a proprietary dynamic library. As soon as I run the library's initialize function, the behavior of logging and printing of numbers changes.
Commas have been inserted at every third decimal. i.e.
cout << 123456789 << endl
used to print out 123456789 and now it prints 123,456,789. This is horribly annoying, because this behavior is not what I want.
This issue is not only apparent in the binary I am compiling, but also shows up in all the couts and stringstreams in the libraries that I link to it.
I have tried using this line of code after calling the initialize function:
setlocale(LC_ALL,"C");
thinking it might reset my locale to the default but to no avail. The commas persist!
This snippet of code:
std::cout.imbue(std::locale("C"));
works to reset the locale of couts and every stringstream I apply it too. However, do I really need to call imbue on EVERY stringstream declared in EVERY library I link to? There are some libraries that are proprietary and I cannot actually change their source code.
There must be a way to reset the locale back to "C" globally?
I believe std::locale::global(std::locale("C")); should do the trick. See http://en.cppreference.com/w/cpp/locale/locale/global
Note that this only affects streams created after this call.
Any streams such as cout that the other library already imbued would have to be re-imbued back to the desired default locale.
And finally I would strongly suggest filing a defect report against the library you're using because it's pretty unjustifiable to to unilaterally make such striking global changes within your initialization function.
In C++, I can set the current locale like this:
std::locale::global(std::locale(name))
But how can I get the current global locale?
In my code, I need to get the current locale, save it to a tmp var, set the global locale to something else, output something, then set it back to the previous locale.
If you call the default constructor of std::locale, you get it.
std::locale the_global_locale; // <-- automatically correct to std::locale::global
// or a copy of std::locale::classic
More info here: http://en.cppreference.com/w/cpp/locale/locale/locale
Its return value is the old locale, so just use that.
locale l = locale::global(locale(name));
//do some stuff here
locale::global(l);
Edit: Potentially useful: http://en.cppreference.com/w/cpp/locale/locale/global
As ipc says, the default constructor for std::locale gives you a copy of the current global locale, but why do you need to cache and then reset the global locale?
C++ routines that use a locale can typically use a C++ locale object you specify, so you don't have to mess with the global locale at all. Using locale objects should be preferred to using the C++ global locale.
I have been unable to find any decent documentation on this function. The code base I am working with uses a function from winuser.h called LoadStringW which takes as arguments: (HINSTANCE hInstance, UINT uID, LPWSTR lpBuffer, int cchBufferMax).
What is this function? What is it for? When might it return 0?
It might be worth a mention that nearly all Win32 APIs that deal with strings have an 'A' and a 'W' variant.
The variant actually called is determined by the definition of macros that don't end in 'A' or 'W' - those macro names are what you might usually think of as the API function's name (LoadString() in this case). UNICODE builds will use the 'W' names and non-UNICODE builds will use the 'A' names.
https://learn.microsoft.com/en-us/windows/desktop/Intl/unicode-in-the-windows-api
There are times when you might want to call a Unicode version of an API even if the build isn't Unicode, in which case you just directly use the name with the 'W' tacked on to the end (it's less often necessary to need to call the non-Unicode APIs in a Unicode build, but it's just as possible). Since the non-Unicode versions of Windows are obsolete, Microsoft has started more and more to implement only Unicode versions of APIs. Note that in nearly all cases, all that the non-Unicode versions of the APIs do is to convert the ANSI/MBCS strings to Unicode, call the 'W' function, then clean up afterward.
LoadStringW is the Unicode version of LoadString.
The documentation states "If the function succeeds, the return value is the number of TCHARs copied into the buffer, not including the terminating NULL character, or zero if the string resource does not exist. To get extended error information, call GetLastError."
Here is the documentation for LoadString(): http://msdn.microsoft.com/en-us/library/ms647486%28VS.85%29.aspx
.. and here is the documentation explaining the differences between ANSI and Unicode functions in the Windows API: http://msdn.microsoft.com/en-us/library/cc500321.aspx.
Basically, the function LoadString comes in two flavours, ANSI and Unicode. LoadStringW is the Unicode-specific version of LoadString.
Edit: Just to be clear, there aren't really two completely separate functions. The ANSI version really just converts the string and calls the unicode version, which does all of the real work.
LoadStringW() is the WideCharacter version of the LoadString function.
See MSDN
It loads a widestring from a stringtable resource using the Windows Unicode Layer for Win95 and NT 3.51. See MSDN for details (see the remarks section).
For the umpteenth time, I just confirmed that when the resource compiler is instructed to null terminate the strings, the count returned by LoadString includes the terminal NULL character. I did so by examining the output buffer that I made available to LoadString.
Resource strings are not null terminated by default. In that case, the returned count excludes the terminal null character, as described in the documentation, because the null is appended by the function after the string is copied into the output buffer.
I suspect this behavior is due to the fact that LoadString disregards the fact that the resource compiler was instructed to null terminate the strings. Indeed, I suspect that it has no way of knowing that they were.
With respect to why you would want to null terminate your resource strings in the first place, when they work just fine without them, and your PE file is thereby a tad smaller, the reason is that the wide character implementation of LoadString, at the LoadStringW entry point, returns a pointer to the string, rather than copying it into a buffer, if the buffer address passed into it is a NULL pointer. Unless your strings are null terminated, using LoadString in this way produces quite unwelcome results.
Since resource strings are always stored as Unicode (wide character) strings, the ANSI implementation of LoadString cannot return a pointer, as the string must be converted to ANSI; hence, it cannot simply be copied.