How to convert std::string to wchar_t* - c++

std::regex regexpy("y:(.+?)\"");
std::smatch my;
regex_search(value.text, my, regexpy);
y = my[1];
std::wstring wide_string = std::wstring(y.begin(), y.end());
const wchar_t* p_my_string = wide_string.c_str();
wchar_t* my_string = const_cast<wchar_t*>(p_my_string);
URLDownloadToFile(my_string, aDest);
I'm using Unicode, the encoding of the source string is ASCII, UrlDownloadToFile expands to UrlDownloadToFileW (wchar_t*) the code above compiles in debug mode, but with a lot of warnings like:
warning C4244: 'argument': conversion from 'wchar_t' to 'const _Elem', possible loss of data
So do I ask, how I could convert a std::string to a wchar_t?

First off, you don't need the const_cast, as URLDownloadToFileW() takes a const wchar_t* as input, so passing it wide_string.c_str() will work as-is:
URLDownloadToFile(..., wide_string.c_str(), ...);
That being said, you are constructing a std::wstring with the individual char values of a std::string as-is. That will work without data loss only for ASCII characters <= 127, which have the same numeric values in both ASCII and Unicode. For non-ASCII characters, you need to actually convert the char data to Unicode, such as with MultiByteToWideChar() (or equivilent), eg:
std::wstring to_wstring(const std::string &s)
{
std::wstring wide_string;
// NOTE: be sure to specify the correct codepage that the
// str::string data is actually encoded in...
int len = MultiByteToWideChar(CP_ACP, 0, s.c_str(), s.size(), NULL, 0);
if (len > 0) {
wide_string.resize(len);
MultiByteToWideChar(CP_ACP, 0, s.c_str(), s.size(), &wide_string[0], len);
}
return wide_string;
}
URLDownloadToFileW(..., to_wstring(y).c_str(), ...);
That being said, there is a simpler solution. If the std::string is encoded in the user's default locale, you can simply call URLDownloadToFileA() instead, passing it the original std::string as-is, and let the OS handle the conversion for you, eg:
URLDownloadToFileA(..., y.c_str(), ...);

There is a cross-platform solution. You can use std::mbtowc.
std::wstring convert_mb_to_wc(std::string s) {
std::wstring out;
std::mbtowc(nullptr, 0, 0);
int offset;
size_t index = 0;
for (wchar_t wc;
(offset = std::mbtowc(&wc, &s[index], s.size() - index)) > 0;
index += offset) {
out.push_back(wc);
}
return out;
}
Adapted from an example on cppreference.com at https://en.cppreference.com/w/cpp/string/multibyte/mbtowc .

Related

Can I define a Shift-JIS string literal in c++?

I'm writing a program which involves comparing some strings which will be encoded in Shift-JIS. I only need to compare against a few possible values, and I'd like to encode them as string constants/literals.
I know how to do this with a user-defined literal using iconv or the MultiByteToWideChar/WideCharToMultiByte functions, but I'm wondering if there's some other way to do this. Or maybe a simpler way to define my own literal.
Some context if it's useful:
The program will only be compiled for Windows.
I'm using the Mingw64 cross compiler to build from Linux.
The strings I'm checking against are all exactly 16 bytes long.
I can use c++17 features, but probably not anything from c++20.
The user-defined literal I'm using now:
auto operator "" _sjs(const char* in, std::size_t len)
{
#ifdef HAS_ICONV
auto t = iconv_open("SHIFT-JIS", "UTF-8");
// max length of BS header name + 1 for terminator
char output[17] = "";
memset(output, ' ', 16);
size_t outLeft = 16;
char* tmpOut = output;
iconv(t, &in, &len, &tmpOut, &outLeft);
iconv_close(t);
return std::string{output};
#else
wchar_t output[34];
memset(output, 0, 34);
MultiByteToWideChar(CP_UTF8, 0, in, -1, output, len);
char output2[17] = "";
memset(output2, ' ', 16);
WideCharToMultiByte(932, 0, output, -1, output2, len*2, NULL, NULL);
return std::string{output2};
#endif
}

Is it possible to concatenate string and wstring? [duplicate]

string s = "おはよう";
wstring ws = FUNCTION(s, ws);
How would i assign the contents of s to ws?
Searched google and used some techniques but they can't assign the exact content. The content is distorted.
Assuming that the input string in your example (おはよう) is a UTF-8 encoded (which it isn't, by the looks of it, but let's assume it is for the sake of this explanation :-)) representation of a Unicode string of your interest, then your problem can be fully solved with the standard library (C++11 and newer) alone.
The TL;DR version:
#include <locale>
#include <codecvt>
#include <string>
std::wstring_convert<std::codecvt_utf8_utf16<wchar_t>> converter;
std::string narrow = converter.to_bytes(wide_utf16_source_string);
std::wstring wide = converter.from_bytes(narrow_utf8_source_string);
Longer online compilable and runnable example:
(They all show the same example. There are just many for redundancy...)
http://ideone.com/KA1oty
http://ide.geeksforgeeks.org/5pRLSh
http://rextester.com/DIJZK52174
Note (old):
As pointed out in the comments and explained in https://stackoverflow.com/a/17106065/6345 there are cases when using the standard library to convert between UTF-8 and UTF-16 might give unexpected differences in the results on different platforms. For a better conversion, consider std::codecvt_utf8 as described on http://en.cppreference.com/w/cpp/locale/codecvt_utf8
Note (new):
Since the codecvt header is deprecated in C++17, some worry about the solution presented in this answer were raised. However, the C++ standards committee added an important statement in http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2017/p0618r0.html saying
this library component should be retired to Annex D, along side , until a suitable replacement is standardized.
So in the foreseeable future, the codecvt solution in this answer is safe and portable.
int StringToWString(std::wstring &ws, const std::string &s)
{
std::wstring wsTmp(s.begin(), s.end());
ws = wsTmp;
return 0;
}
Your question is underspecified. Strictly, that example is a syntax error. However, std::mbstowcs is probably what you're looking for.
It is a C-library function and operates on buffers, but here's an easy-to-use idiom, courtesy of Mooing Duck:
std::wstring ws(s.size(), L' '); // Overestimate number of code points.
ws.resize(std::mbstowcs(&ws[0], s.c_str(), s.size())); // Shrink to fit.
If you are using Windows/Visual Studio and need to convert a string to wstring you could use:
#include <AtlBase.h>
#include <atlconv.h>
...
string s = "some string";
CA2W ca2w(s.c_str());
wstring w = ca2w;
printf("%s = %ls", s.c_str(), w.c_str());
Same procedure for converting a wstring to string (sometimes you will need to specify a codepage):
#include <AtlBase.h>
#include <atlconv.h>
...
wstring w = L"some wstring";
CW2A cw2a(w.c_str());
string s = cw2a;
printf("%s = %ls", s.c_str(), w.c_str());
You could specify a codepage and even UTF8 (that's pretty nice when working with JNI/Java). A standard way of converting a std::wstring to utf8 std::string is showed in this answer.
//
// using ATL
CA2W ca2w(str, CP_UTF8);
//
// or the standard way taken from the answer above
#include <codecvt>
#include <string>
// convert UTF-8 string to wstring
std::wstring utf8_to_wstring (const std::string& str) {
std::wstring_convert<std::codecvt_utf8<wchar_t>> myconv;
return myconv.from_bytes(str);
}
// convert wstring to UTF-8 string
std::string wstring_to_utf8 (const std::wstring& str) {
std::wstring_convert<std::codecvt_utf8<wchar_t>> myconv;
return myconv.to_bytes(str);
}
If you want to know more about codepages there is an interesting article on Joel on Software: The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets.
These CA2W (Convert Ansi to Wide=unicode) macros are part of ATL and MFC String Conversion Macros, samples included.
Sometimes you will need to disable the security warning #4995', I don't know of other workaround (to me it happen when I compiled for WindowsXp in VS2012).
#pragma warning(push)
#pragma warning(disable: 4995)
#include <AtlBase.h>
#include <atlconv.h>
#pragma warning(pop)
Edit:
Well, according to this article the article by Joel appears to be: "while entertaining, it is pretty light on actual technical details". Article: What Every Programmer Absolutely, Positively Needs To Know About Encoding And Character Sets To Work With Text.
Windows API only, pre C++11 implementation, in case someone needs it:
#include <stdexcept>
#include <vector>
#include <windows.h>
using std::runtime_error;
using std::string;
using std::vector;
using std::wstring;
wstring utf8toUtf16(const string & str)
{
if (str.empty())
return wstring();
size_t charsNeeded = ::MultiByteToWideChar(CP_UTF8, 0,
str.data(), (int)str.size(), NULL, 0);
if (charsNeeded == 0)
throw runtime_error("Failed converting UTF-8 string to UTF-16");
vector<wchar_t> buffer(charsNeeded);
int charsConverted = ::MultiByteToWideChar(CP_UTF8, 0,
str.data(), (int)str.size(), &buffer[0], buffer.size());
if (charsConverted == 0)
throw runtime_error("Failed converting UTF-8 string to UTF-16");
return wstring(&buffer[0], charsConverted);
}
Here's a way to combining string, wstring and mixed string constants to wstring. Use the wstringstream class.
This does NOT work for multi-byte character encodings. This is just a dumb way of throwing away type safety and expanding 7 bit characters from std::string into the lower 7 bits of each character of std:wstring. This is only useful if you have a 7-bit ASCII strings and you need to call an API that requires wide strings.
#include <sstream>
std::string narrow = "narrow";
std::wstring wide = L"wide";
std::wstringstream cls;
cls << " abc " << narrow.c_str() << L" def " << wide.c_str();
std::wstring total= cls.str();
From char* to wstring:
char* str = "hello worlddd";
wstring wstr (str, str+strlen(str));
From string to wstring:
string str = "hello worlddd";
wstring wstr (str.begin(), str.end());
Note this only works well if the string being converted contains only ASCII characters.
This variant of it is my favourite in real life. It converts the input, if it is valid UTF-8, to the respective wstring. If the input is corrupted, the wstring is constructed out of the single bytes. This is extremely helpful if you cannot really be sure about the quality of your input data.
std::wstring convert(const std::string& input)
{
try
{
std::wstring_convert<std::codecvt_utf8_utf16<wchar_t>> converter;
return converter.from_bytes(input);
}
catch(std::range_error& e)
{
size_t length = input.length();
std::wstring result;
result.reserve(length);
for(size_t i = 0; i < length; i++)
{
result.push_back(input[i] & 0xFF);
}
return result;
}
}
using Boost.Locale:
ws = boost::locale::conv::utf_to_utf<wchar_t>(s);
You can use boost path or std path; which is a lot more easier.
boost path is easier for cross-platform application
#include <boost/filesystem/path.hpp>
namespace fs = boost::filesystem;
//s to w
std::string s = "xxx";
auto w = fs::path(s).wstring();
//w to s
std::wstring w = L"xxx";
auto s = fs::path(w).string();
if you like to use std:
#include <filesystem>
namespace fs = std::filesystem;
//The same
c++ older version
#include <experimental/filesystem>
namespace fs = std::experimental::filesystem;
//The same
The code within still implement a converter which you dont have to unravel the detail.
For me the most uncomplicated option without big overhead is:
Include:
#include <atlbase.h>
#include <atlconv.h>
Convert:
char* whatever = "test1234";
std::wstring lwhatever = std::wstring(CA2W(std::string(whatever).c_str()));
If needed:
lwhatever.c_str();
String to wstring
std::wstring Str2Wstr(const std::string& str)
{
int size_needed = MultiByteToWideChar(CP_UTF8, 0, &str[0], (int)str.size(), NULL, 0);
std::wstring wstrTo(size_needed, 0);
MultiByteToWideChar(CP_UTF8, 0, &str[0], (int)str.size(), &wstrTo[0], size_needed);
return wstrTo;
}
wstring to String
std::string Wstr2Str(const std::wstring& wstr)
{
typedef std::codecvt_utf8<wchar_t> convert_typeX;
std::wstring_convert<convert_typeX, wchar_t> converterX;
return converterX.to_bytes(wstr);
}
If you have QT and if you are lazy to implement a function and stuff you can use
std::string str;
QString(str).toStdWString()
Here is my super basic solution that might not work for everyone. But would work for a lot of people.
It requires usage of the Guideline Support Library.
Which is a pretty official C++ library that was designed by many C++ committee authors:
https://github.com/isocpp/CppCoreGuidelines
https://github.com/Microsoft/GSL
std::string to_string(std::wstring const & wStr)
{
std::string temp = {};
for (wchar_t const & wCh : wStr)
{
// If the string can't be converted gsl::narrow will throw
temp.push_back(gsl::narrow<char>(wCh));
}
return temp;
}
All my function does is allow the conversion if possible. Otherwise throw an exception.
Via the usage of gsl::narrow (https://github.com/isocpp/CppCoreGuidelines/blob/master/CppCoreGuidelines.md#es49-if-you-must-use-a-cast-use-a-named-cast)
method s2ws works well. Hope helps.
std::wstring s2ws(const std::string& s) {
std::string curLocale = setlocale(LC_ALL, "");
const char* _Source = s.c_str();
size_t _Dsize = mbstowcs(NULL, _Source, 0) + 1;
wchar_t *_Dest = new wchar_t[_Dsize];
wmemset(_Dest, 0, _Dsize);
mbstowcs(_Dest,_Source,_Dsize);
std::wstring result = _Dest;
delete []_Dest;
setlocale(LC_ALL, curLocale.c_str());
return result;
}
Based upon my own testing (On windows 8, vs2010) mbstowcs can actually damage original string, it works only with ANSI code page. If MultiByteToWideChar/WideCharToMultiByte can also cause string corruption - but they tends to replace characters which they don't know with '?' question marks, but mbstowcs tends to stop when it encounters unknown character and cut string at that very point. (I have tested Vietnamese characters on finnish windows).
So prefer Multi*-windows api function over analogue ansi C functions.
Also what I've noticed shortest way to encode string from one codepage to another is not use MultiByteToWideChar/WideCharToMultiByte api function calls but their analogue ATL macros: W2A / A2W.
So analogue function as mentioned above would sounds like:
wstring utf8toUtf16(const string & str)
{
USES_CONVERSION;
_acp = CP_UTF8;
return A2W( str.c_str() );
}
_acp is declared in USES_CONVERSION macro.
Or also function which I often miss when performing old data conversion to new one:
string ansi2utf8( const string& s )
{
USES_CONVERSION;
_acp = CP_ACP;
wchar_t* pw = A2W( s.c_str() );
_acp = CP_UTF8;
return W2A( pw );
}
But please notice that those macro's use heavily stack - don't use for loops or recursive loops for same function - after using W2A or A2W macro - better to return ASAP, so stack will be freed from temporary conversion.
std::string -> wchar_t[] with safe mbstowcs_s function:
auto ws = std::make_unique<wchar_t[]>(s.size() + 1);
mbstowcs_s(nullptr, ws.get(), s.size() + 1, s.c_str(), s.size());
This is from my sample code
use this code to convert your string to wstring
std::wstring string2wString(const std::string& s){
int len;
int slength = (int)s.length() + 1;
len = MultiByteToWideChar(CP_ACP, 0, s.c_str(), slength, 0, 0);
wchar_t* buf = new wchar_t[len];
MultiByteToWideChar(CP_ACP, 0, s.c_str(), slength, buf, len);
std::wstring r(buf);
delete[] buf;
return r;
}
int main(){
std::wstring str="your string";
std::wstring wStr=string2wString(str);
return 0;
}
string s = "おはよう"; is an error.
You should use wstring directly:
wstring ws = L"おはよう";

Unexpected results with wchar_t and c_str string conversion

Based on this answer to a related question, I tried to write a method that converts a standard string to a wide string, which I can then convert into a wchar_t*.
Why aren't the two different ways of creating the wchar_t* equivalent? (I've shown the values that my debugger gives me).
TEST_METHOD(TestingAssertsWithGetWideString)
{
std::wstring wString1 = GetWideString("me");
const wchar_t* wchar1 = wString1.c_str(); // wchar1 = "me"
const wchar_t* wchar2 = GetWideString("me").c_str(); // wchar2 = "ﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮﻮ#" (Why?!)
}
where GetWideString is defined as follows:
inline const std::wstring GetWideString(const std::string &str)
{
std::wstring wstr;
wstr.assign(str.begin(), str.end());
return wstr;
};
Note: the following doesn't work either.
const wchar_t* wchar2 = GetWChar("me");
const wchar_t *GetWChar(const std::string &str)
{
std::wstring wstr;
wstr.assign(str.begin(), str.end());
return wstr.c_str();
};
Each time you call GetWideString(), you are creating a new std::wstring, which has a newly allocated memory buffer. You are comparing pointers to different memory blocks (assuming Assert::AreEqual() is simply comparing the pointers themselves and not the contents of the memory blocks that are being pointed at).
Update: const wchar_t* wchar2 = GetWideString("me").c_str(); does not work because GetWideString() returns a temporary std::wstring that goes out of scope and gets freed as soon as the statement is finished. Thus you are obtaining a pointer to a temporary memory block, and then leaving that pointer dangling when that memory gets freed before you can use the pointer for anything.
Also, const wchar_t* wchar2 = GetWChar("me"); should not compile. GetWChar() returns a std::wstring, which does not implement an implicit conversion to wchar_t*. You have to use the c_str() method to get a wchar_t* from a std::wstring.
Because the two pointers aren't equal. A wchar_t * is not a String, so you get the generic AreEqual.
std::wstring contains of wide characters of type wchar_t. std::string contains characters of type char. For special characters stored within std::string a multi-byte encoding is being used, i.e. some characters are represented by 2 characters within such a string. Converting between these thus can not be easy as calling a simple assign.
To convert between "wide" strings and multi-byte strings, you can use following helpers (Windows only):
// multi byte to wide char:
std::wstring s2ws(const std::string& str)
{
int size_needed = MultiByteToWideChar(CP_UTF8, 0, &str[0], (int)str.size(), NULL, 0);
std::wstring wstrTo(size_needed, 0);
MultiByteToWideChar(CP_UTF8, 0, &str[0], (int)str.size(), &wstrTo[0], size_needed);
return wstrTo;
}
// wide char to multi byte:
std::string ws2s(const std::wstring& wstr)
{
int size_needed = WideCharToMultiByte(CP_ACP, 0, wstr.c_str(), int(wstr.length() + 1), 0, 0, 0, 0);
std::string strTo(size_needed, 0);
WideCharToMultiByte(CP_ACP, 0, wstr.c_str(), int(wstr.length() + 1), &strTo[0], size_needed, 0, 0);
return strTo;
}

One file lib to conv utf8 (char*) to wchar_t?

I am using libjson which is awesome. The only problem I have is I need to convert an utf8 string (char*) to a wide char string (wchar_t*). I googled and tried 3 different libs and they ALL failed (due to missing headers).
I don't need anything fancy. Just a one way conversion. How do I do this?
If you're on windows (which, chances are you are, given your need for wchar_t), use MultiByteToWideChar function (declared in windows.h), as so:
int length = MultiByteToWideChar(CP_UTF8, 0, src, src_length, 0, 0);
wchar_t *output_buffer = new wchar_t [length];
MultiByteToWideChar(CP_UTF8, 0, src, src_length, output_buffer, length);
Alternatively, if all you're looking for is a literal multibyte representation of your UTF8 (which is improbable, but possible), use the following (stdlib.h):
wchar_t * output_buffer = new wchar_t [1024];
int length = mbstowcs(output_buffer, src, 1024);
if(length > 1024){
delete[] output_buffer;
output_buffer = new wchar_t[length+1];
mbstowcs(output_buffer, src, length);
}
Hope this helps.
the below successfully enables CreateDirectoryW() to write to C:\Users\ПетрКарасев , basically an easier-to-understand wrapper around the MultiByteTyoWideChar mentioned by someone earlier.
std::wstring utf16_from_utf8(const std::string & utf8)
{
// Special case of empty input string
if (utf8.empty())
return std::wstring();
// Шаг 1, Get length (in wchar_t's) of resulting UTF-16 string
const int utf16_length = ::MultiByteToWideChar(
CP_UTF8, // convert from UTF-8
0, // default flags
utf8.data(), // source UTF-8 string
utf8.length(), // length (in chars) of source UTF-8 string
NULL, // unused - no conversion done in this step
0 // request size of destination buffer, in wchar_t's
);
if (utf16_length == 0)
{
// Error
DWORD error = ::GetLastError();
throw ;
}
// // Шаг 2, Allocate properly sized destination buffer for UTF-16 string
std::wstring utf16;
utf16.resize(utf16_length);
// // Шаг 3, Do the actual conversion from UTF-8 to UTF-16
if ( ! ::MultiByteToWideChar(
CP_UTF8, // convert from UTF-8
0, // default flags
utf8.data(), // source UTF-8 string
utf8.length(), // length (in chars) of source UTF-8 string
&utf16[0], // destination buffer
utf16.length() // size of destination buffer, in wchar_t's
) )
{
// не работает сука ...
DWORD error = ::GetLastError();
throw;
}
return utf16; // ура!
}
Here is a piece of code i wrote. It seems to work well enough. It returns 0 on utf8 error or when the value is > FFFF (which cant be held by a wchar_t)
#include <string>
using namespace std;
wchar_t* utf8_to_wchar(const char*utf8){
wstring sz;
wchar_t c;
auto p=utf8;
while(*p!=0){
auto v=(*p);
if(v>=0){
c = v;
sz+=c;
++p;
continue;
}
int shiftCount=0;
if((v&0xE0) == 0xC0){
shiftCount=1;
c = v&0x1F;
}
else if((v&0xF0) == 0xE0){
shiftCount=2;
c = v&0xF;
}
else
return 0;
++p;
while(shiftCount){
v = *p;
++p;
if((v&0xC0) != 0x80) return 0;
c<<=6;
c |= (v&0x3F);
--shiftCount;
}
sz+=c;
}
return (wchar_t*)sz.c_str();
}
The following (untested) code shows how to convert a multibyte string in your current locale into a wide string. So if your current locale is UTF-8, then this will suit your needs.
const char * inputStr = ... // your UTF-8 input
size_t maxSize = strlen(inputStr) + 1;
wchar_t * outputWStr = new wchar_t[maxSize];
size_t result = mbstowcs(outputWStr, inputStr, maxSize);
if (result == -1) {
cerr << "Invalid multibyte characters in input";
}
You can use setlocale() to set your locale.

C++ Convert string (or char*) to wstring (or wchar_t*)

string s = "おはよう";
wstring ws = FUNCTION(s, ws);
How would i assign the contents of s to ws?
Searched google and used some techniques but they can't assign the exact content. The content is distorted.
Assuming that the input string in your example (おはよう) is a UTF-8 encoded (which it isn't, by the looks of it, but let's assume it is for the sake of this explanation :-)) representation of a Unicode string of your interest, then your problem can be fully solved with the standard library (C++11 and newer) alone.
The TL;DR version:
#include <locale>
#include <codecvt>
#include <string>
std::wstring_convert<std::codecvt_utf8_utf16<wchar_t>> converter;
std::string narrow = converter.to_bytes(wide_utf16_source_string);
std::wstring wide = converter.from_bytes(narrow_utf8_source_string);
Longer online compilable and runnable example:
(They all show the same example. There are just many for redundancy...)
http://ideone.com/KA1oty
http://ide.geeksforgeeks.org/5pRLSh
http://rextester.com/DIJZK52174
Note (old):
As pointed out in the comments and explained in https://stackoverflow.com/a/17106065/6345 there are cases when using the standard library to convert between UTF-8 and UTF-16 might give unexpected differences in the results on different platforms. For a better conversion, consider std::codecvt_utf8 as described on http://en.cppreference.com/w/cpp/locale/codecvt_utf8
Note (new):
Since the codecvt header is deprecated in C++17, some worry about the solution presented in this answer were raised. However, the C++ standards committee added an important statement in http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2017/p0618r0.html saying
this library component should be retired to Annex D, along side , until a suitable replacement is standardized.
So in the foreseeable future, the codecvt solution in this answer is safe and portable.
int StringToWString(std::wstring &ws, const std::string &s)
{
std::wstring wsTmp(s.begin(), s.end());
ws = wsTmp;
return 0;
}
Your question is underspecified. Strictly, that example is a syntax error. However, std::mbstowcs is probably what you're looking for.
It is a C-library function and operates on buffers, but here's an easy-to-use idiom, courtesy of Mooing Duck:
std::wstring ws(s.size(), L' '); // Overestimate number of code points.
ws.resize(std::mbstowcs(&ws[0], s.c_str(), s.size())); // Shrink to fit.
If you are using Windows/Visual Studio and need to convert a string to wstring you could use:
#include <AtlBase.h>
#include <atlconv.h>
...
string s = "some string";
CA2W ca2w(s.c_str());
wstring w = ca2w;
printf("%s = %ls", s.c_str(), w.c_str());
Same procedure for converting a wstring to string (sometimes you will need to specify a codepage):
#include <AtlBase.h>
#include <atlconv.h>
...
wstring w = L"some wstring";
CW2A cw2a(w.c_str());
string s = cw2a;
printf("%s = %ls", s.c_str(), w.c_str());
You could specify a codepage and even UTF8 (that's pretty nice when working with JNI/Java). A standard way of converting a std::wstring to utf8 std::string is showed in this answer.
//
// using ATL
CA2W ca2w(str, CP_UTF8);
//
// or the standard way taken from the answer above
#include <codecvt>
#include <string>
// convert UTF-8 string to wstring
std::wstring utf8_to_wstring (const std::string& str) {
std::wstring_convert<std::codecvt_utf8<wchar_t>> myconv;
return myconv.from_bytes(str);
}
// convert wstring to UTF-8 string
std::string wstring_to_utf8 (const std::wstring& str) {
std::wstring_convert<std::codecvt_utf8<wchar_t>> myconv;
return myconv.to_bytes(str);
}
If you want to know more about codepages there is an interesting article on Joel on Software: The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets.
These CA2W (Convert Ansi to Wide=unicode) macros are part of ATL and MFC String Conversion Macros, samples included.
Sometimes you will need to disable the security warning #4995', I don't know of other workaround (to me it happen when I compiled for WindowsXp in VS2012).
#pragma warning(push)
#pragma warning(disable: 4995)
#include <AtlBase.h>
#include <atlconv.h>
#pragma warning(pop)
Edit:
Well, according to this article the article by Joel appears to be: "while entertaining, it is pretty light on actual technical details". Article: What Every Programmer Absolutely, Positively Needs To Know About Encoding And Character Sets To Work With Text.
Windows API only, pre C++11 implementation, in case someone needs it:
#include <stdexcept>
#include <vector>
#include <windows.h>
using std::runtime_error;
using std::string;
using std::vector;
using std::wstring;
wstring utf8toUtf16(const string & str)
{
if (str.empty())
return wstring();
size_t charsNeeded = ::MultiByteToWideChar(CP_UTF8, 0,
str.data(), (int)str.size(), NULL, 0);
if (charsNeeded == 0)
throw runtime_error("Failed converting UTF-8 string to UTF-16");
vector<wchar_t> buffer(charsNeeded);
int charsConverted = ::MultiByteToWideChar(CP_UTF8, 0,
str.data(), (int)str.size(), &buffer[0], buffer.size());
if (charsConverted == 0)
throw runtime_error("Failed converting UTF-8 string to UTF-16");
return wstring(&buffer[0], charsConverted);
}
Here's a way to combining string, wstring and mixed string constants to wstring. Use the wstringstream class.
This does NOT work for multi-byte character encodings. This is just a dumb way of throwing away type safety and expanding 7 bit characters from std::string into the lower 7 bits of each character of std:wstring. This is only useful if you have a 7-bit ASCII strings and you need to call an API that requires wide strings.
#include <sstream>
std::string narrow = "narrow";
std::wstring wide = L"wide";
std::wstringstream cls;
cls << " abc " << narrow.c_str() << L" def " << wide.c_str();
std::wstring total= cls.str();
From char* to wstring:
char* str = "hello worlddd";
wstring wstr (str, str+strlen(str));
From string to wstring:
string str = "hello worlddd";
wstring wstr (str.begin(), str.end());
Note this only works well if the string being converted contains only ASCII characters.
This variant of it is my favourite in real life. It converts the input, if it is valid UTF-8, to the respective wstring. If the input is corrupted, the wstring is constructed out of the single bytes. This is extremely helpful if you cannot really be sure about the quality of your input data.
std::wstring convert(const std::string& input)
{
try
{
std::wstring_convert<std::codecvt_utf8_utf16<wchar_t>> converter;
return converter.from_bytes(input);
}
catch(std::range_error& e)
{
size_t length = input.length();
std::wstring result;
result.reserve(length);
for(size_t i = 0; i < length; i++)
{
result.push_back(input[i] & 0xFF);
}
return result;
}
}
using Boost.Locale:
ws = boost::locale::conv::utf_to_utf<wchar_t>(s);
You can use boost path or std path; which is a lot more easier.
boost path is easier for cross-platform application
#include <boost/filesystem/path.hpp>
namespace fs = boost::filesystem;
//s to w
std::string s = "xxx";
auto w = fs::path(s).wstring();
//w to s
std::wstring w = L"xxx";
auto s = fs::path(w).string();
if you like to use std:
#include <filesystem>
namespace fs = std::filesystem;
//The same
c++ older version
#include <experimental/filesystem>
namespace fs = std::experimental::filesystem;
//The same
The code within still implement a converter which you dont have to unravel the detail.
For me the most uncomplicated option without big overhead is:
Include:
#include <atlbase.h>
#include <atlconv.h>
Convert:
char* whatever = "test1234";
std::wstring lwhatever = std::wstring(CA2W(std::string(whatever).c_str()));
If needed:
lwhatever.c_str();
String to wstring
std::wstring Str2Wstr(const std::string& str)
{
int size_needed = MultiByteToWideChar(CP_UTF8, 0, &str[0], (int)str.size(), NULL, 0);
std::wstring wstrTo(size_needed, 0);
MultiByteToWideChar(CP_UTF8, 0, &str[0], (int)str.size(), &wstrTo[0], size_needed);
return wstrTo;
}
wstring to String
std::string Wstr2Str(const std::wstring& wstr)
{
typedef std::codecvt_utf8<wchar_t> convert_typeX;
std::wstring_convert<convert_typeX, wchar_t> converterX;
return converterX.to_bytes(wstr);
}
If you have QT and if you are lazy to implement a function and stuff you can use
std::string str;
QString(str).toStdWString()
Here is my super basic solution that might not work for everyone. But would work for a lot of people.
It requires usage of the Guideline Support Library.
Which is a pretty official C++ library that was designed by many C++ committee authors:
https://github.com/isocpp/CppCoreGuidelines
https://github.com/Microsoft/GSL
std::string to_string(std::wstring const & wStr)
{
std::string temp = {};
for (wchar_t const & wCh : wStr)
{
// If the string can't be converted gsl::narrow will throw
temp.push_back(gsl::narrow<char>(wCh));
}
return temp;
}
All my function does is allow the conversion if possible. Otherwise throw an exception.
Via the usage of gsl::narrow (https://github.com/isocpp/CppCoreGuidelines/blob/master/CppCoreGuidelines.md#es49-if-you-must-use-a-cast-use-a-named-cast)
method s2ws works well. Hope helps.
std::wstring s2ws(const std::string& s) {
std::string curLocale = setlocale(LC_ALL, "");
const char* _Source = s.c_str();
size_t _Dsize = mbstowcs(NULL, _Source, 0) + 1;
wchar_t *_Dest = new wchar_t[_Dsize];
wmemset(_Dest, 0, _Dsize);
mbstowcs(_Dest,_Source,_Dsize);
std::wstring result = _Dest;
delete []_Dest;
setlocale(LC_ALL, curLocale.c_str());
return result;
}
Based upon my own testing (On windows 8, vs2010) mbstowcs can actually damage original string, it works only with ANSI code page. If MultiByteToWideChar/WideCharToMultiByte can also cause string corruption - but they tends to replace characters which they don't know with '?' question marks, but mbstowcs tends to stop when it encounters unknown character and cut string at that very point. (I have tested Vietnamese characters on finnish windows).
So prefer Multi*-windows api function over analogue ansi C functions.
Also what I've noticed shortest way to encode string from one codepage to another is not use MultiByteToWideChar/WideCharToMultiByte api function calls but their analogue ATL macros: W2A / A2W.
So analogue function as mentioned above would sounds like:
wstring utf8toUtf16(const string & str)
{
USES_CONVERSION;
_acp = CP_UTF8;
return A2W( str.c_str() );
}
_acp is declared in USES_CONVERSION macro.
Or also function which I often miss when performing old data conversion to new one:
string ansi2utf8( const string& s )
{
USES_CONVERSION;
_acp = CP_ACP;
wchar_t* pw = A2W( s.c_str() );
_acp = CP_UTF8;
return W2A( pw );
}
But please notice that those macro's use heavily stack - don't use for loops or recursive loops for same function - after using W2A or A2W macro - better to return ASAP, so stack will be freed from temporary conversion.
std::string -> wchar_t[] with safe mbstowcs_s function:
auto ws = std::make_unique<wchar_t[]>(s.size() + 1);
mbstowcs_s(nullptr, ws.get(), s.size() + 1, s.c_str(), s.size());
This is from my sample code
use this code to convert your string to wstring
std::wstring string2wString(const std::string& s){
int len;
int slength = (int)s.length() + 1;
len = MultiByteToWideChar(CP_ACP, 0, s.c_str(), slength, 0, 0);
wchar_t* buf = new wchar_t[len];
MultiByteToWideChar(CP_ACP, 0, s.c_str(), slength, buf, len);
std::wstring r(buf);
delete[] buf;
return r;
}
int main(){
std::wstring str="your string";
std::wstring wStr=string2wString(str);
return 0;
}
string s = "おはよう"; is an error.
You should use wstring directly:
wstring ws = L"おはよう";