I want to write special charaters like ô. ö‚`a¹½ˆ in a file. I am working in MFC and using UNICODE character set. While showing string in message box is working good but its not writing the characters to the file.
Here is parts of my code:
CString abc=_T("hello");
CString xyz=compress(abc); //compressing value and return special characters
CStdioFile file_object(_T("abc.txt"),CFile::modeCreate|CFile::modeWrite);
file_object.WriteString(xyz);
It seems that CStdioFile class does not support Unicode characters directly. You can use this workaround (from this CodeProject article)
// Open the file with the specified encoding
FILE *fStream;
errno_t e = _tfopen_s(&fStream, _T("abc.txt"), _T("wt,ccs=UTF-8"));
if (e != 0) return; // failed..
CStdioFile f(fStream); // open the file from this stream
f.WriteString(xyz);
f.Close();
//
// For Reading
//
// Open the file with the specified encoding
FILE *fStream;
errno_t e = _tfopen_s(&fStream, _T("abc.txt"), _T("rt,ccs=UTF-8"));
if (e != 0) return; // failed..CString sRead;
CStdioFile f(fStream); // open the file from this stream
CString sRead;
f.ReadString(sRead);
f.Close();
Instead of using "UTF-8" encoding, you can also use the following encodings:
“ccs=UNICODE” => UTF-16 (Big endian)
“ccs=UTF-8” => UTF-8
“ccs=UTF-16LE” => UTFS-16LE (Little endian)
“ccs=ANSI” => ANSI (default encoding of the OS)
I found one more method. It is working good...
CString text=_T("HelloÄ^°H©º+");
CString strFilePath=_T("C:\\try.txt");
CFile theFile(strFilePath, CFile::modeReadWrite | CFile::modeCreate);
theFile.Write( (LPCTSTR) text, text.GetLength() * sizeof(TCHAR));
theFile.Close();
Related
I cannot get it work to write a correct utf-8 string to a powershell sub-process. ASCII characters work but utf-8 chars, e.g. 'ü', will be interpreted differently. Same problem when reading from the same powershell sub-process.
Summarized: I want to use powershell through my program with utf-8 encoding.
Update:
Allocating a console with AllocConsole(); and then calling SetConsoleCP(CP_UTF8); and SetConsoleOutputCP(CP_UTF8);, as #mklement mentioned in his answer, worked for me, if you have a GUI application without any console. If you have a console application you don't have to allocate the console manually.
Update 2:
If you have a GUI and called AllocConsole(), you can just call ShowWindow(GetConsoleWindow(), SW_HIDE); afterwards to hide the console, as mentioned here.
What I have tried so far:
Setting Input and Output encoding to utf-8
$OutputEncoding = [System.Console]::OutputEncoding = [System.Console]::InputEncoding = [System.Text.Encoding]::UTF8 within the process
Doing the same with UTF-16 in case there is a bug, e.g. ...ext.Encoding]::Unicode
Doing the same with ISO-Latin 1 (cp1252)
Using wchar_t as buffer and input for all tested encodings
Testing byte order for the given string
Testing Unicode (4 byte per character instead of 2)
Building the string bit by bit by myself
Setting compiler flag to \D UNICODE
Code Example for writing:
std::string test("ls ä\n");
DWORD ret = WriteFile(std_in_write, test.c_str(), test.size(), &number_of_bytes_written, nullptr);
if (ret == 0) {
throw PowershellHelper::Exception(PowershellHelper::Exception::Error::COULD_NOT_WRITE_TO_FILE, GetLastError());
}
Output: ls ä
Example Code:
HANDLE std_in_read = nullptr;
HANDLE std_in_write = nullptr;
HANDLE std_out_read = nullptr;
HANDLE std_out_write = nullptr;
SECURITY_ATTRIBUTES security_attr;
STARTUPINFO startup_info;
PROCESS_INFORMATION process_information;
DWORD buffer_size = 1000000;
security_attr = {sizeof(SECURITY_ATTRIBUTES), nullptr, true};
if (!CreatePipe(&std_in_read, &std_in_write, &security_attr, buffer_size)) {
throw PowershellHelper::Exception(PowershellHelper::Exception::Error::COULD_NOT_CREATE_IN_PIPE, GetLastError());
}
if (!CreatePipe(&std_out_read, &std_out_write, &security_attr, buffer_size)) {
throw PowershellHelper::Exception(PowershellHelper::Exception::Error::COULD_NOT_CREATE_OUT_PIPE, GetLastError());
}
GetStartupInfo(&startup_info);
startup_info.dwFlags = STARTF_USESTDHANDLES | STARTF_USESHOWWINDOW;
startup_info.wShowWindow = SW_HIDE;
startup_info.hStdOutput = std_out_write;
startup_info.hStdError = std_out_write;
startup_info.hStdInput = std_in_read;
if (!CreateProcess(TEXT(default_powershell_path), nullptr, nullptr, nullptr, TRUE, 0, nullptr, TEXT(default_windows_path), &startup_info, &process_information)) {
throw PowershellHelper::Exception(PowershellHelper::Exception::Error::COULD_NOT_CREATE_PROCESS, GetLastError());
}
std::string test("ls ä\n");
DWORD ret = WriteFile(std_in_write, test.c_str(), test.size(), &number_of_bytes_written, nullptr);
if (ret == 0) {
throw PowershellHelper::Exception(PowershellHelper::Exception::Error::COULD_NOT_WRITE_TO_FILE, GetLastError());
}
DWORD dword_read;
while (true) {
DWORD total_bytes_available;
if (PeekNamedPipe(std_out_read, nullptr, 0, nullptr, &total_bytes_available, nullptr) == 0) {
throw PowershellHelper::Exception(PowershellHelper::Exception::Error::COULD_NOT_COPY_FROM_PIPE, GetLastError());
}
if (total_bytes_available != 0) {
DWORD minimum = min(buffer_size, total_bytes_available);
char buf[buffer_size];
if (ReadFile(std_out_read, buf, minimum, &dword_read, nullptr) == 0) {
throw PowershellHelper::Exception(PowershellHelper::Exception::Error::COULD_NOT_READ_FILE, GetLastError());
}
std::string tmp(buf);
std::cout << tmp << std::endl;
}
if (total_bytes_available == 0) {
break;
}
std::this_thread::sleep_for(std::chrono::milliseconds(1000));
}
Note: No duplicate of redirect-input-and-output-of-powershell-exe-to-pipes-in-c, since the code only works for ASCII characters and won't handle utf-8 characters at all.
Also no duplicate of c-getting-utf-8-output-from-createprocess, because the suggested solutions won't work as mentioned above and I want to input utf-8 as well as read utf-8.
You need to set the console in- and output code pages to 65001 (UTF-8) before creating your PowerShell process, via the SetConsoleCP and SetConsoleOutputCP WinAPI functions, because the PowerShell CLI uses them to decode its stdin input and to encode its stdout output.
(By contrast, $OutputEncoding = [System.Console]::OutputEncoding = [System.Console]::InputEncoding = [System.Text.Encoding]::UTF8 only applies intra-PowerShell-session when making external-program calls from PowerShell.)
Note: If the calling process isn't itself a console application, you may have to allocate a console before calling SetConsoleCP and SetConsoleOutputCP, using the AllocConsole WinAPI function, but I'm frankly unclear on (a) whether that makes this console instantly visible (which may be undesired) and (b) whether the CreateProcess call then automatically uses this console.
It that doesn't work, you can call via cmd.exe and call chcp before calling powershell.exe, along the lines of cmd /c "chcp 65001 >NUL & powershell -c ..."; chcp 65001 sets the console code pages to 65001, i.e. UTF-8.
(This introduces extra overhead, but a cmd.exe process is relatively light-weight compared to a powershell.exe process, and so is chcp.com).
Here's a sample command you can run from PowerShell to demonstrate:
& {
# Save the current code pages.
$prevInCp, $prevOutCp = [Console]::InputEncoding, [Console]::OutputEncoding
# Write the UTF-8 encoded form of string 'kö' to a temp. file.
# Note: In PowerShell (Core) 7+, use -AsByteStream instead of -Encoding Byte
Set-Content temp1.txt -Encoding Byte ([Text.UTF8Encoding]::new().GetBytes('kö'))
# Switch to UTF-8, pipe the UTF-8 file's content to PowerShell's stdin,
# verify that it was decoded correctly, and output it, again encoded as UTF-8.
cmd /c 'chcp 65001 >NUL & type temp1.txt | powershell -nop -c "$stdinLine = #($input)[0]; $stdinLine -eq ''kö''; Write-Output $stdinLine" > temp2.txt'
# Read the temporary file as UTF-8 and echo its content.
Get-Content -Encoding Utf8 temp2.txt
# Clean up.
Remove-Item temp[12].txt
# Restore the original code pages.
[Console]::InputEncoding = $prevInCp; [Console]::OutputEncoding = $prevOutCp
}
This outputs the following, indicating that the powershell call both correctly read the UTF-8-encoded input and also output it as UTF-8:
True
ö
Note:
You can bypass character encoding problems by using the in-process PowerShell SDK as an alternative to creating a powershell.exe child process, though I don't know how painful that is from C++. For a C# example, see this answer.
I am working in Windows,using vc++2010 and MFC.
Following is my code:
CFile File;
TCHAR lpCause[1024];
CFileException eException;
CString strErrorMessage;
// a very long file path name means a file name over 255 characters
if (!File.Open(_T("a very long file path name"), CFile::modeCreate, &eException))
{
eException.GetErrorMessage(lpCause, 1024);
strErrorMessage = lpCause;
}
else
File.Close();
When I run the code, I got error message:"a very long file path name contains an incorrect path".
My questions are:
How to modify my code to make it work?
I learn that CreateFile() function can add "\\\\?\" in the beginning of file path, then it will extend this limit to 32767 wide characters.How can I do the same thing in MFC?
Cause
In the source of CFile::Open(), there is an explicit check if the path length exceeds _MAX_PATH:
if (lpszFileName != NULL && SUCCEEDED(StringCchLength(lpszFileName, _MAX_PATH, NULL)) )
If _MAX_PATH is exceeded, the function sets pException->m_cause = CFileException::badPath and returns FALSE.
This is true even for the MFC version that comes with VS2017.
So the standard technique to circumvent the _MAX_PATH limit, that is prefixing the path with \\?\ won't work.
Possible Solutions
Call CreateFileW() directly to pass it a path with \\?\ prefix. Use the CFile constructor that accepts a HANDLE to manage the file through a CFile object. The CFile object will take ownership of the handle so you must not call CloseHandle() on the handle.
HANDLE hFile = CreateFileW( L"\\\\?\\a very long file path name", GENERIC_WRITE, 0, nullptr, CREATE_ALWAYS, 0, NULL );
if( hFile != INVALID_HANDLE_VALUE )
{
// Manage the handle using CFile
CFile file( hFile );
// Use the file...
// The CFile destructor closes the handle here.
}
else
{
DWORD err = GetLastError();
// TODO: do your error handling...
}
Another possibility is to derive a class from CFile that overrides CFile::Open() (which is virtual). For the implementation copy/paste the MFC source, but leave out the _MAX_PATH check. For a big project, this class could be a drop-in replacement for CFile to enable long paths. You could even go so far to prepend the \\?\ prefix if it isn't already there (but that is more involved as the prefix also disables the regular transformations from a Win32 path to a NT-style path, like converting / to \, resolving dots and so on).
Im trying to save one pdf in path that contains japanese username. In this case, HPDF_SaveToFile is doing crash my app on windows. Any options to compile or other thing? Any idea to support Unicode filenames with libhaur? I not want to create pdf with japanese encode, I want to write pdf with japanese filename.
A solution in Qt. If you use C++, you can use fstream/ofstream(::write). If you use C, you can use fwrite.
QFile file(path);
if (file.open(QIODevice::WriteOnly))
{
HPDF_SaveToStream(m_pdf);
/* get the data from the stream and write it to file. */
for (;;)
{
HPDF_BYTE buf[4096];
HPDF_UINT32 siz = 4096;
HPDF_STATUS ret = HPDF_ReadFromStream(m_pdf, buf, &siz);
if (siz == 0)
{
break;
}
if (-1 == file.write(reinterpret_cast<const char *>(buf), siz))
{
qDebug() << "Write PDF error";
break;
}
}
}
HPDF_Free(m_pdf);
Refrence: Libharu Usage examples
I want to open a file with QFile::Open where my file name is unicode:
QString fname(QFile::decodeName("D:/أحدالأنشطة.txt"));
QFile qFile(fname);
bool b=qFile.open(QIODevice::ReadOnly);
if(b)
{
FILE* filedesc = fdopen(qFile.handle(), "rb");
if(filedesc!=NULL)
{
char* nb=(char*)malloc(2*sizeof(char));
qDebug()<<"opened ";
size_t size=fread(nb,sizeof(char),2,filedesc);
fclose(filedesc);
qDebug()<<"filedesc closed size "<<size<<"nb "<<QString::fromAscii(nb,2);
nb=NULL;
free(nb);
}else qDebug()<<"filedesc failed error"<<strerror(errno);
}else
qDebug()<<"qFile failed error"<<strerror(errno);
It failed and I get:
qFile failed error No error
any help will be appreciated.
If the data is in WCHAR array than just use QString filename((QChar*) yourWcharData);
If your source file is UTF-8 encoded, then you might be able to do this:
QString fname(QString::fromUtf8("D:/أحدالأنشطة.txt"));
If it's UTF-16, then:
QString fname(QString::fromUtf16("D:/أحدالأنشطة.txt"));
If the source file is neither UTF-8 not UTF-16, try this instead:
QString fname(QString::fromLocal8Bit("D:/أحدالأنشطة.txt"));
If that also doesn't work, then you need to find out the character set your editor is using.
I use the fragment of code as follows to get the contents of a text file. However the buffer buff at the end has only the number 8 at one place and nothing else. The file being opened has the word "Project" as the only content. How can I handle (i.e. print) the content or the result I should receive? What is wrong with the following code:
TCHAR buff[20];
DWORD dwNumRead;
CString finalPath = path + L"\\" + fileName.c_str();
HANDLE hfile=CreateFile(finalPath ,GENERIC_READ|GENERIC_WRITE,0,NULL,OPEN_EXISTING,FILE_ATTRIBUTE_NORMAL,NULL);
if(ReadFile(hfile,buff,20,&dwNumRead,NULL))
{
CString temp;
temp.Format(L"%s",&buff[0]);
ATLTRACE(L"Success %s", temp);
}
CloseHandle(hfile);
The trouble is that you are trying to print the MFC CString which is composed of wide character with %s macro. You need the %S macro to print the wide character.
This works :
char buff[20] = "";
DWORD dwNumRead;
CString finalPath = path + L"\\" + fileName.c_str();;
HANDLE hfile=CreateFile(finalPath ,GENERIC_READ|GENERIC_WRITE,0,NULL,OPEN_EXISTING,FILE_ATTRIBUTE_NORMAL,NULL);
if(ReadFile(hfile,buff,20,&dwNumRead,NULL))
{
CString temp = buff;
ATLTRACE("Success %S", temp);
}
CloseHandle(hfile);
Otherwise, compile your program in unicode with the following extra C++ defs.
UNICODE,_UNICODE