write Unicode strings into a txt file - c++

Question

I made my simple txt scanner who writes the text into a file that matches my selection. The problem is writing to file when instead of the pen writes, for example, 洀漀. On picture you can see for example:
#include <iostream>
#include <fstream>
#include <string>
using namespace std;
int main()
{
int offset;
wstring DBSearchLine, ScanLine;
wifstream ScanFile, DBSearchFile;
wofstream ResultFile;
ScanFile.open("ScanFile.txt", ios_base::binary);
ResultFile.open("ResultFile.txt", ios::out, ios_base::binary);
if (ScanFile.is_open())
{
while (!ScanFile.eof())
{
DBSearchFile.open("DBSearchFile.txt", ios_base::binary);
if (!DBSearchFile.is_open())
{
cout << "Error open DBSearchFile.txt" << "\n";
break;
}
getline(ScanFile, ScanLine);
wcout << "Scan line is - " << ScanLine << "\n";
while (!DBSearchFile.eof())
{
getline(DBSearchFile, DBSearchLine);
wcout << "DBSearchLine is -" << DBSearchLine << "\n";
if ((offset = ScanLine.find(DBSearchLine, 0)) != string::npos)
{
ResultFile << ScanLine << L"\n";
}
}
DBSearchFile.close();
}
ScanFile.close();
}
else
{
cout << "Error open ScanFile.txt" << "\n";
}
system("PAUSE");
return 0;
}

Answer 1

#include <iostream>
#include <fstream>
#include <string>
#include <locale>
#include <codecvt>
using namespace std;
int main()
{
/* via http://stackoverflow.com/a/5105192/4005233
changes the encoding of the console and all subsequently opened
files */
std::locale::global(std::locale(""));
wifstream ScanFile;
ScanFile.open("ScanFile.txt", ios_base::binary);
if (!ScanFile.is_open()) {
cout << "Error open ScanFile.txt" << "\n";
return 1;
}
wofstream ResultFile("ResultFile.txt", ios::out);
while (!ScanFile.eof())
{
wifstream DBSearchFile;
DBSearchFile.open("DBSearchFile.txt", ios_base::binary);
if (!DBSearchFile.is_open())
{
cout << "Error open DBSearchFile.txt" << "\n";
break;
}
wstring ScanLine;
getline(ScanFile, ScanLine);
wcout << "Scan line is - " << ScanLine << "\n";
do
{
wstring DBSearchLine;
getline(DBSearchFile, DBSearchLine);
// have all lines been read?
if(!DBSearchLine.length())
break;
wcout << "DBSearchLine is -" << DBSearchLine << "\n";
if (ScanLine.find(DBSearchLine, 0) != string::npos)
{
ResultFile << ScanLine << L"\n";
break; // found a match, no need to search further
}
}while(1);
DBSearchFile.close();
}
ScanFile.close();
return 0;
}
This was tested using files with and without a BOM.
The innermost loop had to be changed to handle files with a newline character at the end; if I hadn't done that it would have match with an empty string which is always true.
(I've also changed a few other things according to my coding style, the important change is the one right at the top)