How to put UTF-8 text into std::string through linux sockets - c++

Question

I made a simple C++ server program, which works just fine as long as I use it with simple tools like telnet, however when I use for example .Net (C#) that would connect to it and send it some strings, the text is somewhat corrupted. I tried multiple encodings on C# side, and only result was that it was corrupted in a different way.
I belive that main problem is in this function that is meant to read a line of text from socket:
std::string Client::ReadLine()
{
std::string line;
while (true)
{
char buffer[10];
read(this->Socket, buffer, 9);
int i = 0;
while (i < 10)
{
if (buffer[i] == '\r')
{
i++;
continue;
}
if (buffer[i] == '\0')
{
// end of string reached
break;
}
if (buffer[i] == '\n')
{
return line;
}
line += buffer[i];
i++;
}
}
return line;
}
This is a simple output of program into terminal, when I send it string "en.wikipedia.org" using telnet I see:
Subscribed to en.wikipedia.org
When I use C# that open a stream writer using this code
streamWriter = new StreamWriter(networkStream, Encoding.UTF8);
I see:
Subscribed to en.wiki,pedia.org,
When I use it without UTF-8 (so that default .net encoding is used, IDK what it is)
streamWriter = new StreamWriter(networkStream);
I see:
Subscribed to en.wiki�pedia.org�
However, in both cases it's wrong. What's a most simple way to achieve this, using just standard C++ and linux libraries? (no boost etc - I can do this using some framework, like Qt, boost etc, but I would like to understand this). Full code #http://github.com/huggle/XMLRCS

Answer 1

A UTF-8 string is just a series of single bytes, basically just wnat std::string is supposed to handle. You have two other problems:
The first is that you don't actually check ho many characters was actually read, you always loop over ten characters. Since you don't loop over the actual number of characters read (and don't check for error or end of connection) you might read data in the buffer beyond what was written by read and you have undefined behavior.
The second problem is kind of related to the first, and that is that you have a buffer of ten characters, you read up to nine characters into the buffer, and then loop over all ten characters in the buffer. The problem with this is that since you only read up to nine characters, the tenth character will always be uninitialized. Because the tenth entry in the buffer is always uninitialized, its value will be indeterminate and reading it will again lead to undefined behavior.