Clearing buffer while reading in binary data from a server in C++ - c++

I have a server that sends raw binary data to print a "map" that a user must traverse through, however, I am having trouble clearing out my buffer after each line read and thus keep getting residual data printed at the end of the shorter lines. In the screenshot below you can see my output on the left, and what the output should be on the right. What is the best way to solve this? I feel like I am missing something but cant seem to find a solution.
And the code that is reading/printing this is below:
char* mapData = NULL;
string command = "command> ";
size_t dataSize = 0;
while(mapData != command.c_str()) {
unsigned char* buffer = (unsigned char*) &dataSize;
connection = read(mySocket, buffer, 8);
if(connection == -1 || connection < 0) {
cerr << "**Error: could not read text size" << endl;
return 1;
}
mapData = (char*)malloc(dataSize);
buffer = (unsigned char*) mapData;
while((connection = read(mySocket, buffer, dataSize)) != -1) {
if(connection == -1 || connection < 0) {
cerr << "**Error: could not read text size" << endl;
return 1;
}
if(dataSize != 1) {
cout << buffer;
}
free(buffer);
buffer = NULL;
}
}

You are ignoring the return value of read() to know how many bytes are in the buffer.
read() returns the actual number of bytes that were read, which may be fewer than you requested. So you need to call read() in a loop until you have read all of the bytes you are expecting, eg:
int readAll(int sock, void *buffer, size_t buflen)
{
unsigned char* pbuf = reinterpret_cast<unsigned char*>(buffer);
while (buflen > 0) {
int numRead = read(sock, pbuf, buflen);
if (numRead < 0) return -1;
if (numRead == 0) return 0;
pbuf += numRead;
buflen -= numRead;
}
return 1;
}
Also, after reading the buffer, you are treating it as if it were null-terminated, but it is not, which is why you get extra garbage in your output.
More importantly, mapData != command.c_str() will ALWAYS be true, so your while loop iterates indefinitely (until a socket error occurs), which is not what you want. You want the loop to end when you receive a "command> " string instead.
mapData is initially NULL, and c_str() NEVER returns NULL, so the loop ALWAYS iterates at least once.
Then you allocate and free mapData but don't reset it to NULL, so it is left pointing at invalid memory. Which doesn't really matter, since your while loop is just comparing pointers. c_str() will NEVER return a pointer to memory that mapData ever points to.
To end your loop correctly, you need to compare the contents of mapData after reading, not compare its memory address.
Try this instead:
char *mapData = NULL;
uint64_t dataSize = 0;
const string command = "command> ";
bool keepLooping = true;
do {
if (readAll(mySocket, &dataSize, sizeof(dataSize)) <= 0) {
cerr << "**Error: could not read text size" << endl;
return 1;
}
if (dataSize == 0)
continue;
mapData = new char[dataSize];
if (readAll(mySocket, mapData, dataSize) <= 0) {
cerr << "**Error: could not read text" << endl;
delete[] mapData;
return 1;
}
cout.write(mapData, dataSize);
keepLooping = (dataSize != command.size()) || (strncmp(mapData, command.c_str(), command.size()) != 0);
delete[] mapData;
}
while (keepLooping);
Alternatively:
string mapData;
uint64_t dataSize = 0;
const string command = "command> ";
do {
if (readAll(mySocket, &dataSize, sizeof(dataSize)) <= 0) {
cerr << "**Error: could not read text size" << endl;
return 1;
}
mapData.resize(dataSize);
if (dataSize > 0) {
if (readAll(mySocket, &mapData[0], dataSize) <= 0) {
cerr << "**Error: could not read text" << endl;
return 1;
}
cout << mapData;
}
}
while (mapData != command);

like #eozd pointed out, calling malloc and free in your loop is a bad idea since you use return statements. Your code may leak memory. You should ensure you call free before returns. Even better, you could declare your buffer outside of while loop, and use break instead of return, and call free if there was en error
Looking at your solution, it seems that the communication protocol involves sending data size first, followed by the actual data. How is data size written to the wire? You may need to convert it from network byte order.
To debug, you could print out the value of dataSize before every read to make sure that it is what you expect

You should clear the buffer too. Add:
memset(mapData, 0, dataSize);
after the malloc.

Related

Socket Communication Data Corruption on Write/Read

I've got a C++ server that communicates with multiple clients. It uses a vector to store the handles to the sockets for those clients (playerSockets in the code below). At the end of the "game" I want the server to loop through that vector and write the same string to each client. However, sometimes the data that the client reads (and then displays) is "corrupted" as you can see in the screenshot, but this doesn't happen for the first client, only the second. I can't figure out why this is happening! I use this same technique (looping and writing) earlier in the program and it always works fine in that instance.
Here is what it is supposed to be shown:
Here and here's what I get:
Here is the server code that writes:
std::string announcement = "";
if (playerWon) {
...
}
} else {
announcement = "?No one won the game!\nGAME BOARD: " + cn.getGameBoard();
for (int player : gameData->playerSockets) {
write(player, announcement.c_str(), announcement.size() + 1);
}
}
And here's the client code that reads. Keep in mind that more than one client is running and connected to the server, and this issue only happens with a client OTHER THAN the first client in the server's loop:
static bool readMyTurn(int clientSd) {
...
char buf[BUFSIZE];
read(clientSd, buf, BUFSIZE);
string myTurn(buf);
cout << "MYMYMYMY: " << myTurn << endl;
myTurn.erase(0, 1);
cout << myTurn << endl;
...
}
UPDATE
Here is my current code to read until encountering the null-terminator character.
string readOneStringFromServer(int clientSd, string &leftovers) {
ssize_t nullTerminatorPosition = 0;
std::string stringToReturn = "";
do {
char buf[BUFSIZE];
ssize_t bytesRead = read(clientSd, buf, BUFSIZE);
nullTerminatorPosition = findPositionOfNullTerminator(buf, bytesRead);
// found a null terminator
if (nullTerminatorPosition != -1) {
// create a buffer to hold all of the chars from buf1 up to and including the null terminator
char upToNullTerminator[nullTerminatorPosition + 1];
// get those chars from buf1 and put them into buf2 (including the null terminator)
for (int i = 0; i < nullTerminatorPosition + 1; ++i) {
upToNullTerminator[i] = buf[i];
}
// use buf2 to create a string
stringToReturn += upToNullTerminator;
// check if there are leftover bytes after the null terminator
int leftoverBytes = bytesRead - nullTerminatorPosition - 1;
if (leftoverBytes != 0) {
// if there are, create a char array of that size
char leftoverChars[leftoverBytes];
// loop through buf1 and add the leftover chars to buf3
for (int i = nullTerminatorPosition + 1; i < bytesRead; ++i) {
leftoverChars[i - (nullTerminatorPosition + 1)] = buf[i];
}
// make a string out of those leftover chars
leftovers = leftoverChars;
} else {
// if there are no leftover bytes, then we want to "erase" what is currently held in leftovers so that
// it doesn't get passed to the next function call
leftovers = "";
}
// didn't find one
} else {
stringToReturn += buf;
}
} while (nullTerminatorPosition == -1);
return stringToReturn;
}

Sub-functions to send and receive string over socket

I assume that for messages that are of only 1 byte (a char), I will use read() and write() directly.
For those messages having size > 1 bytes, I use two subfunctions to read and write them over sockets.
For example, I have the server construct a string called strcities (list of city) and print it out --> nothing strange. Then send the number of bytes of this string to the client, and then the actual string.
The client will first read the number of bytes, then the actual city list.
For some reason my code sometimes work and sometimes doesn't. If it works, it also prints out some extra characters that I have no idea where they come from. If it doesn't, it hangs and forever waits in the client, while the server goes back to the top of the loop and wait for next command from the client. Could you please take a look at my codes below and let me know where I did wrong?
Attempt_read
string attempt_read(int rbytes) { // rbytes = number of bytes of message to be read
int count1, bytes_read;
char buffer[rbytes+1];
bool notdone = true;
count1 = read(sd, buffer, rbytes);
while (notdone) {
if (count1 == -1){
perror("Error on write call");
exit(1);
}
else if (count1 < rbytes) {
rbytes = rbytes - count1; // update remaining bytes to be read
count1 = read(sd, buffer, rbytes);
}
else {notdone = false;}
} // end while
string returnme;
returnme = string(buffer);
return returnme;
}
Attempt_write
void attempt_write(string input1, int wbytes) { // wbytes = number of bytes of message
int count1;
bool notdone = true;
count1 = write(sd, input1.c_str(), wbytes);
while (notdone) {
if (count1 == -1){
perror("Error on write call");
exit(1);
}
else if (count1 < wbytes) {
wbytes = wbytes - count1;
count1 = write(sd, input1.c_str(), wbytes);
}
else {notdone = false;}
} // end while
return;
}
1) string class has a method size() that will return the length of the string, so you do not actually need a second attempt_write parameter.
2) You can transfer length of message before message or you can transfer a terminating 0 after, if you only will sent an ASCII strings. Because your connection could terminate at any time, it is better to send exact length before sending the string, so your client could know, what to expect.
3) What compilator do you use, that would allow char buffer[rbytes+1]; ? A standard c++ would require char buffer = new char[rbytes+1]; and corresponding delete to avoid a memory leaks.
4) In your code, the second read function call use same buffer with no adjustment to length, so you, practically, overwrite the already received data and the function will only work, if all data will be received in first function call. Same goes for write function
I would suggest something like this:
void data_read(unsigned char * buffer, int size) {
int readed, total = 0;
do {
readed = read(sd, buffer + total, size - total);
if (-1 == writted) {
perror("Error on read call");
exit(1);
}
total += readed;
} while (total < size);
}
string attempt_read() {
int size = 0;
data_read((unsigned char *) &size, sizeof(int));
string output(size, (char) 0x0);
data_read((unsigned char *) output.c_str(), size);
return output;
}
void data_write(unsigned char * buffer, int size) {
int writted, total = 0;
do {
writted = write(sd, buffer + total, size - total);
if (-1 == writted) {
perror("Error on write call");
exit(1);
}
total += writted;
} while (total < size);
}
void attempt_write(string input) {
int size = input.size();
data_write((unsigned char *) &size, sizeof(int));
data_write((unsigned char *) input.c_str(), size);
}

InternetReadFile not getting entire file

I have the following code to download some rss files from servers, but so far I'm just getting incomplete version of my rss file.(?) The code is as follows -
#include<iostream>
#include<conio.h>
#include<stdio.h>
#include<string>
#include<cstring>
#include<wininet.h>
using namespace std;
const int _SIZE = 307200;
int WEB_GET_DATA(char* WEB_URL){
HINTERNET WEB_CONNECT = InternetOpen("Default_User_Agent",INTERNET_OPEN_TYPE_PRECONFIG,NULL, NULL, 0);
if(!WEB_CONNECT){
cout<<"Connection Failed or Syntax error";
return 0;
}
HINTERNET WEB_ADDRESS = InternetOpenUrl(WEB_CONNECT,WEB_URL, NULL, 0, INTERNET_FLAG_KEEP_CONNECTION, 0);
if(!WEB_ADDRESS){
cout<<"ERROR...\n";
return 0;
}
char _DATA_RECIEVED[_SIZE];
DWORD NO_BYTES_READ = 0;
while(InternetReadFile(WEB_ADDRESS,_DATA_RECIEVED,_SIZE,&NO_BYTES_READ)&&(NO_BYTES_READ)){
cout<<_DATA_RECIEVED;
}
InternetCloseHandle(WEB_ADDRESS);
InternetCloseHandle(WEB_CONNECT);
return 0;
}
int main(){
WEB_GET_DATA("http://themoneyconverter.com/rss-feed/AED/rss.xml");
getch();
return 0;
}
I'm getting only almost half of my file and not from start but my output is seeming to be starting from somewhere in between the file and then to it's end.
So where I'm going wrong? I checked that my rss file is at least gonna be 30kb large. So I have given the _SIZE const 307200 (300kb) and still it is not working? Please help me.
Try this instead:
int WEB_GET_DATA(char* WEB_URL)
{
HINTERNET WEB_CONNECT = InternetOpen("Default_User_Agent", INTERNET_OPEN_TYPE_PRECONFIG, NULL, NULL, 0);
if (!WEB_CONNECT)
{
cout << "Connection Failed or Syntax error" << endl;
return 0;
}
HINTERNET WEB_ADDRESS = InternetOpenUrl(WEB_CONNECT, WEB_URL, NULL, 0, INTERNET_FLAG_KEEP_CONNECTION, 0);
if (!WEB_ADDRESS)
{
cout << "ERROR..." << endl;
InternetCloseHandle(WEB_CONNECT);
return 0;
}
DWORD DATA_SIZE = _SIZE;
char *_DATA_RECIEVED = new char[DATA_SIZE];
DWORD NO_BYTES_READ = 0;
do
{
if (InternetReadFile(WEB_ADDRESS, _DATA_RECIEVED, DATA_SIZE, &NO_BYTES_READ))
{
if (NO_BYTES_READ == 0)
break;
cout << string(_DATA_RECIEVED, NO_BYTES_READ);
}
else
{
if (GetLastError() != ERROR_INSUFFICIENT_BUFFER)
{
cout << "Read error" << endl;
break;
}
delete[] _DATA_RECIEVED;
DATA_SIZE += _SIZE;
_DATA_RECIEVED = new char[DATA_SIZE];
}
}
while (true);
InternetCloseHandle(WEB_ADDRESS);
InternetCloseHandle(WEB_CONNECT);
return 0;
}
char buffer[200000];
DWORD bytes_read = 0;
DWORD currbytes_read;
do
{
bRead = InternetReadFile(file_handle, buffer + bytes_read, 200000 - bytes_read, &currbytes_read);
bytes_read += currbytes_read;
} while (bRead && currbytes_read);
buffer[bytes_read] = 0;
First of all, the problem you are having is that you are overwriting the same buffer and you are not clearing the data before each call of InternetReadFile. You also have not cleared the buffer before your first call. You are then throwing a potentially garbled mess of string and memory into a cout. This is very bad.
A quick fix would be to do this:
BYTE _DATA_RECIEVED[_SIZE]; // BYTE is a char, but its clearer now its not guaranteed to be a string!
BOOL ret = TRUE;
DWORD NO_BYTES_READ = 0;
while(ret){
memset(_DATA_RECIEVED, 0, _SIZE); // clear the buffer
ret = InternetReadFile(WEB_ADDRESS,_DATA_RECIEVED,_SIZE,&NO_BYTES_READ);
if(NO_BYTES_READ > 0)
cout<<_DATA_RECIEVED;
}
This is not the most elegant way of doing it (far from it), but at least you should get the data you expect back.
Remember, InternetReadFile passes back a buffer of data, not necessarily a string! It could be an image, junk, and even if it is a string, in your case, it wont have a null byte to close it off. InternetReadFile reads raw bytes, NOT text.
A more elegant solution might start like this:
std::string resultRss;
BYTE _DATA_RECIEVED[_SIZE];
DWORD NO_BYTES_READ = 0;
while(InternetReadFile(WEB_ADDRESS,_DATA_RECIEVED,_SIZE,&NO_BYTES_READ)){
resultRss.append((char*)_DATA_RECIEVED, NO_BYTES_READ); //doesn't matter about null-byte because we are defining the number of bytes to append. This also means we don't NEED to clear the memory, although you might want to.
}
//output final result
cout << resultRss;
Also, as a commenter added, you need to lay off the ALLCAPS for variables.
Hope this helps.

Error in receiving file

I have this code:
while (1) {
char buffer[4096];
memset(buffer,0,4096);
int bytes_read = recv(client_fd, buffer, sizeof(buffer),0);
if (bytes_read == 0)
break;
if (bytes_read < 0) {
std::cout<< "Error "<<endl;
}
void *p = buffer;
int dest;
dest=open("/root/hello.txt",O_WRONLY);
while (bytes_read > 0) {
int bytes_written = send(dest, buffer, bytes_read,0);
if (bytes_written <= 0) {
std::cout<< "Error2 "<<endl;
}
bytes_read -= bytes_written;
p += bytes_written;
}
}
I receive the file correctly.
The program goes in loop where the cout is "Error2", so the send returns -1.
The problem is to write the file in a new file descriptor, in this case the variable called dest.
How can I solve this problem?
According to the manual-page for send(2), it can only write to a socket, not to a regular file; if you had checked errno (using e.g. perror), you would have seen that it gets set to ENOTSOCK. You should always see what error is being set, otherwise debugging is just shooting in the dark.
Also, since this is C++, I really think you should use C++-style I/O (std::ofstream and so on) to write to a regular file.
Check the return value of
dest=open("/root/hello.txt",O_WRONLY);
It should be a positive integer; everything else indicates an error. My guess is that the file can't be opened for whatever reason.

unix domain stream sockets sending more data then it should be

I have two simple programs set up that share data through a unix domain socket. One program reads data out of a Queue and sends it to the other application. Before it is sent each piece of data is front-appended by four bytes with the length, if it is less then four bytes the left over bytes are the '^' symbol.
The client application then reads the first four bytes, sets a buffer to the appropriate size and then reads the rest. The problem that I'm having is that the first time through the message will be sent perfectly. Every other time after that there is extra data being sent so a message like "what a nice day out" would come out like "what a nice day out??X??". So I feel like a buffer is not being cleared correctly but I can't seem to find it.
Client code:
listen(sock, 5);
for (;;)
{
msgsock = accept(sock, 0, 0);
if (msgsock == -1)
perror("accept");
else do
{
char buf[4];
bzero(buf, sizeof(buf));
if ((rval = read(msgsock, buf, 4)) < 0)
perror("reading stream message");
printf("--!%s\n", buf);
string temp = buf;
int pos = temp.find("^");
if(pos != string::npos)
{
temp = temp.substr(0, pos);
}
int sizeOfString = atoi(temp.c_str());
cout << "TEMP STRING: " << temp << endl;
cout << "LENGTH " << sizeOfString << endl;
char feedWord[sizeOfString];
bzero(feedWord, sizeof(feedWord));
if ((rval = read(msgsock, feedWord, sizeOfString)) < 0)
perror("reading stream message");
else if (rval == 0)
printf("Ending connection\n");
else
printf("-->%s\n", feedWord);
bzero(feedWord, sizeof(feedWord));
sizeOfString = 0;
temp.clear();
}
while (rval > 0);
close(msgsock);
}
close(sock);
unlink(NAME);
Server Code
pthread_mutex_lock(&mylock);
string s;
s.clear();
s = dataQueue.front();
dataQueue.pop();
pthread_mutex_unlock(&mylock);
int sizeOfString = strlen(s.c_str());
char sizeofStringBuffer[10];
sprintf(sizeofStringBuffer, "%i", sizeOfString);
string actualString = sizeofStringBuffer;
int tempSize = strlen(sizeofStringBuffer);
int remainder = 4 - tempSize;
int x;
for(x =0; x < remainder; x++)
{
actualString = actualString + "^";
}
cout << "LENGTH OF ACTUAL STRING: " << sizeOfString << endl;
actualString = actualString + s;
cout << "************************" << actualString << endl;
int length = strlen(actualString.c_str());
char finalString[length];
bzero(finalString, sizeof(finalString));
strcpy(finalString, actualString.c_str());
if (write(sock, finalString, length) < 0)
perror("writing on stream socket");
Rather than padding your packet length with '^', you'd be far better off just doing:
snprintf(sizeofStringBuffer, 5, "%04d", sizeOfString);
so that the value is 0 padded - then you don't need to parse out the '^' characters in the receiver code.
Please also edit out your debug code - there's only one write() in the current code, and it doesn't match your description of the protocol.
Ideally - split your sending routine into a function of its own. You can also take advantage of writev() to handle coalescing the string holding the "length" field with the buffer holding the actual data and then sending them as a single atomic write().
Untested code follows:
int write_message(int s, std::string msg)
{
struct iovec iov[2];
char hdr[5];
char *cmsg = msg.c_str();
int len = msg.length();
snprintf(hdr, 5, "%04d", len); // nb: assumes len <= 9999;
iov[0].iov_base = hdr;
iov[0].iov_len = 4;
iov[1].iov_base = cmsg;
iov[1].iov_len = len;
return writev(s, iov, 2);
}
You have to check return values of both write and read not only for -1 but for short (less then requested) writes/reads. You also seem to just continue after printing an error with perror - do an exit(2) or something there.
Two things:
First - on the Server side you are writing off the end of your array.
char finalString[length];
bzero(finalString, sizeof(finalString));
strcpy(finalString, actualString.c_str());
The strcpy() will copy length+1 characters into finalString (character pull the null terminator).
Second (and most likely to be the problem) - on the client side you are not null terminating the string you read in, therefore the printf() will print your string, and then whatever is on the stack up to the point it hits a null.
Increase both buffers by one, and you should be in better shape.