ProtocolBuffer can't define data although it is received - c++

I'm trying to use Google's ProtocolBuffer to send/receive data in a server/client architecture. I am able to connect the two with winsock and I am able to send and receive data from ProtocolBuffer with it but there seems to be a problem with deserializing the data on the server (I haven't done the opposite so I can't tell if that works yet)
Here's the code I use to create a sample packet of data to send to the server:
MessageID *message = new MessageID();
message->set_type(MessageID::Type::MessageID_Type_PLAYERDATA);
PlayerData::Vector2 *position = new PlayerData::Vector2();
position->set_x(10.0f);
position->set_y(25.0f);
PlayerData *data = new PlayerData();
data->set_health((UINT32)50);
data->set_allocated_position(position);
auto inventory = data->add_items();
inventory->set_itemid((INT32)1);
inventory->set_count((INT32)5);
message->set_allocated_playerdata(data);
m_pNetworkObject->Send(message);
This is the code that actually sends the data over a TCP connection:
// Serialize to string.
std::string sendData;
data->SerializeToString(&sendData);
// Convert data to const char*
const char* dataToSend = sendData.c_str();
int iResult;
iResult = send( ConnectSocket, dataToSend, data->ByteSize(), 0 );
if (iResult == SOCKET_ERROR)
{
printf("send failed with error: %d\n", WSAGetLastError());
closesocket(ConnectSocket);
WSACleanup();
exit(1);
}
So, this was code that is run on the client. Now, when I receive this on the server I get the following errors from ProtocolBuffer:
Can't parse message of type "MessageID" because it is missing required fields: type
Can't parse message of type "PlayerData" because it is missing required fields: Position, Health
The code on the server that decompiles the winsock data is like this:
// decompile to a proto file and check ID
MessageID* receivedData = new MessageID();
receivedData->ParseFromArray(&recv, strlen(recv));
// check and redeserialize
switch (receivedData->type())
{
case MessageID::Type::MessageID_Type_PLAYERDATA:
{
PlayerData* playerData = new PlayerData();
playerData->ParseFromArray(&recv, strlen(recv));
UsePlayerData(sock, playerData);
break;
}
case MessageID::Type::MessageID_Type_WORLDDATA:
{
WorldData* worldData = new WorldData();
worldData->ParseFromArray(&recv, strlen(recv));
UseWorldData(sock, worldData);
break;
}
}
The weird thing is that, although the error says that the data doesn't contain the Type value it does set it to 1 and the switch statement works. After that when I try to take a look at the Position and Health data everything is 0.
I have no idea what I'm doing wrong. I read most of the ProtocolBuffer tutorials and help but it seems to no avail.
EDIT:
I tried serializing and then deserializing the data in the same application (without the networking involved) and with the following code it worked to do just that:
std::string sendData;
message->SerializeToString(&sendData);
MessageID* receivedData = new MessageID();
receivedData->ParseFromString(sendData);
If I check the data in receivedData it is exactly the same (so, the right values) as in the original object "message"

Your problem is that you are using strlen() to find the length of the message. strlen() just looks for the first zero-valued byte and assumes that's the end. This is the convention for text strings, but does not apply to binary messages like Protocol Buffers. In this case, strlen() is returning a size that is too short and so some fields are missing from the message, but the opposite can happen too -- strlen() can run off the end of the buffer and return a size that is too long, or even crash your program.
So, you need to make sure the actual exact size of the message is communicated from the sender to the receiver.

Related

Winsock 2, condensing variables into a string, sending it out, then recieving it and reading it back

I am writing some code that involves using an inertia cube tracker, that actively changes its yaw pitch and roll (in degrees) and I need to set up a server that reads that information in order to network the info. So far I have created a client and server, but the problem I am having is either to send the information in one chunck then read it back as three and print it, or to specify which send matches with which recieve.
if( currentTrackerH > 0 )
{
int iSendResult1;
int iSendResult2;
int iSendResult3;
char EulerBuffer0[64];
char EulerBuffer1[64];
char EulerBuffer2[64];
showStationData( currentTrackerH, &TrackerInfo,
&Stations[station-1], &data.Station[station-1],
&StationsHwInfo[currentTrackerH-1][station-1],
showTemp);
//send to the server
do{
sprintf(EulerBuffer0, "%f", data.Station[station-1].Euler[0]);
iSendResult1= send(Connection, EulerBuffer0, sizeof(data.Station[station-1].Euler[0]), NULL);
sprintf(EulerBuffer1, "%f", data.Station[station-1].Euler[1]);
iSendResult2= send(Connection, EulerBuffer1, sizeof(data.Station[station-1].Euler[1]), NULL);
sprintf(EulerBuffer2, "%f", data.Station[station-1].Euler[2]);
iSendResult3= send(Connection, EulerBuffer2, sizeof(data.Station[station-1].Euler[2]), NULL);
}while ((iSendResult1 || iSendResult2 || iSendResult3)>0);
//shutdown the socket when there is no more data to send
iSendResult1 = shutdown(Connection, SD_SEND);
if (iSendResult1 == SOCKET_ERROR)
{
printf("shutdown failed with error: %d\n", WSAGetLastError());
closesocket(Connection);
WSACleanup();
return 1;
}
}
}
This is my client side and here I will put my server side. The networks connect and my tracker code works just fine but sending and recieving is where it all gets wonky.
//begin recieving data
char yaw[256];
char pitch[256];
char roll[256];
int iResult1;
int iResult2;
int iResult3;
float fyaw, fpitch, froll;
do{
do {
iResult1= recv(newConnection, yaw,sizeof(yaw),NULL);
} while( iResult1 == 0 );
fyaw = atof(yaw);
do {
iResult2= recv(newConnection, pitch,sizeof(pitch),NULL);
} while( iResult1 == 0 );
fpitch = atof(pitch);
do {
iResult3= recv(newConnection, roll,sizeof(roll),NULL);
} while( iResult1 == 0 );
froll = atof(roll);
printf("(%f,%f,%f)deg \n",
fyaw, fpitch, froll);
}while(1);
my knowledge of c++ is not fantastic and any help would be lovely. Thanks!
There is all kinds of wrong in your code. Let's try to break down and correct misconceptions (I assume you're using TCP.) You are sending buffers of one size, but recv'ing potentially a buffer of another size. sizeof(yaw) which is a float, is not the same as the size of the string representation of this float.
Calling send/recv for individual item is slow. Ideally you would define a simple protocol. A message in this protocol would be a string containing all the values you wish to transmit. You send that message using a single send() On the receiving side you read in the stream of data, and look for specific markers that tell you when you have received a complete message. You then process that message, splitting out the different components into your yaw/pitch/roll variables.
An example of a string message would be: "{yaw=1.34;pitch=2.45;roll=5.67}"
Then on the client you continually read into a buffer your data until you reach the "}" Then you can process this message and parse out the different components.

Determine Data Type on a TCP Socket in Qt

I am writing a program to send images captured from an OpenCV window over a TCP connection, using Qt libraries to setup the connections etc.
I have to functions (below) which are both working to send either text or a byte array. The problem I have is at the other end how can I tell if the data coming in is plain text, or an array containing an image. Is there an inbuilt way to do this, or do I need to put a byte at the start of the data to tell the receiver what data is coming? I already put the array length at the start of the serialized image data.
void Screenshot_controller::sendText(std::string textToSend)
{
if(connectionMade)
{
std::string endLine = "\r\n";
textToSend = textToSend + endLine;
const char * textChar = textToSend.c_str();
sendSocket->write(textChar);
sendSocket->flush();
qDebug() << "Text Sent from Server";
}
}
void Screenshot_controller::sendData(QByteArray dataToSend)
{
if(connectionMade)
{
sendSocket->write(dataToSend);
sendSocket->flush();
qDebug() << "Data Sent from Server";
}
}
You need to define the protocol yourself, whether that's with a byte, string, JSON header or any other method. The Tcp socket will allow you to transfer the data, but doesn't care what that data is; it's up to you to handle that.

Socket Write Failure when sending File

I've been stuck on this issue for awhile where I'm unable to send a file through a socket. I've sent other information just fine using this method, but the problem seems to appear when I try to send a PNG file as a string.
These are the methods I use to to send and receive information:
// Sends a Message to the specified Socket
void Server::sendMessage(int socket, string message)
{
// Write the Message Size to the Socket
send(socket, itoa((message.length() + 1)), sizeof(size_t));
// Wait for Write Confirmation
bool response;
receive(socket, &response, 2);
// Write the Message to the Socket
send(socket, (char*) message.c_str(), message.length() + 1);
// Wait for Write Confirmation
receive(socket, &response, 2);
}
// Receives Message from the specified Socket
string Server::receiveMessage(int socket)
{
// Read the Message Size from the Socket
int size;
receive(socket, &size, sizeof(size_t));
// Send Write Confirmation
send(socket, itoa(true), 2);
// Receive the Message from the Socket
char message[size];
receive(socket, message, size);
// Send Write Confirmation
send(socket, itoa(true), 2);
// Return the Message as a String
string msg(message);
return msg;
}
The send and receive methods are just relays for write and read respectively. I'm only doing error checking in those methods, and it's the send method that's telling me that the write isn't working. In case it matters, this is my send method:
// Sends a Data Packet to the specified Socket
int Server::send(int socket, void* data, int size)
{
// Write the Data to the Socket
int count = write(socket, data, size);
// Make sure the Write Succeeded
if(count == -1)
{
print("$f1Error: $f0Unable to Write to Socket $t1%i$t0\n", socket);
exit(1);
}
return count;
}
I should note that the Server operates as a Thread, therefore the above three functions are static. The Client also contains the same four networking functions.
The command line breaking this happens in a separate static function which I use to handle Clients. Here is the relevant portion of said method:
// Handles each Client with a Thread
void* Server::server_handleClient(void* arg)
{
// Determine the Socket Descriptor
int socket = *((int*) arg);
free(arg);
// Create the Rover
Rover* rover = new Rover();
// Loop Indefinitely
while(true)
{
...
// Take a Picture and Send it
sendMessage(socket, rover -> takePicture());
...
}
// Delete the Rover
delete rover;
// Close the Socket
close(socket);
// Return a Successful Status
return (void*) new int(0);
}
Here you can see that I make use of a method from another class I've created. Here is the takePicture method from the Rover class, which is where I actually grab the picture:
// Takes a Picture and Returns the Photo as a String
inline string Rover::takePicture()
{
// Open the Picture File
ifstream picture;
string filepath = "./Server/Pictures/" + getDirection() + ".png";
picture.open(filepath.c_str());
// Make sure the File Opened
if(!picture.is_open())
return "";
// Read the File into a String Buffer
stringstream buffer;
buffer << picture.rdbuf();
return buffer.str();
}
So in short, the Server gets a picture from the Rover which it then sends to a Client. When I check the contents of the string for the photo, it's all there. All possible photos are reasonable in size (the photo used for testing is 674,962 bytes, and the buffer size sent is 674,963 which is expected).
I've used these methods for sending various messages, and all of that worked fine. I'm able to send strings (Like "Hello World!") and integers just fine.
Is there something that I'm doing wrong? Is the file that I'm trying to send simply too large? Is there some information that I'm missing? I need help...
Edit:
I've made a few changes with a little progress. I made one small change to the sendMessage command. The current problem is that the picture isn't being sent properly.
New sendMessage function:
// Sends a Message to the specified Socket
void Server::sendMessage(int socket, string message, bool data = false)
{
// Write the Message Size to the Socket
send(socket, itoa((message.length() + 1)), sizeof(size_t));
// Wait for Write Confirmation
bool response;
receive(socket, &response, 2);
// Determine the Type of Data to Send
if(data)
{
// Write the Message Data to the Socket
send(socket, (char*) message.data(), message.length() + 1);
}
else
{
// Write the Message to the Socket
send(socket, (char*) message.c_str(), message.length() + 1);
}
// Wait for Write Confirmation
receive(socket, &response, 2);
}
The Client's copy of this function has been updated to match as well.
Now that we're working on getting the PNG file saved, here's the function that deals with that as well:
// Handles each Client with a Thread
void* Client::client_handleServer(void* arg)
{
// Define Socket Variables
int socket = *((int*) arg);
free(arg);
...
// Export the Picture to the Client's Directory
message = receiveMessage(socket);
ofstream picture;
picture.open("./Client/Pictures/Picture.png", std::ifstream::binary);
picture << message;
picture.close();
...
}
Currently you are opening the file in textmode. that means any characters in the files which contain newlines "\n" are converted to new line + carriage returns "\r\n".
Open your file in binary mode, like so
picture.open(filepath.c_str(), std::ifstream::binary);
then it may work.
void Server::sendMessage(int socket, string message)
The problem is right here. Don't use string as a container for binary data. Pass the image around as a byte array. Same applies to this:
string Server::receiveMessage(int socket)
I eventually figured everything out in the long run.
Pictures are binary files, and I was using Strings which use ASCII Characters. The issue with this is that binary data does not always translate to ASCII, and Strings are terminated by null characters, whereas binary data can contain null data within it. Long story short, strings do not work.
To preserve the message handling I had in place, I ended up just converting the binary data to hexadecimal data (0-F) which could be displayed in a String.

Authenticating users on a Qt Server

I am trying to implement an authentication system using C++/QtTcpSocket for a personal project (A Multiplayer Chess Game).
My friend suggested a method for verifying a user but I wanted to ask if there was an easier or better way. Coming from a Python background and mostly doing this project to develop a deeper understanding of C++.
I will post the method my friend suggested and ask for maybe a better solution.
He built it in a kind of pseudo code fashion. The server is mostly built, I am now hoping to implement Authentication
*cheers
void process_packet(PACKET *pkt)
{
switch(pkt->PacketID)
{
case 0: // let's say packet id 0 is the logon packet; packet contents are username and password
{
//let's say packet size is 101 bytes; packet id was already received, so get the other 100 bytes
unsigned char BUFFER[101] = {0}; // i always add an extra byte to the end of the buffer to allow for off-by-one errors ^_^
int result = recv_packet(pkt->cSocket, 100, BUFFER);
if(result <= 0)
return; // connection error; no packet data was received
unsigned char *UserName = BUFFER+0; //+0 is not neccessary, but the username starts at the beginning. just getting the point across.
unsigned char *PassWord = BUFFER+50;
//side note: if we did "unsigned long *blah = BUFFER+4" or something, we would have to make sure the byte order is right. network byte order is BIG ENDIAN
// WINDOWS byte order is LITTLE ENDIAN
result = QueryDatabase("SELECT username, password FROM chess_players WHERE username = '%s'", FILTER_INVALID_CHARS(UserName));
// check result
unsigned char ServerResponse[2] = {0};
if(result['password'] == PassWord)
{
ServerResponse[0] = 1; // packet id will be 1. the next byte can be 1 or 0 to indicate logon success or failure.
ServerResponse[1] = true; // so packet 0x0101 mean logon success, packet 0x0100 means logon failure
send_packet(pkt->cSocket, ServerResponse, 2);
} else {
ServerResponse[0] = 1;
ServerResponse[1] = false;
send_packet(pkt->cSocket, ServerResponse, 2);
}
}
break;
default:
{
// received an unknown packet id; send a packet to the client that indicates an error_status_t
unsigned char *ServerResponse[2] = {0};
ServerResponse[0] = 2; // packet id 2 means server error
ServerResponse[1] = 0; // error code 0 means 'unknown packet id'
send_packet(pkt_cSocket, ServerResponse, 2);
}
break;
}
delete pkt; // must delete pkt, was created with 'new' in get_client_packets()
}
This seems rather C-stylish and not like the Qt way of doing things.
There is no general answer to your question but my suggestions are the following:
Listen to the newConnection() signal of the QTcpServer. Your handler has to call the nextPendingConnection() to get the next client waiting in the queue. The first thing you will do is probably your user authentication.
Once authenticated, you keep the QTcpSocket in your list of active connections.
Take a look at e.g. the fortune client/server examples how to actually write/read packets.
You might also want to look into the stream operators << to serialize your objects. This is much easier and less error prone than the low-level method you posted. ALso, QDataStream will take care of host and network byte orders automatically.
If you have followed the fortune client/server examples, you should have a QTcpServer (Rfserver) with a QThread subclass (Rfdevice, its instance variable is called thread in the following code) that contains a QTcpSocket (listenSocket).
Having said that, in your server class, listen for incoming connections, my setup looks like this:
void Rfserver::incomingConnection(int socketDescriptor){
if(thread){ //if thread exists, there is probably still an open connection
if(thread->listenSocket){//if thread exists and the listenSocket is filled, there is definately an open connection
if(thread->listenSocket->state() == QAbstractSocket::UnconnectedState){
//but alas, it could just be in the unconnected state, if so kill it.
this->disconnect();
thread->terminate();
thread=0;
connected=false;
}//otherwise, do nothing, because the software is happily connected to a device
}
}
if(!thread){ //if no thread exists, we are by no means connected
thread = new rfdevice(socketDescriptor, this); //set up a new thread
//this first connection communicates the string from your socket to the server parent...use it if you want.
connect( thread, SIGNAL(RemoteButton(QString)),this,SLOT(remoteButton(QString)),Qt::BlockingQueuedConnection);
connect( thread, SIGNAL(error(QTcpSocket::SocketError)),this,SLOT(tcpError(QTcpSocket::SocketError)),Qt::AutoConnection);
connect( thread, SIGNAL(finished()), this, SLOT(threadZero())); //I have a threadZero function that deletes all the data then schedules the socket for deletion.
thread->start();
connected=true;
QString *welcome = new QString("Enter your password:\r\n");
echoCommand(welcome); //this is a function you will implement that sends the welcome message to the pending device.
}
}
Okay, so now, when a device tries to connect to the server the device is presented with "Enter your password:\r\n". Your device will respond to this with a password and username perhaps. But the Qt side of things would look like this:
/*
FUNCTION:read
this is a polling runloop that listens for data as long as the socket is connected or connecting. If a
write is ever scheduled, it will be called from this runloop..
*/
void Rfdevice::read(void){
while((listenSocket->state() == QAbstractSocket::ConnectedState) || (listenSocket->state() == QAbstractSocket::ConnectingState)){
//if there is data available to send write it to the socket
if(dataToSend) this->write();
if(listenSocket->waitForReadyRead(50)) readBytes();
//wait for 50ms for data from the device
//if there is ever data available to be read, read it.
}
}
Your device responds with a username/password in the format username---password\r\n. Then the socket does this:
/*
FUNCTION:readBytes
this is like a callback function because it only gets called when there is data available for read.
It basically converts the data to a string.
*/
void Rfdevice::readBytes(void){
QByteArray newData;
newData = listenSocket->readAll();
QString *recieved = new QString(newData);
QStringList userAndPass = recieved.split("---");//this is your delimiter
QString username = userAndPass.at(0);
QString password = userAndPass.at(1);
//NOW, check the username and password vs your SQL or wherever it's saved.
}
The pseudo-code is pretty complete on the particulars. Hopefully you can put it all together! Let me know if you need more code.

Losing characters in TCP Telnet transmission

I'm using Winsock to send commands through Telnet ; but for some reason when I try to send a string, a few characters get dropped occasionally. I use send:
int SendData(const string & text)
{
send(hSocket,text.c_str(),static_cast<int>(text.size()),0);
Sleep(100);
send(hSocket,"\r",1,0);
Sleep(100);
return 0;
}
Any suggestions?
Update:
I checked and the error still occurs even if all the characters are sent. So I decided to change the Send function so that it sends individual characters and checks if they have been sent:
void SafeSend(const string &text)
{
char char_text[1];
for(size_t i = 0; i <text.size(); ++i)
{
char_text[0] = text[i];
while(send(hSocket,char_text,1,0) != 1);
}
}
Also, it drops characters in a peculiar way ; i.e. in the middle of the sentence. E.g.
set variable [fp]exit_flag = true
is sent as
ariable [fp]exit_flag = true
Or
set variable [fp]app_flag = true
is sent as
setrable [fp]app_flag = true
As mentioned in the comments you absolutely need to check the return value of send as it can return after sending only a part of your buffer.
You nearly always want to call send in a loop similar to the following (not tested as I don't have a Windows development environment available at the moment):
bool SendString(const std::string& text) {
int remaining = text.length();
const char* buf = text.data();
while (remaining > 0) {
int sent = send(hSocket, buf, remaining, 0);
if (sent == SOCKET_ERROR) {
/* Error occurred check WSAGetLastError() */
return false;
}
remaining -= sent;
buf += sent;
}
return true;
}
Update:
This is not relevant for the OP, but calls to recv should also structured in the same way as above.
To debug the problem further, Wireshark (or equivalent software) is excellent in tracking down the source of the problem.
Filter the packets you want to look at (it has lots of options) and check if they include what you think they include.
Also note that telnet is a protocol with numerous RFCs. Most of the time you can get away with just sending raw text, but it's not really guaranteed to work.
You mention that the windows telnet client sends different bytes from you, capture a minimal sequence from both clients and compare them. Use the RFCs to figure out what the other client does different and why. You can use "View -> Packet Bytes" to bring up the data of the packet and can easily inspect and copy/paste the hex dump.