I'm using a TCP library with a Server and Client class called Lacewing http://lacewing-project.org/
I have noticed that when I send a client several separate messages, it sometimes bundles them and the only way I can separate them is by parsing them again. When sending binary data, it is very hard to parse though.
Does anyone know why it might do this? I tried the DisableNagling() on both client and server but it still does it.
Here is an example of something I might send:
void ServerCore::loginRequestC( const std::string& userName, const std::string& password )
{
std::cout << "Got request" << std::endl;
ServerPlayer* player = (ServerPlayer*)getServerClient()->Tag;
player->setUsername(userName);
for (size_t i = 0; i < m_players.size(); ++i)
{
if(m_players[i] != player)
{
std::cout << "Telling new about " << m_players[i]->getUsername() << std::endl;
m_enc.playerJoinedS(m_players[i]->getUsername());
player->getClient()->Send(m_enc.getLastMessage().c_str());
}
}
m_enc.loginResultS(true,"");
player->getClient()->Send(m_enc.getLastMessage().c_str());
m_enc.playerJoinedS(userName);
for (size_t i = 0; i < m_players.size(); ++i)
{
if(m_players[i] != player)
m_players[i]->getClient()->Send(m_enc.getLastMessage().c_str());
}
}
So if I intended to have:
MSG_A
MSG_B
MSG_C
It might send:
MSG_A
MSG_BMSG_C
The messages get randomly bundled together. It is not my code that is the problem. I have checked.
Although it is useful to bundle messages, I want to control when it happens, not the library.
Thanks
This is not the library, but the TCP protocol itself that is doing it. TCP socket is a bi-directional stream of bytes. One write might result in multiple reads on the other side, and vise versa.
You have to design you application-level protocol so that it's easy to split that stream into messages. Common approaches are to use a length prefix or a message delimiter.
TCP is stream oriented protocol, thus it has no build in message boundaries. You will need to split the stream to messages by yourself.
UDP in opposite is datagram protocol and each packet has a separate message. A disadvantage is that it's not reliable.
Related
I am trying to implement some keep-alive service in UDP using BOOST::ASIO, these are the general steps:
Sending keep-alives to 2 processes on the same machine, they are listening on the same ip with a different port.
Loop to send async_send_to to both, and the callback is a function that calls async_receive_from with a callback F().
Both refer to the same endpoint and data buffers.
while loop with io_service.run_one() inside.
The processes reply immediately.
The issue is that sporadically I either get the 2 differing ports when I check the endpoints' ports (the wanted case) F() runs, or, I get twice the same port.
It seems as the endpoint buffer (and probably the data) is getting overwritten by the later packet.
I was thinking the since I'm using run_one() the packets should be processed one by one and there will be no overwriting.
Initial send -
void GetInstancesHeartbeat(udp::endpoint &sender_endpoint)
{
int instanceIndex = 0;
for (; instanceIndex <= amountOfInstances ; instanceIndex++)
{
udp::endpoint endpoint = udp::endpoint(IP, Port+ instanceIndex);
m_instancesSocket->async_send_to(
boost::asio::buffer((char*)&(message),
sizeof(message)),endpoint,
boost::bind(&ClusterManager::handle_send_to_instance,
this, boost::asio::placeholders::error,
boost::asio::placeholders::bytes_transferred,
sender_endpoint));
}
}
Then the handler -
void handle_send_to_instance(const boost::system::error_code& error, size_t
bytes_recvd, udp::endpoint &sender_endpoint)
{
m_instancesSocket->async_receive_from(
boost::asio::buffer(m_dataBuffer, m_maxLength), m_endpoint,
boost::bind(&ClusterManager::handle_receive_from_instance, this,
boost::asio::placeholders::error,
boost::asio::placeholders::bytes_transferred,
sender_endpoint));
}
While loop -
while(true){
io_service.run_one();
}
And the handle receive where the port results twice the same -
void handle_receive_from_instance(const boost::system::error_code& error, size_t
bytes_recvd, udp::endpoint&sender_endpoint)
{
if (!error && bytes_recvd > 0)
{
int instancePort = m_endpoint.port();
} else {
//PRINT ERROR
}
}
The actual operations are asynchronous, so there's no telling when the endpoint reference gets written to. That's the nature of asynchronous calls.
So, what you need to have is an endpoint receiving variable per asynchronous call (you might store it per instance index).
There are a number of other really suspicious bits:
what's the type of message? For most types you'd write just boost::asio::buffer(message) (which deals with T [], std::vector<T>, array<T> etc). This works when T is char or any POD type.
If message is actually a struct of some type, consider using a single-element array to avoid having to dangerous casting:
Live On Coliru
POD message[1] = {pod};
s.async_send_to(boost::asio::buffer(message), udp::endpoint{{}, 6767}, [](boost::system::error_code ec, size_t transferred) {
std::cout << "Transferred: " << transferred << " (" << ec.message() << ")\n";
});
(Sends 12 bytes on a typical system).
Whatever you do, don't write the unsafe C-style cast (Why use static_cast<int>(x) instead of (int)x?).
You have while(true) { io.run_one(); } which is an infinite loop. A better way to write it would be: while(io.run_one()) {}
However, that would basically be the same as io.run();, but less correctly and less efficiently (see https://www.boost.org/doc/libs/1_68_0/boost/asio/detail/impl/scheduler.ipp line 138), so why not use it?
I'm trying to synchronise 4 clients to one server. I want to send a message to the server when the client is ready to move on, then the server counts how many requests it gets and sends a message back to the clients to say it's ready.
What I've done so far is use REQ/REP:
while(1){
int responses = 0;
while(responses<numberOfCameras){
for(int i=0; i<numberOfCameras;i++){
cout<<"waiting"<<endl;
if(sockets[i]->recv(requests[i], ZMQ_NOBLOCK)){
responses ++;
cout<<"rx"<<endl;
}
}
}
for(int i=0; i<numberOfCameras;i++){
cout<<"tx"<<endl;
sockets[i]->send("k",1);
cout<<"Sent"<<endl;
}
}
With more than one camera, this produces the expected error:
Operation cannot be accomplished in current state
Because it cannot do anything until it's replied to the REQ, right?
How can I modify this to work with multiple clients?
EDIT:
I have attempted to implement a less strict REQ REP with PUSH PULL. The meat is:
Server:
while(1){
int responses = 0;
while(responses<numberOfCameras){
for(int i=0; i<numberOfCameras;i++){
cout<<"waiting"<<endl;
if(REQSockets[i]->recv(requests[i], ZMQ_NOBLOCK)){
responses ++;
cout<<"rx"<<endl;
}
}
}
boost::this_thread::sleep(boost::posix_time::milliseconds(200));
for(int i=0; i<numberOfCameras;i++){
cout<<"tx"<<endl;
REPSockets[i]->send("k",1);
cout<<"Sent"<<endl;
}
boost::this_thread::sleep(boost::posix_time::milliseconds(200));
}
Clients:
for (;;) {
std::cout << "Requesting permission to capture"<< std::endl;
REQSocket.send ("?", 1);
// Get the reply.
zmq::message_t reply;
REPSocket.recv (&reply);
std::cout << "Grabbed a frame" << std::endl;
boost::this_thread::sleep(boost::posix_time::seconds(2));
}
I have outputted all of the ports and addresses to check that they're set right.
The server program hangs with the output:
...
waiting
rx
tx
This means that the program is hanging on the send, but I can't see for the life of me why
EDIT 2:
I have made a github repo with a compilable example and linux makefile and converted to use REP REQ again. The issue is that the client doesn't accept the message from the server, but again, I don't know why.
The answer was to use two REP REQ sockets as in edit 2. I had made a stupid typo for "REQ" instead of "REP" in one of the variable usages and hadn't noticed. I was therefore connecting and then binding the same socket.
I will leave the github repo up as I think the question is long enough already.
I am writing a program to send images captured from an OpenCV window over a TCP connection, using Qt libraries to setup the connections etc.
I have to functions (below) which are both working to send either text or a byte array. The problem I have is at the other end how can I tell if the data coming in is plain text, or an array containing an image. Is there an inbuilt way to do this, or do I need to put a byte at the start of the data to tell the receiver what data is coming? I already put the array length at the start of the serialized image data.
void Screenshot_controller::sendText(std::string textToSend)
{
if(connectionMade)
{
std::string endLine = "\r\n";
textToSend = textToSend + endLine;
const char * textChar = textToSend.c_str();
sendSocket->write(textChar);
sendSocket->flush();
qDebug() << "Text Sent from Server";
}
}
void Screenshot_controller::sendData(QByteArray dataToSend)
{
if(connectionMade)
{
sendSocket->write(dataToSend);
sendSocket->flush();
qDebug() << "Data Sent from Server";
}
}
You need to define the protocol yourself, whether that's with a byte, string, JSON header or any other method. The Tcp socket will allow you to transfer the data, but doesn't care what that data is; it's up to you to handle that.
I would like to connect to a listening server and transmit some data. I looked at the examples available but they seem to have extra functions that do not seem very helpful to me (i.e. connect, fortune, etc.). This is the code I have so far:
QTcpSocket t;
t.connectToHost("127.0.0.1", 9000);
Assuming the server is listening and robust, what do I need to implement to send a data variable with datatype QByteArray?
very simple with QTcpSocket. Begin as you did...
void MainWindow::connectTcp()
{
QByteArray data; // <-- fill with data
_pSocket = new QTcpSocket( this ); // <-- needs to be a member variable: QTcpSocket * _pSocket;
connect( _pSocket, SIGNAL(readyRead()), SLOT(readTcpData()) );
_pSocket->connectToHost("127.0.0.1", 9000);
if( _pSocket->waitForConnected() ) {
_pSocket->write( data );
}
}
void MainWindow::readTcpData()
{
QByteArray data = pSocket->readAll();
}
Be aware, though, that for reading from the TcpSocket you may receive the data in more than one transmission, ie. when the server send you the string "123456" you may receive "123" and "456". It is your responsibility to check whether the transmission is complete. Unfortunately, this almost always results in your class being stateful: the class has to remember what transmission it is expecting, whether it has started already and if it's complete. So far, I haven't figured out an elegant way around that.
In my case I was reading xml data, and sometimes I would not get all in one packet.
Here is an elegant solution. WaitForReadyRead could also have a time out in it and
then some extra error checking in case that timeout is reached. In my case I should never
receive an incomplete xml, but if it did happen this would lock the thread up indefinetly
without the timeout:
while(!xml.atEnd()) {
QXmlStreamReader::TokenType t = xml.readNext();
if(xml.error()) {
if(xml.error() == QXmlStreamReader::PrematureEndOfDocumentError) {
cout << "reading extra data" << endl;
sock->waitForReadyRead();
xml.addData(sock->readAll());
cout << "extra data successful" << endl;
continue;
} else {
break;
}
}
...
I'm using Winsock to send commands through Telnet ; but for some reason when I try to send a string, a few characters get dropped occasionally. I use send:
int SendData(const string & text)
{
send(hSocket,text.c_str(),static_cast<int>(text.size()),0);
Sleep(100);
send(hSocket,"\r",1,0);
Sleep(100);
return 0;
}
Any suggestions?
Update:
I checked and the error still occurs even if all the characters are sent. So I decided to change the Send function so that it sends individual characters and checks if they have been sent:
void SafeSend(const string &text)
{
char char_text[1];
for(size_t i = 0; i <text.size(); ++i)
{
char_text[0] = text[i];
while(send(hSocket,char_text,1,0) != 1);
}
}
Also, it drops characters in a peculiar way ; i.e. in the middle of the sentence. E.g.
set variable [fp]exit_flag = true
is sent as
ariable [fp]exit_flag = true
Or
set variable [fp]app_flag = true
is sent as
setrable [fp]app_flag = true
As mentioned in the comments you absolutely need to check the return value of send as it can return after sending only a part of your buffer.
You nearly always want to call send in a loop similar to the following (not tested as I don't have a Windows development environment available at the moment):
bool SendString(const std::string& text) {
int remaining = text.length();
const char* buf = text.data();
while (remaining > 0) {
int sent = send(hSocket, buf, remaining, 0);
if (sent == SOCKET_ERROR) {
/* Error occurred check WSAGetLastError() */
return false;
}
remaining -= sent;
buf += sent;
}
return true;
}
Update:
This is not relevant for the OP, but calls to recv should also structured in the same way as above.
To debug the problem further, Wireshark (or equivalent software) is excellent in tracking down the source of the problem.
Filter the packets you want to look at (it has lots of options) and check if they include what you think they include.
Also note that telnet is a protocol with numerous RFCs. Most of the time you can get away with just sending raw text, but it's not really guaranteed to work.
You mention that the windows telnet client sends different bytes from you, capture a minimal sequence from both clients and compare them. Use the RFCs to figure out what the other client does different and why. You can use "View -> Packet Bytes" to bring up the data of the packet and can easily inspect and copy/paste the hex dump.