The way my game and server work is like this:
I send messages that are encoded in a format I created. It starts with 'p' followed by an integer for the message length then the message.
ex: p3m15
The message is 3 bytes long. And it corresponds to message 15.
The message is then parsed and so forth.
It is designed for TCP potentially only sending only 1 byte (since TCP only has to send a minimum of 8 bits).
This message protocol I created is extremely lightweight and works great which is why I use it over something like JSON or other ones.
My main concern is, how should the client and the server start talking?
The server expects clients to send messages in my format. The game will always do this.
The problem I ran into was when I tested my server on port 1720. There was BitTorrent traffic and my server was picking it up. This was causing all kinds of random 'clients' to connect to my server and sending random garbage.
To 'solve' this, I made it so that the first thing a client must send me is the string "Hello Server".
If the first byte ever sent is != 'H' or if they have sent me > 12 bytes and it's != "Hello Server" then I immediately disconnect them.
This is working great. I'm just wondering if I'm doing something a bit naive or if there are more standard ways to deal with:
-Clients starting communication with server
-Clients passing Hello Server check, but somewhere along the line I get an invalid message. I can assume that my app will never send an invalid message. If it did, it would be a bug. Right now if I detect an invalid message then I disconnect the client.
I noticed BitTorrent was sending '!!BitTorrent Protocol' before each message. Should I do something like that?
Any advice on this and making it safer and more secure would be very helpful.
Thanks
perhaps a magic number field embedded in your message.
struct Message
{
...
unsigned magic_number = 0xbadbeef3;
...
};
so first thing you do after receive something, is checking whether the magic_number field is 0xbadbeef3.
Typically, I design protocols with a header something like this:
typedef struct {
uint32_t signature;
uint32_t length;
uint32_t message_num;
} header_t;
typedef struct {
uint32_t foo;
} message13_t;
Sending a message:
message13_t msg;
msg.foo = 0xDEADBEEF;
header_t hdr;
hdr.signature = 0x4F4C494D; // "MILO"
hdr.length = sizeof(message13_t);
hdr.message_num = 13;
// Send the header
send(s, &hdr, sizeof(hdr), 0);
// Send the message data
send(s, &msg, sizeof(msg), 0);
Receiving a message:
header_t hdr;
char* buf;
// Read the header - all messages always have this
recv(s, &hdr, sizeof(hdr), 0);
// allocate a buffer for the rest of the message
buf = malloc(hdr.length);
// Read the rest of the message
recv(s, buf, hdr.length, 0);
This code is obviously devoid of error-checking or making sure all data has been sent/received.
Related
I have been reading some socket guides such as Beej's guide to network programming. It is quite clear now that there is no guarantee on how many bytes are received in a single recv() call. Therefore a mechanism of e.g. first two bytes stating the message length should be sent and then the message. So the receiver receives the first two bytes and then receives in a loop until the whole message has been received. All good and dandy!?
I was asked by a colleague about messages going out of sync. E.g. what if, somehow, I receive two bytes in once recv() call that are actually in the middle of the message itself and it would appear as a integer of some value? Does that mean that the rest of the data sent will be out of sync? And what about receiving the header partially, i.e. one byte at a time?
Maybe this is overthinking, but I can't find this mentioned anywhere and I just want to be sure that I would handle this if it could be a possible threat to the integrity of the communication.
Thanks.
It is not overthinking. TCP presents a stream so you should treat it this way. A lot of problems concerning TCP are due to network issues and will probably not happen during development.
Start a message with a (4 byte) magic that you can look for followed by a (4 byte) length in an expected order (normally big endian). When receiving, read each byte of the header at the time, so you can handle it anyway the bytes were received. Based on that you can accept messages in a lasting TCP connection.
Mind you that when starting a new connection per message, you know the starting point. However, it doesn't hurt sending a magic either, if only to filter out some invalid messages.
A checksum is not necessary because TCP shows a reliable stream of bytes which was already checked by the receiving part of TCP, and syncing will only be needed if there was a coding issue with sending/receiving.
On the other hand, UDP sends packets, so you know what to expect, but then the delivery and order is not guaranteed.
Your colleague is mistaken. TCP data cannot arrive out of order. However you should investigate the MSG_WAITALL flag to recv() to overcome the possibility of the two length bytes arriving separately, and to eliminate the need for a loop when receiving the message body.
Its your responsibility to make you client and server syncing together, how ever in TCP there is no out of order delivery, if you got something by calling recv() you can think there isn't anything behind that that you doesn't received.
So the question is how to synchronize sender and receiver ? its easy, as stefaanv said, sender and receiver are knowing their starting point. so you can define a protocol for your network communication. for example a protocol could be defined this way :
4 bytes of header including message type and payload length
Rest of message is payload length
By this, you have to send 4 byte header before sending actual payload, then sending actual payload followed.
Because TCP has garauntied Inorder reliable delivery, you can make two recv() call for each pack. one recv() call with length of 4 bytes for getting next payload size, and another call to recv() with size specified in header. Its necessary to make both recv() blocking to getting synchronized all the time.
An example would be like this:
#define MAX_BUF_SIZE 1024 // something you know
char buf[MAX_BUF_SIZE];
int recvLen = recv(fd, buff, 4, MSG_PEEK);
if(recvLen==4){
recvLen = recv(fd, buff, 4);
if(recvLen != 4){
// fatal error
}
int payloadLen = extractPayloadLenFromHeader(buf);
recvLen = recv(fd, buff, payloadLen, MSG_PEEK);
if(recvLen == payloadLen){
recvLen = recv(fd, buff, payloadLen); // actual recv
if(recvLen != payloadLen){
// fatal error
}
// do something with received payload
}
}
As you can see, i have first called recv with MSG_PEEK flag to ensure is there really 4 bytes available or not, then received actual header. same for payload
i am trying to send data via tcp socket to a server. The idea behind that is a really simple chat programm.
The string I am trying to send looks like the following:
1:2:e9e633097ab9ceb3e48ec3f70ee2beba41d05d5420efee5da85f97d97005727587fda33ef4ff2322088f4c79e8133cc9cd9f3512f4d3a303cbdb5bc585415a00:2:xc_[z kxc_[z kxc_[z kxc_[==
As you can see there a few unprintable characters which I don't think are a problem here.
To send this data I am using the following code snippet.
bool tcp_client::send_data(string data)
{
if( send(sock , data.c_str(), strlen(data.c_str()) , 0) < 0)
{
perror("Send failed : ");
return false;
}
return true;
}
After a few minutes of trying things out I came up, that data.c_str() cuts my string of.
The result is:
1:2:e9e633097ab9ceb3e48ec3f70ee2beba41d05d5420efee5da85f97d97005727587fda33ef4ff2322088f4c79e8133cc9cd9f3512f4d3a303cbdb5bc585415a00:2:xc_[z
I think that there is some kind of null sequence inside my string which is a problem for the c_str() function.
Is there a way to send the whole string as I mentioned aboved without cutting it off?
Thanks.
Is there a way to send the whole string as I mentioned aboved without cutting it off?
What about:
send(sock , data.c_str(), data.size() , 0);
There are only two sane ways to send arbitrary data (such as a array of characters) over stream sockets:
On the server: close the socket after data was sent (like in ftp, http 0.9, etc). On the client - read until socket is closed in a loop.
On the server: prefix the data with fixed-length size (nowadays people usualy use 64 bit integers for size, watch out for endiannes). On the client - read the size first (in a loop!), than read the data until size bytes are read (in a loop).
Everything else is going to backfire sooner or later.
I am doing a simple TCP server in C++ for Windows to echo incoming data. I have a problem with it. Before I explain my problem, I have to say that Winsock is properly set up and this problem happens with any IP address as the source.
The general behaviour when a connection is established is this:
In the loop that runs while connection still alive, it must echo data, and precede it with REPLY word.
To do that, I'm currently using two send() calls:
One call sending "REPLY " alone.
Another call just sending back received data.
But using Putty Client, I'm getting this:
REPLY data_echoed REPLY.
Why REPLY is sent after the last send call if it was the first??? I'll post a little code where the problem happens:
//Reply to client
message = "HELLO CLIENT!! READY TO ECHO.\n";
send(new_socket, message, strlen(message), 0);
///Enter into a loop until connection is finished.
printf("%s \n\n", "Incoming data goes here: ");
do{
///Clear buffer and receive data.
memset(buffer, 0, sizeof(buffer));
ret = recv(new_socket, buffer, sizeof(buffer), 0);
printf("%s", buffer);
///Send a REPLY WORD and the data echoed.
send(new_socket, "REPLY\r\n", 7, 0);
send(new_socket, buffer, sizeof(buffer), 0);
}while(ret != SOCKET_ERROR);
What is wrong with that? I'll remove the first call, and the double effect disappears. Why I can't do two send calls one after other?
You ignore the return value of recv until after you send REPLY, so no matter what happens, you send REPLY followed by the contents of the buffer. After you echo the first time, something happens, and no matter what it is, you send REPLY.
Bluntly, it doesn't look like you understand the very basics of TCP. We used to have a rule that before anyone can do anything with TCP, they must memorize and completely understand the following statement: "TCP is a byte-stream protocol that does not preserve application message boundaries."
Your code pretends that it is receiving and echoing application-level messages. But there is no code to actually implement application-level messages. TCP has no support for application-level messages, so if you want them, you have to implement them. You clearly want them. You also have not implemented them.
Do newline characters delimit messages? If so, where's the code to find them in the data stream? If not, what does?
I'd like to make a chatting program using win socket in c/c++. (I am totally newbie.)
The first question is about how to check if the client receives packets from server.
For instance, a server sends "aaaa" to a client.
And if the client doesn't receive packet "aaaa", the server should re-send the packet again.(I think). However, I don't know how to check it out.
Here is my thought blow.
First case.
Server --- "aaaa" ---> Client.
Server will be checking a sort of time waiting confirm msg from the client.
Client --- "I received it" ---> Server.
Server won't re-send the packet.
The other case.
Server --- "aaaa" ---> Client.
Server is waiting for client msg until time out
Server --- "aaaa" ---> Client again.
But these are probably inappropriate.
Look at second case. Server is waiting a msg from client for a while.
And if time's out, server will re-send a packet again.
In this case, client might receive the packet twice.
Second question is how to send unlimited size packet.
A book says packet should have a type, size, and msg.
Following it, I can only send msg with the certain size.
But i want to send msg like 1Mbytes or more.(unlimited)
How to do that?
Anyone have any good link or explain correct logic to me as easy as possible.
Thanks.
Use TCP. Think "messages" at the application level, not packets.
TCP already handles network-level packet data, error checking & resending lost packets. It presents this to the application as a "stream" of bytes, but without necessarily guaranteed delivery (since either end can be forcibly disconnected).
So at the application level, you need to handle Message Receipts & buffering -- with a re-connecting client able to request previous messages, which they hadn't (yet) correctly received.
Here are some data structures:
class or struct Message {
int type; // const MESSAGE.
int messageNumber; // sequentially incrementing.
int size; // 4 bytes, probably signed; allows up to 2GB data.
byte[] data;
}
class or struct Receipt {
int type; // const RECEIPT.
int messageNumber; // last #, successfully received.
}
You may also want a Connect/ Hello and perhaps a Disconnect/ Goodbye handshake.
class Connect {
int type; // const CONNECT.
int lastReceivedMsgNo; // last #, successfully received.
// plus, who they are?
short nameLen;
char[] name;
}
etc.
If you can be really simple & don't need to buffer/ re-send messages to re-connecting clients, it's even simpler.
You could also adopt a "uniform message structure" which had TYPE and SIZE (4-byte int) as the first two fields of every message or handshake. This might help standardize your routines for handling these, at the expense of some redundancy (eg in 'name' field-sizes).
For first part, have a look over TCP.
It provides a ordered and reliable packet transfer. Plus you can have lot of customizations in it by implementing it yourself using UDP.
Broadly, what it does is,
Server:
1. Numbers each packet and sends it
2. Waits for acknowledge of a specific packet number. And then re-transmits the lost packets.
Client:
1. Receives a packet and maintains a buffer (sliding window)
2. It keeps on collecting packets in buffer until the buffer overflows or a wrong sequenced packet arrives. As soon as it happens, the packets with right sequence are 'delivered', and the sequence number of last correct packet is send with acknowledgement.
For second part:
I would use HTTP for it.
With some modifications. Like you should have some very unique indicator to tell client that transmission is complete now, etc
I am learning how to network program using c/c++ and I have created a server(TCP) that is suppose to respond in specific ways to messages from a client in order to do this I created a class that the server class passes the message to and returns a string to report back to the client.
Here is my problem sometimes it reports the correct string back other times if just repeats what I sent to the message handler. Which no where in the code do I have it return what was passed in. So I am wondering am I handling getting the message correctly?
Secondly, I am unsure of how to keep a connection open in a while loop to continually pass messages back and forth. You can see how I did it in the code below but I am pretty sure this is incorrect, any help on this would be great. Thanks!
if (!fork())
{ // this is the child process
close(sockfd); // child doesn't need the listener
while ((numbytes = recv(new_fd, buf, MAXDATASIZE-1, 0)) > 0)
{
//numbytes = recv(new_fd, buf, MAXDATASIZE-1, 0);
buf[numbytes-1] = '\0';
const char* temp = ash.handleMessage(buf).c_str();
int size_of_temp = ash.handleMessage(buf).length();
send(new_fd, temp, size_of_temp, 0);
//send(new_fd, temp, size_of_temp+1, 0);
}
}//end if
Please excuse my ghetto code
Handles the message
Class Method handler uses
If your learning about sockets you should also know that you can't assume that what you send() as a "complete message", will be delivered as a complete message.
If you send() some big data from your client you might need to use multiple recv() on the server (or vice versa) to read it all. This is a big difference of how files usually work...
If you'r designing your own protocol you can opt to also send the messages length, like [LEN][message]. An easy example would be if the strings you send are limited to 256 bytes you can start with send()ing a short representing the strings length,
Or easier, decide that you use line-feeds (newline - \n) to terminate messages. The the protocol would look like
"msg1\nmsg2\n"
then you would have to recv(), and append the data, until you get a newline. This is all I can muster right now, there are a lot of great examples on the internet, but I would recommend getting the source of some "real" program and look at how it handles its network.
You are calling handleMessage twice. You didn't post the code, but it looks like you're returning a string. It might be better to do:
string temp = ash.handleMessage(buf);
int size_of_temp = temp.length();
This would avoid repeating any action that takes place in handleMessage.