I am developing the protocol for my network Asteroids game. The first version will just allow each player have a ship shooting opponents in a scrolling world. Later I will add rocks, many rocks.
It seems non trivial to implement the protocol on top of UDP, however UDP is unquestionably the way to go in real time games. I have studied Gaffers articles on networking for games and so I think I will have to implement the following on top of UDP:
Reliablilty (making sure certain packets arrive by sending
acknowledgments. This implies also making sure that the
acknowledgements arrive).
Fragmentation and reassembly of packets that are larger than the
recommended UDP packet size.
Detecting dead clients.
Have I left anything major out?
The issue is, will you simply end up re-inventing TCP / stream socket functionality? Once you start considering ACK/NAK, packet ordering and re-assembly, and endpoint integrity, you're essentially re-inventing the wheel. Not only that, but you're doing it in user space, and without the extensive path optimizations and memory management strategies provided by the kernel. I'm not saying UDP is the wrong choice, but have a look at the source for multiplayer / network code in the Q3 engine, etc. A lot depends on payload contents and redundancy. i.e., can you afford to drop frames / packets?
Related
We decided to use UDP to send a lot of data like coordinates between:
client [C++] (using poll)
server [JAVA] [Apache MINA]
My datagrams are only 512 Bytes max to avoid as possible the fragmentation during the transfer.
Each datagram has a header I added (with an ID inside), so that I can monitor :
how many datagrams are received
which ones are received
The problem is that we are sending the datagrams too fast. We receive like the first ones and then have a big loss, and then get some, and big loss again. The sequence of ID datagram received is something like [1], [2], [250], [251].....
The problem is happening in local too (using localhost, 1 network card only)
I do not care about losing datagrams, but here it is not about simple loss due to network (which I can deal with)
So my questions here are:
On client, how can I get the best :
settings, or socket settings?
way to send as much as I can without being to much?
On Server, Apache MINA seems to say that it manage itself the ~"size of the buffer socket"~ but is there still some settings to care about?
Is it possible to reach something like 1MB/s knowing that our connection already allow us to have at least this bandwidth when downloading regular files?
Nowadays, when we want to transfer a ~4KB coordinates info, we have to add sleep time so that we are waiting 5 minutes or more to get it to finish, it's a big issue for us knowing that we should send every minute at least 10MB coordinates informations.
If you want reliable transport, you should use TCP. This will let you send almost as fast as the slower of the network and the client, with no losses.
If you want a highly optimized low-latency transport, which does not need to be reliable, you need UDP. This will let you send exactly as fast as the network can handle, but you can also send faster, or faster than the client can read, and then you'll lose packets.
If you want reliable highly optimized low-latency transport with fine-grained control, you're going to end up implementing a custom subset of TCP on top of UDP. It doesn't sound like you could or should do this.
... how can I get the best settings, or socket settings
Typically by experimentation.
If the reason you're losing packets is because the client is slow, you need to make the client faster. Larger receive buffers only buy a fixed amount of headroom (say to soak up bursts), but if you're systematically slower any sanely-sized buffer will fill up eventually.
Note however that this only cures excessive or avoidable drops. The various network stack layers (even without leaving a single box) are allowed to drop packets even if your client can keep up, so you still can't treat it as reliable without custom retransmit logic (and we're back to implementing TCP).
... way to send as much as I can without being to much?
You need some kind of ack/nack/back-pressure/throttling/congestion/whatever message from the receiver back to the source. This is exactly the kind of thing TCP gives you for free, and which is relatively tricky to implement well yourself.
Is it possible to reach something like 1MB/s ...
I just saw 8MB/s using scp over loopback, so I would say yes. That uses TCP and apparently chose AES128 to encrypt and decrypt the file on the fly - it should be trivial to get equivalent performance if you're just sending plaintext.
UDP is only a viable choice when any number of datagrams can be lost without sacrificing QoS. I am not familiar with Apache MINA, but the scenario described resembles the server which handles every datagram sequentially. In this case all datagrams arrived while the one is serviced will be lost - there is no queuing of UDP datagrams. Like I said, I do not know if MINA can be tuned for parallel datagram processing, but if it can't, it is simply wrong choice of tools.
I am coding a real-time, network program with UDP protocol using SFML libraries. The server will be handling all of the processing, sending packets to the client and vice versa.
I need a method to synchronize the screen updates, because there will be a real-time user interface on both sides of the network. What I am thinking is, I will have the client and server open two ports, 1 for sending the packet and the other as a sort of 'verifying' port.
Once the updates are made and verified, both loops will send a byte over the network indicating that their side is 'ready'. Once a 'ready' side receives that byte, it will know the screen is ready to update on both sides, and then render the updates.
My question is, how would I do this in C++ using SFML libraries. Is the logic correct, will I encounter network errors etc.
Comment if anything needs clearing up.
You should read on the subject to see how it really works, like How do the protocols of real time strategy games such as Starcraft and Age of Empires look?
There is nothing specific to know in terms of SFML as the theory is applicable to every programming languages (that uses sockets). And you've not provided the code that you're struggling with, so I can't help without Minimal, Complete, and Verifiable example.
But one thing I would say is to never wait on the network to render. This will be laggy as hell and if you want "real-time", you must simulate it without real "real-time" as there will always be latency with networking.
And if you go with UDP, you don't have a choice but to open 2 ports, one on the server and one on the client. But I would argue against using a
"confirmation" port as this is useless since you'll be sending multiple times a second anyway.
See how to use sockets with SFML. I guess you already know this but it might be useful for people coming from google.
See SFML networking rts which explains way better than me what to do.
From #muhwu
Generally if you are going for a real-time game with timing critical elements, you should be placing the commands player make slightly to the future to compensate for the latency. This can be few hundred milliseconds at most.
From #sean-middleditch:
Note also that many articles are just really out of date. TCP for instance is a perfectly good choice for networking in a game like an RTS, especially with lock-step networking, and is even the protocol of choice for many smaller MMOs. Despite this, almost every article around on networking is about building a custom UDP protocol.
Speaking of gamedev.stackexchange.com, you might want to read the questions and answers, and if that does not help, then ask one there.
Asking the same question on SO might not be the way to go, as your question is too broad and as already been answered a number of times (like the first one I linked).
I'm working on reliable UDP networking and I have to know something. I think UDP reliable algorithm works like that (IDK, I guess);
Server send: (header:6)abcdef
Client receive: (header:6)abdf, sends back "I got 4 data, they are abdf"
Server send: (header:2)ce
Client receive: (header:2)ce, OK I'm going to combine them!
Now is this true way to reliable UDP?
EDIT (after answer, maybe this can be helpful for someone): I'm goint to use TCP because reliable UDP is not a good way to handle my operations. I'll send position like un-important, temporal variables. Maybe if I create a algorithm for reliable UDP, this reliable process will took 3-4 UDP send-recv that means I can send 3-4 other unreliable position data at this time and I'm sending small datas which is can be more efficiency than reliable UDP.
The "true way" to get reliable UDP is to use TCP.
If you still want to do it over UDP, you can verify the integrity of the message by sending a checksum with the message, and then recalculating the checksum at the other end to see if it matches the checksum you sent.
If it doesn't match, request the packet again. Note that this is essentially reinventing TCP.
Well, even with:
- Client receive: (header:6)abdf, sends back "I got 4 data, they are abdf"
- Server send: (header:2)ce
what if server will not receive your response (which may happen in UDP)? So switching to TCP is much better option, if you're not concerned about speed of connection.
Your problem sounds like it's tailor-made for the Data Distribution Service.
I'll send position like un-important, temporal variables
In fact, location ordinates are the popular examples for many of its vendors. RTI has a demonstration which goes well with your use case.
Yeah, a lot of folks groan when they hear "IDL" but I'd recommend that you give it a fair shake. DDS is unlike very many popular pub-sub/distribution/etc protocols in that it's not a simple encapsulation/pipeline.
I think the really cool thing is that often a lot of logic and design elements go into the problem of "how do I react the underlying network or my peer(s) misbehave(s)?" DDS offers a quality of service negotiation and hooks for your code to react when the QoS terms aren't met.
I would recommend against taking this decision lightly, it's a good deal more complex than TCP, UDP, AMQP, etc. But if you can afford the complexity and can amortize it over a large enough system -- it can pay real dividends.
In the end, DDS does deliver "reliable" messages over UDP. It's designed to support many different transports, and many different dimensions of QoS. It's truly dizzying when you see all the different dimensions of QoS that are considered by this service.
I'm working on an online turn based card game for the PC. It features a lobby which will automatically update the list of active games so I'll be sending many updates to many clients. I will have a game server for this. Lag is not too big a deal for me, if players have to wait an extra 1/4 second sometimes until their card is shown, it dosn't really concern me. What concerns me is reliability and stability. I want to be able to host many 4 player games, I also allow people to watch a specific game too.
I also will need to log them in, and remember their session if they disconnect so they can get back in the game if they get disconnected.
What I am debating is if I should go with Enet which is UDP based with reliability, or plain old TCP/IP.
I will eventually need to be able to send them additional content created such as extra decks which will be in the form of a zip file. But for that I'm sure there is a library that could help me get these from an HTTP source.
If anyone has experience with either or both of these, I'd appreciate your input.
Thnks
I'd stick with classic TCP, not some library that has hacked in correct ordering and packet delivery reliability into a protocol not designed for it.
Enet's claim is to have a few of the benefits of TCP, while delivering speed for "real-time" games. That phrase makes me think of FPS, which isn't your app at all.
I haven't worked with ENet, but I had a quick read of it's documentation, and it seems like it could be useful in certain circumstances.
First question: Do you need a reliable packet delivery mechanism, or is unreliable OK? ENet can do reliable, but at that point it's getting kind of similar to TCP, and unless you need it's multi-stream support, I'm not sure it's worth fooling with.
If unreliable is OK, then the next question is: do you need the packet fragmentation support that ENet has? If your packets are going to be small, then I'd say just use UDP directly.
I'm unclear on how ENet uses UDP sockets and how it manages connections, so I'm unclear whether you're better off with lots of open TCP sockets or lots of open ENet connections.
Lots of questions, I am sorry!
I am doing a voice-chat (VoIP) application and I was thinking of doing a custom implementation of the IP&UDP headers, along with small, extra information mainly seq number. Sounds alot like RTP yes, but I'm mainly just interested in the seq number or timestamp, and trying to implement my own whole RTP sounds like a nightmare with all the complexity involved and data im not likely to use.
Target OS for the application is windows xp and above. I have read http://msdn.microsoft.com/en-us/library/ms740548%28v=vs.85%29.aspx on the topic of Raw sockets in windows, and now I just want some confirmation.
I also have some general networking questions.
Here's the following questions;
1) According to MSDN, you cannot send custom IP packets with a source that is not on the network list. I understand it from a security PoV, but is there any way around this? My idea was to have for example two clients open UDP communication to a non-NAT protected server, and then have the clients spoof the source-header to make it look like packets come from the server instead of each other, thereby eliminating the need for a server as a relay of data to get through NAT, which would improve latency.
I have heard of winpcap but I don't want each client to have to install any 3rd party apps. Considering the number of DoS attacks surely there must be some way around this, like spoofing the network table the OS uses to check if source-header is legit? Will this trigger anti-virus systems?
I feel it would be really fun to actually toy with IP headers and above instead of just using predefined headers.
2) I've been having issues with free RTP libraries like JRTPLIB(which probably is very good anyway it just dosn't want to work for me) to make them work, more than I could almost tolerate, and am thinking of just writing my own interpretation ontop of UDP. Does application-level protcols like RTP simply build their header directly inside the UDP payload with the actual data afterwards? I suspect this considering the encapsulation process but just want to make sure.
If so, one does not need to create a RAW socket to implement application-level protocol, just an ordinary UDP socket and then your own payload interpretation above?
3) RTP does not give any performance boost compared to UDP since it adds more headers, all it does is making sure packets arrive in a sort-of correct manner based on timestamps and sequence numbers, right?
Is it -really- that usefull to use an RTP implementation for your basic VoIP project needs instead of adding basic sequencing yourself? I realise for video conferencing perhaps you reaally don't want frames to play out of order, but in audio conversations, would you really notice it?
4) If my solution in #1 is not applicable and I would have to use a server as a data relay between clients, would multicast be a good solution to reduce server loads? Is multicast supported enough in routing hardware?
5) It is related to question 1). Why do routers/firewalls allow things like UDP hole punching? For example, two clients first conenct to the server, then the server gives a client port / ip on to other clients, so the clients can talk to each other on those ports.
Why would firewalls allow data to be received from another IP than the one used in making the connection on that very port? Sounds like a big security hole that should easly be filtered? I understand that source IP spoofing would trick it, but this?
6) To set up a UDP session between two parties (the client which is behind NAT, server whic his non-NAT) does the client simply have to send a packet to the server and then the session is allowed through the firewall? Meaning the client can receive too from the server.
Based on article at wiki, http://en.wikipedia.org/wiki/UDP_hole_punching
7) Is SIP dependant on RTP? For some reason I got this impression but I cant find data to back it up. I may plan to add softphone functionality to my VoIP client in the future and want to make sure I have a good foundation (RTP if I really must, otherwise my own UDP interpretation)
Thanks in advance!
1, Raw sockets seems unnecessary for this application
2, Yes
3, RTP runs on top of UDP, of course it adds overhead. In many ways RTP (ignoring RTCP) is pretty much the bare minimum already and if you implemented a half-way decent alternative it would save you a few bytes at best and you wouldn't be able to use any of the many RTP test tools.
7, SIP is completely independent of RTP. SIP is used to Initiate Sessions. SDP is the protocol commonly transported by SIP, and it is SDP that negotiates and controls RTP video/voice voice.