I would like to have some advices about my project. I am currently developing a musical application/game in C++/Qt, with a multiplayer mode, and I have the following requirements (ideally):
I want to be able to have friends contacts, and be able to chat and play with them
I need to send/receive realtime datas to these contacts (music notes)
I don't want to create a server application
What would you recommend to do this?
I was thinking of using xmpp protocol, so I can connect to google/jabber and retrieve contacts, chat with them. Actually this part works, but then I don't konw how to send/receive realtime datas. For the realtime part, I was thinking of using direct TCP communication, but I need to know external ip of my contacts, and I have no idea how to do it. I was thinking to automatically send my external ip and tcp port to all my contacts every time I connect, but I didn't find solution to retrieve external ip from code. So I'm a bit stuck. Any advice?
Is there alternative solutions? alternative protocols?
Thanks you,
Laurent
You're going to have a really hard time avoiding writing a server, for realistic, practical, and performance reasons:
Many residential internet connections are behind firewalls (at the ISP, local router, or OS level) that limit accepting connections from outside the network. NAT further complicates accepting connections from the internet on a LAN.
There are precious few methods of internet communication that are serverless, and those that are are subject to using local peer discovery to find peers. Most LPD traffic will not make it off your lan, the ISP will filter it (otherwise you'd be able to "locally" discover peers on the entire internet).
Bandwidth can be a concern for games. Not everyone has a high speed internet connection yet (though market penetration of fiber optics and fast DSL is pretty high at this point), and you'd end up with problems connecting slower hosts to a large swarm.
Servers facilitate star-like networks, which are very efficient. Other network topologies exist, but many suffer from drawbacks that severely inhibit their ability to scale.
Star networks, for example, for clients, require O(connections) =
O(1), O(bandwidth) = O(1), and O(latency) = O(1).
Fully connected networks require every client to be connected to
every other client, so O(connections) = O(bandwidth) = O(n), and
O(latency) = O(1).
In ring networks, each client connects to 2 neighbors and messages of
distant clients are forwarded. Therefore, they have O(connections) =
O(1), but O(bandwidth) = O(latency) = O(n).
If all you need is a chat system, or want badly enough not to write your own server that you're willing to piggyback the entire online experience over a chat server, you could probably rely on something like an XMPP server.
If you choose to go that route, make sure that proper authentication and encryption is used wherever necessary to protect user's private data (passwords, etc). I recommend using a cryptographic authentication scheme that allows clients to authenticate other clients (such as a challenge/response scheme, or something else). Or, you could mediate all authentication with a central service.
Keep in mind that many chat services will not want to provide your project with free bandwidth. Even if you do decide to use XMPP as the heart of your multiplayer protocol, expect to be running your own server.
Related
I am developing a multiplayer game (client-server model) and I am stuck when it comes to scaling its servers.
I understand that most games never even reach 10 000+ players, and I don't think mine will either.
Though if I would be very lucky to get that I want to develop the servers so they cannot become a huge obsticle later.
I have searched a lot for a solution to my problem on the internet, checking GDC talks about it and checking other posts on this website, but none of them seems to solve my specific problem.
My current setup is below and all servers are written in C++ using ENet as my network library.
Game server
This server handles the actual gameplay of the game and requires quite a lot of CPU and packages being sent between the server and its connected clients.
But this dedicated server is hosted by the players themselves, so I don't have to think about scaling it at all.
Lobby server
This server handles the server list, containing all servers currently up.
All game servers are sending a UDP package to this server every 5 seconds to say they are still alive.
This is so the lobby server can keep an updated list of all servers currently online.
All clients are sending a UDP package to this server when they want to fetch all servers (which is only in the server
list screen), and the lobby server sends back a list of all servers.
This does not happen that often and the lobby server is limited to send 4 servers per second to a client (and not a huge package containing all servers).
Login server
This server handles creating accounts, lost password, logins, friends and their current game status,
private messages to other logged in players and player profiles that specifies what in-game items they have.
All clients are sending a UDP package to this server every 5 seconds to say they are still alive, while also
sending what game they are currently in. The server then sends back their friend lists online/offline/in-game statuses.
This is so their friends can keep an updated list of which friend is online/offline/in-game.
It sends messages only with player actions otherwise, like creating an account, logging in, changing/resetting password,
adding/removing/ignoring a friend, private messages to friends, etc.
My questions
What I am worried about is that my lobby and login server might not be scalable and that they would have too much traffic on them.
1. Could they in theory be hosted on just a single computer? Or would it be too much traffic for 10 000+ players?
2. If they can be hosted on a single computer, will the servers still not have issues for people that live far away?
Would it be better to have the lobby and login servers per region of the world in that case?
The bad thing about that is that the players would not be able to see servers in the US if they live in Europe, and that their account and items would not exist on the other servers.
3. Might be far-fetched, but if I would rewrite both servers to instead be on a website with a database and make the client/game server do
web requests instead (such as HTTPS or calling a php with specific headers),
would it help in solving my problems somehow?
All your problems and questions are solved by serverless cloud based solution AWS Lambda e.g. or similar. In this case the scalability is not your problem. Just develop the logic. This will save you much time.
If you would like to make servers as single app hosted by your own server. Consider using something like e.g. Go instead of C++. It's designed exactly for these purposes. I mean highly loaded web/network services.
Well, this is c++ and i code in java, but maybe the logic is useful for you any way so i will tell you how i end up implementing something similar but in a casino.
In my case I have 2 diferrent sockets in the same server program, one of the sockets is TCP and it handles all logins, registers and payments, while the second socket is UDP and it handle the actual game multiple payers are playing, then you could group internally all those UDP connection in groups (probably arrays of sockets) to generate those lobbies. Doing that all your server is just one class that could run in a single pc using 2 ports (one for each socket) However this do not solve the problem of the ping for people who live far away.
If ping is a problem (not my case in a casino) you could probably host your server region base but removing the login, registration and paymets of your server and replace it for a connection to a central server (this central server should be TCP and you could also implement a https socket to also allow your webpage to connect to this server and create accounts or pay you directly from the browser)
sorry to mess your life even more, but i hope it helps
I am running an Openfire server on a AWS EC2 instance and am able to connect to the server from my mobile devices and send messages back and forth. Of course, since XMPP is a client-server based protocol, I incur costs for running this traffic over the AWS server. However, for most use cases, this cost is not very high at all, as normal XMPP stanzas rarely seem to go above ca. 1 KB, so from this end all is ok.
I would now, however, like to include the ability to send images from one client to another. One way would be to use an HTTP server, to which user A uploads the picture and then sends the URL of the image to user B via XMPP, so that the user can now get the image via HTTP. There are also several other methods for sending images via XMPP. However, I am interested in doing this via Jingle.
As far as as I understand, Jingle is an out of band peer-to-peer extension to XMPP. My simple question is, since Jingle communicates peer-to-peer, i.e. without the use of a server, for the multimedia aspect of the session, will I even incur any data cost on AWS for transferring multimedia from one client to another using Jingle? Or put differently, if Jingle is peer-to-peer, does any data go via my AWS server using Jingle (except the session initiate, ack, session terminate stanzas)? If not, what kind of route does this data take, and how can anyone be billed for this traffic cost, if it is peer-to-peer?
Jingle is a negotiation mechanism, and there are a couple of different transports it could negotiate for file transfer. The most common transport is peer to peer bytestreams defined in http://xmpp.org/extensions/xep-0260.html - here the only traffic you'd see via the server would be the jingle negotiation, which is a similar sort of volume to other XMPP traffic). There is also an in-band bytestream transport defined in http://xmpp.org/extensions/xep-0261.html that some clients will use - typically for smaller transfers as it's inefficient, but has the advantage of working in hostile networks with NAT and firewalls. If you control the clients, simply not supporting IBB would be your best bet for ensuring the traffic doesn't travel via the server. If you don't, I'd suggest configuring your server to block IBB traffic.
I note as well that running a server-side proxy will drastically increase the odds of the out-of-band mechanism in 260 working in the face of hostile networks, at the cost of server bandwidth.
There is also the not-widely-deployed http://xmpp.org/extensions/xep-0343.html out of band transport.
I want people to controll my arduino robot via the internet. It's important that the controlling reacts very fast. The user may send many requests per second.
Let me explain my architecture:
The user connects to a web-frontend. He can use a virtual joystick and buttons. The frontend will then send orders (like "motor1:255" "motor2:0"....) to an application server (Wildfly).
When a frontend-session starts, Wildfly will establish a connection to my computer or smartphone using a socket. The orders will be passed to the arduino using bluetooth. When a frontend-session is not longer active, the socket will be closed.
One Wildfly should be able to controll up to 10 robots. One robot can be controlled by exactly one user. Some developers use a mysql table and add a row for each incoming order. I don't think this would work in my case.
Is it okay to use REST to send the orders from the frontend to the application server? Is there any other fast and secure way to transport the user input from the frontend to the business logic?
Goot
REST, when properly understood and applied, is a solution for problems of long-term evolution and maintenance of your application. It doesn't seem to be your case.
What you mean is probably if an HTTP API is a good solution if you have lots of requests, and the answer is, it depends. I'd probably look into something like ZeroMQ for doing what you want.
I've been reading around on the www but just can't get the most important basics of P2P.
The diagram is like this:
[peer1]<-->[dsl-router1]<-->[central server]<-->[dsl-router2]<-->[peer2]
I'm developing a chat software on the central server. Chat messages being transfered thru' the central server well by now, however, I need to make the p2p file sharing feature because the bandwidth (the cable bandwith, not the transfer limit) of the server supposed for transfering chat messages only.
The problem is that, my software on central server knows the IPs and ports of router1 and router2, but not the peer1 and peer2 as these peers are behind the routers and don't have IP addresses.
How to actually transfer some data from peer1 to peer2 and vice versa without having this data passing thru' central server?
(and the worst case is that there is a wireless router between peer and dsl-router)
There are two basic ways of doing this. The new way is to use IGDP (opening a port via uPnP). This is described quite well here:
http://www.codeproject.com/Articles/13285/Using-UPnP-for-Programmatic-Port-Forwardings-and-N
If neither of the two nodes have a router supporting uPnP then another alternative is TCP hole punching, which is not perfect but works quite well in practice. This is described here:
http://www.brynosaurus.com/pub/net/p2pnat/
During some situations, "routers" supplied by the ISP may run on bridge mode, which directly exposes the peer computer on the internet (the computer gets a public internet address). If at least one side has this configuration (or in a similar situation that the peer client is not behind another device), then things should be rather straight forward: simply assign the central server's job to whoever that have this privilege.
In the other case where both peers only have a local address (e.g. 192.168.0.2) assigned to their computers, it would then be rather difficult to get through the routers; clients behind routers are for the most part unreachable from the outside unless they originated the request. Then, one solution to the problem is port forwarding. By doing port forwarding, either through explicitly written rules or UPnP, some ports on the peer computer is exposed to the public internet, as in the first situation where instead of only some ports the entire computer is exposed.
If you are without either of these, then there is no simple way to avoid sending through the central server. Though you could, potentially, find other peers who have the capability to transfer for others.
I have a C++ application that currently uses a simple TCP/IP client/server model to communicate between 2 instances of itself. This works fine on a local network, but I would like this to be used across an external network. Currently, maybe due to firewall issues, it is not able to connect across an external network.
I am not an expert on networking, but I was thinking about having a dedicated server in the middle acting as a hub for communications. Will this mitigate firewall issues?
How do networked games communicate with each other? Is there usually a server in the middle or is it peer-to-peer?
In any case, I'd appreciate any advice on protocols and infrastructure to implement a network enabled application.
Regards
I think the problem is dedicated to the NAT as mentioned by cnicutar.
Maybe you want to have a look at libupnp for automatic port forwarding in the hardware firewalls (your router at home)
There is no de facto architecture for multiplayer network games. Both client-server (most MMOs, most PC FPS's and RTS's) and Peer-to-Peer (most console games) are valid approaches.
Juoni Smed's survey in his book "Algorithms and Networking for Computer Games" is a pretty good overview of the different architectures in the wild.
For the specific issues you're talking about, your need for a proxy server, as others have noted, is probably down to NAT issues - the two machines you're trying to get talking do not have public IP addresses. If you want to pursue a Peer-to-Peer architecture (or to have one of your clients act as the server, as many modern Client-Server games do) you will need your clients to talk directly to each other. This can be achieved with NAT Traversal, unfortunately this is a fiddly process.
Luckily you can use a modern framework like the excellent Raknet which includes State Synchronisation, Remote Procedure Calls AND NAT Traversal out of the box. It's free for hobbyist use and is incorporated in to several modern industrial-grade game engines.
The bane of modern internet communications is NAT. Due to NAT (which shouldn't be confused with a simple firewall) a large portion of hosts on the internet don't have a public address and thus can't (easily) accept incoming connections. NAT breaks the internet so badly that people are moving to a totally different scheme, with slightly different semantics, just to get rid of it.
There are basically two class of solutions
NAT traversal which is sometimes used for peer-to-peer communication. Usually NAT traversal schemes require some publicly accessible server for brokering the initial connection, but the actual communication is done peer-to-peer
Client-server communication. This is easier (since the server generally should have a publicly accessible address) but also kind of inefficient. For instance, say you've got a peer on the same 10Gb LAN. If you want to send him a file through the server (which happens to be in another country) it's going to take ages instead of seconds.
I'm not sure which one is "generally used". But think of it this way:
If there is the logical need for a "controller" (say 8 people are playing a strategy game) then you probably need a server
If any two peers can logically interact without a "controller", you probably want peer-to-peer communication
If you need to transfer LOTS of data fast (file transfer), you almost surely want p2p.
The easiest way to accomplish what you want is by using sockets(in case you are doing it differently). The way you are connecting your app is usually how it's done. Also if it work sin a local network and it does not over the Internet it must be a firewall issue so try opening ports in your router configuration.
You will have to give more info about your program in order to explain if you should go with peer-to-peer or with a server.