I am implementing a c++ application which involves multiply users(for example. 128 users) with asymmetric roles(they all have different jobs). In such scenario, every user has to communicate with each other. Thus each pair of users need a bi-direction (virtual) communication channel between them.
There are three popular messaging patterns in this application.
Exchange: each user i has a message m_ij for user j != i to send. The length of m_ij is a public constant value. These messages m_ij are independent and have no relation with each other. This is something like "everyone has something for everyone".
Distribute: a (per-determined) user i_0 has a message m_j for ever other user j!=i_0. The length of the messages is a public constant value. It is a little similar to broadcast but the receivers are not receiving the same message.
Gather: a (per-determined) user i_0 receives a message m_j from ever other user j!=i_0. The length of the messages is a public constant value. This is very similar to vote mechanism.
Besides, there are also a small amount of two-party communication between some of the users.
The round-trip cost is very sensitive in the application. Thus a one-roundtrip implementation for these communication patterns is very desirable.
Besides, the bandwidth cost of the application is very high thus a non-blocking implementation is almost a must-have.
I first tried the classic server/client socket (https://www.geeksforgeeks.org/socket-programming-cc/) by having multiply ports and deploy a server/client pair between every two users. However it turns out to be a failure.
I also investgate the ZMQ library. But to my poor understanding I have to somehow handle "routing" on my own, which I am not capable of.
Nanomsg is another candidate to go with but none of the patterns it provides seem to match the requirements.
So, could anyone provide any idea about this challenge? Thanks in advance!
We have an actor structure where there is a cluster-sharded actor that calculates a parametric matrix of about 7 meg and has to distribute it to five other nodes for consumption. First of all, there are the following constraints:
The matrix has to be generated in one place.
The matrix is needed on all nodes to handle user load.
The matrix will be periodically generated again to handle changing variables and then sent off to all nodes to handle user load.
Saving the matrix in a database is probably not viable as it merely shifts the networking load and the database would get very large very fast. The database only saves the input parameters.
We changed the Akka max message size to 10 meg to accomplish this, but that feels a bit odd and we didn't see another choice. Normally this works fine even though passing 10 meg messages around a distributed pub-sub seems odd to me. However, on startup, the system has to start up 2000 of these all at once. As a result the sharding coordinators scream at us about buffered messages. It eventually calms down and life resumes, but I would love to be able to do this without the bloodbath in the logs.
Can someone recommend an alternative strategy for handling the distribution of the parametric matrix that gets it to every node but doesn't cause a shard coordinator complaint bloodbath?
I'm trying to implement this pattern on a "smart building" system design (using STL library). Various "sensors" placed in rooms, floors etc, dispatch signals that are handled by "controllers" (also placed in different rooms, floors etc.). The problem I'm facing is that the controller's subscription to an event isn't just event based, it is also location based.
For example, controller A can subscribe to a fire signal from room #1 in floor #4 and to a motion signal in floor #5. A floor-based subscription means that controller A will get an motion event about every room in the floor he's subscribed to (assuming the appropriate sensor is placed there). There's also a building-wide subscription for that matter.
The topology of the system is read from a configuration file at start up, so I don't want to map the whole building, just the relevant places that contain sensors and controllers.
What I've managed to think of :
Option 1: MonitoredArea class that contains the name of the area (Building1, Floor 2, Room 3) and a vector where the vector's index is an enumerated event type each member of the vector contains a list of controllers that are subscribed to this event. The class will also contain a pointer to a parent MonitoredArea, in the case it is a room in a floor, or a floor in a building.
A Sensor class will dispatch an Event to a center hub along with the sensor's name. The hub will run it through his sensor-name-to-location map, acquire the matching MonitoredArea and will alert all the controllers in the vector.
Cons:
Coupling of the location to the controller
Events are enumerated and are hard coded in the MonitoredArea class, adding future events is difficult.
Option 2:
Keeping all the subscriptions in the Controller class.
Cons:
Very inefficient. Every event will make the control center to iterate through all the controller and find out which are subscribed to this particular event.
Option 3:
Event based functionality. Event class (ie. FireEvent) will contain all the locations it can happen in (according to the sensor's setup) and for every location, a list of the controllers that are subscribed to it.
Cons:
A map of maps
Strong data duplication
No way to alert floor-based subscriptions about events in the various rooms.
As you can see, I'm not happy with any of the mentioned solutions. I'm sure I've reached the over-thinking stage and would be happy for a feedback or alternative suggestions as to how I approach this. Thanks.
There is design pattern (sort of speak) used a lot in game development called "Message Bus". And it is sometimes used to replace event based operations.
"A message bus is a connection between one or more senders and/or receivers. Think of it like a connection between computers in a bus topology: Every node can send a message by passing it to the bus, and all connected nodes will receive that message. If the node is processed and if a reply is sent is completely up to each receiver itself.
Having modules connected to a message bus gives us some advantages:
Every module is isolated, it does not need to know of any others.
Every module can react to any message that’s being sent to the bus; that means you get extra flexibility for free, without increasing dependencies at all.
It’s much easier to follow the YAGNI workflow: For example you’re going to add weapons. At first you implement the physics, then you add visuals in the renderer, and then playing sounds. All of those features can be implemented independently at any time, without interrupting each other.
You save yourself from thinking a lot about how to connect certain modules to each other. Sometimes it takes a huge amount of time, including drawing diagrams/dependency graphs."
Sources:
http://gameprogrammingpatterns.com/event-queue.html
http://www.optank.org/2013/04/02/game-development-design-3-message-bus/
Question: Is it possible to update 100+ objects in the Flash Player over Socket Connections? More details and my own try's below!
Details
For my internship I got the time to create a multiplayer physics game. I worked for a steady three months on it. My internship is coming to an end and I couldn't finish my game.
My problem is that its hard to send multiple packets each time-step to the server and back. The packets I send are position updates of the objects and mouse of other clients.
I will try to explain the network/game flow.
Client connects to the server using the binary Socket class in AS3
Server ask for verification and client sends name and thumbnail.
Server waits until 4 clients are connected (Some matchmaking etc)
Server picks 4 clients and makes them run on a separate Thread(Combined as a Team)
Client sends his performance score to the server range 1-100.
Server makes the best client the host machine for the physics and the other 3 slaves
Host game sets up the level and makes around 1-100 shapes in the level(primary shapes and complex shapes like bridges, motors, springs)
Every time-step the host gets all updated property's of the shapes and sends them to the clients (x, y, rotation, sleep)
The client applys all the shape property's to the correct shapes
I tried different time-steps and noticed that until a time-step of 1/15 second the client(slave) won't notice any lagging in the game. I also tried to pick a lower time-step and tween the movement of the shapes but that did give some strange movement on the client(slave) side.
I will give an example of a single object update packet.
<O|t=s:u|x=201|y=202|f=automaticoo</O
<O|t=m:p|x=100|y=345|f=automaticoo</O
I noticed that the Flash Player can stack a lot of packets in the buffer before sending. For example if I send a lot of packets at once it stacks them up and send them together to the server. With faster time-steps you don't get more updates on the client(slave) side but more updates in the same packet row.
Tries
Use the new RTMFP(udp & p2p) protocol for updates. (little bit better in performance but less in reliability)
Code my entire socket server in c++ instead of Air(with the ServerSocket) (better in performance but noticed the lagging part is not the server but the Flash Player)
Use the ByteArray compress method and the AMF serialized format (performance about the same except the c++ server can't unserialize the messages)
Do you guys think it is possible in the Flash Player too handle so many update request each time-step.
Discoveries
There is a stick arena game that is multiplayer in ActionScript 3.0. They used a lot of tricking and even then I get a ping of about 300ms and it only updates the players constantly (4 players in a lobby).
Sorry for the long post.
I believe your problem comes down to type of game and data.
This again is broken up into:
server speed (calculations need in CPU + RAM requirements for world/player data)
connection speed (bandwidth on server)
data size (how much info is needed and how often)
player interaction form (event or FPS)
distance from client to server (ping)
MMORG
Eg. World Of Warcraft, is to my knowledge on the none-PVP worlds using a "client is a viewport" and "client sends keystrokes", "server validates and performs, and tells client what happens" on a CLIENT-SERVER base.
This gives the game a lot of acceptable latency as you only need to transfer commands from client and then results to the client. The rest is drawn on the client.
Its very event driven and from you click an icon or press a key, its okay that your "spell" needs some time to fire on the server. Secondly there is no player collision needed. This lets the server process less data too and keeps the requirements to the server CPU smaller.
Counter-Strike / Battlefield etc.
FPS, fast paced action, with quick response needs to get information about every detail all the time. This makes a higher demand on precision. Collision is a must for both player and weapons.
This sort of game usually doesnt handle more than 32 players on a single map, as they all need to be able to share their positions, bullets, explosions etc. very fast and all this data has to go through the server-validation which again is a bottleneck for any type of online game.
Network latency
In a perfect world this would be 0 ms, but as we all know. All the hardware from client to the server and back takes time. Both going through the network stacks and through the internet connection (switch, router, modem, fiber centrals etc) so the way many modern real time games fixes this is by "prediction". The let the server look at your direction and speed. Then they try to predict (much like the GPS do in a tunnel too) that you were last seen moving forward with a speed of +4 so given timeframe you have moved (timeframes x 4) - but what if you had slowed down or speeded up? then they either instantly "hyperjump" you from A to B in a split second and this you feel like a lagging game or they easy up to the real position so your "hero" slides a little faster or slower into the right possition.
This technique is explained many places on the net, so no need for details in here, but it takes time and tweaks to get a good performance from this - but it works and saves a lot of headaches for the programmers.
What network data is needed?
I read your question and thought: that could be compressed quite a lot. Secondly, I have made a Flash socket chat with pure ByteStream and that worked awesome. It was hard to get running for a start, but once I got it up and running it was fast.
The flash client/player itself isnt the biggest networking client, so expect a lot of lost speed there too. I would go for 10-15 fps for the networking part and then use a more RAW approach for the data sent back and forth.
Lastly, try to keep the data as simple as possible.
Eg. use COMMANDS/SHORTCUTS for certain data/events.
Like a server data bytestring could be: 0x99, 0x45,0x75,0x14,0x04,0x06
Where 0x99 means : BIG EXPLOSION at the following COORDS: (0x45,0x75)
Then 0x14 means: PLAYER 0x14 (player 20 in decimal) has moved to (0x04, 0x06)
So the staring opcode tells the networking protocol handler in your client and server what to expect next. (Its how a CPU knows how to read memory btw.)
For my chat I had commands for each type of data parsed. One for login, one for broadcasting, for for telling the name of a user etc. So once the client made a login, the client received a command + a packed of online users. This was only transfed once to the client. After that each attached client received a "new user online" command too with the name of the new user. Each client maintained its own list with current users and ID's so that I only needed to tell which client number say the text. This kept the traffic at a minimum. Same would go for coordinates or commands of what to do. "Player #20 goes north" etc. could be be 0x14, 0x41, 0xf0 (0x41 could be MOVE, 0xf0 could be NORTH, 0xf1 EAST etc.)
This physical distance to the game
This one you cant change, but you can put in some restrictions or make the servers run in multiple locations worldwide, depending on what type of game you wanna make. Amazon EC2 is a great platform for such projects as they have data-centers all over the world and then you can benchmark the users network against these and then redirect the users to the nearest datacenter where you are running a server.
Hacking/cheating
Also remember, if something gets popular and you start earning money on it, sooner or later SOMEONE will try to break the protocols or break down the accounts to gain access to servers, informations or cheat to get further items/points in the games. You could also be attacked by DDOS where they bomb your network with wrong data just to crash everything and render the game unusable.
Dont mind it so much for a start, just remember that once you go online, you NEVER know who in the world or where in the world they are. I'm not trying to make you paranoid, but there are sick people who will try to earn money by cheating others.
So think this into your structures, dont show data in network packages that isnt needed. Dont believe data from client always is correct. Validate data on server-side.
This also takes time if you have 100 active players at the same time.
But once you do it, you can sleep much better if it gets to be a big success for you, which I really hope.
That was my thoughts from experience. Hope some of it might be usefull eventhough I didnt quite answer if 100 players are possible.
Infact I would say: YES 100 players is possible, but it depends if they all move at the same time and there is collission testing involved and if you will accept lag or not.
Question: Is it possible to update 100+ objects in the Flash Player over Socket Connections?
Phosphor 2 seems to pull it off.
Maybe the best option was do the physics on server AND on each client, with synchronization (server object positions are overwriting client's). This way all clients get equal lags. Until discrepancy is low (as it should be) corrections will not be noticeable. If you use Box2D, you have both AS3 and C++ version ready. But this is totally different architecture, worth 3 month to implement by itself. What lag do you get on empty/simple arena? In limited time, simplification may be your only option.
I'm coding simple game which I plan to make multiplayer (over the network) as my university project.
I'm considering two scenarios for client-server communication:
The physics (they're trivial! I should call it "collision tests" in fact :) ) are processed on server machine only. Therefore the communication looks like
Client1->Server: Pressed "UP"
Server->Clients: here you go, Client1 position is now [X,Y]
Client2->Server: Pressed "fire"
Server->Clients: Client1 hit Client2, make Client2 disappear!
server receives the event and broadcasts it to all the other clients.
Client1->Server: Pressed "UP"
Server->Clients: Client1 pressed "UP", recalculate his position!!
[Client1 receives this one as well!]
Which one is better? Or maybe none of them?
The usual approach is to send this information:
Where the player is
At what time he is there (using the game's internal concept of time, not necessarily real time)
The player's movement vector (direction and speed)
Then the clients can use dead reckoning to estimate where the other players are, so that the network latency will disturb the game less. Updates need to be send only when the player changes his direction or speed of movement (which the other clients cannot predict), so also network bandwidth will be saved.
Here are some links on dead reckoning. The same web sites contain probably also more articles on it.
http://www.gamasutra.com/view/feature/3230/dead_reckoning_latency_hiding_for_.php
http://www.gamedev.net/reference/articles/article1370.asp
i think the first approach is better. so you have equal data on all clients.
when the physics are simple and the results of the calculations are always the same the second approach is ok too. but if there are random numbers possible you will have different effects on all clients.