1 Scalability and Consistency in Peer-to-Peer Based Network Gaming Author, Wei Li, Student of MSc Computer Games Engineering, Newcastle University Abstract This article presents two key features, scalability and consistency in modern Peer-to-Peer (P2P) based network games, discuss the challenges game industry facing concerns to these two issues and the reasonable solutions for creating a stable and scalable online gaming environment for players, and study the case from the recent team project by the author, a game called sky-race, which is a Multi-platform, (PC, PS3 and Android) experimental video game. On the PC version, the game is based on Peer-to-Peer network as an online racing game with a game-website for players and background data-base to manage all account information. The study case as an instance is to verify and to extend the conclusion of this article. Furthermore, the conclusion aims to find out the possibility to improve the game to be in a higher level with better experience for ordinary players. I. INTRODUCTION Since the concept Network was brought into games industry, Peer-to-Peer has become the most popular way for multiplayer gaming experience. From the famous milestone StarCraft to the latest Multi-console game Titian Fall, Peer-to- Peer is the absolute way for organizing a well-controlled, limited-size online gaming environment. With the participation of all peers, the players thus contribute the computing resource as the memory, CPU cycles and bandwidth to manage the whole shared game states [1]. Unlike server-based massive multiplayer online games (MMO) such as world of Warcraft, Peer-to-Peer based network games have certain number of players for each separate game (always less than 64 players for each gaming area). Hence there are limited internet traffics and pre-created game states could be well-calculated and efficientlyorganized. Even under an Internet environment with restricted resource, a both scalable and consist gaming experience could be built and provided to players. With the possibility of every client could be the server (host) and all game state is controlled by client itself there is not much limit for hosting game in a local area network. Furthermore, the server-based game will limits the deployment for user-involved game design such as Mod and self-made map script as another trend of game style. In the new concept for game design an open mind game will meet the need for gaming community to be part of the game they designed and participate, which Peer-to- Peer based game could support this endless possibility for new elements and ideas to keep the game alive. The scalability, as one of the most essential feature of modern games could go both horizontally and vertically [2], [3]. As game could support more game nodes in numbers or increase the resource on each game node or even create new game states which in this case for online games means it could support more players for the same game or it could putting more content to each player or the original game states. Either of these is the way to put the game to be a new level for gaming experience and keeps the game to be alive and attracted to players. Mod and player-involved gaming level design are good examples as providing the ability for people to design themselves in modern game market, and become a lot more important as the reason for players to play these games. Consistency, on the other hand, is the guarantee for creators ideas built to be reasonable and believed. To keep the game state changed at the same point for each client s timeline [4] is the purpose for consistency and the reason why players could have well-interaction with the both the environment and other players in the online game world. Not like a server-based network game all the game stated are stored on the server and clients are just work as different control nodes and renderers running on the same timeline, consistency on Peer-to-Peer based network game depends on the way network programmers understand the network topology handling all the events and various game state implement the right algorithm for latency compensation or future prediction to keep the game consistent [5], which matters a lot in real-time games with fast gaming pace and even competition like FPS or ACT. Under unknown network situation it is a main issue to keep consistency and keep the game to run smoothly like it designed to be. A not well-organized Peer-to-Peer based network game will easily end up as different situation or even different game state on each gaming peer, which is a total disaster for both game designers and players. The key for a Peer-to-Peer based network game to be both scalable and consistent is to have an efficient game model as the hosting network, making sure each way of internet traffic will not end up jammed and solve out all the potential events that could go wrong on multiple timelines. This paper is organized as these following: Next section II presents the topology on Peer-to-Peer based online games comparing with a server based game on a network layer level and a game design level.section III is based on the scalability for a game on both horizontal and vertical method.
2 Respectively, Section IV describes the algorithm to keep the game consistent. In the following section, will be a case study about our own work. And the end of the paper will be the conclusion for the future of our work through the whole study case. II. PEER-TO-PEER IN NETWORK AND GAMING First thing to be clear is that Peer-to-Peer (P2P) network and the online game based on Peer-to-Peer network game cannot be confused together to be the exactly same concept. Without any doubt that P2P network is based on the network layer of the OSI seven-layer model while any kind of online game is implemented on application layer only. [6] A P2P network refers to a type of decentralized and distributed network architecture in which individual nodes as Peer could be both suppliers and receivers for all online resources, comparing to traditional server-client model while all clients act as consumers only applying for the resources provided by the server. A.P2P network topology A P2P network is originally designed that all nodes could have full functions as both client and server at the same time. This form of design is based on the topology of P2P network while all nodes are equal. As Shown above is a typical Peer-to-Peer network topology while any of the PC/Laptop refers to a node or Peer in the network. Because of the decentralized distribution for all peers within the network any of the peers could be replaced or even removed which will lead to no significant change for the whole architecture of the network and will not all other nodes function as they used to. [5] This unstructured network does not have a particular form to be under maintenance. Hence there are fewer limits for P2P network to form and organized than traditional server-based network since the server itself is always indispensable. With P2P network architecture the amount for network Fig. 1. Network topology of a typical Peer-to-Peer Network. Note that all nodes within the network have accesses to every other node directly, which the main difference comparing with a server-client model. traffic could be noteworthy. Assuming that one node with the average amount of data for both downloads and uploads in a certain time of period as N and there are n nodes in the switching network and the maximum amount of network traffic could be N*(n-1) for one node and N*n*(n-1) for the whole network. B. Comparing with Server-based network The traditional client-server model is a network structure while the server the key node or the central node for the whole network. The information transferring is only through server to client, while the clients themselves cannot communicate with each other without server. With unequal role for all nodes in the network the server is Fig. 2. Network topology of a typical Server-Client model Network. As it presenting above, all nodes within the network have the only access to the server the key to collect and transfer information. This sort of network structure has to be built based on the server first and then connect with all its clients to keep every function mechanism. Comparing with P2P network topology, server based network is stable in structure and easy to control the network traffic as well. The whole amount of network traffic could be maximum as N*n. And the larger size and more amount of nodes in the same switching area the heavier tasks for routers and switches on the switching layer, leading the network unstable. C.P2P as a concept for gaming At the very beginning of multiplayer games the Peer-to- Peer network was the structure get used the most by game programmers, with each computer exchanging data with other in the area in a fully connected topology. Still today s RTS and even PS3 games with small amount of player in the same game are using this kind of model because it is easy to form and scale. As mentioned at first of this section a Peer-to-Peer game is not equal to a Peer-to-Peer network. At first the network game is between two players where they exchange information with each other, so it is based on Peer-to-Peer network. However by the increasing numbers of players in the game, the Peer-to-
3 Peer network structure is causing some sort of issues as latency by overload network traffic or unorganized players data by lack of central node. So there is need for a node to control all information called the host. With the host node collect all the updates from other nodes while the rest of nodes could still communicate with each other. Technically speaking in communication field this type of topology network is not purely Peer-to-Peer because there is the key node as host which if shut down the whole game state will not run properly. Fig. 3. The Network topology for a Peer-to-Peer based network game. Notice that the network structure are the same with a Peer-to-Peer network, however there is one of the nodes acting as the host taking most of the network traffic transferring them to the right destination. In the topology network for Peer-to-Peer gaming the host is the most important role for the game state to switch between each players. Once the host player leaves the game the game in this network area will shut down unless another player turned into host again. With the virtual server, the host managing the information that needs to be received by every player like Player s ID or other update and players themselves send and receive information that only designed to certain player like in FPS game a shot hit. Under the Peer-to-Peer concept in gaming the update information taking the most part for internet traffic is handled by the host reducing latency and the unique information could still be send to certain players fast and efficient while there is no need to put dedicate server as the traditional server based network. The different between a Peer-to-Peer and Server based game are whether the non-host/non-server client is able to communicate directly to other client and whether each client contains the whole game states. In Peer-to-Peer game players are allowed to set a game with their own client while on Server based game player are only allowed to join a game that was already hosted on the server. Not like MMO games are based on gaming server, the Peerto-Peer network game is able to form and shut down the host any time while having no influence on other gaming area. Thus it becomes the best choice for small network games that only holds in small size. III. SCALABILITY, GO MORE OR GO LARGER? A. Scale horizontally. The scalability is the term to indicate the capability for network to handle increasing uncertain number of elements. A stable network gaming system that could be scalable at the same time will meet the requirement for real gaming situation and what the players need for keep the game alive and fun to continue playing. While in Peer-to-Peer based network game players not only shared information but also contribute their CPU cycle, memory and most important the network resource (network bandwidth). However sometime the P2P based online game could have limit of scalability. [7]Within the structure of Peer-to-Peer, there will be higher delays when the number of connected nodes increased also as when the network scales to support more nodes. This is a network issues because once the network scales horizontally the more nodes are connected to each other the more network traffic need to be handled, the more amount of data each client need to processed on the network layer. By adding one single node into the network there will be 2N*n more amount of data going through the network. Fig. 5. The horizontal scalability for a Peer-to-Peer based network game. Notes the fact that once there is a new node the network, not only the node sets connection to the host but also connect other nodes in the network. Fig. 4. The game areas that separate from each other so that if one host shut down it will only have influence on the area that host located, while the games in other areas will continue. The solution for reducing the limit is based on the modern server-based structure. [7] By setting region for each game, players got divided and one got chosen as the server of the region and manages most of the data that need to be exchanged. Without the problem of resources limit, the network game could just scale horizontally like a server-based network while still holding all the advantages from Peer-to- Peer based network game [9].
4 Although, it is possible to achieve the scalability for Peerto-Peer Network game, it is still limited by the hardware condition for the host has been chosen, such as the CPU the running memory or even the bandwidth to handle network data. Therefore with limited hardware, the horizontally scalability of Peer-to-Peer based network game cannot compare with a Server based network game which hosted by a dedicate server. B. Scale vertically As it mentioned in this section before, the scalability is limited horizontally since the client worked as host cannot meet the endless needs. Therefore the vertical scalability should be considered by game designers to make a Peer-to- Peer based network game. The definition for vertical scalability is to put more resources to a single node in the network, in the filed for game means putting more game state or in-game multiplayer function to scale up the game. Even though the games are held in different game regions and totally separate from each other, one way to scale up is that the host of the game could act as the client on a larger size server-based network where all the in game content could be extended. One decent case is the Battlelog and the in-game ranking/weapon system that was used by Dice in battlefield series, after the each game finished the related information such as the game record the ranks and the achievements are collected by the host or the game-server and send to the dataserver which hold a data-base which all the players could share whether they use Peer-to-Peer or dedicate Server to hold the games. Battlelog works as an interface for player to access their in-game content with a website browser and based on the huge data-base that controlling all the information collected and processed by the host and the dedicate server. Fig. 6. Battlelog and the games based on Battlelog sharing the same database Another method for the game to scale up is based on any of the peer in a Peer-to-Peer game. With the development of mobile devices and Wi-Fi network, these devices could easily works as additional elements or even interfaces for the network game. Also taking battlefield as an instance, there is the APP on ipad where player could use it as a second screen to locate the map of the real-time game and even play the commander mode of the game with ipad. By using the left resources from nodes connecting to additional nodes the network could go scale up, and have more possibilities for game designs. Fig. 7. Two different methods to extend the scalability vertically: from the host to upper level or from the client to lower level. IV. CONSISTENCY FOR PEER-TO-PEER NETWORK The game state must stay to be consistent within the same region from the game, which is the guarantee for the players to interact with each other and the environment. Depending on the network architecture for Peer-to-Peer game and the mechanism of the game state changed is all client-side. Therefore the consistency for game states should be decided mostly on the client/peer side among all the players. Since they are organized in the network that could directly communicate with each other, the most important task to ensure the consistency of game states is to classify the games states are whether local or global or should be just existed among the players who is in the interaction of the state. Global state/events which will related to all the players among the network like each player s position and facing direction in a FPS game. On each of the players screen the other players position and facing direction should be exactly the same so the game can run properly. Events like a bomb explodes on the map destroy the building and hurts all the players are also considered as global state. These global states are the information that should take almost no fault-tolerance and should be replicated and updated as soon as it happens since they need to be consistent all the time. Events or states that should be controlled over the whole network handled differing from each other could use the Distributed Hash Table (DHT) as in [8], listened by the virtual server and send the feedback to all the Peers in the game. The unique identifier from the hash table will ensure that the same event replication will not receive twice by the same node on the Peer-to-Peer network controlled by the host, which solves the most difficult game state consistency issue. By contrast, local states such as the physics between player and the environment or the local value with no need to inform other players do not need to be consistent among all players. These game states are able to tolerant some of the fault in the virtual world. Such as the amount of force the player received from the environment when it interacts with one static object in the game. [10]Other players have no need to know the actual force which is the local state, and they just need to
5 know the player s position changes after the event in this case the collision happen which is the global state. Identification on all the potential events/states to be whether global or local will not only reduce the network traffic, but also is the key to make sure that among the whole network all game states are consistent and traceable. Furthermore, focus on reducing global states to be the minimum and with the host player as the virtual server, the Peer-to-Peer based network game could be fully functional as the same way the server based network game maintaining its consistency. V. Case Study The case this article going to discuss about is a racing game with online game mode as a team project from university. The author is one of the team members who in charge for network and multiplayer game design. Unlike traditional car racing game, the team project sky race sets the game style as fighter jet racing in canyon during air combat. As the combination for racing game and FPS game, the game is aimed to provide fast-pace gaming experience for players and build graphics physics and network engine for future games. The multiplayer game, as one of the most important part for the game has met a huge amount of scalability and consistency issues during the development. Hence it is worthy to discuss and gain experience from the solutions. The gaming network is designed based on TCP nonblocking socket programing rather than the solution with UDP because it is easier to manage and meet the original game design objectives. Since the game is required to support at least four players in the local network area, choosing TCP will not cause problems such as latency in an unideal network environment and the programmer could focus on the scalability and consistency issues. collision detection,we will discuss that in the later part of this section.) from client to other clients without getting the information processed on game state level. Clients hold all the game states and are able to communicate with each other through virtual connection which making this case a Peer-to- Peer based network game. Besides the clients connected to the server, there are also a database and mobile devices connected to the server. They are all the parts for the scalability elements for the gaming network. The smart phone, is working as the second screen for the player in game providing a game map and the access to deploy a bomb based on the touch on the screen of the smartphone. The database is in charge to store the information of the game which we will discuss next. B. Scalable network game. The single player mode of the game is designed to be one player playing against 1-7 AI players. While the multiplayer game mode supports 4 players most and could extend to support 8 or more players after adjusting in game balance content (such as the width of canyon and the spawn position for 8 players) Before the actual game start, there is the game state called game lobby where players choose their host and connected with each other before the game. There could be 2 to 4 players in the game lobby and all player press the ready button the game will start. A. Network architecture and the virtual server Fig. 9. The game lobby screenshot. There are the host of the player name and the names for players who are already in the room which the all players in the lobby can see. There is also the chat box for players chatting with each other. Once all players in the room press the I m ready button the game will start at the same time. Fig. 8. The network topologic used in the game sky race. The lines indicate the connection between nodes while the virtual lines between players indicate the route for unprocessed message from one player to another transferring by server. The center of the network structure is the communication server, which is the application that could be deployed in any player s PC or on a dedicate server. The client run by player holds almost all the game states while the server is in charge for transferring information(also a very small size for physics The horizontal scalability is based on the communicate server in the room. As for the communicate server which is running on the host player are capable, it could technically contains more nodes in the network and supports for players in the same game. With the update rate of 30HZ on the communicate server and 60HZ game graphics frame rate, it is well-tested that the whole network is able to support four to eight players and keep the game states in each player s client working properly.
6 Furthermore the communicate server runs with the host client so all the players outside the running game is still able to player the game and forming endless gaming areas. On the other hand the game itself could scale vertically as we already referred before in the background research. Fig. 10. The screenshot of the android game. In the middle there is a sky radar work as the mini-map showing all the players location and the player and enemy are marked separately as red and blue points. So the player could drop a bomb by touching it on the screen according to relative positions. From the client side of view, the game supports the mobile device (with Android OS and Wi-Fi access) to be a second screen to support the player. After authoring on the communicate server the mobile device will be connected to the player s client that it is going to support with as the new extended content for the game. purchase weapons jet models or even DLC online using virtual currency which significantly scale up the game in scalability. As the article indicates above even the scalability on horizon for Peer-to-Peer based network game will reach a limit for lacking of resources and may not be able to compare with a server-based network game in number. However as we presents in this section the game still extend itself to a new level. With the android-host-database-website model the game could scales and even with PS3 project, providing players with experience more than just a game. C. Consistency challenges and solutions Since the game sky race have elements on both racing game and FPS game the consistency has become harder to solve and verify according to a huge amount of data need to be transferred rather a traditional single game just updating position. It is the first priority problem to identify the game state whether to be global or local. The basic game state model on peer side is the playerenemy model. The player refers to the actual jet controlled by the player and the enemy is the replication of any other player in the same network game. Both player and enemy contain a unique ID send by host while initiation. And every message transferring on the network could be transform to be a packet with data structure for message type, ID, and actual in game stats. When the game is finished, there are 25 data types and more than 30 different values in a single packet indicate different situation and game states that need to be exchanged and notified through the network. Fig. 11. The screenshot for player profile page of the website. All these data are collected and sent automatically by the host and players from different gaming area could share the same database and website. From the host side of view, all the hosts are connected to one shared database which contains all players information. The database on the other side is also connected with the website which players themselves can scan and manage their in game account. After the game finished, the host will collect and send all the information to the database while the website can print all these information such as winner record immediately. There is also the online store for players to Fig. 12. The flow chart of how the game handles constantly values like positions and health all the time. Just like all the game containing multiplayers that elements like position, facing direction, and health should be always updates to make them consistent through the whole network. By uploading own information to everyone else among the network, everyone else will receive the exactly same player information to update their replications and keep consistent all the time. These game states that need to be exactly the same are surely global states. And the update rate is also controlled on client-side with time steps keeping it to be 30HZ in order to keep the traffic. At the first of the data structure for each update, the network packet will contain the player s ID and the
7 packet type called action means it should be processed immediately and constantly so the receivers could end up with the same result, and create the same consistent game world. The android device, on the other hand is using almost the same mechanism transferring and updating the actual players position all the time. By calculating the position on the cellphone screen it shows up on the mini-map just like it did in game. One thing need to be noticed is that the update rate on android is 10HZ since the Wi-Fi cannot hold too much network packets in a certain time of period. Fig. 13. The screenshot for player shooting and getting hit by machine gun As the features for fighter-jets, there are various weapons provided for players to shot against each other while the race. Getting the shooting action and hit feedback consistent and not to be over transferred is the main challenge for reducing network data and make the game logically correct. If one player in game got hit by another, the damage should be noticed in all the clients and should not be over made in any single client (like that 1 shot causing double damages bug in the early lunch of battlefield 4). The damage model should be accurate otherwise it is meaning less to play an online shooting game. Hence the consistency for weapon and damage is the major issue for network and the programmer has put a lot of work on it. to everyone in the network because the damage will decrease the health point which will update at 30HZ all the time. Another type of weapon is the missile which functioning differently to machine gun. Once the player fires the missile, the missile will be created and move to the direction that player aimed at. The missile will hit the player only when it explodes and the player is inside the exploding radium. So there is chance that the missile will not hit the player after it is launched. While it is a process for missile flying to the target and player got chance to dodge from being hit. If nothing to do through network, and all collision done by client itself, the hit result might be different because the latency issue, which the player plays the in different timeline. All these game state might cause network inconsistent problem and furthermore some memorize leak on the client side. It is complicated but still could be solved by identifying all the potential game state among the network. The solution for missile to be consistent through the network is to make two different types of missiles. The one created after player press fire button is player missile (m_0 in the chart) and the replicates created after that in other clients are enemy missile (m_0* in the chart). Every player missile is created with one unique hash key while all its missile replicates are holding the same value. Therefore once the missile got explodes by hitting player, the program could locate the certain missile and its replicates making them explodes with consistency. To make it reasonable for player to dodge from the missile, the physics system on the client is only able to detect enemy missile hit the player s (the actual player jet but not the enemy jet) physics collision sphere. So if the player dodges the enemy missile the missile will not explode at other client as well because the program will not detect the player missile against enemy or enemy missile against enemy. And keep the dodging to be consistent. If the enemy missile hit the player, the client will send the missile s hash key to everyone else and other clients get the hash key locate the missile replicates and display the same explode effects among the network. Fig. 14. The flow chart for consistency of machine gun Machine gun as one of the weapon in the game holds a unique game state; the damage got detected and calculated immediately after player press the fire button. Therefore once the system detected the player hit the enemy on the one client, the client will send that message through host to the one client with the same ID as the one enemy got hit by the player. The player, who got the message, gets hit and put the damage to the player s jet. There is no need for client to send that damage Fig. 15. The screenshot of same missile explode on the terrain from two players view. Another situation need to be considered is the missile explodes when it hit the terrain, if we use the same hash key model to cast the information across the network may cause the explode happen before the missile replicate hits the terrain in term of the network latency and that replicates runs in the time line later than the one send the hash key. Ending up the missile explodes at the incorrect position before hits the terrain. If the enemy missile hit the terrain collision are enabled without sending hash key through network, the time they explode across different time line might be different but
8 the position will be the same since they are the same replicates. And that is totally fault-tolerance by the requirement of consistency. Fig. 16. The flow chart for consistency of a missile Furthermore the over send hash key may lead to multiple exploding order for the same missile replicate which may cause not only inconsistency or even more serious the memory leak. This model meets the requirements for consistency reduces unnecessary network data to be transferred and is totally reasonable for game logic. method should be implemented since there are two clients calculated the collision. To make it simple and reasonable the programmer move the player-player collision to the host, so only the host is checking for the detection and impulse method calculation and send feedback to other clients, with which makes the impulse method consistent. VI. Conclusion Based on the background research and the case study in the article, the scalability and consistency for a Peer-to-Peer network game are worthy to discuss. To make the game scalable, there are two different methods as extending horizontally by adding more nodes (players) into the game, or scale the game up by holding game in different area supporting for additional game content or even second device for the game. One thing need to be mentioned is that the scalability is not only a technology issues but also a game design state. Only by combining the technology with reasonable game content, the scalability of the game could really meets the need for the players. On the other hand, the consistency has always been a quite important challenge for network game and especially for Peerto-Peer game since all the game states are managed on the client side. Even though, we still got the solution. With identifying potential problem to be globe and local and applying with different strategies, it is still possible to keep the game consistent. The case we discussed before, due to limited develop time and lack of experience may not be perfect to solve the scalability and consistency, however it has become a good start for the team to get into the real game design and implement. REFERENCES Fig. 17. The flow chart for consistency of player-player collision Even through the Peer-to-Peer based network game could keep most of the game state to be consistent, it s hard to solve the physics collision between the player and their enemy. Because the enemy is the replicate of other player and update only position but not velocity. Therefore it will not return proper physics impulse method since we don t have the velocity on one of the two objects. It is able to solve that by containing the velocity as just a value for calculation but still it is hard to decide which impulse [1] B. Knutson, H. Lu, W. Xu and B. Hopkins Peer-to-Peer Support for Massively Multiplayer Games IEEE 2004. [2] M. Maged, M. Jose E. S. Doron and W. Robert W. Scale-up x Scale-out: A Case Study using Nutch/Luncene 2007 IEEE international parallel and distributed processing symposium. [3] H. EI_Rewini and M. Abd-EI-Barr Advanced Computer Architecture and Parallel Processing ISBN 978-0-471-47839-3 Oct 2013. [4] G. Morgan Highly Interactive Scalable Online Worlds,Advances in Computers (2009),Elsevier [5] C. Neumann, N. Prigent, M. Varvello and K. Suh Challenges in Peer-to- Peer Gaming. http://ccr.sigcomm.org/online/files/p79-v37n1p-neumann.pdf [6] G Fiedler, What every programmer needs to know about game networking 2010. http://gafferongames.com/networking-for-game-programmers/what-everyprogrammer-needs-to-know-about-game-networking/ [7] A. P. Yu and S. T. Vuong. Mopar: a mobile peer-to-peer overlay architecture for interest management of massively multiplayer online games. In Proceedings of NOSSDAV 05,pages 99 104, june 2005. [8] A. Rowstron and P. Druschel. Pastry: Scalable, distributed object location and routing for large-scale peer-to-peer systems. In Proceedings of the IFIP/ACM International Conference on Distributed Systems Platforms (Middleware), November 2001. [9] A. Rowstron and P. Druschel. Pastry: Scalable, distributed object location and routing for large-scale peer-to-peer systems. In Proceedings of the IFIP/ACM International Conference on Distributed Systems Platforms (Middleware), November 2001. [10] B. N. Levine, J. Crowcroft, C. Diot, J. Garcia-Luna-Aceves, and J. F. Kurose. Consideration of receiver interest for ip
multicast delivery. In Proceedings of IEEE INFOCOM 00, Mar. 2000. 9