A Measurement Study of a Massive Multi-Player Online First Person Shooter Game in Play-Station Networks

Transcription

1 A Measurement Study of a Massive Multi-Player Online First Person Shooter Game in Play-Station Networks Mohammad Z Masoud, Xiaojun Hei and Wenqing Cheng Department of Electronics and Information Engineering Huazhong University of Science and Technology, Wuhan, China eng.mohammad.masoud@gmail.com, heixj@hust.edu.cn, chengwq@hust.edu.cn Abstract Massive multi-player online games (MMOGs) have been attracting thousands of millions of participants on the Internet in the past decades. The newly emerging smart phones and game consoles together with PCs have enlarged the player base to an even larger scale. The increasing game traffic may generate significant real-time traffic across different ISP networks. In this paper, we conducted a measurement study of the traffic locality property of a popular online game, Call-of-Duty (CoD), which is a hybrid peer-to-peer (P2P) client/server massive multi-player online first person shooter (MMOFPS) game in the play station network (PSN) over the Internet. To facility our measurement, we designed and implemented a peer crawler over the PSN. Our instrumented crawler applies the principle of the ARP poisoning attack in our justified scenario so that our crawler is able to penetrate the PSN to harvest player s information successfully. We analyzed the measurement results for finer granularity at the autonomous system (AS) level compared with previous measurement studies. Our results show that the sessions in this CoD game are constructed with players locality in mind. Nevertheless, optimized locality-aware game sessions are yet to be found. Insights obtained from this study may be valuable for the development and deployment of future P2P online gaming systems. Index Terms Call-of-Duty, Peer-to-Peer, Locality, Massive Multi-player Online First Person Shooter, Play-Station Network (PSN) I. INTRODUCTION A new class of real-time interactive social networking games, namely, Massive Multi-player Online Games (MMOGs), have attracted thousands or millions of users playing together at the same time. MMOGs have four types of games: role-playing (MMORPG) (i.e. Worldof-Warcraft (WoW) []), real-time strategy (MMORTS), first person shooter (MMOFPS) (e.g Call-of-Duty [2]) and social networking games (MMOSG) (e.g. second life [3]). The performance requirements of MMOFPS deviate very much from other MMOGs. MMORPG and M- MORTS can tolerate seconds of network delay while MMOFPS cannot tolerate delay more than 80 milliseconds [4]. MMORPG and MMORTS allow thousand of players playing together in the same time, while MMOFPS divides the players into small farms or groups with 0 50 players in each group. The session s duration of MMOFPS games is in the order of minutes. Dedicated computer systems have served as the game platform for MMOGs in the past decades. Nevertheless, mobile phones and other game consoles have started to support MMOGs with a fast increasing number. Microsoft Corporation and Sony Corporation are two major device manufacturers and service providers for MMOGs. In 2005, Microsoft released Xbox 360 and Xbox Live network, which allows players to play online for a small fee. In 20, Microsoft claimed that 53 million Xbox 360 consoles were used worldwide [5]. In 2006, Sony launched the Play-Station Network (PSN), which allows users to play online freely. PSN claims that more than 77 million online registered users have registered and played through approximately 30 servers [6]. It is technical challenging to develop suitable networking architectures so that 30 servers in PSN are able to provide satisfactory gaming experiences for 77 millions users. These games have been supported conventionally by the client/server architecture, while some game providers have moved from client/server to P2P architecture or hybrid architectures [4]. In particular, hybrid P2P and client/server architectures [7] have been promising since peers capacity can be utilized to provide self-scalability and the delicate infrastructure can provides game controllability and performance guarantee [8]. Successful deployments of hybrid architectures have enabled game providers to provide MMOGs for thousands or millions of players simultaneously. On the other hand, these thousands or millions of players may generate huge traffic load for ISPs. We are interested to examine whether these games consider peer locality to reduce the transit load across ISPs. A number of studies have been carried out to study the traffic loads of P2P file sharing, P2P live streaming systems and P2P VoD applications and to introduce locality-awareness into these systems to reduce cross- ISP traffic load. A few measurement studies have also been conducted for online gaming applications to examine user behaviors [9] and performance issues [0].

2 However, gaming traffic locality issues remain to be investigated at large. In [], the authors reviewed the challenges for P2P online game developers with the focus on performance issues from the player/developer s perspective without considering ISPs at all. In this paper, we conducted a measurement study of Call-of-Duty Modern Warfare 2 (CoDMW2), a popular MMOFPS game that has recently achieved a Guinness world record for the most played online game in the history [2] with the case study of peer locality issues on PSN. We examined PSN for two reasons: first, it is a large-scale free live gaming network, which has been attracting millions of players; second, unlike CoD in the PC platform that depends on public or private game servers, CoD in PSN applies a hybrid C/S-P2P architecture [3], [6]. In this architecture, players are divided into groups of ten to eighteen. Players in each group elect one player to serve as the session host. All traffic in the session pass through this elected host with no server support except very minimal signaling traffic to the game server to manage player s list and scores. Our work in this paper can be summarized in two-fold as follows: We measured the CoD traffic at two critical points in the game: constructing the player list and selecting the session host. Our instrumented crawler applies the principle of the ARP poisoning attack so that our crawler is able to penetrate the PSN to harvest player s information successfully. We developed a tool to measure the AS hops between any two IPs to study the traffic locality characteristics of the game sessions in this CoD game. The rest of this paper is organized as follows. We end this section with the related work. In Section II, we present an overview of this CoD game and player s operations. In Section III, we outline the measurement settings, data collection and measurement experiments. Then, we proposed the locality metric in Section IV. In Section V, we describe the measurement results and finally we conclude the paper in Section VI. A. Related Work In the past decade, many measurement studies have been conducted for MMOGs to study various aspects such as player s behaviors [4], system architecture and communication protocols [5] and their impact on the quality-of-experience (QoE) [6], [7]. MMOFPS is the fastest growing category in MMOG [8] because it is an attractive option for developers and publishers. To the best of our knowledge, we are the first to conduct a measurement study the traffic locality issue in a MMOG over PSN. Previous measurement studies on MMOFPS games can mainly be classified into two categories: active and passive. Active measurements can usually be conducted in three approaches. First, the log files recorded by game servers have been analyzed heavily to study the game traffic characteristics. In [4], the authors analyzed the logs files recorded in the HalloFit project from the main game servers of Halo3 [9] to study the gaming performance of the teams and players. Nevertheless, server log files are usually not accessible for most practitioners. Second, public game servers can be instrumented and provide hosting service over the Internet to collect user s game playing data. In [5], the authors provisioned a public game server for Counter Strike (CS) and then examined the logs files of this server to study the player behaviors of the game. However, this approach may lead to the potential piracy concern since game providers require a legitimate authorization to provide the gaming service on the Internet. Besides, the above two approaches usually only study the game server side without considering the client side. Third, game testbeds can be constructed with both clients and servers to study the complete picture of the games.in [20], a lab with four clients and a server of Unreal Tournament 2003 (UT2003) was implemented to study the impact of packet loss and latency on the game performance. In [2], the authors implemented a gaming lab with 8 clients and a server. These clients were divided into two groups with a NIST Net router placed between them. This router was used to configure various network parameters. Different games of different categories were used: CS, UT2004, Need For Speed Underground 2 and Warcraft 3. The main focus of this study, though, was to demonstrate how different games behave under different network settings. In [7], the authors conducted a measurement study of Quake III based on their collected 3500 hours of the game data. The authors deployed their game testbed in LAN with three nodes (2 clients and server). Different networking scenarios (wired and wireless) with underlying physical layers were configured for studying the game playing performance. Unfortunately, articulated game testbeds in LAN only capture parts of the actual game scenarios on the Internet. Besides, it is difficult to examine the scalability issues of massive MMOFPS such as the impact of the number of players on the gaming performance. In passive measurements, a traffic sniffer is deployed at appropriate vantage points to capture the game traces. In [22], the authors captured traces of 20 players in 8 games. These traces were analyzed to study the caching efficiency of game proxies to enhance the playing experience and to reduce the traffic load. The authors reported that these proxies are able to reduce the traffic load up to 50%. Note that none of these studies examined the locality

3 issues of game traffic and these studies only focus on the PC platform. Nevertheless, other platforms (i.e., smart phones and game consoles) have shown a much large penetration into players lives. We are motivated to conduct a measurement study of CoDMW2 on a hybrid client/server P2P architecture on the console platform over the PSN. PSN Servers Server... Server 2 3 II. CALL-OF-DUTY (COD) CoD is a massive multi-player online first person shooter game with stand-alone and online versions. This game was played initially from PCs and then moved to game consoles and mobile devices. Over years, 4 major versions have been released starting from CoD, 2 and 3, through to (Modern WarFare and 2) and finally (Black Ops) [23]. In this paper, we conducted a measurement study of the CoDMW2 online traffic in PSN. In this section, we fist outline the structure and signaling protocols of CoDMW2 on PSN. In CoDMW2 on PSN, each player has a unique PSN ID. To join a CoDMW2 game, a player should first log in PSN with his/her PSN ID. Then, the player selects a CoD task to join the game. Subsequently, the player configures a game difficulty level to play and finally starts the game. When a player logs in the PSN, he/she receives some gaming statistics, such as, the number of online players at that moment. The player also retrieves his/her friend list and the profiles of these friends as shown in Fig. (Step ). CoDMW2 is a MMOFPS game, in which the players are divided into groups and each group forms a game session sharing the same game map. The total number of players in a game session varies from 8 2 or 2 8 players for two game levels. Each game session usually lasts up to 20 minutes. A player can initiate a CoDMW2 game or join an ongoing game. If a player joins an on-going game, after the player selects a game level, the server sends a list of on-going games to this player (Step 2). Then, the player selects a game with empty player slots for participation (Step 3). After all players in a game are in position, the game starts. Each game has a host peer, which is elected by the players in the session based on the network performance. This host peer communicates with all other peers in the game session for updating their states, the players locations and the re-born locations (when an avatar dies). When the game starts, players will be divided randomly into two teams of avatars. Every avatar from each team tries to shoot as many avatars as he/she can from the opposing team (enemies) without dying. Each time an avatar shoots an enemy, his/her team receives one point. The game session ends when any team reaches the threshold value of the points for the session (from one to two hundreds points) or when the session s duration reaches its deadline (from ten to twenty minutes). Fig.. The major steps to join a CoDMW2 game in PSN The simple architecture that this game uses poses a question about the locality of its peers and if it is concerned about its load on ISPs. To answer such questions, we have to know where the locality can be implemented in the game. These games have two possible position points to implement locality: in the peer list and in the process of selecting the host. The first point is comparable the locality in P2P file sharing and VoD applications. The second point is similar to the locality in selecting relay nodes in VoIP applications and NAT relays. The first point of locality is hard to implement for two reasons. First, players want to find a session to join as soon as possible otherwise the start up time will increase to minutes rather than seconds until free peers are found near to one another in the underlying network. Second, this game allows players to join a session via friend list. The list allows players to follow their friends to sessions where these friends are currently participating. The player, can join their friends in these sessions and play if there is a free space in these sessions. Keep in mind that it is imposable to control the location of the player s friends. III. MEASUREMENT This study consider an indirect edge sniffer method to collect data packets and traces from console. To collect the traces, we used a PC together with console. We tried to make the PC a gateway for by connecting it to the Internet and connecting to the PC. Unfortunately, the could not login to PSN. To collect traces we needed the to login to PSN then start collecting the data, therefore, we moved the PC and connected it to the same switch as the. We used an ARP poisoning technique to collect our data [24]. The steps that we used were as follows: ) Connect the PC and to the same switch. Close all applications on the PC 2) Start the game by logging on to the PSN network Fig. 2(step ). 3) On successful login, we implemented the ARP attack on machine (step 2). The purpose of it is to spoof data in LANs. The principle of this spoofing depends on the lack of security in the ARP protocol. Our PC sends a fake ARP

4 PSN Servers 3 Our Clawler Internet Switch 2 Fig under our contro The Environment reply packet that maps the gateway IP to its MAC address to the under our control. In this way, our crawler poisons the ARP table in the and introduces itself as its gateway. This spoofing attack will direct all traffic to pass through our crawler (step 3) 4) Start collecting packets before finding a game and stop collecting data after finishing of one game. This what we call a session To collect the packets, we used Wireshark. To implement the ARP poisoning attack we used Cain & Abel software [25]. We collected data for one week, that is from, Friday 26/8/20 to Friday 2/9/20. Table I shows some of the data taken by the traces. TABLE I TRACES Parameter Value Number of Session 700 Number of unique IPs 3250 Number of unique ASN 68 Number of the extracted 74 AS paths A. Analyzing the Traces To analyze the traces, we implemented our own service using Perl scripting language. Our service takes two IP addresses, maps them into their compounding AS numbers and then calculates the AS hops count between them. To extract the AS hop count between nodes, we harvested 68 unique ASes. 74 different paths were constructed. To map these paths into their corresponding AS paths, two methods were used. First, a traceroute was executed between our host and the other nodes. The output IP addresses were mapped into their corresponding AS numbers by using our own mapping tool [26]. Subsequently, the unique ASes in the output files were counted to construct the AS path. This method is popular in locality study [27]. Unfortunately, 5.8% of the AS paths have been extracted with this method. Second, we crawled CIDR-report [28] ASes information. This report has the IP addresses that belong to each AS, the up and the download neighbor AS s to any AS. We used these data to construct an AS graph. Subsequently, Dijkstra algorithm was used to extract shortest paths between ASes. this method was used to extract the remaining 94.2% AS paths. We have used these the two methods together for two different reasons. First, the tracerout software and the IPto-AS mapping process method is limited in way that the source address of any measuring process must be accessible to run the tracing process. Second, the graph based method that we created by using CIDR-report data is effected with Internet BGP path inflation issue which makes its accuracy questionable. B. Harvest Player s IP Addresses Unfortunately, the returned peer lists from the server are encrypted. We could not read the peer list from the packets directly but after 700 sessions, we observed that our peer after receiving the list from the server started to negotiate with multiple peers before beginning a further session. It used small-sized UDP packets for this purpose. Subsequently, the peers commenced the further phase of negotiation by using bigger sized packets with a small fraction of the peers. The size of this small latter group of peers is equal to the number of players in the session. This negotiation takes place in order to select a host. Whenever a host is selected, the size of UDP packets becomes very big for a short period. These packets are responsible of the synchronization of the environment and the initialization of the game. These packets are only received from one peer in the game with whom continued to communicate until the end of the session (the host). These peers, therefore, in the negotiation are the peers from the list. We counted them in all sessions, deleted duplications and converted IPs into AS Numbers through the service that we wrote. Then we converted these ASN s into country names with our service. IV. LOCALITY METRIC The performance metric that we used is the average AS hops count in a session ( D). As we have shown, CoD uses the idea of a small swarm or groups that play together in one session. For each game session, a peer will be selected to act as a host. The host has a coordinating and controlling duty and all players in a session are directly connected to it. To have a

5 localized session, the host must be selected dependent upon localized criteria. To do this we used the minimum average AS hop count between each peer and all other players in the session (Equation (c)). To calculate it, we need to measure the AS hop count between each peer and all other players in a given session, then add these AS hop counts and divide them by the number of players. Subsequently, for every peer, we selected the minimum average AS hop count to be the optimum host for this session (optimum depending on the location). X N, (a) i d(x, Y (i)) D(X) =, (b) T Y (i) N X, i =, 2, 3,..., T, Host smallest( D(X)), (c) Where d(x, Y ) is the AS hop count between Peer X and Peer Y and T is the total number of player in the list N (in one session). This metric is valuable. If the D of any peer is less than one, it has a number of important meanings. First, the peer has many players in the same session from it s AS. Secondly, peers that connect from outside this peer s AS are close with AS hop counts. V. RESULTS Our results are divided into two sections: the locality in peer list and locality in selecting the host. In the peer list, we measured whether the peers tend to select peers from the same AS or if the game itself enforces the peer to select from the same AS. In the host selecting section, we will use our metric to measure the average AS hop count between any two peers in any session. After that, we will see if the host of the game is the one with the smallest average AS hop count between other peers. A. Player List addresses from China and only from two AS s. This graph shows that the game has sense of locality when sending the list of players. On the other hand, Fig. 4 demonstrated another point of view. This figure shows the top 20 AS s with the highest number of IP s we have played with. From this figure we can see that Japan took the first rank with the highest number of IP s and the highest number of AS s with whom we played. We can also observe that we moved from number one with the highest number of negotiated peers to number three with only 80 players against 800 players from Japan and 2000 player from USA. Fig. 4. Top 20 AS s with the highest number of player we have played with Fig. 5 shows the CDF of the total number of IPs we played with. This figure shows that more than 95% of the AS s with whom we played have a total number of IPs less than 00. This means that the players are distributed among a big number of AS. This leads us to conclude that even if the game concern about locality, it needs more tuning CDF Total number of IPs From the Same AS Fig. 5. Total number of IPs within the same AS Fig. 3. Total number of IPs of Top 20 ASes in the harvested dataset Fig. 3 shows the total number of IPs which we negotiated with and the number of ASs from the same country. This figure shows only the top 20 ASs with the highest number of IP addresses. The graph shows that we have negotiated with the highest number of IP B. The Selection of the Host Two methods have been used to measure the average AS hop counts. We used trace route and the graph method. Unfortunately, 5.8% of the AS hop counts that we collected can only use the trace route method. In total 94.2% of the paths needs the graph way to estimate their AS hop count path. To prove the accuracy of this method we have compared our 5.8% AS hop count paths that we calculate by using the other methods with the graph

6 method. We observed that 8% of the paths have the same length. Another 2% has AS hop count path with an extra AS hop. The rest 7% has more than one extra AS hop. Fig. 6 shows the CDF of the average AS hops count. Three values have been used: the smallest, the highest and the real value of the real host of the session. From this figure we can observe that, 50% of the sessions selected a host with 2.2 average AS hops count. On a second occasion, we observed that the minimum number could be reduced to.5 if the players selected another host. Fortunately, The hosts that have been selected in all session took an average AS hops count closer to the minimum value but far from the maximum value, which has 3.5 average AS hops count for 50%. This also leads to the same conclusion that the game concern about locality but it needs more tuning. CDF Min Selected MAX Avg AS Hop Count Between the Host and The Other Players Fig. 6. CDF of the average AS hops between the session host and other players VI. CONCLUSION In this paper, we conducted a measurement study of a MMOFPS game, CoDMW2, on the PlayStation Network with the focus on the traffic locality. CoDMW2 deploys a hybrid P2P and client/server architecture to support a large number of players while maintaining good gaming experiences. To facility our measurement, we designed and implemented a peer crawler over the PSN. Our instrumented crawler applies the ARP poisoning attack in our justified scenario to penetrate the PSN to harvest player s information at two critical gaming points. We analyzed the measurement results at the autonomous system (AS) level. Our results show that the CoD sessions initiate the game considering players locality at certain level. Nevertheless, optimized localityaware game sessions are yet to be found. We plan to study the optimal locality-ware session construction algorithms and their impact on the delay performance and system scalability for MMOFPS games. ACKNOWLEDGEMENT This work has been partially supported by the NSFC ( ), the Fundamental Research Funds for the Central Universities (HUST:20QN05) and the Technology Support Plan of the National Twelfth Five-Year- Plan of China (20BAK08B00). REFERENCES [] World of Warcraft, [2] Call of Duty official website, [3] Second life, [4] B. Knutsson, H. Lu, W. Xu, and B. Hopkins, Peer-to-peer support for massively multiplayer games, in IEEE INFOCOM, march [5] Xbox live network, [6] PlayStation network, hhttp://en.wikipedia.org/wiki/playstation- Network. [7] I. Barri, F. Gine, and C. Roig, A scalable hybrid P2P system for MMOFPS, in IEEE ICPDP, Feb [8] J. Jardine and D. Zappala, A hybrid architecture for massively multiplayer online games, in ACM NetGames, [9] M. Suznjevic, O. Dobrijevic, and M. Matijasevic, Hack, slash, and chat: A study of players behavior and communication in MMORPGs, in ACM NetGames, Nov [0] K. J. Shim, M. Ahmad, N. Pathak, and J. Srivastava, Inferring player rating from performance data in massively multiplayer online role-playing games (MMORPGs), in IEEE ICCSE, Aug [] C. Neumann, N. Prigent, M. Varvello, and K. Suh, Challenges in peer-to-peer gaming, in ACM SIGCOMM, January [2] Guinness-world-records, / Guinness -World-Records-Announces-Awards-at-E3/. [3] Call of Duty at Wikipedia, [4] K. J. Shim, K.-W. Hsu, S. Damania, C. DeLong, and J. Srivastava, An exploratory study of player and team performance in multiplayer first-person-shooter games, in IEEE PASSAT, 20. [5] F. Wu-chang, C. Francis, F. Wu-chi, and W. Jonathan, Provisioning on-line games: A traffic analysis of a busy counter-strike server, in ACM SIGCOMM, [6] X. Wang, H. Kim, A. V. Vasilakos, T. Kwon, Y. Choi, S. Choi, and H. Jang, Measurement and analysis of World of Warcraft in mobile WiMAX networks, in ACM Netgames, [7] M. Bredel and M. Fidler, A measurement study regarding quality of service and its impact on multiplayer online games, in ACM NetGames, 200. [8] F. Cifaldi, Analysts: FPS most attractive genre for publishers, in GamaSutra, Feb 2006, bin/news index.php?story=824. [9] C. DeLong, K. Erickson, E. Perrino, K. Shim, and N. Pathak, Project halofit data repository, 2009, [20] T. Beigbeder, R. Coughlan, C. Lusher, J. Plunkett, E. Agu, and M. Claypool, The effects of loss and latency on user performance in unreal tournament 2003, in ACM NetGames, [2] M. Dick, O. Wellnitz, and L. Wolf, Analysis of factors affecting players performance and perception in multiplayer games, in ACM NetGames, [22] J. Saldana, J. Fernandez-Navajas, J. Ruiz-Mas, J. I. Aznar, E. Viruete, and L. Casadesus, First person shooters: can a smarter network save bandwidth without annoying the players? IEEE Communications Magazine, 20. [23] Call of Duty, [24] R. Froom, B. Sivasubramanian, and E. Frahim, Implementing Cisco IP Switched Networks (SWITCH) Foundation Learning Guide. Cisco Press, 200. [25] Cain and Abel, [26] IP-to-AS mapping tool, mohammad/ip2as.php. [27] B. Liu, Y. Cui, Y. Lu, and Y. Xue, Locality-awareness in BitTorrent-like P2P applications, IEEE Trans. Multimedia, Apr [28] CIDR-report,