COMP 3801 Final Project. Deducing Tier Lists for Fighting Games Mathieu Comeau

Transcription

1 COMP 3801 Final Project Deducing Tier Lists for Fighting Games Mathieu Comeau

2 Problem Statement Fighting game players usually group characters into different tiers to assess how good each character is in general when playing at the highest level under official tournament rules. In theory, a tier list should be representative of the viability of each character in a tournament setting, assuming that each character is played as optimally as possible. However, most tier lists made by a game s player base are highly subjective, and either done with votes, or by experimentation and thousands of hours of collective gameplay and analysis of toplevel players. Is it possible, then, to provide a more objective measure of the top-level viability of each character in a fighting game? The aim of this project is to use algorithms and data analysis to create a more objective tier list for the popular fighting game Super Smash Bros. for Wii U. This game was chosen due to its popularity, as well as the amount of available statistics and data for the game and its characters. Background Work There are two areas that I considered when coming up with a character s tier list placing. The first is tournament performance. Tournament performance involves both the highest ranking of a character at a given tournament, as well as the character s overall win percentage against the other characters. This data is tracked using tournament-hosting websites such as Smash.gg (Smash.gg, 2017), as well as the wiki SmashWiki (SmashWiki, s.d.) and can be accessed by scraping the web page of the tournament and analyzing the raw HTML, and working with that. The second area is character data. This involves analyzing various attributes of each character, as well as for each of their attacks and movement options. Each character has a weight, a gravity coefficient, jump height, and number of jumps. Some characters can jump off walls or cling to them, and others have long-range grappling hooks so that they can grab the ledge from a further distance than would normally be possible, which is crucial for not getting KO d. Attacks and movement options have damage and knockback data, as well as various data regarding the start-up lag, ending lag, intangibility/invincibility frames, and shield advantage. These values are more-or-less hardcoded into the game, and do not change during regular gameplay. This data is not published by the developers of the game, but rather it is gathered by members of the fanbase who play and analyze everything, and upload their findings to the Internet. The data I used came from the website Kurogane Hammer (KuroganeHammer, s.d.), as well as the Google Sheet SSB4 Frame Data [1.16] (SSB4 Frame Data [1.16], s.d.)for frame data for moves when the opponent is shielding. To scrape the web pages, I used the JSoup library to visit and collect the required data from a set of 162 total webpages in under a minute. One challenge that I faced was dealing with the layout of webpages being inconsistent. For example, on Kurogane Hammer, every character has a section named Grabs that contains frame data about the character s grabbing motion. For Pac-Man, however, the section is titled Useless Tractor Beams, in reference to Pac-Man grab, which moves really slowly, has a lot of ending lag, and leaves Pac-Man open to punishes, and is thus not a move that is used often by Pac- Man players. There were other character pages, such as Zero Suit Samus and Ike, where a frame data table appears to be one table, but actually ends up being 2-3 tables within another table.

3 Experimental Design & Theoretical Analysis Tournament performance is split into two categories: ranking and win percentage. Ranking refers to the highest placing that a particular character received in a tournament. Since it s possible that multiple players use the same character, we must only consider the highest ranking that a character achieved in the tournament, since considering multiple rankings will skew the results. For each tournament, each character will be awarded an amount of points based on their highest ranking. In order to calculate the maximum point value, I counted all of the individual placings, and noted how many of the placings were eligible for bonus points (explained in greater detail later). The maximum point value is 40 x number of bonus placings + 20 x number of other placings. The 1 st place character is awarded the maximum point value. For each lower ranking, the amount of points will decrease by 20, with the following exceptions: 1st and 2 nd, 2 nd and 3rd, 4th and 5th, and 7th and 9th, where the points will decrease by 40 instead of 20. This is to represent bonuses given to characters that win the whole tournament, make it to Grand Finals, make it to Loser Finals, making it to top 4, and making it to top 8. In addition, each tournament will be given a weight based on how long ago the tournament occurred. This is because the metagame is constantly developing, and players are constantly learning and adjusting to new strategies and techniques. Once these points are assigned, the characters are ranked by total number of points gained throughout the tournaments measured. The above ranking system used is similar to the method used by the IIHF to rank its hockey teams based on performance at the World Hockey Championships and the Olympics. (IIHF World Ranking, s.d.) The IIHF method was chosen because it provides an example of both weighting results from older tournaments less than newer tournaments, as well as its handling of reward points for teams(characters) achieving certain rankings. Another statistic that is of importance to this project is win percentage, that is to say, the percentage of matches won against each character. It can be noted that the win percentages between characters can be represented by a Markov chain, where the win percentage represents the probability of changing state. Below is a sample of a Markov Chain between four characters, Pikachu, Zero-Suit Samus, Mario, and Cloud (any graph with more than 4 characters would get messy and hard to follow visually) Through data scraping, I found that, for example, Cloud has a 90% win percentage vs. Pikachu, while Pikachu only has a 10% win percentage vs. Cloud. This means that if we were to simulate a battle between the two, based on this data, Cloud has a 0.9 probability of defeating Pikachu, and Pikachu has a 0.1 probability of defeating Cloud. With this in mind, we can make use of the PageRank algorithm to rank the characters from best to worst, according to their matchup win percentage.

4 I used the GraphStream library to build the graph, as well as calculate the PageRank of each character. While tournament win rates and rankings give a measure of how well each character performs against other characters in practice, this still does not necessarily measure the potential of each character. Character choice in tournaments is dominated by popular characters. It is very possible that there are characters that are not commonly seen at tournaments, but might do well if players dedicated enough time to learning how to maximize the character s potential. For this reason, we examine certain in-game data values, such as damage and knockback data, as well as various data regarding the start-up lag, ending lag, intangibility/invincibility frames, and shield advantage. A character s frame data consists of which frame of animation that the hitbox becomes active( start-up ), how many frames that the hitbox is active for, as well as how many frames it takes from the time the hitbox is done until the character is able to perform another action( ending lag ). This is a very important metric for a fighting game, because a move with less start-up frames can be used to counter a move with more start-up frames. Also, a character is completely vulnerable during their

5 recovery frames. As a general rule, moves that are stronger tend to have longer start-up and recovery periods than weaker moves in order to balance the game. The following formula is how the game calculates the final knockback on a character after they ve been hit. p is the percentage of damage that the player being hit has already sustained. d is the amount of damage, b is the attack s base knockback value, that is to say, the general amount of knockback generated by a hit, regardless of how much damage the opponent has sustained. w is the opponent s physical weight. s is the knockback growth value, or how much the knockback scales depending on the damage already taken by the opponent. r is a set of ratios based on factors such as AI difficulty, damage ratio, damage taken by move user, etc. For simplicity, since r directly scales the value, I assumed that each possible ratio that could be in r is 1:1, and thus does not affect the rest of the score. (SmashWiki, s.d.) Knockback formula

6 I assigned a numerical score for each character s move set by taking the average damage, knockback growth, base knockback, start-up frames, ending lag, intangibility frames, frame advantage vs. Out of Shield options, and frame advantage vs. shield drop. Then, I took the weighted mean of the average damage, knockback growth, and base knockback. I used the values from the knockback formula above as weights, assuming p = 1(i.e., the opponent has taken 1% of damage before getting hit). The base damage was weighted with p/20(from the p x d / 20), the knockback growth value was weighted with (((p/10 + p x d / 20) x 200/100+w x 1.4) + 18)), and the base knockback was weighted with ((((p/10 + p x d / 20) x 200/100+w x 1.4) + 18)) x s). It is important to note that, for the w variable, I used the median weight of all of the characters in order to get an idea of how well the move set performs on average against any character. After finding the first weighted mean, I then found the weighted mean of the first score, and the frame-data average scores calculated earlier. The characters are then ranked by their move score. Next, we need to rank the characters according to their recovery ability, and jump ability. I examined two different types of jumps: full hops and aerial jumps. Jumping is important for recovering, as well as for going off of the stage to interrupt an opponent s recovery. To measure jumping, the following formula is used: fullhopheight + (aerialjumpheight (maxnumofjumps 1)) Note that the maxnumofjumps includes the initial full hop, which is why the aerialjumpheight is multiplied by the maxnumofjumps 1, since a character has at most maxnumofjumps 1 aerial jumps. The character is also given a bonus 0.5 points if they are capable of performing a wall jump. Wall jumps can be useful for characters to get extra height when their regular jumps won t cut it. The recovery score is the jump score above with another bonus of 0.5 points given to characters that have a grappling hook. Grappling hooks allow characters to reach the ledge even when they may be too far to jump back. The use of weighted means is worth noting here. Weighted means are known to be affected by outliers more than the median. However, for example, in theory, a character could have a minimal number of useful actions, and take advantage of these outlier actions to win a match. For this reason, it makes sense to use the weighted mean to measure the above scores. In order to obtain the final ranking, I took the median of the above rankings for each character, and then ranked the characters according to their median ranking. The media ranking was used here because, in this case, we do not want the outlier rankings to affect the final ranking of a character. The median gives a much better picture of the ranking overall. Results Analysis I compared my results to the currently accepted tier list developed by top-level players and tournament organizers. (Smash, 2016) The one above is the currently accepted tier list, and the one on the bottom is the result of my ranking system. One immediately obvious problem with comparing the two lists is that the current list is essentially completely subjective. There is no real metric to evaluate whether either list is correct. However, given that the current tier list was compiled by experts who have logged thousands of hours, this is the best thing to compare my list against.

7 My top-tier section(s and A) was actually fairly similar to the currently accepted tier list, with a difference of only 2 entries. Charizard and Luigi are considered bottom-tier and mid-tier, respectively.

8 Charizard has a lot of hard-hitting moves, and Luigi has good frame data. However, in the game, Luigi has a bit more difficulty stringing together hits, while Charizard is a big target and easy to hit. Hitbox and hurtbox data is available on the Internet. However, this data is usually represented visually, and it would be impractical to measure the size of each hitbox and hurtbox for each character based on animated.gif files. My high-tier section(b and C) differs remarkably from the current tier list, with only 5 similarities. It is worth noting that two of the characters in my high-tier section, Mario and Fox, are listed as top-tier characters in the current tier list. My mid-tier section(d and E) only had 3 similarities when compared to the current tier list. Most of the other characters in this section were off by only one section, so this section can be considered close to the current list, but not quite. Once again, data for hitboxes and hurtboxes would have helped here. My low-tier section(f and G) has 8 similarities to the current tier list. This is where we begin to see less popular characters, as well as some of the bigger, slower characters with inferior frame data. My bottom-tier section(h) is remarkably different from the current tier list, with only 2 similarities. This seems to reflect the lack of popularity of the characters in this tier more than anything else(see graphic below), although these characters are not known to have great frame data either.

9 In general, one of the challenges with this approach is the lack of data available. Not every match from every bracket is recorded, so there happens to be a lot of match data missing that could have been useful to better reflect a character s win percentage. On top of that, tier lists tend to

10 reinforce themselves. High-tier and top-tier characters become more popular with players wishing to replicate the success that top-level player enjoy. As a result, there is far less matchup data available for characters like Zelda and Ganondorf, compared with Zero Suit Samus or Diddy Kong(reference the graphic above). This, combined with missing attack data, such as hitbox and hurtbox data, and KO percentage, would have been very helpful and might have gotten my list closer to the current accepted tier list. Conclusion I believe that the approaches I took to rank the characters and sort them depending on how viable they are or could potentially be were reasonable approaches to solving the problem of creating an objective tier list for a fighting game. Even though there is no theoretical framework established in this area, I believe that, since the PageRank algorithm and IIHF ranking system are proven to work in very similar contexts, they should work fine here with a few tweaks. Future work would involve finding ways to gather missing data, tweaking values and weights with my own solution, as well as sharing my solution with the community of top players to make more adjustments. Alternative ranking methods can also be explored. For example, genetic algorithms to rank characters based on certain desirable traits, for example, low frame lag, high knockback and damage, good jumping and recovery. My approach could also be used in a sports scenario to measure how suitable a prospect might be for a top-level professional league. Sports use statistics and past performance to measure how good players are, but they rarely seem to factor in a player s physical capabilities in an objective manner(for example, a player doesn t skate at 20 mph, they skate fast ). Furthermore, a player s statistics can be strongly affected by the team they play for, as well as the calibre of the teams they play against most often. Teams are often grouped into divisions, and they play against the teams in their division more often than the rest of the teams. This can affect the statistics of players if the rest of the teams in their division perform significantly better/worse. Using ice hockey as an example, statistics from all prospective NHL players currently playing major junior hockey(ohl, QMJHL, WHL, etc.) could be combined with measurements of a player s actual physical abilities (skating speed, agility, shot power, shot accuracy, reach + stick length, body checking ability) to get an idea of the potential that a junior player has to excel at the NHL level.

11 References IIHF World Ranking. (n.d.). Retrieved from IIHF: Knockback - SmashWiki. (n.d.). Retrieved from SmashWiki: KuroganeHammer. (n.d.). Smash 4 - Kurogane Hammer. Retrieved from Kurogane Hammer: Smash. (2016, August 22). Retrieved from Amino Apps: Smash.gg. (2017, 11 01). Retrieved from Smash.gg: SmashWiki. (n.d.). Retrieved from SmashWiki: SSB4 Frame Data [1.16]. (n.d.). Retrieved from SSB4 Frame Data [1.16]: VDOhZI/edit?pref=2&pli=1#gid=