A Empirical Study for Quatitative Evaluatio of Game Satisfactio HyugBum Ham*, YagSu Lee** Divisio of Mathematics ad Iformatio Statistics, at Seokyeog Uiversity * Dept. of Computer Egieerig, at Seokyeog Uiversity ** 16-1 Jugeug-Dog Sugbuk-Ku Seoul, 136-704 Korea {hbham, yslee}@skuiv.ac.kr Abstract Improvemet of electroic games, ad icreased trade ad competitiveess of the electroic gamig idustry require evaluatio of the satisfactio level of the users. Statistical aalysis is the key to quatitatively assessig both existig games ad to help develop ew oes. This research paper proposes a system i which game satisfactio ca be quatitatively evaluated accordig to a stadardized scale ad that presets a ratioal aalysis of games. This method will eable the idustry to aticipate, for example, the varyig degrees of game satisfactio, ad their distributio, ad ay techical improvemets that eed to be implemeted. 1. Itroductio With ecoomic ad social advacemets rapidly comes a lifestyle preferece of quality over quatity. Sice the gamig idustry is a core part of the etertaimet culture, it cotiues to expad as a multifaceted etertaimet busiess comprised of cultural elemets/ cotext, high techology, IT, ad art. The ecoomic size of the global gamig market, which was $56,200M i 2004, is aticipated to reach $75,700M i 2007. The Korea gamig market has achieved a ecoomic value of $3,597M, which, based o the 2004 statistics, has grow 9.6% from last year. The 124% icrease i the exportatio of $387.69M comprises 41.3% of the total gamig/etertaimet busiess, which raks at the top of all other etertaimet categories. Moreover, such a overall boost i its exportatio ca be explaied by the 91% icrease of its biggest trade product, gamig Therefore, future etertaimet busiess will be cetered o the exportatio of games, which is expected to icrease each year by 20% [7, 8]. Although game compaies come up with ew games every year, which are loaded with all kids of advaced game-related techology, ot all of them will be able to survive i the busiess. I order to produce a game that ca icrease its value added compoet ad ecoomic competitiveess, scietific research ad effort are ecessary to determie how much satisfactio, i terms of a scaled scorig system, a game ca offer to users. It is obvious that the value added compoet, popularity, ad sales of a game, which are the ultimate cocers of all game developers, will icrease with improved user satisfactio. There has bee much discussio o how to develop games with higher degrees of satisfactio. Most of them focus o aalyzig the gamig market ad user s treds, especially i these areas: uderstadig the scale of the market ad the people that work i the idustry, user demographics, actual aalysis of the games beig played, ad preferred game platforms ad geres [7]. I additio, a method of evaluatig user satisfactio has bee researched i [2, 3] to establish a more efficiet game developmet strategy. This paper would like to itroduce ad ivestigate a techique that reasoably evaluates the satisfactio degree of games. By adoptig such techiques as a extesio to the research of [2, 3], it is possible to aticipate a upcomig game s satisfactio degree ad its distributio, to come up with the eeded techological improvemet to icrease its appeal, ad to predict its potetial salability. 2. Research Model 2.1. Stadardizig the Satisfactio Degree
The followig satisfactio meter ca be obtaied by cosiderig a umber of factors i determiig a specific game s satisfactio degree. First of all, a satisfactio score out of 5 is collected from users for each of the categories. Secodly, if the obtaied data is x 1, x 2, L, x, the satisfactio degree is computed by arithmetic average, x= xi /. Although the factor s weight, 1/, has bee cosistetly applied to all the satisfactio degrees obtaied from the arithmetic meas, the stadard weight is differet i ormal situatios. A more effective way of producig satisfactio degrees is to apply the correspodig weights games of differet geres as well as similar geres. Therefore, usig weights to calculate the weighted mea is a more effective way of determiig satisfactio degrees rather tha by directly computig the arithmetic meas. I other words, if factors weights are ω, ω, 1 2 L, ω, the satisfactio degree is evaluated as ω ix. i 2.2. Estimatig the Weights We The foremost task i stadardizig a game s satisfactio degree is calculatig the weight of the game s satisfactio degree factors. This research proposes a calculatio techique for the weights that uses the AHP s eigevector method. The AHP, developed by Saaty [12], is a solutio to the decisiomakig process used i the judgmet ad aalysis of evaluators who possess eough experiece i the idustry to make decisios based o the results. The eigevector method uses a eigevector that correspods to the pairwise compariso matrix s maximum eigevalue, which is used as the weight value i the decisio-makig process. I other words, if the umber of a game s satisfactio degree factors is, a pairwise compariso value a is obtaied by askig ij C 2 = ( 1) / 2 umber of pairwise compariso questios. a, a value that idicates the importace of i ij over j, or j over i, has the values show i Table 1 whe the scale is up to 9. Table 1. Scale for Pairwise Compariso Defiitio umerical values equal importace 1 moderate importace 3 strog importace 5 very strog importace 7 extreme importace 9 itermediate values 2, 4, 6, 8 By pairwise comparig the umber of elemets, the dimesio of the pairwise compariso obtaied is ad lets call this matrix A= { a ij }. A are a reciprocal matrix that has the followig features. a ji = 1/ aij, a = 1, ii i, j= 1,2, L, i= 1,2, L, ω = ω1, ω2, L, ω to the required weight vectors of a game s satisfactio degree, ω is computed by a characteristic equatio, A ω = λmaxω. Here, λ is the maximum eigevalue max of A ad the eigevector that correspods to it becomesω. I the actual decisio-makig questio, each factor s weight cotrols the eigevector, which correspods to the characteristic equatio s maximum eigevalue, causig the sum of ω s elemets to be 1. Moreover, based o the fact that the pairwise compariso values of A will have more cosistecy as λ approaches max, the cosistecy ca be obtaied from the cosistecy idex ad the cosistecy ratio. By assigig the value ( ) T CI = ( λ max ) /( 1), CR= CI / RI Here, RI stads for radom idex ad has the followig values. I the AHP, the pairwise compariso matrix is desiged to be cosistet whe the CR is below 0.1. Table 2. Radom Idex 1 2 3 4 5 6 7 8 RI 0 0 0.58 0.90 1.12 1.24 1.32 1.41 9 10 11 12 13 14 15 RI 1.45 1.49 1.51 1.48 1.56 1.57 1.59 The previous process shows how the fial weight ca be obtaied after costructig a pairwise compariso matrix from oe perso s data to extract the game s satisfactio factors, ad the extractig other people s weights to combie all of them. A uified sigle pairwise compariso matrix eeds to be composed to do this. The uified pairwise compariso matrix computes the geometric average of each of its elemets, except those that do ot fulfill its cosistecy ratio. The rest of the weights are calculated by the same method used i
costructig a sigle pairwise compariso matrix. The geometric average is used because it is the oly way to maitai the iverse property of the matrix[6]. 3. Empirical Aalysis 3.1. Collectio of Data Ever sice 2002, the Korea gamig idustry has stood i first place amog five gamig platforms with a marked $849M worth of productio. It also comprises 61.9% of the etire platform productio ad icreased 35% from last year. Agai, the most frequetly used game platform, was games at 68.3%, followed by PC games at 18.7%, ad mobile games at 6.6% [7]. This project presets the game s satisfactio degrees based o the satisfactio survey results for RPG, FPS, sports, ad board games. The 7 factors that were icorporated are sceario, graphics, soud, game speed, game cotrol, character, ad commuity. These elemets were selected upo referecig game-related papers [2,3,7], gamig policy sites [4,5], customer loyalty, ad flow theory [1,9,10,11]. People with gamig experiece have also bee iterviewed for referece. The survey questios are classified ito 2 major sectios. The first sectio is a set of pairwise compariso questios that icorporate all possible pairs of the 7 factors to obtai the weights ad ask which factor of each pair is more importat. For the secod sectio, the users have to select from various gamig sites what games are widely played ad the evaluate, for each game, the satisfactio degree of each of the 7 correspodig factors. Data was gathered from survey of 400 people, with 100 for each gere, usig these questios, For the pairwise questios, 5 poits was set as the iterval scale ad 7 poits was set for the questios that measured the satisfactio level of each factor. The games selected for the survey were as follows: RPG games (Project A3, RF Olie, Gersag, GooZu, Ragarok, Lieage, Lieage2, Mabiogi, Mu, Yulgag, WOW, Rose Olie), FPS games (Couter Strike, Special Force, Dispatch of Army, Heat Project, Guz Olie, Lock o Target, Vital Sig, Rommel, AA Olie, Karma Olie), sports games (Kart Rider, Free Style, Pag Ya, You re GolfKig, Shot Olie, Accro Race, Ha Qua, ASDF Hover Board), ad board games (Go Stop, Poker, Baduk, Omok, Chess, Bigo, Mahjog, Reble Pebble, Tetris, Fid Differet Poit). 3.2. Yieldig Weights Table 3 shows the estimated weight for each satisfactio factor of RPG, FPS, sports, ad board games. The values i the parethesis represet the rakig of each factor as the differece of importace betwee factors of differet geres. For example, i RPG games the most importat factors were graphics ad game cotrol, while i FPS ad sports games the most importat factors were character ad graphics, ad i board games game cotrol ad game speed were the most importat. A sigificat gap i the weights exists betwee the most importat factor ad the rest of the factors, especially i FPS ad sports games. Table 3. Weights of Satisfactio Degree Factors RPG FPS Sports Board Sceario 0137(5) 0.091(5) 0.077(7) 0.141(4) Graphic 0171(1) 0.198(2) 0.187(2) 0.148(3) Soud 0135(6) 0.075(6) 0.092(5) 0.128(6) Game Speed 0.140(4) 0114(4) 0.143(4) 0.153(2) Game Cotrol 0.143(2) 0186(3) 0167(3) 0.172(1) Character 0.142(3) 0274(1) 0246(1) 0.137(5) Commuity 0.133(7) 0062(7) 0088(6) 0.122(7) Total 1 1 1 1 3.3. Aalysis of Satisfactio Degree Figures 1 to 4 show the satisfactio degree for each of the 40 games (maximum score = 7). The results were obtaied by multiplyig the arithmetic meas of each of the game s 7 factors by the weights listed i Table 3. It is clear that, although the satisfactio degrees of RPG ad board games are similar, there is a sigificat gap betwee FPS ad sports games as show i Figures 1 to 4. This observatio becomes more vivid by lookig at the magitude of Table 4 s stadard deviatio.
Figure 1. Satisfactio Degrees of RPG Games Figure 2. Satisfactio Degrees of FPS Games Figure 4. Satisfactio Degrees of Board Games Table 4 is a summary of the satisfactio degree of the games i each gere with the mea, stadard deviatio(sd), ad both the miimum ad maximum values of overall satisfactio degree. The satisfactio degree of RPG, at 4.408, was the highest of the geres ad the average satisfactio degree of all 40 games tured out to be 4.19. Although 50% of the etire set of games achieved a media score of 4~4.5, Kart Rider (5.606) ad Pag Ya (5.094), both sports games, were the oly two games that scored above 5 poits. Figure 5 represets, as a histogram, the distributio of the satisfactio degree of all games. Table 5-8 shows the satisfactio degree rakig by sortig ad summig the 7 factors of RPG, FPS, sports, ad board games. This allows oe to idetify how much each factor scored i terms of satisfactio degree ad also shows the differece of each factor s rakig betwee differet games. For example, from Table 5, we ca see that all 7 factors of WOW rak first i RPG games. I Table 6, the overall umber 1 rakig Kart Rider trails Pag Ya s character satisfactio degree oly slightly, while rakig umber 1 i the rest of the categories. O the other had, Pag Ya trails Kart Rider s rakigs oly slightly with the exceptio of game speed, i which it was far behid, explaiig its overall rakig of secod place. Figure 3. Satisfactio Degrees of Sports Games Table 4. Satisfactio Degree Statistics RPG FPS Sport Board Total mea 4.408 4.187 4.053 4.039 4.186 sd 0.187 0.508 0.990 0.095 0.521 mi 4.139 3.386 3.067 3.921 3.067 max 4.825 4.951 5.606 4.197 5.606 Figure 5. Histogram for Satisfactio Degrees of All Games
Table 5. Factor Satisfactio Degree Distributio of RPG Games Sceario Graphic Speed Cotrol acter Game Game Char- Commuity Soud Total WOW 0.652 0.836 0.673 0.659 0.706 0.694 0.605 4.825 Lie age2 0.627 0.814 0.615 0.608 0.643 0.682 0.598 4.587 Mu 0.585 0.798 0.609 0.614 0.640 0.668 0.591 4.505 Mabi Nogi Lie age Yul gag Rose Rag arok Project A3 Goo Zu RF Ger sag Kart Rider Pag Ya Free Style You're Golf Kig Shot Ha Qua ASDF Hover Board Accro Race 0.575 0.785 0.600 0.622 0.647 0.655 0.591 4.475 0.611 0.792 0.571 0.593 0.643 0.667 0.593 4.470 0.577 0.771 0.597 0.607 0.667 0.674 0.557 4.450 0.571 0.737 0.593 0.601 0.653 0.631 0.562 4.348 0.570 0.754 0.573 0.584 0.633 0.630 0.579 4.323 0.544 0.747 0.582 0.607 0.621 0.641 0.574 4.316 0.535 0.711 0.574 0.580 0.620 0.621 0.603 4.244 0.529 0.730 0.582 0.579 0.607 0.623 0.559 4.209 0.519 0.713 0.565 0.579 0.613 0.613 0.537 4.139 Table 6. Factor Satisfactio Degree Distributio of Sports Games Sceario Character Game Soud Speed Game Cotrol Total 0.369 1.118 0.431 0.825 1.019 1.466 0.378 5.606 0.365 1.098 0.401 0.435 0.956 1.476 0.363 5094 0.359 0.925 0.399 0.440 0.914 1.298 0.362 4.696 0.308 0.830 0.362 0.374 0.851 1.092 0.341 4.157 0.259 0.621 0.302 0.357 0.768 0.925 0.327 3.559 0.241 0.584 0.307 0.288 0.612 0.800 0.308 3.139 0.253 0.568 0.298 0.305 0.668 0.738 0.278 3.109 0.234 0.530 0.268 0.349 0.654 0.728 0.304 3.067 Table 7. Factor Satisfactio Degree Distributio of FPS Games Couter Strike AA Special Force Heat Project Karma Guz Sceari o Character Commuity Graphic Soud Game Speed Com m- Total uity 0.428 1.005 0.361 0.532 0.875 1.431 0.319 4.951 0.459 1.062 0.375 0.430 0.786 1.420 0.203 4.735 0.408 0.867 0.307 0.512 0.947 1.181 0.295 4.516 0.397 0.958 0.357 0.410 0.781 1.381 0.213 4.498 0.397 0.882 0.320 0.466 0.744 1.146 0.231 4.187 0.357 0.868 0.312 0.482 0.730 1.201 0.215 4.165 Rommel 0.387 0.871 0.259 0.513 0.884 1.014 0.195 4.122 Lock o Target 0.326 0.759 0.294 0.371 0.713 1.073 0.217 3.753 Dispatch 0.393 0.642 0.294 0.442 0.744 0.833 0.208 3.556 of Army Vital Sig 0.319 0.660 0.250 0.437 0.651 0.868 0.202 3.386 0.519 0.611 0.524 0.696 0.800 0.555 0.492 4.197 Go Stop 4 4 1 1 1 2 1 1 Tetris 0.524 0.629 0.524 0.671 0.766 0.553 0.471 4.138 1 1 1 3 3 3 2 2 Fid Differe 0.514 0.614 0.521 0.678 0.780 0.560 0.468 4.135 t Poit 6 2 3 2 2 1 3 3 Poker Reble Pebble 0.523 0.614 0.512 0.671 0.761 0.550 0.465 4.096 2 2 4 3 4 4 4 4 0.517 0.594 0.503 0.655 0.754 0.530 0.458 4.011 5 6 6 8 5 6 6 5 0.502 0.597 0.508 0.661 0.744 0.540 0.452 4.004 Baduk 10 5 5 6 7 5 8 6 Game Character Cotrol Table 8. Factor Satisfactio Degree Distributio of Board Games (7 poits max) Sceariphic Speed Cotrol acter uity Gra- Game Game Char- Comm- Soud Total Full 0.987 1.036 0.896 1.071 1.204 0.952 0.854 7 Marks Mahjog 0.510 0.583 0.499 0.656 0.751 0.530 0.457 3.986
8 8 7 7 6 6 7 7 0.514 0.579 0.492 0.662 0.733 0.529 0.448 3.957 Omok 6 9 9 5 10 8 9 8 Chess Bigo 0.521 0.586 0.498 0.638 0.737 0.525 0.442 3.947 3 7 8 10 8 9 10 9 0.506 0.564 0.490 0.646 0.737 0.518 0.460 3.921 4. Coclusio 9 10 10 9 8 10 5 10 This research substatially aalyzed 4 game geres for the purpose of proposig a way to evaluate game satisfactio degrees usig a fixed method based o survey results. The weights of each elemet for the satisfactio degree tured out to be differet from what was hypothesized i the itroductio because RPG, FPS, sports, ad board games raked highest i graphics, character, ad cotrol cofiguratio, respectively. The average satisfactio degrees of the geres were, i decreasig order, RPS (4.408), FPS (4.187), sports (4.053), ad board games (4.039). The overall average satisfactio degree of all 40 games was 4.19 ad oly two games were able to score above 5. It ca be cocluded that the respodets satisfactio degree is ot too high, as evideced by the 5% ratio of games that scored above 5. Furthermore, the distributio of elemets allows oe to idetify the scores o each elemet ad to compare the values with those of other games. The iformatio gathered by aalyzig the game satisfactio degree i terms of frequecy distributio, regressio aalysis, Structural Equatio Model, ad other statistical aalyses ca be used to aticipate the satisfactio of users, to idetify techical aspects that eed improvemet, ad to predict the sales volume. These, i tur, ca be used as resources to stregthe the competitiveess of the gamig idustry ad to icrease its value added compoet. However, the game satisfactio degrees aalyzed i this research paper are subject to deviatio from the fidigs of other gamig sites ad this largely depeds o the differeces i betwee sample space ad size. Refereces [1] B. Mittal, "Why Do Customers Switch versus Loyalty", The Joural of Marketig, 12(3), 1998, pp. 177-194. [2] Hyug-Bum Ham, Yag-Su Lee, "A Study o Assessmet Method of Game Satisfactio Degree", Joural of Korea Iformatio Processig Society: Research i Game, 1(1), 2004, pp.33-39. [3] Hyug-Bum Ham, Yag-Su Lee ad Chag Ho A, "A Assessmet System Model for Game Satisfactio Degree to Establish Game Developmet Strategy", Joural of Korea Multimedia Society, 7(11), 2004, pp. 1630-1638. [4] http:// www.gitiss.org [5] http://www.kocca.or.kr [6] J. Aczel ad T. L. Satty, Procedures for sythesizig ratio judgmets, Joural of Mathematical Psychology, 27(1), 1983, pp. 99-102. [7] Korea Game Developmet & Promotio Istitute, 2005 Game White Paper Korea, 2005. [8] Korea Miistry of Culture & Tourism, 2005 Culture Idustry White Paper Korea, 2006. [9] K. Ruyter ad M. Wetzels, "O the relatioship betwee perceived service quality, service loyalty ad switchig costs", Iteral Joural of Service Idustry Maagemet, 9(5), 1998, pp. 436-453. [10] L.K. Trevio ad J. Webster, "Flow i Computer- Mediated Commuicatio: Electroic mail ad voice mail evaluatio ad impacts", Commuicatio research, 19(5), 1992, pp.539-573. [11] M. Csikszetmihalyi, Flow: The Psychology of optimal experiece, Harper ad Row, 1990. [12] T. L. Saaty, The Aalytic Hierarchy Process, McGraw-Hill, New York, 1980.