< AIIDE 2011, Oct. 14th, 2011 > Detecting Real Money Traders in MMORPG by Using Trading Network

Size: px

Start display at page:

Download "< AIIDE 2011, Oct. 14th, 2011 > Detecting Real Money Traders in MMORPG by Using Trading Network"

Kelley Pearson
6 years ago
Views:

1 < AIIDE 2011, Oct. 14th, 2011 > Detecting Real Money Traders in MMORPG by Using Trading Network Atsushi FUJITA Hiroshi ITSUKI Hitoshi MATSUBARA Future University Hakodate, JAPAN

2 Focusing on Massively Multiplayer Online RPG Thousands of players co-exist in one virtual world cf. millions of registered players

3 To facilitate further growth Effect of features Extended game fields, one-shot events Influential players Mentoring, intermediation, trades To maintain the order of the virtual world Harassments between players Player killing, occupation of specific locations Causes that lead unfairness and crisis of virtual economy Real Money Trading, use of bots, cheat

4 Real money Virtual properties Currency, items, status, functions, avatars, etc. Observed in other online services, e.g., SNS, auction Two opposing attitudes (sometimes ambivalent) Positive: Means of augmenting the real world e.g., Second Life Negative: Source of serious problems e.g., Most MMORPGs in Japan

5 Automatic detection of RMTers Actual log data is available Now with TECMO KOEI GAMES CO., LTD. Prefer title independent features Operators want no arms race Desire un-cheatable features Operators verification is indispensable To avoid ruling out honest players The amount of human resource depends on situation Title (scale, seriousness) and budget for operation Prefer unsupervised or semi-supervised methods

6 Outline 1. Introduction 2. Approach 3. Procedure 4. Experiment 5. Conclusion

7 Classify each character into RMTer or non-rmter Supervised machine learning [Ahmad+, 09] Naïve Bayes, k-nn, AdaBoost, etc. Various features (incl. those specific to the title) Not flexible: Too much/less positive class Positive Negative

8 Sort characters according to their suspiciousness Using cumulative features [Itsuki+, 10] Handled currency dealing with enormous virtual currency Volume of actions working hard to earn virtual currency Activity hours Not thoroughly studied Ranking 1 st 2 nd 3 rd 4 th 5 th 6 th 7 th

9 Ranking 1 st 2 nd 3 rd 4 th 5 th 6 th 7 th Connection between pairs of characters Extremely low exchange rate, e.g., full of wallet = Division of RMT labor & frequent trade Infrastructure for trading log data are available Volume of individual trade

10 RMTers and their trading partners in one timeframe Division of labor of RMTers Typical roles Seller Earner Collector Tight connection

11 Possibility of wholesale arrest 11

12 Outline 1. Introduction 2. Approach 3. Procedure 4. Experiment 5. Conclusion

13 Raw log data 1 st 2 nd 3 rd 4 th 5 th 6 th 7 th 8 th

14 Graph partitioning / graph clustering Node: Character Edge: Trade between two characters

16 Modularity [Newman+, 04] Q E: Set of all edges in the network E i : Set of edges within i th community A i : Set of edges connecting to a node in i th community

17 Finding a partitioning that maximizes Q: NP-hard A bottom-up greedy algorithm [Clauset+, 04] 1. Regard each node as a community and calculate ΔQ for each connected community pair 2. Merge two communities whose ΔQ is largest (and >0) 3. Update ΔQ for the merged communities 4. Repeat steps 2 & 3 while Q gains

18 Frequent and/or large-scale trades RMT 1. Ranking communities In-community trades 2. Ranking characters in each community Trades of individual character 3 rd 1 st 1 st 2 nd 4 th 2 nd 1 st 4 th 2 nd 3 rd 1 st 2 nd 3 rd 4 th 5 th 6 th 7 th 8 th

19 Outline 1. Introduction 2. Approach 3. Procedure 4. Experiment 5. Conclusion

manually 29 130 (<1%) within 15,249 18,745 characters Actual action log data in the

20 Uncharted Waters Online Exploration, naval battle and trading in mid-ages RMT is prevalent 4 timeframes (15 23 days, no overlap) RMTers are identified (& banned) manually (<1%) within 15,249 18,745 characters Actual action log data in the same period million records (C) TECMO KOEI GAMES CO., LTD. All rights reserved.

# of characters 20000 16000 12000 8000 47% 23% Non traders Traders w/o currency Currency

4000 0 30% (29) 33% (52) 34% (106) 30% (129) Period A Period B Period C Period D Half of

21 # of characters % 23% Non traders Traders w/o currency Currency traders 43% 24% 45% 21% 45% 24% (1) Obs % (29) 33% (52) 34% (106) 30% (129) Period A Period B Period C Period D Half of all characters traded something 1/3 of all characters traded virtual currency Most of RMTers traded virtual currency Only 1 exception in period D

22 # of extracted communities tb tt cb ct cv tb tt cb ct cv tb tt cb ct cv tb tt cb (*) ct (*) cv (*) Period A Period B Period C Period D Target characters and weight of trade All traders Currency traders Obs. tb: binary cb: binary tt: # of times ct: # of times Weights of trades / focusing on currency fine-grained RMTers only a few communities (1-8) cv: volume

Recall (R): how exhaustively RMTers are identified Precision

23 Available human resource is unknown It varies depending on the situation Two measures Balance between Recall and Precision Recall (R): how exhaustively RMTers are identified Precision (P): how correctly system identifies RMTers Avg. Precision at various recall

24 Representation of trading network All traders tb: binary tt: # of times Currency traders cb: binary ct: # of times cv: volume Measure for in-community trades tt: # of trade transactions ct: # of currency transactions cv: Total volume of traded currency Measure for trades of individual character tt: # of trade transactions ct: # of currency transactions cv: Total volume of traded currency

25 45 combinations 10 Representation of trading network (5) Different network achieved the best result in different period Measure for in-community trades (3 1) Volume of traded currency (cv) > # of transactions (tt, ct) Measure for trades of individual character (3 2) Traded currency (ct, cv) > All trade (tt) Implications Large amount of currency is exchanged for RMT RMTers dealt with more than 1/3 of total currency trades Virtual currency is popular in RMT Buyers want virtual currency

26 Sort characters based on handled currency (cv) Two supervised methods (w/o constants) Naïve Bayes with multinomial distribution [McCallum+, 98] Support Vector Machines [Vapnik, 99] with linear kernel (SVM light is used) Feature: all of 338 types of actions trade, attack to other player, find an item, invest for a ship

27 Several versions beat all the baselines But nothing significantly wins in all periods

28 Precision Precision Win Lose Best Proposed (.764) Best BL (.356) Win Lose Best Proposed (.900) Best BL (.567) Recall 1 Recall Precision Precision Win Lose Best Proposed (.524) Best BL (.484) Recall Win Lose Best Proposed (.648) Best BL (.599) Recall

1 1 0.8 0.8 Precision 0.6 0.4 Precision 0.6 0.4 0.2 0 Win Lose Best Proposed (.764) Best BL (.356) 0 0.2 0.4 0.6 0.8 1 Recall Significant improvement Both on R-P curves and Avg. Prec. Most RMTers a single, small, and top-rank community 0.

29 Precision Precision Win Lose Best Proposed (.764) Best BL (.356) Recall Significant improvement Both on R-P curves and Avg. Prec. Most RMTers a single, small, and top-rank community 0.2 Period A: 29 RMTers Period B: 52 RMTers Some are still difficult to detect 0 Win Lose Best Proposed (.900) Best BL (.567) Recall

30 Weak for plural RMTer communities Period C: 106 RMTers Period D: 130 RMTers Need a more intelligent ranking e.g., Combination of ranks (community, character) e.g., Re-ranking based on operators judge 1 st (207) 2 nd (107) 3rd (175) st (180) 2 nd (67) 3rd (85) Precision Precision Win Lose Best Proposed (.524) Best BL (.484) Recall Win Lose Best Proposed (.648) Best BL (.599) Recall

Evaluation using actual log data Better performance than separately assessing

31 Detection of RMTers in MMORPG As a ranking problem Wholesale arrest thru capturing communities Low exchange rate division of labor & frequent trade Evaluation using actual log data Better performance than separately assessing each char. w/ a room of further improvement Ranking 1 st 2 nd 3 rd 4 th 5 th 6 th 7 th

32 Technical aspect Further investigation into trading network Mixture models [Newman+, 07] Augmentation with other components [Ahmad+, 11] Apply state-of-the-art machine learning techniques Evaluation Is arms race really overcome? e.g., Robustness against disposal use of characters Application to other MMORPGs

Chapter 5: Game Analytics

Lecture Notes for Managing and Mining Multiplayer Online Games Summer Semester 2017 Chapter 5: Game Analytics Lecture Notes 2012 Matthias Schubert http://www.dbs.ifi.lmu.de/cms/vo_managing_massive_multiplayer_online_games