Technical Appendix Catch the Pink Flamingo Analysis. Acquiring, Exploring and Preparing the Data

Transcription

1 Technical Appendix Catch the Pink Flamingo Analysis Produced by: Greg B Watson Acquiring, Exploring and Preparing the Data The purpose of this appendix is to ensure that the analysis of the game Catch the Pink Flamingo produced for Eglence can be repeated, either by other teams to verify our findings, or by the same team in the future to continue the analysis. In the following sections, we used the following analysis tools: Splunk data exploration and data set overview Knime Categorization with a decision tree pyspark Cluster Analysis Neo4j Graph Analysis Data Exploration (Splunk) Data Set Overview The table below lists each of the Pink Flamingo game log files available for analysis with a short description of what is found in each one. File Name Description Fields ad clicks.csv A line is added to this file when a player clicks on an advertisement in the Flamingo app timestamp when the click occurred txid a unique id (within ad clicks.log) for the click usersessionid the id of the user session for the user who made the click

2 buy clicks.csv A line is added to this file when a player makes an in app purchase in the Flamingo app teamid the current team id of the user who made the click userid the user id of the user who made the click adid the ID of the ad clicked adcategory the category/type of the ad clicked timestamp when the purchase was made txid a unique id (within buy clicks.log) for the purchase usersessionid the id of the user session for the user who made the purchases team the current team of the user who made the purchase userid the user id of the user who made the purchase buyid the id of the item purchased price the price of the item purchased game clicks.csv A line is added to this file each time a user performs a click in the game time when the click occurred clickid a unique id for the click userid the id of the user performing the click usersessionid the id of the session of the user when the click is performed

3 ishit denotes if the hit is on a flamingo (value is 1) or missed the flamingo (value is 0) teamid the id of the team of user teamlevel the current level of the team of the user level events.csv A line is added to this file each time a team starts or finishes a level in the game time when the event occurred eventid a unique id for the event teamid the id of the team level the level started or completed eventtype the type of event, either start or end team assignme nts.csv team.csv A line is added to this file each time a user joins a team. One user can be a member of at most one team at one time. This file contains one line for each team terminated in the game time when the user joined the team team the id of the team userid the id of the user assignmentid a unique id for this assignment teamid the id of the team name the name of the team teamcreationtime the timestamp when the team was created teamendtime the timestamp when the last team member left the team currentlevel the last level achieved by the team

4 user session.cs v users.csv Each line in this file describes a user session, which denotes when one user started and stopped playing the game. Additionally, when a team goes to the next level in the game, the session is ended for each user in the team and a new session is started. This table contains one line for each user playing the game usersessionid a unique id for the session assignmentid the team assignment id for the user to the team starttimestamp a timestamp denoting when the session started endtimestamp a timestamp denoting when the session ended team level the level of the team during the session platformtype the type of platform of the user during the session timestamp when the user first played the game id the userid assigned to the user nick the nickname chosen by the user twitter the twitter handle of the user dob the date of birth of the user country the two letter country code where the user lives Aggregation (Splunk) Amount spent buying items 21407

5 # Unique items available to be purchased 6 A histogram showing how many times each item is purchased: A histogram showing how much money was made from each item:

6 Filtering A histogram showing how many times each category of advertisement was clicked on:

7 The following table shows the user id, platform, and hit ratio percentage for the top three buying users: Rank User Id Platform Hit Ratio (%) iphone 12% 2 12 iphone 92% IPhone 15%

8 Data Classification Analysis (Decision Tree with Knime) Data Preparation Analysis of combined_data.csv Sample Selection Item Amount # of Samples 4619 # of Samples with Purchases 1411 Attribute Creation A new categorial attribute was created to enable analysis of players as broken into 2 categories (HighRollers and PennyPinchers). A screenshot of the attribute follows:

9 The attribute avg_price indicates if the average prices of ingame purchases exceeds $5 ($avg_price$ > 5 => HighRoller ) or is $5 or less ($avg_price$ <= 5 => PennyPincher ) The creation of this new categorical attribute was necessary because our assigned task is to identify the attributes that can predict if a player will purchase items worth more than $5. The decision tree algorithm will fail if we target a continuous variable Our algorithm will run faster if we store the category as part of the dataset rather than evaluate the category as part of building the decision tree For the decision we week to make (choosing the approach for advertising in game purchases to gamers), predicting a category is sufficient. Attribute Selection The following attributes were filtered from the dataset for the following reasons: Attribute avg_price Rationale for Filtering We excluded rows where avg_price is missing because these rows cannot train or test our model for predicting avg_price_binned. We excluded the column, avg_price, because the target, avg_price_binned_ is calculated from it. usersessionid userid This is a made up foreign key used to merge user and session records. We are better to match on the raw data from each record userid is a made up foreign key. Any relationship we found with respect to a made up key is accidental and hard to interpret. For example, if the existing keys have been created in sequence, then the numeric value of the userid may vary with the number of months for which the user has been a member. This would be an accidental relationship since we do not know of a rule that says userid s have to be created in the sequence in which users join. If we want to check for a relationship between months as member and dollar value of purchases, we are better to calculate months as member from the user create date. Data Partitioning and Modeling The data was partitioned into train and test datasets. The train data set was used to create the decision tree model.

10 The trained model was then applied to the test dataset. This is important because we avoid overfitting (building a model that exactly fits the data used to build the model, but is less accurate than possible in predicting new instances) and can calculate a level of confidence in predicting the outcome of new instances of game players. When partitioning the data using sampling, it is important to set the random seed because other analysts who repeat the same steps will get the same answer (i.e. results are reproducible ). Other researchers are thereby able to verify that we have done the steps correctly. A screenshot of the resulting decision tree can be seen below: Evaluation A screenshot of the confusion matrix can be seen below:

11 As seen in the screenshot above, the overall accuracy of the model is 88.5% = ( ) / ( ) x 100 Which is also shown in the screenshot of accuracy statistics, below The cells in the confusion matrix have the following meaning: 308 time the decision tree predicted the player would be a PennyPincher and the player was, in fact, a PennyPincher 38 times the decision tree predicted the player would be a PennyPincher and the player was, in fact, a HighRoller 27 times the decision tree predicted the player would be a Highroller and the player was, in fact, a PennyPincher 192 times the decision tree predicted the player would be a HighRoller and the player was, in fact, a HighRoller. Analysis Conclusions The final KNIME workflow is shown below:

12 What makes a HighRoller? iphone users tend to be HighRollers. All other users tend to pinch pennies. Specific Recommendations to Increase Revenue 1. Display ads for high price items to iphone users. 2. For Mac and Android users, start with displaying ads for low price items, but add occasional ads for high price items the longer they play. 3. For Windows and Linux users, start with displaying ads for low price items, but an occasional ad for an iphone as they spend more months with the game.

13 Clustering Analysis (pyspark) Attribute Selection Goal: To maximize the value of the game to Eglence, vendors of in game purchases, and advertisers by encouraging the users who make purchases to stay engaged with the game for more hours per day, and stay active as members for more months. Attribute timeofday Rationale for Selection Description Extract time of day from the timestamp for the start of each session. Months Member As Rationale 1) Eglence can easily price advertising by time of day charging more for the time of day that provides the most sales. This will provide an early revenue stream with which to finance further development. 2) The logic for this measure can be leveraged to calculate other measures going forward, such as the ratio of buy clicks to game clicks. Thus the logic we deploy now, will provide a foundation for further calculations going forward Description Calculate the number of months that the user has been a member of Catch the Pink Flamingo, then count the number of purchases per session for each member TeamSz Rationale Players will follow a life cycle with use initially rising as they gain experience, and then dropping off as they tire with the game. Explicitly monitoring the life cycle will help Eglence design the game to engage users faster and keep them longer Description Count the number of members in each team Rationale Social interaction can make the game more interesting and promote member retention. Bad behavior can also cause players to leave sooner. By specifically monitoring team size, Eglence can develop an understanding of features that promote positive social interaction among members

14 PlatformCnt Description Count the number of different platform types a user uses Rationale Developing for a platform that generates few purchases (like Windows mobile and Linux) may seem unprofitable. However, before we reduce the development budget for Windows and Linux, we will check to see if users mix platforms. Training Data Set Creation The training data set used for this analysis is shown below (first 5 lines): Dimensions of the training data set (rows x columns) : 590 x 5 # of clusters created: 6

15 Cluster Centers Cluster # Cluster Center 1 monthsasmember=11, timeofday=5, teamsz=130, platformcnt= monthsasmember=20, timeofday=11, teamsz=111, platformcnt= monthsasmember=9, timeofday=11, teamsz=149, platformcnt= monthsasmember=34, timeofday=10, teamsz=143, platformcnt= monthsasmember=39, timeofday=11, teamsz=118, platformcnt= monthsasmember=11, timeofday=17, teamsz=130, platformcnt=1.05 These clusters can be differentiated from each other as follows: Cluster 1 is different from the others in that Players start playing at 5:00 am, and have been members for 11 months

16 Cluster 2 is different from the others in that Players have been members for 20 months, have teams of 111 users (smallest team size of the clusters), and start playing mid morning Cluster 3 is different from the others in that Players have been members for 9 months, have teams of 149 users (largest team size of the clusters), and start playing mid morning Cluster 4 is different from the others in that Players have been members for 34 months, have teams of 143 users, and start playing mid morning Cluster 5 is different from the others in that Players have been members for 39 months, have teams of 118 users, and start playing mid morning Cluster 6 is different from the others in that Players have been members for 11 months, start playing in the evening, and have teams of 130 users. Recommended Actions Goal: Keep users engaged in the game so they play regularly and remain active longer. Predictability and consistency makes the community attractive to advertisers and vendors. Action Recommended Drop Windows Mobile and Linux Differentiate users by number of months as active members Differentiate users by time of day of play Consider real world physical activities to appeal to older players Rationale for the action Most users never changed platform and the number of users a user used does not appear to have a significant impact on clustering. I suggest Eglence not spend money developing for a platform that attracts few users, vendors, and advertisers. There seems to be significant clustering around 9, 11, 20, and 34+ months. I suggest Eglence offer recognition/promotions for users (and advertise the promotions to all users) who reach 6 months, 1 year, 2 years, 3 years of active play, and make sure that these users find each other by offering special teams and competitions. This will encourage users to stay engaged There seems to be significant clustering around 5:00am, mid morning, and 5:00pm. Suggest Eglence conduct live events around these hours to encourage users to interact with each other as this will keep users engaged in the game. The reported date of birth shows an even distribution of players aged from 18 to 70 with 20% being over the age of 55. The pattern of engagement active use lasting years and play starting mid morning with a cluster at 5:00am suggests retired

17 people. This is older than the DOB in users.csv, but who doesn t want to be perceived as younger than they really are? I suggest Eglence provide activity guides that care givers for seniors can download from the Eglence website for physical real world activities to accompany playing the game online, and write blog posts on the mental benefits of game play. This would make more seniors aware of the game and encourage players to stay engaged.

18 Graph Analytics Analysis (neo4j) Modeling Chat Data using a Graph Data Model The image below shows the nodes and relationships that Neo4j will contain after the chat data is loaded. The table explains the relationships and nodes. Entity Description Relationships TeamChatSessio n ChatItem Team User A session in which team members can see each others chats One instance of a chat text (:user) [:CreateSession] >(:TeamChatSession) (:TeamChatSession)< [:OwnedBy] (:Team) (:user) [:Joins] >(:TeamChatSession) (:user) [:Leaves] >(:TeamChatSession) (:user) [:CreateChat] >(:ChatItem) (:ChatItem) [:PartOf] >(:TeamChatSession) (:ChatItem) [:Mentioned] >(:user) (:ChatItem) [:RespondsTo] >(:ChatItem) (:user) [:InteractsWith] >(:user)

19 Creation of the Graph Database for Chats Describe the steps you took for creating the graph database. 1. The table below describes the schema of the csv files that contain the chat data CSV File Description Fields chat_create_ team_chat.csv chat_item_ team_chat.csv chat_join_ team_chat.csv chat_leave_ team_chat.csv A line is added to this fiel when a player creates a new chat session for their team. Each record creates a node labeled, TeamChatSession, and edges labeled, CreateSession from user OwnedBy to Team A line is added to this file when a user adds a chat item to a team chat session. This record creates the node labeled chatitem and the edges labeled createchat from user, PartOf to teamchatsession, and Joins from user to teamchatsession A line is added to this file when a player joins a team chat session. This record creates an edge labeled joins from User to teamchatsession A line is added to this file when a player leaves a team chat session. This record creates an userid : id for the user who initiated the session teamid : id for the team teamsessionid : id for the session Timestamp : create date/time for the session userid : id for the user who created the chat item teamchatsessionid : id for the chat session in which the chat item was created chatitem : id the cat item Timestamp : create date/time for the chat item Userid : id of the user who joined the team chat session TeamChatSessionId : id of the team chat session Timestamp : data/time user left Userid : id of the user who left the team chat session

20 chat_mention _ team_chat.csv chat_respond_ team_chat.csv edge labeled leaves from User to teamchatsession A line is added to this file when a player mentions another player in a chat post. The record creates an edge labeled Mentioned from chatitem to User. A line is added to this file when a player responds to a chat post. The record creates an edge labeled, ReponseTo from ChatItem to ChatItem TeamChatSessionId : id of the team chat session Timestamp : data/time user left chatitemid : id of chat item userid : id of user mentioned in chat timestamp : date/time of mention chatitemid : source chat item chatitemid : chat responding to source chat item Timestamp : date/time of response 2. To load the chat data, execute the following steps Download the data from the designated repository Unzip the chat data to the neo4j default import folder D:\Users\Greg.B.Watson\Documents\Neo4j\default.graphdb\Impor t You should have the six files described in the table above Execute the following commands in Neo4j to load the csv files to the graph database. USING PERIODIC COMMIT 1000 LOAD CSV FROM "file:///chat_create_team_chat.csv" AS row MERGE (u:user {id: toint(row[0])}) MERGE (t:team {id: toint(row[1])}) MERGE (c:teamchatsession {id: toint(row[2])}) MERGE (u) [:CreatesSession{timeStamp: row[3]}] >(c) MERGE (c) [:OwnedBy{timeStamp: row[3]}] >(t) USING PERIODIC COMMIT 1000 LOAD CSV FROM "file:///chat_join_team_chat.csv" AS row MERGE (u:user {id: toint(row[0])}) MERGE (s:teamchatsession {id:

21 toint(row[1])}) MERGE (u) [:Joins{timeStamp: row[2]}] >(s) USING PERIODIC COMMIT 1000 LOAD CSV FROM "file:///chat_leave_team_chat.csv" AS row MERGE (u:user {id: toint(row[0])}) MERGE (s:teamchatsession {id: toint(row[1])}) MERGE (u) [:Leaves{timeStamp: row[2]}] >(s) USING PERIODIC COMMIT 1000 LOAD CSV FROM "file:///chat_item_team_chat.csv" AS row WITH row LIMIT 0 MERGE (:User {id: toint(row[0])}) [:CreateChat{timeStamp: row[3]}] >(:ChatItem {id: toint(row[2])}) MERGE (:ChatItem {id: toint(row[2])}) [:PartOf{timeStamp: row[3]}] >(:TeamChatSession {id: toint(row[1])}) MERGE (:User {id: toint(row[0])}) [:Joins{timeStamp: row[3]}] >(:TeamChatSession {id: toint(row[1])}) USING PERIODIC COMMIT 1000 LOAD CSV FROM "file:///chat_item_team_chat.csv" AS row MERGE (u:user {id: toint(row[0])}) MERGE (s:teamchatsession {id: toint(row[1])}) MERGE (c:chatitem {id: toint(row[2])}) MERGE (u) [:CreateChat{timeStamp: row[3]}] >(c) MERGE (c) [:PartOf{timeStamp: row[3]}] >(s) MERGE (u) [:Joins{timeStamp: row[3]}] >(s) USING PERIODIC COMMIT 1000 LOAD CSV FROM "file:///chat_mention_team_chat.csv" AS row MATCH (c:chatitem {id: toint(row[0])}) MATCH (u:user {id: toint(row[1])}) MERGE (c) [:Mentioned{timeStamp: row[2]}] >(u) USING PERIODIC COMMIT 1000 LOAD CSV FROM "file:///chat_respond_team_chat.csv" AS row MATCH (a:chatitem {id: toint(row[0])}) MATCH (b:chatitem {id: toint(row[1])}) MERGE (a) [:ResponseTo{timeStamp: row[2]}] >(b) The screenshot below was created with the following query that displays every node and relationship in the database. Teams (purplish pink) create team chat sessions (green) and users (orangish pink) create a cloud of chat items (grey)

22 The image below shows the same query with the limit set to 100 so that we can see the relationships.

23 Finding the longest conversation chain and its participants Report the results including the length of the conversation (path length) and how many unique users were part of the conversation chain. Include an image of the graph with the longest conversation chain. Ten (10) chats were in the longest conversation chain. As shown in the screenshot below, the 10 chats involved 5 users.

24 Analyzing the relationship between top 10 chattiest users and top 10 chattiest teams Include your table containing the top 3 chattiest users and teams below, and report whether or not any of the chattiest users are part of any of the chattiest teams. Chattiest Users Users Userid 394 (team 63) 115 Userid 2067 (team 7) 111 Userid 209 (team 7) 109 Number of Chats Chattiest Teams Teams Number of Chats Userid 999 belongs to team 52. Both user 999 and team 52 were in the top 10 chattiest users and teams respectively. User 999 is the only one of the top 10 chattiest users to belong to one of the top 10 chattiest teams. How Active Are Groups of Users? Report the top 3 most active users in the table below. Steps to rank the activeness of groups of users:

25 1. Construct neighborhoods of users who either mention each other or respond to each other: A user is a member of neighborhood if the user has an 'interactswith' relationship with anyone in the neighborhood. The number of distinct users in the neighbourhood is k. k(k 1) is the maximum number of 'interactswith' relationships one user can have. We subtract 1 from k because the user cannot chat with himself. We multiply because it does not matter which user initiated the interaction. The number of distinct users one user actually interactswith is n MATCH (u1:user) [:CreateChat] >(:ChatItem) [:Mentioned] >(u2:user) CREATE UNIQUE((u1) [:InteractsWith] >(u2)) MATCH (u1:user) [:CreateChat] >(:ChatItem) [:ResponseTo] >(:ChatItem)< [:CreateChat] (u2:use r) CREATE UNIQUE((u1) [:InteractsWith] >(u2)) 2. Remove any InteractsWith relationships where the user interacts with himself or herself. An InteractsWith relationship exists when a ChatItem either Mentioned another user or RepondedTo a ChatItem from another user. Since it is technically possible for a user to respond to his/her own chat items, and we are not interested in users influencing themselves, we have to remove the self loops. MATCH (u1) [r:interactswith] >(u1) DELETE r 3. Calculate the clustering coefficient. The clustering coefficient ranges from 0 (an isolated user) to 1 (a user in a group where every user is connected to every other user) The ratio n/k(k1) is the cluster coefficient ToDo: Find the number of users in the same neighborhood that each user DID NOT chat with, add that to the number each did chat with to get the number of neighbors The query to find the top 3 most active users is as follows. Note that to make this query efficient, we assume that the top 3 most active users are among the top 10 chattiest: We hardcode the top 10 list and check that the users are in that list before calculating the coefficient. MATCH (u1:user) [r:interactswith] (u2:user) WHERE u1.id IN [394,2067,209,1087,554,516,1627,999,668,461] WITH u1 AS u, COLLECT(DISTINCT(u2.id)) AS neighbors MATCH (u1:user),(u2:user) WHERE u1.id IN neighbors AND u2.id IN neighbors RETURN u.id, TOFLOAT(SUM(CASE WHEN (u1) (u2) THEN 1 ELSE 0 END)) /(SIZE(neighbors)*(SIZE(neighbors) 1)) AS coefficient ORDER BY coefficient DESC

26 Most Active Users (based on Cluster Coefficients) User ID / Coefficient Recommended Actions Finally, make recommendations to Eglence, Inc. and include examples of how your findings support them. Include this information in Slide 6 of your final presentation. Give promotions to the users with the highest chat coefficient in each time of day cluster : High chat coefficient users are likely to influence the largest number of other users to buy, and are most likely to influence users who regularly log on at the same time of day as they do. Analyze further which users get chat mentions from other teams and why: We can improve game play by discouraging behavior that puts off players, and encourage players with whom players on other teams want to play and/or want to imitate