UX Research Samples Hamilton Hernandez, PhD

UX Research Samples Hamilton Hernandez, PhD Postdoctoral Fellow Bloorview Research Institute - University of Toronto 150 Kilgour Road, Toronto, Ontario, Canada M4G 1R8 In my work over the last 8 years, I have exercised UX research and design from a truthful user-centered approach with a mix of quantitative and qualitative research methods to conceive and design systems that accurately target the needs of users in multiple domains while providing a fulfilling user experience. I have led and been engaged in various projects in the interdisciplinary fields of Human-Computer Interaction (HCI), Computer Games and Rehabilitation, and as a freelance designer, I have worked in domains including real state, automotive, tourism, and retail. The stakeholders, end-users and members of the teams I have been part of come from technical, medical and academic backgrounds and present different needs (e.g. kids with motor disabilities). The following are samples of the UX research and design conducted in some of the most recent projects I have worked on. More samples can be found on my personal website www.hamilan.com. 1. Liberi: a cycling-based exercise video game for kids with cerebral palsy The goal of this project was to provide kids with cerebral palsy (CP) a way to exercise and socialize in an accessible and engaging way. The research associated with this project covered multiple dimensions such as hardware design, software design and evaluation and optimization of the resulting system. The research I led for this project tried to answer multiple questions in multiple stages, i.e.: What are the requirements, limitations and opportunities for designing an accessible bike for kids with CP? What are the requirements, challenges and possibilities for designing an online cycling-based exercise video game (exergame) that kids with CP can play? What algorithm for mapping pedalling cadence to the movement of the game avatar offers the best user experience? What is the effectiveness of the game at eliciting significant amounts of physical activity? What is the effectiveness of the game at supporting social interactions? To answer these questions, a variety of quantitative and qualitative research methods were necessary. For example, we gathered requirements and iteratively got feedback on prototypes of the bike and game using focus groups, surveys, interviews, controlled usability studies (at the lab and at the hospital), and remote (home-based) user studies. I also designed and conducted an A/B study to identify the pedalling cadence algorithm that offered the best user experience. In the next section I present two samples of the insights derived from my research on this project. After these samples, I present a list of the publications resulting from the research on this project. 1

1.1. Effectiveness of the game at eliciting significant amounts of physical activity The idea behind the game is to engage kids with CP in physical activity that can lead to health benefits (150 minutes of pedaling with a heart rate over 64% of maximum heart rate per week) and to serve as a platform for social interaction. Research goal: To evaluate the effectiveness of Liberi at eliciting amounts of physical activity associated with health benefits, and to identify opportunities for improvement. Research questions: 1) How do each of the activities in Liberi impact levels of physical activity? 2) Are there opportunities for improving the way Liberi impacts levels of physical activity? Methodology The system: Liberi takes place in an island with a central plaza containing shops for avatars customization (Figure 1) and 6 minigames with different play styles (racing, battle, catch them all, puzzle, defense, and building). Players use a stationary recumbent bike, a commercial gamepad, and wear a headset, and a heart rate monitor to play. Study setup: 8-week home-based study with 6 kids with CP. Participants could play the game every day of the study. Figure 1 Screenshot of Liberi Data: game logs registering the player s location in the game, pedaling speed, and heart rate level at every second; notes from weekly calls to the participants; notes from research assistants that listened to the game s chat channel each session of the study; postintervention surveys on game enjoyment; and semi-structured interviews. Effectiveness was measured mainly as percentage of time over target heart rate in game activities. Insights Insights/findings Derived recommendations for designers In general, the game elicited vigorous play that elevates heart rate to the target zone, but the percentage of time players maintain their heart rate in the zone is low. Heart rate levels outside of minigames are low and therefore do not contribute to the target amounts of vigorous physical activity. There are opportunities for improvement in several minigames that allow periods of leisure pedaling that lead to low hr levels. Games with gameplay containing actionoriented components were enjoyed higher and led to more vigorous play than slow-paced games. Players play longer when they have somebody to play with Need to increase the percentage of time that players heart rate is in the target zone. See below for recommendations. Aim to reduce time players spend out of minigames. Reduce opportunities for leisure pedaling in minigames. E.g. create urgency for continuous vigorous pedaling, and reward vigorous play. Focus design of minigames on action-oriented gameplay and/or integrate action-oriented games components into minigames. Implement strategies to allow players to meet online more often. 2

Results: Most of these design recommendations were implemented in a later version of the game. The recommendation Reward vigorous play was found particularly successful by a successor team of researchers in a follow up study. Their results were reported in the following research paper: http://dx.doi.org/10.1145/2793107.2793122. The insights about the potential of action-oriented gameplay to elicit enjoyable and vigorous play were considered a priority for the re-design of the game; and therefore new research was conducted to identify how to approach this design recommendation. Below is a description of this research process. 1.2. Designing action-oriented games that kids with CP can play Research goal: To identify the way to integrate action-oriented gameplay in exergames for kids with CP in order to gain increased game enjoyment without losing playability due to limitations in physical abilities. Hypothesis: It is possible to design action-oriented games that kids with CP can play and enjoy. Methodology Setup: iterative participatory design process with 10 monthly play sessions with kids with CP. Data: game logs, notes on direct user observation, game enjoyment surveys, and interviews. Insights/Findings: This research produced low and high level insights about the challenges and ways of implementing actionoriented gameplay in games for kids with CP. Here are some samples of both types of findings: Low-level insights Derived recommendations for designers Kids with CP have limited manual ability and eye-hand coordination which makes it hard for them to time jumps and dodge enemies in platform games (a Simplify the game level geometry. Reduce consequences of errors. popular type of action game). Limited manual ability makes it difficult to rapidly select between multiple actions via a game controller (required in multiple types of action games). Limited manual ability and gross motor skills make it hard to quickly position an avatar close to a target and point in the correct direction to be able to attack an enemy. High-level insights Simplify the control scheme by removing the need for multiple active buttons on the game controller Remove need for precise positioning and aiming at a target. Following traditional guidelines for the design of accessible video games too closely leads to slowpaced games not suitable for the purpose of motivating vigorous play. Games can have fast-paced action and time sensitive and rapid interactions as long as the impact of play errors is low, the flow of the level is forgiving, the control scheme is simple and the game state is clearly visible. The action components in games were successful at creating the urgency for continuous vigorous pedaling and were enjoyed highly by participants of the sessions. Techniques for balancing differences in physical ability are needed so that players making similar effort move at a similar speed and perform similarly to enjoy the social play experience. Results: The resulting product of this process is an enhanced game whose action-oriented games were enjoyed by all the kids who participated in the iterative design process and that motivates fast-paced 3

gameplay that can lead to health benefits. Professor Nick Graham at Queen s University has created a startup company to pursue the commercialization of the game. Additional insights (low and high level) from this research and the process of how the design recommendations were derived can be found in the following research paper http://www.hamilan.com/files/liberi_action_games.pdf 1.3. Publications from the project Liberi Here are select publications resulting from this research. 1. Hamilton A. Hernandez, T.C. Nicholas Graham, Darcy Fehlings, Lauren Switzer, Zi Ye, Quentin Bellay, Md Ameer Hamza, Cheryl Savery, and Tadeusz Stach. 2012. Design of an exergaming station for children with cerebral palsy. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '12). ACM, New York, NY, USA, 2619-2628. DOI=http://dx.doi.org/10.1145/2207676.2208652 2. Hamilton A. Hernandez, Zi Ye, T.C. Nicholas Graham, Darcy Fehlings, and Lauren Switzer. 2013. Designing action-based exergames for children with cerebral palsy. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13). ACM, New York, NY, USA, 1261-1270. DOI: http://dx.doi.org/10.1145/2470654.2466164 3. Knights, Shannon and Graham, T.C. Nicholas and Switzer, Lauren and Hernandez, Hamilton and Ye, Zi and Findlay, Briar and Xie, Wen and Wright, Virginia and Fehlings, Darcy. An innovative cycling exergame to promote cardiovascular fitness in youth with cerebral palsy: a brief report. Developmental Neurorehabilitation 2014:1 6. 4. Hamilton A. Hernandez, Mallory Ketcheson, Adrian Schneider, Zi Ye, Darcy Fehlings, Lauren Switzer, Virginia Wright, Shelly K. Bursick, Chad Richards, and T.C. Nicholas Graham. 2014. Design and evaluation of a networked game to support social connection of youth with cerebral palsy. In Proceedings of the 16th international ACM SIGACCESS conference on Computers & accessibility (ASSETS '14). ACM, New York, NY, USA, 161-168. DOI: http://dx.doi.org/10.1145/2661334.2661370 2. The Falcon Project The goal of this project was to develop a station that supports hand and arm therapy exercises while playing video games using a force feedback game controller (Novint Falcon). The research associated with this project covered hardware design, software design and clinical evaluation of the resulting system. The research questions I tackled in this project were: What therapeutic goals relevant to CP can be addressed using a low-cost commercial force feedback controller (Novint Falcon)? Figure 2. Standard Novint Falcon bundle What hardware/software adaptations are needed to support these therapeutic goals? How usable and therapeutically relevant is the resultant system for therapists and kids with CP? 2.1. Therapy goals and necessary adaptations To address the first 2 questions, I led focus groups with therapists, and medical and mechanical engineers where we identified the requirements for the station. We then employed design thinking to identify potential solutions to adapt the controller to the therapeutic needs. 4

I built and specified quick physical prototypes to replace the original grip of the Falcon (not suitable for hand exercises) and tested them with the therapists and kids with CP, rapidly identifying which ones had potential and which ones needed to be discarded (Figure 3). The software component of this station was also constructed iteratively with quick prototypes that allowed testing the basic therapy goals. More features allowing setting the difficulty of the exercises were added later on at the therapists request. Finally, a set of commercial games were tested with the therapists and kids with CP in a guerrilla testing fashion (without conducting a proper usability session or study), to identify games that could potentially motivate the kids to perform the therapy exercises. Research insights This stage of the project produced a set of general recommendations and lessons for designers of gaming therapy systems. These recommendations include the adaptation or design of hardware and software, and the selection of games that effectively target therapeutic goals. The following are excerpts from the resulting design recommendations: Recommendations for adapting/designing a physical game controller Provide physical grips that are easy to grab and that allow exercises associated with the therapeutic goals. Provide flexibility in the positioning/orientation of the grips to allow different therapeutic goals and abilities between patients. Recommendations for designing a therapy management application Provide an easy way to calibrate the range of movement required by the patient and the weight and/or resistance used to play. Allow easy identification of the games therapeutic purpose. Recommendations for choosing commercial games for a gaming therapy station Usability/accessibility for patient and therapist: Should allow easy and fast initiation of gameplay so as not to impede on therapy time (e.g limited navigation of menus and storytelling). Game mechanics: Should be limited to simple navigation in a 2D plane and maximum one additional switch action. Game Challenges: Figure 3. Prototypes of grips for the Falcon and new necessary base Length/duration: should allow continuous gameplay with minimum pauses to encourage endurance, momentum and sustained interest in the activity. 5

2.2. Usability and therapeutic relevance of the resultant station In order to identify whether the resulting station was ready for deployment at the clinic, I conducted a usability study with 10 therapists and employed a validated scale (the System Usability Scale (SUS)), custom surveys and interviews to gather the therapists opinions on the system. Research Insights Some of the insights from this usability study confirmed the effectiveness of the software and hardware to target therapy goals, while others informed necessary adjustments that could improve the experience for the therapists and save time when conducting therapy sessions with the system. Insights Therapists avoid using a combination of difficulty settings and stick to only one of the available options. They say that the effect of some of the settings is confusing or not clear. Therapists spend several minutes swapping the different grips for the Falcon losing time that is valuable for the therapy sessions. Several therapists forget to save the difficulty settings for a patient before closing the app Therapists can t remember the goals of the games and how each of them is played The therapists rendered the therapy station ready for deployment at the clinic by giving it a score of 76.7 on the SUS. Derived recommendations for designers Provide an easy way to determine the difficulty settings for the patient. Options: allow setting difficulty through a single scale with clear options (easy, medium, hard), or provide automatic difficulty setting. Another alternative would be to improve the description of the different settings on the screen, and or provide training for using multiple difficulty settings. Redesign grips to provide an easier way to swap between them Implement auto save of difficulty settings Provide a way in the user interface to identify the game goals, therapy goals, and actions required to play each game. No recommendation. To evaluate the therapeutical relevance of the system, I designed and conducted a controlled longitudinal user study where 7 kids with CP went to the hospital to use the station over 12 weeks. I automated the collection and analysis of gaming-therapy data to learn about the usage of the system. In collaboration with a therapist, we also collected quantitative clinical data to determine hand and arm function before and after the 12-week intervention. Some of these measures included grip strength, the Canadian Occupational Performance Measure (COPM) and the Quality of Upper Extremity Skills Test (QUEST)). Research Insights The data collected and analyzed automatically by the system showed that the system elicits a volume of hand and arm therapeutic movements beneficial for motor learning. For example: on average, participants completed 355.5 wrist extensions in 23.1 minutes (the necessary daily number of movements to cause changes in the brain is 400 to 600), 6

12 16 19 23 27 30 34 38 41 45 49 52 56 60 63 67 71 75 78 82 86 89 93 97 100 104 108 111 115 119 122 126 130 133 137 Falcon movement units Y axis 27.5% of the wrist extensions achieved while playing the games were greater than the maximum wrist extension measured during the calibration phase, which highlights the motivation potential in the gaming aspect of the station (see Figure 4), and participants tended to score higher in a wrist speed assessment game as the weeks went by. Example of wrist extension in the game Funky Karts (P2) 2.5 2.0 1.5 1.0 0.5 0.0-0.5-1.0-1.5-2.0-2.5-3.0-3.5-4.0-4.5-5.0-5.5-6.0-6.5 Seconds Falcon Y Required Y to play (75% max) Max Y (From calibration) Figure 4. Example of wrist extension in game versus wrist extension during calibration Additionally, the function measures showed that most of the participants (6 out of 7) had improved their grip strength and scores in the COPM and one of the dimensions in the QUEST associated to grasp quality at a clinically significant level. After the positive results of this first clinical evaluation, the therapists suggested that the gaming therapy station be made available for regular hand and arm therapy at the hospital. Future research efforts will focus on determining the necessary considerations for deploying the station for unsupervised use at a home setting. Additional information on the research conducted in the Falcon project will be found in a research paper currently under revision by one of the world class journals in Human Computer Interaction (ACM Transactions on Computer-Human Interaction (TOCHI)). 7