TaleBlazer: Using ibeacons for Indoor Location-Based Augmented Reality Games. Ellen Yongin Finch

Transcription

1 TaleBlazer: Using ibeacons for Indoor Location-Based Augmented Reality Games by Ellen Yongin Finch Submitted to the Department of Electrical Engineering and Computer Science in partial fulfillment of the requirements for the degree of Master of Engineering in Electrical Engineering and Computer Science at the MASSACHUSETTS INSTITUTE OF TECHNOLOGY June 2015 c Ellen Yongin Finch, MMXV. All rights reserved. The author hereby grants to MIT permission to reproduce and to distribute publicly paper and electronic copies of this thesis document in whole or in part in any medium now known or hereafter created. Author Department of Electrical Engineering and Computer Science May 18, 2015 Certified by Professor Eric Klopfer Director, MIT Teacher Education Program Thesis Supervisor Accepted by Professor Albert R. Meyer Chairman, Masters of Engineering Thesis Committee

2 2

3 TaleBlazer: Using ibeacons for Indoor Location-Based Augmented Reality Games by Ellen Yongin Finch Submitted to the Department of Electrical Engineering and Computer Science on May 18, 2015, in partial fulfillment of the requirements for the degree of Master of Engineering in Electrical Engineering and Computer Science Abstract TaleBlazer is a platform for creating and playing location-based educational augmented reality games. This thesis describes the design and implementation of new indoor location-based functionality in TaleBlazer, based on the use of ibeacon technology. It describes how the new functionality can be used in indoor location-based games, and presents results from a pilot indoor game conducted with the Harvard Museum of Natural History. Thesis Supervisor: Professor Eric Klopfer Title: Director, MIT Teacher Education Program 3

4 4

5 Acknowledgments I d like to thank Eric Klopfer, Judy Perry, and Lisa Stump for giving me the opportunity to work on this project. It s been an experience unlike any of my others at MIT, and I greatly appreciate having gotten to learn so much about something so far from my specialty. I d like to thank Judy Perry, Lisa Stump, and Arielle Ascrizzi for their help in designing and implementing the indoor functionality, and especially for all their hard work in developing such a wonderful pilot game for us to test my work with. I d like to thank the rest of the TaleBlazer team for the good company while working on this project, and wish Bobby Fortanely the best of luck with his ambitious plans for continuing the indoor work. Finally, I d like to thank my family and friends, because as trite as the phrase is, it is also strictly and literally true that I wouldn t be where I am without their support. 5

6 6

7 Contents 1 Introduction Motivations Research Question Chapter Summary Background TaleBlazer TaleBlazer Gameplay TaleBlazer Platform Structure Using TaleBlazer Previous Indoor Games the Institute Mystery at the Museum Indoor Navigation Determining Design Goals Partnership with HMNH Approaches Considered QR Codes, Dead Reckoning, and WiFi Positioning Indoor Google Maps Approach Selected: Bluetooth Beacons Introduction to Beacons Advantages of Beacons

8 3.3.3 Drawbacks of Beacons Implementation Preliminary Testing Design Decisions Software Changes Game Creation Gameplay BLE Support Check Pilot with HMNH Game Design Game Overview Game Mechanics Pre-pilot tests First pre-pilot test Second pre-pilot test Pilot Results Observation Post-game Survey Focus Group Pilot Conclusions Future Work Editor Usability Improvements Adding Android Support Further Tests Using Beacons for Real-Time Player Positioning Conclusion 65 A Mobile Device Battery Drain Tests 67 8

9 B Research Instruments 69 B.1 Observation Protocol B.2 Post-game Survey B.3 Focus Group Questions

10 10

11 List of Figures 2-1 Outdoor game example Bluetooth beacons Beacons tab in TaleBlazer editor Beacon agent bump settings in TaleBlazer editor Agent beacon settings in TaleBlazer editor Agent ordering example Android beacons not supported message ios beacons not supported message ios prompt to turn on Bluetooth Super Survivor biome assignment screen Super Survivor trait selection screen Super Survivor trait feedback screens Super Survivor end screens Super Survivor directions screens Players lean in to closely examine animals

12 12

13 List of Tables 3.1 Indoor Navigation Goals Purchased Beacons Post-game Survey Results for Family Questions Post-game Survey Results for Adult Questions Post-game Survey Results for Child Questions A.1 Battery Drain Test Results

14 14

15 Chapter 1 Introduction TaleBlazer is a platform for creating and playing location-based educational augmented reality (AR) games on Android and ios devices, created by MIT s Scheller Teacher Education Program (STEP) Lab. TaleBlazer is intended to help informal learning spaces provide enhanced educational experiences which leverage real-world locations. The goal of TaleBlazer games is to engage players with the physical space they are visiting, making them look more closely at and think more deeply about exhibits or other aspects of their surroundings. The current version of TaleBlazer supports both indoor and outdoor gameplay. However, because TaleBlazer s source of location information is GPS, when players are indoors, the app cannot passively detect their locations. Thus, indoor TaleBlazer gameplay has limited functionality compared to outdoor TaleBlazer gameplay. This thesis will describe the design and implementation of extended indoor functionality for TaleBlazer. It will describe background research on different approaches to indoor positioning, explain the approach selected and how it was integrated into TaleBlazer, and present the results of a pilot game run at the Harvard Museum of Natural History (HMNH) with the help of the museum staff. 15

16 1.1 Motivations TaleBlazer games provide a structured learning experience within the engaging framework of a location-based AR game. In outdoor TaleBlazer regions, a player interacts with game elements, referred to as agents, by approaching their GPS locations. For example, at a zoo, a character agent could be located next to the lion enclosure; when the player walks by the lions, the app detects that their GPS coordinates have reached the area where the character is located, and causes the character to come up on the screen. Because GPS does not work inside buildings, the indoor mode of TaleBlazer needs to provide some other way to interact with agents. The current solution is to allow players to tap on agents to interact with them. If the game designer wants to ensure that the player has reached the proper location, they can secure agents by using a password the player can only obtain at the physical location of the agent, such as a word on a poster at that location. Both of these approaches require more work from the player than passive location sensing does, and the password-free approach runs the risk that players will tap through the game without visiting any of the exhibits. Ideally, indoor TaleBlazer games would be able to provide similar location-based functionality to outdoor TaleBlazer games. This goal is of particular importance because many informal learning spaces are entirely indoors, and so can use only the indoor functionality of TaleBlazer in any games they might create. Providing a more fully-featured indoor mode would thus greatly increase the utility of TaleBlazer for many of its intended users, as well as broadening TaleBlazer s audience to include indoor locations which would not consider using TaleBlazer without this functionality. 1.2 Research Question In designing the new indoor functionality, our goal was to enable TaleBlazer to be used for creating a location-based experience indoors, while remaining feasible for typical educational institutions given their logistical and other practical constraints. 16

17 Constraints include cost of implementation, visual impact of any new technology used for indoor positioning, and visitor experience. A TaleBlazer game should have high learnability and usability, because learning the platform should not be a distraction from playing the game and learning about the space. To help guide our work on the new functionality, we decided to partner with museum staff at the Harvard Museum of Natural History with the goal of producing a pilot indoor TaleBlazer game. The question my work sought to answer was, could TaleBlazer be modified to create indoor location-based games while meeting these constraints and goals in a typical informal learning institution such as HMNH? 1.3 Chapter Summary Chapter 2 provides background on the TaleBlazer platform and related indoor locationbased work done by the STEP Lab. Chapter 3 discusses the goals of TaleBlazer s indoor navigation and explains the selection of ibeacons as the approach. Chapter 4 details the implementation of the new indoor functionality. Chapter 5 describes the HMNH pilot game and presents the results of the pilot. Chapter 6 suggests possible future work on TaleBlazer s indoor functionality. Chapter 7 concludes. 17

18 18

19 Chapter 2 Background The MIT STEP lab has been working on educational AR games for more than a decade [11], although TaleBlazer itself is only a few years old. The new indoor functionality work on TaleBlazer builds off of both the existing functionality of TaleBlazer and previous indoor AR games run by the lab. 2.1 TaleBlazer TaleBlazer is a platform for creating and playing location-based AR games. As such, its target audience includes both game creators and game players. The target audience of game creators is primarily children and non-technical curriculum writers, so the game creation software is designed to be simple for a non-technical audience to understand and use. The target audience of game players includes both children playing games they have created, and visitors to sites using a TaleBlazer game as an educational activity. The complexity of a game is determined by the game s creator TaleBlazer Gameplay TaleBlazer gameplay is based on interaction with agents, which represent objects such as characters or items in the game. Interacting with an agent is referred to as bumping the agent. 19

20 Figure 2-1: Outdoor game example In an outdoor game region, as the player walks around the real location, their position is tracked by GPS and displayed as a moving dot on the game map. When the player is detected to have reached an agent s location, they bump the agent, causing the agent s bump script to execute whatever behavior the game designer has specified for this event. Figure 2-1 shows an example outdoor game; the circle icon represents the player and will move as the player moves, while the triangle icon represents an agent and is fixed in place. In an indoor game, the player s position is not shown on the map and the player bumps agents by tapping their icons on the map. As described above, depending on the design of the game, tapping can either be all that is needed to interact with an agent or the player may be required to enter a password to demonstrate that they are in fact at the correct location. 20

21 2.1.2 TaleBlazer Platform Structure The TaleBlazer platform consists of five parts: a mobile app for playing games, an online editor for creating games, a website that hosts the editor and stores account and game data, a multiplayer server, and an analytics server. The work in this thesis involved modifications to the mobile app, the editor, and the website. The TaleBlazer mobile app is built in Appcelerator Titanium, a platform that allows a single JavaScript codebase to be compiled into both an ios and an Android app. The editor is a web application written in JavaScript with an HTML/CSS front-end, and the website s back-end is PHP/MySQL Using TaleBlazer The first step in creating a TaleBlazer game is registering for an account on the TaleBlazer website. Using this account, a game designer can sign on to the TaleBlazer website and create games. To create a game, the game designer opens the TaleBlazer editor on the website and selects a region to be the map for their game. They then add agents to the game by placing them on the map. For an outdoor game, the software automatically calculates the GPS coordinates of the agents based on their location on the map. For an indoor game, the designer simply marks a region as indoors, in which case the mobile app will not attempt to locate the agents using GPS. The game designer can easily create scripts for agent interaction using the blocks-based scripting provided in the editor. When the game is saved, it is stored on the TaleBlazer server and can be downloaded and played. To play a TaleBlazer game, a user installs the TaleBlazer mobile app and then uses the app to download the particular game file from the TaleBlazer server. There are many different ways to download TaleBlazer games. If a user has a game code, they can enter it to download that game. If they are near a partner location, the app will suggest the featured game for that location. If they have their own TaleBlazer account, they can log in and download any games they have created. 21

22 2.2 Previous Indoor Games The STEP lab has conducted two major indoor AR games in the past, the Institute and Mystery at the Museum. Both games used room-level indoor positioning to provide location-based functionality. Mystery at the Museum also provided close interactions with particular museum exhibits the Institute the Institute was an indoor AR game conducted on campus at MIT in 2004 [9]. The game scenario was an outbreak of bird flu on campus; both game characters and the players themselves could fall ill from exposure to sick characters or players. The players goal, in their roles as doctors, field technicians, or public health officials, was to contain the outbreak as well as possible. The game was played using WiFi signal strength for indoor positioning. The game mechanics supported by the indoor positioning were using or acquiring items if in the same room as them, and being at risk of becoming infected if in the same room as an infected person [10]. The mechanics were based only on what rooms players were in because the WiFi positioning could only locate players to the precision of what room they were in, not where in the room they were, but this precision was sufficient for the purposes of the game Mystery at the Museum Mystery at the Museum was an indoor AR game conducted at the Museum of Science Boston in 2003 [8]. In design, this game was more similar to a TaleBlazer game than the Institute, because inspecting and interacting with the physical contents of the rooms was relevant to the gameplay. This game s storyline involved recovering a stolen artifact and apprehending the thieves who had stolen it. The gameplay consisted of both room-level interactions, such as interacting with characters and collecting clues, and close interactions with specific exhibits. Like the Institute, Mystery at the Museum used WiFi signal strength 22

23 for room-level positioning. For the close interactions, items were tagged with infrared tags that the players could scan with their mobile devices [6]. Although Mystery at the Museum was a successful game, setting up the infrastructure to support the indoor location components was difficult, requiring manual placement and mapping of the tags and the use of non-adjacent rooms to avoid bleed-over in the WiFi positioning. These difficulties made using a similar approach for other indoor games impractical. 23

24 24

25 Chapter 3 Indoor Navigation There are two components to navigation in TaleBlazer: wayfinding, which is important for guiding the player between agents, and positioning, which is important for determining that the player has reached an agent s location. In outdoor TaleBlazer, GPS serves both functions as it both provides the moving dot on the map that helps the player locate themself, and enables the mobile app to determine when the player should bump an agent. Approaches to indoor navigation can either handle wayfinding and positioning separately, or handle both with the same technology. 3.1 Determining Design Goals To help determine the goals of TaleBlazer s indoor navigation, I conducted interviews with TaleBlazer partners who have worked on indoor and outdoor TaleBlazer games, as well as with HMNH, our partner for the indoor project. These partners included both educational institutions focused on creating games for visitors to play and an after school program focused on having children create and play their own TaleBlazer games. The interviews took the form of general questions on their goals for indoor TaleBlazer games and specific questions about potential approaches to the indoor navigation problem. The common goal these partners had for their TaleBlazer games was getting players to spend more time making detailed observations of specific objects, rather than 25

26 rushing through a space and only glancing at everything. For the educational institutions, creating engaging activities to help bring visitors back for return visits was also a major goal. Everyone interviewed found that simply supplying directions had served well for wayfinding in existing indoor TaleBlazer games. In a case where the game involved working to find specific objects, so that detailed directions could not be given, giving a general description and general location had worked well. Replacing the directions approach with a moving dot on a map was appealing to the partners, but most had concerns about the level of precision possible since poor execution would likely frustrate or confuse players. Because encouraging visitors to make detailed observations is such a common goal, many partners desired a positioning system that could pinpoint individual objects. Suggested levels of accuracy included within a few feet and within inches, although some said they could work with gallery-level precision. One partner noted that an overly fine-grained position sensor could be frustrating for players, if they were unable to bump an agent because they had to be within inches of it. While many partners found that using password protected agents or simply tappable agents had worked well for past indoor games, all of them were very enthusiastic about the idea of a passive location sensing system like the outdoor GPS location. Having the mobile application sense the player s location was noted as one of the most engaging aspects of TaleBlazer, so passive location sensing would add a lot of value for indoor games. A commonly mentioned concern was the visibility of any physical infrastructure required for indoor navigation. Many partners are not in a position to easily post visible codes or pieces of technology around a space, because of the nature of the displays or the rules of the institution or both. Additionally, some partners were concerned about ensuring that people would actually take the time to think about their actions in the game, instead of running around to find everything included in the game. Having the indoor positioning infrastructure be readily visible could be a disadvantage, leading players to take more of a scavenger-hunt approach, even 26

27 for games not designed as scavenger hunts. However, it was also noted that having objects clearly marked as in-game could be useful in some cases. Although cost was not raised as a concern by most of the partners, the potential financial cost of an indoor navigation system requiring new technology is also a major concern. Because TaleBlazer is meant to be usable by a broad range of users, keeping game implementation costs down is a priority for the project. Thus, requiring expensive additional technology for indoor games should be avoided. In a similar vein, another important priority is keeping game creation simple. There should not be much overhead in setting up an indoor TaleBlazer game compared to an outdoor TaleBlazer game, both in terms of physical setup of infrastructure and any extra software configuration required. Table 3.1 summarizes the goals for TaleBlazer s indoor navigation. Table 3.1: Indoor Navigation Goals 1) passive location sensing 2) no overly visible infrastructure 3) fine-grained location sensing 4) low cost 5) easy to set up Partnership with HMNH In order to test indoor TaleBlazer in an authentic environment, we decided to partner with the Harvard Museum of Natural History to develop both the new indoor functionality and a pilot game for end user testing. Because of the partnership, HMNH s goals for TaleBlazer s indoor navigation were given particular weight in the design process. The partnership also led us to focus on developing a polished pilot game and collecting end user data, rather than focusing on experimenting with the technology. HMNH uses a fairly traditional approach to exhibits, primarily consisting of specimens in glass cases. Because of this exhibit design, HMNH s existing visitor activities are focused on getting visitors to look closely at exhibits and make detailed observa- 27

28 tions. HMNH s goals for an indoor TaleBlazer game are the same as for their existing activities. They would like to get visitors to look more closely at exhibits and draw connections between exhibits, while having fun and hopefully deciding to return for a second visit. The goal of getting visitors to spend more time looking at and thinking about specific exhibits instead of looking at everything quickly is particularly important to the museum. HMNH was less concerned with the visibility of infrastructure than some of our other partners. Although visible codes were rejected, small pieces of (sometimes visible) technology were considered acceptable. They were willing to work with whatever level of precision we could offer, although they required some degree of passive location sensing to make using TaleBlazer seem worth the effort to them. With no passive location sensing, they felt they could do as well with the paper activities they already had. Ease of setup was not mentioned as a concern, likely because they knew that they would be working closely with TaleBlazer staff to implement their game. 3.2 Approaches Considered Indoor navigation is an active research area, so there are many approaches available for consideration. In this section, I will discuss various approaches that were considered for use in TaleBlazer, and explain the benefits and drawbacks of each QR Codes, Dead Reckoning, and WiFi Positioning QR codes, dead reckoning, and WiFi positioning were all considered briefly. The primary benefit of these approaches is that they are low cost and capable of providing a high level of accuracy in position. QR codes are square barcodes that can be scanned by a smartphone. A major benefit of using QR codes for positioning would have been ease of configuration. TaleBlazer would generate a code for each agent, and then the game designer could simply print out the codes and place them appropriately. QR codes would provide good precision, because players would need to be close enough to scan the code, and 28

29 the codes could be printed at various sizes to be more or less easy to find. This approach was largely rejected because of the difficulty partners said they would have in getting permission to post something so visible and unattractive. Additionally, this approach would not really provide much more functionality than the existing password-protected agents because it uses essentially the same idea of verifying indoor positions. Dead reckoning refers to using a device s accelerometer and compass to determine a user s position. A dead reckoning approach calculates a user s movement away from a known starting position based on the motion of the device. Dead reckoning has been shown to be able to track a user with only 6% error from the path walked in a test run, and could potentially be used not only to locate the player but also to provide a continuous moving-dot display of their position [2]. The dead reckoning approach was discarded because it requires training of models to achieve its high level of accuracy. The accuracy is also heavily dependent on a highly accurate starting position, which could be difficult to ensure in a TaleBlazer game. There is also a possibility of variation in accuracy between devices, because of differences in hardware. WiFi positioning, which uses detected strengths of WiFi signals to triangulate a user s position, is a popular idea in indoor navigation. This form of indoor positioning was in fact used in both previous indoor games run by the STEP Lab, though in those games it was only able to determine what room a player was in and not their position within the room. In more recent years, WiFi positioning has been used to create highly accurate indoor positioning systems with a 1.5-meter level of precision, which could be useful for TaleBlazer [7]. Unfortunately, WiFi positioning requires a great deal of setup, because the position calculation is based off of a table of fingerprints of WiFi signal strengths at various known locations, which requires someone to collect that data. Also, inconveniently for TaleBlazer, this approach works best when there are many different WiFi access points available around an area. For a small museum with a single WiFi network and few routers, WiFi positioning is unlikely to offer the level of accuracy researchers have accomplished on college campuses. 29

30 3.2.2 Indoor Google Maps Indoor Google Maps uses a combination of WiFi positioning, GPS, and sensor data for indoor navigation [4]. The intended functionality of Indoor Google Maps is to provide the same sort of navigation that Google Maps already provides for outdoor areas, with a moving dot displaying the user s current position on the map. This technology was a strong contender for use as a component of TaleBlazer s indoor navigation solution. We theorized that if Indoor Google Maps could supply the 6- meter accuracy they claimed [4], it would be quite simple to integrate the Indoor Maps API into TaleBlazer, which already uses the standard Google Maps API. Although 6 meters is not precise enough for TaleBlazer s indoor positioning goals, because at that distance in a typical museum a player would not necessarily be in visual range of specific artifacts, this approach seemed like a promising solution for wayfinding which could be supplemented with some other approach for positioning. A drawback of this approach was that getting an indoor space on Indoor Google Maps requires submitting floorplans to Google and working with them to map the space, so there could be a slow turnaround on actually getting an indoor map. To determine the viability of using Indoor Google Maps in TaleBlazer, I visited various locations offering Indoor Maps to see what the user experience was like. My initial test, in the spring of 2014, was a visit to the Smithsonian Museum of Natural History in DC. This museum seemed like a fair example of a space that an indoor TaleBlazer game might be played in. Unfortunately, the indoor positioning was not successful. For most of the test, the phone I had at the time (an HTC Inspire running Android) either incorrectly placed me outside, or placed me indoors but on the wrong floor of the building. The user dot appeared in the correct room and tracked me across it only once. Because technology progresses so rapidly, we hoped that Indoor Google Maps might improve in accuracy by the time I would be implementing the indoor functionality, so we continued this investigation in the fall. During the fall semester, we contacted Google employees who worked on Indoor Maps to ask for a good example 30

31 of Indoor Maps in the Boston metropolitan area. At their suggestion, we visited the Boston Public Library. Unfortunately, even on a new HTC One Android phone and an iphone 6, this location had the same problems as the Smithsonian in level of accuracy. Although there remains a possibility that the locations we visited were simply poorly mapped, we concluded that the technology was not yet mature enough for use in TaleBlazer. However, it continues to be a possibility for future versions of TaleBlazer s indoor navigation. 3.3 Approach Selected: Bluetooth Beacons The approach we decided to use was Bluetooth beacons, also known as ibeacons after the Apple specification that they follow. In this thesis, I will be referring to them simply as beacons. Figure 3-1 shows some examples of beacons, specifically the models that we purchased for testing. From left to right, the brands are Estimote, kontakt.io, GeLo, and PassKit. Figure 3-1: Bluetooth beacons 31

32 3.3.1 Introduction to Beacons Beacons are small Bluetooth devices that use a technology called Bluetooth Low Energy (BLE) and regularly broadcast packets in the ibeacon format. ibeacon packets contain the beacon s identifying information and the power level at which the beacon is transmitting. By comparing the advertised power level with the received signal strength of the packet, a mobile device can approximately calculate its distance from a beacon. The ios and Android operating systems both provide libraries for determining three proximity levels based on signal strength: immediate, near, and far. According to Apple, immediate is physically very close to the beacon. Very likely being held directly up to the beacon, near is 1-3 meters with a clear line of sight, and far is given when the device is not confident enough in the accuracy of the reading to claim immediate or near [3]. Beacons vary in size from just over the size of a quarter to the size of a deck of cards. The variability in sizing is somewhat tied to choice of batteries: a beacon can run on a coin cell and be quite small, or run on easier to replace AA batteries and be much larger. USB beacons also exist, although in a museum space they would likely be difficult to place, because they require a computer or at least a power outlet and adapter. USB beacons do however have the advantage of not requiring batteries, eliminating both the hassle of replacing batteries and concerns about beacons dying unexpectedly. The advertised battery life for beacons ranges from 6 months to over 5 years; it is difficult to gauge how long any given beacon will actually last, because they are such a new product that no one has had beacons for long enough to verify the battery life claims. However, it seems reasonable to expect that beacons will achieve at least half their advertised battery life, and possibly more. Some companies are also introducing the ability to put beacons to sleep during non-business hours, which could lead to significant improvements in beacon battery life. Beacons range in price from small stickers for $10 each to rugged outdoor beacons for $35 each, with most options around $

33 3.3.2 Advantages of Beacons Beacons have several advantages for a platform like TaleBlazer. They are easily repositioned and reused in different games. They are easier to set up than most other possible approaches, because they require no calibration. A game designer can simply enter the IDs of their beacons in Taleblazer and place the beacons around their space, and immediately be able to detect when a player is within immediate, near, or far range of those points. Compare this easy setup to WiFi fingerprinting, which requires taking measurements of WiFi signal at every area in the game. Beacons are also much less obtrusive than the other easy-to-set-up approach, QR codes, because they do not need to be visible to the player to be detected by the player s mobile device. In terms of gameplay functionality, beacons are particularly promising because they offer passive location sensing and multiple levels of precision. As mentioned above, the passive location sensing of outdoor TaleBlazer games is considered one of the most engaging aspects of the games. Being able to bump agents without requiring user action in the UI is a major plus for TaleBlazer s indoor functionality. The multiple levels of precision are useful because they can support different game mechanics. An immediate trigger could be set for a game where finding a specific small item among other small items was important, whereas a near trigger could be used for finding a specific large item in a gallery. Far triggers could be used to give hints or warn users they were moving too far away from a game area Drawbacks of Beacons Although beacons have many advantages as an indoor navigation approach for Tale- Blazer, they also have some drawbacks. Since BLE is a new technology, only newer smartphones feature the option to listen for beacons. However, we feel that because progress in smartphones is so rapid, in the near future most smartphone users will have BLE-enabled phones. Similarly, the cost of purchasing beacons is a concern that we hope will be mitigated with time as beacons become more common and inexpensive. Although at the current prices, beacons are too expensive to be practical for 33

34 TaleBlazer s non-professional target audience, this barrier will hopefully be removed in the future. 34

35 Chapter 4 Implementation The implementation of the indoor functionality had three distinct phases. The first phase was conducting simple feasibility tests with the beacons. The second phase was deciding what exact functionality the beacons would provide. The final phase was the implementation of the chosen functionality in the TaleBlazer software. 4.1 Preliminary Testing We purchased an assortment of beacons from various companies, selected based on their different sizes, price points, and claimed battery life. Most of our purchased beacons were also waterproof, which is of minor interest, as it might become relevant if someone were to want to use the beacon functionality in an outdoor game or even for an indoor use such as a game in a humid greenhouse. Table 4.1 summarizes our beacon purchases. The first round of basic feasibility testing consisted of simply detecting the beacons in a Titanium app. Although ibeacon functionality is provided by both the ios and Android operating systems, it is not directly supported by Titanium s core libraries. Instead, it is necessary to use what Titanium calls modules, libraries extending Titanium s functionality to include additional features from a particular operating system. Modules can be created by Appcelerator or by third-party developers. Although Titanium does not supply ibeacon modules, both Android and ios third-party ibeacon 35

36 Table 4.1: Purchased Beacons Type Estimote beacon kontakt.io GeLo Passkit GemTot Price $99/3 $135/5 $35 each $25/2 Battery life 3 years 6 months 2 years N/A Power source Single-use battery CR2477 battery AA batteries USB Size 5.3 x 3.5 cm 5.5 x 5.5 cm 8.7 x 4 cm 1.8 x 1.3 cm Waterproof Yes Yes Yes No modules are available for use: liferay-android-beacons for Android [1] and TiBeacons for ios [5]. We were thus able to use ibeacon functionality without needing to develop our own modules. After creating a Titanium app that could detect the beacons, the next step was checking what distances were reported as immediate, near, and far. I found that on my Android device, an HTC One smartphone, the distance at which the readings switched from immediate to near was generally around 3 feet, but varied from 14 inches to 56 inches, while far was hardly ever detected by my phone. On ios devices, I found that getting an immediate reading tended to require the device to be held closer to the beacon than with the Android, and sometimes also for a longer period of time before it would register. The major discrepancy between ios and Android was the outer boundary of near readings in open space, which on Android devices was sometimes far out as 75 feet, while the ios devices needed to be within 25 feet to get a near reading. Since our primary concern was close interaction, which we envisioned as using the immediate reading, we decided that these levels of precision were good enough to go forward with. 4.2 Design Decisions Once we had determined that using beacons for indoor positioning would be feasible, we had to decide how precisely to integrate them into TaleBlazer. The first major decision was whether to use the beacons only for positioning, in the form of point 36

37 interactions, or to try to use them for both wayfinding and positioning by triangulating the player s position and creating a moving-dot positioning system like the one provided by GPS. We decided to use beacons for point interactions because that is the use they are designed for. Additionally, I expect that Indoor Google Maps will become viable in the near future, and preferred to focus on an approach that could be supplemented by the use of Indoor Google Maps rather than being replaced by it. Once we had decided to use point interactions, we had to decide how beacons would be linked to agents. Our initial thought was that a beacon could be the location of a single agent, allowing a simple one-to-one association between them. However, we soon realized that that would be impractical, in large part because beacons, though relatively inexpensive, are not so cheap that it is reasonable to require a game designer to use one beacon for each agent in their game. It is not uncommon for even small TaleBlazer games to have 20 agents. Also, in some TaleBlazer games, it may be useful to have two different agents appear at the same location at different points in the game. Because of these considerations, we decided to allow each beacon to be associated with multiple agents. Our third decision was whether and how to use the immediate, near, and far readings. Initially, I had thought we would only use the immediate reading as a trigger, because I had been focused on enabling close interactions, which are a major strength of the beacons. However, we realized that for our pilot version of the software we could learn more by allowing agents to be set to trigger at any of the readings and testing the different behaviors, so we decided to allow the game designer to set which reading should trigger an agent. 4.3 Software Changes The first step in adding beacons to TaleBlazer was enabling beacons to be added to a game, by adding beacons to the game file format and adding beacon configuration functionality to the editor. The next step was adding beacon sensing to the mobile app so that the beacons could be used for gameplay. The final change was adding a 37

38 check for beacon support, so that players can avoid downloading a game with beacons if their device does not support the beacon functionality Game Creation A beacon is identified by a three-part ID, which takes the form of a 128-bit UUID, a 16-bit major, and a 16-bit minor. The UUID is generally shared by all beacons created by the same manufacturer, while the major/minor pair uniquely identifies the beacon within the set of beacons created by that manufacturer. In order to use beacons, a game must contain a list of beacon IDs, so the very first step of implementation was adding a list of beacons to the game file format. To help game creators keep track of their beacons, a beacon in TaleBlazer also has a nickname field, which can be used to store a description of the beacon s physical location. Once beacons were included in the game file format, the next step was adding the user interface for configuring beacons to the editor. This step was done very simply, with a focus on enabling a pilot in the mobile app as quickly as possible. As such, the usability and learnability of this editor feature was not given much thought, and improving the UI is high on the list of future priorities (see Section 6.1). To configure beacons, there is a new beacons tab, analogous to the already existing agents tab (Figure 4-1). The beacons identifying fields UUID, major, minor, and nickname can all be entered on this tab. This tab also contains the bump settings for beacon agents, an overall control for the behavior of agents triggered by beacons separate from the overall settings for the behavior of agents triggered in the standard way (Figure 4-2). On the agents tab, the agents now have the option to be triggered by a beacon (Figure 4-3). With the bump this agent via a beacon option selected, the game designer can select which beacon to trigger on, and whether to trigger on immediate, near, or far. 38

39 Figure 4-1: Beacons tab in TaleBlazer editor Figure 4-2: Beacon agent bump settings in TaleBlazer editor 39

40 Figure 4-3: Agent beacon settings in TaleBlazer editor To configure a beacon in a game, the game designer first needs to find the beacon s identifying information. In the current implementation, they will need to use software provided by the beacon manufacturer to collect this information, and then enter it into the beacons tab by hand. Once the beacon has been entered, it will appear as an option in the dropdown list of beacons when bump this agent via a beacon is selected for an agent, and the game designer can proceed to select the range reading which should trigger an agent bump. When all the beacons and agents have been configured and the game has been saved, the game is ready to be downloaded and played in the mobile app Gameplay The changes for gameplay were straightforward. As mentioned above, there are existing Android and ios modules for using beacons in Titanium, which were used in the feasibility testing stage. To access beacons from within TaleBlazer, these modules simply needed to be integrated into the TaleBlazer app. Listening for beacons has two modes: monitoring and ranging. Monitoring tracks when a device moves in and out of range of hearing a beacon, while ranging produces the immediate, near and far readings which are used in TaleBlazer. An app can listen for individual beacons by UUID, major and minor, or for all beacons with the same 40

41 Figure 4-4: Agent ordering example UUID and major, or for all beacons with the same UUID. For simplicity s sake, since a TaleBlazer game file stores beacons as individual entities and does not know about shared UUIDs or majors, TaleBlazer turns on ranging for each beacon in the game separately using all three ID fields. When an app enables ranging for a particular beacon, the mobile device listens for ibeacon packets containing that beacon s identifying information. When it receives such a packet, it compares the received signal strength of the packet against the power level advertised in the packet, and uses that comparison to estimate the distance to the beacon. The estimated distance is then translated into an immediate, near, or far reading which is passed to the app. TaleBlazer keeps track of the most recent reading it has heard for each beacon in the game. When the app checks to see if any agents should be bumped, it runs through the list of agents, and for each beacon agent compares the trigger setting to the stored reading for the corresponding beacon. Agents are bumped if the stored reading is within the range of the trigger, so a near agent triggers on both immediate and near readings, and a far agent triggers on all readings. The check for agent bumps was initially implemented on a timer, but for better responsiveness the code was changed to run the check as soon as the reading for a beacon changes. The order in which agents trigger is the order in which they appear in the game file, which is the same as the order of the agents as displayed in the agent ribbon in the editor. Figure 4-4 gives an example of agent ordering in the editor. In this example, all the agents are associated with the same beacon, but set to trigger at 41

42 different ranges. If the player were in immediate range of the beacon associated with these agents, the game on the left would display the immediate agent, then the near agent, then the far agent, while the game on the right would display the far agent, then the near agent, then the immediate agent. After this functionality was implemented, I conducted some simple battery life tests to determine how much ranging for beacons drains a mobile device s battery. I found that a game with 3 beacons was no different in battery drain from a game with no beacons, while a game with 9 beacons only reduced the total battery life of the device by 15%. For more details on the results, see Appendix A BLE Support Check The check for BLE support is designed to give the user meaningful feedback before they try to play a game that will not work on their device. The check provides a graceful failure mode for devices which do not have BLE enabled; for devices with BLE support, it serves as a check to ensure that Bluetooth is turned on. To support this check, we first added a beacon flag to the backend. TaleBlazer game information is downloaded in two steps. First, the app downloads a small amount of basic information on the game, such as its name and file size. This information is displayed to the user before they confirm that they want to download the game. With the beacon flag added to this information, TaleBlazer can run the BLE check when the user asks to download a game with beacons, and let them know that they cannot play the game without downloading the game file. Running the check at this stage also means that the check can be run only for games that actually use the beacon functionality, and not inconvenience users who only play games without beacons. The check itself is provided by the Titanium modules for handling beacons, which both include a function to check if the device the app is running on has BLE. Unfortunately, the platforms provide different levels of detail in the result of the check. On both platforms, the check can only pass if Bluetooth is currently on, but on Android the check has only a single failure type. Thus, an app cannot tell if the check failed 42

43 Figure 4-5: Android beacons not supported message because Bluetooth was off or because BLE was unsupported. However, this complication is not currently relevant, because for reasons which will be explained in section 5.2, we decided to drop the Android functionality from this version of TaleBlazer Indoor. On Android devices, regardless of whether the device has BLE support, the BLE check result screen displays Sorry, you can t play this game because it uses beacons and TaleBlazer does not support beacons on Android at this time. (Figure 4-5). On ios devices, the check can determine that BLE is not supported even with Bluetooth off, in which case the check result screen displays Sorry, you can t play this game because it uses beacons and this device can t communicate with beacons. (Figure 4-6). If the device does have BLE but Bluetooth is off, the BLE check will find that Bluetooth is off and TaleBlazer will prompt the user to turn Bluetooth on and re-run the check (Figure 4-7). 43

44 Figure 4-6: ios beacons not supported message Figure 4-7: ios prompt to turn on Bluetooth 44

45 Chapter 5 Pilot with HMNH In order to test the new indoor functionality, we conducted a pilot game with HMNH. The game, Super Survivor, was designed for 6 to 10 year olds to play with their parents in about half an hour. 5.1 Game Design As mentioned above, HMNH uses a traditional approach to exhibits, consisting mainly of specimens in glass cases. Many of the exhibits are arranged by some classification, such as ungulate mammals, instead of in dioramas, and most of the exhibits are non-interactive and cannot be touched by visitors. Because of these constraints, HMNH s existing visitor activities focus on having visitors make close observations of exhibits. A benefit they hope to derive from using TaleBlazer is the ability to provide an overarching story to guide the activity, instead of simply directing visitors to look at various exhibits. We worked closely with Arielle Ascrizzi, a member of the Digital Initiatives staff at the museum, to design our pilot game, Super Survivor. The target age group for Super Survivor was tweens between 6 and 10 years old, because HMNH would like to have more draw for that demographic. Like HMNH s other activities, the goal of the game was to make visitors make close observations of particular specimens. 45

46 5.1.1 Game Overview Super Survivor is set in HMNH s Africa, New England Forests, and Great Mammal Hall animal exhibit halls, and teaches players about various adaptations animals can make to survive in different environments. The goal of the game is to design an animal that would be able to survive in one of three biomes the rainforest, the tundra, or the desert which the player is randomly assigned at the beginning of the game. The biome assignment screen includes some basic information about the biome, to help players figure out what traits might be useful in that environment. Figure 5-1 shows the tundra assignment screen. Figure 5-1: Super Survivor biome assignment screen 46

47 Figure 5-2: Super Survivor trait selection screen The player designs their animal by selecting traits from animals in the exhibit halls. For each of four traits size, body covering, teeth, and feet the player is sent to look at three different neighboring animals and select which animal they would like to mimic for that particular trait. For example, for size the player can select between the size of a moose, of a sheep, and of a mouse deer, while for feet the player can select jumping feet like a wallaby, digging feet like a wombat, or webbed feet like a platypus. Figure 5-2 shows the feet selection screen. As players make their decisions, they receive feedback on how well an animal with their chosen adaptation would survive in their assigned environment. Based on feedback from our pre-pilot testing (see Section 5.2.2), the final pilot game also included the option to go back and make a different choice of adaptation (Figure 5-3). Pre-pilot feedback also led to the inclusion of a star rating at the end of the game, which let the player know how well they did on a 3-point scale (Figure 5-4). 47

48 Figure 5-3: Super Survivor trait feedback screens Figure 5-4: Super Survivor end screens 48

49 5.1.2 Game Mechanics Figure 5-5: Super Survivor directions screens Super Survivor s game interaction model was heavily influenced by the arrangement of HMNH s display cases. Because most exhibits are behind glass, we decided it would be difficult to support the very close interaction model of tagging a specific animal in a case with a beacon and requiring the player to bring their device close that animal. Additionally, we did not have enough beacons to tag all twelve animals in the game. It was also unclear how we could effectively direct players to find the animals in the close-interaction approach, because finding them by only their name seemed too difficult, while displaying a picture would remove the need to physically find the animal to examine it. Our solution was to direct players to look for a familiar landmark animal in the vicinity of the three animals the player could select from for each trait. This direction takes the form of an icon on the map and an image of the animal to look for (Figure 5-5). Instead of requiring the players to get very close to the landmark 49

50 animal, the game considers them close enough when they stand near that animal, thus avoiding the difficulty of placing the beacon extremely close to the landmark animal or any of the selectable animals. The flow of the game is that a player selects a trait to choose, is directed to the landmark animal, finds the landmark animal, and is directed to find the three choice animals nearby and then make a decision. For example, when the player selects choose your feet, they are directed to look for a koala; when they reach the koala, they are prompted to look nearby for a wallaby, a platypus, and a wombat, and then choose which animal s feet they would like to have, with the option to change that choice based on the feedback provided by the game. 5.2 Pre-pilot tests In order to have as smooth a run of the pilot game as possible, we conducted multiple rounds of testing before the pilot. Our first test was conducted with only TaleBlazer staff and Arielle, and simply checked the behavior of the beacon triggers. Our second test was conducted with members of the museum staff, and consisted of having them play through the first complete version of the game First pre-pilot test This test was our first experiment with the beacons placed as they would be in the game, with some beacons inside cases and behind glass. This test had a very significant result, which was that we determined that the discrepancy between ios and Android was too extreme to continue using the beacons in the same way across both platforms. In particular, from our preliminary testing in the lab, we had thought that the approximately 3-foot radius of the immediate reading on Android was shared with ios, and so used that as our main proximity check, while near was used to give intermediate directions. In practice, with the glass cases attenuating the signal, the ios devices had to be held within a foot of the beacon to get an immediate reading, while the Android devices continued to register immediate within 3 feet of the 50

51 beacon. More jarringly, the near reading registered at very inconsistent distances on the different Android devices, and sometimes as far away as halfway across a different room. The ios devices were much more consistent, giving immediate readings within approximately half a meter, near within 3 meters, and far within 20 meters of the beacon. Based on these results, we decided that it was not practical to present the same game on both platforms, because the user experience would be so different. Because the ios devices were consistent with each other, we decided to use only ios devices for the pilot. In the future, hopefully newer iterations of Android BLE support will allow beacons to be used on Android without requiring calibration of the distances for the readings, which is currently the only possible solution to the discrepancy. Based on the distances we measured for the ios range readings, we decided that our planned game flow could implemented using the near and far triggers instead of the immediate and near triggers. In order to also test the immediate trigger, because the close interaction that provides could be useful in a TaleBlazer game, we added a new game component in which leaning close to the beacon triggered a fun fact popup Second pre-pilot test Our second pre-pilot test was a play-through with members of the museum staff. This test was intended primarily to find any problems with the game flow. We had two main game-design takeaways from this test: that the game would be improved by adding the option to redo the choices after making them, and that the immediate trigger fun facts were not well integrated into the game. However, although the immediate trigger component of the game stood out as a late addition that did not mesh with the flow of this particular game, no one had been confused by the mechanic of leaning in to trigger the popup. Thus, we concluded that the mechanic worked, but since the gameplay was not well served by our use of it we removed the fun facts from the game. As mentioned above, we added the requested option to redo the choices, which we expected would improve the educational value of the game by allowing players to learn from their mistakes if they made a poor choice of adaptation. 51

52 Another simple game mechanic change that was requested and implemented for the pilot was the star rating at the end of the game. Another major result of this test was the experience of losing a beacon. For this test, the beacons were simply placed where they would be for the game, with no signage indicating that the beacons were supposed to be in those locations. As a result, one of the beacons was picked up by a visitor and returned to the gift shop. The disappearance of the beacon was discovered when some of the testers complained that they had been in the correct location for a long time without having the game register that they were there. When we investigated, we found that the game had not detected their location because the beacon was gone. This event raises some concerns about using beacons in a real game. It may be unlikely that this would occur in a real game, as in a real game the beacons would be secured to their locations. However, it will likely be important to place the beacons as out-of-reach as possible, and provide some indication that they belong where they are and should not be moved by visitors. 5.3 Pilot Results We conducted our pilot test with nine families who visited the museum on a Saturday. Most of these families were recruited for the pilot by an from the museum, but some were recruited from the regular museum visitors to fill in for families who failed to show up. Among the nine families, we had seven groups with one adult and two groups with two adults. We had one family with three children, four families with two children, and four families with one child. The children s ages ranged from 6 to 12, with an average age of 8 and a half. The game was played in families, with one device per family. The devices were ipads and iphones provided by TaleBlazer and museum staff, because the indoor version of TaleBlazer was a test build and thus due to the constraints of the Apple development environment could not be easily installed on the participants personal devices. In order to determine how successful the game was, we followed an observation protocol while the families played the game, administered a post-game survey, and 52

53 Figure 5-6: Players lean in to closely examine animals conducted a focus group with the families after each run. The observation protocol, post-game survey, and focus group questions are given in Appendix B. We conducted three separate runs in order to have an observer for each family. The amount of time each family took to complete the game ranged from 11 minutes to 31 minutes; the average duration was 20 minutes Observation Our observation data showed that the children were for the most part highly engaged by the game. Only two children were observed to be off-task, one of whom was visibly not feeling well during the game. Three of the children spent a fair amount of time on what we considered related tasks, looking at exhibits but not focusing on the game. The others were on-task for the vast majority of the game. The adults frequently took an active role in playing the game, reading the text to the children and discussing the choices with them. In observing the gameplay, we were particularly pleased to note that the players were looking very closely at the exhibits- many players bent down to the ground to get a closer look at an animal s teeth or feet (Figure 5-6). One potential issue that was noticed during the observation was that the location- 53