Overcoming World in Miniature Limitations by a Scaled and Scrolling WIM

Transcription

1 Please see supplementary material on conference DVD. Overcoming World in Miniature Limitations by a Scaled and Scrolling WIM Chadwick A. Wingrave, Yonca Haciahmetoglu, Doug A. Bowman Department of Computer Science Center for Human Computer Interaction, Virginia Tech 660 McBryde Hall Blacksburg, VA 24060, USA {cwingrav,yoncah,bowman}@vt.edu ABSTRACT The World In Miniature (WIM) technique has effectively allowed users to interact and travel efficiently in Virtual Environments. However, WIM fails to work in worlds with tasks at various levels of scale. Such an example is using the WIM to arrange furniture and then leaving the room to travel the city using the WIM for navigation and wayfinding. To address this problem, scaling and scrolling were added to the WIM to create the Scaled Scrolling World In Miniature (SSWIM). The interface and testbed were iteratively created under expert evaluation and multiple formative user evaluations led to the final design. The WIM and SSWIM were then compared inside three differently sized cities by users who located a sphere and traveled into it to read the label at the sphere s center. Users were administered two standard psychology tests to account for spatial orientation (Cube Comparison Test) and spatial scanning (Maze Tracing Test) factors. The results show that the SSWIM s added functionality, and hence complexity, caused no significant hit in user performance and additionally that users were able to use SSWIM effectively after a short instructional period. To better understand the effect of experience, a follow-up experiment was performed showing performance plateaued after ten to fifteen minutes of use. ACM Classification: H5.2 [Information interfaces and presentation]: User Interfaces - Graphical user interfaces, - Usercentered design General terms: Design, Human Factors, Performance, Experimentation Keywords: Virtual Reality, interface design, travel, WIM, SSWIM. 1. INTRODUCTION The standard World in Miniature (WIM) technique [1] has been successfully used to manipulate objects and travel in Virtual and Augmented [2] Environments. This benefit is accorded by a separate exocentric view of the world placed in the palm of their hand. The exocentric view allows user to understand the arrangements of objects and themselves by rotating and moving the WIM to view from different angles and at different distances. When the world becomes very large or very small, the fixed scale of the WIM makes it hard to interact with objects and to travel. The current work has been the addition of scaling and scrolling to the WIM to overcome these limitations with a new technique called the Scaled Scrolling WIM (SSWIM) (Figure 1). This technique was created over a series of formative expert and user evaluations to reach the final design. Since the WIM can only function in single level of detail worlds, a summative comparison evaluation of the SSWIM against the WIM was conducted in a world where the WIM could function to determine the amount of overhead the additional SSWIM functionality added. A follow-up study evaluated the effect of experience. It should be noted that the SSWIM allows the WIM technique to be applied to new tasks because it can change scales and scroll. A comparison of the WIM and SSWIM techniques in these new tasks are not possible since the WIM can not operate at scales much larger or much smaller than to what it is originally set. The potential for the SSWIM is higher than the standard WIM because of the ability to change its view. This view can be of a location in another part of the environment by scrolling or at different scales by scaling. This enables new types of interaction and work using crystalball-like metaphors and magic lenses [3] to visualize richer information such as radiation levels, heat, smoke or building stress. Both WIMs can also create versions of the space that are spatially out of sync with the virtual world or temporally out of sync for prototyping or reviewing tasks Related Work The WIM concept was originally created by Stoakley and Paush [1] who discussed object selection, object manipulation, user travel, visualization issues and props in the WIM. A qualitative evaluation was performed which found that users easily understood object mappings between the virtual world and the proxy WIM objects. The major issue recognized was problematic user orientation during WIM travel. A later short paper [6] discussed how flying into the WIM reduced this problem. IEEE Symposium on 3D User Interfaces 2006 March 25-26, Alexandria, Virginia, USA /06/$ IEEE Figure 1 The SSWIM technique enables immersed users to scale and scroll a WIM in a Virtual Environment. 1 11

2 Figure 2. The user s WIM position was shown as a cube with drop-down lines and a bull s-eye to help show its position. A large arrow showed the scroll direction when scrolling. Another WIM, the STEP WIM [7], is located on the floor and interacted with by the feet. The main advantage is the freeing of the hands for other tasks. The STEP WIM, though it has no scrolling, has two methods of scaling the scene which are 1) an explicit mode entered by clicking the user's heels together and 2) scaling when the user stands on their toes or crouches. The methods of scaling, though creative, require encumbering the user with special slippers and an extra belt tracker as well as possibly mistakenly mapping a user standing on their toes and crouching with scalings. The STEP WIM was used successfully in multiple projects, but its effectiveness was not evaluated formally. 2. THE SCALED SCROLLING WORLD IN MINIATURE Figure 1 shows the Virtual Environment setup where the user has donned a Virtual Research V8 head-mounted display in nonstereo mode and is being tracked by an InterSense IS-900 tracker. The expectation is that general results of this work are applicable to WIMs used in other 3D environments such as Augmented and Mixed Reality and Ubiquitous and Tangible applications. In the rest of the paper, the term standard WIM will refer to the original WIM technique without scaling and scrolling and WIM will refer to both SSWIM and WIM implementations The Design Process of SSWIM The design of the SSWIM went through three evaluations after being tuned by the original designers. The process is included here because the resulting design for this virtual world might not be as effective in another world with different tasks and goals because of small details of the world. In this case, the method here can be repeated to create a successful SSWIM technique in that new world. The first evaluation was an expert evaluation by two experts that uncovered issues specific to the interface. The experts had been working with 3D interfaces for several years and were familiar with the WIM technique. Usability issues were resolved based upon the expert and implementer's discussions and fed into a pilot study of four participants. The comments and feedback, in addition to making it more usable, led to a better understanding of how to evaluate the interface. After resolving the issues from the pilot study, a second pilot study with four participants incrementally improved the design and led to an additional understanding of user's mental models of the technique and its usage. A description of issues that arose during this process follows World Aligned WIM A major design decision was to attach the WIM to the user's non-dominant hand and have it constantly world aligned. This was out of a concern of the SSWIM interrupting the user's world knowledge by their cognitive efforts focusing on the SSWIM functionality. By keeping the WIM aligned, it could help orient the user as they moved about the environment so as to better transform landmark and route knowledge into world knowledge. This alignment had the unfortunate effect of reducing the motion parallax of objects inside the WIM because hand rotations did not rotate the WIM object. Since motion parallax in a non-stereo world is a strong depth indicator, there were increased difficulties distinguishing between the world and the WIM proxy objects, as well as increased positioning errors in depth. The expert evaluation led us to place transparent walls around the WIM to cut the proxy objects from the world objects as per the Silk Cursor [8]. The position of the user in the WIM was also augmented with a line dropped from the user position to the WIM to show height (as per Figure 3). A circle placed on the WIM shows where the dropped line and WIM collide to better emphasize the positioning along the XZ plane. After doing this, later evaluation found the Figure 3. The world alignment of the WIM led to many design issues. 2 12

3 transparent walls around the WIM were not useful but reduced the view of the scene. So a wireframe outline was placed around the WIM and the transparent walls were removed Scaling The controls for scaling in the SSWIM changed through the study. At first, hand flex of the non-dominant hand as measured by 5DT Gloves was used to scale the world. When the hand was flat, the world scaled up at a constant rate and when the hand was clenched it scaled down at a constant rate. Expert evaluations increased the neutral space where scaling did not occur but it was still too difficult to hold the non-dominant hand in a neutral position and the gloves were also encumbering. The scaling was then mapped to two buttons on the wand. This did not allow users to change scale with one hand while positioning or flying since multiple buttons were required. The hope was that users would learn to do both scaling and positioning simultaneously. So, another method was required which did not use as many buttons. In the final pilot study, scale was mapped to the mouse wheel of a wireless mouse held in the non-dominant hand. The wheel s motion had a natural mapping to scale and was found to be effective. Additionally, the global scale was bounded to settings reasonable to all the worlds with the initial starting scale being midway between the maximum and minimum values Scrolling Scrolling implementation changed little during the iterations. There was a large dead zone in the center of the SSWIM where no scrolling occurred, outlined by a white line after the first pilot evaluation. When the user moved to a position outside of the dead zone while positioning, the SSWIM scrolled the world in that direction (see Figure 2). The rate of scrolling was at first linear but was changed to a cubic function of the distance out of the box. This allowed for a smooth transition from slow to fast scrolling and small accidental movements outside the box led to such small scrollings that users did not realize they had scrolled. Additionally, the scrolling was at rates where users could easily position themselves while scrolling was occurring. Cubic was used instead of quadratic because it maintained the sign. An arrow was also added after the first pilot evaluation that extended from the center of the SSWIM to the positioning location to show the direction of scrolling, letting the users know when scrolling was occurring Flying and Positioning Users were able to use the WIM technique to position themselves at a given location in the environment by releasing their red cube representation where they wanted to be. The technique then placed them at this location (see Updating the User s Position below). It was determined that flying for small and precise updates needed to be added to the WIM. This created the problem of determining if the user intended to fly or position themselves. The first implementation used two different buttons on the wand which was difficult for users to keep the functionality differentiated. More importantly though, it did not become easier with experience. To reduce the complexity, one button was used in the final implementation. Flying occurred when the wand was outside the WIM and positioning occurred when the wand was inside the WIM. Pilot studies reduced the positioning space to the size of the non-scrolling center of the SSWIM and validated that users adjusted to the dual usage of the button after a few trials Updating the User s Position The method of updating the user while they were positioning themselves in the WIM changed during the evaluations. At first, immediate update was used to directly attach world and WIM positions. This led to orientation issues as well as increased user discomfort as hand and tracker jitter were scaled by the WIM. This caused sharp and jumpy movements. The real world was largely ignored by the user during this time as much as possible for good reason. Instead of trying to smooth the positioning to solve the jumpiness, we switched to post-mortem update so that the user was only animated into the position in the WIM after they released the button as was done in [6]. The method of animating the user into position was at first set to a constant time period but large distances created choppy animations due to reduced frame rates and the travel was too rapid to easily comprehend. The animating method was changed to a constant speed and then modified to include ramping up and slowing down at the beginning and end of travel. This created a much smoother transition, leading to less user discomfort and a better understanding of their position. To further reduce orientation problems, we tried in the last pilot study to change the points the animation occurs from. So, instead of animating users from their current position to the target position, we animated from their viewpoint into the WIM to the target position, similar to [6]. However, this was not found to be effective, probably due to the earlier design decision to align the WIM with the hand. By doing this, we gave the metaphor to the user of positioning on a map and not transitioning into the WIM as in [6]. The final design reverted to animating from user s current position to their target position. The switch away from immediate update also reduced the problems distinguishing between world objects and WIM objects, which helped in the previously mentioned decision to remove the transparent walls of the WIM Ring and City Worlds and Users Task As important to the iterative design process as the technique itself is the world used in the evaluation. So it evolved as our understanding of the domain improved. At first, a world of rings was used to evaluate the SSWIM, which users navigated through. The ring path looped around a virtual world, rising and falling, even looping around itself. During the first pilot evaluation, we noticed that the ring world was the wrong world in which to evaluate the SSWIM since WIM techniques have more in common with teleportation and travel across vast distances, and are not suited for precise incremental travel. Using the SSWIM technique to precisely position was not evaluating it on its typical usage. We replaced the ring world with three city worlds (see Figure 4) sized small, medium and large with 8, 13 and 24 buildings respectively. The task of the user became to find the blue target sphere (10 meters in diameter) in the city, travel to its center and read back to the researchers the label positioned there (see Figure 5). Ten trials per city were performed where the researchers brought up a target sphere one at a time. The spheres were placed at different heights such as ground level, building tops or floated in the air. A second travel technique of flying was used outside of the WIM for the smaller and more precise movements. Flying was directed by the orientation of wand and was paced to approximate the speed of jogging which was purposefully slow to motivate accuracy in the positioning task. For the standard WIM technique, the scale was the same across all conditions. It was set such that the large city world would fit on the map all at once. For the small city world, there was excess space on the WIM. 3. COMPARISON STUDY BETWEEN WIMS A study comparing the performance of the SSWIM and standard WIM techniques in the city worlds was conducted to determine the impact the addition of scaling and scrolling 3 13

4 Figure 4 The city world was used in the evaluation shown from street level on the left and from up top on the right. Users had to perform search and navigation tasks with the WIM techniques. functionality had on the SSWIM. The participants were split equally into SSWIM and WIM users controlling for equal Cube Comparison and Maze Tracing test scores (discussed below). Each user performed the 10 tasks in a randomized order in each of the three cities in a counterbalanced order; a between subject three by two factorial design. Thirteen male participants were recruited from computer science classes (some for class credit), and university graduate Listservs. The average age was 28. One individual was replaced because he never used the functionality of the SSWIM to scale so his behavior and strategies for search and travel were completely different than any other participant or pilot user. The experimenters first collected user demographic information about computer usage and game playing. Two standard psychology tests were administered to account for spatial orientation (Cube Comparison Test) and spatial scanning (Maze Tracing Test) [4]. The participants had their technique explained to them and became familiar with it in a city of two buildings with ten trials during a training session. The experiment was then run for all three cities in a random order. Additionally, because the SSWIM reduces the amount of the city shown at any given time, users were asked to draw maps of the medium sized city immediately after completing that city. Lastly, a post questionnaire collected user comments and ratings using seven point scales. The software used the Simple Virtual Environment (SVE) library [9] running on a dual processor 2.0 GHz Apple G5 tracking the user with an InterSense IS-900 tracker and a Logitech wireless mouse. Frame rates were generally above 30 fps but dipped occasionally to 10fps momentarily Quantitative Results User subjective ratings of the technique Despite the extra complexity of the SSWIM technique and testing on an optimal environment for WIM usage, users of both techniques rated equally their preference of the technique and its ease of use. There was no significant difference between the low ratings of user difficulty with the task in the cities for the standard WIM and SSWIM techniques (mean WIM =2.17, mean SSWIM =2.50, p=.747). The only significant difference reported in a city was a weakly significant difference in the small city (mean WIM =1.17, mean SSWIM =2.33, p=.073). Both techniques were rated highly by the users and had no significant difference between them (mean WIM =6.17, mean SSWIM =6.33, p=.756). Users commented, [SSWIM] was pretty intuitive and worked well as well as I liked having the control to put myself anywhere on the map and also at any height. Average Trial Time per City Time (s) WIM SSWIM 5 0 All Small Medium Large Figure 5. Trials consisted of finding a blue sphere in the world and traveling to it and reading a label in the middle of the sphere. Figure 6. There was no significant difference between city size and trial time. 4 14

5 Time to complete a trail Performance measures had equally positive results (Figure 6). In time to complete a trial per technique, there was no overall significant difference in the cities (WIM mean=16.93s, std dev 7.56; SSWIM mean=17.93s, std dev=13.21; p=.376). This also held for the small city (WIM mean=13.94s, std dev 6.47; SSWIM mean=15.68s, std dev 13.63; p =.373), medium city (WIM mean=16.67s, std dev 6.85; SSWIM mean=18.23s, std dev 14.81; p =.461) and the large city (WIM mean=20.17s, std dev 8.06; SSWIM mean=19.89s, std dev 10.72; p =.868) Last positioning distance (LPD) There was a significant difference in mean last positioning distance (LPD). The LPD is the last positioning performed by a user in a trial and it s distance from the center of the target sphere. SSWIM had a lower overall error (mean WIM =10.87m, mean SSWIM =8.39m; p=.029) with WIM errors strongly significantly larger in the Y or height dimension (WIM mean=8.40m, std dev ; SSWIM height mean error=4.60m, std dev=5.43; p=.001) and SSWIM errors strongly significantly larger in the XZ plane (WIM XZ mean error=4.88m, std dev 4.54; SSWIM XZ mean error=6.35m, std dev 5.40, p=.005). The difference in error could be attributed to SSWIM users using a larger scale factor (small city mean=.00247, medium city mean=.00262, large city mean=.00216) compared to WIM (constantly.00180). This would increase accuracy of height and the XZ plane but this would not account for the WIM's increased accuracy in the XZ plane. Since depth has been shown to be the major dimension of error in VEs, one possibility would be that users looked into a SSWIM and down on a WIM with observations of users supporting this conclusion Length of time after the last positioning (LTLP) For trials with the last positioning occurring outside of the target sphere (non- bull s-eye trials) the length of time after the last positioning (LTLP) to complete the trial correlates inversely with Cube Comparison scores (r 2 =.102, p =.001) and inversely with Maze Tracing scores (r 2 =.069, p =.007) but does not significantly correlate in WIM trials. This leads us to believe that something about the SSWIM technique allows users to better apply their innate abilities after the last positioning. Indeed, there is a significant difference in time after the last positioning of a trial between WIM and SSWIM techniques (respectably seconds, std dev ; seconds, std dev ; p=.047). It could be that multiple positioning were occurring with SSWIM which helped to increase user understanding when they were close to the sphere. So only trials of a single positioning were considered and the performance increase of SSWIM over WIM for LTLP only increased (WIM seconds, std dev ; SSWIM seconds, std dev ; p=.031). Two possible explanations are that the scrolling functionality of the SSWIM provides more motion parallax for world-aligned WIMs or that the ability to scale the world increased the ability of the users to place themselves more precisely (a small but significant correlation exists between the scale at the end of a trial and the distance of error of the last positioning of r 2 =.015, p =.019) Correlation between LPD and LTLP It was expected that reducing the LPD would be the optimal strategy to reduce the LTLP and thus the overall trial time. There is in fact a strongly significant good correlation between trial time and LPD (r 2 =.164, p<.001) as well as a very good strongly significant correlation between LTLP and LPD (r 2 =.536, p<.001). Multiple users however did not attempt to position themselves inside the target sphere for a bull s-eye. When we consider the LPD for bull s-eyes, we find that there is only a small negative correlation approaching significance between LTLP and LPD (r 2 =.018, p=.107) and the correlation of LTLP to LPD for nonbull s-eyes is strongly significant and strongly correlated (r 2 =.648, p<.001) Effect of spatial orientation capability If the problem of why positioning inside the sphere does not reduce trial time can be attributed to orientation issues, we would expect higher Cube Comparison scores to correlate with less LTLP which it does with only weak significance and a small negative correlation (r 2 =.019, p=.093). The effect of Cube Comparison scores outside of the sphere was more pronounced with a strongly significant negative correlation (r 2 =.088, p<.001). We observed many users were confused as to the direction of the center of the target sphere when inside the cube, due most likely to being unable to see the outside world. Because of this, orientation problems inside the cube might be just an artifact of the experiment and not the technique itself Maps The maps of the medium sized cities were for the most part unrepresentative of the cities. Users did not need to understand the city to perform their task since their main concentration was to find the target sphere. So, they did not need to develop world knowledge of the cities Follow-up Study on Experience We were interested in how users acquired expertise with the SSWIM technique due to such comments as, It worked pretty well, was a little difficult at first, but over time it got easier. and quite a bit of a learning curve. So, we performed a follow-up study using just the SSWIM technique with four sets of ten trials in the same medium sized city. Four male users were taken from undergraduate classes and received extra class credit for participating and had an average age of Performance Users reached performance plateaus while their proficiency with scaling and scrolling increased throughout (see Figure 7). The mean time of the first set of ten non-tutorial trials was seconds while the third set of ten was seconds. The extra trial we added was a mean seconds which could be attributed to users playing with the technique or losing motivation for the task. At the end, users were comfortable with scaling and scrolling, performing these operations easily. Based upon this, we would expect users to reach a level of expertise after about 20 uses of the technique which took about ten to fifteen minutes. 5 15

6 Time (s) Figure 7. In an extended study, users reached a level of expertise with the SSWIM technique in the third set of trials or, after 20 trials Qualitative Observations Average Time Per Set of Trials 1st 2nd 3rd 4th Despite the iterative design, there are still possibilities for improving the interface. There were problems with the button causing both flying and positioning but they tended to fall off with time as users learned to lower the WIM from their view when flying. Feedback to let users know when the button is in the positioning state as opposed to the flying state would most likely solve this issue. Users did not scale on a regular basis. They generally set the scale of the world as appropriate to the city and rarely adjusted the scale afterwards. The second experiment showed that with experience, users felt more comfortable scaling. A task that required the user to shift between scales or could better make use of different scales would better evaluate the scaling functionality of the SSWIM. Depth continues to be a problem with this experiment, even with a height line that drops from the user's WIM position. One problem was the WIM position cube, height line and circle on the WIM surface were the same red color, causing users looking down onto a WIM to see just a mass of similar color (lighting differentiated the color of the surfaces somewhat). The most common strategy overcoming depth in WIMs is to move the user's position cube along the ray from the eye to the target position until the target occludes the user's cube and then backup. Another strategy is to rotate around the WIM to get another viewpoint so the error due to depth now occurs along a dimension that has already been aligned. Scrolling seemed to be used successfully by users who didn't seem to mind it or mention it in comments. In larger cities, scrolling was used more prevalently so scales could be kept larger. Additionally, due to the non-linear scrolling function, many users were able to position themselves while scrolling and, when needed, scroll quickly across large distances. The simplicity of scrolling might have been a reason users kept the scale factor larger for SSWIM. One problem users experienced were microphones on the trackers being occluded by the user or the other tracker. The users also bump the accelerometers in the tracker causing jitter. To solve this, the WIM can be offset from the hand horizontally, keeping the tracker from being occluded and likely will not be noticed by the user [4]. Many users changed the size of the WIM by bringing the WIM closer to their face to get a better view. This in effect scales the visual size of the WIM. This intuitive motion can be augmented by scaling the WIM additionally as a function of the WIM's distance from the user's head. This will easily increase the scale of the WIM when needed by the user and also makes moving the WIM out of the way easier since the WIM is smaller when the user puts their hand down. 4. CONCLUSION WIM limitations due to the inability to work at multiple levels of scale were successfully overcome with the addition of scaling and scrolling in the SSWIM technique. This has created a more generally applicable WIM implementation. A thorough design process produced a robust interface and a valid testbed to evaluate the scaling and scrolling behavior in traditional WIM domains. Continuing work is to validate the SSWIM in domains not possible for WIMs such as environments with details at multiple levels of scale and very large worlds. Future investigations need to be conducted into non-world aligned WIMs and its impact on user mental models and post-positioning travel methods. A classification of these design decision implications would be invaluable to implementers. Additionally, user strategies with WIMs and multiple WIMs to address existing and new work tasks, such as visualization, needs to be explored with the new scaling and scrolling functionality. 5. REFERENCES 1. Stoakley, C. and Pausch, R. "Virtual Reality on a WIM: Interactive Worlds in Miniature," CHI (1995), Bell, B., Feiner, S. and Höllerer, T. View Management for Virtual and Augmented Reality, In Proc. UIST (2001), Bier, E., Stone, M., Pier, K., Buxton, W. and DeRose. T. Toolglass and magic lenses: the see-through interface. Proceedings of SIGGRAPH (1993), Burns, E., Razzaque, S., Whitton, M.C., McCallus, M.R., Panter, A.T., and Brooks, F.P. The Hand is Slower than the Eye: A quantitative exploration of visual dominance over proprioception. Proceedings of IEEE Virtual Reality (2005). 5. Ekstrom, R. French, J. & Harman, H. Manual for Kit of Factor Referenced Cognitive Tests. Educational Testing Service, Princeton, NJ (1976). 6. Pausch, R. F., Burnette, T., Brockway, D., and Weiblen, M. E., Navigation and locomotion in virtual worlds via flight into hand-held miniatures, SIGGRAPH (1995), LaViola, J., Feliz, D., Keefe, D. and Zeleznik, R. Hands-Free Multi- Scale Navigation in Virtual Environments. Symposium of Interactive 3D Graphics (2001), Zhai, S., Buxton, W. and Milgram, P. The partial-occlusion effect: Utilizing semitransparency in 3D human-computer interaction. ACM Transactions on Computer-Human Interaction (1996), 3(3), Kessler, D., Bowman, D. & Hodges, L. The simple virtual environment library: an extensible framework for building VE applications. Presence: Teleoperators and VEs (2000), 9,