Real-time Information Management System Final Report August 8, 2003

Transcription

1 Real-time Information Management System Final Report August 8, 2003 Heather Frantz Albert Lo Elizabeth Mauer Jessica Mignone Kathryn Rieger

2 Table of Contents Chapters Page 1. Executive Summary Design Challenge Identifying Current Problems Initial Designs Research Iterative Design: Navigation Module Iterative Design: Communication Module Feasibility Future Exploration Appendices A. B. C. D. E. F. G. H. I. J. K. L. M. Brainstorming Ideas Down-selection Matrix Concept Scenarios Surveys and Analyses Interviews and Analyses Contextual Inquiry and Analyses Iterative Design Analyses Prototype Implementation References Workflow Model RIMS Interaction Flow Sketches RIMS Initial Prototype Sketches RIMS Mobile Interaction Flow Sketches

3 1. Executive Summary In response to a challenge from General Motors, our student team from the Human-Computer Interaction Institute at Carnegie Mellon University s School of Computer Science designed a Real-time Information Management System (RIMS) for in-vehicle and pedestrian usage, with a production date of The system networks vehicles to multiple handheld users, transferring location, navigation, and communication data, while providing an appropriate level of global and local context via an interface that minimizes cognitive strain. Our team consisted of five masters students: two cognitive psychologists, one human factors engineer, one technical writer, and one electrical engineer and interaction designer. This diverse team has a broad variety of skill sets from across the Human-Computer Interaction discipline, including design, analysis, and implementation expertise. We used an iterative process to arrive at the final design. During this process, we applied a number of usability evaluation methods, such as affinity diagramming, contextual inquiry and design, think-aloud protocol, and formal user testing. The RIMS prototype consists of two modules: a navigation module and a communication module. The navigation module provides real-time, turnby-turn directions to a destination, routing a driver around traffic backups due to construction, accidents, or other congestion. The communications module permits messaging between drivers and handheld users networked to the vehicle. The following report details the design, our implementation methods, the testing procedures employed throughout the seven month development phase, and the system feasibility. 1

4 2. Design Challenge In January of 2003, General Motors presented us with the following design challenge: design, prototype, and test an onboard application for a vehicle to be produced in They also asked us to assume the following parameters: unlimited electrical power and a broadband wireless connection available in vehicle. The target audience was 30 year olds and older. The identified target audience made this project particularly appealing and relevant to us, since we will become part of the targeted audience in the next thirteen years. Prototype testing on the Carnegie Mellon campus made it easier to find subjects who fell within the desired age group. These users insights and potential acceptance of integrated technology in vehicles became extremely pertinent in the interface development of our application. Brainstorming As Human-Computer Interaction practitioners, we recognized the need to explore the relationship between the user and the driving task - specifically, information that vehicles can provide the user. We sought to reinvent existing and uncover new relationships between the vehicle and the driver, keeping in mind that our task was to ultimately implement a system to facilitate these relationships. Using GM s guidelines as a starting point, we conducted a traditional brainstorming session in which we individually listed potential design tangents on sticky notes for three minutes, and then read them aloud to the group, placing them on a whiteboard working surface. There was no evaluation or filtering of ideas during this phase of contextual design. We did not constrain ourselves to existing technology, resulting in a range of ideas from self-cleaning upholstery to flying cars. While seemingly unrealistic and impractical, these more outlandish suggestions helped us to develop more focused and refined concepts. The session resulted in a list of 100+ ideas, which are listed in appendix A. Affinity diagramming Organizing the ideas into a hierarchy revealed commonalities among ideas, and helped us to discover areas with potential design applications. We constructed the hierarchy bottom-up by organizing the notes along common themes until we defined categories. Once we had grouped the 2

5 ideas, we named the categories. The fifteen categories defined during our affinity diagramming were: 1. Navigation 2. Personal safety 3. Entertainment 4. Comfort 5. Car controls 6. Utility 7. Personal communication 8. Cleaning 9. Car status 10. Driver enhancement 11. Bond / classified 12. Scheduling 13. Car security 14. Customization 15. Systems communication Figure 1: Part of our affinity diagram 3

6 GM reviewed our list of categories and ideas and then used their own set of criteria to determine areas of interest for system development. These criteria were: 1. Rationality of the technology within the context of the world 2. Nature of the technology - emerging preferred This client evaluation resulted in the following thirteen specific development ideas: 1. Voice-controlled navigation system 2. In-depth engine status system that is contextually aware 3. Verbal / visual warnings when you drive recklessly 4. Information integration with PDAs and cell phones 5. Voice browsing 6. Biometrics 7. Car networks and v-commerce 8. Contextual awareness of cities 9. Speakerphone so drivers and passengers can talk hands-free 10. Sharing data like music and movies between cars 11. Contextually aware cell phone 12. Collecting data on elderly and teen drivers 13. Chemical air testing Matrix ranking We constructed a selection matrix to identify the concept for development and then rated ideas across the eight criteria listed below. The weight given to each of the criteria is also listed in percentage form. Both the client and our team assigned this weighting based on various assumptions and project objectives. The matrix is located in appendix B. 1. Revolutionary: how mature is the technology required? [17%] 2. Marketability: How high is perceived demand? [10%] 3. Perceived cost: How high is the cost for GM? [7%] 4. Feasibility: How feasible is it for us to implement? [16%] 5. Lifestyle: How much will it positively impact lifestyles? [15%] 6. Driving enhancement: How much will it affect driving? [15%] 7. Safety: How much will it positively affect driving safety? [7%] 8. Fun: how interested are we in pursuing this topic? [13%] 4

7 We developed scenarios for the resulting top five concepts and presented them to GM. These concepts are as follows (full descriptions of these scenarios are located in appendix C): 1. Context aware navigation 2. Context aware vehicle status system 3. Context aware vehicle commerice 4. Networked vehicles that can share movie and music files 5. Information integration between car and handheld devices In collaboration with the client, we narrowed these five concepts down to one topic: a context aware vehicle. In our system, we decided context awareness would consist of three subsystems: 1. Information exchange 2. Vehicle commerce 3. Dynamic navigation 5

8 3. Identifying Current Problems Surveys During the second phase of our design process we conducted surveys and interviews of drivers between the ages of 25 and 35 to discover driving habits, errand and entertainment planning behavior, and current use of technology off and onboard vehicles. The first survey (see appendix D) sought to identify basic driver information, such as age, marital status, family size, pets, vehicle type, and cell phone, PDA, or use. In addition to these basics, we also asked drivers how they used the Internet, if they shopped online, and what they thought might be the advantages and problems of integrating their current mobile and online technologies with their vehicles. The results of this survey are also included in appendix D. An open-ended question at the end of the survey asked drivers to think about additional functionality they would like their vehicles to have that their car does not currently offer. The answers we received to this question, although varied, interestingly had a common theme - drivers seemed to want their cars to be smarter. The responses ranged from automatic reminders to pick people up to personalized alerts about areas of interest in the car s current location, as well as integration with AAA and other trip planning resources to get real-time reservations and entertainment planning information. Drivers interest in real-time information and intelligent, adaptable vehicles prompted us to create a more detailed interview to probe into the areas of navigation, entertainment planning, shopping, and vehicle maintenance. Interviews We developed an and in-person interview that asked drivers to comment on their planning habits before and after entering their vehicles (see appendix E). Sections covered buying movie tickets, eating out at restaurants, car maintenance, traveling in groups, navigating when lost, re-routing through traffic and road construction, music, long trips, shopping, and emotional driving. We used these interviews to determine facets of driving that might be the most apt to benefit from implementation of real-time services. Complete results of this survey are listed in appendix E. From these interviews we learned that while some trip planning starts outside the vehicle, many drivers will spontaneously try to correct errors, change plans, or make new plans while driving. For example, when asked what they did when stuck in traffic, some drivers said that they would try 6

9 to take alternate routes that they knew about or simply try to take the next exit to get out of the traffic jam, without having an alternate route in mind. In terms of entertainment planning, many people made decisions on the fly, spontaneously choosing restaurants and movie theaters based availability or location. For example, many people said that if a restaurant was booked or movie tickets were sold out, they would drive around looking for new places. People in separate cars who were traveling as a group and trying to meet at a single destination would follow each other in traffic (risking separation) or call each other on their mobile phones in an attempt to coordinate a meeting place. After conducting these interviews, we decided that we had a significant amount of information surrounding navigational and entertainment planning behavior, and decided to design our contextual inquiries to extract details about these tasks. Contextual inquiry Contextual inquiry is an HCI method that involves observing people in their day-to-day activities; in our case, we observed people perform a specific driving task. Contextual inquiry aims to provide a more thorough understanding of driving practices across a number of subjects through observation, conversation with the subject, interpretation of important behaviors, and a guiding purpose (Beyer and Holtzblatt, 1998). We focused on discovering navigational driving behaviors and entertainment planning techniques across a broad range of drivers. We conducted contextual inquiries with eleven (11) subjects who ranged between the ages of 20 and 30, 6 female and 5 male. All subjects had valid driver licenses and owned their own vehicles. The subjects were paid $30 for their time and $5 for fuel. Drivers started outside of their vehicles at the HCI department on the Carnegie Mellon campus at 5000 Forbes Ave., Pittsburgh, PA. The first seven drivers were given the task shown in appendix F. We asked drivers to navigate to an unfamiliar location (Chapel of Blues restaurant) about six miles from campus, during rush hour traffic (4:30 p.m. to 6:30 p.m.), and to find out the wait time at the restaurant for a dinner party of two. Although not given the address, drivers were told they could use any method they wanted of finding the address and/or directions to the specified location. We also asked drivers to attempt to get to the 7

10 destination within 20 minutes while still obeying traffic laws. The time limit encouraged drivers to try to navigate around traffic and road construction. The next four drivers were given the same task, except these drivers were told they were trying to meet a friend at the restaurant at the same time. We sent two drivers out at the same time, one driver navigating from campus and one driver navigating from a familiar starting location (the driver s home). The drivers were provided with each others mobile phone numbers. The solution was open-ended, that is, drivers could have used the mobile phones or could have coordinated following each other to the restaurant. We allowed the drivers to decide. The second task assignment is also available in appendix F. During the contextual inquiries, we focused on observing drivers apply their typical driving practices and method of navigation. We observed drivers before getting into their cars (using MapQuest, calling friends, instant messaging) and while they were driving. At the start of the observation, we allowed drivers an out of vehicle preparation phase. During this phase, drivers could use any means necessary to plan their route to the target destination, and we asked questions about their methods and the people they contacted, if any. Once drivers were navigating in their vehicles, we asked them questions related to their decisions (why did you decide to turn right out of the parking lot instead of left?) as well as questions unrelated to their current task (do you ever listen to traffic reports on the radio?). We aimed to discover the most common methods people used of finding an unknown destination and rerouting themselves around traffic and road construction detours. We also asked people about their techniques for finding restaurants and coordinating with friends in other vehicles. We discovered several common driving themes during contextual inquiry phase. All drivers did some type of pre-planning before getting into their vehicles. Pre-planning consisted of using the web (MapQuest), calling friends on mobile phones, or using an instant messaging program to ask friends for directions. Some drivers printed out MapQuest directions while others copied down directions by hand onto a piece of paper. When asked why individuals copied directions instead of printing, some remarked that the map was not helpful and they only needed directions from the point where they became unfamiliar with the route. In their vehicles, drivers varied in their approach to traffic, construction, and detours. Drivers either chose to stay in traffic on a known route, avoid traffic by taking an alternate known route, or used physical maps to try 8

11 and find a new alternate route. When drivers got lost, which they frequently did due to road construction detours and poor or missing signage, some relied on their own sense of direction and knowledge of the surrounding streets to find their way back, while others used maps and some even stopped to ask for directions. Some problems the drivers encountered were caused by their sources of information. The street directions given by internet sources and friends were often unreliable. Drivers became frustrated quickly and had little patience to understand maps and directions that did not seem to match the actual streets. Drivers attention was often divided between the driving task and the reading of maps and street signs. Most drivers did not even attempt to find out if the restaurant had a wait time; they stated that they would simply drive to the location and find out when they got there. See appendix F for statistical contextual inquiry results. Workflow modeling After reviewing the contextual inquiry results, we developed a workflow model to visualize the drivers plans, tasks, and actions and the artifacts and breakdowns involved in navigating. Workflow models are designed to describe tasks from the point of view of the people interviewed, and help to show work strategy and informal structures contained in the work process (Beyer and Holtzblatt, 1998). The workflow model in appendix J shows the driver at the center of the flow and the driver s tasks, along with other individuals who interact with the driver in circles around the flow. Artifacts the driver interacts with are shown in rectangles. Flow, or the communication between people (which may include the passing of artifacts) is shown with black arrows. Breakdowns are shown with red arrows and lightning bolts. From the flow model, we identified six major breakdowns that could potentially benefit from enhancement of the driving experience: 1. Drivers became stuck in traffic. 2. Poor and transient signage misled drivers. 3. Information received from the Internet is not real-time. 4. Sources of information were inaccurate or unavailable. 5. Drivers had a lack of interest to plan or call ahead. 6. Physical location of information was unsafe or not useful. From the contextual inquiries and workflow modeling, we discovered several key issues that required attention. Firstly, drivers previous experience impacted what might be considered necessary and desirable 9

12 information. For example, some drivers already knew alternate routes around traffic, while other drivers chose to stay on familiar routes. One driver might need alternate route information while another driver might not. Secondly, risk management appeared to be an important aspect of driver satisfaction. Drivers indicated that they were constantly weighing the potential risks of taking alternate or unfamiliar routes. I m going to stay on this road because if I go another way I ll probably get lost. Also, I won t usually go this way during rush hour because I know there will be traffic. Decreasing driver frustration is also an important issue. Frustrated drivers, even after reaching their destination, were more relieved than satisfied with their driving experience. During the contextual inquiries where we required separate drivers to meet at a specific destination, we learned that current communication methods between drivers are poor and any kind of improvement is desirable. We used these concepts to develop our main goals for a new in-car navigation/communication system. Our goals were to: 1. Increase user confidence. 2. Decrease user frustration. 3. Improve user efficiency. Targeted design areas The goals we developed prompted us to target specific design areas that would not only address breakdowns in the driving flow, but would also incorporate some of the important concepts above that make driving a more pleasurable and efficient experience. We decided to target four design areas: 1. Directions to match user knowledge. 2. Real-time directions based on current conditions. 3. Contextual awareness of commerce information 4. Status indication and information sharing between vehicles. 10

13 4. Initial Designs We began our initial system designs by sketching interaction flow diagrams of how we envisioned users accessing and utilizing the various modules in our system. After reviewing these flow diagrams, we narrowed down our scope to create the flow for the first navigation module prototype. Interaction flow sketches The interaction flow sketches shown in appendix K illustrate the possible user interactions with various components of RIMS. The modules include navigation, communication, shopping, entertainment, and vehicle status. First prototype sketches For Prototype I, we sketched out user flows for the initial navigation module before developing the Flash think aloud prototypes. These flow sketches illustrate how drivers could use the system to guide them to familiar and new destinations, and how the system would intelligently save this information to make more educated decision in the future about driver preferences. To see these sketches, please refer to appendix L. 11

14 5. Research Map studies In order to examine the current ways in which drivers navigate, we tested ten individuals on short tasks and used those results to make recommendation for the design of the system. The subjects consisted of five males and five females, with half being familiar with the Pittsburgh and Carnegie Mellon area and the other half unfamiliar. The test consisted of two tasks, one concentrating on finding map orientation and the other focused on determining the amount of context need in a map to navigate. However, we did discover other interesting findings in addition to these two objectives. For the first task, subjects were given a map of Carnegie Mellon s campus and asked to provide driving directions from the sorority houses on one side of campus to the Physical Plant on the opposite side. We observed the orientation that the subjects held their maps when it was first given to them. We found that subjects who are familiar with the campus rotated the map so that its layout fit with their mental map of the campus. Subjects who were not familiar with the campus mainly did not rotate the map. This mostly likely is because they had no previous mental map of the campus. Based upon the subjects actions, we recommended that whenever a map is shown, it should be oriented with respect to the car and not to cardinal directions. The next task required subjects to give driving directions to the experimenters using only a map from MapQuest. The subjects started at an address roughly 60 miles from Pittsburgh (219 S Edgewood Ave, Somerset, PA 15501) and had to give the directions to a location in the middle of Pittsburgh (2715 Murray Ave, Pittsburgh, PA 15217). These two locations were picked because the directions would involve traveling on a combination of local roads and highways. Also, while the initial route was provided for the subjects, in the middle of the task, they were told that there was a major accident at New Stanton along the PA Turnpike and so would have to find an alternate route. This allowed us to see how much context a subject would need when doing spontaneous navigation. Subjects could only use the map to give the directions, but could zoom in and out as much as they liked. We recorded the directions that they gave and also how much and to what extent they zoomed in. From this task, we found many interesting implications. First, subjects would zoom in closer when they were traveling shorter distances, such as on the local roads, and zoom out when they were traveling longer distances, such as highway driving. This implies that for the amount of 12

15 context needed in our maps, shorter periods of travel require more context and longer periods require less context. Similarly, whenever subjects had to navigate around the traffic accident, they zoomed in more. Thus, for spontaneous navigation, more detail about the surrounding area would be needed in a map. An interesting finding that we were not looking for was how the subjects used the color scheme of the map given to them. Whenever subjects had to navigate around the accident, they would often look for other highways not by the number icon on the map, but by the color of the lines. For instance, if they were traveling on a road that was represented as a red line, they would look for another red line when finding a new route since they had learned red represented a highway. Based upon this finding, we recommended that the map used in the system should employ the same idea of color schemes. The final recommendations were that when the map is shown, the car s current position should also be shown with the appropriate amount of context so that the drivers are able to easily create a mental map. While all of these recommendations are geared to being used in a map, there is an underlying theme that it would be possible for there not to be a map at all times since the only time that a map would be needed is when traveling smaller distances or when the driver has to turn off a planned route. Navigation literature review In order to design a better initial prototype, we completed a literature of current studies that have dealt with navigation systems. Based upon these articles, we were able to make several recommendations. The first recommendation is that the system should have a multi-modal display that will use both the visual and auditory channels. The reasons for this are that using both channels will reduce the cognitive load of the user as prescribed by multiple resource theory (Wickens et al. 1983) and a multi-modal display yields better performance results for both younger and older drivers, although the younger drivers perform better than the older ones (Liu 2001). Also, the multi-modal display receives lower stress ratings, lower workload ratings, and higher preference ratings for both younger and older drivers (Liu 2001) and an auditory channel is better suited to delivering guidance information; turn information, and 13

16 emergency information because user response time is faster than displaying this information through a visual channel (Liu 2001). A final reason is information displayed in an auditory channel should have redundant elements in the visual channel because auditory short-term memory is shorter than visual short-term memory. The next recommendation to come from the literature review is that the system should display minimal information through either channel. One finding suggested that by presenting minimal information, the cognitive load would be minimized (Jackson 1998). Also, there are findings that less detailed instructions linked with landmarks or turn-by-turn instructions lead to better user performance in the safety domain (Jackson 1998, Eby & Kostyniuk 1999). Also, detailed driving systems are especially detrimental to older drivers (Jackson 1998). We also recommended that the system should display the distance to a maneuver (either via text or a status bar) before the actual maneuver instructions. One finding to support this is that when displaying directional information, the distance to a maneuver should be displayed before the actual maneuver so users can determine the urgency of the information (Jackson 1998). The final recommendation from the literature review is that the system should gain user trust. This can be accomplished by beginning any guidance or trip instructions before the user starts driving, accepting input regardless of car status (e.g. on or off) and not reorienting the car position on a visual display during turns so that it is more true to life. Other ways to do this are providing guidance instructions in a timely manner, especially when turns and lane changes are involved, highlighting the route in an overview with the start, and having the destination clearly marked so the user is confident the vehicle knows where it is and where it is going (Ross & Burnett 2001). Fisheye literature review A cursory examination of four articles on fisheye displays reveals that fisheye displays allow users to complete navigation and steering tasks faster than other, non-distorted displays. Furnas (1986) stated that fisheye displays are useful for displaying large information structures such as programs, databases, and online texts. This implies that a fisheye display could also benefit navigation tasks, since a geographical area is also a complex information structure. 14

17 Fisheye displays also strike a good balance between displaying global and local detail (Furnas 1986, Sarkar & Brown 1992), making them preferable over displays that show only global or only local context. Results from our map studies and interface testing (see section 6) demonstrate the need for the availability of both global and local context on the same display. Using a fisheye display, users were able to complete tasks faster than with a non-distorted display (Schaffer et al. 1996, Gutwin & Skopik 2003) and there were no significant differences in task accuracy versus a non-distorted display (Schaffer et al. 1996, Gutwin & Skopik 2003). Subjective data from Schaffer et al. (1996) showed that users were able to focus directly on the task and were not as distracted by the need to visualize the display mentally with the fisheye display. This finding is especially significant for our design: since navigation is a secondary task in the vehicle, our system must use minimal cognitive resources to allow the user to drive safely. Dimension of View and HUD literature review When testing and developing a navigation system, Steinfeld and Green found that users preferred the use of either aerial or plan views to display navigation information as opposed to using information given in perspective or 3-D. Based on this finding, we recommended that whenever a view of landmark was presented to users, it should be shown 2-D. These experimenters also tested presenting information on HUDs. They found that HUDs have significantly lower reaction times than Instrument panels. Thus, we decided to use HUDs to convey our information. Researchers also found that HUDs allow drivers to keep their eyes on the road more than instrument panel presentations. Also, HUDs lead to better and faster detection of outside objects and events and drivers need less time to obtain information from a display that is head-up more than one from head-down (Tufano 1997). Also supporting this decision is a problem that arises when using HUDs. Users often become confused about the difference between the perspective of the world around them and the perspective represented on the HUD. Thus, we recommended that the views presented to users once again not use perspective to avoid this confusion. 15

18 Another problem with HUDs is that often times, users have a lot of their attention taken up by HUDs. Thus, we recommended that when using this kind of display, only minimal information should be presented. Touchscreen literature review We completed a literature review to determine how users should interact with the interface. When researching touch screens, we found that minimum instruction is needed for use (McGuire et al. 2000) and also touch screens are faster than using button or arrow keys (Wright et al. 2000, Yarnold et al. 1996). 16

19 6. Iterative Design: Navigation Module Iteration 1: Circle Our initial design idea consisted of circle with hash marks and the hollow circle would fill in as the driver progressed along his route. The hash marks represented turns and separate arrows would appear in the middle of the HUD when a driver was approaching a turn. When an accident occurred, a red dot along with the word Accident would appear at the accident s location on the route (see Figures 2A-C). Figure 2A: Circle design at start of route Figure 2B: Circle design with an accident 17

20 Figure 2C: Circle design at end of route Testing method The Think Aloud protocol was conducted with the users to determine how understandable the design was. This procedure consisted of taking users through a scenario of the system by showing them different views from different parts of a route. The users were not able to interact with the prototype. We interviewed ten subjects over the course of three different days about this design. The interviews lasted approximately a half an hour each and the subjects were paid $5 for their time. One member of the group worked as the interviewer and asked the users the questions from the script and any other she may have thought were relevant. Another member acted as the computer and operated the prototype. Finally, a member took notes while the remaining members observed and helped ask questions. Results From the Think Alouds on the Circle Design, we received a lot of interesting information and became aware of problems that users had with the design. The first result was all of the users except for one initially thought that the circle represented something other than their progress, such as speed or even the angle to turn the steering wheel. However, by the end of the session, all of the users realized that the circle actually represented the route. To counteract this problem, we added more design elements to the circle, such as a mileage indicator. Another problem that was brought to our attention by users and the professors is that it would be hard to display long or complex trips on the small circle because there could be a lot of hash marks that might overlap. In other words, while the circle showed the global context of the 18

21 entire route, the local context could easily get lost if the route was not simple. We redesigned the prototype so that is consisted of two circles with the route being spread over both. However, this confused the users in the second session of Think Alouds even more, so the redesign was dropped. Finally, another problem that arose with the design was the use of the right and left arrows to represent turns. While users had no problem understanding what the arrows were conveying, they did point out to us that turns were not always a straight forward left or right. Because of this, we developed a more complex set of arrows than just simple left or right that helped indicate the turn direction better. Iteration 2: Line For our next design, we moved away from the circle idea in hopes of something that would show more local context for the users. The design we created consisted of a long bar on the left of the windshield. The bar would move down as the driver progressed along the route so that only approximately two miles of a route would be shown at one time. On this bar were arrows that indicated which way the user should turn in addition to the arrows that appeared in the middle of the screen, much like those of the circle design. The user s car was represented by a small green triangle that would always be at the bottom of the bar and as arrows approached it that would signal to the driver that a turn was approaching. When the user approached his or her destination, the line would end with a small green rectangle representing the destination. When an accident occurred, a blinking red circle would appear on the on the bar at the appropriate location (see Figures 3A-C). 19

22 Figure 3A: Line design at start of route Figure 3B: Line design with accident Figure 3C: Line design at end of route Testing method For the line design, we once again conducted Think Aloud protocols, this time on four users. The same setup from the circle design was used with the participants being asked questions from a similar script with all of the questions about the circle being adapted to about the line. Once again, the sessions were half an hour long and the participants were paid for their feedback. Results Once again, the Think Alouds provided us with a lot of good feedback about the design. One of the most interesting findings was that unlike the circle design, all of the users realized right away what the line represented. In addition to that, all of the users had no problem whatsoever realizing what everything in the bar, such as the small triangle and the arrows, also represented. Another positive finding was that the line provided users with a preview of upcoming turns so that they would be able to prepare themselves for navigations in the near future. This also was a double-edged sword 20

23 though because this information was at times too much for the users and thus went against our goal of only providing minimal information. Finally, three of the users commented on the location of the bar on the left side of the windshield and indicated they liked the location. However, there were also problems with this design. First, 3 users complained they became distracted by the bar since the entire bar constantly moved. Second, while the bar showed detailed local context, it did not show any global context, something that our research indicated helped users navigate better. All of the users mentioned a lack of overview of route when asked how far they were to their destination and often didn t notice the mileage counter. In the line design, users had no concept of the scale of the bar so that even though a mileage indicator was given, they had no idea how long the route was or how long till they reached their destination. Iteration 3: Comparative interface test After designing two ideas and gathering feedback about them, we found that while both ideas had their strong points, they also both had more weaknesses than we would have liked. Because of this we designed a third idea we felt had the strengths of both earlier designs and not all of the weaknesses. This idea centered on the use of a fisheye. The fisheye was a small oval that would travel over a line, magnifying the details inside of the oval. The fisheye would travel up the line as the driver progressed along the route and a small green triangle in the bottom of the fisheye represented the driver s car. A small green rectangle at the top of the bar represented the end of the route. All of the turns were represented once again by hash marks on the line, but would gradually grow bigger and start to flash as they entered the fisheye. Like the previous two designs, there were also bigger arrows in the center of the screen to direct users and an arrow library of different directional turns was used. Finally, an accident was represented on the line by a blinking red dot and the word Accident much like the previous two designs (see Figures 4A- C). 21

24 Figure 4A: Fisheye design at start of route Figure 4B: Fisheye design with accident Figure 4C: Fisheye design at end of route The fisheye design allowed users to have both global and local context, something that both of the previous designs lacked. Since the entire route was shown at all times, users were able to glance at the display and have a rough estimate of how much of their route was remaining. However, the fisheye aspects allowed for local context since it zoomed in on information that was relevant at that time. We accomplished both these views using minimal information, our overall goal in developing this system. However, while we believed that this third design would be the most intuitive and reach our goals the best, we wanted to test it on users to find out. Thus, we decided to do a comparative study between all three designs and a fourth control design, a map. For the map, we took screen shots of a map provided by MapQuest and marked the location of the user s car on a set of five maps (see Figure 5). By comparing all four designs, we would be able to finalize the overall design of the navigation system. 22

25 Figure 5: MapQuest map given to users for map condition Testing method In previous user testing, the users always sat and observed the interface and did not interact with it. For this comparative study, we changed the setup so that it was more interactive and also measured more variables, such as accuracy and attention in order to find out not only how much of the interfaces the users understood, but also obtain how cognitively demanding each of the interfaces were. We did this by having users play a driving game while interpreting the low-fi static images that were shown to them (see Figure 6). Sessions were approximately 30 minutes long and users were compensated. Figure 6: Lo-fi static interface taped onto laptop 23

26 We showed 7 users were shown all four interface designs with the presentation order varied so there would be no order effect. First, a brief explanation of each interface was given since the static images didn t provide much feedback. The users were then allowed to practice driving through the game on a laptop until they felt comfortable. They were subsequently shown each interface one at a time by having a series of static images from each design placed next to the driving game on the laptop. While users were driving and being shown the interfaces, a series of questions about each interface were asked and accuracy of responses was measured. Also, members of the group watched the user to measure variables such as eye glances and swerving or stopping while driving and / or being asked a question. This procedure was used for each of the interfaces and afterwards, we asked the users which interface they preferred the most and least. Results To measure the available accuracy, each subject was rated on a 3-point scale (1-low, 3-high) across three categories: glancing, swerving, and stopping the driving task. Two observers who took notes on the users driving determined the ratings. (see appendix G for more results). Mean Percentage of Attention Free Circle Line Fisheye Map Amount of Context Figure 7: Mean percentage attention free by interface 24

27 Mean Percentage of Accurate Answers Circle Line Fisheye Map Amount of Context Figure 8: Mean percentage accuracy by interface Thus, we found that the design with the most available attention was the circle design (see Figure 7). We also rated users on the accuracy with which they answered question and were given a score of one for correct answer and zero for incorrect answers (see appendix G for more results). From the results of these two variables, we concluded that the fisheye design would be the best design idea for the navigation system since it not only had the best overall results of out of all of the interfaces, but also had a lot of literature supporting its use. There are problems with the fisheye design though. First, some users thought since the bar is straight, their entire path was straight. However, once users became familiar with the system, they will most likely not have this problem. Another problem is that there is no fixed position for the user to glance at when navigating since the fisheye moves up the bar. However, the fisheye moving up the bar is less cognitively demanding than the entire bar moving. Next, many of the users did not know during the testing where they were along their path or have any idea of how far they had traveled. By 25

28 providing some kind of feedback about progress made along the route, much like in the circle design, this problem was solved. Finally, when asked not all of the users understood the fisheye metaphor being presented. By redesigning the shape of the fisheye, this issue was resolved. Iteration 4: Fisheye For the next iteration of the fisheye design, a few features were changed based upon information from the comparative user testing. First, the shape of the fisheye was modified so that it would be clearer to users what was taking place in the fisheye. Next, the small individual arrows were removed and replaced by simple hash marks, similar to those from the circle design. This was done because the small arrows were not important to the users and only taking up extra attention. Next, we dropped the use of the larger arrows in the center of the screen because it was redundant information and so not minimizing the cognitive load. However, to make up for this subtraction of information, we changed the arrows on the destination bar to be multimodal. Before the arrows grew and flashed as they entered the fisheye. With this new redesign, the arrows now also emitted an alert noise. Figure 9: Next iteration of fisheye design 26

29 Testing method After deciding upon a final design idea, we started to test the fisheye to refine different aspects of the design and also make sure that users were more confident using it than a map. We mapped out a route in the driving game and implemented a prototype of the fisheye that would navigate users through this route. We tested six users, 3 females and 3 males with ages ranging from 19 to 23 years old. The test consisted of users driving the simulator game on the laptop while at the same time navigating with two different conditions, our navigation system on another laptop or a paper map with written driving instructions, much like those given on MapQuest. The test was a within-subjects design and the users were paid for their participation. Users were first given a few minutes to become familiar with driving the game. After they stated they felt comfortable driving, we presented them with one of the conditions and instructed them to start driving. For the navigation system, only minimal information was explained to the users such as where their car was represented on the system. For the map, users were only told which direction their car was oriented in relation to the map. During both conditions, users were asked to rate their confidence on a 5- point scale with 1 representing very unconfident and 5 representing extremely confident. If users made a wrong turn, we would first ask how confident they felt and then tell them they were off route before helping to return the users to the original route. Results After compiling the results for the confidence ratings of the users, we found that there was a difference between confidence levels for the map and the navigation system (as seen in Figure 10). More detailed findings can be found in appendix G. 27

30 Average Confidence Ratings RIMS Map Figure 10: Average confidence ratings for RIMS and map The total average confidence rating of the users for RIMS was 3.35 and the confidence rating for the map was 2.86, a difference of However, all users except one were more confident on average for the navigation system than the map. We performed a one-way ANOVA on the averages and received a p- value of 0.08, meaning that the difference between the two conditions was not significant. However, we suspect that if the sample size were to be bigger, this result could become significant. The final area of interest we found while performing these tests was a continued pattern of confidence for each of the conditions. Often times, users would be more confident starting with the map than the navigation system. Yet over time, their confidence in the map would reduce while their confidence in the navigation system would increase. This pattern suggests that while users may at first not trust our system, they eventually start to gain confidence as time goes on. In addition to the confidence ratings, we also refined the design by asking users questions about the system and found that there were different aspects that needed to be changed. Before, users often turned too soon or too late because they were not sure which intersection the arrows on the destination bar were referring to. However, with the multimodal alerts implemented, users turned at the 28

31 correct intersections more often. However, there were still some users who turned at the incorrect intersections or turned onto the wrong roads. While all of the users eventually realized that the fisheye provided a zoomed-in view of the route, they still did not realize this at first. Thus, the shape of the fisheye needed to be revised once again. Next, we found that users did not always know their final destination or the amount of their trip completed so far. While this information appeared at the bottom of the destination bar, many users did not notice it. Iteration 5: Fisheye After the previous set of user tests, several elements of the interface were changed to reflect the feedback we had received. The first major change was to implement a set of intersection views. These views were simple 2-D maps of the intersections where users had to turn. We decided on both an aerial and planar view based on previous literature research. They indicated which direction the user came from along with which road the user should turn onto in the intersection. These views appear right when the users approached the designated intersection (see Figure 11). Next, we changed the location of the destination and mileage information. This information moved to the top of the screen since it was often ignored when at the bottom of the screen. Also, the color of the destination circle changed so that it would stand out more to users. Finally, in order to provide more feedback to users, we changed the appearance of the destination bar that included the part of the route they had already completed. As the fisheye moved up the destination toolbar, the part of the bar behind it turned blue to indicate to the user how much of the path had already been traveled. 29

32 Figure 11: Fisheye design with intersection preview and multi-directional arrows displayed Testing method For this iteration, we did some comparative initial testing to see how our navigation system compared to most users current method of navigation, a map such as one from MapQuest. For this test there were 6 users (3 males and 3 females) with ages ranging from 19 to 23 years old. There were two conditions and the test was a within-subject design. Users were compensated for about a half hour of their time. The two conditions were driving through the game using the navigation system or a paper map that showed the surrounding area in the game and also gave text directions. For both conditions, users were driving the game on the laptop using a steering wheel. While driving, the only variable measured was user understanding of the interface and also how users liked the interface compared to the physical map. Results Much of the feedback from these tests helped to confirm and support our design decisions. First, half of the users specifically mentioned the helpfulness of the intersection views and the amount of wrong turns decreased. When asked about the amount of their route already completed or what their final destination was, many more users answered correctly then before, signaling that the mileage and destination information was no longer being ignored by users. 30

33 Finally, users had a better idea of how much of the route had been completed because of the feedback provided by the progress display provided by the blue line. Iteration 6: Re-routing The next step in refining the fisheye design was to test the rerouting system that would be used if drivers went off route. This system consisted of two main components. The first was a red screen that appeared over the fisheye that told the user he or she was off route. The second was a red rectangular bar that appeared across the bottom of the screen that also repeated the off route message while giving directions to return to route via right, left, and forward arrows. Figure 12A: Re-routing system Testing method In order to test and refine this design, we once again completed a comparative study of the fisheye and a map. 31