Modern UX research in action: 10 research stories

Transcription

1 Modern UX research in action: 10 research stories

2 Contents 03 INTRO YELP S MOBILE APP DESTINATION STUDY FITBIT DEVICE AND APP LONGITUDINAL STUDY NEST LEARNING THERMOSTAT UNBOXING APPLE WATCH EXPLORATORY STUDY SNAPCHAT DEMOGRAPHIC COMPARISON GOOGLE WALLET DESTINATION STUDY VIRGIN AMERICA COMPETITOR STUDY NETFLIX REDESIGN USABILITY STUDY STARBUCKS APP DESTINATION STUDY IPHONE 3D TOUCH EXPLORATORY STUDY Modern UX research in action: 10 research stories 02

3 The customer journey is often more complex than simply visiting a website. Intro When most people think of remote usability studies, they think of testing the user experience on websites and apps with participants sitting at home with their device or in front of a computer. And while it s important to provide an awesome online experience, the customer journey is often more complex than simply visiting a website for 15 minutes and making a purchase or exploring a new app for the first time. These days, there s a lot more you can do to uncover insights about how your target customers interact with your brand. With modern UX research tools, it s possible to test customer experiences across all channels, from websites and mobile apps to in-person experiences and even physical products. For example, UserTesting s mobile recorder allows you to record experiences beyond the device, so you can observe the participants surroundings and watch them interact with items in the real world. You can also study customer experiences in specific locations, like retail stores, restaurants, and public transit. Plus, by getting creative with different methodologies, you can observe how people interact with your product over time, how different audience segments perceive your brand differently, and how you measure up to the competition in the eyes of your target market. At UserTesting, we ve set out to push the limits of our own platform, craft some unusual studies, and gather insights beyond traditional website usability testing. Here are some of our favorite stories from the research. Modern UX research in action: 10 research stories 03

4 Yelp s mobile app destination study WHAT WE STUDIED Yelp s mobile app is a bit of an enigma. It s beloved by many, and it has tens of millions of downloads in the App Store and the Play Store... as well as thousands of 1-star reviews. However, reading through the one star reviews, it s clear that many are actually reviews for businesses that users must have accidentally submitted via Yelp s in-app Leave a review about this app messaging. Oops! The reviews of the Yelp app in the App Store don t tell the full story about the user experience on the app itself. So the UserTesting team decided to conduct a study to explore how people use the app out in the real world. HOW WE DID THE RESEARCH Rather than doing a traditional usability study, we decided to launch a destination study using UserTesting s mobile recorder to get some folks out on the street in search of a restaurant to grab a bite to eat. With their smartphone in hand, we asked our participants to tell us a bit about how they typically use Yelp, and then we had them use the app to find a lunch spot nearby. Modern UX research in action: 10 research stories 04

5 WHAT WE LEARNED Participants liked Yelp because it offered all the important information they would need to choose a restaurant or other local business: hours, ratings, phone numbers, a map feature, etc. The users felt like the pictures and reviews in Yelp helped to set realistic expectations of what they would actually encounter when they arrived at the restaurant in person. Because there is so much information available in Yelp, our participants pointed out that having filters is a necessity but the filters can get a little overwhelming and confusing. WRAP-UP Because we ran this study with users walking around on the street, it felt like we were actually walking alongside our users as they took action. We got realistic feedback on how users interact with the app s features in a distracting setting like a busy street while they re hungry. We also got to see users interacting with the map and directions in real time. Super Duper Burger has breakfast? I ve never been there before. Modern UX research in action: 10 research stories 05

6 Fitbit device and app longitudinal study WHAT WE STUDIED Wearable technology for health and fitness has been on the rise for a while, and Fitbit s products are some of the most popular. We wondered: Does a band around your wrist really motivate you to exercise? How drastic is the impact on behavior changes? What causes people to keep wearing this device? HOW WE DID THE RESEARCH We ran a longitudinal study with five users and measured how (if at all) user behavior changed while wearing the Fitbit. During the first part of the study, we asked users to demonstrate how they interact with their Fitbit and the accompanying app. In the second part of the study, we checked in a week later to find out how often they had used their Fitbit in the last week and what, if anything, had changed in their behavior or habits since wearing the Fitbit. Modern UX research in action: 10 research stories 06

7 WHAT WE LEARNED After one week of using the Fitbit, all five users stated they had noticed a change in their behavior and habits since using the Fitbit. These changes included users stating they felt more inclined to stay active, especially in terms of how much they walked, and they felt motivated to keep track of what they ate throughout the day. However, there was still room for improvement. Users mentioned some of the app s features could have been explained more thoroughly, and several users wanted a tutorial to help them set the device up and become familiar with the functions. WRAP-UP For the users in our study, a simple wristband really did impact how much they decided to exercise each day. The constant feedback and accountability users got from Fitbit helped them stick to their goals longer and made them more aware of their eating and exercising habits. It s not just the device that leads people to have healthier life; it s the fact that Fitbit alerts people to lead a healthier life and gives them the awareness to change. It makes me want to get up and move around when I have it on. Modern UX research in action: 10 research stories 07

8 Nest Learning Thermostat unboxing WHAT WE STUDIED Watching customers unbox, assemble, and install your physical product can be as insightful as following what people are doing on your website. We ran an unboxing study with users who recently purchased the Nest Learning Thermostat to see how the product compared with their expectations. HOW WE DID THE RESEARCH Unlike ordinary usability studies that involve digital properties like websites and apps, for this study, we were only interested in the real-world experience with the product, so we launched a beyond-thedevice study. Participants recorded their unboxing experience via their smartphone camera and UserTesting s mobile recorder. They spoke their thoughts aloud as they opened the box, assembled the thermostat, and installed it in their home. Modern UX research in action: 10 research stories 08

9 WHAT WE LEARNED Our participants found several unexpected delights as they installed their devices. One customer was happy the device had a back plate because it wouldn t damage the paint on the wall. Another was pleased that it came with a built-in level, making installation a snap. It s valuable to see how customers discover your product s features in real-time to catch their initial impressions, rather than waiting for them to write a review or complete a follow-up survey. WRAP-UP Having users record the whole process of unboxing lets you get a peek into the world of first impressions and fresh eyes. It ll help you better understand what delights and frustrates users, and you ll figure out how to improve different aspects of your product. There s a level on it too, right on the device, right? That s awesome. Modern UX research in action: 10 research stories 09

10 Apple Watch exploratory study WHAT WE STUDIED The Apple Watch was one of the most anticipated product releases of We wondered if it would live up to the hype. Would it make users lives easier? Which parts of the user experience would be awesome, and which could use some improvement? HOW WE DID THE RESEARCH We ran a qualitative, remote usability study with seven early adopters. Our participants ranged in age from 18 to 54 and were located in the US and the UK. WHAT WE LEARNED Most participants agreed that the Apple Watch takes a little while to get used to, since the interface is so different from other products. Some of the users said they wished the Apple Watch had certain features, like the ability to adjust the screen brightness and timeout, that it actually does have; they just weren t immediately apparent. Three of our participants didn t receive notifications on the watch when they got a text message, and two weren t sure whether setting Do Not Disturb on their iphone would affect their Apple Watch, too. (It turns out that it will if you select the Mirror iphone setting.) Modern UX research in action: 10 research stories 10

11 Still, participants agreed that the Apple Watch did a great job with little things that make life easier. Sending a text was easy, thanks to good voice-to-text recognition, emojis, and pre-programmed common replies like Hey and On my way. One user liked the ability to control the music playing from his iphone through his Apple Watch, especially if his iphone is in his pocket or plugged into speakers across the room. Several users noted they didn t have to look at their phone as often, so they wouldn t be distracted by other apps every time they wanted to read a notification. WRAP-UP In spite of the initial learning curve and the fact that many third-party apps weren t up to snuff yet all seven of our participants said they would use the Apple watch on a daily basis, and they d be very likely to recommend it to a friend. And, of the five participants who had used other smartwatches before, all five preferred the Apple Watch. Controlling my iphone (and what songs are playing) through my wrist is really cool and useful. Modern UX research in action: 10 research stories 11

12 Snapchat demographic comparison WHAT WE STUDIED If you re over 25, you may find Snapchat to be a bit of a mystery. It has a reputation as the app that teenagers use to share self-destructing pictures in a place where parents won t find them. Its interface seems to be designed for a crowd that has grown up using mobile devices and apps rather than the desktop web. Plus, its website, billboards, and marketing materials tend to lack a clear explanation of why you should use it. Here at UserTesting, this made us wonder how users of different age groups interact with the app for the first time. Do they get it? Do they like it? Do they feel like it was made for them? HOW WE DID THE RESEARCH We ran a remote usability study with 18 first-time Snapchat users: six who were between the ages of 18 and 24, six who were 25-40, and six who were over 40. We wanted to find out how the different age groups explored and got the hang of the app and whether or not they felt like the app was made for them. Modern UX research in action: 10 research stories 12

13 WHAT WE LEARNED When our users opened the app for the first time, they arrived straight on the camera screen, and they saw their own face via the front-facing camera. (One groaned, The first thing I see is myself with a double chin. ) A few wished there had been a tutorial to help them get started. It turns out there is a video tutorial, but only three participants discovered it, and one didn t understand how to play it. Older participants acted more irritated than their younger counterparts by the lack of instructions and labels on the app s icons. By contrast, the youngest age group was more confident poking around the app to discover each feature and the meanings behind the icons. Even though the under-25 age group had the easiest time navigating Snapchat, a few of them still weren t convinced that they needed it, noting that they could already share pictures using other apps. Older users were less likely to want to continue using the app, and several said it seemed like it was for kids these days instead of users their own age. WRAP-UP Of the users who did enjoy Snapchat and wanted to keep using it, the primary reason was because they found it fun. None of the users described the app using words like useful or helpful. But Snapchat s millions of users prove that sometimes, having a point isn t the point. It s about providing a fun and addictive service that resonates with just the right audience. I don t know what the number 1 means... what the ghost means... what the rectangular smiley face means. Modern UX research in action: 10 research stories 13

14 Google Wallet destination study WHAT WE STUDIED In a world of smartphones, smartwatches, and the Internet of Things, credit cards may soon be a thing of the past. Mobile payment apps like Google Wallet store users debit, credit, and gift card information, allowing them to make purchases online and in stores that use NFC without the need for a plastic card. We decided to explore how first-time users interact with Google Wallet. How do they feel about keeping their payment information in their phones? What s it like to make a purchase in a store using only the Google Wallet app? HOW WE DID THE RESEARCH To answer these questions, we ran a destination study with five users. We asked them to install and explore the app on their phones, and we gave them each a $10 Walgreens gift card to load into the app. Then we asked them to go to their nearest Walgreens and buy something under $10 using the app. Modern UX research in action: 10 research stories 14

15 WHAT WE LEARNED In general, users were excited about the app. It was easy to set up, the interface was intuitive, and our participants immediately understood the navigation. While Google Wallet requires users to select and enter a PIN to access the app, a couple of our participants wanted more reassurance that their information would be safe. Making a purchase in a brick-and-mortar store was the most confusing part of the experience. Two participants said they wished they d had clearer instructions while making an in-store payment for the first time. Three users noted that the store employees weren t familiar with Google Wallet, so they weren t able to offer any additional help to the users. WRAP-UP Mobile payment apps like Google Wallet face the challenge of balancing convenience and security. While some users are quick to adopt technology that makes their lives easier and eliminates the need for repetitive tasks, others prefer to wait until more people they know have tried it and found it to be safe. Most folks aren t throwing away their plastic cards just yet. But if Google Wallet and its competitors continue to focus on providing a secure, easy-to-understand, and delightful user experience, we may start to use mobile payment apps more often than credit cards in the very near future. I don t know if you re able to scan the bar code directly. You may have to type it in. Modern UX research in action: 10 research stories 15

16 Virgin America competitor study WHAT WE STUDIED Many travel companies have a strikingly similar user experience. When users go to book a flight online, they tend to expect a website that looks more or less the same as all of the other travel companies out there. Virgin America has turned those expectations upside down in the hopes of creating a distinctive, memorable, and hopefully excellent customer experience. Everything from their ads to their in-flight safety messaging goes against the traditional industry norms. But is it effective? Do users understand how to choose a flight? Do they find the workflow helpful or confusing? We wanted to find out if Virgin America s design choices create a delightful user experience or just disorient new customers. HOW WE DID THE RESEARCH We ran a direct comparison study with six first-time Virgin America users. We asked each of them to select and book a flight from Chicago to San Francisco on VirginAmerica.com, and then select and book a similar flight using a different website of their choice. We measured how easy or difficult they found the flight booking process on each website, and we tracked the time it took them to select a flight on each website. Finally, we asked them to verbally compare their experiences on the two websites. Modern UX research in action: 10 research stories 16

17 WHAT WE LEARNED Users noticed right away that the experience on Virgin America was different. While some features were confusing at first (such as selecting departure dates on the calendar and picking a flight out of a list of options), the users reported that the process felt streamlined and enjoyable. All six participants were able to successfully select and book the flight. It took users a longer time on average to select a flight on Virgin America than on the other websites, but users still thought it was easy to use. WRAP-UP Conventional wisdom tells companies to find out their users expectations and then match them. But many customers are more open-minded about online experiences than companies expect. Brilliant visual design, straightforward copy, and an intuitive interface can go a long way in helping users adapt to a new way of doing things. And, as we saw with our study of Virgin America, users are willing to excuse some hiccups in their workflow if the design appeals to their emotions and makes them feel delighted. I m guessing I have to click on the Main Cabin. There we go; now we re moving. Modern UX research in action: 10 research stories 17

18 Netflix redesign usability study WHAT WE STUDIED With over 62 million users worldwide and 10 billion hours of movies and shows streamed in Q1 2015, it s clear that Netflix has been doing something right. That s why their website redesign in Q could be considered a risky move. By changing something so popular, Netflix ran the risk of confusing and alienating existing users who loved the old design. It was critical for Netflix to keep their existing customers happy while also attracting new ones. HOW WE DID THE RESEARCH We recruited five Netflix subscribers to participate in a remote usability study. We screened for people who primarily watch Netflix on their desktop computer. WHAT WE LEARNED Netflix s new website was designed to be more consistent with its mobile, console, and smart TV apps. This creates a more unified omnichannel experience for customers who use Netflix on different devices. Most of our participants said the new design felt newer, sleeker, and more up to date. Modern UX research in action: 10 research stories 18

19 In the new design, users can hover over an image to expand a slideshow of select clips, along with more information on the movie. While users liked the idea of being able to learn more about a movie at a glance, the interface may take some getting used to. Four participants accidentally started playing the movie when they were attempting to find more info. Others lost their place when they accidentally hovered over a different movie while viewing details of one. All five of our participants pointed out that they were glad the redesign still offered a great browsing experience, and they found it very easy to find movies that matched their personal taste preferences. Still, they agreed that there was one area that had room for improvement: the My List feature could be a little overwhelming, and several users had trouble locating it. WRAP-UP With more and more people interacting with companies on multiple devices, it s more important than ever to create a consistent experience across all channels. While no design is perfect, users appreciate knowing what to expect from a company. The modern, sleek feel that users noticed in Netflix s new site may be doing just that. Knowing where I am in this list is great. It would be better if I could click on it and jump to the end. Modern UX research in action: 10 research stories 19

20 Starbucks App destination study WHAT WE STUDIED The Starbucks mobile app introduced an exciting new feature in Fall of 2015: the ability to order and pay for drinks ahead of time, skipping the line in the store. While this seems like a huge convenience for many coffee drinkers, we wondered: Will Starbucks customers understand how to use the feature right away? How smoothly will the process of ordering, paying, and picking up go in the real world? What will people like and dislike about the experience? HOW WE DID THE RESEARCH We recruited six Starbucks mobile app users from around the United States for a two-part study. The first part of the study was a remote user interview. We asked participants how they typically use the Starbucks app, when and where they use it, and what value they find in it. The second part of the study was a destination study in which participants ordered a drink via the app and picked it up in a Starbucks location. Modern UX research in action: 10 research stories 20

21 WHAT WE LEARNED In the interviews, we found that the two main reasons why participants used the app were status and convenience. Participants said they felt like VIPs when they used the app to gain rewards and load up their Starbucks card. They also felt like it was convenient to be able to keep their Starbucks card on their phone, where it was always handy. In the destination study, we found that while it was easy to locate the nearest Starbucks, actually placing the order presented more challenges. Some users weren t sure whether they needed to pick a location or a drink first. Two encountered errors stating their order could not be placed, but no explanation of why. Picking up the drink was the most confusing part of the process: some participants understood where to pick up the drink, others didn t, one participant s drink wasn t ready when she arrived, and one had to start the process all over by getting in line to order her drink. WRAP-UP Overall, participants had positive experiences, although no order and pick-up was seamless. Participants were excited about the idea to use the app to enjoy their favorite drink, even if it took a little longer than expected. Even though the ordering process wasn t perfect, all the participants got what they wanted in the end their favorite Starbucks fix! When technology didn t quite work as planned, great service and loyalty to the brand stepped in to compensate. I don t see my drink anywhere. Maybe I should talk to the cashier? Modern UX research in action: 10 research stories 21

22 iphone 3D Touch exploratory study WHAT WE STUDIED With the introduction of 3D Touch for iphone 6s and 6s Plus, the iphone screen acts as a pressure sensitive surface, making way for an additional dimension of interaction beyond swiping, pinching, and tapping. Users can now apply different amounts of pressure to initiate different types of interaction on their phone. HOW WE DID THE RESEARCH We invited six iphone 6s users and two iphone 6s Plus users to explore the feature in a remote usability study. We asked them to try out the Quick Actions in the home screen, the Peek and Pop gestures in a dummy inbox, and the keyboard-turned-trackpad in Notes. All participants had already played around with their new phones, so they all had some degree of familiarity with 3D Touch or Force Touch prior to the study. WHAT WE LEARNED Most participants admitted they were still getting used to the feature and felt they d need to break their old habits from prior iphones before they could take full advantage of the 3D Touch. The majority of participants encountered some trouble in applying the right amount of pressure. Two participants wanted to see some type of tutorial for 3D Touch to help them better understand how to correctly apply the different amounts of pressure. Modern UX research in action: 10 research stories 22

23 Even without mastery of 3D Touch, participants acknowledged that this new system allowed for smoother transitions from one interaction to another, and it saved them time by cutting out multiple steps in an action. One participant likened the 3D Touch to an advanced right click, while several other participants admired the additional layer or depth of navigation that it allowed. WRAP-UP More than anything else, the novelty of this technology stuck with participants as they worked to get used to the 3D Touch. When we asked participants to choose adjectives to describe the technology, a majority chose words like, inventive, impressive, creative, and fun. Not only did these adjectives reflect an overall positive experience, but it emphasized how fresh the experience was for them. If I press on it, there s a little arrow up top there. If I swipe it, I can do these different tasks. Modern UX research in action: 10 research stories 23

24 Create great experiences UserTesting is the fastest and most advanced user experience research platform on the market. We give marketers, product managers, and UX teams on-demand access to people in their target audience who deliver audio, video, and written feedback on websites, mobile apps, prototypes, and even physical products and locations Bayshore Parkway, Mountain View, CA