Matt Bernius Lead Experience Planner Kristin Youngling Sr. Director, Data Strategy
When it comes to purchasing user experience design strategy and services, how do you know you re getting the results you paid for? Good design is fundamental to the creation of successful products and services. But how exactly do you determine what is good design? We are fast approaching a time when good in an abstract or an aesthetic sense will no longer be enough. Clients will expect the same level of accountability from the design process that they receive from any other professional service providers. In order to meet that need, designers will need to be able to both measure the effectiveness of a design solution and directly tie that measure back to their core business goals. PG 2
At Effective, we deliver that accountability to our clients through our measurable design framework Inform Assess + define Benchmark Forecast Design + develop Test + measure Analyze + apply Assess current-state measurement and translate business objectives into UX KPIs and metrics Measure the current experience to establish baseline UX metrics Predict the likely impact of design improvements Design and development activities guided by and tested against UX KPIs Test against UX KPIs to evaluate performance Analyze results to provide actionable insights and quantify value of sucess Refine UX KPIs, benchmarks, measurement tools and goals as needed The six steps of measurable design 1. Assess and define a scalable measurement plan To kick off the process, we hold a collaborative workshop in which we partner with a client to surface and align on business goals and define a measurement plan whether that be a holistic, end-to-end strategy or more focused, ad hoc measurement. Based on business objectives, we help clients identify a set of user experience goals, key performance indicators (UX KPIs), and measurement methodologies to benchmark and set improvement goals. Like other business KPIs, UX KPIs are specific goals tied to metrics that can be measured over time. For example, Improve overall customer experience is a KPI that can be tied to a specific metric like the Net Promoter Score, Task Success Rate, or the System Usability Scale. 2. Benchmark the current experience Once UX KPIs and metrics have been established, we leverage a mix of existing data and new measurement techniques to baseline the current experience. If the project involves the development of a new product or service, we look at relevant industry benchmarks as a foundation. 3. Forecast impact We make predictions about how much improvement we can potentially bring to the current experience based on the experience benchmarks PG 3
and our understanding of the project. We also use data from past experiences solving similar challenges to help shape the predictions. 4. Design and develop Once we and our client have agreed upon goals and success metrics, design and development commences. Based on agreed upon success criteria, our team leverages proven best practices to inform our approach. 5. Test and measure Guided by established UX KPIs, our team employs a range of qualitative and quantitative methodologies to test our work and ensure each design decision is getting us closer to achieving both business and user experience goals. 6. Analyze and apply Our team evaluates test results against established benchmarks and past KPI measurements to provide actionable insights that inform and quantify the impact of our design. This analysis enables project stakeholders to track how the new design is delivering against goals, ensures decision-making is compelled by data rather than intuition, and allows clients to clearly articulate the value of success. The measurable design framework has helped our clients achieve their business goals in a number of key ways. In particular, measurable design: Aligns design and business strategy The measurable design framework aligns design decision-making with business objectives. UX KPIs act as constructive constraints that guide the entire process, bringing clarity to a process that can often feel very abstract to our clients. Having empirically informed, business driven criteria helps projects avoid going down ineffective design paths. Most importantly, UX KPIs make it easier for our clients to communicate the value of a design project s successes to their management and other internal stakeholders. Precisely defines the challenges The metrics that are tied to UX KPIs illuminate specific performance factors, helping focus an abstract concern into a definable challenge that can be addressed. If a KPI is broad, like lower support costs, PG 4
specific metrics such as decrease the number of incoming customer calls by 10% can help focus the design solution and ultimately provide a repeatable means of measuring the effect a design solution has on the business s bottom line value. Helps avoid surprises Our rigorous approach, driven by UX KPI s and iterative testing, ensures that stakeholders have, at all times, a strong sense of how an engagement is progressing against its goals. Analysis of prototype test results against the UX KPIs keeps concepts aligned to the project s ultimate objectives. And, by testing early, we can better surface issues and fix them with minimal impact on timelines or budgets, avoiding end of project surprises. Enables what s measured to be managed The measureable design framework also creates an opportunity to track key metrics over time including after the experience launches surfacing underperforming experiences and driving ongoing improvement initiatives. This scalable approach ensures clients are empowered to measure impact throughout the design and development process as well as post launch. In fact, we offer clients training in conducting UX measurements on their own, so that even after we complete an engagement, they can continue to track progress against UX KPIs and make adjustments as needed. Adapts as needed This is not a one-size fits all approach. Just as the UX KPIs change from project to project depending on specific business conditions, the Measurable Design approach is adaptable to the particular needs of each client and each project. Some clients use every step. Others leverage only the core components that best support their project s goals and constraints. What are you doing to measure the ROI of design in your product design process? Are you doing anything to specifically validate your design work and investments? Are you seeking proof that product advances will be found valuable by customers and yield a measurable return? Measuring the value of user experience design enables your organization to understand how design improves top and bottom line performance while providing greater value to your users and customers. PG 5
Matt Bernius Lead Experience Planner Matt Bernius is a lead experience planner at Effective, where he uses a range of methods to allow clients to better understand their end-users. He helps clients transform research findings into executable strategies to improve their products, services and end-user outreach. Matt has over 18 years experience in the industry, and expertise in anthropology and qualitative research methods, including contextual inquiry, ethnography and interviewing; usability testing; research synthesis and presentation; user experience (UX) and user interface (UI) design and strategy. Kristin Youngling Sr. Director, Data Strategy With extensive experience in data-driven design, Kristin Youngling is responsible for our in-house Data Strategy practice. She drives strategy and execution across accounts with the goal of bringing greater accountability to our design offerings. Through her oversight, our teams provide measurement plans and strategies that allow us to effectively communicate design decisions through the lens of real data and ladder up UX KPIs to business value. effectiveinc.com 888.310.5327 partner@effectiveinc.com Effective is an experience agency that embraces digital complexity through a measurable, human-centered design approach. We use comprehensive insight to help clients discover unmet opportunities and build transformative customer experiences. With more than a decade of raising the bar, our purposedriven, diverse teams share their expertise in design thinking, customer insights, data strategy, and technology to empower clients and achieve long-term success.