Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

What's Next: Measurement in the Design Process

948 views

Published on

In this webinar we take you through practical ways to collect and use data to drive better and more effective designs.

Published in: Marketing
  • As a single mother every little bit counts! This has been such a great way for me to earn extra money. As a single mother every little bit counts! Finally, a vehicle for making some honest to goodness real money to make life easier and happier now that I don't have to pull my hair out budgeting every penny every day.Thanks for the rainbow in my sky. ▲▲▲ https://bit.ly/2Ruzr8s
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

What's Next: Measurement in the Design Process

  1. 1. Powered by MEASUREMENT IN THE DESIGN PROCESS
  2. 2. Hello! Amanda Gagliardi VP, Experience Design at Effective Rob Blackie Digital Strategy
  3. 3. What’s the weather like in your city? Tell us where you’re dialing in from!
  4. 4. Want this deck? It will be available for download shortly after the webinar on: slideshare.net/socialogilvy Ogilvy staff: It’s also on The Market! themarket.ogilvy.com Are you on the go? You can join our webinars on mobile, too! Download the GoToWebinar app from the App Store or Google Play
  5. 5. Measurement in 
 the Design Process
  6. 6. What to Expect 1 MEASUREMENT BASICS How and why we use measurement in the design of new materials, tools and products. 2 HELP GETTING TO CONSENSUS How measurement can make decision making easier. 3 PRACTICAL WAYS TO USE MEASUREMENT We’ll provide tips and tools that anyone can utilize to measure their initiatives.
  7. 7. At Effective, 
 we use data to inform decision making + measure the impact of solutions
  8. 8. Why measure? 1 DEFINE SUCCESS Simplify complexity and articulate a clear definition of success. 2 INFORM DESIGN Ensure decisions are compelled by data vs. intuition. 3 ALIGN STAKEHOLDERS Increase confidence in and efficiency of stakeholder decision making. 4 MEASURE ROI Quantify impact of design and product investments on desired business outcomes.
  9. 9. Data you can use DATA FROM EXISTING CHANNELS Are you changing something that exists or moving it to a new channel? Leverage data from current channels. TESTING Data from A/B or other audience testing approaches. AUDIENCE + STAKEHOLDER FEEDBACK Feedback from existing surveys, call centers or social channels. MARKETING + ADVERTISING DATA Performance data and best practice captured across both traditional and digital initiatives. DIGITAL ANALYTICS Even for analog initiatives, digital behavior patterns can be valuable in learning about audience preference and behavior. BENCHMARK DATA Publicly available studies on performance of marketing and digital materials.
  10. 10. Make sure your initiative 
 has clear business objectives. Translate those into measurable design goals and key performance indicators
 (KPIs). No data? Here’s how to start Form clear objectives Tools like Google Analytics are easy to implement and free. Codes can easily be added to print materials. Capture and store all performance metrics! Measure your outputs When you can, produce multiple versions with slight differences and test against your KPIs to determine what performs best. Create multiple versions 1 2 3
  11. 11. Understanding of the target audience, business goals, and constraints. Discovery Creating concepts and design directions. Definition Testing and revising selected direction. Iteration Putting the final design into market. Launch Design Process
  12. 12. Understanding 
 the target audience, business goals 
 + constraints. DISCOVERY
  13. 13. Sound Familiar? AUDIENCE BEHAVIORS ARE DISTRUSTED Teams often argue about the validity/ frequency of audience pain points or problem statements. INDIVIDUAL PRIORITIES TAKE CENTER STAGE Priorities of stakeholders with the loudest voices or most pull take priority over audience needs or business goals. WHY ARE WE DOING THIS? Lack of clarity around why the project or initiative is happening or what problem it’s trying to solve.
  14. 14. Later that day Two weeks later One month later 3 hours laterThe next day “I am so frustrated that I am left to pay this large amount of money with no help from my insurance provider. I feel like they don’t care about me. I don’t know if I have the money to pay this bill!” “I don’t trust my current health insurance provider, I am excited to enroll with a different provider. I hope they treat me better.” TOUCHPOINTS ACTIONS PAIN POINTS QUANT DATA EMOTIONS PERSONA 1 3 54 James, 32 Goal Reduce the number of calls to customer service during the online find care journey 30% 45% Goal Likelihood to recommend insurance provider *Based on the number of customers that have the ability to find care onlineRe-enroll rate 45% 65% Goal James injures his wrist skiing. He is not sure if he has broken any bones and needs to see a specialist. One of his friends refers him to Dr. Spencer. James logs into his health insurance account to check if Dr. Spencer is “on his insurance” and to see how much an office visit will cost. He is confused because he sees that Dr. Spencer is “in-network,” but an office visit will still cost him around $300. Members are confused by insurance jargon and terminology. 2 It is difficult for members to find clear information on what a deductible is and how it affects their "out-of-pocket" costs. Jame is frustrated with how long he waits for his appointment. If members do not regularly check their email, they may easily miss important communication and documents. Members are not notified or alerted of the increased costs when they are selecting care from an "out-of-network" provider. 6 There is no forgiveness or amnesty for members who make unintentional mistakes when finding and selecting care. James calls his insurance provider’s customer service center and speaks with a representative who explains to him that under his current insurance plan, he has to meet ther $5000 deductible before his costs are covered. James attends his appointment with Dr. Spencer, who writes an order for James to have his wrist x-rayed. Dr. Spencer provides a list of imaging facilities to James. James has his wrist x-rayed at one of the facilities Dr. Spencer recommends to him. James visits Dr. Spencer's office, where he hears that, his wrist is only sprained, not broken. James’ wrist is stabilized with a brace. James' insurance provider sends him an email that includes the Explanation of Benefits of his wrist x-ray. It is sent to an email he rarely checks and James never sees or opens it. James receives a bill in the mail from the imaging facility for $1500 for his wrist x-ray. He is angry and confused as to why he is being charged this much if he has health insurance. He calls the insurance provider’s customer service center and speaks with a representative who tells him that the facility that conducted his x-rays was “out-of-network,” which is why he is responsible for paying most of the costs. During the next year's open enrollment period, James intentionally selects a competitor's plan instead of his current provider. + – 30k Page views 59CSAT industry avg. 0:46 Avg. pg duration 20% Navigate to "Find Care" 10% Take an action $78,300K cost/mo 10K calls/mo 7:50m avg call time $7.83 avg call cost 19m avg wait in the US Average satisfaction of health plan customers. 59 CSAT 89 Goal $20,660K cost/mo 2K calls/mo 10:20m avg call time $10.33 avg call cost The next day Six months laterA few days later 8% Open rate 55m avg wait time 55% Re-enrollment rate 45% Drop off rate “Ouch! My wrist really hurts, I am bummed I won’t be able to ski anymore this season.” “ I am frustrated that I have to pay so much money, for an office visit” “Why am I being charged so much money? I am shocked, frustrated and confused.” “At least my wrist isn’t broken. I am relieved to know it will heal in a few weeks.” 45CSAT -15 below avg 30% bounce rate
  15. 15. Validating design 1 2 3 4 Define how to measure KPIs Identify user experience KPIs Map user experience goals Align on business goals 
 + metrics
  16. 16. Creating concepts + design directions. DEFINITION
  17. 17. CHOOSING A CONCEPT You and your team have been given choices but there is no clear winner. 
 How do you decide?
  18. 18. Choosing the right direction BUSINESS GOALS EXISTING DATA LIGHT TESTING It seems straightforward but often times we get swept away by something beautiful, new or flashy and forget about what our goals were in the first place. Understand how the designs meet your business goals If you have insights on words, placement, etc that have driven best performance in the past, choose a design that incorporates the same or similar elements. Use lessons learned from the data you already have Ask some test subjects a set of measured questions and use the KPIs you identified at kickoff to score each concept (more on this in the next section). Perform lightweight tests with all the concepts
  19. 19. Identify Benchmark Design + Test 1 2 3 Example: Website Concept Test 4 Analyze Identify priority goals and KPIs Benchmark the current site Test new design concepts Analyze performance and apply
  20. 20. 1 WHAT’S DEFINED Monthly website sales as a business KPI Task success rate as a UX KPI
 IDENTIFY BUSINESS + UX GOALS Increase online sales + Streamline website purchase flow
  21. 21. 2 WHAT’S DETERMINED 45% of users complete successful task/purchase BENCHMARK Measure existing site and set baseline
  22. 22. 3DESIGN + MEASURE Two design concepts are created and tested against our defined UX KPIs WHAT’S MEASURED
 Concept One - 75% | Concept Two - 60% of users complete successful task/purchase
  23. 23. 4ANALYZE A definitive top performer is determined WHAT’S FOUND
 Existing Site - 45% Concept One - 75% Concept Two - 60% of users complete successful task/purchase
  24. 24. Quantify value + tie back EXISTING SITE Monthly Visits = 100K Avg. Cart Size = $100 Task Success Rate = 45% Revenue = $4.5 M CONCEPT ONE Monthly Visits = 100K Avg. Cart Size = $100 Task Success Rate = 75% Revenue = $7.5 M All else equal, the improvement in task success rate translates into $3.0 million of incremental monthly revenue
  25. 25. Revising selected direction. ITERATION
  26. 26. “The concepts were so clean and simple, but by the time we got all the required items in, it seemed too complicated.
  27. 27. 1 USE AN EXISTING METHODOLOGY Tests like the System Usability Scale are easy to administer and have benchmark data available. 2 …OR MAKE YOUR OWN Create a simple set of statements related to your KPIs. Ask your audience to rate their agreement on a Likert scale (strongly disagree to strongly agree) and convert ratings to 1-5 numbers. 3 ADMINISTER THE TEST Give the test to five or more audience members, allow them to provide context for their ratings so you can understand why the iterations is working (or not). 4 Test each iteration with your audience TRACK CHANGE OVER ITERATIONS Average scores and track for each iteration. GETTING IT RIGHT
  28. 28. Example: System Usability Scale
  29. 29. Putting the final design into market. LAUNCH
  30. 30. We’re done, right? SET IT AND FORGET IT Once an initiative is launched, everyone moves on to the next thing and hopes that it does what it’s supposed to. IT DOESN’T PERFORM AS EXPECTED The initiative doesn’t drive the results that everyone had hoped but nobody knows why or what to do. IT WORKS! The initiative performs great but there’s no way to quantify the value or repeat the success.
  31. 31. Ongoing measurement 51 2 3 4 Define how
 to measure KPIs Identify user experience KPIs Map user experience goals Align on business goals 
 + metrics Leverage framework for ongoing optimization
  32. 32. KPI dashboard Task success rate 0% 25% 50% 75% 100% Q1 Q2 Q3 Q4 Baseline Redesign Goal Task abandonment rate 0% 25% 50% 75% 100% Q1 Q2 Q3 Q4 Baseline Redesign Goal Error rate 0% 25% 50% 75% 100% Q1 Q2 Q3 Q4 Baseline Redesign Goal Time on task 15 22.5 30 37.5 45 Q1 Q2 Q3 Q4 Baseline Redesign Goal Product sales Add-On Traditional Conversion rate 0% 5% 10% 15% 20% Baseline Final Redesign
  33. 33. Use data to define clear customer and business KPIs that you can measure throughout the process. Discovery Use data from previous efforts to design to known successes and best practices. Test concepts against each other measuring your key KPIs. Definition Test each iteration to ensure that you are not degrading the design experience. Iteration Track performance once the product is in market and record learnings for improvement and future efforts. Launch Design Process
  34. 34. Questions? Amanda Gagliardi VP, Experience Design at Effective Rob Blackie Digital Strategy
  35. 35. Want this deck? It will be available for download shortly after the webinar on: slideshare.net/socialogilvy Ogilvy staff: It’s also on The Market! themarket.ogilvy.com Are you on the go? You can join our webinars on mobile, too! Download the GoToWebinar app from the App Store or Google Play

×