Metrics Drive Design - Eewei Chen


Published on

Presented at the UXPA September event on User-Centred Design and Agile

Published in: Design, Technology, Travel
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Metrics Drive Design - Eewei Chen

  1. 1. / @emotivu / Metrics Driven Design Better define success, deliver impact, make money Eewei Chen @Ultraman “Efforts and courage are not enough without purpose and direction.” John F. Kennedy
  2. 2. 20 years
  3. 3. What is success?
  4. 4. Stories, workflows, wireframes, interactions My Flights ALERT: Manila to Cebu Book A Flight Get The Best Exchange Rates Flight Status CP100 - OK Check-in Available (1) Destination Guides CEB Club Commercial Partners About Cebu Pacific Sign-in Register Shake Iphone to view more Special Flight Offers MANILA TO BORACAY One way from P780 DISRUPTION: Manila to Cebu delayed Abstract Concrete UX Assets?
  5. 5. Working software Burn down Collaboration Refactoring Agile good enough
  6. 6. Disruption no longer exist no one can live without Survival?
  7. 7. Can UX success be measured?
  8. 8. Pack selection + 213% Checkout completion + 252% User satisfaction* + 237% Sky Upgrades Free Trial uptake + 137% Subscriptions uptake + 187% User satisfaction* + 223% Now TV Subscriptions Query resolution + 120% Info retrieval speed + 239% User satisfaction* + 174% Nav Problem: Poor trial and subs uptake Solution: • New marketing animation • More effective brand messages • Timed marketing triggers • Remote usability test (2K users) • 4 week POC, 12 weeks Problem: Too many choices, low upgrade conversion Solution: • Usage based recommendations • 9 packs down to 3 • Quick check out • MVT Testing to 1% traffic • 3 week POC, 8 weeks delivery Problem: Inaccurate FAQs and MySKY tasks Solution: • Super nav drop down • Quick tasks • Simplified IA • Remote usability test (2K users) • 2 week POC, 6 weeks delivery *In App customer survey feedback form
  9. 9. BUT what if there isn’t much to go on? (Case study)
  10. 10. Digital Airline Experiences are fragmented Fear & Punishment works?? Apple wants it all Many platforms
  11. 11. Stay connected with travellers • Data: Passenger info • Need: More connected mobile digital experience • ROI: Mobile Ad wallet share • UX: Useful traveller engagement throughout lifecycle Airline Airline Airline Advertiser Advertiser Advertiser
  12. 12. Opportunity Mobile Advertising US$11.4 Billion(i) Mobile Ad spend in 2013 £500 million UK spend in 2012 is most per user (ii) (i)  -­‐  h'p:// (ii)  -­‐  h'p://­‐adverDsing-­‐500-­‐million-­‐pounds-­‐2012 “Connecting with Travellers” Trend Gap Channel Vision
  13. 13. Data (any) Insight Business Traveller going on a Business Trip buys a ticket 7 Days or less before travel 52/41 = (127) index
  14. 14. Business Traveller • 23% of all travellers, Frequent flyer (175) • male (123), 50 Years Old (139), Company Chairman (233) • Earns 90K / month (133) • Android smartphone / tablet user (NA) • Unaware of Airline Loyalty Program (109) Behaviours(unique insight) • Books travel & hotel < 7 days in advance (127) • Pays for tickets at airline ticket office (115), airport (117) airline call centre (111) and travel agent (115). • Checks-in at airline desk (106) • Buys advertised goods at airport (128), in-flight (113), later (117). Spends over 250 (191) on perfume, electronics, watches, sunglasses, toys, DVDs/Music • Research In-Flight magazine (114) and travel sites (132). • Plans next trip going to airport (129), in airport lounge (129) and on the flight (111) • Uses laptop on the plane (124) • Doesn’t value airline membership (67% of business) Personality • Last minute business trip planner • Appreciates quality • Will use something if it makes his life easier • Wants respect and a personalised service. • Prefers human face-to-face contact • Impulse buyer, influenced by advertising • Likes to stay informed and up-to-date • Wants to maximise work time Barriers • Does not trust social media recommendations • Does have time to do much research • Money not an issue but wants to know it is well spent Output, Outcome, Impact • Deal alerts<7 days before travel to increase Deal Section Visits by 25%& Ad CPM by 25% • Improve mobile usage (095) and mobile boarding pass value (101) to reduce non-web and non-mobile bookings (all of above) • Faster mobile check-in (092) to reduce physical in-person check-ins (by all of above) and check-in time by 25%. • Product and service recommendations (200): business (1.35), food (1.06), hotel (1.03) and consumer products(1.04), entertainment (1.05) to improve Ancillary CPM by 200% • Personalised Ads: Automotive (1.31), Corporate (1.29), Conference (1.19), Electronics (1.10), Tourism (1.01) and in-flight retail choices (0.81) to increase online/mob CPM and conv (TBD). • Travel updates (0.94) and info to increase recommend to friend (092) and repeat booking purchase (101) • In-flight wifi to increase laptop usage (124) •(numbers)  represent  indexed  data  analyDcs Deal Alerts<7 Days in advance of travel Data
  15. 15. Empathy Books travel & hotel <7 Days in advance (127) Deal Alerts<7 Days in advance of travel Increase Ancillary CPM rate by 25% Does not have time to do research Increase PVs by 25% Last minute planner OUTPUT OUTCOME IMPACT Barriers Personality Behaviour (numbers)  represent  indexed  GPS  data
  16. 16. Success Output Software we build. The materials we produce. Outcome The immediate change due to delivering output. Impact The larger change we see over time Outcome Impact Output
  17. 17. Assumptions We believe by Output, we will Outcome* leading to Impact* We believe by creating deal alerts, we will increase visits to deals section by 25% leading to an increase in Ad CPM by 25% *QualitaDve  or  quanDtaDve  measurement Outcome Impact Output
  18. 18. What do we do first?
  19. 19. Value (relative) EASY HIGH VALUE DIFFICULT Quick wins Differentiator Parity / building blocks Bin LOW VALUE Deal Alerts Home screen advert Basic geo-location *if  we  had  to  build  the  capability  ourselves.  Therefore...  Get  client  to  build  and  enable Mobile CMS In-flight wifi* iOS Push Alerts ROI: Customer value x Personas Business value A Business value B Ease of implementation Dependency level
  20. 20. Wireframes Deal Alerts <7 Days in advance of travel My Flights ALERT: Manila to Cebu Book A Flight ALERT / ADVERT Flight Status CP100 - OK Check-in Available (1) Destination Guides CEB Club Commercial Partners About Cebu Pacific Sign-in Register Shake Iphone to view more Special Flight Offers MANILA TO BORACAY One way from P780 DISRUPTION: Manila to Cebu delayed Flight notifications Ancillary service notifications OFF Connect with Facebook Disconnect Twitter Connect with LinkedIn Settings Share this app Contact us Edit your personal details CEB Club profile Get in touch Social Networks Push notifications ON CEBU PACIFIC: You've just arrived. Book a taxi and save 15% today CEBU PACIFIC: Discounted fares to Boracay this week starting from PHP 780 SPECIAL OFFERS Discount Airtime Tariffs SPECIAL OFFERS ALERT / ADVERT MANILA TO BORACAY One way from P780 ..... Terms & Conditions Additional info on Special offers BOOK NOW Done Outcome Set explicit notifications (time, location, price etc.) Receive well timed push notification See on page alerts on key pages* (e.g. HP) Click to view target page (Destination Page + AD) *Assume  persistent  signed-­‐in  state Impact Output Output Output
  21. 21. What more can we measure?
  22. 22. Passenger  info  from  Airline Origina'on Des'na'on Gender Departure  date Return  date Language Cabin  class Age Airline  loyalty  level Ticket  type  (open  or  return) Na'onality Implicit Click  path  analysis Page  view  /  Preview Time  on  page Shared  on  FB  and  TwiHer Email Geo-­‐locate  /  Check-­‐in Time  of  day Implicit  but  heavy  AI,   Machine  Learning* Sentence  analysis Mood  analysis Other  behavioural  paHern   analysis Explicit Wizards Ques'onnaires Booking Search Save SeSngs  /  Preferences Likes Good Start Differentiation Track it
  23. 23. What about making money?
  24. 24. Success can be Everywhere* Triggers* Holistic AARRR!** *BJ  Fogg,  Behavioral  Analysis:  h'p:// **  Dave  McClure:  Pirate  Metrics:  h'p://­‐metrics-­‐for-­‐pirates-­‐long-­‐version
  25. 25. Don’t forget to experiment
  26. 26. *In App customer survey feedback form Validation • Guess • Consequence • Experiment Cornell U, 1964 • Assumption • Validate • MVP September 2011 Richard Fenyman Eric Ries If it disagrees with experiment, it’s WRONG!
  27. 27. In Conclusion • Problem / Goal • Opportunity • Data • Success= Outputs, Outcomes, Impact • Assumptions • Empathy • Value Map • Wireframes • Track & Capture • Make Money • Experiment
  28. 28. Metrics Driven Design How to define success and deliver more impact Eewei Chen @Ultraman