0
eewei@emoti.vu / @emotivu / Emoti.vu
Metrics Driven Design
Better define success, deliver impact, make money
Eewei Chen
@U...
20 years
What is success?
Stories, workflows, wireframes, interactions
My Flights
ALERT:
Manila to Cebu
Book A Flight
Get The Best Exchange Rates
Fl...
Working software
Burn down
Collaboration
Refactoring
Agile
good enough
Disruption
no longer exist
no one can live without
Survival?
Can UX success be measured?
Pack selection
+ 213%
Checkout completion
+ 252%
User satisfaction*
+ 237%
Sky Upgrades
Free Trial uptake
+ 137%
Subscript...
BUT what if there isn’t much to go on?
(Case study)
Digital Airline Experiences are fragmented
Fear & Punishment works??
Apple wants it all
Many platforms
Stay connected with travellers
• Data: Passenger info
• Need: More connected mobile digital experience
• ROI: Mobile Ad wa...
Opportunity
Mobile Advertising
US$11.4 Billion(i)
Mobile Ad spend in 2013
£500 million
UK spend in 2012 is most per user (...
Data (any)
Insight
Business Traveller
going on a Business Trip
buys a ticket 7 Days or
less before travel
52/41 = (127) in...
Business Traveller
• 23% of all travellers, Frequent flyer (175)
• male (123), 50 Years Old (139), Company Chairman (233)
...
Empathy
Books travel & hotel <7 Days in
advance (127)
Deal Alerts<7 Days
in advance of travel
Increase Ancillary CPM
rate ...
Success
Output
Software we build. The materials we produce.
Outcome
The immediate change due to delivering output.
Impact
...
Assumptions
We believe by Output,
we will Outcome*
leading to Impact*
We believe by creating deal alerts,
we will increase...
What do we do first?
Value (relative)
EASY
HIGH VALUE
DIFFICULT
Quick wins Differentiator
Parity / building blocks Bin
LOW VALUE
Deal Alerts
Ho...
Wireframes
Deal Alerts <7 Days in
advance of travel
My Flights
ALERT:
Manila to Cebu
Book A Flight
ALERT / ADVERT
Flight S...
What more can we measure?
Passenger	
  info	
  from	
  Airline
Origina'on
Des'na'on
Gender
Departure	
  date
Return	
  date
Language
Cabin	
  class
...
What about making money?
Success can be Everywhere*
Triggers*
Holistic
AARRR!**
*BJ	
  Fogg,	
  Behavioral	
  Analysis:	
  h'p://www.behaviormodel....
Don’t forget to experiment
*In App customer survey feedback form
Validation
• Guess
• Consequence
• Experiment
Cornell U, 1964
• Assumption
• Validat...
In Conclusion
• Problem / Goal
• Opportunity
• Data
• Success= Outputs, Outcomes, Impact
• Assumptions
• Empathy
• Value M...
Metrics Driven Design
How to define success and deliver more impact
Eewei Chen
@Ultraman
http://101DesignIngredients.com
Upcoming SlideShare
Loading in...5
×

Metrics Drive Design - Eewei Chen

481

Published on

Presented at the UXPA September event on User-Centred Design and Agile

Published in: Design, Technology, Travel
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
481
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
4
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Transcript of "Metrics Drive Design - Eewei Chen"

  1. 1. eewei@emoti.vu / @emotivu / Emoti.vu Metrics Driven Design Better define success, deliver impact, make money Eewei Chen @Ultraman http://www.ml.sun.ac.za/wp-content/uploads/2009/08/social_graph.png “Efforts and courage are not enough without purpose and direction.” John F. Kennedy
  2. 2. 20 years
  3. 3. What is success?
  4. 4. Stories, workflows, wireframes, interactions My Flights ALERT: Manila to Cebu Book A Flight Get The Best Exchange Rates Flight Status CP100 - OK Check-in Available (1) Destination Guides CEB Club Commercial Partners About Cebu Pacific Sign-in Register Shake Iphone to view more Special Flight Offers MANILA TO BORACAY One way from P780 DISRUPTION: Manila to Cebu delayed Abstract Concrete UX Assets?
  5. 5. Working software Burn down Collaboration Refactoring Agile good enough
  6. 6. Disruption no longer exist no one can live without Survival?
  7. 7. Can UX success be measured?
  8. 8. Pack selection + 213% Checkout completion + 252% User satisfaction* + 237% Sky Upgrades Free Trial uptake + 137% Subscriptions uptake + 187% User satisfaction* + 223% Now TV Subscriptions Query resolution + 120% Info retrieval speed + 239% User satisfaction* + 174% Sky.com Nav Problem: Poor trial and subs uptake Solution: • New marketing animation • More effective brand messages • Timed marketing triggers • Remote usability test (2K users) • 4 week POC, 12 weeks Problem: Too many choices, low upgrade conversion Solution: • Usage based recommendations • 9 packs down to 3 • Quick check out • MVT Testing to 1% traffic • 3 week POC, 8 weeks delivery Problem: Inaccurate FAQs and MySKY tasks Solution: • Super nav drop down • Quick tasks • Simplified IA • Remote usability test (2K users) • 2 week POC, 6 weeks delivery *In App customer survey feedback form
  9. 9. BUT what if there isn’t much to go on? (Case study)
  10. 10. Digital Airline Experiences are fragmented Fear & Punishment works?? Apple wants it all Many platforms
  11. 11. Stay connected with travellers • Data: Passenger info • Need: More connected mobile digital experience • ROI: Mobile Ad wallet share • UX: Useful traveller engagement throughout lifecycle Airline Airline Airline Advertiser Advertiser Advertiser
  12. 12. Opportunity Mobile Advertising US$11.4 Billion(i) Mobile Ad spend in 2013 £500 million UK spend in 2012 is most per user (ii) (i)  -­‐  h'p://www.gartner.com/newsroom/id/2306215 (ii)  -­‐  h'p://www.guardian.co.uk/media/2012/oct/09/mobile-­‐adverDsing-­‐500-­‐million-­‐pounds-­‐2012 “Connecting with Travellers” Trend Gap Channel Vision
  13. 13. Data (any) Insight Business Traveller going on a Business Trip buys a ticket 7 Days or less before travel 52/41 = (127) index
  14. 14. Business Traveller • 23% of all travellers, Frequent flyer (175) • male (123), 50 Years Old (139), Company Chairman (233) • Earns 90K / month (133) • Android smartphone / tablet user (NA) • Unaware of Airline Loyalty Program (109) Behaviours(unique insight) • Books travel & hotel < 7 days in advance (127) • Pays for tickets at airline ticket office (115), airport (117) airline call centre (111) and travel agent (115). • Checks-in at airline desk (106) • Buys advertised goods at airport (128), in-flight (113), later (117). Spends over 250 (191) on perfume, electronics, watches, sunglasses, toys, DVDs/Music • Research In-Flight magazine (114) and travel sites (132). • Plans next trip going to airport (129), in airport lounge (129) and on the flight (111) • Uses laptop on the plane (124) • Doesn’t value airline membership (67% of business) Personality • Last minute business trip planner • Appreciates quality • Will use something if it makes his life easier • Wants respect and a personalised service. • Prefers human face-to-face contact • Impulse buyer, influenced by advertising • Likes to stay informed and up-to-date • Wants to maximise work time Barriers • Does not trust social media recommendations • Does have time to do much research • Money not an issue but wants to know it is well spent Output, Outcome, Impact • Deal alerts<7 days before travel to increase Deal Section Visits by 25%& Ad CPM by 25% • Improve mobile usage (095) and mobile boarding pass value (101) to reduce non-web and non-mobile bookings (all of above) • Faster mobile check-in (092) to reduce physical in-person check-ins (by all of above) and check-in time by 25%. • Product and service recommendations (200): business (1.35), food (1.06), hotel (1.03) and consumer products(1.04), entertainment (1.05) to improve Ancillary CPM by 200% • Personalised Ads: Automotive (1.31), Corporate (1.29), Conference (1.19), Electronics (1.10), Tourism (1.01) and in-flight retail choices (0.81) to increase online/mob CPM and conv (TBD). • Travel updates (0.94) and info to increase recommend to friend (092) and repeat booking purchase (101) • In-flight wifi to increase laptop usage (124) •(numbers)  represent  indexed  data  analyDcs Deal Alerts<7 Days in advance of travel Data
  15. 15. Empathy Books travel & hotel <7 Days in advance (127) Deal Alerts<7 Days in advance of travel Increase Ancillary CPM rate by 25% Does not have time to do research Increase PVs by 25% Last minute planner OUTPUT OUTCOME IMPACT Barriers Personality Behaviour (numbers)  represent  indexed  GPS  data
  16. 16. Success Output Software we build. The materials we produce. Outcome The immediate change due to delivering output. Impact The larger change we see over time Outcome Impact Output
  17. 17. Assumptions We believe by Output, we will Outcome* leading to Impact* We believe by creating deal alerts, we will increase visits to deals section by 25% leading to an increase in Ad CPM by 25% *QualitaDve  or  quanDtaDve  measurement Outcome Impact Output
  18. 18. What do we do first?
  19. 19. Value (relative) EASY HIGH VALUE DIFFICULT Quick wins Differentiator Parity / building blocks Bin LOW VALUE Deal Alerts Home screen advert Basic geo-location *if  we  had  to  build  the  capability  ourselves.  Therefore...  Get  client  to  build  and  enable Mobile CMS In-flight wifi* iOS Push Alerts ROI: Customer value x Personas Business value A Business value B Ease of implementation Dependency level
  20. 20. Wireframes Deal Alerts <7 Days in advance of travel My Flights ALERT: Manila to Cebu Book A Flight ALERT / ADVERT Flight Status CP100 - OK Check-in Available (1) Destination Guides CEB Club Commercial Partners About Cebu Pacific Sign-in Register Shake Iphone to view more Special Flight Offers MANILA TO BORACAY One way from P780 DISRUPTION: Manila to Cebu delayed Flight notifications Ancillary service notifications OFF Connect with Facebook Disconnect Twitter Connect with LinkedIn Settings Share this app Contact us Edit your personal details CEB Club profile Get in touch Social Networks Push notifications ON CEBU PACIFIC: You've just arrived. Book a taxi and save 15% today CEBU PACIFIC: Discounted fares to Boracay this week starting from PHP 780 SPECIAL OFFERS Discount Airtime Tariffs SPECIAL OFFERS ALERT / ADVERT MANILA TO BORACAY One way from P780 ..... Terms & Conditions Additional info on Special offers BOOK NOW Done Outcome Set explicit notifications (time, location, price etc.) Receive well timed push notification See on page alerts on key pages* (e.g. HP) Click to view target page (Destination Page + AD) *Assume  persistent  signed-­‐in  state Impact Output Output Output
  21. 21. What more can we measure?
  22. 22. Passenger  info  from  Airline Origina'on Des'na'on Gender Departure  date Return  date Language Cabin  class Age Airline  loyalty  level Ticket  type  (open  or  return) Na'onality Implicit Click  path  analysis Page  view  /  Preview Time  on  page Shared  on  FB  and  TwiHer Email Geo-­‐locate  /  Check-­‐in Time  of  day Implicit  but  heavy  AI,   Machine  Learning* Sentence  analysis Mood  analysis Other  behavioural  paHern   analysis Explicit Wizards Ques'onnaires Booking Search Save SeSngs  /  Preferences Likes Good Start Differentiation Track it
  23. 23. What about making money?
  24. 24. Success can be Everywhere* Triggers* Holistic AARRR!** *BJ  Fogg,  Behavioral  Analysis:  h'p://www.behaviormodel.org/ **  Dave  McClure:  Pirate  Metrics:  h'p://www.slideshare.net/dmc500hats/startup-­‐metrics-­‐for-­‐pirates-­‐long-­‐version
  25. 25. Don’t forget to experiment
  26. 26. *In App customer survey feedback form Validation • Guess • Consequence • Experiment Cornell U, 1964 • Assumption • Validate • MVP September 2011 Richard Fenyman Eric Ries If it disagrees with experiment, it’s WRONG!
  27. 27. In Conclusion • Problem / Goal • Opportunity • Data • Success= Outputs, Outcomes, Impact • Assumptions • Empathy • Value Map • Wireframes • Track & Capture • Make Money • Experiment
  28. 28. Metrics Driven Design How to define success and deliver more impact Eewei Chen @Ultraman http://101DesignIngredients.com
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×