Designing & Optimizing the DNA of a Killer App Dan Olsen, CEO & Founder, YourVersion Startonomics  · San Francisco · Oct 2...
Building the DNA of a Killer App with Metrics and Optimization <ul><li>How to integrate metrics & optimization </li></ul><...
Adding Metrics and Optimization to your Product Process Copyright © 2008 Olsen Solutions LLC Plan Design Develop Business ...
Optimization through Iteration: Continuous Improvement Copyright © 2008 Olsen Solutions LLC Measure the metric Analyze the...
Optimizing Business Value Creation with Metrics Copyright © 2008 Olsen Solutions LLC
Approaching Business as an Optimization Exercise <ul><li>Given reality as it exists today, </li></ul><ul><li>optimize our ...
<ul><li>Profit =  Revenue  - Cost </li></ul>Define the Equation of your Business “Peeling the Onion” Unique Visitors   x  ...
How to Track Your Metrics <ul><li>Track each metric as daily time series </li></ul><ul><li>Create ratios from primary metr...
Sample Signup Page Yield Data Copyright © 2008 Olsen Solutions LLC
Using Metrics to Optimize the Equation of your Business <ul><li>What are the metrics for your business? </li></ul><ul><li>...
Case Study: Optimizing Viral Loop Metrics <ul><li>Which metric has highest ROI opportunity? </li></ul>Copyright © 2008 Ols...
Maximum Upside Potential of a Metric Copyright © 2008 Olsen Solutions LLC 0 100% 0 100% 0 ? Registration conversion rate %...
Copyright © 2008 Olsen Solutions LLC Doubling Number of Invitations Sent per Sender by Adding Address Book Importer
Optimizing Customer Value Creation with Metrics Copyright © 2008 Olsen Solutions LLC
The Fuzzy Math of Customer Value <ul><li>You create customer value with a product that </li></ul><ul><ul><li>Satisfies cus...
What is Your Value Proposition? <ul><li>Which user benefits are most important? </li></ul><ul><li>How well does your produ...
Analyzing Product Ideas by ROI Copyright © 2008 Olsen Solutions LLC Investment (developer-weeks) Return (Value Created) Id...
Prioritizing Product Ideas by ROI Copyright © 2008 Olsen Solutions LLC Investment (developer-weeks) Return (Value Created)...
The UI Design Iceberg Copyright © 2008 Olsen Solutions LLC Visual Design Interaction Design Information Architecture Conce...
Approaching UI Design Analytically <ul><li>Typical UI design question: </li></ul><ul><li>“ When using web pages, do users ...
Put Key Conversion Actions Above The Fold Copyright © 2008 Olsen Solutions LLC Landing Page A Landing Page B Key conversio...
The Fold Isn’t Binary Either Copyright © 2008 Olsen Solutions LLC 600 px 768 px 1024 px Data courtesy of ClickTale Free tr...
Case Study: Account Signup Process Redesign Copyright © 2008 Olsen Solutions LLC % of Users Sign in / Registration Account...
Analysis of Sign In/Registration Flow Copyright © 2008 Olsen Solutions LLC Open Account Sign in Account Selection Register...
Redesigned User Flow Improved Registration Conversion Rate Copyright © 2008 Olsen Solutions LLC 37% improvement in convers...
Summary: Building the DNA of a Killer App <ul><li>Define what success means </li></ul><ul><ul><li>Customer value propositi...
[email_address] Questions? www.yourversion.com
Appendix Copyright © 2008 Olsen Solutions LLC
Taking Optimization to The Next Level <ul><li>Multivariate testing, aka A/B testing </li></ul><ul><li>New challenges emerg...
Olsen’s Hierarchy of Web User Needs (adapted from Maslow) Copyright © 2008 Olsen Solutions LLC Is the site up when I want ...
Kano Model: User Needs & Satisfaction Copyright © 2008 Olsen Solutions LLC User Satisfaction User Dissatisfaction Performa...
Importance vs. Satisfaction  Ask Users to Rate for Each Feature Copyright © 2008 Olsen Solutions LLC Recommended reading: ...
Upcoming SlideShare
Loading in...5
×

Designing & Optimizing the DNA of a Killer App (Dan Olsen, Startonomics SF 2008)

4,219

Published on

Presentation by Dan Olsen, YourVersion, on how to plan, design, & optimize product development by focusing on measurable events & user actions, business value creation.

Published in: Technology, Economy & Finance
0 Comments
11 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
4,219
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
246
Comments
0
Likes
11
Embeds 0
No embeds

No notes for slide
  • Designing & Optimizing the DNA of a Killer App (Dan Olsen, Startonomics SF 2008)

    1. 1. Designing & Optimizing the DNA of a Killer App Dan Olsen, CEO & Founder, YourVersion Startonomics · San Francisco · Oct 2, 2008
    2. 2. Building the DNA of a Killer App with Metrics and Optimization <ul><li>How to integrate metrics & optimization </li></ul><ul><li>Using metrics to optimize value creation </li></ul><ul><ul><li>Business Value </li></ul></ul><ul><ul><li>Customer Value </li></ul></ul><ul><ul><li>UI Design </li></ul></ul><ul><li>Illustrate with real world examples </li></ul>Copyright © 2008 Olsen Solutions LLC
    3. 3. Adding Metrics and Optimization to your Product Process Copyright © 2008 Olsen Solutions LLC Plan Design Develop Business Objectives Product Objectives Prioritized Feature List Scoping Requirements & Design Code Test Launch Site Level Feature Level Optimize Metrics & User Feedback
    4. 4. Optimization through Iteration: Continuous Improvement Copyright © 2008 Olsen Solutions LLC Measure the metric Analyze the metric Identify top opportunities to improve Design & develop the enhancement Launch the enhancement <ul><li>Learning </li></ul><ul><ul><li>Gaining knowledge: </li></ul></ul><ul><ul><li>Market </li></ul></ul><ul><ul><li>Customer </li></ul></ul><ul><ul><li>Domain </li></ul></ul><ul><ul><li>Usability </li></ul></ul>
    5. 5. Optimizing Business Value Creation with Metrics Copyright © 2008 Olsen Solutions LLC
    6. 6. Approaching Business as an Optimization Exercise <ul><li>Given reality as it exists today, </li></ul><ul><li>optimize our business results </li></ul><ul><li>subject to our resource constraints. </li></ul>Copyright © 2008 Olsen Solutions LLC
    7. 7. <ul><li>Profit = Revenue - Cost </li></ul>Define the Equation of your Business “Peeling the Onion” Unique Visitors x Ad Revenue per Visitor Impressions/Visitor x Effective CPM / 1000 Visits/Visitor x Pageviews/Visit x Impressions/PV New Visitors + Returning Visitors Invited Visitors + Uninvited Visitors # of Users Sending Invites x Invites Sent/User x Invite Click-through Rate Copyright © 2008 Olsen Solutions LLC
    8. 8. How to Track Your Metrics <ul><li>Track each metric as daily time series </li></ul><ul><li>Create ratios from primary metrics: X / Y </li></ul><ul><ul><li>Example: How good is your registration page? </li></ul></ul><ul><ul><li>Okay: # of registered users per day </li></ul></ul><ul><ul><li>Better: registration conversion rate = </li></ul></ul><ul><ul><li># registered users / # uniques to reg page </li></ul></ul>Copyright © 2008 Olsen Solutions LLC Date Unique Visitors Page views Ad Revenue New User Sign-ups … 4/24/08 10,100 29,600 25 490 4/27/08 10,500 27,100 24 480 …
    9. 9. Sample Signup Page Yield Data Copyright © 2008 Olsen Solutions LLC
    10. 10. Using Metrics to Optimize the Equation of your Business <ul><li>What are the metrics for your business? </li></ul><ul><li>Where is current value for each metric? </li></ul><ul><li>How many resources to “move” each metric? </li></ul><ul><ul><li>Developer-hours, time, money </li></ul></ul><ul><li>Which metrics have highest ROI opportunities? </li></ul>Copyright © 2008 Olsen Solutions LLC Return Investment Return Investment Return Investment Metric A Good ROI Metric B Bad ROI Metric C Great ROI
    11. 11. Case Study: Optimizing Viral Loop Metrics <ul><li>Which metric has highest ROI opportunity? </li></ul>Copyright © 2008 Olsen Solutions LLC Active Users Prospective Users Invite Click Succeed Invite click-through rate Conversion rate Don’t Click Fail Invites per sender % of users sending invites <ul><li>Multiplied together, these metrics determine your viral ratio </li></ul>Users % of users who are active = 15% = 2.3 = 85% Registration Process
    12. 12. Maximum Upside Potential of a Metric Copyright © 2008 Olsen Solutions LLC 0 100% 0 100% 0 ? Registration conversion rate % of users sending invitations Avg # of invites sent per sender 2.3 85% 15% Max possible improvement 0.15 / 0.85 = 18% 0.85 / 0.15 = 570% ? / 2.3 = ?%
    13. 13. Copyright © 2008 Olsen Solutions LLC Doubling Number of Invitations Sent per Sender by Adding Address Book Importer
    14. 14. Optimizing Customer Value Creation with Metrics Copyright © 2008 Olsen Solutions LLC
    15. 15. The Fuzzy Math of Customer Value <ul><li>You create customer value with a product that </li></ul><ul><ul><li>Satisfies customers’ needs </li></ul></ul><ul><ul><li>Is easy to use </li></ul></ul><ul><ul><li>Has a good price </li></ul></ul><ul><ul><li>Is better than other alternatives </li></ul></ul><ul><li>Customer value = customer satisfaction </li></ul><ul><li>Not fuzzy: # of users and frequency of use </li></ul><ul><li>Understanding user needs and satisfaction </li></ul><ul><ul><li>Quantitative: site analytics, usage metrics, surveys </li></ul></ul><ul><ul><li>Qualitative: support emails, usability sessions (best) </li></ul></ul>Copyright © 2008 Olsen Solutions LLC
    16. 16. What is Your Value Proposition? <ul><li>Which user benefits are most important? </li></ul><ul><li>How well does your product deliver? </li></ul>Copyright © 2008 Olsen Solutions LLC Benefit 1 Importance to User Current User Satisfaction Benefit 2 User Benefit Upside Potential Benefit 4 Benefit 5 Benefit 6 High High Med Low Low Low High – + Doesn’t matter Benefit 3 Low opportunity opportunity ? ?
    17. 17. Analyzing Product Ideas by ROI Copyright © 2008 Olsen Solutions LLC Investment (developer-weeks) Return (Value Created) Idea C Idea B Idea D Idea A Idea F 1 1 2 3 4 2 3 4 ?
    18. 18. Prioritizing Product Ideas by ROI Copyright © 2008 Olsen Solutions LLC Investment (developer-weeks) Return (Value Created) Idea B Idea C Idea A 1 1 2 3 4 2 3 4 5 6 7 8 5
    19. 19. The UI Design Iceberg Copyright © 2008 Olsen Solutions LLC Visual Design Interaction Design Information Architecture Conceptual Design Recommended reading: Jesse James Garrett’s “Elements of User Experience” chart, free at www.jjg.net What most people see and react to What good PMs and Designers think about
    20. 20. Approaching UI Design Analytically <ul><li>Typical UI design question: </li></ul><ul><li>“ When using web pages, do users scroll down?” </li></ul><ul><ul><li>- Yes </li></ul></ul><ul><ul><li>- No </li></ul></ul>Copyright © 2008 Olsen Solutions LLC <ul><li>UI questions are never yes/no! (not binary) </li></ul><ul><li>Should ask: “What percentage of users …?” </li></ul><ul><li>UI changes impact your metrics </li></ul><ul><ul><li>Impact can be positive, negative, small, large </li></ul></ul><ul><ul><li>Seek high-ROI UI changes </li></ul></ul>
    21. 21. Put Key Conversion Actions Above The Fold Copyright © 2008 Olsen Solutions LLC Landing Page A Landing Page B Key conversion action is above the fold Key conversion action is below the fold The Fold
    22. 22. The Fold Isn’t Binary Either Copyright © 2008 Olsen Solutions LLC 600 px 768 px 1024 px Data courtesy of ClickTale Free trial at www.clicktale.com % of Users The chrome steals about 170 pixels
    23. 23. Case Study: Account Signup Process Redesign Copyright © 2008 Olsen Solutions LLC % of Users Sign in / Registration Account Type Cash vs. Margin 5 Partner Pages 3 Partner Pages Biggest drop
    24. 24. Analysis of Sign In/Registration Flow Copyright © 2008 Olsen Solutions LLC Open Account Sign in Account Selection Register 56% 44% Forget Password Registration Process 45% drop off (20% of total) 36% overall drop off for this step 70% (32% of Total) 17% drop off (10% of total) 20% drop off (6% of total) 30% (14% of Total) 80% (26% of Total) 55% (24% of Total) 64% of Total Change Password 83% (46% of Total)
    25. 25. Redesigned User Flow Improved Registration Conversion Rate Copyright © 2008 Olsen Solutions LLC 37% improvement in conversion rate Released New Design
    26. 26. Summary: Building the DNA of a Killer App <ul><li>Define what success means </li></ul><ul><ul><li>Customer value proposition </li></ul></ul><ul><ul><li>Equation of your business </li></ul></ul><ul><li>Instrument your site and track key metrics </li></ul><ul><li>Identify opportunities and prioritize by ROI </li></ul><ul><li>Launch, learn, and iterate </li></ul>Copyright © 2008 Olsen Solutions LLC
    27. 27. [email_address] Questions? www.yourversion.com
    28. 28. Appendix Copyright © 2008 Olsen Solutions LLC
    29. 29. Taking Optimization to The Next Level <ul><li>Multivariate testing, aka A/B testing </li></ul><ul><li>New challenges emerge </li></ul><ul><ul><li>Are we at local vs. global optimum? </li></ul></ul><ul><ul><li>How do we identify top new ideas to test? (creativity and innovation) </li></ul></ul><ul><ul><li>Are there assumptions we should challenge? </li></ul></ul><ul><ul><li>Are we missing the forest for the trees? </li></ul></ul>Copyright © 2008 Olsen Solutions LLC
    30. 30. Olsen’s Hierarchy of Web User Needs (adapted from Maslow) Copyright © 2008 Olsen Solutions LLC Is the site up when I want to use it? Is the site fast enough? Does the functionality work? How compelling and easy to use is the functionality? Customer’s Perspective What does it mean to us? Uptime Page Load Time Absence of Bugs Feature Set Usability & Design Decreasing Dissatisfaction Increasing Satisfaction
    31. 31. Kano Model: User Needs & Satisfaction Copyright © 2008 Olsen Solutions LLC User Satisfaction User Dissatisfaction Performance (more is better) Delighter (wow) Need not met Need fully met Must Have Needs & features migrate over time
    32. 32. Importance vs. Satisfaction Ask Users to Rate for Each Feature Copyright © 2008 Olsen Solutions LLC Recommended reading: “What Customers Want” by Anthony Ulwick Bad Great
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×