Your SlideShare is downloading. ×
0
Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen
Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen
Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen
Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen
Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen
Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen
Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen
Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen
Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen
Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen
Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen
Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen
Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen
Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen
Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen
Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen
Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen
Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen
Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen
Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen
Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen
Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen
Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen
Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen
Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen
Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen
Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen
Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen
Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen
Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen
Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen
Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen
Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen
Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen
Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen
Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen
Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen
Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen
Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen
Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen

37,739

Published on

Best practices in using metrics to optimize your web product. I gave this webinar on Dec 17, 2008, as part of FeaturePlan's series "The Product Management View".

Best practices in using metrics to optimize your web product. I gave this webinar on Dec 17, 2008, as part of FeaturePlan's series "The Product Management View".

Published in: Business, Technology
7 Comments
126 Likes
Statistics
Notes
  • If you enjoyed this SlideShare, I'll be teaching a 5-hour workshop in San Francisco on October 10, 2013: 'Building Great Products the Lean Startup Way'. More info at http://bit.ly/163BfCE Hope to see you there!
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Thank you for this presentation. I found it succinct and thorough. Nice work!
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Hi. If you like my presentations, I encourage you to check out my startup YourVersion http://www.yourversion.com where I've been applying the product management principles that I talk about in my SlideShares.
    YourVersion is 'Pandora for your real-time web content': a personalized discovery engine that brings you the latest relevant news, blogs, tweets, and videos tailored to your specific interests.
    We launched to the public at the 2009 TechCrunch50 Conference where we won the People’s Choice Award.

    In addition to the website, we also offer a free iPad app http://bit.ly/yv-hd, a free iPhone app http://bit.ly/yv-app, and a free Android app so you can discover on the go. We also offer free browser extensions for Firefox, Chrome, and Safari browsers. Learn more at http://yourversion.com

    I'd love to hear any feedback or suggestions you have on our products.

    Thanks!

    Dan Olsen
    CEO & Cofounder, YourVersion
    http://www.yourversion.com
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • slideshare.com is the best site to find study matarial .
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • 連結策略的網站產品設計,有工具和數字,太棒了!
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total Views
37,739
On Slideshare
0
From Embeds
0
Number of Embeds
11
Actions
Shares
0
Downloads
3,205
Comments
7
Likes
126
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Product Management by Numbers: Using Metrics to Optimize your Product Dan Olsen CEO & Founder, YourVersion Dec 17, 2008 1 Copyright © 2008 Olsen Solutions LLC Copyright 
  • 2. Using Metrics to Optimize your Product Product Management’s Role Integrating metrics & optimization Using metrics to optimize value creation Business Value Customer Value UI Design Illustrate with real world examples Copyright © 2008 Olsen Solutions LLC
  • 3. My Background Education BS, Electrical Engineering, Northwestern MS, Industrial Engineering, Virginia Tech MBA, Stanford PHP, MySQL, JavaScript, XHTML, CSS, UI design 17 years of Product Management Experience Managed submarine design for 5 years 5 years at Intuit, led Quicken Product Management Led Product Management at Friendster Olsen Solutions LLC, PM consultant for start‐ups CEO & Founder of YourVersion, Social Discovery start‐up
  • 4. Product Management’s  Role Copyright © 2008 Olsen Solutions LLC
  • 5. A Product Manager by any Other  Name Would Smell as Sweet Product managers are sometimes called Product marketing manager Program manager Project manager Label and definition of role can vary Based on industry or company Based on B2C (consumer) vs. B2B (enterprise) Based on stage of company Can be area of responsibility vs. actual position Copyright © 2008 Olsen Solutions LLC
  • 6. Product Management is Critical Link in Value Creation Market Product  Development  • Current  Management Team customers • Prospective customers • Competitors Copyright © 2008 Olsen Solutions LLC
  • 7. A Process View of Product Management “Inbound” Product “Outbound” Management Product Long Business Product Management Term Strategy Strategy Market/ Sell Short Business Product Product Term Objectives Objectives Development Service/ Support Copyright © 2008 Olsen Solutions LLC
  • 8. PM’s Job: A Successful Product Know market and customer better than anyone else Translate business objectives and customer needs  into product requirements Be the clearinghouse for all product ideas  Identify, plan & prioritize product ideas to maximize  ROI on engineering resources Work with team to design and build a great product Evangelize product within and outside the company Keep the product team motivated Copyright © 2008 Olsen Solutions LLC
  • 9. Integrating Metrics and  Optimization into your  Product Process Copyright © 2008 Olsen Solutions LLC
  • 10. Adding Metrics and Optimization to  your Product Process Site Level Business Product Prioritized  Plan Objectives Objectives Feature List Scoping Feature  Level Requirements  Design & Design Code Test Launch Develop Metrics & User  Optimize Feedback Copyright © 2008 Olsen Solutions LLC
  • 11. Optimization through Iteration: Continuous Improvement Measure the metric Analyze Learning the metric Gaining knowledge: • Market Identify top  • Customer opportunities to improve • Domain • Usability Design & develop   the enhancement Launch the enhancement Copyright © 2008 Olsen Solutions LLC
  • 12. Optimizing Business Value Creation  with Metrics Copyright © 2008 Olsen Solutions LLC
  • 13. Approaching Business as an  Optimization Exercise Given reality as it exists today, optimize our business results subject to our resource constraints. Copyright © 2008 Olsen Solutions LLC
  • 14. Define the Equation of your Business “Peeling the Onion” Profit = Revenue ‐ Cost Unique Visitors x  Ad Revenue per Visitor Impressions/Visitor x  Effective CPM / 1000 Visits/Visitor  x  Pageviews/Visit  x  Impressions/PV New Visitors + Returning Visitors Invited Visitors + Uninvited Visitors # of Users Sending Invites  x  Invites Sent/User  x  Invite Click‐through Rate Copyright © 2008 Olsen Solutions LLC
  • 15. How to Track Your Metrics Track each metric as daily time series Unique  Page  Ad  New User  … Date Visitors views Revenue Sign‐ups 4/24/08 10,100 29,600 25 490 4/25/08 10,500 27,100 24 480 … Create ratios from primary metrics:  X / Y Example: How good is your registration page? Okay: # of registered users per day Better: registration conversion rate = # registered users / # uniques to reg page Copyright © 2008 Olsen Solutions LLC
  • 16. Sample Signup Page Yield Data Daily Signup Page Yield vs. Time New Registered Users divided by Unique Visitors to Signup Page 100% 90% 80% Daily Signup Page Yield 70% 60% 50% 40% 30% Started requiring registration 20% Changed Added questions messaging to signup page 10% 0% 1/31 2/14 2/28 3/14 3/28 4/11 4/25 5/9 5/23 6/6 6/20 7/4 7/18 8/1 8/15 8/29 9/12 9/26 10/1 0 Copyright © 2008 Olsen Solutions LLC
  • 17. Using Metrics to Optimize the  Equation of your Business What are the metrics for your business? Where is current value for each metric?  How many resources to “move” each metric? Developer‐hours, time, money Which metrics have highest ROI opportunities? Metric A Metric B Metric C Good ROI Bad ROI Great ROI Return Return Return Investment Investment Investment Copyright © 2008 Olsen Solutions LLC
  • 18. Case Study: Optimizing Viral Loop Metrics % of users sending = 15% Invites per Invite invites sender = 2.3 click-through rate Active Invite Prospective Click Registration Fail Users Users Process % of users who are Succeed Don’t active Click Conversion rate = 85% Users • Multiplied together, these metrics determine your viral ratio • Which metric has highest ROI opportunity? Copyright © 2008 Olsen Solutions LLC
  • 19. Maximum Upside Potential of a Metric ? 100% 100% 85% 15% 2.3 0 0 0 Registration % of users sending Avg # of invites conversion rate invitations sent per sender Max possible 0.15 / 0.85 = 18% 0.85 / 0.15 = 570% ? / 2.3 = ?% improvement Copyright © 2008 Olsen Solutions LLC
  • 20. Doubling Number of Invitations Sent per Sender by  Adding Address Book Importer Copyright © 2008 Olsen Solutions LLC
  • 21. Optimizing Customer Value Creation  with Metrics Copyright © 2008 Olsen Solutions LLC
  • 22. The Fuzzy Math of Customer Value You create customer value with a product that Satisfies customers’ needs Is easy to use Has a good price Is better than other alternatives Not fuzzy: # of users and frequency of use Applying metrics approach to customer value Need framework of user needs & satisfaction Quantitative: site analytics, usage metrics, surveys Qualitative: 1‐on‐1 user feedback sessions  are best Copyright © 2008 Olsen Solutions LLC
  • 23. Olsen’s Hierarchy of Web User Needs (adapted from Maslow) Customer’s Perspective What does it mean to us? How easy to use is it? Usability & Design Satisfaction Increasing Does the functionality Feature Set meet my needs? Does the functionality work? Absence of Bugs Dissatisfaction Decreasing Is the site fast enough? Page Load Time Is the site up when I want to use it? Uptime Copyright © 2008 Olsen Solutions LLC
  • 24. Kano Model: User Needs & Satisfaction User Satisfaction Delighter (wow) Performance  (more is better) Need Need not met fully met Must Have Needs & features  migrate over time User Dissatisfaction Copyright © 2008 Olsen Solutions LLC
  • 25. Importance vs. Satisfaction Ask Users to Rate for Each Feature 100 98 Great 95 84 87 90 Bad 86 85 79 84 55 70 80 Importance 80 75 72 80 70 75 65 60 55 41 50 40 50 60 70 80 90 100 Satisfaction Recommended reading: “What  Customers Want” by Anthony Ulwick Copyright © 2008 Olsen Solutions LLC
  • 26. What is Your Value Proposition? Which user benefits are most important? How well does your product deliver? Importance Current User Upside User Benefit to User Satisfaction Potential Low High – + Benefit 1 High Benefit 2 High Benefit 3 Low ? opportunity Benefit 4 Med Benefit 5 Low ? opportunity Benefit 6 Low Doesn’t matter Copyright © 2008 Olsen Solutions LLC
  • 27. Return (Value Created) Analyzing Product Ideas by ROI ? 4 Idea D 3 Idea A Idea B 2 Idea C 1 Idea F 1 2 3 4 Investment (developer-weeks) Copyright © 2008 Olsen Solutions LLC
  • 28. Return (Value Created) Prioritizing Product Ideas by ROI Idea C 8 7 Idea B 6 5 4 Idea A 3 2 1 1 2 3 4 5 Investment (developer-weeks) Copyright © 2008 Olsen Solutions LLC
  • 29. The UI Design Iceberg What most people see and react to Visual Design What good PMs and Designers Interaction think about Design Information Architecture Conceptual Design Recommended reading: Jesse James Garrett’s “Elements of User Experience” chart, free at www.jjg.net Copyright © 2008 Olsen Solutions LLC
  • 30. Approaching UI Design Analytically Typical UI design question: “When using web pages, do users scroll down?” ‐ Yes ‐ No UI questions are never yes/no! (not binary) Should ask: “What percentage of users …?” UI changes impact your metrics Impact can be positive, negative, small, large Seek high‐ROI UI changes Copyright © 2008 Olsen Solutions LLC
  • 31. Put Key Conversion Actions Above The Fold Landing Page A Landing Page B The Fold Key conversion action is above the fold Key conversion action is below the fold Copyright © 2008 Olsen Solutions LLC
  • 32. The Fold: “Chrome” Takes Away Pixels Only 768 600 pixels pixels 1024 pixels
  • 33. The Fold Isn’t Binary Either 768 px The chrome steals  % of Users about 170 pixels 600 px 1024 px Data courtesy of ClickTale Free trial at www.clicktale.com Copyright © 2008 Olsen Solutions LLC
  • 34. Analyze User Screen Height Distribution to Select Design Height for Key Pages 100% 600, 100% 768, 92% 90% % of Users with Given (or Higher) Screen Height 80% 70% Going above 60% 768 px = big drop  50% 40% 772, 38% 864, 29% 30% 807, 29% 1024, 20% 866, 25% 20% 1000, 20% 10% 1030, 5% 0% 600 650 700 750 800 850 900 950 1000 1050 1100 Page Design Height (pixels) Monitor Screen Height (pixels)
  • 35. Case Study: Account Signup Process Redesign 100% 100% Biggest  80% drop % of Users 62.3% 58.8% 60% 50.9% 40% 34.4% 32.7% 20% 0% Sign in / Account  Cash vs. 5 Partner 3 Partner Registration Type Margin Pages Pages Copyright © 2008 Olsen Solutions LLC
  • 36. Analysis of Sign In/Registration Flow Open Account 55% 44% Register Registration  (24% of Total) Process 45% drop off 64% (20% of total) of Total Account  36% overall  Selection 83% 30% drop off for  56% (46% of Total) (14% of Total) this step Sign in Forget  70% Change  80% Password (32% of Total) Password (26% of Total) 17% drop off  20% drop off (10% of total) (6% of total) Copyright © 2008 Olsen Solutions LLC
  • 37. Redesigned User Flow Improved  Registration Conversion Rate Abandonment Rate (7 Day Moving Average) Steps 1-2 80% 70% Abandonment Rate (7 Day Moving Average) 60% 50% 40% 30% Released 37% improvement New Design in conversion rate 20% 10% 0% 10/14/02 10/21/02 10/28/02 11/11/02 11/18/02 11/25/02 12/16/02 12/23/02 12/30/02 10/7/02 11/4/02 12/2/02 12/9/02 1/13/03 1/20/03 1/6/03 Copyright © 2008 Olsen Solutions LLC
  • 38. Product Management by Numbers Cheat Sheet Define what success means Customer value proposition Equation of your business Instrument your site and track key metrics Identify opportunities and prioritize by ROI Launch, learn, and iterate Copyright © 2008 Olsen Solutions LLC
  • 39. YourVersion proactively discovers relevant web content for you based on your interests. Discover Bookmark Share
  • 40. Questions? www.yourversion.com dan@yourversion.com 40 Copyright © 2008 Olsen Solutions LLC Copyright 

×