Early stage-web-product-management-by-dan-olsen-090728040916-phpapp02

799 views

Published on

Published in: Technology, Business
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
799
On SlideShare
0
From Embeds
0
Number of Embeds
11
Actions
Shares
0
Downloads
47
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Early stage-web-product-management-by-dan-olsen-090728040916-phpapp02

  1. 1. Early Stage Web Product ManagementDan OlsenCEO, YourVersionJuly 24, 2009 Copyright © 2009 YourVersion
  2. 2. What You Wanted to Hear About 1 2 3 4 5 6 7 8 9 10 Usability Understanding user needs Prioritizing features Translating learnings from metrics into action Maximizing ROI on development resources Setting up and tracking the right metrics UI design Copyright © 2009 YourVersion
  3. 3. AgendaUnderstanding customer needsPrioritizing, scoping, and maximizing ROI on engineering resourcesUI designUsability testingUsing metrics for optimization Will post slides to slideshare.net/dan_o Copyright © 2009 YourVersion
  4. 4. My BackgroundEducation BS, Electrical Engineering, Northwestern MS, Industrial Engineering, Virginia Tech MBA, Stanford PHP, MySQL, JavaScript, XHTML, CSS, UI design18 years of Product Management Experience Managed submarine design for 5 years 5 years at Intuit, led Quicken Product Management Led Product Management at Friendster Olsen Solutions LLC, PM consultant for startups CEO & Founder of YourVersion, real‐time discovery startup Copyright © 2009 YourVersion
  5. 5. For an Early Stage Startup, Which is More Important? or Qualitative  Quantitative Information? Information? “When you’re Small,  start with Qual!” Copyright © 2009 YourVersion
  6. 6. How is developing Early Stage Products  Different from Later Stage Products?With an early stage product, you haveWAY MORE uncertainty about: Who your target customers really are Which customer needs you should address How to best meet those needs What design works bestThese are qualitative learnings/decisionsQuantitative is also valuable (later) Copyright © 2009 YourVersion
  7. 7. What is Product Management? Copyright © 2009 YourVersion
  8. 8. Product Management is Critical Link in Value CreationMarket Product  Development • Current  Management Team customers• Prospective customers• Competitors Copyright © 2009 YourVersion
  9. 9. The Product Manager’s Job: A Successful ProductBe the expert on the market and the customerTranslate business objectives and customer needs into product requirementsBe the clearinghouse for all product ideas Work with team to design & build great productDefine and track key metricsIdentify, plan & prioritize product ideas to maximize ROI on engineering resources Copyright © 2009 YourVersion
  10. 10. What’s the Formula for a Winning Product?A product that: Meets customers’ needs Is better than other alternatives Is easy to use Has a good value/price Copyright © 2009 YourVersion
  11. 11. Problem Space vs. Solution Space Problem Space Solution Space A customer problem,  A specific  need, or benefit that the  implementation to  product should address address the need or  A product requirement product requirementExample: Ability to write in space  NASA: space pen (zero gravity) ($1 M R&D cost) Russians: pencil Copyright © 2009 YourVersion
  12. 12. Problem Space vs. Solution Space Product LevelProblem Space Solution Space (user benefit) (product) Pen and Prepare paper my taxes TurboTax File my taxes TaxCut Copyright © 2009 YourVersion
  13. 13. Problem Space vs. Solution Space Feature LevelProblem Space Solution Space (user benefit) (feature) Gmail Make it easy importer to share a link with my friends Design Design Design #1 #2 #3 Preview with  User can edit  Allow me to Design checkboxes before import reuse my email #1 No No contacts #2 Yes No #3 Yes Yes Copyright © 2009 YourVersion
  14. 14. Types of Customer BenefitsFunctional Benefits Allow the user to do something they couldn’t do before Deliver a benefit more quickly, conveniently, or cheaply Booking travel more easily (travel sites) Talking for less money (VOIP software)Emotional Benefits Control (Quicken) Feeling informed (news sites) Confidence / security (anti‐virus software) Enjoyment / entertainment (YouTube) Self‐expression (MySpace, Facebook) Copyright © 2009 YourVersion
  15. 15. The Customer Benefits “Ladder” …which means one less thing to  Higher‐level worry about in my hectic life benefit(more abstract) …which makes me feel more in  control of my finances …which gives me a clear picture  of how much money I have Lower‐level Quicken makes it easy for me  benefit to balance my checkbook(more specific) Copyright © 2009 YourVersion
  16. 16. How Do You Prioritize User Benefits  and Product Features?Need a framework for prioritization Which user benefits should you address? Which product features to build (or improve)?Importance vs. Satisfaction Importance of user need (problem space) Satisfaction with how well a product meets the  user’s need (solution space)Opportunity =High Importance need with low Satisfaction Copyright © 2009 YourVersion
  17. 17. What is Your Value Proposition? Which user benefits are you going to provide? How will your product be better than rest? Importance Current User UpsideUser Benefit to User Satisfaction Potential Low High – +Benefit 1 HighBenefit 2 HighBenefit 3 Low ? opportunityBenefit 4 MedBenefit 5 Low ? opportunityBenefit 6 Low Doesn’t matter Copyright © 2009 YourVersion
  18. 18. Importance vs. Satisfaction Ask Users to Rate for Each Feature 100 98 Great 95 84 87 90 Bad 86 85 79 84 55 70 80 Importance 80 75 72 80 70 75 65 60 55 41 50 40 50 60 70 80 90 100 SatisfactionRecommended reading: “What Customers Want” by Anthony Ulwick Copyright © 2009 YourVersion
  19. 19. Kano Model: User Needs & Satisfaction User Satisfaction Delighter (wow) Performance  (more is better) Need Neednot met fully met Must Have Needs & features  migrate over time User Dissatisfaction Copyright © 2009 YourVersion
  20. 20. Olsen’s Hierarchy of Web User Needs (adapted from Maslow) Customer’s Perspective What does it mean to us? How easy to use is it? Usability & Design Satisfaction Increasing Does the functionality Feature Set meet my needs? Does the functionality work? Absence of Bugs Dissatisfaction Decreasing Is the site fast enough? Page Load TimeIs the site up when I want to use it? Uptime Copyright © 2009 YourVersion
  21. 21. Prioritization and ScopeCustomer value is only half the equationHow much engineering effort will it take?Need to consider value and effort (ROI)Ruthlessly prioritize: rank orderBe deliberate about scope & keep it small It’s easy to try to do too much Strategy = deciding what you’re NOT doing Break features down into smaller chunks Smaller scope → faster iterations → better Copyright © 2009 YourVersion
  22. 22. Have to Prioritize Across Multiple  Dimensions At The Same Time Ease of UseCustomer Value Quality Functionality Customer  Understanding Time Copyright © 2009 YourVersion
  23. 23. Prioritizing Product Ideas by ROI ?Return (Value Created) 4 Idea D 3 Idea A Idea B 2 Idea C 1 Idea F 1 2 3 4 Investment (developer‐weeks) Copyright © 2009 YourVersion
  24. 24. User Benefits vs. Ease of UseQ: If two products equally deliver the exact same user benefits, which product is better?A: The product that’s easier to use‘Ease of use’ provides benefits Saves time Reduces cognitive load Reduces frustrationNot many companies excel at UI design‘Ease of use’ can be differentiator Copyright © 2009 YourVersion
  25. 25. The Design Gap at Most Start‐upsLevel Define Design Code 1 Product Mgmt Engineering 2 Product Mgmt Engineering Product Mgmt Engineering 3 PM Eng UI 4 PM Eng Copyright © 2009 YourVersion
  26. 26. The UI Design Iceberg What most people see and react to Visual Design What good PMs and Designers Interaction think about Design Information Architecture Conceptual DesignRecommended reading: Jesse James Garrett’s“Elements of User Experience” chart, free at www.jjg.net Copyright © 2009 YourVersion
  27. 27. Elements of User Interface DesignConsists of Three Distinct Elements: Information Architecture Structure and layout at both site and page level How site is structured (sitemap) How site information is organized (site layout) How each page is organized (page layout) Interaction Design How user and product interact with one another User flows (e.g., navigation across multiple pages) User input (e.g., controls and form design) Visual Design “How it looks” vs. “What it is”, often called “chrome” Fonts, colors, graphical elements Copyright © 2009 YourVersion
  28. 28. Information ArchitectureDocuments used Sitemap Show how sections of website are organized Show major navigation patterns Wireframes Show the layout of components on a page Does NOT focus on visual design Black & White No graphics Templates for overall website and individual pagesTools:  Visio, OmniGraffle, Axure, Powerpoint, Word, Excel, Photoshop, Balsamiq, WriteMaps, whiteboard Copyright © 2009 YourVersion
  29. 29. Information Architecture Sitemap Copyright © 2009 YourVersion
  30. 30. Information Architecture Sitemap Copyright © 2009 YourVersion
  31. 31. Wireframe Copyright © 2009 YourVersion
  32. 32. Napkin Wireframe Copyright © 2009 YourVersion
  33. 33. Interaction DesignDocuments used Flowchart Combination of Wireframes & FlowchartsTools:  Visio, OmniGraffle, Powerpoint, Photoshop, whiteboardMay build prototype using HTML, jQuery, Ruby on Rails, Flash, or paperUsability testing can help find problems Copyright © 2009 YourVersion
  34. 34. Flowchart showing conditional logic Copyright © 2009 YourVersion
  35. 35. Napkin Flowchart Copyright © 2009 YourVersion
  36. 36. Combo Wireframe & Flowchart Copyright © 2009 YourVersion
  37. 37. Visual DesignDocuments used Mockups (aka comps)Tools used:  Photoshop, Illustrator, FireworksDesigner may also deliver HTML & CSS version (with no back‐end code) Copyright © 2009 YourVersion
  38. 38. Visual Design Mockup Copyright © 2009 YourVersion
  39. 39. Visual Design UI Spec Copyright © 2009 YourVersion
  40. 40. Visual DesignSpecifying Color Palette Copyright © 2009 YourVersion
  41. 41. The Value of Usability TestingCritical to talk with customers 1‐on‐1Gain better understanding of Usability issues with your product Customer needs and problems What alternatives customers are using, pros & cons of each, customer preferences QA: use cases & bugs you haven’t seenReally a “user learning” sessionMake test as real for user as possible Copyright © 2009 YourVersion
  42. 42. “Ramen” Usability for StartupsAnyone can do it!Ingredients: 1 user with their laptop 1 desk 1 person to conduct usability Pen and paper N optional observers N+2 chairs Copyright © 2009 YourVersion
  43. 43. Typical Format for Customer Session5 ‐ 10 min: Ask questions to understand user needs and solutions they currently use30 ‐ 50 min: Usability test Non‐directed as much as possible When necessary, direct user to attempt to  perform a specific task5 ‐ 10 min: Wrap‐up Answer any user questions that came up Point out/explain features you want to highlight Copyright © 2009 YourVersion
  44. 44. Dos & Don’ts of Conducting UsabilityDo Explain to the user: Their usability test will help improve the product Not to worry about hurting your feelings “Think Aloud Protocol” Ask user to attempt the task, then be a fly on the wall Ask non‐leading, open‐ended questions Take notes and review them afterwards for take‐awaysDon’t Ask leading questions “Help” the user or explain the UI (e.g., “click over here”) Respond to user frustration or questions (until test is over) Get defensive Blame the user Copyright © 2009 YourVersion
  45. 45. Usability Case Study: Travel SitesConsider 3 major travel sites Expedia, Travelocity, OrbitzAll 3 try to provide same user benefits Easily find flights that match your schedule Easily find the lowest price flightsUI Design differences make some betterCase study: round trip from San Fran to NYC Want to find best combo of price and airports 3 possible airports for each city 9 possible one‐way airport combinations 81 possible round‐trip airport combinations Copyright © 2009 YourVersion
  46. 46. Expedia:Only One Airport Combo at a Time • Have to  manually  check all 9  combos •3 clicks to  change airport •Then wait for  new results •24 clicks + 8 page reloads  to see all 9  combos Copyright © 2009 YourVersion
  47. 47. Travelocity:Can Select which Airports to Include Copyright © 2009 YourVersion
  48. 48. Travelocity:Can only view results 1 combo at a time • Clicking button brings up  results for this combo •Problem: browser Back  button loses other  airports! •Have to go through  ‘Change Search’ process = 9 clicks + 4 page reloads  for each combo •72 clicks + 32 page loads  to see other 8 combos Copyright © 2009 YourVersion
  49. 49. Orbitz:Binary choices to include nearby airports • Good news: all combos  appear in results •80‐mile radius might  include airports I don’t  want (3 for SFO, 5 for NYC)  but turned out not to be  an issue Copyright © 2009 YourVersion
  50. 50. All 3 Travel Sites have Identical “Airline vs. Number of Stops” Price GridsExpediaOrbitzTravelocity Copyright © 2009 YourVersion
  51. 51. Options for Sorting Flight Results WebsiteCan Sort By Expedia Travelocity OrbitzAirline YDeparture Time Y Y YArrival Time Y YTravel Time Y Y YPrice Y Y Y Copyright © 2009 YourVersion
  52. 52. Summary Comparison of Travel SitesUser Benefit Expedia Travelocity OrbitzAbility to include other  Yes Yes Yesnearby airportsAbility to pick specific  High High Lownearby airports (by changing) (can pre‐select)Ease of seeing results  Med Low Highfor multiple airportsEase of trading off  Low High Medairport combos vs. priceAirline vs. Number of  Yes Yes YesConnections Price GridFlight Results Sorting  Med High LowOptionsOverall ability to easily  Med Low Highfind best airport combo Copyright © 2009 YourVersion
  53. 53. And now,for something completely  different… Quantitative! (finally) Copyright © 2009 YourVersion
  54. 54. Approaching Business as an  Optimization ExerciseGiven reality as it exists today,optimize our business resultssubject to our resource constraints. Copyright © 2009 YourVersion
  55. 55. Define the Equation of your Business “Peeling the Onion”Advertising Business Model:Profit = Revenue ‐ Cost Unique Visitors x  Ad Revenue per Visitor Impressions/Visitor x  Effective CPM / 1000 Visits/Visitor  x  Pageviews/Visit  x  Impressions/PV New Visitors + Returning Visitors Invited Visitors + Uninvited Visitors # of Users Sending Invites  x  Invites Sent/User  x  Invite Conversion Rate Copyright © 2009 YourVersion
  56. 56. Equation of your Business Subscription Business ModelProfit = Revenue ‐ Cost Paying Users x  Revenue per Paying User New Paying Users +  Repeat Paying Users Trial Users x  Conv Rate Previous Paying Users  x  ( 1 – Cancellation Rate ) ( SEO Visitors + SEM Visitors + Viral Visitors )  x  Trial Conversion Rate Copyright © 2009 YourVersion
  57. 57. How to Track Your MetricsTrack each metric as daily time series Unique  Page  Ad  New User  … Date Visitors views Revenue Sign‐ups 4/24/08 10,100 29,600 25 490 4/25/08 10,500 27,100 24 480 …Create ratios from primary metrics:  X / Y Example: How good is your registration page? Okay: # of registered users per day Better: registration conversion rate = # registered users / # uniques to reg page Copyright © 2009 YourVersion
  58. 58. Sample Signup Page Yield Data Daily Signup Page Yield vs. Time New Registered Users divided by Unique Visitors to Signup Page 100% 90% 80%Daily Signup Page Yield 70% 60% 50% 40% 30% Started requiring registration 20% Changed Added questions messaging to signup page 10% 0% 1/31 2/14 2/28 3/14 3/28 4/11 4/25 5/9 5/23 6/6 6/20 7/4 7/18 8/1 8/15 8/29 9/12 9/26 10/1 0 Copyright © 2009 YourVersion
  59. 59. Identifying the  “Critical Few” Metrics What are the metrics for your business? Where is current value for each metric?  How many resources to “move” each metric? Developer‐hours, time, money Which metrics have highest ROI opportunities? Metric A Metric B Metric C Good ROI Bad ROI Great ROIReturn Return Return Investment Investment Investment Copyright © 2009 YourVersion
  60. 60. Using Web Analytics Tools to  Understand Your UsersTracking visitors and trafficSeeing where users are clickingMeasuring key conversionsMonitoring user feedback Copyright © 2009 YourVersion
  61. 61. Basic Tracking of Traffic: Google Analytics•Unique visitors•New vs. returning•Pageviews•Time on site•Top referrers•Top geos Copyright © 2009 YourVersion
  62. 62. Seeing Where Users are Clicking:  CrazyEgg Heatmap•Shows Click Density•Color indicates % of clicks•See which links perform best•See impact of UI changes Copyright © 2009 YourVersion
  63. 63. Measuring Key Conversions: Conversion Funnel•Tie user actions to business goals•Instrument key steps in user flow•See where users are dropping off•Quantify improvement from changes Copyright © 2009 YourVersion
  64. 64. Monitoring User Feedback  Quantitatively with Kampyle Unobtrusive Solicitation Average Grade over time Simple popupAverage Grade for each page  on your site Copyright © 2009 YourVersion
  65. 65. Approaching UI Design Analytically Typical UI design question: “When using web pages, do users scroll down?” ‐ Yes ‐ No UI questions are never yes/no! (not binary) Should ask: “What percentage of users …?” UI changes impact your metrics Impact can be positive, negative, small, large Seek high‐ROI UI changes Copyright © 2009 YourVersion
  66. 66. The Fold: “Chrome” Takes Away Pixels Only 768 600 pixels pixels 1024 pixels Copyright © 2009 YourVersion
  67. 67. Put Key Conversion Actions Above The Fold Landing Page A Landing Page B The FoldKey conversion actionis above the fold Key conversion action is below the fold Copyright © 2009 Olsen Solutions LLC Copyright © 2009 YourVersion
  68. 68. The Fold Isn’t Binary Either 768 px The chrome steals % of Users about 170 pixels 600 px 1024 px Data courtesy of ClickTale Free trial at www.clicktale.com Copyright © 2009 YourVersion
  69. 69. Analyze User Screen Height Distribution to Select Design Height for Key Pages 100% 600, 100% 768, 92% 90% % of Users with Given (or Higher) Screen Height 80% 70% Going above 60% 768 px = big drop  50% 40% 772, 38% 864, 29% 30% 807, 29% 1024, 20% 866, 25% 20% 1000, 20% 10% 1030, 5% 0% 600 650 700 750 800 850 900 950 1000 1050 1100 Page Design Height (pixels) Monitor Screen Height (pixels) Copyright © 2009 YourVersion
  70. 70. Marrying UI Design with MetricsThink of “Ease of use”as distinct from the functionalityUser Interface elements matter Position Layout Size Color Font Text copy Interaction design Navigation Copyright © 2009 YourVersion
  71. 71. Automated Multivariate Testing: Google Website Optimizer Copyright © 2009 YourVersion
  72. 72. Case Studies:Translating Metrics Into Action Copyright © 2009 YourVersion
  73. 73. Case Study 1: Quicken Brokerage Optimizing Sign In/Registration Flow 100% 100% Biggest  80% drop% of Users 62.3% 58.8% 60% 50.9% 40% 34.4% 32.7% 20% 0% Sign in / Account  Cash vs. 5 Partner 3 Partner Registration Type Margin Pages Pages Copyright © 2009 YourVersion
  74. 74. Mapping the Flow to See Where Users Were Dropping Off OpenAccount 55% 44% Register Registration  (24% of Total) Process 45% drop off 64% (20% of total) of Total Account  36% overall  Selection 83% 30% drop off for  56% (46% of Total) (14% of Total) this step Sign in Forget  70% Change  80% Password (32% of Total) Password (26% of Total) 17% drop off  20% drop off (10% of total) (6% of total) Copyright © 2009 YourVersion
  75. 75. Redesigned User Flow Improved Registration Conversion Rate 37% Released 37% improvement New Design in conversion rate Copyright © 2009 YourVersion
  76. 76. Case Study 2: Friendster Optimizing Viral Growth % of users sending = 15% Invites per Invite invites sender = 2.3 click-through rate Active Invite Prospective Click Registration Fail Users Users Process% of users who are Succeed Don’t active Click Conversion rate = 85% Users• Multiplied together, these metrics determine your viral ratio• Which metric has highest ROI opportunity? Copyright © 2009 YourVersion
  77. 77. The Upside Potential of a Metric ? 100% 100% 85% 15% 2.3 0 0 0 Registration  % of users sending  Avg # of invites  Process Yield invitations sent per senderMax possible  0.15 / 0.85 = 18% 0.85 / 0.15 = 570% ? / 2.3 = ?%improvement Copyright © 2009 YourVersion
  78. 78. Okay, so how can we improve the metric? How do we increase the average number of  invites being sent out per sender? For each idea: What’s the expected benefit? (how much will it  improve the metric?) What’s the expected cost? (how many engineer‐ hours will it take?) You want to identify highest benefit/cost idea Copyright © 2009 YourVersion
  79. 79. Case Study 2: FriendsterDoubled Number of Invitations Sent per Sender Copyright © 2009 YourVersion
  80. 80. ContinuousImprovement Copyright © 2009 YourVersion
  81. 81. Adding Metrics and Optimization to  your Product Process Site Level Business Product Prioritized Plan Objectives Objectives Feature List Scoping Feature  Level Requirements Design & Design Code Test LaunchDevelop Metrics & User Optimize Feedback Copyright © 2009 YourVersion
  82. 82. Optimization through Iteration: Continuous Improvement Measure the metric Analyze Learning the metric Gaining knowledge: • Market Identify top  • Customer opportunities to improve • Domain • Usability Design & develop   the enhancement Launch the enhancement Copyright © 2009 YourVersion
  83. 83. Early Stage Product Management  Cheat SheetTruly understand your customer needsRuthlessly prioritize and launch v1Talk with users 1‐on‐1 and get feedbackDefine equation of your businessIdentify and track key metricsIdentify opportunities & prioritize by ROILaunch, learn, and iterate Copyright © 2009 YourVersion
  84. 84. Questions?@danolsendan@yourversion.com www.yourversion.com Copyright © 2009 YourVersion

×