Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Using Feedback to De-risk your Roadmap

234 views

Published on

For more information about UserVoice: www.uservoice.com

Join Richard White, Founder and CEO of UserVoice, as he dives into how to make data driven product decisions to help inform what to build in your product.

Published in: Software
  • Be the first to comment

  • Be the first to like this

Using Feedback to De-risk your Roadmap

  1. 1. Data-Informed Product Decisions Using Feedback to De-risk your Roadmap Richard White, Founder & CEO 1
  2. 2. 2 © Copyright 2016 UserVoice. Confidential and proprietary information. 90% of product management is building the right things in the right order
  3. 3. Step 1 - Define Your Goals Increase Win-Rate? Reduce Churn?
  4. 4. Step 2 - Estimate Impact Against Goals Goal A Goal B Effort Confidence Score Accounts % Account $ % Feature A 0 2 2 50% 0.5 30 15% $300,000 19% Feature B 0 2 2 75% 0.8 3 2% $300,000 19% Feature C 1 1 1 50% 1.0 60 30% $600,000 38% Feature D 2 1 4 90% 0.7 90 45% $900,000 56%
  5. 5. Step 2 - Estimate Engineering Effort Goal A Goal B Effort Confidence Score Accounts % Account $ % Feature A 0 2 2 50% 0.5 30 15% $300,000 19% Feature B 0 2 2 75% 0.8 3 2% $300,000 19% Feature C 1 1 1 50% 1.0 60 30% $600,000 38% Feature D 2 1 4 90% 0.7 90 45% $900,000 56%
  6. 6. Step 3 - Estimate Confidence Goal A Goal B Effort Confidence Score Accounts % Account $ % Feature A 0 2 2 50% 0.5 30 15% $300,000 19% Feature B 0 2 2 75% 0.8 3 2% $300,000 19% Feature C 1 1 1 50% 1.0 60 30% $600,000 38% Feature D 2 1 4 90% 0.7 90 45% $900,000 56%
  7. 7. Step 4 - Score & Rank Goal A Goal B Effort Confidence Score Accounts % Account $ % Feature C 1 1 1 50% 1.0 60 30% $600,000 38% Feature B 0 2 2 75% 0.8 3 2% $300,000 19% Feature D 2 1 4 90% 0.7 90 45% $900,000 56% Feature A 0 2 2 50% 0.5 30 15% $300,000 19% (GOAL A + GOAL B) x CONFIDENCE / EFFORT = SCORE
  8. 8. (REVENUE LIFT + REFERRAL LIFT) x CONFIDENCE EFFORT GOAL A GOAL B
  9. 9. 100% BEHAVIORAL 100% QUALITATIVE
  10. 10. 100% BEHAVIORAL 100% QUALITATIVE Most Companies
  11. 11. 100% BEHAVIORAL 100% QUALITATIVE Most Companies BEHAVIORAL ANALYTICS CUSTOMER FEEDBACK
  12. 12. Step 4 - Score & Rank Goal A Goal B Effort Confidence Score Accounts % Account $ % Feature C 1 1 1 50% 1.0 60 30% $600,000 38% Feature B 0 2 2 75% 0.8 3 2% $300,000 19% Feature D 2 1 4 90% 0.7 90 45% $900,000 56% Feature A 0 2 2 50% 0.5 30 15% $300,000 19% (GOAL A + GOAL B) x CONFIDENCE / EFFORT = SCORE
  13. 13. Step 5 - Connect to Customer Feedback Goal A Goal B Effort Confidence Score Customers % Account $ % Feature C 1 1 1 50% 1.0 60 30% $600,000 38% Feature B 0 2 2 75% 0.8 3 2% $300,000 19% Feature D 2 1 4 90% 0.7 90 45% $900,000 56% Feature A 0 2 2 50% 0.5 30 15% $300,000 19%
  14. 14. De-risk #1: Do asks back up estimates? Goal A Goal B Effort Confidence Score Customers % Account $ % Feature C 1 1 1 50% 1.0 60 30% $600,000 38% Feature B 0 2 2 75% 0.8 3 2% $300,000 19% Feature D 2 1 4 90% 0.7 90 45% $900,000 56% Feature A 0 2 2 50% 0.5 30 15% $300,000 19%
  15. 15. De-risk #1: Do asks back up estimates? Goal A Goal B Effort Confidence Score Customers % Account $ % Feature C 1 1 1 50% 1.0 60 30% $600,000 38% Feature B 0 2 2 75% 0.8 3 2% $300,000 19% Feature D 2 1 4 90% 0.7 90 45% $900,000 56% Feature A 0 2 2 50% 0.5 30 15% $300,000 19% Why are we so confident in the impact with so few customer asks?
  16. 16. Step 6 - Add in CRM Data Goal A Goal B Effort Confidence Score Customers % Customer $ % Feature C 1 1 1 50% 1.0 60 30% $600,000 38% Feature B 0 2 2 75% 0.8 3 2% $300,000 19% Feature D 2 1 4 90% 0.7 90 45% $900,000 56% Feature A 0 2 2 50% 0.5 30 15% $300,000 19% Ah, that’s because these are our largest Enterprise customers! Are we overestimating the impact?
  17. 17. Step 7 - Add in CRM & NPS Data Reduce Churn Goal B Effort Confidence Score Detractor & Churned Customers % Detractor & Churned Customer $ % Feature C 1 1 1 50% 1.0 5 10% $100,000 10% Feature B 0 2 2 75% 0.8 0 0% $0 0% Feature D 2 1 4 90% 0.7 60 40% $650,000 50% Feature A 0 2 2 50% 0.5 12 9% $90,000 5%
  18. 18. De-risk #2: Do asks back up impacts? Reduce Churn Goal B Effort Confidence Score Detractor & Churned Customers % Detractor & Churned Customers $ % Feature C 1 1 1 50% 1.0 5 10% $100,000 10% Feature B 0 2 2 75% 0.8 0 0% $0 0% Feature D 2 1 4 90% 0.7 60 40% $650,000 50% Feature A 0 2 2 50% 0.5 12 9% $90,000 5% Are we sure of our estimate for C to Reduce Churn? What about A?
  19. 19. De-risk #3: Confirm with Accounts Goal A Goal B Effort Confidence Score Customers % Customers $ % Feature C 1 1 1 50% 1.0 60 30% $600,000 38% Feature B 0 2 2 75% 0.8 3 2% $300,000 19% Feature D 2 1 4 90% 0.7 90 45% $900,000 56% Feature A 0 2 2 50% 0.5 30 15% $300,000 19% Before we build C let’s review the feedback from those 60 accounts, contact them to give feedback on solution mock ups, beta, etc.
  20. 20. Takeaways Use a goal-based scoring framework for making product decisions - Make sure that decision framework is consistent and understood across teams. AKA Show Your Work - Use your database of feedback to de-risk functionality you put on at the top of the TODO list.
  21. 21. 21 © Copyright 2016 UserVoice. Confidential and proprietary information. For more details read the eBook How do we build that comprehensive database of customer feedback that informs good product management?
  22. 22. Need to tap into both direct & indirect feedback channels
  23. 23. Need to tap into both direct & indirect feedback channels
  24. 24. Getting Feedback from Users
  25. 25. Feedback directly from end-users Methods: NPS, Idea Boards, In-app Solicitation Key Challenge: Low Response Rates Feedback from only a small vocal minority
  26. 26. Feedback directly from end-users Methods: NPS, Idea Boards, In-app Solicitation Key Challenge: Low Response Rates Feedback from only a small vocal minority CAUSES - Lack of awareness - Poor promotion / visibility
  27. 27. Feedback directly from end-users Methods: NPS, Idea Boards, In-app Solicitation Key Challenge: Low Response Rates Feedback from only a small vocal minority CAUSES - Lack of awareness - Poor promotion / visibility - Too much work to give feedback
  28. 28. Feedback directly from end-users Methods: NPS, Idea Boards, In-app Solicitation Key Challenge: Low Response Rates Feedback from only a small vocal minority CAUSES - Lack of awareness - Poor promotion / visibility - Too much work to give feedback - No value for giver - “What’s in it for me”
  29. 29. Feedback directly from end-users Methods: NPS, Idea Boards, In-app Solicitation Key Challenge: Low Response Rates Feedback from only a small vocal minority CAUSES - Lack of awareness - Poor promotion / visibility - Too much work to give feedback - No value for giver - “What’s in it for me”
  30. 30. Feedback directly from end-users Methods: NPS, Idea Boards, In-app Solicitation Key Challenge: Low Response Rates Feedback from only a small vocal minority CAUSES - Lack of awareness - Poor promotion / visibility - Too much work to give feedback - No value for giver - “What’s in it for me”
  31. 31. Feedback directly from end-users Methods: NPS, Idea Boards, In-app Solicitation Key Challenge: Low Response Rates Feedback from only a small vocal minority SOLUTION - Promote that you’re listening - “We want your feedback to help us improve the product. Click here”
  32. 32. Feedback directly from end-users Methods: NPS, Idea Boards, In-app Solicitation Key Challenge: Low Response Rates Feedback from only a small vocal minority SOLUTION - Promote that you’re listening - “We want your feedback to help us improve the product. Click here” - Single question survey if possible
  33. 33. Feedback directly from end-users Methods: NPS, Idea Boards, In-app Solicitation Key Challenge: Low Response Rates Feedback from only a small vocal minority SOLUTION - Promote that you’re listening - “We want your feedback to help us improve the product. Click here” - Single question survey if possible - Explain what the user gets in return - Ideally influence and not prizes. Set expectations and close the loop.
  34. 34. Feedback directly from end-users Methods: NPS, Idea Boards, In-app Solicitation Key Challenge: Low Response Rates Feedback from only a small vocal minority SOLUTION - Promote that you’re listening - “We want your feedback to help us improve the product. Click here” - Single question survey if possible - Explain what the user gets in return - Ideally influence and not prizes. Set expectations and close the loop.
  35. 35. Getting Feedback from Internal Teams
  36. 36. Feedback from customer teams Methods: Meetings, Spreadsheets Sales Feedback Success Feedback 1. This thing 2. That thing 3. Some other thing 1. Not even a thing 2. Big thing 3. Small thing VS
  37. 37. Feedback from customer teams Methods: Meetings, Spreadsheets Key Challenges: Inability to Aggregate How can you compare between top asks? How do you merge with direct feedback? Sales Feedback Success Feedback 1. This thing 2. That thing 3. Some other thing 1. Not even a thing 2. Big thing 3. Small thing VS
  38. 38. Feedback from customer teams Methods: Meetings, Spreadsheets Key Challenges: Inability to Aggregate How can you compare between top asks? How do you merge with direct feedback? Hard to Trust Second hand reports instead of primary sources. Sales Feedback Success Feedback 1. This thing 2. That thing 3. Some other thing 1. Not even a thing 2. Big thing 3. Small thing VS
  39. 39. Feedback from customer teams Methods: Meetings, Spreadsheets Key Challenges: Inability to Aggregate How can you compare between top asks? How do you merge with direct feedback? Hard to Trust Second hand reports instead of primary sources. Time Consuming & Frustrating For PMs and customer facing teams. Sales Feedback Success Feedback 1. This thing 2. That thing 3. Some other thing 1. Not even a thing 2. Big thing 3. Small thing VS
  40. 40. Feedback from customer teams Method: Custom or 3rd party tool for logging feedback directly into system of record. SOLUTION - Know originating end-user - Feedback through teams instead of from.
  41. 41. Feedback from customer teams Method: Custom or 3rd party tool for logging feedback directly into system of record. SOLUTION - Know originating end-user - Feedback through teams instead of from. - Store primary source feedback - What did the user say in their own words.
  42. 42. Feedback from customer teams Method: Custom or 3rd party tool for logging feedback directly into system of record. SOLUTION - Know originating end-user - Feedback through teams instead of from. - Store primary source feedback - What did the user say in their own words. - Conserve limited Product Manager time - Crowdsource organization to customer facing teams + Skip the weekly feedback meeting.
  43. 43. Putting it All Together
  44. 44. 60%% of customers we have feedback from
  45. 45. Now all feedback regardless of channel can be aggregated in a single system USER FEEDBACK IDEAS PM team mines for problems to solve. Organized into ideas by users or customer team members.
  46. 46. Now all feedback regardless of channel can be aggregated in a single system USER FEEDBACK IDEAS PM team mines for problems to solve. Organized into ideas by users or customer team members. FEATURES Solutions spec’d by PM team to address one or more ideas
  47. 47. Now all feedback regardless of channel can be aggregated in a single system USER FEEDBACK IDEAS PM team mines for problems to solve. Organized into ideas by users or customer team members. FEATURES Solutions spec’d by PM team to address one or more ideas CUSTOMERS CRM data about customers
  48. 48. Takeaways Build a system of record for all feedback - Use best practices to get comprehensive feedback both directly and through customer facing teams. Avoid anecdotes and small numbers. - Make sure the data structure enables you to combine that feedback. Avoid death by a thousand spreadsheets. - Invest early. Cost is lower than you think and the payoff grows over time. Use a goal-based scoring framework for making product decisions - Make sure that decision framework is consistent and understood across teams. AKA Show Your Work - Use your database of feedback to de-risk functionality you put on at the top of the TODO list.
  49. 49. Inform & de-risk the product roadmap Increase internal confidence in the roadmap Exposing your decision-making framework and the underlying data makes generating buy from customer facing as well as engineering teams easy. Focus PM time Our survey found that PMs spend 20-25% of their time gathering and organizing feedback. Free them up to focus on higher order functions. Develop more evangelist customers People are often blown away when companies close the loop on product feedback (even if many months later). Don’t discount this aspect of being customer-centric. Benefits Thank You! FOR FURTHER READING eBook - Build a Customer Feedback Machine eBook - The Product Manager's Guide to Smart Product Roadmap Prioritization webinar - Best Practices for Managing Customer Feedback at Scale video - How UserVoice uses UserVoice

×