Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
BWM Product Development Process<br />Presented by<br />1<br />
Target Process Model – Single Team<br />Choosing, Planning, Building & Releasing<br />Each arrow represents a scheduled Ev...
3<br />Target Process Model – Multiple Teams<br />Consumer web team<br />Partner services team<br />Merchant/Backend team<...
 Value focus built into organizational structure (team  value stream)
 Teams more focused on value stream than on functional specialization</li></li></ul><li>Product Focus Areas – (Map to Team...
ROI calc & deal templates
Deal execution
Deal postmortem
Redemption technologies
POS integration
Partner platform
 Co-branded exp
Social Media apps
SEO platform
Landing page testing platform
Core web exp
Deal exp
 Account management
Offer dashboard
2nd ary offer market
Loyalty/reten-tion programs
Mobile/smart-phone exp
Deal production platform
Admin  platform
 Sales support
CS support
Business Intelligence
Offer Science
Email communications</li></ul>4<br />
Teams<br />5<br />
Teams & Emphasis<br /><ul><li> Right now we have
 Team 1 – core consumer experience
Team 2 – operations
 Team 3 – merchant experience/backend
 As we grow our team size through March …
 Team 1 – deal merchandising, exp, & fulfillment
 Team 2 – operations
 Team 3 – backend
 Team 4 – consumer exp, social, & mobile access
 Team 5 -  merchant experience
Teams consist of at least 1 engineer, ½ designer, ½ product manager
 Lean, feedback driven, 2-week iteration based approach</li></ul>6<br />
Target– team composition<br /><ul><li>Teams are cross-functional
Teams are not ‘owned’ by an individual
Teams have the freedom to choose approaches,  but not objectives.
Teams are accountable for the results of their choices
Product Management
(opportunity & solution identification, product performance against kpis)
Visual and interactive design
(usability analysis, solution identification & definition)
Software development
(solution definition, design, & construction)
Quality assurance
(solution definition & correctness)
Process facilitation
(process effectiveness oversight)</li></ul>7<br />
Target– governance<br /><ul><li>Teams will publish calendar with fixed product & sprint complete dates.
Upcoming SlideShare
Loading in …5

Empirical Product Development


Published on

Overview of my approach to product strategy and implementation

  • Be the first to comment

Empirical Product Development

  1. 1. BWM Product Development Process<br />Presented by<br />1<br />
  2. 2. Target Process Model – Single Team<br />Choosing, Planning, Building & Releasing<br />Each arrow represents a scheduled Event<br />Weekly Status meeting for each team<br />Executives must participate in “Choose” mtg<br />Choose Product 2<br />Choose Product 3<br />Plan Product 1<br />Plan Product 2<br />Plan Product 3<br />Product 1<br />Product 2<br />Product 3<br />Plan Sprint b<br />Plan Sprint c<br />Plan Sprint d<br />Plan Sprint a<br />Plan Sprint b<br />Plan Sprint c<br />Plan Sprint d<br />Plan Sprint a<br />Plan Sprint b<br />Plan Sprint a<br />Demo Sprint a<br />Demo Sprint b<br />Demo Sprint c<br />Demo Sprint a<br />Demo Sprint b<br />Demo Sprint c<br />Demo Sprint a<br />Release Product 1<br />Release Product 2<br />Release Product 3<br />Weekly Status<br />2<br />June<br />July<br />August<br />September<br />October<br />November<br />Time<br />
  3. 3. 3<br />Target Process Model – Multiple Teams<br />Consumer web team<br />Partner services team<br />Merchant/Backend team<br />Operations team<br />Email team<br /><ul><li> Products developed in parallel
  4. 4. Value focus built into organizational structure (team  value stream)
  5. 5. Teams more focused on value stream than on functional specialization</li></li></ul><li>Product Focus Areas – (Map to Teams)<br />Consumers<br />Merchants<br />Operations<br />Partners<br /> REMOVE<br /><ul><li>Merchant portal
  6. 6. ROI calc & deal templates
  7. 7. Deal execution
  8. 8. Deal postmortem
  9. 9. Redemption technologies
  10. 10. POS integration
  11. 11. Partner platform
  12. 12. API
  13. 13. Widgets
  14. 14. Co-branded exp
  15. 15. Portal
  16. 16. Social Media apps
  17. 17. SEO platform
  18. 18. Landing page testing platform
  19. 19. Core web exp
  20. 20. Deal exp
  21. 21. Account management
  22. 22. Offer dashboard
  23. 23. 2nd ary offer market
  24. 24. Loyalty/reten-tion programs
  25. 25. Mobile/smart-phone exp
  26. 26. Deal production platform
  27. 27. Admin platform
  28. 28. Sales support
  29. 29. CS support
  30. 30. Business Intelligence
  31. 31. Offer Science
  32. 32. Email communications</li></ul>4<br />
  33. 33. Teams<br />5<br />
  34. 34. Teams & Emphasis<br /><ul><li> Right now we have
  35. 35. Team 1 – core consumer experience
  36. 36. Team 2 – operations
  37. 37. Team 3 – merchant experience/backend
  38. 38. As we grow our team size through March …
  39. 39. Team 1 – deal merchandising, exp, & fulfillment
  40. 40. Team 2 – operations
  41. 41. Team 3 – backend
  42. 42. Team 4 – consumer exp, social, & mobile access
  43. 43. Team 5 - merchant experience
  44. 44. Teams consist of at least 1 engineer, ½ designer, ½ product manager
  45. 45. Lean, feedback driven, 2-week iteration based approach</li></ul>6<br />
  46. 46. Target– team composition<br /><ul><li>Teams are cross-functional
  47. 47. Teams are not ‘owned’ by an individual
  48. 48. Teams have the freedom to choose approaches, but not objectives.
  49. 49. Teams are accountable for the results of their choices
  50. 50. Product Management
  51. 51. (opportunity & solution identification, product performance against kpis)
  52. 52. Visual and interactive design
  53. 53. (usability analysis, solution identification & definition)
  54. 54. Software development
  55. 55. (solution definition, design, & construction)
  56. 56. Quality assurance
  57. 57. (solution definition & correctness)
  58. 58. Process facilitation
  59. 59. (process effectiveness oversight)</li></ul>7<br />
  60. 60. Target– governance<br /><ul><li>Teams will publish calendar with fixed product & sprint complete dates.
  61. 61. Product release dates are refined as sprints are completed. If they are fixed, scope is managed appropriately
  62. 62. Each team will publish calendar with fixed meeting dates
  63. 63. Weekly sprint progress will be published
  64. 64. Executives will be invited to choice meetings
  65. 65. Each team will conduct demonstrations of built functionality at the end of every sprint (~2 weeks) and prior to release
  66. 66. Information published using SharePoint (or another document management system)</li></ul>8<br />
  67. 67. Target- portfolio & roadmap<br /><ul><li> Manage product roadmaps & portfolio based on available teams
  68. 68. Each product category or value stream requires a roadmap.
  69. 69. Each team requires a roadmap
  70. 70. If fewer teams than product categories, group product categories and combine roadmaps
  71. 71. Mapping teams to value streams allows the team to drive against quantifiable success criteria associated with value stream
  72. 72. e.g. visitor to subscriber conversion
  73. 73. e.g. # purchases per session per subscriber
  74. 74. e.g # of purchased *offer science* recommended deals
  75. 75. e.g. # of subscribers per API partner</li></ul>9<br />
  76. 76. Partner Services & API<br />Objective<br /> KPIs<br /> Provide the necessary platform to<br />Extend BWM marketplace into partner properties and communities<br />Syndicate deals<br />Customize core consumer experience for partner community members<br /><ul><li>Maximize [#subscribers] per partner per unit time
  77. 77. Maximize [ARPS ($ per sub)] per partner per unit time
  78. 78. Minimize [integration time] per partner </li></ul>Areas of influence/roadmap<br />Team<br /><ul><li>API
  79. 79. Widgets & JavaScript/HTML objects
  80. 80. Partner tools & analytics
  81. 81. Automated Partner signup
  82. 82. Partner deal management capabilities
  83. 83. Product expert - <tbh>
  84. 84. Interactive/visual designer – <tbh>
  85. 85. Software development – <tbh>,<tbh>
  86. 86. ½ Quality assurance – <tbh></li></ul>10<br />
  87. 87. Consumer Web Experience<br />Objective<br /> KPIs<br />Create an amazing web based experience for users that<br />Showcases deals elegantly<br />Creates user opportunities for new life-style experiences<br />Puts the right deal in front of the right user at the right time<br /><ul><li>Maximize [#subscribers LTV]
  88. 88. Minimize [#subscribers churn/month] ratio
  89. 89. Maximize [#purchases/subscriber] ratio
  90. 90. Maximize [csat/net promoter score per user] ratio</li></ul>Areas of influence/roadmap<br />Team<br /><ul><li>Deal discovery, experience, & participation
  91. 91. Community experience
  92. 92. Account management
  93. 93. Offer Dashboard
  94. 94. Secondary market
  95. 95. Product expert - <tbh>
  96. 96. Interactive/visual designer – <tbh>
  97. 97. Software development – <tbh>, <tbh>
  98. 98. Quality assurance – <tbh></li></ul>11<br />
  99. 99. Merchant Services<br />Objective<br /> KPIs<br />Create tools & services to provide merchants <br />Easy ways to predict campaign value<br />Amazing campaign production tools<br />Analytics for assessing campaign performance<br />Consumer insight & marketing tools<br /><ul><li>Maximize [#merchants]
  100. 100. Maximize [#deals/merchant] ratio
  101. 101. Maximize [commision$/deal] ratio
  102. 102. Maximize [merchant LTV]</li></ul>Areas of influence/roadmap<br />Team<br /><ul><li>Merchant Portal
  103. 103. BoostYourBusiness and associated web collateral
  104. 104. Referral features
  105. 105. Conversion funnels
  106. 106. Product expert - <tbh>
  107. 107. Interactive/visual designer – <tbh>
  108. 108. Software development – <tbh>. <tbh>
  109. 109. ½ Quality assurance – <tbh></li></ul>12<br />
  110. 110. Backend & Admin Services<br />Objective<br /> KPIs<br />Provide the tools and services necessary to support effective and efficient BWM business operations<br /><ul><li>Minimize[time to produce deal]
  111. 111. Minimize[time to respond to customer inquiry] ratio</li></ul>Areas of influence/roadmap<br />Team<br /><ul><li>Sales Production tools
  112. 112. Customer Service tools
  113. 113. Financial Operations support
  114. 114. ½ Product expert - <tbh>
  115. 115. Software development – <tbh>, <tbh>
  116. 116. ½ Quality assurance – <tbh></li></ul>13<br />
  117. 117. Business Intelligence<br />Objective<br /> KPIs<br />To develop and report on a collection of measures that provide meaningful insight into the health and performance of the business and that provide support for the marketing, merchandising, sales, product, & executive function <br /><ul><li>Maximize [#positive outcome decisions/#published KPIs] ratio</li></ul>Areas of influence/roadmap<br />Team<br /><ul><li>Data warehouse
  118. 118. Web analytics
  119. 119. Other internal & external data-sources relevant to our business operations
  120. 120. Database/Report development – <tbh>,<tbh>
  121. 121. Software development – <tbh>
  122. 122. Business analytics – <tbh></li></ul>14<br />
  123. 123. Email & Subscriber Communications<br />Objective<br /> KPIs<br /><ul><li>Drive member engagement through active communication of relevant deals
  124. 124. Maximize the value of each subscriber communication
  125. 125. Leverage email to extend base service offerings
  126. 126. Maximize [Commission $/1000 sends] ratio
  127. 127. Maximize [# clicks/1000 sends] ratio
  128. 128. Maximize [# opens/1000 sends] ratio
  129. 129. Maximize [# inboxs/1000 sends]</li></ul>Areas of influence/roadmap<br />Team<br /><ul><li>Transactional Emails
  130. 130. Email campaigns
  131. 131. Email management tools
  132. 132. Deliverability management tools
  133. 133. Data extraction associated with email campaigns
  134. 134. CRM expert - <tbh>
  135. 135. Visual designer – <tbh>
  136. 136. Software development – <tbh>, <tbh>
  137. 137. ½ Quality assurance – <tbh></li></ul>15<br />
  138. 138. Core Services<br />Objective<br /> KPIs<br />Maximize [#releases/month]<br />Maximize [#items/release]<br />Minimize [#bugs/release]<br />Maximize [$ value/release]<br />Support must do and discretionary short term product releases that support ongoing business operations on a repeating 2 week release cycle<br />Proactively recommends optimizations against key metrics across ALL business concerns<br />Areas of influence/roadmap<br />Team<br /><ul><li>Pixel and landing page optimizations
  139. 139. Bug fixes & enhancements
  140. 140. Admin app support
  141. 141. Web production & publishing
  142. 142. Payment system changes
  143. 143. Product expert - <tbh>
  144. 144. Interactive/visual designer – <tbh>
  145. 145. Software development – <tbh>, <tbh>
  146. 146. ½ Quality assurance – <tbh></li></ul>16<br />
  147. 147. Choosing & defining<br />17<br />
  148. 148. Choosing what to do<br />Ideas come from everywhere<br />Product roadmap<br />Technology roadmap<br />Business development<br />Customer feedback<br />Funnel optimization analysis<br />Metric review<br />Competitive research<br />Prototyping and brainstorming<br />How do we decide which ones are the most valuable?<br />How do we decide which ones to do?<br />
  149. 149. Choosing what to do<br />Release targets are fixed and occur on a recurring basis for each team<br />This schedule puts pressure on the ‘ideation process’ (product identification and prioritization)<br />Each team must have a “backlog” of product ideas ready to build<br />The process of generating, capturing, and evaluating ideas must be as disciplined and normal as the process of implementing them <br />
  150. 150. Choice Cycle<br />Sprint & release cycle (building)<br />sprint 4<br />sprint 1<br />sprint 2<br />sprint 3<br />release 1<br />release 3<br />release 2<br />plan release 2<br />plan release 3<br />plan release 4<br />Identification & evaluation cycle (choosing)<br />= concept doc<br />
  151. 151. Identifying<br />Teams are empowered to work with executive management in order pick the products or projects that will have the greatest impact on their value stream<br />During a release cycle, the team’s product manager must identify and evaluate 2-5 items as candidates to be built during the next release<br />Identification primarily involves picking opportunities to realize or problems to solve<br />
  152. 152. Identifying<br />Opportunities or problems to address can come from at least the following places<br />Roadmap<br />Internal business intelligence<br />Business development opportunity<br />Competitive research & analysis<br />Market study research & analysis<br />Customer feedback<br />*intuition*<br />An opportunity is identified once a ‘concept document’ has been written<br />1- 2 page document with an opportunity description (quantitative), an objective section, and a candidate solution (with cost) section<br />Should provide enough information to evaluate and prioritize item<br />
  153. 153. Choosing<br />Choosing what to do is a critical decision process at the end of which a commitment is made to invest time and money into an endeavor with an uncertain outcome<br />Consequently, the team must be joined by the appropriate set of executive stakeholders when choosing what items to tackle in the upcoming release<br />Choosing and prioritization occurs at the “concept review meeting” which takes place at least 1 week prior to the end of the current release<br />
  154. 154. Choosing<br />In the concept review meeting the conceptual docs for the candidate initiatives are presented and discussed<br />At the end of the concept review meeting 1 or more projects are chosen to be built in the upcoming release. <br />If more than 1 project is chosen, a priority order is defined<br />Multiple methods for choosing may be employed<br />Simple ROI<br />NPV<br />IRR<br />Scorecards<br />Qualitative analysis<br />
  155. 155. Defining<br />Once a project has been chosen it must be ‘minimally’ elaborated before the start of the next release. <br />Elaborating a project is known as ‘release planning’ or creating the ‘product backlog’<br />It is very different from creation a PRD; It is a simple list<br />With a ‘product backlog’ the objective is not completeness<br />Items in a product backlog are called ‘user stories’ or ‘features’ <br />
  156. 156. Defining<br />User stories or features are simple descriptions of application behavior or user interactions<br />They are intended to be place holders for further conversation<br />Each must have a unique value and a time estimate<br />Defining<br />
  157. 157. Planning<br />Releases<br />Releases are divided into sprints<br />During sprints, the team decides how many features or user stories it believes it can accomplish in a fixed amount of time<br />During sprints, the team elaborates the features covered in the sprint<br />Sample release plan<br />Release (29 features)<br />Sprint 1 (12 features)<br />Sprint 2 (17 features)<br />4 weeks<br />4 weeks<br />
  158. 158. Building<br />28<br />
  159. 159. Process model<br />Scrum<br />15 – minute daily meeting<br />Team members respond to basics<br />- What did you do since last meeting?<br />- any obstacles?<br />- What will you do before next meeting?<br />Sprint<br />Every 24 hours<br />Backlog items expanded by team<br />Backlog<br />Features assigned to iteration<br />30-day sprint<br />New Functionality is demonstrated at the end of iteration<br />Scrum<br />Pull next<br />Product Backlog<br />Priority 1<br />Priority 2<br />Priority 3<br />Define<br />Build<br />Test<br />Evaluate<br />Pull top item<br />Fail<br />
  160. 160. Sprinting/Iterating<br />The heart beat of agile product development is the ‘sprint’<br />A release consists of 1 or more sprints<br />The team delivers an integrated, release ready set of features at the end of a sprint.<br />Features and user stories built during a sprint are taken from the ‘product backlog’<br />A sprint is composed of the highest priority items from the backlog that the team believes can be accomplished within the sprint time frame<br />
  161. 161. Sprinting<br />Sprints are always time boxed<br />At the beginning of a sprint the team engages in ‘sprint planning’ and associates tasks (each task has ‘effort points’) with each user story or feature in the sprint<br />The most important user stories or features are built first<br />Features are defined, built, and tested by the team one at a time<br />Once a sprint is fixed, only the team can add new features<br />
  162. 162. Daily Stand-up<br />The daily standup or ‘scrum’ is the primary vehicle for team communication<br /> The goal is for the team to remove any obstacles preventing it from succeeding in delivering the sprint<br />Feature ambiguity or complexity<br />Technical ambiguity or complexity<br />Lack of critical information<br />External distractions<br />Momentum problems<br />The daily standup meeting takes place every day and must be attended by all members on the team<br />
  163. 163. Daily Stand-up<br />Roles<br />Scrum facilitator – facilitates scrum<br />Product expert – defines and refines application behavior<br />Team – realizes application behaviors<br />Daily standup is a realization of ‘empirical process control’<br />Visibility – intermediate process steps are visible<br />Inspection – intermediate results can be evaluated<br />Adaptation - changes can be made as a result of visibility & inspection<br />In this way the product solution can be achieved through a series of small activities, each which can be measured against objectives<br />
  164. 164. Story Boards & Burn-down Charts<br />Story boards and burn down charts are the ‘vehicles’ upon which the sprint and daily stand-ups rely<br />Story boards list user stories or features on index cards and categorizes them as ‘not started’, ‘in progress’, & ‘completed’<br />Through the course of the sprint, index cards migrate from ‘not started’ to ‘completed’<br />The burn down chart captures this progress in a, hopefully downward sloping, two dimensional graph<br />
  165. 165. Story boards<br />Story Boards<br />Not Started<br />In Progress<br />Completed<br />Purchase flow<br />Account History<br />Sprint day 1<br />Not Started<br />In Progress<br />Completed<br />Buy btn – LFM mode<br />Purchase flow 1 trk<br />Purchase flow >1 trk<br />Mar com – splash pg<br />Omniture tracking<br />Account History<br />Buy btn – visit mode<br />Sprint day 22<br />
  166. 166. Burn-down Charts<br />Sprint 2 is a two week sprint which started with 11 features and 63 effort points. <br />Do we have a problem?<br />
  167. 167. Testing & releasing<br />37<br />
  168. 168. Testing in Agile is Different<br />Traditionally, test and development organizations are separate<br />Traditionally, development hands off large amounts of theoretically working, but largely untested code to a test organization<br />With agile, testing and development are peas in a pod<br />With agile, tests are developed concurrently with coding and requirements generation<br />
  169. 169. Testing in Agile is Different<br />Features or user-stories are validated as they move from ‘not started’ to ‘completed’<br />The define/build/test cycle occurs on a feature by feature basis<br />The embedding of a ‘test driven’ mentality throughout the whole ‘build’ process results in systems which are built to be verified<br />This leads to higher overall quality<br />
  170. 170. Agile Testing Principles<br />All code is tested code<br />Teams get no credit for delivering functionality that has been coded but not tested<br />Tests are written before, or concurrently with, the code itself<br />Testing is a team effort; testers, developers, and product experts all write tests<br />Automation is the rule, not the exception<br />
  171. 171. Agile testing strategy<br />
  172. 172. Measuring Success<br />42<br />
  173. 173. Success<br />Success in a world of value stream orientation is the creation of value<br />Each team is successful as a unit if it positively impacts its KPIs<br />Working software products that conform to their intended purpose is the first necessary condition for success<br />Discovering the right products through frequent release and measurement is the other necessary condition for success<br />
  174. 174. Success<br />Every member of the team is exposed to the performance of the products or projects they are building through a team dashboard<br />Success is NOT<br />Completing the step in a work order<br />A PRD or a technical specification<br />Adherence to internal milestone dates<br />A wireframe<br />A complete test plan<br />Positive performance of products is the primary measure of success for the entire team<br />
  175. 175. Metrics<br />Although the primary measures of success are direct product performance metrics, other internal efficiency and effectiveness measures are valuable<br />Project metrics allow the team to perform project kaizen (to get better at doing)<br />There are two kinds of project metrics<br />Sprint assessment metrics<br />Release assessment metrics<br />
  176. 176. Sprint assessment metrics<br />
  177. 177. Release assessment metrics<br />
  178. 178. Process Metrics<br />Project metrics help individual teams improve their own performance at the project level<br />Process metrics provide global insight into process effectiveness across six dimensions<br />Product management capability<br />Release planning and tracking capability<br />Sprint planning and tracking<br />Team effectiveness<br />Testing practices<br />Development practices/infrastructure<br />
  179. 179.
  180. 180.
  181. 181.
  182. 182.
  183. 183. Sharing Information<br />53<br />
  184. 184. Publication<br /><Discuss SharePoint><br /><per team><br /><performance metrics><br /><competitive/market analysis & artifacts<br /><Exploratory comps/wireframes/descriptions etc><br /><backlog><br /><release plan><br /><weekly sprint progress – via backlog><br />
  185. 185. Recurring Meetings<br /><outline recurring meetings><br /><commitment meeting><br /><release planning meeting><br /><sprint planning meeting><br /><sprint delivery meeting><br /><pre-release meeting><br /><monthly open-forum area performance meeting><br />
  186. 186. Status<br /><Define how teams bubble status to exec committee on weekly basis – pre operations meeting><br /><Multi-tabbed spreadsheet (ala ru)><br />
  187. 187. Governance<br />57<br />
  188. 188. Stakeholder Involvement<br /><characterize appropriate involvement levels><br /><examples of *good* involvement><br /><examples of *bad* involvement><br /><emphasize persuasion & coaxing & evangelizing as *good*><br /><emphasize explicit direction & imperatives as *bad*><br /><discuss pathologies that arise as a result of this technique><br />
  189. 189. Process Impediments<br />Iterative Process<br />People arrive late to meetings<br />Meetings take too long. People become bored and devalue meetings<br />Scrum master dictates design decisions or micromanages<br />Teams are too large for daily scrum and sprint planning<br />Teams do not report task remaining time for burn down analysis<br />People Practices<br />Individuals interrupted by non-sprint based work<br />Team members isolated physically<br />Team members not accountable for sprint commitments<br />Individuals multiplexed across too many teams<br />
  190. 190. Process Imediments<br />Product Engineering Practices<br />Resources necessary for definition/design/implementation/testing not present on team<br />Sprints do not fully implement & test deployable increments of customer valued features<br />Product owner not available/integral to team<br />System integration not forced at each sprint<br />Product owner wont split up large product backlogs<br />Features introduced into sprints after sprints begin<br />Organizational issues<br />Software process police regulate to ineffective process<br />Management assumes fixed cost, fixed scope delivery <br />Individual rather than team behavior rewarded<br />Rules or capitalization structures demand water-fall<br />Teams not collocated to maximum extent feasible<br />