Your SlideShare is downloading. ×
BWM Product Development Process<br />Presented by<br />1<br />
Target Process Model – Single Team<br />Choosing, Planning, Building & Releasing<br />Each arrow represents a scheduled Ev...
3<br />Target Process Model – Multiple Teams<br />Consumer web team<br />Partner services team<br />Merchant/Backend team<...
 Value focus built into organizational structure (team  value stream)
 Teams more focused on value stream than on functional specialization</li></li></ul><li>Product Focus Areas – (Map to Team...
ROI calc & deal templates
Deal execution
Deal postmortem
Redemption technologies
POS integration
Partner platform
 Co-branded exp
Social Media apps
SEO platform
Landing page testing platform
Core web exp
Deal exp
 Account management
Offer dashboard
2nd ary offer market
Loyalty/reten-tion programs
Mobile/smart-phone exp
Deal production platform
Admin  platform
 Sales support
CS support
Business Intelligence
Offer Science
Email communications</li></ul>4<br />
Teams<br />5<br />
Teams & Emphasis<br /><ul><li> Right now we have
 Team 1 – core consumer experience
Team 2 – operations
 Team 3 – merchant experience/backend
 As we grow our team size through March …
 Team 1 – deal merchandising, exp, & fulfillment
 Team 2 – operations
 Team 3 – backend
 Team 4 – consumer exp, social, & mobile access
 Team 5 -  merchant experience
Teams consist of at least 1 engineer, ½ designer, ½ product manager
 Lean, feedback driven, 2-week iteration based approach</li></ul>6<br />
Target– team composition<br /><ul><li>Teams are cross-functional
Teams are not ‘owned’ by an individual
Teams have the freedom to choose approaches,  but not objectives.
Teams are accountable for the results of their choices
Product Management
(opportunity & solution identification, product performance against kpis)
Visual and interactive design
(usability analysis, solution identification & definition)
Software development
(solution definition, design, & construction)
Quality assurance
(solution definition & correctness)
Process facilitation
(process effectiveness oversight)</li></ul>7<br />
Target– governance<br /><ul><li>Teams will publish calendar with fixed product & sprint complete dates.
Upcoming SlideShare
Loading in...5

Empirical Product Development


Published on

Overview of my approach to product strategy and implementation

1 Like
  • Be the first to comment

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • partner services &amp; apiacquisition &amp; conversion toolsConsumer webmobilemerchant servicesBackend admin servicesBIEmail
  • Transcript of "Empirical Product Development"

    1. 1. BWM Product Development Process<br />Presented by<br />1<br />
    2. 2. Target Process Model – Single Team<br />Choosing, Planning, Building & Releasing<br />Each arrow represents a scheduled Event<br />Weekly Status meeting for each team<br />Executives must participate in “Choose” mtg<br />Choose Product 2<br />Choose Product 3<br />Plan Product 1<br />Plan Product 2<br />Plan Product 3<br />Product 1<br />Product 2<br />Product 3<br />Plan Sprint b<br />Plan Sprint c<br />Plan Sprint d<br />Plan Sprint a<br />Plan Sprint b<br />Plan Sprint c<br />Plan Sprint d<br />Plan Sprint a<br />Plan Sprint b<br />Plan Sprint a<br />Demo Sprint a<br />Demo Sprint b<br />Demo Sprint c<br />Demo Sprint a<br />Demo Sprint b<br />Demo Sprint c<br />Demo Sprint a<br />Release Product 1<br />Release Product 2<br />Release Product 3<br />Weekly Status<br />2<br />June<br />July<br />August<br />September<br />October<br />November<br />Time<br />
    3. 3. 3<br />Target Process Model – Multiple Teams<br />Consumer web team<br />Partner services team<br />Merchant/Backend team<br />Operations team<br />Email team<br /><ul><li> Products developed in parallel
    4. 4. Value focus built into organizational structure (team  value stream)
    5. 5. Teams more focused on value stream than on functional specialization</li></li></ul><li>Product Focus Areas – (Map to Teams)<br />Consumers<br />Merchants<br />Operations<br />Partners<br /> REMOVE<br /><ul><li>Merchant portal
    6. 6. ROI calc & deal templates
    7. 7. Deal execution
    8. 8. Deal postmortem
    9. 9. Redemption technologies
    10. 10. POS integration
    11. 11. Partner platform
    12. 12. API
    13. 13. Widgets
    14. 14. Co-branded exp
    15. 15. Portal
    16. 16. Social Media apps
    17. 17. SEO platform
    18. 18. Landing page testing platform
    19. 19. Core web exp
    20. 20. Deal exp
    21. 21. Account management
    22. 22. Offer dashboard
    23. 23. 2nd ary offer market
    24. 24. Loyalty/reten-tion programs
    25. 25. Mobile/smart-phone exp
    26. 26. Deal production platform
    27. 27. Admin platform
    28. 28. Sales support
    29. 29. CS support
    30. 30. Business Intelligence
    31. 31. Offer Science
    32. 32. Email communications</li></ul>4<br />
    33. 33. Teams<br />5<br />
    34. 34. Teams & Emphasis<br /><ul><li> Right now we have
    35. 35. Team 1 – core consumer experience
    36. 36. Team 2 – operations
    37. 37. Team 3 – merchant experience/backend
    38. 38. As we grow our team size through March …
    39. 39. Team 1 – deal merchandising, exp, & fulfillment
    40. 40. Team 2 – operations
    41. 41. Team 3 – backend
    42. 42. Team 4 – consumer exp, social, & mobile access
    43. 43. Team 5 - merchant experience
    44. 44. Teams consist of at least 1 engineer, ½ designer, ½ product manager
    45. 45. Lean, feedback driven, 2-week iteration based approach</li></ul>6<br />
    46. 46. Target– team composition<br /><ul><li>Teams are cross-functional
    47. 47. Teams are not ‘owned’ by an individual
    48. 48. Teams have the freedom to choose approaches, but not objectives.
    49. 49. Teams are accountable for the results of their choices
    50. 50. Product Management
    51. 51. (opportunity & solution identification, product performance against kpis)
    52. 52. Visual and interactive design
    53. 53. (usability analysis, solution identification & definition)
    54. 54. Software development
    55. 55. (solution definition, design, & construction)
    56. 56. Quality assurance
    57. 57. (solution definition & correctness)
    58. 58. Process facilitation
    59. 59. (process effectiveness oversight)</li></ul>7<br />
    60. 60. Target– governance<br /><ul><li>Teams will publish calendar with fixed product & sprint complete dates.
    61. 61. Product release dates are refined as sprints are completed. If they are fixed, scope is managed appropriately
    62. 62. Each team will publish calendar with fixed meeting dates
    63. 63. Weekly sprint progress will be published
    64. 64. Executives will be invited to choice meetings
    65. 65. Each team will conduct demonstrations of built functionality at the end of every sprint (~2 weeks) and prior to release
    66. 66. Information published using SharePoint (or another document management system)</li></ul>8<br />
    67. 67. Target- portfolio & roadmap<br /><ul><li> Manage product roadmaps & portfolio based on available teams
    68. 68. Each product category or value stream requires a roadmap.
    69. 69. Each team requires a roadmap
    70. 70. If fewer teams than product categories, group product categories and combine roadmaps
    71. 71. Mapping teams to value streams allows the team to drive against quantifiable success criteria associated with value stream
    72. 72. e.g. visitor to subscriber conversion
    73. 73. e.g. # purchases per session per subscriber
    74. 74. e.g # of purchased *offer science* recommended deals
    75. 75. e.g. # of subscribers per API partner</li></ul>9<br />
    76. 76. Partner Services & API<br />Objective<br /> KPIs<br /> Provide the necessary platform to<br />Extend BWM marketplace into partner properties and communities<br />Syndicate deals<br />Customize core consumer experience for partner community members<br /><ul><li>Maximize [#subscribers] per partner per unit time
    77. 77. Maximize [ARPS ($ per sub)] per partner per unit time
    78. 78. Minimize [integration time] per partner </li></ul>Areas of influence/roadmap<br />Team<br /><ul><li>API
    79. 79. Widgets & JavaScript/HTML objects
    80. 80. Partner tools & analytics
    81. 81. Automated Partner signup
    82. 82. Partner deal management capabilities
    83. 83. Product expert - <tbh>
    84. 84. Interactive/visual designer – <tbh>
    85. 85. Software development – <tbh>,<tbh>
    86. 86. ½ Quality assurance – <tbh></li></ul>10<br />
    87. 87. Consumer Web Experience<br />Objective<br /> KPIs<br />Create an amazing web based experience for users that<br />Showcases deals elegantly<br />Creates user opportunities for new life-style experiences<br />Puts the right deal in front of the right user at the right time<br /><ul><li>Maximize [#subscribers LTV]
    88. 88. Minimize [#subscribers churn/month] ratio
    89. 89. Maximize [#purchases/subscriber] ratio
    90. 90. Maximize [csat/net promoter score per user] ratio</li></ul>Areas of influence/roadmap<br />Team<br /><ul><li>Deal discovery, experience, & participation
    91. 91. Community experience
    92. 92. Account management
    93. 93. Offer Dashboard
    94. 94. Secondary market
    95. 95. Product expert - <tbh>
    96. 96. Interactive/visual designer – <tbh>
    97. 97. Software development – <tbh>, <tbh>
    98. 98. Quality assurance – <tbh></li></ul>11<br />
    99. 99. Merchant Services<br />Objective<br /> KPIs<br />Create tools & services to provide merchants <br />Easy ways to predict campaign value<br />Amazing campaign production tools<br />Analytics for assessing campaign performance<br />Consumer insight & marketing tools<br /><ul><li>Maximize [#merchants]
    100. 100. Maximize [#deals/merchant] ratio
    101. 101. Maximize [commision$/deal] ratio
    102. 102. Maximize [merchant LTV]</li></ul>Areas of influence/roadmap<br />Team<br /><ul><li>Merchant Portal
    103. 103. BoostYourBusiness and associated web collateral
    104. 104. Referral features
    105. 105. Conversion funnels
    106. 106. Product expert - <tbh>
    107. 107. Interactive/visual designer – <tbh>
    108. 108. Software development – <tbh>. <tbh>
    109. 109. ½ Quality assurance – <tbh></li></ul>12<br />
    110. 110. Backend & Admin Services<br />Objective<br /> KPIs<br />Provide the tools and services necessary to support effective and efficient BWM business operations<br /><ul><li>Minimize[time to produce deal]
    111. 111. Minimize[time to respond to customer inquiry] ratio</li></ul>Areas of influence/roadmap<br />Team<br /><ul><li>Sales Production tools
    112. 112. Customer Service tools
    113. 113. Financial Operations support
    114. 114. ½ Product expert - <tbh>
    115. 115. Software development – <tbh>, <tbh>
    116. 116. ½ Quality assurance – <tbh></li></ul>13<br />
    117. 117. Business Intelligence<br />Objective<br /> KPIs<br />To develop and report on a collection of measures that provide meaningful insight into the health and performance of the business and that provide support for the marketing, merchandising, sales, product, & executive function <br /><ul><li>Maximize [#positive outcome decisions/#published KPIs] ratio</li></ul>Areas of influence/roadmap<br />Team<br /><ul><li>Data warehouse
    118. 118. Web analytics
    119. 119. Other internal & external data-sources relevant to our business operations
    120. 120. Database/Report development – <tbh>,<tbh>
    121. 121. Software development – <tbh>
    122. 122. Business analytics – <tbh></li></ul>14<br />
    123. 123. Email & Subscriber Communications<br />Objective<br /> KPIs<br /><ul><li>Drive member engagement through active communication of relevant deals
    124. 124. Maximize the value of each subscriber communication
    125. 125. Leverage email to extend base service offerings
    126. 126. Maximize [Commission $/1000 sends] ratio
    127. 127. Maximize [# clicks/1000 sends] ratio
    128. 128. Maximize [# opens/1000 sends] ratio
    129. 129. Maximize [# inboxs/1000 sends]</li></ul>Areas of influence/roadmap<br />Team<br /><ul><li>Transactional Emails
    130. 130. Email campaigns
    131. 131. Email management tools
    132. 132. Deliverability management tools
    133. 133. Data extraction associated with email campaigns
    134. 134. CRM expert - <tbh>
    135. 135. Visual designer – <tbh>
    136. 136. Software development – <tbh>, <tbh>
    137. 137. ½ Quality assurance – <tbh></li></ul>15<br />
    138. 138. Core Services<br />Objective<br /> KPIs<br />Maximize [#releases/month]<br />Maximize [#items/release]<br />Minimize [#bugs/release]<br />Maximize [$ value/release]<br />Support must do and discretionary short term product releases that support ongoing business operations on a repeating 2 week release cycle<br />Proactively recommends optimizations against key metrics across ALL business concerns<br />Areas of influence/roadmap<br />Team<br /><ul><li>Pixel and landing page optimizations
    139. 139. Bug fixes & enhancements
    140. 140. Admin app support
    141. 141. Web production & publishing
    142. 142. Payment system changes
    143. 143. Product expert - <tbh>
    144. 144. Interactive/visual designer – <tbh>
    145. 145. Software development – <tbh>, <tbh>
    146. 146. ½ Quality assurance – <tbh></li></ul>16<br />
    147. 147. Choosing & defining<br />17<br />
    148. 148. Choosing what to do<br />Ideas come from everywhere<br />Product roadmap<br />Technology roadmap<br />Business development<br />Customer feedback<br />Funnel optimization analysis<br />Metric review<br />Competitive research<br />Prototyping and brainstorming<br />How do we decide which ones are the most valuable?<br />How do we decide which ones to do?<br />
    149. 149. Choosing what to do<br />Release targets are fixed and occur on a recurring basis for each team<br />This schedule puts pressure on the ‘ideation process’ (product identification and prioritization)<br />Each team must have a “backlog” of product ideas ready to build<br />The process of generating, capturing, and evaluating ideas must be as disciplined and normal as the process of implementing them <br />
    150. 150. Choice Cycle<br />Sprint & release cycle (building)<br />sprint 4<br />sprint 1<br />sprint 2<br />sprint 3<br />release 1<br />release 3<br />release 2<br />plan release 2<br />plan release 3<br />plan release 4<br />Identification & evaluation cycle (choosing)<br />= concept doc<br />
    151. 151. Identifying<br />Teams are empowered to work with executive management in order pick the products or projects that will have the greatest impact on their value stream<br />During a release cycle, the team’s product manager must identify and evaluate 2-5 items as candidates to be built during the next release<br />Identification primarily involves picking opportunities to realize or problems to solve<br />
    152. 152. Identifying<br />Opportunities or problems to address can come from at least the following places<br />Roadmap<br />Internal business intelligence<br />Business development opportunity<br />Competitive research & analysis<br />Market study research & analysis<br />Customer feedback<br />*intuition*<br />An opportunity is identified once a ‘concept document’ has been written<br />1- 2 page document with an opportunity description (quantitative), an objective section, and a candidate solution (with cost) section<br />Should provide enough information to evaluate and prioritize item<br />
    153. 153. Choosing<br />Choosing what to do is a critical decision process at the end of which a commitment is made to invest time and money into an endeavor with an uncertain outcome<br />Consequently, the team must be joined by the appropriate set of executive stakeholders when choosing what items to tackle in the upcoming release<br />Choosing and prioritization occurs at the “concept review meeting” which takes place at least 1 week prior to the end of the current release<br />
    154. 154. Choosing<br />In the concept review meeting the conceptual docs for the candidate initiatives are presented and discussed<br />At the end of the concept review meeting 1 or more projects are chosen to be built in the upcoming release. <br />If more than 1 project is chosen, a priority order is defined<br />Multiple methods for choosing may be employed<br />Simple ROI<br />NPV<br />IRR<br />Scorecards<br />Qualitative analysis<br />
    155. 155. Defining<br />Once a project has been chosen it must be ‘minimally’ elaborated before the start of the next release. <br />Elaborating a project is known as ‘release planning’ or creating the ‘product backlog’<br />It is very different from creation a PRD; It is a simple list<br />With a ‘product backlog’ the objective is not completeness<br />Items in a product backlog are called ‘user stories’ or ‘features’ <br />
    156. 156. Defining<br />User stories or features are simple descriptions of application behavior or user interactions<br />They are intended to be place holders for further conversation<br />Each must have a unique value and a time estimate<br />Defining<br />
    157. 157. Planning<br />Releases<br />Releases are divided into sprints<br />During sprints, the team decides how many features or user stories it believes it can accomplish in a fixed amount of time<br />During sprints, the team elaborates the features covered in the sprint<br />Sample release plan<br />Release (29 features)<br />Sprint 1 (12 features)<br />Sprint 2 (17 features)<br />4 weeks<br />4 weeks<br />
    158. 158. Building<br />28<br />
    159. 159. Process model<br />Scrum<br />15 – minute daily meeting<br />Team members respond to basics<br />- What did you do since last meeting?<br />- any obstacles?<br />- What will you do before next meeting?<br />Sprint<br />Every 24 hours<br />Backlog items expanded by team<br />Backlog<br />Features assigned to iteration<br />30-day sprint<br />New Functionality is demonstrated at the end of iteration<br />Scrum<br />Pull next<br />Product Backlog<br />Priority 1<br />Priority 2<br />Priority 3<br />Define<br />Build<br />Test<br />Evaluate<br />Pull top item<br />Fail<br />
    160. 160. Sprinting/Iterating<br />The heart beat of agile product development is the ‘sprint’<br />A release consists of 1 or more sprints<br />The team delivers an integrated, release ready set of features at the end of a sprint.<br />Features and user stories built during a sprint are taken from the ‘product backlog’<br />A sprint is composed of the highest priority items from the backlog that the team believes can be accomplished within the sprint time frame<br />
    161. 161. Sprinting<br />Sprints are always time boxed<br />At the beginning of a sprint the team engages in ‘sprint planning’ and associates tasks (each task has ‘effort points’) with each user story or feature in the sprint<br />The most important user stories or features are built first<br />Features are defined, built, and tested by the team one at a time<br />Once a sprint is fixed, only the team can add new features<br />
    162. 162. Daily Stand-up<br />The daily standup or ‘scrum’ is the primary vehicle for team communication<br /> The goal is for the team to remove any obstacles preventing it from succeeding in delivering the sprint<br />Feature ambiguity or complexity<br />Technical ambiguity or complexity<br />Lack of critical information<br />External distractions<br />Momentum problems<br />The daily standup meeting takes place every day and must be attended by all members on the team<br />
    163. 163. Daily Stand-up<br />Roles<br />Scrum facilitator – facilitates scrum<br />Product expert – defines and refines application behavior<br />Team – realizes application behaviors<br />Daily standup is a realization of ‘empirical process control’<br />Visibility – intermediate process steps are visible<br />Inspection – intermediate results can be evaluated<br />Adaptation - changes can be made as a result of visibility & inspection<br />In this way the product solution can be achieved through a series of small activities, each which can be measured against objectives<br />
    164. 164. Story Boards & Burn-down Charts<br />Story boards and burn down charts are the ‘vehicles’ upon which the sprint and daily stand-ups rely<br />Story boards list user stories or features on index cards and categorizes them as ‘not started’, ‘in progress’, & ‘completed’<br />Through the course of the sprint, index cards migrate from ‘not started’ to ‘completed’<br />The burn down chart captures this progress in a, hopefully downward sloping, two dimensional graph<br />
    165. 165. Story boards<br />Story Boards<br />Not Started<br />In Progress<br />Completed<br />Purchase flow<br />Account History<br />Sprint day 1<br />Not Started<br />In Progress<br />Completed<br />Buy btn – LFM mode<br />Purchase flow 1 trk<br />Purchase flow >1 trk<br />Mar com – splash pg<br />Omniture tracking<br />Account History<br />Buy btn – visit mode<br />Sprint day 22<br />
    166. 166. Burn-down Charts<br />Sprint 2 is a two week sprint which started with 11 features and 63 effort points. <br />Do we have a problem?<br />
    167. 167. Testing & releasing<br />37<br />
    168. 168. Testing in Agile is Different<br />Traditionally, test and development organizations are separate<br />Traditionally, development hands off large amounts of theoretically working, but largely untested code to a test organization<br />With agile, testing and development are peas in a pod<br />With agile, tests are developed concurrently with coding and requirements generation<br />
    169. 169. Testing in Agile is Different<br />Features or user-stories are validated as they move from ‘not started’ to ‘completed’<br />The define/build/test cycle occurs on a feature by feature basis<br />The embedding of a ‘test driven’ mentality throughout the whole ‘build’ process results in systems which are built to be verified<br />This leads to higher overall quality<br />
    170. 170. Agile Testing Principles<br />All code is tested code<br />Teams get no credit for delivering functionality that has been coded but not tested<br />Tests are written before, or concurrently with, the code itself<br />Testing is a team effort; testers, developers, and product experts all write tests<br />Automation is the rule, not the exception<br />
    171. 171. Agile testing strategy<br />
    172. 172. Measuring Success<br />42<br />
    173. 173. Success<br />Success in a world of value stream orientation is the creation of value<br />Each team is successful as a unit if it positively impacts its KPIs<br />Working software products that conform to their intended purpose is the first necessary condition for success<br />Discovering the right products through frequent release and measurement is the other necessary condition for success<br />
    174. 174. Success<br />Every member of the team is exposed to the performance of the products or projects they are building through a team dashboard<br />Success is NOT<br />Completing the step in a work order<br />A PRD or a technical specification<br />Adherence to internal milestone dates<br />A wireframe<br />A complete test plan<br />Positive performance of products is the primary measure of success for the entire team<br />
    175. 175. Metrics<br />Although the primary measures of success are direct product performance metrics, other internal efficiency and effectiveness measures are valuable<br />Project metrics allow the team to perform project kaizen (to get better at doing)<br />There are two kinds of project metrics<br />Sprint assessment metrics<br />Release assessment metrics<br />
    176. 176. Sprint assessment metrics<br />
    177. 177. Release assessment metrics<br />
    178. 178. Process Metrics<br />Project metrics help individual teams improve their own performance at the project level<br />Process metrics provide global insight into process effectiveness across six dimensions<br />Product management capability<br />Release planning and tracking capability<br />Sprint planning and tracking<br />Team effectiveness<br />Testing practices<br />Development practices/infrastructure<br />
    179. 179.
    180. 180.
    181. 181.
    182. 182.
    183. 183. Sharing Information<br />53<br />
    184. 184. Publication<br /><Discuss SharePoint><br /><per team><br /><performance metrics><br /><competitive/market analysis & artifacts<br /><Exploratory comps/wireframes/descriptions etc><br /><backlog><br /><release plan><br /><weekly sprint progress – via backlog><br />
    185. 185. Recurring Meetings<br /><outline recurring meetings><br /><commitment meeting><br /><release planning meeting><br /><sprint planning meeting><br /><sprint delivery meeting><br /><pre-release meeting><br /><monthly open-forum area performance meeting><br />
    186. 186. Status<br /><Define how teams bubble status to exec committee on weekly basis – pre operations meeting><br /><Multi-tabbed spreadsheet (ala ru)><br />
    187. 187. Governance<br />57<br />
    188. 188. Stakeholder Involvement<br /><characterize appropriate involvement levels><br /><examples of *good* involvement><br /><examples of *bad* involvement><br /><emphasize persuasion & coaxing & evangelizing as *good*><br /><emphasize explicit direction & imperatives as *bad*><br /><discuss pathologies that arise as a result of this technique><br />
    189. 189. Process Impediments<br />Iterative Process<br />People arrive late to meetings<br />Meetings take too long. People become bored and devalue meetings<br />Scrum master dictates design decisions or micromanages<br />Teams are too large for daily scrum and sprint planning<br />Teams do not report task remaining time for burn down analysis<br />People Practices<br />Individuals interrupted by non-sprint based work<br />Team members isolated physically<br />Team members not accountable for sprint commitments<br />Individuals multiplexed across too many teams<br />
    190. 190. Process Imediments<br />Product Engineering Practices<br />Resources necessary for definition/design/implementation/testing not present on team<br />Sprints do not fully implement & test deployable increments of customer valued features<br />Product owner not available/integral to team<br />System integration not forced at each sprint<br />Product owner wont split up large product backlogs<br />Features introduced into sprints after sprints begin<br />Organizational issues<br />Software process police regulate to ineffective process<br />Management assumes fixed cost, fixed scope delivery <br />Individual rather than team behavior rewarded<br />Rules or capitalization structures demand water-fall<br />Teams not collocated to maximum extent feasible<br />
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.