Your SlideShare is downloading. ×

ETCA_8

315

Published on

Published in: Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
315
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
13
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. 1 Page
  • 2. A framework to manage multiple releases using Agile Scrum and KanbanVenkatraman L – Head of Project Management, InMobi
  • 3. Contents 1. Abstract ......................................................................................................................................... 4 2. How to read this paper ................................................................................................................ 4 3. Terminologies to get started ....................................................................................................... 4 4. How it all began ............................................................................................................................ 5 5. The Framework ............................................................................................................................ 6 Step 1: Gap Analysis ................................................................................................................... 6 Step 2: Plan the Phase 1 ............................................................................................................ 7 The Execution Framework - Sprints, Stand-ups... .................................................................. 9 The Simple 3-Step Framework (push & pull) ......................................................................... 10 The Pull – Kanban ...................................................................................................................... 10 Arriving at Release Dates ......................................................................................................... 11 Managing Release-Level Visibility – The Key to Success! .................................................. 11 The Leap into Phase 2 .............................................................................................................. 11 6. Key Benefits of the framework ................................................................................................. 12 7. Key Challenges Overcome ....................................................................................................... 13 8. Conclusion .................................................................................................................................. 13 9. References .................................................................................................................................. 13 10. Author’s Profile: ........................................................................................................................ 143 Page
  • 4. 1. AbstractThe key to any startup is speed against any process / framework, to begin with. However, as timeprogresses and the startup suddenly scales leaps and bounds in terms of people and business, theneed for a structured approach starts to emerge. The situation can be rightly called “The ControlledChaos”. The paper describes a startup with multiple stack teams (teams are grouped by architecturalcomponents forming a clean workflow) aiming to deliver multiple simultaneous releases at asustainable pace. Almost a significant number of releases cut across multiple stack teams and thatposed a huge challenge as teams were used to working in their own styles and priorities. Theframework that was designed to manage these releases focused on release dependency matrix, interand intra team dependencies, release planning, sprint planning, without compromising on the velocityof the teams. One thing to call out was that, there was no luxury of picking a “pilot” team as in otherAgile rollouts as there was no single team who could be picked. In short, the mandate was to fix theengine mid-flight. Release lifecycle was just broken down into steps of 3 to make the entire processswift and lean. A mix of push (to stack team backlogs) and a pull (Kanban mode duringexecution) adds flavor to the whole framework.2. How to read this paperThe paper begins with outlining few terminologies that are specific to the framework. The next sectiondetails the scenario that needed a robust framework to manage releases (30+ in parallel). The sectionpost that details how the framework evolved and where it stands now. The final section talks aboutkey challenges and the volume of releases that the teams are handling.3. Terminologies to get startedAll the terminologies described below, though they are generic, have some nuances when it comes tothe context of this paper.Core TeamCore team is a key team during the release ideation and discovery process. This team is responsibleto define the business specs for each release, elaborate the architectural requirements, and suggestany changes to the designs, identify and call out the technical dependencies between the stackteams and also with the platform teams. The participants are the Product Owner (biz specs),Architect (technical dependencies) and UX (as needed by the release)Scrum TeamBy Scrum team, we mean the Product Owner (PO), Scrum Master, Team (Development, TechnicalOperations and Test Engineering) who would spend substantial amount of time in the 2-week sprints.This team (w/ the support of the architect) is responsible for building the product right.4 Page
  • 5. Role of Architects in the Scrum TeamArchitects would also be an integral part of the planning meetings and will be available during theentire sprint for answering queries and modifying or making changes to the architecture and design.Architects would be part of multiple projects (shared across the projects) but their time availability iscritical to the success of each of the teams they interface with.Portfolio Items (PI) & ThemesEach Portfolio Item is equivalent of a release in the system. The reason to move into a portfolio viewof releases is due to the sheer volume of releases that were in execution at any given point in time. Agroup of PI constitutes the release backlog of the organization. These are prioritized every fortnight.sThe entire set of releases was aligned to few Business Themes (Marketplace, Sustenance, Systemhealth, Improve Operational Capabilities, to name a few) so that the executive team can take aninformed decision when the releases are approved.Release v/s DeploymentRelease is a time-boxed (feature rich) deliverable that works end-to-end in the system. Each releasederives out of a business need. Deployment is termed as a push to production by individual stackteams. Multiple deployments put together constitute a release in the system.Backlog GroomingThe process of (but not limited to) elaborating, clarifying, adding acceptance criteria and planninglong-range solutions could be called as the Backlog Grooming. This is ideally done during the stagewhen the team starts looking at the Release Backlog and tries to provide feedback to the PO byasking the right questions and identifying if the product envisioned is feasible.Sprint / IterationSprint is a time-boxed iteration (recommended is 2 weeks but could potentially be of any longerduration too). During this period, the team works uninterrupted (by anybody outside the Scrum) withthe help of the Scrum Master who ensures that the team focuses on what has been committed to (thesprint backlog) be delivered. Each sprint is expected to deliver a potentially shippable productincrement. However, it is the decision of the PO that decides if the product is shippable.4. How it all beganPlanning & Execution StatusDuring Q2 of 2011, though there was enough momentum in the system (small teams) and so was alot of churn in the backlog due to business priorities. Many stack teams were not having a focusedplanning and execution cycles. They were either going by the flavor of the month (driven bycompetition, state of union) or the sudden priorities that crept up. The iteration and deploymentrhythm was broken and they were just executing on a daily / weekly priorities. There were instanceswhere the teams dropped what they were working and moved on to an item with higher priority asbusiness was suffering. Amidst all these, there were some teams who were diligently doing weekly5 Page
  • 6. releases. All teams however wanted a clear priority to execute so that all teams can align and delivertimely releases. In short, the X (read 20+) number of releases that were signed up for the quarter washeading in multiple directions.What were the problems? 1. Clarity on the teams contributed to the release. This was a key gap b/w the PO and Engineering due to handoffs, discussions that were not followed up etc. 2. Flesh the details and own the technical dependencies (between stack teams) – Which team owned it? 3. Status of each of the stack team deliverable and in turn the entire release? 4. How to breakdown of the business ask (of the release) into smaller requirements that each stack team can independently work on? 5. What will the backlog for the stack teams to pick during the sprints? – Who defines their priorities? Do they follow stack prioritization or release level prioritization? What if one of the items was a pre-requisite for the release that needed prioritization? 6. Who and how will we tie up the dependencies to form a big-bang release? 7. How do we bring in the visibility into the upcoming sprints (specifically for resource planning than just for technical dependencies) 8. How do we communicate the status of all releases that were being executed in parallel 9. How to run the stack sprints with production fixes in parallel? 10. How do we manage Code merge and branch out when parallel product initiatives are being run. a. When and how to manage the code merge? b. Who manages the code merges? c. Who manages the merge down and merge up issues? d. What if there has been an update to the production by the time the merge happens?5. The FrameworkAs in Agile, it’s all about “experimenting”. There is no one silver bullet framework that can apply tothese situations. Few team members from Product, Engineering, QA and Project Managementdebated, deliberated and devised the framework that works on delivering multiple releases in parallelwith the finite set of resources. The input from the Senior Management / Executive Management wasthe priority list of releases (with no specific absolute priority as the company had to go in for flankattack)Step 1: Gap AnalysisThe key thing was to identify the gaps in the system before the teams could adopt the new way ofrelease planning and execution. To that effect, a quick retrospective was put between the mid andsenior leadership team to analyze what were the core areas that needed attention. The focus wasdeliberately not on tools or techniques but to address those bottlenecks that would help the teams toplan and execute faster and better. The Pareto principle of identifying those 20% things that wouldgive 80% lift was the focus of the team.6 Page
  • 7. Few top-level things that were identified were: Release Dependencies that currently go unnoticed and also unattended mid-flight (once the releases were in execution). It was a given that 90% of releases had dependencies. Inter team sequencing of the dependencies and deployments Intra-team sequencing of code merges, release signoffs by QA, regressing merged builds Constant and consistent visibility to teams and upper management related to state of releases Coordination with technical and business operations in communicating the release calendarsStep 2: Plan the Phase 1There was a concise quarterly plan to attack all of the problems above. The first quarter was totallydedicated towards ensuring Develop a plan of record for tracking all releases (fix the communication issue) Analysis of the current state of the release (many were struck at 10%, some at 80% for weeks) – find out the dependencies Identify the hardware requirements delaying releases and address them on war footing basis Framework of release thrashing where all stakeholders across various stacks contributed Production push process that outlined basic requirements that should be met (outlining the code changes, install steps, pre-requisites, notification and approvals)Step 3: The push / pullReleases were tracked as “Epics” (Parent User Stories in Agile terminology). Product Owners own theEpics and would work with Architects (forming the core team) to break it down into child user storiesthat carried the technical details related to how the business requirement can be met. These “childuser stories” would reside in the team backlogs, get executed and deployed in the sequence thatwould be defined by the lead architect of the release. So, as you can imagine, stack team A wouldhave an a-la-carte of requirements from various releases.The focus during the first quarter was not on any agile tool but a simple Google Doc that was sharedwith the stack team members. The Google Docs used to have the child user stories (the sprintbacklog), the capacity allocation per stack team, the vacation plans of team members, customizedburn down charts using the planned v/s actual values. The Scrum masters were the glue between theteams and used to manage the coordination between the various teams (using scrum of scrums as aforum)Tracking releases in executionThe key thing by which releases could be tracked was to park them in the right state. Each releasetraversed through:Pipelined > Approved > In Ideation > In Execution > Deployed > Under Review > Closed7 Page
  • 8. Each of these release states had a gating criterion through which the states moved by the releasemanagers.States Explained - Pipelined, Approved & In IdeationEach release gets pipelined for approval to the executive team where the releases are scrutinizedfor completeness, business need and ROI and then approved. The approvals are obtained in a largerforum where all other releases compete in the portfolio and stakeholders can substantiate their case.This used to be the global prioritization meeting chaired by the executive management (Chief ProductOfficer and Chief Technology Officer)Process: (Primarily driven by core team with help of Scrum Master) - Iterative Process 1. Ideation between Product Owner, UX and Architect (Core team) to arrive at the first level Release Backlog (Owner: PO) 2. Iterate on the low-fidelity mocks by the UX team, if applicable. (Owner: PO and UX) 3. Lead Architects discusses with their peers (within other stack teams) to arrive at the dependencies between stack teams and document them (Owner: Architect) 4. Generate artifacts on the system level design changes (Owner: Architect + Lead Engineers) 5. Discuss with Engineering managers, Leads, Technology operations and other stakeholders (Owner: Architect) (Iterative Process) 6. Discuss with Biz Ops, Business, Sales (Owner: PO) (Iterative Process) a. Does sales have a need for this release? Do they have a potential customer base? b. Does this require a PR? Does this require an organization wide training? 7. Discuss with PO continuously to identify if the system that is being developed is what is really being envisioned, tying everything together (Owner: Architect)Output : Release moves into Ideation 1. Child User Stories under the PI (Owner: Architect) 2. High-level Technical Specifications - Architectural / Design Documents (Owner: Architect + required members of technical team) 3. Low Fidelity Mocks (Owner: UX)States Explained - In ExecutionThis is the phase where the team can begin the sprints using the release backlog. This is essentiallythe sprint planning and execution cycle where there is a constant collaboration between Product,Scrum Team, Core Team and other stakeholders with the help of daily standup meetings, statusreporting, retrospectives and demos. The key to this phase is that the teams self-organize, worktogether, inspect and adapt using all the agile tools (planning, daily reporting, retrospectives). Eachstack team would use the pull method to move the requirements from phases (dev > qa >deployment)Input: 1. Child User Stories updated from the previous stage 2. Architectural Artifacts (Design WIKI, diagrams) 3. Sign-offs from the dependent stakeholders (Ops, Sales)8 Page
  • 9. Process: (Primarily moderated / driven by Scrum Master) 1. Release Planning (with few stakeholders) - To identify the tentative TTM / number of sprints (Owner: Architect + PO + Scrum Master + Few key stakeholders from the scrum team + Tech and Biz Ops) 2. Capacity Planning - To identify the availability of time of each team member (Owner: Engineering Manager(s)) 3. Sprint Planning and Sprint ExecutionOutput: 1. Deployments based on Release Dependency Matrix (Owner: Scrum Team) 2. Updated Communication Plan (blocking calendars for demos, look ahead planning for the future sprints etc) (Owner: Scrum Master)States Explained - Deployed / Under ReviewEach release post-deployment (of all stacks) moves and stays in Deployed till reviewed by theTechnology QA team. Under Review is the phase where the release is reviewed for its adoptionmetrics in the field.The Execution Framework - Sprints, Stand-ups...Typically 50% of releases would span across multiple sprints across multiple teams. The core andscrum teams perform a high level release planning (and breaks it to sprints) to identify the sequenceof execution, the dependencies etc. Each sprint typically would require engineers across the stacks tocontribute. In order to ensure the technical dependencies were managed well, each contributing stackteam has to agree to a “contract” that gets defined between the teams that would define how arelease would cut across the entire system. The lead architect who is part of the core team woulddefine this contract and pass it on to the teams to elaborate and suggest alternatives or changes.Also, the expectation is that technical bottleneck of the management of code branches and mergewould need to be handled within the stack teams.9 Page
  • 10. The Simple 3-Step Framework (push & pull)Figure 1 – A 3 Step Process of Release (push to backlog)Each team used to run their sprint planning and sprints and the Scrum masters used to align thedeployment sequencing with the help of key members in the teams. There were trials done in terms ofmaking all releases begin and end on the same dates but realized that the key stakeholders whoneed to be involved in sprint planning (architects, engineering owners, line managers) were stretchedthin between meetings. Hence we realized that we needed to have the teams run at their own paceand it was left to the Scrum masters who doubled up as release managers to tie up the deploymentsequencing.The Pull – KanbanWithin the backlog, the entire stack workflow uses the Kanban mode of “PULL”-ing the user storiesfrom one state to another. Once US is ready for QA, the QA team will pull it into QA based on the exitcriteria specified as a team.10 Page
  • 11. Arriving at Release DatesOnce the dates are arrived at each of the stacks (Code Complete, QA signoff, Deployment), thelongest pole in the entire release including integration testing timelines was taken as the final“Targeted Release Date” and communicated to the field teams. The sprints continued to move andthe dates for the releases are arrived at as and when the deployment dates are available. ScrumMasters owned the diligence in tying up these deployment dates to deliver releases, consistently.Figure 2 – Arriving at Release Date based on Deployment SequencingManaging Release-Level Visibility – The Key to Success!Though the engineers from the stack teams worked within the stack, the daily stand-up meetings,dependencies and the status would be looked at also from the release level. The standup meetingsproved extremely important to discuss if there were any slippages, contract issues and blockers. Afocused release-level daily stand-up meetings (also called the Scrum of Scrums) had keyrepresentatives from each of the stacks coming together in analyzing how the releases wereprogressing and how the teams could help each other move forward towards the release. Thesemeetings helped in reporting the release level status, which became the consistent focus within theorganization.The Leap into Phase 2Taking the learning from Phase 1 using retrospectives with teams and executive sponsors, the teamsmoved into Phase 2 around end of Q4. By then, the teams had multiplied thrice in size and thenumber of releases had doubled (posed a big challenge). The teams moved into managingreleases as individual Portfolio Items but the core push/pull framework remained intact (just theterminologies were changed). The reason to move into Portfolio Management was to provide the11 Page
  • 12. visibility to the executive team on where the team and their capacities were being utilized (bucketinginvestments)Figure 3 – The current framework that is based on PI and US(PIxyz denotes the epic and USxyz denotes the child user stories below the release)6. Key Benefits of the frameworkThere was couple of benefits (both qualitative and quantitative): 1. The framework has been time tested and proved that teams could still get releases done even if there were multiple dependencies. 2. Teams not just in Bangalore but also across the globe could adopt to the framework and had clear visibility into where their dependencies were (which stack, what sprint, when it will be deployed, to name a few) 3. The framework enabled us to scale to around 50 parallel releases during Q2 of 2012 4. In the last 4 quarters, the number of releases that were done was close to 120+ (with the last 2 quarters contributing to 60%). Even when the initiatives were brought in to accelerate the execution, the framework stayed put and guided the releases to closure.12 Page
  • 13. 7. Key Challenges OvercomeThough the framework sounds very straightforward and commonplace, the following were thechallenges that had to be mitigated Operate from week 1 in a no-pilot mode (implement the framework across the teams at one go) without slowing down the existing releases. Experiment, learn, adapt, revise and re-implement in fast turnaround cycles (across 12+ teams) Sudden surge in the team sizes due to business needs (adoption of framework, adapting to new business asks and timelines) o Pressure on teams to deliver more due to the surge in team sizes and in turn the capacity Executive team willing to provide leeway from the existing framework to see if that would speed up execution (this put things out of gear which had to be brought back in) Stack teams having sudden priority changes (tech debts and scalability issues) thereby jeopardizing the entire release timelines Adoption issues due to business pressures Frequent shift in business priorities (which is a given in any industry) leading to randomization of releases o All of a sudden towards end of Q1 2012, the organization had thrust a mandate of 30 releases in 30 days and teams had to huddle to accomplish them. The framework still came in handy to lay out the dependencies, t-shirt sizing and release level tracking rd o The teams are in the 3 such initiative post that8. ConclusionIt is strongly recommended that the team who defines the framework needs to start off with an openmind for feedback and failures. The first step to defining any framework is a complete analysis of theexisting bottlenecks than to go with a silver bullet of Scrum, XP or mix of other methodologies. Thesecond significant step is to work with key stakeholders to see if all their demands have been met(else the adoption becomes bleak). It is extremely important to set expectations with senior andexecutive management on what milestones will be achieved in the adoption cycle. This frameworkabove helps teams who have dependencies in managing releases. In a nutshell, the chain is asstrong as the weakest link. Hence, it is imperative that there is a unified process across the entireorganization so that everybody moves at a sustainable pace. It is equally important that allstakeholders play their part well so that things do not break down.9. References 1. Scrum Guide: http://www.scrum.org/storage/scrumguides/Scrum%20Guide.pdf 2. Scrum Primer: http://assets.scrumfoundation.com/downloads/1/scrumprimer121.pdf?129464083813 Page
  • 14. 10. Author’s Profile: 13 years of IT experience with excellent exposure to various Project Management Methodologies - Agile Scrum (Iterative and Incremental) with XP, Lean, Kanban and Traditional. Has strong experience and exposure in both product and service companies. Currently heading the Technology Project Management Office at InMobi responsible for driving the portfolio of releases across 18+ engineering teams. In the last two engagements, has owned the responsibility for coaching, mentoring teams to adopt Agile Scum as the delivery framework. Leading a team of 4 Project Managers who are playing both the scrum master and release manager roles. Email: venkat@inmobi.com14 Page

×