The Straight Jacket of Agile Iteration


Published on

My presentation on Agile Vancouver conference in 2011

As the goal of Agile evangelists was to convince people to switch from long Waterfall projects, the main message was to think small – short iterations, no upfront design, and requirements that fit on a card. This presentation explores limitations and pitfalls of a purely iteration focused approach and discuss different ways to address them while still retaining the speed and flexibility of the Agile approach.

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Periodically present on Agile and Dev conferencesMy experience with introducing Agile at WebCTTalk about Partnerpedia How easy to get Agile going in RonRIt comes natural to them they don’t know any other way
  • Agile started 10 years ago and it was a quite radical changeMy experience with introducing Agile at WebCTPortfolio projectDone in waterfall Product mng/Architects/DevelopmentYear long release Change in the middle of the projectCould not do it the old way
  • I master printing this long MS Project files and placing them on the wallLooked very impressive
  • Piles of document at WebCT too many documents that never made it into the codeProduct management did not get everything they wanted in the first Agile project but confirmed that the most important features made it
  • Architects that forgot how code looks like
  • Not small deedThis is why so much efforts went into this
  • When you ask somebody what process you use in many cases you hear we are doing iterations
  • Create enough requirements for one iterationDo enough design to get you through the stories for this iterationBoard is a sufficient planning tool for one iteration
  • And it workedAnd it was funYou can see something accomplishedI started to think in a different way about how to split stories vertical vs. horizontal approach
  • Main message was think small focus on iterationFocus all energy on one iteration No point in longer planning as we are agile and everything is going to change anywayAny attempt to think ahead were claimed by purists non-agile and watterfallishThe blinders cover the rear vision of the horse, forcing it to look only in a forward direction and keeping it on track.
  • Some of agile evangelists assumptions are not true in real world Customers and management want to know exact dates and scope and ask for your commitment upfrontThere is no customer on siteYou cannot refactor your way out of the  mess
  • Some people in agile community took this message think small too literallyWith this think small message are we missing big picture
  • Waterfal problem was not in defining the big picture and long view of a projectProblem was it asked to define each small detail in this long view
  • Taking to literally an iteration limits our vision and Create enough requirements for one iterationDo enough design to get you through the stories for this iterationBoard is a sufficient planning tool for one iteration
  • Lean approach is offering a better approach focusing on value and optimizing the way to deliver itAny misbalance in the flow reduces the value deliveredExample from WebCT waterfall – Product management vs. ArchitectsCan this happen in Agile?
  • Lean approach is offering a better approach focusing on value and optimizing the way to deliver itAny misbalance in the flow reduces the value deliveredExample from WebCT waterfall – Product management vs. ArchitectsCan this happen in Agile?
  • Famous backlogIn simplified Agile project examples life is simple enough and stories for the next iteration can be determined discussed in clarified on the iteration planning meeting Product owner is present and knowledgeable and answer to all questions from the team can be easily answeredAs we all know it is not always the case in the real worldNot all knowledge is in the roomSome issues require research and deep understanding of the market requirements and use casesCustomers, partners, and other third party may need to be consulted or even approve the work before team can startBusiness Domain may be new for the teamBacklog needs some work before it is ready to be presented to the team, in some cases a lot
  • If everybody is focused on the current iteration who is preparing for the next
  • This is the process I find usefulYou don’t need to have all requirements ready for a year worth of work but keeping 1-2 month worth of Ready-Ready stories is a good practice
  • Usually, the same people are involved in running / supporting the current iteration and planning the next oneAll our instincts and training tell us NOWBut if we follow them we pay for it later
  • Other thing we need to do as part of planning to achieve the flow
  • - As Agile was fighting for recognition in waterfall world it steered away from Big Design Up Front emphasizing incremental design and ad-hoc decisions done by the team- Many people interpreted this as NO DESIGN no design is needed we have refactoring and it works on small and relatively simple projects or when team has enough experience and doing the work that is not completely new - In small teams, that process can be fluid and ad-hoc, often verbal.  - As the number of developers increased, it creates the need for more structure around communicating architectural decisions.  This communication need creates the need to do more architecture ahead of each iteration. - With big teamArchitecture is so much more important because designing and communicating the architecture is so important to keeping the software consistent.  - Because the architecture needs to be communicated more broadly, it has to be defined earlier in the process.  
  • Architects that forgot how code looks like
  • Time pressure is always a part of s/w development projectEven if we realize the need to refactor we may be pressed into continue with a wrong design pattern making future refactoring more and more costly
  • Examples:Multitenancy Price lists
  • Architecture envisioning - It's important that the technical team establishes a high level architectural vision, including user interface flow, the organization and behavior of high level components and services, and a core domain model. You don't need, nor do you want, a complete and detailed architectural roadmap. But you need a consistent technology vision shared across them team. As the application grows, develop a system of checks and balances that activates the lifecycle to maintain your architectural resiliency.
  • So how can you be smart about all of this?  Although you don’t want to overbuild your system based on future/mythical requirements there isn’t anything wrong with thinking about the future.  It is a two phase strategy:Initial modeling. Do some initial architecture envisioning to think things through, to explore critical concepts and thereby get you going in the right direction.  This doesn't mean that you need to create mounds of architectural documentation, although it is likely that you will do some modeling and yes, egads!, even create sufficient architectural documentation to meet your needs. Defer design decisions.  One of the principles of lean software development is to defer commitment to the last possible moment that you need to make the decision, thereby increasing your flexibility and raising your chances of success. 
  • Level of details depends on how close you are to start implementation
  • As lean software development tells us, we shouldn't commit early to an architectural strategy but instead should consider several alternatives and to keep those alternatives "open" to us as long as they remain viable.  The implication is that when you are envisioning the architecture early in the project you should really be envisioning several possible architectures.  To be fair this strategy isn't new and in fact is one that has been promoted, although not always followed, within the IT architecture community for decades.
  • Multitenancy as an example
  • Define spike – the goal is to gain knowledge not to produce codeDo them in different iterationsDocument resultsResist temptation to fit implementation into the iteration even if successfulExample – selecting report building tool1st iteration to do tool selection2nd iteration to spike a solution using 1-2 top candidate3rd iteration to build into your product
  • Regression is not automatic and is time consumingIt does not fit into the iterationThere is a technical dept to pay and it cannot be done in one iteration
  • Paying technical debt need to be planned, sometimes across multiple releasesUAT for customer – working with waterfall enterpriseTools selectionUnderstanding of business value of each s/w component, associated risks, and current test coverage
  • Do we like it or not we may be faced with the need to do a full regression test after the iteration is complete but before new version can be released
  • Use performance spike to validate design decisions Do focus performance testing for completed features
  • CEO Our financial results for iteration 24 are …Agile evangelist assumed that we development can make business to adopt our speak and habitsFat chanceBusiness plans in quarters, financial years, conferences, industry shows, REVENU
  • Does not matter if we had not have a chance to look at requirements, create stories, play estimation pokerHow do we do it?
  • Not much advice in Agile community. One of the well groomed elephantsInstead books are going into length why this is a bad ideaAs this is any help, I know this is a bad idea. I still need to give a dateFormal way – Business points Gut feel Guru estimations Short spike to figure it out
  • Production points
  • People tend to focus on development only, look at the flow instead
  • Understand dependenciesDon’t forget the ground work as server setup, updates, etc, testing, code merges
  • In real life software development does not happen in a walled garden of one team and one iterationAn ability to wall garden your development project is very limited in most of organizations
  • UAT by customerWaiting for a decision to be made by management, customer, partner, 3rd party library release
  • In conclusion
  • The Straight Jacket of Agile Iteration

    1. 1. The Straight Jacket of Agile Iteration Michael Vax VP Engineering, Partnerpedia
    2. 2. About myself• 30 years in s/w development• Developer, architect, manager, VP, CTO• Discovered Agile in 2005• Implemented Agile process in 4 companies• Involved in Agile Vancouver for last six years• Currently - VP of Engineering at Partnerpedia
    3. 3. Waterfall  Agile Large  Small
    4. 4. The main goal of Agile’sevangelists was to prove that small works
    5. 5. That required simplifications Green field projects Customer on site Development team can define deadlines Simple design Easy to change
    6. 6. Agile = Iterations
    7. 7. Iteration – Focal Point of Go Small Message Iteration n-1 Iteration n Iteration n+1 Plan Implement Demo
    8. 8. DisclaimerI love iterations!!!I love iterations!!!I love iterations!!!
    9. 9. Horse Blinder Vision Iteration Plan Implement Demo
    10. 10. Are real-life projects andsituations more complex thanexamples used in Agile books?
    11. 11. Are we missing the Bigpicture if we focus on iteration too much?
    12. 12. There is a life beyond iterationWhat needs to How to split time between work forbe done before current iteration and an iteration? future planning? Iteration n-1 Iteration n Iteration n+1 Plan Implement Demo What needs to be done after the iteration? How to see the big picture while staying Agile?
    13. 13. Lean Concept of Flow And Optimizing the Whole Require Idea Design Dev Test Release Product mentsInvestment Value The goal of the development process is to optimize delivery of the business value
    14. 14. Lean Concept of Flow And Optimizing the Whole Require Idea ments Design Dev Test Release ProductInvestment Value Can this happen in Agile?
    15. 15. Requirements
    16. 16. Sub-optimized Flow Require Idea Design Dev Test Release Product mentsInvestment Value If everybody is focused on the current iteration, who is preparing for the next one?
    17. 17. Ready-Ready to be Done-Done The Product Owner is asked, “Are you ready-ready?” and the team is asked “Are you done-done?”.“When teams concurrently make work ready fornext iterations during development, we find quite often they can cut the iteration length in half”
    18. 18. Not all features are the sameFeature CharacteristicsMinor • Done it many times • Can be assigned to almost anybody on the team • no room to misinterpret requirementSmall • Enhancement to existing feature • Minor UI changes consistent with previous implementations • No technical challengesMedium • Enhancement to existing feature • Crosscutting multiple modules • Some level of technical uncertaintyLarge • Completely new feature or re-architecture of existing functionality with functional enhancements • Unclear domain questions – is bundle a product? • Requires selection of new technology or design patterns • New UI paradigm
    19. 19. Paths to code ready stories User MinorFeature Story Small User User User UserFeature Story Story Story StoryMedium User User User User User UserFeature Story Story Story Story Story Story User User User Story Story Story Large Epic Epic EpicFeature User User Story StoryLegend User Initial Ready for Definition Review Ready Story
    20. 20. Ready-Ready Epic• Domain questions are identified and answered• Usage scenarios are understood• Main UI flow is defined (mock-ups)• Core technical decisions are made and validated
    21. 21. Code Ready Story• Can be estimated by team in no more that 15 minutes• Is unlikely to be stopped or canceled due to unclear requirements or technical challenges• Tests are defined• Domain Changes are understood, reviewed, and consensus reached• UI mockups are reviewed by both end-users and designers
    22. 22. Metric to measure Ready-Ready• Flow in implementation of a story: Is story implemented without breaks in calendar time and context shift?• Example: Assume a story is 3 points story. However, for various reasons it becomes 9 points story. The flow in this case is defined as 3/9 or 33% Optimize the flow!
    23. 23. Iteration n Iteration n+1 Iteration n+2Plan Implement Demo Focus on NOW or look ahead?
    24. 24. Design & Architecture
    25. 25. No BDUF != NDUG (Big Design Up Front) (No Design Up Front) Every system has an architecture
    26. 26. Sub-optimized Flow Require Idea Design Dev Test Release Product mentsInvestment Value
    27. 27. We can always refactor later
    28. 28. Complex changes can takes weeks or even months to complete
    29. 29. High level architectural visioninstead of detailed architectural Roadmap
    30. 30. Think About The Future But Don’t Overbuild Lean Principle - (Defer Commitment)
    31. 31. Prioritization of Design Work
    32. 32. Consider Several Alternatives Lean Principle - consider several alternatives and to keep those alternatives "open" to us as long as they remain viable
    33. 33. Change Case ModelingDescription(A couple of sentences or a paragraph describing the basic idea)Likelihood(Rating of how likely the change case is to occur (e.g. High, Medium, Low or %))Timeframe(Indication of when the change is likely to occur)Potential Impact(Indication of how drastic the impact will be if the change occurs
    34. 34. Use Spikes to separate research from Implementation
    35. 35. Not enough planning smells• Difficulties with story estimations on iterationplanning meeting• Complete change of technical directions in the middleof iteration• Inability to answer developers questions whenimplementation starts• Abandon and incomplete stories• Idle development team waiting for requirements
    36. 36. Testing
    37. 37. Most of s/w projects have not been developed with TDD
    38. 38. Test Strategy cannot be limited to an iteration
    39. 39. Regression & Manual Testing Have not Disappeared
    40. 40. Performance Testing in Agile Project Benchmark Test Benchmark Test Benchmark Test Iter.0 Iter.1 Iter.2 Iter.3 .... Iter.x-1 Iter.x• Test environment setup • Load tests• Tools selection • Stress tests• Performance requirements • Capacity testing• Test data preparation Focus Test Focus Test Focus Test Focus Test • Resilience testing
    41. 41. Performance Testing and Iterations Performance Spike Iter. n-1 Iter. n Iter n+1 Feature A Feature B Feature CFeature A Feature A - Focus test Feature B - Focus test• Test requirements• Test development Feature B Feature C • Test requirements • Test requirements • Test development • Test development
    42. 42. Performance Testing - Agile Best Practices• Put performance tasks on a SCRUM board• Make fixing functional bugs that block performance tests a high priority• Do not delay fixing performance issues identified by tests• Retest after fixes are applied to know if they worked• You cannot guess where the next bottleneck is• Don’t pre-optimize before testing
    43. 43. Business vs. Development
    44. 44. Business does not plan in iterations
    45. 45. Business needs a Date and Commitment
    46. 46. How to do business level estimations in Agile?
    47. 47. Units of Estimations High Hours Level of Understanding Days Weeks Months Low
    48. 48. Look at the Flow Require Idea Design Dev Test Release Product mentsInvestment Value
    49. 49. Make it VisualGet a picture of wall fromPartnerpedia
    50. 50. We don’t Operate in a Walled Garden
    51. 51. Coordinating Branches & Teams
    52. 52. External Dependencies Customers, Partners,Other Departments
    53. 53. Find the Balance Iteration n Iteration n+1 Iteration n+2Plan Implement Demo Maintain long term vision but don’t focus on small things when looking ahead
    54. 54. Questions? Getting in touch: Email: Twitter: michaelvaxLinkedIn: