Scrumban

Improving speed and effectiveness of inhouse software development Scrum(ban)*at Zahnärztekasse AG *and Crystal Clear Jens Meydam http://jmeydam.blogspot.com/

Zahnärztekasse AG? Scrum? Scrumban???

Zahnärztekasse AG ● around 35 employees ● offers financial services to dentists, mainly in connection with processing invoices ● total value of processed invoices: over CHF 250 million per year ● IT (obviously) plays a central role ● more information: http://www.zakag.ch/

Scrum How many of you have been to a Scrum course or have read a book on Scrum?

Scrum – elevator pitch You have thirty seconds to convince your boss to try Scrum. What would you say?

Scrum – elevator pitch (1) “Agile Software Development with Scrum” by Ken Schwaber and Mike Beedle, p. 82: Jeff Sutherland, VP of Engineering, went to the President of Easel Corporation, and asked him, "Have you ever seen a development team to meet a traditional project plan."

Scrum – elevator pitch (2) He said, "No."

Scrum – elevator pitch (3) Jeff said, "I think the only thing you can trust is working software that the team can demo. If you give the team complete freedom for 30 day Sprints, at the end of the Sprint you will see exactly where the product is, for better or worse."

Scrum – elevator pitch (4) He said, "Fine, for the first time I will know where product development really stands and can make the appropriate decisions. I'm willing to take the risk of giving the team autonomy for defined periods."

Scrum – roles ● Product Owner: decides on the direction of development – what will be developed in what order ● Scrum Master: helps the team to be successful, in particular by removing obstacles to higher productivity ● Team: is responsible for delivering the software with top quality in small increments

Scrum is a broad church There is more variation out there – and for good reasons! – than some people might like

Scrumban ● Scrumban = Scrum + kanban :) ● the name was coined by Corey Ladas in a slightly inflammatory article: http://leansoftwareengineering.com/ksse/scrumban/ ● here kanban = kanban system for software development (a Lean tool)

Kanban systems ● kanban systems for software development are task/story boards on steroids ● you saw a very basic one on the first slide:

Kanban systems ● there are some good introductions by well known members of the agile community: – http://blog.crisp.se/henrikkniberg/2009/04/03/1238795520000.html – http://agileproductdesign.com/blog/2009/kanban_over_simplified.html ● on InfoQ you can find a presentation by David Anderson, one of the leading lights of the kanban enthusiasts: – http://www.infoq.com/presentations/kanbanforsoftware

Now you know ... ● what kind of company I work for ● how to describe Scrum in thirty seconds ● where this horrible new term “Scrumban” comes from

Inhouse software development ● most of our software development enables and supports internal operations ● most of our users are in the same building ● there is only one installation ● the senior developers have full control over the production environment (of course, with great power comes great responsibility!)

Inhouse software development ● all this simplifies software development a lot (I know I used to work for a software company with commercial products) ● the problem of inhouse software development is that its importance tends to be underrated (because it "just costs" money) ● it is much harder to justify the kind of budget needed for serious development

Before Scrum ● at Zahnärztekasse AG, the dominant pattern had been to define small development projects and to assign these projects to individual members of the staff (who thus became "project managers", but this was usually just one of many tasks for them) ● this had worked reasonably well, but there were several disadvantages:

Before Scrum ● productivity suffered from multitasking (too many projects stayed "90% complete" for too long) ● there was too little collaboration among the development staff, which had a negative impact on design quality ● there was usually not enough momentum for a systematic improvement of the infrastructure

A new project ● in the first half of 2008 a consensus emerged that the time had come for modernizing the core infrastructure and the user interface ● we had many ideas for improvements that together would have a measurable positive effect on the business

A new project ● however, the project manager (myself), doubling as architect and developer, was mostly busy with another project ● the other core developer was also busy with other projects and tended to prefer to work remotely ● plus, we didn't have an expert in the GUI technology

A new project ● progress on this supposedly very important project was predictably slow

Introducing Scrum ● I had experience with a "homegrown" agile process and automated regression tests (in a domain where such tests are still uncommon) ● I was determined to introduce a "real" agile development process

Introducing Scrum ● I considered Extreme Programming, but feared that management would not see the point ● I also feared that we would never manage to faithfully stick to all those "extreme" practices

Introducing Scrum ● Scrum is more "managementoriented" there is nothing that doesn't make sense to a good manager ● in particular in our case, Scrum and the basic assumptions behind Scrum were a very good fit for the management philosophy of the CEO

Introducing Scrum ● the CEO had already played a Product Owner like role, including shielding developers from direct "urgent" feature requests by stakeholders ● the CEO had already come to value progress in small steps and the ability to rapidly react to market opportunities

Introducing Scrum ● we invited Peter Stevens to introduce the development team and in particular the CEO to the key concepts of Scrum, and then officially decided to use Scrum ● in October 2008 I took a Scrum Master class with Jeff Sutherland ● I was determined to implement Scrum properly

First steps ● Product: all internally developed software, divided into subsystems ● Product Owner: CEO ● Scrum Master: myself ● Team: development staff including myself

Lack of progress on “main” project ● the first three sprints were definitely “ScrumButt” (but also an exercise in the “art of the possible”) ● we continued to work more or less like before, until it was painfully obvious that progress on the "main" project was far too slow ● it is one thing to know that focus and collocation are beneficial, it is another thing to make it happen

Sprint 1

Sprint 2

Sprint 3

Impediments get removed key decisions in February 2009: ● the CEO as the Product Owner saw that insufficient capacity was partly to blame and increased the budget ● I myself in my reincarnation as a Scrum Master insisted on the developers working together in the same room at least most of the time

Collocation ● I promised the CEO 40% higher productivity if we work together in an adequately equipped team room (that was Jeff Sutherland's estimate) ● collocated development (three days per week) started in March 2009

Team room

View from team room :)

Sprint 4

Sprint 4 adjusting estimates is not worth the effort – in the end I decided to always use the original estimates for my bookkeeping purposes

Sprint 5

Sprint 5 some features in the Sprint backlog were changed – this is strongly discouraged in Scrum (note that we only track features, not tasks)

Sprint 6

Collocation ● comparing the three Sprints before and the three Sprints after partial! collocation, average velocity has actually doubled * * based on data as of 20.05.2009

Collocation ● although there is one more fulltime team member there are still only two developers who do most of the actual work ● there has been a genuine and substantial improvement in productivity per developer, and this has been recognized by everybody involved ● there are more factors involved, but 40% can safely be attributed to collocation

Collocation ● having the team work together in one room, insisting on concentrated work and keeping distractions away is the key for improving efficiency and quality ● close collaboration with users is another key ingredient for success ● if you have read about Crystal, this will sound familiar :)

Collocation Without collocation at least most of the time, you are likely to suffer from all of the following: ● lack of commitment ● slowdown due to inefficient communication ● lack of discussion of design issues leading to poor design choices (software development is essentially design work at multiple levels there are a lot of opportunities for bad decisions)

Bandwidth of communication if the three people involved had been working in the same room all this would have taken less than ten minutes

Removal of further impediments ● one developer (“MaCa”) was and is critical for rapid progress ● the dramatic improvement in productivity is to some extent due to the optimization of the whole environment for this key developer

Removal of further impediments

Project health ● the Product Owner/sponsor is happy with our progress, and has great expectations for the future ● morale is high – the team takes pride in the quality of the design and in adopting established professional practices, in particular test and build automation

Project health ● mind you this is a domain where automated regression tests are still not mainstream – indeed many would think that in our case automated tests are just too much work or even downright impossible ● kudos to Gojco Adzic for DbFit, which has been a godsend for us I don't know what we would have done without it!

“Something old, something new, something borrowed and something blue” ● what I have described so far is more or less standard Scrum (or Crystal) ● now comes the kanban part

Kanban board ● in the first collocated Sprint I introduced a kanban board in order to make our whole "value stream" visible ● as a first approximation, you may think of this kanban board as a standard Scrum story board that is extended to the left to include steps to prepare backlog items for development and to the right to include the steps coming after development until a feature is finally in production

Kanban board

Kanban board ● ESTIMATED ● PREPARE ● READY ● NEXT ● DEVELOPMENT ● DEVELOPED ● INSTALLED ● BEING USED

Kanban board ● this kanban board allows me to visually control the flow of all of our work in particular: – I want to see where work "gets stuck" – I want to see if downstream steps risk getting starved

Kanban board ● note that there is a rough space limit on the number of items in each column ● (I've cheated a bit, actually) ● kanban experts take this a lot further and make an art of getting the limits right

Lean ● in Lean thinking there is an emphasis on speed, on competing on speed ● the ideal is to get "from concept to cash" in a rapid flow ● improving quality and decreasing cost are so to speak side effects of this quest for speed ● ask General Motors if it works :)

Lean ● in our case, the "concepts" are small features and feature enhancements, normally estimated at between 1 and 8 ideal person days of effort ● small or at least uniformly sized units of work can help to improve throughput ● in our case, "cash" is whatever benefit is realized by the use of the feature

Inventory ● at the risk of oversimplifying a little: In Lean, inventory is the enemy, is "waste" ● this is an interesting idea in the context of software development: what is our inventory? ● our inventory is features that have been thought about and worked on, but are not yet in production

Inventory ● in particular, preparing a month's worth of backlog items for development – as needed for Sprint planning – means building up inventory ● I feared this would require a lot of effort and would distract us from development ● it would be more in the spirit of Lean to prepare each backlog item just in time for development

Before Sprint planning meeting

During Sprint

Inventory ● as it turns out, towards the end of the Sprint it is usually relatively easy to see how the items at the top of the backlog should be implemented ● this is due to the increase of knowledge during the development of the previous features

Inventory ● this can be compared to driving a car at night – as one drives, the headlights always show the next part of the road

Inventory ● the monthly rhythm of the review and planning meetings with our CEO has served us well ● since building up the Sprint planning inventory doesn't cost us much effort and since it is usually (almost) entirely consumed at the end of a Sprint we don't need to be too worried about this inventory

Inventory ● in fact, the monthly planning rhythm allows us to concentrate on development for at least three weeks in a row ● in our case this is probably more efficient

Inventory ● remember though that we only track small features and feature enhancements, not tasks ● features are decomposed into tasks justintime – a second Sprint planning meeting by the book for four weeks' worth of work would kill us :( ● shorter iterations are not an option – our Product Owner is the CEO, we have to keep the overhead for him as small as possible

Inventory ● much more serious are delays and inventory further downstream features that are already developed, but not yet in production

Inventory ● every day a “done” feature is not actually used in production the business loses a day's worth of benefit from using the feature ● perhaps more importantly, the developers and the Product Owner! don't get any feedback from production and may make bad decisions based on false assumptions

Cumulative flow diagram ● cumulative flow diagrams are a great way to visualize the flow in a kanban system ● I introduced a cumulative flow diagram at the beginning of our second collocated Sprint

Kanban bookkeeping

Burndown: DEVELOPED vs. INSTALLED

Cumulative flow diagram Cumulative Flow Diagram 160 140 120 100 ESTIMATED IN PROGRESS 80 DEVELOPED INSTALLED 60 BEING USED 40 20 0 01.04.2009 06.04.2009 11.04.2009 16.04.2009 21.04.2009 26.04.2009 01.05.2009 06.05.2009 11.05.2009 16.05.2009

Cumulative flow diagram ● it came as a slight shock to see how slowly features move into production ● here is certainly plenty of room for improvement ● however, in cases where a set of small features is only relevant as a whole it is natural for the feature set to "build up" in a staging area such as DEVELOPED before the set as a whole moves into production

Efficiency and effectiveness ● what ultimately matters to the sponsor is not “requirements” or “features” but whether the changes to the system have the desired effect on the business (effectiveness) ● if features and feature enhancements move into production fast enough (efficiency), the sponsor and the developers have a chance to maximize effectiveness based on feedback

Efficiency and effectiveness I would like to conclude the main part of my talk with a passage from Extreme Programming Explained: Embrace Change by Kent Beck

Learning to drive (1) “Extreme Programming explained” (1st edition) by Kent Beck, p. 27+28: [...] I jerked back to attention as the car hit the gravel. My mom (her courage now amazes me) gently got the car back straight on the road.

Learning to drive (2) Then she actually taught me about driving.

Learning to drive (3) “Driving is not about getting the car going in the right direction. Driving is about constantly paying attention, making a little correction this way, a little correction that way.”

Learning to drive (4) This is the paradigm of XP. [...] The driver of a software project is the customer. [...] Our job as programmers is to give the customer a steering wheel and give them feedback about where we are on the road.

Bonus Material: Crystal Clear ● Crystal Clear is a minimalist methodology for small teams extracted by Alistair Cockburn (remember: the best frameworks are extracted) ● The result of ten years of debriefing successful small teams – originally for IBM ● If you are trying to implement Scrum, you have started to deliver good results, and everybody on the project is happy, you are probably already using it :)

Crystal Clear – favorite quotes “This is the methodology I never knew I was already using. [...] illuminates why the things I've been doing work, and also why some of the things I was doing didn't work.” Jeff Patton “Consultants can tell you what we are. I think the most accurate term would be 'Crystal'.” Chris Matts

Crystal Clear – roles ● sponsor (corresponds to Product Owner – depending on how Product Owner role is implemented) ● at least one senior designer (often also coordinating the project) ● team of designerprogrammers collaborating with expert users

Crystal Clear – targeted properties ● frequent delivery ● reflective improvement ● osmotic communication ● personal safety (a basic level of trust) ● focus ● easy access to expert users ● technical environment with automated tests, configuration management, and frequent integration

Crystal Clear in a nutshell (1) The lead designer and two to seven other developers

Crystal Clear in a nutshell (2) in a large room or adjacent rooms,

Crystal Clear in a nutshell (3) using information radiators such as whiteboards and flip charts,

Crystal Clear in a nutshell (4) having easy access to expert users,

Crystal Clear in a nutshell (5) distractions kept away,

Crystal Clear in a nutshell (6) deliver running, tested, usable code to the users every month or two (quarterly at worst),

Crystal Clear in a nutshell (7) reflecting and adjusting their working conventions periodically.

Reading tips Agile Software Development and Crystal Clear* by Alistair Cockburn and Agile Project Management by Jim Highsmith – arguably the best books on Scrum around – after Ken Schwaber's writings of course :) *chapter 1 is the most lighthearted and enchanting introduction to agile software development you can imagine

Reading tips Organizational Patterns of Agile Software Development by Jim Coplien and Neil Harrison – based on “empirical studies of about 100 organizations”, originally for Bell Laboratories Jim Coplien is one of the true thought leaders of the agile movement (he seems to have a high opinion of Scrum, by the way)

Reading tips Lean Software Development and Implementing Lean Software Development by Mary and Tom Poppendieck – guess who wrote the forewords :) Scrumban – Essays on Kanban Systems for Lean Software Development by Corey Ladas – some very clever writing questioning mainstream agile thinking

Reading tips Scaling Lean & Agile Development by Craig Larman and Bas Vodde – a very recent (2009) book by solid agile practitioners with a deep understanding of Lean and the application of systems thinking and queueing theory – cautioning against going overboard with isolated Lean techniques (in particular kanban)

Reading tips About Face – The Essentials of Interaction Design by Alan Cooper et al. the ideas in this book can help a lot to make software development more effective (as opposed to merely efficient) also have a look at Jeff Patton's blog: http://agileproductdesign.com/blog/

Questions?

http://www.slideshare.net	14
http://twitter.com	11
http://zootool.com	1

Scrumban

by Jens Meydam on Aug 18, 2009

Accessibility

Categories

Upload Details

Usage Rights

3 Embeds 26

Statistics

Scrumban Presentation Transcript