Why Agile? Why Now?

  • 422 views
Uploaded on

An overview of Agile and Scrum practices, and a comparison of 2 different, real-life Agile adoption experiences in a University setting. Presented at the Wharton Web Conference, July 2011

An overview of Agile and Scrum practices, and a comparison of 2 different, real-life Agile adoption experiences in a University setting. Presented at the Wharton Web Conference, July 2011

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
422
On Slideshare
0
From Embeds
0
Number of Embeds
1

Actions

Shares
Downloads
5
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Why Agile? Why Now? Jorj Bauer <jorj@upenn.edu> @bozoskeleton Mike Toppa <toppa@upenn.edu> @mtoppa Wharton Web Conference – July 14, 2011Thursday, July 14, 2011
  • 2. Overview We’re going to talk through two perspectives on the adoption of Agile methodologies What drove us Why we approached Agile How we approached adoption Our perspective from where we are nowThursday, July 14, 2011
  • 3. First: who are we?Thursday, July 14, 2011
  • 4. Jorj Bauer Manager of Engineering, Research and Development for Penn/ISC Professional software developer since 1994 Co-owner of DejaVu Software, Inc. 2004-2008: Director of Networking, Penn/SEAS 2008-present: Manager of ERD in Penn/ISCThursday, July 14, 2011
  • 5. Jorj: The Past Most of my software development background has been ad-hoc, with little to no supporting infrastructure Goals and requirements have been very fluid, with small teams ... until my current positionThursday, July 14, 2011
  • 6. Mike Toppa Director, Web Applications for Penn/SOMIS 15 years of experience in web programming, IT project management, and functional management Georgetown University; Stanford University; University of Pennsylvania E*Trade; Ask Jeeves 2 failed start-ups (Finexa, Kai’s Candy Co.)Thursday, July 14, 2011
  • 7. Mike: The Past None of the places I’ve worked followed any formal development methodologyThursday, July 14, 2011
  • 8. Mike: Becoming Director My team was overwhelmed, but didn’t fully realize it Like boiling frogsThursday, July 14, 2011
  • 9. Mike: The Clients Clients come from across the School’s administrative, academic, and research organizations Hybrid funding model Clients were working directly with developers Usually no project managers Reactive planningThursday, July 14, 2011
  • 10. Mike: The Projects A large number of projects Little to no documentation No one in a position to prioritizeThursday, July 14, 2011
  • 11. Mike: The Team Talented Independent Customer service oriented Focused on fast deliveryThursday, July 14, 2011
  • 12. Mike: The Dev Environment No specific development procedures An aging development framework No automated testing Non-critical bugs and missed requirements common Daily firefighting Difficulty planningThursday, July 14, 2011
  • 13. Jorj’s challengesThursday, July 14, 2011
  • 14. Jorj: The New Department In 2008, I became manager of 5 direct reports, in a department of ~30, full of dotted-lines and large projects Part of a reorganization of the department Two large projects-in-progress with campus-wide scope were instantly “mine”Thursday, July 14, 2011
  • 15. Jorj: The Projects Rapid shifting of priorities Large, with no distinct requirements Disconnect between technical requirements for a good service, level of effort to learn new services, and management expectations of level of effort required Everything is behind schedule A lot of fire-fighting related to the reorg as well as project management culture in the departmentThursday, July 14, 2011
  • 16. Jorj: The Team Very talented and conscientious Not a lot of project focus; rapid shifting of priorities A lot of unrest due to uncertainty and perceived political stupidityThursday, July 14, 2011
  • 17. Jorj: The Dev Environment Everyone For Themselves Central RCS and CVS repositories, so some people chose to set up their own SVN repos No specific development procedures No automated testing Projects begin and take priority based on political demandThursday, July 14, 2011
  • 18. Jorj: The Biggest Challenge “Project Management” happened between developers and their direct management – no higher, and no wider The culture of “how project management works” was dysfunctionalThursday, July 14, 2011
  • 19. Prioritizing Challenges Both of us approached our situations similarly: 1.Get a handle on existing and upcoming projects 2.Improve predictability 3.Improve qualityThursday, July 14, 2011
  • 20. Three Approaches... In software development, you’re looking at one of these three general approaches: Ad-hoc (informal) Waterfall (anticipation) Agile (adaptation)Thursday, July 14, 2011
  • 21. WaterfallThursday, July 14, 2011
  • 22. Waterfall: why so pervasive? “Nobody ever got fired for buying IBM” Many people understand Waterfall It seems sensible to wait for one thing to be complete before working on the next The customer gets a whole solution, all at onceThursday, July 14, 2011
  • 23. Waterfall: low success rates http://www.ambysoft.com/surveys/Thursday, July 14, 2011
  • 24. Waterfall: why does it often fail? Customers never know what they want up front; requirements always change Customers and developers speak two different languages Progress reports are often wrong, or set up a false expectation If I’m 100% done the first task of 4... am I 25% done now?Thursday, July 14, 2011
  • 25. Ad-hoc Home grown workflows, if any Prioritization is arbitrary or politicized Often hard to do long term planning Success dependent on personalities and circumstancesThursday, July 14, 2011
  • 26. Agile We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value. Individuals and interactions over processes and tools Working software over comprehensive documentation Customer collaboration over contract negotiation Responding to change over following a plan That is, while there is value in the items on the right, we value the items on the left more. ©2001 authors of The Agile Manifesto agilemanifesto.orgThursday, July 14, 2011
  • 27. Agile is incremental and iterative Graphic by Jeff PattonThursday, July 14, 2011
  • 28. Agile: “inspect and adapt”Thursday, July 14, 2011
  • 29. The Agile Umbrella “Agile” applies to a large swath of development methodologies, many of which overlap Scrum; Kanban; Crystal; Extreme Programming (XP); Lean... And their hybrids Scrum-Ban; ADROIT; WetAgile/Agile Waterfall... Disclaimer: not all of these may be “good”Thursday, July 14, 2011
  • 30. Scrum: overview Graphic by Skip AngelThursday, July 14, 2011
  • 31. Slice vertically, not horizontallyThursday, July 14, 2011
  • 32. Thursday, July 14, 2011
  • 33. Mike: Why Agile? Why Scrum? Maintain frequent interactions with clients Provides quick feedback Existing good relationships But have them take more ownership of their business processThursday, July 14, 2011
  • 34. Mike: Why Agile? Why Scrum? Put structure around our work Enable planning Make commitments, and meet them Reduce need for firefighting Reduce risk Have more than one person know a projectThursday, July 14, 2011
  • 35. Mike: Why Agile? Why Scrum? Improve quality Reduce misunderstandings Reduce missed requirements Have fewer bugs Increase technical knowledge sharingThursday, July 14, 2011
  • 36. Authority vs. Responsibility Cartoon by Mike Lynch Used with permissionThursday, July 14, 2011
  • 37. What makes a job enjoyable? Autonomy Reward for effort Challenging/complex work “Work that fulfills these three criteria is meaningful.” – Malcolm Gladwell, “Outliers: The Story of Success”Thursday, July 14, 2011
  • 38. Scrum has 3 roles Product owner Authority and responsibility for “what” Programming team Authority and responsibility for “how” Scrum Master Authority over the Scrum process Responsibility for driving continuous improvementThursday, July 14, 2011
  • 39. Scrum role: Product Owner Responsible for what the team will work on, but now how the work is doneThursday, July 14, 2011
  • 40. Scrum role: The Team Takes collective responsibility for doing the workThursday, July 14, 2011
  • 41. Scrum role: Scrum Master A “servant-leader” for the teamThursday, July 14, 2011
  • 42. Agile Adoption Patterns Top-down vs. Bottom-up “All in” vs. incrementalThursday, July 14, 2011
  • 43. Mike: top-down Upper management: supportive Team: uncertain They recognized certain problems But learning and conforming to a process feels like a loss of autonomy and yet another thing that has to be doneThursday, July 14, 2011
  • 44. Mike: all-in Team learned core scrum concepts I explained the new process to clients A small team did a pilot project first Then the whole group switchedThursday, July 14, 2011
  • 45. Mike: initial transition It did not go very well Team not used to working together I misunderstood the roles I overemphasized doing only one project at a time Too many “scrum-buts”Thursday, July 14, 2011
  • 46. Mike: a visualization Cartoon by Esther Derby Used with permissionThursday, July 14, 2011
  • 47. Mike: using scrum coaches I brought in coaches They provided good training Led candid discussion of problems Most long pre-dated my introduction of scrumThursday, July 14, 2011
  • 48. Should you use a coach? A skilled, external coach is often key for driving change If you haven’t done it before, it’s surprisingly easy to introduce Agile the wrong way You need at least enough management support to pay them You need to make sure you’re bringing in someone goodThursday, July 14, 2011
  • 49. But it’s still difficult “In my Scrum classes I warn attendees of what I call the Scrum Promise: If you adopt Scrum, there will be a day you come into the office nearly in tears over how hard the change can be. This is because Scrum doesn’t solve problems, it uncovers them and puts them in our face. Then, through hard work we address them.” – Mike Cohn, Agile TrainerThursday, July 14, 2011
  • 50. Jorj: Bottom-up, IncrementalThursday, July 14, 2011
  • 51. Jorj: Bottom-up, Incremental Ad-hoc The “Cloak of Confusion” surrounded project management; most staff couldn’t see the problems, just the symptoms The staff perceived control in their current methodology; nobody wants to give up controlThursday, July 14, 2011
  • 52. Why Incremental? Convincing staff to change their culture is hard Resources completely overcommitted It’s difficult to “sell” a change without having data on how it will work It’s impossible to show results for a system without using itThursday, July 14, 2011
  • 53. NES’ Transition Phase 1: adopt formal project workflow Built a formal Iterative Waterfall process within the existing framework A lot of resistance and much more project planning but improved results for existing projects Increased awareness around product ownership, sources of delay, and available resources Warning: this will not make you new friendsThursday, July 14, 2011
  • 54. Incrementally Improving “Iterative Waterfall” is single- looped: allowed the team to abandon the waterfall process at many points (causing the project manager lots of additional work) Continue working on the process itselfThursday, July 14, 2011
  • 55. Showing Results This slowed everything down 3-4x the project management work (for me) Allowed us to find and address deep-seated problems in workflow Yields an inventory of projects Gave me the ability to say “No” to new workThursday, July 14, 2011
  • 56. Stepping back a moment...Thursday, July 14, 2011
  • 57. Saying “No” Is Important If you take on an infinite number of projects, then your staff are constantly task-swapping (unless you have infinite staff) Context switching between two projects eats about 20% of a full-time worker’s schedule Cutting back on the number of active projects is key to getting back lost productivity from a bottlenecked staffThursday, July 14, 2011
  • 58. Danger Signs of Meetings Being unavailable, or arriving In the meeting but doing late, for regularly scheduled other work while present meetings Suggesting radical departures from the current Never speaking in a meeting plan, at a stage that’s too you attend regularly lateThursday, July 14, 2011
  • 59. Reduce Meeting Complexity Agile methodologies adopt more frequent meetings (generally daily). Staff specifically talk about: What I did yesterday What I’m doing today What I’m waiting for This is a more effective use of tech staff time (But meetings may be a band-aid for other problems)Thursday, July 14, 2011
  • 60. Time/Project Management Brooks’ Law adding manpower to a late project will make it later Eliyahu Goldratt’s Theory of Constraints Identify the bottlenecks and use them to predict and control the workflowThursday, July 14, 2011
  • 61. Thursday, July 14, 2011
  • 62. Thursday, July 14, 2011
  • 63. NES’ Turning Point Two services: one brings in 2/3 of your department’s budget, and one brings in almost nothing. How do you prioritize the work? Answer: both have to be done as soon as possible. Better answer: look at the costs, benefits, and staff resources for both and be able to shelve oneThursday, July 14, 2011
  • 64. No Looking Back From here, big gains were made fairly quickly on teams that adopted these Agile practices: Daily stand-ups; reduced queue sizes; more direct interaction with people “outside” the team Better results on a shorter timeline within 4 months Managers discussing and inspecting the process itself and its results, and are improving upon it Not Done Yet!Thursday, July 14, 2011
  • 65. Mike: taking inventoryThursday, July 14, 2011
  • 66. Mike: taking inventory 9 developers, 2 product owners, and me supporting 22 clients with 124 applications 3 designers and 1 product owner supporting about 200 static content web sitesThursday, July 14, 2011
  • 67. Mike: our maintenance curveThursday, July 14, 2011
  • 68. Mike: project commitments for first 6 months... oh crapThursday, July 14, 2011
  • 69. Mike: the first 6 months Working through our over-commitment While learning a new way to work The continuous 2 week delivery cycle highlighted process weaknesses Staff changesThursday, July 14, 2011
  • 70. Planning and estimating How did we come up with that chart? Agile requirements gathering Agile estimating techniquesThursday, July 14, 2011
  • 71. Requirements gathering Breaking down a project into feature areas Epics Breaking down feature areas into individual features User stories How do you know when you’re done? Conditions of satisfactionThursday, July 14, 2011
  • 72. Estimating: story points and planning poker Photo by Kelly Hirano Used with permissionThursday, July 14, 2011
  • 73. Velocity enables schedulingThursday, July 14, 2011
  • 74. Mike: one year later We can plan and estimate better Our programmers work together Our next challenges A project prioritization and intake process Improving our technical practicesThursday, July 14, 2011
  • 75. Manifesto for Half-Arsed Agile Software Development We have heard about new ways of developing software by paying consultants and reading Gartner reports. Through this we have been told to value: Individuals and interactions over processes and tools and we have mandatory processes and tools to control how those individuals (we prefer the term ‘resources’) interact Working software over comprehensive documentation as long as that software is comprehensively documented Customer collaboration over contract negotiation within the boundaries of strict contracts, of course, and subject to rigorous change control Responding to change over following a plan provided a detailed plan is in place to respond to the change, and it is followed precisely That is, while the items on the left sound nice in theory, we’re an enterprise company, and there’s no way we’re letting go of the items on the right. http://www.halfarsedagilemanifesto.org/Thursday, July 14, 2011
  • 76. Technical practices: ridiculously brief overview “Specification by Example” by Gojko Adzic “Clean Code” and “Agile Software Development, Principles, Patterns, and Practices” by Bob Martin “Refactoring” by Martin Fowler “Agile Database Techniques” by Scott Ambler “Growing Object Oriented Software, Guided by Tests” by Steve Freeman and Nat PryceThursday, July 14, 2011
  • 77. Agile is relevant beyond software development “Angry Dinosaurs: Accelerating Change and Institutional Incompetence” Cory Ondrejka, Wharton Web Conference, 2010 http://bit.ly/bY4E15 “Let it Roll: Why more companies are abandoning budgets in favor of rolling forecasts” CFO Magazine, May 1, 2011 http://bit.ly/k94SfcThursday, July 14, 2011
  • 78. Other References Eliyahu M. Goldratt, “The Goal” (Theory of Constraints) Fred Brooks, “The Mythical Man-Month” Gerald Weinberg, “Quality Software Management: Systems Thinking” “Convincing Management That Context Switching Is a Bad Idea”, Johanna Rothman http:// www.jrothman.com/Papers/context-switching.html “In Praise of Bad Programmers” http://corvusintl.com/Thursday, July 14, 2011
  • 79. Any Questions? Jorj Bauer <jorj@upenn.edu> @bozoskeleton Mike Toppa <toppa@upenn.edu> @mtoppa Wharton Web Conference – July 14, 2011Thursday, July 14, 2011