Agile Governance – Managing the Enterprise Issues


Published on

IT governance models have been developed largely to deal with traditional waterfall-style development. The rapid increase in the adoption of Agile software development raises a variety of important questions about governance:

• Can existing governance models deal with Agile development?
• How well have those existing governance models been dealing with IT performance and risk management?
• What new patterns for IT governance might be necessary to realise the benefits of faster time-to-market and better IT business alignment promised by more Agile delivery models?

This seminar will explore these and related questions from the perspective of lessons learned from enterprise Agile adoption.

• ThoughtWorks has been assisting organisations in Agile adoption for over 10 years and deals regularly with the challenges of governance and compliance protocols
• Suncorp is carrying out the largest Agile change programme in Australia and has had to grapple with numerous governance concerns
• Lonely Planet’s award-winning web site was recently relaunched after an extensive retooling of its development practices, featuring both Agile adoption and fundamental changes to its operational model

About the Speakers
Nigel Dalton, GM IT, Lonely Planet
When Nigel joined Lonely Planet in 2007, the seeds of an Agile IT organisation had just been planted. With ThoughtWorks’ assistance, Nigel introduced Agile practices across the enterprise and was instrumental at introducing and embedding Agile governance at the board level.

Josh Melville, Executive Manager, Suncorp
Josh is Executive Manager, Portfolio and Performance Services within Suncorp Business Technology and is responsible for portfolio and performance reporting, strategy coordination and risk and compliance. Josh was previously a Change Leader on the Suncorp BT Agile Change Program, responsible for maintaining the overall program of work, tracking performance and managing relationships with the Agile strategic partners.

Lindy Stephens, Professional Services Manager, ThoughtWorks
Lindy has over 10 years experience in working as a Project / Programme Manager for software delivery projects, mostly working with large Australian financial institutions. During this time, Lindy was often called upon to help organisations transition from more traditional software development approaches to what is now colloquially known as Agile software development.

Published in: Technology, Business
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • HISTORYAgile or iterative software development has been around since the 1960s and throughout the 60s and 70s various people were experimenting with ways to improve the way software was delivered. Interestingly, some of the earliest work in iterative and incremental software delivery was done at IBM and NASA, so hardly small companies!The Waterfall model of software delivery was a response to the ad hoc ‘code and fix’ practices of the early days (60s) that lacked rigour and controls. Interestingly, if you study the history of the rise to prominence of waterfall delivery practices, you find that many of the papers and standards used to support them are actually much more luke warm than they seem. This includes the US Department of Defence standards that were seen as the model to follow by other large organisations, and the famous paper by Winston Royce in 1970 that outlines the what became the basis of the waterfall methodology.In his book “Agile & Iterative Development”, Craig Larman refers to this as the Historical Accident of Waterfall Delivery.The dominance of waterfall style governance of projects really came into its own in the 90s, in response to the increasing proportion of failed software projects. It was seen as the control methodology that could get things back on track. We hoped that stricter governance could put a lid on these spiralling failure rates.The thing about waterfall delivery and the governance that surrounds it is that it sounds like it ‘should’ work. On paper it makes sense. It is simple and linear, whereas agile iterative development is more complex to understand. But we are delivering projects in the real world, and real world issues get in the way of well crafted simple plans every time.PICTURE – IBM AN/FSQ-7 was by far the largest computer ever built, and is expected to hold that record. It consisted of two complete Whirlwind II computers installed in a 4-story building Each AN/FSQ supported more than 100 users. IBM had about 60 employees at each site for round-the-clock maintenance.Keeping one unit operating and one on hot standby (to allow for switchover when vacuum tubes failed) resulted in better than 99% uptime. The roles of the two units were reversed at regular intervals, allowing diagnostics and maintenance to be carried out on the standby unit. There were usually several hundred tube failures each day, replaced by workers racing up and down the tube racks with shopping carts full of replacements. Automated tests run by the computer itself would cycle the voltage to the tube racks down and back up to induce marginal tubes to fail early, so that the computer would normally run correctly for the rest of the day. Without this process, the MTBF would have been a few minutes. By the time SAGE was deployed (22 or 23 stations in the period 1959-1963; sources disagree) it was nearly obsolete, since it was designed to detect bombers, not the new ICBMs. Nevertheless it was operational until 1979,
  • ILLUSIONSingle pass waterfall delivery gives the illusion of orderly, predictable, accountable and measureable process, with easy to define milestones that can be defined by simple deliverables (e.g. requirements document complete). “Plan the work, work the plan”The problem is that this is an illusion.For example, using a gating process to allocate money to a project creates an illusion of less risk at each stage as you are achieving a ‘milestone’ before more money is handed out. And you are assessing the progress of the project at each gate. The trouble is that in reality what actually happens is no-one is ever brave enough to cancel a project where several hundred thousands of $$ have been spent and the only thing we have to show for it is a couple of documents and some code that isn’t integrated with anything. The numbers are fudged at each ‘gate’ because no-one can bear the idea of so much lost money.Contrast this to an agile project where working software is delivered every few weeks. If a project is deemed to no longer be worth spending money on, you at least have something to show for all your hard work. The ‘milestones’ on agile projects are not documents, but working software that delivers value to the business.As an example, I was the project manager last year on a project for a TW client that was cut short for financial reasons at the beginning of the global financial crisis. We were part of a programme of work consisting of half a dozen projects, of which ours was the only one using agile. The other projects were all abandoned with nothing delivered, whereas our project was able to go live a few weeks later with only a small amount of additional money spent on the various deployment processes required in the large organisation. TW Australia has been involved in many similar stories over the years.
  • CONTROL & VALUETell story about TomDeMarco change of heart on control.Another problem is that we are often trying to control the wrong things. Software projects are not business as usual processes. Measuring timesheet data, amount of money spent, and how we are progressing against a schedule doesn’t actually tell us much about how we are progressing in our goal to deliver valuable working software. These questions tell us how the project is proceeding, not whether or not it will be successful.Instead we should be focussing on:Are there any changes in objectives or scope?Has the benefits model changed?QualityIs the backlog of scope items appropriately prioritised by business value?Are the project risks being managed?Are stakeholder involvement & comms being managed?These are the measures that will tell us whether or not the project is on a successful path.Tom De Marco bioAdd more here for my benefitPeopleWare & Waltzing With BearsWinner of 1986 Warnier Prize for “lifetime contribution to the field of software development”“The best bang-per-buck risk mitigation strategy we know is incremental delivery” – from Waltzing With Bears (De Marco & Tim Lister)
  • VISIBILITYOne of the main differences between agile governance and governance in more traditional project structures is VISIBILITY.Agile project delivery methodologies use highly visible tracking mechanisms such as: story walls, stand-ups, burnup charts, showcases, etc. These visible trackers are available for anyone associated with the project to see at any time, and provide real time data. There is nowhere to hide with agile.Compare this to a waterfall project. Reports on progress are made to the steering committee and sponsor by the project manager. They therefore only reflect one person’s view of the project – one person’s fiction (whether intentionally or not). You can say or not say anything that suits your needs. The same applies to the gantt chart, which rarely reflects what anyone truly believes will happen (why it is sometimes know as the “can’t chart’).
  • <0.5 minute slide> Hernando Cortes in Mexico in 1519. There are other routes (see Josh)! You might think agile was natural to a bunch of hippies – sort of ‘culturally appropriate’ in a way. Truth is LP came to agile in a very hard nosed fashion. Show the Vignette clock.
  • <3 minutes> Gladwell’s New Yorker speech on the GFC and the illusion of control (the stockbroker example; incompetence vs overconfidence). Governance is no longer ‘rules for losing’, it is operational transparency, accountability and responsibility. Transparency on the card walls; accountability on estimates; responsibility for quality via broken builds (red lights). How do we get to believe that waterfall = watertight?
  • <2 minutes> Standups, cards, pair programming, embrace change, retros, speed dating, poker, Fibonacci 1, 2, 3, 5, 8, 13, 21. We’re moving the finance guy to sit just to the left of this wall. Other examples: GIS implementation; SAP enhancements; New book design (6 mths to 6 wks)
  • <1 minute> Catches bugs, design better, code shorter, problem solving, learning, shared understanding, communication, enjoyment. Costs 15% more – repaid in shorter testing, less QA staff, and ongoing support costs. Not compatible with seagull product management – reveals weaknesses with a delivery every 2 weeks. Reminder – people don’t learn in a waterfall fashion.
  • <3 minutes> What’s the velocity of your end of month finance reporting? The day the LP chef asked me why we only managed 18 points in the last release was a watershed. But it doesn’t stop people sniping from the sidelines without engaging – walk them round the boards regularly. Also apply lean thinking to the software delivery process.
  • <0.5 minute> Ford 6 times a day = crisis; Toyota = 27,000 time a day at Toyota City. You’ll need deviants to make this work.
  • <2 minutes> Explain the comfort lawyers and CEOs have with contracts, and the changes we had to make to our MSAs and work orders! Depreciation rules cause problems – ‘build, freeze the asset, depreciate’ actually defies reality that 80% of the costs of owning s/w is in the production period.Accountants get very excited about our metrics (eg $/point) – use them to write ‘rules for losing’. Tell story of the CFO calling the website launch a ‘fail’, was not part of the prioritisation that sent various cards to the bottom of the pile.The best bang-per-buck risk mitigation strategy we know is incremental delivery” – from Waltzing With Bears (De Marco & Tim Lister)
  • <2 minutes> Lean, agile and ITIL (sdm, change and release), talent, process … it’s about who has lunch with whom. Ask yourself why ITIL works a lot more than software development?
  • <0.5 minute slide>
  • <3 minutes> Acid test for Lonely Planet agile (this and using it to make books). TW as Partners, longitude on our side. The daily rhythm working to your advantage. Lean = send them your best person, the one it really hurts to lose. Then standups, cards, prioritisation, ops in the room.
  • HandoverThank you NigelCan a large, corporate environment really sustain a dynamic, highly interactive way of working ? Surely, as most corporates tend to orientate around mitigating risk to the n’th degree through too much process and too much consistency, the creativity and freedom Agile engenders is severely restricted ?<Next>
  • Suncorp is a large corporate (17,000 people or so..)Our business technology team is about 1,400 permanent peopleWe’re trans-Tasman and work in a distributed environmentFor those uninitiated, the 2007 acquisition of the Promina Group by Suncorp resulted in a significant integration programOut of this, the business technology team was a fusion of nine pre-merger groupsAgile was to be the common binding cultural framework, with explicit encouragement from our Executive Leaders, particularly our Group Executive, to ‘make it happen’For the overwhelming majority of us in business technology, it was a new way of workingOur application of Agile certainly picks up all of the development approaches for which it has become well regardedBut at Suncorp, our application of Agile is broader, covering a value set, principles and practices. Challengingly for a large corporate, the principles include Speed, Flexibility and Simplicity !A transformation of this scale, considered in the context of a large, corporate entity, could have died a painful death.At Suncorp Business Technology, we have been on this Agile journey for two years now, momentum has been maintained and our capabilities continue to mature.So what lessons can I share about introducing Agile to a large corporate, with particular emphasis on the typical corporate pain-points of project governance ?<Next>
  • I have three stories to share with you.I will be starting with Agile and the Beast; all about PMOs and GovernanceI will then talk about Agile’s Adventures in Wonderland – a place where all the people are different !And lastly,The Wizard of Agile; a tale of fear, myths and self-discovery (and maybe even a pair of red shoes) involving Risk, Compliance and Audit<Next>
  • PMO and Governance..I think PMOs and the project governance frameworks they foster are living.Divide and multiply, divide and multiply – These things just grow. Every ‘incident’ translates to a fresh raft of ‘improvements’.I must admit that having, as I do, a PMO background, I love a good framework.But I’ve had to seriously revisit what needs to be in a project governance framework.My new catch-cry is ‘light yet sufficient’ – in everything.Suncorp has a corporate project governance framework that all projects must align to; but it’s not overly prescriptive. It has minimum-mandatories like there should be a compelling business case to make the investment. That there should be some defined, accountable governance roles aligning with our organisational structure (like a steering committee) if we’re to spend the big bucks.But this is not a detailed, checklist type framework. And as a PMO leader in Business Technology, I don’t feel compelled to overlay an additional level of governance across the project space.I understand that Agile projects at Suncorp are different. I appreciate that the business and technology folk are closely entwined and challenging what they’re doing every day. I have confidence that when something looks a bit off, we investigate why, we have a conversation. Mostly though, I marvel at how the self-organising teams employ structure and governance that’s fit for their context – if I was a PM, that’s how I would want it to be !
  • So how does the PMO, historically a control function, respond to such challenge ?I find it’s now less about mandating adherence to standards and more about understanding the decisions that need to be made and providing the best information I have available to make those decisions.The best place for a PMO to start is to ruthlessly challenge what they are doing; ask how does this add value and for whom ?I know there have been some instances where I have been unable to answer these questions, so the process and/or report in question gets turned off, or replaced with something simpler and more streamlined.For the PMO, it’s about finding new ways to add value to their executive stakeholders.
  • But if for no other reason, the PMO needs to change because of this !In all honesty, as a PMO manager how can I compete with the richness of this experience. With an Executive leadership team who can read and interpret such a story wall, a little status indicator doesn’t quite cut it ? By the time I consolidate a report, our Group Exec can visit the project, talk with the project team, remove obstacles and has moved on to assist the next project.So it changes the nature of my role.Add to this the social networking, micro-blogging and collaboration sites and I’m fighting a losing battle. People in Business Technology promote their own good work and learnings themselves; I get pointers from the Exec team about what needs to be in our monthly performance reporting before I even start to piece it together.PMOs need to move beyond process-centricity and embrace a more relationship-led model of gathering management information.
  • Agile truly is another world. It’s got it’s own postcode.So what can impede an organisation as it attempts to transition to Agile like nothing else ? It’s People !I believe corporate hierarchy to be the last bastion of change resistance. So even if you tackle the governance elements to enable a move to Agile, there are some other places where resistance may impede your efforts.The Project ManagerNot just running the schedule and whipping the team into the ground anymore ! The performance of the team becomes particularly important, as is the mood of the team. So some of the ‘softer’ leadership competencies become all the more necessary. Project Managers as people leaders.. How many of your PMs have that ability ?The Team MemberIt’s pretty confronting in an Agile team. Every day, the project stand-up is a very public forum where lack of delivery becomes very noticeable. Not everyone will embrace that – initially.. !The PartnerTypical arrangements where delivery partners have grown fat and tasty on a time and materials basis need to be judiciously assessed. Pick the right partner to help you on your Agile journey, one who shares your values and has the right motivations.The ManagerSenior Leaders cast long shadows and may resist initial efforts to changeAgile projects need rapid, timely decisions to maintain pace. They tend to get on and do just that – they are self-directed.Senior Managers have long played a role of the decision maker in project delivery. Whilst their oversight is required from a governance perspective, most notably via steering committees, they are more valuable as true leaders; removing obstacles for the team, providing guidance and finding more resources. This includes expertise and experience that they can bring to the Project Teams as stakeholders –not as ‘the’ lone decision maker !Watch for other management dysfunctions too, such as proxy project management, corridor interventions, pressure for improved performance (velocity) and pressure for ‘accurate’ estimates.
  • The final area I would like to talk about are the Risk, Compliance and Audit functions.Risk and Compliance functions in the corporate enterprise have the unenviable role of identifying systemic risks and the areas of non-compliance to ever-changing regulations, then intervening to prevent said risk becoming a big issue !How does Agile help to manage risk ?Each story is tied to discrete business value; we know what we’re doing every step of the way.We then assess the relative risks of each story – so it’s very comprehensive.. !Agile projects are structured to tackle the riskiest components first. This approach tests our assumptions up-front and we get much clearer on the success potential of a project.If something is just not going to work and will fail, then we want it to fail fast and fail cheaply. In this context, failure is as good outcome.. !Lastly, iterations also help with the changing business environment including legislative change, we can be flexible and responsive.So if you were a risk manager, would you like Agile ?Our experience suggests that whatever requirements are made by the risk and compliance functions, our Agile projects can respond with appropriate structure and rigour.<Next>
  • What is it about the word ‘audit’ that strikes fear into the heart of a project manger ?I recall the apprehension of wondering, what are our auditors going to make of this ? Sure, Agile works well in those neat web applications that exist outside of the highly regulated Australian Financial Services industry; but surely, here’s where we’re going to come unstuck ?You know what, both Suncorp internal and external audit teams were really keen to understand ‘Agile’ and how it would lend itself for review.So our internal and external audit teams participated together in an Agile training session tailored for them. We then worked with the audit team to translate their checklists into an Agile context.This is still very much an education journey. Executive support was needed was when audit was first engaged. Fully seized of the myths around an Agile approach, we needed to counter the initial response of “doesn’t Agile introduce greater risk ?” and concerns about how to audit a project that has “no documentation”. Whilst bottom up support was there, we needed backing from our leaders to lend credibility. That education journey continues.I would still say that most PMs still fear the audit teams (some would say this is healthy !) but fundamentally, there is no disconnect in terms of adopting Agile in a corporate from an audit perspective.
  • So in the end, to a great degree it comes down to changing your context.Agile projects still have project charters, release plans and test plans. They may not have big, chunky requirements documents and we may not tack on a three month ‘big up-front design’ phase so everyone can prod and poke the thing to see if it will fly, but there is structure, rigour and discipline.You just have to learn a new lingo.Every Agile journey is different. Every organisation will adapt Agile to meet it’s specific needs.But I believe that Agile can be effective, irrespective of the size of the organisation.I come from a hardened PMO background and would not try to replicate what I’ve done before now that I’ve been introduced to Agile. To be totally frank, I find it not only easier to work Agile but so much more stimulating !I appreciate that having an Executive Sponsored Agile Change Program is somewhat of a unique situation. But I am adamant that what I have stepped through today is possible in any context;You can question your governance frameworksYou can challenge your PMO to simplify and streamline – ask where the value is !You can look to change the way you work and what you expect from your team membersYou can work with your audit teams to help them understand Agile (and support you to adopt it !)These are not always easy areas to tackle, in fact far from it, these elements need to be challenged if you are contemplating the Agile way of working. But they are well worth pursuing !Thank you.HandoverI will now hand over to Lindy who is going to share some suggestions about how to start an Agile transformation in your own organisation.
  • CHANGEToday we’re talking about how to adapt your companies governance in order to support agile project delivery. But perhaps you’re not lucky enough to work for an organisation that is interested in making these changes. Can you still deliver individual projects in an agile way? Of course you can, TW works with clients doing this all the time. But you will lose some of the benefits.For example, it can be time consuming for a project manager to convert the sort of data that comes out of an agile project into a proscribed template. And hard to align agile project deliverables with traditional funding milestones. The main impacts of this are usually a time overhead for the PM, the need for extra communication and potential loss of productivity of the team waiting for ‘gates’.But what can you do in your organisation to start to bring about some of the changes we are talking about today? How can you influence the gate keepers?One of the best ways to highlight the fallacies associated with traditional governance is to encourage better post project reviews in your organisation. Focus on true return on investment. Were the benefits in the business case actually realised? Highlight what proportion of the software delivered by your projects is actually useful to the people who use it. Without these measures, it is easy to call a project a ‘success’ based on the fact that it has simply gone live. Value becomes irrelevant. Encourage others around you to see value to the business of the project success criteria, not whether or not the project came in under some budget number handed down from above.By getting people around you to think about what actually constitutes a successful project, you might also start to get them thinking about the best ways to measure that success during the project.
  • Agile Governance – Managing the Enterprise Issues

    1. 1. Quarterly Technology Briefing<br />Agile Governance - Managing The Enterprise Issues<br />
    2. 2. AN/FSQ-7<br />
    3. 3.
    4. 4. To my mind, the question that’s much more important than how to control a software project is, why on earth are we doing so many projects that deliver such marginal value? <br /> - Tom DeMarco, July 2009<br />
    5. 5.
    6. 6. Lonely Planet’s route to agile<br />
    7. 7. The illusion of control in IT projects<br />I’d strongly suggest that we pick a date to catalyse our efforts and hold ourselves accountable to delivering <br />full scope on budget. It will be a rallying call for <br />all of us and <br />something to shift our culture to holding deliverables to deadlines. <br />
    8. 8. The new look of governance<br />
    9. 9. Governance at the coal face<br />
    10. 10. Agile’s ultra-transparency invites critique<br />17 days<br />4 days<br />Initiation<br />Detailed Product Design<br />Prep<br />Develop and Test (usually 2 weeks)<br />Release<br />Idea<br />Concept to Cash<br />Average Cycle Time from Initiation to Release: 78 days!<br />25% of this time is spent actually creating value. The rest is waste.<br />Often there is an additional 2 weeks after deployment before advertisements generate cash.<br />
    11. 11. Essential ‘coal-face’ governance equipment<br />
    12. 12. CFO’s & Lawyers: where’s the guarantees?<br />
    13. 13. ITIL governance plugs right in<br />
    14. 14. Communication &gt; governance<br />
    15. 15. Outsourcing agile? Can it work?<br />
    16. 16. How does Agile survive and thrive in a corporate ? <br />
    17. 17.
    18. 18. Three stories about Agile..<br />1<br />2<br />3<br />Agile and the Beast<br />Agile’s Adventures in Wonderland<br />The Wizard of Agile<br />
    19. 19. Agile and The Beast<br />PMOs and the ‘G’ word..<br />How much governance does a self-directed,self-organising project team need ?..<br />
    20. 20. Agile and The Beast<br />Don’t you know who I am.. ?!<br />The role of the PMO has to change..<br />
    21. 21. Agile and The Beast<br />Don’t you know who I am.. ?!<br />How can a PMO compete with this ?<br />
    22. 22. Agile’s Adventures in Wonderland<br />What about the people.. ?<br />It’s a truly wonderful place,<br />but requires people to be, well, different..<br />
    23. 23. The Wizard of Agile<br />What about the risk.. ?<br />Actually, Agile is all about risk mitigation..<br />
    24. 24. The Wizard of Agile<br />“Audit says no”<br />The intent of an audit is to assess if<br />you did what you said you would do..<br />
    25. 25. In the end..<br />You Say..<br />I Say..<br />In Agile, it’s just different, that’s all..<br />Bridging the gap is easier than you might think !<br />