Agenda What’s the problem? What’s the metaphor? – Building Construction versus Jazz Improvisation Agile Methods Examples Agile Manifesto XP as an example of agile method Four Values Basic Principles Planning games, pair programming and metaphor. Critique
The Problem Failure of software project is endemic. Schedules slip, Projects are cancelled, Business needs never met, Changes made faster than system can be developed, Testing is inadequate, Quality not managed, Staff turnover, Systems become too large to manage.
Computer systems cannot cope with change. Difficult to change, Cannot adapt to organisational change, Require certainty. But organisations’ and businesses’ problems change New customers, Transactions changes, New products, Demands of Internet-based systems
How do we deal with change? Large scale project management approaches try to: Reduce change Increase certainty Eliminate risk by planning, documentation and contracts. We do what it says on the contract We freeze requirements and we formalise such freezing though stage meetings. Changes become rewrites. Our metaphor is the construction industry and our project management practices are drawn from the industry. What if the metaphor is wrong?
The Construction Analogy Assumption : sequential development process, predictable and stable environment, goals of achieving efficiency and reducing uncertainty Structure IS stable artifacts? Requirements ‘fixed in concrete’ IS as social artifacts Process Linear? Formal join points? SSADM-like? Roles Cultural ghettos, divergent, conflict and formalisation. Blinkered view.
Film Production Script based on literary sources Revision during production Interpretation by different creative artists Pre-production, Filming, Post-Production
Structure More pliable product Builds on body of work But little active role of users. Process Dynamic development Greater scene of teamwork, changing roles, continuous involvement of many creative staff. Roles Variety of roles. Wide-skill set. Some role changing.
Jazz Improvisation Use of minimalist musical structures including harmonies, melodies and rhythm. Small team elaborating on simple structures in complex ways Musicians operate with a set of social norms, with changing roles and intense interaction. Minimalist Structures Constrain the turbulence of the jazz process by specifying particular ways of inventing and co-ordinating musical ideas’
Refashioning performance in response to colleagues and audience Risk-taking and continuous learning Experimenting with new instruments, styles and textures of sound Alternate soloing and comping. For leadership and personal development Knowledge of music technology and instrumentation Trust within wide zones of manoeuvre and constructive controversy Wide stock of talent and performative competence Partnering in autonomous ensemble Templates of songs, choruses etc on which to improvise Communicative Codes Keys, Chord progressions, repertoire Behavioural Norms Technical Structure Social Structure
Structure Dynamic systems, subject to change in response to organisational change Minimal componentised structures Patterns Process Variation with socially determined process structured Iterative development and continuous delivery Theme development Role Role rotation. Importance of mentors Continuous communication and listening.
Focus on creativity and development of new ideas within a minimal design framework Focus on technology and rigorous adherence to pre-defined design. Technology adaptation occurs as new tools and methods are explored. Technology fixed at start of project User input critical throughout project. User input limited temporally and functionally Changing structure and new structure is the norm Changes difficult to achieve. Roles rotate and project involvement is continuous throughout the project's lifetime Roles fixed in skill base and temporal involvement with project Product structure evolves in response to new ideas and environmental change Product structure defined and fixed at start Iterative dynamic progression Sequential progression Film Production/ Jazz Improvisation Analogy Construction Analogy
Agile Software Development Approaches Ancestry in Rapid Application Development Held up as antithesis of traditional software development . Focus on: Early delivery priority business requirements. Dealing with partial knowledge Reduced documentation Small groups of programmers Iteration Continuous testing Integral customer involvement
The Agile Alliance Snowbird 2001 : Meeting of representatives of agile methods Purpose to get all the leaders of lightweight methods into one room ‘ Define a developer community freed from the baggage of Dilbertesque corporations.’ [Respond to] ‘the failure of the standard "fixed" process mindset that so frequently plagues our industry.’ ‘ The Agile movement is not anti-methodology, in fact, many of us want to restore credibility to the word methodology. We want to restore a balance. We embrace modeling, but not in order to file some diagram in a dusty corporate repository. We embrace documentation, but not hundreds of pages of never-maintained and rarely-used tomes. We plan, but recognize the limits of planning in a turbulent environment .’
Agile Methods Adaptive Software Development Feature Driven development Crystal Clear Method Dynamic Systems Development Method Rapid Application Development (James Martin) Scrum Pragmatic Programming Extreme Programming
Agile Alliance Values Individuals and interactions over processes and tools Working software over comprehensive documentation Customer collaboration over contract negotiation Responding to change over following a plan
Principle 1 Our highest priority is to satisfy the customer through early and continuous delivery of valuable software. Software as the business output. Continuous delivery provide regular feedback. Enables customer evaluation
Principle 2 Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage. Promoting software ‘evolvability’ But still avoiding radical changes at the end of the development project.
Principle 3 Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale. Rapid feedback. Daily build Requirements can be tested and altered quickly. Key importance of iterative and incremental development From fixed pricing to adaptive pricing? Defining releases to match business deadlines and user ability to absorb changes.
Principle 4 Business people and developers must work together daily throughout the project. Role of developers changing towards business orientation. Development managers with both business and technical understanding and still writing code
Principle 5 Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done. Agile approaches put emphasis on people factors – sociability, talent, skill communication. ‘ Personnel attributes and human relation activities provide by far the largest source of opportunity for improving software productivity ’ Barry Boehm XP very demanding of people skills. We’re not all Kent Becks or Ward Cunninghams! Need for extraordinary skills, strong tacit knowledge and discipline
Principle 6 The most efficient and effective method of conveying information to and within a development team is face-to-face conversation. Stand-up meetings. Informal communication Tacit knowledge transfer Role of written communication. Problem of unrecognised shortfalls of tacit knowledge
Principle 7 Working software is the primary measure of progress. Early delivery Measurement. .. Not lines of code Measurements of what? Functions? Passed tests?
Principle 8 Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely. Workaholism. Ineffectiveness of work-all-hours mentality. Creativity needs recreation Sustained overtime is a bad sign
Principle 9 Continuous attention to technical excellence and good design enhances agility. Refactoring and personal quality requirements Expertise and attitude of the programmer
Principle 10 Simplicity--the art of maximizing the amount of work not done--is essential. Avoiding bells and whistle. Simple solutions are easy to understand Who decides what is simple?
Principle 11 The best architectures, requirements, and designs emerge from self-organizing teams. Self-governing teams Emergent behaviour The ‘jelled’ team Complexity Theory and Post-modernism
Principle 12 At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly. Value of organisational learning. Time cost of review
eXtreme Programming Now let’s see the Agile Manifesto Principles in Action. Comes from a group of American programmers : – Kent Beck Ward Cunningham Ron Jefferies. Assumes an object-oriented approach. Closely connected with the patterns movement.
Extreme Programming Little documentation. Source code is only documentation No software specification No separate design and testing phase Design for change prohibited No formal reviews or inspections
Extreme Programming A view of what development is. Four values. Some basic principles. Twelve practices. An approach to development .. The development episode.
What’s important in software development management? Cost Time Quality Scope What’s important in the process? Coding! Testing Listening Designing.
Extreme Programming: Four Values Communication Making communication essential. Face-to-face. Environment. Pair programming. Simplicity What is the simplest thing that can possibly work? Removing complexity (by refactoring) Challenging complexity (by continuous integration, pair programming) Feedback Coaching. Testing. Starting with test case. Daily integration. Quick releases. Courage. Make a decision. Team support. Ready to start again.
Extreme Programming. Fundamental principles. Rapid Feedback . Learn immediately. Stimulus/response. Days not weeks/ months. Assume simplicity . Sort today’s problems today. No belts and braces. Using tests. No design for reuse. Incremental Change . Series of small changes. Evolve solutions. Plan, design, team, change a little at a time. XP adopted a little at a time. Embrace Change . No frozen specs. Quality work . How. Constant review (pair programming) refactoring. Removing redundancy. Pride in craftsmanship. Art.. Others (see for yourself) Teaching learning, Small initial investment, play to win (not (not to loose)), Concrete experiments (leave no abstract decisions untested), open, honest communication, Work with people instincts, accepted responsibility, local adaptation, travel light, honest measurement.
Extreme Programming : Basic Practices The Planning Game - Developers and customers scope and plan using story cards Small releases - Release as small as sensibly possible. Metaphor – Identify overarching metaphor e.g. contracts and contract management, tracking robot. Simple design Design as simple as possible at any given moment. Only leave whatever is really needed. Eliminate complexity Testing – Write the test first. These must run flawlessly for development to continue. Refactoring – restructuring system without changing its behaviour. Pair Programming – All production code is written with two programmers at one machine.
Extreme Programming : Basic Practices Collective Ownership – Anyone can change code anywhere in the system at anytime. Opposite of Software Configuration Management. Works if there’s continuous integration, simple writing, complete visibility. Continuous integration . Integrate and build system many times a day, every time a task is complete. 40-hour week . Never two weeks in a row overtime. On-site customer – real live customer (not line manager) sitting with team all the time. More value out of system with more business contribution. Coding standards – programmers write all the code in accordance with rule emphasizing communication though the code.
Extreme Programming : Roles Programmer – adds actual value, at heart of development Customer – importance of customer engagement Tester – helps customer write functional tests Coach – responsible for process as a whole – XP expert. Provides toys and food. Tracker – conscience of XP team. ‘Is the feedback loop being closed? Consultant – ‘wizard’ supplies deep technical knowledge. Big Boss – Champion of development team.
The Planning Game Business writes stories. Estimate time for each story. Break down stories if necessarily. Clarify with customer. Sort stories and decide what’s in the first iteration. Sort on value and risk. Turn stories into tasks.
Programmer Accepts tasks and estimates. Finds a partner. Writes test cases (which will all fail at the start because there’s nothing there). Gets all tests working. Get something, probably with many small classes, up and running as fast as possible. ‘Go like the clappers’ Minimalist design.
Critique Origins – OO, Patterns movement, key group of ‘gurus’ Refactoring – programmer psychology. Programmer versus manager. Business / IT gap approach to crossing it? Scalability Debate. Size does matter. Reaction to outsourcing and ‘big thump’ delivery of outsourcing. Philosophy: Chaos and Complexity, Emergence, Post-modernist, Eastern Philosophy. Culture . ‘Biggest barrier to XP is culture’ Beck. Precedence of code as the lingua franca. A programming high priesthood?
Bibliography and Sources Beck, K. (1999) Embracing change with extreme programming. IEEE Computer, October 1999, 70-77. Beck, K. (2000) Extreme Programming Explained. Addison Wesley Agile Methods List (CERN) Agile Software Development Ecosystems Build better software Agile Alliance Questioning Extreme Programming Agile Software Development in Theory and Practice (MSc Thesis, Finland) Martin Fowler articles on XP and Agile Methods Boehm, B and Basili, V. Gaining Intellectual Control Over Software Development
Bibliography and Sources Jim Highsmith Links and resources Xtreme programming.com (Ron Jefferies) Extreme Programming Roadmap (Ward Cunningham) Scrum Crystal Clear Extreme Programming Case Study - Teaching Extreme Programming in a university DaimlerChrysler C3 Case Study On the Productivity of Agile Software Practices: An Industrial Case study Hi-Tech Workplace no better than factories (BBC News) Avison, D and Wilson, D. (2001) A viewpoint on software engineering and information systems: what we can learn from the construction industry. Information and Software Technology 43, 795 - 799. Kamoche,K and Pina e Cunha, M. (2001) Minimal Structures: From Jazz Improvisation to Product Innovation. Organisational Studies 22(5) 733 - 764.