Agile SE -> Agile SD How many have heard of Agile Software Development? What does the word Agile mean? Anyone? Latin word agilis, which means “easily moved, light, nimble, active”. 3. Agile Software development – defined as easily moved, light, nimble, and active software processes. (Agile processes are not unique to software development. They appeared in mainstream business literature in 1991 in the form of Agile manufacturing.) 4. Fitting the process… Not project to the process. Battlefield commanders plan extensively when going to war -> but realize that their plans are just the beginning. Creating and responding to change are very important. Success is defined by accomplishing the mission (defeating the enemy) not conforming to the plan. Can you image a commander saying, “We lost the battle but by golly we were successful because we followed our plan to the letter.” Or “If we just plan this battle long and hard enough, and put repeatable processes in place, we can eliminate change early and not have to deal with it later.” Pretty unreasonable.
Chaordic Perspective – due to the unpredictability of the environment. Goals are achievable, but project details are often unpredictable, the goal of repeatable processes is unattainable. Collaborative Values - capitalization on each individual's and each team's unique strengths. The process is adapted to the people, who indeed are part of the project. Process -> adapts -> project. Project understanding comes more from face-to-face interaction than from documentation. Agilists do not believe that a reliance on heavy processes makes up for lack of skill, talent, and knowledge. Streamlined Methodology – a balance of flexibility and structure in the process.
Idealisms Over instead of versus is because they are all good ideals Rework vs. Reuse – everything should be production quality, but reworkable for flexibility. Don’t be afraid of partial solutions. Military Adoption of the 80% solution. Responding to Change – in line with Lucy Suchman’s Plans and Situated Actions, with the trukese and european navigators.
How did Agile development come about? Welcome to the Jungle. Diagram -> Started in the 80s and evolved from a combination of practices such as Crystal Methods, XP, ASD, Scrum, and PP. Crystal Methods – involves the use of customizable development methods that are color coded based on the heaviness of a project with regard to size and complexity. XP – Extreme Programming – is a collection of well-known software practices that are applied on an individual bases. (short iterations, small releases, rapid feedback. Adaptive Software Development – hallmarked by adaptive processes including incremental and iterative development with constant prototyping. Scrum – management of the software process in a volatile environment. It has an Empirical approach that leaves room for developers to choose the techniques and methods for implementation. Requires frequent management oversight. PP – Pragmatic programming – the use of a set of 70 programming best practices for incremental and iterative development, rigorous testing, and user-centered design. Test-Driven methods of development are building blocks of agile processes. 4. As shown by the diagram, the evolution of Agile Software Development is complex. And as we’ll talk about later, because of its roots, it has some issues. 5. In any case, what’s the motive behind the maddness? Developers sick of late deliveries and lost opportunities. Sometimes making the customer follow your software development processes is like banging your head into a brick wall. Is essence it is all about flexibility and leanness. -> for the Development side and the Customer side.
Extreme Projects – Not extreme programming projects but rather projects that use leading or bleeding edge tech, Involve erratic requirements changes, and deliver quickly. Multi-Sized Corps – Nokia story. Government Contracts – Earned Value Management Even Intel has used Agile Methodology for Embedded Development.
Embedded Software for buses and coaches. Time Constraints – hours or days, not weeks or months 50% rejection rate - Documentation flaws and incomprehensible function code 8% rejection rate - Functional unit fails test cases and integration errors. Current SE practices Inadequate.
Solution: Classical - systematic documentation and measurement. Agile – Test First Process Test First Practices gave way to more process improvements such as: quicker simulation setups faster module tests Quality assurance effort spread out Test case scope – too detailed, not enough pruning of inputs. Concerns over business risk of a whole adoption.
Again, agile practices treat change the same anywhere in the development process.
Collaborative schema – Developers made changes to database to make application work, then changes were approved by the team and modified/abstracted to fit other developers needs. Rework vs. Reuse Leaner database schema – obsolete areas of schema were easily phased out. No lingering question “What are these fields for?”
Moral of the story is that, just like anything, Agile processes are not just a hack job that require no preparation.
Why not more adoption (in SE? in corporate? In universities?) Inconsistent and diverse defs – need more quality methodology not just quantity. Lack of Theoretical Grounding – current concept of agility based on experience, not on underlying concepts like flexibility and leanness. Agile methods like SCRUM and XP are derived from subjective practical experience and not reliable systematic research. But there is hope… Conboy et. Al. and their framework of agility for software development (based on underlying concepts); Role changes – management role less prominent and active, more like a coach Situational customization – no out of the box operation. Takes time to customize a process and get good at it. Difficult to quantify – Agile practices are more philosophy based not activity based. Risks – Making the customer understand the tradeoffs for following them down a rabbit hole can be difficult. How do you budget a project if you can’t get the all/most requirements upfront?
Good for dynamic.. - Simple design and refactoring principles Boehm and Turner propose these risk factors based on observation, not from experience.
Boehm and Turner Risk assessment model (5 factors)
May be somewhat conflicting with Boehm and Turner. Agile processes at worst can be plan driven processes if your project demands such a process. If you can’t revert back to plan driven processes you are not using Agile processes. Again Agile Software Processes are simply about flexibility and leanness. It is not about a lack of structure, it is about adaptability in the presence and kind of structure for a process. Developers have been pretty Agile for a while, it is just that now they have more influence over the extent of their Agile practices. House metaphor Architect Builder All use agile processes.
As a kid learning to draw, I would often indulgently render small parts of a drawing in rich detail, and pay less attention to how these parts related to each other. Lacking an overall spatial plan, I often ended up with gross inaccuracies that ruined the drawing. My instructional books recommended starting with a light sketch to plan the composition, and then to gradually commit to areas of light and shade over the entire surface to evenly develop the composition. Maintaining such a balanced overview of the drawing would reveal planning flaws sooner and allow their correction. Initially I resisted drawing these lines that would later need to be erased, but the advice eventually worked; results became more predictable and the eraser became my friend.
Agile Software Engineering
OverviewAgile Methodology– Concepts and Ideology– Origins– Applicable DomainsCase StudiesProcess ExampleAdoption DetractorsAgile vs. Plan Driven ProcessesFuture Headings
What is Agile Software Development?Easily moved, light, nimble, activesoftware processesFitting the process to the projectAvoidance of things that waste time References: , 
Agile Software Development Ecosystem“Chaordic” PerspectiveCollaborative Values and PrinciplesStreamlined Methodology Reference: 
Agile Software Development Ideals Individuals and interactions over process and tools Working Software over comprehensive documentation – Rework vs. Reuse Customer collaboration over contract negotiation – Solutions vs. Products Responding to change over following a plan References: , 
Substantive Definition“the continual readiness of an entity to rapidly or inherently, proactively or reactively, embrace change, through high quality, simplistic, economical components and relationships with its environment”. Reference: 
Origins and Foundations Scrum Crystal Fam. XP Adaptive SD Agile PP Agile Modelling References:  (image), , 
Applicable DomainsMulti-Sized Corporations– Multi-team environments using overlapping cross-team communitiesGovernment ContractsMost effective for extreme projectsProjects that do not work well in rigorousplan-driven processes References: , , , , 
Case StudiesDaimler-Chrysler Embedded SoftwareCaribouLake.com DatabaseNuclear Control Systems Manufacturer References: , , 
Daimler-Chrysler Case StudyProblem: Customer specific add-ons a hassleReasons1. Rising workload2. Frequent late changes3. Difficult time and quality constraints4. Late delivery Solutions Image From: 1. Implement highest initial value item first2. Identify discrepancies of Work Products to Specifications as automated as possible3. Identify discrepancies of Work Products to Specifications as soon as possible4. Fix discrepancies of Work Products to Specifications as soon as possible
Daimler-Chrysler Case Study ResultsOverall Solution: Combination of classical and agilepractices– Test First practices– Unit TestingChallenges– No external support– Test cases were difficult to scope– Developer transitions implement->document->test test->implement->documentEffects– Substantial increase in flexibility, quality and timeliness– Greater acceptance of agile practices
CaribouLake.com Case StudyProblem: Database design during shortdevelopment iterationsReasons– Traditional up-front database development impractical Late requirements changes Database changes more costly than application changes
CaribouLake.com Case Study ResultsSolution: Collaborative schema evolution– Formalized Refactoring– Test Suites– Database coding standardsEffects– Distributed knowledge of database design– Faster development– Leaner database schema
Nuclear Control Systems Manufacturer Case StudyProblem: Difficulty replacing old control software– Heavyweight waterfall process too inflexibleSolution: Independent research and wholesale adoptionof agile processes– No upfront training of team– Absence of common working area, workstation configurations and an integration and build environmentResults– Five development iterations produced < 20% projected business value– Agile process approach “canned”– Team was demoralized
Adoption DetractorsInconsistent and diverse definitionsLack of theoretical groundingDifferent way of thinking– Role changes– Situational customizationSolid people skills requiredShort iterations inhibit long-term perspectiveRisks– Harder to manage feature creep and customer expectations– Difficult to quantify cost, time, quality. References: , , , , , 
Agile vs. Plan Driven Processes1. Small products and 1. Large products and teams; scalability limited teams; hard to scale2. Untested on safety- down critical products 2. Handles highly critical3. Good for dynamic, but products; hard to scale expensive for stable down environments. 3. Good for stable, but4. Require experienced expensive for dynamic Agile personnel environments throughout 4. Require experienced5. Personnel thrive on personnel only at start if freedom and chaos stable environment 5. Personnel thrive on structure and order Reference: 
Use of Agile and Plan Driven ProcessesEach haveappropriate roles insoftware developmentMost use Agile-PlanDriven HybridWhen should each beused? References:  (image), , 
My Agile SynopsisNo such thing as Agile hybrid.Agility is about flexibility and leanness.Agility != Lack of StructureChange of process control from Top-Downto Bottom-Up
An Eye on the FutureVolatility and uncertainty in projectenvironments– Increased demand for more adaptive processesMore theoretical research in AgileMethodsCombined methods emerging (Agile RUP)Expanded fields of use (anywhere designis present) References: , , , , , 
References1) Abrahamsson, P. et al. New Directions on Agile Methods: A Comparative Analysis. In Proceedings of the 25th International Conference on Software Engineering . IEEE 244-256, Portland, Oregon, May 2003. May be found at: http://csdl.computer.org/comp/proceedings/icse/2003/1877/00/18770244abs.htm2) Alleman, G. B. and Henderson, M. Making Agile Development Work in a Government Contracting Environment. In Proceedings of the Agile Development Conference (ADC’03). IEEE 114-120, Salt Lake City, Utah, June 2003. May be found at: http://csdl.computer.org/comp/proceedings/adc/2003/2013/00/20130114abs.htm3) Armitage, J. Are Agile Methods Good for Design? Interactions. ACM 14-23. 11,1 January 2004. May be found at: http://portal.acm.org/citation.cfm?id=962342.9623524) Beck, K. et al. Manifesto for Agile Software Development. Last Access: 02-7-2005. May be found at: http://www.agilemanifesto.org/5) Boehm, B. and Turner, R. Using Risk to Balance Agile and Plan-Driven Methods. IEEE Computer. IEEE 57-66, 36,6, June 2003. May be found at: http://csdl.computer.org/comp/mags/co/2003/06/r6057abs.htm6) Conboy, K. and Fitzgerald, B. Toward a Conceptual Framework of Agile Methods: A Study of Agility in Different Disciplines. In Proceedings of the 2004 ACM Workshop on Interdisciplinary Software Engineering Research. ACM 37-44, Newport Beach, CA. November 2004. May be found at: http://portal.acm.org/citation.cfm?id=1029997.10300057) Derbier, G. Agile Development in the Old Economy. In Proceedings of the Agile Development Conference (ADC’03). IEEE 125-132, Salt Lake City, tah, June 2003. May be found at: http://csdl.computer.org/comp/proceedings/adc/2003/2013/00/20130125abs.htm8) Green, B. Agile Methods Applied to Embedded Firmware Development. In Proceedings of the Agile Development Conference (ADC’04). IEEE 71-77, Salt Lake City, Utah, June 2004. May be found at: http://csdl.computer.org/comp/proceedings/adc/2004/2248/00/22480071abs.htm9) Harriman, A., Hodgetts, P. and Leo, M. Emergent Database Design: Liberating Database Development with Agile Practices. In Proceedings of the Agile Development Conference (ADC’04). IEEE 100-105, Salt Lake City, Utah, June 2004. May be found at: http://csdl.computer.org/comp/proceedings/adc/2004/2248/00/22480100abs.htm
References (Cont.)10) Highsmith, J. What is Agile Software Development? CrossTalk: The Journal of Defense Software Engineering. Oct. 2002. May be found at: http://www.stsc.hill.af.mil/crosstalk/2002/10/highsmith.html11) Hirsch, M. Making RUP Agile. In Practitioners Reports of the Conference on Object Oriented Programming Systems Languages and Applications (OOPSLA 2002) . ACM 1-28. Seattle, Washington, November 2002. May be found at: http://portal.acm.org/citation.cfm? id=604251.60425412) Hodgetts, P. Refactoring the Development Process: Experiences with the Incremental Adoption of Agile Practices. In Proceedings of the Agile Development Conference (ADC’04). IEEE 106-113, Salt Lake City, Utah, June 2004. May be found at: http://csdl.computer.org/comp/proceedings/adc/2004/2248/00/22480106abs.htm13) Huo, M. et. al. Software Quality and Agile Methods. In Proceedings of the 28th Annual International Computer Software and Applications Conference (COMPSAC04). IEEE 520- 525, Hong Kong. September 2004. May be found at: http://csdl.computer.org/comp/proceedings/compsac/2004/2209/01/220910520abs.htm14) Kähkönen, T. Agile Methods for Large Organizations – Building Communities of Practice. In Proceedings of the Agile Development Conference (ADC’04). IEEE 2-11, Salt Lake City, Utah, June 2004. May be found at: http://csdl.computer.org/comp/proceedings/adc/2004/2248/00/22480002abs.htm15) Manhart, P. and Schneider, K. Breaking the Ice for Agile Development of Embedded Software: An Industry Experience Report. In Proceedings of the 26 th International Conference on Software Engineering (ICSE’04). IEEE 378-386, Edinburgh, Scotland, UK. May 2004. May be found at: http://csdl.computer.org/comp/proceedings/icse/2004/2163/00/21630378abs.htm16) Schneider, J. and Johnston, L. eXtreme Programming at Universities – An Educational Perspective. In Proceedings of the 25th International Conference on Software Engineering . IEEE 594-599, Portland, Oregon, May 2003. May be found at: http://csdl.computer.org/comp/proceedings/icse/2003/1877/00/18770594abs.htm