Formal Versus Agile: Survival of the Fittest? (Paul Boca)

3,662 views

Published on

The potential for combining agile and formal methods holds promise. Although it might not always be an easy partnership, it will succeed if it can foster a fruitful interchange of expertise between the two communities. In this talk I explain how formal methods can complement agile practices and vice versa. There are no pre-requisites for this talk, except an open mind and a desire to make software development more reliable. Leave any pre-conceptions at home, and be prepared for myths to be dispelled.

Published in: Technology

Formal Versus Agile: Survival of the Fittest? (Paul Boca)

  1. 1. Agile versus Formal Methods: Survival of the Fittest? Paul Boca
  2. 2. Collaborators <ul><li>Sue Black </li></ul><ul><li>Mike Hinchey </li></ul><ul><li>Jonathan Bowen </li></ul><ul><li>Jason Gorman </li></ul>(Scrum mistress) (Formal Methodist) ( Z z z z z z z z z z z )
  3. 3. About Me <ul><li>PhD in program transformation (categorical stuff) </li></ul><ul><li>Hardware Compilation R&D (industry light) </li></ul><ul><li>High-level synthesis (start up) </li></ul><ul><li>Was R&D manager at UK static analysis vendor </li></ul><ul><li>Currently Quality Engineering Manager (SME) </li></ul><ul><li>Formal Methods enthusiast </li></ul><ul><li>Organizer of formal methods seminars </li></ul>
  4. 4. Outline <ul><li>Agile background </li></ul><ul><li>Formal Methods background </li></ul><ul><li>Co-existence of Formal Methods and Agile </li></ul><ul><li>The Cost of Agile </li></ul><ul><li>Summary </li></ul><ul><li>But first... </li></ul>
  5. 5. Quiz #1 <ul><li>First Prize: Toyota Prius </li></ul><ul><li>Second Prize: Sony PlayStation 3 </li></ul><ul><li>Runner up: Toyota Prius and PlayStation 3  </li></ul>
  6. 6. Quiz #1 <ul><li>63,000 ? </li></ul><ul><li>2,000+ ? </li></ul><ul><li>700 ? </li></ul><ul><li>280210 ? </li></ul><ul><ul><li>The number of bugs in Windows 2000 </li></ul></ul><ul><ul><li>The number of pages of Z specifications in a certain air-traffic control system </li></ul></ul><ul><ul><li>Number of lines of code written per month by an agile developer </li></ul></ul><ul><ul><li>Sony PlayStation 3 bug revealed! </li></ul></ul><ul><ul><li>What do each of the numbers below correspond to </li></ul></ul>
  7. 7. Agile Manifesto <ul><li>Individuals and interactions over processes and tools </li></ul><ul><li>Working software over comprehensive documentation </li></ul><ul><li>Customer collaboration over contract negotiation </li></ul><ul><li>Responding to change over following a plan </li></ul><ul><li>http://agilemanifesto.org/ </li></ul>
  8. 8. Approaches + Techniques <ul><li>Approaches </li></ul><ul><ul><li>Extreme Programming </li></ul></ul><ul><ul><li>Scrum </li></ul></ul><ul><ul><li>DSDM </li></ul></ul><ul><li>Techniques </li></ul><ul><ul><li>Pairwise Programming </li></ul></ul><ul><ul><li>Test Driven Development </li></ul></ul>
  9. 9. Quiz #2 <ul><li>Agile is about producing software quickly </li></ul><ul><li>Agile is about being responsive to change </li></ul>
  10. 10. Some Agile Success Stories
  11. 11. But Agile is not perfect <ul><li>Lacks comprehensive documentation </li></ul><ul><li>Writing tests upfront is great, but it’s not possible to test everything </li></ul><ul><li>Rapidly changing requirements can be hard to trace. </li></ul><ul><li>Refactoring code can introduce defects </li></ul><ul><li>We’ll return to these later... </li></ul>
  12. 12. Formal Methods <ul><li>The application of a fairly broad variety of theoretical computer science fundamentals, in particular logic calculi, formal languages, automata theory, and program semantics, but also type systems and algebraic data types to problems in software and hardware specification and verification </li></ul><ul><li>http://en.wikipedia.org/wiki/Formal_methods </li></ul>
  13. 13. Formal Methods Timeline Z 1977 B 1986 CSP 1978 Circus 2000 DbC 1986 1970 1980 1990 2000 2010 X machines 1974 VDM 70s Alloy 1997 FM + Agile 2003 not complete but sound 
  14. 14. Formal Methods Success Stories
  15. 16. Formal Methods Success Stories
  16. 17. Quiz #3 <ul><li>Agile will phase out Formal Methods </li></ul><ul><li>Formal Methods will phase out Agile </li></ul><ul><li>Formal Methods and Agile can coexist </li></ul><ul><li>What’s more, Agile and Formal Methods need one another </li></ul>
  17. 18. Agile is not perfect <ul><li>Lacks comprehensive documentation </li></ul><ul><li>Writing tests upfront is great, but it’s not possible to test everything </li></ul><ul><li>Rapidly changing requirements can be hard to trace. </li></ul><ul><li>Refactoring code can introduce defects </li></ul>It's like déjà vu all over again Yogi Berra
  18. 19. Agile is not perfect <ul><li>Lacks comprehensive documentation </li></ul><ul><li>Writing tests upfront is great, but it’s not possible to test everything </li></ul><ul><li>Rapidly changing requirements can be hard to trace. </li></ul><ul><li>Refactoring code can introduce defects </li></ul>
  19. 20. Why we need documentation <ul><li>Code bases need to be maintained and extended </li></ul><ul><li>Some systems will be in operation for decades </li></ul><ul><li>Knowledge transfer to prepare for personnel leaving </li></ul><ul><ul><li>Pairwise programming = Knowledge in silos </li></ul></ul>
  20. 21. Design by Contract to the rescue <ul><li>Preconditions and postconditions provide information on the intent of a function </li></ul><ul><li>Generating loop invariants can help with retrospective documentation </li></ul><ul><li>Contracts can be checked (semi-)automatically </li></ul>
  21. 22. Agile is not perfect <ul><li>Lacks comprehensive documentation </li></ul><ul><li>Writing tests upfront is great, but it’s not possible to test everything </li></ul><ul><li>Rapidly changing requirements can be hard to trace. </li></ul><ul><li>Refactoring code can introduce defects </li></ul>
  22. 23. To test or not to test? <ul><li>Writing tests is definitely a good thing </li></ul><ul><li>When is enough enough? </li></ul><ul><li>Can only test the finite, and you can’t test everything </li></ul><ul><li>Testing sequential programs is hard enough, so what happens when parallelism is introduced? </li></ul><ul><li>What about safety critical applications? </li></ul>
  23. 24. Formal Methods to the Resue <ul><li>Automated generation of test cases (e.g. Perl, Python, Functional Programming languages) </li></ul><ul><li>Use static analysers (e.g. Coverity) </li></ul><ul><ul><li>Finds issues in code, but can’t find everything </li></ul></ul><ul><ul><li>Complements dynamic testing </li></ul></ul><ul><li>Use model checkers (e.g. FDR) </li></ul><ul><li>Mutation testing </li></ul><ul><ul><li>Modify the code base introducing “mutants” </li></ul></ul><ul><ul><li>See whether the test suite “kills” the mutant </li></ul></ul><ul><ul><li>Helps to identify gaps in test suites </li></ul></ul>
  24. 25. Agile is not perfect <ul><li>Lacks comprehensive documentation </li></ul><ul><li>Writing tests upfront is great, but it’s not possible to test everything </li></ul><ul><li>Rapidly changing requirements can be hard to trace </li></ul><ul><li>Refactoring code can introduce defects </li></ul>
  25. 26. Agile is not perfect <ul><li>Lacks comprehensive documentation </li></ul><ul><li>Writing tests upfront is great, but it’s not possible to test everything </li></ul><ul><li>Rapidly changing requirements can be hard to trace </li></ul><ul><li>Refactoring code can introduce defects </li></ul>
  26. 27. Refactoring Code <ul><li>Refactor to </li></ul><ul><ul><li>Remove code smells </li></ul></ul><ul><ul><li>Beautify code </li></ul></ul><ul><li>IDEs can carry out certain refactoring steps automatically. </li></ul><ul><li>Refactoring to change underlying algorithms is a manual process – needs a “Eureka” </li></ul><ul><li>Manual refactoring is error prone </li></ul>
  27. 28. Refactoring = program transformation <ul><li>Correctness preserving transformations are present under the hood of refactoring systems </li></ul><ul><li>New transformations can be added </li></ul><ul><li>Proved sound </li></ul><ul><li>Completeness issues </li></ul><ul><li>Decidability issues </li></ul>
  28. 29. Avoiding Continuous Dis integration! <ul><li>Continuous Integration </li></ul><ul><ul><li>Frequent checkins to source control system </li></ul></ul><ul><ul><li>Build frequently (triggered after commit) </li></ul></ul><ul><ul><li>Run tests </li></ul></ul><ul><ul><li>Fix defects immediately </li></ul></ul>Avoid the wrath of Agnes!!
  29. 30. Continuous Integration (CI) Subversion server build server test server Feedback mechanism commit commit commit check Model checker Theorem Prover Analysis tools
  30. 31. Enabling CI+FM <ul><li>Faster tools </li></ul><ul><li>Harness multi core architectures </li></ul><ul><li>Deploy machines on demand in the cloud </li></ul><ul><ul><li>No need to use physical machines </li></ul></ul><ul><ul><li>Provision servers in the cloud to meet demand </li></ul></ul><ul><ul><li>Cost effective solution </li></ul></ul>
  31. 32. The Cost of Agile Time Time Effort Effort Formal development XP development
  32. 33. The Cost of Agile Time Time Effort Effort Formal development Scrum development
  33. 34. The Cost of Agile Time Time Effort Effort Formal development DSDM
  34. 35. Agile and FM: survey <ul><li>We’re putting together a survey to gain further insight into how formal methods and agile can work together </li></ul><ul><li>Areas of interest include: </li></ul><ul><ul><li>Suitability of FMs Suitable Agile approaches </li></ul></ul><ul><ul><li>Application areas Success Stories </li></ul></ul><ul><ul><li>Demographics Reasons for Failure </li></ul></ul><ul><ul><li>Team sizes Team skills </li></ul></ul><ul><ul><li>Tools used Lessons learned </li></ul></ul><ul><ul><li>Defect rates Maintainability of code </li></ul></ul>
  35. 36. Summary <ul><li>Agile has drawbacks, but these can be overcome with formal methods </li></ul><ul><li>Formal methods opens up areas for agile, e.g. Safety Critical Systems </li></ul><ul><li>Continuous Integration can be supplemented with formal methods, harnessing cloud computing </li></ul><ul><li>Dispelled myths about the costs of Agile and Formal Methods </li></ul><ul><li>Surveying the landscape: formal methods and agile usage in industry </li></ul>
  36. 37. Take Away Messages <ul><li>Formal Methods can add value in the agile domain, acting as a sanity check and safety net </li></ul><ul><li>Formal methods provides reliability, assurance and good documentation, whilst agile provides flexibility, customer satisfaction and tangible progress </li></ul>
  37. 38. Agile and FM: survey <ul><li>If you would like to participate in the survey, please email me at [email_address] </li></ul><ul><li>I look forward to hearing from you  </li></ul><ul><li>Thank you in advance. </li></ul>
  38. 39. FM + AM 2010 <ul><li>17 September, Pisa, Italy </li></ul><ul><li>One of the workshops at SEFM 2010: </li></ul><ul><ul><li>http://www.sefm2010.isti.cnr.it/ </li></ul></ul><ul><li>See http://fm-am-2010.tripod.com/index.html for further details of the workshop </li></ul><ul><li>SUBMIT SUBMIT SUBMIT SUBMIT SUBMIT SUBMIT (even though you’ll be reducing the chances of my paper being accepted!) </li></ul>
  39. 40. <shameless plug/> 20% reduction for delegates. If you are interested, please pick up a flyer. All royalties ploughed back into seminar series
  40. 41. AGILE FM The perfect partnership

×