Formal Versus Agile: Survival of the Fittest? (Paul Boca)

Agile versus Formal Methods: Survival of the Fittest? Paul Boca

Collaborators
Sue Black
Mike Hinchey
Jonathan Bowen
Jason Gorman
(Scrum mistress) (Formal Methodist) ( Z z z z z z z z z z z )

About Me
PhD in program transformation (categorical stuff)
Hardware Compilation R&D (industry light)
High-level synthesis (start up)
Was R&D manager at UK static analysis vendor
Currently Quality Engineering Manager (SME)
Formal Methods enthusiast
Organizer of formal methods seminars

Outline
Agile background
Formal Methods background
Co-existence of Formal Methods and Agile
The Cost of Agile
Summary
But first...

Quiz #1
First Prize: Toyota Prius
Second Prize: Sony PlayStation 3
Runner up: Toyota Prius and PlayStation 3 

Quiz #1
63,000 ?
2,000+ ?
700 ?
280210 ?
The number of bugs in Windows 2000
The number of pages of Z specifications in a certain air-traffic control system
Number of lines of code written per month by an agile developer
Sony PlayStation 3 bug revealed!
What do each of the numbers below correspond to

Agile Manifesto
Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan
http://agilemanifesto.org/

Approaches + Techniques
Approaches
Extreme Programming
Scrum
DSDM
Techniques
Pairwise Programming
Test Driven Development

Quiz #2
Agile is about producing software quickly
Agile is about being responsive to change

Some Agile Success Stories

But Agile is not perfect
Lacks comprehensive documentation
Writing tests upfront is great, but it’s not possible to test everything
Rapidly changing requirements can be hard to trace.
Refactoring code can introduce defects
We’ll return to these later...

Formal Methods
The application of a fairly broad variety of theoretical computer science fundamentals, in particular logic calculi, formal languages, automata theory, and program semantics, but also type systems and algebraic data types to problems in software and hardware specification and verification
http://en.wikipedia.org/wiki/Formal_methods

Formal Methods Timeline Z 1977 B 1986 CSP 1978 Circus 2000 DbC 1986 1970 1980 1990 2000 2010 X machines 1974 VDM 70s Alloy 1997 FM + Agile 2003 not complete but sound 

Formal Methods Success Stories

Quiz #3
Agile will phase out Formal Methods
Formal Methods will phase out Agile
Formal Methods and Agile can coexist
What’s more, Agile and Formal Methods need one another

Agile is not perfect
It's like déjà vu all over again Yogi Berra

Why we need documentation
Code bases need to be maintained and extended
Some systems will be in operation for decades
Knowledge transfer to prepare for personnel leaving
Pairwise programming = Knowledge in silos

Design by Contract to the rescue
Preconditions and postconditions provide information on the intent of a function
Generating loop invariants can help with retrospective documentation
Contracts can be checked (semi-)automatically

To test or not to test?
Writing tests is definitely a good thing
When is enough enough?
Can only test the finite, and you can’t test everything
Testing sequential programs is hard enough, so what happens when parallelism is introduced?
What about safety critical applications?

Formal Methods to the Resue
Automated generation of test cases (e.g. Perl, Python, Functional Programming languages)
Use static analysers (e.g. Coverity)
Finds issues in code, but can’t find everything
Complements dynamic testing
Use model checkers (e.g. FDR)
Mutation testing
Modify the code base introducing “mutants”
See whether the test suite “kills” the mutant
Helps to identify gaps in test suites

Rapidly changing requirements can be hard to trace

Refactoring Code
Refactor to
Remove code smells
Beautify code
IDEs can carry out certain refactoring steps automatically.
Refactoring to change underlying algorithms is a manual process – needs a “Eureka”
Manual refactoring is error prone

Refactoring = program transformation
Correctness preserving transformations are present under the hood of refactoring systems
New transformations can be added
Proved sound
Completeness issues
Decidability issues

Avoiding Continuous Dis integration!
Continuous Integration
Frequent checkins to source control system
Build frequently (triggered after commit)
Run tests
Fix defects immediately
Avoid the wrath of Agnes!!

Continuous Integration (CI) Subversion server build server test server Feedback mechanism commit commit commit check Model checker Theorem Prover Analysis tools

Enabling CI+FM
Faster tools
Harness multi core architectures
Deploy machines on demand in the cloud
No need to use physical machines
Provision servers in the cloud to meet demand
Cost effective solution

The Cost of Agile Time Time Effort Effort Formal development XP development

The Cost of Agile Time Time Effort Effort Formal development Scrum development

The Cost of Agile Time Time Effort Effort Formal development DSDM

Agile and FM: survey
We’re putting together a survey to gain further insight into how formal methods and agile can work together
Areas of interest include:
Suitability of FMs Suitable Agile approaches
Application areas Success Stories
Demographics Reasons for Failure
Team sizes Team skills
Tools used Lessons learned
Defect rates Maintainability of code

Summary
Agile has drawbacks, but these can be overcome with formal methods
Formal methods opens up areas for agile, e.g. Safety Critical Systems
Continuous Integration can be supplemented with formal methods, harnessing cloud computing
Dispelled myths about the costs of Agile and Formal Methods
Surveying the landscape: formal methods and agile usage in industry

Take Away Messages
Formal Methods can add value in the agile domain, acting as a sanity check and safety net
Formal methods provides reliability, assurance and good documentation, whilst agile provides flexibility, customer satisfaction and tangible progress

Agile and FM: survey
If you would like to participate in the survey, please email me at [email_address]
I look forward to hearing from you 
Thank you in advance.

FM + AM 2010
17 September, Pisa, Italy
One of the workshops at SEFM 2010:
http://www.sefm2010.isti.cnr.it/
See http://fm-am-2010.tripod.com/index.html for further details of the workshop
SUBMIT SUBMIT SUBMIT SUBMIT SUBMIT SUBMIT (even though you’ll be reducing the chances of my paper being accepted!)

<shameless plug/> 20% reduction for delegates. If you are interested, please pick up a flyer. All royalties ploughed back into seminar series

AGILE FM The perfect partnership

Formal Versus Agile: Survival of the Fittest? (Paul Boca)

by AdaCore on Mar 18, 2010

Accessibility

Categories

Tags

Upload Details

Usage Rights

2 Embeds 89

Statistics

Formal Versus Agile: Survival of the Fittest? (Paul Boca) — Presentation Transcript

http://www.open-do.org	85
http://www.slideshare.net	4