Ten Commandments of
 Formal Methods: A decade later
Jonathan P. Bowen Michael G. Hinchey
Museophile Limited, UK          L...
Dagstuhl Seminar 06191
Rigorous Methods for Software Construction
       and Analysis, 8–12 May 2006
Choosing a formal method – difficult

                        ASM
               GC6
        B

                  Z




  ...
Background – formal methods
  Academics vs. industrial practitioners
  Theory vs. practice
  Still little used in general ...
The Flat Earth
   Society
Cf. formal methods
community…
— Gerard J. Holzmann




     FMICS 2005 (Lisbon)
     conference ...
Ten Commandments
  … ten years later
J.P. Bowen & M.G. Hinchey,
IEEE Computer,
April 1995 & January 2006

He proclaimed to...
Thou shalt choose an
   appropriate notation.
Notations are a frequent complaint…
but the real problem is to understand
th...
Which notation?

Various notations:
   ASM (testing?)
   B (development?)
   Z (specification?)
   Etc. – 95 under
   vl.f...
Beware
Panaceas!



 Cf. Formal
  methods
Caviat
Emptor!
Cf. Software
Name       Combines        Advantage          Ref.
Combined            Temporal   B, temporal    Adds time to the Bonnet e...
Thou shalt formalize
        but not overformalize.

    Need for formality
    Formality vs. informality
    Levels of us...
Level     Name             Involves

Levels          0       Formal        Formal notation
                      Specifica...
Thou shalt estimate costs.
   Estimation models (CoCoMo II, …)
   Total cost of ownership (TCO)
   Quality of people varie...
200
                                                                           Requirements
                              ...
Cost of proofs
Mathematics – simple theorems,
deep proofs (decades or centuries)
Cf. software – complicated specs
& progra...
Hand vs.
                                   machine
                                   checked
                           ...
Thou shalt have a formal
       methods guru on call.
 Communication/understanding important
 Project management
 Technolo...
Technology transfer
E.g.: Z notation
Courses (academia & industry)
Textbooks (good choice)
Tools (type-checkers, provers, ...
Formal Methods Europe
FME: started with European funding
Industry, academia and government
Now more international in scope...
ForTIA
Formal Techniques Industry Association
Founded through European CoLogNET
Computational Logic Network and FME
at FM2...
Thou shalt not abandon
        thy traditional
        development methods.
      UML
      Object-orientation
      Model...
UML & OO methods
Unified Modeling Language
    pUML (precise UML)
    Combined with B-Method tools
Object-Z
Perfect Develo...
Thou shalt document
     sufficiently.

Case studies – success & failure
Process important
Textbooks (c10 Z vs. c1000 Java...
Google Book Search   books.google.com
Textbooks
for courses
 Resistance by
 students
 Resistance even
 by academics
 Professional
 society
 accreditation
 (e.g....
Software Specification Methods
                       Henri Habrias &
                       Marc Frappier (eds.)
        ...
Wikipedia
                         Z notation category.
                         Add ASM, B-Method,
                      ...
Thou shalt not compromise
          thy quality standards.
$360B loses due to poor software quality (2002)
ISO 9000 revise...
Z Standard
ISO/IEC 13568
Long process (1990s)
Final Committee Draft
– accepted in 2001!
Important for tools and
industrial...
Thou shalt not
          be dogmatic.
Listen to industry’s problems
Choice may depend on expertise
Good tool support impor...
Community Z Tools
Open systems model – e.g.,
Community Z Tools (CZT) initiative
Sourceforge project:
czt.sourceforge.net
Open source initiatives
European RODIN project (2004–2007):
   Rigorous Open Development Environment
   for Complex System...
Thou shalt test,
        test, and test again.
Even short programs complex
Small changes can cause large problems
Easy to ...
FORTEST Network
 Formal methods and testing


           www.fortest.org.uk
UK academia and industry (3 years funding)
Reg...
Formalization of testing criteria
  Z notation – readable
      Existing criteria (e.g., MC/DC)
      Modified Condition/D...
Formalization of testing criteria

 Using the Z notation. E.g.:
Thou shalt reuse.

  Possible if “formal”
  Cheaper at higher levels of abstraction
  Levels of complexity

The biggest di...
Levels of complexity
25 lines of informal requirements
250 lines of (formal) specification
2,500 lines of design descripti...
Reflection
Oui, l'œuvre sort plus belle
D'une forme au travail
Rebelle,
Vers, marbre, onyx, émail.
[Yes, the work comes ou...
Grand Challenge 6
1 of 7: Dependable Systems Evolution

Sir Tony Hoare et al.

Verifying Compiler (this century!)

Worksho...
Verified Software Repository
 Cf. QED Pro Quo repository – www.qpq.org
 Case study software, tools, challenges
     Mondex...
Conclusion
   Continued niche market for critical systems
      Especially safety and security
      Hardware as well as s...
Applied Formal Methods

"You know my methods.
     Apply them."
             — Sir Arthur Conan Doyle
               The S...
SEFM 2007 conference
IEEE conference on Software
Engineering and Formal Methods
Keyworth Centre, London South Bank
Univers...
ABZ08: ASM, B, Z meeting
 ASM, B, Z user groups & VSR-net
 2008 Jean-Raymond Abrial’s 70th birthday
 (inventor of Z and B)...
ASM, B, Z meeting – people
ASM – Egon Börger (Pisa)

B – Michael Butler (Southampton)

Z – Jonathan Bowen (London)

VSR-ne...
www.fmnet.info


The End                Keyworth Centre ▲
                    Hubble in clean room ▼




sel.gsfc.nasa.gov
Upcoming SlideShare
Loading in …5
×

Ten Commandments of Formal Methods: A decade later

5,439 views

Published on

In 1995, a paper "Ten Commandments of Formal Methods" suggested some guidelines to help ensure the success of a formal methods project. It proposed ten requirements (or “commandments”) for formal developers to consider and follow, based on our knowledge of several industrial application success stories, most of which have been reported in more detail in two books. The paper was surprisingly popular, is still widely referenced, and used as required reading in a number of formal methods courses. However, not all have agreed with some of the commandments, feeling that they may not be valid in the long-term. We re-examine the original commandments a decade later, and consider their validity in the light of industrial best practice and experiences, especially with respect to formal notations such as B and Z. We also cover the activities of the UK Verified Software Repository Network (VSR-net) in the context of Grand Challenge 6 on Dependable Systems Evolution.

Published in: Technology
7 Comments
6 Likes
Statistics
Notes
No Downloads
Views
Total views
5,439
On SlideShare
0
From Embeds
0
Number of Embeds
78
Actions
Shares
0
Downloads
0
Comments
7
Likes
6
Embeds 0
No embeds

No notes for slide

Ten Commandments of Formal Methods: A decade later

  1. 1. Ten Commandments of Formal Methods: A decade later Jonathan P. Bowen Michael G. Hinchey Museophile Limited, UK Loyola College in Maryland Baltimore, USA www.jpbowen.com (Also visiting academic, (Also NASA) University College London) See IEEE Computer, 39(1):40–48, January 2006. Based on Dagstuhl workshop, Germany, 8–12 May 2006.
  2. 2. Dagstuhl Seminar 06191 Rigorous Methods for Software Construction and Analysis, 8–12 May 2006
  3. 3. Choosing a formal method – difficult ASM GC6 B Z VSR-net
  4. 4. Background – formal methods Academics vs. industrial practitioners Theory vs. practice Still little used in general practice Size of community critical It is clear to the best minds in the field that a more mathematical approach is needed for software to advance much. ― Bertrand Meyer
  5. 5. The Flat Earth Society Cf. formal methods community… — Gerard J. Holzmann FMICS 2005 (Lisbon) conference queue! ►
  6. 6. Ten Commandments … ten years later J.P. Bowen & M.G. Hinchey, IEEE Computer, April 1995 & January 2006 He proclaimed to you his covenant, which he commanded you to keep: the Ten Commandments, which he wrote on two tablets of stone. “Can’t I just read your URL?” ― Deuteronomy 4:13, 10:4, Ex.34:28 vl.fmnet.info/moses-url
  7. 7. Thou shalt choose an appropriate notation. Notations are a frequent complaint… but the real problem is to understand the meaning and properties of the symbols … … you will cultivate an appreciation of mathematical elegance and style. By that time, the symbols will be invisible ... The great advantage of mathematics is that the rules are simpler than those of natural language ― C.A.R. Hoare
  8. 8. Which notation? Various notations: ASM (testing?) B (development?) Z (specification?) Etc. – 95 under vl.fmnet.info
  9. 9. Beware Panaceas! Cf. Formal methods
  10. 10. Caviat Emptor! Cf. Software
  11. 11. Name Combines Advantage Ref. Combined Temporal B, temporal Adds time to the Bonnet et B logic B-Method al. (1995) formal methods ZCCS Z, CCS Combines CCS Galloway process algebra and add to the and state based Stoddard confusion! aspects of Z (1997) CSP OZ Z, CSP Combines Z and Fischer CSP (2000) If I could say it [13] in words there Object Z Z, OO Adds OO to Z Smith would be no principles, (2000) temporal reason to paint. logic ― Edward Hopper Object-Z, π- Adds π-calculus PiOZ Taguchi calculus style dynamic et al. (1882–1967) comm. (2004) capabilities to Object-Z
  12. 12. Thou shalt formalize but not overformalize. Need for formality Formality vs. informality Levels of use Strange as it seems, no amount of learning can cure stupidity, and formal education positively fortifies it. ― Stephen Vizinczey
  13. 13. Level Name Involves Levels 0 Formal Formal notation Specification used for specifying of use requirements only; no analysis/proof 1 Formal Proving properties Development / and applying Verification refinement calculus Cost vs. 2 Machine Use of theorem correctness Checked prover/checker (quality) Proofs / Model tool to prove checking consistency/ integrity.
  14. 14. Thou shalt estimate costs. Estimation models (CoCoMo II, …) Total cost of ownership (TCO) Quality of people varies (c10:1?) Cost (salary) varies (c2:1?) Still an inexact “science” I think that God in creating Man somewhat overestimated his ability. ― Oscar Wilde (1854–1900)
  15. 15. 200 Requirements GRO78 Target Cost Overrun, Percent phase costs OMV TDRSS compared with 160 IRAS project overrun Gali HST costs GOES I-M TETH 120 (source: NASA) LAND76 CEN EDO (recent start) MARS ACTS ERB77 COBE 80 STS CHA.REC LAND78 GRO82 ERB80 SEASAT 40 UARS VOYAGER HEAO EUVE/EP DE Ulysses ISEE SMM PIONVEN IUE 0 Ref: NASA/W. Gruhl 0 5 10 15 20 Requirements Cost/Program Cost, percent
  16. 16. Cost of proofs Mathematics – simple theorems, deep proofs (decades or centuries) Cf. software – complicated specs & programs, shallow proofs (B, 90–95% automated, 5–10% manual, weeks or months). Fermat’s Last Theorem (in Toulouse) an + bn ≠ cn (n>2) — Pierre de Fermat (1601–1635)
  17. 17. Hand vs. machine checked proofs Blackboard at Dagstuhl workshop!
  18. 18. Thou shalt have a formal methods guru on call. Communication/understanding important Project management Technology transfer Support organizations (FME, ForTIA, …) An expert is a person who has made all the mistakes that can be made in a very narrow field. ― Niels Bohr (1885–1962)
  19. 19. Technology transfer E.g.: Z notation Courses (academia & industry) Textbooks (good choice) Tools (type-checkers, provers, …) Web resources – vl.fmnet.info Discussion – comp.specification.* User Group (meetings) Standards (see later)
  20. 20. Formal Methods Europe FME: started with European funding Industry, academia and government Now more international in scope FM’06: 14th Symposium Hamilton, Canada, 21–27 Aug 2006 www.fmeurope.org FME Wiki: www.fmeurope.org/twiki/bin/view
  21. 21. ForTIA Formal Techniques Industry Association Founded through European CoLogNET Computational Logic Network and FME at FM2003 symposium, Pisa Subgroup of FME Technology transfer to industry See: www.fortia.org
  22. 22. Thou shalt not abandon thy traditional development methods. UML Object-orientation Model-Based Development (MBD) A great many of those who ‘debunk’ traditional... values have in the background values of their own which they believe to be immune from the debunking process. ― C. S. Lewis (1898–1963) The Abolition of Man
  23. 23. UML & OO methods Unified Modeling Language pUML (precise UML) Combined with B-Method tools Object-Z Perfect Developer (Java/C++) Escher Technologies Applied to self, proving c95% of approx. 130,000 verification conds Cf. Atelier-B tool?
  24. 24. Thou shalt document sufficiently. Case studies – success & failure Process important Textbooks (c10 Z vs. c1000 Java!) I have always tried to hide my own efforts and wished my works to have the lightness and joyousness of a springtime which never lets anyone suspect the labours it cost. ― Henri Matisse (1869–1954)
  25. 25. Google Book Search books.google.com
  26. 26. Textbooks for courses Resistance by students Resistance even by academics Professional society accreditation (e.g., BCS)
  27. 27. Software Specification Methods Henri Habrias & Marc Frappier (eds.) Springer-Verlag, 2001 and ISTE, 2006 Z, SAZ, B, OMT, Action Systems, UML, VHDL, Estelle, SDL, E-LOTOS, JSD, CASL, Coq, Petri Nets, TLA. Process of producing a formal spec…
  28. 28. Wikipedia Z notation category. Add ASM, B-Method, … categories? en.wikipedia.org/wiki/Formal_methods See also: en.wikipedia.org/wiki/Category:Formal_methods
  29. 29. Thou shalt not compromise thy quality standards. $360B loses due to poor software quality (2002) ISO 9000 revised (2000) IEC 61508-3 functional safety standard (1998) 00-55 UK MoD standard updated (1997) 00-56 Issue 3 for hardware-software (2005) FMs mandated for safety-related software If people knew how hard I worked to get my mastery, it wouldn't seem so wonderful at all. ― Michelangelo Buonarroti (1475–1564)
  30. 30. Z Standard ISO/IEC 13568 Long process (1990s) Final Committee Draft – accepted in 2001! Important for tools and industrial use ASM, B, … ?
  31. 31. Thou shalt not be dogmatic. Listen to industry’s problems Choice may depend on expertise Good tool support important Combined theorem proving/model checking (e.g., Yices from SRI) … And I am unanimous in that! ― Molly Sugden, a.k.a. Mrs. Slocombe Are You Being Served? BBC TV (1972–1993)
  32. 32. Community Z Tools Open systems model – e.g., Community Z Tools (CZT) initiative Sourceforge project: czt.sourceforge.net
  33. 33. Open source initiatives European RODIN project (2004–2007): Rigorous Open Development Environment for Complex Systems rodin.cs.ncl.ac.uk Support for B# (“B sharp”, cf. C#) rodin-b-sharp.sourceforge.net See also B4free: www.b4free.com HOL 4: hol.sourceforge.net Jape: sourceforge.net/projects/jape
  34. 34. Thou shalt test, test, and test again. Even short programs complex Small changes can cause large problems Easy to change, not easy to be correct I believe the hard part of building software to be the specification, design and testing of this conceptual construct, not the labor of representing it and testing the fidelity of the representation. ― Frederick P. Brooks, Jr., No Silver Bullet
  35. 35. FORTEST Network Formal methods and testing www.fortest.org.uk UK academia and industry (3 years funding) Regular workshops (last 19 Dec 2005, London) “Landscapes” ACM Surveys paper to appear Book in preparation for Springer LNCS (2007)
  36. 36. Formalization of testing criteria Z notation – readable Existing criteria (e.g., MC/DC) Modified Condition/Decision Coverage New criteria (e.g., RC/DC) Reinforced Condition/Decision Coverage (false actuation type errors detected) Reduces ambiguity, increases understanding See: Formal Aspects of Computing, 18(1):42–62, March 2006 & STVR, 15(1):21–40, March 2005 [Work with Sergiy Vilkomir & Kalpesh Kapoor] See: www.cafm.lsbu.ac.uk/fortest
  37. 37. Formalization of testing criteria Using the Z notation. E.g.:
  38. 38. Thou shalt reuse. Possible if “formal” Cheaper at higher levels of abstraction Levels of complexity The biggest difference between time and space is that you can't reuse time. ― Merrick Furst
  39. 39. Levels of complexity 25 lines of informal requirements 250 lines of (formal) specification 2,500 lines of design description 25,000 lines of high-level program code 250,000 machine instructions of object code 2,500,000 CMOS transistors in hardware!
  40. 40. Reflection Oui, l'œuvre sort plus belle D'une forme au travail Rebelle, Vers, marbre, onyx, émail. [Yes, the work comes out more beautiful from a material that resists the process, verse, marble, onyx, or enamel.] — Théophile Gautier (1811–1872) L'Art
  41. 41. Grand Challenge 6 1 of 7: Dependable Systems Evolution Sir Tony Hoare et al. Verifying Compiler (this century!) Workshops: e.g., Zurich, Dagstuhl Further information: www.fmnet.info/gc6
  42. 42. Verified Software Repository Cf. QED Pro Quo repository – www.qpq.org Case study software, tools, challenges Mondex Electronic Purse (security) Dagstuhl Seminar (10–14 June 2006) UK EPSRC VSR-net network (2005–2008) EPSRC project proposal Last meeting (York, UK, 5–6 October 2006) Further information: www.fmnet.info/vsr-net
  43. 43. Conclusion Continued niche market for critical systems Especially safety and security Hardware as well as software (model checking) Tools very important (open source?) Breakthrough with theorem proving/model checking? Breaking the “5,000” glass ceiling? … in this area my academic colleagues are doing exactly what they should do: developing and propagating an indispensable technology so that it will be available when “the world out there” undeniably needs it. ― Edsger W. Dijkstra (1930–2002)
  44. 44. Applied Formal Methods "You know my methods. Apply them." — Sir Arthur Conan Doyle The Sign of Four (1890) URL: vl.fmnet.info Virtual Library
  45. 45. SEFM 2007 conference IEEE conference on Software Engineering and Formal Methods Keyworth Centre, London South Bank University, UK, 10-14 September 2007 URL: www.iist.unu.edu/SEFM07 Submission deadline: 31 March 2007
  46. 46. ABZ08: ASM, B, Z meeting ASM, B, Z user groups & VSR-net 2008 Jean-Raymond Abrial’s 70th birthday (inventor of Z and B) BCS London offices, 15-18 September 2008 c/o BCS Formal Aspects of Computing Science (FACS) Specialist Group Free venue for BCS SGs (120 people max) 1 day joint, 2 days in parallel, 1 day VSR-net workshop (space dividable)
  47. 47. ASM, B, Z meeting – people ASM – Egon Börger (Pisa) B – Michael Butler (Southampton) Z – Jonathan Bowen (London) VSR-net – Jim Woodcock (York) Local organization – Paul Boca (London) Industrial case study – Ian Oliver (Nokia, Helsinki)
  48. 48. www.fmnet.info The End Keyworth Centre ▲ Hubble in clean room ▼ sel.gsfc.nasa.gov

×