Successfully reported this slideshow.
Your SlideShare is downloading. ×

(Moonconf 2016) Fetching Moths from the Works: Correctness Methods in Software

Ad

FETCHING MOTHS
FROM THE WORKS
CORRECTNESS METHODS IN SOFTWARE

Ad

- or -

Ad

An Investigation into the
Nature of Software with
a Particular Concern
toward its Effective
Construction

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Check these out next

1 of 114 Ad
1 of 114 Ad

(Moonconf 2016) Fetching Moths from the Works: Correctness Methods in Software

Download to read offline

We live in a nice world. There’s a wealth of historical thought on achieving correctness in software–shipping code that does only what is intended, not less and not more–and there are a whole bunch of methods available to us as practitioners. Some of these are hard to apply, some are easy. For instance, case testing is widely used and considered standard practice. Property testing is understood to exist but not widely used. The application of advanced logics? Way out there.

If you look around you’ll find a lot of software fails a lot of the time. Why is that?

In this talk I’ll give an overview of the methods for producing correct systems and will discuss each in its historical context. With each method, we’ll keep an eye out for present applications and the difficulty of doing so. We’ll discuss why there’s so much buggy software in the world. I expect there will be talk of spaceships a bit. By the end of this talk you ought to be able to make reasoned decisions about applying correctness methods in your own work and have a good shot at building better software.

We live in a nice world. There’s a wealth of historical thought on achieving correctness in software–shipping code that does only what is intended, not less and not more–and there are a whole bunch of methods available to us as practitioners. Some of these are hard to apply, some are easy. For instance, case testing is widely used and considered standard practice. Property testing is understood to exist but not widely used. The application of advanced logics? Way out there.

If you look around you’ll find a lot of software fails a lot of the time. Why is that?

In this talk I’ll give an overview of the methods for producing correct systems and will discuss each in its historical context. With each method, we’ll keep an eye out for present applications and the difficulty of doing so. We’ll discuss why there’s so much buggy software in the world. I expect there will be talk of spaceships a bit. By the end of this talk you ought to be able to make reasoned decisions about applying correctness methods in your own work and have a good shot at building better software.

Advertisement
Advertisement

More Related Content

Advertisement

(Moonconf 2016) Fetching Moths from the Works: Correctness Methods in Software

  1. 1. FETCHING MOTHS FROM THE WORKS CORRECTNESS METHODS IN SOFTWARE
  2. 2. - or -
  3. 3. An Investigation into the Nature of Software with a Particular Concern toward its Effective Construction
  4. 4. - or -
  5. 5. Why do Computers Fail and What can be Done About It?
  6. 6. Why do Computers Stop and What Can be Done About It? Jim Gray - 1986 @bltroutwine Moonconf, 2016
  7. 7. “The resulting systems have hardware MTBF measured in decades or centuries. ” @bltroutwine Moonconf, 2016
  8. 8. “Unfortunately, it says nothing about tolerating the major sources of failure. . .” @bltroutwine Moonconf, 2016
  9. 9. “Software” @bltroutwine Moonconf, 2016
  10. 10. design failures, open doors -- @bltroutwine Moonconf, 2016
  11. 11. The Case of the Three Engineers vs. BART Gordon Friedlander - 1974 @bltroutwine Moonconf, 2016
  12. 12. agile iteration takes to the sky -- @bltroutwine Moonconf, 2016
  13. 13. SIGSOFTVol. 6 No. 2: Frontmatter *gSoft Editor - 1981 @bltroutwine Moonconf, 2016
  14. 14. a bug affects the staging prototype -- @bltroutwine Moonconf, 2016
  15. 15. The BUG Heard 'Round the World Discussion of The Software Problem Which Delayed the First Shuttle Orbital Flight John Garman - 1981 @bltroutwine Moonconf, 2016
  16. 16. “Maintaining software systems in the field, absorbing large changes or additions in the middle of development cycles. . . @bltroutwine Moonconf, 2016
  17. 17. . . . reconfiguring software systems to ‘fit’ never-quite- identical vehicles or missions are our real problems today.” @bltroutwine Moonconf, 2016
  18. 18. That was the late 1970s, have we made progress? @bltroutwine Moonconf, 2016
  19. 19. Yes! @bltroutwine Moonconf, 2016
  20. 20. Sorta! @bltroutwine Moonconf, 2016
  21. 21. ‘Correct’ is not a state, it’s a goal. @bltroutwine Moonconf, 2016
  22. 22. What’s needed is an understanding of how we fail to achieve it. @bltroutwine Moonconf, 2016
  23. 23. Analyzing Software Requirements Errors in Safety-Critical, Embedded Systems Robin R. Lutz - 1993 @bltroutwine Moonconf, 2016
  24. 24. “Few internal faults were uncovered during integration and system testing.” @bltroutwine Moonconf, 2016
  25. 25. “Functional faults are the most common kind of software error.” @bltroutwine Moonconf, 2016
  26. 26. What kind of software faults are there? @bltroutwine Moonconf, 2016
  27. 27. Program Faults • Internal mistakes • Interface violations • Functional violations @bltroutwine Moonconf, 2016
  28. 28. Program Faults • Internal • Interface • Functional Bugs @bltroutwine Moonconf, 2016
  29. 29. Human Error • Intra-team comms. • Extra-team comms. • Misunderstanding spec. • Mishandling spec. @bltroutwine Moonconf, 2016
  30. 30. Human Error • Intra-team comms. • Extra-team comms. • Misunderstanding spec. • Mishandling spec. Comm. Problems@bltroutwine Moonconf, 2016
  31. 31. Process Error • Inadequate testing • Inadequate specs. • Unknown requirements • Incorrect requirements @bltroutwine Moonconf, 2016
  32. 32. Process Error • Inadequate testing • Inadequate specs. • Unknown requirements • Incorrect requirements Org. Goofs@bltroutwine Moonconf, 2016
  33. 33. ‘Correct’ breaks down into two sub-goals. @bltroutwine Moonconf, 2016
  34. 34. - Validation - @bltroutwine Moonconf, 2016
  35. 35. - Verification - @bltroutwine Moonconf, 2016
  36. 36. What steps can we take today? @bltroutwine Moonconf, 2016
  37. 37. Step 0. @bltroutwine Moonconf, 2016
  38. 38. Convince your organization to invest. @bltroutwine Moonconf, 2016
  39. 39. Eliminating Embedded Software Defects Prior to Integration Test Ted Bennett, Paul Wennberg - 2005 @bltroutwine Moonconf, 2016
  40. 40. “The more faults that pass undetected into integration test and beyond, the more the project will cost and the longer it will take to complete.” @bltroutwine Moonconf, 2016
  41. 41. Step 1. @bltroutwine Moonconf, 2016
  42. 42. Aim to make systems both safe and reliable. @bltroutwine Moonconf, 2016
  43. 43. Engineering a Safer World Systems Thinking Applied to Safety Nancy Leveson - 2011 @bltroutwine Moonconf, 2016
  44. 44. Step 2. @bltroutwine Moonconf, 2016
  45. 45. Be clear on what your system must and mustn’t do. @bltroutwine Moonconf, 2016
  46. 46. The Role of Software in Spacecraft Accidents Nancy Leveson - 2004 @bltroutwine Moonconf, 2016
  47. 47. “. . .software specifications often describe nominal behavior well but are very incomplete with respect to required software behavior under off-nominal conditions . . . @bltroutwine Moonconf, 2016
  48. 48. “Most safety-related requirements. . .are best described using. . .design constraints.” @bltroutwine Moonconf, 2016
  49. 49. Step 3. @bltroutwine Moonconf, 2016
  50. 50. "We don't want nobody that nobody sent." @bltroutwine Moonconf, 2016
  51. 51. The Role of Software in Spacecraft Accidents Nancy Leveson - 2004 @bltroutwine Moonconf, 2016
  52. 52. “It is widely believed that because software has executed safely in other applications, it will be safe in the new one. . . @bltroutwine Moonconf, 2016
  53. 53. (M)ost accidents involve software that is doing exactly what it was designed to do (but) it reliably performs the wrong function.” @bltroutwine Moonconf, 2016
  54. 54. Step 4. @bltroutwine Moonconf, 2016
  55. 55. Audit and review all code. Aid with automated tests. @bltroutwine Moonconf, 2016
  56. 56. The OpenBSD Culture David Gwynne - 2006 @bltroutwine Moonconf, 2016
  57. 57. Going Fast Slowly Poul-Henning Kamp, 2016 @bltroutwine Moonconf, 2016
  58. 58. How SQLite is Tested Dwayne Hipp - 2009 @bltroutwine Moonconf, 2016
  59. 59. Step 5. @bltroutwine Moonconf, 2016
  60. 60. Use randomized testing and track coverage. @bltroutwine Moonconf, 2016
  61. 61. An Evaluation of Randomized Testing Joe Duran, *meon Ntafos - 1984 @bltroutwine Moonconf, 2016
  62. 62. “Our experiments have shown that random testing can discover some relatively subtle errors without a great deal of effort.” @bltroutwine Moonconf, 2016
  63. 63. QuickCheck A Lightweight Tool for Random Testing of Haskell Programs Coen Claessen, John Hughes - 2000 @bltroutwine Moonconf, 2016
  64. 64. Step 6. @bltroutwine Moonconf, 2016
  65. 65. Be willing to change your approach. @bltroutwine Moonconf, 2016
  66. 66. An Experimental Evaluation of the Assumption of Independence in Multiversion Programming Nancy Leveson, John Knight - 1986 @bltroutwine Moonconf, 2016
  67. 67. Step 7. @bltroutwine Moonconf, 2016
  68. 68. Use tools amenable to formal methods. @bltroutwine Moonconf, 2016
  69. 69. Rigorous Software Development An Introduction to ProgramVerification Jose Almedia et al., 2011 @bltroutwine Moonconf, 2016
  70. 70. Building High Integrity Applications with SPARK John McCormick, Peter Chapin - 2015 @bltroutwine Moonconf, 2016
  71. 71. Step 9. @bltroutwine Moonconf, 2016
  72. 72. Use formal methods. @bltroutwine Moonconf, 2016
  73. 73. Formal Specification and Documentation with Z A Case Study Approach Jonathan Bowen, 2003 @bltroutwine Moonconf, 2016
  74. 74. Moving Fast with Software Verification Cristiano Calcagno et al., 2015 @bltroutwine Moonconf, 2016
  75. 75. Step 9. @bltroutwine Moonconf, 2016
  76. 76. Build simple. @bltroutwine Moonconf, 2016
  77. 77. Out of the Tar Pit Ben Moseley, Peter Marks - 2006 @bltroutwine Moonconf, 2016
  78. 78. Normal Accidents Living with High-Risk Technologies Charles Perrow - 1986 @bltroutwine Moonconf, 2016
  79. 79. Step 10. @bltroutwine Moonconf, 2016
  80. 80. Build for failure. @bltroutwine Moonconf, 2016
  81. 81. Crash-Only Software George Candea, Armando Fox - 2003 @bltroutwine Moonconf, 2016
  82. 82. Making Reliable Distributed Systems in the Presence of Software Errors Joe Armstrong - 2003 @bltroutwine Moonconf, 2016
  83. 83. “We assume that such programs do contain errors, and investigate methods for building reliable systems despite such errors.” @bltroutwine Moonconf, 2016
  84. 84. What must we invent? @bltroutwine Moonconf, 2016
  85. 85. Formal specification tools a project manager can love. @bltroutwine Moonconf, 2016
  86. 86. Effective system modeling tools. @bltroutwine Moonconf, 2016
  87. 87. Methods for the effective analysis of running systems. @bltroutwine Moonconf, 2016
  88. 88. A techno-political culture of excellence. @bltroutwine Moonconf, 2016
  89. 89. What can we study? @bltroutwine Moonconf, 2016
  90. 90. Lots! @bltroutwine Moonconf, 2016
  91. 91. The End!

×