Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Notes on teaching software testing


Published on

Published in: Technology
  • Login to see the comments

  • Be the first to like this

Notes on teaching software testing

  1. 1. Notes on Teaching Software Testing Alpen-Adria-Universität, Klagenfurt, 28 March 2013Ladislav Samuelis, Technical University of Košice, Slovakia
  2. 2. Motivation• For years I was notoriously curious about the latest updates in program development.• The main question: – How a program “is born”. Later changed to “what is software evolution”.• Challenge: grant of the Slovak Ministry of Education: “Implementation of Current Software Engineering Courses for Computer Science Programs …” Ladislav Samuelis: Notes on Teaching 2 of 23 Software Testing
  3. 3. Dawn of software testing• P. Naur, 1969: “If you develop programs systematically, you can dispense with testing altogether.”• This hope to achieve the goal is still present.• However, in practice testing cannot be avoided by any other means, whether it is the most careful design, construction, or formal analysis of the requirements.• Elimination of testing from the development process just postpones the test with reality until the product is put into production. Ladislav Samuelis: Notes on Teaching 3 of 23 Software Testing
  4. 4. Lehman’s law• E-type software, software implementing real world computer applications, must be continually evolved to maintain user satisfaction, generally over several years and releases.• It is likely that the responsible organisation will, sooner or later, consider its replacement through redevelopment or acquisition. (1974) Ladislav Samuelis: Notes on Teaching 4 of 23 Software Testing
  5. 5. Specific of software testing• Testing is a part of the development process, not the manufacturing process.• This testing is different from many other engineering products, where the emphasis is on testing the correct reproduction. Ladislav Samuelis: Notes on Teaching 5 of 23 Software Testing
  6. 6. What are the questions?• What are we teaching in software testing courses and why?• What are the scientific foundations of testing? Ladislav Samuelis: Notes on Teaching 6 of 23 Software Testing
  7. 7. Why should we have courses on software testing?• Developers spend 50% of their time testing, and reliability requirements are steadily increasing.• All programmers spend a significant amount of time to trial their software.• It gets negligible significance in most software development curricula.• Accidents, ARIANE … Ladislav Samuelis: Notes on Teaching 7 of 23 Software Testing
  8. 8. What are we teaching in software testing courses?• Techniques, measurements, management … – Plenty of comprehensive books …• ISTQB (International Software Testing Qualification Board) – Effective for undergraduate informatics study- programmes – Required by the industry Ladislav Samuelis: Notes on Teaching 8 of 23 Software Testing
  9. 9. Notes on obstacles in building software testing course• How to avoid the phenomenon of “bag of tricks”?• It is not easy to design a specific curriculum with adequate labs and assessment procedures.• The border between the program analysis and testing is fuzzy. Ladislav Samuelis: Notes on Teaching 9 of 23 Software Testing
  10. 10. How do we cooperate with the industry?• RWE IT local company provides selected lectures on: – testing of large projects – testing techniques – test cases by examples• Diploma theses and internships• Experimental data published in papers• Workshop biannually for prospective employees Ladislav Samuelis: Notes on Teaching 10 of 23 Software Testing
  11. 11. Dijkstra’s “law”• “Program testing can be used to show the presence of bugs, but never to show their absence”. (1969)• Donald Knuth: – Beware of bugs in the above code; I have only proved it correct, not tried it.This law is expresses the limitations of black-boxtesting.Question:How do we explain the theory of this “law” tostudents? Ladislav Samuelis: Notes on Teaching 11 of 23 Software Testing
  12. 12. On the limits of formal methods (D.L.Parnas)Simplification (model) is achieved by ignoringcertain facts such as the limits in the sizes ofdata elements and the errors in arithmeticoperations. Unfortunately, these are exactly thetype of details that can cause faults and lead tofailures.No formal analysis of such a model that leavesout critical limits can reveal faults attributableto those limits. Ladislav Samuelis: Notes on Teaching 12 of 23 Software Testing
  13. 13. The theory of program testing• The idea of program testing is as old as programming.• 1970 – new field of research called testing theory emerged – efficiency and effectivity• What are the main questions? – How to design test cases from different sources: specification, source code, input and output domains of programs – How to select a subset of test cases from the set of all possible test cases? Ladislav Samuelis: Notes on Teaching 13 of 23 Software Testing
  14. 14. A scheme of testing(a mathematical approach) Ladislav Samuelis: Notes on Teaching 14 of 23 Software Testing
  15. 15. The theory of Goodenough and Gerhart (very simplified explanation of the ideal test)P “runs” and gives B number of bugs.We create new set Tk until Bk ≥ B. Thenit is an ideal test. • D = input domain, T≤D, d є D • P= program, result is P(d) • d is true, if output P(d) is acceptable Ladislav Samuelis: Notes on Teaching 15 of 23 Software Testing
  16. 16. Other testing theories• Theory of Weyuker and Ostrand, Gourlay• Dines Bjørner: On a Triptych of Software Development – Domain engineering, – Requirements engineering, – Design• SEMAT initiativeQuestion:How much testing theory do we have to teach? Ladislav Samuelis: Notes on Teaching 16 of 23 Software Testing
  17. 17. Power lawThe notion of power laws has been around formore than a century. P(x) ~ c.x−kQuestion:What does it mean in software testing? Ladislav Samuelis: Notes on Teaching 17 of 23 Software Testing
  18. 18. Power law and software testing• Object-oriented software ~ network of objects.• It is worth to test less complex classes than many trivial classes.Note:This is an empirical “law” or concept , rarelytaught in testing courses. Ladislav Samuelis: Notes on Teaching 18 of 23 Software Testing
  19. 19. Gödel’s incompleteness theorem• No computer program is capable of proving all programs.• For any consistent system of axioms there will always be statements that are improvable within the system. Ladislav Samuelis: Notes on Teaching 19 of 23 Software Testing
  20. 20. Teaching of Gödels incompleteness theorems?• Elaine Weyuker - Evaluating Software Test Data Adequacy (1988)Question:Where are the places of Gödel’s theorems insoftware engineering courses? Ladislav Samuelis: Notes on Teaching 20 of 23 Software Testing
  21. 21. Note on testing and deductive reasoning• Deductive reasoning; – Major premise – initial set of facts – Minor premise – goal fact• For example, PROLOG program – the inference engine gives the verdict.• Not-PROLOG program – no inference engine, program is executed and tester “gives a verdict” against requirements. Ladislav Samuelis: Notes on Teaching 21 of 23 Software Testing
  22. 22. Questions on the role of observation in software testing Questions: Why do we select observations? Where do we perform the observation? In which phase of the product lifecycle do we perform observations? Ladislav Samuelis: Notes on Teaching 22 of 23 Software Testing
  23. 23. Notes on the history of testing concurrent programs• Before object-oriented programming: – mechanisms for ensuring mutual exclusion (Dijkstra), – monitors (Hoare) and – critical regions (Brinch-Hansen)Question:What topics to teach in concurrent programtesting? Ladislav Samuelis: Notes on Teaching 23 of 23 Software Testing