Software Testing Techniques Dan Berger


Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Software Testing Techniques Dan Berger

  1. 1. Software Testing Techniques Dan Berger [email_address] Peter Fröhlich [email_address] edu
  2. 2. Outline <ul><li>What is Testing? </li></ul><ul><li>Why Test? </li></ul><ul><li>How To Test The Tests? </li></ul><ul><li>Code Coverage (Peter) </li></ul><ul><li>How To Decide What To Test? </li></ul><ul><li>Automation Tools </li></ul><ul><li>Writing Test Cases </li></ul>
  3. 3. What Is Testing? <ul><li>In this context – confirming correct behavior of a piece of software. </li></ul>
  4. 4. Why Test? <ul><li>We know what we intended the code to do – we need to confirm that it does what we intended. </li></ul>
  5. 5. How To Test Our Tests? <ul><li>In other words: how can we be sure our tests are: </li></ul><ul><ul><li>testing the right (relevant) “stuff”? </li></ul></ul><ul><ul><li>testing “most” of the code and not just repeatedly testing the same code? </li></ul></ul>
  6. 6. Code Coverage <ul><li>Take it away, Peter… </li></ul>
  7. 7. How To Decide What To Test? Low-Level Classes Application Subsystems Your Program
  8. 8. How To Decide (II) <ul><li>The trick is drawing a line across that triangle. </li></ul><ul><ul><li>Draw it too low and you have too many classes to test – and the tests aren’t meaningful. </li></ul></ul><ul><ul><li>Draw it too high and a test failure leads you on a wild goose chase… </li></ul></ul><ul><li>The bad news is – deciding where to draw that line is an art, not a science. </li></ul>
  9. 9. Automation Tools <ul><li>How many times have you written code and then sat in front of the keyboard running through test cases? </li></ul><ul><ul><li>How many times have you actually thought about test cases? </li></ul></ul><ul><li>The good news is that there is a better way. </li></ul><ul><ul><li>And it’s never too late for a new beginning… </li></ul></ul>
  10. 10. Automation To The Rescue <ul><li>A few key tools: </li></ul><ul><ul><li>make </li></ul></ul><ul><ul><ul><li>make running tests a part of your build </li></ul></ul></ul><ul><ul><li>scripting </li></ul></ul><ul><ul><ul><li>if you can automate it, automate it </li></ul></ul></ul><ul><ul><li>expect </li></ul></ul><ul><ul><ul><li>a powerful tool </li></ul></ul></ul>
  11. 11. Make <ul><li>Add a test rule to your makefile – it should run all the tests you’ve written and log their results. </li></ul><ul><li>If they all pass, you’re done – if not, figure out why not. </li></ul>
  12. 12. Scripting <ul><li>Imagine you have a bunch of executables that test various parts of your program. </li></ul><ul><ul><li>We’ll talk about how they might be built shortly. </li></ul></ul><ul><li>Imagine you’ve written a shell script to run them, one at a time. </li></ul><ul><li>How do you know when one fails? </li></ul>
  13. 13. Scripting (II) <ul><li>You can check the exit code of a program running from a shell script: </li></ul><ul><ul><li>if [ $? –ne 0 ]; then … else … fi </li></ul></ul><ul><li>if the executable exits abnormally, the then clause executes. </li></ul>
  14. 14. Expect <ul><li>Written by Don Libes – an extension to Tcl </li></ul><ul><ul><li>What, you’ve never heard of tcl? </li></ul></ul><ul><ul><li>That’s OK – there’s a python version as well. </li></ul></ul><ul><ul><ul><li>If you’ve never heard of python, you should have attended our python seminar… </li></ul></ul></ul>
  15. 15. Expect (II) <ul><li>It’s be installed on the lab machines </li></ul><ul><li>Basic commands: </li></ul><ul><li>spawn </li></ul><ul><ul><li>run’s a new command under script control. </li></ul></ul><ul><li>send </li></ul><ul><ul><li>provides input to the command </li></ul></ul><ul><li>expect </li></ul><ul><ul><li>matches a pattern in the command output </li></ul></ul>
  16. 16. Expect (Example) <ul><li>#!/usr/bin/expect </li></ul><ul><li>set timeout 5 </li></ul><ul><li>spawn ssh –x [lindex $argv 0] “ls /” </li></ul><ul><li>expect { </li></ul><ul><li>timout {puts “timeout”} </li></ul><ul><li>eof {exit 0} </li></ul><ul><li>“ continue connecting (yes/no)? ” {send “yes” } </li></ul><ul><li>} </li></ul><ul><li>expect { </li></ul><ul><li>eof {exit 0} </li></ul><ul><li>} </li></ul>
  17. 17. Python Expect <ul><li>Pexpect – http:// pexpect </li></ul><ul><li>Good News: Provides *most* of expect’s functionality from within python. </li></ul><ul><li>Bad News: it’s not installed on the lab machines (yet?). </li></ul>
  18. 18. Writing Test Cases <ul><li>Write a simple executable that instantiates a class, and “drives” that instance. </li></ul>
  19. 19. Test Cases (Cont.) <ul><li>Imagine you have a linked list class… </li></ul><ul><ul><li>If you wrote your own linked list class – and you’re working in C++ - you should have been in our STL seminar. </li></ul></ul><ul><li>What should your test program test? </li></ul>
  20. 20. Parting Words <ul><li>Be a skeptic – you, as the developer, are the only one who can crystal-box test your code. </li></ul><ul><ul><li>Don’t test the easy cases – test the ones you’re not sure about. </li></ul></ul><ul><ul><li>Don’t waste time testing the same thing more than once – break input into “equivalence classes” </li></ul></ul>