• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Why you should use a testing framework
 

Why you should use a testing framework

on

  • 1,894 views

The case for using a testing framework to make your software robust, with a simple example in MATLAB.

The case for using a testing framework to make your software robust, with a simple example in MATLAB.

Statistics

Views

Total Views
1,894
Views on SlideShare
1,894
Embed Views
0

Actions

Likes
0
Downloads
10
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as OpenOffice

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • In this short presentation, I'm going to try and persuade you that you should be using a testing framework for any software that you write. The example we'll look at uses MATLAB, but the argument is applicable to any software development.
  • Hopefully you'll be looking at this problem and thinking “this is easy, I learnt Pythagoras's theorem when I was in school”. That's good. In that case our first attempt at solving the problem will look familiar to you.
  • As Pythagoras said, “the square of the longest side is equal to the sum of the squares of the other two sides”, or whatever the Greek equivalent is. This looks like we might have cracked it straight away. Let's test it and find out.
  • First we test the function under normal conditions. As every school-kid knows, for a right-angled triangle with sides of length 3 and 4, the hypotenuse squared is 3 squared plus 4 squared, which is 25 and so the hypotenuse is of length 5. Likewise, the hypotenuse of the unit square is root 2. So far, so good.
  • What if we don't pass the function any arguments? This test shows that an error is thrown when no arguments are passed to the function. You may decide that that is the most appropriate behaviour. On this occasion, let's decide that we want default values.
  • Here's our second attempt. The additional code sets a default value of 1 when no input was passed for that length.
  • Now that we've changed the function, we need to check that our new code passes the test. Happily, in this case it does.
  • Another test is to see what happens when you pass empty inputs in to the function. Empty inputs should probably work in the same way as no inputs, so we have a problem here.
  • Here's our third attempt. We need to test that this version works with empty inputs but we also need to check that we haven't introduced new bugs. Running your old tests to make sure that there are no new bugs is called regression testing.
  • Here is our set of tests again. As you can imagine, typing in all these tests every time you change something is already really time consuming, and we're not done yet.
  • If you've done any numeric programming before, you'll have realised that bad things often happen when you introduce very big or very small inputs. In this case, squaring the numbers causes them to over or underflow.
  • This fourth attempt is the version that most modern scientific software uses. Explaining how it works is beyond the scope of this presentation. The important thing is that we've changed the function, so we need to run all the tests again.
  • I'm now really bored of typing in tests, and yet we still haven't exhausted the test possibilities, e.g., What happens when non-numeric arguments are passed? What happens if non-scalar arguments are passed? What happens in the 3D (or higher dimension) case?
  • So we've established two things. That we need to write lots of tests, and that we need to run each test lots of times. In order to save our sanity, we need a way of easily creating and running and maintaining these tests. Otherwise, laziness will take over and we'll just not bother.
  • We could write our own function to run the tests. This is much better, but notice that there's a lot of effort involved in displaying the results, when we really just want to think about the tests themselves.
  • A testing framework is just a formalised version of the contents of the previous slide. They generally consist of: Lots of short functions or methods representing individual tests
  • A suite function that runs some or all of the tests.
  • And some utility functions for handling errors and displaying results.
  • Here's an example using the MATLAB xUnit framework. To use it, first, we create some test functions, and place them in the same directory. Here you can see the five tests we thought of earlier.
  • We just need two more lines of code. The first line creates a suite of tests from each function, and the second line runs them all, telling us that we passed them.
  • If we run the tests on our original version of the function instead, then the test suite shows us where our problems lie.
  • To summarise: Firstly, thorough testing is necessary, even when you think the answer should be easy. In this case, the overflow bug was quite obscure, and you might not have picked up on it without testing.
  • Secondly, if we use a testing framework, then thorough testing needn't be onerous. Each test was a one line function, creating and running the suite were one line each.
  • These two ideas taken together imply that testing frameworks are a great idea.
  • If you want to find out more, then check out these resources. Thanks for listening.

Why you should use a testing framework Why you should use a testing framework Presentation Transcript

    • Why you should use a testing framework
      by Richard Cotton
  • Our goal: write a function to calculate the length of the hypotenuse of a right-angled triangle from the other two sides
  •  
  •  
  •  
    • ( nargin returns the number of arguments passed into the function)
  •  
    • In MATLAB, [] represents an empty matrix.
  •  
  •  
    • Uh-oh. The answers should be 1.414e300 and 1.414e-300 respectively.
  •  
  •  
    • The story so far
    • We need to do lots of tests.
    • We need to do each test lots of times.
      This means that ...
    • It would be a good idea to have a suite of tests that can be easily created/run/maintained.
  •  
    • Lots of short functions/methods representing individual tests
    • Lots of short functions/methods representing individual tests
    • A suite function that runs some or all of the tests.
    • Lots of short functions/methods representing individual tests
    • A suite function that runs some or all of the tests.
    • Some utility functions for handling errors and displaying results.
  •  
  •  
  •  
    • Thorough testing is necessary.
    • Thorough testing is necessary.
    • If we use a testing framework, then thorough testing needn’t be onerous.
    • Thorough testing is necessary.
    • If we use a testing framework, then thorough testing needn’t be onerous.
      1 & 2 => Testing frameworks are a great idea.
    • Resources
      Lists of testing frameworks
      http://c2.com/cgi/wiki?TestingFramework http://en.wikipedia.org/wiki/List_of_unit_testing_frameworks
      Testing Strategies
      " Testing Computer Software " by Kaner et al, especially Ch. 3 http://en.wikipedia.org/wiki/Software_testing http://software-carpentry.org/4_0/test/