1. Lecture-09
DEVELOPMENT, TESTING and DEBUGGING
4FTC2064 Introduction to Programming and Discrete Structures
Presented by:
Dr. Mohamed Tahoun
(m.tahoun@gaf.edu.eg)
Lecture notes:
Dr. Steve Hunt
Dept of Computer Science
University of Hertfordshire
CS (AI) – Level-4
2022/2023
2. Correctness testing vs debugging
• Correctness testing: detecting errors
• Run a program – or part of a program – and compare the results it
actually gives with the results it is expected to give
• Provide data for which we know what the result should be
• If the actual result does not match the expected result the program is
incorrect, so it needs to be debugged
• Debugging: fixing errors
• Cannot debug until we have found one or more errors
• Use diagnostic techniques to find the programming mistakes that cause
the errors we have found
• Correct programming mistakes to remove errors
3. Broad categories of error
• Syntax errors
• What the programmer has written is not a legal Python program, so
Python cannot make sense of it
• Run-time errors
• The running program has told Python to do something impossible (e.g.
adding an int to a str, or indexing beyond the end of a list)
• These are the ones that give red error messages in IDLE
• Semantic errors
• Programmer has asked Python to do the wrong thing (something that
makes sense, but that will give an incorrect result)
• Sometimes caused by a misunderstanding of what is required,
sometimes by a misunderstanding of how to achieve it
4. Can we find the errors in this function?
= len(sl)
def fs (sl) :
l
ss = sl[0]
for i in range (l) :
cs = sl[i]
if len(cs) :
len(ss) <
cs = ss
cs
return
• No way to tell what it’s supposed to do, though we might guess
that sl, ss and cs are either strings or lists (how??)
5. Could we work out what the function does?
• We could use our knowledge of the Python language
• We could conduct some experiments
• Compile the code
• Attempt to run it with different data items passed as parameters
• We may be able to get an idea of what the function actually
does, but this doesn’t tell us what it was supposed to do
6. How about now?
# Finds
def fs (sl) :
the shortest string in a list of strings
l = len(sl)
ss = sl[0]
for i in range (l) :
cs = sl[i]
if len(cs) :
len(ss) <
cs = ss
cs
return
• At least there’s a clue to guide our experiments
7. We still cannot test this effectively
• Why not?
• Because the function definition contains a single-line description that is
too ambiguous to tell whether or not it's correct :
# Finds the shortest string in a list of strings
• This may seem clear
• We can devise some of lists of strings and apply the function to each of
them in turn
• We can work out what the function does by examining the return values
• We can determine whether this matches our ideas about "finding the
shortest string"
• But this doesn't tell us what the function was supposed to do
8. Every function needs a specification
• A function's specification tells us – in precise terms – what the
function is supposed to do
• The specification is required to tell us what results the function should
give, but not how it should work them out
• A correct function is one that gives correct results in all cases
• A precise specification is essential
• A specification should state
• What data items the function will need to be given as parameters (how
many, and what kinds of item)
• Any restrictions on those data items (pre-conditions)
• The precise relationship between the data items the function is given and the
results it should return (post-conditions)
9. What did the specification say?
• Write a function called findShortest that
• takes a list of strings as its parameter
• the list must contain at least one string
• finds the shortest string in the list
• returns the index of the shortest string
• Remember: a specification doesn’t tell us how the function is
supposed to work, but it does tell us what it is supposed to do
10. Re-written using pre- and post-conditions
• function findShortest
• Parameters:
• sl, a list of character strings
• Result returned:
• i, an integer
• Pre-condition:
• len(sl) >= 1
• Post-condition:
• i is the index of the shortest character string in the list sl
11. Correctness tests
• We can devise correctness tests now that we know what the
code is supposed to do
• A correctness test for a function has two components
1. Test data (a set of values for all of its parameters)
2. Expected result (the value that should be returned, given those
parameter values)
• To run a test we apply the function to the test data and
examine the return value
• If it matches the expected result the test is passed
• If it does not match the test is failed
12. The limits of testing
• It is not possible to test a function completely, because there
are too many different data items that might be passed as
parameters
• So we cannot prove that a program is correct by testing it, even if it
passes all of the tests we throw at it
• If a program fails a test we know one of two things
1. Either the program contains an error
2. Or the test contains an error
• But we do not know what caused the problem, or how to fix it.
Finding and fixing errors is called debugging
13. Correctness tests for our function?
• Use the specification to help devise test cases
• Case 1: argument passed to sl is a list that contains only one string
• Expected result: 0
• How many different examples of Case 1 should we test?
• Case 2: argument passed to sl is a list that contains several strings
• Expected result: the index of the shortest string
• How many different examples of Case 2 should we test?
14. Some tests for Case 1
• fs ( ["fred"] ) Expected result: 0 Actual result: ?
• fs ( ["steve"] ) Expected result: 0 Actual result: ?
• We can automate testing by including the tests in the script that contains
the function definition
list1 = ["fred"]
expected1 = 0
print ("Argument:",list1,"Expected:",expected1,"Actual:",fs(list1))
list2 = ["steve"]
expected2 = 0
print ("Argument:",list2,"Expected:",expected2,"Actual:",fs(list2))
16. The function doesn't meet the spec
• We could tell very quickly that this function didn't meet its
specification
• It returns a string rather than the index of a string
• The string returned is sometimes – but not always – the shortest string
• But how do we fix it?
• We need to work through the code to see what has gone
wrong
• Need to develop an understanding of what is happening, and why it is the
wrong thing
• Use this to work out – and fix – the causes of any errors
• We need more tests, based on the code
17. Does this help?
# Finds the shortest string in
def findShortest (stringList) :
a list of strings
length = len(stringList)
shortest = stringList[0]
for index in range(length) :
current = stringList[index]
if len(shortest) < len(current) :
current = shortest
return current
• All I’ve done is replace one set of names with another
18. Yes: it helps
• Just by using sensible names for things we have made it
clearer what is going on – and it's obvious that the writer hasn't
implemented the spec we have
• This is a really important lesson
• Poor choice of variable names can make it much harder to understand a
program
• If it's hard to understand what's going on it's hard to devise tests, and it's
even harder to debug the code
19. Black Box vs White Box testing
• Correctness tests that are based only on what the program is
supposed to do are called black box tests
• we behave as if the program is a box into which we cannot see
• we can check the relationship between what goes in and what comes out,
but we have no way of checking how the result was obtained
• Correctness tests should also take some account of how the
program works. Such tests are called white box tests or glass
box tests
• In white box testing we look inside the code to see if there are any design
choices that are likely to lead to errors cropping up
• Sometimes we will spot the errors before we even write the tests!
20. Some common sources of coding errors
• Loops
• Misunderstanding how many times a loop body should be executed
• Misunderstanding what a loop should produce
• Wrong initialization (starting in the wrong place)
• Wrong test used in loop (continue / end under wrong circumstances)
• Wrong code in loop body (doing the wrong thing each time)
• Selection statements
• Misunderstanding the nature of the selection / what the alternatives are
• Wrong test used
• Wrong ordering of clauses (if .. elif .. else)
• Wrong code in one or more alternatives
21. Devise additional (white box) test cases
• The function uses a loop to work through the list, so we need
tests that will expose at least the following errors if they exist
1. Starting after the beginning of the list
2. Finishing before the end of the list
3. Attempting to index off the end of the list
4. Doing the wrong thing inside the loop body
• It also uses an if statement, so we need tests that will expose
at least the following errors if they exist
1. Using the wrong truth-valued condition in the 'if'
2. Doing the wrong thing when the condition is true
22. Correctness tests for our function?
• Case 1
• Input: A list that contains only one string
• Expected result: 0
• How many different examples should we test?
23. Correctness tests for our function?
• Case 2
• Input: A list that contains several strings, all of different lengths
• Expected result: the index of the shortest string
• How many different examples should we test?
• Different lengths of list
• Different positions for the shortest string (beginning, middle, end, of list)
24. Correctness tests for our function?
• Case 3:
• Input: A list that contains several strings, two or more of which are of the
same length
• Expected result: the index of the shortest string
• How many different examples should we test?
• Different lengths of list
• All strings of the same length (do we care what index we get back?)
• Lists with one shortest string, several longer strings, two or more of which are
of the same length
• Lists with two or more shortest strings (do we care what index we get back?)
25. • We looked at a function that was poorly defined and poorly
written
# Finds
def fs (sl) :
the shortest string in a list of strings
l = len(sl)
ss = sl[0]
for i in range (l) :
cs = sl[i]
if len(cs) :
len(ss) <
cs = ss
cs
return
26. • We found that using meaningful names helped with our
understanding of the code
# Finds the shortest string in
def findShortest (stringList) :
a list of strings
length = len(stringList)
shortest = stringList[0]
for index in range(length) :
current = stringList[index]
if len(shortest) < len(current) :
current = shortest
return current
27. • We found that the specification could be used to help
derive test cases:
• Write a function called findShortest that
• takes a list of strings as its parameter
• the list must contain at least one string
• finds the shortest string in the list
• returns the index of the shortest string
28. • We saw that it can be hard to determine what has gone wrong,
even with proper testing
• The results of the tests are a long way from what we expected,
so when attempting to correct the code it is difficult to know
where to start
• In fact once we have the results of the first few tests it may
even hard to decide how to go about further testing
29. Suggestion
• We need to use diagnostic methods that help us to understand
how the function is getting its result
• The simplest of these is to insert print statements to help us
track the execution of the function
30. Tracing with print statements ....
• If we insert print statements at strategic places we can
determine what is going on, and this should help us to identify
the mistakes that are causing errors (bugs)
# Finds the shortest string in
def findShortest (stringList) :
a list of strings
length = len(stringList)
shortest = stringList[0]
for index in range(length) :
current = stringList[index]
if len(shortest) < len(current) :
current = shortest
return current
31. Tracing with print statements ....
# Finds the shortest string in
def findShortest (stringList) :
a list of strings
length = len(stringList)
shortest = stringList[0]
print ("Shortest so far is",shortest)
for index in range(length) :
current = stringList[index]
print ("Current string is",current)
if len(shortest) < len(current) :
is:",shortest)
print ("New shortest string
current = shortest
current
return
32. Tracing with print statements ....
• But for this to be effective we need to know what we should be
seeing at each stage
• We need to understand how the programmer intended to solve
the problem in order to understand what's gone wrong
• What algorithm was the programmer trying to implement?
• Was that algorithm correct?
• What does it say should happen at each stage?
• Does the implementation follow the algorithm?
33. We need to understand what was intended
1. We need a specification of what the function should do
2. We need an algorithm that says how the result should be
calculated
3. We need an implementation of the algorithm in Python
(ideally commented to explain what's being done and why)
Many experienced programmers express their algorithms in their
target programming language (combining 2 and 3). This is ok, as
long as you know the target language well enough to be certain
of what you are doing
34. Every function needs an algorithm
• The algorithm tells us the sequence of steps that will be
performed in order to work out the results
• An algorithm is required to tell us how the function should work
out its results, but not what the correct results should be!
• A correct algorithm is one that provides the results set out in
the specification
• There will typically be several algorithms that give the correct results for
any specification
• A faithful implementation of a correct algorithm should result in a correct
function
35. Every function needs an implementation
• The implementation is an expression of an algorithm in a
programming language (in our case the language is Python)
• This takes the form of program code
• A correct function is a faithful implementation of a correct
algorithm
• There will typically be several implementations of any algorithm
• We can examine the code to determine whether it implements
the algorithm, BUT
• the implementation needs to be tested against the specification
36. Specification of findShortest
• function findShortest
• Parameters:
• stringList, a list of character strings
• Result returned:
• indexofshortest, an integer
• Pre-condition:
• len(stringList) >= 1
• Post-condition:
• indexofshortest is the index of the shortest character string in the list
stringList
37. Developing an algorithm for findShortest
• function findShortest
• Parameter: stringList
• Work through stringList one string at a time to find the shortest, and
return its index
• It's easy to go through the list one string at a time, but how do we find the
shortest? And how do we get its index? Need more detail
• Use a loop to work through the list
• Work through the indexes one at a time
• Use indexes to obtain individual strings
• Compare each string against the others to see if it's the shortest
• Return the index of the shortest one we find
38. A more detailed algorithm
• function findShortest
• Parameter: stringList
• Find the length of stringList
• For each index in the sequence 0 to length-1
• Get the string at this index position
• If it is shorter than the shortest string found so far, record its index
• Return the index of the shortest string found so far
• Still not enough detail
• How do we keep track of the shortest string found so far?
How do we record the index of the shortest string found so far?
39. More detailed still
• function findShortest
• Parameter: stringList
• Set the index of the shortest string found so far to 0
• Set the shortest string so far to stringList [0]
• Find the length of stringList
• For each index in the sequence 1 to length-1
• Get the string at this index position (current string)
• If the current string is shorter than the shortest string found so far
• Set the shortest string so far to the current string
• Set the index of the shortest string found so far to the index of the
current string
• Return the index of the shortest string found so far
40. Correctness tests for our function?
• We can devise a set of correctness tests based on the
specification and the algorithm
• Ideally these should be tests that will help us identify the causes
of errors
• Let's give it a go
41. Correctness tests for our function?
• Case 1
• Input: A list that contains only one string
• Expected result: 0
• How many different examples should we test?
• One or two should be enough
list1 = ["fred"]
list2 = ["bob"]
expected result = 0
expected result = 0
42. Correctness tests for our function?
• Case 2
• Input: A list that contains several strings, all of different lengths
• Expected result: the index of the shortest string
• How many different examples should we test?
• Different lengths of list
• Different positions for the shortest string (beginning, middle, end, of list)
list3 = ["fred","bob"]
list4 = ["bob","fred"]
list5 = ["fred","bob","peter"]
list6 = ["bob","peter","fred"]
list7 = ["peter","fred","bob"]
expected result = 1
expected result = 0
expected result = 1
expected result = 0
expected result = 2
43. Correctness tests for our function?
• Case 3:
• Input:A list that contains several strings, two or more of which are of the
same length
• Expected result: the index of the shortest string
• How many different examples should we test?
• Different lengths of list
• All strings of the same length (do we care what index we get back?)
• Lists with one shortest string, several longer strings, two or more of which are of the
same length
• Lists with two or more shortest strings (do we care what index we get back?)
list8 = ["bill","roy","ray"]
list9 = ["bill","fred","peter"]
list10 = ["bill","bob","fred","peter"]
list11 = ["bill","peter","bob","roy","ray"]
expected result = 1
expected result = 0
expected result = 1
expected result = 2
44. Implementing the function
• We have
• A specification
• An algorithm
• A test plan
• Now we can implement the function in Python
• A little at a time
• Test and debug as we go
45. Implementing this algorithm
• Baby steps first
def findShortest (stringList) :
indexofshortest = 0
= stringList[0]
length of stringList
index in the range 1 to length-1
shortest
# Find the
# For each
#
#
#
#
Get the string
If the current
Set shortest
Set index of
at this index position (current string)
string is shorter than the shortest so far
to the current string
shortest to index of current string
return indexofshortest
# Correct return value is 0, no matter what stringList is
46. Implementing this algorithm
• Oops! Correct the syntax and we get
def findShortest (stringList) :
indexofshortest = 0
= stringList[0]
length of stringList
index in the range 1 to length-1
shortest
# Find the
# For each
#
#
#
#
Get the string
If the current
Set shortest
Set index of
at this index position (current string)
string is shorter than the shortest so far
to the current string
shortest to index of current string
return indexofshortest
# Correct return value is 0, no matter what stringList is
47. Implementing this algorithm
• Add a print statement for tracing purposes
def findShortest (stringList) :
indexofshortest = 0
shortest = stringList[0]
print ("shortest so far is",shortest)
# Find the length of stringList
# For each index in the range 1 to length-1
#
#
#
#
Get the string
If the current
Set shortest
Set index of
at this index position (current string)
string is shorter than the shortest so far
to the current string
shortest to index of current string
return indexofshortest
# Print statement should show the first string in the list
# Correct return value is 0, no matter what stringList is
48. Implementing this algorithm
• If it works correctly, add a little more
def findShortest (stringList) :
indexofshortest = 0
shortest = stringList[0]
length = len (stringList)
# For each index in the range 1 to length-1
#
#
#
#
Get the string
If the current
Set shortest
Set index of
at this index position (current string)
string is shorter than the shortest so far
to the current string
shortest to index of current string
return indexofshortest
# Correct return value is still 0, no matter what stringList is
49. Implementing this algorithm
• And another print statement
def findShortest (stringList) :
indexofshortest = 0
shortest = stringList[0]
length = len (stringList)
print ("length of the list is",length)
# For each index in the range 1 to length-1
#
#
#
#
Get the string
If the current
Set shortest
Set index of
at this index position (current string)
string is shorter than the shortest so far
to the current string
shortest to index of current string
return indexofshortest
# Print statement should show the length of the list
# Correct return value is still 0, no matter what stringList is
50. Implementing this algorithm
• Now add the loop with an empty body
def findShortest (stringList) :
indexofshortest = 0
shortest = stringList[0]
length = len (stringList)
in range (1,length-1)
# Added because a lop body cannot be empty
for index
pass
Get the
#
#
#
#
string
If the current
Set shortest
Set index of
at this index position (current string)
string is shorter than the shortest so far
to the current string
shortest to index of current string
return indexofshortest
# Correct output is still 0, no matter what stringList is
51. Implementing this algorithm
• Add a print statement to print out indexes one at a time
def findShortest (stringList) :
indexofshortest = 0
shortest = stringList[0]
length = len (stringList)
in range (1,length-1)
("current index is",index)
for index
print
Get the
#
#
#
#
string at this index position
string is shorter than
to the current string
(current string)
the shortest so far
If the current
Set shortest
Set index of shortest to index of current string
return indexofshortest
# Print should show the index of each string in turn
# Correct output is still 0, no matter what stringList is
52. Implementing this algorithm
• Correct the code, and test again
def findShortest (stringList) :
indexofshortest = 0
shortest = stringList[0]
length = len (stringList)
in range (1,length)
("current index is",index)
for index
print
Get the
#
#
#
#
string at this index position
string is shorter than
to the current string
(current string)
the shortest so far
If the current
Set shortest
Set index of shortest to index of current string
return indexofshortest
# Print should show the index of each string in turn
# Correct output is still 0, no matter what stringList is
53. Implementing this algorithm
• Print out the strings as you go through the list
def findShortest (stringList) :
indexofshortest = 0
shortest = stringList[0]
length = len (stringList)
for index in range (1,length)
current = stringList [index]
print ("current string is",current)
# If the current string is shorter than the shortest so far
# Set shortest to the current string
# Set index of shortest to index of current string
return indexofshortest
# Print should show each string in turn
# Correct output is still 0, no matter what stringList is
54. Implementing this algorithm
• Now add the if statement
def findShortest (stringList) :
indexofshortest = 0
shortest = stringList[0]
length = len (stringList)
for index in range (1,length)
current = stringList [index]
if len(current) < len(shortest)
shortest = current
indexofshortest = index
return indexofshortest
# Correct output is different for different stringLists
55. Implementing this algorithm
• Add sufficient print statements to trace what's happening
def findShortest (stringList) :
indexofshortest = 0
shortest = stringList[0]
length = len (stringList)
for index in range (1,length)
current = stringList [index]
print ("current string is",current)
if len(current) < len(shortest)
print ("this is shorter than",shortest)
shortest = current
indexofshortest = index
return indexofshortest
# Correct output is different for different stringLists
56. Automate the tests
• Tests may be automated by adding a series of Python statements
that will be executed when the module is run
def findShortest (stringList) :
indexofshortest = 0
shortest = stringList[0]
length = len (stringList)
for index in range (1,length)
current = stringList [index]
print ("current string is",current)
if len(current) < len(shortest)
print ("this is shorter than",shortest)
shortest = current
indexofshortest = index
return indexofshortest
# Correct output is different for different stringLists
