1.
Computing Science 1P Lecture 21: Wednesday 18 th April Simon Gay Department of Computing Science University of Glasgow 2006/07
2.
What's coming up? Wed 18 th April (today): lecture Fri 20 th April: lecture Mon 23 rd April, 12.00-14.00: FPP demo session; PRIZES! Mon 23 rd – Wed 25 th April: labs: exam preparation / lab exam preparation Wed 25 th April: lecture / tutorial Friday 27 th April: lecture: revision / questions with Peter Saffrey Mon 30 th April – Wed 2 nd May: Lab Exam Wed 2 nd May: No lecture Fri 4 th May: No lecture THE END
3.
Lab Exam 2 As you know, there will be a second lab exam, during the week beginning 30 th April. It is worth 10% of the module mark. The question has been available since Monday afternoon. In the first lab exam, although there were many very good submissions, a worrying number of submissions showed very little evidence of advance preparation. You can prepare for the lab exam in any way you want to , including asking a friend to explain the solution to you. On the day, of course, you are on your own. Trying to solve the problem from scratch in the exam is just making life difficult for yourself.
4.
Algorithms We have looked at algorithms for sorting ; we saw that choosing a better algorithm can have a dramatic effect on the efficiency of a program. Now let's consider searching , another basic computing task. The general problem: find a desired item of data in a collection. Example: in a collection of (name,address) pairs, find a particular person's address.
5.
Searching in unstructured data Imagine that we have a list of (key,value) pairs and we do not know anything about the order. We can easily define: def find(key,data): for i in data: if i[0] == key: return i[1] raise "KeyNotFound",key
6.
Searching in unstructured data What can we say about the time taken by find ? Just as for sorting, the relevant measure is the number of comparisons. Clearly it is possible that the key we are looking for is at the end of the list. In that case we have to compare the given key with every key in the list. If we imagine testing find repeatedly with a large number of random lists, on average it will have to search half way along the list. When analysing algorithms, sometimes we talk about the average case and sometimes the worst case . In this situation they are both the same: order n , where n is the length of the list.
7.
Searching in unstructured data It's obvious that we can't do better than order n for searching in an unstructured list, because we can't avoid the possibility that the desired key is at the end. Remarkably, there is an algorithm for quantum computers which only takes square root of n operations to search in an unstructured list. However, quantum computers of a useful size have not yet been built. To find out more, look up Grover's algorithm . But let's stick to conventional algorithms…
8.
More efficient search If we can't improve the algorithm for search in an unstructured list, the only alternative is to change the data structure: don't use an unstructured list! The first idea is quite simple: use an ordered list instead. In other words, put the data in the list in such a way that the keys are in order. Often this means alphabetical order or numerical order, but other more complex orders could be defined. Example: in a dictionary, the words are in alphabetical order, and we can take advantage of this to find words quickly.
10.
Binary search 0 1 2 3 4 5 6 7 8 9 10 11 Search for the key: cat It could be anywhere in the list. android badger cat door ending fireman garage handle iguana jumper kestrel lemon
11.
Binary search 0 1 2 3 4 5 6 7 8 9 10 11 Search for the key: cat It could be anywhere in the list. The list has length 12. Divide it by 2 and look at position 6. android badger cat door ending fireman garage handle iguana jumper kestrel lemon
12.
Binary search 0 1 2 3 4 5 6 7 8 9 10 11 Search for the key: cat It could be anywhere in the list. The list has length 12. Divide it by 2 and look at position 6. cat < garage android badger cat door ending fireman garage handle iguana jumper kestrel lemon
13.
Binary search 0 1 2 3 4 5 6 7 8 9 10 11 Search for the key: cat Because the list is ordered, we now know that cat must be before garage , i.e. it is in the first half of the list. android badger cat door ending fireman garage handle iguana jumper kestrel lemon
14.
Binary search 0 1 2 3 4 5 6 7 8 9 10 11 Search for the key: cat Because the list is ordered, we now know that cat must be before garage , i.e. it is in the first half of the list. android badger cat door ending fireman garage handle iguana jumper kestrel lemon
15.
Binary search 0 1 2 3 4 5 6 7 8 9 10 11 Search for the key: cat Now repeat, searching in a list of length 6. android badger cat door ending fireman garage handle iguana jumper kestrel lemon
16.
Binary search 0 1 2 3 4 5 6 7 8 9 10 11 Search for the key: cat Now repeat, searching in a list of length 6. Divide by 2 and look at position 3. android badger cat door ending fireman garage handle iguana jumper kestrel lemon
17.
Binary search 0 1 2 3 4 5 6 7 8 9 10 11 Search for the key: cat Now repeat, searching in a list of length 6. Divide by 2 and look at position 3. cat < door android badger cat door ending fireman garage handle iguana jumper kestrel lemon
18.
Binary search 0 1 2 3 4 5 6 7 8 9 10 11 Search for the key: cat We now know that cat must be before door . android badger cat door ending fireman garage handle iguana jumper kestrel lemon
19.
Binary search 0 1 2 3 4 5 6 7 8 9 10 11 Search for the key: cat We now know that cat must be before door . android badger cat door ending fireman garage handle iguana jumper kestrel lemon
20.
Binary search 0 1 2 3 4 5 6 7 8 9 10 11 Search for the key: cat Now repeat, searching in a list of length 3. android badger cat door ending fireman garage handle iguana jumper kestrel lemon
21.
Binary search 0 1 2 3 4 5 6 7 8 9 10 11 Search for the key: cat Now repeat, searching in a list of length 3. Divide by 2 and look at position 1. android badger cat door ending fireman garage handle iguana jumper kestrel lemon
22.
Binary search 0 1 2 3 4 5 6 7 8 9 10 11 Search for the key: cat Now repeat, searching in a list of length 3. Divide by 2 and look at position 1. cat > badger android badger cat door ending fireman garage handle iguana jumper kestrel lemon
23.
Binary search 0 1 2 3 4 5 6 7 8 9 10 11 Search for the key: cat We now know that cat must be after badger . android badger cat door ending fireman garage handle iguana jumper kestrel lemon
24.
Binary search 0 1 2 3 4 5 6 7 8 9 10 11 Search for the key: cat We now know that cat must be after badger . We have narrowed down the possible position of cat to just one place. And in fact cat is there, so we have found it. android badger cat door ending fireman garage handle iguana jumper kestrel lemon
25.
Binary search 0 1 2 3 4 5 6 7 8 9 10 11 Search for the key: cat We now know that cat must be after badger . We have narrowed down the possible position of cat to just one place. And in fact cat is there, so we have found it. If a different word is there, then cat is not in the list. android badger cat door ending fireman garage handle iguana jumper kestrel lemon
26.
Binary search The idea of binary search is very simple, but implementing it correctly requires care: there are many possibilities for "off by one" errors. Searching in a dictionary is often used as an example of binary search, but we don't really use dictionaries in exactly this way. Usually we flick through the pages quickly to find the right letter, then do something similar to binary search. A typical dictionary has extra structure to support this process (e.g. words in the page headers; thumbholes for indexing).
27.
Another example 0 1 2 3 4 5 6 7 8 9 10 11 Search for the key: handle android badger cat door ending fireman garage handle iguana jumper kestrel lemon
28.
Another example 0 1 2 3 4 5 6 7 8 9 10 11 Search for the key: handle android badger cat door ending fireman garage handle iguana jumper kestrel lemon
29.
Another example 0 1 2 3 4 5 6 7 8 9 10 11 Search for the key: handle android badger cat door ending fireman garage handle iguana jumper kestrel lemon
30.
Another example 0 1 2 3 4 5 6 7 8 9 10 11 Search for the key: handle android badger cat door ending fireman garage handle iguana jumper kestrel lemon
31.
Another example 0 1 2 3 4 5 6 7 8 9 10 11 Search for the key: handle android badger cat door ending fireman garage handle iguana jumper kestrel lemon
32.
Another example 0 1 2 3 4 5 6 7 8 9 10 11 Search for the key: handle We could stop here, but if we follow the algorithm strictly, we continue dividing the region in two android badger cat door ending fireman garage handle iguana jumper kestrel lemon
33.
Another example 0 1 2 3 4 5 6 7 8 9 10 11 Search for the key: handle android badger cat door ending fireman garage handle iguana jumper kestrel lemon
34.
Another example 0 1 2 3 4 5 6 7 8 9 10 11 Search for the key: handle android badger cat door ending fireman garage handle iguana jumper kestrel lemon
35.
Another example 0 1 2 3 4 5 6 7 8 9 10 11 Search for the key: handle Now we can certainly stop android badger cat door ending fireman garage handle iguana jumper kestrel lemon
36.
Analysing binary search Remember that we are interested in the number of comparisons. Suppose that we are searching in a list of length n . We compare the middle item with the search key. The result might tell us we have found the key, but in general it narrows down the region of the list in which we are searching. The possible region of the list is now half the size it was.
37.
Analysing binary search We keep halving the size of the region, until we narrow it down to a single position in which the key should be found. How many times do we have to halve the size? n = 16: 8, 4, 2, 1 4 comparisons n = 64: 32, 16, 8, 4, 2, 1 6 comparisons It is the logarithm of n to base 2 , i.e. the power of 2 which gives n. Binary search is an order log n algorithm.
38.
Analysing binary search We can compare the efficiency of an order n algorithm with that of an order log n algorithm: 10 sec 10 000 000 21 microsec 21 10 000 000 1 sec 1 000 000 18 microsec 18 1 000 000 100 millisec 100 000 15 microsec 15 100 000 10 millisec 10 000 12 microsec 12 10 000 1 millisec 1 000 9 microsec 9 1 000 100 6 100 10 3 10 time n time log n n
41.
Termination When writing programs with loops, we have to be sure that they terminate , i.e. eventually stop. In almost all of our previous programs, it has been obvious that loops terminate. In a for loop, the number of iterations is known before we start, e.g. for x in range(10) In a while loop, the condition can be anything, but we have always used a simple structure: i = 0 while i < 10: # code inside the loop, not changing i i = i + 1
42.
Termination Binary search uses a while loop with a more complex structure: length = upper - lower + 1 while length > 1 : midpoint = lower + length/2 if key < data[midpoint]: upper = midpoint - 1 else: lower = midpoint length = upper - lower + 1 Let's prove that eventually length <= 1 , so that the loop terminates.
43.
Proving termination Consider one iteration of the loop. At the beginning we have values lower , upper and length . At the end of the body of the loop we have new values lower', upper' and length'. We will prove that length' < length . Therefore as we go round the loop repeatedly, length gets smaller and smaller. It is always an integer value, so eventually it must reach 1 or less.
44.
Proving termination At the top of the loop we have values lower , upper. We have length = upper – lower + 1 Assume that length > 1 , so that we go into the loop. At the bottom of the loop we have new values lower' , upper' and we have a new value length' = upper' – lower' + 1 we also have midpoint = lower + length/2 Now we consider the possible ways of calculating lower', upper'
45.
Proving termination Case 1: we take the first branch of the if statement. upper' = midpoint – 1 lower' = lower length' = upper' – lower' + 1 = midpoint – 1 – lower + 1 = midpoint – lower = lower + length/2 – lower = length/2 < length because length > 1
46.
Proving termination Case 2: we take the second branch of the if statement. upper' = upper lower' = midpoint length' = upper' – lower' + 1 = upper – midpoint + 1 = upper – (lower + length/2) + 1 = upper – lower – length/2 + 1 = length – length/2 < length because length > 1
47.
Proving termination We have proved that whichever path we take through the body of the while loop, length decreases. Therefore the loop must terminate. With further calculation of a similar kind ( exercise !) we can prove that when the loop terminates, length = 1 (not < 1 ), meaning that we really have identified one location in the list where the key should be found if it is present at all.
48.
Refining binary search It might turn out that when we look at the midpoint of the list, the key we want happens to be there. We might as well take advantage of that case: while length > 1: midpoint = lower + length/2 if key < data[midpoint]: upper = midpoint - 1 elif key > data[midpoint]: lower = midpoint else: return midpoint but notice that we have introduced an extra comparison. What can we do about this?
49.
The function cmp The problem is that when we compare two values, there are three possible results: equal, first one smaller, second one smaller The comparisons < <= > >= == return a boolean result, so they only tell us one of two possible results. To solve this problem, Python provides the function cmp . cmp(a,b) returns 0 if a == b -1 if a < b 1 if a > b
50.
Using cmp while length > 1: midpoint = lower + length/2 r = cmp(key,data[midpoint]) if r == -1 : upper = midpoint - 1 elif r == 1 : lower = midpoint else: return midpoint
A particular slide catching your eye?
Clipping is a handy way to collect important slides you want to go back to later.
Be the first to comment