Intro to Python (High School) Unit #2

<High School>
Computer Science:
Introduction to Python
Unit #2
(Covering CPython 3)
Spring 2017
License: CC BY-SA 4.0 (except images), by Jay Coskey 1

Unit #2
Software Engineering
2

Software Engineering
3
• Docstrings
• Lambdas, map, and filter
• Classes & Python special functions
• Caching implemented using class-level data
• Logging with the package "logging"
• Unit testing with the package "unittest"
• Debugging

Docstrings
• Q: Where do the help strings associated with these functions come from?
• A: From docstrings! These are comments inside triple quotes. Recall this example:
def get_pi_estimate(sample_count):
"""Use a Monte Carlo technique to approximate the area of the unit circle"""
• Docstrings can be attached to individual functions, classes, or modules.
• As shown above, docstrings for functions appear just under the first line of the definition.
• The first line of the docstring becomes the short description of the function.
• There are conventions for what goes into a docstring, but we won't worry about that now.
ipython: For a short description of any of these:
>>> ? function_name
For a longer description:
>>> help(function_name)
License: CC BY-SA 4.0 (except images), by Jay Coskey
4

Lambdas (a.k.a. "function expressions") (1)
• We've been defining functions like this
def square(x):
return x**2 # After the colon is a block (often indented) of statements
• We can instead define "lambda functions" (or just "lambdas") like this:
square = lambda x: x**2 # No "def" needed. After the colon is an expression; no return needed.
square(5) # Value is 25
• Lambdas can also be functions of multiple arguments:
area = lambda x, y: x * y
area(5, 4) # Value is 20
• As with all functions, lambdas can be stored in variables or passed as function arguments:
def replace(f, items):
return [f(x) for x in items]
square = lambda x: x**2
items = [1,2,3,4,5]
print(replace(square, items)) # Output: [1, 4, 9, 16, 25]
5

Lambdas (a.k.a. "function expressions") (2)
• Lambdas are functions defined as expressions. They can be called "function expressions".
• They're also sometimes called "function literals".
• The term "lambda" comes from the word of Alonzo Church (1903-1995), who in the 1930s
used the Greek letter lambda (λ) to represent such functions in his writings on the lambda
calculus (within the field of mathematical logic), which in some ways is still used today.
• The Lisp programming language, which we mentioned on the first day of this course, was
basically a translation into software of Church's lambda calculus. So the lambda calculus
was basically the first written practical programming language.

map and filter
• The function map applies a function to each item in a list.
So map(f, items) acts like the list comprehension [f(x) for x in items]
For example, print(map(square, [1,2,3,4,5]))
prints out [1,4,9,16,25].
print(map(lambda s: s[::-1], ['Hello', 'world']))
prints out ['olleH', dlrow']
• The function filter results only those items in a list for which the filter function returns True.
So filter(p, items) acts like [x for x in items if p(x)]
For example, print(filter(lambda x: x % 2 == 0, [1,2,3,4,5]))
prints out [2,4].
print(filter(lambda s: len(s) > 3, ['Hello','to','the','world']))
prints out ['Hello', 'world']
p items

Software design: Components
• Suppose you're writing a program for a small game.
• It has game logic.
• It has player interaction.
• Probably some miscellaneous functionality.
• We can think of these as
• A Game Logic component,
• A Player Interaction component, and
• A Utility component.
• It's helpful to write the program in different sections,
where each section written corresponds to one of the
components being modeled.
9
Game
Game Logic
Player
Interaction
Utility

Software design: Classes
• Object-oriented ("OO") software development became popular in the 1980s with the
programming language C++ (*). In short, the OO approach:
• Breaks a large system down into smaller components
• Takes the bits of software that modeled each of those components
• Assembles those bits of software into a program dealing with the original system.
• In OO software, the bits of software used are called "objects".
• Objects generally consist of data and the functions that operate on that data.
• The definitions of those objects are called classes.
• The objects themselves are sometimes called class instances.
10
(*) But OO software goes back at least to 1965, when the Simula language first appeared.
Game System
Game Logic
Player
Interaction
Utility
Game Program
Game class Player class Util class
(a game that
people play)
(software that
people run)

Class example: BankAccount (use, not definition)
• Here is an example of how we might use a class called BankAccount.
• Note: We haven't defined the class yet. We'll do that soon.
if __name__ == '__main__':
ba = BankAccount(100.00)  This line creates and initializes the object.
ba.print_balance()  This line calls a function belonging to that object.
Output:
Balance=$100.00
11
Note: It looks like a BankAccountobject stores one
piece of information — the account balance,
which is printed by print_balance.
"Let's play the
BankAccount game!"
said no one ever.

class BankAccount:
def __init__(self, balance=0):  This initializes the object, with a default balance of zero.
self.balance = balance  Here "balance" is the argument, while "self.balance" belongs to the object.
def print_balance(self):
print('Balance=${0:.2f}'.format(self.balance))
def deposit(self, amount):
self.balance += amount
def withdraw(self, amount):
assert(amount <= self.balance) # Why is this line here?
self.balance -= amount
if __name__ == '__main__':
checking = BankAccount(100)  checking is an object of class BankAccount, or an instantiation of it.
checking.print_balance()  Does this remind you of string's format, lower, & upper functions?
savings = BankAccount()  savings is initialized without a balance, so it's set to the default of $0.
savings.deposit(250)
savings.print_balance()
Output:
Balance=$100.00
Balance=$250.00
Class example: BankAccount definition
12
Data
Functions
The term "self" refers to
the current object.

Class example: Greeter (use, not definition)
if __name__ == '__main__':
greeter = Greeter('Hello', 'world')  This line creates and initializes the object.
greeter.greet()  This line calls a function belonging to that
Output:
Hello, world!
13
Note: This shows how the Greeter class is used.
It looks like a Greeter object has two pieces of
information:
(1) the greeting, and (2) the thing being greeted.
We'll see shortly how its defined.

class Greeter:
def __init__(self, greeting='Hello', name ='world'):  This initializes the object with default args.
self.greeting = greeting  Here "greeting" is the argument, while "self.greeting" belongs to the object.
self.name = name  Same here: Value of the function argument is put into the object attribute
def greet(self):
print(self.greeting + ', ' + self.name + '!')
if __name__ == '__main__':
hw = Greeter('Hello', 'world')  hw is called a "Greeter object", or an instantiation of class Greeter.
sp = Greeter('Hola', 'amigos')
hw.greet()
sp.greet()
print('{0:s}, {1:s}!'.format(hw.greeting, sp.name))  You can access attributes directly
Output:
Hello, world! # hw.greet()
Hola, amigos! # sp.greet()
Hello, amigos! # print statement
Class example: Greeter (2)
14
Data
Function

Helpful(?) example: Biological taxonomies
• Consider this abridged taxonomy of the animal kingdom
15
DogCat Cow
Mammals
Animals
Deer Goat Pig
rover porkybessiescampers buck rambrandt
Object/Instance
attributes:
individual_name
height
weight
owner
Life
Class attributes:
species_name
is_domesticated
is_endangered
How are these taxons different than the ones below
(Cat, Cow, etc.)? No one says:
"I saw a mammal while driving today!"
These taxons represent categories rather than types
of individual animals. In other words, they aren't
ready to be instantiated in our minds as animals.

Class attributes vs. instance attributes (1)
• In HW #2, we saw that the contant data value pi (i.e., π) is math.pi.
• There are also oodles of math functions in the same module, such as:
• exp(x), sqrt(x), sin(x), cos(x), tan(x), log2(x), log10(x), factorial(x), gcd(a, b), etc.
• None of these require that you create an object of type math.
• Instead, these are attributes that belong to the class itself, rather than to
instances of the class. Let's call them class-level attributes, because they
belong to the class itself, not to individual instances of the class.
• The class Greeter that we defined earlier has these instance-level attributes:
• greeting and name (both data attributes of type string)
• greet (a function attribute)
16

Class attributes vs. instance attributes (2)
•Examples:
17
Attribute type Class-level
(i.e., one shared by all objects)
Instance-level
(i.e., one per object)
Data math.pi balance
greeting
name
Function math.cos print_balance
greet
(sometimes called methods)

Class example: Hello
(different than Greeting)
class Hello:
greeting = 'Hello'
def __init__(self, name='world'):
self.name = name
def greet(self):
print(Hello.greeting + ', ' + self.name + '!')
If __name__ == '__main__':
hw = Hello('world')  hw is a "Hello object", or an instantiation of class Hello.
sp = Hello('amigos')
hw.greet()
sp.greet()
print('Greeting={0:s}'.format(Hello.greeting))  The class-level greeting is Hello.
print('Spanish name={0:s}'.format(sp.name))  This particular instance-level name is "amigos".
Output:
Hello, world!
Hello, amigos!
Greeting=Hello
Spanish name=amigos
18
Instance
data
Class
data
• "greeting" belongs to the class itself, not each object.
• Everybody gets the same greeting.
• "name" is class instance data.
• Every object greets someone different.
function

Q: Can class data point to a specific object? A: Yes
class Human:
tallest = None # Equalling None is like not having a value
def get_tallest():
return Human.tallest
def __init__(self, name, height):
self.name = name
self.height = height
if Human.tallest == None or height > Human.tallest.height:
Human.tallest = self
def greet(self):
print('Hi, {0:s}'.format(self.name))
if __name__ == '__main__':
amy = Human('Amy', 5.5)
bob = Human('Bob', 6.0)
cindy = Human('Cindy', 7.5)
print('Tallest: {0:s}'.format(Human.get_tallest().name))
Output:
Tallest: Cindy
19

Class example: fibonacci revisited
Recall that we saw this recursive function fibonacci earlier. We can make it a function attribute of a class.
class Fibonacci:
def __init__(self):
pass
def fibonacci(self, n):
assert( type(n) == int and n >= 0 )
if n == 0 or n == 1:
return 1
else:
return fibonacci(n – 2) + fibonacci(n – 1)
if __name__ == '__main__':
f = Fibonacci( )
for n in range(0, 32):
print('Fibonacci #{0:d} = {1:d}'.format(n, f.fibonacci (n)))
Output:
Fibonacci #0 = 1
Fibonacci #1 = 1
Fibonacci #2 = 2
Fibonacci #3 = 3
etc.
20
No instance data, so nothing
to initialize, so __init__ isn't needed. This is an
"instance function".
That just mean that each object created
gets its own copy.
So what?
Is there any benefit to putting this function in a class?
Not yet.
But wait, there's more!....

Class example: Cached fibonacci function
We can save return values as we compute them, and reuse them. This saves on computation time, and speeds up our program.
class CFibonacci:
computed_values = dict()
def __init__(self):
pass
def fibonacci(self, n):
assert( type(n) == int and n >= 0 )
if n in CFibonacci.computed_values.keys():
return CFibonacci.computed_values[n]
elif n == 0 or n == 1:
return 1
else:
val = self.fibonacci(n – 2) + self.fibonacci(n – 1)
CFibonacci.computed_values[n] = val
return val
if __name__ == '__main__':
cf = CFibonacci()
for n in range(0, 5000):
print(cf.fibonacci(n))
Output:
Same as before, but much faster.
21
Class-level data
(no use of 'self')
Instance-level function
(self is used)
An instance-level
function can use
class data
This range is far more than the
previous version could have handled,
and it's done in a fraction of a second.

Python operators and special functions (a partial list)
(See Python 3 docs at https://docs.python.org/3.6/library/operator.html)
Operation Special object-level function (a.k.a. "dunder" function)
abs(), +, -
*, /, //
**, %, @, divmod
__abs__, __pos__, __add__, __concat__, __neg__, __sub__,
__mul__, __truediv__, __floordiv__,
__pow__, __mod__, __matmul__, __divmod__
+, -, *, /, **, etc. __radd__, __rsub__, __rmul__, __rtruediv__, __rpow__, etc.
+=, -=, *=, /=, etc. __iadd__, __isub__, __imul__, __itruediv__, etc.
<, <=, ==, !=, >, >= __lt__, __le__, __eq__, __ne__, __gt__, __ge__ (__cmp__ is not in 3.5)
See functools.total_ordering
and, or, not Logical: __not__ (Also, not_). No special functions for the first two: see PEP 335.
&, |, ^, ~ Bitwise: __and__, __or__, __xor__, __invert__ (Also and_, or_)
<<, >> __lshift__, __rlshift, __rshift__, __rrshift__
bool(obj)
len(obj)
If __bool__ is not defined, but __len__ is, then an object is true if len != 0.
If neither __bool__ nor __len__ is defined, then all instances are considered true.
self[key] __getitem__, __setitem__, __delitem__ (Both indexing and slicing)
for word in words __contains__, __iter__, __next__
is_, is_not Test object type
22

Python operators and special functions (2)
Operation Special function (a.k.a. "dunder" function)
abs(), +, -
*, /, //
**, %, @, divmod
__abs__, __pos__, __add__, __concat__, __neg__, __sub__,
__mul__, __truediv__, __floordiv__,
__pow__, __mod__, __matmul__, __divmod__
Etc. Etc.
23
• Suppose we define our own class, Vector2d:
class Vector2d:
def __init__(self, x, y):
self.x = x
self.y = y
def __add__(self, other):
new_x = self.x + other.x
new_y = self.y + other.y
return Vector2d(new_x, new_y)
With Vector2d defined like this, whenever Python sees:
• a Vector2d called a
• in the expression a + b
it converts that expression into a.__add__(b).
Otherwise, when it sees:
• a Vector2d called b
• in the expression a + b
it converts that expression into b.__radd__(a).
Lastly, when it sees:
• a Vector2d called a
• in the expression a += b
it converts that expression into a.__iadd__(b).
• Which of the operators we just saw would be convenient to define for this new class?
r => "right-hand side"
i => "increment"

IndividualCheckingAcct
Class hierarchies (not a focus of this course)
• Just like the tree biological taxonomy forms a hierarchy, you can also form a
tree of classes modeling some sort of hierarchy.
• For example, you can define a class called Mammal, and then define another
class called Dog, and tell Python that class Dog inherits all the attributes in
class Mammal.
• We won't be focusing on class hierarchies or class inheritance in this course.
• Sorry, rover!
24
Dog
Mammals IndividualBankAcct CommercialBankAcct CheckingAcct SavingsAcct
BankAcct
class IndividualCheckingAcct( IndividualBankAcct, CheckingAcct ):

Logging

Logging (1)
• Print statements can be useful for debugging, but there is room for improvement. There could be a way to:
• save the output to files, since a text window isn't always available (e.g., window-based apps)
• associate each message with a level of importance (DEBUG, INFO, WARNING, ERROR, CRITICAL)
• log only those messages that are important enough within the current "context", i.e., that meet a given "threshold"
• always log the time that the message was logged (and perhaps the host, for a server).
• Enter the logging package.
import logging
logging.basicConfig(level=logging.WARNING) # Only log msgs @ level WARNING or higher
logger = logging.getLogger(__name__)
logger.info('Starting to read from database') # Not logged, since INFO < WARNING
logger.error('Could not read from database') # Logged, since ERROR > WARNING
Output:
ERROR:__main__:Could not read from database
1. Within the software industry, a standard has emerged for web servers: See Common_Log_Format on Wikipedia.
2. If your logged messages are aggregated across timezones, then you should use Coordinated Universal Time (UTC).
3. Protip: Never use the filename logging.py: this confuses the logging infrastructure.
26

Logging (2)
• We just saw that the logging system can be configured with the statement that sets the logging "threshold".
logging.basicConfig(level=logging.WARNING)
• Here are some of the many other things that can be done with a basicConfig statement.
• Specify a filename, with the filename argument (By default, output is printed on the "console".)
• Specify the output format, with the format argument (This can include the user name, host name, etc.)
• Specify the date/time format, with the datefmt argument (default format: 2010-09-06 22:38:15,292)
In case it wasn't clear from the previous slide:
• The statement logging.debug(msg) logs the given message at level DEBUG.
• The statement logging.info(msg) logs the given message at level INFO.
• The statement logging.warning(msg) logs the given message at level WARNING.
• The statement logging.error(msg) logs the given message at level ERROR.
• The statement logging.critical(msg) logs the given message at level CRITICAL.
Note: The logging framework is designed to be thread-safe by default.
Note: Logging systems have been around for ages, including syslog (since the '80s) and Java's log4j (since 2001).
27

Unit testing

Unit testing: Introduction
• Q: How do you know if your code is working right?
• A: You don't.
• Q: What types of testing can increase your
confidence that it's working right?
• A: There are many. Unit testing is an important one.
• Unit testing—Apply tests to small units of code.
• Integration testing—Check that the different pieces of
code are working together as they're supposed to.
• Regression testing—Check to see that the bugs that have
been seen in the past haven't returned.
• Acceptance testing—Check that the customer's
requirements for sign-off are being met.
• And many, many more.
29

Unit testing: TestCase using assertEqual (1)
• Suppose your program has as many lines of testing code as there are of "production" code.
Keeping them separate helps simplify the code. (See "Separation of concerns" on Wikipedia.)
import unittest
class MyClassTest(unittest.TestCase):
def test_increment(self):
x = 0
x += 1
self.assertEqual(x, 1)
def test_decrement(self):
x = 0
x -= 1
self.assertEqual(x, -1)
if __name__ == '__main__':
unittest.main()
This main function discovers and runs all the tests it finds.
Details on how this works can be found at
https://docs.python.org/3/library/unittest.html#unittest.main
Important: Each test name starts with 'test_'
Output:
..
----------------------------------------
Ran 2 tests in 0.000s
OK

Unit testing: TestCase using assertEqual (2)
• Here is a similar unit testing example that uses the BankAccount class we saw earlier.
import unittest
class BankAccountTest(unittest.TestCase):
<...functions that we saw earlier...>
def test_deposit(self):
TEST_AMOUNT = 100
acct = BankAccount(0)
acct.deposit(TEST_AMOUNT)
self.assertEqual(acct.get_balance(), TEST_AMOUNT)
<...more test functions...>
if __name__ == '__main__':
unittest.main()
Oops: class BankAccount doesn't have a
get_balance function. We'll need to add it.

Unit testing: expectedFailure
• Unittest has support for many, many types of assertions, including "expected failures".
import unittest
class BankAccountTest(unittest.TestCase):
<...functions we saw earlier...>
@unittest.expectedFailure
def test_withdraw(self):
TEST_AMOUNT = 100
acct = BankAccount(0)
acct.withdraw(TEST_AMOUNT) # Ack! Insufficient funds!
self.assertEqual(acct.get_balance(), -TEST_AMOUNT)
<...more test functions...>
if __name__ == '__main__':
unittest.main()
This demonstrates the use of Python
decorators, which we haven't seen before.
With this, the assertion failure within this
function actually causes the test to succeed.

Unit testing: setUp and tearDown
• Lastly, the unittest package has support for organized groups of tests, or "test suites".
• Suppose you're testing a weather model that requires a lot of effort to set up.
• Then you'd want to:
• Set up a weather model before running a the test suite.
• Run each test that uses this weather model.
• Dispose of ("tear down") the weather model.
import unittest
class MyClassTest(unittest.TestCase):
def setUp(self):
create_weather_model()
def tearDown(self):
delete_weather_model()
def weather_test_1(self):
pass # TODO: Add test code here
def weather_test_2(self):
pass # TODO: Add test code here
• setUp is run first.
• Next, tests #1 and #2 are run.
• tearDown is run last

Debugging
“As soon as we started programming, we found to our surprise that it wasn’t as easy to get
programs right as we had thought. We had to discover debugging. I can remember the exact
instant when I realized that a large part of my life from then on was going to be spent in finding
mistakes in my own programs.”
—Maurice Wilkes (1913-2010), 1949
"Program testing can be used to show the presence of bugs, but never to show their absence!"
—Edsger W. Dijkstra (1930-2002), 1970
“Everyone knows that debugging is twice as hard as writing a program in the first place. So if
you're as cleverly as you can be when you write it, how will you ever debug it?”
— Brian W. Kernighan (b. 1942), 1974

Debugging: An overview
• When your program doesn't work the way you expect (or hope), it can be difficult to tell what's doing wrong.
• The first 18 chapters of Think Python v2 each have a section on Debugging, plus an Appendix on debugging.
• The debugging mindset. Some bugs are easy. Some require that you become a detective.
• Top 10 types of bugs. There are some common mistakes that are made that you can watch out for.
• Bug reports. There are standard ways to report bugs, in order to help teams debug effectively.
• Prevent recurrence. Sever bugs warrant a process or activity to prevent recurrence.
• Custom debugging code. Debugging is something that's often worth investing in.
• Design for Testability and Automation. Make it easier and cheaper to find bugs in the future.

Debugging: The debugging mindset
Bugs are an inevitable byproduct of the optimal pace of development.
• It might be possible to avoid almost all bugs by proceeding very cautiously, or even by
mathematically providing the correctness of the algorithms involved, but development
would be too slow for commercial purposes.
Some bugs are "shallow", and just require you to fix the error reported by the
Python interpreter, or correct a typo.
Some other bugs are more subtle, and require that you become a detective, and
• imagine what are the most likely things that could have gone wrong
• question your assumptions about how your program works
• get help from others on your team, or other teams, or from the internet.

Debugging: Top 10 types of bugs
Keeping possibilities like this in mind can help you find the bug more quickly.
• Indentation. Mixed spaces and tabs that confused the Python interpreter?
• Missing import. Tried a new function but didn't import the required module?
• No return value. Trying to use the return value of a function that doesn't return one.
• Off-by-one error. Did you get the right the begin and end loop bounds? (See § 8.11 of TP)
• Varying constants. Was a "constant" defined differently in two different places?
• Missing dependencies. Did you neglect to set up the weather model before using it?
• Version incompatibility. You're calling version 3.6 of a function, but you need to call v3.8.
• Recursion depth exceeded. Did a recursive function fail to stop calling itself?
• Didn't remove temporary code. Forgot to remove something temporary?
• Undertested error-handling code.
• Suppose there are checks to handle invalid user input.
• Such code is often not tested as well as the "mainstream" part of the program.

Debugging: Bug reports
Reporting a bug to others (for resolution, analysis, or archiving)? Some info to include:
• Reproduction ("repro") steps.
• Can the bug be reproduced?
• If so, how? (See "Platform".) If it's too difficult to reproduce, is there a "minimal test
case" that can trigger the same type of bug? Maybe a short program?
• Description.
• What (program) behavior are you expecting? A shared expectation, or disagreement?
• What behavior was actually observed? How do expectation and observation differ?
• Do the details of the bug change from run to run (e.g., timing issues)?
• Platform.
• What operating system was being used when the bug was found?
• What version of Python? "python --version" or "import sys; print(sys.version)"
• What program was run? How was it run? With what arguments?
• If it was found through a web browser, which browser type and version?

Debugging: Prevent recurrence
• "Low-hanging fruit".
• Suppose a function you use was updated from taking two arguments to taking three.
• When one call of a function uncovers the error, go ahead and fix all the calls.
• The systematic "Root Cause Resolution" (RCR) approach.
• If the bug was serious enough, you can take steps to prevent a recurrence.
• Gather info on the details. The bigger the program/system, the more effort this takes.
• Understand people's actions, but make it about the system, not the people. ("Ego free")
• List what steps could be taken to avoid recurrence.
• Follow through and see that those steps are carried out.

Debugging: Custom debugging code
• Companies often invest a great deal of effort in custom software to help them fix bugs in
their own code.
• Below is a function that prints out the important attributes of a given object. It's a fancier
version of the print_attributes function from Section 17.10 of ThinkPython, 2nd ed.
def dump_attributes(obj):
assert(hasattr(obj, '__class__'))
print('Dump of instance of {0:s}'.format(repr(obj.__class__)))
for attr_name in dir(obj):
if hasattr(obj, attr_name) and '__' not in attr_name:
attr_val = str(getattr(obj, attr_name))
attr_val = '<method>' if 'method' in attr_val else attr_val
print(' obj.{0:s} = {1:s}'.format(attr_name, attr_val))
• Code like this that prints out the details of object is sometimes called "pretty printing".

Debugging: Design for Testability and Automation
• Testability = Controllability + Visibility
• The Lunar Lander program we saw earlier in the course had options to set the initial
values of fuel, altitude, and velocity. This made it easy to test the effect of specific burn
rates on specific states of the Lunar Lander.
• The BankAccount class we saw earlier started without a get_balance function, which we
had to add in order to perform a specific test we had in mind.
• When breaking up functionality into classes and functions, choose wisely.
• Classes that are too "large" allow for too much "hidden" interaction within the class.
• Classes that are too "fine-grained" require excessive "bookkeeping".
• Automation of testing.
• Testing is expensive. It makes sense to invest effort into automating tests over time.
• unittest is one example of support for improving test automation over time.

Intro to Python (High School) Unit #2

More Related Content

What's hot

Similar to Intro to Python (High School) Unit #2

More from Jay Coskey

Recently uploaded

Intro to Python (High School) Unit #2

Editor's Notes