Elegant Solutions for Everyday Python Problems Pycon 2018 - Nina Zakharenko

Elegant
Solutions for
Everyday
Python
Problems
Nina Zakharenko
@nnja
h!p://bit.ly/elegant-
python
ℹ There are links in these slides.

slides: h!p://bit.ly/elegant-python
This talk is for you if:
You're an intermediate python programmer
You're coming to python from another language
You want to learn about language features like: magic
methods, iterators, decorators, and context managers
@nnja

How do we make code elegant?
We pick the right tool for the job!
Resources for converting from Python 2 -> 3

Beauty is
in the eye of
the beholder

Magic methods start and end with a double underscore
(dunder)
By implementing a few straightforward magic methods,
you can make your objects behave like built-ins such as:
numbers
lists
dictionaries
and more...
@nnja

class Money:
currency_rates = {
'$': 1,
'€': 0.88,
}
def __init__(self, symbol, amount):
self.symbol = symbol
self.amount = amount
def __str__(self):
return '%s%.2f' % (self.symbol, self.amount)
@nnja

class Money:
# deﬁned currency_rates, __init__, and str above...
def convert(self, other):
"""Convert other amount to our currency"""
new_amount = (
other.amount / self.currency_rates[other.symbol]
* self.currency_rates[self.symbol])
return Money(self.symbol, new_amount)
@nnja

__str__ in action
>>> soda_cost = Money('$', 5.25)
>>> print(soda_cost)
$5.25
>>> pizza_cost = Money('€', 7.99)
>>> print(pizza_cost)
€7.99
@nnja

class Money:
def __add__(self, other):
""" Add 2 Money instances using '+' """
new_amount = self.amount + self.convert(other).amount
return Money(self.symbol, new_amount)
@nnja

>>> print(soda_cost + pizza_cost)
$14.33
More on Magic Methods: Dive into Python3 - Special Method
Names

>>> print(soda_cost + pizza_cost)
$14.33
>>> print(pizza_cost + soda_cost)
€12.61
More on Magic Methods: Dive into Python3 - Special Method
Names

some magic methods map to symbols
>>> d = {'one': 1, 'two': 2}
>>> d['two']
2
>>> d.__getitem__('two')
2
@nnja

other magic methods map to built-in functions
class SquareShape:
def __len__(self):
""" Return the number of sides in our shape """
return 4
>>> my_square = SquareShape()
>>> len(my_square)
4
@nnja

Making classes iterable
In order to be iterable, a class needs to implement
__iter__()
__iter__() must return an iterator
In order to be an iterator a class needs to implement
__next__() which must raise StopIteration when
there are no more items to return
Great explanation of iterable vs. iterator vs. generator

Scenario..
We have a Server instance running services on
different ports.
Some services are active, some are inactive.
When we loop over our the Server instance, we only
want to loop over active services.
@nnja

class IterableServer:
services = [
{'active': False, 'protocol': 'ftp', 'port': 21},
{'active': True, 'protocol': 'ssh', 'port': 22},
{'active': True, 'protocol': 'http', 'port': 80},
]
@nnja

class IterableServer:
def __init__(self):
self.current_pos = 0
def __iter__(self):
# can return self, because __next__ implemented
return self
def __next__(self):
while self.current_pos < len(self.services):
service = self.services[self.current_pos]
self.current_pos += 1
if service['active']:
return service['protocol'], service['port']
raise StopIteration
@nnja

>>> for protocol, port in IterableServer():
print('service %s on port %d' % (protocol, port))
service ssh on port 22
service http on port 80
loops over all active services ... not bad
@nnja

tip: use a generator
when your iterator doesn't need to
maintain a lot of state
(which is most of the time)
@nnja

class Server:
services = [
{'active': False, 'protocol': 'ftp', 'port': 21},
{'active': True, 'protocol': 'ssh', 'port': 22},
{'active': True, 'protocol': 'http', 'port': 21},
]
def __iter__(self):
for service in self.services:
if service['active']:
yield service['protocol'], service['port']
@nnja

Why does this work?
use single parenthesis ( ) to create a generator
comprehension
^ technically, a generator expression but I like this term better, and so does Ned
Batchelder
>>> my_gen = (num for num in range(1))
>>> my_gen
<generator object <genexpr> at 0x107581bf8>
@nnja

An iterator must implement __next__()
>>> next(my_gen) # __next__() maps to built-in next()
0
and raise StopIteration when there are no more elements
>>> next(my_gen)
... StopIteration Traceback (most recent call last)
see itertools for working with iterators

alias methods
class Word:
def __init__(self, word):
self.word = word
def __repr__(self):
return self.word
def __add__(self, other_word):
return Word('%s %s' % (self.word, other_word))
# Add an alias from method __add__ to the method concat
concat = __add__
@nnja

We can add an alias from __add__ to concat because
methods are just objects
>>> # remember, concat = __add__
>>> first_name = Word('Max')
>>> last_name = Word('Smith')
>>> first_name + last_name
Max Smith
>>> first_name.concat(last_name)
Max Smith
>>> Word.__add__ == Word.concat
True
@nnja

Dog class
>>> class Dog:
sound = 'Bark'
def speak(self):
print(self.sound + '!', self.sound + '!')
>>> my_dog = Dog()
>>> my_dog.speak()
Bark! Bark!
read the docs

getattr(object, name, default)
>>> class Dog:
sound = 'Bark'
def speak(self):
print(self.sound + '!', self.sound + '!')
>>> my_dog = Dog()
>>> my_dog.speak()
Bark! Bark!
>>> getattr(my_dog, 'speak')
<bound method Dog.speak of
<__main__.Dog object at 0x10b145f28>>
>>> speak_method = getattr(my_dog, 'speak')
>>> speak_method()
Bark! Bark!

Example: command line tool with dynamic commands
class Operations:
def say_hi(self, name):
print('Hello,', name)
def say_bye(self, name):
print('Goodbye,', name)
def default(self, arg):
print('This operation is not supported.')
if __name__ == '__main__':
operations = Operations()
# let's assume error handling
command, argument = input('> ').split()
getattr(operations, command, operations.default)(argument)
read the docs

Output
$ python demo.py
> say_hi Nina
Hello, Nina
> blah blah
This operation is not supported.
✨
additional reading - inverse of getattr() is setattr()

functools.partial(func, *args, **kwargs)
Return a new partial object which behaves like func
called with args & kwargs
if more arguments are passed in, they are appended
to args
if more keyword arguments are passed in, they extend
and override kwargs
read the docs on partials

functool.partial(func, *args, **kwargs)
# We want to be able to call this function on any int
# without having to specify the base.
>>> int('10010', base=2)
18
>>> from functools import partial
>>> basetwo = partial(int, base=2)
>>> basetwo
<functools.partial object at 0x1085a09f0>
>>> basetwo('10010')
18
read the docs on partials

magic methods
& method magic
in action!
photo credit

library I
!
: github.com/mozilla/agithub
agithub is a (poorly named) REST API client with
transparent syntax which facilitates rapid prototyping
— on any REST API!
Implemented in ~400 lines.
Add support for any REST API in ~30 lines of code.
agithub knows everything it needs to about protocol
(REST, HTTP, etc), but assumes nothing about your
upstream API.
@nnja

deﬁne endpoint url & other connection properties
class GitHub(API):
def __init__(self, token=None, *args, **kwargs):
props = ConnectionProperties(
api_url = kwargs.pop('api_url', 'api.github.com'))
self.setClient(Client(*args, **kwargs))
self.setConnectionProperties(props)
@nnja

then, start using the API!
>>> gh = GitHub('token')
>>> status, data = gh.user.repos.get()
>>> # ^ Maps to GET /user/repos
>>> data
... ['tweeter', 'snipey', '...']
github.com/mozilla/agithub

404 if we provide a path that doesn't exist:
>>> gh = GitHub('token')
>>> status, data = gh.this.path.doesnt.exist.get()
>>> status
... 404
github.com/jpaugh/agithub

class API:
def __getattr__(self, key):
return IncompleteRequest(self.client).__getattr__(key)
__getitem__ = __getattr__
class IncompleteRequest:
def __getattr__(self, key):
if key in self.client.http_methods:
htmlMethod = getattr(self.client, key)
return partial(htmlMethod, url=self.url)
else:
self.url += '/' + str(key)
return self
__getitem__ = __getattr__
class Client:
http_methods = ('get') # ...
def get(self, url, headers={}, **params):
return self.request('GET', url, None, headers)
github.com/jpaugh/agithub source: base.py

When should I use one?
Need to perform an action before and/or after an
operation.
Common scenarios:
Closing a resource after you're done with it (ﬁle,
network connection)
Perform cleanup before/after a function call
@nnja

Example Problem: Feature Flags
Turn features of your application on and off easily.
Uses of feature ﬂags:
A/B Testing
Rolling Releases
Show Beta version to users opted-in to Beta Testing
Program
More on Feature Flags

class FeatureFlags:
SHOW_BETA = 'Show Beta version of Home Page'
flags = {
SHOW_BETA: True
}
@classmethod
def is_on(cls, name):
return cls.flags[name]
@classmethod
def toggle(cls, name, value):
cls.flags[name] = value
feature_flags = FeatureFlags()
@nnja

How do we temporarily turn features on and off when
testing ﬂags?
Want:
with feature_ﬂag(FeatureFlags.SHOW_BETA):
assert '/beta' == get_homepage_url()
@nnja

Using Magic Methods __enter__ and __exit__
class feature_flag:
""" Implementing a Context Manager using Magic Methods """
def __init__(self, name, on=True):
self.name = name
self.on = on
self.old_value = feature_flags.is_on(name)
def __enter__(self):
feature_flags.toggle(self.name, self.on)
def __exit__(self, *args):
feature_flags.toggle(self.name, self.old_value)
See: contextlib.contextmanager

The be!er way: using the contextmanager decorator
from contextlib import contextmanager
@contextmanager
def feature_flag(name, on=True):
old_value = feature_flags.is_on(name)
feature_flags.toggle(name, on)
yield
feature_flags.toggle(name, old_value)

The be!er way: using the contextmanager decorator
@contextmanager
""" The easier way to create Context Managers """
# behavior of __enter__()
yield
# behavior of __exit__()

Note: yield?
@contextmanager
""" The easier way to create Context Managers """
feature_ﬂags.toggle(name, on) # behavior of __enter__()
yield
feature_ﬂags.toggle(name, old_value) # behavior of __exit__()

either implementation
def get_homepage_url():
""" Returns the path of the page to display """
if feature_ﬂags.is_on(FeatureFlags.SHOW_BETA):
return '/beta'
else:
return '/homepage'
def test_homepage_url_with_context_manager():
# saw the beta homepage...
assert get_homepage_url() == '/beta'
with feature_ﬂag(FeatureFlags.SHOW_BETA, on=False):
# saw the standard homepage...
assert get_homepage_url() == '/homepage'

either implementation
def get_homepage_url():
""" Returns the path of the page to display """
if feature_ﬂags.is_on(FeatureFlags.SHOW_BETA):
return '/beta'
else:
return '/homepage'
def test_homepage_url_with_context_manager():
print('seeing the beta homepage...')
with feature_ﬂag(FeatureFlags.SHOW_BETA, on=False):
assert get_homepage_url() == '/homepage'
print('seeing the standard homepage...')

Decorators
The simple explanation:
Syntactic sugar that allows modiﬁcation of an underlying
function.
@nnja

Decorators:
Wrap a function in another function.
Do something:
before the call
after the call
with provided arguments
modify the return value or arguments
@nnja

class User:
is_authenticated = False
def __init__(self, name):
self.name = name
Throw an exception if trying to access data only for
logged in users:
def display_profile_page(user):
""" Display profile page for logged in User """
if not user.is_authenticated:
raise Exception('User must login.')
print('Profile: %s' % user.name)

def enforce_authentication(func):
def wrapper(user):
return func(user)
return wrapper
the important logic:
def wrapper(user):
return func(user)
return wrapper
@nnja

Using enforce_authentication without a decorator:
enforce_authentication(display_profile_page)(some_user)
Or, as a decorator:
@enforce_authentication
def display_profile_page(user):
print('Profile: %s' % user.name)
Now this raises an Exception if unauthenticated:
user = User('nina')
display_profile_page(nina)

Problem: lost context using a decorator
>>> display_proﬁle_page.__name__
'wrapper'
>>>> display_proﬁle_page.__doc__
# ... empty
Solution: Use contextlib.wraps
from contextlib import wraps
@wraps(func)
def wrapper(user):
# ... rest of the code

Decorators: Common uses
logging
timing
validation
rate limiting
mocking/patching
@nnja

ContextDecorators
ContextManagers
+ Decorators combined.
@nnja

By using ContextDecorator you can easily write
classes that can be used both as decorators with @
and context managers with the with statement.
ContextDecorator is used by contextmanager(),
so you get this functionality automatically .
Or, you can write a class that extends from
ContextDecorator or uses ContextDecorator as a
mixin, and implements __enter__, __exit__ and
__call__
@nnja

Remember @contextmanager from earlier?
@contextmanager
yield
@nnja

use it as a context manager
or use as a decorator
@feature_flag(FeatureFlags.SHOW_BETA, on=False)
def get_profile_page():
beta_flag_on = feature_flags.is_on(
FeatureFlags.SHOW_BETA)
if beta_flag_on:
return 'beta.html'
else:
return 'profile.html'

library I
!
: freezegun
lets your python tests ❇ travel through time! ❇
from freezegun import freeze_time
# use it as a Context Manager
def test():
with freeze_time("2012-01-14"):
assert datetime.datetime.now() == datetime.datetime(
2012, 1, 14)
assert datetime.datetime.now() != datetime.datetime(2012, 1, 14)
# or a decorator
@freeze_time("2012-01-14")
def test():
assert datetime.datetime.now() == datetime.datetime(2012, 1, 14)
read the source sometime, it's mind-bending!

NamedTuple
Useful when you need lightweight representations of
data.
Create tuple subclasses with named ﬁelds.
@nnja

Example
from collections import namedtuple
CacheInfo = namedtuple(
"CacheInfo", ["hits", "misses", "max_size", "curr_size"])
@nnja

Giving NamedTuples default values
RoutingRule = namedtuple(
'RoutingRule',
['preﬁx', 'queue_name', 'wait_time']
)
(1) By specifying defaults
RoutingRule.__new__.__defaults__ = (None, None, 20)
(2) or with _replace to customize a prototype instance
default_rule = RoutingRule(None, None, 20)
user_rule = default_rule._replace(
preﬁx='user', queue_name='user-queue')

NamedTuples can be subclassed and extended
class Person(namedtuple('Person', ['first_name', 'last_name'])):
""" Stores first and last name of a Person"""
def __str__(self):
return '%s %s' % (self.first_name, self.last_name)
>>> me = Person('nina', 'zakharenko')
>>> str(me)
'nina zakharenko'
>>> me
Person(first_name='nina', last_name='zakharenko')

New Tools
Magic Methods
make your objects behave like builtins (numbers,
list, dict, etc)
Method ❇Magic❇
alias methods
getattr
@nnja

ContextManagers
Manage resources
Decorators
do something before/after call, modify return value
or validate arguments
ContextDecorators
ContextManagers + Decorators in one
@nnja

Iterators & Generators
Loop over your objects
yield
NamedTuple
Lightweight classes
@nnja

"Perfection is achieved, not when
there is nothing more to add, but
when there is nothing left to take
away."
— Antoine de Saint-Exupery
@nnja

Be an
elegant
Pythonista!
@nnja

Thanks!
slides: bit.ly/elegant-python
python@ microso! aka.ms/python
@nnja
@nnja

Elegant Solutions for Everyday Python Problems Pycon 2018 - Nina Zakharenko

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Elegant Solutions for Everyday Python Problems Pycon 2018 - Nina Zakharenko

Similar to Elegant Solutions for Everyday Python Problems Pycon 2018 - Nina Zakharenko (20)

More from Nina Zakharenko

More from Nina Zakharenko (7)

Recently uploaded

Recently uploaded (20)

Elegant Solutions for Everyday Python Problems Pycon 2018 - Nina Zakharenko