Beaker is probably the most widespread cross-framework solution to manage sessions and caching in the python web ecosystem. Born in 2005 from Pylons author has been historically maintained together with the Pylons framework. Since Pylons has been deprecated in favour of Pyramid, the Pylons Project team decided to write a custom session management solution for Pyramid and let the user handle more advanced backends. Since 2015 beaker maintenance has been passed to the TurboGears project who ported it to a fully native Python3 solution and it’s stille the de-facto standard for Sessions and Caching on Bottle. The talk will cover the advantages and drawbacks of the architectural decisions behind the 10 years of history of Beaker, the drawbacks we discovered while porting it to Python3 and which have been the “wrong choices” and how we plan to solve them and make it shine again as the best whole-around solution for Sessions and Caching on WSGI.

1. BEAKER - MAKING SESSIONS AND
CACHING ROOMMATES
Alessandro Molina
@__amol__
amol@turbogears.org

2. Who am I
● CTO @ AXANT.it, mostly Python company
(with some iOS and Android)
● TurboGears2 dev team member
● Took over Beaker maintenace in 2015
● Mostly contributed to web python libraries:
Formencode, MING MongoDB ODM, ToscaWidgets2

3. What’s Beaker
● Framework for handling Caching and
Sessions in web applications
● Used by many different frameworks:
TurboGears, Bottle, Pyramid, etc...
● Created by Pylons Author to solve the
dogpile effect.

4. DogPile!

5. The Good
● Handles the DogPile effect
● Provides many backends: Memcache, File,
MongoDB, Redis, SQLAlchemy
● A single tool to handle both caching and
sessions with common backends.

6. The Bad
● File based synchronization, what about
distributed solutions?
● Session as a big BLOB of data doesn’t
perform well in case of big BLOBs of data
● Updated for too long without breaking
backward compatibility: Too many APIs

7. Getting on project: PY3 Support
● Beaker supported Python3 using 2to3
● This lead to some bugs (not everything got
properly converted)
● Made really hard to run the test suite and
maintain the project.

8. So my first action as new
maintainer...

9. Rewriting for Py3 a project that
involves serializing/deserializing

10. Porting to PY3: #1 Snakes speak ?
Beaker uses BASE64 to encode pickled data
in sessions.
>>> import base64, pickle
>>> data = dict(key='value', otherkey='othervalue')
>>> base64.b64encode(pickle.dumps(data))
'KGRwMApTJ290aGVya2V5JwpwMQpTJ290aGVydmFsdWUnCnAyCnN
TJ2tleScKcDMKUyd2YWx1ZScKcDQKcy4='

11. BASE64: rfc4648
Here are a few requirements that
determine which alphabet should be
used:
o Handled by humans.

12. Human handle?
TEXT!

13. Human handle?
TEXT!

14. Human handle?
BYTES!
>>> import base64, pickle
>>> base64.b64encode(pickle.dumps(‘HELLO’))
b'gANYBQAAAEhFTExPcQAu'

15. Porting to PY3: #2 Session Cookies?
● Session ID is stored in cookies
● CookieSession stores even the whole
session in a cookie
○ Great idea, makes really simple to scale
○ HTTP Headers are plain text
○ We are pickling data and base64 so already ASCII
○ Just need to encrypt it to avoid people from
messing with them.

16. HTTP Cookies
HTTP has allowed field content with text
in the ISO-8859-1 charset [ISO-8859-1],
supporting other charsets only through
use of [RFC2047] encoding. In practice,
most HTTP header field values use only a
subset of the US-ASCII charset [USASCII].
Newly defined header fields SHOULD limit
their field values to US-ASCII octets

17. Encoding/Encripting data on Py3
● Beaker expected to be able to perform
text operations on the result of base64
● Relied on AES to encrypt cookies, pbkdf2
to generates keys and base64/binascii
to make it text.
● Reads the encrypt_key from config file

18. 4 lines of code == 10 doubts
● Even Python has no clear idea of what is
text and what are bytes:
○ Both BASE64 and BINASCII work with ASCII
○ BINASCII accepts unicode (text)
○ BASE64 accepts bytes
○ Between Python3.2 and Python3.3 the binascii
changed behaviour of accepting unicode vs bytes
○ ConfigParser returns text on py3, bytes on py2

19. Now crying in a corner...

20. Let’s go for something simpler...
● Review patch to caching decorator
● To cache a function, just apply a decorator
that calls function and caches the result
● Caching decorator generates cache key
○ Cache Key from the function name
○ convert parameters to strings
○ add parameters to cache key

21. Getting the Cache Key
def cache(self, *args, **kwargs):
"""Decorate a function to cache itself with supplied parameters
:param args: Used to make the key unique for this function, as in region()
above.
:param kwargs: Parameters to be passed to get_cache(), will override defaults
Example::
# Assuming a cache object is available like:
cache = CacheManager(dict_of_config_options)
def populate_things():
@cache.cache('mycache', expire=15)
def load(search_term, limit, offset):
return load_the_data(search_term, limit, offset)
return load('rabbits', 20, 0)
"""
return _cache_decorate(args, self, kwargs, None)

22. Well, maybe...

23. Unknown parameters
● Decorator gets *args and **kwargs
● Makes sense... doesn’t know the real
function arguments
● Not so much in fact… func(1) and func
(a=1) end up with two different cache
keys

24. Avoid the doubt
● Beaker solved this by accepting only
positional arguments on cached functions
.. note::
The function being decorated must only be called with
positional arguments.
● Users not so happy, breaks people code
when they introduce caching:
○ https://github.com/bbangert/beaker/issues/62

25. What to do?
● Leave it as is… As been like that for years
● Add **kwargs and just document the issue
● Add **kwargs and try to be smart using
inspect.getargspec / inspect.
getcallargs but that would be slow

26. Leave it as is… at least isn’t broken

27. Has been a great trip!
● Making sessions and caching in a single
solution seemed simple, but has actually a
lot of corner case
● Beaker Co-Author thrown in the towel and
created dopgile.cache which is based on
beaker code but is much simpler and
doesn’t provide sessions

28. Still is incredibily convenient
● There are more shared features than
differences (especially in cookie based
marshalled sessions).
● It’s really convenient, write backends once
and have support for both sessions and
caching on them.

29. Questions?