If someone tells you that it is impossible to reimplement some piece of software to be compatible with a newer technology, that is most often not truth. This is the story about porting RestrictedPython, one of the core packages of Zope to be compatible with Python 3.

1. Doing the Impossible
Porting RestrictedPython to Python 3
The way to Python 3 for Plone?
Alexander Loechel
PloneConf 2017 - Barcelona
1

2. –Johnny Appleseed
“Type a quote here.”
2
End of Life
for Python 2
by 2020 PyCon
(April 12th 2020)

3. PloneConf 2014 Bristol - Plone Roadmap Discussion
“Plone on Python 3
is impossible,
there are too
many blockers in Zope.
Especially RestrictedPython
& AccessControl”
3

4. Clarke's First Law – Arthur C. Clarke
“When a distinguished but elderly
scientist states that something is
possible, he is almost certainly right.
When he states that something is
impossible, he is very probably wrong.”
4

5. – 2015 - Start looking into RestrictedPython
“Why should RestrictedPython
be the major blocker for
Plone on Python 3?”
5

6. Hanno Schlichting - Zope Release Manager
„Every piece of Zope that was not
adopted by Plone is literally dead.“
6

7. „Debugging is twice as hard as writing
the code in the first place. Therefore, if
you write the code as cleverly as
possible, you are, by definition, not
smart enough to debug it.“
7
- Brian Kernighan
Almost no documentation
low test coverage

8. Python community - around 2000
„Where Zope leads,
Python follows“
8
Zope specific leftovers in the Python 2 Standard Library
Deprecated Python Standard Library Modules:
• compiler Package (no described upgrade path, almost no documentation)

9. Clarke's Second Law – Arthur C. Clarke
“The only way of discovering the
limits of the possible is to
venture a little way past them
into the impossible.”
9

10. Clarke's Third Law – Arthur C. Clarke
“Any sufficiently
advanced technology
is indistinguishable
from magic.”
10

11. • TTW (Through-The-Web)
• PythonScripts
• DocumentTemplate
• Workflows
• Zope ZCatalog
11

12. What is
RestrictedPython
and what is it not?
12

13. A Workshop on Python at NIST
SPAM 1 - First PyCon (1994) - Topics
I've assembled a strawman list of topics for discussion. New topics
and comments on the topics listed are certainly welcome.
A.Requirements for a "Safe" Python interpreter
B. A standard GUI module interface definition for Python
C.The requirements for persistent objects in Python
D.A Python engineering graphs package
E. The standard Python WWW interface
F. Embedding Python in a WWW client
G.Technical information management using Python
H.Support for dynamic loading of foreign language modules in Python
I. Replacing make, rcs, and cvs with Python
J. An Electronic Data Interchange library for Python
K. Discussing the formation of a Python Consortium
13

14. Rich Hickey
“Most of the biggest problem
in software are problems of
misconception.”
14

15. No Sandbox
Sandboxes most often don’t work
15

16. Limited, „Safe“ Subset
of the Python Programming
Language / Grammar
• No more Turing Complete
• Common Technique (Ada RavenScar Profile)
16

17. The Way to
RestrictedPython 4.0
and Python 3
17

18. - John Johnson
“First, solve the problem.
Then, write the code.”
18

19. • Python 2 Standard Library Module compiler and its Class ast
• Not fully documented
• No upgrade path for Python 3 described
• manual Byte-Code generator
• Interpreter specifics —> Only CPython 2 compatible
19

20. If it is not documented it is not usable
if it is not tested it did not work
if it is not checked into version control it did not exists
if it is not repeatable it is not science
• Compiler knowledge was necessary to port RestrictedPython
• I did have a degree in Informatics (Computer Science) and
learned how to build and manipulate compilers, so give it a try.
20

21. Plone Open Garden 2015
Start looking into RestrictedPython
21

22. Kent Beck
“Any fool can write code that a
computer can understand.
Good programmers write code
that humans can understand.”
22
Reading &
understanding the Code

23. Louis Sreygley
“Without requirements or design,
programming is the art of adding
bugs to an empty text file.”
23
Documentation First +
Test Driven Development

24. • compiler.ast => ast.AST
• builtin compile() function since Python 2.6 accepts ast.AST as input
and compiles to Python Byte-Code —> Interpreter independence
• compile(source, filename, mode [, flags=ast.PyCF_ONLY_AST [, dont_inherit]])
or ast.parse() return ast.AST
With Python 2.6+ / 3.4+ RestrictedPython could be easier to implement
24

25. Got Access to Zope Foundation Repositories via Tres Seaver at Plone Symposium Tokyo 2015
“Access Required.”
25

26. >>> import this
The Zen of Python, by Tim Peters
Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to
break the rules.
…
PEP 20 - 19. Aug. 2004 https://www.python.org/dev/peps/pep-0020/
26
Apply Python’s PEP8 and
Plone’s Coding Conventions
—> Make the Code more readable

27. – Richard Feynman
“The first principle is that you
must not fool yourself — and you
are the easiest person to fool.”
27
Writing Tests & raising test coverage
—> Functional Test of all Python Syntax cases

28. tox & .
• tox: local tests of multiple python versions
• pytest: modern Python testing framework, that allows:
• parameterisation (inputs and functions)
• skip_if
• assert
Talk: Modern Python Testing
28

29. Test example (pytest)
c_exec = (c_exec‘, [
RestrictedPython.compile.compile_restricted_exec,
RestrictedPython.RCompile.compile_restricted_exec,
])
@pytest.mark.parametrize(*c_exec)
def test_(c_exec):
result = c_exec('a = max([1, 2, 3])')
assert result.errors == ()
loc = {}
exec(result.code, {}, loc)
assert loc['a'] == 3
assert result.used_names == {'max': True}
29

30. Alpine City Sprint 2016 / 2017
Join in of Gocept & other Zope Core Developers
30

31. Ian Hickson
„Things that are impossible
just take longer.“
31

32. May 2017 Release of RestrictedPython 4.0a1
https://pypi.python.org/pypi/RestrictedPython —> 4.0.b2
Welcome to the Python 3 Wonderland32

33. Philip J. Eby - Python Core Developer - Author PEP 333/3333 - WSGI
„Those who do not study Zope,
are condemned to reinvent it.“
33

34. Bjarne Stroustrup
“If you think it’s simple,
then you have
misunderstood the problem.”
34

35. My Wish
Make RestrictedPython more known
enable other Projects and Frameworks to use
a „Safe“ Python Interpreter through the Web

36. John Gall - Systemantics: How Systems Really Work and How They Fail
A complex system that works is invariably
found to have evolved from a simple system
that worked. A complex system designed from scratch
never works and cannot be patched up to make it work.
You have to start over with a working simple system.
36

37. Lessons Learned while Porting RestrictedPython
• Don’t take »„impossible“ to port / fix« statements serious
• Adopt modern tools and frameworks that helps:
• tox & pytest
—> Update Best Practices for Plone development
37

38. Some further Talk recommendations
• Zope on Python 3
Hanno Schlichting - Friday 14:30-15:15 - Auditori
• Modern Python Testing
Alexander Loechel - Thursday 14:30-15:15 - Sala d'actes
• subtemplates in bobtemplates.plone or on the way to plonecli
Maik Derstappen - Friday 15:25-15:55 - Sala d'actes
38

39. Clarke's Law of Revolutionary Ideas – Arthur C. Clarke
Every revolutionary idea
— in science, politics, art, or whatever —
seems to evoke three stages of reaction.
They may be summed up by the phrases:
1. "It's completely impossible — don't waste my time"
2. "It's possible, but it's not worth doing"
3. "I said it was a good idea all along"
39