Talk at Pycon India 2016 CC BY-SA 4.0

1. @jargnar
github.com/jargnar
Hacking the Python AST

2. 01010101010101010101010101...

3. A primer on compiling computer languages
Front end Middle end Back end
C
JAVA
ARM
x86
IR IR

4. Intel’s developer manual

5. A primer on compiling computer languages
Front end Middle end Back end
C
JAVA
ARM
x86
IR IR

6. @jargnar
github.com/jargnar
The
Dragon
Book

7. Grammars
S –> AB
A –> aA | ε
B –> b | bB
S
AB S –> AB
aAB A –> aA
aaAB A –> aA
aaaAB A –> aA
aaaεB A –> ε
aaab B –> b
A parse of aaab
Does aaab belong to this
Grammar?

8. @jargnar
github.com/jargnar
LL(1) Grammar
Left-to-right scan
Leftmost derivation

9. @jargnar
github.com/jargnar
Let’s take a quick look at Python’s Grammar
(Grammar/Grammar)

10. Design of the CPython Compiler
Parser/
pgen.c
Python/
ast.c
Python/
compile.c
Python/
compile.c
Parse
Tree
.py
file
Byte
code
AST CFG

11. @jargnar
github.com/jargnar
dis

12. Design of the CPython Compiler
Parser/
pgen.c
Python/
ast.c
Python/
compile.c
Python/
compile.c
Parse
Tree
.py
file
Byte
code
AST CFG

13. Python to x86 and then the 01010101s
Parser/
pgen.c
Python/
ast.c
Python/
compile.c
Python/
compile.c
Parse
Tree
.py
file
Byte
code
AST CFG
Python/
ceval.c

14. @jargnar
github.com/jargnar

15. Accessing the AST
a = 26
b = 0
print(a + b)
example.py
In [1]: import ast
In [2]: source =
open('example.py').read()
In [3]: tree = ast.parse(source)

16. Accessing the AST
a = 26
b = 0
print(a + b)
example.py
Module(body=[
Expr(value=Str(s='Docstring')),
Assign(
targets=[Name(
id='a',
ctx=Store())],
value=Num(n=26)),
Assign(
targets=[Name(
id='b',
ctx=Store())],
value=Num(n=0)),
Expr(value=Call(
func=Name(
id='print',
ctx=Load()),
args=[BinOp(
left=Name(
id='a',
ctx=Load()),
op=Add(),
right=Name(
id='b',
ctx=Load()))],
keywords=[]))])

17. Having a little fun, now
In [3]: tree = ast.parse('x = 5')
In [4]:
print(astunparse.dump(tree))
Module(body=[Assign(
targets=[Name(
id='x',
ctx=Store())],
value=Num(n=5))])
In [5]: tree.body[0].targets[0].id
Out[5]: 'x'
In [6]: astunparse.unparse(tree)
x = 5
In [7]:
tree.body[0].targets[0].id = 'y'
In [8]:
print(astunparse.unparse(tree))
y = 5

18. @jargnar
github.com/jargnar
Let’s do absolutely silly things with
ast.NodeVisitor
ast.NodeTransformer

19. @jargnar
github.com/jargnar
Even more silly things with
Flake8

20. @jargnar
github.com/jargnar
Hacking is a mindset, not a skill.
suhas.org