@jargnar
github.com/jargnar
Hacking the Python AST
01010101010101010101010101...
A	primer	on	compiling	computer	languages
Front end Middle end Back end
C
JAVA
ARM
x86
IR IR
Intel’s developer manual
A	primer	on	compiling	computer	languages
Front end Middle end Back end
C
JAVA
ARM
x86
IR IR
@jargnar
github.com/jargnar
The
Dragon
Book
Grammars
S –> AB
A –> aA | ε
B –> b | bB
S
AB	 S	–>	AB
aAB A	–>	aA
aaAB A	–>	aA
aaaAB A	–>	aA
aaaεB A	–>	ε
aaab B	–>	b
A parse of aaab
Does aaab belong to this
Grammar?
@jargnar
github.com/jargnar
LL(1) Grammar
Left-to-right scan
Leftmost derivation
@jargnar
github.com/jargnar
Let’s	take	a	quick	look	at	Python’s	Grammar
(Grammar/Grammar)
Design	of	the	CPython Compiler
Parser/
pgen.c
Python/
ast.c
Python/
compile.c
Python/
compile.c
Parse
Tree
.py
file
Byte
code
AST CFG
@jargnar
github.com/jargnar
dis
Design	of	the	CPython Compiler
Parser/
pgen.c
Python/
ast.c
Python/
compile.c
Python/
compile.c
Parse
Tree
.py
file
Byte
code
AST CFG
Python	to	x86	and	then	the	01010101s
Parser/
pgen.c
Python/
ast.c
Python/
compile.c
Python/
compile.c
Parse
Tree
.py
file
Byte
code
AST CFG
Python/
ceval.c
@jargnar
github.com/jargnar
Accessing	the	AST
a = 26
b = 0
print(a + b)
example.py
In [1]: import ast
In [2]: source =
open('example.py').read()
In [3]: tree = ast.parse(source)
Accessing	the	AST
a = 26
b = 0
print(a + b)
example.py
Module(body=[
Expr(value=Str(s='Docstring')),
Assign(
targets=[Name(
id='a',
ctx=Store())],
value=Num(n=26)),
Assign(
targets=[Name(
id='b',
ctx=Store())],
value=Num(n=0)),
Expr(value=Call(
func=Name(
id='print',
ctx=Load()),
args=[BinOp(
left=Name(
id='a',
ctx=Load()),
op=Add(),
right=Name(
id='b',
ctx=Load()))],
keywords=[]))])
Having	a	little	fun,	now
In [3]: tree = ast.parse('x = 5')
In [4]:
print(astunparse.dump(tree))
Module(body=[Assign(
targets=[Name(
id='x',
ctx=Store())],
value=Num(n=5))])
In [5]: tree.body[0].targets[0].id
Out[5]: 'x'
In [6]: astunparse.unparse(tree)
x = 5
In [7]:
tree.body[0].targets[0].id = 'y'
In [8]:
print(astunparse.unparse(tree))
y = 5
@jargnar
github.com/jargnar
Let’s	do	absolutely	silly	things	with
ast.NodeVisitor
ast.NodeTransformer
@jargnar
github.com/jargnar
Even	more	silly	things	with	
Flake8
@jargnar
github.com/jargnar
Hacking is a mindset, not a skill.
suhas.org

Hacking the Python AST