This document discusses simulating a CPU using Python. It describes the Epiphany manycore chip architecture, which has 16-64 simple cores that each have 32kB of RAM. The Revelation simulator of the Epiphany was written in RPython (statically typed Python) and has around 1,000 lines of code. It uses the Pydgin framework to generate instruction set simulators from simple architecture descriptions. The document outlines how to test the simulator by writing unit tests against individual instructions and integration tests against compiled ELF files, and compares traces with an existing reference simulator to check for differences.

1. Simulating a CPU
with Python
or: surprising programs
you might have thought
were better written in C
Sarah Mount @snim2

2. Data from Stanford VLSI Group CPUdb

3. Left: late 1980s INMOS Transputer (image © Konstantin
Lanzet - CPU collection. Licensed under CC BY 3.0 via
Wikimedia Commons)
Right: a 2011 Parallella board, the spiritual successor to
the INMOS Transputer (image © Adapteva Inc)

4. Adapteva Epiphany III chip
A new style of energy-efficient manycore
computer?
● 16 / 64 cores, theoretical maximum of 4095 cores
● 1 GHz
● <2 Watts
● Whole Parallella board: 5 Watts
● 19 GFLOPS / Watt
● 32 kB RAM / core

5. Packing code into that small space
● Most 32-bit Epiphany core instructions have
a 16-bit equivalent
● Compiler chooses the 16-bit instruction
whenever possible

6. Derek Lockhart, Berkin Ilbeyi, and Christopher Batten. (2015) Pydgin: Generating Fast Instruction Set
Simulators from Simple Architecture Descriptions with Meta-Tracing JIT Compilers. IEEE International
Symposium on Performance Analysis of Systems and Software (ISPASS).

7. Revelation: a new Epiphany sim
● Written using the Pydgin framework in
RPython (statically typed Python)
● All instructions implemented except some
multicore
● Small code base: ~1k Source Lines of Code
● http://www.revelation-sim.org/
Still a work in progress though (i.e. don’t expect
it to work!)

8. ● Clone pypy:
$ hg clone https://bitbucket.org/pypy/pypy
● Clone Pydgin:
$ git clone ...github.com/cornell-brg/pydgin.git
● Put Pydgin on your PYTHONPATH:
$ export PYTHONPATH=${PYTHONPATH}:.../pydgin/:
● Write a simple simulator!
Write your own Pydgin simulator!

9. Instruction model

10. Decode and implementation (add32)

11. Machine model

12. Simulator

13. Some oddities about the Epiphany
● The chip has a flat memory map. Everything
(registers, flags) is a location in RAM.
● Each core has 32kB local memory,
addressed as 0x0…0xFFFFF
● The same memory can be globally
addressed, and accessed by other cores, as
(COREID << 20) | local address, e.g.
0xF0408 (the PC) is globally 0x808F0408
for chip 0x808 (on row 32, column 8 of the
chip.

14. More oddities
● There are many instructions, almost all of
which have 16-bit and 32-bit versions. This
could lead to duplicate code.
● Thankfully, we can use closures to create all
the versions of an instruction implementation
that we need in one go:

15. Implementation of JUMP instructions

16. Test, compile, iterate
To compile your simulator (with a JIT):
$ .../rpython/bin/rpython -Ojit sim.py
To compile with the ability to print an instruction
trace:
$ .../rpython/bin/rpython -Ojit sim.py
--debug

17. Test, compile, iterate
To run your simulator:
$ ./pydgin-sim-jit --help
Then iterate!

18. But you said TEST, compile, iterate!
● Compiling is slow, so you want to avoid
compiling every time you make a change to
the code base
● That means you want to test your RPython
code dynamically (untranslated)
● Because RPython is a subset of Python, you
can use Python test tools, such as py.test,
which means you can unit tests your
untranslated simulator

19. Unit tests
● “Proper” unit tests, which test small program
units (such as a single instruction)
● The Epiphany Architecture Reference
Manual has many examples (1 or more for
each instruction). Each of these is written
into an assembler file and used as the basis
of a (more complex) unit test.
● Currently >500 unit and integration tests in
Revelation, testing a very tiny part of the
state space!

20. A “basic” unit test

21. An assembler file

22. A unit test to go with the assembler

23. Integration tests
● Integration tests typically load and execute a
full ELF file (compiled from C code). They
then check one or more of:
○ The machine state
○ The number of instructions executed
○ Output on STDOUT
● ELF files need to be cross-compiled with the
Epiphany SDK (e-gcc, etc.) and so are
checked into version control

24. Hello, world!

25. “Hello, world!” the integration test

26. Using a test oracle
● A test oracle is a source of truth, for example
an existing, complete simulator
● Adapteva e-sim:
○ https://github.com/adapteva/epiphany-cgen
○ https://github.com/adapteva/epiphany-gdb

27. e-sim trace file (4.5MB, 46265 lines)
0x000000 b.l 0x0000000000000100 - pc <-
0x100
0x000100 mov.l r3,0x138 - registers <-
0x138
0x000104 movt r3,0x0 - registers <-
0x138
0x000108 jalr r3 - registers <-
0x10a, pc <- 0x138
0x000138 --- _epiphany_star mov.l sp,0x7ff0 - registers <-
0x7ff0
0x00013c --- _epiphany_star movt sp,0x0 - registers <-
0x7ff0
0x000140 --- _epiphany_star mov.l fp,0x0 - registers <-
0x0
0x000144 --- _epiphany_star mov.l r0,0x58 - registers <-
0x58
0x000148 --- _epiphany_star movt r0,0x0 - registers <-
0x58

28. Revelation trace (6.7MB, 46096 lines)
0 000080e8 bcond32 0 AN=False AZ=False
AC=False AV=False AVS=False BN=False BZ=False BIS=False
BUS=False BV=False BVS=False
100 0012670b movimm32 1 :: WR.RF[3 ] = 00000138
104 1002600b movtimm32 2 :: RD.RF[3 ] = 00000138
:: WR.RF[3 ] = 00000138
108 00000d52 jalr16 3 :: WR.RF[14] = 0000010a
:: RD.RF[3 ] = 00000138
138 27f2be0b movimm32 4 :: WR.RF[13] = 00007ff0
13c 3002a00b movtimm32 5 :: RD.RF[13] = 00007ff0
:: WR.RF[13] = 00007ff0
140 2002e00b movimm32 6 :: WR.RF[15] = 00000000
144 00020b0b movimm32 7 :: WR.RF[0 ] = 00000058
148 1002000b movtimm32 8 :: RD.RF[0 ] = 00000058
:: WR.RF[0 ] = 00000058

29. Example trace diff (4.MB, 51845 lines)
Semantics of instruction at 0x44e differ:
Revelation: 44e 2f8b0417 fsub16 35097 :: RD.RF[0
] = 3f800000 :: RD.RF[1 ] = 5f800000 :: RD.RF[0 ] = 3f800000 ::
WR.RF[0 ] = 5f800000 :: RD.RF[0 ] = 5f800000 :: RD.RF[0 ] =
5f800000 :: RD.RF[1 ] = 5f800000 :: RD.RF[0 ] = 5f800000
AN=False AZ=False AC=False AV=False AVS=True BN=False BZ=False
BIS=False BUS=False BV=False BVS=False
e-sim: 0x00044e --- main fsub r0,r1,r0 -
bzbit <- 0x0, bnbit <- 0x0, bvbit <- 0x0, bvsbit <- 0x0, busbit
<- 0x0, bisbit <- 0x0, registers <- 0x5f7fffff
Registers differ. Revelation: rf<-0x5f800000 e-sim: rf<-0x5f7fffff
Semantics of instruction at 0x8e006d1e differ:
Revelation: 8e006d4c 0652268b movimm32 35845 :: WR.RF[1
] = 00006534
e-sim: 0x8e006d1e mov.l r1,0x6534
- registers <- 0x6534
Program counters differ. Revelation: 0x8e006d4c, e-sim: 0x8e006d1e

30. A specious difference between traces
Semantics of instruction at 0x8e009980 differ:
Revelation: 8e009980 6dda1fe2 trap16 990 :: RD.RF[3
] = 00000005 :: RD.RF[0 ] = 00000001 :: RD.RF[1 ] = 8f000008 ::
RD.RF[2 ] = 00000001 :: WR.RF[0 ] = 000
00001 :: WR.RF[3 ] = 00000000
e-sim: 0x8e009980 trap 0x7
- registers <- 0x1
Registers differ. Revelation: rf<-0x0 rf<-0x1 e-sim: rf<-0x1

31. A simple logger

32. Things I haven’t tried but might...
● American Fuzzy Lop:
○ http://lcamtuf.coredump.cx/afl/
● Hypothesis
○ http://hypothesis.works/
● ...any other form of automated testing

33. Thank you.

34. Opcode factory

35. New state with given contents

36. State checker

37. Using the state checker

38. Mock simulator

39. Using the mock simulator