A proof-of-concept random test generator for RISC-V ISA is presented. The test generator uses constraint programming for specification of relationships between instructions and operands. Example scenarios to cover basic instruction randomization, data hazards, and non-sharing are presented. The tool integrates the RISC-V instruction set simulator to enable the generation of self-checking tests. The tool is implemented in Python using a freely-available constraint solver library. A summary of problems encountered is provided and next steps are discussed.

1. Practical RISC-V Random Test Generation using Constraint
Programming
Edmond A. Cote – http://www.edmondcote.com
ABSTRACT
A proof-of-concept random test generator for RISC-V ISA is
presented. The test generator uses constraint programming for
specification of relationships between instructions and operands.
Example scenarios to cover basic instruction randomization, data
hazards, and non-sharing are presented. The tool integrates the
RISC-V instruction set simulator to enable the generation of self-
checking tests. The tool is implemented in Python using a freely-
available constraint solver library. A summary of problems
encountered is provided and next steps are discussed.
BACKGROUND
Constrained random verification is an important and effective tool
for functional verification of microprocessors. Simple random
stimulus is not sufficient to verify a processor fully [1]. The use of
constraint programming for functional verification has
traditionally been limited to domain-specific languages such as e,
OpenVera, and SystemVerilog. In the context of a lean
semiconductor development model, the cost to acquire licenses of
such tools may be prohibitive. The freely-available SystemC
Verification Library offers limited supports for constraints. Python
verification frameworks exist such as cocotb [2] and Migen [3] but
do not offer support for constraint programming. This project
aims to explore the possibility of extending Python-based
hardware verification frameworks with support for constrained
random verification.
OBJECTIVES
• Learn about RISC-V architecture ahead of workshop
• Learn about low-level constraint programming
• Demonstrate proof-of-concept random test generator
• Complete interesting technical project outside of work
METHODS CONCLUSIONS AND NEXT STEPS
REFERENCES
• [1] Applying Constrained-Random Verification to
Microprocessors - Jason C. Chen, Synopsys Inc.
• [2] cocoTB documentation -
https://media.readthedocs.org/pdf/cocotb/latest/cocotb.pdf
• [3] Migen manual http://m-labs.hk/migen.pdf
• [4] Constraint Satisfaction Problem Wiki -
http://en.wikipedia.org/wiki/Constraint_satisfaction_problem
Constraint Programming
• Used Python language and python-constraint CSP solver library
• CSPs defined by triple {X,D,C} [4]:
X = {X1,…, Xn} is a set of variables
D = {D1,…,Dn} is a set of respective domain values
C = {C1,…,Cm} is a set of constraints
• Problem is input to tool using this paradigm
• MinConflictSolver in library provides a single valid random
solution
Examples
• Define 10 variables (instructions) and their domain
(set of all instructions)
i_var = ["i[{}]".format(x) for _ in range(10)]
i_dom = ['sw','lw','add','sub', …] # RISC-V instrs
csp.addVariables(i_var, i_dom)
• Constrain instruction 0 and 2 to be either load or store
csp.addConstraint(InSetConstraint(['sw','lw']),
['i[0]','i[2]'])
• Constrain for probable WAW hazard
rd_var = ["rd[{}]".format(x) for _ in range(10)]
rd_dom = ['ra','s0','s1', …] # RISC-V registers
csp.addVariables(rd_vars, rd_dom)
# At least 5 values of rd are in set
csp.addConstraint(SomeInSetConstraint(['s1'], 5), rd_var)
• Use of anonymous functions non shared addresses
csp.addVariable('addr', range(256)) # fixed domain issue
csp.addConstraint(lambda addr: # non sharing
addr > cpu*256/ncpu and addr <= (cpu+1)*256/ncpu))
Test Generation
• Generate random initial values for all registers
• Solve CSP; use csp.getSolution()
• Generate instruction sequence based on instruction type, save to
temporary file, and compile.
Self Checking
• Test generator instructs ISS to execute ELF
p = Popen('spike -d /path/to/elf'.split(' '),
stdin=subprocess.PIPE, stderr=subprocess.PIPE)
• ISS runs until magic number reached
until reg 0 gp DEADBEEF
• Test generator inserts reg commands to ISS and captures values of
registers
• Self checking code appended to test
• Generated self checking tests pass on RISC-V ISS
• Random stability checked by varying random seed
• Low level constraint programming is difficult, but
interesting exercise
• Fixed domains are problematic for integer space
• Access to alternative solvers can provide the entire
solution space. This presents interesting opportunities
for functional verification (e.g. analysis of state space,
creation of exhaustive tests, etc.).
• Use of Python subprocess module to communicate with
ISS was effective, but required patch for tool to accept
commands from stdin
• Explore Google or-tools and SystemC Verification Library
• Find a project to apply such a methodology to
Example test case :
RVTEST_CODE_BEGIN
// Randomly generated initial values (subset shown)
li s0, 0x8e73c
li s1, 0xa46d6
// Random instructions
or s0 , s0, s1
// Magic number register write
li gp, 0xDEADBEEF
// Generated self checking code, values from ISS
li gp, 0xAE7FE
bne s0, gp, fail
li gp, 0xa46d6
bne s1, gp, fail
TEST_PASSFAIL
RVTEST_CODE_END