This document provides an overview of a tutorial on getting started with RISC-V verification. The tutorial will cover issues in verifying RISC-V CPU designs, RISC-V compliance and its relationship to verification, reference model requirements, simulators for RISC-V CPUs, components for building verification testbenches, instruction stream generators, and a demonstration of a UVM testbench for a RISC-V core. It will also discuss running benchmarks and operating systems on RISC-V designs.

1. Information Classification: General
December 8-10, 2020 | Virtual Event
Getting Started with RISC-V Verification
Simon Davidmann, Lee Moore
Imperas Software
info@imperas.com
#RISCVSUMMIT
Tutorial
mp

2. Information Classification: General
Getting started with RISC-V Verification 2
RISC-V Verification Tutorial
Items to be covered, main takeaways
• Overview of the issues when verifying the design of a RISC-V CPU
• Status of RISC-V compliance and its relationship to verification
• Discussion of different reference model requirements
• Introduction to simulators for RISC-V CPUs
• Use of various virtual platform components in verification
• Main components of a verification testbench
• Introduction to instruction stream generators
• Pointers to some useful architectural validation test suites
• Understanding a complete UVM testbench via a detailed walk through demonstration

3. Information Classification: General
Getting started with RISC-V Verification 3
Introduction to Imperas
Our involvement with RISC-V
• Imperas develops simulators, tools, debuggers, modelling technology, and models to help embedded systems developers get
their software running…
• and hardware developers get their designs correct
• 12+ years, self funded, profitable, UK based, team with much EDA (simulators, verification), processors, and embedded
experience
• Staff worked in Arm, Mips, Tensilica, Cadence, Synopsys
• and in verification in EDA on development of Verilog, VCS, SystemVerilog, Verisity and their methodologies
• Started work with RISC-V in 2017 customers
• Contributed to RISC-V compliance since 2018
• Our RISC-V focus is CPU verification
• We provide configurable reference models, the fastest highest quality simulators, advanced development tools and the absolute
best solution for RISC-V hardware design verification
• 20+ of the leading RISC-V CPU developers use and rely on Imperas solutions
• www.Imperas.com
• www.OVPworld.org

4. Information Classification: General
Getting started with RISC-V Verification 4
Agenda
• Imperas Introduction – our experience in verification
• CPU verification – the challenges and process
• RISC-V and compliance (Sail & riscvOVPsim reference models)
• Required technologies
• Configuring reference models
• Simulators and other tools
• RISC-V Architectural Validation Test Suites
• Instruction Stream Generators
• Verification IP & setting up a testbench
• Walkthrough of OpenHW CORE-V-VERIF UVM testbench for CV32E40P RISC-V core
• Running benchmarks and operating systems

5. Information Classification: General
Getting started with RISC-V Verification 5
RISC-V Presents New Challenges
• RISC-V is a new ISA – an open standard ISA
• Managed by the non-profit RISC-V International (riscv.org) [a Linux foundation project]
• This means any designer can build a processor implementation
• (Oct 2020 – there are over 100 RTL designs including open source and proprietary)
• Traditionally (for CPUs)
• processor IP comes from, and is maintained by the ISA owner
• is single sourced
• comes fully verified and compliant to that specific ISA (Arm tests for 10**15 instructions)
• ‘all’ user needs to do is to verify using integration tests
• processor verification has no public ‘standard’ approach and there are few available public tools
• The RISC-V industry / eco-system needs to adopt its best practices for hardware verification and adapt them to processor
verification
• There is a lot to learn for RISC-V…

6. Information Classification: General
Getting started with RISC-V Verification 6
The CPU HW DV Process
• Source/build/hire/allocate the expert team to do the work…
• Develop Verification Plan
• Focus on what needs to be verified – develop measurement metrics
• Determine EDA tools and models and methods to be used
• Simulation: open source, commercial, bespoke, SystemVerilog, UVM, FPGA, Emulation
• Formal…
• Get tools, testbenches, models in place
• Obtain tests, create tests
• Generate huge number of (pseudo random) tests
• Verify
• … continue while measuring until functional and code coverage metrics reached
• Benchmark, soak and integration testing

7. Information Classification: General
Getting started with RISC-V Verification 7
Agenda
• Imperas Introduction – our experience in verification
• CPU verification – the challenges and process
• RISC-V compliance (Sail & riscvOVPsim reference models)
• Required technologies
• Configuring reference models
• Simulators and other tools
• RISC-V Architectural Validation Test Suites
• Instruction Stream Generators
• Verification IP & setting up a testbench
• Walkthrough of OpenHW CORE-V-VERIF UVM testbench for CV32E40P RISC-V core
• Running benchmarks and operating systems

8. Information Classification: General
Getting started with RISC-V Verification 8
Compliance and sail walkthrough
• Compliance charter

9. Information Classification: General
Getting started with RISC-V Verification 9
Agenda
• Imperas Introduction – our experience in verification
• CPU verification – the challenges and process
• RISC-V and compliance (Sail & riscvOVPsim reference models)
• Required technologies
• Configuring reference models
• Simulators and other tools
• RISC-V Architectural Validation Test Suites
• Instruction Stream Generators
• Verification IP & setting up a testbench
• Walkthrough of OpenHW CORE-V-VERIF UVM testbench for CV32E40P RISC-V core
• Running benchmarks and operating systems

10. Information Classification: General
Getting started with RISC-V Verification 10
Important Technology needed
• Reference model of RISC-V CPU
• Including all required ISA extensions and modes (e.g. Hypervisor, Debug)
• With extensibility for adding custom instructions, CSRs, accelerators
• With full configurability of ISA specification including subsets, options, and versions
• Range of simulators from free to commercial
• With verification, analysis, profiling, debug, and software development capabilities
• Collection of models to build simulation platform / testbench for device
• Other CPUs, accelerators, behavioural components
• Ability to easily add own models and extend/configure existing models
• Verification IP
• CPU reference model encapsulation, integration into SystemVerilog
• Micro-architectural synchronisation for multi-core, async interrupts, debug mode
• Tools to create functional coverage code
• Instruction Stream Generator to create (random) test suites
• Test suites covering required ISA and ISA versions and device configuration

11. Information Classification: General
Getting started with RISC-V Verification 11
Agenda
• Imperas Introduction – our experience in verification
• CPU verification – the challenges and process
• RISC-V and compliance (Sail & riscvOVPsim reference models)
• Required technologies
• Reference models
• Simulators and other tools
• RISC-V Architectural Validation Test Suites
• Instruction Stream Generators
• Verification IP & setting up a testbench
• Walkthrough of OpenHW CORE-V-VERIF UVM testbench for CV32E40P RISC-V core
• Running benchmarks and operating systems

12. Information Classification: General
Getting started with RISC-V Verification 12
Reference models and accuracy
• Reference models are different to Instruction Set Simulators (ISS)
• ISS is typically pretty simplistic and relatively inflexible and is a standalone executable (e.g. graduate student project)
• Good for some use models, but not much use for detailed advanced verification (like encapsulated step/compare)
• A useful reference model for Hardware Design Verification is very sophisticated
• Must accurately implement full ISA specification
• Must be configurable to all of the RISC-V ISA spec (Imperas reference model has 130+ detailed overrides)
• Must be configurable to all different versions of ISA specs (each subsection selectable to a different version)
• e.g. User 2.3, Priv 1.10, Debug 13.2, Vector 0.8, Bitmanip 0.93, Crypto 0.7.1 etc…
• Must include methodology and have been designed to be extendable for added instructions, state, modes, asynch. behavior
• Having source is not enough – as users do not want to modify & maintain base model, just their extension
• Ideally should be used previously by many users as a reference validating their RTL
• Must be designed to work tightly coupled into 3rd party simulation environments
• Standalone executables running independently or controlled with threads / sockets is never reliable / good enough
• Needs to be a linkable object that can seamlessly become part of 3rd party simulation. Co-simulation is not good enough
• For example Imperas models can be linked to simulators using C, C++, SystemC, SystemVerilog, proprietary, …
• Must be able to
• Control (and set model into any state and control every operation incrementally)
• Observe (all internal state and operational sequences)
• Debug (have triggers on anything, including source code / state, and be able to set state / values and send asynch. events)
• Extend (add user behaviors and state to explore)

13. Information Classification: General
Getting started with RISC-V Verification 13
Agenda
• Imperas Introduction – our experience in verification
• CPU verification – the challenges and process
• RISC-V and compliance (Sail & riscvOVPsim reference models)
• Required technologies
• Configuring reference models
• Simulators and other tools
• RISC-V Architectural Validation Test Suites
• Instruction Stream Generators
• Verification IP & setting up a testbench
• Walkthrough of OpenHW CORE-V-VERIF UVM testbench for CV32E40P RISC-V core
• Running benchmarks and operating systems

14. Information Classification: General
Getting started with RISC-V Verification 14
Simulators and other tools
• Free GitHub riscvOVPsim – for simple runs and compliance checking
• Free OVPworld riscvOVPsimPlus - for test development and debug and some initial verification
• Imperas commercial M*SIM
• Imperas Multi-Processor Debugger
• Imperas Verification, Analysis, and Profiling (VAP) tools
• Including OS aware
• Extending tools and creating your own tools

15. Information Classification: General
Getting started with RISC-V Verification 15
Free tool walkthough

16. Information Classification: General
Getting started with RISC-V Verification 16
Simulators and other tools
• Free GitHub riscvOVPsim – for simple runs and compliance checking
• Free OVPworld riscvOVPsimPlus - for test development and debug and some initial verification
• Imperas commercial M*SIM
• Imperas Multi-Processor Debugger
• Imperas Verification, Analysis, and Profiling (VAP) tools
• Including OS aware
• Extending tools and creating your own tools

17. Information Classification: General
Getting started with RISC-V Verification 17
Web browse of Imperas tools

18. Information Classification: General
Getting started with RISC-V Verification 18
Simulators and other tools
• Free GitHub riscvOVPsim – for simple runs and compliance checking
• Free OVPworld riscvOVPsimPlus - for test development and debug and some initial verification
• Imperas commercial M*SIM
• Imperas Multi-Processor Debugger
• Imperas Verification, Analysis, and Profiling (VAP) tools
• Including OS aware
• Extending tools and creating your own tools

19. Information Classification: General
Getting started with RISC-V Verification 19
Agenda
• Imperas Introduction – our experience in verification
• CPU verification – the challenges and process
• RISC-V and compliance (Sail & riscvOVPsim reference models)
• Required technologies
• Configuring reference models
• Simulators and other tools
• RISC-V Architectural Validation Test Suites
• Instruction Stream Generators
• Verification IP & setting up a testbench
• Walkthrough of OpenHW CORE-V-VERIF UVM testbench for CV32E40P RISC-V core
• Running benchmarks and operating systems

20. Information Classification: General
Getting started with RISC-V Verification 20
RISC-V Architectural Validation & Test Suites
Directed Tests
• Collections of ‘bare metal’ test programs that utilize specific instructions in specific ways to stimulate DUT architecture
• Purpose is to explore behaviors and architectural choices for DUT
• Focus is common instruction usage, various values, and corner cases
• Often packaged targeting ISA subsets, e.g. Integer, Mathematics, Vectors, Interrupts, Debug mode, …
• Vary from simplistic to very thorough, from general ISA specification through bespoke detailed microarchitectural focus
• When evaluating effectiveness need to consider configurability, functional coverage, quality
• Configurability needed for optional ISA choices, address space relocation, …
• Functional coverage indicates focus and completeness
• Quality needs evaluating to ensure that test in fact does what is intended, and can measure results
• Validation achieved by being self checking, requiring signature compare, or requiring reference model compare
Examples
• Berkeley – original self checking tests, evolving: github.com/riscv/riscv-tests
• RISC-V Compliance Group - initial base ISA tests – phase 1 (phase 2 under development Dec. 2020), signature based
• Imperas
• Extended base ISA validation tests (RV32/64 I,M,C)
• Initial port of B-bitmanip, K-crypto (scalar) [these are a work in progress by the voluntary RISC-V Task Groups]
• V-vector tests (configurable through full spec options and versions from 0.8 -> 1.0draft) [7,00 tests]
• Bremen Fuzz Tests – explore legal and illegal ISA spec conditions [8,380 legal, 13,540 testing illegal conditions]

21. Information Classification: General

22. Information Classification: General
Getting started with RISC-V Verification 22
Agenda
• Imperas Introduction – our experience in verification
• CPU verification – the challenges and process
• RISC-V and compliance (Sail & riscvOVPsim reference models)
• Required technologies
• Configuring reference models
• Simulators and other tools
• RISC-V Architectural Validation Test Suites
• Instruction Stream Generators
• Verification IP & setting up a testbench
• Walkthrough of OpenHW CORE-V-VERIF UVM testbench for CV32E40P RISC-V core
• Running benchmarks and operating systems

23. Information Classification: General
Getting started with RISC-V Verification 23
Constrained Random Instruction Stream
Test Generators (ISG)
Introduction & basic operation
randomization
constraints
DUT
specification
configuration
instructions
to be targeted
reference
model
configuration
test list
configuration
TESTS

24. Information Classification: General
Getting started with RISC-V Verification 24
Constrained Random Instruction Stream
Test Generators (ISG)
• From Google presentation 2019 RISC-V Summit

25. Information Classification: General
Getting started with RISC-V Verification 25
Constrained Random Instruction Stream
Test Generators (ISG)
Example Generation Flow
• From Google presentation 2019 RISC-V Summit

26. Information Classification: General
Getting started with RISC-V Verification 26
Constrained Random Instruction Stream
Test Generators (ISG)
• Introduction & basic operation
• Open Source
• Berkeley (Scala) riscv-torture (early)
• Google (SystemVerilog) RISCV-DV
• Chips Alliance/PerfectVIPs (python) RISCV-DV
• OpenHW (C++) FORCE-RISCV
• ISP-RAS (nML, Ruby) MicroTESK
• Commercial
• Valtrix STING

27. Information Classification: General
Getting started with RISC-V Verification 27
Walkthrough of ISG
• d

28. Information Classification: General
Getting started with RISC-V Verification 28
Agenda
• Imperas Introduction – our experience in verification
• CPU verification – the challenges and process
• RISC-V and compliance (Sail & riscvOVPsim reference models)
• Required technologies
• Configuring reference models
• Simulators and other tools
• RISC-V Architectural Validation Test Suites
• Instruction Stream Generators
• Verification IP & setting up a testbench
• Walkthrough of OpenHW CORE-V-VERIF UVM testbench for CV32E40P RISC-V core
• Running benchmarks and operating systems

29. Information Classification: General
Getting started with RISC-V Verification 29
Verification IP & Setting up a testbench
Agenda
• Encapsulation of reference model
• Functional coverage
• Imperas simulators
• SystemVerilog (Imperas can generate and provide source for SystemVerilog covergroups / coverpoints of ISA extensions)
• Verification IP blocks needed
• Core Introspection / tracing
• Synchronizing DUT and reference model
• Comparing state of DUT and reference model

30. Information Classification: General
Getting started with RISC-V Verification 30
Example Step / Compare testbench
Unlike post simulation trace of signature compare – this testbench configuration checks each instruction as it goes – more effective
Imperas Reference
(OVP model of cpu)
SystemVerilog module
DUT: RISC-V RTL
(cpu)
DUT
memory
Ref
memory
RISC-V.S /
RISC-V.c
GCC/
LLVM/
ASM
RISC-V.elf results.log
SystemVerilog Testbench
Step
Synch
Tracer
Compare

31. Information Classification: General
Getting started with RISC-V Verification 31
Example Step / Compare testbench
Control Blocks
Comparator
• Consumer of ‘tracer’ data
• Requires consideration at start of project
Compare
Step
Synch
Tracer
Synchronizer
• DUT is in control
• Top level testbench controller
• scheduling of both DUT and Reference model
• Tracks instruction retires
• Monitors internals of RTL for async. Events
• Controls and synchronizes between DUT and
reference model
Tracer / Introspector
• In the RTL
• Developed by DUT developers
• Created in the RTL with the purpose of DV in mind
• Must be correct be every instruction boundary
• Provides data up to the testbench

32. Information Classification: General
Getting started with RISC-V Verification 32
Agenda
• Imperas Introduction – our experience in verification
• CPU verification – the challenges and process
• RISC-V and compliance (Sail & riscvOVPsim reference models)
• Required technologies
• Configuring reference models
• Simulators and other tools
• RISC-V Architectural Validation Test Suites
• Instruction Stream Generators
• Verification IP & setting up a testbench
• Walkthrough of OpenHW CORE-V-VERIF UVM testbench for CV32E40P RISC-V core
• Running benchmarks and operating systems
• Summary

33. Information Classification: General
• Step and compare of the Imperas architecturally derived cv32e40p, against the SystemVerilog RTL microarchitectural
representation
Getting started with RISC-V Verification 33
Demo, Step and Compare
Imperas Reference
(OVP model of cpu)
SystemVerilog module
DUT: RISC-V RTL
(cpu)
DUT
memory
Ref
memory
RISC-V.S /
RISC-V.c
GCC/
LLVM/
ASM
RISC-V.elf results.log
Step
Synch
Tracer
Compare

34. Information Classification: General
• Running The Continuous Integration Check-In tests
• ci_check
Getting started with RISC-V Verification 34
Demonstration :
cv32e40p Step/Compare UVM Testbench

35. Information Classification: General

36. Information Classification: General
• Debug environment for Applications running in the UVM Testbench
• Software View
• Reference Model
• Hardware View
• RTL
• Reference Model
Demonstration :
cv32e40p Step/Compare UVM Testbench
Getting started with RISC-V Verification 36

37. Information Classification: General
SV - DPI Wrapper
make test SIM=<simulator> TEST=<test_case>
Instruction Set Simulator
CV RTL
Functional Coverage
SVA
GCC
compile
elf2hex
testcase.S
mm_ram
dp_ram
vp
Compiler Toolchain
Step
&
Compare
Compare all CSRs,
GPRs and PC
Demonstration :
cv32e40p Step/Compare UVM Testbench
Getting started with RISC-V Verification 37
Golden ISS Reference Model

38. Information Classification: General

39. Information Classification: General

40. Information Classification: General

41. Information Classification: General
• Step and compare of the Imperas architecturally derived cv32e40p, against the SystemVerilog RTL microarchitectural
representation
Getting started with RISC-V Verification 41
Imperas Reference
(OVP model of cpu)
SystemVerilog module
DUT: RISC-V RTL
(cpu)
DUT
memory
Ref
memory
RISC-V.S /
RISC-V.c
GCC/
LLVM/
ASM
RISC-V.elf results.log
Step
Synch
Tracer
Compare
Imperas Step and Compare

42. Information Classification: General
Getting started with RISC-V Verification 42
Agenda
• Imperas Introduction – our experience in verification
• CPU verification – the challenges and process
• RISC-V and compliance (Sail & riscvOVPsim reference models)
• Required technologies
• Configuring reference models
• Simulators and other tools
• RISC-V Architectural Validation Test Suites
• Instruction Stream Generators
• Verification IP & setting up a testbench
• Walkthrough of OpenHW CORE-V-VERIF UVM testbench for CV32E40P RISC-V core
• Running benchmarks and operating systems
• Summary

43. Information Classification: General
Getting started with RISC-V Verification 43
Agenda
• Imperas Introduction – our experience in verification
• CPU verification – the challenges and process
• RISC-V and compliance (Sail & riscvOVPsim reference models)
• Required technologies
• Configuring reference models
• Simulators and other tools
• RISC-V Architectural Validation Test Suites
• Instruction Stream Generators
• Verification IP & setting up a testbench
• Walkthrough of OpenHW CORE-V-VERIF UVM testbench for CV32E40P RISC-V core
• Running benchmarks and operating systems
• Summary

44. Information Classification: General

45. Information Classification: General
Getting started with RISC-V Verification 45
Summary / Wrap up
Topics introduced and explored
• Methodologies, flows and challenges with RISC-V CPU verification
• Reference models and introduced the Imperas configurable RISC-V Envelope model
• Free simulators, test suites, and compliance
• RISC-V Instruction Stream Generators
• Commercial simulators and test suites
• Discussed advanced verification tools and methodologies
• Explored UVM testbench with Imperas step/compare reference simulator and OpenHW RTL core

46. Information Classification: General
Getting started with RISC-V Verification 46
Imperas RISC-V Verification Solutions
If you want to explore verification of RISC-V cores further, then please contact Imperas: info@imperas.com
Methodology
Collaboration with customers, vendor ecosystem
Tools
Leading simulation, debug, software
development tools.
Advanced HW DV tools
Models
250+ CPU models
300+ peripheral models
50+ platforms
Golden reference models for HW DV
Training
Imperas & partners
Customer customized agendas
Resources
Imperas & partners
Model development
Tool development

47. Information Classification: General
Getting started with RISC-V Verification 47
For more information:
Imperas / OVP
• www.Imperas.com
• www.OVPworld.org
Tests
• https://github.com/riscv-ovpsim/imperas-riscv-tests
• http://systemc-verification.org/risc-v (Bremen fuzz tests)
• https://github.com/riscv/riscv-tests
• https://github.com/riscv/riscv-compliance
Instruction Test Generators
• www.valtrix.in/sting
• https://github.com/google/riscv-dv
• https://github.com/openhwgroup/force-riscv
• https://forge.ispras.ru/projects/microtesk
Testbenches
• https://github.com/openhwgroup/core-v-verif
Reference Models / Simulators
• https://github.com/riscv-ovpsim
• www.OVPworld.org/riscv-ovpsim-plus
• https://github.com/rems-project/sail-riscv
Videos
• www.imperas.com/imperas-videos
• www.OVPworld.org/demosandvideos
Contact info@imperas.com for more information:
• reference models
• simulators, tools
• validation tests
• CPU verification
• software/OS bring up