This document summarizes several papers related to accelerating neural network inference using RISC-V processors. The papers propose and evaluate different approaches including implementing hardware accelerators integrated with RISC-V cores, extending the RISC-V instruction set with new instructions for common neural network operations, and designing specialized barrel processors for neural network workloads. Evaluations show these approaches can significantly accelerate neural network operations compared to software implementations on standard RISC-V cores.

1. VISpro V i s i o n P r o c e s s i n g L a b
Literature Summary

2. 2
List
1. Implementation of a RISC-V Processor with Hardware
Accelerator
2. A Reconfigurable Convolutional Neural Network-
Accelerated Coprocessor Based on RISC-V Instruction Set
3. Open-Source RISC-V Processor IP Cores for FPGAs –
Overview and Evaluation
4. Extending a Soft-Core RISC-V Processor to Accelerate
CNN Inference
5. XpulpNN: Accelerating Quantized Neural Networks on
RISC-V Processors Through ISA Extensions
6. RISC-V Barrel Processor for Deep Neural Network
Acceleration
7. Wip: The RISC-V Instruction Set Architecture Optimization
and Fixed-point Math Library Co-design
(BE Thesis: Chalmers uni 2019)
(MDPI Electronics 2020)
(MECO 2019)
(MDPI Electronics 2020)
(DATE 2020)
(ISCAS 2021)
(CODES+ISSS 2021)

3. 3
Implementation of a RISC-V Processor with Hardware Accelerator
■ Design HW accelerator for matrix operations and implement it together with a
RISC-V processor.
■ Support a set of operations.
■ Evaluated and compared against the RISC-V core doing in software.
■ Compared with other alternative ways of doing equivalent matrix calculations.

4. 4
A Reconfigurable Convolutional Neural Network-Accelerated Coprocessor
Based on RISC-V Instruction Set
■ Proposed CNN accelerator is connected to the E203 core as a coprocessor,
and the corresponding custom coprocessor instructions are designed.
■ Implement the four basic algorithms of convolution, pooling, ReLU, and matrix
addition.
■ The compiler environment and library functions are established for the designed
instructions, the hardware and software design of the coprocessor is completed,
and the implementation process of common algorithms in the IoT system on the
coprocessor is described.
■ Finally, resource consumption evaluation and performance analysis of the
coprocessor are completed on a Xilinx FPGA.
■ Acceleration of convolution has reached 6.27 times that achieved by standard
instruction set.

5. 5
Open-Source RISC-V Processor IP Cores for FPGAs – Overview and Evaluation
■
Comparison of open-source processor IP cores. Only open-source cores with free isa, 32-bit data width, available standard bus interface
synchronous design, tool- and technology independent code base, and existing compiler support were considered here. The first four cores are
listed for comparison only.

6. 6
Extending a Soft-Core RISC-V Processor to Accelerate CNN Inference
■ Investigate the potential of extending RISC-V ISA for accelerating the inference
of a CNN using in-pipeline hardware blocks and custom instructions.
■ Preliminary designs have a small footprint and minimal impact on maximum
core frequency.

7. 7
XpulpNN: Accelerating Quantized Neural Networks on RISC-V Processors
Through ISA Extensions
■ A set of extensions to the RISC-V ISA, aimed at boosting the energy efficiency
of low bit width QNNs on low-power microcontroller-class cores.
■ Integrate proposed extensions RTL description and GCC toolchain of an open-
source RISC-V processor targeting energy-efficiency.
■ Implement a full microcontroller system (based on open-source PULPissimo)
■ Extended processor features 11.1% area, 5.9% power overhead, and negligible
timing overhead compared to the original one;

8. 8
RISC-V Barrel Processor for Deep Neural Network Acceleration
■ Designed a barrel RISC-V processor to control NN processing elements (PEs).
■ For lower Area, Processor is an implementation of RV32I + custom CSRs for
controlling the PEs.
■ To demonstrate the capabilities of design, we computed a GEMV operation with
an input matrix size of 8 by 128 and a weight matrix size of 128 by 128 with two-
bit precision in only 16 clock cycles
(ISCAS 2021)
CSRs Control and Status Registers PE performs arbitrary precision GEneral Matrix Vector (GEMV) operations.

9. 9
Work-in-Progress: The RISC-V Instruction Set Architecture Optimization and
Fixed-point Math Library Co-design
■ Found: current RISC-V instruction set has room of improvement for lightweight
mathematical operations.
■ Introduce A set of custom RISC-V ISA designs for fixed-point math libraries
■ Perform performance evaluations to demonstrate the optimization results.

10. End