This document proposes a methodology to use Smalltalk for hardware/software co-design of FPGA applications. Smalltalk would be used as a high-level language to describe and test FPGA hardware designs. Critical parts of a Smalltalk robotic application are identified and projected onto an FPGA to gain performance. An experiment projects image processing from a camera onto an FPGA, achieving a processing time of 2.5ms compared to 73ms in Smalltalk. Future work includes developing the modeling methodology for hardware design using Smalltalk and integrating Pharo and FPGA interaction for software/hardware co-design.

1. TOWARD A METHODOLOGY TO TURN
SMALLTALK CODE INTO FPGA
LE Xuan Sang1,2
Loïc LAGADEC1, Luc FABRESSE2,
Jannik LAVAL2 and Noury BOURAQADI2
1 Lab-STICC, ENSTA Bretagne
2 Institut Mines-Telecom, Mines Douai

2. Robotic application requirements
Sensors Sensing Planification Control Actuators
2
• Robotics applications demand :
• Real-time + amount of data : Processing power
Example in vision : 20 images of 320x240/s ≃ 37Mbps.
• Flexibility to evolution and unforeseen change of hardware
(adding more device/sensor/actuators, improvement of circuit etc.)

3. • Simplicity & rich semantic
• High-level abstraction
• Smalltalk is also an IDE:
• Support Agile methodology
• Valuable ability of debugging & testing application
• BUT : Time-consuming for mass data processing
3
Smalltalk in Robotic Software Development

4. • Configurable hardware/chip
• Parallel processing
• FPGA circuits are designed using Hardware
Description Language (HDLs).
• BUT: HDL-based design is unsuitable for
hardware/software co-design
4
Field Programmable Gate Array (FPGA)

5. 5
FPGA HDL-BASED DESIGN
1. Hardware Design (HDL)
2. Write tech-bench and
perform Register Transfer
Level simulation (RTL)
3. Synthesis and code
generation
4. Deployment on FPGA
HW Design
Synthesis
and
code generation
Deployment on
FPGA
Test bench
RTL
simulation
Vendor
toolchain
dependent

6. • HDLs support specification up to Register
Transfer Level (RTL). But :
• Lack abstractions to implement high-level algorithms
• Debugging is really hard (waveforms)
• Not adequate for high-level modelling/programming
• Hardware dependency → limit reusability
6
FPGA HDL-BASED DESIGN

7. OBJECTIVE
Using Smalltalk in hardware/
software co-design
• Hardware : Smalltalk as a high-level
description and verification
language
!
• Software : Smalltalk robotic
application that interact with
FPGA
7
Smalltalk application
HW/SW Partitioning
SW HW
Interface
Pharo VM FPGA

8. 8
SMALLTALK FOR HARDWARE DESIGN
• Smalltalk-based design
• Hardware design abstraction layer
• RTL simulation
• Waveform tracing
• Unit Test
• Smalltalk-VHDL conversion
• Reusability & extensibility
• Vendor’s tool interaction
Pharo VM
Smalltalk code
Hardware design abstraction layer
Smalltalk to VHDL RTL Simulation
Vendor’s tools chain FPGA

9. SMALLTALK FOR SOFTWARE
PROGRAMMING ON FPGA (1)
• Standalone FPGA
• Operate independently with the host
system
• Communicate with the host system via
interfaces of communication : USB,
RS232, etc.
• Smalltalk application talk to FPGA via
Foreign Function Interfaces (such as
Native Boost)
• Problem : bandwidth bottleneck
9
Host System
Pharo VM
FPGA
FFI/ Language Binding
Interface
(USB,
UART, etc.)

10. SMALLTALK FOR SOFTWARE
PROGRAMMING ON FPGA (2)
• FPGA-ARM SoC/SoM
• FPGA System on Chip/Module : all in
one system
• Accelerate FPGA - ARM
communication →reduced bottleneck.
• Direct access to FPGA registers via
system library
• Smalltalk application talk to FPGA
registers via Register interaction
abstraction layer which uses FFI
10
Linux Embedded System
Pharo
Smalltalk application
Register interaction abstraction layer
FFI/ Language Binding
Drivers / Extension Processing Platform
FPGA
Architecture
ARM

11. EXPERIMENT
•Build a Pharo robotic
application
•Identify critical parts
•Project the critical parts
on FPGA
•Evaluation of performance
gain/loss
11

12. VIDEO
http://car.mines-douai.fr/2014/04/pharos-based-tracker-robot/
12

13. 13
EXPERIMENT
Camera RGB →HSV
HSV filtrer
Laser
sensor
Object
detector Motors
Sensor Smalltalk application Actuators

14. 14
EXPERIMENT
Camera RGB →HSV
HSV filtrer
Laser
sensor
Object
detector
Critical part !
Motors
Sensor Smalltalk application Actuators

15. 15
PERFORMANCE COMPARISON
RGB →HSV HSV filtrer
192x128, 32 bit
Pharo Smalltalk C(OpenCV) FPGA circuits
73 ms 1.5ms 2.5ms

16. 16
PERFORMANCE COMPARISON
RGB →HSV HSV filtrer
192x128, 32 bit
Pharo Smalltalk C(OpenCV) FPGA circuits
73 ms 1.5ms 2.5ms
! Bottleneck problem

17. CONCLUSION
• Future works :
• Modelling methodology
of hardware design using
Smalltalk.
• Pharo and FPGA
interaction.
• Software/hardware co-design
integration.
17
Smalltalk application
HW/SW Partitioning
SW HW
Interface
Pharo VM FPGA

18. TOWARD A METHODOLOGY TO TURN
SMALLTALK CODE INTO FPGA
LE Xuan Sang1,2
Loïc LAGADEC1, Luc FABRESSE2,
Jannik LAVAL2 and Noury BOURAQADI2
1 Lab-STICC, ENSTA Bretagne
2 Institut Mines-Telecom, Mines Douai