PhD Slides

Checkpointing for Virtual Platforms and SystemC-TLM-2.0 Màrius Montón Directors: Dr. Mark Burton Dr. Jordi Carrabina December 2010 Univ. Autònoma Barcelona

Outline
Introduction
Objectives
Virtual Platforms and SystemC
Checkpointing for SystemC
Conclusions

Introduction
Objectives
Conclusions
Introduction

Introduction – Virtual Platforms
Functional models of physical platforms
Target SW unable to distinguish virtual platform from real HW
Run SW or OS on Virtual HW
Develop SW for non-existing HW
Simulate complex system interconnectivity

Introduction – Virtual Platforms

Virtual Platform Design Hardware Software Integration & Test time Engineering Resources © WindRiver (Virtutech)

Virtual Platform Design Hardware Software Integration & Test time Engineering Resources time Engineering Resources Resources Time to Market Hardware Software Integration & Test © WindRiver (Virtutech)

Generic Virtual Platform Diagram Virtual Platform ISS ISS VP Back-end functions Devices Devices Memories Target Drivers Boot code Target Operating System User User User

Introduction – SystemC & TLM-2
SystemC language for systems description (HW mainly)
OSCI simulator
TLM-2 standarizes communication model
Sockets to emulate any memory-mapped bus
De facto standard for system modeling

Introduction – Checkpointing
Ability to stop a execution and save it to disk
Retrieve the execution from a disk
Useful for simulations of large systems
avoid repetitive tasks (boot OS)
bug fixing (modify a simulation value & continue)
reverse execution
multi-site development teams

Introduction – VP Languages
Different Virtual Platforms uses different languages (C, C++, DLM, ...) and APIs
HW engineers know (or should) SystemC, not other languages
Different VP -> Rewrite own models
No interoperability

Introduction
Objectives
Conclusions
Objectives

Objectives
Add SystemC-TLM to any Virtual Platform
Different strategies to add two simulators
Add SystemC-TLM support to an open-sourced VP
Add checkpointing support to SystemC
C++ not checkpointable
Overcome limitations

Generic Virtual Platform Diagram Virtual Platform ISS ISS VP Back-end functions Devices Devices Memories Target Drivers Boot code Target Operating System User User User SystemC SystemC Devices Sync

Introduction
Objectives
Conclusions

Link together two simulators
Synchronization strategy
Generic bridge
Support for generic TLM-2 devices
LT, AT, DMI...

Link together two simulators
Virtual Platforms and SystemC Synch VP SystemC Virtual time

Virtual Platforms and SystemC Execution time Synch VP SystemC Virtual execution

QEMU-SC
Transition from RTL to TLM
Generic fabric bus (TLM) -> any architecture
QEMU simul. master and SystemC slave
QEMU manages simulation
Focus on few SystemC devices

QEMU-SC Linux Driver VP SystemC module SC_Bridge SC_Link TLM Socket Application

Synchronization
Only synchronize when needed ( sc_start() )
When SystemC devices are accessed
When pending event in current simulation time
SystemC events list
When I/O to/from SystemC device
Capture or notify all I/O in SystemC device
QuantumKeeper asks to
To adhere to standard

Communication
QEMU works with zero-delay communication
CPU accesses one device and the device responds immediately
Fit to LT devices
Need to manage AT devices
Special synchronization
Finish all protocol phases before return to VP

Communication - LT

Communication - AT

Synchronization
Manage external I/O from/to SC device
Devices use QEMU callbacks for I/O
Bridge knows when a I/O is performed
Synchronize simulators
Continue SystemC simulation when
VP time arrives to first SC event
Every quantum time (TLM-2)
Advance SystemC time until transaction ends

QBox
Change simulation manager
QEMU becomes simulator slave
SystemC manages simulation
QEMU is a TLM-2 Initiator module
Easy integration
Focus on many SystemC
models

QBox internal architecture

QBox complex example QEMU Wrapper NIC TILE TILE TILE QEMU NoC

Test & results
Different tests and examples
Validate implementation
Extract performance metrics
Results for performance
relatives to same system in native language
C in QEMU & Qbox

QEMU-SC Test System TLM Socket QEMU SystemC iDCT acc. SC_Bridge SC_Link DMI Pointer Reg. File IRQ IRQ Linux Driver MPEG accelerator irq_event MPEG-2 decoder

QEMU-SC Results
No acc.
No acc. w/ drivers
QEMU style 1 acc.
SystemC 1 acc.
QEMU style 2 acc.
SystemC 2 acc.
SystemC skel. (1 acc.)

QEMU-SC Results
No acc.
No acc. w/ drivers
QEMU style 1 acc.
SystemC 1 acc.
QEMU style 2 acc.
SystemC 2 acc.
Penalty 8% ~ 14%

Complex QBox system DMI Pointer TLM 2 Socket SignalSocket (IRQ) QEMU Wrapper GreenRouter TLM 2 Socket (Ethernet) Linux QEMU NE2000 NE2000 Testbench

QBox Test System TLM Socket SystemC iDCT acc. Reg. File IRQ SignalSocket (IRQ) QEMU Wrapper GreenRouter Linux Driver irq_event QEMU MPEG-2 decoder

QBox Results
No acc.
No acc. w/ drivers
QEMU style 1 acc.
SystemC 1 acc.
QEMU style 2 acc.
SystemC 2 acc.

QBox Results
No acc.
No acc. w/ drivers
QEMU style 1 acc.
SystemC 1 acc.
QEMU style 2 acc.
SystemC 2 acc.
Penalty ~ 100%!!!

QBox Test System TLM Socket SystemC iDCT acc. Reg. File IRQ SignalSocket (IRQ) QEMU Wrapper GreenRouter Linux Driver irq_event QEMU MPEG-2 decoder Penalty ~ 100%!!!

Introduction
Objectives
Conclusions

Add checkpointing capabilities to SystemC language
Two approaches
Process save and restore
State-variables save and restore

Process save checkpointing Save Restore O.S. HW O.S. HW VP VP

Process save checkpointing Save Restore O.S. HW O.S. HW HW VP VP

Process save checkpointing Save Restore O.S. HW O.S. HW HW O.S. VP VP

State save checkpointing Save Restore O.S. HW O.S. HW VP VP chkp_mng chkp_mng

State save checkpointing Save Restore O.S. HW O.S. HW HW VP VP chkp_mng chkp_mng

State save checkpointing Save Restore O.S. HW O.S. HW HW O.S. VP VP chkp_mng chkp_mng

Inspection to OSCI kernel to get/set:
Dynamic & static SC_METHOD events list
Non-destructive access
Text conversion for easy management
Channels (SC_FIFO<>, SC_MUTEX, etc.)
state
stored values

User code needs to give its internal state
Implicit
Designer use special types to publish state variables
gs_param<> from GreenSocs
Explicit
Designer use special callbacks to publish state variables
do_checkpoint()
do_restore()

Checkpoint manager
New class to OSCI kernel
Offers simulation checkpoint to external control
checkpoint()
restore()

chk_mng::checkpoint()
Collects data from:
Module state variables
Channels state and data
Pending events lists (for SC_METHOD)
Static sensibility
Dynamic sensibility
Converts al l data to text
Portable
Editable

chk_mng::restore()
Restores all data, converting from text to:
module state variables
channels state and data
events (for SC_METHOD)
Static sensibility
Dynamic sensibility
Simulation state ready to continue
sc_start() again

Restrictions to device modeling style
SC_METHOD only
SC_THREAD left to designer
Guidelines
To model FSM using SC_METHOD
Avoid usage of SC_THREAD (in literature)
Enough for VP modeling

Checkpointing for SystemC VP Device Reg. File
Member Variables
gs_param<>
Callbacks functions

Checkpointing for SystemC VP Device Reg. File
FSM
SC_THREAD to SC_METHOD
Member Variables
S3 S1 S2

Checkpointing for SystemC VP Device Reg. File IRQ
FSM
Member Variables
Events
from OSCI events lists
IRQ generator S3 S1 S2

Checkpointing for SystemC VP Device Reg. File IRQ
FSM
Member Variables
Events
I/O
Save Channels
IRQ generator I/O manag S3 S1 S2

Simulation time
No performance penalty on OSCI kernel
New functions only used when doing checkpointing
Main performance loss is due to overhead in gs_param<>
Mosts cases less than 1%
Other cases about a 50%

Checkpoint data size
Checkpoint data depends on
state variable
pending events
channels data
Usually order of ~kB vs ~GB other solutions
Easily portable
Fast saving to disk time
Reverse execution

Checkpoint data # OSCI data 55000 60000;Initiator.initiator_func; # FIFO “my_fifo” Target.my_fifo=14;Target.my_fifo=0; Target.my_fifo=0.5;Target.my_fifo=1; ... Target.my_fifo=6.5; # gs_params Initiator.initiator_func_fsm_param 0 Initiator.int_param 6 Target.data_gs_param {"0" "0.5" "1" "1.5" "2" "2.5" "0" "0" "0" "0"} Target.data_gs_paramB.0 0 Target.data_gs_paramB.1 0.5 Target.data_gs_paramB.2 1 Target.data_gs_paramB.3 1.5 Target.data_gs_paramB.4 2 ... Target.data_gs_paramB.9 0 OSCI kernel data Channels data User modules data

Introduction
Objectives
Conclusions
Conclusions

Joined SystemC to two Virtual Platforms
Tested two different strategies for joining two simulators
QEMU-SC
QBox
Minor performance impact SystemC bridge
Published as open-sourced projects www.greensocs.com
Conclusions – SystemC

Conclusions – SystemC Simulation Manager Penalty System for QEMU-SC Yes 10% SW QBox No 25~30% HW

Conclusions – Simics Virtutech
Add SystemC capabilities to a commercial VP
Close source -> new device manages SystemC
Same strategy than QEMU-SC

Conclusions – Simics Results
SystemC QT = 1 sec.
SystemC QT = 1 ms.
SystemC QT = 1 μ s.
Simics DML
Microbenchmark

Conclusions – Simics Results
SystemC QT = 1 sec.
SystemC QT = 1 ms.
SystemC QT = 1 μ s.
Simics DML
Microbenchmark Penalty ~ 5%

Conclusions – Industry
R&D from university to industry
Industry project involves PhD
Commercial tool: Simics
Other vendors using our work
Synopsys Innovator and QBox

Conclusions – Checkpointing
Added checkpointing to SystemC
Saving only necessary data, not the entire process
Focused on VP modeling
With limitations but enough for VP devices
No simulation speed penalty due to Checkpointing
Use of callbacks
Inspect OSCI kernel when checkpointing

Future Work (light)
Automagic configuration of QBox systems
Manage map-address in QEMU, Router, BIOS, etc.
Enhance QEMU time management
Hard to measure virtual time in QEMU
Add multiple instances from QEMU
Current QBox library allows one
Explore QEMU user mode
Simplified version, only ISS, run applications

Future Work (medium)
Merge SystemC methods into QEMU kernel
Remove OSCI simulator and write an API SystemC <-> QEMU
Merge QEMU functionality into OSCI kernel
Make QEMU a truly SystemC ISS
Both merges increase simulation speed due to removed synchronization

Future Work (hard)
Develop an automatic tool for SC_THREAD to SC_METHOD translation
Build a SystemC Virtual Machine
allows native checkpointing
performance issues

Thank you! Questions?

BACKUP SLIDES

Conclusions – Simics Virtutech

QEMU ecosystem

http://blogtlm.mariusmonton.com	270
http://mariusmonton.com	61
http://www.mariusmonton.com	8
http://static.slidesharecdn.com	1

PhD Slides

by Màrius Montón on Dec 20, 2010

Accessibility

Categories

Tags

Upload Details

Usage Rights

4 Embeds 340

Statistics

PhD Slides — Presentation Transcript