Presented at Storage Solution Meetup 2020-07-21

1. Introduction to OmniXtend
Wesley Terpstra
SiFive
wesley@sifive.com

2. CHIPS Alliance – who are we?
› Develop open source hardware RTL and open source software design tools
› A barrier free environment for collaboration
Wilson Snyder Olof Kindgren

3. What is OmniXtend?
3
An open, heterogeneous, data centric architecture
Main Memory can be shared equally by everyone

4. 4
OmniXtend vs. other memory-centric concepts
› Memory fabric may mean
different things to different
people:
› Page fault trap leading to RDMA request
(incurs context switch and SW overhead)
› Global address translation management in
SW, leading to LD/ST across global memory
fabric
› Coherence protocol scaled out, global page
management and no context switching
Context switch cost comparable
to memory access latency
No rewriting of software, scalable like the algorithm
This is OmniXtend
Require software/kernel support
and/or rewriting of applications

5. OmniXtend Overview
5
› OmniXtend is based off of TileLink
› TileLink is an open, coherent bus used to connect Cores with Memory
OmniXtend enhances TileLink
and serializes it over Ethernet

6. Example OmniXtend Implementation
6
L1 $ L1 $ L1 $ L1 $
Internal Cache Coherence Switch
Cache Coherence SerializerL2 cache
Coherence Manager 802.3 PHY
DRAM
OmniXtend

7. Example OmniXtend System Diagram
7
ML
Accelerator
802.3 Phy
Internal Cache Coherence Switch
Cache Coherence Serializer L2 cache
Coherence Manager802.3 PHY
DRAM
Programmable
(P4) Switch
TofinoTM
Ethernet
with Cache
Coherency
L1 $ L1 $ L1 $ L1 $
L1 $ L1 $ L1 $ L1 $
Internal Cache Coherence Switch
Cache Coherence SerializerL2 cache
Coherence Manager 802.3 PHY
DRAM
NVM
802.3 Phy
NVM
NVM – Main
Memory

8. TileLink Cache Coherence Requests and States
› Acquire—sent by master to obtain access permissions from slave.
› Probe—sent by slave to master to obtain remove access permissions.
› Release—sent by master to slave to relinquish all access permissions.
› Grant—sent by slave to master to grant access permissions.
› TileLink Cache Coherence supports four different primary cache states:
› M Exclusive modified (read/write access with obligation to write data back upon eviction from cache).
› E Exclusive clean (read/write access, data can be discarded upon eviction).
› S Shared clean (read-only access, data can be discarded upon eviction).
› I No access.
› There is no shared dirty state in TileLink v1.8

9. Github: github.com/chipsalliance/omnixtend
9

13. First “real world” measurements
RISC-V SoC with
OmniXtend running in
FPGA
Tofino Switch programmed
with P4 code to support
OmniXtend
1 1
2
1
2

14. 14
First “real world” OmniXtend 0.1.1 measurements
Global memory, no local DRAM like in RDMA
L1 DRAM
L
1
L
2
DRAM

15. SoC with OmniXtend support
› Interconnect Workgroup in CHIPS Alliance
› SiFive & Western Digital collaborating on SoC
› SoC architecture:
› Quad core, Linux capable U74 RV64GC cores
› 64KB private L1I$ and L1D$; 2MB shared L2$
› LPDDR4 controller
› FPGA validation in progress

16. Workgroups
Tools-WG:Cores-WG:
Chisel-WGInterconnect:
›Verilator
›FuseSOC
›Cocotb-verilator
›SweRVTM
›TileLink 2.0
›AIB (Chiplets)
›OmniXtendTM
Rocket SoC AI
accelerator
Join us!
• Share resources to lower the
cost of development
• Receive CPU/SoC designs,
complex IP blocks & SW
design tools
• Monetize your product with a
Redhat like model
• Have your brand associated
with a forward-thinking
organization
See more:
https://chipsalliance.org/join/

17. Summary
› OmniXtend is an open, data centric memory fabric
› Continued development ongoing
› Collaborate and join us to unleash memory architectures for next generation data
centers
17
See more: www.chipsalliance.org

18. OmniXtend Reference Design: Memory Fabric Innovation Platform
Allegro files now in: