Tom Lyon is an expert in data storage technologies and the founder of DriveScale. NVMe over Fabrics allows high-speed SSDs to be disaggregated from servers over fast networks like Ethernet using the NVMe protocol. Composable infrastructure further disaggregates other resources like CPUs, memory, GPUs, and HDDs so they can be allocated on demand to workloads running in virtual machines. DriveScale provides a software platform that implements composable infrastructure to allow flexible resource allocation for data-intensive workloads typically seen in hyperscale environments.

1. NVMe over Fabrics and
Composable Infrastructure –
What do they mean for SDS?
Tom Lyon
DriveScale, Inc.

2. Who Is Tom Lyon?
§ Co-Founder and Chief Scientist, DriveScale
§ Distinguished Engineer, Cisco
§ Co-Founder and Chief Scientist, Nuova Systems [-> Cisco UCS]
§ CTO, Nokia Internet Communications
§ Founder & CTO, Ipsilon Networks [-> Nokia IP Routing & Security]
§ Distinguished Engineer, Sun Microsystems [employee #8]
§ Systems Architect, Amdahl Corp.
§ BSE in EECS, Princeton Univ.
§ Hacking on UNIX since 1975
§ UNIX, UTS, SunOS, NFS, SPARC, SunLink, IP Switching, ATM, UCS, VFIO …
2
©2018 DriveScale Inc. All Rights Reserved.

3. Why Software-Defined Storage?
§ SANs originally provided separation of compute & storage resources
– This is a very good thing – consolidation, simpler servers, …
§ But SANs now are $$$$$! and don’t scale
§ Centralized storage creates network bottlenecks
§ SAN hardware evolved to just being servers anyways
§ But:
– there is no right size for an SDS server/storage combo
– Every cluster needs different cpu/storage ratios
– CPUs and drives have different “natural” life-cycles
3
©2017 DriveScale Inc. All Rights Reserved.

4. NVMe enabled SSDs
§ Crazy fast SSDs:
– 500K IOPS
– 3GB/s
§ But:
– Bottleneck shifts to software
– Hard to buy small & fast SSDs
– 2TB and rising for good performance
§ NVMe itself is just an interface protocol enabling deep & parallel queues
4
©2017 DriveScale Inc. All Rights Reserved.

5. NVMe Over Fabrics
§ Substitute network for PCIe – keep NVMe basics
§ Fibre Channel or RDMA networks
– Ethernet RoCE is leader; iWarp has lost out; IB still niche
– NVMe over TCP coming because RDMA is “very difficult”
– Typically for 25Gb Ethernet and beyond
§ NVMe-o-F aspects:
– Data Plane – highly standardized
– Control Plane - some standardization
– Manageability aspects – don’t hold your breath
5
©2017 DriveScale Inc. All Rights Reserved.

6. NVMe-o-F Use Cases
§ 1. Just another protocol between servers & SAN
– Esp NVMe on FC
§ 2. Protocol embedded in backend network of storage system
– NVMe devices not visible to general network
§ 3. Protocol for direct dis-aggregation of drives from servers
– Target devices are real network citizens
– Discovery, security, monitoring, … are very important
6
©2017 DriveScale Inc. All Rights Reserved.

7. NVMe-o-F Targets
§ Of course, x86 servers can be NVMe targets
§ But lots of new SOC chips for more efficient hardware:
– Broadcom Stingray
– Mellanox Bluefield
– Kazan – pure hardware data path
§ Redundant controllers needed to avoid new SPOFs
§ So software needs to live on the server, not on the target
§ Servers become stateless, homogeneous, re-deployable
7
©2017 DriveScale Inc. All Rights Reserved.

8. Server Disaggregation
§ If SSDs outside the server make sense, why not
– Hard Drives
– GPUs?
– DRAM?
§ Disaggregation requires re-Composition
8
©2017 DriveScale Inc. All Rights Reserved.

9. Composable Infrastructure: Definition
9
©2018 DriveScale Inc. All Rights Reserved.

10. Composable Infrastructure: Large System Vendors
§ Cisco started it, but dropped the ball
§ HPE is promoting it with Synergy blade system
§ DellEMC entering market
– PowerEdge MX composable system
– “Kinetic Infrastructure”
§ WDC – OpenFlex Composable Infrastructure
10
©2018 DriveScale Inc. All Rights Reserved.

11. Composable Infrastructure: Startups
§ DriveScale: “Software Composable Infrastructure for Your Data Center”
– HDD, SSD, NVMe - Ethernet
§ Liqid: “On-Demand Composable Infrastructure”
– NVMe & GPU – External PCIe
§ H3Platform: NVMe and GPU disaggregation
– NVMe & GPU – External PCIe
§ Other NVMe over Fabrics:
– Apeiron, Attala, Excelero, E8 Storage, Pavilion Data, …
11
©2018 DriveScale Inc. All Rights Reserved.

12. ©2018 DriveScale Inc. All Rights Reserved.
What is DriveScale?
12
DAS
Composable
DriveScale was born to radically transform how modern
applications are deployed in the data center.
Seeing how hyperscale companies were reaping the
benefits of analytics and AI with massive data sets on their
scale-out infrastructure, DriveScale set out to make it
viable for any enterprise to deploy this infrastructure
on- premises and reap the same reward.
DriveScale designed a software platform that creates a
way to run these same data-intensive applications
unchanged with cloud scale and efficiency at a
significantly lower cost.

13. DriveScale Composability Platform
13
Bare Metal
DriveScale Composability Platform
PCIe Ethernet
iSCSI RoCE v2 NVME/TCP
Rack Servers
HDD SSD NVMe
Blade Servers
Modular
Servers
GenZ
NoSQL