The Nitro system is a rich collection of building block technologies that include hardware offload and security components built on AWS. It is powering the next generation of EC2 instances with an ever-broadening selection of compute, storage, memory, and networking options. In this session, we deep dive into the Nitro system, explore its design and architecture, discover how it enables innovative new EC2 instances, and understand how it has made the seemingly impossible, possible.

2. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Powering Next-Gen EC2 Instances
Deep Dive into the Nitro System
Anthony Liguori
Sr. Principal Engineer
EC2
C M P 3 0 3

3. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Agenda
Nitro Overview
Nitro Security Chip Deep Dive
Recent Launches
What’s Next

4. After ten years of Amazon Elastic
Compute Cloud (Amazon EC2), if we
applied all of our learnings, what
would a hypervisor look like?

5. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Nitro: One year later
Launched in November 2017
In development since 2013
All new launches use Nitro
Purpose-built hardware/software
Hypervisor built for AWS
AWS Nitro

6. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Nitro in three parts
Nitro Cards Nitro Security Chip Nitro Hypervisor
VPC Networking
Amazon Elastic Block Store
(Amazon EBS)
Instance Storage
System Controller
Integrated into motherboard
Protects hardware resources
Hardware Root of Trust
Lightweight hypervisor
Memory and CPU allocation
Bare Metal-like performance

7. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Nitro Cards
Nitro
Control
Instance
Storage
ENA PCIe Controller
VPC Data Plane
NVMe PCIe Controller
EBS Data Plane
NVMe PCIe Controller
Transparent Encryption
System Control
Root of Trust

8. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Nitro Card for VPC
ENA Controller
Drivers available for all major operating systems
Independent of fabric
VPC Data Plane
Encapsulation
Security Groups
Limiters
Routing

9. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Nitro Card for EBS
NVMe Controller
Standard drivers broadly available
EBS Data Plane
Encryption support
NVM to remote storage protocol

10. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Nitro Card for Instance Storage
NVMe Controller
Standard drivers broadly available
Instance
Storage
Instance Storage Data Plane
Transparent Encryption
Limiters
Drive monitoring

11. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Nitro Card Controller
System Control
Provides passive API endpoint
Coordinates all other Nitro Cards
Coordinates with Nitro Hypervisor
Coordinates with Nitro Security ChipNitro
Controller
Hardware Root of Trust
Provides measurement and attestation

12. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Nitro Security Chip
Custom microcontroller
that traps all I/O to non-
volatile storage
Controllable from the
Nitro Controller to hold
system boot
Provides a simple, hardware-based root of trust

13. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
UEFI Secure Boot
Boot starts untrusted and must
prove that system is trustworthy.
Deep complexity with millions of
lines of code.
Unavoidable complexity due to
need to support legacy and general
purpose workloads.
Properly Signed?PK/KEK
Early Firmware
Fail Boot!
Yes
No

14. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
UEFI Secure Boot
Boot starts untrusted and must
prove that system is trustworthy.
Deep complexity with millions of
lines of code.
Unavoidable complexity due to
need to support legacy and general
purpose workloads.
Properly Signed?
Properly Signed?
PK/KEK
Early Firmware
UEFI Boot Manager
Fail Boot!
Yes
No
No
Yes

15. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
UEFI Secure Boot
Boot starts untrusted and must
prove that system is trustworthy.
Deep complexity with millions of
lines of code.
Unavoidable complexity due to
need to support legacy and general
purpose workloads.
Properly Signed?
Properly Signed?
Properly Signed?
PK/KEK
Early Firmware
UEFI Boot Manager
Fail Boot!
UEFI Applications
Yes
No
No
No
Yes
Yes

16. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
UEFI Secure Boot
Boot starts untrusted and must
prove that system is trustworthy.
Deep complexity with millions of
lines of code.
Unavoidable complexity due to
need to support legacy and general
purpose workloads.
Properly Signed?
Properly Signed?
Properly Signed?
PK/KEK
Early Firmware
UEFI Boot Manager
Fail Boot!
Properly Signed?UEFI Applications
UEFI Drivers
Yes
No
No
No
No
Yes
Yes
Yes

17. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
UEFI Secure Boot
Boot starts untrusted and must
prove that system is trustworthy.
Deep complexity with millions of
lines of code.
Unavoidable complexity due to
need to support legacy and general
purpose workloads.
Properly Signed?
Properly Signed?
Properly Signed?
PK/KEK
Early Firmware
UEFI Boot Manager
Fail Boot!
Properly Signed?UEFI Applications
Properly Signed?UEFI Drivers
Properly Signed?Operating System
Yes
No
No
No
No
No
No
Yes
Yes
Yes
Yes

18. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Nitro Hardware Root of Trust
Radical simplification enabled by
Nitro Cards.
All write access to non-volatile
storage is blocked in hardware.
Simple to understand security due
to lack of legacy.
Instance
Nitro
Controller
BIOS, BMC,
et al

19. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Nitro Hardware Root of Trust
Radical simplification enabled by
Nitro Cards.
All write access to non-volatile
storage is blocked in hardware.
Simple to understand security due
to lack of legacy.
Instance
Writes
Nitro
Controller
BIOS, BMC,
et al

20. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Nitro Hypervisor
KVM-based hypervisor
with custom MM and
small userspace
Only executes on behalf
of instance, quiescent.
With Nitro, the hypervisor can be fast and simple

21. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Nitro Hypervisor Jitter
0
100
200
300
400
500
600
700
800
Microseconds
Wake up Delay
Series1 Series2 Series3
SLA

22. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Continuous Innovation via Nitro

23. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
C5d, M5d, R5d
Instance storage types
NVMe-based
Instance Storage
High throughput and
low latency
Ideal for fast access to scratch or replicated storage

24. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
I3.metal
Bare Metal instance types
First Nitro
Bare Metal platform
Enables custom
hypervisors and
micro-VM runtimes
Ideal for Android emulation and Micro-VMs

25. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
R5, M5, and T3 instances with AMD EPYC processors
AMD instances for cost-optimized compute
10% cost savings
compute and memory
Seamless application
migration
Ideal for general purpose and memory optimized workloads

26. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
C5n
100 Gbps instance types
100 Gbps network
throughput including
data transfer to S3
Super low latency
message passing
Ideal for HPC workloads and distributed ML

27. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS support for Arm-based applications
Up to 45% cost savings,
higher price/performance
AWS Graviton Processor
with Arm-based cores and
customized silicon
A1
Ideal for scale-out workloads including web and e-commerce sites

28. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Firecracker Micro-VMs
Security Speed by design Scale and efficiency
Minimal device model reduces
memory footprint and attack
surface area
User-space code in <125ms,
150 microVM
per second per host
Low memory overhead with high
density (thousands) of microVMs on
each server

29. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Firecracker Open Source Project
Open Source and on GitHub
Built for Serverless
AWS Lambda and AWS Fargate
Written in Rust
Still very early, much more to do
Containerd integration
Linux improvements
Kata integration

30. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What’s Next?

31. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Nitro: Anywhere you need it
AWS Outposts
Nitro hardware and software in your data center
Access via standard AWS API and console
Deploy apps to Outposts using AWS services

32. Thank you!
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Anthony Liguori
aliguori@amazon.com

33. Please complete the session
survey in the mobile app.
!
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.