Towards User-Defined SLA in Cloud Flash Storage Paper presentation

1. Towards User-Defined SLA in
Cloud Flash Storage
Speaker: Po-Chaun, Chen

2. • Abstract
• Introduction
• Background & related work
• User-Defined Storage
• Tail Latency Problem with NAND Flash
• SLA Curve (Arithmetic)
• SLA Curve (System Design)
• SLA-Aware Data Placement
• SLA-Enforcing I/O scheduling
• Implementation
• Evaluation
• Resource Utilization
• Performance SLA
• Discussion
• Conclusion

3. Abstract
• Difference between users and cloud providers
For users,
storage applications
often request
asymmetric read and write
bandwidth, with tail read
latency guarantee.
For cloud providers,
typical cloud SSD offerings either provide
read-write aggregate throughput
guarantee or only specifies peak pure-
read and write throughput.

4. Problems
It is also hard for cloud NAND flash SSDs to provide tail latency
guarantees because of their notorious read-write interference problem.
As a result, users have to over-provision SSD resource to satisfy their
service level agreement (SLA), leading to potential under-utilization.

5. Regulator
It define their SLA for NAND flash based cloud storage
Regulator first proposes
 a formalization of user-defined SLA as SLA Curve
 an SLA-aware data placement algorithm
 a runtime QoS module
Regulator can increase cloud flash
utilization by 15%~44% while
satisfying user-defined SLAs.

6. Introduction
NAND flash SSD is receiving increasingly wider adoption in cloud data centers
 provide much higher throughput
 much lower access latency
 much better random access performance

7. Different needed
• Bandwidth
• Users require various kinds of performance guarantees
• Cloud SSDs cannot provide such flexibility
• Throughput
• Users require distinguish reads and writes
• Cloud SSDs only offer overall throughput guarantee
• Latency
• Users expect their storage devices to have tail latency guarantee
• NAND flash suffers read-write interference

8. Used : runs read-write mixed application
Reserved : maximum aggregate throughput under all read-write-ratios
Wasted : some situation that its application never reaches
Actual SSD : available bandwidth of the physical SSD

9. Why ?
• The SSD is not able to serve the tenant even though it only actually
uses a small portion of its bandwidth capability
• Their actual SSD throughput area would shrink dramatically, because
only little write would be allowed under read-write interference.

10. How ?
As resource disaggregation breaks server boundaries, cloud operators
are able to do more fine-grained resource allocation according to user
requirements while achieving better utilization.

11. Regulator
 SLA Curves :
express both read-write asymmetry and read latency sensitivity in
applications and SSDs.
 A novel resource allocation algorithm :
increase flash utilization, efficiently place different kinds of tenants onto
physical SSDs.
 QoS module :
schedules I/O requests sent to SSDs to enforce SLA.
This paper prototype Regulator using SPDK
Storage Performance Development Kit
provides a set of tools and libraries for writing high
performance, scalable, user-mode storage applications.

12. • Abstract
• Introduction
• Background & related work
• User-Defined Storage
• Tail Latency Problem with NAND Flash
• SLA Curve (Arithmetic)
• SLA Curve (System Design)
• SLA-Aware Data Placement
• SLA-Enforcing I/O scheduling
• Implementation
• Evaluation
• Resource Utilization
• Performance SLA
• Discussion
• Conclusion

13. User-Defined Storage
• User Defined Cloud (UDC), which is quite similar to our user-defined
SLA methodology.
• Instead of letting cloud providers offer fixed hardware options, UDC
proposes that users can customize virtual hardware according to their
own requirement, which would increase overall resource utilization.

14. Tail Latency Problem
NAND flash based SSDs are notorious for their bad tail latency because
of the read-write interference problem.
Using Samsung PM963, which is a widely
adopted NAND flash SSD in data center.
We choose to focus on read tail
latency here because in read-
write interference,
only read is influenced by write
but not vice versa

15. Observation
• Caused by SSD background operations incurred by writes, including write
buffer flush and garbage collection.
• By controlling read and write throughput, we can make SSD internal
activities happen less frequently, thus improving tail latency, which is a
statistical result.

16. Some solutions
 Smart IO/FIOS : I/O scheduler
 ReFlex : Model (assigned weight to writes)
 Flash on Rails : split SSD work into different slices
 LinnOS : lightweight neural network
 SWAN : split all-flash array into frontend SSD
(log-structured writes & backend GC)

17. • Abstract
• Introduction
• Background & related work
• User-Defined Storage
• Tail Latency Problem with NAND Flash
• SLA Curve (Arithmetic)
• SLA Curve (System Design)
• SLA-Aware Data Placement
• SLA-Enforcing I/O scheduling
• Implementation
• Evaluation
• Resource Utilization
• Performance SLA
• Discussion
• Conclusion

18. SLA Curve
• Represent tenants’ bandwidth requirements as well as storage
devices’ service capability
• Describes the maximum read throughput for each write throughput
while achieving a certain read tail latency guarantee
• This paper define SLA Curve as a function of write throughput.

19. SLA Curve
By ensuring read and write throughput under the SLA Curve, we can
keep the frequency of SSD’s internal activities low enough, therefore
guaranteeing tail latency.
This paper derive the SLA curve with
a convex hull algorithm on the sampled points.

20. SLA Arithmetic
When multiple tenants are sharing the same storage device, we need to
know their aggregate bandwidth requirements.
When one tenant is assigned to a storage device, We need to calculate the
remaining service capability of the device.

21. • Abstract
• Introduction
• Background & related work
• User-Defined Storage
• Tail Latency Problem with NAND Flash
• SLA Curve (Arithmetic)
• SLA Curve (System Design)
• SLA-Aware Data Placement
• SLA-Enforcing I/O scheduling
• Implementation
• Evaluation
• Resource Utilization
• Performance SLA
• Discussion
• Conclusion

22. SLA Curve based system design

23. SLA-Aware Data Placement
• Multiple tenants typically share multiple physical storage units
• The multi-tenant resource placement problem is often modeled as a
bin packing problem (NP-hard)
• Using best-fit
It greedily selects the bin with the minimum capacity remaining that
can fit the item.
Then, if no available bin can fit the item, it starts a new bin.

24. Difference between traditional way
• For the first phase, the "size" and "capacity" of tenants and storage units
are described using SLA Curves instead of a scalar value
• And we need to consider which type of storage unit to select when we
need a new one
• Specifically, we consider not only whether the device’s curve fits tenant’s
demand, but also how well the two curves align with each other

25. Metrics can be used for SLA Curve alignment
 Dot product which emphasizes the shape similarity between SLA Curves
 L2-norm which represents the distance between SLA Curves
 Ratio between areas of two SLA Curves

26. And then
 Start a new storage unit in the second phase, the Regulator data
placement algorithm greedily chooses the storage unit that minimizes
the cost for the current tenant.
 Specifically, it calculates the cost needed to fit the tenant’s SLA
requirement for each type of storage unit.
 If a single storage unit can not meet the need of the tenant, the
algorithm strips the data across multiple storage units to avoid hotspots.

27. About tail latency guarantees
• Some tenants may require tail latency guarantees so strict that it is
hard to provide with plain NAND SSDs.
• To satisfy such tail requirements, we leverage to construct redundant
SSD arrays without read-write interference.

28. • Abstract
• Introduction
• Background & related work
• User-Defined Storage
• Tail Latency Problem with NAND Flash
• SLA Curve (Arithmetic)
• SLA Curve (System Design)
• SLA-Aware Data Placement
• SLA-Enforcing I/O scheduling
• Implementation
• Evaluation
• Resource Utilization
• Performance SLA
• Discussion
• Conclusion

29. SLA-Enforcing I/O scheduling
• The SLA needs to be actually enforced at runtime
• Regulator’s I/O scheduler does so by controlling the read and write
throughput of each tenant sharing a same storage unit
• To control the bandwidth of both reads and writes, the Regulator I/O
scheduler performs curve-based rate limiting

30. Rule
In each time slice, it tracks the read and write bandwidth usage of each
tenant.
When the request comes :
while (current bandwidth usage > SLA curve) {
Waiting in queue
}
Submits the requests

32. But actually …
• The method do not recommend choosing an SLA Curve that covers all the
bandwidth outliers because that will likely cause a waste of bandwidth
• The Regulator scheduler can optionally allow
tenants to temporarily burst above their SLA Curves

33. Dispatch
• The bursty accesses are only scheduled when no non-bursty request
is waiting to be dispatched.
• When multiple tenants burst above their curve, the Regulator
scheduler uses weighted round robin to share the extra bandwidth.
• Larger than 64KB will be spliced

34. The I/O scheduling
• The I/O scheduling is done on the storage units
 Because they have full knowledge about all tenants that are accessing them
 It can make sure the storage device always runs under its desired SLA curve

35. • Abstract
• Introduction
• Background & related work
• User-Defined Storage
• Tail Latency Problem with NAND Flash
• SLA Curve (Arithmetic)
• SLA Curve (System Design)
• SLA-Aware Data Placement
• SLA-Enforcing I/O scheduling
• Implementation
• Evaluation
• Resource Utilization
• Performance SLA
• Discussion
• Conclusion

36. Brief
• It implemented a prototype system for Regulator mainly based on
Storage Performance Development Kit (SPDK)
• It implemented the Regulator controller in C++.
• The Regulator I/O scheduler is implemented as an extension of SPDK’s
bdev layer at the storage server.

37. Resource allocation

38. Performance SLA

39. • Abstract
• Introduction
• Background & related work
• User-Defined Storage
• Tail Latency Problem with NAND Flash
• SLA Curve (Arithmetic)
• SLA Curve (System Design)
• SLA-Aware Data Placement
• SLA-Enforcing I/O scheduling
• Implementation
• Evaluation
• Resource Utilization
• Performance SLA
• Discussion
• Conclusion

40. Discussion
• Obtaining SLA Curves
• Hardware acceleration
• Generalize to other types of storage

41. User-Defined SLA in Cloud Flash Storage