Tutorial Description: Big data, virtualization, OLTP systems, and a host of performance-intensive applications have led to much greater use of flash memory in data centers. Data center managers and engineers must therefore focus on when to use flash memory, how to keep costs at a reasonable level, and where to employ flash (whether in caches, in tiers, or as main storage). This session focuses on why flash is being used and what benefits it brings to data centers. Topics: The impact of flash memory in data center infrastructure Benefits of flash technology Flash and the data center ecosystem Growing flash utilization within data centers and IT infrastructures Intended Audience: Storage specialists; storage engineers; data center, network, and storage managers; product designers and managers; system analysts; computer managers and engineers; data center engineers

1. Achieving Flash Memory’s Full Potential
Darpan Dinker
Senior Director, Engineering
SanDisk
Flash Memory Summit 2012
Santa Clara, CA 1

2. Forward Looking Statement
During our meeting today we will be making forward-looking statements.
Any statement that refers to expectations, projections or other
characterizations of future events or circumstances is a forward-looking
statement, including those relating to revenue, pricing, market share, market
growth, product sales, industry trends, expenses, gross margin, future
memory technology, production capacity and technology transitions and future
products.
Actual results may differ materially from those expressed in these forward-
looking statements due to the factors detailed under the caption “Risk
Factors” and elsewhere in the documents we file from time-to-time with the
SEC, including our annual and quarterly reports.
We undertake no obligation to update these forward-looking statements,
which speak only as of the date hereof.
Flash Memory Summit 2012 2
Santa Clara, CA

3. Agenda
§ Flash memory application in the data-center
§ Why focus on balanced systems
§ Architectural approaches to exploit flash
§ Scenarios to leverage flash
§ Lessons learned
Flash Memory Summit 2012 3
Santa Clara, CA

4. Flash Memory Application
in Data Centers
VDI
Cloud Databases
Virtualization
Hadoop
Big Data SharePoint
Multi-tier storage NoSQL
SAN NAS Memcached
MongoDB
Exchange MySQL
Flash Memory Summit 2012 4
Santa Clara, CA

5. Why Focus on Balanced Systems
10 ms
§ CPU Magnetic Storage
1 ms
§ Main-Memory Remote memory
100 µs Flash memory
§ Storage 10GbE, InfiniBand
10 µs
§ Network PCIe
1 µs
100 ns DRAM
10 ns CPU Cache
CPU Register
1 ns
Flash Memory Summit 2012 5
Santa Clara, CA

6. Architectural Approaches to Exploiting
Flash Memory
1. Augment main-memory
• Directly (mmap, system paging,
custom kernel) Memcached
• Indirectly (object cache)
Flash Memory Summit 2012 6
Santa Clara, CA

7. Architectural Approaches to Exploiting
Flash Memory
1. Augment main-memory
• Directly (mmap, system paging,
custom kernel)
• Indirectly (object cache)
2. Cache blocks of magnetic storage
• Disjoint from storage (WB or WT)
• Integrated with storage
Flash Memory Summit 2012 7
Santa Clara, CA

8. Architectural Approaches to Exploiting
Flash Memory
1. Augment main-memory
• Directly (mmap, system paging,
custom kernel)
• Indirectly (object cache)
2. Cache blocks of magnetic storage
• Disjoint from storage (WB or WT)
• Integrated with storage
3. Replace magnetic storage
• DAS
• NAS or SAN
Flash Memory Summit 2012 8
Santa Clara, CA

9. Scenarios for Leveraging Flash Memory
1. Applications are IO bound
2. Lots of random IO
3. Prepare for data growth
4. Heavily virtualized environment
5. Prepare for unpredictable load
6. Improved response time / SLAs
7. Improved reliability
8. Save on space, power, cooling
Flash Memory Summit 2012 9
Santa Clara, CA

10. Scenarios for Leveraging Flash Memory
1. Applications are IO bound
§ Common symptoms
• Low CPU usage
• High io-wait
• Sluggish response time (are locks to blame?)
Flash Memory Summit 2012 10
Santa Clara, CA

11. Scenarios for Leveraging Flash Memory
2. Lots of random IO
Databases
§ Magnetic drive heads cannot MongoDB
keep up with random IO
(~200 IOPS/drive) MySQL
§ Common symptoms observed
with ‘iostat’
• High IO service time, queue size, wait time
• Low IO bandwidth
Flash Memory Summit 2012 11
Santa Clara, CA

12. Scenarios for Leveraging Flash Memory
3. Prepare for data growth
Big Data
§ Data growth can increase
demand for IO
Hadoop
§ Common symptoms
• Database queries slow down as table size increases
• Wrong trends observed with ‘iostat’ as data grows
Flash Memory Summit 2012 12
Santa Clara, CA

13. Scenarios for Leveraging Flash Memory
4. Heavily virtualized environment
Cloud
§ Individual VMs with low IO
requirements build up a heavy VDI
IO workload for a shared storage Virtualization
system
§ Many individual sequential accesses create a
random IO workload
§ IO path is not particularly optimized in VMs
Flash Memory Summit 2012 13
Santa Clara, CA

14. Scenarios for Leveraging Flash Memory
5. Prepare for unpredictable load
§ DAS with magnetic disks have
SAN
limited upper bound for IOPS. In
order to size systems for peak NAS
load, the “work” per server may
require substantial throttling à resulting in
underperforming servers
Flash Memory Summit 2012 14
Santa Clara, CA

15. Scenarios for Leveraging Flash Memory
6. Improved response time / SLAs
§ Many applications require disk SharePoint
access. Disk access can be
responsible for significant portion Exchange
of the response time
§ Sequential and random access times are
asymmetric for magnetic drives. Defining SLAs
is thus non trivial
Flash Memory Summit 2012 15
Santa Clara, CA

16. Scenarios for Leveraging Flash Memory
7. Improved reliability
§ Moving mechanical parts make magnetic
drives quite susceptible to failure. RAID is
a common practice even if data is made
redundant on another server
Flash Memory Summit 2012 16
Santa Clara, CA

17. Scenarios for Leveraging Flash Memory
8. Save on space, power, cooling
§ Magnetic drives take power and dissipate
a lot of heat (>10X that of flash memory)
§ Random IOPS are expensive with
magnetic drives
Flash Memory Summit 2012 17
Santa Clara, CA

18. Lessons Learned:
Don’t Trust the Folk Lore
§ Can observe response time and throughput
improvements for most IO-bound workloads after
exploiting flash, but…
• Nothing close to 100X theoretical improvement
(5-10ms à 50-100µs)
• Most existing software applications unable to
push IOPS
• May observe “write cliff” for workloads with
heavy writes
Flash Memory Summit 2012 18
Santa Clara, CA

19. Lessons Learned:
Your Mileage May Vary
§ Workload: DBT2 open-source
OLTP version of TPC-C
75 • 1000 warehouses, 32 connections
• 0 think-time
#2
• Result metric: TPM (new order)
TPM (in thousands)
50
§ Measurement Configuration
• 2 node Master-Slave configuration
25
• 2 socket 6C Westmere,
#1 72GB DRAM
0 #1 Replace HDD with flash
MySQL 5.5 SchoonerSQL
#2 Understand application architecture & design.
HDD Flash Re-implement portions and integrate with flash.
Flash Memory Summit 2012 19
Santa Clara, CA

20. Lessons Learned:
Tweak or Rewrite Application SW
§ Tips
• Consider bypassing OS IO schedulers (noop)
• When IO throughput is more important, employ larger
block size
• Target keeping high number of outstanding IOs
§ To push 10k-50k IOPS, tweak storage access layer
of application
§ To push 50k+ IOPS, consider rewriting
parallelization and storage access layer
Flash Memory Summit 2012 20
Santa Clara, CA

21. Lessons Learned:
Know the Technology
§ Consumer grade vs. enterprise grade device
§ Random writes and non-uniform size writes
• Algorithmic differences between vendors for space and
wear management
§ IOPS is not everything in production
• Determine distribution and sustained latency
• Does the device really offer “durability”?
§ Compression in devices fare differently based on
data (e.g. jpeg, mpeg)
§ Monitor SMART metrics
Flash Memory Summit 2012 21
Santa Clara, CA

22. Lessons Learned:
Know About the Form Factor
§ 1 SAS with 1 PCIe flash device is apples : oranges
comparison for throughput
§ SAS/SATA based flash
• Easier to RAID
• Typically hot swappable and easy to replace
• Not all HBAs are built to match flash memory speeds
§ PCIe based flash
• Typically offer IOs with lower latency
• Convenient to add to existing systems for caching
Flash Memory Summit 2012 22
Santa Clara, CA

23. acquires
F R E E YOUR DATA
June 21, 2012
Flash Memory Summit 2012 23
Santa Clara, CA

24. SanDisk Flash-Accelerated Products
Schooner Membrain Enterprise NoSQL Cache/Store
SchoonerSQL Enterprise SQL Database
FlashSoft™ Caching software Enterprise Storage Caching
Lightning® SAS Enterprise SSD
Enterprise Flash Hardware
Lightning® PCIe Enterprise SSA
Flash Memory Summit 2012 24
Santa Clara, CA