Sanjay Sabnis presented on next generation storage solutions for modern big data applications. He discussed how NVMe storage provides significantly higher performance than SATA, with speeds over 6x faster for reads and over 40x faster for writes. Pavilion Data offers an all-NVMe rack scale storage array that provides 120GB/s of throughput with DAS-level latency. This solution can meet the performance and scalability demands of big data workloads like MongoDB, Splunk, and containerized applications.

1. Revolutionary Storage
For Modern Applications
Sanjay Sabnis
@sabhub1
Big Data Science Meetup
@Paviliondata
05/25/2018

2. Agenda
• Welcome
• Big Data Application Demands
• Next Generation Storage for Big Data Applications
• Big Data Use Cases

3. It is About Data

4. Expectations
Speed
Performance
Latency
Accuracy

5. Solving the Scalability Problem
RACK
Adding More Nodes
Add More memory
Upgrade Networking
• Need more storage? Add more nodes ßà Comes with Compute As
well
• Under-utilization of storage - Islands of storage still exist
• Limited by Rack Level Data Management at Scale
• Network is not up to date to utilize new features of hardware

6. Infrastructure
Connectivity Cognition
2.0 3.0
How do we connect
the world?
How do we make
sense of the world?

7. Crossing the Chasm
AI
Autonomous Vehicles
Image Recognition
Von Neumann
Time to rethink infrastructure, tooling, and
development practices.

8. Modern Applications Requirements
• Compute/Network/Storage
• Rack Awareness/DC Awareness – Data Locality/HA
• Master/Slave - Scalable
• Master-less - Scalable
• IOPS, Bandwidth – High Data Transfer Rates – 25/50/100 GB is
Standard now.
• Storage Awareness? – This is something new!
• Non-compute Centric Data Management

9. The Compute & Storage Disconnect
• Compute and Storage Age Differently
• Compute has Moore’s Law, what about storage?
• Replacing Compute calls for replacing disks – Fixed Density, more $$$
• We all have been using SATA drives
• There is a new interface called NVMe for SSD (Non Volatile Memory
Express)
It is a logical device interface specification

10. Storage Protocol Differences
SATA NVMe
End of Freeway

11. 11
Comparing NVMe to SATA
SATA SSD NVMe SSD NVMe Difference
Read BW (MB/s) 500 3300 6.6X
4K Read IOPS 64K 830K 13X
Write BW (MB/s) 475 2100 4.4X
4K Write IOPS 5K 200K 40X
4K Mixed IOPS (70:30 R:W) 11K 550K 55X

12. DATA INTENSIVE WORKLOADS
Analytics, IoT, Streaming Media, AI/ML, Databases
NVMe-oF SSD Array
• High Performance
• Cost Efficient
• High Utilization
• Scalable (14 TB to 1 PB)
• HA
• PaYG Model (Pay as You Grow)
NVMe-Based Storage for Big Data

13. Pavilion All-NVMe Storage Array
13

14. Advantages for Big Data Deployments
Reduce per-rack costs up to 72%
Improve Storage Utilization 2X+
Free up stranded capacity residing on DAS
Management Flexibility
Less raw storage deployed lowers IT Admin Costs
Move data sets from one server to another without copying
Reduce Infrastructure
Less Servers required, or consolidate more DB instances per
server
Eliminate DAS SSDs
Leverage Full-Performance, Space-Efficient Copies
14
Performance (Latency & Bandwidth) of Direct Attached Storage
Serviceability and Data Management of Shared Storage
DAS

15. Performance and Cost Advantages
Index More Data With Splunk
Lower Costs of
noSQL Deployments
Using networked Pavilion storage instead of direct-attached SSDs gives better performance per
server, allowing you to reduce server count and size, plus gain the cost advantages of a SAN
15

16. Pavilion All-NVMe Storage for Big Data
120 GB/S
PERFORMANCE
Up to 40 x 100GE
Ports
MODULAR
14TB – 1PB
CAPACITY
Up To 20, Active-
active Controllers
RESILIENCY
4 RU
DENSITY
Raid-6,
Snapshots, Thin
Provisioning
DATA MANAGEMENT
NVME & NVMEOF
100% STANDARDS COMPLIANT
X86, 2.5” NVMe
SSD
STANDARD OFF-THE-SHELF
COMPONENTS
1/10TH
$COST/IOPS
DISRUPTIVE ECONOMICS

17. Shared Block Storage For Big Data Applications
ü Hosts connected using
25/40/50/100Gb Ethernet
ü NVMe block storage presented to
host servers using
community/standard NVMeoF driver
ü No custom host software required
ü 10s of micro-second latency
ü Latency of DAS SSDs
ü Full HA capability and hot-pluggable
components
Thin-Provisioned
NVMe volumes
presented to the
host server
17

18. Management Integration
18
Rest API

19. Use Cases

20. Cassandra
C*
C*
C*
1
2
3
Volumes for Node2
a2 b2 c2
a Commit Log
b Data
c Log
Volumes for Node1
a1 b1 c1
Rack Scale
• Dense Compute Rack
• Easy to Add or Replace nodes
• Integrates into DevOps using Rest API
• Thin provisioning to save flash resources
• Increase Volume Size Dynamically
• Manage instant data copies using Rest API
~ 1PB Storage
Snapshot/Clone
Data Backup/Restore
Rack 1 Rack 2

21. Adding a New Shard • Adding Shard to the Cluster
• Add shard to scale the MongoDB cluster horizontally
• Affects the balance of chunks among the shards of a cluster for all
existing sharded collections.
• The balancer will begin migrating chunks so that the cluster will
achieve balance
• Rebalance will affect existing Read/Write and IOPS performance.
PRIMARY
SECONDARY
SECONDARY
SHARD 1
PRIMARY
SECONDARY
SECONDARY
SHARD 2
APP SERVER APP SERVERAPP SERVER
REPLICA SET 1 REPLICA SET 2
Present Individual Volumes
For each node
PRIMARY
SECONDARY
SECONDARY
SHARD 3
REPLICA SET 3
New Shard
>
> Speed Shard Rebalancing
• Pavilion Advantages
• No sizing activity required.
• No impact of no. of parallel chunk migrations, same IOPS for all with 40 ports
• Pre Configure Pavilion volumes for future shard expansion to automate the
scaling activity
• Over provision the volume size to alleviate IOPS performance as data grows
>

22. MongoDB - Leveraging Snapshots and Clones
PRIMARY
SECONDARY
SECONDARY
.
.
SECONDARY
Instant Clone
Point in Time Instant Pavilion Snapshots
PRODUCTION
PRIMARY
SECONDARY
SECONDARY
.
.
.
DEV/QA/PREPROD
Backup/Archive
Instant Clone
Use Clone to Scale Replica Set Use Clone to spin up DEV/QA/PREPROD quickly
Pavilion Instant Clones
SECONDARY
Replication
• Scale MongoDB infrastructure without downtime
• Rapid volume cloning capabilities allow for new backup and deployment strategies
• Instant cloning makes node recovery and replacement easy

23. Reduce Splunk Indexer Sprawl
PAVILION DATA CONFIDENTIAL & PROPRIETARY 23
HOT
WARM
COLD
FROZEN
Tier 1 - $$$$
Tier 2 - $$$
Tier 3 - $$
Tier 4 - $
Backup
Read-Only Snapshots
QA/Dev/PreProd Testing
R/W Clones
Consolidate All Splunk Data on One High-Speed Storage Platform, Simplify Backup and Copy Management

24. Addressing Splunk Challenges
24
Splunk Solution Design Considerations
Insufficient disk I/O is the most common limitation in Splunk infrastructure
Pavilion delivers over 100 GB/s of bandwidth, and 20 Million IOPS from a
compact, 4U Chassis, which can power even the largest Splunk
deployments
Review the disk subsystem requirements before provisioning your hardware
Pavilion’s scalable platform allows you to focus on the needs on the
compute infrastructure instead of storage
More disks (specifically, more spindles) are better for indexing performance
Pavilion’s low latency storage platform eliminates storage as the indexing
bottleneck
Total throughput of the entire system is important.
Pavilion delivers significant improvements in performance and improves
decision times.
The ratio of disks to disk controllers in a particular system should be higher,
similar to how you provision a database host
Pavilion’s performance and capacity allows for easy storage configuration.
Hot Bucket – Cannot Backup
Take backup of any volume any time without performance overhead on
indexing nodes by using the Pavilion Snapshot feature

25. Modernize Database Deployments
25
ü Simplify Infrastructure by disaggregating storage
into a centralized, rack-scale appliance
ü Leverage shared storage resources at the speed
and latency of local SSDs
ü Reduce raw flash required
ü Independently scale compute, networking and
storage to maximize flexibility
ü Move to ‘storage-less’ 1U servers to increase
compute density per rack
ü Centralize storage resources to facilitate easy
backup and restore
ü Instantly deploy new copies of the database for
test/dev/QA purposes
DENSE
Compute CLUSTER

26. Other Use Cases
…………

27. New Data Architectures
Centralized Logging
“We are a log Management Company that happens
To Stream Videos”
-Netflix Chief Architect
Log Monitoring/Forwarding/….
No Log Forwarding from each Node
Save CPU Cycles

28. Container Architecture - Cloud
• Fits into Kubernetes or OpenStack implementations
• Integrate Pavilion REST API with Cinder Wrapper provided by the Pavilion
• Storage can be used as Static or Dynamic Volume provisioning
• Fits readily into DevOps CI/CD setup with provided REST API interfaces
• Utilize the Pavilion Snapshot, Clone and volume migration features to manage data beyond lifecycle of the virtual image
• Supports Block Storage, NFS ( S3 support in near future ).
Kubernetes
Pod
Nova
KeyStone
Boot
Launch
Authentication
Persistent
Volume
Docker
Kubernetes Cluster - Datacenter
OpenStack
CSI
Wrapper
Cinder Block
Storage Volumes
Rack Scale Flash Array
Docker’s Containers-as-a-service
(CaaS) platform that can run atop
cloud-based infrastructure such as
OpenStack, or on bare metal
infrastructure, providing complete
application lifecycle management
for container deployments.

29. HiBD (Hi-Performance Big Data)
• NVMe-oF opens up opportunity for commoditizing the HiBD
• RDMA + NVMe = Killer IOPS & Bandwidth
• Lots of Development has been done using RDMA-based HiBD
Apache Crail - Incubating
Pavilion - 120 GB/S
With DAS Latency
Crail is designed from ground up
for modern high-performance
networking and storage
hardware (RDMA, NVMe, NVMf,
etc.). It leverages user-level I/O to
access hardware directly from
the application context, providing
bare-metal I/O performance to
analytics workloads.
Storage Awareness