As software eats the world, open source always follows. Storage is being fundamentally disrupted by open source. This presentation covers software-defined storage and open source trends and how they affect traditional storage.
3. THE RISE OF OPEN SOURCE STORAGE PROJECTS
- RETHINKING STORAGE
SESSION OUTLINE
STORAGE MARKET DYNAMICS
4. DATA STORAGE “SHORTFALL”
Data stores are growing
exponentially, while IT budgets are
not
20232013
IT BUDGET
2% CAGR
(2013-2023)
IT Professionals
Will Shoulder a Greater Storage Burden
STORAGE CAPACITY
62% CAGR
(2013-2023)
230 GB
Per IT Pro
2013
1,231
GB
Per IT Pro
2023
HDDs are becoming more dense,
but $/GB decline is slowing
Software and hardware advances
are needed to close the gap
Source: IDC, The Digital Universe of Opportunities: Rich Data
and the Increasing Value of the Internet of Things, April 2014
5. ORGANIZATIONS ARE RETHINKING STORAGE
$0
$5,000
$10,000
$15,000
$20,000
$25,000
$30,000
$35,000
$40,000
$45,000
$50,000
2016 2017 2018 2019 2020
Hyperscale Software-defined Storage
Enterprise Software-defined Storage
Enterprise Traditional Storage
TAM(millions)
Source: Wikibon
Traditional SAN/NAS storage
market is in accelerating decline
Software-defined storage (SDS)
grows from 12% to 49% between
2016-2020
Most of the SDS market
is hyperscale today
Enterprise SDS is smaller to date,
but where the faster growth is
expected
6. SOFTWARE-DEFINED STORAGE (SDS)
SDS ENABLES ABSTRACTION OF THE
HARDWARE
FROM THE SOFTWARE
Independence from
high-cost proprietary
HW + SW systems
leads to better
pricing and freedom
of choice
Hardware is treated
as a pool, scalable
across multiple
storage systems
Innovative
functionality, easier
to manage,
faster to implement,
and more flexible to
scale-out
Integral part of the
software-defined
data center (SDDC)
KEY BENEFITS
7. TRADITIONAL VS. SDS
Lower cost, standardized supply chain
Increased operational flexibility
More programmability, agility, and control
More flexible, well-integrated technology
TRADITIONAL SDS
Proprietary
Hardware
Scale-Up
Architecture
Hardware-Based
Innovation
Closed Development
Process
Common,
Off-The-Shelf Hardware
Scale-Out
Architecture
Software-Based
Innovation
Open Development
Process
9. OPEN SOURCE =
GOOD BUSINESS
NEARLY 9in 10 SAY THAT OPEN SOURCE
GREATLY IMPACTS THE SPEED OF INNOVATION –
ALIGNING WITH TIME TO MARKET FOR NEW PRODUCTS
UP FROM 45% IN 2014
NEARLY 8 in 10 SAY THAT OPEN
SOURCE IMPROVES MARGINS
AND GROWS REVENUE
OVER
65%GAIN COMPETITIVE
ADVANTAGE
10. COTS HARDWARE + OPEN SOURCE SOFTWARE + COMMUNITY
WHAT IS OPEN SOURCE STORAGE?
Support & Maintenance
Proprietary Software
Proprietary Hardware
CPU
CPU
CPU
Disk
Disk
Disk
Enterprise Products & Services
Open Source Software
COTS Hardware
By 2018 Open Source Storage will gain 20% of the market, approx. $10B*
CPU
CPU
CPU
Disk
Disk
Disk
*(Gartner, “2014 Strategic Roadmap for Storage”, March 2014)
11. OPEN SOURCE STORAGE STACK
DISTRIBUTED
PROCESSING
CLOUD
COMPUTING
OS
STORAGE
DATABASE
SWIFT
12. TOP OPEN SOURCE STORAGE PROJECTS
• Number one storage choice for OpenStack installations
• Contributors include RedHat, Intel, SanDisk, CERN, and Yahoo
• Block, File, Object: RADOS, RBD, Ceph FS, S3, Swift
• Over 11M downloads in the last 12 months
• File: GlusterFS, CIFS, NFS
• RedHat ownership increasing project interest
92 authors/last mo
1,139 commits/last
mo
39 authors/last mo
234 commits/last
mo
• Part of the OpenStack project
• Contributors include SwiftStack, RedHat, HPE, Intel, Rackspace,
IBM
• Object: Swift
30 authors/last mo
48 commits/last mo
SWIFT
13. CEPH INTRODUCTION
• Open source software-defined storage
• Massively scalable
• Object, block and file system storage in a single platform
• Runs on COTS hardware – saving cost and offering improved
flexibility
OBJECT
RADOS S3/SWIFT
Object
Object
Object
Object
Object
Object
Object
FILE
NFS CIFS
BLOCK
RBD
14. CEPH BASICS
• Massively Scalable to Exabytes
• Self-healing and Self-managing
• Automatic distribution of replicas
• Auto-rebalancing
• Redundancy though data replication
• Rack and data center aware
• No need for RAID
• Fully distributed – no single point of failure
• APIs and cloud integration for self-service
RGW
A web
services
gateway for
object storage
LIBRADOS
A library allowing apps to directly access RADOS
RADOS
A software-based reliable, autonomous, distributed
object store comprised of self-healing, self-managing,
intelligent storage nodes and lightweight monitors
RBD
A reliable,
fully
distributed
block device
CEPH FS
A distributed
file system
with POSIX
semantics
Application Host / VM Client
15. WHAT ABOUT THE HARDWARE?
Original Design Manufacturers (ODM)
• Provide a more cost effective storage infrastructure
- ODMs operate on 10% GM versus 60-70%
• Strong ecosystem of design and build partners
• Have built strength in parts depots and field support
• Through necessity with many large web-scale
companies buying direct
• Intel white-box design helps fast-track their designs
• Quick release cycles and adopt newer technology
quickly
17. File & Object Services Node
(Provides S3, Swift, NFS, CIFS)
Performance Data Node
(2 SSDs, 10 HDDs)
Capacity Data Node
(6 SSDs, 30 HDDs)
Starts small,
Expands to
multiple
racks
Applications
OPEN
EXASCAL
E
STORAGE
18. OPEN
EXASCAL
E
STORAGE
RADOS RBD NFS CIFSS3 Swift
Object Pools Block Pools
Self-
Healing
Auto-
Balancing
Scrubbing
Thin
Provisioning
Snapshots
HA Ethernet
CRUSH Map & Placement Groups
VolumesBuckets VolumesBuckets
Cap Data Nodes Perf Data NodesHigh Cap Data Nodes
SSDs HDDs SSDs HDDsSSDs HDDs
Software-defined Storage
Delivered on Qualified Hardware
Support for Hardware & Software
19. of the Top 25
Service Providers
in North America
Several of the Top
Service Providers
in Europe
6
Including
TRUST
VISION
PERFORMANCE
COST
Proven Performance
VOD, Cloud DVR
need consistent speed
Multi-workload &
Open Source
Economics of SDS
are Strong
FEATURES
Fast-paced community
development
WHY?
CUSTOMERS
20. VIDEO OPENSTACK SYNC-N-SHARE BACK-UP / DR
OPEN STORAGE USE CASES
Power
next generation
video services with
advanced, proven
storage.
Simplify
storage management
and service provisioning
for OpenStack.
Solve
the “shadow IT” problem
– maintain compliance
and secure intellectual
property
Utilize
the cost economics and
scaling of object on one
multi-workload system
21. • Media file storage for customer-facing application
• Drop-in replacement for legacy object backend
• Minimal resistance to increasing scale
• Multi-protocol capable for future service
• Fast transaction capable
FOR MEDIA