Storage Architectures For The Modern Data Centre presented by Chris Evans in London, UK on February 25, 2016

1. STORAGE ARCHITECTURES FOR THE MODERN DATA
CENTRE
Chris
M
Evans
Langton
Blue
Ltd
Copyright
©
2016
Langton
Blue
Ltd 1

2. A
WORD ABOUT ME
Copyright
©
2009-­‐2016 Langton
Blue
Ltd 2
§ 28
years
experience
in
the
IT
industry
across
all
platforms,
including
IBM
mainframe,
Windows
&
Open
Systems.
§ Co-­‐founder
and
independent
consultant
at
Langton
Blue
Ltd,
a
specialist
consulting
company
in
the
UK.
§ Blogger
and
part-­‐time
analyst
at
Architecting
IT
§ Twitter:
@chrismevans,
@architectingIT,
@langtonblue
§ Web:
www.architecting.it,
www.langtonblue.com

3. AGENDA
§ Evolution
of
The
Data
Centre
§ Understanding
Customer
Needs
§ Storage
Architectures
§ Product
Marketplace
§ Q&A
Copyright
©
2009-­‐2016 Langton
Blue
Ltd 3

4. EVOLUTION
OF
THE
DATA
CENTRE
A
quick
primer
on
storage
heritage
Copyright
©
2009-­‐2016 Langton
Blue
Ltd 4

5. 20TH CENTURY DESIGN
§ Monolithic
§ Bespoke
hardware
§ Expensive
to
acquire,
maintain
&
manage
§ Static
designs
and
configuration
§ (Think
time
taken
to
run
binfile
change)
§ Limited
or
no
data
services
§ No
RAID,
replication,
snapshots
§ All
features
that
developed
over
time
Copyright
©
2009-­‐2016 Langton
Blue
Ltd 5

6. 20TH CENTURY ARCHITECTURE
§ Custom
hardware
designs
&
components
still
reign
§ Hosts
own
LUNs
mapped
from
physical
RAID
groups
§ Performance
defined
by
RAID
group
size
and
disk
speed
§ Wide
striping
for
performance
§ Throughput
&
I/O
latency
influenced
(and
restricted)
by
controller
capabilities
§ Lots
of
manual
effort
to
rebalance
systems,
especially
at
scale
§ LUN
migration,
rebuilds,
reliance
on
host-­‐level
LVMs
for
a
lot
of
the
work
§ Lots
of
risk
fragmentation
and
orphan
storage
§ Wasted
space
in
RAID
groups
Copyright
©
2009-­‐2016 Langton
Blue
Ltd 6
CONTROLLER
A CONTROLLER
B
BED BED BED BED BED BED BED BED

7. 21ST CENTURY TECHNOLOGY
§ Vendors
have
moved
away
from
bespoke
hardware
designs
§ x86
Standardisation
§ Commodity,
reliable
components
§ High
performance
processors
&
memory
– Moore’s
Law
§ Interoperability
§ Components
generally
work
together
without
problems
§ Storage
protocols
are
robust
and
mature
(still
also
developing
– NVMe)
§ New
Technology
§ Flash
§ NVDIMM
§ 3D
Xpoint
§ Migration
of
“smarts”
to
software
-­‐ Software
Defined
Storage
§ Features
implemented
in
code,
not
microcode
Copyright
©
2009-­‐2016 Langton
Blue
Ltd 7

8. FLASH &
NEW MEDIA
§ Designs
§ SLC
(Single
Level
Cell),
MLC
(Multi-­‐level
Cell),
TLC
(Triple
Level
Cell)
§ 3D-­‐NAND
– 3-­‐D
rather
than
planar
(2D)
technology
for
higher
density
§ 3D-­‐Xpoint
– faster
and
more
scalable
persistent
storage
than
Flash,
slower
than
DRAM
and
in
between
on
price
§ Intel/Micron
joint
development
§ NVRAM/NVDIMM
– DIMM
form
factor
products
delivering
persistent
flash
memory
on
the
motherboard
§ NVMe – new
connectivity
standard
for
writing
to
persistent
storage
§ PCIe and
SDD
type
form
factors
(not
yet
widely
adopted)
Copyright
©
2009-­‐2016 Langton
Blue
Ltd 8

9. PROCESSOR &
MEMORY PERFORMANCE
Copyright
©
2009-­‐2016 Langton
Blue
Ltd 9
§ Processor,
memory
and
bus
speeds
are
increasing
§ Processor
performance
and
bus
speeds
follow
Moore’s
Law
§ Bus
speeds
double
with
each
release
§ PCIe
v1.x
=
250MB/s
(per
lane)
-­‐ 2003
§ PCIe
v2.x
=
500MB/s
(per
lane)
-­‐ 2007
§ PCIe
v3.0
=
985MB/s
(per
lane)
-­‐ 2010
§ PCIe
v4.0
=
1969MB/s
(per
lane)
– 2014
§ Features
much
easier
to
deliver
from
software
on
x86
§ New
instruction
sets
like
SSSE3
and
AVX2
allow
storage
computations
like
erasure
coding/Reed
Solomon
to
be
done
in
software
§ Fewer
reasons
to
develop
bespoke
ASICs
and
FGPAs

10. SOFTWARE DEFINED STORAGE(KEY FEATURES)
§ Abstraction – I/O
services
should
be
delivered
independent
of
the
underlying
hardware,
through
logical
constructs
like
LUNs,
volumes,
file
shares
and
repositories.
§ Automation – resources
should
be
consumed
using
CLIs
and
APIs
rather
than
manually
allocated
through
a
GUI.
§ Policy/Service
Driven
– the
service
received
(IOPS,
latency)
should
be
established
by
policies
that
implement
Quality
of
Service,
availability
and
resiliency.
§ Scalable – solutions
should
enable
performance
&
capacity
scaling
independent
of
I/O
delivery.
Copyright
©
2009-­‐2016 Langton
Blue
Ltd 10

11. MAJOR STORAGE THEMES
§ More
choice
in
products
(both
hardware
and
software
than
ever
before)
§ Divergence
– storage
platforms
for
specific
purposes
(no
single
platform
for
all
requirements)
§ Object
storage,
cloud,
backup,
primary
§ Convergence
– collapsing
the
storage/server
hardware
into
a
single
unit
integrating
new
technology
(converged
&
hyper-­‐converged)
§ Integration
– VVOLs,
APIs,
platform
drivers
(e.g.
Cinder),
hypervisor
extensions
(VASA,
VAAI)
§ New
Media
§ Flash
and
its
successors
Copyright
©
2009-­‐2016 Langton
Blue
Ltd 11

12. UNDERSTANDING
CUSTOMER
NEEDS
The
changing
face
of
application
deployment
Copyright
©
2009-­‐2016 Langton
Blue
Ltd 12

13. WHAT ARE CUSTOMERS’
PAIN POINTS
§ Cost
§ Performance
§ Operational
Complexity
§ Reliability
§ Floor
Space
§ Power
Consumption
Copyright
©
2009-­‐2016 Langton
Blue
Ltd 13
Source:
Tintri State
of
Storage
Report,
March
2015
https://www.tintri.com/sites/default/files/fie ld/pdf/whitepapers/state_of_storage_ infographic_150513t10216.pdf

14. WHAT DO CUSTOMERS CARE ABOUT
§ Efficiency
§ Cost
($/GB,
effective
or
raw)
§ Physical
media
§ Performance
§ Latency,
IOPS,
throughput,
Quality
of
Service
§ Management
§ Multi-­‐tenancy,
APIs
(Cinder),
Automation,
Abstraction
(VVOLs)
§ Reliability
§ Resiliency,
Availability
§ Features
§ Data
Services,
Data
Protection
§ Breaking
the
buying
cycle
§ No
forklift
upgrades
or
3-­‐4
year
refresh
projects
Copyright
©
2009-­‐2016 Langton
Blue
Ltd 14

15. PREDICTABLYUNPREDICTABLE
§ Workloads
becoming
more
random
in
nature
§ I/O
blender
§ Peaks
and
troughs
in
demand
§ VDI
(Bootstorms,
logoff
storms)
§ Increase
in
I/O
Density
§ Driven
by
virtualisation
and
soon
containers
§ Agility
§ Desire
to
move
data
around,
spin
up
copies,
create
test
dev environments,
destroy
and
recreate
§ Persistence
§ Applications
expecting
to
be
transient
in
nature,
but
more
need
for
storage
to
be
100%
available
across
the
entire
infrastructure
Copyright
©
2009-­‐2016 Langton
Blue
Ltd 15

16. VIRTUAL FIRST STRATEGY
§ Provided
the
ability
to
consolidate
over-­‐provisioned
server
resources
§ Systems
typically
25%
allocated
§ Reduced
delivery
time
for
(virtual)
servers
from
days/months
to
hours
§ Assuming
sufficient
capacity
available
§ Enabled
high
availability
through
infrastructure
§ No
need
to
deploy
clustered
systems
§ Centralised
data
protection
(e.g.
VADP)
§ Centralised
storage
and
virtualisation
don’t
play
well
together
§ Storage
doesn’t
understand
VMs
§ Storage
sees
LUNs
&
volumes
§ Virtualisation
sees
VMs
§ LUNs
encapsulate
many
volumes
– no
visibility
of
VMs
§ Problems
being
worked
around
with
VVOLs
and
other
solutions
that
integrate
directly
into
hyper-­‐converged
platforms.
Copyright
©
2009-­‐2016 Langton
Blue
Ltd 16

17. TECHNICAL REQUIREMENTS– QUALITY OF SERVICE
§ Eliminate
the
“noisy
neighbour”
problem
§ Provide
storage
resources
based
on
policy
§ Ensure
storage
isn’t
under-­‐delivering
§ Guarantee
IOPS,
performance
minima
§ Ensure
storage
isn’t
over-­‐delivering
§ 1st on
the
array
gets
best
performance
(until
there
are
more)
§ Set
limits
and
prioritisation
§ Apply
to
VM/VMDK
where
possible
Copyright
©
2009-­‐2016 Langton
Blue
Ltd 17

18. TECHNICAL REQUIREMENTS-­‐ API
DRIVEN
§ Admin-­‐based
functionality
driven
by
API
§ Ability
to
create,
manage,
destroy
LUNs
from
code
§ Integrate
storage
functions
into
workflow
§ Cloud,
Automation,
Self
Service
§ Reduce
dependence
on
manual
process
§ Agility
Copyright
©
2009-­‐2016 Langton
Blue
Ltd 18

19. TECHNICAL REQUIREMENT – SUPPORT VIRTUAL
§ Reduce
waste
from
over-­‐provisioning
§ Support
– object
(e.g.
VM-­‐based)
management
§ Apply
policies
at
the
VM
level
§ Data
location,
performance,
latency,
backup
§ VM
friendly
for
data
protection
(snapshots/replication)
§ Offload
heavy
lifting
tasks
(e.g.
VAAI)
Copyright
©
2009-­‐2016 Langton
Blue
Ltd 19

20. STORAGE
ARCHITECTURES
A
few
words
about
design
Copyright
©
2009-­‐2016 Langton
Blue
Ltd 20

21. DIVERSITY OF STORAGE PLATFORMS
Copyright
©
2009-­‐2016 Langton
Blue
Ltd 21
HYBRID
COMMODITY
OPEN-­‐
SOURCE
APP/DATA
AWARE
SMART
STORAGE
OBJECT
STORES
SCALE-­‐OUT
SAN
SCALE-­‐OUT
NAS
ALL-­‐FLASH
SOFTWARE
DEFINED
SCALE-­‐OUT
BACKUP
ULTRA-­‐FAST
FLASH
HIGH-­‐CAPACITY
FLASH
HPC

22. INFRASTRUCTURE CONSUMPTION CONTINUUM
Copyright
©
2009-­‐2016 Langton
Blue
Ltd 22
As-­‐a-­‐Service
Hyper-­‐
converged
Converged
Proprietary
Appliance
Software
on
Commodity
SMB/Start-­‐ups Large
Enterprise Hyperscale &
SP
Ease
of
Implementation
Implementation
Flexibility
Degree
of
Vendor
Lock-­‐in
Cost
Efficiency
Harder
More
Flexible
Less
Lock-­‐in
At
Large
Scale
Easier
Less
Flexible
More
Lock-­‐in
At
Small
Scale

23. SCALE UP
§ Typically
Dual
controller
implementation
§ Active/Active
&
Active/Passive
§ Throughput
limited
by
controller
performance
§ More
throughput
means
better
controller
capability
(CPU/memory)
§ I/O
performance
limited
by
media
§ Flash
based
implementations
failed
to
use
flash
to
fullest
capability
§ Older
designs
don’t
cater
for
write
endurance
issues
§ Automated
tiering capabilities
to
place
data
on
most
appropriate
technology
Copyright
©
2009-­‐2016 Langton
Blue
Ltd 23
CONTROLLER
A CONTROLLER
B

24. SCALE OUT (CLOSELY COUPLED)
§ Scale
out
through
adding
nodes
and/or
capacity
§ Tight
node
integration
either
paired
or
with
bespoke
backplane
(e.g.
PCIe/Infiniband)
§ Symmetric
design
– scale
with
nodes
of
similar
size/capacity
§ LUNs/Volumes
delivered
from
specific
nodes
– not
fully
load
balanced
§ Scale
out
limited
by
backplane
traffic
§ Add/remove
nodes
dynamically
Copyright
©
2009-­‐2016 Langton
Blue
Ltd 24
CTRL
A CTRL
B CTRL
C CTRL
D CTRL
E CTRL
F

25. SCALE OUT (LOOSELY COUPLED)
§ Fully
distributed
“Shared
Nothing”
architecture
§ Loose
node
coupling
(usually
IP-­‐based)
§ All
nodes
contribute
storage
(no
disk
shelves)
§ At
least
one
node
of
“spare”
capacity
required
across
the
cluster
§ More
nodes
means
more
efficiency
in
redundancy
§ Data
resiliency
based
on
replicating
data
between
nodes
§ Standard
hardware
(typically
1U
server)
design
§ Node
power
efficiency
&
size
become
important
§ High
scalability
§ Smaller
increments
in
adding
capacity/performance
§ More
complex
design
Copyright
©
2009-­‐2016 Langton
Blue
Ltd 25
NODE
A
NODE
B
NODE
C
NODE
D
NODE
E
NODE
F

26. HYPER-­‐CONVERGED
§ Single
hardware
device
for
storage
and
server
§ Node-­‐based
scale-­‐out
§ Integrated
hypervisor
&
storage
services
§ Storage
implemented
in
VM
or
hypervisor
kernel
§ Distributed
file
system
(scale-­‐out
shared
nothing)
§ Efficient
use
of
resources
§ Simplified
management
§ At
scale
probably
more
expensive
than
separate
components
§ Reduced
VM
density
§ Issues
managing
performance
vs
capacity
scaling
Copyright
©
2009-­‐2016 Langton
Blue
Ltd 26
NODE
A
NODE
B
NODE
C
NODE
D
NODE
E
NODE
F

27. HYPER-­‐CONVERGENCE -­‐ TCO
§ Simplified
§ Less
“plan”
time
-­‐ no
requirement
to
certify
and
test
components
§ Less
“build”
time
– deploy
a
new
node,
power
up
and
add
to
the
configuration
§ Less
“operate”
time
– single
management
console,
typically
integrated
with
the
hypervisor,
tight
hypervisor
awareness
§ Efficient
§ Scale
per
node;
less
wastage,
no
up-­‐front
hardware
acquisition
§ “Decommission”
time
almost
eliminated;
simply
add
new
nodes
and
evacuate/remove
old
nodes
over
time
§ Provides
overall
TCO
savings
(not
storage
specific)
§ Not
typically
suited
to
high-­‐scale
environments
Copyright
©
2009-­‐2016 Langton
Blue
Ltd 27

28. STORAGE VIRTUALISATION-­‐ EVOLUTION
§ Monolithic
– single
central
controller
§ Inline
in
the
data
path
§ Controller
maps
logical
to
“physical”
storage
§ data
management
features
built
into
the
controller
§ Not
highly
scalable
§ Centralised Metadata
– Distributed
Data
§ Central
metadata
functions
§ Data
distributed/replicated
across
many
devices
§ System
not
in
the
data
path
§ Separation
of
control
and
data
planes
§ Totally
Distributed
§ No
central
metadata
or
data
§ Data
distributed
across
many
devices
§ Fully
scalable
architecture
Copyright
©
2009-­‐2016 Langton
Blue
Ltd 28

29. ADVANCED SERVICES
§ Space
Efficiency
§ Compression
§ De-­‐duplication
§ Thin
Provisioning
§ Quality
of
Service
§ Object
abstraction
(VVOLS,
per
VM
services)
§ Application
consistent
snapshots/backup
Copyright
©
2009-­‐2016 Langton
Blue
Ltd 29

30. SPACE OPTIMISATION/EFFICIENCY
Copyright
©
2009-­‐2016 Langton
Blue
Ltd 30
Feature Compression De-­‐duplication Thin
Provisioning
Physical
space
reduction ✔ ✔ ❌
Architecture/metadata
dependent ❌ ✔ ✔
CPU Friendly ❌ ✔ ✔
Traditional
OLTP
Database
friendly ✔ ❌ ❌
VDI/VSI Friendly ❌ ✔ ✔
HDD
Friendly ✔ ❌ ✔
Flash
Friendly ✔ ✔ ✔

31. QUALITY OF SERVICE (AGAIN)
§ Assign
service-­‐based
metrics
to
application
workloads
§ Reduce/eliminate
the
“noisy
neighbour”
effect
§ Set
application
owner
“expectations”
on
performance
§ Avoid
first/last
application
syndrome
from
legacy
arrays
§ Metrics
§ Latency
§ IOPS
§ Throughput
§ Establish
maximum/minimum/burst
values
&
priority
Copyright
©
2009-­‐2016 Langton
Blue
Ltd 31

32. OBJECT ABSTRACTION
§ Implement
per-­‐VM
services
§ Abstract
away
from
LUNs/Volumes
§ Implement
object-­‐specific
services
§ Snapshots
§ Replication
§ Cloning
§ Application-­‐consistent
data
management
§ Examples
§ VVOLs
§ Object
Stores
Copyright
©
2009-­‐2016 Langton
Blue
Ltd 32

33. VENDOR
LANDSCAPE
A
look
at
the
product
marketplace
Copyright
©
2009-­‐2016 Langton
Blue
Ltd 33

34. STORAGE NASCAR
SLIDE!
Copyright
©
2009-­‐2016 Langton
Blue
Ltd 34

35. PUTTING IT ALL TOGETHER
§ Start
with
Requirements
§ Don’t
buy
what
you
don’t
need
§ Focus
on
your
sensitivities
(cost,
performance,
scalability)
§ See
the
“bigger
picture”
§ Don’t
just
storage
in
isolation
§ Don’t
buy
a
technology
§ Look
past
the
glamour
of
shiny
boxes!
§ Evaluate
your
options
§ Use
tools
to
create
an
equitable
test
environment
§ Use
online
resources
and
other
views
§ Wisdom
of
the
crowd
applies
Copyright
©
2009-­‐2016 Langton
Blue
Ltd 35