Traditional databases struggle to scale with the rapid growth of e-commerce, leading to performance issues that can result in lost sales and profits. The document explores various scaling methodologies, their pros and cons, and highlights how newer solutions like ClustrixDB address these limitations by offering better performance, uptime, and adaptability for high traffic scenarios. Ultimately, it emphasizes the importance of selecting the right database technology to ensure business success in e-commerce.

1. Why
Tradi*onal
Databases
Fail
so
Miserably
to
Scale
With
E-­‐Commerce
Growth

2. Dragon
Slayer
Consul*ng
–
Marc
Staimer
CDS
Dragon
Slayer
Consul9ng
All
Rights
Reserved
©
2014
2
marcstaimer@me.com
Analyst/Consultant
IT
Industry
Experience
Publish
w/TT
&
GigaOM
Speak
@
Trade
Shows
Improve
Vendors
Marke9ng
Help
EUs
w/Problems

3. Agenda
Storage
Decisions
Conference
|
©
TechTarget

4. It’s
What
We
Know
Is
True
When
It’s
Not
That
Causes
The
Most
Difficult
Problems
Storage
Decisions
Conference
|
©
TechTarget
If
I
Catch
You,
You’re
Mine

5. SQL
Database
E-­‐Commerce
Performance
Problem
! E-­‐Commerce
mission
cri9cal
! Slow
performance
and/or
outages
=
! Lost
sales
! Lost
revenue
! Lost
profits
8/5/14
Dragon
Slayer
Consul9ng
All
Rights
Reserved
©
2014
5

6. When
The
IOPS
BoOleneck
is
Storage
! >
DRAM
! Fast
with
lots
of
random
IOPS
! Expensive
&
vola9le
! >
HDDs
! Rela9vely
inexpensive
! Not
as
many
random
IOPS
so
need
lots
of
them
! >
suppor9ng
infrastructure
! >
exper9se
required
(short-­‐stroking)
8/5/14
Dragon
Slayer
Consul9ng
All
Rights
Reserved
©
2014
6

7. The
New
Darling:
Flash
Storage
! DDR
DIMM
! PCIe
! SAS/SATA
SSD
! Hybrid
array
! All
flash
array
(AFA)
8/5/14
Dragon
Slayer
Consul9ng
All
Rights
Reserved
©
2014
7

8. Flash
ShiRs
SQL
DBMS
Performance
BoOleneck
Fall
2014
Dragon
Slayer
Consul9ng
All
Rights
Reserved
©
2014
8

9. Making
SQL
DBMS
Performance
A
“Scale”
Problem
! Complicated
! Requiring
a
lot
of
exper9se
! Costs
are
excessive
! Many
tasks
are
con9nuously
manually
labor-­‐intensive
! Reduced
Up9me

10. Common
SQL
Database
Scaling
Workarounds
! Scale-­‐Up
! Scale-­‐Out
! Sharding
! Read-­‐only
Slaves
! Peer-­‐to-­‐Peer
Replica9on
! Linked
Servers
&
Distributed
Queries
! Distributed
Par99on
Views
! Data
Par99on
Rou9ng
! NoSQL
Databases
Fall
2014
Dragon
Slayer
Consul9ng
All
Rights
Reserved
©
2014
10

11. Scale-­‐Up
! More
cores
! More
DRAM
! More
system
IO
bandwidth
! More
power
Fall
2014
Dragon
Slayer
Consul9ng
All
Rights
Reserved
©
2014
11

12. Scale-­‐Up
Pros
&
Cons
Pros
! Simpler
–
easier
! No
sodware
changes
! Nothing
addi9onal
to
manage
! Or
train/learn
Cons
! Cost
$$$$
! >
Server
HW
CapEx
! >
Server
HW
OpEx
! (Servers
oden
proprietary)
! >
DBMS
licensing
costs
! >
HA
costs
Fall
2014
Dragon
Slayer
Consul9ng
All
Rights
Reserved
©
2014
12

13. Scale-­‐Out:
Sharding
! Breaks
up
the
database
! Into
separate
instances
! Easy
to
understand
! Fairly
common
especially
with
! MySQL
! SQL
Server
! PostgreSQL
Fall
2014
Dragon
Slayer
Consul9ng
All
Rights
Reserved
©
2014
13

14. Sharding
Pros
&
Cons
Pros
! Data
split
along
natural
fault
lines
! Highly
scalable
! Many
DBAs
know
how
! Non-­‐DBAs
do
not
Cons
! Mul9ple
DBMS
instances
>
license
$
! Requires
lots
of
exper9se
! App
&
SQL
DBMS
data
structures
! Time
consuming:
labor
intensive
! Error
prone
! Non-­‐dynamic
! Rela9onship
loss
! Too
many
SPOFs
Fall
2014
Dragon
Slayer
Consul9ng
All
Rights
Reserved
©
2014
14

15. Scale-­‐Out:
Read-­‐Only
Slaves
! Master
database
that
replicates
! To
a
series
of
slave
databases
! Slaves
are
read-­‐only
! #
of
slaves
varies
! Limited
only
by
master’s
replica9on
ability
! Speed
! Affect
on
DBMS
performance
Fall
2014
Dragon
Slayer
Consul9ng
All
Rights
Reserved
©
2014
15
Slaved
SQL
DBMS
Read-­‐Only
Database
Files

16. Read-­‐Only
Slaves
Pros
&
Cons
Pros
! Nominal
setup
! Simple
! App/DBMS
transparent
! Incredibly
Popular
Cons
! Master
database
bolleneck
! Not
good
for
write
intensive
apps
! Master
database
SPOF
! Patches/upgrades/fixes
! Mul9ple
DBMS
instances
=
>
$$$$
Fall
2014
Dragon
Slayer
Consul9ng
All
Rights
Reserved
©
2014
16

17. Scale-­‐Out:
Peer-­‐to-­‐Peer
w/Replica*on
! Mul9ple
database
copies
! Each
database
engine
manages
and
maintains
its
own
DBMS
copy
! Replica9on
updates
other
database
engines
when
there’s
a
change
Fall
2014
Dragon
Slayer
Consul9ng
All
Rights
Reserved
©
2014
17

18. Peer-­‐to-­‐Peer
w/Replica*on
Pros
&
Cons
Pros
! Mul9ple
database
copies
! No
SPOF
! Good
for
infrequent
updates
Cons
! Complicated
opera9ons
! Conflict
resolu9on
is
difficult
! Stewardship:
labor-­‐intensive
! Merge
replica9on:
high
overhead
! Mul9ple
DBMS
instances
=
>
$$$$
Fall
2014
Dragon
Slayer
Consul9ng
All
Rights
Reserved
©
2014
18

19. Scale-­‐Out:
Linked
Servers
&
Distributed
Queries
! U9lized
when:
! Databases
are
split
into
func9onal
areas
! Requiring
minimal
coupling
! Or
when
spliong
data
by
type
Fall
2014
Dragon
Slayer
Consul9ng
All
Rights
Reserved
©
2014
19
Client
App
Server
DBMS
Server
1
DBMS
Server
2

20. Linked
Servers
&
Distributed
Queries
Pros
&
Cons
Pros
! Makes
mul9-­‐DBMS
appear
as
1
! DBMS
update
frequency
! Has
no
impact
! DBMS
split
into
func9onal
areas
! Minimal
coupling
! When
spliong
data
by
type
Cons
! Referen9al
integrity
constraints
! Queries
spanning
mul9ple
DBMS
! =
>
latencies
<
performance
! Significant
exper9se
required
! Difficult
to
pull
off
successfully
! Mul9ple
DBMS
instances
=
>
$$$$
Fall
2014
Dragon
Slayer
Consul9ng
All
Rights
Reserved
©
2014
20

21. Scale-­‐Out:
Distributed
Par**on
Views
(DPV)
! Table
data
par99oned
among
tables
in
numerous
distributed
databases
! Based
on
a
par99oning
key
! Update-­‐intensive
apps
best
target
for
DPVs
! Generally
good
E-­‐Commerce
query
performance
! Few
rows
impacted
–
likely
processed
in
single
DBMS
Fall
2014
Dragon
Slayer
Consul9ng
All
Rights
Reserved
©
2014
21

22. Distributed
Par**on
Views
(DPV)
Pros
&
Cons
Pros
! Good
for
update
intensive
Apps
! App
transparent
! Scale-­‐out
par99oned
data
! Table
data
par99oned
into
tables
! Mul9ple
DBMS
Cons
! Data
movement
between
par99ons
! Slows
performance
! Ltd
by
table’s
par9onability
! Mul9ple
SPOFs
! Significant
exper9se
required
! Difficult
to
pull
off
successfully
! Mul9ple
DBMS
instances
=
>
$$$$
Fall
2014
Dragon
Slayer
Consul9ng
All
Rights
Reserved
©
2014
22

23. Scale-­‐Out:
Data
Dependent
Rou*ng
(DDR)
! E-­‐Commerce
app
or
middleware
responsible
for
DBMS
query
rou9ng
! Premise:
they
have
deeper
understanding
of
the
data
! Popular
when
queries
distributed
across
hundreds
or
more
databases
Fall
2014
Dragon
Slayer
Consul9ng
All
Rights
Reserved
©
2014
23
Client
Middleware
or
data
layer
routed
to
SQL
DBMS
w/data
requested

24. Data
Dependent
Rou*ng
(DDR)
Pros
&
Cons
Pros
! Good
when
queries
spread
! Across
100s
or
>
DBMS
! Apps
know
their
own
data
! Beler
query
decisions
Cons
! Significant
exper9se
required
! Deep
understanding
of
apps
! Or
middleware
! Strong
architectural
skills
! Massively
complicated
! Based
on
#
of
DBMS
servers
! ALL
apps
must
be
customized
! Mul9ple
DBMS
instances
=
>
$$$$
Fall
2014
Dragon
Slayer
Consul9ng
All
Rights
Reserved
©
2014
24

25. Scale-­‐Out:
NoSQL
Databases
! Excellent
for
large
data
sets
and
analy9cs
in-­‐general
! OLAP
! Data
Warehousing
! Unstructured
data
analy9cs
Fall
2014
Dragon
Slayer
Consul9ng
All
Rights
Reserved
©
2014
25

26. NoSQL
Databases
Pros
&
Cons
Pros
! Data
Ingest
! Unstructured
data
analy9cs
! OLAP
! Data
Warehousing
! Historical
data
Cons
! Poor
transac9onal
performance
! Lack
real-­‐9me
dashboards
! &
Real-­‐9me
repor9ng
! Not
good
@
scanning
data
! As
it’s
changing
“NOW”
! Lacks
common
SQL
DBMS
tools
! Technology
s9ll
immature
Fall
2014
Dragon
Slayer
Consul9ng
All
Rights
Reserved
©
2014
26

27. Conclusion
&
Summary
! Tradi9onal
scaling
methodologies
have
serious
issues
ranging
from
! Too
much
cost
$$$$
! Too
complicated
! Up
front
&
ongoing
! Too
labor
intensive
! Too
much
down9me
! Trouble
shoo9ng
excessively
difficult
! Too
much
frustra9on
Fall
2014
Dragon
Slayer
Consul9ng
All
Rights
Reserved
©
2014
27

28. The
Test
of
Three
Fall
2014
Dragon
Slayer
Consul9ng
All
Rights
Reserved
©
2014
28

29. © 2014 CLUSTRIX

30. Software
launched
San Francisco
San Jose
Seattle
New York
London
Company
Offices Worldwide Top-Tier Venture Funding
1 Trillion+ transactions
per month
100s of installations worldwide
Clustrix
founded
Mission-critical
production
deployments
20142006 2010 2013
ClustrixDB Overview30

31. Without the right database,
e-commerce success
leads to website slowdown or failure
Slowdowns
Outages
Unhappy
Customers
Lost
Revenue
ClustrixDB Overview31

32. More
Uptime
More
Capacity
More
Business
The first e-commerce database
purpose-built for today’s online retailers
ClustrixDB Overview32

33. ClustrixDB Overview
Today’s E-Commerce Database
• Readily handles e-commerce growth
• Easy accommodates traffic spikes
• Meets no-downtime demands
• Supports on-the-fly updates
• Enables live e-commerce reporting
• Integrates with e-commerce stack
• Replaces MySQL with minimal effort
ClustrixDB Overview33

34. Serve fast-growing mobile
and social channels
Business Advantages
Ready for
Revenue Growth
Peak Performance
at Peak Times
Capture More
Business
Scale effortlessly with
sales success
Avoid lost revenue
during traffic spikes
Readily manage
unpredictable demand
Analyze and act on
real-time sales data
Better orchestrate
omni-channel shopping
ClustrixDB Overview34

35. E-commerce leaders across retail, travel, digital services,
and social commerce rely on ClustrixDB
Customer Highlights
ClustrixDB Overview35

36. E-Retailer Takes 600% Sales Spike in Stride
“Three years ago, Black Friday was horrible. We wasted money
having database issues. With Clustrix, those issues are gone.”
Challenge ClustrixDB
• Second-fastest growth rate in
Internet Retailer Top 500
• Loads spike 20X or more
during holidays
• Hours-long downtime incidents,
with $20K-$60K in lost revenue
per hour
• Flexed up and down to
handle 600% Cyber Monday
sales spike with no downtime
• Real-time catalog updates
during flash sales
• Live reporting and analytics
on e-commerce business
Keith Bussey
VP Technology
ClustrixDB Overview36

37. Online Textbook Company Graduates from MySQL
“We wouldn’t be able to grow our business as easily
without ClustrixDB.”
Challenge ClustrixDB
• 3X revenue growth strained
chaotic MySQL backend
• Significant administrative
overhead required for
redundancy and uptime
• Large schema changes needed
system-wide downtime
• 100% increase in database
load with no performance hits
• Simplified operations with
easy expansion and
seamless datacenter failover
• Went from 10 hours of
downtime to no downtime
with on-the-fly changes
Paolo Resmini
VP of Engineering
ClustrixDB Overview37

38. Picture-Perfect Performance for Digital Photo Service
“Clustrix has given us one advantage over our competitors…
the ability to stay open at peak times with great service levels.”
Challenge ClustrixDB
• Highly seasonal business
with 10X spike in December
• Need to ensure uptime
during peak loads for
30 million members
• Reached capacity limits of
conventional database
• Gained performance and
scale with no app rewrite
• 89% reduction in December
service incidents
• More time on innovation,
less time on database worries
Graham Hobson
CTO
ClustrixDB Overview38

39. Online Travel Site Increases Uptime, Lowers TCO
“MakeMyTrip frequently experiences 50% spikes in traffic that
no longer cause slow page load times and costly downtime.”
Challenge ClustrixDB
• Two server MySQL config
with single point of failure
• Slow page loads and costly
downtime
• Sharding / MySQL Cluster
too expensive to implement
and maintain
• Drop-in replacement with
no single point failure
• $100K cost reduction with
easy expansion
• Handles massive spikes in
traffic with faster web
response times
Sanjay Kharb
Director Technical Ops
ClustrixDB Overview39

40. Automatic,
100% fault tolerance
Flex up and down,
in minutes
Massive, linear scalability
Technical Advantages
Capacity Availability Productivity
Extreme concurrency
No single point of failure
Battle-tested performance
Eliminates re-architecting
the database
Plug-in MySQL compatibility
Self-managing operation
ClustrixDB Overview40

41. Clustrix Design!
Intelligent Data
Distribution
Massively Parallel
Query Processing
Shared Nothing
Architecture
SQL
SQL
SQL
SQL
SQL
ClustrixDB Overview41
Query Compiler
Data Map
Database Engine
Query Compiler
Data Map
Database Engine
Query Compiler
Data Map
Database Engine

42. Signs It’s Time for ClustrixDB
• Capacity planning for a busy shopping season
• Gearing up for mobile and social demand
• Issues with website slowdowns or outages
• Scaling is too costly, complex and time consuming
• Database is a single point of failure
• Business wants new e-commerce capabilities
ClustrixDB Overview42

43. Thank You.
facebook.com/clustrix
ecommerce.clustrix.com
@clustrix
linkedin.com/clustrix
ClustrixDB Overview43