At Meet Magento NY 2015 Kevin Bortnick, Senior Magento Solutions Architect of Clustrix, hosted a workshop titled, "Scaling Techniques to Increase Magento Capacity." There he spoke about scaling strategies used to overcome performance bottlenecks associated with the MySQL database used by most Magento implementations. Kevin highlighted the shortcomings of ‘read slaves’, ‘multiple masters’ and ‘sharding’ and shared is his real world experiences. Check out Kevin's presentation notes on how scale-out database opens new possibilities for scaling to meet these demands either in the datacenter or cloud.
2. PROPRIETARY AND CONFIDENTIAL2
• Kevin is a Certified Magento Developer. He has built and maintained some of the
largest and most complex Magento-powered E-commerce sites.
• Kevin lives in Las Vegas and is currently the Magento Solutions Architect for Clustrix.
3. As a Magento Architect — I think about the future
How easy is it to maintain?
Can someone else support it?
Can it handle growth?
Can it be used for something else?
When is it going to break?
PROPRIETARY AND CONFIDENTIAL3
4. Symptoms of a Problem?
• Increased page load time
•
• Spikes and Errors
•
• Site Crashes
•
• Support Tickets
PROPRIETARY AND CONFIDENTIAL4
5. If you are seeing issues…
• Look in the obvious places
– Scale your Magento web nodes
– Add Memcached or Redis
– Add Varnish
• If everything else is tuned right, the
database becomes the next
bottleneck
PROPRIETARY AND CONFIDENTIAL5
6. Ways of Extending MySQL
PROPRIETARY AND CONFIDENTIAL6
Faster Hardware?
Read Slaves?
Multi Master? Partitioning?
Percona?
NoSQL?
Clustrix?
7. Report Card Grading
7
Report Card
MySQL Compatible ✔
Improves Performance ✔
Scales Reads ✔
Scales Writes ✔
No Application Changes
No Single Point of Failure
8. Percona
• A drop-in replacement for MySQL
• Pros:
– Easy to set up and use
– Retains consistent
performance at high resource
utilization
• Cons:
– Still has MySQL's architecture
limitations
– No high availability or disaster
recovery
8
Report Card
MySQL Compatible ✔
Improves Performance ✔
Scales Reads ✔
Scales Writes Limited
No Application Changes ✔
No Single Point of Failure
9. Faster Hardware
• Pros:
– Immediate performance improvements
• Cons:
– Limit to how large you can go
– High-end hardware can get very
expensive, especially when you include
backup servers
– Excess hardware during off peak
season
9
Report Card
MySQL Compatible ✔
Improves Performance ✔
Scales Reads Limited
Scales Writes Limited
No Application Changes ✔
No Single Point of Failure
10. Master with Read Slaves
• Pros:
– Easy to do
– Magento has this feature built in
• Cons:
– Does not help solve write
checkout bottlenecks
– Eventual consistency
10
Master
SlaveSlave
Report Card
MySQL Compatible ✔
Improves Performance ✔
Scales Reads ✔
Scales Writes
No Application Changes ✔
No Single Point of Failure
11. Master/Master, Store/Admin
• Pros:
– Works well if you do a lot of
updates
– Helps for 24 hour stores
• Cons:
– Have to accommodate edge
cases
– DBA needs to monitor for
replication issues
– Won't fix front-end only
bottlenecks
11
Store
SlaveSlave
Admin
Report Card
MySQL Compatible ✔
Improves Performance ✔
Scales Reads ✔
Scales Writes
No Application Changes
No Single Point of Failure
12. True Multi Master
• Pros
– Can deliver better overall
performance
• Cons
– Application must keep track of
orders and inventory
– Latency can cause sync issues
– Expensive to develop and maintain
– Complexity increases and stability
decreases
12
1
1b1a
2
2b2a
Report Card
MySQL Compatible ✔
Improves Performance ✔
Scales Reads ✔
Scales Writes ✔
No Application Changes
No Single Point of Failure
13. Partitioning (Magento 2 feature CQRS)
• Pros
– Able to distribute load
• Cons
– Can’t join between partitioned
segments
– Amount of tables able to be
partitioned is limited
– Harder to maintain—need to track
configurations
13
Report Card
MySQL Compatible ✔
Improves Performance ✔
Scales Reads ✔
Scales Writes Limited
No Application Changes Limited
No Single Point of Failure
14. NoSQL
• Pros:
– Runs very fast in non-ACID mode
• Cons:
– Gains speed from short-cutting of ACID
– Commits before writing to disk
– No cross-document transactions
– NoSQL requires significant application
development to swap-out MySQL
14
ACID = Transaction and Data Safety
Report Card
MySQL Compatible
Improves Performance
Scales Reads In non-
ACID
mode
Scales Writes ✔
No Application Changes Limited
No Single Point of Failure
Don’t Drop
ACID
15. ClustrixDB
15
• Pros
– Drop-In Replacement for MySQL
– Scales READS and WRITES
– Distributed
– Shared Nothing
– Data Redundancy
– Built in high availability solution
– Low cost disaster recovery
– Elastic in the cloud and data
center
– Seasonal Flex, Self-Managing
Report Card
MySQL Compatible ✔
Improves Performance ✔
Scales Reads ✔
Scales Writes ✔
No Application Changes ✔
No Single Point of Failure ✔
20. Wrap Up – Think About the Future
How easy is it to maintain
Can someone else support it
Can it handle growth
Can it be used for something else
When is it going to break
20
Report Card
MySQL Compatible ✔
Improves Performance ✔
Scales Reads ✔
Scales Writes ✔
No Application Changes ✔
No Single Point of Failure ✔
23. ClustrixDB Key Differentiators
Automatic, 100%
fault tolerance
Availability
No single
point of failure
Battle-tested
performance
Flex up and
down, in minutes
Massive,
linear scalability
Capacity
Extreme
concurrency
Productivity
Eliminates
re-architecting
the database
Plug-in MySQL
compatibility
Self-managing
operation
PROPRIETARY AND CONFIDENTIAL23
Engineered
for any cloud
One-click
installation
Cloud
Seamless
scale-out
Editor's Notes
Ecommerce success creates site failure
Holiday
Flash sale
You will hit something
Ways to fix your upcoming capacity problem
Cloud
Designed for seamless installation and scale-out on any cloud-based infrastructure
Capacity
Flex up and down, in minutes - If you need more capacity or performance, just connect and go.
Massive, linear scalability - Readily handles massive volumes of customers, carts, orders, products, and business performance reporting
Extreme concurrency - Manages millions of concurrent actions without impacting site response time
Availability
Automatic, 100% fault tolerance – High availability architecture to meet always-on demands of business-critical operations
No single point of failure – No impact from hardware outages; zero downtime
Battle-tested performance – Proven rock solid performance at some the world’s fastest-growing companies. E.g., maintained superior service levels during 600% Cyber Monday sales spike (Choxi)
Productivity
Plug-in MySQL compatibility - Deploy in days, with few or no code changes. ClustrixDB is compatible with any application that uses MySQL, including the popular Magento platform for e-commerce and internally developed solutions.
Eliminates re-architecting the database - Does away with complicated scaling strategies like sharding and replication, which are expensive, labor-intensive, and ultimately unsustainable
Self-managing operation – Virtually eliminates DBA operations tasks because the management is built into the database itself