M|18 How Copart Switched to MariaDB and Reduced Costs During Growth

1. Copart at M|18
Copart + MariaDB
Feb 27th 2018

2. WHAT IS COPART?

3. Copart, Inc.
• Copart is a publicly traded company, founded in 1982 by Willis J. Johnson.
• Copart, Inc., headquartered in Dallas, Texas and provides real-time online vehicle auction and
remarketing services globally.
• Copart provides vehicle sellers with a range of services to process and sell salvage and clean title
vehicles over the Internet using its patented virtual auction technology.
151,000+ Vehicles on sale now!
200+ locations across 12 countries
120+ Auctions with 18,000 Cars sold per day
250,000 Live bids from 1,000+ Participants per day
A car sold every 50 secs
• Visit www.Copart.com for more details.
3

4. Copart, Inc.
4

5. COPART: A TECHNOLOGY LEADER

6. SOME OF THE TECHNOLOGIES USED @ Copart
6

7. DATABASE @ Copart
• Our AS400 data is currently estimated at 33 TB and growing.
• We currently have 3 TB in MariaDB and growing rapidly.
• On an average, we facilitate 300 TPS on AS400 every day.
• We currently have 160 TPS on MariaDB every day.
• Our technology ecosystem consists of our cutting-edge auctioning
system.
7

8. ORGANIZATIONAL DIRECTION
• Light and agile ecosystem to facilitate global expansion.
• Existing technology is becoming legacy.
• The organization is promoting open source technology.
• Operational and administrative costs are high.
• International market has stricter data compliance standards.
8

9. TECHNICAL REQUIREMENTS
• Highly scalable database platform.
• Technology solution should be relevant for long term.
• SQL and logical objects should be compatible to prevent full re-
architecture.
• Operational costs should be low and system stability and
availability should be high.
• Need encryption data at rest and in motion for international data
compliance.
9

10. MariaDB + REQUIREMENTS
• Technology is open source with an active community.
• MariaDB is SQL compliant.
• Lower operational cost and high stability.
• The platform supports functions and stored procedures.
• Supports encryption of data at rest and in motion.
• MariaDB fast tracks important features and patches.
• Minimum retraining required for the Developers.
• DBAs can pick up technology easily.
• MariaDB can be installed on commodity hardware.
1
0

11. WHAT WAS THE SELLING POINT?
• MariaDB is based on MySQL which has been a popular RDBMS for
a long time.
• Platform is becoming more popular and we want to be ahead of
the curve.
• Migrating structure and data from AS400 to MariaDB is fairly
simple.
• The cluster solution has very good HA/DR features.
• MariaDB enterprise support is GREAT!!!
1
1

12. LEARNINGS & WORKAROUNDS
• As ‘Sequence’ does not exist in MariaDB 10.1/10.2, we had to
create additional application logic to generate and manage
sequences.
• Mandatory primary key requirement for Galera cluster needed
tables and application logic redesign.​
• Galera is not optimized to support bulk loads, so ETL needed to be
configured for smaller commit sizes.
• Hibernate(JPA) needed to be customized to use the Read/Write
split feature in Galera. 1
2

13. MariaDB @ Copart
• Its been 3 years since we started with the implementation of MariaDB.
• We started with implementing MariaDB for non-critical applications.
• Today many of our business critical applications run off of MariaDB.
• Realtime pipeline standalone implementation including data replication,
StarQuest. Data will flow from source to website within a few seconds.
• We currently have 15 instances of Galera Clusters(3 nodes each).
• We also have multiple master-master and master-slave implementations.
1
3

14. MASTER-MASTER/MASTER-SLAVE PROD
IMPLEMENTATION

15. 1
5

16. GALERA CLUSTER MASTER-SLAVE
IMPLEMENTATION

17. 1
7

18. GALERA CLUSTER MULTI-INSTANCE
IMPLEMENTATION

19. 1
9

20. WHY MULTI-INSTANCE CLUSTER?
• Copart has multiple Non-Prod environments which need dedicated
clusters.
• Implementing 14 Non-Prod clusters on 42+14 servers is an
administrative overhead.
• This is a heavy resource utilization setup.
• Reduced to 9+3 powerful machines.
• MaxScale for read-write split and full load balanced.
2
0

21. COPART+MARIADB: FUTURE!!!

22. WHAT'S BAKING?
• ColumnStore and MariaDB AX implementation for reporting to
replace our In-Memory solution. Benchmarking results were good
and pretty close to that of an In-Memory database platform.
o Hardware and Software cost.
o System stability.
o Better performance.
• JSON NoSQL implementation for unstructured data.
2
2

23. WHAT'S IN THE PIPELINE?
• We are planning to use MaxScale as a CDC tool to replace our
existing inhouse developed solution.
• MariaDB with CentOS Cluster.
• MariaDB is working on providing us a connector for Pentaho ETL.
• With the next major release of MaxScale, we are looking at
implementing:
o AD authentication for DB access
o Limiting DB user connections
2
3

24. Q & A
2
4