This document summarizes a workshop agenda on MemSQL, an in-memory distributed SQL database. The agenda covers an introduction to MemSQL as a company and software, a discussion of current data challenges, and a demonstration of MemSQL's architecture, features like transactions and high availability, system requirements, licensing, and a speed test. Hands-on exercises are also included to showcase MemSQL's capabilities.

1. The Database for Real-Time & Historical Big Data Analytics
MemSQL Workshop
Carlos Bueno (15 Jun 2015)

2. 2
Workshop Agenda
▪ The Company
▪ The Landscape
▪ The Software
▪ Hands-on
▪ Scaling up
▪ Q & A

3. 3
▪ Experienced leadership from Facebook,
SQL Server, Oracle, Fusion-io
▪ In-Memory, distributed, relational
database
▪ Solving the Enterprise Architecture Gap
▪ Horizontal scale-out with modern
database innovation
▪ $50 million in funding
MemSQL the company

4. Going Real-Time is the Next Phase for Big Data
More
Sensors
More
Interconnectivity
More
User Demand
…and companies are at risk of being left behind
4

5. Current Data Management Challenges
ETL
Batch Processing
Big Iron Appliances
5

6. MemSQL the software
▪ Distributed and Parallel
▪ Shared-Nothing, Lock-Free
▪ Data in memory or SSD
▪ SQL all the way down
6

7. MemSQL Engine: “memsqld”
▪ Basic scaling unit of a cluster
▪ A full, independent RDBMS
▪ 50,000 inserts / sec on wide table
▪ ~1M inserts / sec on skinny table
▪ Millions of primary-key lookups / sec
MemSQL
7

8. MemSQL Engine: Aggregators Aggregate
Agg 1 Agg 2
Leaf 1 Leaf 2 Leaf 3 Leaf 4
8

9. MemSQL Engine: Leaves Hold Data
Agg 1 Agg 2
Leaf 1 Leaf 2 Leaf 3 Leaf 4
9

10. MemSQL Engine: Sharding and Joins
Agg 1 Agg 2
Leaf 1 Leaf 2 Leaf 3 Leaf 4
select * from lineitem L, orders O
where L.orderkey = O.orderkey...
leaf1> using memsql_demo_0
select * from lineitem L, orders O
where L.orderkey = O.orderkey...
leaf2> using memsql_demo_1
select * from lineitem L, orders O
where L.orderkey = O.orderkey...
10

11. MemSQL Engine: Compiled Queries
Parse
In Cache?
Execute
Codegen
select * from foo where id=1234
and name like ‘%jingleheimer%’;
SELECT * FROM foo WHERE id = @
AND name LIKE ^
11

12. Durability: Transactions (MVCC)
Every write creates a new version of row
Old versions get garbage-collected
Reads are never blocked
Row-level locking for writes
Allows online ALTER TABLE!
Multi-statement transactions
v1
v2
v3
v0
v4
readers
readers
writer readers
(waiting writer)
12

13. Durability: Logging and Snapshots
Every write saved to transaction log on disk:
/var/lib/memsql/data/logs
Periodic compaction into a snapshot file:
/var/lib/memsql/data/snapshots
On restart data is loaded into RAM
Latest two snapshots are kept by default
13

14. Durability: High Availability
Leaves are paired up
Partitions replicated async
Automatically fails over
Uses 2X space
Leaf 1 Leaf 2 Leaf 4Leaf 3
Agg 1 Agg 2
14

15. Minimum System Requirements
▪ 8GB RAM (32+ recommended)
▪ 4X 64-bit Intel CPU cores (8+ recommended)
▪ 2X RAM free disk space (for backups & logs)
▪ 10GB swap space
▪ Hyperthreading OFF (physical CPU cores)
▪ Modern Linux (Centos 6+, Ubuntu 12+)
▪ 1gbps switched network
15

16. Licensing
▪ Community Edition
• Free Forever, Unlimited Scale
• Full SQL features
▪ Enterprise Edition
• Subscription basis, by RAM capacity
• No limit on disk storage
• Enterprise support
• Replication / High Availability
16

17. 17
MemSQL “Cluster in a box”
▪ AWS m4.2xlarge: 8 cores, 32GB RAM

18. 18
MemSQL “Cluster in a box”

19. 19
MemSQL “Cluster in a box”

20. 20
MemSQL “Cluster in a box”

21. 21
MemSQL “Cluster in a box”

22. 22
MemSQL “Cluster in a box”

23. 23
MemSQL Speed Test

24. 24
MemSQL Web Console

25. Thank You
www.memsql.com