Co-Founder Peter Mattis presented this deck to the NYC PostgreSQL User Group on Nov. 4, 2015. It compares PostgreSQL to CockroachDB SQL- logical data structures, kv layers, data storage, online schema changes, and more.

1. @cockroachdb
PostgreSQL meetup, November 2015
CockroachDB
presented by Peter Mattis / Co-Founder

2. @cockroachdb
1.Overview of CockroachDB
2.SQL Data Model
3.Logical Data Storage
4.Online/Concurrent Schema Change
Agenda

3. @cockroachdb
What is CockroachDB?
■Scale out SQL
■Distributed
■Survivable
■Consistent
■Open source

4. @cockroachdb
CockroachDB: Architecture
■Layered abstractions
■SQL is starting point
■Distributes at map
■Replicates at physical layer
SQL
Transactional KV
Monolithic Map
Raft

5. @cockroachdb
CockroachDB: Architecture
■Layered abstractions
■SQL is starting point
■Distributes at map
■Replicates at physical layer
Transactional KV
Monolithic Map
Raft
GraphSQL

6. @cockroachdb
CockroachDB: Architecture
■Layered abstractions
■SQL is starting point
■Distributes at map
■Replicates at physical layer
Transactional KV
Monolithic Map
Raft
SQL Graph Doc

7. @cockroachdb
CockroachDB: Architecture
■Layered abstractions
■SQL is starting point
■Distributes at map
■Replicates at physical layer
Transactional KV
Monolithic Map
Raft
SQL

8. @cockroachdb
CockroachDB: Architecture
■Layered abstractions
■SQL is starting point
■Distributes at map
■Replicates at physical layer
Transactional KV
Monolithic Map
Raft
SQL
Physical

9. @cockroachdb
SQL Data Model

10. @cockroachdb
■Tables
SQL Data Model

11. @cockroachdb
■Tables
SQL Data Model
Inventory

12. @cockroachdb
■Tables
■Rows
SQL Data Model
Inventory

13. @cockroachdb
■Tables
■Rows
■Columns
SQL Data Model
Inventory
ID Name Price
1 Glove 1.11
2 Ball 2.22
3 Shirt 3.33
4 Shorts 4.44
5 Bat 5.55
6 Shoes 6.66

14. @cockroachdb
■Tables
■Rows
■Columns
■Indexes
SQL Data Model
Inventory
ID Name Price
1 Glove 1.11
2 Ball 2.22
3 Shirt 3.33
4 Shorts 4.44
5 Bat 5.55
6 Shoes 6.66
Name
Ball
Bat
Glove
Shirt
Shoes
Shorts
Name_Idx

15. @cockroachdb
PostgreSQL: Logical Data Storage

16. @cockroachdb
■Rows are stored in an unordered heap
■Indexes are btrees
■Primary key is a unique index
PostgreSQL: Data Storage

17. @cockroachdb
CREATE TABLE test (
id INTEGER PRIMARY KEY,
name VARCHAR,
price FLOAT,
);
PostgreSQL: Example Table

18. @cockroachdb
INSERT INTO test VALUES (1, “ball”, 3.33);
PostgreSQL: Logical Data Storage

19. @cockroachdb
INSERT INTO test VALUES (1, “ball”, 3.33);
PostgreSQL: Logical Data Storage
Tuple ID (Page# / Item#) Row
(0, 1) (1, “ball”, 3.33)
test (heap)

20. @cockroachdb
INSERT INTO test VALUES (1, “ball”, 3.33);
PostgreSQL: Logical Data Storage
Tuple ID (Page# / Item#) Row
(0, 1) (1, “ball”, 3.33)
Index Key Tuple ID
1 (0, 1)
test (heap)test_pkey (btree)

21. @cockroachdb
INSERT INTO test VALUES (1, “ball”, 3.33);
INSERT INTO test VALUES (2, “glove”, 4.44);
PostgreSQL: Logical Data Storage
Tuple ID (Page# / Item#) Row
(0, 1) (1, “ball”, 3.33)
(0, 2) (2, “glove”, 4.44)
Index Key Tuple ID
1 (0, 1)
2 (0, 2)
test (heap)test_pkey (btree)

22. @cockroachdb
CockroachDB: Logical Data Storage

23. @cockroachdb
■Keys and values are strings
■Monolithic, sorted map
CockroachDB: KV

24. @cockroachdb
Get(key)
Put(key, value)
ConditionalPut(key, value, expValue)
Scan(startKey, endKey)
CockroachDB: KV Primitives

25. @cockroachdb
Get(key)
Put(key, value)
ConditionalPut(key, value, expValue)
Scan(startKey, endKey)
Del(key)
CockroachDB: KV Primitives

26. @cockroachdb
■All tables have a primary key
■One key/value pair per column
CockroachDB: Row Storage

27. @cockroachdb
■All tables have a primary key
■One key/value pair per column
■Key anatomy:
/<table>/<index>/<pkey>/<column>
CockroachDB: Row Storage

28. @cockroachdb
CREATE TABLE test (
id INTEGER PRIMARY KEY,
name VARCHAR,
price FLOAT,
);
CockroachDB: Example Table

29. @cockroachdb
INSERT INTO test VALUES (1, “ball”, 2.22);
CockroachDB: Key Anatomy
Key: /<table>/<index>/<key>/<column> Value
/test/primary/1/name “ball”
/test/primary/1/price 2.22

30. @cockroachdb
INSERT INTO test VALUES (1, “ball”, 2.22);
INSERT INTO test VALUES (2, “glove”, 3.33);
CockroachDB: Key Anatomy
Key: /<table>/<index>/<key>/<column> Value
/test/primary/1/name “ball”
/test/primary/1/price 2.22
/test/primary/2/name “glove”
/test/primary/2/price 3.33

31. @cockroachdb
INSERT INTO test VALUES (1, “ball”, 2.22);
INSERT INTO test VALUES (2, “glove”, 3.33);
CockroachDB: Key Anatomy
Key: /<table>/<index>/<key>/<column> Value
/test/primary/1/name “ball”
.../price 2.22
.../2/name “glove”
.../price 3.33

32. @cockroachdb
INSERT INTO test VALUES (1, “ball”, 2.22);
INSERT INTO test VALUES (2, “glove”, 3.33);
CockroachDB: Key Anatomy
Key: /<table>/<index>/<key>/<column> Value
/1000/1/1/1 “ball”
.../2 2.22
.../2/1 “glove”
.../2 3.33

33. @cockroachdb
■Key encoding
■NULL column values
■Unique indexes
■Non-unique indexes
CockroachDB: The Details

34. @cockroachdb
■Keys and values are strings
■Columns are typed data
■???
CockroachDB: Key Encoding

35. @cockroachdb
■NULL indicates value does not exist
■NULL is weird: NULL != NULL
CockroachDB: NULL Column Values

36. @cockroachdb
■NULL indicates value does not exist
■NULL is weird: NULL != NULL
■CockroachDB: NULL values are not explicitly stored
CockroachDB: NULL Column Values

37. @cockroachdb
INSERT INTO test VALUES (1, “ball”, NULL);
CockroachDB: NULL Column Values
Key: /<table>/<index>/<key>/<column> Value
/test/primary/1/name “ball”

38. @cockroachdb
INSERT INTO test VALUES (1, “ball”, NULL);
INSERT INTO test VALUES (2, NULL, NULL);
CockroachDB: NULL Column Values
Key: /<table>/<index>/<key>/<column> Value
/test/primary/1/name “ball”
??? ???

39. @cockroachdb
INSERT INTO test VALUES (1, “ball”, NULL);
INSERT INTO test VALUES (2, NULL, NULL);
CockroachDB: NULL Column Values
Key: /<table>/<index>/<key>[/<column>] Value
/test/primary/1 Ø
/test/primary/1/name “ball”
/test/primary/2 Ø

40. @cockroachdb
CREATE UNIQUE INDEX bar ON test (name);
■Multiple table rows with equal indexed values are
not allowed
CockroachDB: Unique Indexes

41. @cockroachdb
CREATE UNIQUE INDEX bar ON test (name);
INSERT INTO test VALUES (1, “ball”, 2.22);
CockroachDB: Unique Indexes
Key: /<table>/<index>/<key> Value
/test/bar/”ball” 1

42. @cockroachdb
CREATE UNIQUE INDEX bar ON test (name);
INSERT INTO test VALUES (1, “ball”, 2.22);
INSERT INTO test VALUES (2, “glove”, 3.33);
CockroachDB: Unique Indexes
Key: /<table>/<index>/<key> Value
/test/bar/”ball” 1
/test/bar/”glove” 2

43. @cockroachdb
CREATE UNIQUE INDEX bar ON test (name);
INSERT INTO test VALUES (1, “ball”, 2.22);
INSERT INTO test VALUES (2, “glove”, 3.33);
INSERT INTO test VALUES (3, “glove”, 4.44);
CockroachDB: Unique Indexes
Key: /<table>/<index>/<key> Value
/test/bar/”ball” 1
/test/bar/”glove” 2
/test/bar/”glove” 3

44. @cockroachdb
CREATE UNIQUE INDEX bar ON test (name);
■NULL is weird: NULL != NULL
CockroachDB: Unique Indexes (NULL Values)

45. @cockroachdb
CREATE UNIQUE INDEX bar ON test (name);
INSERT INTO test VALUES (3, NULL, NULL);
CockroachDB: Unique Indexes (NULL Values)
Key: /<table>/<index>/<key> Value
/test/bar/NULL 3

46. @cockroachdb
CREATE UNIQUE INDEX bar ON test (name);
INSERT INTO test VALUES (3, NULL, NULL);
INSERT INTO test VALUES (4, NULL, NULL);
CockroachDB: Unique Indexes (NULL Values)
Key: /<table>/<index>/<key> Value
/test/bar/NULL 3
/test/bar/NULL 4

47. @cockroachdb
CREATE UNIQUE INDEX bar ON test (name);
INSERT INTO test VALUES (3, NULL, NULL);
CockroachDB: Unique Indexes (NULL Values)
Key: /<table>/<index>/<key>[/<pkey>] Value
/test/bar/NULL/3 Ø

48. @cockroachdb
CREATE UNIQUE INDEX bar ON test (name);
INSERT INTO test VALUES (3, NULL, NULL);
INSERT INTO test VALUES (4, NULL, NULL);
CockroachDB: Unique Indexes (NULL Values)
Key: /<table>/<index>/<key>[/<pkey>] Value
/test/bar/NULL/3 Ø
/test/bar/NULL/4 Ø

49. @cockroachdb
CREATE INDEX foo ON test (name);
■Multiple table rows with equal indexed values are
allowed
CockroachDB: Non-Unique Indexes

50. @cockroachdb
CREATE INDEX foo ON test (name);
■Multiple table rows with equal indexed values are
allowed
■Primary key is a unique index
CockroachDB: Non-Unique Indexes

51. @cockroachdb
CREATE INDEX foo ON test (name);
INSERT INTO test VALUES (1, “ball”, 2.22);
CockroachDB: Non-Unique Indexes
Key: /<table>/<index>/<key>/<pkey> Value
/test/foo/”ball”/1 Ø

52. @cockroachdb
CREATE INDEX foo ON test (name);
INSERT INTO test VALUES (1, “ball”, 2.22);
INSERT INTO test VALUES (2, “glove”, 3.33);
CockroachDB: Non-Unique Indexes
Key: /<table>/<index>/<key>/<pkey> Value
/test/foo/”ball”/1 Ø
/test/foo/”glove”/2 Ø

53. @cockroachdb
CREATE INDEX foo ON test (name);
INSERT INTO test VALUES (1, “ball”, 2.22);
INSERT INTO test VALUES (2, “glove”, 3.33);
INSERT INTO test VALUES (3, “glove”, 4.44);
CockroachDB: Non-Unique Indexes
Key: /<table>/<index>/<key>/<pkey> Value
/test/foo/”ball”/1 Ø
/test/foo/”glove”/2 Ø
/test/foo/”glove”/3 Ø

54. @cockroachdb
■Keys and values are strings
■NULL column values
■Unique indexes
■Non-unique indexes
CockroachDB: Logical Data Storage

55. @cockroachdb
Logical Data Storage
PostgreSQL CockroachDB
Keys are composite structures Keys are strings
Heap storage for rows Required primary key
Per-table heap/indexes Monolithic map

56. @cockroachdb
Online Schema Change

57. @cockroachdb
Schema Change Operations
CREATE INDEX foo ON test (col1, col2, …);
ALTER TABLE test DROP col1;
ALTER TABLE test ADD col3 INTEGER;
...

58. @cockroachdb
Schema Change (the easy way)
1. Lock table
2. Adjust table data (add column, populate index, etc.)
3. Unlock table

59. @cockroachdb
Schema Change (the easy way)
1. Apologize for down time
2. Lock table
3. Adjust table data (add column, populate index, etc.)
4. Unlock table

60. @cockroachdb
Schema Change (the MySQL way)
1. Create new table with altered schema
2. Capture changes from source to the new table
3. Copy rows from the source to the new table
4. Synchronize source and new table
5. Swap/rename source and new table

61. @cockroachdb
Schema Change (the PostgreSQL way)
1. CREATE INDEX CONCURRENTLY

62. @cockroachdb
CockroachDB: Schema Change
■TableDescriptor contains table schema
■TableDescriptor replicated on every node
■Distributed atomic updates are difficult
■Distributed locking is difficult
■The easy way isn’t feasible

63. @cockroachdb
CockroachDB: CREATE INDEX
CREATE INDEX foo ON TEST
1. Backfill index entries
2. Add index to TableDescriptor

64. @cockroachdb
CockroachDB: CREATE INDEX
T1 T2
CREATE INDEX foo ON test… INSERT INTO test…

65. @cockroachdb
CockroachDB: CREATE INDEX
CREATE INDEX foo ON TEST
1. Add index to TableDescriptor as write-only
2. Backfill index entries
3. Mark index as read-write

66. @cockroachdb
CockroachDB: CREATE INDEX
T1 T2
CREATE INDEX foo ON test… INSERT INTO test…
or
UPDATE test…
or
DELETE FROM test…

67. @cockroachdb
CockroachDB: CREATE INDEX
CREATE INDEX foo ON TEST
1. Add index to TableDescriptor as delete-only
2. Wait for descriptor propagation
3. Mark index as write-only
4. Wait for descriptor propagation
5. Backfill index entries
6. Mark index as read-write

68. @cockroachdb
Online Schema Change
Online schema change is difficult
The database should do the heavy lifting

69. @cockroachdb
The End
SQL databases are KV stores on steroids

70. @cockroachdb
github.com/cockroachdb/cockroach
CockroachLabs.com
@cockroachdb
Thank You