This document provides an overview of schema design considerations for MongoDB. It discusses modeling different types of relationships like one-to-one, one-to-many, and many-to-many. It also covers topics like embedding vs referencing, indexing, and common patterns like trees and queues. The key message is that schema design should focus on how the data will be used by applications rather than how it is stored, as the document model provides more flexibility compared to relational databases.

1. #MongoDC
Schema Design
Gary J. Murakami, Ph.D.
Lead Engineer and Ruby Evangelist

2. Algorithms – Data Structures xkcd.com
=?
Schema Design – Gary Murakami

3. Chess (Northwestern University)
Larry Atkin & Dave Slate
Schema Design – Gary Murakami

4. Agenda
• Working with documents
• Evolving a Schema
• Queries and Indexes
• Common Patterns
Schema Design – Gary Murakami

5. RDBMS MongoDB
Database ➜ Database
Table ➜ Collection
Row ➜ Document
Index ➜ Index
Join ➜ Embedded Document
Foreign Key ➜ Reference
Terminology
Schema Design – Gary Murakami

6. Working with
Documents

7. Modeling Data
Schema Design – Gary Murakami

8. Documents
Provide flexibility and
performance
Schema Design – Gary Murakami

9. Normalized Data
Schema Design – Gary Murakami

10. De-Normalized (embedded)
Data
Schema Design – Gary Murakami

11. Relational Schema Design
Focus on data storage
Schema Design – Gary Murakami

12. Document Schema Design
Focus on data use
Schema Design – Gary Murakami

13. Schema Design
Considerations
• How do we manipulate the data?
– Dynamic Ad-Hoc Queries
– Atomic Updates
– Map Reduce
• What are the access patterns of the application?
– Read/Write Ratio
– Types of Queries / Updates
– Data life-cycle and growth rate
Schema Design – Gary Murakami

14. Data Manipulation
• Query Selectors
– Scalar: $ne, $mod, $exists, $type, $lt, $lte, $gt, $gte
– Vector: $in, $nin, $all, $size
• Atomic Update Operators
– Scalar: $inc, $set, $unset
– Vector: $push, $pop, $pull, $pushAll, $pullAll, $addToSet
Schema Design – Gary Murakami

15. Data Access
• Flexible Schemas
• Ability to embed complex data structures
• Secondary Indexes
• Multi-Key Indexes
• Aggregation Framework
– $project, $match, $limit, $skip, $sort, $group, $unwind
• No Joins
Schema Design – Gary Murakami

16. Endgame tablebase
Play chess with God – Ken
Thompson
Schema Design – Gary Murakami

17. Getting Started

18. Library Management
Application
• Patrons
• Books
• Authors
• Publishers
Schema Design – Gary J. Murakami

19. An Example
One to One Relations
Schema Design – Gary Murakami

20. Modeling Patrons
patron = {
patron = { _id: "joe",
_id: "joe", name: "Joe Bookreader",
name: "Joe Bookreader” address: {
} street: "123 Fake St. ",
city: "Faketon",
address = { state: "MA",
patron_id = "joe", zip: 12345
street: "123 Fake St. ", }
city: "Faketon", }
state: "MA",
zip: 12345
}
Schema Design – Gary Murakami

21. One to One Relations
• Mostly the same as the relational approach
• Generally good idea to embed “contains”
relationships
• Document model provides a holistic
representation of objects
Schema Design – Gary Murakami

22. An Example
One To Many Relations
Schema Design – Gary Murakami

23. Modeling Patrons
patron = {
_id: "joe",
name: "Joe Bookreader",
join_date: ISODate("2011-10-15"),
addresses: [
{street: "1 Vernon St.", city: "Newton", state: "MA", …},
{street: "52 Main St.", city: "Boston", state: "MA", …}
]
}
Schema Design – Gary Murakami

24. Publishers and Books
• Publishers put out many books
• Books have one publisher
Schema Design – Gary Murakami

25. Book
MongoDB: The Definitive Guide,
By Kristina Chodorow and Mike Dirolf
Published: 9/24/2010
Pages: 216
Language: English
Publisher: O’Reilly Media, CA
Schema Design – Gary Murakami

26. Modeling Books – Embedded
Publisher
book = {
title: "MongoDB: The Definitive Guide",
authors: [ "Kristina Chodorow", "Mike Dirolf" ],
published_date: ISODate("2010-09-24"),
pages: 216,
language: "English",
publisher: {
name: "O’Reilly Media",
founded: "1980",
location: "CA"
}
}
Schema Design – Gary Murakami

27. Modeling Books & Publisher
Relationship
publisher = {
name: "O’Reilly Media",
founded: "1980",
location: "CA"
}
book = {
title: "MongoDB: The Definitive Guide",
authors: [ "Kristina Chodorow", "Mike Dirolf" ],
published_date: ISODate("2010-09-24"),
pages: 216,
language: "English"
}
Schema Design – Gary Murakami

28. Publisher _id as a Foreign
Key
publisher = {
_id: "oreilly",
name: "O’Reilly Media",
founded: "1980",
location: "CA"
}
book = {
title: "MongoDB: The Definitive Guide",
authors: [ "Kristina Chodorow", "Mike Dirolf" ],
published_date: ISODate("2010-09-24"),
pages: 216,
language: "English",
publisher_id: "oreilly"
}
Schema Design – Gary Murakami

29. Book _id as a Foreign Key
publisher = {
name: "O’Reilly Media",
founded: "1980",
location: "CA"
books: [ "123456789", ... ]
}
book = {
_id: "123456789",
title: "MongoDB: The Definitive Guide",
authors: [ "Kristina Chodorow", "Mike Dirolf" ],
published_date: ISODate("2010-09-24"),
pages: 216,
language: "English"
}
Schema Design – Gary Murakami

30. Where Do You Put the Foreign
Key?
• Array of books inside of publisher
– Makes sense when many means a handful of items
– Useful when items have bound on potential growth
• Reference to single publisher on books
– Useful when items have unbounded growth (unlimited # of
books)
• SQL doesn’t give you a choice, no arrays
Schema Design – Gary Murakami

31. Longest “Database Endgame”
Mate
• Retrograde analysis
– Distance to mate (DTM)
– Distance to conversion (DTC)
• Tablebase analysis
– 6 piece – 262 moves, KRNKNN
– 7 piece – 517 moves, so far
• estimate 2015
Schema Design – Gary Murakami

32. Another Example
One to Many Relations
Schema Design – Gary Murakami

33. Books and Patrons
• Book can be checked out by one Patron at a time
• Patrons can check out many books (but not
1000’s)
Schema Design – Gary Murakami

34. Modeling Checkouts
patron = {
_id: "joe",
name: "Joe Bookreader",
join_date: ISODate("2011-10-15"),
address: { ... }
}
book = {
_id: "123456789",
title: "MongoDB: The Definitive Guide",
authors: [ "Kristina Chodorow", "Mike Dirolf" ],
...
}
Schema Design – Gary Murakami

35. Modeling Checkouts
patron = {
_id: "joe",
name: "Joe Bookreader",
join_date: ISODate("2011-10-15"),
address: { ... },
checked_out: [
{ _id: "123456789", checked_out: "2012-10-15" },
{ _id: "987654321", checked_out: "2012-09-12" },
...
]
}
Schema Design – Gary Murakami

36. Denormalization
Provides data locality
De-normalize for speed
Schema Design – Gary Murakami

37. Modeling Checkouts:
Denormalized
patron = {
_id: "joe",
name: "Joe Bookreader",
join_date: ISODate("2011-10-15"),
address: { ... },
checked_out: [
{ _id: "123456789",
title: "MongoDB: The Definitive Guide",
authors: [ "Kristina Chodorow", "Mike Dirolf" ],
checked_out: ISODate("2012-10-15")
},
{ _id: "987654321"
title: "MongoDB: The Scaling Adventure",
...
}, ...
]
}
Schema Design – Gary Murakami

38. Referencing vs. Embedding
• Embedding is a bit like pre-joined data
• Document-level ops are easy for server to
handle
• Embed when the 'many' objects always appear
with (i.e. viewed in the context of) their parent
• Reference when you need more flexibility
Schema Design – Gary Murakami

39. An Example
Single Table Inheritance
Schema Design – Gary Murakami

40. Single Table Inheritance
book = {
title: "MongoDB: The Definitive Guide",
authors: [ "Kristina Chodorow", "Mike Dirolf" ],
published_date: ISODate("2010-09-24"),
kind: "loanable",
locations: [ ... ],
pages: 216,
language: "English",
publisher: {
name: "O’Reilly Media",
founded: "1980",
location: "CA"
}
}
Schema Design – Gary Murakami

41. Can We Solve Chess
One Day?
• Chess tablebase problem
– search is not localized, poor cache performance, seeks
– competitive chess programs often play worse
• Endgame database size
– 5 piece – 7 GB compressed 75%
• 157 MB Shredderbase – 1000x
• 441 MB Shredderbase – 10,000x
– 6 piece – 1.2 TB compressed
– 7 piece – 70 TB estimated by 2015
Schema Design – Gary Murakami

42. An Example
Many to Many Relations
Schema Design – Gary Murakami

43. Relational Approach
Schema Design – Gary Murakami

44. Books and Authors
book = {
title: "MongoDB: The Definitive Guide",
authors = [
{ _id: "kchodorow", name: "K-Awesome" },
{ _id: "mdirolf", name: "Batman Mike" },
]
published_date: ISODate("2010-09-24"),
pages: 216,
language: "English"
}
author = {
_id: "kchodorow",
name: "Kristina Chodorow",
hometown: "New York"
}
Schema Design – Gary Murakami

45. An Example
Trees
Schema Design – Gary Murakami

46. Parent Links
book = {
title: "MongoDB: The Definitive Guide",
authors: [ "Kristina Chodorow", "Mike Dirolf" ],
published_date: ISODate("2010-09-24"),
pages: 216,
language: "English",
category: "MongoDB"
}
category = { _id: MongoDB, parent: "Databases" }
category = { _id: Databases, parent: "Programming" }
Schema Design – Gary Murakami

47. Child Links
book = {
_id: 123456789,
title: "MongoDB: The Definitive Guide",
authors: [ "Kristina Chodorow", "Mike Dirolf" ],
published_date: ISODate("2010-09-24"),
pages: 216,
language: "English"
}
category = { _id: MongoDB, children: [ 123456789, … ] }
category = { _id: Databases, children: ["MongoDB", "Postgres"}
category = { _id: Programming, children: ["DB", "Languages"] }
Schema Design – Gary Murakami

48. Modeling Trees
• Parent Links
- Each node is stored as a document
- Contains the id of the parent
• Child Links
- Each node contains the id’s of the children
- Can support graphs (multiple parents / child)
Schema Design – Gary Murakami

49. Array of Ancestors
book = {
title: "MongoDB: The Definitive Guide",
authors: [ "Kristina Chodorow", "Mike Dirolf" ],
published_date: ISODate("2010-09-24"),
pages: 216,
language: "English",
categories: ["Programming", "Databases", "MongoDB” ]
}
book = {
title: "MySQL: The Definitive Guide",
authors: [ "Michael Kofler" ],
published_date: ISODate("2010-09-24"),
pages: 216,
language: "English",
parent: "MongoDB",
ancestors: [ "Programming", "Databases", "MongoDB"]
}
Schema Design – Gary Murakami

50. An Example
Queues
Schema Design – Gary Murakami

51. Book Document
book = {
_id: 123456789,
title: "MongoDB: The Definitive Guide",
authors: [ "Kristina Chodorow", "Mike Dirolf" ],
published_date: ISODate("2010-09-24"),
pages: 216,
language: "English",
available: 3
}
db.books.findAndModify({
query: { _id: 123456789, available: { "$gt": 0 } },
update: { $inc: { available: -1 } }
})
Schema Design – Gary Murakami

52. It’s All About Your
Application
• Programs + Databases = Applications
• Your Schema is the impedance matcher
– Melds programming with MongoDB
– Capitalizes on both programming language & DB
– Maximizes power transfer – best of both working together
– Flexible for development and change
• Programs X MongoDB = Great Big Applications
• Play chess with God
Schema Design – Gary Murakami

53. It’s All About Your
Application
• Programs + Databases = Applications
• Your Schema is the impedance matcher
– Melds programming with MongoDB
– Capitalizes on both programming language & DB
– Maximizes power transfer – best of both working together
– Flexible for development and change
• Programs X MongoDB = Great Big Applications
• Play music with God – AAC / PAC
Schema Design – Gary Murakami

54. #MongoDC
"His pattern indicates
two-dimensional thinking.”
-Spock
Star Trek II:
The Wrath of Khan
Thank You
Gary J. Murakami, Ph.D. www.3dchessfederation.com
Lead Engineer and Ruby Evangelist, 10gen (the MongoDB
company)