MongoDB .local London 2019: A Complete Methodology to Data Modeling for MongoDB

#MDBlocal
A Complete Methodology to
Data Modeling for MongoDB
Jim Blackhurst and Eugene Bogaart,
Solution Architects, MongoDB
LONDON

@
#MDBlocal
Eugene Bogaart
Senior Solution Architect, NEUR, MongoDB
Bogaart

@
#MDBlocal
Jim Blackhurst
Principal Solution Architect, NEUR, MongoDB
jimthree

Goals of the Presentation
Document vs Tabular
Recognize the
differences
Methodology
Summarize the steps
when modeling for
MongoDB
Patterns
Recognize when to apply

Document versus
Tabular
Recognize the differences when modeling for a Document
Database versus a Relational/Tabular Database

#MDBLocal
Thinking in Documents
• Polymorphism
• different documents may contain
different fields
• Array
• represent a "one-to-many" relation
• index entry separately
• Sub Document
• grouping some fields together
• JSON/BSON
• documents shown as JSON
• BSON is the physical format

#MDBLocal
Example: Modeling a blog

#MDBLocal
CRDs: Collection-Relationship-Diagrams for two solutions
ORSolution A Solution B
Queries by
articles or
users
Queries by
articles
Duplication
of users
information
Simpler

#MDBLocal
Example: Modeling a Social Network
Solution A Solution B

#MDBLocal
Example: Modeling a Social Network
ü Slower writes
ü More storage space
ü Duplication
ü Faster reads
Pre-aggregated
Data
Solution A Solution B
(Fan Out on writes)(Fan Out on reads)

#MDBLocal
Tabular MongoDB
Steps to create the
model
1 – define schema
2 – develop app and queries
1 – identifying the queries
2 – define schema
Differences: Tabular vs Document

#MDBLocal
Tabular MongoDB
Steps to create the
model
1 – define schema
2 – define schema
Initial schema • 3rd normal form
• one possible solution
• many possible solutions

#MDBLocal
Tabular MongoDB
Steps to create the
model
1 – define schema
2 – define schema
Final schema • likely denormalized • few changes

#MDBLocal
Tabular MongoDB
Steps to create the
model
1 – define schema
2 – define schema
Schema evolution • difficult and not optimal
• likely downtime
• easy
• no downtime

#MDBLocal
Tabular MongoDB
Steps to create the
model
1 – define schema
2 – define schema
Schema evolution • difficult and not optimal
• likely downtime
• easy
• no downtime
Performance • mediocre • optimized

Methodology
Summarize the steps of a methodology when modeling for
MongoDB

#MDBLocal
Main Tradeoff in Modeling

Methodology
1. Describe the
Workload

Methodology
1. Describe the
Workload
2. Identify and Model
the Relationships

#MDBLocal
Actors, Movies and Reviews
actor_name
date_of_birth
movie_title
revenues
reviewer_name
rating

#MDBLocal
Actors, Movies and Reviews actors
name
date_of_birth
movies
title
revenues
reviews
name
rating
actor_name
date_of_birth
movie_title
revenues
reviewer_name
rating

#MDBLocal
Actors, Movies and Reviews actors
name
date_of_birth
movies : [ .. ]
movies
title
revenues
actors: [ ..]
name
rating
actor_name
date_of_birth
movie_title
revenues
reviewer_name
rating

Methodology
1. Describe the
Workload
2. Identify and Model
the Relationships
3. Apply Patterns

#MDBLocal
Flexible Methodology

Use Case
Let's start a franchise of coffee shops…

#MDBLocal
Case Study: Coffee Shop Franchises
Name: Beyond the Stars Coffee

#MDBLocal
Objective:
• 10 000 stores in the United States

#MDBLocal
Objective:
• … then we expend to the rest of the World

#MDBLocal
Objective:
• … then we expand to the rest of the World
Keys to success:
1. Best coffee in the world

#MDBLocal
Objective:
• … then we expand to the rest of the World
Keys to success:
1. Best coffee in the world
2. Best Technology

#MDBLocal
Make the Best Coffee in the World
23g of ground coffee in, 20g of extracted coffee
out, in approximately 20 seconds
1. Fill a small or regular cup with 80% hot
water (not boiling but pretty hot). Your cup
should be 150ml to 200ml in total volume,
80% of which will be hot water.
2. Grind 23g of coffee into your portafilter
using the double basket. We use a scale that
you can get here.
3. Draw 20g of coffee over the hot water by
placing your cup on a scale, press tare and
extract your shot.

#MDBLocal
Key to Success 2: Best Technology
a) Intelligent Shelves
• Measure inventory in real time

#MDBLocal
b) Intelligent Coffee Machines
• Weightings, temperature, time to produce, …
• Coffee perfection

#MDBLocal
b) Intelligent Coffee Machines
• Weightings, temperature, time to produce, …
• Coffee perfection
c) Intelligent Data Storage
• MongoDB

#MDBLocal
1 – Workload: List Queries
Query Operation Description
1. Coffee weight on the shelves write A shelf send information when coffee bags are
added or removed

#MDBLocal
added or removed
2. Coffee to deliver to stores read How much coffee do we have to ship to the store in
the next days

#MDBLocal
added or removed
the next days
3. Anomalies in the inventory read Analytics

#MDBLocal
added or removed
the next days
4. Making a cup of coffee write A coffee machine reporting on the production of a
coffee cup

#MDBLocal
added or removed
the next days
coffee cup
5. Analysis of cups of coffee read Analytics

#MDBLocal
added or removed
the next days
coffee cup
5. Analysis of cups of coffee read Analytics
6. Technical Support read Helping our franchisees

#MDBLocal
1 – Workload: quantify/qualify the queries
Query Quantification Qualification
1. Coffee weight on the shelves 10/day*shelf*store
=> 1/sec
<1s
critical write
2. Coffee to deliver to stores 1/day*store
=> 0.1/sec
<60s
3. Anomalies in the inventory 24 reads/day <5mins
"collection scan"
4. Making a cup of coffee 10 000 000 writes/day
115 writes/sec
<100ms
non-critical write
… cups of coffee at rush hour 3 000 000 writes/hr
833 writes/sec
<100ms
non-critical write
5. Analysis of cups of coffee 24 reads/day stale data is fine
"collection scan"
6. Technical Support 1000 reads/day <1s

#MDBLocal
Disk Space
Cups of coffee
• one year of data
• 10000 x 1000/day x 365
• 3.7 billions/year
• 370 GB (100 bytes/cup of
coffee)
Weighings
• one year of data
• 10000 x 10/day x 365
• 365 billions/year
• 3.7 GB (100 bytes/weighings)

#MDBLocal
2 - Relations are still important
Type of Relation -> one-to-one/1-1 one-to-many/1-N many-to-many/N-N
Document
embedded in the
parent document
• one read
• no joins
• one read
• no joins
• one read
• no joins
• duplication of
information
Document
referenced in the
parent document
• smaller reads
• many reads
• smaller reads
• many reads
• smaller reads
• many reads

#MDBLocal
2 - Entities for Beyond the Stars Coffee
Entities:
• Coffee cups
• Stores
• Coffee machines
• Shelves
• Weighings
• Coffee bags

Patterns
Recognize the need and when to apply Schema Design Patterns

#MDBLocal
Schema Design Patterns Resources
A. Advanced Schema Design Patterns
• MongoDB World 2017
B. Blogs on Patterns, with Ken Alger
• https://www.mongodb.com/blog/post/building-
with-patterns-a-summary
C. MongoDB University: M320 – Data Modeling
• https://university.mongodb.com/courses/M320/about
D. Schema Design, Builder Fest PODs
• Wednesday, with our Consulting Engineers

#MDBLocal
Bucket Pattern
{
"device_id": 000123456,
"type": "2A",
"date": ISODate("2018-03-02"),
"temp": [ [ 20.0, 20.1, 20.2, ... ],
[ 22.1, 22.1, 22.0, ... ],
...
]
}
{
"device_id": 000123456,
"type": "2A",
"date": ISODate("2018-03-03"),
"temp": [ [ 20.1, 20.2, 20.3, ... ],
[ 22.4, 22.4, 22.3, ... ],
...
]
}
{
"device_id": 000123456,
"type": "2A",
"date": ISODate("2018-03-02T13"),
"temp": { 1: 20.0, 2: 20.1, 3: 20.2, ... }
}
{
"device_id": 000123456,
"type": "2A",
"date": ISODate("2018-03-02T14"),
"temp": { 1: 22.1, 2: 22.1, 3: 22.0, ... }
}
Bucket per
Day
Bucket per
Hour

#MDBLocal
Solution with Patterns
• Schema Versioning
• Subset
• Computed
• Bucket

#MDBLocal
https://university.mongodb.com/courses/M320/about
Data Modeling Patterns Use Cases

Takeaways from the Presentation
Document vs Tabular
Recognize the
differences
Methodology
Summarize the steps
when modeling for
MongoDB
Patterns
Recognize when to apply

Thank you for taking our FREE
MongoDB classes at
university.mongodb.com

Register Now!
https://university.mongodb.com/courses/M320/about

#MDBlocal
A Complete Methodology to
Data Modeling for
MongoDB [DEV]
Jim Blackhurst + Eugene Bogaart
https://www.surveymonkey.com/r/KG5MP96

Appendix A
Schema Versioning Pattern

#MDBLocal
Nightmare: Alter Table

#MDBLocal
This is what your dreams should be when
thinking about a schema upgrade !

#MDBLocal
Schema Revision
Relational MongoDB
Versioned Unit Schema Document
Migration Procedure Difficult Easy
Service Uptime Interrupted No interruption
Rollback Difficult to
nightmare-ish
Easy

#MDBLocal
Application Lifecycle
Modify Application
• Can read/process all versions of documents
• Have different handler per version
• Reshape the document before processing
it
Update all Application servers
• Install updated application
• Remove old processes
Once migration completed
• remove the code to process old versions.

#MDBLocal
Document Lifecycle
New Documents:
• Application writes them in latest version
Existing Documents
A) Use updates to documents
• to transform to latest version
• keep forever documents that never
need an update
B) or transform all documents in batch
• no worry even if process takes days

#MDBLocal
Timeline of the migration

Problem Solution
Use Cases Examples Benefits and Trade-Offs
Schema Versioning Pattern
● Avoid downtime while doing schema
upgrades
● Upgrading all documents can take hours,
days or even weeks when dealing with big
data
● Don't want to update all documents
No downtime needed
Feel in control of the migration
Less future technical debt
! May need 2 indexes for same field while
in migration period
● Each document gets a "schema_version"
field
● Application can handle all versions
● Choose your strategy to migrate the
documents
● Every application that use a database,
deployed in production and heavily used.
● System with a lot of legacy data

#MDBLocal
Mathematical Operations

#MDBLocal
"Fan Out" Operations

#MDBLocal
"Roll Up" Operations

Problem Solution
Use Cases Examples Benefits and Trade-Offs
Computed Pattern
● Costly computation or manipulation of
data
● Executed frequently on the same data,
producing the same result
Read queries are faster
Saving on resources like CPU and Disk
! May be difficult to identify the need
! Avoid applying or overusing it unless
needed
● Perform the operation and store the
result in the appropriate document and
collection
● If need to redo the operations, keep the
source of them
● Internet Of Things (IOT)
● Event Sourcing
● Time Series Data
● Frequent Aggregation Framework
queries

#MDBLocal
But if you must, you can have maximum of two lines in the
title or this becomes too much for your audience

#MDBLocal
Title and content – tables, charts, graphics, and text

#MDBLocal
First level content is paragraph style, without bullets. To insert
bullets, click on "Increase list level" to create your list. Use bold and
green color treatment to emphasis copy.
• Second level at 28-point font
• Third level at 24-point font
• Fourth level at 20-point font
• Fifth level – do not go over fifth level
Title and text only slide

#MDBLocal
Title and subtitle slide for diagrams or graphics
Subhead goes here and keep this to one line of text, short and sweet

#MDBLocal
This has default animation
Go to Animations panel and
remove if not needed
• Bullet one
• Bullet two
Title with left graphic white space and right text

#MDBLocal
This layout is for slides that do
not have a title
It has a default appear on click
animation for each text level
Remove animation from the
Animations panel if not needed

#MDBLocal
Product showcase
on laptop
Description of the screenshot goes here.
Click on the icon to add your screenshot.
Bullets are not styled in this text box so
please refrain from using them.
Instead, use a hard return to create your
content list. It's a much cleaner look and
doesn't compete with the screenshot.

#MDBLocal
Product showcase
on mobile
Description of the screenshot goes here.
Click on the icon to add your screenshot.
Bullets are not styled in this text box so
please refrain from using them.
Instead, use a hard return to create your
content list. It's a much cleaner look and
doesn't compete with the screenshot.

#MDBLocal
Title with code
Click to add code. This slide is for title and code only. Do not use
this for anything else.

Divider Slide
Subtitle can go here if needed

DEMO
Demo title goes here
Presenter Name

#MDBLocal
Doughnut chart
First level content is paragraph
style, without bullets. To insert
bullets, click on "Increase list
level" to create your list. Use bold
and green color treatment to
emphasis copy.

#MDBLocal
Table
Column 1 Column 2 Column 3 Column 4 Column 5 Column 6
Row one 48 66 75 45 66 34
Row two 35 75 89 58 48 66
Row three 66 34 55 35 35 75
Row four 55 70 84 52 55 35

#MDBLocal
Mega Table
Column 1 Column 2 Column 3 Column 4 Column 5 Column 6
Row one 48 66 75 45 66 34
Row two 35 75 89 58 48 66
Row three 66 34 55 35 35 75
Row four 55 70 84 52 55 35
Row five 48 66 75 45 66 34
Row six 35 75 89 58 48 66
Row seven 66 34 55 35 35 75
Row eight 55 70 84 52 55 35
Row nine 48 66 75 45 66 34
Total Row 456 556 681 425 474 454

MongoDB .local London 2019: A Complete Methodology to Data Modeling for MongoDB

MongoDB .local London 2019: A Complete Methodology to Data Modeling for MongoDB

Recommended

Recommended

More Related Content

Similar to MongoDB .local London 2019: A Complete Methodology to Data Modeling for MongoDB

Similar to MongoDB .local London 2019: A Complete Methodology to Data Modeling for MongoDB (20)

More from Lisa Roth, PMP

More from Lisa Roth, PMP (10)

Recently uploaded

Recently uploaded (20)

MongoDB .local London 2019: A Complete Methodology to Data Modeling for MongoDB