Tugdual Grall, profile picture
Uploaded byTugdual Grall
1,334 views

Drop your table ! MongoDB Schema Design

The document discusses MongoDB document design and contrasts it with relational database design, emphasizing the flexibility and structure of document models for complex data such as product catalogs and contact information. It provides examples of how to model various products and contact details using documents, highlighting the benefits of embedding and referencing data. Overall, the document serves as a guide to understanding document schemas and their advantages in handling data efficiently.

Embed presentation

@SoftShakeEvent DROP TABLE ! Overview of Document Design Tugdual Grall Technical Evangelist MongoDB @tgrall Soft Shake ‘14
{“about” : “me”} Tugdual “Tug” Grall • MongoDB Join me at the Gōng-fu I/O • Technical Evangelist • Couchbase • Technical Evangelist • eXo • CTO • Oracle • Developer/Product Manager • Mainly Java/SOA • Developer in consulting firms • Web • @tgrall • http://blog.grallandco.com • tgrall • NantesJUG co-founder • Pet Project : • http://www.resultri.com • tug@mongodb.com • tugdual@gmail.com
With “Relations” Between Rows Customer ID First Name Last Name City 0 John Doe New York 1 Mark Smith San Francisco 2 Jay Black Newark 3 Meagan White London 4 Edward Daniels Boston Account Number Branch ID Account Type Customer ID 10 100 Checking 0 11 101 Savings 0 12 101 IRA 0 13 200 Checking 1 14 200 Savings 1 15 201 IRA 2
But, some things are hard to model • Let’s look at a product catalog:
Baseball Bat ! -3 length to weight ratio 2-5/8" barrel diameter Two-piece construction R2 alloy barrel provides outstanding durability, performance and "pop" R2 composite handle shifts weight into the bat's knob for ultra-fast swing speeds Rifle Barrel design removes weight from the barrel for thinner wall thickness Acoustic barrel offers that sweet-sounding "ping" Contact grip helps eliminate sting and vibration AIR Elite is RIP-IT's® fastest BBCOR bat and the one with the most performance BBCOR certified - approved for high school and collegiate play Includes RIP-IT's® "Love It Or Return It" 30 Day Refund Policy with free return shipping Manufacturer's warranty: 400 days Made in the USA Model: B1403E
Bat Product Table Category Model Name Brand Length to weight ratio Barrel Dia Type Barrel Handle Cert. Country Price Bat B1403E Air Elite RIP-IT -3 2 5/8 Composite R2 Alloy R2 composite BBCOR USA $399.99 Bat B1403 Prototype RIP-IT -3 2 5/8 One-piece R1 Alloy R1 Alloy BBCOR USA $199.99 Bat MCB1B One Marucci -3 2 5/8 One-piece AZ3000 aluminum AZ3000 aluminum BBCOR Imported $199.99 Bat BB14S1 S1 Easton -3 2 5/8 Composite IMX SIC Black Carbon BBCOR China $399.99
Lets Add Gloves Size: 12" Infield/Outfield/Pitcher model 2-Piece Web pattern Most popular MLB® pattern among pitchers Pro Stock® American steerhide leather offers rugged durability and a superior feel Dual-Welting™ on "exposed edges" of the fingers helps maintain pocket shape and durability Pro Stock™ hand-designed pattern for unbeatable craftsmanship Dri-Lex® ultra-breathable wrist lining repels moisture from your hand Black leather with rich brown embellishments Pattern: B212 Model: WTA2000BBB212 Wilson
Bat and Glove Product Table Category Model Name Brand Length to weight ratio Barrel Dia Type Barrel Handle Cert. Country Price Bat B1403E Air Elite RIP-IT -3 2 5/8 Composite R2 Alloy R2 composite BBCOR USA $399.99 Bat B1403 Prototype RIP-IT -3 2 5/8 One-piece R1 Alloy R1 Alloy BBCOR USA $199.99 Bat MCB1B One Marucci -3 2 5/8 One-piece AL AL BBCOR Imported $199.99 Bat BB14S1 S1 Easton -3 2 5/8 Composite IMX SIC Black Carbon BBCOR China $399.99 Category Model Name Brand Size Position Pattern Web Pattern Material Color Country Price Glove WTA2000B BB212 A2000 Wilson 12" Infield B212 2-piece Leather black Vietnam $299.99 Glove PRO112PT HOH Pro Rawlings 11.25" Outfield Pro taper Modified Trap-Eze Horween Leather black China $229.99
Add some baseballs Cover: Full grain leather for excellent durability Core: Cushioned cork core Additions/Technologies: Made to the exact specifications of MLB Stitching/Seams: 108 classic red stitches/Rawlings® Major League seaming League/Certification(s): MLB Balls included per purchase: individual Recommended Age: All ages Model : ROMLB Rawlings
Bat and Glove and Ball Product Table Category Model Name Brand Length to weight ratio Barrel Dia Type Barrel Handle Cert. Country Price Bat B1403E Air Elite RIP-IT -3 2 5/8 Composite R2 Alloy R2 composite BBCOR USA $399.99 Bat B1403 Prototype RIP-IT -3 2 5/8 One-piece R1 Alloy R1 Alloy BBCOR USA $199.99 Bat MCB1B One Marucci -3 2 5/8 One-piece AL AL BBCOR Imported $199.99 Bat BB14S1 S1 Easton -3 2 5/8 Composite IMX SIC Black Carbon BBCOR China $399.99 Categor y Model Name Brand Size Position Pattern Web Pattern Material Color Country Price Glove WTA200 0BBB21 2 A2000 Wilson 12" Infield B212 2-piece Leather black Vietnam $299.99 Glove PRO112 PT HOH Pro Rawling s 11.25" Outfield Pro taper Modifie d Trap- Eze Horwee n Leather black China $229.99 Category Model Name Brand Color Cover Core Cert. Country Price Baseball DICRLLB1 PBG Little League Rawlings white Leather Cork rubber Little League China $4.99 Baseball ROML MLB Rawlings white Leather cork China $6.99
Sparse Table Category Model Name Brand Length to weight ratio Barrel Dia Type Barrel Handle Certificati on Country Price Size Position Pattern Web Pattern Material Color Cover Core Bat B1403E Air Elite RIP-­‐IT -­‐3 2 5/8 Composite R2 Alloy R2 composite BBCOR USA $399.99 Bat B1403 Prototype RIP-­‐IT -­‐3 2 5/8 One-­‐piece R1 Alloy R1 Alloy BBCOR USA $199.99 Bat MCB1B One Marucci -­‐3 2 5/8 One-­‐piece AZ3000 aluminum AZ3000 aluminum BBCOR Imported $199.99 Bat BB14S1 S1 Easton -­‐3 2 5/8 Composite IMX SIC Black Carbon BBCOR China $399.99 Glove WTA2000BB B212 A2000 Wilson Vietnam $299.99 12" Infield B212 2-­‐piece Leather black Glove PRO112PT HOH Pro Rawlings China $229.99 11.25" Outfield Pro taper Modified Trap-­‐Eze Horween Leather black Baseball DICRLLB1PB G Little League Rawlings Little League China $4.99 white Leather cork and rubber Baseball ROML MLB Rawlings China $6.99 white Leather cork Continue adding columns as you add new products
Maybe this design will work better prodID property value 1 length/ weight -3 1 barrel dia 2 5/8 1 type composite 1 certification BBCOR … 5 size 12 5 position infield 5 pattern B212 5 material leather 5 color black … 8 color white 8 cover leather 8 core cork prodID Category Model Name Brand Country Price 1 Bat B1403E Air Elite RIP-IT USA $399.99 2 Bat B1403 Prototype RIP-IT USA $199.99 3 Bat MCB1B One Marucci Imported $199.99 4 Bat BB14S1 S1 Easton China $399.99 5 Glove WTA2000B BB212 A2000 Wilson Vietnam $299.99 6 Glove PRO112PT HOH Pro Rawlings China $229.99 DICRLLB1 7 Baseball PBG Little League Rawlings China $4.99 8 Baseball ROML MLB Rawlings China $6.99
It is hard to iterate New Column Name Age Phone Email New Table New Table New Column
Have to Manage Changes in 3 Places Code XML Config DB Schema Relational Database Object Relational Application Mapping
Working with Documents
Match the Data in your Application Relational MongoDB { first_name: ‘Paul’, surname: ‘Miller’ city: ‘London’, location: [45.123,47.232], cars: [ { model: ‘Bentley’, year: 1973, value: 100000}, { model: ‘Rolls Royce’, year: 1965,! value: 330000}! } }
Let’s go back to our product catalog
How would we model this in MongoDB? Size: 12" Infield/Outfield/Pitcher model 2-Piece Web pattern Most popular MLB® pattern among pitchers Pro Stock® American steerhide leather offers rugged durability and a superior feel Dual-Welting™ on "exposed edges" of the fingers helps maintain pocket shape and durability Pro Stock™ hand-designed pattern for unbeatable craftsmanship Dri-Lex® ultra-breathable wrist lining repels moisture from your hand Black leather with rich brown embellishments Pattern: B212 Model: WTA2000BBB212 Wilson
We use a document ! { category: “glove”, model: “PRO112PT”, name: “Air Elite”, brand: “Rawlings”, price: 229.99, available: Date(“2013-03-31”) } Fields Values Field values are typed string number date
Documents are rich structures ! { category: “glove”, model: “PRO112PT”, name: “Air Elite”, brand: “Rawlings”, price: 229.99, available: Date(“2013-03-31”), position: [“infield”, “outfield”, “pitcher”] } Fields can contain arrays
Documents are rich structures ! { category: “glove”, model: “PRO112PT”, name: “Air Elite”, brand: “Rawlings”, price: 229.99, available: Date(“2013-03-31”), position: [“infield”, “outfield”, “pitcher”], endorsed: {name: “Ryan Howard”, team: “Phillies”, position: “first base”}, } Fields can contain sub-documents
` { category: “glove”, model: “PRO112PT”, name: “Air Elite”, brand: “Rawlings”, price: 229.99, available: Date(“2013-03-31”), position: [“infield”, “outfield”, “pitcher”], endorsed: {name: “Ryan Howard”, team: “Phillies”, position: “first base”}, history: [{date: Date(“2013-03-31”), price: 279.99}, {date: Date(“2013-06-01”), price: 259.79}, {date: Date(“2013-08-15”), price: 229.99}] } Fields can contain an array of sub-documents
Variation is easy with document model { category: bat, model: B1403E, name: Air Elite, brand: “Rip-IT”, price: 399.99 ! diameter: “2 5/8”, barrel: R2 Alloy, handle: R2 Composite, type: composite, } ! { category: glove, model: PRO112PT, name: Air Elite, brand: “Rawlings”, price: “229.99” ! size: 11.25, position: outfield, pattern: “Pro taper”, material: leather, color: black } { category: ball, model: ROML, name: MLB, brand: “Rawlings”, price: “6.99” ! cover: leather, core: cork, color: white }
Relational Schema Design Focus on data storage
Document Schema Design Focus on data use
Document Design
Schema Design Considerations • How do we manipulate the data? – Dynamic Ad-Hoc Queries – Atomic Updates – Aggregation • What are the access patterns of the application? – Read/Write Ratio – Types of Queries / Updates – Data life-cycle and growth rate
Let's model something together How about a business card?
Business Card
Referencing Addresses! ! {! ! “_id”: 1,! ! “street”: “10260 Bandley Dr”,! ! “city”: “Cupertino”,! ! “state”: “CA”,! ! “zip_code”: ”95014”,! ! “country”: “USA”! } Contacts! ! {! “_id”: 2,! “name”: “Steven Jobs”,! “title”: “VP, New Product Development”,“company”: “Apple Computer”,! “phone”: “408-996-1010”,! “address_id”: 1! }
Embedding Contacts! ! {! “_id”: 2,! “name”: “Steven Jobs”,! “title”: “VP, New Product Development”,! “company”: “Apple Computer”,! “address”: {! ! “street”: “10260 Bandley Dr”,! ! “city”: “Cupertino”,! ! “state”: “CA”,! ! “zip_code”: ”95014”,! ! “country”: “USA”! },! “phone”: “408-996-1010”! }!
! Contact Relational Schema ! • name • company • title • phone Address ! • street • city • state • zip_code
! Contact ! • name • company • adress address • Street street • City city • State • Zip zip_code • title • phone Document Schema
Schema Flexibility {! “name”: “Steven Jobs”,! “title”: “VP, New Product Development”,! “company”: “Apple Computer”,! “address”: {! ! “street”: “10260 Bandley Dr”,! ! “city”: “Cupertino”,! ! “state”: “CA”,! ! “zip_code”: ”95014”! },! “phone”: “408-996-1010”! }! {! “name”: “Larry Page”,! “url”: “http://google.com/”,! “title”: “CEO”,! “company”: “Google!”,! “email”: “larry@google.com”,! “address”: {! “street”: “555 Bryant, #106”,! “city”: “Palo Alto”,! “state”: “CA”,! “zip_code”: “94301”! }! “phone”: “650-618-1499”,! “fax”: “650-330-0100”! }
Example
Let’s Look at an Address Book
Groups N N ! • name Contacts • name • company • title Addresses • type • street • city • state • zip_code Phones • type • number Emails • type • address Twitters • name • location • web • bio Thumbnails • mime_type • data Portraits • mime_type • data N 1 1 1 1 1 1 1 1 N N 1 Address Book Entity-Relationship
Associating Entities
! Contacts • name • company • title One to One Addresses • type • street • city • state • zip_code Phones • type • number Emails • type • address Thumbnails • mime_type • data Portraits • mime_type • data Groups • name N 1 N 1 N N N 1 1 1 1 1 Twitters • name • location • web • bio 1 1
One to One General Recommendation • Full contact info all at once – Contact embeds twitter • Parent-child relationship – “contains” • twitter twitter 1 • No additional data duplication • Can query or index on embedded field – e.g., “twitter.name” – Exceptional cases… Contact • Reference portrait which has very large data
! Contacts • name • company • title One to Many Addresses • type • street • city • state • zip_code Phones • type • number Emails • type • address Thumbnails • mime_type • data Portraits • mime_type • data Groups • name N 1 N 1 N N N 1 1 1 1 1 Twitters • name • location • web • bio 1 1
One to Many General Recommendation • Full contact info all at once – Contact embeds multiple phones • Parent-children relationship – “contains” • No additional data duplication • Can query or index on any field – e.g., { “phones.type”: “mobile” } – Exceptional cases… Contact • phones phone N • Scaling: maximum document size is 16MB
! Contacts • name • company • title Many to Many Addresses • type • street • city • state • zip_code Phones • type • number Emails • type • address Thumbnails • mime_type • data Portraits • mime_type • data Groups • name N 1 N 1 N N N 1 1 1 1 1 Twitters • name • location • web • bio 1 1
Many to Many Traditional Relational Association X Join table ! Contacts • name • company • title • phone Groups • name GroupContacts • group_id • contact_id Use arrays instead
Many to Many General Recommendation • Depends on use case 1. Simple address book • Contact references groups 2. Corporate email groups group contact N N • group_ids: [ ] • Group embeds contacts for performance • Exceptional cases – Scaling: maximum document size is 16MB – Scaling may affect performance and working set
Contacts • name • company • title addresses • type • street • city • state • zip_code phones • type • number emails • type • address thumbnail • mime_type • data Groups • name N Portraits • mime_type • data N 1 1 twitter • name • location • web • bio N N N 1 1 Document model - holistic and efficient representation
Contact document example {! ! “name” : “Gary J. Murakami, Ph.D.”,! ! “company” : “MongoDB, Inc.”,! ! “title” : “Lead Engineer”,! ! “twitter” : {! ! ! “name” : “Gary Murakami”, “location” : “New Providence, NJ”,! ! ! “web” : “http://www.nobell.org”! ! },! ! “portrait_id” : 1,! ! “addresses” : [ ! ! ! { “type” : “work”, “street” : ”229 W 43rd St.”, “city” : “New York”, “zip_code” : “10036” }! ! ],! ! “phones” : [ ! ! ! { “type” : “work”, “number” : “1-866-237-8815 x8015” }! ! ],! ! “emails” : [ ! ! ! { “type” : “work”, “address” : “gary.murakami@mongodb.com” },! ! ! { “type” : “home”, “address” : “gjm@nobell.org” }! ! ]! ! }!
General Recommendations
Legacy Migration 1. Copy existing schema & some data to MongoDB 2. Iterate schema design development Measure performance, find bottlenecks, and embed 1. one to one associations first 2. one to many associations next 3. many to many associations 3. Migrate full dataset to new schema New Software Application? Embed by default
Embedding over Referencing • Embedding is a bit like pre-joined data – BSON (Binary JSON) document ops are easy for the server • Embed (90/10 following rule of thumb) – When the “one” or “many” objects are viewed in the context of their parent – For performance – For atomicity • Reference – When you need more scaling – For easy consistency with “many to many” associations without duplicated data
It’s All About Your Application • Programs+Databases = (Big) Data Applications • Your schema is the impedance matcher – Design choices: normalize/denormalize, reference/embed – Melds programming with MongoDB for best of both – Flexible for development and change • Programs×MongoDB = Great Big Data Applications
@SoftShakeEvent Thank You! Tugdual Grall Technical Evangelist MongoDB @tgrall Soft Shake ‘14

More Related Content

PDF
Introduction to Streaming with Apache Flink
byTugdual Grall
 
PDF
Rothco Boot Catalog
bymarketingintern1
 
PDF
Binder1
byGreggS
 
PDF
Introduction to NoSQL with MongoDB - SQLi Workshop
byTugdual Grall
 
PPT
eDays - Quadrem Approach
byJens Brown
 
PPTX
Buying guide of outdoor basketball in 2021
byStacy Robert
 
PDF
MongoDB Schema Design (Event: An Evening with MongoDB Houston 3/11/15)
byMongoDB
 
PPT
Baseball
byguestee9099
 
Introduction to Streaming with Apache Flink
byTugdual Grall
 
Rothco Boot Catalog
bymarketingintern1
 
Binder1
byGreggS
 
Introduction to NoSQL with MongoDB - SQLi Workshop
byTugdual Grall
 
eDays - Quadrem Approach
byJens Brown
 
Buying guide of outdoor basketball in 2021
byStacy Robert
 
MongoDB Schema Design (Event: An Evening with MongoDB Houston 3/11/15)
byMongoDB
 
Baseball
byguestee9099
 

More from Tugdual Grall

PDF
Big Data Journey
byTugdual Grall
 
PDF
Introduction to Streaming with Apache Flink
byTugdual Grall
 
PDF
Fast Cars, Big Data - How Streaming Can Help Formula 1
byTugdual Grall
 
PPTX
Lambda Architecture: The Best Way to Build Scalable and Reliable Applications!
byTugdual Grall
 
PPTX
MongoDB and Hadoop
byTugdual Grall
 
PDF
Neotys conference
byTugdual Grall
 
PDF
Building Your First MongoDB Application
byTugdual Grall
 
PDF
Big Data Israel Meetup : Couchbase and Big Data
byTugdual Grall
 
PDF
FOSDEM 2013 : Getting Started with Couchhbase Server 2.0
byTugdual Grall
 
PDF
Softshake 2013: Introduction to NoSQL with Couchbase
byTugdual Grall
 
PDF
NoSQL Matters 2013 - Introduction to Map Reduce with Couchbase 2.0
byTugdual Grall
 
PDF
Some cool features of MongoDB
byTugdual Grall
 
PDF
Enabling Telco to Build and Run Modern Applications
byTugdual Grall
 
PPT
Big Data Paris : Hadoop and NoSQL
byTugdual Grall
 
PDF
Why and How to integrate Hadoop and NoSQL?
byTugdual Grall
 
PDF
Introduction to NoSQL with Couchbase
byTugdual Grall
 
PDF
Devoxx 2014 : Atelier MongoDB - Decouverte de MongoDB 2.6
byTugdual Grall
 
PDF
Proud to be polyglot
byTugdual Grall
 
PDF
Opensourceday 2014-iot
byTugdual Grall
 
PDF
Proud to be Polyglot - Riviera Dev 2015
byTugdual Grall
 
Big Data Journey
byTugdual Grall
 
Introduction to Streaming with Apache Flink
byTugdual Grall
 
Fast Cars, Big Data - How Streaming Can Help Formula 1
byTugdual Grall
 
Lambda Architecture: The Best Way to Build Scalable and Reliable Applications!
byTugdual Grall
 
MongoDB and Hadoop
byTugdual Grall
 
Neotys conference
byTugdual Grall
 
Building Your First MongoDB Application
byTugdual Grall
 
Big Data Israel Meetup : Couchbase and Big Data
byTugdual Grall
 
FOSDEM 2013 : Getting Started with Couchhbase Server 2.0
byTugdual Grall
 
Softshake 2013: Introduction to NoSQL with Couchbase
byTugdual Grall
 
NoSQL Matters 2013 - Introduction to Map Reduce with Couchbase 2.0
byTugdual Grall
 
Some cool features of MongoDB
byTugdual Grall
 
Enabling Telco to Build and Run Modern Applications
byTugdual Grall
 
Big Data Paris : Hadoop and NoSQL
byTugdual Grall
 
Why and How to integrate Hadoop and NoSQL?
byTugdual Grall
 
Introduction to NoSQL with Couchbase
byTugdual Grall
 
Devoxx 2014 : Atelier MongoDB - Decouverte de MongoDB 2.6
byTugdual Grall
 
Proud to be polyglot
byTugdual Grall
 
Opensourceday 2014-iot
byTugdual Grall
 
Proud to be Polyglot - Riviera Dev 2015
byTugdual Grall
 

Recently uploaded

PDF
BO049FHDMO 0.49 Inch Micro OLED Datasheet
bySyluxdisplay
 
PDF
Unit 1.2 Components of a Computer System.pdf
bySanieBautista
 
PPTX
Cesium formate brine - A brief history - prepared by John Downs in May 2011
byJohn Downs
 
PDF
Xemelgo - RFID Industry Predictions for 2026
byRich Rogers
 
PDF
MAD (1).pdf Mobile application develipment
byprathiT1
 
PPTX
Salesforce Spring 26 Release Key .pptx
byMehediHasan939830
 
PPTX
Document Reconstruction using AI & Deep Learning
bysonjuktaweb
 
PDF
The Future of AI-Powered Fashion Design 10 Top Generators for 2026
bymockitseo
 
PDF
Regenerative Agriculture Finance : Environmental impact
byrose mason
 
PDF
A Brief Introduction About Christopher Elwell Woburn
byChristopher Elwell Woburn
 
PDF
Benefits of Using the IAC 500 Sensor for Ambient Conditions
byCS Instruments
 
PDF
The map to conquer linear algebra for IT engineer
byakipii ogaoga
 
PDF
How Application Performance Monitoring Tools Are Used in Performance Testing
byhenrycavill30521
 
PDF
Industrial RFID Landscape for 2025 - Readers, Tags, Antennas, Printers, etc
byRich Rogers
 
PDF
What is Voice User Interface (VUI) Definition & Examples.pdf
byDesign Studio UI UX
 
PPTX
Cybersecurity Basics: Understanding Threats, Protection Methods, and Safe Dig...
byDhruvManiar3
 
PPT
Information and Communication Technologies and Power
byJeffrey Hart
 
PDF
Digital Marketing Trends in 2026: AI, Data, Content & Future Growth
byConacent Consulting
 
PPTX
UNIT III TYPOLOGY OF CRIME AND CRIMINAL BEHAVIOUR (1).pptx
bybloodyhumanoid
 
PPTX
Digital transformation success powered by EPM and NexInfo.pptx
byjayasuryanexinfo
 
BO049FHDMO 0.49 Inch Micro OLED Datasheet
bySyluxdisplay
 
Unit 1.2 Components of a Computer System.pdf
bySanieBautista
 
Cesium formate brine - A brief history - prepared by John Downs in May 2011
byJohn Downs
 
Xemelgo - RFID Industry Predictions for 2026
byRich Rogers
 
MAD (1).pdf Mobile application develipment
byprathiT1
 
Salesforce Spring 26 Release Key .pptx
byMehediHasan939830
 
Document Reconstruction using AI & Deep Learning
bysonjuktaweb
 
The Future of AI-Powered Fashion Design 10 Top Generators for 2026
bymockitseo
 
Regenerative Agriculture Finance : Environmental impact
byrose mason
 
A Brief Introduction About Christopher Elwell Woburn
byChristopher Elwell Woburn
 
Benefits of Using the IAC 500 Sensor for Ambient Conditions
byCS Instruments
 
The map to conquer linear algebra for IT engineer
byakipii ogaoga
 
How Application Performance Monitoring Tools Are Used in Performance Testing
byhenrycavill30521
 
Industrial RFID Landscape for 2025 - Readers, Tags, Antennas, Printers, etc
byRich Rogers
 
What is Voice User Interface (VUI) Definition & Examples.pdf
byDesign Studio UI UX
 
Cybersecurity Basics: Understanding Threats, Protection Methods, and Safe Dig...
byDhruvManiar3
 
Information and Communication Technologies and Power
byJeffrey Hart
 
Digital Marketing Trends in 2026: AI, Data, Content & Future Growth
byConacent Consulting
 
UNIT III TYPOLOGY OF CRIME AND CRIMINAL BEHAVIOUR (1).pptx
bybloodyhumanoid
 
Digital transformation success powered by EPM and NexInfo.pptx
byjayasuryanexinfo
 

Drop your table ! MongoDB Schema Design

  • 1.
    @SoftShakeEvent DROP TABLE! Overview of Document Design Tugdual Grall Technical Evangelist MongoDB @tgrall Soft Shake ‘14
  • 2.
    {“about” : “me”} Tugdual “Tug” Grall • MongoDB Join me at the Gōng-fu I/O • Technical Evangelist • Couchbase • Technical Evangelist • eXo • CTO • Oracle • Developer/Product Manager • Mainly Java/SOA • Developer in consulting firms • Web • @tgrall • http://blog.grallandco.com • tgrall • NantesJUG co-founder • Pet Project : • http://www.resultri.com • tug@mongodb.com • tugdual@gmail.com
  • 3.
    With “Relations” BetweenRows Customer ID First Name Last Name City 0 John Doe New York 1 Mark Smith San Francisco 2 Jay Black Newark 3 Meagan White London 4 Edward Daniels Boston Account Number Branch ID Account Type Customer ID 10 100 Checking 0 11 101 Savings 0 12 101 IRA 0 13 200 Checking 1 14 200 Savings 1 15 201 IRA 2
  • 4.
    But, some thingsare hard to model • Let’s look at a product catalog:
  • 5.
    Baseball Bat ! -3 length to weight ratio 2-5/8" barrel diameter Two-piece construction R2 alloy barrel provides outstanding durability, performance and "pop" R2 composite handle shifts weight into the bat's knob for ultra-fast swing speeds Rifle Barrel design removes weight from the barrel for thinner wall thickness Acoustic barrel offers that sweet-sounding "ping" Contact grip helps eliminate sting and vibration AIR Elite is RIP-IT's® fastest BBCOR bat and the one with the most performance BBCOR certified - approved for high school and collegiate play Includes RIP-IT's® "Love It Or Return It" 30 Day Refund Policy with free return shipping Manufacturer's warranty: 400 days Made in the USA Model: B1403E
  • 6.
    Bat Product Table Category Model Name Brand Length to weight ratio Barrel Dia Type Barrel Handle Cert. Country Price Bat B1403E Air Elite RIP-IT -3 2 5/8 Composite R2 Alloy R2 composite BBCOR USA $399.99 Bat B1403 Prototype RIP-IT -3 2 5/8 One-piece R1 Alloy R1 Alloy BBCOR USA $199.99 Bat MCB1B One Marucci -3 2 5/8 One-piece AZ3000 aluminum AZ3000 aluminum BBCOR Imported $199.99 Bat BB14S1 S1 Easton -3 2 5/8 Composite IMX SIC Black Carbon BBCOR China $399.99
  • 7.
    Lets Add Gloves Size: 12" Infield/Outfield/Pitcher model 2-Piece Web pattern Most popular MLB® pattern among pitchers Pro Stock® American steerhide leather offers rugged durability and a superior feel Dual-Welting™ on "exposed edges" of the fingers helps maintain pocket shape and durability Pro Stock™ hand-designed pattern for unbeatable craftsmanship Dri-Lex® ultra-breathable wrist lining repels moisture from your hand Black leather with rich brown embellishments Pattern: B212 Model: WTA2000BBB212 Wilson
  • 8.
    Bat and GloveProduct Table Category Model Name Brand Length to weight ratio Barrel Dia Type Barrel Handle Cert. Country Price Bat B1403E Air Elite RIP-IT -3 2 5/8 Composite R2 Alloy R2 composite BBCOR USA $399.99 Bat B1403 Prototype RIP-IT -3 2 5/8 One-piece R1 Alloy R1 Alloy BBCOR USA $199.99 Bat MCB1B One Marucci -3 2 5/8 One-piece AL AL BBCOR Imported $199.99 Bat BB14S1 S1 Easton -3 2 5/8 Composite IMX SIC Black Carbon BBCOR China $399.99 Category Model Name Brand Size Position Pattern Web Pattern Material Color Country Price Glove WTA2000B BB212 A2000 Wilson 12" Infield B212 2-piece Leather black Vietnam $299.99 Glove PRO112PT HOH Pro Rawlings 11.25" Outfield Pro taper Modified Trap-Eze Horween Leather black China $229.99
  • 9.
    Add some baseballs Cover: Full grain leather for excellent durability Core: Cushioned cork core Additions/Technologies: Made to the exact specifications of MLB Stitching/Seams: 108 classic red stitches/Rawlings® Major League seaming League/Certification(s): MLB Balls included per purchase: individual Recommended Age: All ages Model : ROMLB Rawlings
  • 10.
    Bat and Gloveand Ball Product Table Category Model Name Brand Length to weight ratio Barrel Dia Type Barrel Handle Cert. Country Price Bat B1403E Air Elite RIP-IT -3 2 5/8 Composite R2 Alloy R2 composite BBCOR USA $399.99 Bat B1403 Prototype RIP-IT -3 2 5/8 One-piece R1 Alloy R1 Alloy BBCOR USA $199.99 Bat MCB1B One Marucci -3 2 5/8 One-piece AL AL BBCOR Imported $199.99 Bat BB14S1 S1 Easton -3 2 5/8 Composite IMX SIC Black Carbon BBCOR China $399.99 Categor y Model Name Brand Size Position Pattern Web Pattern Material Color Country Price Glove WTA200 0BBB21 2 A2000 Wilson 12" Infield B212 2-piece Leather black Vietnam $299.99 Glove PRO112 PT HOH Pro Rawling s 11.25" Outfield Pro taper Modifie d Trap- Eze Horwee n Leather black China $229.99 Category Model Name Brand Color Cover Core Cert. Country Price Baseball DICRLLB1 PBG Little League Rawlings white Leather Cork rubber Little League China $4.99 Baseball ROML MLB Rawlings white Leather cork China $6.99
  • 11.
    Sparse Table CategoryModel Name Brand Length to weight ratio Barrel Dia Type Barrel Handle Certificati on Country Price Size Position Pattern Web Pattern Material Color Cover Core Bat B1403E Air Elite RIP-­‐IT -­‐3 2 5/8 Composite R2 Alloy R2 composite BBCOR USA $399.99 Bat B1403 Prototype RIP-­‐IT -­‐3 2 5/8 One-­‐piece R1 Alloy R1 Alloy BBCOR USA $199.99 Bat MCB1B One Marucci -­‐3 2 5/8 One-­‐piece AZ3000 aluminum AZ3000 aluminum BBCOR Imported $199.99 Bat BB14S1 S1 Easton -­‐3 2 5/8 Composite IMX SIC Black Carbon BBCOR China $399.99 Glove WTA2000BB B212 A2000 Wilson Vietnam $299.99 12" Infield B212 2-­‐piece Leather black Glove PRO112PT HOH Pro Rawlings China $229.99 11.25" Outfield Pro taper Modified Trap-­‐Eze Horween Leather black Baseball DICRLLB1PB G Little League Rawlings Little League China $4.99 white Leather cork and rubber Baseball ROML MLB Rawlings China $6.99 white Leather cork Continue adding columns as you add new products
  • 12.
    Maybe this designwill work better prodID property value 1 length/ weight -3 1 barrel dia 2 5/8 1 type composite 1 certification BBCOR … 5 size 12 5 position infield 5 pattern B212 5 material leather 5 color black … 8 color white 8 cover leather 8 core cork prodID Category Model Name Brand Country Price 1 Bat B1403E Air Elite RIP-IT USA $399.99 2 Bat B1403 Prototype RIP-IT USA $199.99 3 Bat MCB1B One Marucci Imported $199.99 4 Bat BB14S1 S1 Easton China $399.99 5 Glove WTA2000B BB212 A2000 Wilson Vietnam $299.99 6 Glove PRO112PT HOH Pro Rawlings China $229.99 DICRLLB1 7 Baseball PBG Little League Rawlings China $4.99 8 Baseball ROML MLB Rawlings China $6.99
  • 13.
    It is hardto iterate New Column Name Age Phone Email New Table New Table New Column
  • 14.
    Have to ManageChanges in 3 Places Code XML Config DB Schema Relational Database Object Relational Application Mapping
  • 15.
  • 16.
    Match the Datain your Application Relational MongoDB { first_name: ‘Paul’, surname: ‘Miller’ city: ‘London’, location: [45.123,47.232], cars: [ { model: ‘Bentley’, year: 1973, value: 100000}, { model: ‘Rolls Royce’, year: 1965,! value: 330000}! } }
  • 17.
    Let’s go backto our product catalog
  • 18.
    How would wemodel this in MongoDB? Size: 12" Infield/Outfield/Pitcher model 2-Piece Web pattern Most popular MLB® pattern among pitchers Pro Stock® American steerhide leather offers rugged durability and a superior feel Dual-Welting™ on "exposed edges" of the fingers helps maintain pocket shape and durability Pro Stock™ hand-designed pattern for unbeatable craftsmanship Dri-Lex® ultra-breathable wrist lining repels moisture from your hand Black leather with rich brown embellishments Pattern: B212 Model: WTA2000BBB212 Wilson
  • 19.
    We use adocument ! { category: “glove”, model: “PRO112PT”, name: “Air Elite”, brand: “Rawlings”, price: 229.99, available: Date(“2013-03-31”) } Fields Values Field values are typed string number date
  • 20.
    Documents are richstructures ! { category: “glove”, model: “PRO112PT”, name: “Air Elite”, brand: “Rawlings”, price: 229.99, available: Date(“2013-03-31”), position: [“infield”, “outfield”, “pitcher”] } Fields can contain arrays
  • 21.
    Documents are richstructures ! { category: “glove”, model: “PRO112PT”, name: “Air Elite”, brand: “Rawlings”, price: 229.99, available: Date(“2013-03-31”), position: [“infield”, “outfield”, “pitcher”], endorsed: {name: “Ryan Howard”, team: “Phillies”, position: “first base”}, } Fields can contain sub-documents
  • 22.
    ` { category:“glove”, model: “PRO112PT”, name: “Air Elite”, brand: “Rawlings”, price: 229.99, available: Date(“2013-03-31”), position: [“infield”, “outfield”, “pitcher”], endorsed: {name: “Ryan Howard”, team: “Phillies”, position: “first base”}, history: [{date: Date(“2013-03-31”), price: 279.99}, {date: Date(“2013-06-01”), price: 259.79}, {date: Date(“2013-08-15”), price: 229.99}] } Fields can contain an array of sub-documents
  • 23.
    Variation is easywith document model { category: bat, model: B1403E, name: Air Elite, brand: “Rip-IT”, price: 399.99 ! diameter: “2 5/8”, barrel: R2 Alloy, handle: R2 Composite, type: composite, } ! { category: glove, model: PRO112PT, name: Air Elite, brand: “Rawlings”, price: “229.99” ! size: 11.25, position: outfield, pattern: “Pro taper”, material: leather, color: black } { category: ball, model: ROML, name: MLB, brand: “Rawlings”, price: “6.99” ! cover: leather, core: cork, color: white }
  • 24.
    Relational Schema Design Focus on data storage
  • 25.
    Document Schema Design Focus on data use
  • 26.
  • 27.
    Schema Design Considerations • How do we manipulate the data? – Dynamic Ad-Hoc Queries – Atomic Updates – Aggregation • What are the access patterns of the application? – Read/Write Ratio – Types of Queries / Updates – Data life-cycle and growth rate
  • 28.
    Let's model somethingtogether How about a business card?
  • 29.
  • 30.
    Referencing Addresses! ! {! ! “_id”: 1,! ! “street”: “10260 Bandley Dr”,! ! “city”: “Cupertino”,! ! “state”: “CA”,! ! “zip_code”: ”95014”,! ! “country”: “USA”! } Contacts! ! {! “_id”: 2,! “name”: “Steven Jobs”,! “title”: “VP, New Product Development”,“company”: “Apple Computer”,! “phone”: “408-996-1010”,! “address_id”: 1! }
  • 31.
    Embedding Contacts! ! {! “_id”: 2,! “name”: “Steven Jobs”,! “title”: “VP, New Product Development”,! “company”: “Apple Computer”,! “address”: {! ! “street”: “10260 Bandley Dr”,! ! “city”: “Cupertino”,! ! “state”: “CA”,! ! “zip_code”: ”95014”,! ! “country”: “USA”! },! “phone”: “408-996-1010”! }!
  • 32.
    ! Contact RelationalSchema ! • name • company • title • phone Address ! • street • city • state • zip_code
  • 33.
    ! Contact ! • name • company • adress address • Street street • City city • State • Zip zip_code • title • phone Document Schema
  • 34.
    Schema Flexibility {! “name”: “Steven Jobs”,! “title”: “VP, New Product Development”,! “company”: “Apple Computer”,! “address”: {! ! “street”: “10260 Bandley Dr”,! ! “city”: “Cupertino”,! ! “state”: “CA”,! ! “zip_code”: ”95014”! },! “phone”: “408-996-1010”! }! {! “name”: “Larry Page”,! “url”: “http://google.com/”,! “title”: “CEO”,! “company”: “Google!”,! “email”: “larry@google.com”,! “address”: {! “street”: “555 Bryant, #106”,! “city”: “Palo Alto”,! “state”: “CA”,! “zip_code”: “94301”! }! “phone”: “650-618-1499”,! “fax”: “650-330-0100”! }
  • 35.
  • 36.
    Let’s Look atan Address Book
  • 37.
    Groups N N ! • name Contacts • name • company • title Addresses • type • street • city • state • zip_code Phones • type • number Emails • type • address Twitters • name • location • web • bio Thumbnails • mime_type • data Portraits • mime_type • data N 1 1 1 1 1 1 1 1 N N 1 Address Book Entity-Relationship
  • 38.
  • 39.
    ! Contacts •name • company • title One to One Addresses • type • street • city • state • zip_code Phones • type • number Emails • type • address Thumbnails • mime_type • data Portraits • mime_type • data Groups • name N 1 N 1 N N N 1 1 1 1 1 Twitters • name • location • web • bio 1 1
  • 40.
    One to One General Recommendation • Full contact info all at once – Contact embeds twitter • Parent-child relationship – “contains” • twitter twitter 1 • No additional data duplication • Can query or index on embedded field – e.g., “twitter.name” – Exceptional cases… Contact • Reference portrait which has very large data
  • 41.
    ! Contacts •name • company • title One to Many Addresses • type • street • city • state • zip_code Phones • type • number Emails • type • address Thumbnails • mime_type • data Portraits • mime_type • data Groups • name N 1 N 1 N N N 1 1 1 1 1 Twitters • name • location • web • bio 1 1
  • 42.
    One to Many General Recommendation • Full contact info all at once – Contact embeds multiple phones • Parent-children relationship – “contains” • No additional data duplication • Can query or index on any field – e.g., { “phones.type”: “mobile” } – Exceptional cases… Contact • phones phone N • Scaling: maximum document size is 16MB
  • 43.
    ! Contacts •name • company • title Many to Many Addresses • type • street • city • state • zip_code Phones • type • number Emails • type • address Thumbnails • mime_type • data Portraits • mime_type • data Groups • name N 1 N 1 N N N 1 1 1 1 1 Twitters • name • location • web • bio 1 1
  • 44.
    Many to Many Traditional Relational Association X Join table ! Contacts • name • company • title • phone Groups • name GroupContacts • group_id • contact_id Use arrays instead
  • 45.
    Many to Many General Recommendation • Depends on use case 1. Simple address book • Contact references groups 2. Corporate email groups group contact N N • group_ids: [ ] • Group embeds contacts for performance • Exceptional cases – Scaling: maximum document size is 16MB – Scaling may affect performance and working set
  • 46.
    Contacts • name • company • title addresses • type • street • city • state • zip_code phones • type • number emails • type • address thumbnail • mime_type • data Groups • name N Portraits • mime_type • data N 1 1 twitter • name • location • web • bio N N N 1 1 Document model - holistic and efficient representation
  • 47.
    Contact document example {! ! “name” : “Gary J. Murakami, Ph.D.”,! ! “company” : “MongoDB, Inc.”,! ! “title” : “Lead Engineer”,! ! “twitter” : {! ! ! “name” : “Gary Murakami”, “location” : “New Providence, NJ”,! ! ! “web” : “http://www.nobell.org”! ! },! ! “portrait_id” : 1,! ! “addresses” : [ ! ! ! { “type” : “work”, “street” : ”229 W 43rd St.”, “city” : “New York”, “zip_code” : “10036” }! ! ],! ! “phones” : [ ! ! ! { “type” : “work”, “number” : “1-866-237-8815 x8015” }! ! ],! ! “emails” : [ ! ! ! { “type” : “work”, “address” : “gary.murakami@mongodb.com” },! ! ! { “type” : “home”, “address” : “gjm@nobell.org” }! ! ]! ! }!
  • 48.
  • 49.
    Legacy Migration 1.Copy existing schema & some data to MongoDB 2. Iterate schema design development Measure performance, find bottlenecks, and embed 1. one to one associations first 2. one to many associations next 3. many to many associations 3. Migrate full dataset to new schema New Software Application? Embed by default
  • 50.
    Embedding over Referencing • Embedding is a bit like pre-joined data – BSON (Binary JSON) document ops are easy for the server • Embed (90/10 following rule of thumb) – When the “one” or “many” objects are viewed in the context of their parent – For performance – For atomicity • Reference – When you need more scaling – For easy consistency with “many to many” associations without duplicated data
  • 51.
    It’s All AboutYour Application • Programs+Databases = (Big) Data Applications • Your schema is the impedance matcher – Design choices: normalize/denormalize, reference/embed – Melds programming with MongoDB for best of both – Flexible for development and change • Programs×MongoDB = Great Big Data Applications
  • 52.
    @SoftShakeEvent Thank You! Tugdual Grall Technical Evangelist MongoDB @tgrall Soft Shake ‘14