MongoDB: What,why,when.
SolutionsArchitect,MongoDB Inc.
Massimo Brignoli
#mongodb
Who Am I?
• Solutions Architect/Evangelist in MongoDB Inc.
• 24 years of experience in databases and software
development
• Former MySQL employee
• Previous life: web,web,web
Innovation
Understanding Big Data – It’s Not Very“Big”
from Big Data Executive Summary – 50+ top executives from Government and F500 firms
64% - Ingest diverse,
new data in real-time
15% - More than 100TB
of data
20% - Less than 100TB
(average of all? <20TB)
“I have not failed. I've just found 10,000 ways that won't work.”
― Thomas A. Edison
Back in 1970…Cars Were Great!
Lots of Great Innovations Since 1970
Would you use these
technologies for your
business today?
Including the Relational Database
For which computers the
relational model has been
designed for?
So Were Computers!
And Storage!
RDBMS Makes Development Hard
Relational Database
Object Relational
Mapping
Application
Code XML Config DB Schema
And Even Harder To Iterate
New Table
New Table
New
Column
Name Pet Phone Email
New
Column
3 months later…
RDBMS
From Complexity to Simplicity
MongoDB
{
_id : ObjectId("4c4ba5e5e8aabf3"),
employee_name: "Dunham, Justin",
department : "Marketing",
title : "Product Manager, Web",
report_up: "Neray, Graham",
pay_band: “C",
benefits : [
{ type : "Health",
plan : "PPO Plus" },
{ type : "Dental",
plan : "Standard" }
]
}
MongoDB
The leading NoSQL database
Document
Database
Open-
Source
General
Purpose
7,000,000+
MongoDB Downloads
150,000+
Online Education Registrants
25,000+
MongoDB User Group Members
25,000+
MongoDB DaysAttendees
20,000+
MongoDB Management Service (MMS) Users
Global Community
To provide the best database for how we build and
run apps today
MongoDB Vision
Build
– New and complex data
– Flexible
– New languages
– Faster development
Run
– Big Data scalability
– Real-time
– Commodity hardware
– Cloud
Enterprise Big Data Stack
EDWHadoop
Management&Monitoring
Security&Auditing
RDBMS
CRM, ERP, Collaboration, Mobile, BI
OS & Virtualization, Compute, Storage, Network
RDBMS
Applications
Infrastructure
Data Management
Online Data Offline Data
Agile
MongoDB Overview
Scalable
Operational Database Landscape
Key → Value
• One-dimensional storage
• Single value is a blob
• Query on key only
• No schema
• Value cannot be updated,only replaced
Key Blob
Relational/Wide Column
• Two-dimensional storage (tuples)
• Each field contains a single value
• Query on anyfield
• Very structured schema (table)
• In-place updates
• Normalization process requires many tables, joins,
indexes,and poor data locality
Primary
Key
Document
• N-dimensional storage
• Each field can contain 0,1,
many,or embedded values
• Query on anyfield & level
• Flexible schema
• Inline updates *
• Embedding related data has optimal data locality,
requires fewer indexes,has better performance
_id
Document Data Model
Relational MongoDB
{
first_name: ‘Paul’,
surname: ‘Miller’,
city: ‘London’,
location: [45.123,47.232],
cars: [
{ model: ‘Bentley’,
year: 1973,
value: 100000, … },
{ model: ‘Rolls Royce’,
year: 1965,
value: 330000, … }
]
}
Document Model Benefits
• Agility and flexibility
– Data models can evolve easily
– Companies can adapt to changes quickly
• Intuitive,natural data representation
– Developers are more productive
– Manytypes of applications are a good fit
• Reduces the need for joins,disk seeks
– Programming is more simple
– Performance can be delivered at scale
Developers are more productive
Developers are more productive
Automatic Sharding
• Three types of sharding: hash-based, range-based, tag-
aware!
• Increase or decrease capacity as you go!
• Automatic balancing
Query Routing
• Multiple query optimization models!
• Each sharding option appropriate for different apps!
HighAvailability–Ensure application availabilityduring many
types of failures
!
Disaster Recovery–Address the RTO and RPO goals for business
continuity
!
Maintenance –Perform upgrades and other maintenance
operations with no application downtime
Availability Considerations
Replica Sets
• Replica Set – two or more copies!
• “Self-healing” shard!
• Addresses many concerns:!
- High Availability!
- Disaster Recovery!
- Maintenance
Strong Consistency
Delayed Consistency
Write Concern
• Network acknowledgement
• Wait for error
• Wait for journal sync
• Wait for replication
Unacknowledged
MongoDB Acknowledged (wait for error)
Wait for Journal Sync
Wait for Replication
Tagging
• Control where data is written to,and read from
• Each member can have one or more tags
– tags: {dc:"ny"}
– tags: {dc:"ny",

subnet:"192.168",

rack:"row3rk7"}
• Replica set defines rules for write concerns
• Rules can change without changing app code
{!
_id : "mySet",!
members : [!
{_id : 0, host : "A", tags : {"dc": "ny"}},!
{_id : 1, host : "B", tags : {"dc": "ny"}},!
{_id : 2, host : "C", tags : {"dc": "sf"}},!
{_id : 3, host : "D", tags : {"dc": "sf"}},!
{_id : 4, host : "E", tags : {"dc": "cloud"}}],!
settings : {!
getLastErrorModes : {!
allDCs : {"dc" : 3},!
someDCs : {"dc" : 2}} }!
}!
> db.blogs.insert({...})!
> db.runCommand({getLastError : 1, w : "someDCs"})
Tagging Example
Wait for Replication (Tagging)
Read Preference Modes
• 5 modes
– primary(only)-Default
– primaryPreferred
– secondary
– secondaryPreferred
– Nearest
!
When more than one node is possible,closest node is used for
reads (all modes but primary)
Single Data Center
• Automated failover !
• Tolerates server failures!
• Tolerates rack failures!
• Number of replicas
defines failure tolerance
Primary –A Primary – B Primary – C
Secondary –A Secondary –ASecondary – B
Secondary – BSecondary – CSecondary – C
Active/Standby Data Center
• Tolerates server and rack failure!
• Standby data center
Data Center - West
Primary –A Primary – B Primary – C
Secondary –ASecondary – B Secondary – C
Data Center - East
Secondary –A Secondary – B Secondary – C
Active/Active Data Center
• Tolerates server, rack, data center failures, network
partitions
Data Center - West
Primary –A Primary – B Primary – C
Secondary –A Secondary – BSecondary – C
Data Center - East
Secondary –A Secondary – B Secondary – C
Secondary – B Secondary – C Secondary –A
Data Center - Central
Arbiter –A Arbiter – B Arbiter – C
Global Data Distribution
Real-time
Real-time Real-time
Real-time
Real-time
Real-time
Real-time
Primary
Secondary
Secondary
Secondary
Secondary
Secondary
Secondary
Secondary
Read Global/Write Local
Primary:NYC
Secondary:NYC
Primary:LON
Primary:SYD
Secondary:LON
Secondary:NYC
Secondary:SYD
Secondary:LON
Secondary:SYD
Common Use Cases
High Volume Data Feeds
••More machine forms, sensors & data
••Variably structured
Machine
Generated Data
••High frequency trading
••Daily closing price
Securities Data
••Multiple data sources
••Each changes their format consistently
••Student Scores, ISP logs
Social Media /
General Public
Operational Intelligence
••Large volume of users
••Very strict latency requirements
••Sentiment Analysis
Ad Targeting
••Expose data to millions of customers
••Reports on large volumes of data
••Reports that update in real time
Real time
dashboards
••Join the conversation
••Catered Games
••Customized Surveys
Social Media
Monitoring
Metadata
••Diverse product portfolio
••Complex querying and filtering
••Multi-faceted product attributes
Product
Catalogue
••Data mining
••Call records
••Insurance Claims
Data analysis
••Retina Scans
••Fingerprints
Biometric
Content Management
••Comments and user generated content
••Personalization of content and layout
News Site
••Generate layout on the fly
••No need to cache static pages
Multi-device
rendering
••Store large objects
••Simpler modeling of metadata
Sharing
Questions?
Thanks!
@massimobrignoli
Massimo Brignoli
#MongoDB
SolutionsArchitect,MongoDB Inc.
massimo@mongodb.com

MongoDB: What, why, when

  • 1.
  • 2.
    Who Am I? •Solutions Architect/Evangelist in MongoDB Inc. • 24 years of experience in databases and software development • Former MySQL employee • Previous life: web,web,web
  • 3.
  • 4.
    Understanding Big Data– It’s Not Very“Big” from Big Data Executive Summary – 50+ top executives from Government and F500 firms 64% - Ingest diverse, new data in real-time 15% - More than 100TB of data 20% - Less than 100TB (average of all? <20TB)
  • 5.
    “I have notfailed. I've just found 10,000 ways that won't work.” ― Thomas A. Edison
  • 6.
  • 7.
    Lots of GreatInnovations Since 1970
  • 8.
    Would you usethese technologies for your business today?
  • 9.
  • 10.
    For which computersthe relational model has been designed for?
  • 11.
  • 12.
  • 13.
    RDBMS Makes DevelopmentHard Relational Database Object Relational Mapping Application Code XML Config DB Schema
  • 14.
    And Even HarderTo Iterate New Table New Table New Column Name Pet Phone Email New Column 3 months later…
  • 15.
    RDBMS From Complexity toSimplicity MongoDB { _id : ObjectId("4c4ba5e5e8aabf3"), employee_name: "Dunham, Justin", department : "Marketing", title : "Product Manager, Web", report_up: "Neray, Graham", pay_band: “C", benefits : [ { type : "Health", plan : "PPO Plus" }, { type : "Dental", plan : "Standard" } ] }
  • 16.
    MongoDB The leading NoSQLdatabase Document Database Open- Source General Purpose
  • 17.
    7,000,000+ MongoDB Downloads 150,000+ Online EducationRegistrants 25,000+ MongoDB User Group Members 25,000+ MongoDB DaysAttendees 20,000+ MongoDB Management Service (MMS) Users Global Community
  • 18.
    To provide thebest database for how we build and run apps today MongoDB Vision Build – New and complex data – Flexible – New languages – Faster development Run – Big Data scalability – Real-time – Commodity hardware – Cloud
  • 19.
    Enterprise Big DataStack EDWHadoop Management&Monitoring Security&Auditing RDBMS CRM, ERP, Collaboration, Mobile, BI OS & Virtualization, Compute, Storage, Network RDBMS Applications Infrastructure Data Management Online Data Offline Data
  • 20.
  • 21.
  • 22.
    Key → Value •One-dimensional storage • Single value is a blob • Query on key only • No schema • Value cannot be updated,only replaced Key Blob
  • 23.
    Relational/Wide Column • Two-dimensionalstorage (tuples) • Each field contains a single value • Query on anyfield • Very structured schema (table) • In-place updates • Normalization process requires many tables, joins, indexes,and poor data locality Primary Key
  • 24.
    Document • N-dimensional storage •Each field can contain 0,1, many,or embedded values • Query on anyfield & level • Flexible schema • Inline updates * • Embedding related data has optimal data locality, requires fewer indexes,has better performance _id
  • 25.
    Document Data Model RelationalMongoDB { first_name: ‘Paul’, surname: ‘Miller’, city: ‘London’, location: [45.123,47.232], cars: [ { model: ‘Bentley’, year: 1973, value: 100000, … }, { model: ‘Rolls Royce’, year: 1965, value: 330000, … } ] }
  • 26.
    Document Model Benefits •Agility and flexibility – Data models can evolve easily – Companies can adapt to changes quickly • Intuitive,natural data representation – Developers are more productive – Manytypes of applications are a good fit • Reduces the need for joins,disk seeks – Programming is more simple – Performance can be delivered at scale
  • 27.
  • 28.
  • 29.
    Automatic Sharding • Threetypes of sharding: hash-based, range-based, tag- aware! • Increase or decrease capacity as you go! • Automatic balancing
  • 30.
    Query Routing • Multiplequery optimization models! • Each sharding option appropriate for different apps!
  • 31.
    HighAvailability–Ensure application availabilityduringmany types of failures ! Disaster Recovery–Address the RTO and RPO goals for business continuity ! Maintenance –Perform upgrades and other maintenance operations with no application downtime Availability Considerations
  • 32.
    Replica Sets • ReplicaSet – two or more copies! • “Self-healing” shard! • Addresses many concerns:! - High Availability! - Disaster Recovery! - Maintenance
  • 33.
  • 34.
  • 35.
    Write Concern • Networkacknowledgement • Wait for error • Wait for journal sync • Wait for replication
  • 36.
  • 37.
  • 38.
  • 39.
  • 40.
    Tagging • Control wheredata is written to,and read from • Each member can have one or more tags – tags: {dc:"ny"} – tags: {dc:"ny",
 subnet:"192.168",
 rack:"row3rk7"} • Replica set defines rules for write concerns • Rules can change without changing app code
  • 41.
    {! _id : "mySet",! members: [! {_id : 0, host : "A", tags : {"dc": "ny"}},! {_id : 1, host : "B", tags : {"dc": "ny"}},! {_id : 2, host : "C", tags : {"dc": "sf"}},! {_id : 3, host : "D", tags : {"dc": "sf"}},! {_id : 4, host : "E", tags : {"dc": "cloud"}}],! settings : {! getLastErrorModes : {! allDCs : {"dc" : 3},! someDCs : {"dc" : 2}} }! }! > db.blogs.insert({...})! > db.runCommand({getLastError : 1, w : "someDCs"}) Tagging Example
  • 42.
  • 43.
    Read Preference Modes •5 modes – primary(only)-Default – primaryPreferred – secondary – secondaryPreferred – Nearest ! When more than one node is possible,closest node is used for reads (all modes but primary)
  • 44.
    Single Data Center •Automated failover ! • Tolerates server failures! • Tolerates rack failures! • Number of replicas defines failure tolerance Primary –A Primary – B Primary – C Secondary –A Secondary –ASecondary – B Secondary – BSecondary – CSecondary – C
  • 45.
    Active/Standby Data Center •Tolerates server and rack failure! • Standby data center Data Center - West Primary –A Primary – B Primary – C Secondary –ASecondary – B Secondary – C Data Center - East Secondary –A Secondary – B Secondary – C
  • 46.
    Active/Active Data Center •Tolerates server, rack, data center failures, network partitions Data Center - West Primary –A Primary – B Primary – C Secondary –A Secondary – BSecondary – C Data Center - East Secondary –A Secondary – B Secondary – C Secondary – B Secondary – C Secondary –A Data Center - Central Arbiter –A Arbiter – B Arbiter – C
  • 47.
    Global Data Distribution Real-time Real-timeReal-time Real-time Real-time Real-time Real-time Primary Secondary Secondary Secondary Secondary Secondary Secondary Secondary
  • 48.
  • 49.
  • 50.
    High Volume DataFeeds ••More machine forms, sensors & data ••Variably structured Machine Generated Data ••High frequency trading ••Daily closing price Securities Data ••Multiple data sources ••Each changes their format consistently ••Student Scores, ISP logs Social Media / General Public
  • 51.
    Operational Intelligence ••Large volumeof users ••Very strict latency requirements ••Sentiment Analysis Ad Targeting ••Expose data to millions of customers ••Reports on large volumes of data ••Reports that update in real time Real time dashboards ••Join the conversation ••Catered Games ••Customized Surveys Social Media Monitoring
  • 52.
    Metadata ••Diverse product portfolio ••Complexquerying and filtering ••Multi-faceted product attributes Product Catalogue ••Data mining ••Call records ••Insurance Claims Data analysis ••Retina Scans ••Fingerprints Biometric
  • 53.
    Content Management ••Comments anduser generated content ••Personalization of content and layout News Site ••Generate layout on the fly ••No need to cache static pages Multi-device rendering ••Store large objects ••Simpler modeling of metadata Sharing
  • 54.
  • 55.