SlideShare a Scribd company logo
MongoDB
      at
Community Engine
About me

Lead platform engineer
mathieu.kempe@communityengine.com
@mathieukempe
Agenda

Brain dump on our experience with MongoDB
• Why NoSQL
• Why we chose MongoDB
• Moving away from an hybrid solution
• MongoDB and Amazon
• SOLR and MongoDB
• Ease of development
• Zero downtime database deployment
About Community Engine
• Social network based on locality and
  small business
• Launching in April
• Built on:
    • ASP.NET MVC 3
    • Amazon Web services
    • MongoDB
    • SOLR
    • Mahout
    • …
NoSQL?

• How to store the big amounts of data required
  in social networking applications
• Data complexity, NoSQL handle hierarchical
  and graph data structures better
• Change management is always difficult with
  RDBMS
• Scaling
Why we chose MongoDB?

• Reviewed different products RavenDB, CouchDB...
• Selected MongoDB because we had the best
  experience
   – Very easy to install and get started
   – Great developer experience
   – Replication very easy to setup
   – Good documentation
   – Much of the convenience of SQL, Dynamic
     Queries, Indexing
Why Hybrid?

• Team had a lot of experience with SQL Server
  and Entity Framework
• Reporting
• Transaction
No more SQL Server
• Simplify our infrastructure
• Easier to Backup
• Better performance, not slowed down by SQL
  Server, too many queries joined in the
  application
• Development speed
• Lower cost
Transaction?

Transactions we could go around that using the atomic
  document updates and a good schema design

MongoDB supports atomic operations on single
 document.

When transactions across documents are needed
Two phase commits
Hosting with Amazon Web Service

• Elasticity and scalability
• Configure MongoDB using Amazon EC2 instance
  bundled into an AMI.
• 64 bits EC2 instance
• Raid10 + EBS volumes
• Multi-datacenter 3-node replica set in different
  availability zone
• Use secondaries for zero downtime backup
• We are not yet using sharded replica sets
What about durability?


Use journaling
Use replica sets
Critical writes

Verify that replication is working at write time

mongodb://host1,host2,host3/?safe=true;w=2;

•safe=true : Use safemode
•w=2: wmode, connect to a replica set waiting for
replication to succeed on the majority of nodes
Why we kept SOLR?

•   Right tool for the right job
•   Proven technology
•   SOLR best solution for Full Text Indexing
•   Faceted search, Spelling suggestions…
•   Team already skilled with SOLR
•   SOLR scales well
MongoDB/SOLR How we do it
MongoDB/SOLR How we do it
MongoDB/SOLR How we do it
MongoDB/SOLR How we do it
MongoDB/SOLR How we do it
Ease of development
Hierarchical data in SQL Server
Single table
Mongo Database Schema

Using Type discriminator

{
     "_id" : ObjectId("4f504e7acd3e1c190ce04198"),
     "_t" : "PhotoSpark",
     "Photo" : "MyMotorcycle.png "
    "DateCreated" : ISODate("2012-02-24T09:23:12.246Z")
}
{
     "_id" : ObjectId("4f504e7ccd3e1c190ce04199"),
     "_t" : "PostSpark",
     "Body" : "Hello World“
    " DateCreated" : ISODate("2012-02-28T10:44:12.858Z")
}
Views
                      


                                         




                                         




                
Deployment of database with zero downtime

• We release every week
• We aim at zero downtime
• Our domain model change often
Deployment of database with zero downtime

Make sure that our code can handle both
 "versions" of the data structure
When saving we updates to the new structure
Deployment of database with zero downtime

• Use a migration script
Deployment of database with zero downtime

  Expansion Script   Deploy new version   Compression Script




                                                               t
Questions?
Thank you!

mathieu.kempe@communityengine.com
@mathieukempe

More Related Content

What's hot

Docker y azure container service
Docker y azure container serviceDocker y azure container service
Docker y azure container service
Fernando Mejía
 
Know thy cost (or where performance problems lurk)
Know thy cost (or where performance problems lurk)Know thy cost (or where performance problems lurk)
Know thy cost (or where performance problems lurk)
Oren Eini
 
Architecting on The Cloud
Architecting on The CloudArchitecting on The Cloud
Architecting on The Cloud
Amazon Web Services
 
Application of nodejs in epsilon mobile
Application of nodejs in epsilon mobileApplication of nodejs in epsilon mobile
Application of nodejs in epsilon mobileTony Vo
 
Cloudsolutionday 2016: Docker & FAAS at getvero.com
Cloudsolutionday 2016: Docker & FAAS at getvero.comCloudsolutionday 2016: Docker & FAAS at getvero.com
Cloudsolutionday 2016: Docker & FAAS at getvero.com
AWS Vietnam Community
 
Containerization - The DevOps Revolution
Containerization - The DevOps RevolutionContainerization - The DevOps Revolution
Containerization - The DevOps RevolutionYulian Slobodyan
 
Aws 12 Month Free Tier for Web Designers and Developers
Aws 12 Month Free Tier for Web Designers and DevelopersAws 12 Month Free Tier for Web Designers and Developers
Aws 12 Month Free Tier for Web Designers and Developers
Dylan Burris
 
Cnam cours azure web sites
Cnam cours azure web sitesCnam cours azure web sites
Cnam cours azure web sites
Aymeric Weinbach
 
Running MongoDB in the Cloud
Running MongoDB in the CloudRunning MongoDB in the Cloud
Running MongoDB in the Cloud
Tony Tam
 
Building Codealike: a journey into the developers analytics world
Building Codealike: a journey into the developers analytics worldBuilding Codealike: a journey into the developers analytics world
Building Codealike: a journey into the developers analytics world
Oren Eini
 
How Microsoft learned to love Java
How Microsoft learned to love JavaHow Microsoft learned to love Java
How Microsoft learned to love Java
Brian Benz
 
Optimizing Your WordPress Site
Optimizing Your WordPress SiteOptimizing Your WordPress Site
Optimizing Your WordPress Site
ozzyr
 
Антон Бойко "Разделяй и властвуй — набор практик для построения масштабируемо...
Антон Бойко "Разделяй и властвуй — набор практик для построения масштабируемо...Антон Бойко "Разделяй и властвуй — набор практик для построения масштабируемо...
Антон Бойко "Разделяй и властвуй — набор практик для построения масштабируемо...
Marina Peregud
 
How and when to use NoSQL
How and when to use NoSQLHow and when to use NoSQL
How and when to use NoSQL
Amazon Web Services
 
Customer Presentaion-LittleBigPlanet: Taking an Idea to Market Using AWS
Customer Presentaion-LittleBigPlanet: Taking an Idea to Market Using AWSCustomer Presentaion-LittleBigPlanet: Taking an Idea to Market Using AWS
Customer Presentaion-LittleBigPlanet: Taking an Idea to Market Using AWS
Amazon Web Services
 
Matt Chung (Independent) - Serverless application with AWS Lambda
Matt Chung (Independent) - Serverless application with AWS Lambda Matt Chung (Independent) - Serverless application with AWS Lambda
Matt Chung (Independent) - Serverless application with AWS Lambda
Outlyer
 
How to Build High Performance : WordPress
How to Build High Performance : WordPressHow to Build High Performance : WordPress
How to Build High Performance : WordPress
Dylan Burris
 
Visualization of RDS metrics using AWS CLI and JQuery at AWS Usergroup Leipzig
Visualization of RDS metrics using AWS CLI and JQuery at AWS Usergroup LeipzigVisualization of RDS metrics using AWS CLI and JQuery at AWS Usergroup Leipzig
Visualization of RDS metrics using AWS CLI and JQuery at AWS Usergroup Leipzig
root360 GmbH
 
Azure virtual machine-network
Azure virtual machine-networkAzure virtual machine-network
Azure virtual machine-network
Thi Nguyen Dinh
 

What's hot (19)

Docker y azure container service
Docker y azure container serviceDocker y azure container service
Docker y azure container service
 
Know thy cost (or where performance problems lurk)
Know thy cost (or where performance problems lurk)Know thy cost (or where performance problems lurk)
Know thy cost (or where performance problems lurk)
 
Architecting on The Cloud
Architecting on The CloudArchitecting on The Cloud
Architecting on The Cloud
 
Application of nodejs in epsilon mobile
Application of nodejs in epsilon mobileApplication of nodejs in epsilon mobile
Application of nodejs in epsilon mobile
 
Cloudsolutionday 2016: Docker & FAAS at getvero.com
Cloudsolutionday 2016: Docker & FAAS at getvero.comCloudsolutionday 2016: Docker & FAAS at getvero.com
Cloudsolutionday 2016: Docker & FAAS at getvero.com
 
Containerization - The DevOps Revolution
Containerization - The DevOps RevolutionContainerization - The DevOps Revolution
Containerization - The DevOps Revolution
 
Aws 12 Month Free Tier for Web Designers and Developers
Aws 12 Month Free Tier for Web Designers and DevelopersAws 12 Month Free Tier for Web Designers and Developers
Aws 12 Month Free Tier for Web Designers and Developers
 
Cnam cours azure web sites
Cnam cours azure web sitesCnam cours azure web sites
Cnam cours azure web sites
 
Running MongoDB in the Cloud
Running MongoDB in the CloudRunning MongoDB in the Cloud
Running MongoDB in the Cloud
 
Building Codealike: a journey into the developers analytics world
Building Codealike: a journey into the developers analytics worldBuilding Codealike: a journey into the developers analytics world
Building Codealike: a journey into the developers analytics world
 
How Microsoft learned to love Java
How Microsoft learned to love JavaHow Microsoft learned to love Java
How Microsoft learned to love Java
 
Optimizing Your WordPress Site
Optimizing Your WordPress SiteOptimizing Your WordPress Site
Optimizing Your WordPress Site
 
Антон Бойко "Разделяй и властвуй — набор практик для построения масштабируемо...
Антон Бойко "Разделяй и властвуй — набор практик для построения масштабируемо...Антон Бойко "Разделяй и властвуй — набор практик для построения масштабируемо...
Антон Бойко "Разделяй и властвуй — набор практик для построения масштабируемо...
 
How and when to use NoSQL
How and when to use NoSQLHow and when to use NoSQL
How and when to use NoSQL
 
Customer Presentaion-LittleBigPlanet: Taking an Idea to Market Using AWS
Customer Presentaion-LittleBigPlanet: Taking an Idea to Market Using AWSCustomer Presentaion-LittleBigPlanet: Taking an Idea to Market Using AWS
Customer Presentaion-LittleBigPlanet: Taking an Idea to Market Using AWS
 
Matt Chung (Independent) - Serverless application with AWS Lambda
Matt Chung (Independent) - Serverless application with AWS Lambda Matt Chung (Independent) - Serverless application with AWS Lambda
Matt Chung (Independent) - Serverless application with AWS Lambda
 
How to Build High Performance : WordPress
How to Build High Performance : WordPressHow to Build High Performance : WordPress
How to Build High Performance : WordPress
 
Visualization of RDS metrics using AWS CLI and JQuery at AWS Usergroup Leipzig
Visualization of RDS metrics using AWS CLI and JQuery at AWS Usergroup LeipzigVisualization of RDS metrics using AWS CLI and JQuery at AWS Usergroup Leipzig
Visualization of RDS metrics using AWS CLI and JQuery at AWS Usergroup Leipzig
 
Azure virtual machine-network
Azure virtual machine-networkAzure virtual machine-network
Azure virtual machine-network
 

Viewers also liked

ownCloud, tu nube libre
ownCloud, tu nube libreownCloud, tu nube libre
ownCloud, tu nube libre
Jorge Cacho
 
Entrevista a Jono Bacon
Entrevista a Jono BaconEntrevista a Jono Bacon
Entrevista a Jono Bacon
Jorge Cacho
 
Smarttop: un servidor web (y mucho más) de bajo coste
Smarttop: un servidor web (y mucho más) de bajo costeSmarttop: un servidor web (y mucho más) de bajo coste
Smarttop: un servidor web (y mucho más) de bajo coste
Jorge Cacho
 
Fortifica tu Wordpress con Latch
Fortifica tu Wordpress con LatchFortifica tu Wordpress con Latch
Fortifica tu Wordpress con Latch
Jorge Cacho
 
Raspberry Pi: tutoriales (servidor web, ownCloud y XBMC)
Raspberry Pi: tutoriales (servidor web, ownCloud y XBMC)Raspberry Pi: tutoriales (servidor web, ownCloud y XBMC)
Raspberry Pi: tutoriales (servidor web, ownCloud y XBMC)
Jorge Cacho
 
Raspberry Pi: un ordenador por $35
Raspberry Pi: un ordenador por $35Raspberry Pi: un ordenador por $35
Raspberry Pi: un ordenador por $35
Jorge Cacho
 
Tiny Tiny RSS(bye bye Google Reader)
Tiny Tiny RSS(bye bye Google Reader)Tiny Tiny RSS(bye bye Google Reader)
Tiny Tiny RSS(bye bye Google Reader)
Jorge Cacho
 
Intro to the MongoDB Community
Intro to the MongoDB CommunityIntro to the MongoDB Community
Intro to the MongoDB Community
francescapasha
 
The role of the community manager in your organisation
The role of the community manager in your organisationThe role of the community manager in your organisation
The role of the community manager in your organisationBlaise Grimes-Viort
 

Viewers also liked (9)

ownCloud, tu nube libre
ownCloud, tu nube libreownCloud, tu nube libre
ownCloud, tu nube libre
 
Entrevista a Jono Bacon
Entrevista a Jono BaconEntrevista a Jono Bacon
Entrevista a Jono Bacon
 
Smarttop: un servidor web (y mucho más) de bajo coste
Smarttop: un servidor web (y mucho más) de bajo costeSmarttop: un servidor web (y mucho más) de bajo coste
Smarttop: un servidor web (y mucho más) de bajo coste
 
Fortifica tu Wordpress con Latch
Fortifica tu Wordpress con LatchFortifica tu Wordpress con Latch
Fortifica tu Wordpress con Latch
 
Raspberry Pi: tutoriales (servidor web, ownCloud y XBMC)
Raspberry Pi: tutoriales (servidor web, ownCloud y XBMC)Raspberry Pi: tutoriales (servidor web, ownCloud y XBMC)
Raspberry Pi: tutoriales (servidor web, ownCloud y XBMC)
 
Raspberry Pi: un ordenador por $35
Raspberry Pi: un ordenador por $35Raspberry Pi: un ordenador por $35
Raspberry Pi: un ordenador por $35
 
Tiny Tiny RSS(bye bye Google Reader)
Tiny Tiny RSS(bye bye Google Reader)Tiny Tiny RSS(bye bye Google Reader)
Tiny Tiny RSS(bye bye Google Reader)
 
Intro to the MongoDB Community
Intro to the MongoDB CommunityIntro to the MongoDB Community
Intro to the MongoDB Community
 
The role of the community manager in your organisation
The role of the community manager in your organisationThe role of the community manager in your organisation
The role of the community manager in your organisation
 

Similar to MongoDB at community engine

MongoDB.pptx
MongoDB.pptxMongoDB.pptx
MongoDB.pptx
Sigit52
 
MongoDB World 2019: Using the MongoDB Enterprise Kubernetes Operator to Scale...
MongoDB World 2019: Using the MongoDB Enterprise Kubernetes Operator to Scale...MongoDB World 2019: Using the MongoDB Enterprise Kubernetes Operator to Scale...
MongoDB World 2019: Using the MongoDB Enterprise Kubernetes Operator to Scale...
MongoDB
 
Managing Your Cloud Assets
Managing Your Cloud AssetsManaging Your Cloud Assets
Managing Your Cloud Assets
Amazon Web Services
 
MongoDB MUG Delhi NCR - December 19 2020 (Cloud Security)
MongoDB MUG Delhi NCR - December 19 2020 (Cloud Security)MongoDB MUG Delhi NCR - December 19 2020 (Cloud Security)
MongoDB MUG Delhi NCR - December 19 2020 (Cloud Security)
Shrey Batra
 
Building FoundationDB
Building FoundationDBBuilding FoundationDB
Building FoundationDB
FoundationDB
 
Aws-What You Need to Know_Simon Elisha
Aws-What You Need to Know_Simon ElishaAws-What You Need to Know_Simon Elisha
Aws-What You Need to Know_Simon Elisha
Helen Rogers
 
The Perils and Triumphs of using Cassandra at a .NET/Microsoft Shop
The Perils and Triumphs of using Cassandra at a .NET/Microsoft ShopThe Perils and Triumphs of using Cassandra at a .NET/Microsoft Shop
The Perils and Triumphs of using Cassandra at a .NET/Microsoft Shop
Jeff Smoley
 
Meanstack overview
Meanstack overviewMeanstack overview
Meanstack overview
Adthasid Sabmake
 
Devops continuousintegration and deployment onaws puttingmoneybackintoyourmis...
Devops continuousintegration and deployment onaws puttingmoneybackintoyourmis...Devops continuousintegration and deployment onaws puttingmoneybackintoyourmis...
Devops continuousintegration and deployment onaws puttingmoneybackintoyourmis...
Emerson Eduardo Rodrigues Von Staffen
 
DevOps, Continuous Integration and Deployment on AWS: Putting Money Back into...
DevOps, Continuous Integration and Deployment on AWS: Putting Money Back into...DevOps, Continuous Integration and Deployment on AWS: Putting Money Back into...
DevOps, Continuous Integration and Deployment on AWS: Putting Money Back into...
Amazon Web Services
 
Windows Azure introduction
Windows Azure introductionWindows Azure introduction
Windows Azure introduction
Microsoft Iceland
 
Introduction to MERN Stack
Introduction to MERN StackIntroduction to MERN Stack
Introduction to MERN Stack
Surya937648
 
web development with mern stack in power point
web development with mern stack in power pointweb development with mern stack in power point
web development with mern stack in power point
RAMKUMARRIT20
 
Final year presentation topicssssss in 1
Final year presentation topicssssss in 1Final year presentation topicssssss in 1
Final year presentation topicssssss in 1
RAMKUMARRIT20
 
Scaling with mongo db (with notes)
Scaling with mongo db (with notes)Scaling with mongo db (with notes)
Scaling with mongo db (with notes)
emiltamas
 
Mongo db first steps with csharp
Mongo db first steps with csharpMongo db first steps with csharp
Mongo db first steps with csharpSerdar Buyuktemiz
 
A Case Study of NoSQL Adoption: What Drove Wordnik Non-Relational?
A Case Study of NoSQL Adoption: What Drove Wordnik Non-Relational?A Case Study of NoSQL Adoption: What Drove Wordnik Non-Relational?
A Case Study of NoSQL Adoption: What Drove Wordnik Non-Relational?DATAVERSITY
 
An introduction to Node.js
An introduction to Node.jsAn introduction to Node.js
An introduction to Node.js
Kasey McCurdy
 

Similar to MongoDB at community engine (20)

MongoDB.pptx
MongoDB.pptxMongoDB.pptx
MongoDB.pptx
 
MongoDB World 2019: Using the MongoDB Enterprise Kubernetes Operator to Scale...
MongoDB World 2019: Using the MongoDB Enterprise Kubernetes Operator to Scale...MongoDB World 2019: Using the MongoDB Enterprise Kubernetes Operator to Scale...
MongoDB World 2019: Using the MongoDB Enterprise Kubernetes Operator to Scale...
 
Managing Your Cloud Assets
Managing Your Cloud AssetsManaging Your Cloud Assets
Managing Your Cloud Assets
 
MongoDB MUG Delhi NCR - December 19 2020 (Cloud Security)
MongoDB MUG Delhi NCR - December 19 2020 (Cloud Security)MongoDB MUG Delhi NCR - December 19 2020 (Cloud Security)
MongoDB MUG Delhi NCR - December 19 2020 (Cloud Security)
 
Building FoundationDB
Building FoundationDBBuilding FoundationDB
Building FoundationDB
 
Microservices
MicroservicesMicroservices
Microservices
 
Aws-What You Need to Know_Simon Elisha
Aws-What You Need to Know_Simon ElishaAws-What You Need to Know_Simon Elisha
Aws-What You Need to Know_Simon Elisha
 
The Perils and Triumphs of using Cassandra at a .NET/Microsoft Shop
The Perils and Triumphs of using Cassandra at a .NET/Microsoft ShopThe Perils and Triumphs of using Cassandra at a .NET/Microsoft Shop
The Perils and Triumphs of using Cassandra at a .NET/Microsoft Shop
 
20120306 dublin js
20120306 dublin js20120306 dublin js
20120306 dublin js
 
Meanstack overview
Meanstack overviewMeanstack overview
Meanstack overview
 
Devops continuousintegration and deployment onaws puttingmoneybackintoyourmis...
Devops continuousintegration and deployment onaws puttingmoneybackintoyourmis...Devops continuousintegration and deployment onaws puttingmoneybackintoyourmis...
Devops continuousintegration and deployment onaws puttingmoneybackintoyourmis...
 
DevOps, Continuous Integration and Deployment on AWS: Putting Money Back into...
DevOps, Continuous Integration and Deployment on AWS: Putting Money Back into...DevOps, Continuous Integration and Deployment on AWS: Putting Money Back into...
DevOps, Continuous Integration and Deployment on AWS: Putting Money Back into...
 
Windows Azure introduction
Windows Azure introductionWindows Azure introduction
Windows Azure introduction
 
Introduction to MERN Stack
Introduction to MERN StackIntroduction to MERN Stack
Introduction to MERN Stack
 
web development with mern stack in power point
web development with mern stack in power pointweb development with mern stack in power point
web development with mern stack in power point
 
Final year presentation topicssssss in 1
Final year presentation topicssssss in 1Final year presentation topicssssss in 1
Final year presentation topicssssss in 1
 
Scaling with mongo db (with notes)
Scaling with mongo db (with notes)Scaling with mongo db (with notes)
Scaling with mongo db (with notes)
 
Mongo db first steps with csharp
Mongo db first steps with csharpMongo db first steps with csharp
Mongo db first steps with csharp
 
A Case Study of NoSQL Adoption: What Drove Wordnik Non-Relational?
A Case Study of NoSQL Adoption: What Drove Wordnik Non-Relational?A Case Study of NoSQL Adoption: What Drove Wordnik Non-Relational?
A Case Study of NoSQL Adoption: What Drove Wordnik Non-Relational?
 
An introduction to Node.js
An introduction to Node.jsAn introduction to Node.js
An introduction to Node.js
 

Recently uploaded

Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...
Product School
 
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Ramesh Iyer
 
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
Sri Ambati
 
"Impact of front-end architecture on development cost", Viktor Turskyi
"Impact of front-end architecture on development cost", Viktor Turskyi"Impact of front-end architecture on development cost", Viktor Turskyi
"Impact of front-end architecture on development cost", Viktor Turskyi
Fwdays
 
UiPath Test Automation using UiPath Test Suite series, part 4
UiPath Test Automation using UiPath Test Suite series, part 4UiPath Test Automation using UiPath Test Suite series, part 4
UiPath Test Automation using UiPath Test Suite series, part 4
DianaGray10
 
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
Product School
 
Leading Change strategies and insights for effective change management pdf 1.pdf
Leading Change strategies and insights for effective change management pdf 1.pdfLeading Change strategies and insights for effective change management pdf 1.pdf
Leading Change strategies and insights for effective change management pdf 1.pdf
OnBoard
 
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
UiPathCommunity
 
Knowledge engineering: from people to machines and back
Knowledge engineering: from people to machines and backKnowledge engineering: from people to machines and back
Knowledge engineering: from people to machines and back
Elena Simperl
 
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
James Anderson
 
Connector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a buttonConnector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a button
DianaGray10
 
DevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA ConnectDevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA Connect
Kari Kakkonen
 
FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf
FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdfFIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf
FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf
FIDO Alliance
 
Bits & Pixels using AI for Good.........
Bits & Pixels using AI for Good.........Bits & Pixels using AI for Good.........
Bits & Pixels using AI for Good.........
Alison B. Lowndes
 
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdfFIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance
 
Designing Great Products: The Power of Design and Leadership by Chief Designe...
Designing Great Products: The Power of Design and Leadership by Chief Designe...Designing Great Products: The Power of Design and Leadership by Chief Designe...
Designing Great Products: The Power of Design and Leadership by Chief Designe...
Product School
 
UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3
DianaGray10
 
PHP Frameworks: I want to break free (IPC Berlin 2024)
PHP Frameworks: I want to break free (IPC Berlin 2024)PHP Frameworks: I want to break free (IPC Berlin 2024)
PHP Frameworks: I want to break free (IPC Berlin 2024)
Ralf Eggert
 
Epistemic Interaction - tuning interfaces to provide information for AI support
Epistemic Interaction - tuning interfaces to provide information for AI supportEpistemic Interaction - tuning interfaces to provide information for AI support
Epistemic Interaction - tuning interfaces to provide information for AI support
Alan Dix
 
Accelerate your Kubernetes clusters with Varnish Caching
Accelerate your Kubernetes clusters with Varnish CachingAccelerate your Kubernetes clusters with Varnish Caching
Accelerate your Kubernetes clusters with Varnish Caching
Thijs Feryn
 

Recently uploaded (20)

Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...
 
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
 
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
 
"Impact of front-end architecture on development cost", Viktor Turskyi
"Impact of front-end architecture on development cost", Viktor Turskyi"Impact of front-end architecture on development cost", Viktor Turskyi
"Impact of front-end architecture on development cost", Viktor Turskyi
 
UiPath Test Automation using UiPath Test Suite series, part 4
UiPath Test Automation using UiPath Test Suite series, part 4UiPath Test Automation using UiPath Test Suite series, part 4
UiPath Test Automation using UiPath Test Suite series, part 4
 
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
 
Leading Change strategies and insights for effective change management pdf 1.pdf
Leading Change strategies and insights for effective change management pdf 1.pdfLeading Change strategies and insights for effective change management pdf 1.pdf
Leading Change strategies and insights for effective change management pdf 1.pdf
 
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
 
Knowledge engineering: from people to machines and back
Knowledge engineering: from people to machines and backKnowledge engineering: from people to machines and back
Knowledge engineering: from people to machines and back
 
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
 
Connector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a buttonConnector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a button
 
DevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA ConnectDevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA Connect
 
FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf
FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdfFIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf
FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf
 
Bits & Pixels using AI for Good.........
Bits & Pixels using AI for Good.........Bits & Pixels using AI for Good.........
Bits & Pixels using AI for Good.........
 
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdfFIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
 
Designing Great Products: The Power of Design and Leadership by Chief Designe...
Designing Great Products: The Power of Design and Leadership by Chief Designe...Designing Great Products: The Power of Design and Leadership by Chief Designe...
Designing Great Products: The Power of Design and Leadership by Chief Designe...
 
UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3
 
PHP Frameworks: I want to break free (IPC Berlin 2024)
PHP Frameworks: I want to break free (IPC Berlin 2024)PHP Frameworks: I want to break free (IPC Berlin 2024)
PHP Frameworks: I want to break free (IPC Berlin 2024)
 
Epistemic Interaction - tuning interfaces to provide information for AI support
Epistemic Interaction - tuning interfaces to provide information for AI supportEpistemic Interaction - tuning interfaces to provide information for AI support
Epistemic Interaction - tuning interfaces to provide information for AI support
 
Accelerate your Kubernetes clusters with Varnish Caching
Accelerate your Kubernetes clusters with Varnish CachingAccelerate your Kubernetes clusters with Varnish Caching
Accelerate your Kubernetes clusters with Varnish Caching
 

MongoDB at community engine

  • 1. MongoDB at Community Engine
  • 2. About me Lead platform engineer mathieu.kempe@communityengine.com @mathieukempe
  • 3. Agenda Brain dump on our experience with MongoDB • Why NoSQL • Why we chose MongoDB • Moving away from an hybrid solution • MongoDB and Amazon • SOLR and MongoDB • Ease of development • Zero downtime database deployment
  • 4. About Community Engine • Social network based on locality and small business • Launching in April • Built on: • ASP.NET MVC 3 • Amazon Web services • MongoDB • SOLR • Mahout • …
  • 5. NoSQL? • How to store the big amounts of data required in social networking applications • Data complexity, NoSQL handle hierarchical and graph data structures better • Change management is always difficult with RDBMS • Scaling
  • 6. Why we chose MongoDB? • Reviewed different products RavenDB, CouchDB... • Selected MongoDB because we had the best experience – Very easy to install and get started – Great developer experience – Replication very easy to setup – Good documentation – Much of the convenience of SQL, Dynamic Queries, Indexing
  • 7.
  • 8. Why Hybrid? • Team had a lot of experience with SQL Server and Entity Framework • Reporting • Transaction
  • 9.
  • 10. No more SQL Server • Simplify our infrastructure • Easier to Backup • Better performance, not slowed down by SQL Server, too many queries joined in the application • Development speed • Lower cost
  • 11. Transaction? Transactions we could go around that using the atomic document updates and a good schema design MongoDB supports atomic operations on single document. When transactions across documents are needed Two phase commits
  • 12. Hosting with Amazon Web Service • Elasticity and scalability • Configure MongoDB using Amazon EC2 instance bundled into an AMI. • 64 bits EC2 instance • Raid10 + EBS volumes • Multi-datacenter 3-node replica set in different availability zone • Use secondaries for zero downtime backup • We are not yet using sharded replica sets
  • 13.
  • 14. What about durability? Use journaling Use replica sets
  • 15. Critical writes Verify that replication is working at write time mongodb://host1,host2,host3/?safe=true;w=2; •safe=true : Use safemode •w=2: wmode, connect to a replica set waiting for replication to succeed on the majority of nodes
  • 16.
  • 17.
  • 18.
  • 19.
  • 20.
  • 21. Why we kept SOLR? • Right tool for the right job • Proven technology • SOLR best solution for Full Text Indexing • Faceted search, Spelling suggestions… • Team already skilled with SOLR • SOLR scales well
  • 28. Hierarchical data in SQL Server
  • 30. Mongo Database Schema Using Type discriminator { "_id" : ObjectId("4f504e7acd3e1c190ce04198"), "_t" : "PhotoSpark", "Photo" : "MyMotorcycle.png " "DateCreated" : ISODate("2012-02-24T09:23:12.246Z") } { "_id" : ObjectId("4f504e7ccd3e1c190ce04199"), "_t" : "PostSpark", "Body" : "Hello World“ " DateCreated" : ISODate("2012-02-28T10:44:12.858Z") }
  • 31.
  • 32.
  • 33.
  • 34. Views                                                                    
  • 35. Deployment of database with zero downtime • We release every week • We aim at zero downtime • Our domain model change often
  • 36. Deployment of database with zero downtime Make sure that our code can handle both "versions" of the data structure When saving we updates to the new structure
  • 37. Deployment of database with zero downtime • Use a migration script
  • 38. Deployment of database with zero downtime Expansion Script Deploy new version Compression Script t

Editor's Notes

  1. Brain dump on why we started to use MongoDB Why we moved away from a solution using MongoDB and SQL Server
  2. Moved to this infra
  3. If the primary crash we have the data in at least one of the secondary Second node has priority greater than or equal to other eligible nodes in the set
  4. Update is different as we do sometime Update in place Update in place are a query and an update in one command