SlideShare a Scribd company logo
Geo & capped collections
    with MongoDB:
 Use cases & performance
     in the real world
/usr/bin/whoami
  Russell Smith

  Consultant for UKD1 Limited

  Specialising in helping companies going through rapid growth

  Help with code, architecture, infrastructure, devops, sysops,
  capacity planning, etc

  <3 gearman, mongodb, neo4j, mysql, kohana, riaksearch, php,
  debian
What are capped collections?
  Special option for collections which allow you to;

  Limit the size of a collection

  Limit the number of documents in a collection
Why is this useful?
  Logging

  News feeds

  Caching
Restrictions
  Can’t delete objects (you can drop the collection though)

  Can’t update objects (well...not 100% true - you can only if the size
  doesn’t change)
Neat things
  Objects are returned in the insertion order

  Don’t have to worry about things growing out of control

  No more cronjobs to delete old data
Real world example
  SongFor.com:
  Activity feeds for the front of the site are using capped collections

  StrongHold Kingdoms (MMO):
  Logging of events from syslog / internal events are stored in a
  capped collection (Graylog2 & a custom dashboard system)

  Lioness: cache + log of data, used to cut lag & reduce API access
  requests to Facebook

  Findlunch.in: Caching of page data
How to setup...
  db.createCollection("mycoll", {capped:true, size:100000})

  db.createCollection("mycoll", {capped:true, size:100000, max:100});
  (limit to 100 documents - this is slower as it adds overhead)
What is Geospatial indexing?
  Allows easily finding things close to a point - great for location based
  services

  Insert a document as normal, but with some co-ordinates...

  ensureIndex({location_field: ‘2d’});
Other magic things
  You can search $within a $box, or around $center point
  (circle)...which is handy

  Results are sorted by distance from points

  geoNear (command) will return distance, average distance, etc
Versions & support
  > v1.3.3 for initial version, which is pretty old

  >= 1.7.0 added spherical support (more in a minute)

  < 1.7.2 doesn’t support geo in sharded collections. It still has issues
  see: http://jira.mongodb.org/browse/SERVER-926
and the point is?
  Really really easy to find documents around a specific point

  You can get distances from that point as well

  Lookups are fast and accurate

  This is a pain to do in MySQL
Example
  FindLunch.in lets people find lunch close to where they are.

  Geocode lookup of ~2 million postcodes from the UK

  All London tube stations & most boroughs are also listed

  Average query time Xms over 20k of test documents
Questions?
  Any questions?

More Related Content

What's hot

Database growth
Database growthDatabase growth
Database growth
Akash Pramanik
 
Mission impossible
Mission impossibleMission impossible
Mission impossible
Samantha Billington
 
RDO hangout on gnocchi
RDO hangout on gnocchiRDO hangout on gnocchi
RDO hangout on gnocchi
Eoghan Glynn
 
Google Cloud Storage backup and archive
Google Cloud Storage backup and archiveGoogle Cloud Storage backup and archive
Google Cloud Storage backup and archive
Ido Green
 
21st Athens Big Data Meetup - 1st Talk - Fast and simple data exploration wit...
21st Athens Big Data Meetup - 1st Talk - Fast and simple data exploration wit...21st Athens Big Data Meetup - 1st Talk - Fast and simple data exploration wit...
21st Athens Big Data Meetup - 1st Talk - Fast and simple data exploration wit...
Athens Big Data
 
MongoDB + GeoServer
MongoDB + GeoServerMongoDB + GeoServer
MongoDB + GeoServer
MongoDB
 
Bandwidth Generation Lab
Bandwidth Generation LabBandwidth Generation Lab
Bandwidth Generation Lab
Mike Higgins
 
Bucket your partitions wisely - Cassandra summit 2016
Bucket your partitions wisely - Cassandra summit 2016Bucket your partitions wisely - Cassandra summit 2016
Bucket your partitions wisely - Cassandra summit 2016
Markus Höfer
 
How to Reduce Your Database Total Cost of Ownership with TimescaleDB
How to Reduce Your Database Total Cost of Ownership with TimescaleDBHow to Reduce Your Database Total Cost of Ownership with TimescaleDB
How to Reduce Your Database Total Cost of Ownership with TimescaleDB
Timescale
 
Mongo sharding
Mongo shardingMongo sharding
Mongo sharding
Nik Kul
 
How to calculate the optimal undo retention in Oracle
How to calculate the optimal undo retention in OracleHow to calculate the optimal undo retention in Oracle
How to calculate the optimal undo retention in Oracle
Jorge Batista
 
MongoDB GeoSpatial Feature
MongoDB GeoSpatial FeatureMongoDB GeoSpatial Feature
MongoDB GeoSpatial Feature
Hüseyin BABAL
 
Mongodb
MongodbMongodb
Exploring the Enron Email Dataset with Kiji and Hive
Exploring the Enron Email Dataset with Kiji and HiveExploring the Enron Email Dataset with Kiji and Hive
Exploring the Enron Email Dataset with Kiji and Hive
WibiData
 
Mongodb
MongodbMongodb
Mongodb
Manav Prasad
 
Clickhouse Capacity Planning for OLAP Workloads, Mik Kocikowski of CloudFlare
Clickhouse Capacity Planning for OLAP Workloads, Mik Kocikowski of CloudFlareClickhouse Capacity Planning for OLAP Workloads, Mik Kocikowski of CloudFlare
Clickhouse Capacity Planning for OLAP Workloads, Mik Kocikowski of CloudFlare
Altinity Ltd
 
Cache'owanie danych w PHP
Cache'owanie danych w PHPCache'owanie danych w PHP
Cache'owanie danych w PHP
Patryk Jar
 
2012 apache hadoop_map_reduce_windows_azure
2012 apache hadoop_map_reduce_windows_azure2012 apache hadoop_map_reduce_windows_azure
2012 apache hadoop_map_reduce_windows_azure
DataPlato, Crossing the line
 
Geospatial and MongoDB
Geospatial and MongoDBGeospatial and MongoDB
Geospatial and MongoDB
Norberto Leite
 
Александр Терещук - Memory Analyzer Tool and memory optimization tips in Android
Александр Терещук - Memory Analyzer Tool and memory optimization tips in AndroidАлександр Терещук - Memory Analyzer Tool and memory optimization tips in Android
Александр Терещук - Memory Analyzer Tool and memory optimization tips in Android
UA Mobile
 

What's hot (20)

Database growth
Database growthDatabase growth
Database growth
 
Mission impossible
Mission impossibleMission impossible
Mission impossible
 
RDO hangout on gnocchi
RDO hangout on gnocchiRDO hangout on gnocchi
RDO hangout on gnocchi
 
Google Cloud Storage backup and archive
Google Cloud Storage backup and archiveGoogle Cloud Storage backup and archive
Google Cloud Storage backup and archive
 
21st Athens Big Data Meetup - 1st Talk - Fast and simple data exploration wit...
21st Athens Big Data Meetup - 1st Talk - Fast and simple data exploration wit...21st Athens Big Data Meetup - 1st Talk - Fast and simple data exploration wit...
21st Athens Big Data Meetup - 1st Talk - Fast and simple data exploration wit...
 
MongoDB + GeoServer
MongoDB + GeoServerMongoDB + GeoServer
MongoDB + GeoServer
 
Bandwidth Generation Lab
Bandwidth Generation LabBandwidth Generation Lab
Bandwidth Generation Lab
 
Bucket your partitions wisely - Cassandra summit 2016
Bucket your partitions wisely - Cassandra summit 2016Bucket your partitions wisely - Cassandra summit 2016
Bucket your partitions wisely - Cassandra summit 2016
 
How to Reduce Your Database Total Cost of Ownership with TimescaleDB
How to Reduce Your Database Total Cost of Ownership with TimescaleDBHow to Reduce Your Database Total Cost of Ownership with TimescaleDB
How to Reduce Your Database Total Cost of Ownership with TimescaleDB
 
Mongo sharding
Mongo shardingMongo sharding
Mongo sharding
 
How to calculate the optimal undo retention in Oracle
How to calculate the optimal undo retention in OracleHow to calculate the optimal undo retention in Oracle
How to calculate the optimal undo retention in Oracle
 
MongoDB GeoSpatial Feature
MongoDB GeoSpatial FeatureMongoDB GeoSpatial Feature
MongoDB GeoSpatial Feature
 
Mongodb
MongodbMongodb
Mongodb
 
Exploring the Enron Email Dataset with Kiji and Hive
Exploring the Enron Email Dataset with Kiji and HiveExploring the Enron Email Dataset with Kiji and Hive
Exploring the Enron Email Dataset with Kiji and Hive
 
Mongodb
MongodbMongodb
Mongodb
 
Clickhouse Capacity Planning for OLAP Workloads, Mik Kocikowski of CloudFlare
Clickhouse Capacity Planning for OLAP Workloads, Mik Kocikowski of CloudFlareClickhouse Capacity Planning for OLAP Workloads, Mik Kocikowski of CloudFlare
Clickhouse Capacity Planning for OLAP Workloads, Mik Kocikowski of CloudFlare
 
Cache'owanie danych w PHP
Cache'owanie danych w PHPCache'owanie danych w PHP
Cache'owanie danych w PHP
 
2012 apache hadoop_map_reduce_windows_azure
2012 apache hadoop_map_reduce_windows_azure2012 apache hadoop_map_reduce_windows_azure
2012 apache hadoop_map_reduce_windows_azure
 
Geospatial and MongoDB
Geospatial and MongoDBGeospatial and MongoDB
Geospatial and MongoDB
 
Александр Терещук - Memory Analyzer Tool and memory optimization tips in Android
Александр Терещук - Memory Analyzer Tool and memory optimization tips in AndroidАлександр Терещук - Memory Analyzer Tool and memory optimization tips in Android
Александр Терещук - Memory Analyzer Tool and memory optimization tips in Android
 

Similar to Geo & capped collections with MongoDB

MongoDB - A next-generation database that lets you create applications never ...
MongoDB - A next-generation database that lets you create applications never ...MongoDB - A next-generation database that lets you create applications never ...
MongoDB - A next-generation database that lets you create applications never ...
Ram Murat Sharma
 
Python and MongoDB
Python and MongoDB Python and MongoDB
Python and MongoDB
Norberto Leite
 
Superficial mongo db
Superficial mongo dbSuperficial mongo db
Superficial mongo db
DaeMyung Kang
 
Look Ma! No more blobs
Look Ma! No more blobsLook Ma! No more blobs
Look Ma! No more blobs
Aparna Chaudhary
 
Mongodb in-anger-boston-rb-2011
Mongodb in-anger-boston-rb-2011Mongodb in-anger-boston-rb-2011
Mongodb in-anger-boston-rb-2011
bostonrb
 
MongoDB and Python
MongoDB and PythonMongoDB and Python
MongoDB and Python
Norberto Leite
 
Mongodb
MongodbMongodb
Mongodb
Apurva Vyas
 
MongoDB dessi-codemotion
MongoDB dessi-codemotionMongoDB dessi-codemotion
MongoDB dessi-codemotion
Massimiliano Dessì
 
Mongo db dla administratora
Mongo db dla administratoraMongo db dla administratora
Mongo db dla administratora
Łukasz Jagiełło
 
Einführung in MongoDB
Einführung in MongoDBEinführung in MongoDB
Einführung in MongoDB
NETUserGroupBern
 
171_74_216_Module_5-Non_relational_database_-mongodb.pptx
171_74_216_Module_5-Non_relational_database_-mongodb.pptx171_74_216_Module_5-Non_relational_database_-mongodb.pptx
171_74_216_Module_5-Non_relational_database_-mongodb.pptx
sukrithlal008
 
Building Cloud-Native Applications with Microsoft Windows Azure
Building Cloud-Native Applications with Microsoft Windows AzureBuilding Cloud-Native Applications with Microsoft Windows Azure
Building Cloud-Native Applications with Microsoft Windows Azure
Bill Wilder
 
MongoDB using Grails plugin by puneet behl
MongoDB using Grails plugin by puneet behlMongoDB using Grails plugin by puneet behl
MongoDB using Grails plugin by puneet behl
TO THE NEW | Technology
 
Introduction to Apache Tajo: Data Warehouse for Big Data
Introduction to Apache Tajo: Data Warehouse for Big DataIntroduction to Apache Tajo: Data Warehouse for Big Data
Introduction to Apache Tajo: Data Warehouse for Big Data
Gruter
 
MongoDB Introduction and Data Modelling
MongoDB Introduction and Data Modelling MongoDB Introduction and Data Modelling
MongoDB Introduction and Data Modelling
Sachin Bhosale
 
introtomongodb
introtomongodbintrotomongodb
introtomongodb
saikiran
 
Dev Jumpstart: Build Your First App with MongoDB
Dev Jumpstart: Build Your First App with MongoDBDev Jumpstart: Build Your First App with MongoDB
Dev Jumpstart: Build Your First App with MongoDB
MongoDB
 
Hadoop
HadoopHadoop
Starting with MongoDB
Starting with MongoDBStarting with MongoDB
Starting with MongoDB
DoThinger
 
mongodb tutorial
mongodb tutorialmongodb tutorial
mongodb tutorial
Jaehong Park
 

Similar to Geo & capped collections with MongoDB (20)

MongoDB - A next-generation database that lets you create applications never ...
MongoDB - A next-generation database that lets you create applications never ...MongoDB - A next-generation database that lets you create applications never ...
MongoDB - A next-generation database that lets you create applications never ...
 
Python and MongoDB
Python and MongoDB Python and MongoDB
Python and MongoDB
 
Superficial mongo db
Superficial mongo dbSuperficial mongo db
Superficial mongo db
 
Look Ma! No more blobs
Look Ma! No more blobsLook Ma! No more blobs
Look Ma! No more blobs
 
Mongodb in-anger-boston-rb-2011
Mongodb in-anger-boston-rb-2011Mongodb in-anger-boston-rb-2011
Mongodb in-anger-boston-rb-2011
 
MongoDB and Python
MongoDB and PythonMongoDB and Python
MongoDB and Python
 
Mongodb
MongodbMongodb
Mongodb
 
MongoDB dessi-codemotion
MongoDB dessi-codemotionMongoDB dessi-codemotion
MongoDB dessi-codemotion
 
Mongo db dla administratora
Mongo db dla administratoraMongo db dla administratora
Mongo db dla administratora
 
Einführung in MongoDB
Einführung in MongoDBEinführung in MongoDB
Einführung in MongoDB
 
171_74_216_Module_5-Non_relational_database_-mongodb.pptx
171_74_216_Module_5-Non_relational_database_-mongodb.pptx171_74_216_Module_5-Non_relational_database_-mongodb.pptx
171_74_216_Module_5-Non_relational_database_-mongodb.pptx
 
Building Cloud-Native Applications with Microsoft Windows Azure
Building Cloud-Native Applications with Microsoft Windows AzureBuilding Cloud-Native Applications with Microsoft Windows Azure
Building Cloud-Native Applications with Microsoft Windows Azure
 
MongoDB using Grails plugin by puneet behl
MongoDB using Grails plugin by puneet behlMongoDB using Grails plugin by puneet behl
MongoDB using Grails plugin by puneet behl
 
Introduction to Apache Tajo: Data Warehouse for Big Data
Introduction to Apache Tajo: Data Warehouse for Big DataIntroduction to Apache Tajo: Data Warehouse for Big Data
Introduction to Apache Tajo: Data Warehouse for Big Data
 
MongoDB Introduction and Data Modelling
MongoDB Introduction and Data Modelling MongoDB Introduction and Data Modelling
MongoDB Introduction and Data Modelling
 
introtomongodb
introtomongodbintrotomongodb
introtomongodb
 
Dev Jumpstart: Build Your First App with MongoDB
Dev Jumpstart: Build Your First App with MongoDBDev Jumpstart: Build Your First App with MongoDB
Dev Jumpstart: Build Your First App with MongoDB
 
Hadoop
HadoopHadoop
Hadoop
 
Starting with MongoDB
Starting with MongoDBStarting with MongoDB
Starting with MongoDB
 
mongodb tutorial
mongodb tutorialmongodb tutorial
mongodb tutorial
 

More from Rainforest QA

Machine Learning in Practice - CTO Summit Chicago 2019
Machine Learning in Practice - CTO Summit Chicago 2019Machine Learning in Practice - CTO Summit Chicago 2019
Machine Learning in Practice - CTO Summit Chicago 2019
Rainforest QA
 
CTO Summit NASDAQ NYC 2017: Creating a QA Strategy
CTO Summit NASDAQ NYC 2017: Creating a QA StrategyCTO Summit NASDAQ NYC 2017: Creating a QA Strategy
CTO Summit NASDAQ NYC 2017: Creating a QA Strategy
Rainforest QA
 
Ops Skills and Tools for Beginners [#MongoDB World 2014]
Ops Skills and Tools for Beginners [#MongoDB World 2014]Ops Skills and Tools for Beginners [#MongoDB World 2014]
Ops Skills and Tools for Beginners [#MongoDB World 2014]
Rainforest QA
 
Pragmatic Rails Architecture [SF Rails, 24 Apr 14]
Pragmatic Rails Architecture [SF Rails, 24 Apr 14]Pragmatic Rails Architecture [SF Rails, 24 Apr 14]
Pragmatic Rails Architecture [SF Rails, 24 Apr 14]
Rainforest QA
 
Bitcoin Ops & Security Primer
Bitcoin Ops & Security PrimerBitcoin Ops & Security Primer
Bitcoin Ops & Security Primer
Rainforest QA
 
Pivotal Labs Lunch Talk; 3 Infrastructure and workflow lessons learned at an ...
Pivotal Labs Lunch Talk; 3 Infrastructure and workflow lessons learned at an ...Pivotal Labs Lunch Talk; 3 Infrastructure and workflow lessons learned at an ...
Pivotal Labs Lunch Talk; 3 Infrastructure and workflow lessons learned at an ...
Rainforest QA
 
MongoDB Command Line Tools
MongoDB Command Line ToolsMongoDB Command Line Tools
MongoDB Command Line Tools
Rainforest QA
 
Seedhack MongoDB 2011
Seedhack MongoDB 2011Seedhack MongoDB 2011
Seedhack MongoDB 2011
Rainforest QA
 
An Introduction to Map/Reduce with MongoDB
An Introduction to Map/Reduce with MongoDBAn Introduction to Map/Reduce with MongoDB
An Introduction to Map/Reduce with MongoDB
Rainforest QA
 
How does Riak compare to Cassandra? [Cassandra London User Group July 2011]
How does Riak compare to Cassandra? [Cassandra London User Group July 2011]How does Riak compare to Cassandra? [Cassandra London User Group July 2011]
How does Riak compare to Cassandra? [Cassandra London User Group July 2011]
Rainforest QA
 
London MongoDB User Group April 2011
London MongoDB User Group April 2011London MongoDB User Group April 2011
London MongoDB User Group April 2011
Rainforest QA
 

More from Rainforest QA (11)

Machine Learning in Practice - CTO Summit Chicago 2019
Machine Learning in Practice - CTO Summit Chicago 2019Machine Learning in Practice - CTO Summit Chicago 2019
Machine Learning in Practice - CTO Summit Chicago 2019
 
CTO Summit NASDAQ NYC 2017: Creating a QA Strategy
CTO Summit NASDAQ NYC 2017: Creating a QA StrategyCTO Summit NASDAQ NYC 2017: Creating a QA Strategy
CTO Summit NASDAQ NYC 2017: Creating a QA Strategy
 
Ops Skills and Tools for Beginners [#MongoDB World 2014]
Ops Skills and Tools for Beginners [#MongoDB World 2014]Ops Skills and Tools for Beginners [#MongoDB World 2014]
Ops Skills and Tools for Beginners [#MongoDB World 2014]
 
Pragmatic Rails Architecture [SF Rails, 24 Apr 14]
Pragmatic Rails Architecture [SF Rails, 24 Apr 14]Pragmatic Rails Architecture [SF Rails, 24 Apr 14]
Pragmatic Rails Architecture [SF Rails, 24 Apr 14]
 
Bitcoin Ops & Security Primer
Bitcoin Ops & Security PrimerBitcoin Ops & Security Primer
Bitcoin Ops & Security Primer
 
Pivotal Labs Lunch Talk; 3 Infrastructure and workflow lessons learned at an ...
Pivotal Labs Lunch Talk; 3 Infrastructure and workflow lessons learned at an ...Pivotal Labs Lunch Talk; 3 Infrastructure and workflow lessons learned at an ...
Pivotal Labs Lunch Talk; 3 Infrastructure and workflow lessons learned at an ...
 
MongoDB Command Line Tools
MongoDB Command Line ToolsMongoDB Command Line Tools
MongoDB Command Line Tools
 
Seedhack MongoDB 2011
Seedhack MongoDB 2011Seedhack MongoDB 2011
Seedhack MongoDB 2011
 
An Introduction to Map/Reduce with MongoDB
An Introduction to Map/Reduce with MongoDBAn Introduction to Map/Reduce with MongoDB
An Introduction to Map/Reduce with MongoDB
 
How does Riak compare to Cassandra? [Cassandra London User Group July 2011]
How does Riak compare to Cassandra? [Cassandra London User Group July 2011]How does Riak compare to Cassandra? [Cassandra London User Group July 2011]
How does Riak compare to Cassandra? [Cassandra London User Group July 2011]
 
London MongoDB User Group April 2011
London MongoDB User Group April 2011London MongoDB User Group April 2011
London MongoDB User Group April 2011
 

Recently uploaded

Christine's Supplier Sourcing Presentaion.pptx
Christine's Supplier Sourcing Presentaion.pptxChristine's Supplier Sourcing Presentaion.pptx
Christine's Supplier Sourcing Presentaion.pptx
christinelarrosa
 
"Frontline Battles with DDoS: Best practices and Lessons Learned", Igor Ivaniuk
"Frontline Battles with DDoS: Best practices and Lessons Learned",  Igor Ivaniuk"Frontline Battles with DDoS: Best practices and Lessons Learned",  Igor Ivaniuk
"Frontline Battles with DDoS: Best practices and Lessons Learned", Igor Ivaniuk
Fwdays
 
Taking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdfTaking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdf
ssuserfac0301
 
Northern Engraving | Modern Metal Trim, Nameplates and Appliance Panels
Northern Engraving | Modern Metal Trim, Nameplates and Appliance PanelsNorthern Engraving | Modern Metal Trim, Nameplates and Appliance Panels
Northern Engraving | Modern Metal Trim, Nameplates and Appliance Panels
Northern Engraving
 
GNSS spoofing via SDR (Criptored Talks 2024)
GNSS spoofing via SDR (Criptored Talks 2024)GNSS spoofing via SDR (Criptored Talks 2024)
GNSS spoofing via SDR (Criptored Talks 2024)
Javier Junquera
 
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...
Pitangent Analytics & Technology Solutions Pvt. Ltd
 
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
Jason Yip
 
Choosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptxChoosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptx
Brandon Minnick, MBA
 
Mutation Testing for Task-Oriented Chatbots
Mutation Testing for Task-Oriented ChatbotsMutation Testing for Task-Oriented Chatbots
Mutation Testing for Task-Oriented Chatbots
Pablo Gómez Abajo
 
Biomedical Knowledge Graphs for Data Scientists and Bioinformaticians
Biomedical Knowledge Graphs for Data Scientists and BioinformaticiansBiomedical Knowledge Graphs for Data Scientists and Bioinformaticians
Biomedical Knowledge Graphs for Data Scientists and Bioinformaticians
Neo4j
 
Session 1 - Intro to Robotic Process Automation.pdf
Session 1 - Intro to Robotic Process Automation.pdfSession 1 - Intro to Robotic Process Automation.pdf
Session 1 - Intro to Robotic Process Automation.pdf
UiPathCommunity
 
Nordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptxNordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptx
MichaelKnudsen27
 
5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides
DanBrown980551
 
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-EfficiencyFreshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
ScyllaDB
 
Leveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and StandardsLeveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and Standards
Neo4j
 
A Deep Dive into ScyllaDB's Architecture
A Deep Dive into ScyllaDB's ArchitectureA Deep Dive into ScyllaDB's Architecture
A Deep Dive into ScyllaDB's Architecture
ScyllaDB
 
Demystifying Knowledge Management through Storytelling
Demystifying Knowledge Management through StorytellingDemystifying Knowledge Management through Storytelling
Demystifying Knowledge Management through Storytelling
Enterprise Knowledge
 
inQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
inQuba Webinar Mastering Customer Journey Management with Dr Graham HillinQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
inQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
LizaNolte
 
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
saastr
 
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
Alex Pruden
 

Recently uploaded (20)

Christine's Supplier Sourcing Presentaion.pptx
Christine's Supplier Sourcing Presentaion.pptxChristine's Supplier Sourcing Presentaion.pptx
Christine's Supplier Sourcing Presentaion.pptx
 
"Frontline Battles with DDoS: Best practices and Lessons Learned", Igor Ivaniuk
"Frontline Battles with DDoS: Best practices and Lessons Learned",  Igor Ivaniuk"Frontline Battles with DDoS: Best practices and Lessons Learned",  Igor Ivaniuk
"Frontline Battles with DDoS: Best practices and Lessons Learned", Igor Ivaniuk
 
Taking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdfTaking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdf
 
Northern Engraving | Modern Metal Trim, Nameplates and Appliance Panels
Northern Engraving | Modern Metal Trim, Nameplates and Appliance PanelsNorthern Engraving | Modern Metal Trim, Nameplates and Appliance Panels
Northern Engraving | Modern Metal Trim, Nameplates and Appliance Panels
 
GNSS spoofing via SDR (Criptored Talks 2024)
GNSS spoofing via SDR (Criptored Talks 2024)GNSS spoofing via SDR (Criptored Talks 2024)
GNSS spoofing via SDR (Criptored Talks 2024)
 
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...
 
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
 
Choosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptxChoosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptx
 
Mutation Testing for Task-Oriented Chatbots
Mutation Testing for Task-Oriented ChatbotsMutation Testing for Task-Oriented Chatbots
Mutation Testing for Task-Oriented Chatbots
 
Biomedical Knowledge Graphs for Data Scientists and Bioinformaticians
Biomedical Knowledge Graphs for Data Scientists and BioinformaticiansBiomedical Knowledge Graphs for Data Scientists and Bioinformaticians
Biomedical Knowledge Graphs for Data Scientists and Bioinformaticians
 
Session 1 - Intro to Robotic Process Automation.pdf
Session 1 - Intro to Robotic Process Automation.pdfSession 1 - Intro to Robotic Process Automation.pdf
Session 1 - Intro to Robotic Process Automation.pdf
 
Nordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptxNordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptx
 
5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides
 
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-EfficiencyFreshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
 
Leveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and StandardsLeveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and Standards
 
A Deep Dive into ScyllaDB's Architecture
A Deep Dive into ScyllaDB's ArchitectureA Deep Dive into ScyllaDB's Architecture
A Deep Dive into ScyllaDB's Architecture
 
Demystifying Knowledge Management through Storytelling
Demystifying Knowledge Management through StorytellingDemystifying Knowledge Management through Storytelling
Demystifying Knowledge Management through Storytelling
 
inQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
inQuba Webinar Mastering Customer Journey Management with Dr Graham HillinQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
inQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
 
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
 
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
 

Geo & capped collections with MongoDB

  • 1. Geo & capped collections with MongoDB: Use cases & performance in the real world
  • 2. /usr/bin/whoami Russell Smith Consultant for UKD1 Limited Specialising in helping companies going through rapid growth Help with code, architecture, infrastructure, devops, sysops, capacity planning, etc <3 gearman, mongodb, neo4j, mysql, kohana, riaksearch, php, debian
  • 3. What are capped collections? Special option for collections which allow you to; Limit the size of a collection Limit the number of documents in a collection
  • 4. Why is this useful? Logging News feeds Caching
  • 5. Restrictions Can’t delete objects (you can drop the collection though) Can’t update objects (well...not 100% true - you can only if the size doesn’t change)
  • 6. Neat things Objects are returned in the insertion order Don’t have to worry about things growing out of control No more cronjobs to delete old data
  • 7. Real world example SongFor.com: Activity feeds for the front of the site are using capped collections StrongHold Kingdoms (MMO): Logging of events from syslog / internal events are stored in a capped collection (Graylog2 & a custom dashboard system) Lioness: cache + log of data, used to cut lag & reduce API access requests to Facebook Findlunch.in: Caching of page data
  • 8. How to setup... db.createCollection("mycoll", {capped:true, size:100000}) db.createCollection("mycoll", {capped:true, size:100000, max:100}); (limit to 100 documents - this is slower as it adds overhead)
  • 9. What is Geospatial indexing? Allows easily finding things close to a point - great for location based services Insert a document as normal, but with some co-ordinates... ensureIndex({location_field: ‘2d’});
  • 10. Other magic things You can search $within a $box, or around $center point (circle)...which is handy Results are sorted by distance from points geoNear (command) will return distance, average distance, etc
  • 11. Versions & support > v1.3.3 for initial version, which is pretty old >= 1.7.0 added spherical support (more in a minute) < 1.7.2 doesn’t support geo in sharded collections. It still has issues see: http://jira.mongodb.org/browse/SERVER-926
  • 12. and the point is? Really really easy to find documents around a specific point You can get distances from that point as well Lookups are fast and accurate This is a pain to do in MySQL
  • 13. Example FindLunch.in lets people find lunch close to where they are. Geocode lookup of ~2 million postcodes from the UK All London tube stations & most boroughs are also listed Average query time Xms over 20k of test documents
  • 14. Questions? Any questions?

Editor's Notes

  1. \n
  2. \n
  3. \n
  4. \n
  5. \n
  6. \n
  7. \n
  8. \n
  9. \n
  10. \n
  11. \n
  12. \n
  13. \n
  14. \n