An Evening with MongoDB Detroit 2013


Published on

1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

An Evening with MongoDB Detroit 2013

  1. 1. An Evening with MongoDB Detroit
  2. 2. Agenda• Overview: 10gen and MongoDB• MongoDB and Big Data Processing• MongoDB and Node.js 2
  3. 3. Introduction10gen is the company behind MongoDB –the leading NoSQL database Document- General Open- Oriented Purpose Source 3
  4. 4. 10gen Overview 170+ employees 500+ customers Offices in New York, Palo Alto, Washington Over $81 million in funding DC, London, Dublin, Barcelona and Sydney 4
  5. 5. Leading Organizations Rely on MongoDB 5
  6. 6. Global MongoDB Community41,000+Monthly Unique Downloads24,000+Online Education Registrants12,000+MongoDB User Group Members10,000+Annual MongoDB Days Attendees 6
  7. 7. Relational DatabaseChallenges
  8. 8. Relational Database ChallengesData Types Agile Development• Unstructured data • Iterative• Semi-structured • Short development data cycles• Polymorphic data • New workloadsVolume of Data New Architectures• Petabytes of data • Horizontal scaling• Trillions of records • Commodity servers• Millions of queries per second • Cloud computing 8
  9. 9. MongoDB Solution
  10. 10. MongoDB Solution Agile Scalable Powerful 10
  11. 11. AgilityRDBMS MongoDB { _id : ObjectId("4c4ba5e5e8aabf3"), employee_name: "Dunham, Justin", department : "Marketing", title : "Product Manager, Web", report_up: "Neray, Graham", pay_band: “C", benefits : [ { type : "Health", plan : "PPO Plus" }, { type : "Dental", plan : "Standard" } ] } 11
  12. 12. PowerBetter Data Locality In-Memory Caching RDBMS MongoDB 12
  13. 13. High Availability 13
  14. 14. Scalability 14
  15. 15. Voila! 15
  16. 16. MongoDB Features JSON Data Model with Auto-Sharding for Dynamic Schema Horizontal Scalability Flexible, Full Index Rich, Document-Based Support Queries Built-In Replication and Fast, In-Place High Availability Updates Aggregation Framework and GridFS for Large File Map/Reduce Storage 16
  17. 17. MongoDB Monitoring Service (MMS)Free, cloud-based service for monitoring and alerts• Charts, custom dashboards and automated alerting• Tracks 100+ metrics – performance, resource utilization, availability and response times• 10,000+ users 17
  18. 18. Best Total Cost of Ownership (TCO)Developer and Ops Savings• Less code• More productive development• Easier to maintainHardware Savings• Commodity servers• Internal storage (no SAN)• Scale out, not upSoftware and Support Savings• No upfront license – pay for value DB Alternative over time• Cost visibility for usage growth 18
  19. 19. 10gen Products and Services Subscriptions Professional Support, Subscriber Edition and Commercial License Consulting Expert Resources for All Phases of MongoDB Implementations Training Online and In-Person for Developers and Administrators 19
  20. 20. Customer Use Cases
  21. 21. MongoDB Use Cases Content Management Operational Intelligence E-Commerce User Data Management High Volume Data Feeds 21
  22. 22. MongoDB Adoption 22
  23. 23. Case Study Stores user and location-based data in MongoDB for social networking mobile app Problem Why MongoDB Results• Relational architecture • Auto-sharding to scale • Focus engineering on could not scale high-traffic and fast- building mobile app vs. growing application back-end• Check-in data growth hit single-node capacity • Geo-indexing for easy • Scale efficiently with ceiling querying of location- limited resources based data• Significant work to build • Increased developer custom sharding layer • Simple data model productivity 23
  24. 24. Case Study Relies on a MongoDB-powered, real-time analytics product for SMBs Problem Why MongoDB Results• More than 500,000 • Ability to handle • Shorter feature websites complex data while development cycles maintaining high (e.g., 1 week)• 10 years of complex performance data • 2.5x faster than MySQL • Took 1 week for devs to• Relational database ramp up on MongoDB • High-performance real- took several days to time analytics to over process data • Strong community 500,000 SMBs 24
  25. 25. Case Study Stores billions of posts in myriad formats with MongoDB Problem Why MongoDB Results• 1.5M posts per day, • Flexible document- • Initial deployment held different structures based model over 5B documents and• Inflexible MySQL, 10TB of data • Horizontal scalability lengthy delays for built in • Automated failover making changes provides high• Data piling up in • Easy to use availability production database • Interface in familiar • Schema changes are• Poor performance language quick and easy 25
  26. 26. Get InvolvedSpeak at a MongoDB User GroupGet MongoDB News and Updates 26
  27. 27. For More Information Resource Location MongoDB Downloads Free Online Training Webinars and Events White Papers Case Studies Presentations Documentation Additional Info 27
  28. 28. Case Study Serves targeted content to users using MongoDB- powered identity system Problem Why MongoDB Results• 20M+ unique visitors • Easy-to-manage • Rapid rollout of new per month dynamic data model features enables limitless• Rigid relational schema growth, interactive • Customized, social unable to evolve with content conversations changing data types throughout site and new features • Support for ad hoc queries • Tracks user data to• Slow development increase cycles • Highly extensible engagement, revenue 29
  29. 29. Case Study Uses MongoDB to safeguard over 6 billion images served to millions of customers Problem Why MongoDB Results• 6B images, 20TB of • JSON-based data • 5x cost reduction data model • 9x performance• Brittle code base on top • Agile, high improvement of Oracle database – performance, scalable hard to scale, add • Faster time-to-market features • Alignment with Shutterfly’s services- • Dev cycles in weeks vs.• High SW and HW costs based architecture tens of months 30
  30. 30. Case Study Stores one of world’s largest record repositories and searchable catalogues in MongoDB Problem Why MongoDB Results• One of world’s largest • Fast, easy scalability • Will scale to 100s of TB record repositories by 2013, PB by 2020 • Full query language• Move to SOA required • Searchable catalogue new approach to data • Complex metadata of varied data types store storage • Decreased SW and• RDBMS could not support costs support centralized data mgt and federation of information services 31
  31. 31. Case Study Provides low-latency, high-scale translation management platform built on MongoDB Problem Why MongoDB Results• Old MySQL • Horizontal scale with built- • Simplified scaling and performance in sharding high-performance degradation and high architecture maintenance • High availability with replica sets • Dramatically improved• Complex to scale MySQL developer productivity • Memory-mapped• High- architecture for ingesting • Increased uptime speed, asynchronous content quickly w/out storage and fast read separate caching layer requirements 32
  32. 32. Case Study Uses MongoDB to power real-time ad serving platform Problem Why MongoDB Results• Needed costly SQL • Dynamic schema enables • Billions of requests per architecture to enable continuous algorithm day with sub-ms latency real-time bidding development and customer-specific fields • Inexpensive cost per• Large volumes of data query and queries • Scalability for massive data volumes and low • TB of data stored,• Diverse, evolving latency populated on the fly and schema queried in real-time • Visual monitoring with MMS 33
  33. 33. Case Study Uses MongoDB to underpin social media monitoring and recommendation engine Problem Why MongoDB Results• HBase locked them into • Ease of use and flexibility • Robust queries and rigid data model, stifling of data model dynamic schema ability to create enable higher quality connections between • Powerful indexing and ad recommendations data sets hoc querying, plus integrated MapReduce • Bugs fixed in hours• Single points of failure instead of days with master/slave • High availability with topology replica sets • Dramatically improved uptime• Up to 1M posts per day 34
  34. 34. Case Study Stores 3.5 TB of data in MongoDB to power real-time dictionary Problem Why MongoDB Results• Performance • Easy to • Migrated 5B records in roadblocks with MySQL store, locate, retrieve data 1 day, zero downtime• Massive data ingestion • Eliminated Memcached • Reduced code by 75% led to database outages while increasing performance: up to 2M • Sped up document• Tables locked for tens requests per hour, 8,000 metadata retrieval from of seconds during words inserted per 30 ms to 0.1 ms inserts second • Significant cost • Long runway for scale-out savings, 15% reduction in servers 35
  35. 35. Case Study Self-service product built on MongoDB enables real-time analytics for social marketing Problem Why MongoDB Results• Need for real-time • Real-time aggregation to • Operational cost aggregation and adjust campaigns on the savings analytics fly • Simplified scale-out to• Tried SQL, then • Scalability for persistence support 140M MapReduce – both layer impressions solutions only handled periodic data, could not • Ability to store large • Data flexibility to add scale amounts of data reliably new features and performance gains without overhead 36
  36. 36. Case Study Runs unified data store serving hundreds of diverse web properties on MongoDB Problem Why MongoDB Results• Hundreds of diverse • Flexible schema • Developers can focus on web properties built on end-user features instead Java-based CMS • Rich querying and support of back-end storage for secondary index• Rich documents forced support • Simplified day-to-day into ill-suited model operations • Easy to manage• Adding new data types, replication and scaling • Simple to add new brands, tables to RDBMS killed content types, etc. to read performance platform 37
  37. 37. Case Study MongoDB serves as social gaming platform for hundreds of millions of users Problem Why MongoDB Results• Legacy MySQL • Flexible data model • Improved game hindered development applicable to wide variety performance and end-user speed, could not scale of use cases experience• Needed operational • High availability through • Server cost reduction database that could replica sets on commodity also handle real-time servers • Accelerated development analysis and time-to-market • Size and strength of• Server sprawl MongoDB community 38
  38. 38. Case Study Delivers agile automated supply chain service to retailers powered by MongoDB Problem Why MongoDB Results• RDBMS poorly- • Document-oriented model • Decreased supplier equipped to handle less complex, easier to onboard time by 12x varying data types code (e.g., SKUs, images) • Grew from 400K records to • Single data store for 40M in 12 months• Inefficient use of structured, semi- storage in RDBMS structured and • Significant cost reductions (i.e., 90% empty unstructured data on schema design time, columns) ongoing developer effort, • Scalability and availability and storage usage• Complex joins degraded performance • Analytics with MapReduce 39
  39. 39. Case Study Runs social marketing suite with real-time analytics on MongoDB Problem Why MongoDB Results• RDBMS could not meet • Ease of use, developer • Decreased app speed and scale ramp-up development from months requirements of to weeks measuring massive • Solution maturity – depth online activity of functionality, failover • 30M social events per day• Inability to provide real- stored in MongoDB • High-performance with time analytics and write-heavy system • 6x increase in customers aggregations supported over one year • Queuing and logging for• Unpredictable peak loads easy search at app layer 40
  40. 40. Case Study Real-time server and website monitoring solution runs on MongoDB Problem Why MongoDB Results• Needed to handle • General purpose DB • MongoDB-first policy thousands of requests per second • High-write throughput • 12+ TB ingested per month• MySQL resulted in • Scales easily while millions of rows per maintaining performance • Increased performance, month, per server decreased disk usage • Easy-to-use replication• Difficult to scale MySQL and automated failover • Simplified infrastructure with replication cuts costs, frees up • Native PHP and Python resources for dev drivers 41
  41. 41. Case Study Social e-commerce application built on MongoDB offers 100M+ products from over 30K brands Problem Why MongoDB Results• MySQL could not • Flexible data model to • Boosted developer accommodate growth handle varying product productivity attributes• Significant optimization • Scaled from 5M to required to tune MySQL • Scalability for global reach 100M products with performance minimal work • Ease of maintenance• Database maintenance • Decreased product inhibited development • Consistent performance import time by 90% even when adding data and new features 42