Webinar: NoSQL as the New Normal


Published on

What started as a way for web giants to solve problems of serious scale has become the default way all enterprises manage Big Data. Despite having a catchy, if inaccurate title, there really isn't a coherent "NoSQL" category, nor is there a simple future for the range of NoSQL databases. In this presentation, Matt Asay will outline the reasons for NoSQL's existence and persistence, how the different NoSQL technologies help enterprises get control of Big Data, and will identify the trends that point to a bright future for post-relational databases.

Published in: Technology

Webinar: NoSQL as the New Normal

  1. 1. VP, Corporate Strategy, 10genMatt Asay (@mjasay)NoSQL: The New Normal
  2. 2. 210gen Overview200+ employees 600+ customersOver $81 million in fundingOffices in New York, Palo Alto, WashingtonDC, London, Dublin, Barcelona and Sydney
  3. 3. 34,000,000+MongoDB Downloads50,000+Online Education Registrants15,000+MongoDB User Group Members14,000+MongoDB Monitoring Service Users (MMS)10,000+Annual MongoDB Days AttendeesGlobal Community
  4. 4. 4Indeed.com TrendsTop Job Trends1.  HTML 52.  MongoDB3.  iOS4.  Android5.  Mobile Apps6.  Puppet7.  Hadoop8.  jQuery9.  PaaS10.  Social MediaLeading NoSQL DatabaseLinkedIn Job SkillsMongoDBCompetitor 1Competitor 2Competitor 3Competitor 4Competitor 5All OthersGoogle SearchMongoDBCompetitor 1Competitor 2Competitor 3Competitor 4Jaspersoft Big Data IndexDirect Real-Time DownloadsMongoDBCompetitor 1Competitor 2Competitor 3
  5. 5. The Past…Is Prologue?
  6. 6. 6Back to the Future?What has been will be again,what has been done will be done again;there is nothing new under the sun.(Ecclesiastes 1:9, NIV)
  7. 7. 7What Do You Remember about 1969?
  8. 8. 8What Do You Remember about 1969?
  9. 9. 9nothing? hold that thought
  10. 10. 10•  IBM’s IMS (1969) – Developedas part of the Apollo Project•  IDS (Integrated Data Store),navigational database, 1973•  High performance but:–  Forced developers to worryabout both query design andschema design upfront–  Made it hard to change anythingmid-streamBack to the Future: PreSQL
  11. 11. 11•  Designed to overcome “PreSQL” deficiencies–  Decoupled query design from schema design–  Allowed developers to focus on schema design–  Could be confident that you could query the data asyou wanted later•  30 years of dominance later…Enter SQL
  12. 12. …The Present…
  13. 13. 13RDBMS Is Like a Spreadsheet
  14. 14. 14With “Relations” Between Rows
  15. 15. 15Lots of relations.Lots of rows.
  16. 16. 16It Hides What You’re Really Doing
  17. 17. 17It Makes Development HardRelationalDatabaseObject RelationalMappingApplicationCode XML Config DB Schema
  18. 18. 18And Makes Things Hard to ChangeNewTableNewTableNewColumnName Pet Phone EmailNewColumn3 months later…
  19. 19. 19RDBMS Scale = Bigger Computers“Clients can also opt to run zEC12 without a raiseddatacenter floor -- a first for high-end IBM mainframes.”IBM Press Release 28 Aug, 2012
  20. 20. 20This Was a Problem for GoogleSource: http://googleblog.blogspot.com/2010/06/our-new-search-index-caffeine.html250,000+MBP’s==4.1miles2010 Search Index Size:100,000,000 GBNew data added per day100,000+ GBDatabases they could use0
  21. 21. 21And for Facebook2010: 13,000,000 queries per second
  22. 22. 22And for Facebook2010: 13,000,000 queries per secondTPC Top ResultsTPC #1 DB: 504,161 tps
  23. 23. 23And for Facebook2010: 13,000,000 queries per secondTPC Top ResultsTPC #1 DB: 504,161 tpsTop 10 combined: 1,370,368 tps
  24. 24. 24Big Data Changes Everything…Variety of Data•  Unstructured, semi-structured, polymorphicVolume/Velocity of Data•  Petabytes of data•  Millions of queries persecond; real-timeAgile Development•  Iterative, short developmentcyclesNew Architectures•  Horizontal scalingof commodityservers; cloudSource: NewVantage, 2012
  25. 25. 25Shift in What We’re Computing
  26. 26. 26Living in the Post-transactional FutureOrder-processing systems largely “done” (RDBMS);primary focus on better search and recommendationsor adapting prices on the fly (NoSQL)Vast majority of its engineering is focused onrecommending better movies (NoSQL), notprocessing monthly bills (RDBMS)Easy part is processing the credit card (RDBMS).Hard part is making it location aware, so it knowswhere you are and what you’re buying (NoSQL)
  27. 27. 27“Systems of Engagement are built by front-linedevelopers using modern languages who aredriven by time to market, the need for rapiddeployment and iteration….They value solutionsthat make it easy for them to deploy theirapplication code with as little friction aspossible.” (Forrester 2013)Shift in How We Develop Applications
  28. 28. 29AgileWhy NoSQL?Scalable Lower TCO
  29. 29. 30Operational Database Landscape
  30. 30. What Does This Mean?
  31. 31. 32Developers Are More ProductiveApplicationCodeRelationalDatabaseObject RelationalMappingXML Config DB Schema
  32. 32. 33Developers Are More ProductiveApplicationCodeRelationalDatabaseObject RelationalMappingXML Config DB Schema
  33. 33. 34Developers? They Matter
  34. 34. Some MongoDB Examples
  35. 35. 36RDBMSAgilityMongoDB{_id : ObjectId("4c4ba5e5e8aabf3"),employee_name: "Dunham, Justin",department : "Marketing",title : "Product Manager, Web",report_up: "Neray, Graham",pay_band: “C",benefits : [{ type : "Health",plan : "PPO Plus" },{ type : "Dental",plan : "Standard" }]}
  36. 36. 37Serves targeted content to users using MongoDB-powered identity systemExampleProblem Why MongoDB Results•  20M+ unique visitorsper month•  Rigid relational schemaunable to evolve withchanging data typesand new features•  Slow developmentcycles•  Easy-to-managedynamic data modelenables limitlessgrowth, interactivecontent•  Support for ad hocqueries•  Highly extensible•  Rapid rollout of newfeatures•  Customized, socialconversationsthroughout site•  Tracks user data toincrease engagement,revenue
  37. 37. 38ScalabilityAuto-Sharding•  Increase capacity as you go•  Commodity and cloud architectures•  Improved operational simplicity and cost visibility
  38. 38. 39Manages a wide range of content and servicesfor its web properties using MongoDBCase StudyProblem Why MongoDB Results•  Trouble dealing with ahuge variety of content•  Open-source RDBMSunable to keep up withperformance andscalability requirements•  Problems compoundedby integratinginformation from T-Mobile joint venture•  Move from 6 billion rowsin RDBMS to simplicityof 1 document•  Automated failover andability to add nodeswithout downtime•  “Blazingly fast” queryperformance: “blownaway by [MongoDB’s]performance”•  Significant performancegains despite bigincrease in volume andvariety of data•  Greater agility, fasterdevelopment iteration•  Saved £2m in licensesand hardware
  39. 39. 40Developer/Ops Savings•  Ease of Use•  Agile development•  Less maintenanceHardware Savings•  Commodity servers•  Internal storage (no SAN)•  Scale out, not upSoftware/Support Savings•  No upfront license•  Cost visibility for usage growthBetter Total Cost of Ownership (TCO)DB Alternative
  40. 40. 41Stores one of world’s largest record repositories andsearchable catalogues in MongoDBCase StudyProblem Why MongoDB Results•  One of world’s largestrecord repositories•  Move to SOA requirednew approach to datastore•  RDBMS could notsupport centralized datamgt and federation ofinformation services•  Fast, easy scalability•  Full query language•  Complex metadatastorage•  Delivers high scalability,fast performance, andeasy maintenance, whilekeeping support costs low•  Will scale to 100s of TBby 2013, PB by 2020•  Searchable catalogueof varied data types•  Decreased SW andsupport costs
  41. 41. 42Better DataLocalityPerformanceIn-MemoryCachingIn-PlaceUpdates
  42. 42. 43Uses MongoDB to safeguard over 6 billion imagesserved to millions of customersCase StudyProblem Why MongoDB Results•  6B images, 20TB ofdata•  Brittle code base on topof Oracle database –hard to scale, addfeatures•  High SW and HW costs•  JSON-based datamodel•  Agile, highperformance, scalable•  Alignment withShutterfly’s services-based architecture•  5x cost reduction•  9x performanceimprovement•  Faster time-to-market•  Dev cycles in weeks vs.tens of months
  43. 43. …The Future
  44. 44. 45Leading Organizations Rely on MongoDB
  45. 45. 46NoSQL AdoptionFirstNoSQLProjectMultipleNoSQLProjectsMultipleNoSQLProjectsNoSQLCentre ofExcellenceNoSQLFirstPolicy
  46. 46. 47NoSQL: The New NormalRDBMSs MeetRequirementsKey/Value orColumn StoresMeet RequirementsDocumentStore MeetsRequirements
  47. 47. 48Is It a Polyglot Future?
  48. 48. 49Remember the Long Tail?
  49. 49. 50It Didn’t Work out Well
  50. 50. 51General Purpose, High PerformanceRank Database Database Type Score Changes1Oracle RDBMS 1560.59   +27.2  2MySQL RDBMS 1342.45   +47.24  3Microsoft SQLServer RDBMS 1278.15   -­‐40.21  4PostgreSQL RDBMS 174.09   -­‐3.07  5Microsoft Access RDBMS 161.4   -­‐8.77  6DB2 RDBMS 155.02   -­‐4.31  7MongoDB Document store 129.75   +5.52  8SQLite RDBMS 88.94   +5.68  9Sybase RDBMS 80.16   -­‐5.25  10Solr Search engine 46.15   +2.99  11Teradata RDBMS 44.93  12Cassandra Columnar store 38.57   +2.21  13Redis Key-value store 35.58   +3.15  14Memcached Key-value store 24.8   -­‐0.17  15Informix RDBMS 24   +0.1  Source: DBEngines, Feb 2013Database PopularityJobs, Searches, Mentions, Etc.
  51. 51. 52MongoDB BenefitsIncreased Developer Productivity Better User ExperienceFaster Time to Market Lower TCO