The Past, Present, and Future of NoSQL

1,265 views

Published on

Video and slides synchronized, mp3 and slide download available at http://bit.ly/17ohqr5.

Matt Asay outlines the reasons for NoSQL's existence and persistence, and will identify the trends that point to a bright future for post-relational databases. Filmed at qconlondon.com.

Matt Asay is the VP of Corporate Strategy at 10gen (the MongoDB company). Before 10gen, Matt was Vice President of Business Development at Nodeable (acquired by Appcelerator), a provider of real-time data stream processing for big data applications, and also at Strobe (acquired by Facebook). Twitter: @mjasay

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,265
On SlideShare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
0
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

The Past, Present, and Future of NoSQL

  1. 1. NoSQL: The New NormalMatt Asay (@mjasay)VP, Corporate Strategy, 10gen
  2. 2. Watch the video with slide synchronization on InfoQ.com! http://www.infoq.com/presentations /nosql-history-future InfoQ.com: News & Community Site• 750,000 unique visitors/month• Published in 4 languages (English, Chinese, Japanese and Brazilian Portuguese)• Post content from our QCon conferences• News 15-20 / week• Articles 3-4 / week• Presentations (videos) 12-15 / week• Interviews 2-3 / week• Books 1 / month
  3. 3. Presented at QCon London www.qconlondon.comPurpose of QCon- to empower software development by facilitating the spread ofknowledge and innovationStrategy - practitioner-driven conference designed for YOU: influencers ofchange and innovation in your teams- speakers and topics driving the evolution and innovation- connecting and catalyzing the influencers and innovatorsHighlights- attended by more than 12,000 delegates since 2007- held in 9 cities worldwide
  4. 4. 10gen Overview 180+ employees 500+ customers Offices in New York, Palo Alto, Washington Over $81 million in funding DC, London, Dublin, Barcelona and Sydney 2
  5. 5. Global Community3,800,000+MongoDB Downloads47,000+Online Education Registrants15,000+MongoDB User Group Members14,000+MongoDB Monitoring Service Users (MMS)10,000+Annual MongoDB Days Attendees 3
  6. 6. the past…is prologue?
  7. 7. Back to the Future? What has been will be again, what has been done will be done again; there is nothing new under the sun. (Ecclesiastes 1:9, NIV) 5
  8. 8. What Do You Remember about 1969? 6
  9. 9. nothing? hold that thought 7
  10. 10. Back to the Future: PreSQL •  IBM’s IMS (1969) – Developed as part of the Apollo Project •  IDS (Integrated Data Store), navigational database, 1973 •  High performance but: –  Forced developers to worry about both query design and schema design upfront –  Made it hard to change anything mid-stream 8
  11. 11. Enter SQL•  Designed to overcome “PreSQL” deficiencies –  Decoupled query design from schema design –  Allowed developers to focus on schema design –  Could be confident that you could query the data as you wanted later•  30 years of dominance later… 9
  12. 12. …the present…
  13. 13. RDBMS Is Like a Spreadsheet 11
  14. 14. With “Relations” Between Rows 12
  15. 15. Lots of relations. Lots of rows. 13
  16. 16. It Hides What You’re Really Doing 14
  17. 17. It Makes Development Hard Code XML Config DB Schema Object Relational Relational Application Mapping Database 15
  18. 18. And Makes Things Hard to ChangeNewTable New Column New Table Name Pet Phone Email New Column 3 months later… 16
  19. 19. RDBMS Scale = Bigger Computers “Clients can also opt to run zEC12 without a raised datacenter floor -- a first for high-end IBM mainframes.” IBM Press Release 28 Aug, 2012 17
  20. 20. This Was a Problem for Google2010 Search Index Size: 250,000+ MBP’s == 4.1 miles100,000,000 GBNew data added per day100,000+ GBDatabases they could use0 18 Source: http://googleblog.blogspot.com/2010/06/our-new-search-index-caffeine.html
  21. 21. And for Facebook 2010: 13,000,000 queries per second 19
  22. 22. And for Facebook 2010: 13,000,000 queries per second TPC #1 DB: 504161 tpsTPC Top Results 20
  23. 23. And for Facebook 2010: 13,000,000 queries per second TPC #1 DB: 504161 tpsTPC Top Results Top 10 combined: 1,370,368 tps 21
  24. 24. Big Data Changes Everything…for EveryoneVariety of Data Agile Development•  Unstructured data •  Iterative•  Semi-structured •  Short development data cycles•  Polymorphic data •  New workloadsVolume/Velocity of Data New Architectures•  Petabytes of data •  Horizontal scaling•  Trillions of records •  Commodity servers•  Millions of queries per •  Cloud computing second 22
  25. 25. Shift in What We’re Computing 23
  26. 26. Living in the Post-transactional Future Order-processing systems largely “done” (RDBMS); primary focus on better search and recommendations or adapting prices on the fly (NoSQL)Vast majority of its engineering is focused onrecommending better movies (NoSQL), notprocessing monthly bills (RDBMS) Easy part is processing the credit card (RDBMS). Hard part is making it location aware, so it knows where you are and what you’re buying (NoSQL) 24
  27. 27. Shift in How We Develop Applications “Systems of Engagement are built by front-line developers using modern languages who are driven by time to market, the need for rapiddeployment and iteration….They value solutions that make it easy for them to deploy their application code with as little friction as possible.” (Forrester 2013) 25
  28. 28. Why NoSQL? Agile Scalable Lower TCO 27
  29. 29. what does this mean?
  30. 30. Developers Are More Productive Code XML Config DB Schema Object Relational Relational Application Mapping Database 29
  31. 31. Developers Are More Productive Code XML Config DB Schema Object Relational Relational Application Mapping Database 30
  32. 32. Developers? They Matter 31
  33. 33. some nosql examples
  34. 34. Agility RDBMS MongoDB { _id : ObjectId("4c4ba5e5e8aabf3"), employee_name: "Dunham, Justin", department : "Marketing", title : "Product Manager, Web", report_up: "Neray, Graham", pay_band: “C", benefits : [ { type : "Health", plan : "PPO Plus" }, { type : "Dental", plan : "Standard" } ] } 33
  35. 35. Example Serves targeted content to users using MongoDB- powered identity system Problem Why MongoDB Results•  20M+ unique visitors •  Easy-to-manage •  Rapid rollout of new per month dynamic data model features enables limitless•  Rigid relational schema growth, interactive •  Customized, social unable to evolve with conversations content changing data types throughout site and new features •  Support for ad hoc •  Tracks user data to queries•  Slow development increase engagement, cycles •  Highly extensible revenue 34
  36. 36. Scalability Auto-Sharding•  Increase capacity as you go•  Commodity and cloud architectures•  Improved operational simplicity and cost visibility 35
  37. 37. Case Study Manages a wide range of content and services for its web properties using MongoDB Problem Why MongoDB Results•  Trouble dealing with a •  Move from 6 billion rows •  Significant performance huge variety of content in RDBMS to simplicity gains despite big•  MySQL unable to keep of 1 document increase in volume and up with performance variety of data •  Automated failover and and scalability requirements ability to add nodes •  Greater agility, faster without downtime development iteration•  Problems compounded by integrating •  “Blazingly fast” query •  Saved £2m in licenses information from T- performance: “blown and hardware Mobile joint venture away by [MongoDB’s] performance” 36
  38. 38. Better Total Cost of Ownership (TCO)Developer/Ops Savings•  Ease of Use•  Agile development•  Less maintenanceHardware Savings•  Commodity servers•  Internal storage (no SAN)•  Scale out, not upSoftware/Support Savings DB Alternative•  No upfront license•  Cost visibility for usage growth 37
  39. 39. Case Study Stores one of world’s largest record repositories and searchable catalogues in MongoDB Problem Why MongoDB Results•  One of world’s largest •  Fast, easy scalability •  Will scale to 100s of TB record repositories by 2013, PB by 2020 •  Full query language•  Move to SOA required •  Searchable catalogue new approach to data •  Complex metadata of varied data types store storage •  Decreased SW and•  RDBMS could not •  Delivers high scalability, support costs support centralized data fast performance, and mgt and federation of easy maintenance, while information services keeping support costs low 38
  40. 40. Performance Better Data In-Memory In-Place Locality Caching Updates 39
  41. 41. Case Study Uses MongoDB to safeguard over 6 billion images served to millions of customers Problem Why MongoDB Results•  6B images, 20TB of •  JSON-based data •  5x cost reduction data model •  9x performance•  Brittle code base on top •  Agile, high improvement of Oracle database – performance, scalable hard to scale, add •  Faster time-to-market features •  Alignment with •  Dev cycles in weeks vs. Shutterfly’s services-•  High SW and HW costs tens of months based architecture 40
  42. 42. …the future
  43. 43. Leading Organizations Rely on NoSQL 42
  44. 44. NoSQL Adoption NoSQL First Policy Multiple NoSQL NoSQL of Centre Projects Excellence Multiple NoSQL First Projects NoSQL Project 43
  45. 45. NoSQL: The New Normal Document Store Meets RequirementsRDBMSs MeetRequirements Key/Value or Column Stores Meet Requirements 44
  46. 46. Is It a Polyglot Future? 45
  47. 47. Remember the Long Tail? 46
  48. 48. It Didn’t Work out Well 47
  49. 49. General Purpose, High Performance Database Popularity Jobs, Searches, Mentions, Etc. Rank Database Database Type Score % Change 1 Oracle RDBMS 1567.93 + 8.6 Microsoft SQL 2 Server RDBMS 1310.61 - 0.73 3 MySQL RDBMS 1284.78 - 28.89 4 Microsoft Access RDBMS 186.05 + 15.12 5 PostgreSQL RDBMS 183.47 + 16 6 DB2 RDBMS 162.05 + 8.39 7 MongoDB Document store 116.14 + 20.01 8 Sybase RDBMS 84.52 + 1.07 9 SQLite RDBMS 81.01 + 0.65 10 Solr Search engine 47.35 Wide column 11 Cassandra store 36.16 + 1.24 12 Redis Key-value store 32.03 + 6.06 13 Informix RDBMS 25.43 + 1.52 14 Memcached Key-value store 23.28 - 1.83 Wide column 15 Hbase store 20.74 + 1.05 Source: DBEngines, Feb 2013 48
  50. 50. NoSQL BenefitsIncreased Developer Productivity Better User Experience Faster Time to Market Lower TCO 49
  51. 51. Leading NoSQL Database Google Search LinkedIn Job Skills MongoDB Competitor 1 Competitor 2 Competitor 3 Competitor 4 Competitor 5 MongoDB Competitor 2 Competitor 4 All Others Competitor 1 Competitor 3 Jaspersoft Big Data Index Indeed.com Trends Direct Real-Time Downloads Top Job Trends 1.  HTML 5 MongoDB 2.  MongoDB Competitor 1 3.  iOS 4.  Android Competitor 2 5.  Mobile Apps 6.  Puppet Competitor 3 7.  Hadoop 8.  jQuery 9.  PaaS 51 10.  Social Media

×