OUG Scotland 2014 - NoSQL and MySQL - The best of both worlds
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

OUG Scotland 2014 - NoSQL and MySQL - The best of both worlds

on

  • 595 views

Understand how you can get the benefits you're looking for from NoSQL data stores without sacrificing the power and flexibility of the world's most popular open source database - MySQL.

Understand how you can get the benefits you're looking for from NoSQL data stores without sacrificing the power and flexibility of the world's most popular open source database - MySQL.

Statistics

Views

Total Views
595
Views on SlideShare
572
Embed Views
23

Actions

Likes
2
Downloads
9
Comments
0

2 Embeds 23

https://twitter.com 21
http://tweetedtimes.com 2

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

OUG Scotland 2014 - NoSQL and MySQL - The best of both worlds Presentation Transcript

  • 1. NoSQL & MySQL: The Best of Both Worlds Andrew Morgan (@andrewmorgan) www.clusterdb.com Principal MySQL Product Manager 11th June 2014
  • 2. Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.2 Safe Harbour Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.
  • 3. Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.3 MySQL Customers World’s Most Popular Open Source Database WebSaaS, Hosting Enterprise OEM / ISV’s Telecom
  • 4. Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.4
  • 5. Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.5 Session Agenda  NoSQL – What are people looking for?  RDBMS – What advantages do they still have?  How MySQL Delivers the Best of Both Worlds – MySQL Cluster  NoSQL attributes: Scale-out, performance, ease-of-use, schema flexibility, on-line operations  NoSQL APIs – Key-Value store access to InnoDB (Memcached)
  • 6. Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.6 Types of NoSQL stores Key-Value • Cassandra • Memcached • BigTable • Hadoop • Voldermort Document • MongoDB • CouchDB Graph • Neo4J • FlockDB
  • 7. Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.7
  • 8. Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.8
  • 9. Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.9 Graph Database
  • 10. Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.10 What NoSQL must deliver  Massive scalability – No application-level sharding  Performance  High Availability/Fault Tolerance  Ease of use – Simple operations/administration – Simple APIs – Quickly evolve application & schema Scalability Performance HA Ease of use
  • 11. Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.11
  • 12. Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.12 Still a role for the SQL (RDBMS)? NoSQL Simple access patterns Compromise on consistency for performance Ad-hoc data format Simple operation SQL Complex queries with joins ACID transactions Well defined schemas Rich set of tools  No best single solution fits all  Mix and match Scalability Performance HA Ease of use SQL/Joins ACID Transactions
  • 13. Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.13 The Temptation to Jettison Relational Model  Relational Model: – Data stored in multiple tables – Many lookups to retrieve a user’s timeline – Is it worth the effort in setting up this complex data model?  Simpler just to store as one document? The allure of document stores Examples borrowed from @sarahmei https://speakerdeck.com/sarahmei/switching-data-stores-a-postmodern-comedy user friend post comment like liker commenter many many many one one many
  • 14. Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.14 The Temptation to Jettison Relational Model  Document Model: – Entire timeline in a single document (row) – Single lookup to retrieve the user’s timeline – Brilliantly efficient model when the document truly contains self-contained information  Like a real-world document! The allure of document stores Examples borrowed from @sarahmei https://speakerdeck.com/sarahmei/switching-data-stores-a-postmodern-comedy {name: ‘Joe’, url: ‘…’ stream:[ {friend:{ name: ‘Jane’, posts:[{content: ‘…’, comments:[ {comment: ‘…’, commenter: ‘Joe’}, {…} ], likes: [‘Joe’, ‘Fred’, ‘Billy’] }, {…},{…},… ] }, {…},{…},… }
  • 15. Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.15
  • 16. Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.16 The Temptation to Jettison Relational Model  These are all people who have their own data that users will want to view or click through: – Name – url – Timeline  Easy to represent with FKs in a relational model But when the data isn’t self contained… Examples borrowed from @sarahmei https://speakerdeck.com/sarahmei/switching-data-stores-a-postmodern-comedy user friend post comment like liker commenter many many many one one many
  • 17. Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.17 The Temptation to Jettison Relational Model  Do I store all data for all friends and likers again at every point they appear in the document? – Massive amount of repeated data  Wasted space  Needs to be kept in sync (and you don’t have transactions to provide consistency The allure of document stores Examples borrowed from @sarahmei https://speakerdeck.com/sarahmei/switching-data-stores-a-postmodern-comedy {name: ‘Joe’, url: ‘…’ stream:[ {friend:{ name: ‘Jane’, posts:[{content: ‘…’, comments:[ {comment: ‘…’, commenter: ‘Joe’}, {…} ], likes: [‘Joe’, ‘Fred’, ‘Billy’] }, {…},{…},… ] }, {…},{…},… }
  • 18. Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.18
  • 19. Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.19 The Temptation to Jettison Relational Model The allure of document stores Examples borrowed from @sarahmei https://speakerdeck.com/sarahmei/switching-data-stores-a-postmodern-comedy {name: 83746251, url: ‘…’ stream:[ {friend:{ name: 9384726153, posts:[{content: ‘…’, comments:[ {comment: ‘…’, commenter: 83746251}, {…} ], likes: [83746251, 750730283, 2938493820] }, {…},{…},… ] }, {…},{…},… }  The reality is that the developer will store the user- ids instead  Developer is responsible for implementing ‘joins’ in the application – e.g. scanning all timelines for where a user made a comment or liked a post  No transactions to ensure consistency between documents
  • 20. Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.20
  • 21. Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.21 MySQL Cluster: Overview • Auto-Sharding, Multi-Master • ACID Compliant, OLTP + Real-Time Analytics HIGH SCALE, READS + WRITES • Shared nothing, no Single Point of Failure • Self Healing + On-Line Operations 99.999% AVAILABILITY • Open Source + Commercial Editions • Commodity hardware + Management, Monitoring Tools LOW TCO • Key/Value + Complex, Relational Queries • SQL + Memcached + JavaScript + Java + JPA + HTTP/REST & C++ SQL + NoSQL • In-Memory Optimization + Disk-Data • Predictable Low-Latency, Bounded Access Time REAL-TIME
  • 22. Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.22 Who’s Using MySQL Cluster?
  • 23. Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.23 MySQL Cluster Architecture MySQL Cluster Data Nodes Data Layer Clients Application Layer Management
  • 24. Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.24 MySQL Cluster Architecture MySQL Cluster Data Nodes Data Layer Clients Application Layer Management Management
  • 25. Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.25
  • 26. Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.26 MySQL Cluster Architecture MySQL Cluster Data Nodes Data Layer Application Layer Management Management Clients
  • 27. Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.27
  • 28. Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.28 On-line Operations  Scale the cluster (add & remove nodes on-line)  Repartition tables  Upgrade / patch servers & OS  Upgrade / patch MySQL Cluster  Back-Up  Evolve the schema on-line, in real-time
  • 29. Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.29
  • 30. Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.30
  • 31. Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.31 Scale-Out Reads & Writes on Commodity Hardware  8 x Commodity Intel Servers – 2 x 6-core processors 2.93GHz – x5670 processors (24 threads) – 48GB RAM  Infiniband networking  flexAsynch benchmark (NDB API) 0 200 400 600 800 1,000 1,200 2 4 8 Millions Number of Data Nodes SELECT Queries per Minute 0 20 40 60 80 100 120 4 8 Millions Number of Data Nodes UPDATE Queries per Minute
  • 32. Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.32 JOIN Performance 33K rows over 11 tables Must Analyze tables for best results mysql> ANALYZE TABLE <tab-name>; 0 10 20 30 40 50 60 70 80 90 100 MySQL Cluster 7.1 MySQL Cluster 7.2 Query Execution Time Seconds Scalability a Performance a HA a Ease of use SQL/Joins a ACID Transactions a 70x More performance
  • 33. Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.33 MySQL Cluster 7.4  Performance gain over 7.3 – 47% (Read-Only) – 38% (Read-Write) Better performance and operational simplicity labs.mysql.com  Faster node restarts – Recovering nodes rejoin the cluster faster
  • 34. Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.34
  • 35. Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.35 Creating & running your first Cluster The traditional way (pre-MCM & Auto-Installer) Download & Extract •edelivery.oracle.com •www.mysql.com •dev.mysql.com Configure •Cluster-wide “config.ini” •Per-mysqld “my.cnf” Start processes •Management Nodes •Data Nodes •MySQL Servers
  • 36. Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.36 Scalability a Performance a HA a Ease of use a SQL/Joins a ACID Transactions a
  • 37. Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.37
  • 38. Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.38 MySQL Cluster 7.3: Auto-Installer  Fast configuration  Auto-discovery  Workload optimized  Repeatable best practices Specify Workload Auto- Discover Define TopologyDeploy
  • 39. Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.39
  • 40. Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.40 MySQL Cluster Manager 1. Download MCM/Cluster package from edelivery.oracle.com: 2. Unzip 3. Run agent, define, create & start Cluster! $> binmcmd --bootstrap MySQL Cluster Manager 1.1.2 started Connect to MySQL Cluster Manager by running "D:AndrewDocumentsMySQLmcmbinmcm" -a NOVA:1862 Configuring default cluster 'mycluster'... Starting default cluster 'mycluster'... Cluster 'mycluster' started successfully ndb_mgmd NOVA:1186 ndbd NOVA ndbd NOVA mysqld NOVA:3306 mysqld NOVA:3307 ndbapi * Connect to the database by running "D:AndrewDocumentsMySQLmcmclusterbinmysql" -h NOVA -P 3306 -u root
  • 41. Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.41 MCM: Upgrade Cluster mysql> upgrade cluster --package=7.3 mycluster;
  • 42. Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.42 Scalability a Performance a HA a Ease of use a SQL/Joins a ACID Transactions a
  • 43. Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.43
  • 44. Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.44
  • 45. Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.45 NoSQL Access to MySQL Cluster data ClusterJ MySQL JDBC Apps JPA JNI Python Ruby ClusterJPA Apps Apps Apps Apps Apps Node.js JS Apps mod-ndb Apache Apps ndb-eng Memcached Apps Apps NDB API (C++) MySQL Cluster Data Nodes Apps PHP PERL Apps
  • 46. Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.46 Schema-Free apps  Rapid application evolution – New types of data constantly added – No time to get schema extended – Missing skills to extend schema – Initially roll out to just a few users – Constantly adding to live system
  • 47. Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.47 Cluster & Memcached – Schema-Free <town:maidenhead,SL6> key value <town:maidenhead,SL6> key value Key Value town:maidenhead SL6 generic table Application view SQL view
  • 48. Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.48 Cluster & Memcached - Configured Schema <town:maidenhead,SL6> prefix key value <town:maidenhead,SL6> key value Prefix Table Key-col Val-col policy town: map.zip town code cluste r Config tables town ... code ... maidenhead ... SL6 ... map.zip Application view SQL view
  • 49. Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.49 MySQL 5.6 Memcached with InnoDB 0 10000 20000 30000 40000 50000 60000 70000 80000 8 32 128 512 TPS Client Connections MySQL 5.6: NoSQL Benchmarking Memcach ed API SQL Clients and Applications MySQL Server Memcached Plug-in innodb_ memcached local cache (optional) Handler API InnoDB API InnoDB Storage Engine mysqld process SQL Memcached Protocol Up to 9x Higher “SET / INSERT” Throughput
  • 50. Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.50 MySQL Cluster 7.3: Node.js NoSQL API  Native JavaScript access to MySQL Cluster – End-to-End JavaScript: browser to the app & DB – Storing and retrieving JavaScript objects directly in MySQL Cluster – Eliminate SQL transformation  Implemented as a module for node.js – Integrates Cluster API library within the web app  Couple high performance, distributed apps, with high performance distributed database  Optionally routes through MySQL Server V8 JavaScript Engine MySQL Cluster Node.js Module MySQL Cluster Data Nodes Clients
  • 51. Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.51 MySQL Cluster NoSQL API for Node.js & FKs  FKs enforced on all APIs: { message: 'Error', sqlstate: '23000', ndb_error: null, cause: { message: 'Foreign key constraint violated: No parent row found [255]', sqlstate: '23000', ndb_error: { message: 'Foreign key constraint violated: No parent row found', code: 255, classification: 'ConstraintViolation', handler_error_code: 151, status: 'PermanentError' }, cause: null } }
  • 52. Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.52 Next Steps Learn More • www.mysql.com/cluster • Authentic MySQL Curriculum: http://oracle.com/education/mysql Try it Out • dev.mysql.com/downloads/cluster/ Let us know what you think • clusterdb.com • @clusterdb • forums.mysql.com/list.php?25
  • 53. Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.53