NoSQL and MySQL webinar - best of both worlds
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

NoSQL and MySQL webinar - best of both worlds

on

  • 2,832 views

Copy of webinar slides delivered March 29th 2012

Copy of webinar slides delivered March 29th 2012

Statistics

Views

Total Views
2,832
Views on SlideShare
2,829
Embed Views
3

Actions

Likes
2
Downloads
74
Comments
0

2 Embeds 3

http://www.slashdocs.com 2
http://duckduckgo.com 1

Accessibility

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

NoSQL and MySQL webinar - best of both worlds Presentation Transcript

  • 1. NoSQL Access to MySQL: The Best of Both Worlds Andrew Morgan – andrew.morgan@oracle.com – www.clusterdb.com MySQL Product Management1 29th March 2012 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  • 2. The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle s products remains at the sole discretion of Oracle.2 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  • 3. Session Agenda •  NoSQL – What are people looking for? •  RDBMS – What advantages do they still have? •  How MySQL Delivers the Best of Both Worlds –  MySQL Cluster •  NoSQL attributes: Scale-out, performance, ease-of-use, schema flexibility, on-line operations •  NoSQL APIs –  Key-Value store access to InnoDB (Memcached)3 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  • 4. NoSQL – Why is it needed? 2.1BN USERS 8X DATA GROWTH IN 5 YRS 850M USERS •  Web applications 70+ NEW DOMAINS EVERY 60 SECONDS demanding: 20M APPS PER DAY 40% DATA GROWTH PER YEAR 1 TR VIDEO PLAYBACKS –  Development velocity: $1TR BY 2014 Simplicity & flexibility of data250m TWEETS PER DAY $700BN IN 2011 model & APIs –  Scalability & performance: 5.9BN MOBILE SUBS IN 2010 (78% PENETRATION) high write throughput and 1 BILLION Ios & ANDROIDAPPS DOWNLOADED PER WEEK 370K CALL MINUTES EVERY 60 SECONDS Key/Value access –  Support for “Big Data” 4 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  • 5. What NoSQL must deliver•  Massive scalability –  No application-level sharding•  Performance Scalability  •  High Availability/Fault Tolerance Performance  •  Ease of use HA   –  Simple operations/administration Ease  of  use   –  Simple APIs –  Quickly evolve application & schema 5 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  • 6. Types of NoSQL stores Key-­‐Value   Document   Graph  • Cassandra   • MongoDB   • Neo4J  • Memcached   • CouchDB   • FlockDB  • BigTable  • Hadoop  • Voldermort  6 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  • 7. Still a role for the RDBMS? •  No best single solution fits all •  Mix and matchNoSQL   RDBMS   Simple  access  paGerns   Complex  queries  with  joins   Scalability   Performance   Compromise  on  consistency   ACID  transacJons   HA   for  performance   Ease  of  use   Ad-­‐hoc  data  format   Well  defined  schemas   SQL/Joins   Simple  operaJon   Rich  set  of  tools   ACID  Transac>ons   7 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  • 8. MySQL Cluster introduction8 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  • 9. MySQL Cluster Architecture REST Scalability   Application Nodes Performance   HA   Node Group 1 Node Group 2 Ease  of  use   Node 1 F1 F2 Node 3Cluster Cluster SQL/Joins   "Mgr F3 F4 Mgr ACID  Transac>ons   " Node 2 Node 4 F3 F4 F1 F2 Data Nodes 9 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  • 10. MySQL Cluster Architecture http://clusterdb.com/u/demo REST Scalability   Application Nodes Performance   HA   " Node Group 1 Node Group 2 Ease  of  use   Node 1 F1 F2 Node 3Cluster Cluster SQL/Joins   "Mgr F3 F4 Mgr ACID  Transac>ons   " Node 2 Node 4 F3 F4 F1 F2 Data Nodes 10 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  • 11. Scale-Out: Auto-Partitioning Table T1 Data Node 1 P1 Data Node 2 P2 P3 Data Node 3 P4 Data Node 411 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  • 12. Scale-Out: Auto-Partitioning Table T1 Data Node 1 F1 P1 Data Node 2 P2 P3 Data Node 3 P4 Data Node 412 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  • 13. Scale-Out: Auto-Partitioning Table T1 Data Node 1 F1 P1 Data Node 2 P2 F1 P3 Data Node 3 P4 Data Node 413 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  • 14. Scale-Out: Auto-Partitioning Table T1 Data Node 1 F1 P1 Data Node 2 P2 F3 F1 P3 Data Node 3 P4 Data Node 414 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  • 15. Scale-Out: Auto-Partitioning Table T1 Data Node 1 F1 F3 P1 Data Node 2 P2 F3 F1 P3 Data Node 3 P4 Data Node 415 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  • 16. Scale-Out: Auto-Partitioning Table T1 Data Node 1 F1 F3 P1 Node Group 1 Data Node 2 P2 F3 F1 P3 Data Node 3 F2 F4 P4 Node Group 2 Data Node 4 F4 F216 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  • 17. Scale-Out: Auto-Partitioning Table T1 Data Node 1 F1 F3 P1 Node Group 1 Data Node 2 P2 F3 F1 P3 Data Node 3 F2 F4 P4 Node Group 2 Data Node 4 F4 F217 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  • 18. Scale-Out: Auto-Partitioning Table T1 Scalability   " P1 Performance   P2 HA   " Ease  of  use   P3 SQL/Joins   " P4 ACID  Transac>ons   "18 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  • 19. Scale-Out Reads & Writes on Commodity Hardware SELECT Queries per Minute 1,200 1,000 800Millions 600 400 200 0 2 4 8 Number of Data Nodes •  8 x Commodity Intel Servers UPDATE Queries per Minute –  2 x 6-core processors 2.93GHz 150 –  x5670 processors (24 threads) 100 –  48GB RAM Millions 50 •  Infiniband networking 0 •  flexAsynch benchmark (NDB API) 4 8 Number of Data Nodes 19 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  • 20. JOIN Performance 70x More performance Query Execution Time Seconds100 •  33K rows over 11 tables 90 80 •  Must Analyze tables for best 70 results 60 50 mysql> ANALYZE TABLE <tab-name>; 40 30 Scalability   " 20 10 Performance   " 0 HA   " MySQL Cluster 7.1 MySQL Cluster 7.2 Ease  of  use   SQL/Joins   " ACID  Transac>ons   " 20 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  • 21. Creating & running your first Cluster The traditional way (pre-MCM) – Up and running in 15 mins       Download  &   Configure   Start  processes   Extract   •  Cluster-­‐wide   •  Management  Nodes   •  edelivery.oracle.com   “config.ini”   •  Data  Nodes   •  www.mysql.com   •  Per-­‐mysqld  “my.cnf”   •  MySQL  Servers   •  dev.mysql.com  •  Up & running in 10-15 minutes using Quick Start guides from http://dev.mysql.com/downloads/cluster/ –  Versions for Linux, Windows & Solaris 21 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  • 22. Scalability   " Performance   " HA   " Ease  of  use   " SQL/Joins   " ACID  Transac>ons   "22 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  • 23. MySQL Cluster Manager Bootstrap single host Cluster 1.  Download MCM/Cluster package from edelivery.oracle.com: 2.  Unzip 3.  Run agent, define, create & start Cluster! $> binmcmd –bootstrap MySQL Cluster Manager 1.1.2 started Connect to MySQL Cluster Manager by running "D:AndrewDocumentsMySQLmcmbinmcm" -a NOVA:1862 Configuring default cluster mycluster... Starting default cluster mycluster... Cluster mycluster started successfully ndb_mgmd NOVA:1186 ndbd NOVA ndbd NOVA mysqld NOVA:3306 mysqld NOVA:3307 ndbapi * Connect to the database by running "D:AndrewDocumentsMySQLmcmclusterbinmysql" -h NOVA -P 3306 -u root23 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  • 24. MCM: Upgrade Cluster mysql> upgrade cluster --package=7.1 mycluster;24 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  • 25. MCM: On-Line Add-Node Scale out with no loss of service mysql> add hosts --hosts=192.168.0.35,192.168.0.36 mysite; mysql> add package --basedir=/usr/local/mysql_7_1_9a –hosts=192.168.0.35,192.168.0.36 7.1; mysql> add process --processhosts=mysqld@192.168.0.33,mysqld@192.168.0.34,ndbd@192.168.0.35,ndbd@192.168.0.36 mycluster; mysql> start process --added mycluster; 25 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  • 26. On-line Schema changes On-Line Operations •  Scale the cluster (add & remove nodes on-line) •  Repartition tables •  Upgrade / patch servers & OS •  Upgrade / patch MySQL Cluster •  Back-Up •  Evolve the schema on-line, in real-time26 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  • 27. Scalability   " Performance   " HA   " Ease  of  use   " SQL/Joins   " ACID  Transac>ons   "27 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  • 28. NoSQL Access to MySQL Cluster data Mix & Match! Same data accessed simultaneously through SQL & NoSQL interfaces28 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  • 29. MySQL Cluster 7.1: ClusterJ/JPA •  New Domain Object Model Persistence API (ClusterJ) : –  Java API –  High performance, low latency –  Feature rich •  JPA interface built upon this new Java layer: –  Java Persistence API compliant •  Implemented as an OpenJPA plugin –  Uses ClusterJ where possible, reverts to JDBC for some operations –  Higher performance than JDBC –  More natural for most Java designers –  Easier Cluster adoption for web applications29 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  • 30. ClusterJ•  High Performance, Easy to Use•  In the style of Hibernate / JPA / JDO•  Domain Object Model DataMapper pattern –  Data is represented as domain objects –  Domain objects are separate from business logic –  Domain objects are mapped to database tables•  Built on ndbjtie –  JNI adapter –  integral part of MySQL Cluster –  Straight mapping of MySQL Cluster API (a.k.a NDB API) to Java•  Does not support relationships –  Look at JDO / JPA for these modelling patterns 30 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  • 31. NoSQL with Memcached (MySQL Cluster 7.2) •  Memcached is a distributed memory based hash-key/ value store with no persistence to disk •  NoSQL, simple API, popular with developers •  MySQL Cluster already provides scalable, in-memory performance with NoSQL (hashed) access as well as persistence –  Provide the Memcached API but map to NDB API calls •  Writes-in-place, so no need to invalidate cache •  Simplifies architecture as caching & database integrated into 1 tier •  Access data from existing relational tables31 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  • 32. Traditional Memcached Architecture Two levels of hashing httpd memcached hash keyPHP/Perl ver memcached pick ser to find data key toMemcache hash friends:12389! memcached memcache key 32 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  • 33. Cache hit httpd hash key PHP/Perl se rver memcached to find data ke yto pickMemcache hash friends:12389! VALUE friends:12389 0 31rn! 101, 11009, 11150, 55881, 77798 rn! 33 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  • 34. Cache miss (1): fetch from DB httpd hash keyPHP/Perl ver memcached memcache pick ser to find data key toMemcache hash FOUN D NOT mysql SELECT fri FROM u end_id ! MySQL ser_frien WHERE ds ! user_id = Slave ?!34 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  • 35. Cache miss (2): manage cache httpd hash keyPHP/Perl memcached to find dataMemcache ! ! 31 rn 798 rn s:12389 5881, 77 riend , 11150, 5 set f 009 11 101,35 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  • 36. Data change (1): Write to DB httpdPHP/Perl mysql INSERT INTO user_friends ! (user_id, friend_id) ! VALUES ( 12389, 999101);! MySQL Master36 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  • 37. Data change (2): manage cache httpdPHP/Perl memcached mysql rn! s:1 2389 te f riend dele MySQL Master37 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  • 38. NoSQL with Memcached•  Flexible: set maidenhead 0 0 3 SL6 –  Deployment options STORED –  Multiple Clusters –  Simultaneous SQL Access get maidenhead –  Can still cache in Memcached server VALUE maidenhead 0 3 –  Flat key-value store or map to multiple tables/ SL6 columns END 38 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  • 39. Schema-Free apps •  Rapid application evolution –  New types of data constantly added –  No time to get schema extended –  Missing skills to extend schema –  Initially roll out to just a few users –  Constantly adding to live system39 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  • 40. Cluster & Memcached – Schema-Free key value <town:maidenhead,SL6> Application view SQL view key value <town:maidenhead,SL6> Key   Value   town:maidenhead   SL6   generic table40 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  • 41. Cluster & Memcached - Configured Schema key value <town:maidenhead,SL6> Application view SQL view prefix key value <town:maidenhead,SL6>Prefix   Table   Key-­‐col   Val-­‐col   policy   town   ...   code   ...  town:   map.zip   town   code   cluster   maidenhead   ...   SL6   ...   Config tables map.zip41 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  • 42. Memcached with MySQL Cluster Try it out http://clusterdb.com/u/memcached42 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  • 43. Memcached NoSQL Access with InnoDB •  Memcached as a plugin of MySQL Server; same process space, with very low latency Application access to data •  Memcapable: supports both memcached SQL Memcached protocol ascii protocol and binary protocolmysqld •  Support multiple columns: users can map multiple columns into “value” memcached plugin MySQL Server •  Optional local caching: “innodb-only”, local cache “cache-only”, and “caching” innodb_memcache (optional) •  Batch operations for performance Handler API InnoDB API •  Available from labs.mysql.com. Is not GA. InnoDB Storage Engine 43 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  • 44. Which API to use?44 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  • 45. Scalability   " Performance   " HA   " Ease  of  use   " SQL/Joins   " ACID  Transac>ons   "45 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  • 46. Next Steps •  Guide to MySQL and NoSQL - Delivering the Best of Both Worlds –  http://mysql.com/why-mysql/white-papers/mysql-wp-guide-to- nosql.php •  Evaluate MySQL Cluster 7.2 –  http://www.mysql.com/downloads/cluster/ •  Bootstrap a Cluster –  https://edelivery.oracle.com/ •  Try Memcached API for InnoDB –  http://labs.mysql.com/46 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  • 47. Summary Today’s web workloads demand more from databases Performance, scale-out, simples access patterns & APIs MySQL meets these needs while still delivering benefits of an ACID RDBMS47 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  • 48. MySQL Cluster 7.2 GA Enabling  Next  GeneraJon  Web  Services   •  70x  Higher  Complex  Query  Performance:  AdapJve  Query  LocalizaJon   •  NaJve  Memcached  API   •  MySQL  5.5  Server  IntegraJon   •  MulJ-­‐Threaded  Data  Node  Extensions   •  VM  Support     Enhancing  Cross  Data  Center  Scalability   •  MulJ-­‐Site  Clustering   •  Simplified  AcJve  /  AcJve  ReplicaJon   Ease  of  Use   •  MySQL  Cluster  Manager  1.1.4   •  Consolidated  Privileges  48 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.