Dave StokesMySQL Community ManagerDavid.Stokes@Oracle.com                SQL & NoSQL:                How Big Data & MySQL ...
Program Agenda    •          WHY SQL / NoSQL    •          Alternatives to SQL    •          Big Data    •          Best o...
SQL – Cod and Date    Images from Wikipedia.com3    Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
Codd & Date                                                                           Wikipedia: Edgar Frank "Ted"        ...
Codds Relational Model    The purpose of the relational model is to    provide a declarative method for specifying    data...
Not all data relational or easy to    extract using SQL              Common NoSQL uses    ●         Document Stores / Fuzz...
NoSQL    ●         Database management without relational              model, schema free    ●         Does not use SQL (s...
Common NoSQL Types    ●         Key value stores    ●         Document databases    ●         Graph databases    ●        ...
Infobrights Emerging Database Landscape       http://bit.ly/emerging_db_landscape                                         ...
An example using MongoDB               db.cars.insert(               {make: Ford,               model: F-150,             ...
Hadoop               Part 1 – Reliable                                            Part 2 – Parallel               data sto...
Tilting relational databases on their side               Columnar databases align by column, not               rows     ● ...
When not to use NoSQL               Your data is                   ●         Relational                                 – ...
memcached before MySQL 5.6                                                                                memcached       ...
MySQLs use of memcached for NoSQL     ●         Innodb or NDB storage engines     ●         Access same data (same disks) ...
Diagrammatic Overview16   Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
Why this is cool?     ●          memcached as a daemon plugin of mysqld: both mysqld and              memcached are runnin...
Why is this cool? 2     ●         Optional local caching: three options – “cache-only”, “innodb-               only”, and ...
PHP Example of using memcachedfunction get_foo(int userid) {                                                function get_f...
Example InnoDB/memcached     ●        SQL                  ●         mysql> INSERT INTO demo_test                         ...
Getting it running     ●         Install MySQL 5.6.6     ●         /scripts/innodb_memcached_config.sql                   ...
MySQL Cluster –     NDB and/or memcahced22   Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
MySQL Cluster quick review                                                                            Fault tolerant, auto...
Option 1 co-locate the memcached API     with the data nodes                                                              ...
Separate Layer                                                                             For maximum                    ...
Co locate with Application                                                                             Another simple opti...
In all of the last three examples,     there has been a single source for     the data (it’s all in MySQL Cluster)        ...
OTHER OPTIONS     •           There are other options for Big Data           and NoSQL that are beyond the     •          ...
David.Stokes@Oracle.com@Stoker                                                                                            ...
Sql no sql
Upcoming SlideShare
Loading in …5
×

Sql no sql

1,160 views

Published on

SQL versus NoSQL and how MySQL's InnoDB/Memcached interface can keep you sane.

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,160
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
44
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Sql no sql

  1. 1. Dave StokesMySQL Community ManagerDavid.Stokes@Oracle.com SQL & NoSQL: How Big Data & MySQL Work Together Presenting with David.Stokes@Oracle.Com LOGO 1 MySQL Community Manager Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 8
  2. 2. Program Agenda • WHY SQL / NoSQL • Alternatives to SQL • Big Data • Best of both worlds – InnoDB/memcached • MySQL Cluster – NDB/memcached • Q&A Synopsis – How to use MySQL as a relational data store according to Codd & Date while gaining the ability to access schema-less data and looking cool while doing it.2 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
  3. 3. SQL – Cod and Date Images from Wikipedia.com3 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
  4. 4. Codd & Date Wikipedia: Edgar Frank "Ted" Codd was an English computer scientist who invented the relational model for database management, the theoretical basis for relational databases. ...Codd continued to develop and extend his relational model, sometimes in collaboration with Chris Date4 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
  5. 5. Codds Relational Model The purpose of the relational model is to provide a declarative method for specifying data and queries: users directly state what information the database contains and what information they want from it, and let the database management system software take care of describing data structures for storing the data and retrieval procedures for answering queries. Wikipedia5 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
  6. 6. Not all data relational or easy to extract using SQL Common NoSQL uses ● Document Stores / Fuzzy schemas ● A Facebook Query – find the friends of your friends … and then their friends ● Data size may be too large for RDMS or OS ● Coolness factor6 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
  7. 7. NoSQL ● Database management without relational model, schema free ● Does not use SQL (some retrofitting) ● Usually not ACID ● Eventually consistent data ● Distributed, fault-tolerant ● Large amounts of data7 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
  8. 8. Common NoSQL Types ● Key value stores ● Document databases ● Graph databases ● XML databases ● Distributed peers ● Object stores8 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
  9. 9. Infobrights Emerging Database Landscape http://bit.ly/emerging_db_landscape Row Based Columnar NoSQL – Key Value NoSQL – Dcoument NoSQL Columnar store Store StoreCommon uses Transaction Historical data Cache for storing Web apps or app Real-time data processing analysis, data frequently needing scaling w/o logging warehousing, BI requested data defined schemaBasic Description Data structured in Data structured in Data stored in Persistent storage, Very large data rows columns memory some SQL-like storage, querying MapReduce supportStrengths Capturing/inputting Fast query support Scalability, very fast Persistent store, Very high new records; on data sets; storage/retrieval of scalable; better throughput; strong Robust, proven compression data query support than partitioning; random technology key-value stores read-write accessWeaknesses Scale issues Import/export Usually all data Lack of Low level API; speed; heavy must fit into sophisticated Inability to do computing resource memory, no capabilities complex queries; needed complex queries high query latencyTypical Database Several GB to 50 Several GB to Several TB to Several TB toSize Range TB several TB several PB several PBKey Players MySQL, Oracle, Infobright, Aster Memacached, MongoDB, Hbase, Big Table, SQL Server, Sybase Data, Sybase IQ, Amazon S3, Redis, CouchDB, Cassandra ASE Vertica, ParAccel Voldemort SimpleDB 9 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
  10. 10. An example using MongoDB db.cars.insert( {make: Ford, model: F-150, cylinders: 8}) db.cars.find( cylinders: {$gte: 8})10 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
  11. 11. Hadoop Part 1 – Reliable Part 2 – Parallel data storage using data processing the Hadoop using map/reduce Distributed File System (HDFS) ● Can get expensive for hardware, not for all data11 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
  12. 12. Tilting relational databases on their side Columnar databases align by column, not rows ● High compression possible ● OLAP & Data Warehousing ● MySQL engines ● Calponts InfiniDB ● Infobright12 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
  13. 13. When not to use NoSQL Your data is ● Relational – Some hierarchy – schema ● Need ACID ● Do not like lots of servers, disk farms13 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
  14. 14. memcached before MySQL 5.6 memcached is a general-purpose distributed memory caching system --Wikipedia14 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
  15. 15. MySQLs use of memcached for NoSQL ● Innodb or NDB storage engines ● Access same data (same disks) either through SQL or memcached ● 1,000,000,000+ transactions a minute for MySQL Cluster ● Many sites already using memcached - already in use, well known, easy to implement15 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
  16. 16. Diagrammatic Overview16 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
  17. 17. Why this is cool? ● memcached as a daemon plugin of mysqld: both mysqld and memcached are running in the same process space, with very low latency access to data ● Direct access to InnoDB: bypassing SQL parser and optimizer ● Support standard protocol (memcapable): support both memcached text-based protocol and binary protocol; all 55 memcapable tests are passed ● Support multiple columns: users can map multiple columns into “value”. The value is separated by a pre-defined “separator” (configurable).17 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
  18. 18. Why is this cool? 2 ● Optional local caching: three options – “cache-only”, “innodb- only”, and “caching” (both “cache” and “innodb store”). These local options can apply to each of four Memcached operations (set, get, delete and flush). ● Batch operations: user can specify the batch commit size for InnoDB memcached operations via “daemon_memcached_r_batch_size” and “daemon_memcached_w_batch_size” (default 32) ● Support all memcached configure options through MySQL configure variable “daemon_memcached_option18 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
  19. 19. PHP Example of using memcachedfunction get_foo(int userid) { function get_foo(int userid) { data = db_select("SELECT * FROM /* first try the cache */users WHERE userid = ?", userid); data = memcached_fetch("userrow:" + userid); return data; if (!data) {} /* not found : request database */ data = db_select("SELECT * FROM users WHERE userid = ?", userid); Rewriten to /* then store in cache until next get */ use memcached_add("userrow:" + userid, data); memached } return data; } 19 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
  20. 20. Example InnoDB/memcached ● SQL ● mysql> INSERT INTO demo_test VALUES (dave,it works, 10, 200, NULL)g ● Memcached20 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
  21. 21. Getting it running ● Install MySQL 5.6.6 ● /scripts/innodb_memcached_config.sql ● Creates test.demo_test – Key (c1) – CHAR/VARCAHR – Value (c2) – CHAR/VARCHAR – Flag (c3) – 32bit Integer – CAS (c4) – 64bit Integer – Exp (c5) – 32bit integer ● mysql> install plugin daemon_memcached soname “libmemcached.so”; ● mysql> set session TRANSACTION ISOLATION LEVEL read uncommitted; /* ignore batches */21 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
  22. 22. MySQL Cluster – NDB and/or memcahced22 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
  23. 23. MySQL Cluster quick review Fault tolerant, auto sharding, shared nothing, data on redundant boxes, 99.999% up time, ACID, geographical replication between clusters, & no single point of failure23 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
  24. 24. Option 1 co-locate the memcached API with the data nodes The applications can connect to any of the memcached API nodes – if one should fail just switch to another as it can access the exact same data instantly. As you add more data nodes you also add more memcached servers and so the data access/storage layer can scale out (until you hit the 48 data node limit).24 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
  25. 25. Separate Layer For maximum flexibility, you can have a separate Memcached layer so that the application, the Memcached API & MySQL Cluster can all be scaled independently.25 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
  26. 26. Co locate with Application Another simple option is to co- locate the Memcached API with the application. In this way, as you add more application nodes you also get more Memcached throughput. If you need more data storage capacity you can independently scale MySQL Cluster by adding more data nodes. One nice feature of this approach is that failures are handled very simply – if one App/Memcached machine should fail, all of the other applications just continue accessing their local Memcached API.26 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
  27. 27. In all of the last three examples, there has been a single source for the data (it’s all in MySQL Cluster) . ● If you choose, you can still have all or some of the data cached within the memcached server (and specify whether that data should also be persisted in MySQL Cluster) – you choose how to treat different pieces of your data. If for example, you had some data that is written to and read from frequently then store it just in MySQL Cluster, if you have data that is written to rarely but read very often then you might choose to cache it in memcached as well and if you have data that has a short lifetime and wouldn’t benefit from being stored in MySQL Cluster then only hold it in memcached. The beauty is that you get to configure this on a per-key-prefix basis (through tables in MySQL Cluster) and that the application doesn’t have to care – it just uses the memcached API and relies on the software to store data in the right place(s) and to keep everything in sync. ● Of course if you want to access the same data through SQL then you’d make sure that it was configured to be stored in MySQL Cluster.27 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
  28. 28. OTHER OPTIONS • There are other options for Big Data and NoSQL that are beyond the • Scope of this presentation .. • Although I can not think of • Any thing to point to as an • Example :-) ORACLE PRODUCT LOGO28 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 8
  29. 29. David.Stokes@Oracle.com@Stoker Q&Aslideshare.net/davestokes/presentations 29 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 8

×