0
MySQL Cluster: NoSQL Memcached API
•  Meet New Requirements: Add persistence, sharding, fault tolerance•  Ease-of-Use: Standard Memcached libraries•  Simplif...
Program Agenda•  Overview, MySQL Cluster                          <Insert Picture Here>•  Memcached API   •  Memcache Over...
MySQL Cluster HIGH SCALE,     •  Auto-Sharding, Multi-MasterREADS + WRITES   •  ACID Compliant, OLTP + Real-Time Analytics...
MySQL Cluster 7.2                          70x Faster JOINs                         8x Higher Per Node                    ...
Memcached API            Copyright © 2012 Oracle Corp.
Memcached Overview                                                           memcachedApplication                         ...
Memcache Protocol Elements•  Key           Up to 250 characters•  Expire Time   A number in seconds, up to 30 days•  Flags...
Memcache Protocol Commands (1)•  get key                    Fetch value for this key•  set key value              Store th...
Memcache Protocol Commands (2)•  append key value           Append text to current value•  prepend key value          Inse...
Traditional architecture:   Memcached & MySQLApplication                       t!                         ro  ughpu       ...
NDB & Memcache Architecture:  Memcache protocol + NDB storage                                                MySQLApplicat...
Goals•  Access stored data directly from memcache client  •  Memcached perspective:      •  MySQL Cluster is a write-scala...
Measured Latency                  memcachetest -t 2 -M 7000 -c 25000 !             Copyright © 2012 Oracle Corp.
Configuration OverviewNDB Memcache                Copyright © 2012 Oracle Corp.
Anatomy of a Memcache key                 user:1248    the prefix                                     the database key    ...
A key-prefix mappingMemcache                                                       Cache   key            Cluster         ...
A memcache server role                        key                               Con-    Cache                             ...
Getting StartedMySQL Cluster and Memcached                     Copyright © 2012 Oracle Corp.
Cluster on a laptop•  Configuration file            cluster.ini•  Management server             ndb_mgmd•  Data node      ...
Cluster Configuration FileA single central file for the whole clusterLoaded by the management server on --initialAnd then ...
[MGM] section: Management Server[MGM]	NodeId=1	datadir=/Users/jdd/sandbox/ndb	hostname=localhost	                  Copyrig...
[DB DEFAULT] Section: data nodeparameters[DB DEFAULT]	NoOfReplicas=1	TimeBetweenGlobalCheckpoints=12000	HeartbeatIntervalD...
[NDBD] Sections: individual data node[NDBD]	NodeId=2	              Copyright © 2012 Oracle Corp.
[API] Sections for MySQL &Memcached servers[API]	[API]	[API]	[API]	[API]	[API]	[API]	              Copyright © 2012 Oracle...
Complete cluster.ini file [MGM]	 NodeId=1	 datadir=/Users/jdd/sandbox/ndb	 hostname=localhost	 	 [DB DEFAULT]	 NoOfReplica...
Start the Management Server% ndb_mgmd -f config-file-path	              Copyright © 2012 Oracle Corp.
Start the Data Node% ndbd	              Copyright © 2012 Oracle Corp.
Configure & start mysqld (my.cnf file)[mysqld]	ndbcluster	              Copyright © 2012 Oracle Corp.
Load NDB Memcache configuration  tables% cd /usr/local/mysql/share/memcache-api		% mysql -u root < ndb_memcache_metadata.s...
Sample Application                Copyright © 2012 Oracle Corp.
"Twitter-like" Application•  Two Tables      •  authors      •  tweets•  Three Perl Functions      •  Create a new author ...
Required Perl Modules% cpan install Cache::Memcached		% cpan install UUID::Tiny	                Copyright © 2012 Oracle Co...
Sample Application% mysql -u root		mysql> CREATE DATABASE demo;	       use demo;	                Copyright © 2012 Oracle C...
Data Definition: AuthorsCREATE TABLE authors (	  author varchar(15) not null,	  date_created timestamp, 	  tweets bigint u...
Data Definition: TweetsCREATE TABLE tweets ( 	id varchar(36) primary key,author varchar(15),time_stamp timestamp,tweet var...
Configuration: Use the ndbmemcacheconfig schemamysql> use ndbmemcache;	       show tables;	              Copyright © 2012 ...
Configuration: Create container &    key_prefixesINSERT into containers 	SET 	  name = tweets_table ,	  db_schema = demo ,...
Two key_prefixes for authors table•  newauthor:  •  ADD an author record and set date_created•  tweetcount:  •  INCR tweet...
Configuration: New AuthorINSERT into containers 	SET 	  name = author_new ,	  db_schema = demo , db_table = authors,	 	key...
Configuration: Tweet CountINSERT into containers 	SET 	  name = author_count,	  db_schema = demo, db_table = authors,	  ke...
Memcached: Start the server% cd /usr/local/mysql	% bin/memcached -E lib/ndb_engine.so 		17-Aug-2012 13:53:32 PDT NDB Memca...
Perl: Startup Codeuse Cache::Memcached;	use UUID::Tiny;		our $mc = new Cache::Memcached( {	  servers => [ "localhost:11211...
Perl: Add a new author## Takes one parameter: author	sub new_author {	   my $author = shift;	   my $time = time();		   my ...
Perl: Insert a Tweet## Take two parameters: author and tweet	sub insert_tweet {	   my $author = shift;	   my $tweet = shif...
Perl: Increment Tweet Count## Increment the authors tweet count.	## Take one parameter: author	sub increment_tweetcount {	...
Download MySQL Cluster + Memcached APIhttp://www.mysql.com/downloads/cluster/#downloads
MySQL Cluster CGEMost secure, scalable MySQL Database, Development /Management / Monitoring Tools, backed by Oracle Premie...
Oracle Premier Support for MySQL     Rely on The Experts - Get Unique Benefits•    Straight from the Source•    Largest Te...
Getting Started                                                     Learn More                                    Scaling ...
Summary                                        Scale Web Services with                                          99.999% Av...
MySQL Cluster Architecture                   JPA            REST                         Application Nodes          Node G...
MySQL Cluster - Extreme Resilience                   JPA                 REST                         Application Nodes   ...
MySQL Cluster NoSQL Memcached API
Upcoming SlideShare
Loading in...5
×

MySQL Cluster NoSQL Memcached API

4,341

Published on

MySQL Cluster 7.2 added support for the Memcached API, enabling web services to directly query MySQL Cluster using the Memcached API, and adding a persistent, scalable, fault tolerant backend to Memcached.

The slides take you through the design concepts and introduce a sample social media app built using memcached and MySQL Cluster

Published in: Technology
0 Comments
5 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
4,341
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
104
Comments
0
Likes
5
Embeds 0
No embeds

No notes for slide

Transcript of "MySQL Cluster NoSQL Memcached API"

  1. 1. MySQL Cluster: NoSQL Memcached API
  2. 2. •  Meet New Requirements: Add persistence, sharding, fault tolerance•  Ease-of-Use: Standard Memcached libraries•  Simplification: Eliminate cache-coherency & consolidate data tiers•  Flexible Data Model: Schema / Schema-less storage•  Flexible Access Patterns: SQL and NoSQL•  Flexible Deployments: values stored in Cluster, Memcached or both•  Key / Value Performance: Native access to MySQL Cluster Copyright © 2012 Oracle Corp.
  3. 3. Program Agenda•  Overview, MySQL Cluster <Insert Picture Here>•  Memcached API •  Memcache Overview •  MySQL Cluster (NDB) & Memcache setup •  Sample Application•  Resources to Get Started Copyright © 2012 Oracle Corp.
  4. 4. MySQL Cluster HIGH SCALE, •  Auto-Sharding, Multi-MasterREADS + WRITES •  ACID Compliant, OLTP + Real-Time Analytics 99.999% •  Shared nothing, no Single Point of Failure AVAILABILITY •  Self Healing + On-Line Operations •  High Load, Real Time Performance REAL-TIME •  Predictable Low-Latency, Bounded Access Time •  Key/Value + Complex, Relational Queries SQL + NoSQL •  SQL + Memcached + Java + JPA + HTTP/REST + C++ •  Open Source + Commercial Editions LOW TCO •  Commodity hardware + Management, Monitoring Toos Copyright © 2012 Oracle Corp.
  5. 5. MySQL Cluster 7.2 70x Faster JOINs 8x Higher Per Node Performance Fastest Ramp…Ever NoSQL Memcached API Geo-Distributed Clusters writes
  6. 6. Memcached API Copyright © 2012 Oracle Corp.
  7. 7. Memcached Overview memcachedApplication hash key erver memcached to pick s to find data Memcache hash key Client friends:12389! memcached memcache key Copyright © 2012 Oracle Corp.
  8. 8. Memcache Protocol Elements•  Key Up to 250 characters•  Expire Time A number in seconds, up to 30 days•  Flags 32-bit number for application use•  CAS ID 64-bit incrementing version number on value•  Value Value stored for key; traditionally up to 1 MB. Copyright © 2012 Oracle Corp.
  9. 9. Memcache Protocol Commands (1)•  get key Fetch value for this key•  set key value Store this data•  add key value Store this data (strict insert)•  replace key value Store this data (strict update)•  cas key value cas-id Update with version number check Copyright © 2012 Oracle Corp.
  10. 10. Memcache Protocol Commands (2)•  append key value Append text to current value•  prepend key value Insert text before current value•  incr key amount Add to current stored integer value•  decr key amount Subtract from current stored value•  delete key Delete a key and value•  flush_all Delete all stored values on the server•  stats stat-name Fetch server statistics Copyright © 2012 Oracle Corp.
  11. 11. Traditional architecture: Memcached & MySQLApplication t! ro ughpu hi gher th tency! a lower l memcached Memcache Clientmysql client more sto more po rage (on MySQL werful p disk)! rotocol ( Slave SQL)! Copyright © 2012 Oracle Corp.
  12. 12. NDB & Memcache Architecture: Memcache protocol + NDB storage MySQLApplication Cluster memcached Data Node Memcache Client NDB Engine MySQL Cluster Data Node Copyright © 2012 Oracle Corp.
  13. 13. Goals•  Access stored data directly from memcache client •  Memcached perspective: •  MySQL Cluster is a write-scalable, replicated data store •  with reliable in-memory storage, •  plus on-disk storage when data is too big for memory. •  MySQL Cluster perspective: •  memcache is a high performance API •  providing easy access to in-memory data, •  plus an extra layer of caching when data is on disk. Copyright © 2012 Oracle Corp.
  14. 14. Measured Latency memcachetest -t 2 -M 7000 -c 25000 ! Copyright © 2012 Oracle Corp.
  15. 15. Configuration OverviewNDB Memcache Copyright © 2012 Oracle Corp.
  16. 16. Anatomy of a Memcache key user:1248 the prefix the database key Copyright © 2012 Oracle Corp.
  17. 17. A key-prefix mappingMemcache Cache key Cluster Container Policy prefix Copyright © 2012 Oracle Corp.
  18. 18. A memcache server role key Con- Cache Cluster prefix tainer Policy key Con- Cache Cluster prefix tainer Policy Server Role ID key Con- Cache Cluster prefix tainer Policy key Con- Cache Cluster prefix tainer Policy Copyright © 2012 Oracle Corp.
  19. 19. Getting StartedMySQL Cluster and Memcached Copyright © 2012 Oracle Corp.
  20. 20. Cluster on a laptop•  Configuration file cluster.ini•  Management server ndb_mgmd•  Data node ndbd•  MySQL server mysqld•  Memcached server memcached (All of these are present in MySQL Cluster 7.2 distribution) Copyright © 2012 Oracle Corp.
  21. 21. Cluster Configuration FileA single central file for the whole clusterLoaded by the management server on --initialAnd then stored in cache for future restartsSections: [MGM] [DB DEFAULT] [NDBD] [API] Copyright © 2012 Oracle Corp.
  22. 22. [MGM] section: Management Server[MGM] NodeId=1 datadir=/Users/jdd/sandbox/ndb hostname=localhost Copyright © 2012 Oracle Corp.
  23. 23. [DB DEFAULT] Section: data nodeparameters[DB DEFAULT] NoOfReplicas=1 TimeBetweenGlobalCheckpoints=12000 HeartbeatIntervalDbApi=15000 DataMemory=180M IndexMemory=60M Copyright © 2012 Oracle Corp.
  24. 24. [NDBD] Sections: individual data node[NDBD] NodeId=2 Copyright © 2012 Oracle Corp.
  25. 25. [API] Sections for MySQL &Memcached servers[API] [API] [API] [API] [API] [API] [API] Copyright © 2012 Oracle Corp.
  26. 26. Complete cluster.ini file [MGM] NodeId=1 datadir=/Users/jdd/sandbox/ndb hostname=localhost [DB DEFAULT] NoOfReplicas=1 TimeBetweenGlobalCheckpoints=12000 HeartbeatIntervalDbApi=15000 DataMemory=180M IndexMemory=60M [NDBD] NodeId=2 [API] [API] ... Copyright © 2012 Oracle Corp.
  27. 27. Start the Management Server% ndb_mgmd -f config-file-path Copyright © 2012 Oracle Corp.
  28. 28. Start the Data Node% ndbd Copyright © 2012 Oracle Corp.
  29. 29. Configure & start mysqld (my.cnf file)[mysqld] ndbcluster Copyright © 2012 Oracle Corp.
  30. 30. Load NDB Memcache configuration tables% cd /usr/local/mysql/share/memcache-api % mysql -u root < ndb_memcache_metadata.sql Copyright © 2012 Oracle Corp.
  31. 31. Sample Application Copyright © 2012 Oracle Corp.
  32. 32. "Twitter-like" Application•  Two Tables •  authors •  tweets•  Three Perl Functions •  Create a new author record •  Insert a new tweet •  Increment an authors tweet count Copyright © 2012 Oracle Corp.
  33. 33. Required Perl Modules% cpan install Cache::Memcached % cpan install UUID::Tiny Copyright © 2012 Oracle Corp.
  34. 34. Sample Application% mysql -u root mysql> CREATE DATABASE demo; use demo; Copyright © 2012 Oracle Corp.
  35. 35. Data Definition: AuthorsCREATE TABLE authors ( author varchar(15) not null, date_created timestamp, tweets bigint unsigned not null default 0, followers bigint unsigned not null default 0, following bigint unsigned not null default 0, PRIMARY KEY (author) ) ENGINE=ndb; Copyright © 2012 Oracle Corp.
  36. 36. Data Definition: TweetsCREATE TABLE tweets ( id varchar(36) primary key,author varchar(15),time_stamp timestamp,tweet varchar(140), index(time_stamp), index(author))ENGINE=ndb; Copyright © 2012 Oracle Corp.
  37. 37. Configuration: Use the ndbmemcacheconfig schemamysql> use ndbmemcache; show tables; Copyright © 2012 Oracle Corp.
  38. 38. Configuration: Create container & key_prefixesINSERT into containers SET name = tweets_table , db_schema = demo , db_table = tweets, key_columns = id , value_columns = time_stamp,author,tweet; INSERT into key_prefixes SET key_prefix = tweet:, policy = ndb-only, container = tweets_table; Copyright © 2012 Oracle Corp.
  39. 39. Two key_prefixes for authors table•  newauthor: •  ADD an author record and set date_created•  tweetcount: •  INCR tweet count for author Copyright © 2012 Oracle Corp.
  40. 40. Configuration: New AuthorINSERT into containers SET name = author_new , db_schema = demo , db_table = authors, key_columns = author , value_columns = date_created; INSERT into key_prefixes SET key_prefix = newauthor:, policy = ndb-only, container = author_new; Copyright © 2012 Oracle Corp.
  41. 41. Configuration: Tweet CountINSERT into containers SET name = author_count, db_schema = demo, db_table = authors, key_columns = author, increment_column = tweets; INSERT into key_prefixes SET key_prefix = tweetcount:, policy = ndb-only, container = author_count; Copyright © 2012 Oracle Corp.
  42. 42. Memcached: Start the server% cd /usr/local/mysql % bin/memcached -E lib/ndb_engine.so 17-Aug-2012 13:53:32 PDT NDB Memcache 5.5.25-ndb-7.2.7 ... Contacting primary management server (localhost:1186) ...Connected to "localhost:1186" as node id 4.Retrieved 6 keyprefixes for server role "default_role". Server started with 4 threads. Priming the pump ... done [0.661 sec]. Copyright © 2012 Oracle Corp.
  43. 43. Perl: Startup Codeuse Cache::Memcached; use UUID::Tiny; our $mc = new Cache::Memcached( { servers => [ "localhost:11211" ] }); Copyright © 2012 Oracle Corp.
  44. 44. Perl: Add a new author## Takes one parameter: author sub new_author { my $author = shift; my $time = time(); my $key = "newauthor:$author"; $mc->add($key, $time); } Copyright © 2012 Oracle Corp.
  45. 45. Perl: Insert a Tweet## Take two parameters: author and tweet sub insert_tweet { my $author = shift; my $tweet = shift; my $id = create_UUID_as_string(UUID_V4); my $time = time(); # Insert the tweet my $key = "tweet:$id"; my $value = "$time" ."t". $author ."t". $tweet; $mc->add($key, $value); } Copyright © 2012 Oracle Corp.
  46. 46. Perl: Increment Tweet Count## Increment the authors tweet count. ## Take one parameter: author sub increment_tweetcount { my $author = shift; my $key = "tweetcount:$author"; $mc->incr($key, 1); } Copyright © 2012 Oracle Corp.
  47. 47. Download MySQL Cluster + Memcached APIhttp://www.mysql.com/downloads/cluster/#downloads
  48. 48. MySQL Cluster CGEMost secure, scalable MySQL Database, Development /Management / Monitoring Tools, backed by Oracle PremierLifetime Support Oracle Product Oracle Premier Certifications/Integrations Support MySQL Enterprise MySQL Enterprise Backup Monitor/Query Analyzer MySQL Enterprise MySQL Enterprise Security Scalability MySQL Cluster MySQL Workbench Manager
  49. 49. Oracle Premier Support for MySQL Rely on The Experts - Get Unique Benefits•  Straight from the Source•  Largest Team of MySQL Experts•  Direct Access to MySQL Support Engineers•  Backed by MySQL Developers Only From•  Global 24/7/365, 29 languages•  Forward Compatible Hot Fixes Oracle•  MySQL Maintenance Releases•  Unlimited Incidents•  Knowledge Base "The MySQL support service has been•  MySQL Consultative Support essential in helping us with troubleshooting and providing recommendations for the production cluster, Thanks." -- Carlos Morales – Playfulplay.com
  50. 50. Getting Started Learn More Scaling Web Databases Guide www.mysql.com/cluster/ Get MySQL Cluster 7.2 Review Documentation https://dev.mysql.com/doc/refman/5.5/en/ Download Today mysql-cluster-ndbmemcache.html http://www.mysql.com/ downloads/cluster/Copyright 2011 Oracle Corporation 50
  51. 51. Summary Scale Web Services with 99.999% Availability Native Memcached NoSQL Access Try it out Today!Copyright 2011 Oracle Corporation 51
  52. 52. MySQL Cluster Architecture JPA REST Application Nodes Node Group 1 Node Group 2 F1 F2 Node 1 Node 3Cluster ClusterMgmt F3 F4 Mgmt F3 F4 Node 4 Node 2 F1 F2 Data Nodes
  53. 53. MySQL Cluster - Extreme Resilience JPA REST Application Nodes Node Group 1 Node Group 2 F1 F2 Node 1 Node 3Cluster ClusterMgmt F3 F4 Mgmt F3 F4 Node 4 Node 2 F1 F2 Data Nodes
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×