MEMBASE:                              BE	  RICH	  AND	  FAST                    MEMBASE	  KEY-­‐VALUE	  STORES	  FOR	  INT...
WHAT IS MEMBASE?                                              2Thursday, April 14, 2011
Membase is Simple, Fast, Elastic                           Five minutes or less to a working                           clu...
Membase is Simple, Fast, Elastic                           Predictable                           • “Never keep an applicat...
Membase is Simple, Fast, Elastic                           Zero-downtime elasticity                           • Spread I/O...
USE	  CASES                                          6Thursday, April 14, 2011
Deployments Leading Membase             Leading cloud service (PAAS)    Social game leader – FarmVille,             provid...
Ad targeting                                          40	  milliseconds	  to	  come	  up	                                 ...
Search and Gaming Portal                           Database                                      9Thursday, April 14, 2011
OBSERVATIONS                                          10Thursday, April 14, 2011
Many,	  many	  access	  methods                                  Document	  Store           Column	  Oriented             ...
ENABLING	  POLYGLOT                                                  12Thursday, April 14, 2011
Clustering With Membase                                SET	  request	  arrives	  at	  KEY’s	             SET	  acknowledge...
TAP     • A	  generic,	  scalable	  method	  of	  streaming	  muta5ons	  from	  a	         given	  server              – A...
DEMONSTRATION                                           15Thursday, April 14, 2011
Demo	  Deployment                                                                    Game Application                     ...
Data	  Flow                                      •	  Generates	  TAP_MUTATIONs	  for	  only	  the	  data	  modified        ...
Handling	  a	  TAP_MUTATION  sta5c	  void	  tap_muta5on(libcouchbase_t	  instance,                                        ...
Polyglot	  Possibili5es  • Relational data has many clear uses         - Relationships (duh)         - Need for more compl...
THE	  FUTURE	  WITH	  COUCHBASE                                                     20Thursday, April 14, 2011
What	  is	  Couchbase?                           1+1	  really	  does	  equal	  3                                          ...
Couchbase  • Feb	  08:	  Merger	  Announced  • March	  10:	  Developer	  Preview	  of	  Couchbase	  Mobile	  -­‐	      Apa...
Q	  &	  A                                         23Thursday, April 14, 2011
Data management for interactive web and mobile applications.  MATT	  INGENTHRON  MATT@COUCHBASE.COM  @INGENTHR            ...
Upcoming SlideShare
Loading in …5
×

Be Rich and Fast: Membase and Drizzle - MySQL Conference 2011

1,339 views

Published on

Slides from MySQL Conference 2011 presentation:
http://en.oreilly.com/mysql2011/public/schedule/detail/17499

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,339
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
23
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Be Rich and Fast: Membase and Drizzle - MySQL Conference 2011

  1. 1. MEMBASE: BE  RICH  AND  FAST MEMBASE  KEY-­‐VALUE  STORES  FOR  INTERACTIVITY,   MYSQL  FOR  FULL  QUERYABILITY MYSQL  CONFERENCE  2011 MATT  INGENTHRON 1Thursday, April 14, 2011
  2. 2. WHAT IS MEMBASE? 2Thursday, April 14, 2011
  3. 3. Membase is Simple, Fast, Elastic Five minutes or less to a working cluster • Downloads for Linux and Windows • Start with a single node • One button press joins nodes to a cluster Easy to develop against • Just SET and GET – no schema required • Drop it in. 10,000+ existing applications already “speak membase” (via memcached) • Practically every language and application framework is supported, out of the box Easy to manage • One-click failover and cluster rebalancing • Graphical and programmatic interfaces • Configurable alerting 3Thursday, April 14, 2011
  4. 4. Membase is Simple, Fast, Elastic Predictable • “Never keep an application waiting” • Quasi-deterministic latency and throughput Low latency • Built-in Memcached technology High throughput • Multi-threaded • Low lock contention • Asynchronous wherever possible • Automatic write de-duplication 4Thursday, April 14, 2011
  5. 5. Membase is Simple, Fast, Elastic Zero-downtime elasticity • Spread I/O and data across commodity servers (or VMs) • Consistent performance with linear cost • Dynamic rebalancing of a live cluster All nodes are created equal • No special case nodes • Any node can replace any other node, online • Clone to grow Extensible • Filtered TAP interface provides hook points for external systems (e.g. full-text search, backup, warehouse) • Data bucket – engine API for specialized container types 5Thursday, April 14, 2011
  6. 6. USE  CASES 6Thursday, April 14, 2011
  7. 7. Deployments Leading Membase Leading cloud service (PAAS) Social game leader – FarmVille, provider Mafia Wars, Café World Over 65,000 hosted Over 230 million monthly users applications Membase Server Membase Server serving over is the 500,000 ops-per-second 1,200 Heroku customers (as of database behind FarmVille and June 10, 2010) Café World 7Thursday, April 14, 2011
  8. 8. Ad targeting 40  milliseconds  to  come  up   with  an  answer. 3 profiles,  real  5me  campaign   sta5s5cs 2 1 profiles,  campaigns events 8Thursday, April 14, 2011
  9. 9. Search and Gaming Portal Database 9Thursday, April 14, 2011
  10. 10. OBSERVATIONS 10Thursday, April 14, 2011
  11. 11. Many,  many  access  methods Document  Store Column  Oriented Polyglot  Persistence ACID BASE CAP 11Thursday, April 14, 2011
  12. 12. ENABLING  POLYGLOT 12Thursday, April 14, 2011
  13. 13. Clustering With Membase SET  request  arrives  at  KEY’s   SET  acknowledgement   master  server 1 2 returned  to  applica5on 3 3 Listener-­‐Sender RAM 4 membase  storage  engine Disk Disk Disk Disk Disk Disk Replica  Server  1  for  KEY Master  server  for  KEY Replica  Server  2  for  KEY 13Thursday, April 14, 2011
  14. 14. TAP • A  generic,  scalable  method  of  streaming  muta5ons  from  a   given  server – As  data  opera5ons  arrive,  they  can  be  sent  to  arbitrary  TAP  receivers • Leverages  the  exis5ng  memcached  engine  interface,  and  the   non-­‐blocking  IO  interfaces  to  send  data • Three  modes  of  opera5on Data Mutations Data set Data Mutations Data set Data set 14Thursday, April 14, 2011
  15. 15. DEMONSTRATION 15Thursday, April 14, 2011
  16. 16. Demo  Deployment Game Application Online <play games> Game Users Data Analytics •Using  TAP  from   libcouchbase •Parse  JSON  and  insert  into   Drizzle 16Thursday, April 14, 2011
  17. 17. Data  Flow •  Generates  TAP_MUTATIONs  for  only  the  data  modified •  TAP_MUTATIONS  may  be  acked,  or  not •  TAP  star5ng  point  defined  in  the  protocol •  Receive  TAP  muta5ons •  Possibly  filter libcouchbase TAP client •  Transform •  (polyglot) •  Receives  simple  SQL •  Handles  other,  rich  methods  of  processing  data 17Thursday, April 14, 2011
  18. 18. Handling  a  TAP_MUTATION sta5c  void  tap_muta5on(libcouchbase_t  instance,        len  +=  drizzle_escape_string(sql_buffer  +  len,  anacking,  strlen(anacking));                                                  const  void  *key,        len  +=  (size_t)sprinp(sql_buffer  +  len,  ",");                                                  size_t  nkey,        len  +=  drizzle_escape_string(sql_buffer  +  len,  defending,  strlen                                                  const  void  *data, (defending));                                                  size_t  nbytes,        len  +=  (size_t)sprinp(sql_buffer  +  len,  ",");                                                  uint32_t  flags,        len  +=  drizzle_escape_string(sql_buffer  +  len,  winner,  strlen(winner));                                                  uint32_t  exp,        len  +=  (size_t)sprinp(sql_buffer  +  len,  ",");                                                  const  void  *es,        len  +=  drizzle_escape_string(sql_buffer  +  len,  when,  strlen(when));                                                  size_t  nes)        len  +=  (size_t)sprinp(sql_buffer  +  len,  ",%d);",  damage); {        assert(nbytes  <  sizeof(json_buffer));        drizzle_return_t  ret;        memcpy(json_buffer,  data,  nbytes);        drizzle_query_str(drizzle_con,  result,  sql_buffer,  &ret);        json_buffer[nbytes]  =  0;        if  (ret  !=  DRIZZLE_RETURN_OK)  {        cJSON  *c  =  cJSON_Parse(json_buffer);                fprinp(stderr,  "Failed  to  add  entry  to  the  database:  %sn",                                drizzle_con_error(drizzle_con));        const  char  *anacking  =  get_char_val(c,  "anackingPlayer");        }        const  char  *defending  =  get_char_val(c,  "defendingPlayer");        const  char  *winner  =  get_char_val(c,  "winner");        drizzle_result_free(result);        const  char  *when  =  get_char_val(c,  "when");        cJSON_Delete(c);        int  damage  =  get_int_val(c,  "damage"); }        (void)instance;(void)flags;(void)exp;(void)es;(void)nes;        size_t  len  =  (size_t)sprinp(sql_buffer,                                                                    "REPLACE  INTO  Fight(AJacker,  "                                                                    "Defender,  Winner,  FightTime,  "                                                                    "Damage)  VALUES("); 18Thursday, April 14, 2011
  19. 19. Polyglot  Possibili5es • Relational data has many clear uses - Relationships (duh) - Need for more complex indexes, joins • Various batch processing systems - Hadoop M-R - Greenplum, Infobright, etc. 19Thursday, April 14, 2011
  20. 20. THE  FUTURE  WITH  COUCHBASE 20Thursday, April 14, 2011
  21. 21. What  is  Couchbase? 1+1  really  does  equal  3 21Thursday, April 14, 2011
  22. 22. Couchbase • Feb  08:  Merger  Announced • March  10:  Developer  Preview  of  Couchbase  Mobile  -­‐   Apache  CouchDB  for  iOS  devices • March  15:  Couchbase  1.1  released – Includes  Apache  CouchDB++,  GeoCouch,  ready  for  support • New  Membase  update  in  the  summer – Integrate  some  features  from  CouchDB • Elas5c  Couchbase  later  in  the  year Mobile  Couchbase Couchbase Elas5c  Couchbase* 22Thursday, April 14, 2011
  23. 23. Q  &  A 23Thursday, April 14, 2011
  24. 24. Data management for interactive web and mobile applications. MATT  INGENTHRON MATT@COUCHBASE.COM @INGENTHR 24Thursday, April 14, 2011

×