Naviga&ng	  the	  Transi&on	  from	  Rela&onal	  to	  NoSQL	  Technology	                             Dip&	  Borkar	      ...
WHY	  TRANSITION	  TO	  NOSQL?	                 	                                          2	  
Two	  big	  drivers	  for	  NoSQL	  adop&on	               49%	                                                       35%	...
NoSQL	  catalog	                         Key-­‐Value	     Data	  Structure	     Document	       Column	        Graph	  (me...
DISTRIBUTED	  DOCUMENT	        DATABASES	                                5	  
Document	  Databases	  •  Each	  record	  in	  the	  database	  is	  a	  self-­‐   describing	  document	  	              ...
COMPARING	  DATA	  MODELS	                       7	  
h]p://www.geneontology.org/images/diag-­‐godb-­‐er.jpg	     8	  
Rela&onal	  vs	  Document	  data	  model	              C1	      C2	       C3	      C4	                                    ...
Example:	  User	  Profile	                     User	  Info	                                                    Address	  In...
Document	  Example:	  User	  Profile	  	  {	  	  	  	  	  “ID”:	  1,	                                              =	      ...
Making	  a	  Change	  Using	  RDBMS	                     User	  Table	                                                    ...
Making	  the	  Same	  Change	  with	  a	  Document	  Database	  	                               	  {	                     ...
Document	  modeling	  	  	                       •    Are	  these	  separate	  object	  in	  the	  model	  layer?	  	  	  ...
Document	  Design	  Op&ons	        	  	  	   	  	    •  One	  document	  that	  contains	  all	  related	  data	  	  	    ...
Document	  ID	  /	  Key	  selec&on	   •    Similar	  to	  primary	  keys	  in	  rela&onal	  databases	   •    Documents	  ...
Example:	  En&&es	  for	  a	  Blog	                                                                        BLOG	      •  U...
Blog	  Document	  –	  Op&on	  1	  –	  Single	  document	  	         {	         “UUID ”:	  “2 1 f7 f8 de-­‐8 0 5 1 -­‐5 b89...
Blog	  Document	  –	  Op&on	  2	  -­‐	  Split	  into	  mul&ple	  docs	  	  {	  { !“UUID ”:	  “21f7f8de-­‐8051 -­‐5b89 -­‐8...
Threaded	  Comments	  •  You	  can	  imagine	  how	  to	  take	  this	  to	  a	  threaded	  list	                         ...
COMPARING	  	  SCALING	  MODEL	                         21	  
Rela&onal	  Technology	  Scales	  Up	                                                                                    A...
Couchbase	  Server	  Scales	  Out	  Like	  App	  Tier	                                                                    ...
EVALUATING	  NOSQL	                            24	  
The	  Process	  –	  From	  Evalua&on	  to	  Go	  Live	  	                          No	  different	  from	  evalua&ng	  a	  ...
1	      	  Analyze	  your	  requirements	  	          Common	  applica&on	  requirements	              	              •  R...
2	     	  Find	  solu&ons	  that	  match	  key	  requirements	  •  Linear	  Scalability	  	  •  Schema	  flexibility	      ...
3	        	  Proof	  of	  concept	  /	  Performance	  evalua&on	      Prototype	  a	  workload	      	      •  Look	  for	...
3	     	  Other	  considera&ons	  	  	  	  Accessing	  data	                                                              ...
3	     	  Other	  considera&ons	  	  	  	  Opera&ons	                                                                     ...
4	        	  Begin	  development	      	      	      	                       Data	  Modeling	  and	                       ...
5	        	  Deploying	  to	  staging	  and	  produc&on	      	               •  Monitoring	  the	  system	      	        ...
Couchbase	  Server	  Admin	  Console	                                                33	  
34	  
So	  are	  you	  being	  impacted	  by	  these?	  	                                       Schema	  Rigidity	  problems	  	...
Is	  NoSQL	  the	  right	  choice	  for	  you?	       Does	  your	  applica&on	  need	  rich	  database	  func&onality?	  ...
WHERE	  IS	  NOSQL	  A	  GOOD	  FIT?	                                                  37	  
Market	  Adop&on	           Internet	  Companies	             Enterprises	      •     Social	  Gaming	          • Communic...
Market	  Adop&on	  –	  Customers	          Internet	  Companies	                                         Enterprises	     ...
Applica&on	  Characteris&cs	  -­‐	  Data	  driven	     •    3rd	  party	  or	  user	  defined	  structure	  (Twi]er	  feeds...
Applica&on	  Characteris&cs	  -­‐	  Performance	  driven	       •  Low	  latency	  cri&cal	  (ex.	  1millisecond)	       •...
Common	  Use	  Cases	     Social	  Gaming	                       Ad	  Targe&ng	                   	                       ...
BRIEF	  OVERVIEW	  COUCHBASE	  SERVER	                            43	  
Couchbase	  Server	    NoSQL	  Distributed	  Document	  Database	       for	  interac&ve	  web	  applica&ons	             ...
Couchbase	  Open	  Source	  Project	   •  Leading	  NoSQL	  database	  project	      focused	  on	  distributed	  database...
Couchbase	  Server	                       Easy	                                                    Consistent	  High	     ...
New	  in	  2.0	               JSON	  support	          Indexing	  and	  Querying	                      JSON               ...
Addi&onal	  Couchbase	  Server	  Features	  Built-­‐in	  clustering	  –	  All	  nodes	  equal	     Append-­‐only	  storage...
COUCHBASE	  SERVER	  	    ARCHITECTURE	                                49	  
Couchbase	  Server	  2.0	  Architecture	      8092	                          11211	                       11210	      Quer...
Couchbase	  Server	  2.0	  Architecture	      8092	                          11211	                       11210	      Quer...
Couchbase	  deployment	                            Web	                          Applica&on	                          Couc...
Single	  node	  -­‐	  Couchbase	  Write	  Opera&on	                                                                       ...
Single	  node	  -­‐	  Couchbase	  Update	  Opera&on	                                                                      ...
Single	  node	  -­‐	  Couchbase	  Read	  Opera&on	                                                                        ...
Single	  node	  -­‐	  Couchbase	  Cache	  Evic&on	                                                                        ...
Single	  node	  –	  Couchbase	  Cache	  Miss	                                                                             ...
Cluster	  wide	  -­‐	  Basic	  Opera&on	                              APP	  SERVER	  1	                                   ...
Cluster	  wide	  -­‐	  Add	  Nodes	  to	  Cluster	                                     APP	  SERVER	  1	                  ...
Cluster	  wide	  -­‐	  Fail	  Over	  Node	                                        APP	  SERVER	  1	                       ...
Indexing	  and	  Querying	  	                              APP	  SERVER	  1	                                              ...
Cross	  Data	  Center	  Replica&on	  (XDCR)	        SERVER	  1	  	                           SERVER	  2	   	              ...
THANK	  YOU	            	            	  DIPTI@COUCHBASE.COM	        @DBORKAR	                            63	  
Upcoming SlideShare
Loading in …5
×

Webinar - Transitioning from RDBMS to NoSQL

3,133 views

Published on

While the hype surrounding “NoSQL” (non-relational) database technology has become deafening, there is real substance beneath the often exaggerated claims. “NoSQL” database technologies have emerged as a better match for the needs of modern interactive applications with cost-effective data management. Developers accustomed to relational database technology need to approach things differently.

Published in: Technology
2 Comments
6 Likes
Statistics
Notes
No Downloads
Views
Total views
3,133
On SlideShare
0
From Embeds
0
Number of Embeds
801
Actions
Shares
0
Downloads
87
Comments
2
Likes
6
Embeds 0
No embeds

No notes for slide

Webinar - Transitioning from RDBMS to NoSQL

  1. 1. Naviga&ng  the  Transi&on  from  Rela&onal  to  NoSQL  Technology   Dip&  Borkar   Director,  Product  Management   1  
  2. 2. WHY  TRANSITION  TO  NOSQL?     2  
  3. 3. Two  big  drivers  for  NoSQL  adop&on   49%   35%   29%   16%   12%   11%  Lack  of  flexibility/   Inability  to   Performance   Cost   All  of  these   Other   rigid  schemas   scale  out  data   challenges  Source:  Couchbase  Survey,  December  2011,  n  =  1351.   3  
  4. 4. NoSQL  catalog   Key-­‐Value   Data  Structure   Document   Column   Graph  (memory  only)   Cache   memcached   redis  (memory/disk)   membase   couchbase   cassandra   Neo4j   Database   mongoDB   4  
  5. 5. DISTRIBUTED  DOCUMENT   DATABASES   5  
  6. 6. Document  Databases  •  Each  record  in  the  database  is  a  self-­‐ describing  document     {  •  Each  document  has  an  independent   “UUID”:  “ 21f7f8de-­‐8051-­‐5b89-­‐86 “Time”:   “2011-­‐04-­‐01T13:01:02.42 “Server”:   “A2223E”, structure   “Calling   Server”:   “A2213W”, “Type”:   “E100”, “Initiating   User”:   “dsallings@spy.net”,•  Documents  can  be  complex     “Details”:   { “IP”:  “ 10.1.1.22”,•  All  databases  require  a  unique  key   “API”:   “InsertDVDQueueItem”, “Trace”:   “cleansed”,•  Documents  are  stored  using  JSON  or   “Tags”:   [ “SERVER”,   XML  or  their  deriva&ves   “US-­‐West”,   “API” ]•  Content  can  be  indexed  and  queried     } }•  Offer  auto-­‐sharding  for  scaling  and   replica&on  for  high-­‐availability   6  
  7. 7. COMPARING  DATA  MODELS   7  
  8. 8. h]p://www.geneontology.org/images/diag-­‐godb-­‐er.jpg   8  
  9. 9. Rela&onal  vs  Document  data  model   C1   C2   C3   C4   {   JSON       JSON     }   JSON   Rela&onal  data  model   Document  data  model   Highly-­‐structured  table  organiza&on   Collec&on  of  complex  documents  with   with  rigidly-­‐defined  data  formats  and   arbitrary,  nested  data  formats  and   record  structure.   varying  “record”  format.   9  
  10. 10. Example:  User  Profile   User  Info   Address  Info   KEY   First   Last   ZIP_id   ZIP_id   CITY   STATE   ZIP   1   Dip&   Borkar   2   1   DEN   CO   30303   2   Joe Smith   2   2   MV   CA   94040     3   Ali   Dodson   2   3   CHI   IL   60609   4   John   Doe   3   4   NY   NY   10010   To  get  informa&on  about  specific  user,  you  perform  a  join  across  two  tables     10  
  11. 11. Document  Example:  User  Profile    {          “ID”:  1,   =   +          “FIRST”:  “Dip&”,          “LAST”:  “Borkar”,          “ZIP”:  “94040”,          “CITY”:  “MV”,          “STATE”:  “CA”      }   JSON   All  data  in  a  single  document   11  
  12. 12. Making  a  Change  Using  RDBMS   User  Table   Photo  Table   Country  Table   Country   TEL Country  User  ID   First   Last   Zip   ID   User  ID   3   Photo  ID   Comment   ID   Country  ID   Country  name   2   d043   NYC      001   001   USA   1   Dip&   Borkar   94040    001   2   b054   Bday      007   002   UK   2   Joe   Smith   94040   001   5   c036   Miami      001   003   Argen&na   3   Ali   Dodson   94040   001   7   d072   Sunset      133   004   Australia   5002   e086   Spain      133   4   Sarah   Gorin   NW1   002   005   Aruba   Status  Table   006   Austria   5   Bob   Young   30303   001   Country   User  ID   Status  ID   Text   ID   007   Brazil   6   Nancy   Baker   10010   001   1   a42   At  conf      134   008   Canada   4   b26   excited   007   7   Ray   Jones   31311   001   5   c32   hockey      008   009   Chile   8   Lee   Chen   V5V3M   008   12   d83   Go  A’s      001   •  •          •      5000   e34   sailing      005   •      .   •      .   130   Portugal   •      .   Affilia&ons  Table   Country   User  ID   Affl  ID   Affl  Name   ID   131   Romania  50000   Doug   Moore   04252   001   2   a42   Cal      001   132   Russia   4   b96   USC      001  50001   Mary   White   SW195   002   133   Spain   7   c14   UW      001  50002   Lisa   Clark   12425   001   8   e22   Oxford      002   134   Sweden   12  
  13. 13. Making  the  Same  Change  with  a  Document  Database      {          “ID”:  1,          “FIRST”:  “Dip&”,          “LAST”:  “Borkar”,          “ZIP”:  “94040”,          “CITY”:  “MV”,          “STATE”:  “CA”,          “STATUS”:     }   ,              {    “TEXT”:  “At  Conf”      }              “GEO_LOC”:  “134”  },     “COUNTRY”:  ”USA”   }         JSON   Just  add  informa&on  to  a  document   13  
  14. 14. Document  modeling       •  Are  these  separate  object  in  the  model  layer?         Q   •  •  Are  these  objects  accessed  together?     Do  you  need  updates  to  these  objects  to  be  atomic?   •  Are  mul&ple    people  edi&ng  these  objects  concurrently?        When  considering  how  to  model  data  for  a  given    applica&on   •  Think  of  a  logical  container  for  the  data   •  Think  of  how  data  groups  together         14  
  15. 15. Document  Design  Op&ons             •  One  document  that  contains  all  related  data       –  Data  is  de-­‐normalized   –  Be]er  performance  and  scale   –  Eliminate  client-­‐side  joins       •  Separate  documents  for  different  object  types  with   cross  references     –  Data  duplica&on  is  reduced   –  Objects  may  not  be  co-­‐located     –  Transac&ons  supported  only  on  a  document  boundary   –  Most  document  databases  do  not  support  joins   15  
  16. 16. Document  ID  /  Key  selec&on   •  Similar  to  primary  keys  in  rela&onal  databases   •  Documents  are  sharded  based  on  the  document  ID   •  ID  based  document  lookup  is  extremely  fast     •  Usually  an  ID  can  only  appear  once  in  a  bucket         Q     •         Do  you  have  a  unique  way  of  referencing  objects?   •         Are  related  objects  stored  in  separate  documents?   Op&ons   • UUIDs,  date-­‐based  IDs,  numeric  IDs       • Hand-­‐crajed  (human  readable)     • Matching  prefixes  (for  mul&ple  related  objects)   16  
  17. 17. Example:  En&&es  for  a  Blog   BLOG   •  User  profile   The  main  pointer  into  the  user  data   •  Blog  entries   •  Badge  sekngs,  like  a  twi]er  badge       •  Blog  posts   Contains  the  blogs  themselves       •  Blog  comments   •  Comments  from  other  users   17  
  18. 18. Blog  Document  –  Op&on  1  –  Single  document     {   “UUID ”:  “2 1 f7 f8 de-­‐8 0 5 1 -­‐5 b89 -­‐8 6 “Time”:   “2 0 1 1 -­‐0 4-­‐0 1 T1 3 :0 1 :0 2.4 2 { “Server”:   “A2 2 2 3 E”, ! “Calling   Server”:   “A2 2 1 3 W”, “_id”: “Couchbase_Hello_World”,! “Type”:   “E1 0 0 ”, “author”: “dborkar”, ! “Initiating   Us er”:   “ds allings @s py.net”, “type”: “post”! “D etails ”:   “title”: “Hello World”,! { “format”: “IP”:  “1 0 .1 ! .2 2 ”, “markdown”, .1 “API”:   “Ins ertD VD QueueItem”, “body”: “Hello from [Couchbase](http://couchbase.com).”, ! “Trace”:   “cleans ed”, “html”: “<p>Hello from <a href=“http: …! “Tags ”:   “comments”:[ ! [ [“format”: “markdown”, “body”:”Awesome post!”],! “SERVER”,   “US-­‐Wes t”,   [“format”: “markdown”, “body”:”Like it.” ]! ]! “API” ] }   } } 18  
  19. 19. Blog  Document  –  Op&on  2  -­‐  Split  into  mul&ple  docs    {  { !“UUID ”:  “21f7f8de-­‐8051 -­‐5b89 -­‐86“_id”: “Coucbase_Hello_World”,!“Time”:   “2011 -­‐04-­‐01T13:01:02.42“author”: “A2223E”, !“Server”:   “dborkar”,“Calling   Server”:   “A2213W”,“type”: “E100 ”,“Type”:   “post”!“title”: “Hello World”,! @s py.net”,“Initiating   Us er”:   “ds allings“D etails ”:  “format”: “markdown”, ! {“body”:“IP”:  “10.1.1.22”, “Hello from [Couchbase]( “API”:   “Ins ertDVD QueueItem”,http://couchbase.com).”, ! “Trace”:   “cleans ed”,“html”:“Tags ”:   “<p>Hello from <a href=“http: …! [“comments”:[! “SERVER”,   ! “comment1_Couchbase_Hello_world”! “US-­‐Wes t”,   ! “API” ]! ] {   COMMENT  }! } “UUID ”:  “ 2 1 f7 f8 d e-­‐ 8 0 5 1 -­‐5 b 8 9 -­‐ 8 6 “Time”:   “ 2 0 1 1 -­‐ 0 4 -­‐0 1 T1 3 :0 1 :0 2 .4 2 “Server”:   “A2 2 2 3 E”,} “Callin g   Server”:   “A2 2 1 3 W ”, {! BLOG  DOC   “Typ e”:   “E1 0 0 ”, “In itiatin g   Us er”:   “d s allin gs @s p y.n et”, “_id”: “comment1_Couchbase_Hello_World”,! “D etails ”:   { “IP ”:  “ 1 0 .1 .1 .2 2 ”, “format”: “markdown”, ! “AP I”:   “ In s ertD VD Qu eu eItem”, “Trace”:   “clean s ed ”, “Tags ”:   “body”:”Awesome post!” ! [ “SERVER”,   “US-­‐Wes t”,   }   “AP I” ] } } 19  
  20. 20. Threaded  Comments  •  You  can  imagine  how  to  take  this  to  a  threaded  list   List   First   Reply  to   comment   Blog   List   comment   More   Comments  Advantages  •  Only  fetch  the  data  when  you  need  it   •  For  example,  rendering  part  of  a  web  page  •  Spread  the  data  and  load  across  the  en&re  cluster     20  
  21. 21. COMPARING    SCALING  MODEL   21  
  22. 22. Rela&onal  Technology  Scales  Up   Applica&on  Scales  Out   Just  add  more  commodity  web  servers   System  Cost   Applica&on  Performance    Web/App  Server  Tier   Users   RDBMS  Scales  Up   Get  a  bigger,  more  complex  server   System  Cost   Applica&on  Performance     Won’t   scale   beyond   this  point   Rela&onal  Database   Users   Expensive  and  disrup&ve  sharding,  doesn’t  perform  at  web  scale   22  
  23. 23. Couchbase  Server  Scales  Out  Like  App  Tier   Applica&on  Scales  Out   Just  add  more  commodity  web  servers   System  Cost   Applica&on  Performance    Web/App  Server  Tier   Users   NoSQL  Database  Scales  Out   Cost  and  performance  mirrors  app  &er   System  Cost   Applica&on  Performance     Couchbase  Distributed  Data  Store   Users   Scaling  out  flatens  the  cost  and  performance  curves   23  
  24. 24. EVALUATING  NOSQL   24  
  25. 25. The  Process  –  From  Evalua&on  to  Go  Live     No  different  from  evalua&ng  a  rela&onal  database     1    Analyze  your  requirements       2    Find  solu&ons  /  products  that  match  key  requirements     3    Execute  a  proof  of  concept  /  performance  evalua&on     4    Begin  development  of  applica&on         5    Deploy  in  staging  and  then  produc&on     New  requirements  è  New  solu&ons     25  
  26. 26. 1    Analyze  your  requirements     Common  applica&on  requirements     •  Rapid  applica&on  development   –  Changing  market  needs   –  Changing  data  needs     •  Scalability     –  Unknown  user  demand     –  Constantly  growing  throughput   •  Consistent  Performance     –  Low  response  &me  for  be]er  user  experience   –  High  throughput  to  handle  viral  growth     •  Reliability   –  Always  online     26  
  27. 27. 2    Find  solu&ons  that  match  key  requirements  •  Linear  Scalability    •  Schema  flexibility   NoSQL  •  High  Performance  •  Mul&-­‐document  transac&ons  •  Database  Rollback    •  Complex  security  needs   RDBMS  •  Complex  joins  •  Extreme  compression  needs  •  Both  /  depends  on  the  data   RDBMS   NoSQL   27  
  28. 28. 3    Proof  of  concept  /  Performance  evalua&on   Prototype  a  workload     •  Look  for  consistent  performance…         –  Low  response  &mes  /  latency       •  For  be]er  user  experience   –  High  throughput     •  To  handle  viral  growth     •  For  resource  efficiency   •  …  across   –  Read  heavy  /  Write  heavy  /  Mixed  workloads   –  Clusters  of  growing  sizes     •  …  and  watch  for     –  Conten&on  /  heavy  locking     –  Linear  scalability   28  
  29. 29. 3    Other  considera&ons        Accessing  data   App  Server   –  No  standards  exist  yet   –  Typically  via  SDKs  or  over  HTTP   –  Check  if  the  programing  language  of  your   choice  is  supported.        Consistency   App  Server   –  Consistent  only  at  the  document  level   –  Most  documents  stores  currently  don’t   support  mul&-­‐document  transac&ons   –  Analyze  your  applica&on  needs        Availability   App  Server   –  Each  node  stores  ac&ve  and  replica  data   (Couchbase)   –  Each  node  is  either  a  master  or  slave   (MongoDB)   29  
  30. 30. 3    Other  considera&ons        Opera&ons   App  Server   –  Monitoring  the  system   –  Backup  and  restore  the  system   –  Upgrades  and  maintenance     –  Support        Ease  of  Scaling   App  Server   –  Ease  of  adding  and  reducing  capacity   Client   –  Single  node  type   –  App  availability  on  topology  changes        Indexing  and  Querying   –  Secondary  indexes  (Map  func&ons)   –  Aggregates  Grouping  (Reduce  func&ons)   –  Basic  querying     30  
  31. 31. 4    Begin  development         Data  Modeling  and   Document  Design   31  
  32. 32. 5    Deploying  to  staging  and  produc&on     •  Monitoring  the  system     •  RESTful  interfaces  /  Easy  integra&on  with  monitoring     tools   •  High-­‐availability   •  Replica&on   •  Failover  and  Auto-­‐failover     •  Always  Online  –  even  for  maintenance  tasks     •  Database  upgrades   •  Sojware  (OS)  and  Hardware  upgrades   •  Backup  and  restore   •  Index  building   •  Compac&on   32  
  33. 33. Couchbase  Server  Admin  Console   33  
  34. 34. 34  
  35. 35. So  are  you  being  impacted  by  these?     Schema  Rigidity  problems     •  Do  you  store  serialized  objects  in  the  database?   •  Do  you  have  lots  of  sparse  tables  with  very  few  columns   Q   being  used  by  most  rows?   •  Do  you  find  that  your  applica&on  developers  require  schema   changes  frequently  due  to  constantly  changing  data?       •  Are  you  using  your  database  as  a  key-­‐value  store?   Scalability  problems     •  Do  you  periodically  need  to  upgrade  systems  to  more   powerful  servers  and  scale  up?     Q   •  Are  you  reaching  the  read  /  write  throughput  limit  of  a  single   database  server?     •  Is  your  server’s  read  /  write  latency  not  mee&ng  your  SLA?     •  Is  your  user  base  growing  at  a  frightening  pace?     35  
  36. 36. Is  NoSQL  the  right  choice  for  you?   Does  your  applica&on  need  rich  database  func&onality?       •  Mul&-­‐document  transac&ons   •  Complex  security  needs  –  user  roles,  document  level  security,   authen&ca&on,  authoriza&on  integra&on   •  Complex  joins  across  bucket  /  collec&ons     •  BI  integra&on     •  Extreme  compression  needs   NoSQL  may  not  be  the  right  choice  for  your  applica&on   36  
  37. 37. WHERE  IS  NOSQL  A  GOOD  FIT?   37  
  38. 38. Market  Adop&on   Internet  Companies   Enterprises   •  Social  Gaming   • Communica&ons   •  Ad  Networks   • Retail   •  Social  Networks   • Financial  Services   •  Online  Business   • Health  Care   Services   • Automo&ve/Airline   •  E-­‐Commerce   • Agriculture   •  Online  Media   •  Content  Management   • Consumer  Electronics   •  Cloud  Services   • Business  Systems   38  
  39. 39. Market  Adop&on  –  Customers   Internet  Companies   Enterprises   More  than  300  customers  -­‐-­‐  5,000  produc&on  deployments  worldwide   39  
  40. 40. Applica&on  Characteris&cs  -­‐  Data  driven   •  3rd  party  or  user  defined  structure  (Twi]er  feeds)   •  Support  for  unlimited  data  growth    (Viral  apps)   •  Data  with  non-­‐homogenous  structure     •  Need  to  quickly  and  ojen  change  data  structure   •  Variable  length  documents   •  Sparse  data  records   •  Hierarchical  data       Couchbase  is  a  good  fit     40  
  41. 41. Applica&on  Characteris&cs  -­‐  Performance  driven   •  Low  latency  cri&cal  (ex.  1millisecond)   •  High  throughput  (ex.  200000  ops  /  sec)   •  Large  number  of  users     •  Unknown  demand  with  sudden  growth  of   users/data     •  Predominantly  direct  document  access   •  Read  /  Mixed  /  Write  heavy  workloads     Couchbase  is  a  good  fit     41  
  42. 42. Common  Use  Cases   Social  Gaming   Ad  Targe&ng     Session  store   •  Couchbase  stores       player  and  game   •  Couchbase  stores   •  Couchbase  Server  as  a  key-­‐ data     user  informa&on  for   value  store   fast  access   •  Examples   •  Examples  customers  include:   customers  include:   •  Examples  customers   Concur,  Sabre   Zynga   include:  AOL,   •  Tapjoy,  Ubisoj,   Mediamind,   Convertro       Tencent   Content  &  Metadata     Store   Mobile  Apps   User  Profile  Store     •  Couchbase  document  store         with  Elas&c  Search   •  Couchbase  Server  as  a   •  Couchbase  stores  user   key-­‐value  store   info  and  app  content   •  Examples  customers   •  Examples  customers   include:  McGraw  Hill   •  Examples  customers   include:  Tunewiki     include:  Kobo,  Play&ka     3rd  party  data    aggrega&on           High  availability  cache     •  Couchbase  stores  social  media  and       data  feeds  •  Couchbase  Server  used  as  a  cache  &er  replacement   •  Examples  customers  include:   Sambacloud  •  Examples  customers  include:  Orbitz   42    
  43. 43. BRIEF  OVERVIEW  COUCHBASE  SERVER   43  
  44. 44. Couchbase  Server   NoSQL  Distributed  Document  Database   for  interac&ve  web  applica&ons   2.0 44  
  45. 45. Couchbase  Open  Source  Project   •  Leading  NoSQL  database  project   focused  on  distributed  database   technology  and  the  surrounding   ecosystem   •  Supports  both  key-­‐value  and   document-­‐oriented  use  cases   •  All  components  are  available   under  the  Apache  2.0  Public   License   Couchbase •  Obtained  as  packaged  sojware   Open Source Project in  both  enterprise  and   community  edi&ons.   45  
  46. 46. Couchbase  Server   Easy   Consistent  High   Scalability   PE RF O R M A N C E Performance   Grow  cluster  without   Consistent  sub-­‐millisecond     applica&on  changes,  without   read  and  write  response  &mes     down&me  with  a  single  click   with  consistent  high  throughput   Always  On   Flexible  Data   24x365   Model   JSON JSON JSO N JSON JSON No  down&me  for  sojware   JSON  document  model  with   upgrades,  hardware   no  fixed  schema.   maintenance,  etc.   46  
  47. 47. New  in  2.0   JSON  support   Indexing  and  Querying   JSON JSON JSO JSON N JSON Incremental  Map  Reduce   Cross  data  center  replica&on   47  
  48. 48. Addi&onal  Couchbase  Server  Features  Built-­‐in  clustering  –  All  nodes  equal   Append-­‐only  storage  layer      Data  replica&on  with  auto-­‐failover   Online  compac&on      Zero-­‐down&me  maintenance     Monitoring  and  admin  API  &  UI      Built-­‐in  managed  cached   SDK  for  a  variety  of  languages       48  
  49. 49. COUCHBASE  SERVER     ARCHITECTURE   49  
  50. 50. Couchbase  Server  2.0  Architecture   8092   11211   11210   Query  API   Memcapable    1.0   Memcapable    2.0   Moxi   Query  Engine   REST  management  API/Web  UI   vBucket  state  and  replica&on  manager   Memcached   Global  singleton  supervisor   Rebalance  orchestrator   Configura&on  manager   Node  health  monitor   Process  monitor   Heartbeat   Couchbase  EP  Engine   Data  Manager   Cluster  Manager   storage  interface   New  Persistence  Layer   htp   on  each  node   one  per  cluster   Erlang/OTP   HTTP   Erlang  port  mapper   Distributed  Erlang   8091   4369   21100  -­‐  21199   50  
  51. 51. Couchbase  Server  2.0  Architecture   8092   11211   11210   Query  API   Memcapable    1.0   Memcapable    2.0   Moxi   Query  Engine   REST  management  API/Web  UI   vBucket  state  and  replica&on  manager   Memcached   Global  singleton  supervisor   Rebalance  orchestrator   Configura&on  manager   Node  health  monitor   Process  monitor   Heartbeat   Couchbase  EP  Engine   storage  interface   New  Persistence  Layer   htp   on  each  node   one  per  cluster   Erlang/OTP   HTTP   Erlang  port  mapper   Distributed  Erlang   8091   4369   21100  -­‐  21199   51  
  52. 52. Couchbase  deployment   Web   Applica&on   Couchbase   Client  Library   Data  Flow   Cluster  Management   52  
  53. 53. Single  node  -­‐  Couchbase  Write  Opera&on   2   Doc  1   App  Server   3   2   3   Managed  Cache   To  other  node   Replica&on   Doc  1   Queue   Disk  Queue   Disk   Couchbase  Server  Node   53  
  54. 54. Single  node  -­‐  Couchbase  Update  Opera&on   2   Doc  1’   App  Server   3   2   3   Managed  Cache   To  other  node   Replica&on   Doc  1   Doc  1’   Queue   Disk  Queue   Disk   Doc  1   Couchbase  Server  Node   54  
  55. 55. Single  node  -­‐  Couchbase  Read  Opera&on   2   Doc  1   GET   App  Server   3   2   3   Managed  Cache   To  other  node   Replica&on   Queue   Doc  1   Disk  Queue   Disk   Doc  1   Couchbase  Server  Node   55  
  56. 56. Single  node  -­‐  Couchbase  Cache  Evic&on   2   Doc  6   2 3 4 5 App  Server   3   2   3   Managed  Cache   To  other  node   Replica&on   Queue   Doc  1   Disk  Queue   Disk   Doc  1   Doc  6   Doc  5   Doc  4   Doc  3   Doc  2   Couchbase  Server  Node   56  
  57. 57. Single  node  –  Couchbase  Cache  Miss   2   Doc  1   GET   App  Server   3   2   3   Managed  Cache   To  other  node   Replica&on   Queue   Doc  1   Doc  5   4   4   Doc   Doc   Doc  3   2   Doc   Disk  Queue   Disk   Doc  1   Doc  6   Doc  5   Doc  4   Doc  3   Doc  2   Couchbase  Server  Node   57  
  58. 58. Cluster  wide  -­‐  Basic  Opera&on   APP  SERVER  1   APP  SERVER  2   COUCHBASE  Client  Library   COUCHBASE  Client  Library       CLUSTER  MAP     CLUSTER  MAP     READ/WRITE/UPDATE   SERVER  1     SERVER  2     SERVER  3     •  Docs  distributed  evenly  across     ACTIVE     ACTIVE     ACTIVE   servers     Doc  5   Doc   Doc  4   Doc   Doc  1   Doc   •  Each  server  stores  both  ac&ve  and   replica  docs   Doc  2   Doc   Doc  7   Doc   Doc  2   Doc   Only  one  server  ac&ve  at  a  &me   •  Client  library  provides  app  with   Doc  9   Doc   Doc  8   Doc   Doc  6   Doc   simple  interface  to  database   REPLICA   REPLICA   REPLICA   •  Cluster  map  provides  map     to  which  server  doc  is  on   Doc  4   Doc   Doc  6   Doc   Doc  7   Doc   App  never  needs  to  know   Doc  1   Doc   Doc  3   Doc   Doc  9   Doc   •  App  reads,  writes,  updates  docs   Doc  8   Doc   Doc  2   Doc   Doc  5   Doc   •  Mul&ple  app  servers  can  access  same document  at  same  &me   COUCHBASE  SERVER    CLUSTER  User  Configured  Replica  Count  =  1   58  
  59. 59. Cluster  wide  -­‐  Add  Nodes  to  Cluster   APP  SERVER  1   APP  SERVER  2   COUCHBASE  Client  Library   COUCHBASE  Client  Library       CLUSTER  MAP     CLUSTER  MAP     READ/WRITE/UPDATE   READ/WRITE/UPDATE   SERVER  1     SERVER  2     SERVER  3     SERVER  4     SERVER  5     •  Two  servers  added     ACTIVE     ACTIVE     ACTIVE     ACTIVE     ACTIVE   One-­‐click  opera&on   Doc  5   Doc   Doc  4   Doc   Doc  1   Doc   •  Docs  automa&cally   rebalanced  across   Doc  2   Doc   Doc  7   Doc   Doc  2   Doc   cluster   Even  distribu&on  of  docs   Minimum  doc  movement   Doc  9   Doc   Doc  8   Doc   Doc  6   Doc   •  Cluster  map  updated   REPLICA   REPLICA   REPLICA   REPLICA   REPLICA   •  App  database     Doc  4   Doc   Doc  6   Doc   Doc  7   Doc   calls  now  distributed     over  larger  number  of   Doc  1   Doc   Doc  3   Doc   Doc  9   Doc   servers     Doc  8   Doc   Doc  2   Doc   Doc  5   Doc   COUCHBASE  SERVER    CLUSTER  User  Configured  Replica  Count  =  1   59  
  60. 60. Cluster  wide  -­‐  Fail  Over  Node   APP  SERVER  1   APP  SERVER  2   COUCHBASE  Client  Library   COUCHBASE  Client  Library       CLUSTER  MAP     CLUSTER  MAP     SERVER  1     SERVER  2     SERVER  3     SERVER  4     SERVER  5     •  App  servers  accessing  docs             ACTIVE   ACTIVE   ACTIVE   ACTIVE   ACTIVE   •  Requests  to  Server  3  fail   Doc  5   Doc   Doc  4   Doc   Doc  1   Doc   Doc  9   Doc   Doc  6   Doc   •  Cluster  detects  server  failed   Promotes  replicas  of  docs  to   Doc  2   Doc   Doc  7   Doc   Doc  2   Doc   Doc  8   Doc   Doc   ac&ve   Updates  cluster  map   Doc  1   Doc  3   •  Requests  for  docs  now  go  to   REPLICA   REPLICA   REPLICA   REPLICA   REPLICA   appropriate  server   Doc  4   Doc   Doc  6   Doc   Doc  7   Doc   Doc  5   Doc   Doc  8   Doc   •  Typically  rebalance     would  follow   Doc  1   Doc   Doc  3   Doc   Doc  9   Doc   Doc  2   Doc   COUCHBASE  SERVER    CLUSTER  User  Configured  Replica  Count  =  1   60  
  61. 61. Indexing  and  Querying     APP  SERVER  1   APP  SERVER  2   COUCHBASE  Client  Library   COUCHBASE  Client  Library       CLUSTER  MAP     CLUSTER  MAP     Query   SERVER  1   SERVER  2   SERVER  3     •  Indexing  work  is  distributed   ACTIVE     ACTIVE     ACTIVE     amongst  nodes   Doc  5   Doc   Doc  5   Doc   Doc  5   Doc   •  Large  data  set  possible   Doc  2   Doc   Doc  2   Doc   Doc  2   Doc   •  Parallelize  the  effort   Doc  9   Doc   •  Each  node  has  index  for  data  stored Doc  9   Doc   Doc  9   Doc   on  it   REPLICA   REPLICA   REPLICA   •  Queries  combine  the  results  from   Doc  4   Doc   required  nodes   Doc  4   Doc   Doc  4   Doc   Doc  1   Doc   Doc  1   Doc   Doc  1   Doc   Doc  8   Doc   Doc  8   Doc   Doc  8   Doc   COUCHBASE  SERVER    CLUSTER  User  Configured  Replica  Count  =  1   61  
  62. 62. Cross  Data  Center  Replica&on  (XDCR)   SERVER  1     SERVER  2     SERVER  3       ACTIVE     ACTIVE     ACTIVE   COUCHBASE  SERVER    CLUSTER   Doc     Doc   Doc     NY  DATA  CENTER   Doc  2   Doc     Doc     Doc  9   Doc     Doc  RAM   RAM   RAM   Doc     Doc     Doc   Doc     Doc   Doc     Doc   Doc   Doc   DISK   DISK   DISK   SERVER  1     SERVER  2     SERVER  3       ACTIVE     ACTIVE     ACTIVE   Doc     Doc   Doc     Doc  2   Doc     Doc     Doc  9   Doc     Doc   RAM   RAM   RAM   COUCHBASE  SERVER    CLUSTER   Doc     Doc     Doc   Doc     Doc   Doc     Doc   Doc   Doc   SF  DATA  CENTER   DISK   DISK   DISK   62  
  63. 63. THANK  YOU      DIPTI@COUCHBASE.COM   @DBORKAR   63  

×