Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Couchbase by Tugdual Grall

6,698 views

Published on

Discover the miracle of Couchbase, Document-Oriented NoSQL Database technology. A conference by Tudgdual Tall for Elsass JUG.

Published in: Technology
  • Be the first to like this

Couchbase by Tugdual Grall

  1. 1. Lorraine JUG - 19-JUN-2013 Friday, June 21, 13
  2. 2. Elsass JUG - 20-JUN-2013 Send  a  mail  to: winning@couchbase.com ElsassJUG  in  the  subject... Join  the  French  Couchbase  Meetup h3p://www.meetup.com/Couchbase-­‐France/ Friday, June 21, 13
  3. 3. Elsass JUG - 20-JUN-2013 Introduc)on  to  NoSQL   with  Couchbase Tugdual  ‘Tug’  Grall @tgrall Technical  Evangelist Friday, June 21, 13
  4. 4. Elsass JUG - 20-JUN-2013 About  me... • Tugdual  “Tug”  Grall ­ Couchbase • Technical  Evangelist ­ eXo • CTO ­ Oracle • Developer/Product  Manager ­ Mainly  Java/SOA ­ Developer  in  consul)ng  firms • Web ­  @tgrall ­      hLp://blog.grallandco.com ­      tgrall ­ NantesJUG  co-­‐founder ­ Pet  Project  : ­ hLp://www.resultri.com Friday, June 21, 13
  5. 5. Elsass JUG - 20-JUN-2013 Agenda • Why  NoSQL  ? • NoSQL  Landscape • Document  Design • Use  Cases • Couchbase  Server  2.0 Friday, June 21, 13
  6. 6. Elsass JUG - 20-JUN-2013 $30B  Database  Market  Being  Disrupted <50%? 2027 95% RelaVonal   Technology 2012 All  new  database  growth  will  be  NoSQL RelaVonal   Technology RelaVonal   Technology RelaVonal   Technology NoSQL Technology Other Friday, June 21, 13
  7. 7. Elsass JUG - 20-JUN-2013 Three  Macro  Trends  Driving  DisrupVon Big  Data SaaS/Cloud   CompuHngBig  Users Friday, June 21, 13
  8. 8. Elsass JUG - 20-JUN-2013 Big  Data High  Data  Variety  and  Velocity Source:  IDC  2011  Digital  Universe  Study  (hLp://www.emc.com/collateral/demos/microsites/emc-­‐digital-­‐universe-­‐2011/index.htm) Trillions  of  Gigabytes  (ZeLabytes) 0 0.50 1.00 1.50 2.00 2000 2006 2011 Unstructured  and  Semi-­‐ Structured  Data Structured  Data Text,  Log  Files,  Click   Streams,  Blogs,   Tweets,  Audio,   Video,  etc. More  Flexible  Data  Model  Required Friday, June 21, 13
  9. 9. Elsass JUG - 20-JUN-2013 RDBMS  are  not  Enough? Friday, June 21, 13
  10. 10. Elsass JUG - 20-JUN-2013 Growth  is  the  New  Reality • Instagram  gained  nearly  1  million  users  overnight  when  then  expanded  to  Android Friday, June 21, 13
  11. 11. Elsass JUG - 20-JUN-2013 Draw  Something  Viral  Growth 191715131197533/12826242220181614121082/6 Draw  Something  by  OMGPOP Daily  Ac)ve  Users  (millions) 21 2 4 6 8 10 12 14 16 Friday, June 21, 13
  12. 12. Elsass JUG - 20-JUN-2013 RDBMS  is  good  for  many  thing,  but  hard  to  scale RDBMS  Scales  Up Get  a  bigger,  more  complex  server Users ApplicaVon  Scales  Out Just  add  more  commodity  web  servers Users System  Cost Applica)on  Performance   RelaHonal  Database Web/App  Server  Tier System  Cost Applica)on  Performance   Won’t  scale   beyond  this   point How  do  you  take  this  growth? Friday, June 21, 13
  13. 13. Elsass JUG - 20-JUN-2013 Web/App  Server  Tier Memcached  Tier MySQL  Tier Scaling  out  RDBMS • Run  Many  SQL  Servers • Data  could  be  sharded ­ Done  by  the  applica)on  code • Caching  for  faster  response  Vme Friday, June 21, 13
  14. 14. Elsass JUG - 20-JUN-2013 Scaling  out  fla3ens  the  cost  and  performance  curves NoSQL  Database  Scales  Out Cost  and  performance  mirrors  app  Ver Users NoSQL  Distributed  Data  Store Web/App  Server  Tier ApplicaVon  Scales  Out Just  add  more  commodity  web  servers Users System  Cost Applica)on  Performance   Applica)on  Performance   System  Cost NoSQL  Technology  Scales  Out Friday, June 21, 13
  15. 15. Elsass JUG - 20-JUN-2013 Dynamo October  2007 Cassandra August  2008 Bigtable November  2006 Voldemort February  2009 Very  few  organizaHons  want  to  (fewer  can)  build  and  maintain  database  so]ware  technology. But  every  organizaHon  building  interacHve  web  applicaHons  needs  this  technology. A  New  Technology? • Building  new  database  to  answer  the  following  requirements ­ No  schema  required  before  inser)ng  data ­ No  schema  change  required  to  change  data  format ­ Auto-­‐sharding  without  applica)on  par)cipa)on ­ Distributed  queries ­ Integrated  main  memory  caching ­ Data  synchroniza)on  (  mul)-­‐datacenter) Friday, June 21, 13
  16. 16. Elsass JUG - 20-JUN-2013 Lack  of  flexibility/ rigid  schemas Inability  to  scale  out  data Performance  challenges Cost All  of  these Other 49% 35% 29% 16% 12% 11% What  Is  Biggest  Data  Management  Problem  Driving   Use  of  NoSQL  in  Coming  Year? Source:  Couchbase  Survey,  December  2011,  n  =  1351. Friday, June 21, 13
  17. 17. Elsass JUG - 20-JUN-2013 NoSQL  Catalog Key-­‐Value Memcached Membase Redis Data  Structure Document Column Graph MongoDB Couchbase Cassandra Cache (memory  only) Database (memory/disk) Neo4j HBase InfiniteGraph Coherence Friday, June 21, 13
  18. 18. Elsass JUG - 20-JUN-2013 OperaVonal  vs.  AnalyVc  Databases Cloudera Hortonworks Mapr Couchbase MongoDB Cassandra Hbase AnalyHc Databases Get  insights  from   data Real-­‐Hme,   InteracHve  Databases Fast  access   to  data NoSQL Friday, June 21, 13
  19. 19. Elsass JUG - 20-JUN-2013 Hadoop Friday, June 21, 13
  20. 20. Elsass JUG - 20-JUN-2013 Use  Cases Friday, June 21, 13
  21. 21. Elsass JUG - 20-JUN-2013 Use  Cases Key  Value •  Session  Management •  User  Profile/Preferences •  Shopping  Cart Document •  Event  Logging •  Content  Management   •  Web  Analy)cs •  E-­‐Commerce  Applica)on Columns •  Event  Logging •  Content  Management •  Counters Graph •  Connected  Data  /    Social  Networks •  Rou)ng,  Dispatch •  Recommenda)ons  based  on  Social  Graph Friday, June 21, 13
  22. 22. Elsass JUG - 20-JUN-2013 Data  Models Friday, June 21, 13
  23. 23. Elsass JUG - 20-JUN-2013 RelaVonal  vs  Document  Data  Model RelaHonal  data  model Document  data  model Collec)on  of  complex  documents  with arbitrary,  nested  data  formats  and varying  “record”  format. Highly-­‐structured  table  organiza)on   with  rigidly-­‐defined  data  formats  and   record  structure. C1 C2 C3 C4 JSON JSON JSON { } Friday, June 21, 13
  24. 24. Elsass JUG - 20-JUN-2013 The  RelaVonal  Approach  to  Storing  Data Rela)onal  databases  were  not  designed  with  clusters  in  mind,  which   is  why  people  have  cast  around  for  an  alterna)ve.  Storing  aggregates   as  fundamental  units  makes  a  lot  of  sense  for  running  on  a  cluster.   hLp://mar)nfowler.com/bliki/AggregateOrientedDatabase.html Friday, June 21, 13
  25. 25. Elsass JUG - 20-JUN-2013 Document  Database  by  Comparison o::1001 { uid: “ji22jd”, customer: “Ann”, line_items: [ { sku: 0321293533, quan: 2, unit_price: 48.0 }, { sku: 0321601912, quan: 1, unit_price: 39.0 }, { sku: 0131495054, quan: 1, unit_price: 51.0 } ], payment: { type: “Amex”, expiry: “04/2001”, last5: 12345 } } Easy  to  distribute  data Makes  sense  to  applica)on  programmers Friday, June 21, 13
  26. 26. Elsass JUG - 20-JUN-2013 Objects  Serialized  to  JSON  and  Back   User  Object string uid string firstname string lastname int age array favorite_colors string email u::john@couchbase.com { “uid”:  123456, “firstname”:  “John”, “lastname”:  “Smith”, “age”:  22, “favorite_colors”:  [“blue”,  “black”], “email”:  “john@couchbase.com” } User  Object string uid string firstname string lastname int age array favorite_colors string email u::john@couchbase.com { “uid”:  123456, “firstname”:  “john”, “lastname”:  “smith”, “age”:  22, “favorite_colors”:  [“blue”,  “black”], “email”:  “john@couchbase.com” } set() get() Friday, June 21, 13
  27. 27. Elsass JUG - 20-JUN-2013 Schema  Changes V1 Speaker Robin  Johnson Tug  Grall John  Zablo Event  Info ID Title Speaker_id Frank  Weigel 1 Paris  JUG rbin 2 QCON  UK tgrall 3 NoSQL  CLN johnz 4 Devoxx fweigel ID Name rbin tgrall johnz fweigel Friday, June 21, 13
  28. 28. Speaker Robin  Johnson Tug  Grall John  Zablo Event  Info ID Title Speaker_id Frank  Weigel 1 Paris  JUG rbin 2 QCON  UK tgrall 3 NoSQL  CLN johnz 4 Devoxx fweigel ID Name rbin tgrall johnz fweigel Elsass JUG - 20-JUN-2013 Schema  Changes V2 A  talk  could  be  done  by  one  or  more  speakers! 1.Create  a  new  table 2.“Prepare”  the  Data        (move  to  new  table,  delete  column) 3.Change  the  constraints  and  foreign  keys 4.Change  the  applicaZon  code 5.Hope  that  your  DBA  is  in  a  good  day Friday, June 21, 13
  29. 29. Elsass JUG - 20-JUN-2013 Schema  Changes V1 event:1 { type: “event”, title: “Paris JUG”, speaker: “rbin” } event:1 { type: “event”, title: “Paris JUG”, speaker: “rbin” } event:1 { type: “event”, title: “Paris JUG”, speaker: “rbin” } event:1 { type: “event”, title: “Paris JUG”, speaker: “rbin” } event:1 { type: “event”, title: “Paris JUG”, speaker: “rbin” } user:rbin { type: “user”, name: “Robin Johnson”, } user:rbin { type: “user”, name: “Robin Johnson”, } user:rbin { type: “user”, name: “Robin Johnson”, } user:rbin { type: “user”, name: “Robin Johnson”, } user:rbin { type: “user”, name: “Robin Johnson”, } user:rbin { type: “user”, name: “Robin Johnson”, id: “rbin” } Friday, June 21, 13
  30. 30. Elsass JUG - 20-JUN-2013 Schema  Changes V2 event:1 { type: “event”, title: “Paris JUG”, speaker: “rbin” } event:1 { type: “event”, title: “Paris JUG”, speaker: “rbin” } event:1 { type: “event”, title: “Paris JUG”, speaker: “rbin” } event:1 { type: “event”, title: “Paris JUG”, speaker: “rbin” } event:1 { type: “event”, title: “Paris JUG”, speaker: “rbin” } user:rbin { type: “user”, name: “Robin Johnson”, } user:rbin { type: “user”, name: “Robin Johnson”, } user:rbin { type: “user”, name: “Robin Johnson”, } user:rbin { type: “user”, name: “Robin Johnson”, } user:rbin { type: “user”, name: “Robin Johnson”, } user:rbin { type: “user”, name: “Robin Johnson”, id: “rbin” } A  talk  could  be  done  by  one  or  more  speakers! Friday, June 21, 13
  31. 31. Elsass JUG - 20-JUN-2013 Schema  Changes V2 event:1 { type: “event”, title: “Paris JUG”, speaker: “rbin” } event:1 { type: “event”, title: “Paris JUG”, speaker: “rbin” } event:1 { type: “event”, title: “Paris JUG”, speaker: “rbin” } event:1 { type: “event”, title: “Paris JUG”, speaker: “rbin” } event:1 { type: “event”, title: “Paris JUG”, speakers: {“rbin”, “tgrall”} } user:rbin { type: “user”, name: “Robin Johnson”, } user:rbin { type: “user”, name: “Robin Johnson”, } user:rbin { type: “user”, name: “Robin Johnson”, } user:rbin { type: “user”, name: “Robin Johnson”, } user:rbin { type: “user”, name: “Robin Johnson”, } user:rbin { type: “user”, name: “Robin Johnson”, id: “rbin: } 1.Change  the  App 1.Update  the  data  (‘when  you  want’) 2.Change  your  “index”  to  query  the  new  data You  can  do  it  in  a  “lazy”  way  .. Friday, June 21, 13
  32. 32. Elsass JUG - 20-JUN-2013 QuesVons  ?  Concerns  ? • Data  DuplicaVon? • Schema  ? • TransacVon  ? • DBA  ? • Backup/Restore  ? • ... ? Friday, June 21, 13
  33. 33. Elsass JUG - 20-JUN-2013 Couchbase  Server NoSQL  Document  Database Friday, June 21, 13
  34. 34. Elsass JUG - 20-JUN-2013 Easy  Scalability Consistent  High   Performance Always  On   24x365 Grow  cluster  without  applica)on  changes,   without  down)me  with  a  single  click Consistent  sub-­‐millisecond   read  and  write  response  )mes   with  consistent  high  throughput No  down)me  for  sorware  upgrades,   hardware  maintenance,  etc. Flexible  Data  Model JSON  document  model  with  no  fixed   schema. Couchbase  Server Friday, June 21, 13
  35. 35. Elsass JUG - 20-JUN-2013 Open  Source  Project Friday, June 21, 13
  36. 36. Elsass JUG - 20-JUN-2013 Couchbase  Handles  Real  World  Scale Friday, June 21, 13
  37. 37. Elsass JUG - 20-JUN-2013 Heartbeat Process  monitor Global  singleton  supervisor Configura)on  manager on  each  node Rebalance  orchestrator Node  health  monitor one  per  cluster vBucket  state  and  replica)on  manager hbp REST  management  API/Web  UI HTTP 8091 Erlang  port  mapper 4369 Distributed  Erlang 21100  -­‐  21199 Erlang/OTP storage  interface Couchbase  EP  Engine 11210 Memcapable    2.0 Moxi 11211 Memcapable    1.0 Memcached New  Persistence  Layer 8092 Query  API Query  Engine Data  Manager Cluster  Manager Couchbase  Server  Architecture Friday, June 21, 13
  38. 38. Elsass JUG - 20-JUN-2013 Heartbeat Process  monitor Global  singleton  supervisor Configura)on  manager on  each  node Rebalance  orchestrator Node  health  monitor one  per  cluster vBucket  state  and  replica)on  manager hbp REST  management  API/Web  UI HTTP 8091 Erlang  port  mapper 4369 Distributed  Erlang 21100  -­‐  21199 Erlang/OTP storage  interface Couchbase  EP  Engine 11210 Memcapable    2.0 Moxi 11211 Memcapable    1.0 Memcached New  Persistence  Layer 8092 Query  API Query  Engine Couchbase  Server  Architecture Friday, June 21, 13
  39. 39. Elsass JUG - 20-JUN-2013 Couchbase  Opera)ons Friday, June 21, 13
  40. 40. Elsass JUG - 20-JUN-2013 33 2 Single  node  -­‐  Couchbase  Write  Opera)on Managed  Cache Disk  Queue Disk Replica)on  Queue App  Server Couchbase  Server  Node Doc  1Doc  1 Doc  1 To  other  node Friday, June 21, 13
  41. 41. Elsass JUG - 20-JUN-2013 COUCHBASE  SERVER    CLUSTER Basic  Opera)on • Docs  distributed  evenly  across  servers   • Each  server  stores  both  acZve  and  replica  docs Only  one  server  ac)ve  at  a  )me • Client  library  provides  app  with  simple  interface  to   database • Cluster  map  provides  map   to  which  server  doc  is  on App  never  needs  to  know • App  reads,  writes,  updates  docs • MulZple  app  servers  can  access  same  document  at   same  Zme User  Configured  Replica  Count  =  1 READ/WRITE/UPDATE ACTIVE SERVER  1 ACTIVE Doc  4 Doc  7 Doc Doc Doc SERVER  2 Doc  8 ACTIVE Doc  1 Doc  2 Doc Doc Doc REPLICA Doc  4 Doc  1 Doc  8 Doc Doc Doc REPLICA Doc  6 Doc  3 Doc  2 Doc Doc Doc REPLICA Doc  7 Doc  9 Doc  5 Doc Doc Doc SERVER  3 Doc  6 Doc  5 Doc  2 Doc Doc DocDoc  9 APP  SERVER  1 COUCHBASE  Client  Library CLUSTER  MAP COUCHBASE  Client  Library CLUSTER  MAP APP  SERVER  2 Friday, June 21, 13
  42. 42. Elsass JUG - 20-JUN-2013 Add  nodes  to  the  cluster • Two  servers  added One-­‐click  operaZon • Docs  automaZcally   rebalanced  across  cluster Even  distribu)on  of  docs Minimum  doc  movement • Cluster  map  updated • App  database   calls  now  distributed   over  larger  number  of   servers REPLICA ACTIVE Doc  5 Doc  2 Doc Doc Doc  4 Doc  1 Doc Doc SERVER  1 REPLICA ACTIVE Doc  4 Doc  7 Doc Doc Doc  6 Doc  3 Doc Doc SERVER  2 REPLICA ACTIVE Doc  1 Doc  2 Doc Doc Doc  7 Doc  9 Doc Doc SERVER  3 SERVER  4 SERVER  5 REPLICA ACTIVE REPLICA ACTIVE Doc Doc  8 Doc Doc  9 Doc Doc  2 Doc Doc  8 Doc Doc  5 Doc Doc  6 READ/WRITE/UPDATE READ/WRITE/UPDATE APP  SERVER  1 COUCHBASE  Client  Library COUCHBASE  Client  Library CLUSTER  MAP CLUSTER  MAP APP  SERVER  2 COUCHBASE  SERVER    CLUSTER Friday, June 21, 13
  43. 43. Elsass JUG - 20-JUN-2013 Fail  Over  Node REPLICA ACTIVE Doc  5 Doc  2 Doc Doc Doc  4 Doc  1 Doc Doc SERVER  1 REPLICA ACTIVE Doc  4 Doc  7 Doc Doc Doc  6 Doc  3 Doc Doc SERVER  2 REPLICA ACTIVE Doc  1 Doc  2 Doc Doc Doc  7 Doc  9 Doc Doc SERVER  3 SERVER  4 SERVER  5 REPLICA ACTIVE REPLICA ACTIVE Doc  9 Doc  8 Doc Doc  6 Doc Doc Doc  5 Doc Doc  2 Doc  8 Doc Doc • App  servers  accessing  docs • Requests  to  Server  3  fail • Cluster  detects  server  failed Promotes  replicas  of  docs  to  ac)ve Updates  cluster  map • Requests  for  docs  now  go  to   appropriate  server • Typically  rebalance   would  follow Doc Doc  1 Doc  3 APP  SERVER  1 COUCHBASE  Client  Library CLUSTER  MAP COUCHBASE  Client  Library CLUSTER  MAP APP  SERVER  2 COUCHBASE  SERVER    CLUSTER Friday, June 21, 13
  44. 44. Elsass JUG - 20-JUN-2013 Client  SDK www.couchbase.com/develop Clojure Python Ruby libcouchbase Go Friday, June 21, 13
  45. 45. Elsass JUG - 20-JUN-2013 Demo  Time Friday, June 21, 13
  46. 46. Elsass JUG - 20-JUN-2013 Indexing  and  Querying COUCHBASE  SERVER    CLUSTER ACTIVE Doc  5 Doc  2 Doc Doc Doc SERVER  1 REPLICA Doc  4 Doc  1 Doc  8 Doc Doc Doc APP  SERVER  1 COUCHBASE  Client  Library CLUSTER  MAP COUCHBASE  Client  Library CLUSTER  MAP APP  SERVER  2 Doc  9 • Indexing  work  is  distributed  amongst   nodes • Large  data  set  possible • Parallelize  the  effort • Each  node  has  index  for  data  stored  on  it • Queries  combine  the  results  from   required  nodes ACTIVE Doc  5 Doc  2 Doc Doc Doc SERVER  2 REPLICA Doc  4 Doc  1 Doc  8 Doc Doc Doc Doc  9 ACTIVE Doc  5 Doc  2 Doc Doc Doc SERVER  3 REPLICA Doc  4 Doc  1 Doc  8 Doc Doc Doc Doc  9 Query Friday, June 21, 13
  47. 47. Elsass JUG - 20-JUN-2013 Map  FuncVon Friday, June 21, 13
  48. 48. Elsass JUG - 20-JUN-2013 Cross  Data  ReplicaVon US  DATA  CENTER EUROPE  DATA  CENTER ASIA  DATA  CENTER ReplicaZon ReplicaZon ReplicaZon • Data  close  to  users • MulVple  locaVons  for  disaster  recovery • Independently  managed  clusters  serving  local  data Friday, June 21, 13
  49. 49. Elsass JUG - 20-JUN-2013 www.couchbase.com/download Couchbase  Server   Friday, June 21, 13
  50. 50. Elsass JUG - 20-JUN-2013 Q  &  A Meetup h;p://meetup.com/Couchbase-­‐France/ Contacts @tgrall  /  tug@couchbase.com Friday, June 21, 13
  51. 51. Elsass JUG - 20-JUN-2013 Thank  you! tug@couchbase.com @tgrall Join  the  French  Couchbase  Meetup h3p://www.meetup.com/Couchbase-­‐France/ Friday, June 21, 13
  52. 52. Lorraine JUG - 19-JUN-2013 Friday, June 21, 13

×