Couche Base par Tugdual Grall

1,314 views

Published on

Soirée CouchBase (NoSQL) avec Tugdual Grall

Cette présentation commencera par une introduction au NoSQL. Cette introduction expliquera les raisons pour lesquelles de plus en plus de projets utilisent ce type de solutions et les limitations des architectures classiques. Tugdual fera également le tour des différentes approches (Clé Valeur, Document, Colonnes, Graph) et solutions.

Une fois cette introduction effectuée vous verrez en detail la solution Couchbase Server 2.0. Couchbase est une base de données NoSQL orientée Document développée en Open Source. Tugdual presentera en s’appuyant sur des démonstrations.

le développement d’applications Java (et/ou autres selon les gouts) sur du NoSQL en expliquant les bénéfices des solutions
se basant sur des structures de données flexibles (Document JSON)
l’utilisation de Map Reduce pour la création de requetes complexes
la mise en place et l’administration d’un cluster, et les défis associés


Tugdual « Tug » Grall, est Technical Evangelist chez Couchbase, et un développer passionné. Il travaille actuellement avec les communautés de developpers en Europe pour faciliter l’adoption du NoSQL. Tugdual contribue aux SDKs Java et NodeJS de Couchbase.
Avant de rejoindre Couchbase, Tug était le CTP d’eXo la société qui a créé l’outil de développement sur le cloud www.codenvy.com et la pateforme sociale eXo Platform. Auparavant Tugdual a travaillé comme product manager et développeur dans l’équipe Oracle Fusion Middleware sur la plateforme Java EE.
Tug a participé a de nombreuses conférences DevoxxFR, FOSDEM, Google I/O, JavaOne et de nombreux meetups/groupes utilisateurs. Tugdual est co-fondateur du NantesJUG qui organise depuis 2008 des réunions mensuelles autour l’écosystème Java. Lorsqu’il a le temps, il maintient un blog http://tugdualgrall.blogspot.com et une application déployée sur Google App Engine http://www.resultri.com. Vous pouvez le suivre sur twitter.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,314
On SlideShare
0
From Embeds
0
Number of Embeds
184
Actions
Shares
0
Downloads
8
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Couche Base par Tugdual Grall

  1. 1. Wednesday, March 20, 13
  2. 2. Win some cool stuff, send a mail to winning@couchbase.com with NORMANDYJUG in the subject.Wednesday, March 20, 13
  3. 3. Normandy  JUG  -­‐  March  19th  2013 Introduc)on  to  NoSQL with  Couchbase Tugdual  “Tug”  Grall Technical  Evangelist email:  tug@couchbase.com twi0er:  @tgrallWednesday, March 20, 13
  4. 4. About  me • Tugdual  “Tug”  Grall • Web -­‐ Couchbase -­‐    @tgrall -­‐ Technical  Evangelist -­‐      hQp://blog.grallandco.com -­‐ eXo -­‐      tgrall -­‐ CTO • NantesJUG  co-­‐founder -­‐ Oracle • Pet  Project  : -­‐ Developer/Product  Manager • hQp://www.resultri.com -­‐ Mainly  Java/SOA -­‐ Developer  in  consul)ng  firmsWednesday, March 20, 13
  5. 5. INTRO  TO  NOSQLWednesday, March 20, 13
  6. 6. RDBMS  ARE  NOT  ENOUGH?Wednesday, March 20, 13
  7. 7. Growth  is  the  New  Reality • Instagram gained nearly 1 million users overnight when then expanded to AndroidWednesday, March 20, 13
  8. 8. Draw  Something  -­‐  Social  Game 35 million monthly active users in 1 month !!!Wednesday, March 20, 13
  9. 9. By  contrast.... The Simpson’s : Tapped Out Daily Active Users (Millions)Wednesday, March 20, 13
  10. 10. How  do  you  take  this  growth? Applica@on  Scales  Out Just  add  more  commodity  web  servers System  Cost Applica)on  Performance   Web/App  Server  Tier Users RDBMS  Scales  Up Get  a  bigger,  more  complex  server System  Cost Applica)on  Performance   Won’t  scale   beyond  this   point Rela@onal  Database Users RDBMS  is  good  for  many  thing,  but  hard  to  scaleWednesday, March 20, 13
  11. 11. Scaling  out  RDBMS • Run  Many  SQL  Servers Web/App  Server  Tier • Data  could  be  shared -­‐ Done  by  the  applica)on  code Memcached  Tier • Caching  for  faster  response  )me MySQL  TierWednesday, March 20, 13
  12. 12. NoSQL  Technology  Scales  Out Applica@on  Scales  Out Just  add  more  commodity  web  servers System  Cost Applica)on  Performance   Web/App  Server  Tier Users NoSQL  Database  Scales  Out Cost  and  performance  mirrors  app  @er System  Cost Applica)on  Performance   NoSQL  Distributed  Data  Store Users Scaling  out  fla;ens  the  cost  and  performance  curvesWednesday, March 20, 13
  13. 13. A  New  Technology? Bigtable Dynamo Cassandra Voldemort November  2006 October  2007 August  2008 February  2009 Building  new  database  to  answer  the  following  requirements • No  schema  required  before  inser)ng  data • No  schema  change  required  to  change  data  format • Auto-­‐sharding  without  applica)on  par)cipa)on • Distributed  queries • Integrated  main  memory  caching • Data  synchroniza)on  (  mul)-­‐datacenter) Very  few  organiza@ons  want  to  (fewer  can)  build  and  maintain  database  soQware  technology. But  every  organiza@on  building  interac@ve  web  applica@ons  needs  this  technology.Wednesday, March 20, 13
  14. 14. What  Is  Biggest  Data  Management  Problem  Driving   Use  of  NoSQL  in  Coming  Year? 49% 35% 29% 16% 12% 11% Lack  of  flexibility/ Inability  to  scale   Performance   Cost All  of  these Other rigid  schemas out  data challenges Source:  Couchbase  Survey,  December  2011,  n  =  1351.Wednesday, March 20, 13
  15. 15. NO  SQL  TAXONOMIESWednesday, March 20, 13
  16. 16. NoSQL  Catalog Key-­‐Value Data  Structure Document Column Graph(memory  only) Cache Memcached Redis(memory/disk) Database Membase Couchbase Cassandra Neo4j MongoDBWednesday, March 20, 13
  17. 17. NoSQL  Catalog Key-­‐Value Data  Structure Document Column Graph(memory  only) Cache Memcached Redis Coherence(memory/disk) Database Membase Couchbase Cassandra Neo4j MongoDB HBase InfiniteGraphWednesday, March 20, 13
  18. 18. Hadoop  ?Wednesday, March 20, 13
  19. 19. OperaIonal  vs.  AnalyIc  Databases Real-­‐Cme,   AnalyCc InteracCve  Databases Databases NoSQL Fast  access   Get  insights  from   to  data data Couchbase Cassandra Cloudera MongoDB Hbase Hortonworks MaprWednesday, March 20, 13
  20. 20. COUCHBASEWednesday, March 20, 13
  21. 21. Couchbase  Server  Core  Principles Easy   Consistent  High   Scalability PE RF O R M A N C E Performance Grow  cluster  without   Consistent  sub-­‐millisecond   applica^on  changes,  without   read  and  write  response  ^mes   down^me  with  a  single  click with  consistent  high  throughput Always  On   JSON JSON JSO Flexible  Data   24x365 Model JSON N JSON No  down^me  for  so`ware   JSON  document  model  with  no   upgrades,  hardware   fixed  schema. maintenance,  etc.Wednesday, March 20, 13
  22. 22. Couchbase  2.0  New  Features JSON support Indexing and Querying Incremental Map Cross data center Reduce replicationWednesday, March 20, 13
  23. 23. Couchbase  Handles  Real  World  ScaleWednesday, March 20, 13
  24. 24. 8092 Couchbase  Server  2.0  Architecture 11211 11210 Query  API Memcapable    1.0 Memcapable    2.0 Sub^tle Moxi Query  Engine REST  management  API/Web  UI vBucket  state  and  replica)on  manager Memcached Global  singleton  supervisor Rebalance  orchestrator Configura)on  manager Node  health  monitor Process  monitor Couchbase  EP  Engine Heartbeat Data  Manager Cluster  Manager storage  interface New  Persistence  Layer hdp on  each  node one  per  cluster Erlang/OTP HTTP Erlang  port  mapper Distributed  Erlang 8091 4369 21100  -­‐  21199Wednesday, March 20, 13
  25. 25. Couchbase  Server  2.0  Architecture 8092 11211 11210 Query  API Memcapable    1.0 Memcapable    2.0 Moxi Query  Engine REST  management  API/Web  UI vBucket  state  and  replica)on  manager Object-­‐level  Cache Global  singleton  supervisor RAM  Cache,   Rebalance  orchestrator Configura)on  manager Node  health  monitor Server/Cluster   Process  monitor Indexing  &   Heartbeat Couchbase  EP  Engine Management  &   Persistence   storage  interface Communica@on Management (Erlang) New  (C  Persistence Disk   &  V8) Persistence  Layer hdp on  each  node one  per  cluster Erlang/OTP The Unreasonable Effectiveness of C by Damien Katz HTTP Erlang  port  mapper Distributed  Erlang 8091 4369 21100  -­‐  21199Wednesday, March 20, 13
  26. 26. Open  Source  Project Apache  2.0 hQps://github.com/couchbase/ Gerrit: hQp://review.couchbase.org/ hQps://github.com/couchbaselabs/Wednesday, March 20, 13
  27. 27. SETTING  UP  TO  DEVELOPWednesday, March 20, 13
  28. 28. Install  Couchbase  Server  2.0 Ubuntu RedHat Mac  OS  X Windows or  build  from  sourcesWednesday, March 20, 13
  29. 29. Official  SDKs Ruby libcouchbase Clojure Python www.couchbase.com/develop GoWednesday, March 20, 13
  30. 30. Client  SDKs c)on on n e make  c y ol og Couchbase   eceiv e  top r Couchbase  Client Topology App  Server UpdateWednesday, March 20, 13
  31. 31. COUCHBASE  OPERATIONSWednesday, March 20, 13
  32. 32. Write  OperaIon Doc  1 App  Server 3 3 Managed  Cache 2 To  other  node Replica^on   Doc  1 Queue Disk  Queue Disk Couchbase  Server  NodeWednesday, March 20, 13
  33. 33. Basic  OperaIons APP  SERVER  1 APP  SERVER  2 COUCHBASE  Client  Library COUCHBASE  Client  Library CLUSTER  MAP CLUSTER  MAP • Docs  distributed  evenly  across  servers   READ/WRITE/UPDATE • Each  server  stores  both  ac@ve  and  replica   SERVER  1 SERVER  2 SERVER  3 docs Only  one  doc  ac)ve  at  a  )me ACTIVE ACTIVE ACTIVE • Client  library  provides  app  with  simple   Doc  5 Doc Doc  4 Doc Doc  1 Doc interface  to  database Doc  2 Doc Doc  7 Doc Doc  2 Doc • Cluster  map  provides  map   to  which  server  doc  is  on Doc  9 Doc Doc  8 Doc Doc  6 Doc App  never  needs  to  know • App  reads,  writes,  updates  docs REPLICA REPLICA REPLICA • Mul@ple  app  servers  can  access  same   Doc  4 Doc Doc  6 Doc Doc  7 Doc document  at  same  @me Doc  1 Doc Doc  3 Doc Doc  9 Doc Doc  8 Doc Doc  2 Doc Doc  5 Doc COUCHBASE  SERVER    CLUSTERWednesday, March 20, 13
  34. 34. Store  &  Retrieve  OperaIons • get  (key) –  Retrieve  a  document • set  (key,  value) –  Store  a  document,  overwrites  if  exists • add  (key,  value) –  Store  a  document,  error/excep^on  if  exists • replace  (key,  value) –  Store  a  document,  error/excep^on  if  doesn’t  exist • cas  (key,  value,  cas) –  Compare  and  swap,  mutate  document  only  if  it  hasn’t  changed   while  execu^ng  this  opera^onWednesday, March 20, 13
  35. 35. Atomic  Counter  OperaIons These  opera^ons  are  always  executed  in  order  atomically. • set  (key,  value) –  Use  set  to  ini^alize  the  counter • cb.set(“my_counter”,  1) • incr  (key) –  Increase  an  atomic  counter  value,  default  by  1 • cb.incr(“my_counter”)  #  now  it’s  2 • decr  (key) –  Decrease  an  atomic  counter  value,  default  by  1 • cb.decr(“my_counter”)  #  now  it’s  1Wednesday, March 20, 13
  36. 36. Mental  Adjustments • In  SQL  we  tend  to  want  to  avoid  hilng  the  database  as  much  as   possible – Even  with  caching  and  indexing  tricks,  and  massive   improvements  over  the  years,  SQL  s^ll  gets  bogged  down  by   complex  joins  and  huge  indexes,  so  we  avoid  making   database  calls • In  Couchbase,  get’s  and  set’s  are  so  fast  they  are  trivial,  not   bo0lenecks,  this  is  hard  for  many  people  to  accept  at  first;   Mul^ple  get  statements  are  commonplace,  don’t  avoid  it!  Wednesday, March 20, 13
  37. 37. JSON  Document  Structure meta { Meta  Informa@on   “id”:  “u::jasdeep@couchbase.com”, Including  Key “rev”:  “1-­‐0002bce0000000000”, “flags”:  0, “expira@on”:  0, All  Keys  Unique  and   “type”:  “json” Kept  in  RAM } document { “uid”:  123456, Document  Value “firstname”:  “jasdeep”, “lastname”:  “Jaitla”, Most  Recent  In  Ram   “age”:  22, “favorite_colors”:  [“blue”,  “black”], And  Persisted  To  Disk “email”:  “jasdeep@couchbase.com” }Wednesday, March 20, 13
  38. 38. DEMONSTRATIONWednesday, March 20, 13
  39. 39. Add  Nodes  to  Cluster APP  SERVER  1 APP  SERVER  2 COUCHBASE  Client  Library COUCHBASE  Client  Library CLUSTER  MAP CLUSTER  MAP READ/WRITE/UPDATE READ/WRITE/UPDATE SERVER  1 SERVER  2 SERVER  3 SERVER  4 SERVER  5 ACTIVE ACTIVE ACTIVE ACTIVE ACTIVE • Two  servers  added Doc  5 Doc Doc  4 Doc Doc  1 Doc One-­‐click  opera@on • Docs  automa@cally   Doc  2 Doc Doc  7 Doc Doc  2 Doc rebalanced  across  cluster Even  distribu)on  of  docs Doc  9 Doc Doc  8 Doc Doc  6 Doc Minimum  doc  movement • Cluster  map  updated REPLICA REPLICA REPLICA REPLICA REPLICA • App  database   Doc  4 Doc Doc  6 Doc Doc  7 Doc calls  now  distributed   over  larger  number  of   Doc  1 Doc Doc  3 Doc Doc  9 Doc servers Doc  8 Doc Doc  2 Doc Doc  5 Doc COUCHBASE  SERVER    CLUSTERUser  Configured  Replica  Count  =  1Wednesday, March 20, 13
  40. 40. Fail  Over  Node APP  SERVER  1 APP  SERVER  2 COUCHBASE  Client  Library COUCHBASE  Client  Library CLUSTER  MAP CLUSTER  MAP SERVER  1 SERVER  2 SERVER  3 SERVER  4 SERVER  5 • App  servers  accessing  docs ACTIVE ACTIVE ACTIVE ACTIVE ACTIVE Doc  5 Doc Doc  4 Doc Doc  1 Doc Doc  9 Doc Doc  6 Doc • Requests  to  Server  3  fail Doc  2 Doc Doc  7 Doc Doc  2 Doc Doc  8 Doc Doc • Cluster  detects  server  failed Promotes  replicas  of  docs  to  ac)ve Updates  cluster  map Doc  1 Doc  3 • Requests  for  docs  now  go  to   appropriate  server REPLICA REPLICA REPLICA REPLICA REPLICA Doc  4 Doc Doc  6 Doc Doc  7 Doc Doc  5 Doc Doc  8 Doc • Typically  rebalance   would  follow Doc  1 Doc Doc  3 Doc Doc  9 Doc Doc  2 Doc COUCHBASE  SERVER    CLUSTERUser  Configured  Replica  Count  =  1 Wednesday, March 20, 13
  41. 41. Indexing  and  Querying APP  SERVER  1 APP  SERVER  2 COUCHBASE  Client  Library COUCHBASE  Client  Library CLUSTER  MAP CLUSTER  MAP Query SERVER  1 SERVER  2 SERVER  3 ACTIVE • Indexing  work  is  distributed  amongst   ACTIVE ACTIVE nodes Doc  5 Doc Doc  5 Doc Doc  5 Doc • Large  data  set  possible Doc  2 Doc Doc  2 Doc Doc  2 Doc • Parallelize  the  effort Doc  9 Doc Doc  9 Doc Doc  9 Doc • Each  node  has  index  for  data  stored  on  it REPLICA REPLICA REPLICA • Queries  combine  the  results  from   required  nodes Doc  4 Doc Doc  4 Doc Doc  4 Doc Doc  1 Doc Doc  1 Doc Doc  1 Doc Doc  8 Doc Doc  8 Doc Doc  8 Doc COUCHBASE  SERVER    CLUSTERWednesday, March 20, 13
  42. 42. DEMONSTRATIONWednesday, March 20, 13
  43. 43. Cross  Data  Center  ReplicaIon  (XDCR) Couchbase  Cluster Couchbase  Cluster West  Coast  Data  Center East  Coast  Data  Center SERVER  1 SERVER  2 SERVER  3 SERVER  1 SERVER  2 SERVER  3 RAM  CACHE RAM  CACHE RAM  CACHE RAM  CACHE RAM  CACHE RAM  CACHE Doc  1 Doc Doc  1 Doc Doc  1 Doc Doc  1 Doc Doc  1 Doc Doc  1 Doc Doc  2 Doc Doc  2 Doc Doc  2 Doc Doc  2 Doc Doc  2 Doc Doc  2 Doc Doc  6 Doc Doc  6 Doc Doc  6 Doc Doc  6 Doc Doc  6 Doc Doc  6 Doc DISK DISK DISK DISK DISK DISK Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc DocWednesday, March 20, 13
  44. 44. Map  FuncIonWednesday, March 20, 13
  45. 45. DEMONSTRATIONWednesday, March 20, 13
  46. 46. ElasIc  Search  Adaptor • Elastic Search is good for ad-hoc queries and faceted browsing • Our adapter is aware of changing Couchbase topology • Indexed by Elastic Search after stored to disk in Couchbase Elas@cSearchWednesday, March 20, 13
  47. 47. I’m  Excited  to  See  What  You  Build, Q  &  A Couchbase  Docs Contact  me  on  Twider www.couchbase.com/docs/index-­‐full.html @tgrall Couchbase  Forums Contact  me  by  Email www.couchbase.com/forums tug@couchbase.com IRC Learn  More  About  Design  Paderns #couchbase CouchbaseModels.com #libcouchbase Senng  up  for  Ruby  on  Rails CouchbaseOnRails.comWednesday, March 20, 13
  48. 48. Win some cool stuff, send a mail to winning@couchbase.com with NORMANDYJUG in the subject.Wednesday, March 20, 13
  49. 49. Q&AWednesday, March 20, 13

×