Softshake 2013 Intro to NoSQL

366 views

Published on

Tugdual Grall's Introduction to NoSQL presentation originally given at the Softshake event in Geneva, Switzerland. Follow @couchbase on Twitter and like us on Facebook to stay up on all Couchbase news. Event page: www.couchbase.com/events - don't miss out!

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
366
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
15
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Softshake 2013 Intro to NoSQL

  1. 1. Introduc)on  to  NoSQL with  Couchbase Tugdual  “Tug”  Grall Technical  Evangelist tug@couchbase.com Monday, October 28, 13 TwiBer:  tgrall
  2. 2. {“about”  :  “me”} • Tugdual  “Tug”  Grall -­‐ Couchbase -­‐ -­‐ eXo -­‐ -­‐ Technical  Evangelist CTO Oracle • Web -­‐ -­‐ -­‐    @tgrall      hEp://blog.grallandco.com      tgrall • NantesJUG  co-­‐founder • Pet  Project  : • hEp://www.resultri.com -­‐ • tug@couchbase.com -­‐ -­‐ Developer/Product   Manager Mainly  Java/SOA • tugdual@gmail.com Developer  in  consulAng  firms Monday, October 28, 13
  3. 3. RDBMS  are  not  Enough? Monday, October 28, 13
  4. 4. Growth  is  the  New  Reality • Instagram  gained  nearly  1  million  users  overnight  when  then   expanded  to  Android Monday, October 28, 13
  5. 5. 50  Million  Users  in  50  Days Draw  Something  by  OMGPOP Daily  Ac)ve  Users  (millions) 16 14 12 10 8 6 4 2 2/6 8 10 Monday, October 28, 13 12 14 16 18 20 22 24 26 28 3/1 3 5 7 9 11 13 15 17 19 21
  6. 6. How  do  you  take  this  growth? Applica;on  Scales  Out Just  add  more  commodity  web  servers System  Cost Applica;on  Performance   Web/App  Server  Tier Users RDBMS  Scales  Up Get  a  bigger,  more  complex  server System  Cost Applica;on  Performance   Won’t  scale   beyond  this   point Rela2onal  Database Users RDBMS  is  good  for  many  thing,  but  hard  to  scale Monday, October 28, 13
  7. 7. Scaling  out  RDBMS • Run  Many  SQL  Servers Web/App  Server  Tier • Data  could  be  sharded Memcached  Tier ­ Done  by  the  applicaAon  code • Caching  for  faster  response  )me MySQL  Tier Monday, October 28, 13
  8. 8. NoSQL  Technology  Scales  Out Applica;on  Scales  Out Just  add  more  commodity  web  servers System  Cost Applica;on  Performance   Web/App  Server  Tier Users NoSQL  Database  Scales  Out Cost  and  performance  mirrors  app  ;er System  Cost Applica;on  Performance   NoSQL  Distributed  Data  Store Users Scaling  out  fla?ens  the  cost  and  performance  curves Monday, October 28, 13
  9. 9. A  New  Technology? Bigtable November  2006 Dynamo October  2007 Cassandra August  2008 Voldemort February  2009 Building  new  database  to  answer  the  following  requirements • No  schema  required  before  inser)ng  data • No  schema  change  required  to  change  data  format • Auto-­‐sharding  without  applica)on  par)cipa)on • Distributed  queries • Integrated  main  memory  caching • Data  synchroniza)on  (  mul)-­‐datacenter) Very  few  organiza)ons  want  to  (fewer  can)  build  and  maintain  database  so]ware  technology. But  every  organiza)on  building  interac)ve  web  applica)ons  needs  this  technology. Monday, October 28, 13
  10. 10. What  Is  Biggest  Data  Management  Problem  Driving   Use  of  NoSQL  in  Coming  Year? 49% 35% 29% 16% Lack  of  flexibility/ rigid  schemas Inability  to  scale   Performance  challenges out  data Source:  Couchbase  Survey,  December  2011,  n  =  1351. Monday, October 28, 13 Cost 12% 11% All  of  these Other
  11. 11. NoSQL  Catalog Cache (memory  only) Key-­‐Value Memcached Data  Structure Document Column Graph Redis Database (memory/disk) Coherence Membase Cassandra Neo4j MongoDB Monday, October 28, 13 Couchbase HBase InfiniteGraph
  12. 12. Opera)onal  vs.  Analy)c  Databases Real-­‐Pme,   InteracPve  Databases AnalyPc Databases NoSQL Fast  access   to  data Couchbase MongoDB Monday, October 28, 13 Get  insights  from   data Cloudera Cassandra Hortonworks Hbase Mapr
  13. 13. Document  Database Monday, October 28, 13
  14. 14. Rela;onal  vs  Document  Data  Model C1 C2 C3 C4 { JSON JSON } JSON RelaPonal  data  model Document  data  model Highly-­‐structured  table  organiza;on  with   rigidly-­‐defined  data  formats  and  record   structure. Collec;on  of  complex  documents  with arbitrary,  nested  data  formats  and varying  “record”  format. Monday, October 28, 13
  15. 15. Rela;onal  Approach Monday, October 28, 13
  16. 16. Document  Database o::1001 { uid: “ji22jd”, customer: “Ann”, line_items: [ { sku: 0321293533, qty: 2, price: 48.0 }, { sku: 0321601912, qty: 1, price: 39.0 }, { sku: 0131495054, qty: 1, price: 51.0 } ], payment: { type: “Amex”, expiry: “04/2001”, last5: 12345 } } Aggregate  Oriented  Database hEp://marAnfowler.com/bliki/AggregateOrientedDatabase.html   Monday, October 28, 13
  17. 17. Schema  Update  :  RDBMS Event  Info ID 1 Title Speaker Speaker_id Paris  JUG ID Name rbin rbin Robin  Johnson 1.  Create  a  new  table 2 tgrall QCON  UK 2.“Prepare”  the  Data    (move  to  new  tgrall drop  column) table,   Tug  Grall A  talk  could  be  done  by  one  or  more  speakers! 3.  Change  the  constraints   3 4 NoSQL  CLN johnz johnz John  Zablo fweigel fweigel Frank  Weigel 4.  Change  the  app Devoxx Monday, October 28, 13
  18. 18. Schema  Update  :  Document user:rbin { user:rbin { user:rbin type: “user”, { type: “user”, name: “Robin Johnson”, type: “user”, name: “Robin Johnson”, } } name: “Robin Johnson”, } event:1 { event:1 1. Change  The  ApplicaPon { event:1 type: “event”, { event:1 type: “event”, the   • Update   { event:1 event:1 title: “Paris JUG”, Data  when  “you”  want type: “Paris JUG”, { title: “event”, the  index  to  query  the  new  format “event”, {type: “Paris • speaker: “rbin” JUG”, title: Change   type:“rbin” JUG”, title: “event”, } speaker: “Paris JUG”, type: “Paris title: “event”, } speaker: “rbin” JUG”, title: “rbin” } speaker: “Paris speaker: “rbin” } speaker: [“rbin”,”tgrall”] } } Monday, October 28, 13
  19. 19. Couchbase  Server Monday, October 28, 13
  20. 20. Couchbase  Server  Core  Principles Easy   Scalability PE RF O R M A N C E Grow  cluster  without  applicaAon   changes,  without  downAme  with   a  single  click Always  On   24x365 No  downAme  for  sodware   upgrades,  hardware  maintenance,   etc. Monday, October 28, 13 Consistent  High   Performance Consistent  sub-­‐millisecond   read  and  write  response  Ames   with  consistent  high  throughput JSON JSON JSO JSON N JSON Flexible  Data   Model JSON  document  model  with  no   fixed  schema.
  21. 21. Monday, October 28, 13
  22. 22. Couchbase  Architecture 8092 11211 Query  API Memcapable    1.0 11210 Memcapable    2.0 New  Persistence  Layer vBucket  state  and  replica;on  manager Node  health  monitor Rebalance  orchestrator Global  singleton  supervisor storage  interface Configura;on  manager Data  Manager Process  monitor Couchbase  EP  Engine Heartbeat Memcached REST  management  API/Web  UI Query  Engine Moxi Cluster  Manager hRp on  each  node one  per  cluster Erlang/OTP HTTP 8091 Monday, October 28, 13 Erlang  port  mapper 4369 Distributed  Erlang 21100  -­‐  21199
  23. 23. Couchbase  Architecture 8092 11211 Query  API Memcapable    1.0 11210 Memcapable    2.0 New  Persistence  Layer Disk  Persistence vBucket  state  and  replica;on  manager Node  health  monitor Rebalance  orchestrator Global  singleton  supervisor Configura;on  manager storage  interface Server/Cluster   Management  &   Communica2on (Erlang) hRp Process  monitor RAM  Cache,  Indexing   Couchbase  EP  Engine &  Persistence   Management (C  &  V8) Heartbeat Object-­‐level  Cache REST  management  API/Web  UI Query  Engine Moxi on  each  node one  per  cluster Erlang/OTP The Unreasonable Effectiveness of C by Damien Katz HTTP 8091 Monday, October 28, 13 Erlang  port  mapper 4369 Distributed  Erlang 21100  -­‐  21199
  24. 24. Open  Source  Project Apache  2.0 hEps://github.com/couchbase/ hEps://github.com/couchbaselabs/ Monday, October 28, 13 Gerrit: hEp://review.couchbase.org/
  25. 25. Cluster-­‐wide  Basic  Opera)on App  Server  1 App  Server  2 COUCHBASE  Client  Library COUCHBASE  Client  Library Cluster  Map Cluster  Map READ/WRITE/UPDATE Server  1 Server  2 Server  3 Ac2ve Ac2ve Ac2ve Doc  5 Doc Doc  4 Doc Doc  1 Doc Doc  2 Doc Doc  7 Doc Doc  2 Doc Doc  9 Doc Doc  8 Doc Doc  6 Doc REPLICA REPLICA REPLICA • Docs  distributed  evenly  across   servers   • Each  server  stores  both  ac)ve  and   replica  docs Only  one  server  acAve  at  a  Ame • Client  library  provides  app  with   simple  interface  to  database • Cluster  map  provides  map   to  which  server  doc  is  on Doc  4 Doc Doc  6 Doc Doc  7 Doc Doc  1 Doc Doc  3 Doc Doc  9 Doc • App  reads,  writes,  updates  docs Doc  8 Doc Doc  2 Doc Doc  5 Doc • Mul)ple  app  servers  can  access  same   document  at  same  )me Couchbase  Server    Cluster User  Configured  Replica  Count  =  1 Monday, October 28, 13 App  never  needs  to  know
  26. 26. Add  Nodes  to  Cluster App  Server  1 App  Server  2 COUCHBASE  Client  Library COUCHBASE  Client  Library Cluster  Map Cluster  Map READ/WRITE/UPDATE READ/WRITE/UPDATE Server  1 Server  2 Server  3 Server  4 Server  5 Ac2ve Ac2ve Ac2ve Ac2ve Ac2ve Doc  5 Doc Doc  4 Doc Doc  1 Doc Doc  2 Doc Doc  7 Doc Doc  2 Doc Doc  9 Doc Doc  8 Doc Doc  6 Doc REPLICA REPLICA REPLICA Doc  4 Doc Doc  6 Doc Doc  7 Doc Doc  1 Doc Doc  3 Doc Doc  9 Doc Doc  8 Doc Doc  2 Doc Doc  5 Doc Couchbase  Server    Cluster User  Configured  Replica  Count  =  1 Monday, October 28, 13 • Two  servers  added One-­‐click  opera)on • Docs  automa)cally   rebalanced  across   cluster Even  distribuAon  of  docs Minimum  doc  movement REPLICA REPLICA • Cluster  map  updated • App  database   calls  now  distributed   over  larger  number  of   servers
  27. 27. Fail  Over  Node App  Server  1 App  Server  2 COUCHBASE  Client  Library COUCHBASE  Client  Library Cluster  Map Cluster  Map Server  1 Server  2 Server  3 Server  4 Server  5 Ac2ve Ac2ve Ac2ve Ac2ve Ac2ve Doc  5 Doc Doc  4 Doc Doc  1 Doc Doc  9 Doc Doc  2 Doc Doc  7 Doc Doc  2 Doc Doc  8 Doc Doc  1 Doc  6 Doc Doc Doc  3 REPLICA REPLICA REPLICA REPLICA Doc  4 Doc Doc  6 Doc Doc  7 Doc Doc  5 Doc  1 Doc Doc  3 Doc Doc  9 Doc Doc  2 Couchbase  Server    Cluster User  Configured  Replica  Count  =  1 Monday, October 28, 13 Doc REPLICA Doc  8 Doc Doc • App  servers  accessing  docs • Requests  to  Server  3  fail • Cluster  detects  server  failed Promotes  replicas  of  docs  to   acAve Updates  cluster  map • Requests  for  docs  now  go  to   appropriate  server • Typically  rebalance   would  follow
  28. 28. Indexing  and  Querying APP  SERVER  1 APP  SERVER  2 COUCHBASE  Client  Library COUCHBASE  Client  Library CLUSTER  MAP CLUSTER  MAP Query SERVER  1 SERVER  2 ACTIVE ACTIVE SERVER  3 ACTIVE • Indexing  work  is  distributed  amongst   nodes Doc  5 Doc Doc  5 Doc Doc  5 Doc Doc  2 Doc Doc  2 Doc Doc  2 Doc • Parallelize  the  effort Doc  9 Doc Doc  9 Doc Doc  9 Doc • Each  node  has  index  for  data  stored  on  it REPLICA REPLICA REPLICA Doc  4 Doc Doc  4 Doc Doc  4 Doc Doc  1 Doc Doc  1 Doc Doc  1 Doc Doc  8 Doc Doc  8 Doc Doc  8 Doc COUCHBASE  SERVER    CLUSTER Monday, October 28, 13 • Large  data  set  possible • Queries  combine  the  results  from  required   nodes
  29. 29. Couchbase  SDK Official  SDKs Ruby Python Community  SDKs Go Clojure www.couchbase.com/develop Monday, October 28, 13
  30. 30. Full  Text  Search • Elastic Search is good for ad-hoc queries and faceted browsing • Our adapter is aware of changing Couchbase topology • Indexed by Elastic Search after stored to disk in Couchbase ElasPcSearch Monday, October 28, 13
  31. 31. The  Complete  Mobile  Solu;on Available  now  in  Beta  : hfp://mobile.couchbase.com Monday, October 28, 13
  32. 32. Couchbase  Docs www.couchbase.com/docs/ Couchbase  Forums www.couchbase.com/communi;es Meetup hbp://meetup.com/Couchbase-­‐France/ Mobile  (beta) hbp://mobile.couchbase.com/ N1QL  -­‐  New  Query  Engine  (DP) hbp://query.couchbase.com Monday, October 28, 13 Contact  me  on  Twi?er @tgrall Contact  me  by  Email tug@couchbase.com IRC #couchbase #libcouchbase
  33. 33. Introduc)on  to  NoSQL with  Couchbase Tugdual  “Tug”  Grall Technical  Evangelist tug@couchbase.com Monday, October 28, 13 TwiBer:  tgrall

×