Webinar- Couchbase Mobile Deep Dive

1,069 views

Published on

In this webinar, Jens Alfke, Lead Developer of Couchbase Mobile, will discuss the Couchbase Mobile architecture, key concepts behind Couchbase Lite and Sync Gateway, and real-world examples of using Couchbase Mobile.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,069
On SlideShare
0
From Embeds
0
Number of Embeds
265
Actions
Shares
0
Downloads
14
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Webinar- Couchbase Mobile Deep Dive

  1. 1. Couchbase  Mobile     Deep  Dive Jens  Al4e   jens@couchbase.com V1.0 - 6/17/14
  2. 2. ©2009  Helder  da  Rocha
  3. 3. Architecture   of  Couchbase  Lite  and  &  Sync  Gateway • Document  data  store   Mul?-­‐Version  Concurrency  Control  (MVCC)   • Replica=on  protocol   Conflict  handling   • Querying/filtering   Map/reduce  views   • Client  APIs   REST   Na?ve
  4. 4. Document  Store
  5. 5. Document  Database • Persistent  map  from  IDs  →  documents   • where  documents  are  JSON  objects   • …plus  aLachments   Extra  set  of  proper?es  mapping  name  →  {binary  data,  MIME  type}   • …with  mul=ple  revisions
  6. 6. Mul=-­‐Version  Concurrency  Control • Each  document  has  a  revision  history   • Each  revision’s  ID  has  a  genera=on  number  and  a  digest   e.g.  12-­‐c8c993737bf3e800b0a3d2e864c2039d   • Dele=ons  are  just  special  revisions   “_deleted”:  true   • Every  doc  update  must  specify  its  parent  revision  ID
  7. 7. Sequence  Index • Every  revision  added  gets  a  sequence  number   • Sequence  numbers  are  assigned  consecu=vely   • Local  to  a  single  database   • Documents  are  indexed  by  sequence  number   Abstract  ?meline  of  the  database’s  history   Allows  asking  “what  changed  since  sequence  ____?”
  8. 8. Replica=on
  9. 9. What  Replica=on  Does • Transfers  changed  documents  from  a  source  to  a  target   database   • Incremental   • Asynchronous   • Peer-­‐to-­‐peer   • Semi-­‐op=mized   Doesn’t  send  any  revision  that  the  target  already  has   Doesn’t  send  any  adachment  version  that  the  target  already  has   But  does  send  en?re  JSON  docs  &  adachments  (no  delta  compression)
  10. 10. Replica=on  Conflicts • Conflicts  are  inevitable,  per  the  CAP  Theorem   • Replica=ng  conflic=ng  revision  histories  produces  a  tree   • The  document  model  allows  for  conflicts   A  conflict  is  a  valid  state  of  a  document   • Conflict  resolu=on  is  the  app’s  job   Only  the  app  needs  to  care  about  resolving  conflicts   App  can  use  any  logic  it  wants  to  pick  winners  or  merge  changes   Resolu?on  creates  new  revisions  that  are  replicated Jens — This is too confusing. We really need a diagram here! Thx.
  11. 11. Replica=on  Conflicts  Cause  Branches 1-­‐abcd 2-­‐c124 1-­‐abcd 2-­‐c124 3-­‐beef 1-­‐abcd 2-­‐c124 3-­‐fade 1-­‐abcd 2-­‐c124 3-­‐beef 3-­‐fade
  12. 12. 4-­‐dead 4-­‐feed Resolving  a  Conflict 1-­‐abcd 2-­‐c124 3-­‐beef 3-­‐fade
  13. 13. Replica=on  Conflicts • Conflicts  are  inevitable,  per  the  CAP  Theorem   • Replica=ng  conflic=ng  revision  histories  produces  a  tree   • The  document  model  allows  for  conflicts   A  conflict  is  a  valid  state  of  a  document   • Conflict  resolu=on  is  the  app’s  job   Only  the  app  needs  to  care  about  resolving  conflicts   App  can  use  any  logic  it  wants  to  pick  winners  or  merge  changes   Resolu?on  creates  new  revisions  that  are  replicated
  14. 14. Replica=on  and  Peer-­‐to-­‐Peer • It’s  more  than  just  push/pull  to  a  server   • Any  two  databases  can  replicate  if  they  can  connect   Couchbase  Lite’s  “Listener”  component  allows  connec?ons   • Arbitrary  topologies  (trees,  meshes)  are  supported
  15. 15. Replica=on  Protocol • Extension  of  REST  API   • One  ac=ve  partner  (client),  one  passive  (server)   • Ac=ve  partner  can  push  or  pull   • State  (checkpoint)  is  stored  redundantly  on  both  sides
  16. 16. Pull  Algorithm • Verify  local  &  remote  checkpoints   • Request  the  changes  since  checkpointed  remote  sequence   →  array  of  {sequence,  doc-­‐id,  rev-­‐id}  tuples   • Determine  which  document  revisions  do  not  exist  locally   • Download  leaf  revision(s)  of  each  such  document   Including  revision  history   Skipping  unmodified  adachments   • Update  the  checkpoints   • In  con1nuous  mode,  keep  monitoring  remote  changes
  17. 17. Push  Algorithm • Verify  local  &  remote  checkpoints   • Send  IDs  of  revisions  made  since  checkpoint’s  local  sequence   Server  responds  with  the  subset  that  it  doesn’t  have   • Upload  the  leaf  revision(s)  of  each  such  document   Including  revision  history   Skipping  unmodified  adachments   • Update  checkpoints   • In  con1nuous  mode,  keep  monitoring  local  changes Note to self: verify against https://github.com/couchbase/couchbase-lite-ios/wiki/Replication-Algorithm
  18. 18. Filtered  Replica=on • Implemented  in  Couchbase  Lite,  CouchDB,  PouchDB,  etc.   • App-­‐defined  filter  func=on  can  accept  or  skip  each  outgoing   revision   • Always  runs  on  source  side  of  replica=on   Push:  on  client  side   Pull:  on  server  side   • Simple,  but  not  super  scaleable
  19. 19. Channel-­‐Based  Replica=on • Implemented  by  Couchbase  Sync  Gateway   • Documents  are  tagged  with  channel  IDs   • User  accounts  have  access  to  sets  of  channels   • Documents  are  indexed  by  channel   • Pull  replica=ons  efficiently  look  up  docs  in  available  channels Tell people to watch the Sync Gateway talk next week for more details!
  20. 20. Map/Reduce  Views
  21. 21. Anatomy  of  a  View • A  view  belongs  to  a  database,  has  a:   Name   Map  func?on   Reduce  func?on  (op?onal)   Persistent  index   Checkpoint  sequence   • Views  exist  in:   Couchbase  Lite   Couchbase  Server   not  Sync  Gateway   Also  CouchDB,  PouchDB,  etc. FYI: Last week’s iOS webinar had better slides explaining this stuff.
  22. 22. Indexing  a  View • For  each  document  updated  since  checkpoint  sequence:   1. Remove  all  index  rows  previously  generated  by  that  document   2. Call  map  func?on  with  current  document  proper?es  as  parameter   3. Each  ?me  emit()  is  called,  add  {key,  value,  docID}  tuple  to  the  index   • Save  the  current  sequence  number  as  the  checkpoint   • The  keys  in  the  index  are  sorted  as  JSON
  23. 23. Querying  a  View • Is  its  checkpoint  sequence  the  database’s  latest  sequence?   If  not,  update  the  index  (as  per  previous  slide)   This  can  be  suppressed  by  sejng  query’s  IndexUpdateMode   • Fetch  a  range  of  {key,  value,  docID}  tuples  from  the  index   Honoring  startKey,  endKey,  skip,  limit,  reverse
  24. 24. Client  APIs
  25. 25. REST  API  —  The  Classic • Originally  defined  by  Apache  CouchDB   • Supported  by  Sync  Gateway  (with  slight  differences)   Server  side  of  replica?on   API  for  app  servers   • Supported  by  Couchbase  Lite  Listener   For  JavaScript  app  code  (e.g.  PhoneGap)   Server  side  of  peer-­‐to-­‐peer  replica?on
  26. 26. REST  API • Document  access  (CRUD):   PUT  /  GET  /  DELETE   /dbname/docid   • Query  a  view:   POST  /dbname/_design/name/_view/name   • Replicate:   POST  /dbname/_replicate
  27. 27. Na=ve  APIs  by  Plalorm • Objec=ve-­‐C:  iOS,  Mac  OS  X   • Java:  Android,  Linux   • C#:  Windows,  Windows  Mobile,  Mono          BETA   • Common  API  design  across  languages   We  have  a  single  carefully  designed  language-­‐neutral  API  spec   Matching  class  names,  method  names,  seman?cs   Slight  differences,  to  remain  idioma?c Point them to the “Development with Android / iOS” webinars.
  28. 28. Na=ve  API:  Key  Classes • Manager   • Database   • Document   • View   • Query   • Replica=on
  29. 29. Oops, that’s all I wrote. Stall for time — maybe answer some questions?
  30. 30. THE  END www.couchbase.com/webinars   developer.couchbase.com/mobile/

×