Couchbase 102 - SDK Operations

3,472 views

Published on

Learn the fundamentals of all operations with Couchbase including creating, updating, and deleting Documents, atomic counters, CAS operations for Optimistic Concurrency, Locks for Pessimistic Concurrency, Expirations and Durability.

What will be covered during this training:

Understanding how to setup SDK for languages and different platforms
JSON serialization/deserialization
How Operations work in Couchbase Architecture
Making a Connection to Couchbase
Document Operations in Couchbase
Optimistic Concurrency
Pessimistic Concurrency
Durability with Observe


DEMO: Make a connection; Create, update and delete Documents; Optimistic Concurrency

Published in: Technology
1 Comment
2 Likes
Statistics
Notes
  • très intéressante mais j ai une petite nuance concernant la fonction reduce est ce qu elle s exécute sur les keys de la map ou bien sur la valeur
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total views
3,472
On SlideShare
0
From Embeds
0
Number of Embeds
1,331
Actions
Shares
0
Downloads
175
Comments
1
Likes
2
Embeds 0
No embeds

No notes for slide

Couchbase 102 - SDK Operations

  1. 1. Couchbase  102:  SDK  OperaFons Jasdeep  Jaitla Technical  Evangelist email:  jasdeep@couchbase.com twi0er:  @scalabl3
  2. 2. SETUP  SDK
  3. 3. Supported SDK's www.couchbase.com/communi/es • Each  supported  SDK  page  has  instrucFons  for  setup   • PHP,  Ruby,  NodeJS  and  Python  clients  are  wrappers  around   libcouchbase  C  library,  so  libcouchbase  must  be  installed  first   • For  other  community  clients,  click  on  "All  Clients"  on  leS  nav,  scroll   down  the  page  and  you  can  see  clients  for  Go,  Erlang,  Clojure,  TCL,   other  nodejs  and  Perl.
  4. 4. Installing Libcouchbase www.couchbase.com/communi/es/c/geBng-­‐started • Mac  Tips  before  Libcouchbase  &  SDK  Install   • Make  sure  you  have  XCode  &  Command  Line  Tools  Installed   • Install  Homebrew  if  you  don't  have  it  already   • Do  a  $  brew  update,  $  brew  upgrade  and  $  brew  doctor  to  be  sure  you're   up  to  date
  5. 5. Installing Libcouchbase www.couchbase.com/communi/es/c/geBng-­‐started • Mac  Via  Homebrew   • $  brew  install  libcouchbase   • PC-­‐Windows     • Download  appropriate  Zip  from  website   • Redhat/CentOS   • wget  the  yum  repositories   • $  sudo  yum  install  -­‐y  libcouchbase2-­‐libevent  libcouchbase-­‐devel   • Ubuntu   • wget  ubuntu  repositories   • $  sudo  apt-­‐get  install  libcouchbase2-­‐libevent  libcouchbase-­‐dev
  6. 6. Client Connections 8091 8092 11210 Application Server 11211 Couchbase Server
  7. 7. Client Connections Cluster  Configura/on  and   Cluster  Par//on  Map HTTP 8091 8092 11210 Application Server 11211 Couchbase Server
  8. 8. Client Connections Cluster  Configura/on  and   Cluster  Par//on  Map HTTP 8091 8092 MAP 11210 Application Server 11211 Couchbase Server
  9. 9. Client Connections HTTP 8091 8092 MAP 11210 Application Server 11211 Couchbase Server
  10. 10. Client Connections HTTP Create  Socket  Connec/on  to   Each  Node  in  Cluster 8091 8092 MAP TCP Binary Application Server 11210 11211 Couchbase Server
  11. 11. Client Connections HTTP 8091 8092 MAP TCP Binary Application Server 11210 11211 Couchbase Server
  12. 12. Client Connections View  Querying HTTP 8091 HTTP 8092 TCP Binary 11210 MAP Application Server 11211 Couchbase Server
  13. 13. Client Connections HTTP 8091 HTTP 8092 TCP Binary 11210 MAP Application Server 11211 Couchbase Server
  14. 14. Make a Connection RUBY require 'rubygems'! require 'couchbase'!  ! cb = Couchbase.connect(! :bucket => "default",! :hostname => "localhost")! ! data = { jsonkey: "value", created_at: Time.now }! ! cb.add("mydata", data)! puts cb.get("mydata") #!/usr/bin/env python! from couchbase import Couchbase! from pprint import pprint! from datetime import datetime! ! PYTHON cb = Couchbase.connect(bucket='default')! ! data = { "jsonkey": "value", "created_at": datetime.now() }! ! cb.add('mydata', data)! result = cb.get('mydata')! pprint(result.value, indent=4)!
  15. 15. Make a Connection import import import import import ! com.couchbase.client.CouchbaseClient;! java.net.URI;! java.util.*;! com.google.gson.Gson;! com.google.gson.GsonBuilder;! public class HelloWorld {!  ! public static void main(String[] args) throws Exception {!  ! List<URI> hosts = Arrays.asList(! new URI("http://127.0.0.1:8091/pools")! );! JAVA ! CouchbaseClient cb = new CouchbaseClient(hosts, "default", "");! Gson json = new Gson();! ! Hashtable data = new Hashtable();! data.put("jsonkey", "value");! data.put("created_at", new Date());! ! cb.add("mydata", json.toJson(data))! System.out.println(cb.get("mydata"));! ! cb.shutdown();! }! }!
  16. 16. Make a Connection var Couchbase = require('couchbase');! var cb = new Couchbase.Connection({bucket: "default"}, function(err) { });! NODEJS ! var data = { jsonkey: "value", created_at: new Date().toString() };! cb.add("mydata", data);! ! console.log(cb.get("mydata"));! <?php! // adjust these parameters to match your installation! $cb = new Couchbase("127.0.0.1:8091", "", "", "default");! ! PHP $data = ! array("jsonkey" => "value", ! "created_at" => date("Y-m-d H:i:s"));! $cb->add("mydata",json_encode($data));! var_dump($cb->get("mydata"));! ! ?>!
  17. 17. OPERATIONS
  18. 18. Couchbase Organization • Couchbase operates like a Key-Value Document Store - Simple Datatypes: strings, numbers, datetime (string), boolean, and binary data (string) can be stored - Complex Datatypes: dictionaries/hashes, arrays/lists, can be stored in JSON format (simple lists can be string based with delimiter) - JSON is a special class of string with a specific format for encoding simple and complex data structures • Schema is unenforced and implicit, schema changes are programmatic, done online, and can vary from Document to Document
  19. 19. Store  &  Retrieve  OperaFons • get  (key)   –  Retrieve  a  document   • set  (key,  value)   –  Store  a  document,  overwrites  if  exists   • add  (key,  value)   –  Store  a  document,  error/excepFon  if  exists   • replace  (key,  value)   –  Store  a  document,  error/excepFon  if  doesn’t  exist   • cas  (key,  value,  cas)   –  Compare  and  swap,  mutate  document  only  if  it  hasn’t  changed  while   execuFng  this  operaFon
  20. 20. Atomic Counter Operations These  operaFons  are  always  executed  in  order  atomically.   ! • incr  (key)   –  Increase  an  atomic  counter  value,  default  by  1   • cb.incr(“my_counter”)  #  now  it’s  2   • decr  (key)   –  Decrease  an  atomic  counter  value,  default  by  1   • cb.decr(“my_counter”)  #  now  it’s  1
  21. 21. Non-JSON Operations You  can  use  these  creaFvely!   ! • prepend  (key,  value)   –  prepend  exisFng  string  in  couchbase  with  value   • cb.prepend(“mykey”,  "jumped  over  the  lazy  dog")     • cb.prepend("mykey",  "the  brown  fox  ")     • append  (key,  value)   –  append  exisFng  string  in  couchbase  with  value   • cb.append(“mykey2”,  "oranges")     • cb.append("mykey2",  "  apples  bananas")
  22. 22. Retrieval Operations Couchbase Server RAM Cache EP Engine Disk Write Queue Application Server Replication Queue Replica Couchbase Cluster Machine
  23. 23. Retrieval Operations Couchbase Server SELECT  *  WHERE  KEY  =  “mykey” RAM Cache EP Engine Disk Write Queue get Application Server Replication Queue Replica Couchbase Cluster Machine
  24. 24. Storage Operations - set Couchbase Server RAM Cache EP Engine Disk Write Queue Application Server Replication Queue Replica Couchbase Cluster Machine
  25. 25. Storage Operations - set Couchbase Server INSERT/UPDATE  WHERE  KEY  =  “mykey” RAM Cache EP Engine Disk Write Queue set Application Server Replication Queue Replica Couchbase Cluster Machine
  26. 26. Storage Operations - add Couchbase Server RAM Cache EP Engine Disk Write Queue Application Server Replication Queue Replica Couchbase Cluster Machine
  27. 27. Storage Operations - add Couchbase Server INSERT  KEY  =  VALUE RAM Cache EP Engine Disk Write Queue add Application Server Replication Queue Replica Couchbase Cluster Machine
  28. 28. Storage Operations - add Couchbase Server RAM Cache EP Engine Disk Write Queue Application Server Replication Queue Replica Couchbase Cluster Machine
  29. 29. Storage Operations - add Couchbase Server RAM Cache EP Engine Disk Write Queue add Application Server ALREADY  EXISTS! Replication Queue Replica Couchbase Cluster Machine
  30. 30. Storage Operations - replace Couchbase Server RAM Cache EP Engine Disk Write Queue Application Server Replication Queue Replica Couchbase Cluster Machine
  31. 31. Storage Operations - replace Couchbase Server UPDATE  SET  KEY  =  VALUE  WHERE  KEY  =  “mykey” RAM Cache EP Engine Disk Write Queue replace Application Server Replication Queue Replica Couchbase Cluster Machine
  32. 32. Storage Operations - replace Couchbase Server RAM Cache EP Engine Disk Write Queue Application Server Replication Queue Replica Couchbase Cluster Machine
  33. 33. Storage Operations - replace Couchbase Server RAM Cache EP Engine Disk Write Queue replace Application Server DOESN’T  EXIST! Replication Queue Replica Couchbase Cluster Machine
  34. 34. Consistency Couchbase Server RAM Cache EP Engine Disk Write Queue Application Server Replication Queue Replica Couchbase Cluster Machine
  35. 35. Consistency Couchbase Server RAM Cache get EP Engine Disk Write Queue Application Server Replication Queue Replica Couchbase Cluster Machine
  36. 36. CONCURRENCY
  37. 37. Optimistic Concurrency with CAS • Every storage operation (including touch) creates a new "CAS" value, which is just a long int • The CAS value simply represents the current state of the Document, it's like a version number • You can use this CAS for "Optimistic Concurrency" - value, flags, cas = get("mykey", :extended => true) - This will only succeed if the CAS matches • replace("mykey", newvalue, :cas => cas) - If another process had changed the "mykey" document, a new CAS will have been generated, and that replace operation will fail
  38. 38. Optimistic Concurrency with CAS Couchbase Server RAM Cache EP Engine Disk Write Queue Application Server 1 Application Server 2 Replication Queue
  39. 39. Optimistic Concurrency with CAS CAS: 111111111 Couchbase Server RAM Cache EP Engine Disk Write Queue Application Server 1 Application Server 2 CAS: 111111111 Replication Queue
  40. 40. Optimistic Concurrency with CAS CAS: 111111111 Couchbase Server RAM Cache EP Engine Disk Write Queue Application Server 1 Application Server 2 CAS: 111111111 Replication Queue
  41. 41. Optimistic Concurrency with CAS CAS: 222222222 Couchbase Server RAM Cache EP Engine Disk Write Queue Application Server 1 Application Server 2 CAS: 111111111 Replication Queue
  42. 42. Optimistic Concurrency with CAS CAS: 222222222 Couchbase Server RAM Cache EP Engine Disk Write Queue Application Server 1 Application Server 2 CAS: 111111111 Replication Queue
  43. 43. Optimistic Concurrency with CAS CAS: 222222222 Couchbase Server RAM Cache Application Server 1 Disk Write Queue CAS  MISMATCH set/add/replace with CAS Application Server 2 CAS: 111111111 EP Engine Replication Queue
  44. 44. Optimistic Concurrency with CAS CAS: 222222222 Couchbase Server RAM Cache Application Server 1 Disk Write Queue CAS  MISMATCH set/add/replace with CAS Application Server 2 CAS: 111111111 EP Engine Replication Queue
  45. 45. Optimistic Concurrency with CAS CAS: 222222222 Couchbase Server RAM Cache EP Engine Disk Write Queue Application Server 1 Application Server 2 CAS: 111111111 Replication Queue
  46. 46. Optimistic Concurrency with CAS CAS: 222222222 Couchbase Server RAM Cache EP Engine Disk Write Queue Application Server 1 set/add/replace with CAS Application Server 2 RETRY  MUTATION   WITH  UPDATED  CAS CAS: 222222222 NEW  CAS  GENERATED Replication Queue
  47. 47. Optimistic Concurrency with CAS CAS: 222222222 Couchbase Server RAM Cache EP Engine Disk Write Queue Application Server 1 set/add/replace with CAS Application Server 2 RETRY  MUTATION   WITH  UPDATED  CAS CAS: 222222222 NEW  CAS  GENERATED Replication Queue
  48. 48. Optimistic Concurrency with CAS CAS: 222222222 Couchbase Server RAM Cache EP Engine Disk Write Queue Application Server 1 Application Server 2 CAS: 222222222 Replication Queue
  49. 49. Optimistic Concurrency with CAS CAS: 333333333 Couchbase Server RAM Cache EP Engine Disk Write Queue Application Server 1 Application Server 2 CAS: 333333333 Replication Queue
  50. 50. Pessimistic Concurrency with Lock Couchbase Server RAM Cache EP Engine Disk Write Queue Application Server 1 Application Server 2 Replication Queue
  51. 51. Pessimistic Concurrency with Lock Couchbase Server RAM Cache EP Engine Disk Write Queue Application Server 1 Application Server 2 Replication Queue
  52. 52. Pessimistic Concurrency with Lock Couchbase Server RAM Cache EP Engine Disk Write Queue Application Server 1 Application Server 2 Replication Queue
  53. 53. Pessimistic Concurrency with Lock Couchbase Server RAM Cache EP Engine Disk Write Queue Application Server 1 Application Server 2 Replication Queue
  54. 54. Pessimistic Concurrency with Lock UNLOCKS  AFTER   COMPLETE set/add/replace with CAS Couchbase Server RAM Cache NEW  CAS  GENERATED Application Server 1 Application Server 2 Replication Queue EP Engine Disk Write Queue
  55. 55. EXPIRATIONS
  56. 56. Storage Operations with Expirations • CMS Framework Cache can be configured to use Couchbase - In most frameworks this is simple, as they typically already have memcached support ! • Create/Update the Value and Expiration - [Ruby] set/add/replace("mykey", value, :ttl => 30) - [Java] set/add/replace("mykey", 30, value) ! • Update the expiration only - [Ruby] touch("mykey", :ttl => 30) - [Java] touch("mykey", 30) !
  57. 57. Expiration Couchbase Server RAM Cache EP Engine Disk Write Queue Application Server Replication Queue Replica Couchbase Cluster Machine
  58. 58. Expiration Couchbase Server RAM Cache set/add/replace get with TTL EP Engine Disk Write Queue EXPIRED Application Server DELETED  ON  COMPACTION Replication Queue Replica Couchbase Cluster Machine
  59. 59. Expiration Couchbase Server RAM Cache EP Engine Disk Write Queue Application Server Replication Queue Replica Couchbase Cluster Machine
  60. 60. Expiration Couchbase Server UPDATES  TTL RAM Cache set/add/replace touch with TTL EP Engine Disk Write Queue Application Server Replication Queue Replica Couchbase Cluster Machine
  61. 61. DURABILITY
  62. 62. Using Storage with Observe • Callback when it has been written to disk on active partition • Callback when it has been written to a replica(s) • Callback when it has been written to replica(s) disk • Observe Persisted to Disk and Replicated • [Ruby] set/add/replace("mykey", value, 
 :observe => {:persisted => 1, :replicated => 1}) • [Java] set/add/replace("mykey", 0, value, PersistTo.MASTER, ReplicateTo.ONE) • Observe Replicated Only • [Ruby] set/add/replace("mykey", value, 
 :observe => {:replicated => 1}) • [Java] set/add/replace("mykey", 0, value, ReplicateTo.ONE)
  63. 63. Durability Persist-To Couchbase Server RAM Cache EP Engine Disk Write Queue Application Server Replication Queue Replica Couchbase Cluster Machine
  64. 64. Durability Persist-To Couchbase Server RAM Cache set/add/replace EP Engine Disk Write Queue Application Server Replication Queue Replica Couchbase Cluster Machine
  65. 65. Durability Persist-To Couchbase Server RAM Cache set/add/replace Application Server EP Engine Disk Write Queue Callback Replication Queue Replica Couchbase Cluster Machine
  66. 66. Durability Replicate-To Couchbase Server RAM Cache EP Engine Disk Write Queue Application Server Replication Queue Replica Couchbase Cluster Machine
  67. 67. Durability Replicate-To Couchbase Server RAM Cache set/add/replace EP Engine Disk Write Queue Application Server Replication Queue Replica Couchbase Cluster Machine
  68. 68. Durability Replicate-To Couchbase Server RAM Cache set/add/replace Application Server EP Engine Disk Write Queue Callback Replication Queue Replica Couchbase Cluster Machine
  69. 69. Durability Replicate-To Couchbase Server RAM Cache set/add/replace Application Server EP Engine Disk Write Queue Callback Replication Queue Replica Couchbase Cluster Machine
  70. 70. Durability Replicate-To Couchbase Server RAM Cache set/add/replace Application Server EP Engine Disk Write Queue Callback Replication Queue Replica Couchbase Cluster Machine
  71. 71. DEMO
  72. 72. Q  &  A
  73. 73. Resources Next Webinar: Couchbase 103 - Modeling Main  Resource  Portal   www.couchbase.com/communiFes   ! Code  Samples  Going  through  OperaUons   www.github.com/couchbaselabs/DeveloperDay   ! Couchbase  Q  &  A   www.couchbase.com/communiFes/q-­‐and-­‐a   My  Email:  jasdeep@couchbase.com   My  TwiZer:  @scalabl3

×