Couchbase 102 - Developing with Couchbase

Like this? Share it with your network

Share

Couchbase 102 - Developing with Couchbase

  • 2,778 views
Uploaded on

Learn the fundamentals of creating data models with Couchbase including JSON strategies, common key patterns and handling and various differences from RDBMS systems. ...

Learn the fundamentals of creating data models with Couchbase including JSON strategies, common key patterns and handling and various differences from RDBMS systems.

What will be covered during this training:

Differences with RDBMS
ORM's and ActiveRecord
Simple vs Complex Data Types
JSON serialization/deserialization
Understanding how to setup SDK for languages and different platforms
Making a Connection to Couchbase
Operations in Couchbase
Key Patterns
Advanced Operations
Dynamic Schema Change Strategies
Document Tips that help with Views

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
2,778
On Slideshare
2,691
From Embeds
87
Number of Embeds
2

Actions

Shares
Downloads
56
Comments
0
Likes
0

Embeds 87

http://www.couchbase.com 83
https://twitter.com 4

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Friday, October 11, 13
  • 2. Technical  Evangelist twi0er:  @scalabl3email:  jasdeep@couchbase.com Jasdeep  Jaitla Couchbase  102:  Development Friday, October 11, 13
  • 3. Technical  Evangelist twi0er:  @scalabl3email:  jasdeep@couchbase.com Jasdeep  Jaitla Couchbase  102:  Development Friday, October 11, 13
  • 4. Friday, October 11, 13
  • 5. SETUP  SDK Friday, October 11, 13
  • 6. Supported SDK's www.couchbase.com/communi/es • Each  supported  SDK  page  has  instrucHons  for  setup • PHP,  Ruby,  NodeJS  and  Python  clients  are  wrappers  around   libcouchbase  C  library,  so  libcouchbase  must  be  installed  first • For  other  community  clients,  click  on  "All  Clients"  on  leR  nav,  scroll   down  the  page  and  you  can  see  clients  for  Go,  Erlang,  Clojure,  TCL,   other  nodejs  and  Perl. Friday, October 11, 13
  • 7. Installing Libcouchbase • Mac  Tips  before  Libcouchbase  &  SDK  Install • Make  sure  you  have  XCode  &  Command  Line  Tools  Installed • Install  Homebrew  if  you  don't  have  it  already • Do  a  $  brew  update,  $  brew  upgrade  and  $  brew  doctor  to  be  sure  you're   up  to  date www.couchbase.com/communi/es/c/geBng-­‐started Friday, October 11, 13
  • 8. Installing Libcouchbase • Mac  Via  Homebrew • $  brew  install  libcouchbase • PC-­‐Windows   • Download  appropriate  Zip  from  website • Redhat/CentOS • wget  the  yum  repositories • $  sudo  yum  install  -­‐y  libcouchbase2-­‐libevent  libcouchbase-­‐devel • Ubuntu • wget  ubuntu  repositories • $  sudo  apt-­‐get  install  libcouchbase2-­‐libevent  libcouchbase-­‐dev www.couchbase.com/communi/es/c/geBng-­‐started Friday, October 11, 13
  • 9. Make a Connection require 'rubygems' require 'couchbase'   cb = Couchbase.connect( :bucket => "default", :hostname => "localhost") data = { jsonkey: "value", created_at: Time.now } cb.add("mydata", data) puts cb.get("mydata") #!/usr/bin/env python from couchbase import Couchbase from pprint import pprint from datetime import datetime cb = Couchbase.connect(bucket='default') data = { "jsonkey": "value", "created_at": datetime.now() } cb.add('mydata', data) result = cb.get('mydata') pprint(result.value, indent=4) RUBY PYTHON Friday, October 11, 13
  • 10. Make a Connection import com.couchbase.client.CouchbaseClient; import java.net.URI; import java.util.*; import com.google.gson.Gson; import com.google.gson.GsonBuilder; public class HelloWorld {   public static void main(String[] args) throws Exception {   List<URI> hosts = Arrays.asList( new URI("http://127.0.0.1:8091/pools") ); CouchbaseClient cb = new CouchbaseClient(hosts, "default", ""); Gson json = new Gson(); Hashtable data = new Hashtable(); data.put("jsonkey", "value"); data.put("created_at", new Date()); cb.add("mydata", json.toJson(data)) System.out.println(cb.get("mydata")); cb.shutdown(); } } JAVA Friday, October 11, 13
  • 11. Make a Connection var Couchbase = require('couchbase'); var cb = new Couchbase.Connection({bucket: "default"}, function(err) { }); var data = { jsonkey: "value", created_at: new Date().toString() }; cb.add("mydata", data); console.log(cb.get("mydata")); NODEJS <?php // adjust these parameters to match your installation $cb = new Couchbase("127.0.0.1:8091", "", "", "default"); $data = array("jsonkey" => "value", "created_at" => date("Y-m-d H:i:s")); $cb->add("mydata",json_encode($data)); var_dump($cb->get("mydata")); ?> PHP Friday, October 11, 13
  • 12. Friday, October 11, 13
  • 13. RDBMS  VS  COUCHBASE Friday, October 11, 13
  • 14. RDBMS Organization • RDBMS organizes data as tables -Tables represent data in rows; n columns of m rows -Table rows have a specific schema, each column as a static type -Simple Datatypes: strings, numbers, datetimes, booleans, can be represented by columns in a single table -Complex Datatypes: dictionaries/hashes, arrays/lists cannot be represented in a single table [Impedence Mismatch] • All rows have identical schema, schema changes require taking database offline, migrations, royal pains • Reading/Writing/Transactions require mutex and locking Friday, October 11, 13
  • 15. Couchbase Organization • Couchbase operates like a Key-Value Document Store -Simple Datatypes: strings, numbers, datetime, boolean, and binary data can be stored; they are stored as Base64 encoded strings -Complex Datatypes: dictionaries/hashes, arrays/lists, can be stored in JSON format (simple lists can be string based with delimiter) -JSON is a special class of string with a specific format for encoding simple and complex data structures • Schema is unenforced and implicit, schema changes are programmatic, done online, and can vary from Document to Document Friday, October 11, 13
  • 16. Complex Datatypes • Simple Types are easy, make them columns • Complex Types are more challenging, require separate tables and joins, slower to store and retrieve • ORM's reduce complexity but trade off additional speed/scale, hard to optimize RDBMS public class User { private String name; private String email; private Integer age; private Boolean gender_male; private DateTime created_at; private ArrayList items_viewed; private Hashtable preferences; private ArrayList<Books> authored; public User(...) { ... } ... } Friday, October 11, 13
  • 17. Complex Datatypes • Can represent both simple and complex data types in JSON data structures • Can modify schema on the fly, and Documents of a specific "type" can vary in schema • "Type" is arbitrary, it's a programming strategy, there are no actual "types", but it's typical to embed the class name as a "doctype" json key Couchbase { "doctype": "User", "name": "Jasdeep Jaitla", "email": "jasdeep@couchbase.com", "age": 38, "gender_male": true, "created_at": "2013-09-20 23:59:59", "items_viewed": [ "12345", "23456", 34567" ], "preferences": { "email_notifications": true, "sms_notifications": false }, "authored": [ { "title": "Couchbase Models", "price": 49.95 } ] } Friday, October 11, 13
  • 18. Benefits of JSON • Can Represent Complex Objects and Data Structures • Very simple notation, lightweight, compact, readable • The most common API return type for Integrations -Facebook, Twitter, you name it, return JSON • Native to Javascript (can be useful) • Serialization and Deserialization are very fast Friday, October 11, 13
  • 19. Friday, October 11, 13
  • 20. OPERATIONS Friday, October 11, 13
  • 21. • get  (key) –  Retrieve  a  document • set  (key,  value) –  Store  a  document,  overwrites  if  exists • add  (key,  value) –  Store  a  document,  error/excepHon  if  exists • replace  (key,  value) –  Store  a  document,  error/excepHon  if  doesn’t  exist • cas  (key,  value,  cas) –  Compare  and  swap,  mutate  document  only  if  it  hasn’t  changed  while   execuHng  this  operaHon Store  &  Retrieve  OperaHons Friday, October 11, 13
  • 22. Atomic Counter Operations These  operaHons  are  always  executed  in  order  atomically. • incr  (key) –  Increase  an  atomic  counter  value,  default  by  1 • cb.incr(“my_counter”)  #  now  it’s  2 • decr  (key) –  Decrease  an  atomic  counter  value,  default  by  1 • cb.decr(“my_counter”)  #  now  it’s  1 Friday, October 11, 13
  • 23. Non-JSON Operations You  can  use  these  creaHvely! • prepend  (key,  value) –  prepend  exisHng  string  in  couchbase  with  value • cb.prepend(“mykey”,  "jumped  over  the  lazy  dog")   • cb.prepend("mykey",  "the  brown  fox  ")   • append  (key,  value) –  append  exisHng  string  in  couchbase  with  value • cb.append(“mykey2”,  "oranges")   • cb.append("mykey2",  "  apples  bananas") Friday, October 11, 13
  • 24. Friday, October 11, 13
  • 25. CONCURRENCY Friday, October 11, 13
  • 26. Optimistic Concurrency with CAS • Every storage operation (including touch) creates a new "CAS" value, which is just a long int • The CAS value simply represents the current state of the Document, it's like a version number • You can use this CAS for "Optimistic Concurrency" -value, flags, cas = get("mykey", :extended => true) -This will only succeed if the CAS matches • replace("mykey", newvalue, :cas => cas) -If another process had changed the "mykey" document, a new CAS will have been generated, and that replace operation will fail Friday, October 11, 13
  • 27. Friday, October 11, 13
  • 28. EXPIRATIONS Friday, October 11, 13
  • 29. Storage Operations with Expirations • CMS Framework Cache can be configured to use Couchbase -In most frameworks this is simple, as they typically already have memcached support • Create/Update the Value and Expiration -[Ruby] set/add/replace("mykey", value, :ttl => 30) -[Java] set/add/replace("mykey", 30, value) • Update the expiration only -[Ruby] touch("mykey", :ttl => 30) -[Java] touch("mykey", 30) Friday, October 11, 13
  • 30. Friday, October 11, 13
  • 31. DURABILITY Friday, October 11, 13
  • 32. Using Storage with Observe • Callback when it has been written to disk on active partition • Callback when it has been written to a replica(s) • Callback when it has been written to replica(s) disk • Observe Persisted to Disk and Replicated • [Ruby] set/add/replace("mykey", value, :observe => {:persisted => 1, :replicated => 1}) • [Java] set/add/replace("mykey", 0, value, PersistTo.MASTER, ReplicateTo.ONE) • Observe Replicated Only • [Ruby] set/add/replace("mykey", value, :observe => {:replicated => 1}) • [Java] set/add/replace("mykey", 0, value, ReplicateTo.ONE) Friday, October 11, 13
  • 33. Friday, October 11, 13
  • 34. BASIC  KEY  PATTERNS Friday, October 11, 13
  • 35. Objects Serialized to JSON and Back User  Object string uid string firstname string lastname int age array favorite_colors string email u::jasdeep@couchbase.com { “uid”:  123456, “firstname”:  “jasdeep”, “lastname”:  “Jaitla”, “age”:  22, “favorite_colors”:  [“blue”,  “black”], “email”:  “jasdeep@couchbase.com” } User  Object string uid string firstname string lastname int age array favorite_colors string email u::jasdeep@couchbase.com { “uid”:  123456, “firstname”:  “jasdeep”, “lastname”:  “Jaitla”, “age”:  22, “favorite_colors”:  [“blue”,  “black”], “email”:  “jasdeep@couchbase.com” } add() get() Friday, October 11, 13
  • 36. Basic Keying • Use a Unique value for key (email, username, sku, isbn, etc.) -Users • u::jasdeep@couchbase.com • u::scalabl3 -Products • p::978-0321573513 [isbn] • Predictable Keys can follow Key-Value patterns (Users typically can be done this way and are the most numerous items) • Unpredictable Keys (GUID, UUID, etc.) require Views (Map- Reduce Indexes) to find their documents Friday, October 11, 13
  • 37. Counter-ID ApplicaHon id  =  incr("counter-­‐key") add("key"  +  id,  data) ApplicaHon count  =  get("counter-­‐key") mulH-­‐get(keys[]) Data  Crea/on Iterate  Through  Collec/on Friday, October 11, 13
  • 38. Counter-ID • Similar to IDENTITY column in RDBMS • Creating New Document is a pair of operations, INCR and ADD -Initialize one Key as an Atomic Counter (I do at App Start) -Increment Counter and save new value ✴ id = client.incr("blog::couchbase::comment_count") -Use the id as component of key for new document ✴ client.add(""blog::couchbase::c"::" + id, self.to_json) Friday, October 11, 13
  • 39. Lookup Pattern ApplicaHon add("jasdeep@couchbase.com",   "u::550e8400-­‐e29b-­‐41d4-­‐a716") add("u::550e8400-­‐e29b-­‐41d4-­‐a716",  data) add("scalabl3",   "u::550e8400-­‐e29b-­‐41d4-­‐a716") ApplicaHon key  =  get("jasdeep@couchbase.com") get(key) Data  Crea/on Data  Retrieval Friday, October 11, 13
  • 40. Lookup Pattern • Create simple document that has referential data (Key) to primary document -Primary Document u::a2bf2-23317-2302 -Lookup Document: u::jasdeep@couchbase.com { u::a2bf2-23317-2302 } • Lookup Documents aren't JSON, they should just be the Key as a string so you skip JSON parsing • Requires Two GET operations, first GET Lookup, then GET primary Document -key = client.get("u::jasdeep@couchbase.com") -doc = client.get(key) Friday, October 11, 13
  • 41. User Data Multiple Social Networks & Emails u::count 1001 u::1001 {  "name":  "Jasdeep  Jaitla",      "facebook_id":  16172910,      "email":  "jj@scalabl3.com", “password”:  ab02d#Jf02K      "created_at":  "5/1/2012  2:30am", “facebook_access_token”:  xox0v2dje20, “twi0er_access_token”:  20jffieieaaixixj  } ::16172910 1001   nflx::2939202 1001 twtr::2920283830 1001 em::jj@jasdeep.com 1001 em::jj@scalabl3.com 1001 uname::scalabl3 1001 Friday, October 11, 13
  • 42. Combine Counter-ID and Lookup ApplicaHon add("jasdeep@couchbase.com",  id) add("u::"  +  id,  data) add("scalabl3",  id) ApplicaHon key  =  get("jasdeep@couchbase.com") get(key) Data  Crea/on Data  Retrieval id  =  incr("user::count") Friday, October 11, 13
  • 43. Combine Counter-ID and Lookup Pro's • Binary Operations, overall faster than large volume of View Queries • Essentially creates several ways to find a single document • Is always consistent, just like all other Binary operations Con's • Increases Number of Documents, therefore Metadata usage in RAM -But this is generally a non-issue for most people Friday, October 11, 13
  • 44. User Data (Sample) CREATE TABLE Users id, int, identity(1000) PRIMARY KEY name, nvarchar(100) facebook_id, bigint email, nvarchar(255) created_at, datetime u::count 1 u::1001 {  "name":  "Jasdeep  Jaitla",      "facebook_id":  16172910,      "email":  "jj@scalabl3.com",      "created_at":  "5/1/2012  2:30am"  } ::16172910 1001 em::jj@scalabl3.com 1001 Friday, October 11, 13
  • 45. INSERT INTO Users (name, facebook_id, email, created_at) VALUES ("Jasdeep Jaitla", 16172910, "jj@scalabl3.com", "5/1/2012 2:30am") Get User By FB SELECT * FROM Users WHERE facebook_id = 16172910 Get User by Email SELECT * FROM Users WHERE email = “jj@scalabl3.com” user_data = { "name": "Jasdeep Jaitla", "facebook_id": 16172910, "email": "jj@scalabl3.com", "created_at": "5/1/2012 2:30am" } uid = couch.incr("u::count") + 1000 couch.add ("u::" + uid, user_data) couch.set ("em::" + user_email, uid) couch.set ("fb::" + user_fb, uid) Get User By FB uid = couch.get("fb::16172910") user_data = couch.get ("u::" + uid) Get User By Email uid = couch.get("em::jj@scalabl3.com") user_data = couch.get ("u::" + uid) Each Table Grows and it gets Slower for Each Request RDBMS Couchbase Friday, October 11, 13
  • 46. Friday, October 11, 13
  • 47. MULTIPLE  DOCUMENTS Friday, October 11, 13
  • 48. Aligning Documents to Behaviors user::1 { name: “Jasdeep”, points: 1000, shopping_carts: [ 1000, 1001, 1002 ], products_bought: [ 2000, 2001, 2002], games_won: [ 3000, 3001, 3002, 3004], notifications: [ “Lorem”, “Ipsum”, “docet”, ...] } user::1 { name: “Jasdeep” } user::1::points { points: 1000 } user::1::shopping_carts { carts: [ 1000, 1001, 1002 ], products_bought: [ 2000, 2001, 2002] } user::1::games_won { game_ids: [ 3000, 3001, 3002, 3004] } user::1::notification_count 57 user::1::notifications::57 { message: “Hi Bob” } user::1::notifications::56 { message: “Happy Hour?” } Friday, October 11, 13
  • 49. Behavior Driven Design • Reduce the number of User Actions that affect a single document • Instead, separate that user document in a predictable key structure and make it accessible via getters and setters in your class • Like TDD/BDD encourages smaller, simpler methods, that are easier to write and maintain Friday, October 11, 13
  • 50. Friday, October 11, 13
  • 51. RDBMS  TO  COUCHBASE Friday, October 11, 13
  • 52. The Most Common Mental Adjustments #1 • In SQL we tend to want to avoid hitting the database as much as possible • We know, intuitively, that it’s costly when tying up connection pools, and overloading the db servers • Even with caching and indexing tricks, and massive improvements over the years, SQL still gets bogged down by complex joins and huge indexes • In Couchbase, get’s and set’s are so fast they are trivial, and not bottlenecks, this is hard for many people to accept (reptilian SQL mind) Friday, October 11, 13
  • 53. Complex Joins vs Multiple Gets select * from (((client left join brochure on client.cid = brochure.cid) left join translation on brochure.bid = translation.bid ) left join version on translation.tid = version.tid) left join language on language.lid = translation.lid order by client.cid, brochure.bid, translation.tid, version.vid shopping_cart_id = couch.get(“u::1001::transaction::count”) cart_items = couch.get(“u::sc::” + shopping_cart_id”) foreach item_id in cart_items.items cart_details.push( couch.get(“product::” + item_id) ) end Friday, October 11, 13
  • 54. The Most Common Mental Adjustments #2 • The key to finding data is the Key! • Key design can give you many different ways to access data, being able to predict key values, and use them creatively • Many newcomers see Views as a replacement for key design, because it seems more “SQL”-like • Use Views for what you cannot do with Key Design, and there are lots of things you can't do with Key Design Friday, October 11, 13
  • 55. Friday, October 11, 13
  • 56. Q  &  A Friday, October 11, 13
  • 57. Resources Main  Resource  Portal www.couchbase.com/communiHes Code  Samples  Going  through  Opera/ons www.github.com/couchbaselabs/DeveloperDay Couchbase  Q  &  A www.couchbase.com/communiHes/q-­‐and-­‐a My  Email:  jasdeep@couchbase.com My  Twiher:  @scalabl3 Friday, October 11, 13
  • 58. Couchbase 103: Views Monday  October  14th,  10:00am  PDT Friday, October 11, 13