CouchConf_Getting Started with Couchbase Application Dev

Like this? Share it with your network

Share

CouchConf_Getting Started with Couchbase Application Dev

  • 976 views
Uploaded on

 

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
976
On Slideshare
922
From Embeds
54
Number of Embeds
3

Actions

Shares
Downloads
7
Comments
0
Likes
0

Embeds 54

http://www.couchbase.com 48
http://beta.stage.couchbase.com 5
http://new.stage.couchbase.com 1

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide
  • if you are ingesting Tweets, git commits, and linked-in API data, there ’ s little value in transforming it before you save it. just store it and sort it out later — the same holds for user data
  • schemaless is good as far as it goes, but what it ’ s really saying is: “ don ’ t worry about the database ” so a lot of the patterns move to the application. that ’ s what this section is about.
  • By design, installation of couchbase server is easy.
  • Again, quick. show it is all simple: gem install, easy_install, pecl isntall, mvn isnstall, etc ...
  • under ideal conditions no pauses
  • schemaless is good as far as it goes, but what it ’ s really saying is: “ don ’ t worry about the database ” so a lot of the patterns move to the application. that ’ s what this section is about.
  • A document database is a species of aggregate oriented database. Aggregate is just a term for any complex data object in your runtime. You may be used to building them from relational database queries. With a document database these aggregates are the currency of interchange with the database, instead of breaking it up into tables.
  • there ’ s room for a lot of further optimizations schema is driven by the application
  • "Capture the user's intent" -- if you are gonna import a bunch of Twitter API data and mix it up with Github API data, will you enjoy transforming it to fit a schema on the way through? Same thing with user data. Views are asynchronous, interactive requires a high degree of consistency
  • schemaless is good as far as it goes, but what it ’ s really saying is: “ don ’ t worry about the database ” so a lot of the patterns move to the application. that ’ s what this section is about.
  • this could be the ruby form POST example
  • this could be the ruby form POST example
  • this could be the ruby form POST example
  • why would you use a lock? assuming all your operations are idempotent: ONLY when the cost of CAS failure is greater than the cost of setting the lock. locking is only an optimization, provides no transactional boundaries.
  • schemaless is good as far as it goes, but what it ’ s really saying is: “ don ’ t worry about the database ” so a lot of the patterns move to the application. that ’ s what this section is about.
  • This heavily depends on your application and use case. Are these separate objects in the ORM layer? Are these accessed together What are the atomicity requirements What are concurrency requirements
  • The simpler approach is to embed all related information into 1 document. Data is denormalized and almost represent a pre-computed join across tables. In contrast to this, you could split out objects into separate docs and include references in related objects. The join needs to be processed client-side by the application. Most document databases currently do not support joins.
  • Key selection is very important. Key ’ s are hard to change at a latter point. ID ’ s are similar to the primary key defined when the table is created. Lookups are extremely fast because clients know exactly which server the document belongs to based on consistent hashing. ID ’ s can appear only once per bucket. In couchbase, we call them buckets, A bucket is equivalent to a table or a collection. Selection your ID depends on your document model as well. Questions. Options. UUID…. Hand crafted. In Some NoSQL database systems, data is sorted by ID. If you use prefixes for related objects , you can look up related objects faster. Selecting a clever ID, can make your life a lot easier.
  • You have different entities within the application. - - -
  • It has most of the fields you ’ d expect to have in a blog entry. The comment field is an array within embedded comment objects Easy to get all information about a blog. Issues with this approach. You may not want to display all of the comments on a page. Some blogs may be very popular and have lots commments. So you don ’ t want to get such a large amount of data from the database.
  • It has most of the fields you ’ d expect to have in a blog entry. The comment field is an array within embedded comment objects Easy to get all information about a blog. Issues with this approach. You may not want to display all of the comments on a page. Some blogs may be very popular and have lots commments. So you don ’ t want to get such a large amount of data from the database.
  • If you ’ re expecting a very large number of comments, or want to display them threaded, you can easily imagine doing so by extending the list technique discussed earlier. This makes the application more complicated but load is spread across the cluster.
  • These were simpler times, not everyone used the Internet, there weren ’ t SmartPhones and apps, everything had a single interface, the Browser
  • As the Internet and Internet Usage grew, your app became more popular and you needed to add servers to handle it, App Servers scale linearly so that ’ s no problem Add App Servers as your Requests go up All App Servers still going to single DB Instance
  • As Requests go up even further, lots of data is used over and over, so cache it Database Begins Scaling Vertically, we need more Power!
  • Add multiple platforms and services to the application, now requests are skyrocketing App Servers can still scale linearly but Database Server can ’ t handle the loads First you use Replication, but now Write Throughput is limited to master Replication eventually won ’ t be able to keep up, so you begin losing Consistency Eventually you try to use Sharding, but now you have to re-architect the application because you lose Relational Capabilities, essentially turning your SQL into NoSQL but with all the disadvantages of not being NoSQL to begin with Your architecture now has a lot of moving parts, lots of bottlenecks, every change requires you to take down the application and you ’ re also losing employees, the ones that architected this to begin with

Transcript

  • 1. Getting Started withCouchbase App Development John Zablocki Developer Advocate @codevoyeur 1
  • 2. What we’ll talk about• Setup Development Environment• Intro to Document Databases• Intro to Couchbase Operations• Basic Document Modeling• Modeling Examples 2
  • 3. Basics & Getting started 33
  • 4. Couchbase is Faster, Scales and Simplifies App Servers Couchbase Server Cluster API Servers 4
  • 5. Download & Install Couchbase• Downloads at couchbase.com/download • Linux: Ubuntu and Red Hat/CentOS – Packages for most common distributions. • dpkg -i , rpm -i, etc. • Mac – Download a .zip, open, drag to Applications • Windows – Download a setup.exe, double click • Provision via wizard in Web Console – Or provision via REST interface: operations folks to automate provisioning (i.e Chef, puppet, Rightscale rightscript) 5
  • 6. Setup Smart Client SDK• High performance, official client libraries for Java, .NET, PHP, Ruby, C, Python – Community supported for Clojure, JRuby, Perl, Go, Erlang, Node.js• Head to couchbase.com/develop for SDKs – Client libraries – Getting Started guides – Tutorial with a sample application – A complete API reference 6
  • 7. Client Setup: Getting Cluster Configuration Cluster Configuration Couchbase Client over REST { …     "bucketCapabilities": [         "touch",          "sync",          "couchapi" http://myserver:8091/     ],      "bucketCapabilitiesVer": "sync-1.0",      "bucketType": ”couchbase",     "name": "default",      "nodeLocator": "vbucket",      "nodes": [ …. 7
  • 8. Client Setup: Getting Cluster Configuration Couchbase Client New Node Added to Cluster and Coming Online Couchbase Topology Couchbase Topology Update Update 8
  • 9. Document Database Basics 99
  • 10. The relational approach to model data http://martinfowler.com/bliki/AggregateOrientedDatabase.html 10
  • 11. Compared to a Document Database o::1001 { uid: “ji22jd”, customer: “Ann”, line_items: [ { sku: 0321293533, quan: 3, unit_price: 48.0 }, { sku: 0321601912, quan: 1, unit_price: 39.0 }, { sku: 0131495054, quan: 1, unit_price: 51.0 } ], payment: { type: “Amex”, expiry: “04/2001”, last5: 12345 } • Easy to distribute data • Makes sense to application programmers 11
  • 12. JSON Documents• Maps more closely to external API• CRUD Operations, lightweight and implicit schema myDocument = { “fields” : [“with basic types”, 3.14159, true], “like” : “your favorite language.”, “status”: { “apis” : true, “databases” : “document” } }• Stored under a unique identifier key client.set(“mydocumentid”, myDocument); mySavedDocument = client.get(“mydocumentid”); 12
  • 13. Object to JSON back to ObjectUser Object u::michael.nitschinger@couchbase.co m set()string uid {string firstname “uid”: 123456,string lastname “firstname”: “Michael”,int age “lastname”: “Nitschinger”, “age”: 24,array favorite_colors “favorite_colors”: [“blue”, “black”],string email “email”: “michael.nitschinger@cou…” }User Object u::michael.nitschinger@couchbase.co m get() string uid { string firstname “uid”: 123456, string lastname “firstname”: “Michael”, int age “lastname”: “Nitschinger”, “age”: 24, array favorite_colors “favorite_colors”: [“blue”, “black”], string email “email”: “michael.nitschinger@cou…” } 13
  • 14. Couchbase Operations 14 14
  • 15. The Most Common Mental Adjustment• In SQL we tend to want to avoid hitting the database as much as possible: – Costly, tying up connection pools and overloading db servers – Even with caching and indexing tricks, and massive improvements over the years, SQL still gets bogged down by complex joins and huge indexes• In Couchbase, get’s and set’s are so fast they are trivial, and not bottlenecks, this is hard for many people to accept at first 15
  • 16. Couchbase Basic Operations• get (key) – Retrieve a document• set (key, value) – Store a document, overwrites if exists• add (key, value) – Store a document, error/exception if exists• replace (key, value) – Store a document, error/exception if doesn’t exist• cas (key, value, cas) – Compare and swap, mutate document only if it hasn’t changed while executing this operation 16
  • 17. Couchbase Basic Operations (cont’d)Atomic Counters are a special structure in Couchbase, theyare executed in order and are Positive Integer Values•set (key, value) – Use set to initialize the counter • cb.set(“my_counter”, 1)•incr (key) – Increase an atomic counter value, default by 1 • cb.incr(“my_counter”) # now it’s 2•decr (key) – Decrease an atomic counter value, default by 1 • cb.decr(“my_counter”) # now it’s 1 17
  • 18. INCR / DECR• Maintain a numerical counter• Good for shared counters with tons of increments• Example: game scores• Can INCR by any integer• Accessible as a JSON number in views 18
  • 19. Simple Example in Ruby # example.rb # user.rb require “./user.rb” require “rubygems” require “couchbase” u1 = User.new({ :email => “michael.nitschinger@c…”, class User :name => “Michael Nitschinger”, attr_accessor :name, :email, :title, :twitter :title => “Developer Advocate”, :twitter => “@daschl” def initialize(attr = {}) }) attr.each do |name, value| setter = "#{name}=” u1.save next unless respond_to?(setter) send(setter, value) end end def save client = Couchbase.bucket client.set(@email.downcase, self.to_json) end end 19
  • 20. Add Retrieval# example.rb # user.rbrequire “./user.rb” require “rubygems” require “couchbase”u1 = User.new({ :email => “michael.nitschinger@c…”, class User :name => “Michael Nitschinger”, attr_accessor :name, :email, :title, :twitter :title => “Developer Advocate”, :twitter => “@daschl”}) def initialize(attr = {}) ... endu1.save def save client = Couchbase.bucketu1 = User.find_by_email(“michael.nitschinger@c…”) client.set(@email.downcase, self.to_json)if u1 endputs “User Found!”puts u1.inspect def self.find_by_email(email)else client = Couchbase.bucketputs “User Not Registered!” doc = client.get(email.downcase)end return doc ? User.new(doc) : nil end end 20
  • 21. Add Instance Variables# example.rb # user.rbrequire “./user.rb” require “rubygems” require “couchbase”u1 = User.find_by_email(“michael.nitschinger@c…”) class Userif u1 attr_accessor :name, :email, :title, :twitteru1.fb_id = “682829292”u1.fb_token = “f02jdjd020d8373730djd02” attr_accessor :fb_id, :fb_tokenu1.saveelse def initialize(attr = {}) ... end# create userend def save ... end def self.find_by_email(email) client = Couchbase.bucket doc = client.get(email.downcase) return doc ? User.new(doc) : nil end end 21
  • 22. Distributed System Design: Concurrency Controls# actors.rb Actor 1 Actor 2c = Couchbase.bucketc.set(“mydoc”, { :myvalue => nil } Success CAS mismatchdoc1, flags, cas = c.get(“mydoc”, :extended => true)doc2, flags, cas = c.get(“mydoc”, Couchbase Server :extended => true)#c.set (“mydoc”, { “myvalue”: true }, :cas => cas)# will fail because cas has changedc.set (“mydoc”, { “myvalue”: true }, :cas => cas) 22
  • 23. Document Modeling 23 23
  • 24. Basic Document Modeling • Are these separate objects in the model layer? Q • Are these objects accessed together?When considering how to model data for a given application: • Think of a logical container for the data • Think of how data groups together 24
  • 25. Document Design Options • One document that contains all related data – Data is de-normalized – Better performance and scale – Eliminate client-side joins • Separate documents for different object types with cross references – Data duplication is reduced – Objects may not be co-located – Transactions supported only on a document boundary – Most document databases do not support joins 25
  • 26. Basic Document Design: Document ID / Key selection• Similar to primary keys in relational databases• Documents are sharded based on the document ID• ID based document lookup is extremely fast• ID-Key’s can only appear once in a bucket, unique Q • Do you have a unique way of referencing objects? • Are related objects stored in separate documents?Options• UUIDs, date-based IDs, numeric IDs• Hand-crafted (human readable)• Matching prefixes (for multiple related objects) 26
  • 27. Example: Entities for a Blog • User profile BLOG • • Blog posts – Contain the posts themselves • Blog comments – Comments from other users • Option 1 - Keep it with Blog Document • Option 2 - Separate it Out 27
  • 28. Blog Document – Option 1 – Single document {Blog Post Objectstring id “_id”: “daschl_Hello_World”,string author “author”: “daschl”,string type “type”: “post” “title”: “Hello World”,string title “format”: “markdown”,string format “body”: “Hello from [Couchbase](http://couchbase.com).”,string body “html”: “<p>Hello from <a href=“http: …,string htmlarray comments “comments”: [ [“format”: “markdown”, “body”:”Awesome post!”], [“format”: “plain”, “body”:”Like it.” ] ] } 28
  • 29. Blog Document – Option 2 - Split into Multiple Docs {Blog Post Object “_id”: “daschl_Hello_World”,string id “author”: “daschl”,string author “type”: “post” “title”: “Hello World”,string type “format”: “markdown”,string title “body”: “Hello from [Couchbase](http://couchbase.com).”,string format “html”: “<p>Hello from <a href=“http: …”, “num_comments”: 2string body }string htmlint num_comments { “_id”: “daschl_Hello_World_1”, “format”: “markdown”,Comment Object “body”: “Awesome post!”string format } {string body “_id”: “daschl_Hello_World_2”,int id “format”: “plain”, “body”: “Love It!” } 29
  • 30. Threaded Comments• You can imagine how to take this to a threaded list List First Reply to List comment comment Blog More CommentsAdvantages• Only fetch the data when you need it • For example, rendering part of a web page • Spread the data and load across the entire cluster 30
  • 31. Demo Time/Questions? 31 31