Your SlideShare is downloading. ×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Saving this for later?

Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime - even offline.

Text the download link to your phone

Standard text messaging rates apply

CCSF12_Getting_started_with_Couchbase_App_Development

2,715
views

Published on

Published in: Technology

0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,715
On Slideshare
0
From Embeds
0
Number of Embeds
6
Actions
Shares
0
Downloads
30
Comments
0
Likes
2
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • if you are ingesting Tweets, git commits, and linked-in API data, there’s little value in transforming it before you save it. just store it and sort it out later — the same holds for user data
  • These were simpler times, not everyone used the Internet, there weren’t SmartPhones and apps, everything had a single interface, the Browser
  • As the Internet and Internet Usage grew, your app became more popular and you needed to add servers to handle it, App Servers scale linearly so that’s no problem Add App Servers as your Requests go up All App Servers still going to single DB Instance
  • As Requests go up even further, lots of data is used over and over, so cache it Database Begins Scaling Vertically, we need more Power!
  • Add multiple platforms and services to the application, now requests are skyrocketing App Servers can still scale linearly but Database Server can’t handle the loads First you use Replication, but now Write Throughput is limited to master Replication eventually won’t be able to keep up, so you begin losing Consistency Eventually you try to use Sharding, but now you have to re-architect the application because you lose Relational Capabilities, essentially turning your SQL into NoSQL but with all the disadvantages of not being NoSQL to begin with Your architecture now has a lot of moving parts, lots of bottlenecks, every change requires you to take down the application and you’re also losing employees, the ones that architected this to begin with
  • schemaless is good as far as it goes, but what it’s really saying is: “ don’t worry about the database” so a lot of the patterns move to the application. that’s what this section is about.
  • By design, installation of couchbase server is easy.
  • Again, quick. show it is all simple: gem install, easy_install, pecl isntall, mvn isnstall, etc ...
  • under ideal conditions no pauses
  • schemaless is good as far as it goes, but what it’s really saying is: “ don’t worry about the database” so a lot of the patterns move to the application. that’s what this section is about.
  • A document database is a species of aggregate oriented database. Aggregate is just a term for any complex data object in your runtime. You may be used to building them from relational database queries. With a document database these aggregates are the currency of interchange with the database, instead of breaking it up into tables.
  • there’s room for a lot of further optimizations schema is driven by the application
  • "Capture the user's intent" -- if you are gonna import a bunch of Twitter API data and mix it up with Github API data, will you enjoy transforming it to fit a schema on the way through? Same thing with user data. Views are asynchronous, interactive requires a high degree of consistency
  • schemaless is good as far as it goes, but what it’s really saying is: “ don’t worry about the database” so a lot of the patterns move to the application. that’s what this section is about.
  • this could be the ruby form POST example
  • this could be the ruby form POST example
  • this could be the ruby form POST example
  • this could be the ruby form POST example
  • why would you use a lock? assuming all your operations are idempotent: ONLY when the cost of CAS failure is greater than the cost of setting the lock. locking is only an optimization, provides no transactional boundaries.
  • schemaless is good as far as it goes, but what it’s really saying is: “ don’t worry about the database” so a lot of the patterns move to the application. that’s what this section is about.
  • This heavily depends on your application and use case. Are these separate objects in the ORM layer? Are these accessed together What are the atomicity requirements What are concurrency requirements
  • The simpler approach is to embed all related information into 1 document. Data is denormalized and almost represent a pre-computed join across tables. In contrast to this, you could split out objects into separate docs and include references in related objects. The join needs to be processed client-side by the application. Most document databases currently do not support joins.
  • Key selection is very important. Key’s are hard to change at a latter point. ID’s are similar to the primary key defined when the table is created. Lookups are extremely fast because clients know exactly which server the document belongs to based on consistent hashing. ID’s can appear only once per bucket. In couchbase, we call them buckets, A bucket is equivalent to a table or a collection. Selection your ID depends on your document model as well. Questions. Options. UUID…. Hand crafted. In Some NoSQL database systems, data is sorted by ID. If you use prefixes for related objects , you can look up related objects faster. Selecting a clever ID, can make your life a lot easier.
  • You have different entities within the application. - - -
  • It has most of the fields you’d expect to have in a blog entry. The comment field is an array within embedded comment objects Easy to get all information about a blog. Issues with this approach. You may not want to display all of the comments on a page. Some blogs may be very popular and have lots commments. So you don’t want to get such a large amount of data from the database.
  • It has most of the fields you’d expect to have in a blog entry. The comment field is an array within embedded comment objects Easy to get all information about a blog. Issues with this approach. You may not want to display all of the comments on a page. Some blogs may be very popular and have lots commments. So you don’t want to get such a large amount of data from the database.
  • If you’re expecting a very large number of comments, or want to display them threaded, you can easily imagine doing so by extending the list technique discussed earlier. This makes the application more complicated but load is spread across the cluster.
  • Transcript

    • 1. Developing with Couchbase I Jasdeep Jaitla Technology Evangelist @scalabl3 1 1
    • 2. What we’ll talk about• Setup Development Environment• Intro to JSON Documents• Intro to Couchbase Operations• Basic Document Modeling• Modeling Examples• Agile Development 2
    • 3. What Internet Apps looked like in 1995 Client (Browsers) App Server SQL Database Server 3
    • 4. Grow Linearly (For a While)Client (Browsers) App Servers SQL Database Server 4
    • 5. Speed things up through Caching Client (Browsers) App Servers Memcached SQL Database Server 5
    • 6. Use Replication First and Eventually Sharding App Servers Memcached (Membase) SQL Database Clusters API Servers 6
    • 7. Basics & Getting started 77
    • 8. Couchbase is Faster, Scales and Simplifies App Servers Couchbase Server Cluster API Servers 8
    • 9. Download & Install Couchbase• Downloads at couchbase.com/download • Linux: Ubuntu and Red Hat/CentOS – Packages for most common distributions. • dpkg -i , rpm -i, etc. • Mac – Download a .zip, open, drag to Applications • Windows – Download a setup.exe, double click • Provision via wizard in Web Console – Or provision via REST interface: operations folks to automate provisioning (i.e Chef, puppet, Rightscale rightscript) 9
    • 10. Setup Smart Client SDK• High performance, official client libraries for Java, .NET, PHP, Ruby, C, Python, Go – Community supported for Clojure, JRuby, Perl• Head to couchbase.com/develop for SDKs – Client libraries – Screencasts on Getting Started – Getting Started guides – Tutorial with a sample application – A complete API reference 10
    • 11. Client Setup: Getting Cluster Configuration Cluster Configuration Couchbase Client over REST { …     "bucketCapabilities": [         "touch",          "sync",          "couchapi" http://myserver:8091/     ],      "bucketCapabilitiesVer": "sync-1.0",      "bucketType": ”couchbase",     "name": "default",      "nodeLocator": "vbucket",      "nodes": [ …. 11
    • 12. Client Setup: Getting Cluster Configuration Couchbase Client New Node Added to Cluster and Coming Online Couchbase Topology Couchbase Topology Update Update 12
    • 13. Document Database Basics 1313
    • 14. Couchbase Server is a Document Database http://martinfowler.com/bliki/AggregateOrientedDatabase.html This synergy between the programming model and the distribution model is very valuable. It allows the database to use its knowledge of how the application programmer clusters the data to help performance across the cluster. 14
    • 15. Document Database o::1001 { uid: ji22jd, customer: Ann, line_items: [ { sku: 0321293533, quan: 3, unit_price: 48.0 }, { sku: 0321601912, quan: 1, unit_price: 39.0 }, { sku: 0131495054, quan: 1, unit_price: 51.0 } ], payment: { type: Amex, expiry: 04/2001, last5: 12345 } } • Easy to distribute data • Makes sense to application programmers 15
    • 16. JSON Documents• Maps more closely to external API• CRUD Operations, lightweight schema myDocument = { “fields” : [“with basic types”, 3.14159, true], “like” : “your favorite language.”, “status”: { “apis” : true, “databases” : “document” } }• Stored under a unique identifier key client.set(“mydocumentid”, myDocument); mySavedDocument = client.get(“mydocumentid”); 16
    • 17. Object to JSON back to ObjectUser Object u::jasdeep@couchbase.com { set()string uid “uid”: 123456,string firstname “firstname”: “jasdeep”,string lastname “lastname”: “Jaitla”,int age “age”: 22, “favorite_colors”: [“blue”, “black”],array favorite_colors “email”: “jasdeep@couchbase.com”string email }User Object u::jasdeep@couchbase.com { get() string uid “uid”: 123456, string firstname “firstname”: “jasdeep”, string lastname “lastname”: “Jaitla”, int age “age”: 22, “favorite_colors”: [“blue”, “black”], array favorite_colors “email”: “jasdeep@couchbase.com” string email } 17
    • 18. Couchbase Operations 18 18
    • 19. The Most Common Mental Adjustment• In SQL we tend to want to avoid hitting the database as much as possible – We know, intuitively, that it’s costly when tying up connection pools, and overloading the db servers – Even with caching and indexing tricks, and massive improvements over the years, SQL still gets bogged down by complex joins and huge indexes• In Couchbase, get’s and set’s are so fast they are trivial, and not bottlenecks, this is hard for many people to accept at first 19
    • 20. Couchbase Basic Operations• get (key) – Retrieve a document• set (key, value) – Store a document, overwrites if exists• add (key, value) – Store a document, error/exception if exists• replace (key, value) – Store a document, error/exception if doesn’t exist• cas (key, value, cas) – Compare and swap, mutate document only if it hasn’t changed while executing this operation 20
    • 21. Couchbase Basic Operations (cont’d)Atomic Counters are a special structure in Couchbase, theyare executed in order and are Positive Integer Values•set (key, value) – Use set to initialize the counter • cb.set(“my_counter”, 1)•incr (key) – Increase an atomic counter value, default by 1 • cb.incr(“my_counter”) # now it’s 2•decr (key) – Decrease an atomic counter value, default by 1 • cb.decr(“my_counter”) # now it’s 1 21
    • 22. INCR / DECR• Maintain a numerical counter• Good for shared counters with tons of increments• Example: game scores• Can INCR by any integer• Accessible as a JSON number in views• Well look at maintaining a leaderboard in the next session 22
    • 23. Simple Example in Ruby# example.rb # user.rbrequire “./user.rb” require “rubygems” require “couchbase”u1 = User.new({ :email => “jasdeep@scalabl3.com”, class User :name => “Jasdeep Jaitla”, attr_accessor :name, :email, :title, :twitter :title => “Scalability Sherpa”, :twitter => “@scalabl3 def initialize(attr = {}) attr.each do |name,}) value| setter = "#{name}=" next unless respond_to?(setter) send(setter, value) endu1.save end def save C = Couchbase.bucket C. set(@email.downcase, self.to_json) end end end end 23
    • 24. Add Retrieval# example.rb # user.rbrequire “./user.rb” require “rubygems” require “couchbase”u1 = User.new({ :email => “jasdeep@scalabl3.com”, class User :name => “Jasdeep Jaitla”, attr_accessor :name, :email, :title, :twitter :title => “Scalability Sherpa”, :twitter => “@scalabl3}) def initialize(attr = {}) ... endu1.save def save c = Couchbase.bucketu1 = User.find_by_email(“jasdeep@scalabl3.com”) c.set(@email.downcase, self.to_json) endif u1puts “User Found!”puts u1.inspect def self.find_by_email(email)else c = Couchbase.bucketputs “User Not Registered!” doc = c.get(email.downcase)end return doc ? User.new(doc) : nil end end 24
    • 25. Add Instance Variables# example.rb # user.rbrequire “./user.rb” require “rubygems” require “couchbase”u1 = User.find_by_email(“jasdeep@scalabl3.com”) class Userif u1 attr_accessor :name, :email, :title, :twitteru1.fb_id = “682829292”u1.fb_token = “f02jdjd020d8373730djd02” attr_accessor :fb_id, :fb_tokenu1.saveelse def initialize(attr = {}) ... end# create userend def save ... end def self.find_by_email(email) c = Couchbase.bucket doc = c.get(email.downcase) return doc ? User.new(doc) : nil end end 25
    • 26. Add Instance Variables# example.rb # user.rbrequire “./user.rb” require “rubygems” require “couchbase”u1 = User.find_by_email(“jasdeep@scalabl3.com”) class Userif u1 attr_accessor :name, :email, :title, :twitteru1.fb_id = “682829292”u1.fb_token = “f02jdjd020d8373730djd02” attr_accessor :fb_id, :fb_tokenu1.saveelse def initialize(attr = {}) ... end# create userend def save ... end def self.find_by_email(email) c = Couchbase.bucket doc = c.get(email.downcase) return doc ? User.new(doc) : nil end end 26
    • 27. Distributed System Design: Concurrency Controls# actors.rb Actor 1 Actor 2c = Couchbase.bucketc.set(“mydoc”, { :myvalue => nil }doc1, flags, cas = c.get(“mydoc”, :extended => true) Success CAS mismatchdoc2, flags, cas = c.get(“mydoc”, :extended => true)#c.set (“mydoc”, { “myvalue”: true }, :cas => cas) Couchbase Server# will fail because cas has changedc.set (“mydoc”, { “myvalue”: true }, :cas => cas) 27
    • 28. Document Modeling 28 28
    • 29. Basic Document Modeling • Are these separate object in the model layer? Q • Are these objects accessed together?When considering how to model data for a given application • Think of a logical container for the data • Think of how data groups together 29
    • 30. Document Design Options • One document that contains all related data –Data is de-normalized –Better performance and scale –Eliminate client-side joins • Separate documents for different object types with cross references –Data duplication is reduced –Objects may not be co-located –Transactions supported only on a document boundary –Most document databases do not support joins 30
    • 31. Basic Document Design: Document ID / Key selection• Similar to primary keys in relational databases• Documents are sharded based on the document ID• ID based document lookup is extremely fast• ID-Key’s can only appear once in a bucket, unique Q • Do you have a unique way of referencing objects? • Are related objects stored in separate documents?Options• UUIDs, date-based IDs, numeric IDs• Hand-crafted (human readable)• Matching prefixes (for multiple related objects) 31
    • 32. Example: Entities for a Blog • User profile BLOG • • Blog posts – Contains the blogs themselves • Blog comments – Comments from other users • Option 1 - Keep it with Blog Document • Option 2 - Separate it Out 32
    • 33. Blog Document – Option 1 – Single document {Blog Post Object “_id”: “scalabl3_Hello_World”,string id “author”: “scalabl3”,string author “type”: “post”string type “title”: “Hello World”,string title “format”: “markdown”, “body”: “Hello from [Couchbase]string format (http://couchbase.com).”,string body “html”: “<p>Hello from <a href=“http: …,string htmlarray comments “comments”: [ [“format”: “markdown”, “body”:”Awesome post!”], [“format”: “markdown”, “body”:”Like it.” ] ] } 33
    • 34. Blog Document – Option 2 - Split into Multiple Docs {Blog Post Object “_id”: “scalabl3_Hello_World”,string id “author”: “scalabl3”,string author “type”: “post” “title”: “Hello World”,string type “format”: “markdown”,string title “body”: “Hello from [Couchbase](http://couchbase.com).”,string format “html”: “<p>Hello from <a href=“http: …, “num_comments”: 2string body }string htmlarray comments { “_id”: “scalabl3_Hello_World_1”, “format”: “markdown”,Comment Object “body”: “Awesome post!”string format } {string body “_id”: “scalabl3_Hello_World_2”,int id “format”: “markdown”, “body”: “Love It!” } 34
    • 35. Threaded Comments• You can imagine how to take this to a threaded list List First Reply to List comment comment Blog More CommentsAdvantages• Only fetch the data when you need it • For example, rendering part of a web page • Spread the data and load across the entire cluster 35
    • 36. Q&A 36