CouchbaseDistributed Document Database                  Perry Krug              Sr. Solutions Architect                   ...
Company Background  • Leading NoSQL database company  • Open source development and distribution model  • Provide easy-to-...
Paid Production Deployments (partial list)                                             3
Couchbase Server                                             (a.k.a. Membase)           Simple. Fast. Elastic. NoSQL.   Co...
“ Relational database technology has served us well for 40 years, and will likely continue  to do so for the foreseeable f...
Modern Interactive Software Architecture                                            Application Scales Out                ...
Data Layer Matches Application Logic TierArchitecture                                                 Application Scales O...
Couchbase NoSQL: Simple, Fast, Elastic  • Easily scale apps to an “infinite” number of users     – Simply add nodes with a...
PERFORMANCE              9
Key results of Cisco and Solarflare BenchmarkCouchbase Server demonstrates• Consistent sub-millisecond  latency for mixed ...
Your secret weapon: Sub-millisecond AND consistent latencyLatency (micro seconds)                                         ...
Your secret weapon: Sub-millisecond AND consistent latency                                     High throughput with 1.4   ...
SCALE        13
Draw Something by OMGPOP                           14
Draw Something “goes viral” 3 weeks after launch     Draw Something by OMGPOP     Daily Active Users (millions)16141210864...
As usage grew, game data went non-linear.         Draw Something by OMGPOP         Daily Active Users (millions)    16    ...
The game exploded. But Couchbase did not.                       February February February February     March     March   ...
In contrast.     The Simpson’s: Tapped Out     Daily Active Users (millions)16141210864                  #2 Free app on iP...
NO SCHEMA            19
Document database• Json objects• Flexible schema  {          "_id": "brewery_Cleveland_ChopHouse_and_Brewery",          "_...
COUCHBASE SOLUTION   “THE BASICS”                     21
Basic Operation – scale out                   APP SERVER 1                                     APP SERVER 2               ...
Add Nodes                  APP SERVER 1                                APP SERVER 2                                       ...
Fail Over Node                  APP SERVER 1                                APP SERVER 2                                  ...
Couchbase Server 2.0  •   Next major release of Couchbase Server  •   Currently in Developer Preview, approaching Beta and...
Couchbase Server 2.0 Architecture    8092             11211                11210    Couch View       Memcapable 1.0       ...
Couchbase Server 2.0 Architecture    8092           11211                11210    Couch View     Memcapable 1.0       Memc...
Couchbase Server 2.0 Architecture    8092           11211                11210    Couch View     Memcapable 1.0       Memc...
Partitioning The Data – vbucket map                                      29
Indexing and querying • Build in incremental map reduce • Map functions are written and executed on Java Script   (V8) • I...
Incremental Map reduce using javascript• Document from our sample built in beer database  {           "_id": "brewery_Clev...
Map function  • Map functionsfunction (doc) {  if (doc.country, doc.state, doc.city) {    emit([doc.country, doc.state, do...
Reduce functions• Built in reduce functions   • _count   • _sum   • _stats ({“sum”: 1411, “count”: 1411, “min”: 1, “max”: ...
Indexing and Querying                  APP SERVER 1                                APP SERVER 2                   APP SERV...
Cross Data Center Replication  US DATA                        EUROPE DATA                 ASIA DATA  CENTER               ...
Integration to Analytics systems                                    COUCHBASE SERVER CLUSTER                              ...
Couchbase and Hadoop Integration• Support large-scale analytics on application data by streaming data  from Couchbase to H...
Couchbase Client SDKsJava ClientSDK              User Code.Net SDK       Java client API                                  ...
Couchbase Demonstration• Couchbase ServerTemplate  Demo  – Starting with one database    node under load                  ...
THANK YOU                 COUCHBASE         SIMPLE, FAST, ELASTIC NOSQL                QUESTIONS?@couchbase               ...
COUCHBASE CUSTOMERS                      41
Paid Production Deployments – Social Gaming       iki                                              42
Paid Production Deployments – Key SegmentsAd PlatformsSocial Networks                                             43
Production Deployments – Key Industry SegmentsOnline Biz Services E-CommerceOnline Media                                  ...
Production Deployments – Key Industry Segments   HealthCare   Military/  GovernmentCommunications                         ...
Production Deployments – Key Industry Segments   Online  Education Web Design  Financial  Services                        ...
Production Deployments – Key Industry Segments  Software  Security                                                 47
Production Deployments – Enterprises                                       48
Upcoming SlideShare
Loading in …5
×

London web performance-Couchbase meetup

823 views

Published on

Published in: Technology
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
823
On SlideShare
0
From Embeds
0
Number of Embeds
41
Actions
Shares
0
Downloads
23
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide
  • CAPI interface – basic Couch API of which some goes through the caching layer (CRUD), some goes directly to Couch (Views)
  • CAPI interface – basic Couch API of which some goes through the caching layer (CRUD), some goes directly to Couch (Views)
  • CAPI interface – basic Couch API of which some goes through the caching layer (CRUD), some goes directly to Couch (Views)
  • Not yet enabled in current DP, will be available for Beta
  • London web performance-Couchbase meetup

    1. 1. CouchbaseDistributed Document Database Perry Krug Sr. Solutions Architect 1
    2. 2. Company Background • Leading NoSQL database company • Open source development and distribution model • Provide easy-to-develop and -deploy, high-performance, easily scalable, document database • Focused on internet and mobile applications and cloud computing environments • Most mature, reliable and widely deployed solution – 1000s of production deployments worldwide • Located in Silicon Valley (Mountain View, CA) – 80 employees including 50 in engineering/product – Round C company, VCs includes Accel partners, Mayfield, North Bridge, and Ignition 2
    3. 3. Paid Production Deployments (partial list) 3
    4. 4. Couchbase Server (a.k.a. Membase) Simple. Fast. Elastic. NoSQL. Couchbase automatically distributes data across commodity servers. Built-in caching enables apps to read and write data with sub-millisecond latency. And with no schema to manage, Couchbase effortlessly accommodates changing data management requirements. 4
    5. 5. “ Relational database technology has served us well for 40 years, and will likely continue to do so for the foreseeable future to support transactions requiring ACID guarantees. But a large, and increasingly dominant, class of software systems and data do not need those guarantees. Much of the data manipulated by Web applications have less strict transactional requirements but, for lack of a practical alternative, many IT teams continue to use relational technology, needlessly tolerating its cost and scalability limitations. For these applications and data, distributed document cache and database technologies such as Couchbase’s provide a promising alternative. ” Carl Olofson IDC Research Vice President, Information and Data Management RDBMS HAS DOMINATED FOR 40 YEARS BUT NO LONGER BEST SOLUTION FOR MANY APPS 5
    6. 6. Modern Interactive Software Architecture Application Scales Out Just add more commodity web servers Database Scales Up Get a bigger, more complex server Expensive & disruptive sharding, doesn’t perform at web scale 6
    7. 7. Data Layer Matches Application Logic TierArchitecture Application Scales Out Just add more commodity web servers Database Scales Out Just add more commodity data servers • Horizontally scalable with auto-sharding • High performance at web scale • Schema-less for flexibility Scaling out flattens the cost and performance curves 7
    8. 8. Couchbase NoSQL: Simple, Fast, Elastic • Easily scale apps to an “infinite” number of users – Simply add nodes with a single click – Never need to change your application to scale – Simple development with memcached API • High performance with predictably low latency – Sub millisecond reads and writes – No drop in performance as app scales • Schema-less document database – Flexibility to meet rapidly changing market requirements – Roadmap: Indexing, querying similar to RDBMS capabilities • Low cost solution that economically scales with app 8
    9. 9. PERFORMANCE 9
    10. 10. Key results of Cisco and Solarflare BenchmarkCouchbase Server demonstrates• Consistent sub-millisecond latency for mixed workload• High throughput• Linear scalability http://www.cisco.com/en/US/prod/collateral/switches/ps9441/ps9670/white_paper_c11-708169.pdf 10
    11. 11. Your secret weapon: Sub-millisecond AND consistent latencyLatency (micro seconds) Consistently low latencies in microseconds for varying documents sizes with a mixed workload Object size (Bytes) 11
    12. 12. Your secret weapon: Sub-millisecond AND consistent latency High throughput with 1.4 GB/sec data transfer rate using 4 serversOperations per second Linear throughput scalability Number of servers in cluster 12
    13. 13. SCALE 13
    14. 14. Draw Something by OMGPOP 14
    15. 15. Draw Something “goes viral” 3 weeks after launch Draw Something by OMGPOP Daily Active Users (millions)161412108642 2/6 8 10 12 14 16 18 20 22 24 26 28 3/1 3 5 7 9 11 13 15 17 19 21 15
    16. 16. As usage grew, game data went non-linear. Draw Something by OMGPOP Daily Active Users (millions) 16 14 12 10 8 By March 19, there were 6 over 30,000,000 downloads of the app, over 5,000 drawings being stored per second, 4 over 2,200,000,000 drawings stored,over 105,000 database transactions per second, Instagram (7.5M in 5 wks) 2 and over 3.3 terabytes of data stored. 2/6 8 10 12 14 16 18 20 22 24 26 28 3/1 3 5 7 9 11 13 15 17 19 21 16
    17. 17. The game exploded. But Couchbase did not. February February February February March March March 6 13 20 27 5 12 19Drawings/second 0 3 50 333 1660 3000 5400 Total drawings 0 5 12 50 500 1 2.2 million million million million billion billion R/W latency (usec) 30 40 32 31 38 29 34 Servers 6 6 6 18 54 72 90 Without a second of downtime, and while sustaining front-end performance, the cluster was continuously expanded to support growth, absorbing frequent server hardware failures. 17
    18. 18. In contrast. The Simpson’s: Tapped Out Daily Active Users (millions)16141210864 #2 Free app on iPad2 #3 Free app on iPhone 2/6 8 10 12 14 16 18 20 22 24 26 28 3/1 3 5 7 9 11 13 15 17 19 21 18
    19. 19. NO SCHEMA 19
    20. 20. Document database• Json objects• Flexible schema { "_id": "brewery_Cleveland_ChopHouse_and_Brewery", "_rev": "1-00000061480b50910000000000000000", "city": "Cleveland", "updated": "2010-07-22 20:00:20", "code": "44113", "name": "Cleveland ChopHouse and Brewery", "country": "United States", "phone": "1-216-623-0909", "state": "Ohio", "address": [ "824 West St.Clair Avenue" ], "geo": { "loc": [ "-81.6994", "41.4995" ], "accuracy": "ROOFTOP" }, "$expiration": 0, "$flags": 0 } 20
    21. 21. COUCHBASE SOLUTION “THE BASICS” 21
    22. 22. Basic Operation – scale out APP SERVER 1 APP SERVER 2  Docs distributed evenly across COUCHBASE CLIENT LIBRARY COUCHBASE CLIENT LIBRARY servers in the cluster  Each server stores both active CLUSTER MAP CLUSTER MAP & replica docs  Only one server active at a time  Client library provides app with Read/Write/Update Read/Write/Update simple interface to database  Cluster map provides map to which server doc is on  App never needs to know SERVER 1 SERVER 2 SERVER 3  App reads, writes, updates Active Docs Active Docs Active Docs docs Doc 5 DOC Doc 4 DOC Doc 1 DOC  Multiple App Servers can Doc 2 DOC Doc 7 DOC Doc 3 DOC access same document at Doc 9 DOC Doc 8 DOC Doc 6 DOC same time Replica Docs Replica Docs Replica Docs Doc 4 DOC Doc 6 DOC Doc 7 DOC Doc 1 DOC Doc 3 DOC Doc 9 DOC Doc 8 DOC Doc 2 DOC Doc 5 DOC COUCHBASE SERVER CLUSTERUser Configured Replica Count = 1 22
    23. 23. Add Nodes APP SERVER 1 APP SERVER 2  Two servers added to COUCHBASE CLIENT LIBRARY COUCHBASE CLIENT LIBRARY cluster  One-click operation CLUSTER MAP CLUSTER MAP  Docs automatically rebalanced across cluster  Even distribution of docs Read/Write/Update Read/Write/Update  Minimum doc movement  Cluster map updated  App database calls now distributed over larger # SERVER 1 SERVER 2 SERVER 3 SERVER 4 SERVER 5 of servers Active Docs Active Docs Active Docs Active Docs Active Docs Active Docs Doc 5 DOC Doc 4 DOC Doc 1 DOC Doc 3 Doc 2 DOC Doc 7 DOC Doc 3 DOC Doc 6 Doc 9 DOC Doc 8 DOC Doc 6 DOC Replica Docs Replica Docs Replica Docs Replica Docs Replica Docs Replica Docs Doc 4 DOC Doc 6 DOC Doc 7 DOC Doc 7 Doc 1 DOC Doc 3 DOC Doc 9 DOC Doc 9 Doc 8 DOC Doc 2 DOC Doc 5 DOC COUCHBASE SERVER CLUSTERUser Configured Replica Count = 1 23
    24. 24. Fail Over Node APP SERVER 1 APP SERVER 2  App servers happily accessing docs on Server 3 COUCHBASE CLIENT LIBRARY COUCHBASE CLIENT LIBRARY  Server fails  App server requests to server 3 fail CLUSTER MAP CLUSTER MAP  Cluster detects server has failed  Promotes replicas of docs to active  Updates cluster map  App server requests for docs now go to appropriate server  Typically rebalance would follow SERVER 1 SERVER 2 SERVER 3 SERVER 4 SERVER 5 Active Docs Active Docs Active Docs Active Docs Active Docs Active Docs Doc 5 DOC Doc 4 DOC Doc 1 DOC Doc 9 DOC Doc 6 DOC Doc 3 Doc 2 DOC Doc 7 DOC Doc 3 Doc 8 DOC Doc 6 DOC Replica Docs Replica Docs Replica Docs Replica Docs Replica Docs Replica Docs Doc 4 DOC Doc 6 DOC Doc 7 DOC Doc 5 DOC Doc 8 DOC Doc 7 Doc 1 DOC Doc 3 DOC Doc 9 DOC Doc 2 DOC Doc 9 COUCHBASE SERVER CLUSTERUser Configured Replica Count = 1 24
    25. 25. Couchbase Server 2.0 • Next major release of Couchbase Server • Currently in Developer Preview, approaching Beta and GA. What’s new: • New storage engine technology (Append only b-tree) • Indexing and Querying • Incremental Map Reduce • Cross Data Center Replication • Better memory management, large data sets, and other technological improvments • Fully backwards compatible with existing Couchbase Server 25
    26. 26. Couchbase Server 2.0 Architecture 8092 11211 11210 Couch View Memcapable 1.0 Memcapable 2.0 Moxi REST management API/Web UI vBucket state and replication manager Couch API Memcached Global singleton supervisor Rebalance orchestrator Configuration manager Node health monitor Process monitor Heartbeat Membase EP Engine Data Manager Cluster Manager Membase storage interface Distributed CouchStore Indexing Auto compaction http on each node one per cluster CouchBase Erlang/OTP HTTP Erlang port mapper Distributed Erlang 8091 4369 21100 - 21199 26
    27. 27. Couchbase Server 2.0 Architecture 8092 11211 11210 Couch View Memcapable 1.0 Memcapable 2.0 Moxi REST management API/Web UI vBucket state and replication manager Memcached Interface Couch API Global singleton supervisor Rebalance orchestrator Configuration manager Node health monitor Process monitor Heartbeat Membase EP Engine Cluster Manager Membase storage interface Distributed CouchStore Indexing Auto compaction http on each node one per cluster CouchBase Erlang/OTP HTTP Erlang port mapper Distributed Erlang 8091 4369 21100 - 21199 27
    28. 28. Couchbase Server 2.0 Architecture 8092 11211 11210 Couch View Memcapable 1.0 Memcapable 2.0 Moxi REST management API/Web UI vBucket state and replication manager Memcached Interface Couch API Global singleton supervisor Rebalance orchestrator Configuration manager Node health monitor Process monitor Heartbeat EP Engine storage interface Distributed CouchStore Indexing Auto compaction http on each node one per cluster CouchBase Erlang/OTP HTTP Erlang port mapper Distributed Erlang 8091 4369 21100 - 21199 28
    29. 29. Partitioning The Data – vbucket map 29
    30. 30. Indexing and querying • Build in incremental map reduce • Map functions are written and executed on Java Script (V8) • Index is built incrementally as mutation streams in • Query in a scatter/gather fashion 30
    31. 31. Incremental Map reduce using javascript• Document from our sample built in beer database { "_id": "brewery_Cleveland_ChopHouse_and_Brewery", "_rev": "1-00000061480b50910000000000000000", "city": "Cleveland", "updated": "2010-07-22 20:00:20", "code": "44113", "name": "Cleveland ChopHouse and Brewery", "country": "United States", "phone": "1-216-623-0909", "state": "Ohio", "address": [ "824 West St.Clair Avenue" ], "geo": { "loc": [ "-81.6994", "41.4995" ], "accuracy": "ROOFTOP" }, "$expiration": 0, "$flags": 0 } 31
    32. 32. Map function • Map functionsfunction (doc) { if (doc.country, doc.state, doc.city) { emit([doc.country, doc.state, doc.city], 1); } else if (doc.country, doc.state) { emit([doc.country, doc.state], 1); } else if (doc.country) { emit([doc.country], 1); }} REST call: http://db1.couchbase.com:8092/beer-sample/_design/dev_beer/_view/by_location?limit=10 32
    33. 33. Reduce functions• Built in reduce functions • _count • _sum • _stats ({“sum”: 1411, “count”: 1411, “min”: 1, “max”: 1, “sumsqr”:1411})• Developing procedure • Develop against a subset of the data • Built the index on the entire cluster • Promote a dev_ view to production 33
    34. 34. Indexing and Querying APP SERVER 1 APP SERVER 2 APP SERVER 1 APP SERVER 2  Indexing work is distributed COUCHBASE CLIENT LIBRARY COUCHBASE CLIENT LIBRARY COUCHBASE CLIENT LIBRARY COUCHBASE CLIENT LIBRARY amongst nodes  Large data set possible CLUSTER MAP MAP CLUSTER CLUSTER MAPMAP CLUSTER  Parallelize the effort  Each node has index for data stored on it Query Response  Queries combine the results from required nodes SERVER 1 SERVER 2 SERVER 3 Active Docs Active Docs Active Docs Doc 5 DOC Doc 4 DOC Doc 1 DOC Doc 2 DOC Doc 7 DOC Doc 3 DOC Doc 9 DOC Doc 8 DOC Doc 6 DOC Replica Docs Replica Docs Replica Docs Doc 4 DOC Doc 6 DOC Doc 7 DOC Doc 1 DOC Doc 3 DOC Doc 9 DOC Doc 8 DOC Doc 2 DOC Doc 5 DOCUser Configured Replica Count = 1 34
    35. 35. Cross Data Center Replication US DATA EUROPE DATA ASIA DATA CENTER CENTER CENTER Replication Replication Replication  Data close to users  Multiple locations for disaster recovery  Independently managed clusters serving local data 35
    36. 36. Integration to Analytics systems COUCHBASE SERVER CLUSTER  Use the cross data center SERVER 1 SERVER 2 SERVER 3 interface Active Docs Active Docs Active Docs  Agnostic to topology changes Doc 5 DOC Doc 4 DOC Doc 1 DOC  De-duplication Doc 2 DOC Doc 7 DOC Doc 3 DOC  Effective changes feed of the Doc 9 DOC Doc 8 DOC Doc 6 DOC entire cluster Replica Docs Replica Docs Replica Docs Doc 4 DOC Doc 6 DOC Doc 7 DOC Doc 1 DOC Doc 3 DOC Doc 9 DOC Doc 8 DOC Doc 2 DOC Doc 5 DOC CROSS DATA CENTER CONNETROR Changes feed to consumed by Any destinationUser Configured Replica Count = 1 36
    37. 37. Couchbase and Hadoop Integration• Support large-scale analytics on application data by streaming data from Couchbase to Hadoop – Real-time integration using Flume – Batch integration using Sqoop• Examples – Various game statistics (e.g., monthly / daily / hourly rankings) – Analyze game patterns from users to enhance various game metrics memcached Sqoop TAP protocol listener/sender engine interface Couchbase Storage Engine 37
    38. 38. Couchbase Client SDKsJava ClientSDK User Code.Net SDK Java client API CouchbaseClient cb = new CouchbaseClient(listURIs, "aBucket", "letmein"); // this is all the same as before cb.set("hello", 0, "world"); cb.get("hello"); spymemcached HTTP couchDB Map<String, Object> manyThings =PHP SDK Connection connection cb.getBulk(Collection<String> keys); /* accessing a view View view = cb.getView("design_document", "my_view"); Query query = new Query(); query.getRange("abegin", "theend");Ruby SDK Couchbase ServerPython SDK http://www.couchbase.org/code 38
    39. 39. Couchbase Demonstration• Couchbase ServerTemplate Demo – Starting with one database node under load Application user – Dynamically scaling to two database nodes – Easy management and Web application server monitoring – Not possible any other Couchbase Servers database technology In the EC2 or Datacenter 39
    40. 40. THANK YOU COUCHBASE SIMPLE, FAST, ELASTIC NOSQL QUESTIONS?@couchbase perry@couchbase.com 40
    41. 41. COUCHBASE CUSTOMERS 41
    42. 42. Paid Production Deployments – Social Gaming iki 42
    43. 43. Paid Production Deployments – Key SegmentsAd PlatformsSocial Networks 43
    44. 44. Production Deployments – Key Industry SegmentsOnline Biz Services E-CommerceOnline Media 44
    45. 45. Production Deployments – Key Industry Segments HealthCare Military/ GovernmentCommunications 45
    46. 46. Production Deployments – Key Industry Segments Online Education Web Design Financial Services 46
    47. 47. Production Deployments – Key Industry Segments Software Security 47
    48. 48. Production Deployments – Enterprises 48

    ×