Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Introduction to Couchbase Server 2.0

4,286 views

Published on

Silicon Valley NoSQL Meetup - Nov 2012. View with animations: video version here: https://vimeo.com/54691785

http://www.meetup.com/Silicon-Valley-NoSQL/events/88257222/

For more information visit: www.couchbase.com

Published in: Technology
  • Be the first to comment

Introduction to Couchbase Server 2.0

  1. 1. Introduction toCouchbase Server 2.0 Dipti Borkar Director, Product Management 1
  2. 2. Couchbase Server NoSQL Document Database 2.0 2
  3. 3. Couchbase Server Easy Consistent High Scalability Performance Grow cluster without Consistent sub-millisecond application changes, without read and write response times downtime with a single click with consistent high throughput Always Flexible Data On JSON JSON JSO JSON JSON N Model 24x365 No downtime for software JSON document model with upgrades, hardware no fixed schema. maintenance, etc. 3
  4. 4. Flexible Data Model { “ID”: 1, “FIRST”: “Dipti”, “LAST”: “Borkar”, “ZIP”: “94040”, “CITY”: “MV”, “STATE”: “CA” } JSON JSON JSON JSON • No need to worry about the database when changing your application • Records can have different structures, there is no fixed schema • Allows painless data model changes for rapid application development 4
  5. 5. New in 2.0 JSON support Indexing and Querying JSON JSON JSO JSON N JSON Incremental Map Reduce Cross data center replication 5
  6. 6. Couchbase Server 2.0 Architecture 8092 11211 11210 Query API Memcapable 1.0 Memcapable 2.0 Moxi Query Engine REST management API/Web UI vBucket state and replication manager Memcached Global singleton supervisor Rebalance orchestrator Configuration manager Node health monitor Process monitor Heartbeat Couchbase EP Engine Data Manager Cluster Manager storage interface New Persistence Layer http on each node one per cluster Erlang/OTP HTTP Erlang port mapper Distributed Erlang 8091 4369 21100 - 21199 6
  7. 7. Couchbase Server 2.0 Architecture 8092 11211 11210 Query API Memcapable 1.0 Memcapable 2.0 Moxi Query Engine REST management API/Web UI vBucket state and replication manager Object-level Cache Global singleton supervisor Rebalance orchestrator Configuration manager Node health monitor Process monitor Heartbeat Couchbase EP Engine storage interface New Persistence Layer http on each node one per cluster Erlang/OTP HTTP Erlang port mapper Distributed Erlang 8091 4369 21100 - 21199 7
  8. 8. Couchbase deployment Web Application Couchbase Client Library Data Flow Cluster Management 8
  9. 9. COUCHBASE OPERATIONS 9
  10. 10. Single node - Couchbase Write Operation 2 Doc 1 App Server 3 2 3 Managed Cache To other node Replication Doc 1 Queue Disk Queue Disk Couchbase Server Node 10
  11. 11. Single node - Couchbase Update Operation 2 Doc 1’ App Server 3 2 3 Managed Cache To other node Replication 1 Doc 1’ Queue Disk Queue Disk Doc 1 Couchbase Server Node 11
  12. 12. Single node - Couchbase Read Operation 2 Doc 1 GET App Server 3 2 3 Managed Cache To other node Replication Queue Doc 1 Disk Queue Disk Doc 1 Couchbase Server Node 12
  13. 13. Single node - Couchbase Cache Eviction 2 Doc 6 2 3 4 5 App Server 3 2 3 Managed Cache To other node Replication Doc 1 Queue Disk Queue Disk Doc 1 Doc 6 Doc 5 Doc 4 Doc 3 Doc 2 Couchbase Server Node 13
  14. 14. Single node – Couchbase Cache Miss 2 Doc 1 GET App Server 3 2 3 Managed Cache To other node Replication Queue Doc 1 Doc 5 4 4 Doc Doc Doc 3 2 Doc Disk Queue Disk Doc 1 Doc 6 Doc 5 Doc 4 Doc 3 Doc 2 Couchbase Server Node 14
  15. 15. Cluster wide - Basic Operation APP SERVER 1 APP SERVER 2 COUCHBASE Client Library COUCHBASE Client Library CLUSTER MAP CLUSTER MAP READ/WRITE/UPDATE SERVER 1 SERVER 2 SERVER 3 • Docs distributed evenly across ACTIVE ACTIVE ACTIVE servers Doc 5 Doc Doc 4 Doc Doc 1 Doc • Each server stores both active and replica docs Doc 2 Doc Doc 7 Doc Doc 2 Doc Only one server active at a time • Client library provides app with Doc 9 Doc Doc 8 Doc Doc 6 Doc simple interface to database REPLICA REPLICA REPLICA • Cluster map provides map to which server doc is on Doc 4 Doc Doc 6 Doc Doc 7 Doc App never needs to know Doc 1 Doc Doc 3 Doc Doc 9 Doc • App reads, writes, updates docs Doc 8 Doc Doc 2 Doc Doc 5 Doc • Multiple app servers can access same document at same time COUCHBASE SERVER CLUSTERUser Configured Replica Count = 1 15
  16. 16. Cluster wide - Add Nodes to Cluster APP SERVER 1 APP SERVER 2 COUCHBASE Client Library COUCHBASE Client Library CLUSTER MAP CLUSTER MAP READ/WRITE/UPDATE READ/WRITE/UPDATE SERVER 1 SERVER 2 SERVER 3 SERVER 4 SERVER 5 • Two servers added ACTIVE ACTIVE ACTIVE ACTIVE ACTIVE One-click operation Doc 5 Doc Doc 4 Doc Doc 1 Doc • Docs automatically rebalanced across Doc 2 Doc Doc 7 Doc Doc 2 Doc cluster Even distribution of docs Minimum doc movement Doc 9 Doc Doc 8 Doc Doc 6 Doc • Cluster map updated REPLICA REPLICA REPLICA REPLICA REPLICA • App database Doc 4 Doc Doc 6 Doc Doc 7 Doc calls now distributed over larger number of Doc 1 Doc Doc 3 Doc Doc 9 Doc servers Doc 8 Doc Doc 2 Doc Doc 5 Doc COUCHBASE SERVER CLUSTERUser Configured Replica Count = 1 16
  17. 17. Cluster wide - Fail Over Node APP SERVER 1 APP SERVER 2 COUCHBASE Client Library COUCHBASE Client Library CLUSTER MAP CLUSTER MAP SERVER 1 SERVER 2 SERVER 3 SERVER 4 SERVER 5 • App servers accessing docs ACTIVE ACTIVE ACTIVE ACTIVE ACTIVE • Requests to Server 3 fail Doc 5 Doc Doc 4 Doc Doc 1 Doc Doc 9 Doc Doc 6 Doc • Cluster detects server failed Promotes replicas of docs to Doc 2 Doc Doc 7 Doc Doc 2 Doc Doc 8 Doc Doc active Updates cluster map Doc 1 Doc 3 • Requests for docs now go to REPLICA REPLICA REPLICA REPLICA REPLICA appropriate server Doc 4 Doc Doc 6 Doc Doc 7 Doc Doc 5 Doc Doc 8 Doc • Typically rebalance would follow Doc 1 Doc Doc 3 Doc Doc 9 Doc Doc 2 Doc COUCHBASE SERVER CLUSTERUser Configured Replica Count = 1 17
  18. 18. DEMO TIME 18
  19. 19. Indexing and Querying – The basics • Define materialized views on JSON documents and then query across the data set • Using views you can define • Primary indexes • Simple secondary indexes (most common use case) • Complex secondary, tertiary and composite indexes • Aggregations (reduction) • Indexes are eventually indexed • Queries are eventually consistent with respect to documents • Built using Map/Reduce technology • Map and Reduce functions are written in Javascript 19
  20. 20. Eventually indexed Views – Data flow 2 Doc 1 App Server Couchbase Server Node 3 2 3 Managed Cache To other node Replication Queue Doc 1 Disk Queue Disk Doc 1 View engine 20
  21. 21. Cluster wide - Indexing and Querying APP SERVER 1 APP SERVER 2 COUCHBASE Client Library COUCHBASE Client Library CLUSTER MAP CLUSTER MAP Query SERVER 1 SERVER 2 SERVER 3 • Indexing work is distributed ACTIVE ACTIVE ACTIVE amongst nodes Doc 5 Doc Doc 5 Doc Doc 5 Doc • Large data set possible Doc 2 Doc Doc 2 Doc Doc 2 Doc • Parallelize the effort Doc 9 Doc • Each node has index for data stored Doc 9 Doc Doc 9 Doc on it REPLICA REPLICA REPLICA • Queries combine the results from Doc 4 Doc required nodes Doc 4 Doc Doc 4 Doc Doc 1 Doc Doc 1 Doc Doc 1 Doc Doc 8 Doc Doc 8 Doc Doc 8 Doc COUCHBASE SERVER CLUSTERUser Configured Replica Count = 1 21
  22. 22. Cross Data Center Replication – The basics• Replicate your Couchbase data across clusters• Clusters may be spread across geos• Configured on a per-bucket basis• Supports unidirectional and bidirectional operation• Application can read and write from both clusters (active – active replication)• Replication throughput scales out linearly• Different from intra-cluster replication 22
  23. 23. Cross data center replication – Data flow 2 Doc 1 App Server Couchbase Server Node 3 2 3 Managed Cache To other node Replication Doc 1 Queue Disk Queue Disk Doc 1 XDCR Queue To other cluster 23
  24. 24. Cluster wide - XDCR SERVER 1 SERVER 2 SERVER 3 ACTIVE ACTIVE ACTIVE COUCHBASE SERVER CLUSTER Doc Doc Doc NY DATA CENTER Doc 2 Doc Doc Doc 9 Doc DocRAM RAM RAM Doc Doc Doc Doc Doc Doc Doc Doc Doc DISK DISK DISK SERVER 1 SERVER 2 SERVER 3 ACTIVE ACTIVE ACTIVE Doc Doc Doc Doc 2 Doc Doc Doc 9 Doc Doc RAM RAM RAM COUCHBASE SERVER CLUSTER Doc Doc Doc Doc Doc Doc Doc Doc Doc SF DATA CENTER DISK DISK DISK 24
  25. 25. DEMO TIME 25
  26. 26. Demo: The next big social game 3 Objects (documents) within game: • Players • Monsters • Items Gameplay: • Players fight monsters • Monsters drop items • Players own items 26
  27. 27. Player Document { "jsonType": "player", "uuid": "35767d02-a958-4b83-8179-616816692de1", "name": "Keith4540", "hitpoints": 75, Player ID "experience": 663, "level": 4, "loggedIn": false } 27
  28. 28. Item Document { Item ID "jsonType": "item", "name": "Katana_e5890c94-11c6-65746ce6c560", "uuid": "e5890c94-11c6-4856-a7a6-65746ce6c560", "ownerId": "Dale9887" } Player ID 28
  29. 29. Monster Document { "jsonType": "monster", Monster ID "name": "Bauchan9932", "uuid": "d10dfc1b-0412-4140-b4ec-affdbf2aa5ec", "hitpoints": 370, "experienceWhenKilled": 52, "itemProbability": 0.5050581341872865 } 29
  30. 30. GAME ON! 30
  31. 31. Full Text Search Integration• Elastic Search is good for ad-hoc queries and faceted browsing• Couchbase adapter uses XDCR to push mutations to ESDocs are indexed by Elastic Search• Couchbase ES Adapter is cluster-aware ElasticSearch Unidirectional Cross Data Center Replication 31
  32. 32. Full Text Search Application Server Couchbase SDK ES Queries over HTTP Do ta TS c Da ery Qu Re Qu er fs M R y Couchbase Server Cluster ElasticSearch Server Cluster MR MR MR MR Views Views Views Views XDCR-based Cross Data Center Replication CB-ES Transport 32
  33. 33. Couchbase SDKsJava SDK User Code.Net SDK Java client API CouchbaseClient cb = new CouchbaseClient(listURIs, "aBucket", "letmein"); cb.set("hello", 0, "world"); cb.get("hello"); Couchbase Java LibraryPHP SDK (spymemcached)Ruby SDK Couchbase Server…and manymore http://www.couchbase.com/develop 33
  34. 34. QUESTIONS? 34 3
  35. 35. THANK YOUDIPTI@COUCBASE.COM @DBORKAR 35 3

×