CouchConf Tokyo 2013_Couchbase Server Tour and Demo
 

Like this? Share it with your network

Share

CouchConf Tokyo 2013_Couchbase Server Tour and Demo

on

  • 862 views

 

Statistics

Views

Total Views
862
Views on SlideShare
383
Embed Views
479

Actions

Likes
1
Downloads
10
Comments
0

3 Embeds 479

http://www.couchbase.com 463
http://beta.stage.couchbase.com 15
http://site-qa.cbauthx.com 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • -we’ll be showcasing 8 apps running Couchbase Server (plus you can score more drink tickets by playing the games)-we’re also having an after-after party. Find me towards the end of the party here and we’ll head over to Harrington’s bar
  • Partial listing of companies with paid production deployments – Add roket play!Thousands more using open source
  • Hello everyone, I am diptiborkar, director of product management at couchbase. I manage the product roadmap of couchbase server, our flagship product. And I’m really excited to talk to you about couchbase server 2.0
  • Let me start with the document data model, which is really one of the things that is driving this trend,Most of you are probably familiar with the table layout. A table is defined with a set of column. And each record in the table conforms to the schema. If you wish to capture different data in the future, the table schema must be changed using the ALTER TABLE statement. Typically data is normalized in the 3rd normal form reduce duplication. Large tables are split into smaller tablesusing foreign keys
  • Example. Normalized schema 2 tables Foreign keys (links) connects the two. To get information about a specific error, you will perform and JOIN across the two tables
  • Single doc contains aggregated info that would normallly be distributed across tables. Of course in real use cases it tends to be info spread out over tens, hundresds or even thousands of tables in real world complex systems (like SAP)Example. Normalized schema 2 tables Fk connects the two. To get information about a specific error, you will perform and join across the two tables
  • Rapid app devwithouth the need to perform an expensive alter table operation.
  • All nodes are equal, single node type, easy to scale your cluster. No single point of failoverEvery node manages some active data and some replica data. Data is distributed across the clsuter and hence the load is also uniformly distributed using auto sharding. We have a fixed number of shards that a key get hashed to. 1024 shards, distributed across the cluster. Replication within the cluster for high availability. Number of replicas are configurable with upto 3 replicas. With auto-failiover or manual failover, replica information is immediately promoted to active Add multiple nodes at a time to grow and shrink your cluster.
  • JSON support – natively stored as json, whne you build an app, there is not conversion required. New doc viewing , editing capability. Indexing and querying – look inside your json, build views and query for a key, for ranges or to aggregate data Incremental mapreduce – powers indexing. Build complex views over your data. Great for real-time analytics XDCR – replicate information from one cluster to another cluster
  • All nodes are equal, single node type, easy to scale your cluster. No single point of failoverEvery node manages some active data and some replica data. Data is distributed across the clsuter and hence the load is also uniformly distributed using auto sharding. We have a fixed number of shards that a key get hashed to. 1024 shards, distributed across the cluster. Replication within the cluster for high availability. Number of replicas are configurable with upto 3 replicas. With auto-failiover or manual failover, replica information is immediately promoted to active Add multiple nodes at a time to grow and shrink your cluster.
  • CAPI interface – basic Couch API of which some goes through the caching layer (CRUD), some goes directly to Couch (Views)
  • CAPI interface – basic Couch API of which some goes through the caching layer (CRUD), some goes directly to Couch (Views)
  • 1.  A set request comes in from the application .2.  Couchbase Server responses back that they key is written3. Couchbase Server then Replicates the data out to memory in the other nodes4. At the same time it is put the data into a write que to be persisted to disk
  • 1.  A set request comes in from the application .2.  Couchbase Server responses back that they key is written3. Couchbase Server then Replicates the data out to memory in the other nodes4. At the same time it is put the data into a write que to be persisted to disk
  • 1.  A set request comes in from the application .2.  Couchbase Server responses back that they key is written3. Couchbase Server then Replicates the data out to memory in the other nodes4. At the same time it is put the data into a write que to be persisted to disk
  • 1.  A set request comes in from the application .2.  Couchbase Server responses back that they key is written3. Couchbase Server then Replicates the data out to memory in the other nodes4. At the same time it is put the data into a write que to be persisted to disk
  • 1.  A set request comes in from the application .2.  Couchbase Server responses back that they key is written3. Couchbase Server then Replicates the data out to memory in the other nodes4. At the same time it is put the data into a write que to be persisted to disk
  • Bulletize the text. Make sure the builds work.
  • Bulletize the text. Make sure build work properly.
  • Bulletize the text. Make sure build work properly.
  • 1.  A set request comes in from the application .2.  Couchbase Server responses back that they key is written3. Couchbase Server then Replicates the data out to memory in the other nodes4. At the same time it is put the data into a write que to be persisted to disk
  • Bulletize the text. Make sure the builds work.
  • Overview of what this feature is
  • 1.  A set request comes in from the application .2.  Couchbase Server responses back that they key is written3. Couchbase Server then Replicates the data out to memory in the other nodes4. At the same time it is put the data into a write que to be persisted to disk
  • Break out to demo here
  • ElasticSearch cluster is fed the documents from the Couchbase Server clusterElastic search indexes the fields(configurable which ones) and by default will only store references back to the document idThe application does document access via the Couchbase Server Cluster and uses The Views and incremental map reduce on the Couchbase cluster.For full text queries it queries the Ealstic search cluster directly (simple Http and JSON interface)The full text queries typically returns the ids of the matching documents.Documents are then retrieved from the Couchbase Server cluster.This way the high throughput document access always comes from high performance Couchbase Cluster.

CouchConf Tokyo 2013_Couchbase Server Tour and Demo Presentation Transcript

  • 1. WelcomeFebruary 22, 2013 1
  • 2. When you tweet… #CouchConf 2
  • 3. Thanks to CouchConf Tokyo Translation Sponsor 3
  • 4. Thanks to CouchConf Tokyo Core Sponsor 4
  • 5. Thanks to CouchConf Tokyo Media Sponsors 5
  • 6. Stay for the Q&A Session! Beer. Wine. Snacks. Don’t miss it. 6
  • 7. Submit questions for the Q&A session...Email: couchconftokyo@couchbase.comTweet: @couchbase...or write it down and hand it to any of theCouchbase staff! 7
  • 8. WelcomeFebruary 22, 2013 8
  • 9. Couchbase NoSQL Leadership Leading NoSQL database company Open Source development & business model Behind Couchbase open source project Document-oriented NoSQL database Focused on interactive internet and mobile applications Provide more flexible, higher performance, more scalable database than relational alternative Most mature, reliable and widely deployed solution >5,000 paid production deployments worldwide, over 350 customers Headquarters in Silicon Valley (Mountain View, CA) ~100 employees including >60 in engineering/product >80% of commits to Couchbase, memcached, Apache CouchDB 9
  • 10. Market Adoption – Customers Internet Companies Enterprises More than 350 customers -- 5,000 production deployments worldwide 10
  • 11. Introduction toCouchbase Server 2.0 Sharon Barr VP of Engineering 11
  • 12. Couchbase Server NoSQL Document Database 2.0 12
  • 13. Couchbase Server 2.0 Easy Consistent High Scalability Performance Grow cluster without Consistent sub-millisecond application changes, without read and write response times downtime with a single click with consistent high throughput Always Flexible Data On JSON JSON JSO JSON JSON N Model 24x365 No downtime for software JSON document model with upgrades, hardware no fixed schema. maintenance, etc. 13
  • 14. Flexible Data Model: Relational vs Document Data Model C1 C2 C3 C4 { JSON JSON } JSON Relational data model Document data model Highly-structured table organization Collection of complex documents with with rigidly-defined data formats and arbitrary, nested data formats and record structure. varying “record” format. 14
  • 15. RDBMS Example: User Profile User Info Address Info KEY First Last ZIP_id ZIP_id CITY STATE ZIP 1 Frank Weigel 2 1 DEN CO 30303 2 Ali Dodson 2 2 MV CA 94040 3 Mark Azad 2 3 CHI IL 60609 4 Steve Yen 3 4 NY NY 10010 To get info about specific user, you perform a join across two tables 15
  • 16. Document Example: User Profile { “ID”: 1, “FIRST”: “Frank”, “LAST”: “Weigel”, “ZIP”: “94040”, “CITY”: “MV”, = + “STATE”: “CA” } JSON All data in a single document 16
  • 17. Flexible Data Model { “ID”: 1, “FIRST”: “Dipti”, “LAST”: “Borkar”, “ZIP”: “94040”, “CITY”: “MV”, “STATE”: “CA” } JSON JSON JSON JSON • No need to worry about the database when changing your application • Records can have different structures, there is no fixed schema • Allows painless data model changes for rapid application development 17
  • 18. Couchbase Server Features Built-in clustering – All nodes equal Data replication with auto-failover Zero--downtime maintenance Clone to grow and scale horizontally Built-in managed cached Monitoring and administration APIs and GUI SDK for a variety of languages 18
  • 19. New in 2.0 JSON support Indexing and Querying JSON JSON JSO JSON N JSON Incremental Map Reduce Cross data center replication 19
  • 20. Additional Couchbase Server 2.0 Features Append-only storage layer Online compaction Better working set management Reduce server warm-up time 20
  • 21. Couchbase Server 2.0 Architecture 8092 11211 11210 Couch View Memcapable 1.0 Memcapable 2.0 Moxi REST management API/Web UI vBucket state and replication manager Memcached Interface Couch API Global singleton supervisor Rebalance orchestrator Configuration manager Node health monitor Process monitor Heartbeat Couchbase EP Engine Hash table cache Write/replica Data Manager Queues Cluster Manager storage interface http on each node one per cluster Distributed CouchStore Indexing Auto compaction Erlang/OTP HTTP Erlang port mapper Distributed Erlang 8091 4369 21100 - 21199 21
  • 22. Couchbase deployment Web Application Couchbase Client Library Data ports Data Flow Cluster Management (8091) 23
  • 23. COUCHBASE OPERATIONS 24
  • 24. Single node - Couchbase Write Operation 2 Doc 1 App Server 3 2 3 Managed Cache To other node Replication Doc 1 Queue Disk Queue Disk Couchbase Server Node 25
  • 25. Couchbase single node operations• Write• Update• Evict• Read• Cache miss 26
  • 26. Partitioning The Data – vbucket (internal partitions) map 31
  • 27. Cluster wide - Basic Operation APP SERVER 1 APP SERVER 2 COUCHBASE Client Library COUCHBASE Client Library CLUSTER MAP CLUSTER MAP READ/WRITE/UPDATE SERVER 1 SERVER 2 SERVER 3 • Docs distributed evenly across ACTIVE ACTIVE ACTIVE servers Doc 5 Doc Doc 4 Doc Doc 1 Doc • Each server stores both active and replica docs Doc 2 Doc Doc 7 Doc Doc 2 Doc Only one server active at a time • Client library provides app with Doc 9 Doc Doc 8 Doc Doc 6 Doc simple interface to database REPLICA REPLICA REPLICA • Cluster map provides map to which server doc is on Doc 4 Doc Doc 6 Doc Doc 7 Doc App never needs to know Doc 1 Doc Doc 3 Doc Doc 9 Doc • App reads, writes, updates docs Doc 8 Doc Doc 2 Doc Doc 5 Doc • Multiple app servers can access same document at same time COUCHBASE SERVER CLUSTERUser Configured Replica Count = 1 32
  • 28. Cluster wide - Add Nodes to Cluster APP SERVER 1 APP SERVER 2 COUCHBASE Client Library COUCHBASE Client Library CLUSTER MAP CLUSTER MAP READ/WRITE/UPDATE READ/WRITE/UPDATE SERVER 1 SERVER 2 SERVER 3 SERVER 4 SERVER 5 • Two servers added ACTIVE ACTIVE ACTIVE ACTIVE ACTIVE One-click operation Doc 5 Doc Doc 4 Doc Doc 1 Doc • Docs automatically rebalanced across Doc 2 Doc Doc 7 Doc Doc 2 Doc cluster Even distribution of docs Minimum doc movement Doc 9 Doc Doc 8 Doc Doc 6 Doc • Cluster map updated REPLICA REPLICA REPLICA REPLICA REPLICA • App database Doc 4 Doc Doc 6 Doc Doc 7 Doc calls now distributed over larger number of Doc 1 Doc Doc 3 Doc Doc 9 Doc servers Doc 8 Doc Doc 2 Doc Doc 5 Doc COUCHBASE SERVER CLUSTERUser Configured Replica Count = 1 33
  • 29. Cluster wide - Fail Over Node APP SERVER 1 APP SERVER 2 COUCHBASE Client Library COUCHBASE Client Library CLUSTER MAP CLUSTER MAP SERVER 1 SERVER 2 SERVER 3 SERVER 4 SERVER 5 • App servers accessing docs ACTIVE ACTIVE ACTIVE ACTIVE ACTIVE • Requests to Server 3 fail Doc 5 Doc Doc 4 Doc Doc 1 Doc Doc 9 Doc Doc 6 Doc • Cluster detects server failed Promotes replicas of docs to Doc 2 Doc Doc 7 Doc Doc 2 Doc Doc 8 Doc Doc active Updates cluster map Doc 1 Doc 3 • Requests for docs now go to REPLICA REPLICA REPLICA REPLICA REPLICA appropriate server Doc 4 Doc Doc 6 Doc Doc 7 Doc Doc 5 Doc Doc 8 Doc • Typically rebalance would follow Doc 1 Doc Doc 3 Doc Doc 9 Doc Doc 2 Doc COUCHBASE SERVER CLUSTERUser Configured Replica Count = 1 34
  • 30. DEMO TIME 35
  • 31. Indexing and Querying – The basics • Define materialized views on JSON documents and then query across the data set • Using views you can define • Primary indexes • Simple secondary indexes (most common use case) • Complex secondary, tertiary and composite indexes • Aggregations (reduction) • Indexes are eventually indexed • Queries are eventually consistent with respect to documents • Built using Map/Reduce technology • Map and Reduce functions are written in Javascript 36
  • 32. Eventually indexed Views – Data flow 2 Doc 1 App Server Couchbase Server Node 3 2 3 Managed Cache To other node Replication Queue Doc 1 Disk Queue Disk Doc 1 View engine 37
  • 33. Cluster wide - Indexing and Querying APP SERVER 1 APP SERVER 2 COUCHBASE Client Library COUCHBASE Client Library CLUSTER MAP CLUSTER MAP Query SERVER 1 SERVER 2 SERVER 3 • Indexing work is distributed ACTIVE ACTIVE ACTIVE amongst nodes Doc 5 Doc Doc 5 Doc Doc 5 Doc • Large data set possible Doc 2 Doc Doc 2 Doc Doc 2 Doc • Parallelize the effort Doc 9 Doc • Each node has index for data stored Doc 9 Doc Doc 9 Doc on it REPLICA REPLICA REPLICA • Queries combine the results from Doc 4 Doc required nodes Doc 4 Doc Doc 4 Doc Doc 1 Doc Doc 1 Doc Doc 1 Doc Doc 8 Doc Doc 8 Doc Doc 8 Doc COUCHBASE SERVER CLUSTERUser Configured Replica Count = 1 38
  • 34. Cross Data Center Replication – The basics• Replicate your Couchbase data across clusters• Clusters may be spread across geos• Configured on a per-bucket basis• Supports unidirectional and bidirectional operation• Application can read and write from both clusters (active – active replication)• Replication throughput scales out linearly• Different from intra-cluster replication 39
  • 35. Cross data center replication – Data flow 2 Doc 1 App Server Couchbase Server Node 3 2 3 Managed Cache To other node Replication Doc 1 Queue Disk Queue Disk Doc 1 XDCR Queue To other cluster 40
  • 36. Cluster wide - XDCR SERVER 1 SERVER 2 SERVER 3 ACTIVE ACTIVE ACTIVE COUCHBASE SERVER CLUSTER Doc Doc Doc NEW YORK DATA CENTER Doc 2 Doc Doc Doc 9 Doc DocRAM RAM RAM Doc Doc Doc Doc Doc Doc Doc Doc Doc DISK DISK DISK SERVER 1 SERVER 2 SERVER 3 ACTIVE ACTIVE ACTIVE Doc Doc Doc Doc 2 Doc Doc Doc 9 Doc Doc RAM RAM RAM COUCHBASE SERVER CLUSTER Doc Doc Doc Doc Doc Doc Doc Doc Doc SAN FRANCISCO DATA CENTER DISK DISK DISK 41
  • 37. DEMO TIME 42
  • 38. Demo: The next big social game 3 Objects (documents) within game: • Players • Monsters • Items Gameplay: • Players fight monsters • Monsters drop items • Players own items 43
  • 39. Player Document { "jsonType": "player", "uuid": "35767d02-a958-4b83-8179-616816692de1", "name": "Keith4540", "hitpoints": 75, Player ID "experience": 663, "level": 4, "loggedIn": false } 44
  • 40. Item Document { Item ID "jsonType": "item", "name": "Katana_e5890c94-11c6-65746ce6c560", "uuid": "e5890c94-11c6-4856-a7a6-65746ce6c560", "ownerId": "Dale9887" } Player ID 45
  • 41. Monster Document { Monster ID "jsonType": "monster", "name": "Bauchan9932", "uuid": "d10dfc1b-0412-4140-b4ec-affdbf2aa5ec", "hitpoints": 370, "experienceWhenKilled": 52, "itemProbability": 0.5050581341872865 } 46
  • 42. GAME ON! 47
  • 43. Full Text Search Integration• Elastic Search is good for ad-hoc queries and faceted browsing• Couchbase adapter uses XDCR to push mutations to ESDocs are indexed by Elastic Search• Couchbase ES Adapter is cluster-aware ElasticSearch Unidirectional Cross Data Center Replication 48
  • 44. Full Text Search Application Server Couchbase SDK ES Queries over HTTP Do ta TS c Da ery Qu Re Qu er fs M R y Couchbase Server Cluster ElasticSearch Server Cluster MR MR MR MR Views Views Views Views XDCR-based Cross Data Center Replication CB-ES Transport 49
  • 45. Couchbase SDKsJava SDK User Code.Net SDK Java client API CouchbaseClient cb = new CouchbaseClient(listURIs, "aBucket", "letmein"); cb.set("hello", 0, "world"); cb.get("hello"); Couchbase Java LibraryPHP SDK (spymemcached)Ruby SDK Couchbase Server…and manymore http://www.couchbase.com/develop 50
  • 46. QUESTIONS? 51 5
  • 47. THANK YOU COUCHBASE SIMPLE, FAST, ELASTIC NOSQLsharon@couchbase.com@sharonyb 52