Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Couchbase_John_Bryce_Israel_Training_couchbase_overview

1,013 views

Published on

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Couchbase_John_Bryce_Israel_Training_couchbase_overview

  1. 1. Introduction toCouchbase ServerPerry KrugSr. Solutions Architect
  2. 2. Couchbase Server 2.0 is a high performance, easyto scale and flexible Document “NoSQL” Database.
  3. 3. EasyScalabilityConsistent HighPerformanceAlwaysOn24x365Grow cluster withoutapplication changes, withoutdowntime with a single clickConsistent sub-millisecondread and write response timeswith consistent high throughputNo downtime for softwareupgrades, hardwaremaintenance, etc.Couchbase ServerJSONJSONJSONJSONJSONFlexible DataModelJSON document model withno fixed schema.The NoSQL Promise
  4. 4. Couchbase Feature Set• Flexible Data Model:- JSON Support- Indexing/Querying- Incremental Map-Reduce• Easy Scalability:- “Clone to grow” with auto-sharding- Cross-data center replication• Consistent High Performance:- Built-in Object level cache• Always on 24x365- Zero-downtime maintenance- Built-in data replication with auto-failover- Management and Monitoring UI- Reliable persistence architecture
  5. 5. Couchbase Server ArchitectureReplication, Rebalance,Shard State ManagerREST managementAPI/Web UI8091Admin ConsoleErlang/OTP11210 / 11211Data access portsObject-managedCacheStorage Engine8092Query APIQueryEnginehttpData Manager Cluster Manager
  6. 6. Couchbase Operations
  7. 7. WebApplicationClient InteractionData FlowCluster ManagementWebApplicationCouchbaseClient LibraryWebApplication … …Couchbase Server Couchbase Server Couchbase Server Couchbase ServerReplication Flow
  8. 8. 33 2Write (‘set’) Operation2Managed CacheDiskQueueDiskReplicationQueueApp ServerCouchbase Server NodeDoc 1Doc 1Doc 1To other node
  9. 9. 33 2View processing and XDCR2Managed CacheDiskQueueDiskReplicationQueueApp ServerCouchbase Server NodeDoc 1Doc 1Doc 1To other nodeXDCR QueueDoc 1To other clusterView engineDoc 1
  10. 10. Disk Compaction• Disk writes to data files and index are ‘append-only’• On-disk size increases compared to actual stored data• Compaction defragments data and index information• Operates on a live bucket (no downtime)• Both automatic and manual compaction available• Compaction operates per-shard on each node
  11. 11. CompactionInitial file layout:Update some data:After compaction:Doc A Doc B Doc CDoc C Doc B’ Doc A’’Doc A Doc B Doc A’ Doc B’ Doc A’’Doc A Doc B Doc C Doc A’ Doc DDoc D
  12. 12. GETDoc133 2Read (‘get’) Operation2DiskQueueReplicationQueueApp ServerCouchbase Server NodeDoc 1Doc 1Doc 1Managed CacheDiskTo other node
  13. 13. 33 2Cache Ejection2DiskQueueReplicationQueueApp ServerCouchbase Server NodeDoc 1Doc 6Doc 5Doc 4Doc 3Doc 2Doc 1Doc 6 Doc 5 Doc 4 Doc 3 Doc 2Managed CacheDiskTo other node
  14. 14. 33 2Cache Miss2DiskQueueReplicationQueueApp ServerCouchbase Server NodeDoc 1Doc 3Doc 5 Doc 2Doc 4Doc 6 Doc 5 Doc 4 Doc 3 Doc 2Doc 4GETDoc1Doc 1Doc 1Managed CacheDiskTo other node
  15. 15. COUCHBASE SERVER CLUSTERCluster wide - Basic Operation• Docs distributed evenly acrossservers• Each server stores both active andreplica docsOnly one server active at a time• Client library provides app withsimple interface to database• Cluster map provides mapto which server doc is onApp never needs to know• App reads, writes, updates docs• Multiple app servers can access samedocument at same timeUser Configured Replica Count = 1READ/WRITE/UPDATEACTIVEVB 1VB 7VB 4VB 8VB 14SERVER 1ACTIVEVB 2VB 9VB 5VB 10VB 16SERVER 2VB 15ACTIVEVB 3VB 11VB 6VB 12VB 18REPLICAVB 2VB 9VB 15VB 3VB 11VB 17REPLICAVB 4VB 8VB 13VB 6VB 12VB 18REPLICAVB 5VB 10VB 14VB 7VB 1VB 16SERVER 3VB 17APP SERVER 1COUCHBASE Client LibraryCLUSTER MAPCOUCHBASE Client LibraryCLUSTER MAPAPP SERVER 2VB 13
  16. 16. Cluster wide - Add Nodes to Cluster• Two servers addedOne-click operation• Docs automaticallyrebalanced acrossclusterEven distribution of docsMinimum doc movement• Cluster map updated• App databasecalls now distributedover larger number ofserversREPLICAACTIVEDoc 5Doc 2DocDocDoc 4Doc 1DocDocSERVER 1REPLICAACTIVEDoc 4Doc 7DocDocDoc 6Doc 3DocDocSERVER 2REPLICAACTIVEDoc 1Doc 2DocDocDoc 7Doc 9DocDocSERVER 3 SERVER 4 SERVER 5REPLICAACTIVEREPLICAACTIVEDocDoc 8 DocDoc 9 DocDoc 2 DocDoc 8 DocDoc 5 DocDoc 6READ/WRITE/UPDATE READ/WRITE/UPDATEAPP SERVER 1COUCHBASE Client LibraryCLUSTER MAPCOUCHBASE Client LibraryCLUSTER MAPAPP SERVER 2COUCHBASE SERVER CLUSTERUser Configured Replica Count = 1
  17. 17. Cluster wide - Fail Over NodeREPLICAACTIVEDoc 5Doc 2DocDocDoc 4Doc 1DocDocSERVER 1REPLICAACTIVEDoc 4Doc 7DocDocDoc 6Doc 3DocDocSERVER 2REPLICAACTIVEDoc 1Doc 2DocDocDoc 7Doc 9DocDocSERVER 3 SERVER 4 SERVER 5REPLICAACTIVEREPLICAACTIVEDoc 9Doc 8Doc Doc 6 DocDocDoc 5 DocDoc 2Doc 8 DocDoc• App servers accessing docs• Requests to Server 3 fail• Cluster detects server failedPromotes replicas of docs toactiveUpdates cluster map• Requests for docs now go toappropriate server• Typically rebalancewould followDocDoc 1 Doc 3APP SERVER 1COUCHBASE Client LibraryCLUSTER MAPCOUCHBASE Client LibraryCLUSTER MAPAPP SERVER 2User Configured Replica Count = 1COUCHBASE SERVER CLUSTER
  18. 18. COUCHBASE SERVER CLUSTERIndexing and QueryingUser Configured Replica Count = 1ACTIVEDoc 5Doc 2DocDocDocSERVER 1REPLICADoc 4Doc 1Doc 8DocDocDocAPP SERVER 1COUCHBASE Client LibraryCLUSTER MAPCOUCHBASE Client LibraryCLUSTER MAPAPP SERVER 2Doc 9• Indexing work is distributedamongst nodes• Large data set possible• Parallelize the effort• Each node has index for data storedon it• Queries combine the results fromrequired nodesACTIVEDoc 5Doc 2DocDocDocSERVER 2REPLICADoc 4Doc 1Doc 8DocDocDocDoc 9ACTIVEDoc 5Doc 2DocDocDocSERVER 3REPLICADoc 4Doc 1Doc 8DocDocDocDoc 9Query
  19. 19. • Application can access both clusters (active – active replication)• Scales out linearly• Different from intra-cluster replication (“CP” versus “AP”)XDCR: Cross Data Center Replication
  20. 20. Full Text Search
  21. 21. Documents
  22. 22. •get (key)– Retrieve a document•set (key, value)– Store a document, overwrites if exists•add (key, value)– Store a document, error/exception if exists•replace (key, value)– Store a document, error/exception if doesn’t exist•cas (key, value, cas)– Compare and swap, mutate document only if it hasn’t changedwhile executing this operationStore & Retrieve Operations
  23. 23. Check and Set/Compare and Swap (CAS)• Compares supplied CAS to validate achange to a value:- Client gets key and checksum(cas_token)- Client updates using key and checksum- If checksum doesn’t match, update fails• Client can only update if the key + CASmatch• Used when multiple clients accesssame data• First client with correct CAS wins• Subsequent client updates receiveCAS mismatchActor 1 Actor 2Couchbase ServerCAS mismatchSuccess
  24. 24. Document Driven• Use JSON to store documents- Replace serialized objects- Custom structures• Documents define a "record" of data• Store/Update/Retrieve using same protocol• JSON parsed by the server View system
  25. 25. JSON Document Structuremeta{“id”: “u::jasdeep@couchbase.com”,“rev”: “1-0002bce0000000000”,“flags”: 0,“expiration”: 0,“type”: “json”}document{“uid”: 123456,“firstname”: “jasdeep”,“lastname”: “Jaitla”,“age”: 22,“favorite_colors”: [“blue”, “black”],“email”: “jasdeep@couchbase.com”}Meta InformationIncluding KeyAll Keys Unique andKept in RAMDocument ValueMost Recent In RamAnd Persisted To Disk
  26. 26. A JSON Document{“id": "beer_Hoptimus_Prime",“type”: “beer”,"abv": 10.0,"brewery": "Legacy Brewing Co.","category": "North American Ale","name": "Hoptimus Prime","style": "Imperial or Double India Pale Ale",}The primary keyA floatThe type information
  27. 27. Other Documents and DocumentRelationships{“id": "beer_Hoptimus_Prime",“type” : “beer”,"abv": 10.0,"brewery": ”brewery_Legacy_Brewing_Co","category": "North American Ale","name": "Hoptimus Prime","style": “Double India Pale Ale”}{“id": ”brewery_Legacy_Brewing_Co”,“type” : “brewery”,"name" : "Legacy Brewing Co.","address": "525 Canal StreetReading, Pennsylvania, 19601 United States","updated": "2010-07-22 20:00:20","latitude": -75.928469,"longitude": 40.325725}Afterthought
  28. 28. Simplicity of Document Oriented Datastore• Schema is optional– Technically, each document has an implicit schema– Extend the schema at any time!• Need a new field? Add it. Define a default for similar objects which may not havethis field yet.• Data is self-contained– Documents more naturally support the world around you, the data structuresaround you• Model data for your App/Code instead for the Database• Try to keep documents as small as possible (less than 1MB)• Group data together that fits together, but split out portions that mayhave high levels of contention or are constantly growing
  29. 29. Views/Indexes/Queries• Views create perspectives on a collection of documents- Primary/Secondary/Tertiary/Composite Indexing- Aggregations• Use Incremental Map/Reduce- Map defines the relationship between fields in documents and output table- Reduce provides method for collating/summarizing• VIEWS materialize INDEXES- Data writes are fast (no index)- Index updates all changes since last update- Indexes are eventually indexed- Must be pre-materialized (ad-hoc querying available via full-text indexing)• Applications QUERY the INDEX- Queries are eventually consistent with respect to documents
  30. 30. Cluster Administration
  31. 31. Web Console
  32. 32. BackupData FilescbbackupServerServer Servernetwork networknetwork
  33. 33. Restore2) “cbrestore” used to restore data into live/different clusterData Filescbrestore (-a)
  34. 34. Upgrading2 Methods to upgradeCouchbase Server cluster:In-place (offline) and Rolling (online)
  35. 35. Sizing a ClusterSizing == performance• Serve reads out of RAM• Enough IO for writes and disk operations• Mitigate inevitable failuresReading Data Writing DataServerGive medocument AHere isdocument AApplication ServerAServerPlease storedocument AOK, I storeddocument AApplication ServerA
  36. 36. How many nodes?5 Key Factors determine number of nodes needed:1) RAM2) Disk3) CPU4) Network5) Data Distribution/SafetyCouchbase ServersWeb application serverApplication user
  37. 37. DEMO
  38. 38. EasyScalabilityConsistent HighPerformanceAlwaysOn24x365Grow cluster withoutapplication changes, withoutdowntime with a single clickConsistent sub-millisecondread and write response timeswith consistent high throughputNo downtime for softwareupgrades, hardwaremaintenance, etc.Couchbase ServerJSONJSONJSONJSONJSONFlexible DataModelJSON document model withno fixed schema.Couchbase is the Complete Solution
  39. 39. Thank youCouchbaseNoSQL Document Database

×