Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Couchbase data platform 101: what you need to know to get started – Couchbase Connect New York 2017

288 views

Published on

Application development may be all about introducing change and innovation, but Operations is all about getting applications deployed, running them efficiently, and making sure they never go down. But how do you do that when your operational data platform is new?
This session will introduce you to the Couchbase data platform and the power that lies within. Learn key concepts about the deployment architecture, scaling, high availability, and disaster recovery as well as indexing and querying capabilities demonstrated via a live demo. Get operational best practices used by leading Couchbase customers with years of production experience.

Published in: Software
  • Be the first to comment

Couchbase data platform 101: what you need to know to get started – Couchbase Connect New York 2017

  1. 1. ©2017 Couchbase Inc. Couchbase Data Platform 101 Perry Krug, Principal Architect 1
  2. 2. ©2017 Couchbase Inc. Couchbase • Multiple data models and index types • N1QL – SQL like query language • Built-in query workbench • SDKs, ODBC / JDBC drivers and frameworks • Push-button scalability • Consistent high-performance • Always on 24x7 with HA - DR • Easy Administration with Web UI, Rest API and CLI
  3. 3. ©2017 Couchbase Inc. Couchbase: Full Stack Data Management Lightweight embedded NoSQL database with full CRUD and query functionality. Secure web gateway with synchronization, data access, and data integration APIs for accessing, integrating, and synchronizing data over the web. Highly scalable, highly available, high performance NoSQL database server. Built-in enterprise level security throughout the entire stack includes user authentication, user and role based data access control (RBAC), secure transport (TLS), and 256-bit AES full database encryption. Couchbase Lite Sync Gateway Couchbase Server EMBEDDED DATABASE SYNCHRONIZATION DATABASE SERVER SECURITY CLIENT WEB TIER DATABASEinternet intranet
  4. 4. ©2017 Couchbase Inc. Intranet Internet Database Middle Tier Client Couchbase: Full Stack Data Management Couchbase Lite On Device Couchbase Sync Gateway Over the Web Couchbase Server KV, Query, Search, Replication, Security, Analytics High Performance, Elastic Scalability, Always Available Application
  5. 5. ©2017 Couchbase Inc. Couchbase in the Data Management Landscape Enterprise Applications Social Media Web & Ecommerce Mobile AppsSensor Data MainframeExternal DataSystem Logs Speed Layer Batch Layer Data Lake Business Intelligence Dashboards Query Tools In Memory Cache Data Warehouse N1QL Query Workbench Data Sources Data Integration Data Management Data Access Data Serving Layer Real Time Data Ingestion
  6. 6. ©2017 Couchbase Inc. Couchbase in the Lambda Architecture New Data Stream MergedView All Data Precompute Views (Map Reduce) Process Stream Incremental Views Partial Aggregate Partial Aggregate Partial Aggregate Real-Time Data Batch Recompute BatchViews Real-Time Views Real-Time Increment Merge Batch Layer Serving/Presentation Layer Speed Layer Ingestion Layer
  7. 7. ©2017 Couchbase Inc. Connecting into and out of Couchbase
  8. 8. ©2017 Couchbase Inc. ONE PLATFORM FOR OPERATIONAL & ANALYTICSThe Couchbase Data Platform Data Query Index Search Unified Programming Interface Analytics Unified Administration
  9. 9. ©2017 Couchbase Inc.©2017 Couchbase Inc. Operate At Any Scale
  10. 10. ©2017 Couchbase Inc. Couchbase Architecture • Data Service – Stores documents, builds and maintains Distributed Secondary Indexes (MapReduce Views) • Indexing Service – builds and maintains Global Secondary Indexes • Search Service – builds, maintains and queries Full Text Search indexes • Query Service – plans, coordinates, and executes queries against either Global or Distributed Indexes • Cluster Manager – configuration, heartbeat, statistics, RESTful Management interface
  11. 11. ©2017 Couchbase Inc. Application To Database Interaction
  12. 12. ©2017 Couchbase Inc. Modern Architecture – Multi-Dimensional Scaling MDS is the architecture that enables independent scaling of data, query, and indexing workloads while being managed as one cluster.
  13. 13. ©2017 Couchbase Inc. Modern Architecture – Multi-Dimensional Scaling
  14. 14. ©2017 Couchbase Inc. Modern Architecture – Multi-Dimensional Scaling
  15. 15. ©2017 Couchbase Inc. XDCR: Cross Data Center Replication • Active-Active • Memory-to-Memory • Topology Aware • Filtered • Independently Scalable
  16. 16. ©2017 Couchbase Inc. Data Service: Replication and Persistence APPLICATION SERVER CACHE DISK XDCR QUEUE TO OTHER CLUSTER CACHE DISK DISK QUEUE REPLICATION QUEUE APPLICATION SERVER DISK DISK QUEUE CACHE REPLICATION QUEUE DISK QUEUE DOC DOCDOC DOC DOC CLUSTER 1 NODE 2 CLUSTER 2 NODE … CLUSTER 1 NODE 1 DOC DOC
  17. 17. ©2017 Couchbase Inc. Data Service: Cluster Map and Scaling Out
  18. 18. ©2017 Couchbase Inc. Couchbase Services: Load balancing, HA and DR • Data Service: • LB: Single-hop access based upon cluster map • HA: Automatic replication + Rack/Zone awareness, auto-failover • DR: Cross-cluster replication • Index Service: • LB: Per-query, round-robin and heuristics based • HA: Auto or manual replication + Rack/Zone awareness, auto-failover • DR: Indexes independent across clusters • Query Service: • LB: Round-robin from application • HA: Query engines are stateless, auto-failover • DR: Query service independent across clusters • FullText Service: • LB: Round-robin from application • HA: Auto replication (not Rack/Zone aware yet) • DR: Indexes independent across clusters
  19. 19. ©2017 Couchbase Inc. Backup And RestoreWith Couchbase • Zer0-downtime backup and restore: Documents and index definitions • Built-in utilities: cbbackup / cbrestore / cbbackupmgr • Full, differential and cumulative backup available per-bucket • Restore from any point, to any bucket or topology
  20. 20. ©2017 Couchbase Inc. Security With Couchbase Administrative Security • Role-BasedAccess Control • Authenticate via LDAP, PAM, or built-in • Encryption on RESTAPI andWeb GUI (SSL/TLS or x.509) • Administrative action auditing • Encrypted configuration, passwords, tokens Data Security • Role-BasedAccess Control (per bucket) • Authenticate via LDAP, PAM, or built-in • Encryption: Data-at-Rest and over-the-wire (SSL/TLS)
  21. 21. ©2017 Couchbase Inc. 21 Best Practices for getting started in Production • Minimum Hardware: 8 CPU cores, 8GB RAM • Physical Hardware • VMs • Containers • Cloud Instances • 3 nodes minimum • 1-3 Buckets, 1 replica each • Remove the primary index in production • Covering indexes whenever possible • Incremental backups from the beginning • Keep the defaults! • “T-Shirt” cluster sizing: • Small: • 0-10M Documents • 0-1k writes/sec • 0-1k N1QL queries/sec • 3 nodes, all services all nodes • Medium: • <100M Documents • <10k writes/sec • <5k N1QL queries/sec • 5-7 nodes, “Multi-Dimensional Scaling” • Large: • >500M Documents • >20k writes/sec • >10k N1QL queries/sec • 20+ nodes, MDS • Call Couchbase 
  22. 22. ©2017 Couchbase Inc.©2017 Couchbase Inc. 22 Demo
  23. 23. ©2017 Couchbase Inc. 23 Perry Krug Principal Architect perry@couchbase.com
  24. 24. ©2017 Couchbase Inc.©2017 Couchbase Inc. ThankYou! 24

×