Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Architecting Business-Critical Applications With Couchbase Mobile – Connect Silicon Valley 2018

50 views

Published on

Speakers: Wayne Carter, Chief Architect, Mobile, Couchbase;
Jens Alfke, Mobile Architect, Couchbase; Adam Fraser, Senior Software Architect, Couchbase

In this session you’ll learn how to architect applications to deliver on the specific needs of business-critical applications. We’ll explain how Couchbase Mobile provides everything you need to fully manage your business-critical data from the cloud to the edge. Couchbase Mobile provides fully managed data storage, access, transport, and security throughout the entire application and network stack and is uniquely designed to guarantee data availability and millisecond response times regardless of internet connectivity and speed. We will cover how to synchronize your data with mobile applications, query and search data on device using N1QL, secure your data in motion and at rest, recover data on device, and cloudless peer-to-peer replication.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Architecting Business-Critical Applications With Couchbase Mobile – Connect Silicon Valley 2018

  1. 1. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2018. All rights reserved. How to Architect Business Critical Applications with Couchbase Mobile September 19, 2018 Wayne Carter | Vice President - Mobile Jens Alfke | Architect - Couchbase Lite Adam Fraser | Architect - Sync Gateway
  2. 2. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2018. All rights reserved. AGENDA 1 2 3 4 Introduction Couchbase Lite Sync Gateway Q&A
  3. 3. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2018. All rights reserved. 3 Desired Business Outcome What are businesses trying to do?
  4. 4. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2018. All rights reserved. 4 Desired Business Outcome More Effective Business Better Products & Services More Effective Employees Better Customer Experience 1. Always available when needed 2. Always fast access to information to help buy and use the product or service 1. Always available when needed 2. Always fast access to the information to help do the job at hand Requirements Strategy Objective
  5. 5. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2018. All rights reserved. 5 Technical Approach Capabilities Data Location Network Requirement Application Availability Application Speed Enabling Technology
  6. 6. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2018. All rights reserved. 6 Technical Approach Capabilities Data Location Network Requirement Application Availability Application Speed Enabling Technology Remote Only Always Connected Sometimes Available Sometimes Slow Sometimes Fast REST Services
  7. 7. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2018. All rights reserved. 7 Technical Approach Capabilities Data Location Network Requirement Application Availability Application Speed Enabling Technology Remote Only Always Connected Sometimes Available Sometimes Slow Sometimes Fast REST Services Local Only Never Connected Always Available Always Fast Embedded Database
  8. 8. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2018. All rights reserved. 8 Technical Approach Capabilities Data Location Network Requirement Application Availability Application Speed Enabling Technology Remote Only Always Connected Sometimes Available Sometimes Slow Sometimes Fast REST Services Local Only Never Connected Always Available Always Fast Embedded Database Local & Remote Sometimes Connected Always Available Always Fast Embedded Database + Sync
  9. 9. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2018. All rights reserved. 9 Technical Approach Capabilities Data Location Network Requirement Application Availability Application Speed Enabling Technology Remote Only Always Connected Sometimes Available Sometimes Slow Sometimes Fast REST Services Local Only Never Connected Always Available Always Fast Embedded Database Local & Remote Sometimes Connected Always Available Always Fast Embedded Database + Sync
  10. 10. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2018. All rights reserved. 10 Technical Approach Capabilities Data Location Network Requirement Application Availability Application Speed Enabling Technology Remote Only Always Connected Sometimes Available Sometimes Slow Sometimes Fast REST Services Local Only Never Connected Always Available Always Fast Embedded Database Local & Remote Sometimes Connected Always Available Always Fast Embedded Database + Sync Only local & remote data, enabled using an embedded database + sync, is the only technical approach that can deliver on the requirements for business critical mobile applications.
  11. 11. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2018. All rights reserved. 11 Data Management Concerns
  12. 12. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2018. All rights reserved. 12 Data Management Concerns Functional Performance Scalability Availability Integration Storage Transport Access Control Revocation Storage Access Transfer Operational Security
  13. 13. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2018. All rights reserved. 13 Client WAN Middle Tier LAN Storage Couchbase
  14. 14. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2018. All rights reserved. 14 Client Your Applications WAN Middle Tier LAN Storage Couchbase Server KV, Query, Search, Analytics, Eventing, Replication, Security Couchbase
  15. 15. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2018. All rights reserved. 15 Client Your Applications WAN Middle Tier Couchbase Sync Gateway Over the Web LAN Storage Couchbase Server KV, Query, Search, Analytics, Eventing, Replication, Security Couchbase
  16. 16. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2018. All rights reserved. 16 Client Your Applications Couchbase Lite On Device WAN Middle Tier Couchbase Sync Gateway Over the Web LAN Storage Couchbase Server KV, Query, Search, Analytics, Eventing, Replication, Security Couchbase
  17. 17. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2018. All rights reserved. 17 Couchbase Mobile Couchbase Lite Couchbase Sync Gateway Couchbase Server Lightweight, fully-functional embedded NoSQL database Highly scalable, highly performant NoSQL database server Internet gateway for securely synchronizing data over the web CLIENT SERVERMIDDLE TIERWAN LAN Authentication, access control, encrypted transport, and encrypted storage SECURITY
  18. 18. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2018. All rights reserved. Couchbase Lite 2
  19. 19. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2018. All rights reserved. 19 Designing Mobile Apps … Correctly ● Data Design ● UI Controller Design ● Bundling a Pre-Loaded Database ● Local Data Recovery ● Security
  20. 20. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2018. All rights reserved. Data Design a
  21. 21. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2018. All rights reserved. 21 To Normalize Or Not To Normalize? “zoe_smith” {“name”: “Zoe Smith”, “addresses”: [ {“street”: “123 Main St”, “city”: “Middletown”, “state”: “CA”}, {“street”: “3 Rue de la Chat”, “city”: “Oulala”, “country”: “France”} } “zoe_smith” {“name”: “Zoe Smith”} “addr_C768F07B9B62” {"owner": "zoe_smith", “street”: “123 Main St”, “city”: “Middletown”, “state”: “CA”} “addr_81F542CB738D” {"owner": "zoe_smith", “street”: “3 Rue de la Chat”, “city”: “Oulala”, “country”: “France”} Denormalized Normalized
  22. 22. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2018. All rights reserved. 22 Benefits Of Normalization • Smaller docs use less memory • Lower risk of conflicts • Allows different security (channels) for different portions of the data • More efficient replication (for now) • More efficient queries (for now) “zoe_smith” {“name”: “Zoe Smith”} “addr_C768F07B9B62” {"owner": "zoe_smith", “street”: “123 Main St”, “city”: “Middletown”, “state”: “CA”} “addr_81F542CB738D” {"owner": "zoe_smith", “street”: “3 Rue de la Chat”, “city”: “Oulala”, “country”: “France”}
  23. 23. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2018. All rights reserved. 23 Benefits Of Denormalization • A document is always internally ACID-consistent • Simpler security controls • Often simpler to code for • More “JSON-y” “zoe_smith” {“name”: “Zoe Smith”, “addresses”: [ {“street”: “123 Main St”, “city”: “Middletown”, “state”: “CA”}, {“street”: “3 Rue de la Chat”, “city”: “Oulala”, “country”: “France”} }
  24. 24. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2018. All rights reserved. 24 What To Use Blobs For • Large or binary data • … that doesn’t always need to be loaded with the rest of the doc • … that may change less often • … that doesn’t need to be queried “zoe_smith” {“name”: “Zoe Smith”, “photo”: { “@type”: “blob”, “content_type”: “image/jpeg”, “digest”: "S6kGd7...."} }
  25. 25. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2018. All rights reserved. Reactive UI Programming b
  26. 26. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2018. All rights reserved. 26 Reactive UI Programming Model Controller Views User Input Doc / Query Replicator Other Controller
  27. 27. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2018. All rights reserved. 27 • CBL Listeners only tell you about saved document changes • Your model should synthesize notifications of unsaved changes • Listener tells you the {doc, query} has changed, but not exactly what changed. • Query listener only triggers when query results change • Doc changes not reflected in the query will not trigger a notification • Loading the Document from a QueryResult is often an anti-pattern • Query listeners are not cheap! Reactive UI: Caveats
  28. 28. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2018. All rights reserved. Pre-Loaded Database c
  29. 29. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2018. All rights reserved. 29 Pre-Loaded Database: Use Case On first launch, your app has to pull a lot of infrequently-changing, non-sensitive docs from the server.
  30. 30. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2018. All rights reserved. 30 Pre-Loaded Database: What To Do • At build time: • Create a database and sync it with the server • Copy the database file (directory) into the app itself • On first launch: • Call Database.copyFromPath to install the database into writeable storage • Open the copied database • Start a pull replication to pick up changes
  31. 31. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2018. All rights reserved. 31 Pre-Loaded Database: Pros and Cons • Pros: • Shortens initial pull replication • Ideally, app is useable immediately on first launch • Saves bandwidth • Cons: • Increases app size and download/installation time • Requires extra build steps • Can only contain public documents • Should be updated periodically, or it loses value over time
  32. 32. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2018. All rights reserved. Local Data Recovery d
  33. 33. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2018. All rights reserved. 33 Data Recovery: Use Case Critical data is entered into the app while offline, and there is a danger of the device failing or being damaged before it can sync.
  34. 34. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2018. All rights reserved. 34 Data Recovery: What To Do • Create a backup database on removable storage • Replicate from the main database to the backup • Push only • Continuous is safest
  35. 35. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2018. All rights reserved. Data Security e
  36. 36. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2018. All rights reserved. 36 Data Security: Use Case Your app has real-life users.
  37. 37. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2018. All rights reserved. 37 Security: On The Network • Encryption • Use TLS (of course) • Authenticate the server! • Avoid man-in-the-middle attacks and faked server certs • Best practice: TLS certificate pinning • Authenticate to the server • Store user credentials in secure enclave (Keychain)
  38. 38. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2018. All rights reserved. 38 Security: Local Database Encryption • Best Practice: • Generate a random AES key • …with a cryptographically-secure random number generator • Store key in OS’s secure enclave (keychain) • Extra credit: • Make the OS require a biometric scan (e.g. TouchID) to read the key
  39. 39. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2018. All rights reserved. 39 Security: Local Database Encryption • Best Practice (iOS): • Use enterprise management tools to force users to set strong device passcodes • Consider whether adding a second layer of encryption is worth it • If mandated by law (e.g. HIPPA) • If devices will be seized by highly sophisticated attackers (FBI, foreign intelligence…) • If Marketing wants encryption as a checkbox feature • If not, no need to encrypt
  40. 40. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2018. All rights reserved. 40 Security: Avoid Data Leakage • Does your app support switching between multiple users? • Use a separate local database for each user! • Does your app allow changing the username? • Delete & recreate the database before syncing!
  41. 41. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2018. All rights reserved. 41 Security: Data Recovery • Always encrypt the backup database • Encryption key must be available off-device! • One solution: Store key in main db, synced to server
  42. 42. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2018. All rights reserved. Sync Gateway 3
  43. 43. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2018. All rights reserved. 43 Requirements of Business Critical Applications - Functional - Data management, Document Lifecycle - Security - Access Control - Operational - Performance, Availability, Integration
  44. 44. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2018. All rights reserved. Document Management and Lifecyclea
  45. 45. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2018. All rights reserved. 45 Functional: Shared Bucket Access Use case: The ability to work with your data at any point in the stack - Read or modify data through Couchbase Lite, Sync Gateway, or Couchbase Server - Maintain security and replication guarantees What to do: - Enable shared bucket access on Sync Gateway - Documents written directly to the bucket are imported by Sync Gateway - Security applied by Sync Gateway - Mobile metadata moved out of the document body, maintains data integrity for Couchbase Server-based applications
  46. 46. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2018. All rights reserved. 46 Functional: Document Lifecycle Use Case: Ensure deleted documents don’t degrade performance - Deleted documents (tombstones) must be retained in order to be replicated - Managing the lifespan of obsolete tombstones is critical - Recover bucket storage - Reduce replication overhead What to do: - Set metadata purge interval on Couchbase Server based on your client replication frequency
  47. 47. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2018. All rights reserved. Security b
  48. 48. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2018. All rights reserved. 48 Security: Access Control Document Channel User Role * * * * * * * * Channel assignment Access Grant Role Grant Use Case: Fine-grained control over document access
  49. 49. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2018. All rights reserved. 49 Security: Granting Read Access Use Case: How to grant channel access to users? - Different approaches for different use cases - Static grants - Admin-based static grants - Minimal processing overhead, static naming - Dynamic grants - Data-driven dynamic grants - Grants issued by documents, via Sync Function - Additional processing required to maintain, but enable more flexible use cases
  50. 50. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2018. All rights reserved. 50 Security Patterns: Read-centric application Use case: Shared, read-only data What to do: - Well-known channel names - Documents assigned to one or more channels - Static channel grants, typically at user creation time Key features: - Highly leverages channel cache - Coarse-grained security access
  51. 51. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2018. All rights reserved. 51 Security Patterns: Channel per User Use Case: Managing of user-owned data What to do: - Assign each user their own channel via static grant - Each document specifies the set of users that should have access Key Features: - Static access grants reduce computation - Clean and transparent security model - Requires every document to identify set of users with access - Potential for document bloat for highly shared documents
  52. 52. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2018. All rights reserved. 52 Security Patterns: Channel per Object Use case: Structured data with object-level security What to do: - Assign each document of a given data type its own channel - Add associated/child documents to the same channel - Dynamically grant users access to the channel Key Features: - More resource-intensive on Sync Gateway, in exchange for fine-grained security access and routing
  53. 53. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2018. All rights reserved. Operational c
  54. 54. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2018. All rights reserved. 54 Operational: Cache Tuning Correctly tuning Sync Gateway’s internal caches can significantly improve system capacity and performance - Reducing the amount of work Sync Gateway needs to do per user/request means more capacity per Sync Gateway node - Caches are tuned via Sync Gateway’s config
  55. 55. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2018. All rights reserved. 55 Operational: Cache Tuning: Revision Cache What is it? - LRU Cache of document revisions - Stores revision body and associated metadata - Optimizes read/write operations Tuning Guidelines - Should be sized such that latest revision of a document is resident in the cache until connected clients have replicated - Generally means that rev cache should be sized to least 3 x (system writes/sec) - Additional capacity provides tolerance for latency spikes - Memory usage dependent on document size
  56. 56. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2018. All rights reserved. 56 Operational: Cache Tuning: Channel Cache What is it? - Ordered cache of document mutations by channel - Stores document metadata only (key, revision, sequence, flags) - Optimizes replication, minimizes number of Couchbase queries Tuning depends on use case - Intermittently connecting, read-heavy clients - Optimized when entire channels are resident in cache - Frequently connecting clients - Cache capacity can be reduced to match write throughput - Caveat: Cache includes non-compacted tombstones and removal markers - need to consider these when sizing
  57. 57. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2018. All rights reserved. 57 Operational: Conflicts Use Case: Avoid or resolve conflicts to improve system performance - Unresolved conflicts result in larger document metadata - Impacts storage, transport, processing What to do: - Resolve conflicts - Couchbase Lite 2.0 - Use allow_conflicts:false - Provides CAS-like semantics via Sync Gateway REST API
  58. 58. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2018. All rights reserved. 58 Operational: Availability Use Case: Ensuring system availability through node failure and migration - Couchbase Server node failure - Sync Gateway uses GSI w/ replicas to avoid downtime due to view reindexing - Sync Gateway node failure - Stateless nodes provide redundancy, clients can transfer seamlessly - Upgrade - Query versioning and backwards compatibility avoids downtime, supports rolling upgrade - Coming soon - High availability for feed-based import and sg-replicate - Currently single-node, and will have downtime during Sync Gateway node failure
  59. 59. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2018. All rights reserved. 59 Integration: Couchbase Mobile and Web Applications Use Case: Users need to work with their data across mobile and web applications What to do: - Shared Data - Shared Bucket Access allows mobile and web applications to share data - Import filter lets you identify a subset of mobile-enabled data
  60. 60. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2018. All rights reserved. 60 Integration: External Integrations Use Case: Integrate Couchbase Mobile data with external systems What to do: - Changes worker integration - Guaranteed delivery, fault-tolerant, resumable - Stateful, requires application to manage - Webhooks - Fire-and-forget minimizes processing overhead - HA is receiver responsibility - Stateless, no application coding required
  61. 61. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2018. All rights reserved. Q&A 4
  62. 62. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2018. All rights reserved. 62 Client Your Applications Couchbase Lite On Device WAN Middle Tier Couchbase Sync Gateway Over the Web LAN Storage Couchbase Server KV, Query, Search, Analytics, Eventing, Replication, Security Couchbase Mobile for Business Critical Applications Data Management Concerns ✓ Functional ✓ Operational ✓ Security Business Requirements ✓ Always Available ✓ Always Fast
  63. 63. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2018. All rights reserved. Thank you.

×