Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Sync Gateway: beyond just sync – Connect Silicon Valley 2017

165 views

Published on

Speaker: Fujio Turner

If you’re familiar with it, then you know that a big function of Couchbase Sync Gateway is to sync data between web, mobile, and IoT applications (Couchbase Lite) and the backend database (Couchbase Server).

But did you know Sync Gateway can do a lot more than that? In addition to transparently synchronizing data between edge devices and cloud or vice versa, Sync Gateway’s data access APIs give you fine-grained security to build web, mobile, and IoT applications. It also provides inbound and outbound data integration APIs for interacting with external data stores and services. In this session, Sachin will demonstrate the many ways you can leverage the power of this internet-facing web gateway to build powerful apps.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Sync Gateway: beyond just sync – Connect Silicon Valley 2017

  1. 1. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2017. All rights reserved. The Offline Challenge: Delivering mobile apps that always work COUCHBASE MOBILE
  2. 2. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2017. All rights reserved. 2 Personalized Anywhere Fast Secure Online/Offline Customers now expect exceptional mobile experiences
  3. 3. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2017. All rights reserved. 4 Challenges with creating mobile experiences Data availability for apps that always work Data Integration API’s for integrating into your existing ecosystem Data security from the cloud to the edge Data management from the cloud to the edge
  4. 4. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2017. All rights reserved. 5 A homegrown mobile application platform INTERNET INTRANET S E C U R I T Y  Complex to build and manage  Sync is hard  Offline experience is poor  Custom-build data API’s Middle Tier WEB SERVICE WEB SERVICE WEB SERVICE. . . Database Tier SoRSoRCache SoR. . . S E C U R I T Y S E C U R I T Y Client Tier C O N N E C T E D A P P S
  5. 5. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2017. All rights reserved. 6 Couchbase Mobile The Full-Stack Mobile Data Platform
  6. 6. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2017. All rights reserved. 7 Couchbase Lite Full-featured, embedded NoSQL database Sync Gateway Secure data synchronization over the web Couchbase Server Flexible, Highly Scalable NoSQL Database Couchbase Mobile Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2017. All rights reserved.
  7. 7. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2017. All rights reserved. 8 Couchbase Lite  AES-256  Industry Standard Encryption Sync Gateway  HTTP/TLS  Fine Grain Read/Write Permissions Couchbase Server  Role Base Access  Logging & Auditing  Optional Data Encryption on Disk Couchbase Mobile Security Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2017. All rights reserved.
  8. 8. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2017. All rights reserved. 9 9 Replication with Couchbase Lite Your APP PULL PUSH Replication: Continuous or One Time
  9. 9. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2017. All rights reserved. 10 Sync Gateway & Channels – Pull Feed { “_id”:”12”, “_rev”:”3-hno..”, “channels”:[“yellow”] } { “_id”:”78”, “_rev”:”5-u9b..”, “channels”:[“yellow”] } 78312017
  10. 10. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2017. All rights reserved. 11 Couchbase Sync Gateway
  11. 11. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2017. All rights reserved. 12 12 Sync Gateway Overview ©2017 Couchbase Inc. Synchronize data between Couchbase Lite and Couchbase Server Data integration APIs including REST, stream, batch, and event APIs Fine grained user and role based access control Elastically scalable in real-time Sync Gateway Synchronization
  12. 12. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2017. All rights reserved. 13 13 Role of Couchbase Sync Gateway Request Processing Inbound requests over the internet Security Policy Mgmt Data Access Control
  13. 13. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2017. All rights reserved. 14 Bob Jim channel(doc.owner) Admin Definable Write Access Simple or Fine Grain
  14. 14. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2017. All rights reserved. 15 Authentication 1. Basic Authentication 2. OpenID Connect oAuthorization Code Flow oImplicit Flow 3. Custom Authentication 4. Facebook & Google Login
  15. 15. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2017. All rights reserved. 16 Authentication & Sessions Custom 3rd Party Auth User ? Built-In
  16. 16. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2017. All rights reserved. 17 17 Sync Gateway Security Model JSON { }User (“Pilot”) User User User User User User JSON { } Roles (”West_Pilots”) Channels (“West_Flights”) Doc (“West_Flight_100”) GRANT
  17. 17. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2017. All rights reserved. 18 POST /{db}/_user/ HTTP/1.1 Host: localhost:4985 Content-Type: application/json { "name": "bob", "password": "1234", "admin_roles": [”west_pilots”], "admin_channels":["bob","bob-flights","public"] } POST /{db}/_user/ HTTP/1.1 Host: localhost:4985 Content-Type: application/json { "name": "bob", "password": "1234", "admin_roles": [”west_pilots"], "admin_channels":[”*"] } CREATING USERS
  18. 18. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2017. All rights reserved. 19 Optional – Creating Roles POST /{db}/_role/{name} HTTP/1.1 Host: localhost:4985 Content-Type: application/json { "name": ”west_pilots", "admin_channels":[”west-flight-editor",”west-crew-logs"] }
  19. 19. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2017. All rights reserved. 20 SYNC GATEWAY CRUD CREATE HTTP POST or PUT /{hostname}:{port}/{db}/{docID} + JSON UPDATE HTTP PUT /{hostname}:{port}/{db}/{docID} + JSON READ HTTP GET /{hostname}:{port}/{db}/{docID} >> output JSON DELETE HTTP DELETE /{hostname}:{port}/{db}/{docID} Docs – Replication(YES) CREATE HTTP POST or PUT /{hostname}:{port}/{db}/_local/{docID} + JSON UPDATE HTTP PUT /{hostname}:{port}/{db}/_local/{docID} + JSON READ HTTP GET /{hostname}:{port}/{db}/_local/{docID} >> output JSON DELETE HTTP DELETE /{hostname}:{port}/{db}/_local/{docID} Local Docs – Replication(NO)
  20. 20. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2017. All rights reserved. 21 Sync Gateway REST API https://developer.couchbase.com/documentation/mobile/current/references/sync-gateway/rest-api/index.html
  21. 21. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2017. All rights reserved. 22 Data Lifespan CREATE HTTP POST or PUT /{hostname}:{port}/{db}/{docID} + JSON UPDATE HTTP PUT /{hostname}:{port}/{db}/{docID} + JSON { …… “_exp”:”seconds or timestamp” } Docs with Expiration(TTL) UPDATE HTTP POST /{hostname}:4985/{db}/_purge + JSON { “{docID}”:[”{Doc Revision} or * ”] } Admin Doc Purge (Hard Delete)
  22. 22. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2017. All rights reserved. 23 Sync Function in Sync Gateway { “_id”:”12”, “_rev”:”3-hno..”, ”channels”:[“bob”] , “user”:”bob”, “role”:”west-pilot”, “createdBy”:”bob”, “docType”:”profile” } Where does this document go? ? ? ?
  23. 23. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2017. All rights reserved. 24 Sync Function - Checking Documents • Document Type ? • What channel(s) should it go in? • Are they the author ? • Can they access the channel(s) ? { “_id”:”12”, “_rev”:”3-hno..”, ”channels”:[“bob”] , “user”:”bob”, “role”:”west-pilot”, “createdBy”:”bob”, “docType”:”profile” }
  24. 24. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2017. All rights reserved. 25 Sync Function via JavaScript in Sync Gateway Built-In requireAccess(); requireRole(); requireUser(); access(); role(); throw();
  25. 25. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2017. All rights reserved. 26 Integration
  26. 26. Couchbase Mobile to My Current Systems Your App for: Your Business Logic 242526
  27. 27. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2017. All rights reserved. 28 Simple Data Feed
  28. 28. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2017. All rights reserved. 29 Github https://github.com/fujio-turner
  29. 29. Couchbase Mobile to My Current Systems Stream and/or Batch Data Feed(s) 242526
  30. 30. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2017. All rights reserved. 31 HA in Your App { “dateTime”:”2017-01-15 10:19:55”, “seq”:”1000”, “user”:”7.7.7.7” } PUT {host}:{ip}/{db}/_local/{docID} APP (7.7.7.7) APP (8.8.8.8) Backup
  31. 31. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2017. All rights reserved. 32 Sync Gateway – the “Truth” .v1
  32. 32. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2017. All rights reserved. 33 Conflict Management
  33. 33. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2017. All rights reserved. 34 Conflicts – “It’s ok to have conflicts just:” id:123 city: Boston rev: 1-123.. id:123 city: New York rev: 2-4aq.. id:123 city: Chicago rev: 2-5jo.. 1. Spell out the business logic on how to handle them. 2. Make sure everybody who see them in your systems handles them the same, deterministically.
  34. 34. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2017. All rights reserved. 35 Conflicts id:123 city: Boston rev: 1-123.. id:123 city: New York rev: 2-4aq.. id:123 city: Chicago rev: 2-5jo.. id:123 city: Phoenix rev: 3-c81.. NOT Deleted and Longest History and Highest ASCII Sort Order Deterministic To Show
  35. 35. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2017. All rights reserved. 36 Conflicts id:123 city: Boston rev: 1-123.. id:123 city: New York rev: 2-4aq.. id:123 city: Chicago rev: 2-5jo.. id:123 city: Phoenix rev: 3-c81.. NOT Deleted and Longest History and Highest ASCII Sort Order - “Is the conflict resolved?” Deterministic To Show
  36. 36. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2017. All rights reserved. 37 { “_id”:”123”, “changes”:[ ”2-5jo..”, ”2-4aq..” ] } http://{ip}/{db}/_changes?active_only=true&style=all_docs The default rev to show is always first:
  37. 37. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2017. All rights reserved. 39 Conflicts id:123 city: Boston rev: 1-123.. id:123 city: New York rev: 2-4aq.. id:123 city: Chicago rev: 2-5jo.. id:123 city: Phoenix rev: 3-c81.. DELETE id: 123 , rev: 2-4aq - Resolving (Prune the Branch) id:123 _deleted: true rev: 3-9v8..
  38. 38. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2017. All rights reserved. 40 Github https://github.com/fujio-turner
  39. 39. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2017. All rights reserved. 41 Conflicts id:123 city: Boston rev: 1-123.. id:123 city: New York rev: 2-4aq.. id:123 city: Chicago rev: 2-5jo.. id:123 city: Phoenix rev: 3-c81.. id:123 _deleted: true rev: 3-9v8.. id:123 city: Phoenix , New York rev: 4-t5n.. - Resolving (Prune the Branch) & Merge
  40. 40. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2017. All rights reserved. 42 Convergence
  41. 41. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2017. All rights reserved. 43 Mobile Convergence in CB 5.0 & CB Mobile 1.5 Q: “Can I use the SDK to update keys and sync data?” A: “Yes”
  42. 42. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2017. All rights reserved. 44 Auditing
  43. 43. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2017. All rights reserved. 45 Monitoring & Auditing Sync - “How do I know it synced?” { “_id”:”34”, “_rev”:”5-z9a..”, “channels”:[“bob”] } { “_id”:”12”, “_rev”:”3-hno..”, “channels”:[“tim”,”amy”] } _user:Bob channels[bob] { “_id”:”34”, “_rev”:”5-z9a..”, “channels”:[“jim”] } sent doc:34,ver:5-z9a.. to Bob @ T1 Sync Gateway Logs Bob said he processed doc @ T2
  44. 44. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2017. All rights reserved. 46 46 Why Couchbase Mobile 46 One Platform Complete Secure Couchbase Lite Sync Gateway Couchbase Server IntranetInternet Security ✓ ___ ✓ ___ ✓ ___
  45. 45. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2017. All rights reserved. THANK YOU

×