Ch-ch-ch-ch-changes
Taking Your Stitch Application to the Next Level
with Stitch Triggers
Industry Trends – Leading to Stitch
1370 1990
Industry Trends – Leading to Stitch
19901370 1997
Industry Trends – Leading to Stitch
19971990 2001
Industry Trends – Leading to Stitch
20011997 2007
Industry Trends – Leading to Stitch
20072001 2014
Industry Trends – Leading to Stitch
20142007 2018
Industry Trends – Leading to Stitch
20182014
Intelligent Operational Data Platform
Best way to work
with data
Intelligently put data
where you want it
Freedom to run
anywhere
● MongoDB Stitch
● MongoDB Server 4.0
● MongoDB Compass
● MongoDB Charts
● MongoDB Mobile
● MongoDB Server 4.0
● MongoDB Atlas
● Ops Manager 4.0
● Free Cloud Monitoring
The evolution of MongoDB
3.0 3.2
Document Validation
$lookup
Fast Failover
Simpler Scalability
Aggregation ++
Encryption At Rest
In-Memory Storage Engine
BI Connector
MongoDB Compass
APM Integration
Profiler Visualization
Auto Index Builds
Backups to File System
Doc-Level
Concurrency
Compression
Storage Engine API
≤50 replicas
Auditing ++
Ops Manager
Linearizable reads
Intra-cluster compression
Views
Log Redaction
Linearizable Reads
Graph Processing
Decimal
Collations
Faceted Navigation
Zones ++
Aggregation ++
Auto-balancing ++
ARM, Power, zSeries
BI & Spark Connectors ++
Compass ++
Hardware Monitoring
Server Pool
LDAP Authorization
Encrypted Backups
Cloud Foundry Integration
3.4 3.6
Change Streams
Retryable Writes
Expressive Array Updates
Query Expressivity
Causal Consistency
Consistent Sharded Sec. Reads
Compass Community
Ops Manager ++
Query Advisor
Schema Validation
End to End Compression
IP Whitelisting
Default Bind to Localhost
Sessions
WiredTiger 1m+ Collections
MongoDB BI Connector ++
Expressive $lookUp
R Driver
Atlas Cross Region Replication
Atlas Auto Storage Scaling
4.0
Multi-Document ACID Transactions
Atlas Global Sharding
Atlas HIPAA
Atlas LDAP
Atlas Audit
Atlas Encrypted Storage Engine
Atlas AWS Backup Snapshots
Snapshot Reads
Agg Pipeline Type Conversions
40% Faster Shard Migrations
Non-Blocking Secondary Reads
SHA-2
TLS 1.1+
Compass Agg Pipeline Builder
Compass Export to Code
Charts Beta
Monitoring Cloud Service
Ops Manager K8s & OpenShift
MongoDB Stitch GA
MongoDB Mobile (Private Beta)
“We set out to build a database that we would want to use, so that
whenever developers wanted to build an application, they could
focus on the application, not on working around the database.”
- Eliot Horowitz
Stitch Functions
Stitch is a collection of servers that
process application requests
Requests:
• Single actions for Database or Services
• Or executing a Stitch Function
• Integrated with Stitch’s rules
Functions:
• Scalable, hosted JavaScript (ES6) Functions
• Integrated with application context
• User, Request, Services, Values, etc.
MongoDB Query Language + Native DriversIntegrated Rules
Functions3rd Party Services
Native SDKs (JavaScript, Android, iOS)
Rest API
When do we want functions to execute?
AUTHENTICATION
WEBHOOK/
INTEGRATION
DATABASE EVENTS
3 Contexts Where Functions
Execute
1
2
3
When do we want functions to execute?
client.executeFunction(…)client.loginWithCredential(…)
AUTHENTICATION
WEBHOOK/
INTEGRATION
DATABASE EVENTS
1
2
3
When do we want functions to execute?
Incoming Webhooks
(HTTP, Twilio, Github)
AUTHENTICATION
WEBHOOK/
INTEGRATION
DATABASE EVENTS
1
2
3
When do we want functions to execute?
Database Events
Collection.insertOne(…)
AUTHENTICATION
WEBHOOK/
INTEGRATION
DATABASE EVENTS
1
2
3
Responding to Database Changes in the Past
Responding to Database Changes in the Past
Before Change Streams:
{
"ts" : Timestamp(1526593721, 1),
"t" : NumberLong(1),
"h" : NumberLong("-2175450131096164196"),
"v" : 2,
"op" : "n",
"ns" : "",
"wall" : ISODate("2018-05-17T21:48:41.962Z"),
"o" : {
"msg" : "periodic noop"
}
}
{
"ts" : Timestamp(1526593731, 1),
"t" : NumberLong(1),
"h" : NumberLong("4757850941625744949"),
"v" : 2,
"op" : "n",
"ns" : "",
"wall" : ISODate("2018-05-17T21:48:51.962Z"),
"o" : {
"msg" : "periodic noop"
}
}
OPLOGOPLOG OPLOG
Application
W
db.users.insert({,,,})
PRIMARY SECONDARY SECONDARY
db.users.insert({,,,})
A
C
K
db.users.insert({,,,})
W
db.users.update({,,,}) db.users.update({,,,})
A
C
K
db.users.update({,,,})
What’s an Oplog?
OUT OF DATE OUT OF DATE
OPLOGOPLOG OPLOG
Application
W
db.users.insert({,,,})
PRIMARY SECONDARY SECONDARY
db.users.insert({,,,})
A
C
K
db.users.insert({,,,})
W
db.users.update({,,,}) db.users.update({,,,})
A
C
K
db.users.update({,,,})
Monitor
R R
If (insert) {
aws.s3.insertObject(…);
}
Before ChangeStreams and Triggers
TAILING THE OPLOG
Setting up a Change Stream (NodeJS)
Triggers in Stitch
Traditional Database Triggers
• Triggers are:
• Code executed when a certain database
event occurs
• Executed within the Database
• Trigger uses include:
• Enforcing Referential/Business logic
• Auditing and Logging
• Security/Validation Logic
• Maintaining Replication
Stitch Coordinates Change Streams
1. Sets-up Change Stream
2. Maintains Streams over time
3. Passes Change Events to
Coordinator
Stitch Infrastructure
Stitch Connection
Event Coordinator
CS 1 CS 3
CS 2
Adding Triggers to Stitch
Reduce Time to Market
Functions as a Service: No waiting on
infrastructure
Reduce boilerplate: Save code
Cross-Platform: Develop once
Existing Apps Untouched: No reverse-
engineering
Reduce Ops costs
Serverless: Zero wasted capacity
Payment: No up-front cost
Capacity on Demand: No need to over-
provision in advance
Secure by Default: Less operational effort
Reduce Dev Effort
Backend Integration Built in: No generic
backend code
HA & Scalability Built in: Hard and time
consuming to build Orchestrate Services:
Reuse what's out there
Faster Cheaper Easier
Event Driven Life-cycle
Pending SucceededOwned
Failed Dead
Event is …
Waiting for a ConsumerAttempting ExecutionCompletedAwaiting RetriesRe-Attempting ExecutionUnable to Execute
(Trigger Disabled)
Introducing: Stitch Triggers
Add Trigger
Add Trigger
Event-driven Functions for Communication
FUNCTION
Database
Update
Text/E-mail
EVENT
Push Notification
Event-driven Functions for Data Propagation
FUNCTION
EVENT
Database
Operation
Stitch Ensures Correct Execution
Database Events are:
• Generated by matching
operations ({match: { …}})
• Fully Ordered
1. Event is Generated
2. Event added to Job Queue
3. Event waits for blocking Events
to complete
4. Event waits for consumer to
attempt execution
The MongoDB Swagstore
What is the Swagstore
Standard Retail Store:
• Browse Items
• Add items to cart
• Checkout
• Request Notification for Restock
Additional Features:
• Update/text on item re-stock
• Shipping text/e-mail notification
Stitch Features
• Functions
• Triggers
• 3rd Party Services
Inventory Updates
FUNCTION
EVENT
{
ID: XXXXX
SIZE: M
COUNT: 100
}
{
ID: XXXXX
DESC: “Men’s T-shirt”
SM_COUNT: 12
MED_COUNT: 100
LG_COUNT: 23
}
Shipping Updates
FUNCTION
Shipping Status
Updated
Your package
has shipped
EVENT
Swagstore in Action
AUTHENTICATION
BROWSE
SELECT ITEM
IN
STOCK?
CART
CHECKOUT
UPDATE NOTIFY
Initialization
• Before we leverage any features of Stitch,
we must initialize our API with the Stitch
App Key
import React from 'react'
import ReactDOM from 'react-dom'
import './index.css'
import App from './App'
import registerServiceWorker from './registerServiceWorker'
import { BrowserRouter } from 'react-router-dom'
import { StitchClientFactory } from 'mongodb-stitch'
const appId = 'ecommercechatbot-glwkl'
let stitchClientPromise = StitchClientFactory.create(appId)
stitchClientPromise.then(stitchClient => {
let db = stitchClient.service('mongodb', 'mongodb-
atlas').db('swagstore')
let props = { stitchClient, db }
ReactDOM.render(
<BrowserRouter>
<App {...props} />
</BrowserRouter>,
document.getElementById('root')
)
registerServiceWorker()
})
AUTHENTICATION
BROWSE
SELECT ITEM
IN
STOCK?
CART
CHECKOUT
UPDATE NOTIFY
Authentication
• Leverage integration with Google Sign-in
to authenticate the user and set the user’s
profile context.
AUTHENTICATION
BROWSE
SELECT ITEM
IN
STOCK?
CART
CHECKOUT
UPDATE NOTIFY
Browse
• Leverage SDK db.collection() to access
SWAG-STORE database and products
collection
import React, { Component } from 'react'
import { ProductDeck } from '../components/Products'
class ProductsPage extends Component {
constructor(props) {
super(props)
this.state = {
products: []
}
}
getProducts() {
this.props.db
.collection('products’)
.find({ category: this.props.match.params.category })
.execute()
.then(products => {
this.setState({ products })
})
.catch(err => {
console.log(err)
})
}
componentDidMount() {
this.getProducts()
}
componentDidUpdate(prevProps, prevState, snapshot) {
if (this.props.match.params.category !== prevProps.match.params.category) {
AUTHENTICATION
BROWSE
SELECT ITEM
IN
STOCK?
CART
CHECKOUT
UPDATE NOTIFY
Click on Product
• Reveals Product Page
getProduct() {
this.props.db
.collection('products’)
.findOne({ id: this.props.match.params.id })
.then(product => {
this.setState({
product,
notifying:this.props.notify
.includes(this.props.match.params.id)
})
this.props.handleBrowsedProduct(product)
})
.catch(err => {
console.log(err)
})
}
AUTHENTICATION
BROWSE
SELECT ITEM
IN
STOCK?
CART
CHECKOUT
UPDATE NOTIFY
UPDATE NOTIFY
IN
STOCK?
AUTHENTICATION
BROWSE
SELECT ITEM
CART
CHECKOUT
UPDATE NOTIFY
IN
STOCK?
AUTHENTICATION
BROWSE
SELECT ITEM
CART
CHECKOUT
UPDATE NOTIFY
IN
STOCK?
AUTHENTICATION
BROWSE
SELECT ITEM
CART
CHECKOUT
UPDATE NOTIFY
IN
STOCK?
AUTHENTICATION
BROWSE
SELECT ITEM
CART
CHECKOUT
AUTHENTICATION
BROWSE
SELECT ITEM
IN
STOCK?
CART
CHECKOUT
UPDATE NOTIFY
AUTHENTICATION
BROWSE
SELECT ITEM
IN
STOCK?
CART
CHECKOUT
UPDATE NOTIFY
Event Subscriptions
in App Structure
• Event Subscriptions are a
top-level Stitch asset
• Can be constructed in the UI
or with the CLI
Event Subscriptions
in App Structure
• Event Subscriptions are a
top-level Stitch asset
• Can be constructed in the UI
or with the CLI
Change Streams Future Work
• Non-Atlas/Multiple Application per Application
• Events that span Multiple MongoDB Instances
• Timed Events
• Execute on a certain Interval
• Execute at a given time
• For now, can use Webhooks with CRON service
What do change events look like?
Event Subscription (Trigger) Event Source (Actual event document)
Example Trigger
Name of the Trigger
Type: DATABASE, AUTHENTICATION
Type of Operation to Trigger
Database
Collection
Match Criteria
Full Document or Partial
Function to be called on Trigger
Example Trigger Event Document
Type of Database Operation
Full Document impacted by operation
Database
Collection

MongoDB World 2018: Ch-Ch-Ch-Ch-Changes: Taking Your Stitch Application to the Next Level With Triggers

  • 1.
    Ch-ch-ch-ch-changes Taking Your StitchApplication to the Next Level with Stitch Triggers
  • 2.
    Industry Trends –Leading to Stitch 1370 1990
  • 3.
    Industry Trends –Leading to Stitch 19901370 1997
  • 4.
    Industry Trends –Leading to Stitch 19971990 2001
  • 5.
    Industry Trends –Leading to Stitch 20011997 2007
  • 6.
    Industry Trends –Leading to Stitch 20072001 2014
  • 7.
    Industry Trends –Leading to Stitch 20142007 2018
  • 8.
    Industry Trends –Leading to Stitch 20182014
  • 9.
    Intelligent Operational DataPlatform Best way to work with data Intelligently put data where you want it Freedom to run anywhere ● MongoDB Stitch ● MongoDB Server 4.0 ● MongoDB Compass ● MongoDB Charts ● MongoDB Mobile ● MongoDB Server 4.0 ● MongoDB Atlas ● Ops Manager 4.0 ● Free Cloud Monitoring
  • 10.
    The evolution ofMongoDB 3.0 3.2 Document Validation $lookup Fast Failover Simpler Scalability Aggregation ++ Encryption At Rest In-Memory Storage Engine BI Connector MongoDB Compass APM Integration Profiler Visualization Auto Index Builds Backups to File System Doc-Level Concurrency Compression Storage Engine API ≤50 replicas Auditing ++ Ops Manager Linearizable reads Intra-cluster compression Views Log Redaction Linearizable Reads Graph Processing Decimal Collations Faceted Navigation Zones ++ Aggregation ++ Auto-balancing ++ ARM, Power, zSeries BI & Spark Connectors ++ Compass ++ Hardware Monitoring Server Pool LDAP Authorization Encrypted Backups Cloud Foundry Integration 3.4 3.6 Change Streams Retryable Writes Expressive Array Updates Query Expressivity Causal Consistency Consistent Sharded Sec. Reads Compass Community Ops Manager ++ Query Advisor Schema Validation End to End Compression IP Whitelisting Default Bind to Localhost Sessions WiredTiger 1m+ Collections MongoDB BI Connector ++ Expressive $lookUp R Driver Atlas Cross Region Replication Atlas Auto Storage Scaling 4.0 Multi-Document ACID Transactions Atlas Global Sharding Atlas HIPAA Atlas LDAP Atlas Audit Atlas Encrypted Storage Engine Atlas AWS Backup Snapshots Snapshot Reads Agg Pipeline Type Conversions 40% Faster Shard Migrations Non-Blocking Secondary Reads SHA-2 TLS 1.1+ Compass Agg Pipeline Builder Compass Export to Code Charts Beta Monitoring Cloud Service Ops Manager K8s & OpenShift MongoDB Stitch GA MongoDB Mobile (Private Beta)
  • 11.
    “We set outto build a database that we would want to use, so that whenever developers wanted to build an application, they could focus on the application, not on working around the database.” - Eliot Horowitz
  • 12.
    Stitch Functions Stitch isa collection of servers that process application requests Requests: • Single actions for Database or Services • Or executing a Stitch Function • Integrated with Stitch’s rules Functions: • Scalable, hosted JavaScript (ES6) Functions • Integrated with application context • User, Request, Services, Values, etc.
  • 13.
    MongoDB Query Language+ Native DriversIntegrated Rules Functions3rd Party Services Native SDKs (JavaScript, Android, iOS) Rest API
  • 19.
    When do wewant functions to execute? AUTHENTICATION WEBHOOK/ INTEGRATION DATABASE EVENTS 3 Contexts Where Functions Execute 1 2 3
  • 20.
    When do wewant functions to execute? client.executeFunction(…)client.loginWithCredential(…) AUTHENTICATION WEBHOOK/ INTEGRATION DATABASE EVENTS 1 2 3
  • 21.
    When do wewant functions to execute? Incoming Webhooks (HTTP, Twilio, Github) AUTHENTICATION WEBHOOK/ INTEGRATION DATABASE EVENTS 1 2 3
  • 22.
    When do wewant functions to execute? Database Events Collection.insertOne(…) AUTHENTICATION WEBHOOK/ INTEGRATION DATABASE EVENTS 1 2 3
  • 23.
    Responding to DatabaseChanges in the Past
  • 24.
    Responding to DatabaseChanges in the Past Before Change Streams: { "ts" : Timestamp(1526593721, 1), "t" : NumberLong(1), "h" : NumberLong("-2175450131096164196"), "v" : 2, "op" : "n", "ns" : "", "wall" : ISODate("2018-05-17T21:48:41.962Z"), "o" : { "msg" : "periodic noop" } } { "ts" : Timestamp(1526593731, 1), "t" : NumberLong(1), "h" : NumberLong("4757850941625744949"), "v" : 2, "op" : "n", "ns" : "", "wall" : ISODate("2018-05-17T21:48:51.962Z"), "o" : { "msg" : "periodic noop" } }
  • 25.
    OPLOGOPLOG OPLOG Application W db.users.insert({,,,}) PRIMARY SECONDARYSECONDARY db.users.insert({,,,}) A C K db.users.insert({,,,}) W db.users.update({,,,}) db.users.update({,,,}) A C K db.users.update({,,,}) What’s an Oplog? OUT OF DATE OUT OF DATE
  • 26.
    OPLOGOPLOG OPLOG Application W db.users.insert({,,,}) PRIMARY SECONDARYSECONDARY db.users.insert({,,,}) A C K db.users.insert({,,,}) W db.users.update({,,,}) db.users.update({,,,}) A C K db.users.update({,,,}) Monitor R R If (insert) { aws.s3.insertObject(…); } Before ChangeStreams and Triggers TAILING THE OPLOG
  • 27.
    Setting up aChange Stream (NodeJS)
  • 28.
  • 29.
    Traditional Database Triggers •Triggers are: • Code executed when a certain database event occurs • Executed within the Database • Trigger uses include: • Enforcing Referential/Business logic • Auditing and Logging • Security/Validation Logic • Maintaining Replication
  • 30.
    Stitch Coordinates ChangeStreams 1. Sets-up Change Stream 2. Maintains Streams over time 3. Passes Change Events to Coordinator Stitch Infrastructure Stitch Connection Event Coordinator CS 1 CS 3 CS 2
  • 31.
    Adding Triggers toStitch Reduce Time to Market Functions as a Service: No waiting on infrastructure Reduce boilerplate: Save code Cross-Platform: Develop once Existing Apps Untouched: No reverse- engineering Reduce Ops costs Serverless: Zero wasted capacity Payment: No up-front cost Capacity on Demand: No need to over- provision in advance Secure by Default: Less operational effort Reduce Dev Effort Backend Integration Built in: No generic backend code HA & Scalability Built in: Hard and time consuming to build Orchestrate Services: Reuse what's out there Faster Cheaper Easier
  • 32.
    Event Driven Life-cycle PendingSucceededOwned Failed Dead Event is … Waiting for a ConsumerAttempting ExecutionCompletedAwaiting RetriesRe-Attempting ExecutionUnable to Execute (Trigger Disabled)
  • 33.
  • 36.
    Event-driven Functions forCommunication FUNCTION Database Update Text/E-mail EVENT Push Notification
  • 37.
    Event-driven Functions forData Propagation FUNCTION EVENT Database Operation
  • 38.
    Stitch Ensures CorrectExecution Database Events are: • Generated by matching operations ({match: { …}}) • Fully Ordered 1. Event is Generated 2. Event added to Job Queue 3. Event waits for blocking Events to complete 4. Event waits for consumer to attempt execution
  • 39.
  • 40.
    What is theSwagstore Standard Retail Store: • Browse Items • Add items to cart • Checkout • Request Notification for Restock Additional Features: • Update/text on item re-stock • Shipping text/e-mail notification Stitch Features • Functions • Triggers • 3rd Party Services
  • 41.
    Inventory Updates FUNCTION EVENT { ID: XXXXX SIZE:M COUNT: 100 } { ID: XXXXX DESC: “Men’s T-shirt” SM_COUNT: 12 MED_COUNT: 100 LG_COUNT: 23 }
  • 42.
  • 43.
  • 46.
    AUTHENTICATION BROWSE SELECT ITEM IN STOCK? CART CHECKOUT UPDATE NOTIFY Initialization •Before we leverage any features of Stitch, we must initialize our API with the Stitch App Key import React from 'react' import ReactDOM from 'react-dom' import './index.css' import App from './App' import registerServiceWorker from './registerServiceWorker' import { BrowserRouter } from 'react-router-dom' import { StitchClientFactory } from 'mongodb-stitch' const appId = 'ecommercechatbot-glwkl' let stitchClientPromise = StitchClientFactory.create(appId) stitchClientPromise.then(stitchClient => { let db = stitchClient.service('mongodb', 'mongodb- atlas').db('swagstore') let props = { stitchClient, db } ReactDOM.render( <BrowserRouter> <App {...props} /> </BrowserRouter>, document.getElementById('root') ) registerServiceWorker() })
  • 47.
    AUTHENTICATION BROWSE SELECT ITEM IN STOCK? CART CHECKOUT UPDATE NOTIFY Authentication •Leverage integration with Google Sign-in to authenticate the user and set the user’s profile context.
  • 48.
    AUTHENTICATION BROWSE SELECT ITEM IN STOCK? CART CHECKOUT UPDATE NOTIFY Browse •Leverage SDK db.collection() to access SWAG-STORE database and products collection import React, { Component } from 'react' import { ProductDeck } from '../components/Products' class ProductsPage extends Component { constructor(props) { super(props) this.state = { products: [] } } getProducts() { this.props.db .collection('products’) .find({ category: this.props.match.params.category }) .execute() .then(products => { this.setState({ products }) }) .catch(err => { console.log(err) }) } componentDidMount() { this.getProducts() } componentDidUpdate(prevProps, prevState, snapshot) { if (this.props.match.params.category !== prevProps.match.params.category) {
  • 49.
    AUTHENTICATION BROWSE SELECT ITEM IN STOCK? CART CHECKOUT UPDATE NOTIFY Clickon Product • Reveals Product Page getProduct() { this.props.db .collection('products’) .findOne({ id: this.props.match.params.id }) .then(product => { this.setState({ product, notifying:this.props.notify .includes(this.props.match.params.id) }) this.props.handleBrowsedProduct(product) }) .catch(err => { console.log(err) }) }
  • 50.
  • 51.
  • 52.
  • 53.
  • 54.
  • 55.
  • 56.
  • 57.
    Event Subscriptions in AppStructure • Event Subscriptions are a top-level Stitch asset • Can be constructed in the UI or with the CLI
  • 58.
    Event Subscriptions in AppStructure • Event Subscriptions are a top-level Stitch asset • Can be constructed in the UI or with the CLI
  • 59.
    Change Streams FutureWork • Non-Atlas/Multiple Application per Application • Events that span Multiple MongoDB Instances • Timed Events • Execute on a certain Interval • Execute at a given time • For now, can use Webhooks with CRON service
  • 61.
    What do changeevents look like? Event Subscription (Trigger) Event Source (Actual event document)
  • 62.
    Example Trigger Name ofthe Trigger Type: DATABASE, AUTHENTICATION Type of Operation to Trigger Database Collection Match Criteria Full Document or Partial Function to be called on Trigger
  • 63.
    Example Trigger EventDocument Type of Database Operation Full Document impacted by operation Database Collection