In this presentation I will be covering BetterCloud's approach to creating and finding patterns for complex authentication and identifying key performance and correctness metrics for ensuring API stability. We'll take a deep dive into BetterCloud's cost benefit analysis for migrating automated tests to Postman monitors and go over the challenges and successes faced along the way.

2. D A V I D E S P O S I T O
P L A T F O R M A R C H I T E C T
Migrating Automation Tests to Postman
Monitors and ROI

3. Who am I?
espo@bettercloud.com

4. SaaS Provider
e.g. Slack, Google, Okta, Zendesk
API
Advanced Search
Automation
Bulk Actions
Auditing
Fine Grained Access Controls
Derived Event Feeds
Real Time Security Monitoring
DLP
SaaS Admin Decorator Pattern

5. ~700
Internal API’s
BetterCloud in 2019
100
Engineers
12Integrations
320
Services
1-5 billion
REST calls made
per day

6. APIs at BetterCloud
● 3rd Party
● API Provider
● Internal APIs
● API Ops
SaaS Provider
E.g. Google
Internal Services
Micro1 Micro2
Micro3
PlatformAPI
Customers
& Partners
API Ops

7. How to measure API’s
Performance Correctness

8. How to measure API’s
Performance
Resilience
How does the system perform
under load?
Sampling
How does the system perform in
average or ideal conditions?
Correctness

9. How to measure API’s
Performance
Resilience
How does the system perform
under load?
Sampling
How does the system perform in
average or ideal conditions?
Correctness
Smoke Testing
Does anything explode?
Feature Testing
Does the system adhere to the
contract?

10. How to measure API’s
Performance
Resilience
How does the system perform
under load?
Sampling
How does the system perform in
average or ideal conditions?
Correctness
Smoke Testing
Does anything explode?
Feature Testing
Does the system adhere to the
contract?

11. APIs at BetterCloud
● 3rd Party
● API Provider
● Internal APIs
● API Ops
SaaS Provider
E.g. Google
Internal Services
Micro1 Micro2
Micro3
PlatformAPI
Customers
& Partners
API Ops

12. BetterCloud as an API
Provider
● Tenant specific credentials
● Sampling
○ latency < 500 ms
● Smoke
○ check headers
● Feature
○ GraphQL queries
Internal Services
Micro1
PlatformAPI
Customers
& Partners

13. Internal APIs at
BetterCloud
Internal Services
Micro1 Micro2
Micro3
PlatformAPI
● HMAC Authentication
● Sampling
○ latency < 500ms
● Smoke
○ check headers
● Feature

14. #1 Productivity
#2 Historical Trends
#3 Out of the Box Integrations

15. Psst… Don’t tell...
Scheduled Webhooks as a Service
● Automated Testing (as Designed)
○ Track trends
● Automated Operations
○ Non-critical metrics and alerts, e.g. Kafka
partition skew
● Business Insights
○ Data aggregations
○ Abusive customer alerts

16. What’s Next?
Monitor vs. Newman

18. espo@bettercloud.com