2. Development
• Problem Definition
• Twitter/Salesforce RPC Stack
• Supporting Infrastructure
Logistics – Setting Up a Service
• Buyout
• External Reviews
• Source Control, Packaging and
Deployment
• Running Environments
• Provisioning
• Logging and Monitoring
• System of Record Integration
• Production
3. Microservice – A Simple Definition
Code module deployed in a stand-alone process.
4. Breaking Up The Monolith
• Code modularization.
• Transaction refactoring.
• Deployment/operational logistics.
6. Supporting Infrastructure
Compute (Docker, Mesos/Aurora at Twitter, AWS).
Hardware provisioning automation.
Analytics stack (event collectors, time series storage, dashboards, alerts).
Log management (Splunk, ELK stack, Hadoop, Spark).
Package registry, deployment tools.
Keys/secrets infrastructure.
CI pipeline and tools.
7. Setting Up a Service
1. Buyout
2. External Reviews
3. Source Control, Packaging and Deployment
4. Running Environments
5. Provisioning
6. Logging and Monitoring
7. System of Record Integration
8. Production
Before you even start…
8. Setting Up a Service - Buyout
Define Scope and Lifetime
• The service has a clear scope and expectations for each planned iteration.
Get Management / Executive Sponsorship
• The service is mentioned in the long term plan, and approved by the management chain.
Team Charter
• The new service aligns with the team’s charter.
Clients
• Someone wants/needs your service.
9. Setting Up a Service – External Reviews
Internal Design Reviews
• Senior engineers review your design.
Security
• Product
• Network and infrastructure
Legal & Compliance
• Use of 3rd party/open source software.
• Legal compliance (copyright, taxation, etc.).
• Infrastructure compliance (PCI, service agreements, etc.)
Design, Compliance, Infrastructure
10. Setting Up a Service – Source, Packaging & Deployment
Source
• Build system (Maven, Bazel, Pants, Buck)
• Source versioning repository (monolithic vs. individual repo, branching).
• Package/lib versioning.
• CD/CI pipeline (automate testing and merging).
• Testing (unit, functional, integration, performance).
Packaging
• Some sort of archive (zip, uber jar, container image).
Deployment
• Package registry
• Rollout orchestration
• Security keys
Hello World Logistics
11. Setting Up a Service – Environments
Local
Development
Staging (w/ or w/o production traffic)
Production
Performance
Public vs. Private Cloud
CI/Test/Automatic Builds
It worked on my machine…
12. Setting Up a Service – Provisioning
Hardware provisioning and automation.
Elastic compute capacity (public/private cloud).
Real-time/offline services bandwidth and capacity (stream/batch event processing, etc.)
Storage - (No-)SQL, data lakes, data warehousing, analytics, indexing.
Getting resources
13. Setting Up a Service – Logging and Monitoring
• Event collectors (system and application).
• Time series and structured log storage, indexing, and querying.
• Dashboards.
• Alerts.
• Stream and batch processing.
Events Infrastructure
14. Setting Up a Service – System of Record Integration
• Bug Tracker
• Site Reliability
• Release Management
• On-call Paging
• Root Cause Analysis
15. Setting Up a Service – Production
• Documentation.
• On-Call Playbooks.
• Deployment Dependencies.
• Patch/Rollout Procedures.
• Sign-Offs.