I'm building Convox, an open-source platform that expertly configures AWS to run and manage any application. In doing this I have discovered a minimal set of 11 cloud infrastructure services that form an architecture that supports any application. Here I'll introduce the 11 service types that support our secure computing, application workload, and operational needs. I'll show how these services are used together to manage a secure computing cluster that can build Docker images for an app, roll out new images to production with zero downtime, and scale to handle any load. Finally we'll look at the cloud computing landscape to see how all the cloud providers and software projects compare with offering these 11 service types.

1. Minimum Viable
Infrastructure
Noah Zoschke
noah@convox.com
@nzoschke
LA DevOps
09-26-2016

2. Goal
• Take a web app
• Run it in the cloud forever
• Sleep well every night
10000 Year Clock from http://www.mawer.com/insights/articles/the-10000-year-clock/

3. Strategy
• Developer provides app code
and manifest
• Community develops
strategy and architecture to
run it
• Providers operate cloud
services to support it
• Software automates setup,
monitoring and maintenance
Leroy Buffington skyscraper (1888) from http://patentroom.com/architecture

4. Single Node Architecture
Well Understood
Secure Compute
• CPU
• Memory
• Network
• Crypto
┌────────────────────────┐
┌──┤ HAProxy ├─┐
│┌─┤ Load Balancer ├┐│
││ └────────────────────────┘││
││ ┌──────┐┌──────┐┌──────┐ ││
││ │apache││ php ││python│ ││
││ └──────┘└──────┘└──────┘ ││
││ iptables ││
││ Network Security ││
│└───────────────────────────┘│
│┌───────┐┌────────┐┌────────┐│
││syslogd││collectd││Postgres││
││ Log ││ Metric ││Database││
│└───────┘└────────┘└────────┘│
│┌───────────────────────────┐│
││ ┌──────────┐ ┌──────────┐ ││
││ │ key.pem │ │ /var/ │ ││
││ │ Crypto │ │ Data │ ││
││ └──────────┘ └──────────┘ ││
││ ext4 ││
││ File System ││
│└───────────────────────────┘│
│ Ubuntu OS │
└─────────────────────────────┘
┌───┐┌───┐┌───┐┌───┐
│CPU││RAM││ETH││SSD│
└───┘└───┘└───┘└───┘
App Workload
• Package
• Config
• Data
• Proxy
Visibility
• Logs
• Metrics
• Events

5. Single Node Challenges
Well Understood
• OS and Software Versions
• Single Point of Failure
• Coarse Scaling
• Constant Pricing
┌────────────────────────┐
┌──┤ HAProxy ├─┐
│┌─┤ Load Balancer ├┐│
││ └────────────────────────┘││
││ ┌──────┐┌──────┐┌──────┐ ││
││ │apache││ php ││python│ ││
││ └──────┘└──────┘└──────┘ ││
││ iptables ││
││ Network Security ││
│└───────────────────────────┘│
│┌───────┐┌────────┐┌────────┐│
││syslogd││collectd││Postgres││
││ Log ││ Metric ││Database││
│└───────┘└────────┘└────────┘│
│┌───────────────────────────┐│
││ ┌──────────┐ ┌──────────┐ ││
││ │ key.pem │ │ /var/ │ ││
││ │ Crypto │ │ Data │ ││
││ └──────────┘ └──────────┘ ││
││ ext4 ││
││ File System ││
│└───────────────────────────┘│
│ Ubuntu OS │
└─────────────────────────────┘
┌───┐┌───┐┌───┐┌───┐
│CPU││RAM││ETH││SSD│
└───┘└───┘└───┘└───┘

6. Service Oriented Architecture
Evolving Best Practices
Secure Compute
• CPU/Memory → VM
• Network → VPC
• Crypto → Crypto
App Workload
• Package → Image, Container
• Config → Crypto, Blob
• Data → Database
• Proxy → Load Balancer
Visibility
• Logs → Logs
• Metrics → Metrics
• Events → KV, Blob

7. Minimum Viable Infrastructure
Simple Monolith → Simple SOA
┌────────────────────────┐
┌──┤ HAProxy ├─┐
│┌─┤ Load Balancer ├┐│
││ └────────────────────────┘││
││ ┌──────┐┌──────┐┌──────┐ ││
││ │apache││apache││python│ ││
││ └──────┘└──────┘└──────┘ ││
││ iptables ││
││ Network Security ││
│└───────────────────────────┘│
│┌───────┐┌────────┐┌────────┐│
││syslogd││collectd││Postgres││
││ Log ││ Metric ││Database││
│└───────┘└────────┘└────────┘│
│┌───────────────────────────┐│
││ ┌──────────┐ ┌──────────┐ ││
││ │ key.pem │ │ /var/ │ ││
││ │ Crypto │ │ Data │ ││
││ └──────────┘ └──────────┘ ││
││ ext4 ││
││ File System ││
│└───────────────────────────┘│
│ Linux OS │
└─────────────────────────────┘
┌───┐┌───┐┌───┐┌───┐
│CPU││RAM││ETH││SSD│
└───┘└───┘└───┘└───┘
┌────────────────────────────────────┐
┌┤ Load Balancer ├┐
│└────────────────────────────────────┘│
│┌─────────────────┐┌─────────────────┐│
││┌─────┐┌────────┐││ ┌─────┐ ││
│││web 1││worker 1│││ │web 2│ ││
││└─────┘└────────┘││ └─────┘ ││
││ VM 1 ││ VM 2 ││
│└─────────────────┘└─────────────────┘│
│ ┌────────┐ │
│ │Database│ │
│ └────────┘ │
│ VPC │
└──────────────────────────────────────┘
┌──────┐┌─────┐┌───┐┌──────┐┌──┐┌────┐
│Crypto││Image││Log││Metric││KV││Blob│
└──────┘└─────┘└───┘└──────┘└──┘└────┘

8. SOA Advantages
Somewhat Understood
• Service Level Agreements
• Versioned APIs
• Independent Scaling
• Utility Pricing
┌────────────────────────────────────┐
┌┤ Load Balancer ├┐
│└────────────────────────────────────┘│
│┌─────────────────┐┌─────────────────┐│
││┌─────┐┌────────┐││ ┌─────┐ ││
│││web 1││worker 1│││ │web 2│ ││
││└─────┘└────────┘││ └─────┘ ││
││ VM 1 ││ VM 2 ││
│└─────────────────┘└─────────────────┘│
│ ┌────────┐ │
│ │Database│ │
│ └────────┘ │
│ VPC │
└──────────────────────────────────────┘
┌──────┐┌─────┐┌───┐┌──────┐┌──┐┌────┐
│Crypto││Image││Log││Metric││KV││Blob│
└──────┘└─────┘└───┘└──────┘└──┘└────┘

9. SOA Challenges
Somewhat Understood
• Competing providers, services
and software solutions
• Configuration
• Customization
• Impedance Mismatch
┌────────────────────────────────────┐
┌┤ Load Balancer ├┐
│└────────────────────────────────────┘│
│┌─────────────────┐┌─────────────────┐│
││┌─────┐┌────────┐││ ┌─────┐ ││
│││web 1││worker 1│││ │web 2│ ││
││└─────┘└────────┘││ └─────┘ ││
││ VM 1 ││ VM 2 ││
│└─────────────────┘└─────────────────┘│
│ ┌────────┐ │
│ │Database│ │
│ └────────┘ │
│ VPC │
└──────────────────────────────────────┘
┌──────┐┌─────┐┌───┐┌──────┐┌──┐┌────┐
│Crypto││Image││Log││Metric││KV││Blob│
└──────┘└─────┘└───┘└──────┘└──┘└────┘

10. Virtual Machine (VM)
Secure Compute
A Virtual Machine (VM)
service provides CPU,
Memory, Networking and a
server Operating System.
┏━━━━━━━━━━━━━━━━━┓
┃ ┃
┃ ┃
┃ ┃
┃ VM ┃
┗━━━━━━━━━━━━━━━━━┛

11. Virtual Private Cloud (VPC)
Secure Compute
A Virtual Private Cloud (VPC)
service provides private
networking.
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ ┃
┃ ┃
┃ ┌─────────────────┐ ┃
┃ │ │ ┃
┃ │ │ ┃
┃ │ │ ┃
┃ │ VM │ ┃
┃ └─────────────────┘ ┃
┃ ┃
┃ ┃
┃ VPC ┃
┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛

12. Cryptography
Secure Compute
A Cryptography (Crypto)
service provides a way to
create, import and rotate an
unguessable key for
encrypting data, and
provides a way to get and
audit access to the key for
decrypting data.
┌──────────────────────────────────────┐
│ │
│ │
│ ┌─────────────────┐ │
│ │ │ │
│ │ │ │
│ │ │ │
│ │ VM │ │
│ └─────────────────┘ │
│ │
│ │
│ VPC │
└──────────────────────────────────────┘
┏━━━━━━┓
┃Crypto┃
┗━━━━━━┛

13. Image
App Workload
An Image service provides a
private place to push, store
and pull binary application
and dependency data.
┌──────────────────────────────────────┐
│ │
│ │
│ ┌─────────────────┐ │
│ │ │ │
│ │ │ │
│ │ │ │
│ │ VM │ │
│ └─────────────────┘ │
│ │
│ │
│ VPC │
└──────────────────────────────────────┘
┌──────┐┏━━━━━┓
│Crypto│┃Image┃
└──────┘┗━━━━━┛

14. Container
App Workload
A Container service provides
a way to run many
specialized process types,
defined by Images and
commands, on fewer
homogeneous VMs.
┌──────────────────────────────────────┐
│ │
│ │
│┌─────────────────┐┌─────────────────┐│
││┏━━━━━┓┏━━━━━━━━┓││ ┏━━━━━┓ ││
││┃web 1┃┃worker 1┃││ ┃web 2┃ ││
││┗━━━━━┛┗━━━━━━━━┛││ ┗━━━━━┛ ││
││ VM 1 ││ VM 2 ││
│└─────────────────┘└─────────────────┘│
│ │
│ │
│ VPC │
└──────────────────────────────────────┘
┌──────┐┌─────┐
│Crypto││Image│
└──────┘└─────┘

15. Load Balancer
App Workload
A Load Balancer service
provides a single, stable
hostname that accepts
network requests and
proxies them to one or more
healthy Containers.
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┌┫ Load Balancer ┣┐
│┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛│
│ │
│┌─────────────────┐┌─────────────────┐│
││┌─────┐┌────────┐││ ┌─────┐ ││
│││web 1││worker 1│││ │web 2│ ││
││└─────┘└────────┘││ └─────┘ ││
││ VM 1 ││ VM 2 ││
│└─────────────────┘└─────────────────┘│
│ │
│ │
│ VPC │
└──────────────────────────────────────┘
┌──────┐┌─────┐
│Crypto││Image│
└──────┘└─────┘

16. Database
App Workload
A Database service provides
a single network hostname
that is used to save, update
and delete application data
records concurrently by one
or more Containers.
┌────────────────────────────────────┐
┌┤ Load Balancer ├┐
│└────────────────────────────────────┘│
│┌─────────────────┐┌─────────────────┐│
││┌─────┐┌────────┐││ ┌─────┐ ││
│││web 1││worker 1│││ │web 2│ ││
││└─────┘└────────┘││ └─────┘ ││
││ VM 1 ││ VM 2 ││
│└─────────────────┘└─────────────────┘│
│ ┏━━━━━━━━┓ │
│ ┃Database┃ │
│ ┗━━━━━━━━┛ │
│ VPC │
└──────────────────────────────────────┘
┌──────┐┌─────┐
│Crypto││Image│
└──────┘└─────┘

17. Log
Visibility
A Log service provides a
place to send ordered text
from app Containers’ stdout
and stderr streams, so all the
application events can be
tailed in real-time and
searched later.
┌────────────────────────────────────┐
┌┤ Load Balancer ├┐
│└────────────────────────────────────┘│
│┌─────────────────┐┌─────────────────┐│
││┌─────┐┌────────┐││ ┌─────┐ ││
│││web 1││worker 1│││ │web 2│ ││
││└─────┘└────────┘││ └─────┘ ││
││ VM 1 ││ VM 2 ││
│└─────────────────┘└─────────────────┘│
│ ┌────────┐ │
│ │Database│ │
│ └────────┘ │
│ VPC │
└──────────────────────────────────────┘
┌──────┐┌─────┐┏━━━┓
│Crypto││Image│┃Log┃
└──────┘└─────┘┗━━━┛

18. Metric
Visibility
A Metric service provides a
place to save numerical data
from apps and underlying
cloud services so
operational properties of the
entire system can be
aggregated, analyzed and
graphed in real-time and
reviewed later.
┌────────────────────────────────────┐
┌┤ Load Balancer ├┐
│└────────────────────────────────────┘│
│┌─────────────────┐┌─────────────────┐│
││┌─────┐┌────────┐││ ┌─────┐ ││
│││web 1││worker 1│││ │web 2│ ││
││└─────┘└────────┘││ └─────┘ ││
││ VM 1 ││ VM 2 ││
│└─────────────────┘└─────────────────┘│
│ ┌────────┐ │
│ │Database│ │
│ └────────┘ │
│ VPC │
└──────────────────────────────────────┘
┌──────┐┌─────┐┌───┐┏━━━━━━┓
│Crypto││Image││Log│┃Metric┃
└──────┘└─────┘└───┘┗━━━━━━┛

19. Key-Value (KV)
Visibility
A Key-Value (KV) service
provides a way to save small
amounts of structured data in
a highly-available (HA)
fashion while still being easy
to query for reporting
purposes.
┌────────────────────────────────────┐
┌┤ Load Balancer ├┐
│└────────────────────────────────────┘│
│┌─────────────────┐┌─────────────────┐│
││┌─────┐┌────────┐││ ┌─────┐ ││
│││web 1││worker 1│││ │web 2│ ││
││└─────┘└────────┘││ └─────┘ ││
││ VM 1 ││ VM 2 ││
│└─────────────────┘└─────────────────┘│
│ ┌────────┐ │
│ │Database│ │
│ └────────┘ │
│ VPC │
└──────────────────────────────────────┘
┌──────┐┌─────┐┌───┐┌──────┐┏━━┓
│Crypto││Image││Log││Metric│┃KV┃
└──────┘└─────┘└───┘└──────┘┗━━┛

20. Blob
Visibility
A Blob service provides a
way to save and retrieve
large amounts unstructured
data in a HA fashion.
┌────────────────────────────────────┐
┌┤ Load Balancer ├┐
│└────────────────────────────────────┘│
│┌─────────────────┐┌─────────────────┐│
││┌─────┐┌────────┐││ ┌─────┐ ││
│││web 1││worker 1│││ │web 2│ ││
││└─────┘└────────┘││ └─────┘ ││
││ VM 1 ││ VM 2 ││
│└─────────────────┘└─────────────────┘│
│ ┌────────┐ │
│ │Database│ │
│ └────────┘ │
│ VPC │
└──────────────────────────────────────┘
┌──────┐┌─────┐┌───┐┌──────┐┌──┐┏━━━━┓
│Crypto││Image││Log││Metric││KV│┃Blob┃
└──────┘└─────┘└───┘└──────┘└──┘┗━━━━┛

21. MVI Apps

22. Rails App
Production Grade┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┐
web:
│ build: . │
labels:
│ - convox.port.443.protocol=tls │
- convox.port.443.proxy=true
│ links: │
- db ┌───────────────────┐
│ - redis │ │ TLS Load Balancer │
ports: ┌┤https + websockets ├┐ ┌─────────┐ ┌─────────┐
│ - 80:4000 │ │└────────┬─┬────────┘│ │┌───────┐│ │┌───────┐│
- 443:4001 │ ┌─────┐ │ │ ┌─────┐ │ ││ rake ││ ││ rake ││
│ │ ┌ ─ ─ ─ ─ ─ ─ ─ ─ │ │nginx│ │ │ │nginx│ │ ││resque ││ ││resque ││
worker: Rails Image │ │ └─────┘ │ │ └─────┘ │ │└───────┘│ │└───────┘│
│ build: . │ │ │ ┌─────┐ │ │ ┌─────┐ │ │ worker │ │ worker │
command: rake resque work ──────▶ Ubuntu 16.04 OS │──────▶│ │ruby │ │ │ │ruby │ │ │Container│ │Container│
│ │ │ pg, redis gems │ │puma │ │ │ │puma │ │ └─────────┘ └─────────┘
db: + code │ │ └─────┘ │ │ └─────┘ │ ┌─────────┐ ┌─────────┐
│ image: convox/postgres │ └ ─ ─ ─ ─ ─ ─ ─ ─ │ web │ │ web │ │┌───────┐│ │┌───────┐│
labels: │Container│ │Container│ ││ rake ││ ││ rake ││
│ - convox.health.timeout=60 │ └─────────┘ └─────────┘ ││resque ││ ││resque ││
ports: ┌─────────┐ ┌─────────┐ │└───────┘│ │└───────┘│
│ - 5432 │ │Postgres │ │ Redis │ │ worker │ │ worker │
volumes: │Database │ │Database │ │Container│ │Container│
│ - /var/lib/postgresql/data │ └─────────┘ └─────────┘ └─────────┘ └─────────┘
│redis: │
image: convox/redis
│ ports: │
- 6433
└ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┘

23. Challenges
• Containers, Images, Load balancers and data services only
• Just like Heroku / Twelve-Factor, but more ports and
protocols
• Constraints are good
• Orchestration
• Rolling deploys
• Health Checks
• Resilience is good

24. MVI Systems

25. Build SystemAPI docker build
Load Balancer Blob KV Container daemon Container Image
┌┐ ┌┐ ┌┐ ┌┐ ┌┐ ┌┐ ┌┐
POST code ││ ││ ││ ││ ││ ││ ││
──────────────▶││ ││ ││ ││ ││ ││ ││
││ ││ ││ ││ ││ ││ ││
││ PUT obj ││ ││ ││ ││ ││ ││
││──────────────▶││ ││ ││ ││ ││ ││
││ ││ ││ ││ ││ ││ ││
││ PUT item ││ ││ ││ ││ ││ ││
││───────────────┼┼──────────────▶││ ││ ││ ││ ││
││ ││ ││ ││ ││ ││ ││
││ RUN container ││ ││ ││ RUN container ││ ││ ││
││───────────────┼┼───────────────┼┼──────────────▶││─ ─ ─ ─ ─ ─ ─ ▶││ ││ ││
││ ││ ││ ││ ││ ││ ││
GET logs ││ ││ ││ ││ ││ ││ ││
──────────────▶││─ ─ ─ ─ ─ ─ ─ ─│┼ ─ ─ ─ ─ ─ ─ ─ ┼│─ ─ ─ ─ ─ ─ ─ ─│┼ ─ ─ ─ ─ ─ ─ ─▶││ ││ ││
││ ││ ││ ││ ││ ││ ││
││ ││ ││ ││ ││ GET obj ││ ││
││ ││◀──────────────┼┼───────────────┼┼───────────────┼┼───────────────││ ││
││ ││ ││ ││ ││ ││ ││
││ ││ ││ ││ ││ BUILD ││ ││
││ ││ ││ ││ ││◀──────────────││ ││
││ ││ ││ ││ ││ ││ ││
││ ││ ││ ││ ││ TAG ││ ││
││ ││ ││ ││ ││◀──────────────││ ││
││ ││ ││ ││ ││ ││ ││
││ ││ ││ ││ ││ PUSH ││ ││
││ ││ ││ ││ ││◀──────────────││ ││
││ ││ ││ ││ ││─ ─ ─ ─ ─ ─ ─ ─│┼ ─ ─ ─ ─ ─ ─ ─▶││
││ ││ ││ ││ ││ ││ ││
││ ││ ││ ││ ││ UPDATE item ││ ││
││ ││ ││◀──────────────┼┼───────────────┼┼───────────────││ ││
││ ││ ││ ││ ││ ││ ││
GET result ││ ││ ││ ││ ││ ││ ││
──────────────▶││─ ─ ─ ─ ─ ─ ─ ─│┼ ─ ─ ─ ─ ─ ─ ─▶││ ││ ││ ││ ││
└┘ └┘ └┘ └┘ └┘ └┘ └┘

26. MVI Landscape

27. IaaS Providers
Amazon Google Microsoft
VM EC2 Google Compute Engine Azure Virtual Machines
VPC VPC GCE Firewall Rules Azure Virtual Network
Crypto KMS Azure Key Vault
Image ECR GCE Registry
Container ECS
Google Container Engine
(Kubernetes)
Azure Container Service
(DC/OS or Swarm)
Load Balancer ELB Google Cloud Load Balancing Azure Load Balancer
Database RDS Google Cloud SQL Azure SQL
Log CloudWatch Logs Stackdriver Logging Azure Log Analytics
Metric CloudWatch Metrics Stackdriver Monitoring Azure Monitor
KV DynamoDB Google Cloud Bigtable Azure Tables Storage
Blob S3 Google Cloud Storage Azure Blob Storage
https://cloud.google.com/docs/compare/aws/
https://azure.microsoft.com/en-us/services/

28. Service Providers
VM VPC Crypto Image Container LB DB Log Metric KV Blob
Digital Ocean Heroku CloudFlare
Heroku
Postgres
Sumo logic MongoLab
Rackspace
Cloud Files
Linode CircleCI Compose Loggly DataDog Compose
Rackspace Docker Hub Papertrail Newrelic
Quay Librato

29. Software Projects
VM VPC Crypto Image Container LB DB Log Metric KV Blob
KVM OpenStack Vault
Docker
Distribution
Docker
Swarm
HAProxy Postgres
Elastic
Search
RRD Redis minio
Xen Knox GitLab Kubernetes nginx MySQL Splunk Graphite MongoDB
VMWare Jenkins Mesos Prometheus Cassandra
VirtualBox Nomad CouchDB

30. MVI Open Questions
• Future additions or simplifications?
• Three database types?
• Shared File System
• Lambda
• Anti-patterns?
• Service discovery
• Overlay networking
• Naming Things
• What knobs and gauges?

31. MVI Key Takeaways
• We can support any app or system with 11 component SOA
• Building with MVI services has advantages:
• Off-the-shelf components
• Privacy
• Reliability
• Scale
• Time to first deploy
• Total cost of ownership

32. Thanks!
• Feedback and questions to @nzoschke
• See MVI in action
• https://github.com/convox/rack
• https://github.com/nzoschke/mvi