The document outlines the evolution and architecture of microservices at Squarespace, detailing the transition from monolithic systems to microservices for scalability and reliability. It discusses key pillars such as service discovery, observability, fault tolerance, and the need for efficient monitoring and orchestration using Kubernetes. Additionally, it highlights challenges faced with static infrastructure and emphasizes the adoption of containerization for improved resource management and deployment processes.

2. Part 1: Building the Pillars of
Microservices
Part 2: Containerization and
Orchestration (Kubernetes)
AGENDA

3. Part 1: Building the Pillars
01
The Journey to Microservices
02
Building the Pillars of Microservices

5. Microservices Journey: A Story of Growth
2013: small (< 50 engineers)
build product & grow customer base
whatever works
2014: medium (< 100 engineers)
we have a lot of customers now!
whatever works doesn't work anymore
2016: large (100+ engineers)
architect for scalability and reliability
organizational structures
?: XL (200+ engineers)

6. Challenges with a Monolith
● Reliability
● Performance
● Engineering agility/speed, cross-team coupling
● Engineering time spent fire fighting rather than building new
functionality
What were the increasingly difficult challenges with a
monolith?

7. https://www.squarespace.com/?gclid=<unique-id>
Challenges with a Monolith
Story of an Outage...During the Super Bowl

8. Challenges with a Monolith
● Monitoring typically starts at the edges
○ Think requests in, DB queries out, etc
● What about the guts of the app? How much visibility do you have
there?
● How long does it take you to recover from an issue? Find the cause
and fix the issue?
Challenges with Monitoring/Finding Faults

9. The Journey to Microservices
● Define Pillars: ideas we consider necessary for successful production
microservices
● Implement these pillars as part of our platform
● Reduce boilerplate and reinventing the wheel syndrome
● Service authors get these for free and can focus on their application
domain
Design a Platform for Production, Remove Challenges

10. Pillars
Microservice Framework
HTTP API
Service
Discovery
Software Load
Balancer
Observability Async Client Fault Tolerance
https://engineering.squarespace.com/blog/2017/the-pill
ars-of-squarespace-services

11. Platform Features
Service
Discovery
API
Documentation
Structured
Logging
Metrics &
Dashboards
Distributed
Tracing
Contextual
Information
Alert
Definitions
Standardized
Deployments
Healthchecks
Dynamic
Configuration
Client-Side Load
Balancing
Latency & Fault
Tolerance
Client-Side
Caching
HTTP Request
Builders
Code
Generation
Service
Dashboard
Traffic
Visualization
Server Platform
Client Platform
ToolingTooling

12. Platform Features
Service
Discovery
API
Documentation
Structured
Logging
Metrics &
Dashboards
Distributed
Tracing
Contextual
Information
Alert
Definitions
Standardized
Deployments
Healthchecks
Dynamic
Configuration
Client-Side Load
Balancing
Latency & Fault
Tolerance
Client-Side
Caching
HTTP Request
Builders
Code
Generation
Service
Dashboard
Traffic
Visualization
Async/Reactive
Alert
Management
Log Aggregation

13. Building the Pillars of Microservices
● HTTP + JSON
○ Industry standard. Tons of tools.
● Solid open source Java API server platforms
○ Started with Dropwizard
○ now on Spring Boot (configured to use Jetty and Jersey 2)
Pillar: HTTP APIs

14. Building the Pillars of Microservices
● Swagger (OpenAPI Specification)
● Code generation
○ Swagger spec → models, server API, client
Even Easier APIs

15. Swagger Path Example
paths:
/currency-info:
put:
tags:
- CurrencyInfo
description: "Creates a new {@link CurrencyInfo} resource."
summary: Create a new currency info
operationId: save
parameters:
- name: info
in: body
schema:
$ref: '#/definitions/CurrencyInfo'
responses:
200:
description: ok
schema:
$ref: '#/definitions/CurrencyInfo'

16. Interactive API Documentation

17. Building the Pillars of Microservices
● Services announce themselves, publishing their name and host/port
information
● Started with a simple announcement payload and found that was
enough
● Healthchecks to mark services down
Pillar: Service Discovery

18. Building the Pillars of Microservices
● First: Zookeeper
○ Complicated clients (no HTTP API)
○ Must build discovery on Zookeeper primitives
○ Strong consistency is unnecessary
○ Client heartbeats can’t be expanded upon
○ No great way to support multiple data centers
Service Discovery Systems

19. ● Now: Consul
○ First class discovery support
○ Built in multi-data center support
○ Simple HTTP API
○ Configurable healthchecks
○ key/value store
■ We use for dynamic config and leader election
Building the Pillars of Microservices
Service Discovery Systems

20. DC2DC1
Multi DC with Consul
ConsulConsulConsul ConsulConsulConsul
Service
Announce
Service
Announce
Primary DB Replica DB
Replicate
WAN Gossip
Consistent Set

21. DC2DC1
Multi DC with Consul
ConsulConsulConsul ConsulConsulConsul
Service Service
Primary DB Replica DB
Replicate
Service
Query
?dc=”DC2”
Remote DC forwarding

22. Building the Pillars of Microservices
● Avoid middleware/extra configuration
● Customizable logic
● Connection pooling
● System awareness to increase fault tolerance
● Builds on Netflix Ribbon OSS
Pillar: Software Load Balancers

23. Building the Pillars of Microservices
● Metrics
● Dashboards
● Distributed Tracing
● Structured Logging
● Healthchecks
● Alerts
Pillar: Observability

24. Metrics & Dashboards

25. Distributed Tracing

26. Structured Logging
tail -f /data/logs/taxation-access.log
2017-03-22 07:24:45:026 GMT
thread=jetty-846
contextId=JaOLrH2O
contextSourceType=billing
clientVersion=taxation-client-3.1
level=INFO
class=AccessLog
ip=10.100.101.205
method=GET
uri=/api/1/taxation/rates
queryString=
httpVersion=HTTP/1.1
responseCode=200
responseTimeMs=39

27. Contextual Information
Client
v3.1
Taxation Service
Billing Service
Context IdClient Version
Client Source
Type
JaOLrH2O

28. Building the Pillars of Microservices
● Addresses the Fanout problem, improved latency
● Reactive: RxJava with RxNetty
● Allows greater composition and reuse. Avoid “callback hell”
Pillar: Async Client

29. Fanout Depicted
Client
Service A
Service Z
Application Container
Service B
Service C
Service D

30. Sync Execution
Client
Service A
Service Z
Application Container
Service B
Service C
Service D
1
2
3
4
5
Total Latency = A + B + C + D + Z

31. Async Execution
Client
Service A
Service Z
Application Container
Service B
Service C
Service D
1
2
2
2
1
Total Latency = max(A, Z)
A = max(B, C, D) + A’s latency

32. Building the Pillars of Microservices
● Circuit breakers
● Retry logic
○ Much easier to implement w/ RxJava
● Timeouts
● Fallbacks (cached or static values)
● Netflix Hystrix
Pillar: Fault Tolerance

33. Fault Tolerance
Service B
Service A
Service C
Service A Client
Service B Client
Service C Client
User
Request
Application Container

34. Fault Tolerance
Service B
Service A
Service C
Service A Client
10 Threads
Service B Client
5 Threads
Service C Client
5 Threads
User
Request
Fail fast, fail silent, or fallback
Application Container

35. Pillars
Microservice Framework
HTTP API
Service
Discovery
Software Load
Balancer
Observability Async Client Fault Tolerance
https://engineering.squarespace.com/blog/2017/the-pill
ars-of-squarespace-services

36. Building the Pillars of Microservices
● Entirely Async Systems
○ Async servers, Streaming, gRPC, Netty
● Distributed task management
○ Serverless computing
● Easier/better alert definition and management
● Better tooling to create and deploy services
Future Work

37. Part 2: Containerization & Kubernetes Orchestration
01
The problem with static infrastructure
02
Kubernetes in a datacenter?
03
Challenges

38. Containerization & Kubernetes Orchestration
● Engineering org grows...
● More services…
● More infrastructure to spin up…
● Ops becomes a blocker...
Stuck in a loop

39. Containerization & Kubernetes Orchestration
● Difficult to find resources
● Slow to provision and scale
● Already have discovery!
● Metrics system must support short lived metrics
● Alerts are usually per instance
Static infrastructure and microservices do not mix!

40. Traditional Provisioning Process
● Pick ESX with available resources
● Pick IP
● Register host to Cobbler
● Register DNS entry
● Create new VM on ESX
● PXE boot VM and install OS and base configuration
● Install system dependencies (LDAP, NTP, CollectD, Sensu…)
● Install app dependencies (Java, FluentD/Filebeat, Consul,
Mongo-S…)
● Install the app
● App registers with discovery system and begins receiving traffic

41. Kubernetes Provisioning Process
● kubectl apply -f app.yaml

42. Containerization & Kubernetes Orchestration
● Provisioning/Scaling: Kubernetes
● Monitoring: Prometheus
● Alerting: AlertManager
● Discovery: Consul + Kubernetes
● Decentralization
So how do we make this magic work?

43. Kubernetes in a datacenter?

44. Kubernetes Architecture

45. Spine and Leaf Layer 3 Clos Topology
● Each leaf switch represents a Top-of-Rack switch (ToR)
● All work is performed at the leaf switch
● Each leaf switch is separate Layer 3 domain
● Each leaf is a separate BGP domain (ASN)
● No Spanning Tree Protocol issues seen in L2 networks (convergence
time, loops)
Leaf Leaf Leaf Leaf
Spine Spine

46. Spine and Leaf Layer 3 Clos Topology
● Simple to understand
● Easy to scale
● Predictable and consistent latency (hops = 2)
● Allows for Anycast IPs
Leaf Leaf Leaf Leaf
Spine Spine

47. Calico Networking
● No network overlay required
● Communicates directly with existing L3 mesh network
● BGP Peering with Top of Rack switch
● Calico supports Kubernetes NetworkPolicy firewall rules

48. Monitoring

49. ● Graphite does not scale well with ephemeral instances
● Easy to have combinatoric explosion of metrics
Traditional Monitoring & Alerting
● Application and system alerts are tightly coupled
● Difficult to create alerts on SLAs
● Difficult to route alerts

50. Kubernetes Monitoring & Alerting

51. Kubernetes Monitoring & Alerting

52. Kubernetes Monitoring & Alerting

53. Microservice Pod Definition
resources:
requests:
cpu: 2
memory: 4Gi
limits:
cpu: 2
memory: 4Gi
Microservice Pod
Java Microservice
fluentd consul

54. Challenges

55. Microservice Pod Definition
resources:
requests:
cpu: 2
memory: 4Gi
limits:
cpu: 2
memory: 4Gi
● Kubernetes assumes no other processes are
consuming significant resources
● Completely Fair Scheduler (CFS)
○ Schedules a task based on CPU Shares
○ Throttles a task once it hits CPU Quota

56. Microservice Pod Definition
resources:
requests:
cpu: 2
memory: 4Gi
limits:
cpu: 2
memory: 4Gi
● Shares = CPU Request * 1024
● Total Kubernetes Shares = # Cores * 1024
● Quota = CPU Limit * 100ms
● Period = 100ms

57. Java in a Container
● JVM is able to detect # of cores via sysconf(_SC_NPROCESSORS_ONLN)
● Scales tasks relative to this

58. Java in a Container
● Provide a base container that calculates the container’s resources!
● Detect # of “cores” assigned
○ /sys/fs/cgroup/cpu/cpu.cfs_quota_us divided by
/sys/fs/cgroup/cpu/cpu.cfs_period_us
● Automatically tune the JVM:
○ -XX:ParallelGCThreads=${core_limit}
○ -XX:ConcGCThreads=${core_limit}
○ -Djava.util.concurrent.ForkJoinPool.common.parallelism=${core_limit
}

59. Java in a Container
● Many libraries rely on Runtime.getRuntime.availableProcessors()
○ Jetty
○ ForkJoinPool
○ GC Threads
○ That mystery dependency...

60. Java in a Container
● Use Linux preloading to override availableProcessors()
#include <stdlib.h>
#include <unistd.h>
int JVM_ActiveProcessorCount(void) {
char* val = getenv("CONTAINER_CORE_LIMIT");
return val != NULL ? atoi(val) : sysconf(_SC_NPROCESSORS_ONLN);
}
https://engineering.squarespace.com/blog/2017/understanding-linux-container-scheduling

61. Communication With External Services
● Environment specific services should not be encoded in application
● Single deployment for all environments and datacenters
● Federation API expects same deployment
● Not all applications are using consul

62. Communication With External Services

63. Communication With External Services
apiVersion: v1
kind: Service
metadata:
name: kafka
namespace: elk
spec:
type: ClusterIP
clusterIP: None
sessionAffinity: None
ports:
- port: 9092
protocol: TCP
targetPort: 9092
apiVersion: v1
kind: Endpoints
metadata:
name: kafka
namespace: elk
subsets:
- addresses:
- ip: 10.120.201.33
- ip: 10.120.201.34
- ip: 10.120.201.35
...
ports:
- port: 9092
protocol: TCP

64. So what’s left?

65. Future Work: Enforce Squarespace Standards
● Custom Admission Controller requires all services, deployments, etc.
meet certain standards
○ Resource requests/limits
○ Owner annotations
○ Service labels

66. Future Work: Updating Common Dependencies
● Custom Initializers
○ Inject container dependencies into deployments (consul, fluentd)
○ Configure Prometheus instances for each namespace
● Trigger rescheduling of pods when dependencies need updating
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: location
namespace: core-services
annotations:
initializer.squarespace.net/consul:
"true"

67. QUESTIONS
Thank you!
squarespace.com/careers
Doug Jones
@dougfjones
Kevin Lynch
@kevml