This talk from the virtual 2020 CTO Summit (https://www.ctoconnection.com/summits) covers several architecture lessons to help you survive and thrive through the scaling phase of your company: * Modular Architecture * Event-Driven Communication * Quality and Reliability * Continuous Delivery

1. Scaling Your Architecture
for the Long Term
Randy Shoup
@randyshoup

2. Background
@randyshoup

3. STARTING SCALING OPTIMIZING
https://ittybiz.com/s-curve/@randyshoup

4. STARTING SCALING OPTIMIZING
https://ittybiz.com/s-curve/
Single Team
Rapid
iteration
@randyshoup
Monolithic
architecture

5. STARTING SCALING OPTIMIZING
https://ittybiz.com/s-curve/
Many Teams
Velocity and
Scalability
@randyshoup
Microservice architecture
Single Team
Rapid
iteration
Monolithic
architecture

6. STARTING SCALING OPTIMIZING
https://ittybiz.com/s-curve/
Fewer Teams
Efficiency and
Sustainability
@randyshoup
Consolidating
architecture
Many Teams
Velocity and
Scalability
Microservice architecture
Single Team
Rapid
iteration
Monolithic
architecture

7. STARTING SCALING OPTIMIZING
https://ittybiz.com/s-curve/
Fewer Teams
Efficiency and
Sustainability
@randyshoup
Consolidating
architecture
Many Teams
Velocity and
Scalability
Microservice architecture
Single Team
Rapid
iteration
Monolithic
architecture

8. STARTING SCALING OPTIMIZING
https://ittybiz.com/s-curve/
Fewer Teams
Efficiency and
Sustainability
@randyshoup
Consolidating
architecture
Many Teams
Velocity and
Scalability
Microservice architecture
Single Team
Rapid
iteration
Monolithic
architecture

9. STARTING SCALING OPTIMIZING
https://ittybiz.com/s-curve/
Fewer Teams
Efficiency and
Sustainability
@randyshoup
Consolidating
architecture
Many Teams
Velocity and
Scalability
Microservice architecture
Single Team
Rapid
iteration
Monolithic
architecture
(~1% of all
applications)
SCALING

10. Architecture Evolution
• eBay
• 5th generation today
• Monolithic Perl  Monolithic C++  Java  microservices
• Twitter
• 3rd generation today
• Monolithic Rails  JS / Rails / Scala  microservices
• Amazon
• Nth generation today
• Monolithic Perl / C  Java / C++  microservices

11. No one starts with microservices
…
Past a certain scale, everyone ends
up with microservices
@randyshoup
…
but most never reach that scale

12. Scalability Lessons
•Modular Architecture
•Event-Driven Communication
•Quality and Reliability
•Continuous Delivery

13. Scalability Lessons
•Modular Architecture
•Event-Driven Communication
•Quality and Reliability
•Continuous Delivery

14. Many Autonomous Teams
Idea
Development
Quality
Operations
Idea
Development
Quality
Operations
Idea
Development
Quality
Operations

15. Modular Architecture
• One domain: One team: One / few service(s)
o Organization  reflects  Architecture (“Conway’s Law”)
• Autonomy and Accountability
o Team can independently design, develop, deploy, operate its service(s)
o Team owns its service(s) end to end
@randyshoup

16. Modular Architecture
• Abstraction and Encapsulation
o Fault isolation
o “Optimal” technology stack
o Performance optimization
o Security boundary
• Isolated persistence
o All operations through published service interface
o No backdoor access to database (!)
@randyshoup

17. Modular Architecture
• Strict interface discipline
o Well-specified interface contract
o Forward- and backward-compatibility
o Testable and mockable
• Domains grow and divide over time
o Teams and services split like “cellular mitosis”
• Service ecosystem
o Services call others, which call others, etc.
o Graph, not a strict layering
@randyshoup

18. Scalability Lessons
•Modular Architecture
•Event-Driven Communication
•Quality and Reliability
•Continuous Delivery

19. Event-Driven Communication
• Service publishes an event when state changes
o Statement that some interesting thing occurred
o Represents a semantic domain event
• Events are a first-class part of a service interface
• 0 | 1 | N consumers subscribe to the event
@randyshoup

20. Event-Driven Communication
• Decouple domains and teams
o Abstract
o Asynchronous
• Decouple producer and consumer services
o 0 | 1 | N consumers
o Decoupled availability
o Independent scalability
o Reduced latency
o Smoother load characteristics
@randyshoup

21. Event-Driven Communication
• Formal Interface
o Well-specified event schema
o Testable and mockable
o Forward- and backward-compatibility
@randyshoup

22. Event-Driven Communication
• Standardized event bus
o Event persistence and replays
o Event ordering and duplication
o Delivery guarantees
o …
• Do not roll your own (!)
o This stuff is deceptively complicated
o There are decades of research and experience
o It’s probably not your core competency
@randyshoup

23. Scalability Lessons
•Modular Architecture
•Event-Driven Communication
•Quality and Reliability
•Continuous Delivery

24. Quality and Reliability
• Quality and Reliability are “Priority-0 features”
• Equally important to users as product features and
engaging user experience
@randyshoup

25. Automated Testing
• Tests help you go faster
o Tests are “solid ground”
o Tests are the safety net
• Tests make better code
o Confidence to break things
o Courage to refactor mercilessly
• Tests make better systems
o Catch bugs earlier, fail faster
@randyshoup

26. Developer Productivity
• 75% reading
existing code
• 20% modifying
existing code
• 5% writing new
code
https://blogs.msdn.microsoft.com/peterhal/2006/01/04/what-do-programmers-really-do-anyway-aka-part-2-of-the-yardstick-saga/
@randyshoup

27. Developer Productivity
• 75% reading
existing code
• 20% modifying
existing code
• 5% writing new
code
https://blogs.msdn.microsoft.com/peterhal/2006/01/04/what-do-programmers-really-do-anyway-aka-part-2-of-the-yardstick-saga/
@randyshoup

28. Vicious Cycle of Technical Debt
Technical
Debt
“No time
to do it
right”
Quick-
and-dirty

29. Virtuous Cycle of Investment
Solid
Foundatio
n
Confidence
Faster and
Better
Quality
Investment

30. Scalability Lessons
•Modular Architecture
•Event-Driven Communication
•Quality and Reliability
•Continuous Delivery

31. Continuous Delivery
• Deploy services multiple times per day
o Tighter feedback loop for build -> measure -> learn cycle
o Faster time to value
• More solid systems
o Release smaller units of work
o Smaller changes to roll back or roll forward
o Faster to repair, easier to understand, simpler to diagnose
@randyshoup

32. Continuous Delivery
• Repeatable Deployment Pipeline
o Low-risk, push-button deployment
o Rapid release cadence
o Rapid rollback and recovery
@randyshoup

33. Feature Flags
• Configuration “flag” to enable / disable a feature for a
particular set of users
o Independently discovered at eBay, Facebook, Google, etc.
• More solid systems
o Decouple feature delivery from code delivery
o Rapid on and off
o Develop / test / verify in production
@randyshoup

34. Scalability Lessons
•Modular Architecture
•Event-Driven Communication
•Quality and Reliability
•Continuous Delivery

35. Thank you!
@randyshoup
linkedin.com/in/randyshoup
medium.com/@randyshoup