1. How do we cook highload
microservices at SBTech
by @AntyaDev
Kyiv 28.10.2017

2. genda
• What technology stack did we choose and why?
[Stateful/Stateless Services; Cache; Streaming Data; Structure Logging]
• How do we use/adopt F# as a primary language
and what do we gain?
[DDD; CQRS; ES; Property Based Testing; Dependency Rejection]
• How do we test our microservices?
[Integration/Load/Failover Tests]

3. - One of the biggest provider in sport offering
- Supports a lot of regulated markets
- About 1000 microservices (200 distinct types)
- About 5 datacenters maintained fully by SBTech
- About 500 concurrent live events at pick time
- Design to settle 1 – 2ml bets per event

4. Stateless Services

5. Stateless Services
are Wasteful
• Stateless services have worked well. Storing the
canonical source of truth in the database and
horizontally scaling by adding new stateless service
instances as needed has been very effective.
• The problem is our applications do have state and we
are hitting limits where one database doesn’t cut it
anymore. In response we’re sharding relational
databases or using NoSQL databases. This gives up
strong consistency which causes part of the database
abstraction to leak into services.

6. Actor model
User 1
User 2
User 3

7. Cache

8. Complex
Event Processing

9. Streaming

10. Streaming
• Bridge between datacenters
• Production traffic (integration tests)
• Metrics, logs
• Pick time about 80K logs in sec
• Overall volume about 2TB per day

12. 12:23:10 [INF] Service "loyalty-card-printer" starting up on host 8fd342hkg22u
12:23:11 [INF] Listening on http://8fd342hkg22u.example.com:1234
12:23:20 [INF] Card replacement request received for customer-109972
12:40:55 [INF] Card replacement request received for customer-98048

13. time=12:23:10, level=INF, service=loyalty-card-printer, host=8fd342hkg22u, activity=started
time=12:23:11, level=INF, endpoint=http://8fd342hkg22u.example.com:1234, activity=listening
time=12:23:20, level=INF, customer=customers-1099972, activity=replacement
time=12:40:55, level=INF, customer=customer-98048, activity=replacement

14. Microservice
Serilog

15. Microservice Microservice Microservice Microservice Microservice Microservice

16. Functional
programming

19. LEFT FOLD
fold: ('State -> 'Event -> 'State) -> 'State -> 'Event list -> 'State
let state = loadEvents(streamId) |> List.fold applyEvent zeroState

25. (John Hughes, Koen Claessen)