This document describes a reactive microservices architecture using Kubernetes and cloud PaaS. It discusses the need for scalability and availability driven by mobile growth. It presents microservices and stateful services as functions that produce events from commands to handle state in a distributed manner. The architecture includes reactive services, an API gateway, message bus, monitoring and logging using Prometheus and ELK. Deployments are automated through CI/CD pipelines on Kubernetes leveraging configuration as code.

1. https://github.com/patricknoir/reactive-platform
Using Kubernetes and Cloud PaaS

2. Patrick Di Loreto
CTO Addison Global (MoPlay)
https://github.com/patricknoir
http://patricknoir.blogspot.com
@patricknoir

3. 2007 2012

6.  This architecture was designed for
system pre smart phone generation
 Need for customer acquisition
=> need for SCALABILITY
 Availability became more important
than Consistency (CAP theorem)

7. Response in a
timely manner
Responsive under
failure scenariosResponsive
under load
Enabler for
responsiveness

8. Monolithic Microservices

9.  Average system utilization across organization is 15% to 20%
 The ideal should be around 80%
20% of IT budgets go to operations
“We have created:
Large
Underutilized
Complex
Environments which are costly to maintain”

10. In => Out
In
Stateful
Out
Database
In => Out
In
Stateless
FaaS
Out

11.  Has a name
 Is a function from In to Out
 Service[-In, +Out](name: String, f: In => Out)

12.  Sometime services have to handle a state, so in order to elaborate a
computation: they need to know the current state and they will also
produce an output state.
type StatefulService[S, Input, Output] = Service[(S, Input), (S, Output)]
class Service[In, Out](name: String, f: In => Out)
type In = (S, Input) type Out = (S, Output)

13. type In = (S, I)
type Out = (S, O)
class Service[In, Out](name: String, f: In => Out)
f: (S, I) => (S, O)
f: I => S => (S, O) // Currying: I => (S => (S, O))
f: I => State[S, O] // S => (S, O) = State[S, O]
class StatefulService[S, I, O](name: String, f: I => State[S, O])

15. Instruction Time
Execute typical instruction 1/1,000,000,000 = 1 nanosec
Fetch from L1 cache memory 0.5 nanosec
Branch misprediction 5 nanosec
Fetch from L2 cache memory 7 nanosec
Mutex lock/unlock 25 nanosec
Fetch from main memory 100 nanosec
Send 2K bytes over 1Gbps network 20,000 nanosec (20µs)
Read 1MB sequentially from memory 250,000 nanosec (250µs)
Fetch from new disk location (seek) 8,000,000 (8ms)
Read 1MB sequentially from disk 20,000,000 nanosec (20ms)
Send packet US to Europe and back 150,000,000 nanosec (150ms)

16. Instruction Time
Execute typical instruction 1 s
Fetch from L1 cache memory 0.5 s
Branch missprediction 5 s
Fetch from L2 cache memory 7 s
Mutex lock/unlock ½ s
Fetch from main memory 1½ min
Send 2K bytes over 1Gbps network 5½ hours
Read 1MB sequentially from memory 3 days
Fetch from new disk location (seek) 13 weeks
Read 1MB sequentially from disk 6½ months
Send packet US to Europe and back 5 years

17. class Service[In, Out](name: String, f: In => Out)
type ReactiveService[I, O] = Service[I, Future[O]]
val f: I => Future[O]
type ReactiveStatefulService[S, I, E] = StatefulService[I, Future[State[S, E]]]
val f: I => Future[State[S, E]]
O = State[S, E]

18. StatefulService[S, Command, Event]
Command => Future[State[S, Either[Notification, Event]]]
StatefulService[S, Command, Either[Notification, Event]]
(S, Command) => Either[Notification, Event]
(S, Event) => S

19. (S, Command) => Event
(Wallet(id=123, balance=5), CreditCmd(walletId=123, amount=10)) =>CreditedEvt(walletId=123, amount=10)
f: (S, Event) => S
(Wallet(id=123, balance=5), CreditedEvt(walletId=123, amount=10)) => Wallet(id=123, balance=15)
f: (S, Request) => Response
(Wallet(id=123, balance=15), GetBalanceReq(walletId=123)) => GetBalanceResp(walletId=123, amount=15)

20. Processor
Cmd[Debit(10)]
Evt[Credit(50)]
Evt[Debit(20)]
Evt[Credit(15)]
Evt[Debit(10)]
publish
process
View
request
respond
Client App
notify
Actor
[View1]Actor
[View1][View1]
PROCESSORS
(WRITE)
VIEWS
(READ)
(Wallet, Cmd) => (Wallet Evt)
(List[Wallet], Req) => Res
(List[Wallet], Evt) => List[Wallet]

21. wallet_evts
wallet_cmds
POD
Entity Store
Snapshots Journal
State(evtId=0, balance=0) WalletCreated(id=0,balance=0)
Credited(id=1, amount=50)
WalletService
(S, Cmd) => Either[Ntf, Evt]Stateful Sidercar Agent
(S, Evt) => S
Reactive
Service
service_resps
Debited(id=2, amount=25)
Cmd
(S,
List[Evts)])
(S, Cmd)
(S, Evt)

22.  Deployment CI/CD – Jenkins, CRDs
 Security Auth/Autz – JWT, SSL Offloading: ISTIO
 Monitoring – Federated Prometheus
 Logging - ELK

23.  Simplified version of git flow which provides continuous deployment on DEV
 Fully automated release process on managed environments:
 Integration with JIRA and creation of Release Ticket with release note built via commit
log
 Automated deployment when ticket is progressed on the JIRA board by the Product
manager (Webhook)
 Code as a config leveraging Kubernetes secrets
 Dependency deployment via CRDs (Kafka topics, Cassandra keyspace, RDS
schemas etc…)

24. API
Gateway
Req
Reactive
Service
service_resps
service_cmds
routing
Service
Service
Service
Reactive
Service
Pub
Int
Priv

25. API
Gateway
• SSL Offloading
• Routing Req/Resp to Service
• Routing async Cmd/Evt/Ntf to reactive Services
• Validate JWT Token
SSL
Envoy
JWT
Auth/Authz
JWT
Sign Verification

26. Federated Prometheus

28. Node
/logs
POD PODPOD
Infra Kubernetes

30. MessageBus
APIGateway
Pub
Int
Ext
Computation Farm
RDS KV GRAPH HDFS/S3
Repository
Kubernetes

31.  Mobile growth changed customer behavior, new needs:
 SCALABILITY is a must to run your business
 Shift on priorities: AVAILABILITY over CONSISTENCY (CAP)
 Infrastructure complexity and operational costs increase with MicroService
Architecture vs Monolith
 Serverless is the way to go but state is still a challenge to be solved
 https://cloudstate.io/

32.  https://github.com/patricknoir/reactive-platform