apidays LIVE Australia - Building Business Ecosystems
From micro to macro-coordination through domain-centric DDL pipeline
Alex Khilko, CTO of PlayQ Inc.
apidays LIVE Australia - From micro to macro-coordination through domain-centric DDL pipeline by Alex Khilko
1. From micro to macro
coordination
Through domain centric DDL pipeline
Alex Khilko, CTO
PlayQ Inc.
Alex Khilko, CTO
PlayQ Inc.
2. API Days Live, From Micro to Macro Coordination, Sep 2020
PlayQ
2
Domain: Mobile Games
Distributed Team: California USA,
Ireland, Ukraine, Australia
Tech: Client and back-office facing
platform serving millions of players
3. API Days Live, From Micro to Macro Coordination, Sep 2020
PlayQ Platform
3
Areas Covered: Authentication, Analytics, Cloud Storage,
Real-time leaderboards, Payments, Ad-Networks & attribution
providers integrations, Push Notifications, Orchestration…
Term: 4 years in production
Languages:
4. API Days Live, From Micro to Macro Coordination, Sep 2020 4
20+ High-performance services: 10-70ms response for the 99th
percentile
PlayQ Platform
5. API Days Live, From Micro to Macro Coordination, Sep 2020 5
Services Team: 6 persons
PlayQ Platform
6. API Days Live, From Micro to Macro Coordination, Sep 2020 6
Services API development
7. API Days Live, From Micro to Macro Coordination, Sep 2020 7
API Design Breakdown
Business Logic
Domain Design
Dispatchers
Models & Codecs
Routing
Transport
If user A is of org B, then do C
There are entities: Users, Organizations
Logic execution dispatching
Entity representation: on the wire
Delivery routing: URIs, Path, Versions, ...
Delivery mechanism: HTTP, Sockets, ...
API
8. API Days Live, From Micro to Macro Coordination, Sep 2020 8
Problems
1. Information is fragmented and decentralized Business Logic
Domain Design
Dispatchers
Models & Codecs
Routing
Transport
- Coming from multiple stakeholders
- Written and delivered using Confluence, Spreadsheets, emails,
etc.
- Ad-hoc requests
- Often times is incomplete, especially during design phase
API
9. API Days Live, From Micro to Macro Coordination, Sep 2020 9
Business Logic
Domain Design
Dispatchers
Models & Codecs
Routing
Transport
- Entities are usually complex: hierarchical, relational
- Composition is not engineer friendly
API
Problems
1. Information is fragmented and decentralized
2. No solutions to map business domain to code
10. API Days Live, From Micro to Macro Coordination, Sep 2020 1
0
Problems
1. Information is fragmented and decentralized
2. No solutions to map business domain to code
3. Most work is very mechanical and
time-consuming
Business Logic
Domain Design
Dispatchers
Models & Codecs
Routing
Transport- Once entities are designed, require a lot of mechanical work to reflect them in code
- Coordination with other teams / consumers to make sure implementation is
consistent
- Team interlocking
API
11. API Days Live, From Micro to Macro Coordination, Sep 2020 11
Problems
1. Information is fragmented and decentralized
2. No solutions to map business domain to code
3. Most work is very mechanical and
time-consuming
4. Evolution and Refactoring is very error prone
Business Logic
Domain Design
Dispatchers
Models & Codecs
Routing
Transport
- Any change in the upper layers, has to be propagated down
- Any error in the upper layers, has to be fixed all the way
down
API
12. API Days Live, From Micro to Macro Coordination, Sep 2020 12
Multiplied with every new consumer
Business Logic
Domain Design
Dispatchers
Models & Codecs
Routing
Transport
Business Logic
Domain Design
Dispatchers
Models & Codecs
Routing
Transport
Business Logic
Domain Design
Dispatchers
Models & Codecs
Routing
Transport
Business Logic
Domain Design
Dispatchers
Models & Codecs
Routing
Transport
API
13. API Days Live, From Micro to Macro Coordination, Sep 2020 13
Solution
1. Single source of truth
2. Rich type system
3. Full reliance on code generation
4. CI/CD based changes propagation
https://izumi.7mind.io/idealingua/
API
14. API Days Live, From Micro to Macro Coordination, Sep 2020 14
Single Source of Truth
API
15. API Days Live, From Micro to Macro Coordination, Sep 2020 15
Rich Type System
package user.api
enum Role = Admin | Moderator | User
id EntityID {
uuid: uuid
}
mixin Entity {
id: EntityID
}
mixin Person {
name: str
surname: str
role: Role
}
data User {
& Entity
+ Person
password: str
}
data PublicUser {
+ User
- password: str
}
adt Result = Success | Failure
data Success {
message: str
}
data Failure {
code: int8
}
API
16. API Days Live, From Micro to Macro Coordination, Sep 2020 16
Intuitive & Easy to read definitions
mixin Pet {
}
enum HuntingSkill {
lazy
clueless
}
/**
* A representation of a cat
*/
data Cat {
&Pet
/**
* The measured skill for hunting
* @default lazy
*/
huntingSkill: HuntingSkill
}
"Cat": {
"description": "A representation of a cat",
"allOf": [
{
"$ref": "#/definitions/Pet"
},
{
"type": "object",
"properties": {
"huntingSkill": {
"type": "string",
"description": "The measured skill for hunting",
"default": "lazy",
"enum": [
"clueless",
"lazy"
]
API
vs
17. API Days Live, From Micro to Macro Coordination, Sep 2020 17
Full Reliance on Code Generation
mixin Person {
name: str
surname: str
role: Role
}
data User {
& Entity
+ Person
password: str
}
data PublicUser {
+ User
- password: str
}
API
18. API Days Live, From Micro to Macro Coordination, Sep 2020 18
CI / CD Changes Propagation
mixin Person {
name: str
surname: str
role: Role
}
Change
CI / CD
API
19. API Days Live, From Micro to Macro Coordination, Sep 2020 19
Authentication Service
API Example
20. API Days Live, From Micro to Macro Coordination, Sep 2020 20
Providers support:
1. Support Email login
2. Support Google login
Auth Service
21. API Days Live, From Micro to Macro Coordination, Sep 2020
data LoginWithEmail {
email: string
passHash: string
}
data LoginWithGoogle {
accessToken: string
}
adt LoginRequest {
LoginWithEmail
LoginWithGoogle
}
21
Domain Models
data LoginResponse {
name: string
}
data GenericFailure {
code: int
message: string
}
service AuthService {
def login(data: LoginRequest) => LoginResponse !! GenericFailure
}
Auth Service
22. API Days Live, From Micro to Macro Coordination, Sep 2020 22
Iteration 2 changes:
Add support for companyId, so we can have
multi-tenant deployment.
Auth Service
23. API Days Live, From Micro to Macro Coordination, Sep 2020 23
Domain Models (rev 2)
mixin LoginWith {
companyId: CompanyID
}
data LoginWithEmail {
&LoginWith
email: string
passHash: string
}
data LoginWithGoogle {
&LoginWith
accessToken: string
}
adt LoginRequest {
LoginWithEmail
LoginWithGoogle
}
data LoginResponse {
name: string
}
data GenericFailure {
code: int
message: string
}
service AuthService {
def login(data: LoginRequest) => LoginResponse !! GenericFailure
}
Auth Service
24. API Days Live, From Micro to Macro Coordination, Sep 2020 24
TypeScript Models Generated
mixin LoginWith {
companyId: CompanyID
}
data LoginWithEmail {
&LoginWith
email: string
passHash: string
}
Auth Service
25. API Days Live, From Micro to Macro Coordination, Sep 2020 25
TypeScript Client Generated
service AuthService {
def login(data: LoginRequest) => LoginResponse !! GenericFailure
}
Auth Service
26. API Days Live, From Micro to Macro Coordination, Sep 2020 26
// Client Definition
export declare class AuthServiceClient implements IAuthServiceClient {
login(data: LoginRequest): Promise<LoginResponse | BasicFailure>;
}
// Server Definition
export interface IAuthServiceServer<C> {
login(context: C, data: LoginRequest): Promise<LoginResponse | GenericFailure>;
}
TypeScript Client & Server Generated
Auth Service
27. API Days Live, From Micro to Macro Coordination, Sep 2020 27
Iteration 3 changes:
Add support for 2-factor authentication during login.
Auth Service
28. API Days Live, From Micro to Macro Coordination, Sep 2020 28
Domain Models (rev 3)
data LoginResponse {
name: string
}
data MfaRequest {
message: string
token: string
}
adt LoginResult {
LoginResponse
MfaRequest
}
data LoginWith2FA {
code: string
token: string
}
data GenericFailure {
code: int
message: string
}
adt LoginRequest {
LoginWithEmail
LoginWithGoogle
LoginWith2FA
}
service AuthService {
def login(data: LoginRequest) => LoginResult !! GenericFailure
}
Auth Service
29. API Days Live, From Micro to Macro Coordination, Sep 2020 29
TypeScript Client Change
if (loginResponse instanceof MfaRequest) {
// TODO Ask user for a code and retry login
return;
}
// Successful, continue with execution
Auth Service
30. API Days Live, From Micro to Macro Coordination, Sep 2020 30
Summary
31. API Days Live, From Micro to Macro Coordination, Sep 2020 31
1. Roughly 60-70% of the code is generated (e.g. 250K generated
vs 95K actual business code in TypeScript codebase, 187K vs
120K in Scala)
2. We focus only on business logic and design
3. Every stakeholder contributes in an async way
4. No coordination on UserName or user_name when it comes to
actual implementation
5. Any API / Design change is immediately visible to all consumers
6. Refactoring is orders of magnitude easier
7. New APIs and changes through pull requests
8. Integration is a bliss and error free, no typos or missing data
Business Logic
Domain Design
Dispatchers
Models & Codecs
Routing
Transport
Summary
32. API Days Live, From Micro to Macro Coordination, Sep 2020 32
Q & A
Alex Khilko
akhilko@playq.net
33. API Days Live, From Micro to Macro Coordination, Sep 2020 33
Through evolution:
1. Add new services
2. Add new methods to existing services
3. Add new optional fields to existing models
4. Add new models
Through deployment address:
1. Version 1: /api/v1
2. Version 2: /api/v2
Through custom headers routing
Versioning
34. API Days Live, From Micro to Macro Coordination, Sep 2020 34
1. Extensive primitive types, including Date, Time, DateTime with offsets, signed
/ unsigned numbers, and so on.
2. Unique algebraic and ID types
3. Definitions are parsed and AST is built using all of the inheritance, structural
composition information
4. Code is generated, not templated, making each language transpiler unique
and that follows semantics of the target language
5. Vertically sliced, making switching any layer an easy process: for instance
going from HTTP to WebSockets requires zero changes to the logic
6. Built-in support for HTTP and WebSocket transports, routing, JSON encoding
7. Server to Client requests system
8. Generated codecs tests
9. Converters: expand / contract are generated
Unique Features