Versioning challenges in micro services of Gwent

1. Georgii Korshunov & Krzysztof Sobczak
Software Engineers at Gwent
Webservices

2. Versioning challenges in micro services of

4. When should you
begin API versioning?

5. How long should you
avoid API versioning ?

6. GWENT webservices – GOG
COVERED PARTS:
• MICROTRANSACTIONS
• LEADERBOARDS
• MULTIPLAYER COMPONENTS
• PLAYER’S PROFILES
• CARD COLLECTIONS
• REWARDS AND ACHIEVEMENTS

7. Our beginning
• Only one game version NO VERSIONING

8. Introducing new game version
• Logic changes synced with „fast” game update
• Backwards compatibility
NO VERSIONING

9. Supporting old game versions
VERSIONING REQUIRED• Requirement to support multiple logic paths

11. Summary
• Convenient way of rolling out new game versions
REASONS FOR VERSIONING
• Supporting old versions with different business logic
• Protecting access to old/new features

12. API versioning

13. Microservices versioning
Independent versions per
service
_version=3
_version=1,_version=3,
…
Global version

14. Microservices communication

15. Background tasks

16. Interface
http://service.gog.com/v2/endpoint?param=value
http://service.gog.com/endpoint?param=value
Accept: application/vnd.gog.service+json; version=2.0
X-Api-Version: 2
http://service.gog.com/endpoint?param=value&_version=2
Path
Header
Query

17. Microservices communication
Synchronous — HTTP calls
http://service.gog.com/endpoint?param=value&_version=3
Asynchronous — AMQP messages

18. Implementation
Routing helper
class UserController extends Controller
{
public function getAction($userId)
{}
_version=1
_version>=2 public function getAction_2($userId)
{}

19. Implementation
Strategy class with semanthic methods
class ApiVersionStrategy
{
public function hasNewCardNames()
{
return $this->version >= 3;
}

20. Dropping support for old versions
• Drop support for old version ASAP
• Forbid requests with old versions
• Remove code and tests

21. Data versioning

22. Data versioning

23. Data versioning

24. Data versioning
• API is not changing
• There are several versions of data
• Data versions are used by different versions of client

25. Microservice communication

26. Interface
http://service.gog.com/endpoint?_data_version=2
Query param

27. Implementation

28. Implementation
• Duplicate tables of versioned entities for each version
• Table names depend on current data version
• Foreign keys are duplicated too
• Except for foreign keys from unversioned to versioned data

29. Implementation
• Data version is a request parameter
• Hook into Doctrine metadata loader to change tablenames
• Hook into Doctrine metadata cache loader

30. Other usages — Rankings

31. Other usages — Rankings

32. Supporting
• New data versions: per-user privileges
• Old data versions: forbidding access
• Active data versions: readonly for data safety

33. Game Servers orchestration

34. Game server versions

35. Game servers

36. Game Server – Multiplayer
• Data version
• Game version
Registered version includes:
Additional server data:
• Maximum number of players

37. Server management

38. Server Agent - deamon

39. Server Agent
• Updates server state through HTTP
MANAGEMENT
• Registers the server in a server manager
• Sends commands to server process with bash
github.com: yohang/finite
• Uses state machine for bullet proof state transitions
• States include: starting, running, graceful-stopping, stopped, updating etc.

40. Server Manager
• Provides current tasks for each servers
• HTTP RESTful service
• Analyzes current server requirements and spawns tasks for servers
VERSION
MANAGER
COORDINATOR
• Handles server registration & state updates
• Background process
• Notifies about not enough servers in the pool
• Capable of auto-scaling based on the game servers usage by
players

41. Server Manager
• Manager spawns minimum number of servers
• Human defines minimum/maximum number of servers per game version
• Manager adjusts number of servers to cover current load
SERVER PROVISIONING

42. Tests & Monitoring

43. Our stack
CHARTS - GRAFANA
METRICS - TELEGRAF
LOGS - KIBANA
CHECKS - ICINGA
BOTS WITH UI
HEADLESS BOTS
DEV TESTS - DOCKER
MULTIPLE TESTING ENVS

44. Game release monitoring
Upgrade started Crossed half
(players on game servers by game version)

45. Chart alerts
game version A
game version B
Alerting state Recovery state

46. Questions?

47. Join us!