ScaleDay conference

2. WASILEWSKI Jean
DevOps Bare Metal
Migrating the Online’s console
with Docker
2
Part 1: Previous state of the Online’s Console
Part 2: Dockerized Online’s Console

3. The Online’s Console
3

4. The Online’s Console
Some key values
More than 130k servers
Around 310k users
Driven by 384k lines of code
4

5. The Online’s Console
Architectural design
4 Load Balancers
5 Web Servers
2 Databases
55

6. The Online’s Console
Why we start this work
●
Old servers
●
Various operating systems
●
Unable to create new internal servers
●
Impossible to scale
●
Stop using our homemade deployment system
6

7. The Online’s Console
What we planned to do
●
Old servers
➔
Replace with new servers
●
Various operating systems
➔
Install new ones with the same OS
●
Unable to create new internal servers
➔
Deploy new ones with Ansible
●
Impossible to scale
➔
Scale with Nomad functionalities
●
Stop using our homemade deployment system
➔
Use Gitlab CI and Nomad’s blue/green
7

8. The Online’s Console
Kill the homemade CI (@bob)
Deploy server
8

9. Our new tools
9

10. Our new tools
What is Nomad?
Nomad Servers
Nomad Nodes
10

11. Our new tools
What is Consul?
Nomad Nodes
Consul Servers
11

12. Our new tools
What is blue/green deployment?
v11 v11
Nomad Nodes
12

13. Our new tools
What is blue/green deployment?
v11 v11
Nomad Nodes
v12 v12
13

14. Our new tools
What is blue/green deployment?
v11 v11
Nomad Nodes
v12 v12
14

15. Our new tools
What is blue/green deployment?
Nomad Nodes
v12 v12
15

16. Dockerizing the Console
16

17. Dockerizing the Console
Why choosing Nomad
●
Way less complex to maintain than kubernetes or Mesos
●
Deployable easily with Ansible
●
Good integration with Consul and Vault
●
Consul allows us to use service discovery
●
Contains all the features we need
17

18. Dockerizing the Console
What we did
●
Order new servers
●
Deploy them with Ansible
●
Create a proper architecture based on Nomad and Consul
●
Store all secrets in Vault
●
Harmonize all configurations in one big container
●
Deploy via Gitlab CI
18

19. Dockerizing the Console
New design
19
4 3
22
14
2 3
Databases
Nomad
Nodes
Load Balancers Nomad Servers
Consul Servers
Vault Servers
Transfer Control
Register
Store
Provide
Secrets

20. Dockerizing the Console
How it works now
Test the container
Create container
Create commit and push it
20

21. Dockerizing the Console
What about the preprod?
●
Same infrastructure as before
●
Replicated DBs
●
Isolated APIs
21

22. Dockerizing the Console
What went wrong
●
Some libraries were missing
●
Hidden hard coded path with missing files
●
Issues with Nomad container promotion
22

23. Dockerizing the Console
What went wrong
v11 v11
Nomad Nodes
23

24. Dockerizing the Console
What went wrong
v11 v11
Nomad Nodes
v12 v12
24

25. Dockerizing the Console
What went wrong
Nomad Nodes
v12 v12
25

26. Dockerizing the Console
IPs ACL are bad Mkay?!
●
Will never scale
●
Require a bigger rework to fix
●
Just don’t
26

27. The Online’s Console
Well, now, it works
✔
Replace with new servers
✔
Install new ones with the same OS
✔
Deploy new ones with Ansible
✔
Scale with Nomad functionalities
✔
Use Gitlab CI and Nomad’s blue/green
27

28. THANK YOU
Follow our news and tutorials on Twitter and LinkedIn with
@Scaleway
Follow me on LinkedIn with @jeanwasilewski
And on GitHub via @henyxia
28