The document outlines the lessons learned by Daniel Oliveira in building an automated staging environment using Docker, Kubernetes, and cloud provisioning. It discusses the challenges faced, such as the need for less manual intervention, and presents solutions including integrating Slack for deployment notifications and using AWS for managing resources. Future goals involve reducing application image sizes and enhancing logging and monitoring capabilities.

1. Building an automated
Staging environment
LESSONS LEARNED BY DANIEL OLIVEIRA

2. About Me
danieloliv079
danieloliveira079
df@autobutler.de
Daniel Oliveira
Black Magic Ops

3. About Us
https://www.autobutler.dk

4. Stack

5. Challenge
Provide a staging environment where
we can deploy our applications
requiring as less human
interaction/intervention as possible.

6. Toolbox

7. 1º Step: Dockerize 'Em All
+

8. 2º Step: Cloud Provisioning
+ KOPShttps://github.com/kubernetes/kops
(Network, Postgresql, Salt Master, Salt Minion)

9. 3º Step: From code to K8S
https://Pull Request
Dynamic URLs

10. Challenges / Solutions
■ Developers don't wanna learn shell commands / CD + Github integration
■ Where should I host my docker images? / Docker Cloud Private Registry
■ Docker building time / Base image for gems initial install
■ Database Isolation per deployment / Reuse or on-demand
■ AWS - K8S Cluster CPU and RAM Usage / Ops Microservices - Garbage collector

11. Challenges / Solutions
■ Dynamic URLs / Salt Formulas + AWS Route 53 + NGINX controllers
■ HTTPS / Letsencrypt
■ Jira Integration / Ops Microservice - Update Ticket and includes URLs
■ Logging and debugging / Weavescope
■ DRY / Saltstack states+pillars

12. ■ Integrations / Third-Party APIs (Github, Jira, Internal)
■ Deployments management / Deployment Dashboard (Built in house)
■ Pull request updates (source code) / Redeploy using dashboard or slack bot
■ Linked environments / Rails API + React Front-End
Challenges / Solutions

13. Challenges / Solutions
■ Manual deployments / Slack Bot
■ Manual Destroy / Deployment dashboard option

14. Deployment workflow
Pull Request webhook
Update
Notify
Update

15. Deployment Dashboard

16. Logging and Monitoring - Weavescope

17. Logging and Monitoring - Weavescope

18. Results
■ Sandbox environment
■ Product team can validate features and bug fixes
■ End-to-end testing
■ AWS - K8S Cluster CPU and RAM / Ops Microservices - Garbage collector

19. Future
■ Decrease rails application image size
■ Helm Charts - https://helm.sh/
■ Ops FaaS - https://github.com/alexellis/faas
■ Move off from Heroku to AWS / Kubernetes
■ Logging and Monitoring Aggregation / Prometheus + Sysdig + Datadog

20. From Code to Kubernetes - Slideshare

21. Questions ?

22. Thank you ?