Code, ci, infrastructure - the gophers way
•
1 like•535 views
The presence of golang. Golang becomes pretty powerful. There are plenty of everyday golang tools. I will show you that we built infrastructure mostly consists of tools written on one language - golang. What benefits did we gain from writing an application on the language of its infrastructure. And I will describe how we use golang in production.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (11)
Similar to Code, ci, infrastructure - the gophers way
Similar to Code, ci, infrastructure - the gophers way (20)
Recently uploaded
Recently uploaded (20)
Code, ci, infrastructure - the gophers way
- 1. Code, CI, Infrastructure - the gophers way Alex Baitov Senior backend developer 2gis.com
- 2. About 2GIS International tech company, developing software for convenient city living 2GIS City Information Service (most successful product) ● combines a map with a city directory ● used by over 40 million people ● covers over 300 cities in 9 different countries info.2gis.com/index_en.html
- 3. Content 1. Application written in golang 2. Ordinary infrastructure 3. Unified infrastructure 4. Gitlab CI as a glue 5. Benefits of using one language
- 4. How did we come to golang in production?
- 5. Problem of losing packages Application backend server Main statistics server
- 6. Asynchronous reverse proxy with buffer Application backend server Main statistics server Asynchronous reverse proxy with buffer on golang
- 13. Team infrastructures Ordinary infrastructure Unified infrastructure Monolith Microservice
- 14. Team infrastructures Ordinary infrastructure Unified infrastructure Monolith Microservice Microservice (other teams)
- 15. Why did we switch to another infrastructure?
- 16. Software development life cycle SDLC PlanningPlanning Develop- ment Develop- ment TestingTesting DeliveryDelivery Mainte- nance Mainte- nance Infrastructure is the core of SDLC
- 20. SDLC: substages PlanningPlanning Develop- ment Develop- ment TestingTesting DeliveryDelivery Mainte- nance Mainte- nance Application UnitDeploy Configuring Functional Metrics and Alerting Logging Monitoring Init and keeping alive SDLC
- 21. SDLC: substage tools PlanningPlanning Develop- ment Develop- ment TestingTesting DeliveryDelivery Mainte- nance Mainte- nance Application UnitDeploy Configuring Functional Metrics and Alerting Logging Monitoring Init and keeping alive SDLC jenkins / rundeck / heat templates in openstack chef / ansible zabbix / sensu application sends by protocol / beaver kibana upstart / systemd / monit openstack / own testing frameworks test modules framework / package manager / standards
- 22. Disadvantages of lots of tools ● High entry barrier
- 23. Disadvantages of lots of tools ● High entry barrier ● Simultaneous studying of all this tools takes a long time
- 24. Disadvantages of lots of tools ● High entry barrier ● Simultaneous studying of all this tools takes a long time ● Difficult integration
- 25. Disadvantages of lots of tools ● High entry barrier ● Simultaneous studying of all this tools takes a long time ● Difficult integration ● Specific debugging of each tool
- 26. Disadvantages of lots of tools ● High entry barrier ● Simultaneous studying of all this tools takes a long time ● Difficult integration ● Specific debugging of each tool ● Human factor (sometimes forget something)
- 27. What if you need to look inside a tools or write a custom module for it? You need to know tool’s language
- 28. SDLC: substage tools languages PlanningPlanning Develop- ment Develop- ment TestingTesting DeliveryDelivery Mainte- nance Mainte- nance Application UnitDeploy Configuring Functional Metrics and Alerting Logging Monitoring Init and keeping alive SDLC java / heat templates yaml ruby / python c, php / ruby application language / python javascript c php / scala php / scala / go php / scala
- 29. Disadvantages of different languages of lots of tools ● Different error reporting for each language
- 30. Disadvantages of different languages of lots of tools ● Different error reporting for each language ● Harder understanding of how the tool works
- 31. Disadvantages of different languages of lots of tools ● Different error reporting for each language ● Harder understanding of how the tool works ● Studying different languages to write custom modules
- 32. Advantages of this approach ● Wider knowledge about familiar tools ● Familiar tools help to fast build of minimum viable product (MVP)
- 33. Disadvantages of this approach ● Too much support for one team ● Too much tools for building microservices ● Long everyday releases ● Expensive development process
- 34. How to improve it?
- 35. Unified infrastructure ● DEIS - platform as a service that deploys and scales containers ● Docker - pack, ship and run any application as a lightweight, portable, self-sufficient container ● Prometheus - metrics and alerting system ● Grafana - metrics visualization
- 36. Unified infrastructure ● DEIS - platform as a service that deploys and scales containers ● Docker - pack, ship and run any application as a lightweight, portable, self-sufficient container ● Prometheus - metrics and alerting system ● Grafana - metrics visualization All of them are: ● open-source ● written on go
- 37. Unified infrastructure ● DEIS -> DEIS 2 ( Workflow / Kubernetes ) ● Docker - pack, ship and run any application as a lightweight, portable, self-sufficient container ● Prometheus - metrics and alerting system ● Grafana - metrics visualization All of them are: ● open-source ● written on go
- 38. How do we use mostly golang infrastructure in production?
- 39. SDLC: substage tools PlanningPlanning Develop- ment Develop- ment TestingTesting DeliveryDelivery Mainte- nance Mainte- nance Application SDLC native / framework / vendoring / standards
- 40. SDLC: substage tools PlanningPlanning Develop- ment Develop- ment TestingTesting DeliveryDelivery Mainte- nance Mainte- nance Application Unit Functional SDLC docker-compose / test library test library native / framework / vendoring / standards
- 41. SDLC: substage tools PlanningPlanning Develop- ment Develop- ment TestingTesting DeliveryDelivery Mainte- nance Mainte- nance Application UnitDeploy Configuring Functional SDLC gitlab-ci / deis gitlab-ci / deis (via environment variables) docker-compose / test library test library native / framework / vendoring / standards
- 42. SDLC: substage tools PlanningPlanning Develop- ment Develop- ment TestingTesting DeliveryDelivery Mainte- nance Mainte- nance Application UnitDeploy Configuring Functional Metrics and Alerting Logging Monitoring Init and keeping alive SDLC gitlab-ci / deis gitlab-ci / deis (via environment variables) prometheus deis captures application output deis (via checking health) kibana / grafana docker-compose / test library test library native / framework / vendoring / standards
- 43. SDLC: substage tools languages PlanningPlanning Develop- ment Develop- ment TestingTesting DeliveryDelivery Mainte- nance Mainte- nance Application UnitDeploy Configuring Functional Metrics and Alerting Logging Monitoring Init and keeping alive SDLC go / gitlab-ci yaml go / gitlab-ci yaml go go go / javascript go go go go
- 44. This infrastructure is also used for Scala, Node.js and C# applications
- 45. Comparison of SDLCs Ordinary infrastructure Tools (quantity) Golang infrastructure Languages (quantity) 15 + 8 7 3
- 46. How do we use it together?
- 47. Gitlab workflow ● 2 shared runners with docker: 16 GB, 32 CPU ● Gitlab-ci job isolation with docker or docker-compose ● Latest branch docker images of application - in entire docker registry ● That allows to rollback ● Job parallelisation
- 48. Gitlab pipeline
- 49. Everything in code ● Application ● Tests
- 50. Everything in code ● Application ● Tests ● Deploy ● Configuration
- 51. Everything in code ● Application ● Tests ● Deploy ● Configuration ● Metrics ● Alerts rules
- 52. Everything in code ● Application ● Tests ● Deploy ● Configuration ● Metrics ● Alerts rules Review everything via merge requests Easy to resolve problem
- 53. What are benefits from using golang in application and infrastructure?
- 54. Benefits of language coincidence First official tool client goes on its native language: go
- 55. Benefits of language coincidence First official tool client goes on its native language: go Easy to read and understand tools logic
- 56. Benefits of language coincidence First official tool client goes on its native language: go Easy to read and understand tools logic Easy to contribute in infrastructure
- 57. Benefits of language coincidence First official tool client goes on its native language: go Easy to read and understand tools logic Easy to contribute in infrastructure Concentrate on one language
- 58. Benefits of language coincidence First official tool client goes on its native language: go Easy to read and understand tools logic Easy to contribute in infrastructure Concentrate on one language Fast maintenance reaction
- 59. Benefits of language coincidence First official tool client goes on its native language: go Easy to read and understand tools logic Easy to contribute in infrastructure Concentrate on one language Fast maintenance reaction Fast mvp
- 60. Conclusion Everything in code Mostly one language Golang is good for infrastructure tools Golang is production proved
- 61. Alex Baitov al.baitov@2gis.ru alex.baitov@gmail.com slides & links : goo.gl / LKPmvi 2gis.ru Thank you!
- 62. Extras
- 63. Docker An open-source project that automates the deployment of Linux applications inside software containers Also use: ● Docker-compose ● Docker registry
- 64. Deis An open-source platform as a service (PaaS) that makes it easy to deploy and manage applications on your own servers. Deis builds upon Docker and CoreOS to provide a lightweight PaaS with a Heroku-inspired workflow. Built-in: ● Logging ● Scaling ● Routing ● Fault-tolerance
- 65. Prometheus An open-source metrics and alerting system: ● Implements a highly dimensional model data ● Flexible querying language ● Built-in visualisation ● Federation system ● Push or pull data mining ● Alert manager ● Many client libraries