Deutsche Telekom plans to dramatically improve the processes and the Time to Market of Fiber To The Home Infrastructure deployments in Germany. Working with legacy processes and software doesn’t meet their needs. A brand-new world is needed to sustain these goals. Success key items are automated geospatial data collection and analysis, end-to-end digital pipelines, online collaboration platforms sustaining the workflows and validation systems. Implementing agile workflows for the project requires solid foundations based on DevOps principles of automation and embodied in the role of SRE (Site Reliability Engineer). The deployment of an Open Source Spatial Data Infrastructure on an OpenShift cluster, which enables processes to scale and provides atomic architecture patterns for the deployment of modular components, is a central part of that project. In this talk, we’ll present the key challenges and how Deutsche Telekom and Camptocamp successfully implemented agile workflows for the project.
5. www.camptocamp.com
Measurements inside FTTH-factory
Leverages
Measurements
2D & 3D data
collections
Identification of
objects and surfaces
1
Data
conflation
Shift to UTM
spatial corrections
2
Automated
planning
optimized planning
based on detected
surfaces
3
Simplified
purchase
processes
All inclusive prices
for fixed offers
4
Digital plan
corrections
APP based mapping
of trenches
5
9. www.camptocamp.com
PROVISIONING AS CODE
Cloud Infrastructure Automation
■ TOOLS: OPENSTACK & TERRAFORM
○ Provision networks, security groups, etc. from code
○ Provision VMs and orchestrate their deployment from code
○ Support for multiple cloud providers and third party tools
■ BENEFITS
○ Full control of all cloud resources
○ Reproductibility of environments
○ Easy decommissioning
10. www.camptocamp.com
PROVISIONING AS CODE
Case Study: Provision a Development Environment
■ OBJECTIVE
Deploy Development and Production Environments
in a fully automated way
■ IMPLEMENTATION
Terraform is used to:
○ provision networks and nodes, integrate with Puppet/Ansible
■ BENEFITS
○ Fully automated, extremely fast deployment of deployments
○ All resources managed in one location
11. www.camptocamp.com
CONFIGURATION MANAGEMENT
Maintaining Machine State
■ TOOLS: TWO APPROACHES
○ Pull approach (Cfgmgmt): Cfengine, Puppet, Chef
○ Push approach (deploy): Ansible, Salt
■ BENEFITS
○ No manual management of nodes
○ Reproductibility of nodes
○ Inventory
○ More confidence in environments and deployments
12. www.camptocamp.com
CONFIGURATION MANAGEMENT
Case Study: Manage Server Fleet with Puppet
■ OBJECTIVE
Manage configuration for ~50 servers in two environments
■ IMPLEMENTATION
○ Puppet Open Source (Camptocamp’s Dockerized Stack)
○ DBs managed with Puppet
○ Identity Management
■ BENEFITS
○ Very fast setup of new nodes
○ Dynamic, up-to-date inventory
○ Nodes are kept up-to-date with their target configuration
13. www.camptocamp.com
DEPLOYMENT AS CODE
Deploying Complex Software
■ TOOLS: ANSIBLE, BOLT
○ Orchestration of multi-node deployments with dependencies
■ BENEFITS
○ Reproductibility of deployments
○ More confidence in deployments
14. www.camptocamp.com
DEPLOYMENT AS CODE
Case Study: OpenShift Deployment
■ OBJECTIVE
Deploy an OpenShift cluster, with various roles
and inter-nodes dependencies.
■ IMPLEMENTATION
Red Hat provides Ansible playbooks to deploy an OpenShift cluster.
■ BENEFITS
○ Fully automated OpenShift deployment
○ Much easier/faster to deploy new nodes in the cluster
16. www.camptocamp.com
CONTAINER ORCHESTRATION
Case Study: Puppet Stack Deployment with Helm
■ OBJECTIVE
Deploy GIS applications (geOrchestra) on OpenShift
■ IMPLEMENTATION
○ Helm charts for each component
○ Helmfile for full stacks
■ BENEFITS
○ Easily deploy a complex application on OpenShift
○ Parameters and secrets passed to templates
17. www.camptocamp.com
CONTINUOUS INTEGRATION
CI/CD Pipelines as Code
■ TOOLS: JENKINS
○ Continuous Integration platform
○ Integrates with multiple Version Control Systems
○ Allows to build pipelines
○ CI/CD as Code with Jenkinsfile
○ Extensible with Groovy code
■ BENEFITS
○ Flexible way to program CI/CD tasks
○ Portable to other Jenkins instances
○ Pipeline approach
18. www.camptocamp.com
CONTINUOUS INTEGRATION
Case Study: CI/CD with OpenShift
■ OBJECTIVE
○ Automatically deploy a GIS multi-tiered application
from code with OpenShift
■ IMPLEMENTATION
○ Use ImageStreams with OpenShift builds (using Jenkins)
and Jenkinsfile, along with triggers for deployment.
■ BENEFITS
○ Fully automated pipeline workflow for developers,
with automated checks and triggers.
19. www.camptocamp.com
OBSERVABILITY AS CODE
Logs, Metrics & Monitoring
■ TOOLS: ELK, PROMETHEUS, GRAFANA
○ Gather logs & metrics
○ Provide code for data processing and dashboards
■ BENEFITS
○ Prometheus time series provide
a flexible query language
○ JSON dashboards provide portability
20. www.camptocamp.com
OBSERVABILITY AS CODE
Case Study: Monitor Puppet resources and performance
■ OBJECTIVE
Gather performance metrics for the Puppet stack,
both on the server and agents.
■ IMPLEMENTATION
○ The Puppet prometheus_reporter module
provides a report plugin to collect agent data
○ JMX for internal Puppet Enterprise server metrics,
gathered in Prometheus
■ BENEFITS
Easily deployed dashboards with tuned metrics and monitoring.
21. www.camptocamp.com
BACKUPS AS CODE
Automate Backups of Container Volumes
■ TOOLS: KUBERNETES AND BIVAC
○ Automatic backups of PVCs
○ Tune backup parameters with Labels
■ BENEFITS
○ Allow users of container platforms
○ to choose with Data Volumes to backup
○ Allow users to tune how backups are performed
22. www.camptocamp.com
BACKUPS AS CODE
Case Study: Backup Data Volumes in OpenShift
■ OBJECTIVE
○ Provide an API for developers to backup their Data Volumes.
■ IMPLEMENTATION
○ The Bivac stack is deployed on OpenShift projects
to detect and backup Physical Volume Claims automatically.
■ BENEFITS
○ Backup container data where it is stored and let users
tune how to perform the backup.
23. www.camptocamp.com
CONCLUSION
Everything as Code
■ TOOLS
○ Many open-source tools to automate:
■ Systems Administration
■ Software Testing and Deployment
■ BENEFITS
○ Increased flexibility
○ Increased collaboration between teams
○ DevOps everything
Editor's Notes
<number>
<number>
<number>
* provisioning as code / case study:
- on déploie 2 envs: dev + prod pour FTTH
- terraform + ansible pour conf/paquets base, licences, etc.
- network components (DNS, sg, etc.), nodes & integration Puppet & Ansible
* cfgmgmt case study:
- ~10 machines par env, soit ~20
- DBs gérées par Puppet
- identity management -> cf Mickaël
- stack Puppet docker (docker-compose) c2c
* deployment as code: ansible + puppet pour clés SSH etc.
* container orchestration case study:
- pas puppet
- applications dans openshift: monitoring, logging, backups
- stack CI/CD déployées avec Helm
- au travers CI/CD: applications dev à prod avec Helm
- Georchestra déployé avec Helm
- https://github.com/roboll/helmfile couche au-dessus de helm pour définir un bundle de charts helm par env https://docs.cloudposse.com/tools/helmfile/
* CI/CD case study:
- Gitlab joue un rôle important
- Intégration Gitlab/Jenkins/OpenShift
- utilisé avec merge requests dans GitLab
- client veut utiliser Gitlab
- Jenkins: meilleure intégration dans OpenShift, donc configuré complètement avec Helm
- CD tout géré au travers de git, y compris déploiement des releases dans les envs (merge requests)