Implementing an Automated Staging Environment

•

0 likes•228 views

Slides from my talk that demonstrates the challenges, difficulties, discoveries and joys of building an CI/CD for staging environments using Docker, Kubernetes, Github, Slack, AWS, Terraform and SaltStack. From the opening of a pull request to a running environment, how our process at Autobutler is provisioning and running staging environments using containers.

Technology

Building an automated
Staging environment
LESSONS LEARNED BY DANIEL OLIVEIRA

About Me
danieloliv079
danieloliveira079
df@autobutler.de
Daniel Oliveira
Black Magic Ops

Challenge
Provide a staging environment where
we can deploy our applications
requiring as less human
interaction/intervention as possible.

2º Step: Cloud Provisioning
+ KOPShttps://github.com/kubernetes/kops
(Network, Postgresql, Salt Master, Salt Minion)

3º Step: From code to K8S
https://Pull Request
Dynamic URLs

Challenges / Solutions
■ Developers don't wanna learn shell commands / CD + Github integration
■ Where should I host my docker images? / Docker Cloud Private Registry
■ Docker building time / Base image for gems initial install
■ Database Isolation per deployment / Reuse or on-demand
■ AWS - K8S Cluster CPU and RAM Usage / Ops Microservices - Garbage collector

Challenges / Solutions
■ Dynamic URLs / Salt Formulas + AWS Route 53 + NGINX controllers
■ HTTPS / Letsencrypt
■ Jira Integration / Ops Microservice - Update Ticket and includes URLs
■ Logging and debugging / Weavescope
■ DRY / Saltstack states+pillars

■ Integrations / Third-Party APIs (Github, Jira, Internal)
■ Deployments management / Deployment Dashboard (Built in house)
■ Pull request updates (source code) / Redeploy using dashboard or slack bot
■ Linked environments / Rails API + React Front-End
Challenges / Solutions

Challenges / Solutions
■ Manual deployments / Slack Bot
■ Manual Destroy / Deployment dashboard option

Deployment workflow
Pull Request webhook
Update
Notify
Update

Results
■ Sandbox environment
■ Product team can validate features and bug fixes
■ End-to-end testing
■ AWS - K8S Cluster CPU and RAM / Ops Microservices - Garbage collector

Future
■ Decrease rails application image size
■ Helm Charts - https://helm.sh/
■ Ops FaaS - https://github.com/alexellis/faas
■ Move off from Heroku to AWS / Kubernetes
■ Logging and Monitoring Aggregation / Prometheus + Sysdig + Datadog

What's hot

Demystifying the Nuts & Bolts of Kubernetes Architecture

Ajeet Singh Raina

Have you ever thought how to deploy Cloud Native Java Applications (Spring Boot) on Kubernetes? Kubernetes has now become a de-facto standard for deploying Cloud Native Applications, but still there is myth that they are not ready for Java workloads. The aim of this session is to break that myth to show Kubernetes is well suited for Cloud Native Java applications. The session starts with a brief history of Microservices; the framework, the libraries, the platform and the tools that have been part of every cloud native applications and how they become deprecated with Cloud Native Java applications deployed to Kubernetes. The session explores the cloud native characteristics such as Discovery, Blue/Green Deployments, Elasticity, Canary Deployments, Resiliency, Pipeline(CI/CD), Authentication etc., becomes implicit characteristics to your Spring Boot Java applications that are deployed on Kubernetes/OpenShift. In this session, we will see how to build, debug, deploy and discover Spring Boot applications on Kubernetes, covering in depth details of the tools, libraries and platform that could be used to make your spring boot deployment smooth and easy.

KUBEBOOT - SPRING BOOT DEPLOYMENT ON KUBERNETES HAS NEVER BEEN SO EASY

Red Hat Developers

Building Cloud-Native Applications with Kubernetes, Helm and Kubeless

Bitnami

Kubernetes has been a key component for many companies to reduce technical debt in infrastructure by: • Fostering the Adoption of Docker • Simplifying Container Management • Onboarding Developers On Infrastructure • Unlocking Continuous Integration and Delivery During this meetup we are going to discuss the following topics and share some best practices • What's new with Kubernetes 1.3 • Generate Cluster Configuration using CloudFormation • Deploy Kubernetes Clusters on AWS • Scaling the Cluster • Integrating Ingress with Elastic Load Balancer • Using Internal ELB's as Kubernetes' Service • Using EBS for persistent volumes • Integrating Route53

Running Production-Grade Kubernetes on AWS

DoiT International

Cloud Native Night, January 2018, Mainz: Talk by Jakob Karalus (@krallistic, IT Consultant at codecentric) Join our Meetup: https://www.meetup.com/de-DE/Cloud-Native-Night Abstract: While it's easy to deploy stateless application with Kubernetes, it's harder for stateful software. Since applications often require custom functionality that Kubernetes can't provide, developers want to add more specialized patterns like automatic backups, failover or rebalancing to their Kubernetes deployments. In this talk, we will look at the Operator Pattern and other possibilities to extend the functionality of Kubernetes and how to use them to operate stateful applications.

The Operator Pattern - Managing Stateful Services in Kubernetes

QAware GmbH

Arkena's video-on-demand platform is used as backend by major european channels (TF1 / beIN SPORTS / Elisa) to propose a non-linear experience to their customers. Previously hosted on Heroku, the number of our users is increasing constantly. In order to optimize resources we decided to move on a bare metal infrastructure powered by Kubernetes. We'll share thoughts, feedbacks and technical details about this successful transition. Sched Link:

KubeCon EU 2016: Heroku to Kubernetes

KubeAcademy

4K–Kubernetes with Knative, Kafka and Kamel

Red Hat Developers

Incredibly powerful and flexible, Kubernetes role-based access control (RBAC) is an essential tool to effectively manage production clusters. Yet many Ops and DevOps engineers are still facing barriers to efficiently use it at scale. These include a steep learning curve, YAML-based configuration, lack of standardized best practices, and the general complexity of this functionality at large -- it truly can be somewhat overwhelming. During this meetup Oleg, CTO at Kublr, will discuss Kubernetes RBAC concepts and objects. He'll explore different use cases ranging from simple permission management for in-cluster application accounts to integrations with external identity providers for SSO and enterprise user access management. Leveraging the Kublr Platform, Oleg will demonstrate how it simplifies the management of access and RBAC rules in a cloud native environment while staying vendor-independent and compatible with any Kubernetes distribution.

Introduction to Kubernetes RBAC

Kublr

CI/CD with Openshift and Jenkins

Ari LiVigni

Setting up CI/CD pipeline with Kubernetes and Kublr step-by-step

Oleg Chunikhin

Version control can basically be found in almost every company, team or project – no matter if it is used for creating and releasing software or providing a safe place for your setup’s configurations. A lot of professionals have chosen GitLab for making work life easier, better and more flexible. So, ever wondered why GitLab is so famous? Or why it is easy to use while still being such a feature-rich application? And how you can take a fast start with a new idea? Then come and check out GitLab with us! We will give you an introduction to GitLab and the basics needed for working with this version control software – create your project, write a Dockerfile and power it on with a runner. Git things done fast and efficiently!

OSDC 2018 | Git things done with GitLab by Gabriel Hartmann and Nicole Lang

NETWAYS

Cloud Native Night, January 2018, Munich: Workshop led by Jörg Schad (@joerg_schad, Technical Lead Community Projects at Mesosphere) Join our Meetup: https://www.meetup.com/de-DE/cloud-native-muc PLEASE NOTE: During this workshop, Jörg showed many demos and the audience could participate on their laptops. Unfortunately, we can't provide these demos. Nevertheless, Jörg's slides give a deep dive into the topic. ABSTRACT: Kubernetes has been one of the topics in 2017 and will probably remain so in 2018. In this hands-on technical workshop you will learn how best to deploy, operate, and scale Kubernetes clusters from one to hundreds of nodes using DC/OS. You will learn how to integrate and run Kubernetes alongside traditional applications and fast data services of your choice (e.g. Apache Cassandra, Apache Kafka, Apache Spark, TensorFlow, and more) on any infrastructure. This workshop best suits operators focussed on keeping their apps and services up and running in production and developers focussed on quickly delivering internal and customer facing apps into production. You will learn how to: - Introduction to Kubernetes and DC/OS (including the differences between both) - Deploy Kubernetes on DC/OS in a secure, highly available, and fault-tolerant manner - Solve operational challenges of running a large/multiple Kubernetes cluster - One-click deploy big data stateful and stateless services alongside a Kubernetes cluster

Kubernetes One-Click Deployment: Hands-on Workshop (Munich)

QAware GmbH

Docker provides PODA (Package Once Deploy Anywhere) and complements WORA (Write Once Run Anywhere) provided by Java. It also helps you reduce the impedance mismatch between dev, test, and production environment and simplifies Java application deployment. In this talk, Arun Gupta, Java Champion and Docker Captain and Fabiane Nardon, Java Champion, will explain how to run and package your Java application with Docker including sharing your Java application using Docker Hub. In addition, they will cover: * Deploying your Java application using Maven * Deploying your application using Docker for AWS * Scaling Java services with Docker Engine swarm mode * Packaging your multi-container application and use service discovery * Monitoring your Docker + Java applications * Building a deployment pipeline using common tools.

Docker for Java Developers - Fabiane Nardon and Arun gupta

Docker, Inc.

How to integrate Kubernetes in OpenStack: You need to know these project

inwin stack

Running and Managing Kubernetes on OpenStack

Victor Palma

GlueCon kubernetes & container engine

brendandburns

Mattia Gandolfi - Improving utilization and portability with Containers and C...

Codemotion

GitOps A/B testing with Istio and Helm

Weaveworks

Building Clustered Applications with Kubernetes and Docker

Steve Watt

A Million ways of Deploying a Kubernetes Cluster

Jimmy Lu

What's hot (20)

Demystifying the Nuts & Bolts of Kubernetes Architecture

KUBEBOOT - SPRING BOOT DEPLOYMENT ON KUBERNETES HAS NEVER BEEN SO EASY

Building Cloud-Native Applications with Kubernetes, Helm and Kubeless

Running Production-Grade Kubernetes on AWS

The Operator Pattern - Managing Stateful Services in Kubernetes

KubeCon EU 2016: Heroku to Kubernetes

4K–Kubernetes with Knative, Kafka and Kamel

Introduction to Kubernetes RBAC

CI/CD with Openshift and Jenkins

Setting up CI/CD pipeline with Kubernetes and Kublr step-by-step

OSDC 2018 | Git things done with GitLab by Gabriel Hartmann and Nicole Lang

Kubernetes One-Click Deployment: Hands-on Workshop (Munich)

Docker for Java Developers - Fabiane Nardon and Arun gupta

How to integrate Kubernetes in OpenStack: You need to know these project

Running and Managing Kubernetes on OpenStack

GlueCon kubernetes & container engine

Mattia Gandolfi - Improving utilization and portability with Containers and C...

GitOps A/B testing with Istio and Helm

Building Clustered Applications with Kubernetes and Docker

A Million ways of Deploying a Kubernetes Cluster

Similar to Implementing an Automated Staging Environment

Machine Learning (ML) is quite popular today in the academic/research world. However, it is quite difficult to put into a product, especially a product with a huge customer base. This session will give Kubeflow, the open source ML toolkit on top of Kubernetes, a deep look from the MLOps perspective. Furthermore, we will have a brief look at Distributed MLSys and how Kubeflow copes with scalability. Last, a demo of the Kubeflow stack setup and ML project pipeline deployment will be demonstrated in the cloud.

Bring-your-ML-Project-into-Production-v2.pdf

Liang Yan

HOW TO DRONE.IO IN CI/CD WORLD

Aleksandr Maklakov

Deploying Web Apps with PaaS and Docker Tools

Eddie Lau

What's New in Docker - February 2017

Patrick Chanezon

Getting Started with MariaDB with Docker

MariaDB plc

Detailed Introduction To Docker

nklmish

This talk introduced both OpenFaaS & OpenFaaS Cloud before demonstrating the Kafka Connector. Based on the Connector SDK, the Kafka Connector enables Kubernetes based functions to be triggered using messages consumed from Kafka topics. The connector easily integrates with your existing Kafka installations enabling you to flexibly complement your existing systems' functionality with serverless functions.

Staying on Topic - Invoke OpenFaaS functions with Kafka

Richard Gee

20150425 experimenting with openstack sahara on docker

Wei Ting Chen

Experts Live Europe 2017 - Why you should care about Docker - an introduction

Marc Müller

There is no shortage now of development and CI/CD tools for cloud-native application development. But how do we put the cloud-native concept and think as the cloud-native way on the leftmost side of CI/CD pipeline. During developing phrase, the tools provided with cloud code can help you expedite iteration of source codes, run and debug cloud native applications in an easy and fast way, making cloud-native development turn into real-time process, reduce the gap between deployment and development. 現在不乏用於雲原生應用程序開發的開發和 CI/CD工具。但是，我們如何將雲原生概念放在的 CI/CD 流水線的最左側呢？在開發階段，如何用 Cloud code 協助您加快原始碼的迭代速度，以簡便快捷的方式運行和調用雲原生應用程序，使雲原生開發變為即使過程，縮小開發與部署之間的差

[20200720]cloud native develoment - Nelson Lin

HanLing Shen

Docker Seattle Meetup April 2015 - The Docker Orchestration Ecosystem on Azure

Patrick Chanezon

Combining Docker, cloud infrastructure, and continuous integration and delivery practices can create a highly automated and efficient way to get new applications and features to market. The RightScale development team has been using Docker from development to continuous integration, and now the operations team has taken Docker into the production environment. The Docker in Production: How RightScale Delivers Cloud Applications webinar will cover: Approach and use case for adopting Docker How RightScale has adopted Docker for development, CI, and production Overcoming technical and process challenges The RightScale process before and after Docker Benefits for both developers and operations teams

Docker in Production: How RightScale Delivers Cloud Applications

RightScale

Docker New York Meetup May 2015 - The Docker Orchestration Ecosystem on Azure

Patrick Chanezon

Docker introduction

Marcelo Ochoa

Docker San Francisco Meetup April 2015 - The Docker Orchestration Ecosystem o...

Patrick Chanezon

CCCEU15 run cloudstack in docker

Pierre-Luc Dion

CloudStack Collab Conference 2015 Run CloudStack in Docker

CloudOps2005

Docker Mentorweek beginner workshop notes

Sreenivas Makam

Cloud and agile software projects: Overview and Benefits

Guillaume Berche

OpenCloudDay 2014: Deploying trusted developer sandboxes in Amazon's cloud

Netcetera

Similar to Implementing an Automated Staging Environment (20)

Bring-your-ML-Project-into-Production-v2.pdf

HOW TO DRONE.IO IN CI/CD WORLD

Deploying Web Apps with PaaS and Docker Tools

What's New in Docker - February 2017

Getting Started with MariaDB with Docker

Detailed Introduction To Docker

Staying on Topic - Invoke OpenFaaS functions with Kafka

20150425 experimenting with openstack sahara on docker

Experts Live Europe 2017 - Why you should care about Docker - an introduction

[20200720]cloud native develoment - Nelson Lin

Docker Seattle Meetup April 2015 - The Docker Orchestration Ecosystem on Azure

Docker in Production: How RightScale Delivers Cloud Applications

Docker New York Meetup May 2015 - The Docker Orchestration Ecosystem on Azure

Docker introduction

Docker San Francisco Meetup April 2015 - The Docker Orchestration Ecosystem o...

CCCEU15 run cloudstack in docker

CloudStack Collab Conference 2015 Run CloudStack in Docker

Docker Mentorweek beginner workshop notes

Cloud and agile software projects: Overview and Benefits

OpenCloudDay 2014: Deploying trusted developer sandboxes in Amazon's cloud

Recently uploaded

AI+A11Y 11MAY2024 HYDERBAD GAAD 2024 - HelloA11Y (11 May 2024)

Samir Dash

Key topics covered: - Real-world examples of Choreo's comprehensive coverage from application design and deployment, security, scaling, and monitoring - Running different types of workloads, such as web applications, APIs, microservices, integrations, and tasks at scale, and wire them together to deliver seamless omnichannel digital experiences - How Choreo improves the developer experience by eliminating repetition, silos, and redundancy through enhanced discoverability and self-serviceability

Choreo: Empowering the Future of Enterprise Software Engineering

WSO2

Six Myths about Ontologies: The Basics of Formal Ontology

johnbeverley2021

Decarbonising Commercial Real Estate: The Role of Operational Performance

IES VE

Retrieval augmented generation (RAG) is the most popular style of large language model application to emerge from 2023. The most basic style of RAG works by vectorizing your data and injecting it into a vector database like Milvus for retrieval to augment the text output generated by an LLM. This is just the beginning. One of the ways that we can extend RAG, and extend AI, is through multilingual use cases. Typical RAG is done in English using embedding models that are trained in English. In this talk, we’ll explore how RAG could work in languages other than English. We’ll explore French, Chinese, and Polish.

Introduction to Multilingual Retrieval Augmented Generation (RAG)

Zilliz

Dubai, often portrayed as a shimmering oasis in the desert, faces its own set of challenges, including the occasional threat of flooding. Despite its reputation for opulence and modernity, the emirate is not immune to the forces of nature. In recent years, Dubai has experienced sporadic but significant floods, testing the resilience of its infrastructure and communities. Among the critical lifelines in this bustling metropolis is the Dubai International Airport, a bustling hub that connects the city to the world. This article explores the intersection of Dubai flood events and the resilience demonstrated by the Dubai International Airport in the face of such challenges.

Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...

Orbitshub

Stronger Together: Developing an Organizational Strategy for Accessible Desig...

caitlingebhard1

In this session, we will discuss the journey of API governance from its initial, ungoverned state to the development of sophisticated models that tackle contemporary challenges. We'll explore how APIs have become essential in the intersection of business and technology, adapting to advancements and evolving needs. We'll focus on how organizations have moved from launching to monetizing APIs, using models like pay-per-use and subscriptions, and finding the right balance between technical implementation and business strategy. We'll also highlight the impact of governance on monetization strategies, especially how data security, compliance, and service quality influence pricing. Real-world examples will demonstrate the effective integration of governance with monetization, including AI's role in dynamic pricing. Looking ahead, we'll share insights into future trends in API governance and monetization, emphasizing the importance of adaptability, continuous learning, and innovation.

API Governance and Monetization - The evolution of API governance

WSO2

Webinar Recording: https://www.panagenda.com/webinars/why-teams-call-analytics-is-critical-to-your-entire-business Nothing is as frustrating and noticeable as being in an important call and being unable to see or hear the other person. Not surprising then, that issues with Teams calls are among the most common problems users call their helpdesk for. Having in depth insight into everything relevant going on at the user’s device, local network, ISP and Microsoft itself during the call is crucial for good Microsoft Teams Call quality support. To ensure a quick and adequate solution and to ensure your users get the most out of their Microsoft 365. But did you know that ‘bad calls’ are also an excellent indicator of other problems arising? Precisely because it is so noticeable!? Like the canary in the mine, bad calls can be early indicators of problems. Problems that might otherwise not have been noticed for a while but can have a big impact on productivity and satisfaction. Join this session by Christoph Adler to learn how true Microsoft Teams call quality analytics helped other organizations troubleshoot bad calls and identify and fix problems that impacted Teams calls or the use of Microsoft365 in general. See what it can do to keep your users happy and productive! In this session we will cover - Why CQD data alone is not enough to troubleshoot call problems - The importance of attributing call problems to the right call participant - What call quality analytics can do to help you quickly find, fix-, and prevent problems - Why having retrospective detailed insights matters - Real life examples of how others have used Microsoft Teams call quality monitoring to problem shoot problems with their ISP, network, device health and more.

Why Teams call analytics are critical to your entire business

panagenda

Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows. We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases. This video focuses on the deployment of external web forms using Jotform for Bonterra Impact Management. This solution can be customized to your organization’s needs and deployed to support the common use cases below: - Intake and consent - Assessments - Surveys - Applications - Program registration Interested in deploying web form automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.

Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...

Jeffrey Haguewood

Understanding the FAA Part 107 License ..

Christopher Logan Kennedy

Join our latest Connector Corner webinar to discover how UiPath Integration Service revolutionizes API-centric automation in a 'Quote to Cash' process—and how that automation empowers businesses to accelerate revenue generation. A comprehensive demo will explore connecting systems, GenAI, and people, through powerful pre-built connectors designed to speed process cycle times. Speakers: James Dickson, Senior Software Engineer Charlie Greenberg, Host, Product Marketing Manager

Connector Corner: Accelerate revenue generation using UiPath API-centric busi...

DianaGray10

In the dynamic field of DevOps, the quest for efficiency and productivity is endless. This talk introduces a revolutionary toolkit: Large Language Models (LLMs), including ChatGPT, Gemini, and Claude, extending far beyond traditional coding assistance. We'll explore how LLMs can automate not just code generation, but also transform day-to-day operations such as crafting compelling cover letters for TPS reports, streamlining client communications, and architecting innovative DevOps solutions. Attendees will learn effective prompting strategies and examine real-life use cases, demonstrating LLMs' potential to redefine productivity in the DevOps landscape. Join us to discover how to harness the power of LLMs for a comprehensive productivity boost across your DevOps activities.

ChatGPT and Beyond - Elevating DevOps Productivity

VictorSzoltysek

Discover the innovative features and strategic vision that keep WSO2 an industry leader. Explore the exciting 2024 roadmap of WSO2 API management, showcasing innovations, unified APIM/APK control plane, natural language API interaction, and cloud native agility. Discover how open source solutions, microservices architecture, and cloud native technologies unlock seamless API management in today's dynamic landscapes. Leave with a clear blueprint to revolutionize your API journey and achieve industry success!

WSO2's API Vision: Unifying Control, Empowering Developers

WSO2

In the ever-evolving landscape of data management, Zero-ETL is an approach that is reshaping how businesses handle and integrate their data. This webinar explores Zero-ETL, a paradigm shift from the traditional Extract, Transform, Load (ETL) process, offering a more streamlined, efficient, and real-time data integration method. We will begin with an introduction to the concept of Zero-ETL, including how it allows direct access to data in its native environment and real-time data transformation, providing up-to-date information with significantly reduced data redundancy. Next, we'll take you through several demonstrations showing how Zero-ETL can deliver real-time data and enable the free movement of data between systems. We will also discuss the various tools that support all aspects of Zero-ETL, providing attendees with an understanding of how they can adopt this innovative approach in their organizations. Lastly, the session will conclude with an interactive Q&A segment, allowing participants to gain deeper insights into how Zero-ETL can be tailored to their specific business needs and how they can get started today. Join us to discover how Zero-ETL can elevate your organization's data strategy.

The Zero-ETL Approach: Enhancing Data Agility and Insight

Safe Software

In this keynote, Asanka Abeysinghe, CTO,WSO2 will explore the shift towards platformless technology ecosystems and their importance in driving digital adaptability and innovation. We will discuss strategies for leveraging decentralized architectures and integrating diverse technologies, with a focus on building resilient, flexible, and future-ready IT infrastructures. We will also highlight WSO2's roadmap, emphasizing our commitment to supporting this transformative journey with our evolving product suite.

Platformless Horizons for Digital Adaptability

WSO2

JohnPollard-hybrid-app-RailsConf2024.pptx

JohnPollard37

Corporate and higher education. Two industries that, in the past, have had a clear divide with very little crossover. The difference in goals, learning styles and objectives paved the way for differing learning technologies platforms to evolve. Now, those stark lines are blurring as both sides are discovering they have content that’s relevant to the other. Join Tammy Rutherford as she walks through the pros and cons of corporate and higher ed collaborating. And the challenges of these different technology platforms working together for a brighter future.

Corporate and higher education May webinar.pptx

Rustici Software

Exploring Multimodal Embeddings with Milvus

Zilliz

Quantum Leap in Next-Generation Computing

WSO2

Recently uploaded (20)

AI+A11Y 11MAY2024 HYDERBAD GAAD 2024 - HelloA11Y (11 May 2024)

Choreo: Empowering the Future of Enterprise Software Engineering

Six Myths about Ontologies: The Basics of Formal Ontology

Decarbonising Commercial Real Estate: The Role of Operational Performance

Introduction to Multilingual Retrieval Augmented Generation (RAG)

Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...

Stronger Together: Developing an Organizational Strategy for Accessible Desig...

API Governance and Monetization - The evolution of API governance

Why Teams call analytics are critical to your entire business

Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...

Understanding the FAA Part 107 License ..

Connector Corner: Accelerate revenue generation using UiPath API-centric busi...

ChatGPT and Beyond - Elevating DevOps Productivity

WSO2's API Vision: Unifying Control, Empowering Developers

The Zero-ETL Approach: Enhancing Data Agility and Insight

Platformless Horizons for Digital Adaptability

JohnPollard-hybrid-app-RailsConf2024.pptx

Corporate and higher education May webinar.pptx

Exploring Multimodal Embeddings with Milvus

Quantum Leap in Next-Generation Computing

Implementing an Automated Staging Environment

1. Building an automated Staging environment LESSONS LEARNED BY DANIEL OLIVEIRA

2. About Me danieloliv079 danieloliveira079 df@autobutler.de Daniel Oliveira Black Magic Ops

3. About Us https://www.autobutler.dk

4. Stack

5. Challenge Provide a staging environment where we can deploy our applications requiring as less human interaction/intervention as possible.

6. Toolbox

7. 1º Step: Dockerize 'Em All +

8. 2º Step: Cloud Provisioning + KOPShttps://github.com/kubernetes/kops (Network, Postgresql, Salt Master, Salt Minion)

9. 3º Step: From code to K8S https://Pull Request Dynamic URLs

10. Challenges / Solutions ■ Developers don't wanna learn shell commands / CD + Github integration ■ Where should I host my docker images? / Docker Cloud Private Registry ■ Docker building time / Base image for gems initial install ■ Database Isolation per deployment / Reuse or on-demand ■ AWS - K8S Cluster CPU and RAM Usage / Ops Microservices - Garbage collector

11. Challenges / Solutions ■ Dynamic URLs / Salt Formulas + AWS Route 53 + NGINX controllers ■ HTTPS / Letsencrypt ■ Jira Integration / Ops Microservice - Update Ticket and includes URLs ■ Logging and debugging / Weavescope ■ DRY / Saltstack states+pillars

12. ■ Integrations / Third-Party APIs (Github, Jira, Internal) ■ Deployments management / Deployment Dashboard (Built in house) ■ Pull request updates (source code) / Redeploy using dashboard or slack bot ■ Linked environments / Rails API + React Front-End Challenges / Solutions

13. Challenges / Solutions ■ Manual deployments / Slack Bot ■ Manual Destroy / Deployment dashboard option

14. Deployment workflow Pull Request webhook Update Notify Update

15. Deployment Dashboard

16. Logging and Monitoring - Weavescope

17. Logging and Monitoring - Weavescope

18. Results ■ Sandbox environment ■ Product team can validate features and bug fixes ■ End-to-end testing ■ AWS - K8S Cluster CPU and RAM / Ops Microservices - Garbage collector

19. Future ■ Decrease rails application image size ■ Helm Charts - https://helm.sh/ ■ Ops FaaS - https://github.com/alexellis/faas ■ Move off from Heroku to AWS / Kubernetes ■ Logging and Monitoring Aggregation / Prometheus + Sysdig + Datadog

20. From Code to Kubernetes - Slideshare

21. Questions ?

22. Thank you ?

Implementing an Automated Staging Environment

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Implementing an Automated Staging Environment

Similar to Implementing an Automated Staging Environment (20)

Recently uploaded

Recently uploaded (20)

Implementing an Automated Staging Environment