Bootstrapping a Kubernetes cluster is easy, rolling it out to nearly 200 engineering teams and operating it at scale is a challenge. In this talk, we are presenting our approach to Kubernetes provisioning on AWS, operations and developer experience for our growing Zalando Technology department. We will highlight in the context of Kubernetes: AWS service integrations, our IAM/OAuth infrastructure, cluster autoscaling, continuous delivery and general developer experience. The talk will cover our most important learnings and we will openly share failure stories.
Talk given at Container Days HH (https://containerdays.io/) on 2017-06-20.
From AWS/STUPS to Kubernetes on AWS @Zalando - Berlin Kubernetes MeetupHenning Jacobs
This talk will highlight our challenges while migrating from our STUPS infrastructure (Docker on EC2, Cloud Formation) to Kubernetes on AWS.
Talk was held at Berlin Kubernetes Meetup on 2017-05-18: https://www.meetup.com/Berlin-Kubernetes-Meetup/events/239313998/
Kubernetes on AWS @Zalando - Berlin AWS User Group 2017-05-09Henning Jacobs
In this talk we share our learnings from running Kubernetes on AWS in production and how we are migrating 200+ engineering teams from AWS/STUPS to Kubernetes.
This talk was given at the Berlin AWS User Group meetup on 2017-05-09 hosted by NewStore (https://www.meetup.com/aws-berlin/events/236795816/).
More information on http://kubernetes-on-aws.readthedocs.io/en/latest/admin-guide/kubernetes-in-production.html
Developer Journey at Zalando - Idea to Production with Containers in the Clou...Henning Jacobs
Talk held on R-ETAIL:CODE in London on 2018-03-15.
- The history of how DevOps evolved at Zalando: from on-premise data centers to autonomous teams, microservices and cluster management in the cloud
- How the developer experience looks like for the application lifecycle from idea to production and what our vision for the future is
- Challenges and learnings from our past experiences: why architecture principles and constraints are important to lead 200+ engineering teams
Kubernetes on AWS at Europe's Leading Online Fashion PlatformHenning Jacobs
Henning Jacobs is a Kubernetes on AWS Hacker at Zalando Tech. His talk briefly covers our learnings in Zalando Tech while running Kubernetes on AWS in production.
Topics include:
- Cluster provisioning,
- AWS integration,
- Ingress,
- Cluster autoscaling,
- OAuth/IAM and
- Operations/monitoring.
https://www.meetup.com/Zalando-Tech-Events-Berlin/events/238212872/
Connexion is an open source API first REST framework for Python, built on top of Flask and based on OpenAPI/Swagger, targeted for microservice development. Connexion automagically handles request routing, oauth2 security, request validation and response serialization based on an OpenAPI 2.0 Specification file in YAML, so you don’t have to care about boilerplate anymore.
Because it is based on Flask it supports everything that Flask does, including deployment options and extensions.
At Zalando we’ve adopted “API First” as one of our key engineering principles, to ensure our API are robust, consistent, general and
abstracted from specific implementation and use cases. But when we tried to implement this principle for the first time we were faced with the lack of a python framework to achieve it in a easy fashion - there were several frameworks that produce a swagger definition from the
implementation but none that do it the other way around - so we decided to fill that gap.
Henning will show how to get started with OpenAPI+Connexion, present some real-world use cases and deployment options such as Kubernetes.
Plan B: Service to Service Authentication with OAuthHenning Jacobs
In this talk we show how our 85+ Zalando engineering teams are using OAuth 2.0 to secure their growing fleet of microservices running with Docker on AWS. We describe how and why we build our open source “Plan B” OAuth components from scratch to provide a resilient, highly available infrastructure with no single point of failure.
We built Plan B as a distributed OAuth provider and validation endpoint by using proven technologies such as Cassandra and JSON Web Tokens (JWT). We show how token revocations are possible without any central token storage and how we avoid network latency on token validation.
From AWS/STUPS to Kubernetes on AWS @Zalando - Berlin Kubernetes MeetupHenning Jacobs
This talk will highlight our challenges while migrating from our STUPS infrastructure (Docker on EC2, Cloud Formation) to Kubernetes on AWS.
Talk was held at Berlin Kubernetes Meetup on 2017-05-18: https://www.meetup.com/Berlin-Kubernetes-Meetup/events/239313998/
Kubernetes on AWS @Zalando - Berlin AWS User Group 2017-05-09Henning Jacobs
In this talk we share our learnings from running Kubernetes on AWS in production and how we are migrating 200+ engineering teams from AWS/STUPS to Kubernetes.
This talk was given at the Berlin AWS User Group meetup on 2017-05-09 hosted by NewStore (https://www.meetup.com/aws-berlin/events/236795816/).
More information on http://kubernetes-on-aws.readthedocs.io/en/latest/admin-guide/kubernetes-in-production.html
Developer Journey at Zalando - Idea to Production with Containers in the Clou...Henning Jacobs
Talk held on R-ETAIL:CODE in London on 2018-03-15.
- The history of how DevOps evolved at Zalando: from on-premise data centers to autonomous teams, microservices and cluster management in the cloud
- How the developer experience looks like for the application lifecycle from idea to production and what our vision for the future is
- Challenges and learnings from our past experiences: why architecture principles and constraints are important to lead 200+ engineering teams
Kubernetes on AWS at Europe's Leading Online Fashion PlatformHenning Jacobs
Henning Jacobs is a Kubernetes on AWS Hacker at Zalando Tech. His talk briefly covers our learnings in Zalando Tech while running Kubernetes on AWS in production.
Topics include:
- Cluster provisioning,
- AWS integration,
- Ingress,
- Cluster autoscaling,
- OAuth/IAM and
- Operations/monitoring.
https://www.meetup.com/Zalando-Tech-Events-Berlin/events/238212872/
Connexion is an open source API first REST framework for Python, built on top of Flask and based on OpenAPI/Swagger, targeted for microservice development. Connexion automagically handles request routing, oauth2 security, request validation and response serialization based on an OpenAPI 2.0 Specification file in YAML, so you don’t have to care about boilerplate anymore.
Because it is based on Flask it supports everything that Flask does, including deployment options and extensions.
At Zalando we’ve adopted “API First” as one of our key engineering principles, to ensure our API are robust, consistent, general and
abstracted from specific implementation and use cases. But when we tried to implement this principle for the first time we were faced with the lack of a python framework to achieve it in a easy fashion - there were several frameworks that produce a swagger definition from the
implementation but none that do it the other way around - so we decided to fill that gap.
Henning will show how to get started with OpenAPI+Connexion, present some real-world use cases and deployment options such as Kubernetes.
Plan B: Service to Service Authentication with OAuthHenning Jacobs
In this talk we show how our 85+ Zalando engineering teams are using OAuth 2.0 to secure their growing fleet of microservices running with Docker on AWS. We describe how and why we build our open source “Plan B” OAuth components from scratch to provide a resilient, highly available infrastructure with no single point of failure.
We built Plan B as a distributed OAuth provider and validation endpoint by using proven technologies such as Cassandra and JSON Web Tokens (JWT). We show how token revocations are possible without any central token storage and how we avoid network latency on token validation.
05.10.2017 AWS User Group Meetup - FALLACIES OF DISTRIBUTED COMPUTING WITH KU...Zalando adtech lab
Raffaele Di Fazio from Zalando in Berlin held this presentation on "Fallacies of Distributed Computing with Kubernetes on AWS" on the AWS USER GROUP MEETUP in the Zalando Hamburg Office on 5th October 2017
Vasilii Avdiushkin from Zalando adtech lab in Hamburg held this presentation called "Path to microservices" on the CLOJURE HAMBURG MEETUP in the Zalando adtech lab Office on 18th September 2017
STUPS by Zalando @WHD.local Frankfurt: STUPS.io - an Open Source Cloud Framew...Henning Jacobs
This talk shed lights on how Zalando, Europe’s leading eCommerce platform for fashion, has developed an architecture and set of open source cloud tools that scales as its technology team (100 autonomous engineering teams and counting) grows and evolves. It also describes the challenges regarding compliance and innovation that we face, and how we built our cloud infrastructure on top of AWS and Docker to manage these challenges.
Talk given by Henning Jacobs at the WHD.local event in Frankfurt on September 10th 2015.
Avec des millions de conteneurs démarrés tous les mois pour ses propres services (GMail, Search, Maps…) Google a développé un savoir-faire partagé dans de nombreux papiers de recherche, mais aussi désormais dans des projets open source comme Kubernetes et dans ses produits Google Cloud Platform. Cette session sera l'occasion de comprendre l'intérêt et la portabilité d'applications développées sur Kubernetes, de présenter certaines caractéristiques techniques de son architecture, ses dernières évolutions, quelques exemples de déploiements, et enfin de faire un point sur la communauté grandissante qui développe et enrichi le projet Kubernetes.
Par Alexis Moussine-Pouchkine (Developer Relations @ Google Cloud Platform)
Toutes les vidéos des conférences seront disponibles sur Xebia.tv
Presented at GR8Conf US 2016 in Minneapolis, MN.
You've got your microservices, cloud config & Eureka servers running in development, but how do you take that to production? A single instance of each isn't a viable solution for these critical services. In this talk, we'll walk through how to configure and run the cloud config and Eureka servers in a high availability manner. We will also cover other production tasks like pushing config updates and monitoring all your Hystrix circuit breakers using Turbine. This is not an introductory talk and assumes a basic familiarity with the Spring Cloud Services.
Automating Kubernetes Environments with AnsibleTimothy Appnel
Ansible fits naturally into any Kubernetes environment. Both are very active and widely used open source projects with vibrant communities that help make hard things easier. Here, we explore ways how...
DevOps and Hybrid Applications: What You Need to KnowDevOps.com
Although continuous delivery (CD) and DevOps are growing in popularity, there is not much practical information available about how to get started and build automated pipelines across containers and traditional infrastructure.
Particularly, most applications today follow the hybrid model, combining components deployed on containers with services that run on virtual machines (VMs) or public cloud services provided by Amazon Web Services, Google Cloud Platform, and Microsoft Azure.
In this presentation, we’ll provide an overview on how to automate software delivery for hybrid applications that stretch over containers, VMs, and public clouds—from source code commit to deployment in production using Jenkins, Red Hat OpenShift, and Red Hat Ansible Automation—from the ground up.
We’ll cover the basics around continuous delivery and DevOps concepts. However, no previous experience with these practices is required.
[Workshop] "Vuetify in practice", Alexander StepanovFwdays
How and why to use the Vuetify component library
My personal journey of using Vuetify
What you can build with that library
A brief overview of available components
Customization, validation, i18n, and other aspects
Example on a real project. We will develop a page with a typical booking process using Vuetify
Using source code management patterns to configure and secure your Kubernetes...Giovanni Galloro
In this session we will show how to set up, from scratch, a git repository to centrally manage, with Anthos Config Management, all the configurations and security policies of multiple Kubernetes clusters in different environments, using git as the source of truth and applying the processes typically used in source code lifecycle. We will also explore what is possible to do with ACM Policy Controller, based on Open Policy Agent Gatekeeper, and configure constraints to enforce many of the possible security policies that an enterprise organization would require.
Serverless Workflow: New approach to Kubernetes service orchestration | DevNa...Red Hat Developers
With the rise of Serverless Architectures, Workflows have gained a renewed interest and usefulness. Typically thought of as centralized and monolithic, they now play a key role in service orchestration and coordination as well as modular processing. With many different architecture approaches already in place, the Cloud Native Computing Foundation (CNCF) has started an initiative to specify serverless workflows to ensure portability and vendor neutrality. In this talk, we introduce the CNCF Serverless Workflow specification and provide examples and demos on top of Kogito, Red Hat's business automation toolkit. You will learn: 1- The what, why, and how of the CNCF Serverless Workflow specification 2- Why using the Serverless Workflow specification and orchestration can improve your serverless architecture 3- When to use CNCF Serverless Workflow and Kogito together and the benefits derived.
How Zalando runs Kubernetes clusters at scale on AWS - AWS re:InventHenning Jacobs
Many clusters, many problems? Having many clusters has benefits: reduced blast radius, less vertical scaling of cluster components, and a natural trust boundary. In this session, Zalando shows its approach for running 140+ clusters on AWS, how it does continuous delivery for its cluster infrastructure, and how it created open-source tooling to manage cost efficiency and improve developer experience. The company openly shares its failures and the learnings collected during three years of Kubernetes in production.
AWS re:Invent session OPN211 on 2019-12-05
Large Scale Kubernetes on AWS at Europe's Leading Online Fashion Platform - A...Henning Jacobs
Bootstrapping a Kubernetes cluster is easy, rolling it out to nearly 200 engineering teams and operating it at scale is a challenge.
In this talk, we are presenting our approach to Kubernetes provisioning on AWS, operations and developer experience for our growing Zalando Technology department. We will highlight in the context of Kubernetes: AWS service integrations, our IAM/OAuth infrastructure, cluster autoscaling, continuous delivery and general developer experience. The talk will cover our most important learnings and we will openly share failure stories.
Presented on 2017-09-28 at AWS Tech Community Days in Cologne.
05.10.2017 AWS User Group Meetup - FALLACIES OF DISTRIBUTED COMPUTING WITH KU...Zalando adtech lab
Raffaele Di Fazio from Zalando in Berlin held this presentation on "Fallacies of Distributed Computing with Kubernetes on AWS" on the AWS USER GROUP MEETUP in the Zalando Hamburg Office on 5th October 2017
Vasilii Avdiushkin from Zalando adtech lab in Hamburg held this presentation called "Path to microservices" on the CLOJURE HAMBURG MEETUP in the Zalando adtech lab Office on 18th September 2017
STUPS by Zalando @WHD.local Frankfurt: STUPS.io - an Open Source Cloud Framew...Henning Jacobs
This talk shed lights on how Zalando, Europe’s leading eCommerce platform for fashion, has developed an architecture and set of open source cloud tools that scales as its technology team (100 autonomous engineering teams and counting) grows and evolves. It also describes the challenges regarding compliance and innovation that we face, and how we built our cloud infrastructure on top of AWS and Docker to manage these challenges.
Talk given by Henning Jacobs at the WHD.local event in Frankfurt on September 10th 2015.
Avec des millions de conteneurs démarrés tous les mois pour ses propres services (GMail, Search, Maps…) Google a développé un savoir-faire partagé dans de nombreux papiers de recherche, mais aussi désormais dans des projets open source comme Kubernetes et dans ses produits Google Cloud Platform. Cette session sera l'occasion de comprendre l'intérêt et la portabilité d'applications développées sur Kubernetes, de présenter certaines caractéristiques techniques de son architecture, ses dernières évolutions, quelques exemples de déploiements, et enfin de faire un point sur la communauté grandissante qui développe et enrichi le projet Kubernetes.
Par Alexis Moussine-Pouchkine (Developer Relations @ Google Cloud Platform)
Toutes les vidéos des conférences seront disponibles sur Xebia.tv
Presented at GR8Conf US 2016 in Minneapolis, MN.
You've got your microservices, cloud config & Eureka servers running in development, but how do you take that to production? A single instance of each isn't a viable solution for these critical services. In this talk, we'll walk through how to configure and run the cloud config and Eureka servers in a high availability manner. We will also cover other production tasks like pushing config updates and monitoring all your Hystrix circuit breakers using Turbine. This is not an introductory talk and assumes a basic familiarity with the Spring Cloud Services.
Automating Kubernetes Environments with AnsibleTimothy Appnel
Ansible fits naturally into any Kubernetes environment. Both are very active and widely used open source projects with vibrant communities that help make hard things easier. Here, we explore ways how...
DevOps and Hybrid Applications: What You Need to KnowDevOps.com
Although continuous delivery (CD) and DevOps are growing in popularity, there is not much practical information available about how to get started and build automated pipelines across containers and traditional infrastructure.
Particularly, most applications today follow the hybrid model, combining components deployed on containers with services that run on virtual machines (VMs) or public cloud services provided by Amazon Web Services, Google Cloud Platform, and Microsoft Azure.
In this presentation, we’ll provide an overview on how to automate software delivery for hybrid applications that stretch over containers, VMs, and public clouds—from source code commit to deployment in production using Jenkins, Red Hat OpenShift, and Red Hat Ansible Automation—from the ground up.
We’ll cover the basics around continuous delivery and DevOps concepts. However, no previous experience with these practices is required.
[Workshop] "Vuetify in practice", Alexander StepanovFwdays
How and why to use the Vuetify component library
My personal journey of using Vuetify
What you can build with that library
A brief overview of available components
Customization, validation, i18n, and other aspects
Example on a real project. We will develop a page with a typical booking process using Vuetify
Using source code management patterns to configure and secure your Kubernetes...Giovanni Galloro
In this session we will show how to set up, from scratch, a git repository to centrally manage, with Anthos Config Management, all the configurations and security policies of multiple Kubernetes clusters in different environments, using git as the source of truth and applying the processes typically used in source code lifecycle. We will also explore what is possible to do with ACM Policy Controller, based on Open Policy Agent Gatekeeper, and configure constraints to enforce many of the possible security policies that an enterprise organization would require.
Serverless Workflow: New approach to Kubernetes service orchestration | DevNa...Red Hat Developers
With the rise of Serverless Architectures, Workflows have gained a renewed interest and usefulness. Typically thought of as centralized and monolithic, they now play a key role in service orchestration and coordination as well as modular processing. With many different architecture approaches already in place, the Cloud Native Computing Foundation (CNCF) has started an initiative to specify serverless workflows to ensure portability and vendor neutrality. In this talk, we introduce the CNCF Serverless Workflow specification and provide examples and demos on top of Kogito, Red Hat's business automation toolkit. You will learn: 1- The what, why, and how of the CNCF Serverless Workflow specification 2- Why using the Serverless Workflow specification and orchestration can improve your serverless architecture 3- When to use CNCF Serverless Workflow and Kogito together and the benefits derived.
How Zalando runs Kubernetes clusters at scale on AWS - AWS re:InventHenning Jacobs
Many clusters, many problems? Having many clusters has benefits: reduced blast radius, less vertical scaling of cluster components, and a natural trust boundary. In this session, Zalando shows its approach for running 140+ clusters on AWS, how it does continuous delivery for its cluster infrastructure, and how it created open-source tooling to manage cost efficiency and improve developer experience. The company openly shares its failures and the learnings collected during three years of Kubernetes in production.
AWS re:Invent session OPN211 on 2019-12-05
Large Scale Kubernetes on AWS at Europe's Leading Online Fashion Platform - A...Henning Jacobs
Bootstrapping a Kubernetes cluster is easy, rolling it out to nearly 200 engineering teams and operating it at scale is a challenge.
In this talk, we are presenting our approach to Kubernetes provisioning on AWS, operations and developer experience for our growing Zalando Technology department. We will highlight in the context of Kubernetes: AWS service integrations, our IAM/OAuth infrastructure, cluster autoscaling, continuous delivery and general developer experience. The talk will cover our most important learnings and we will openly share failure stories.
Presented on 2017-09-28 at AWS Tech Community Days in Cologne.
Bitbucket Pipelines - Powered by KubernetesNathan Burrell
This talk covers how pipelines uses Kubernetes to power its builder infrastructure and shares some tips on running Kubernetes at scale in a secure way.
This presentation was presented to the sydney Kubernetes meetup on the 3rd of August 2017.
Come costruire una Platform As A Service con Docker, Kubernetes Go e JavaCodemotion
"Come costruire una Platform As A Service con Docker, Kubernetes Go e Java" by Massimiliano Dessì
Per automatizzare la CI e la CD, durante sviluppo, test, in preproduzione e in produzione si utilizzano le tecniche chiamate attualmente DevOps, in locale con Vagrant oppure su una PAAS su cloud, privati o pubblici. Possiamo costruire una PAAS scalabile utilizzando solo Docker, Docker e Kubernetes oppure soluzioni già pronte come Openshift 3 (che sta sopra Docker e Kubernetes). Nella presentazione vedremo come avere questi tre tipi di PAAS con in più uno strato di orchestrazione in GO/Java e Ansible per automatizzare il comportamento in base ad eventi monitorati
Kubernetes Failure Stories, or: How to Crash Your Cluster - ContainerDays EU ...Henning Jacobs
Bootstrapping a Kubernetes cluster is easy, rolling it out to nearly 200 engineering teams and operating it at scale is a challenge. In this talk, we are presenting our approach to Kubernetes provisioning on AWS, operations and developer experience for our growing Zalando developer base. We will walk you through our horror stories of operating 100+ clusters and share the insights we gained from incidents, failures, user reports and general observations. Our failure stories will be sourced from recent and past incidents, so the talk will be up-to-date with our latest experiences.
In this two part presentation, Faheem Memon, Principal DevOps Architect @ Amobee did a recap of the AWS reInvent 2021 event with focus on cloud-native technologies. Jason Morgan, Tech Evangelist at Buoyant Inc reviewed k9s CLI.
Weaveworks at AWS re:Invent 2016: Operations Management with Amazon ECSWeaveworks
Alfonso described how Weave open source projects (Weave Net and Weave Scope) can help with networking, visualization, and control for ECS. Specifically, Weave acts as a key communicator for networking containers with its multi-host overlay and additional features (including automatic DNS service discovery and multicast).
Containers have been a driving force in this industry for the last 5+ years. In the meanwhile we have seen the raise of other compute patterns, such as serverless. 2020 seems to be the year where the line between containers and serverless starts to blurry. We are seeing the raise of container serverless platforms (e.g. AWS Fargate) as well as the raise of higher order abstractions above container platforms (e.g. OpenFaaS, ECS CLI v2, …) that allows developers to focus on their code instead of managing containers. In this session we will discuss how the serverless benefits are starting to permeate into the container ecosystem and we will provide real life examples of how some AWS and OSS technologies can be used to abstract and remove part of the undifferentiated heavy lifting developers often need to take care of.
Adopting Java for the Serverless world at JUG HamburgVadym Kazulkin
Java is for many years one of the most popular programming languages, but it used to have hard times in the Serverless Community. Java is known for its high cold start times and high memory footprint. For both you have to pay to the cloud providers of your choice. That's why most developers tried to avoid using Java for such use cases. But the times change: Community and cloud providers improve things steadily for Java developers. In this talk we look at the features and possibilities AWS cloud provider offers for the Java developers and look the most popular Java frameworks, like Micronaut, Quarkus and Spring (Boot) and look how (AOT compiler and GraalVM native images play a huge role) they address Serverless challenges and enable Java for broad usage in the Serverless world.
Managed services such as AWS Lambda and API Gateway allow developers to focus on value adding development instead of IT heavy lifting. This workshop introduces how to build a simple REST blog backend using AWS technologies and the serverless framework.
In this Cloud-Native era, it is difficult not to have heard about Kubernetes (k8s). Social media are flooded with technical articles on Kubernes and it is a popular topic at many cloud-related conferences. Major internet companies, such as Reddit or AirBnQ, have been switching their production environment to k8s. But why the fuss? What exactly is Kubernetes? What problem does it solve? Should you learn more about it? If you have asked yourself one of these questions this talk is for you.
During this talk you will:
* Understand the context and problem k8s is trying to solve
* Learn the basic building blocks and features of k8s
* Receive a list of resources to start your own journey with k8s
Recorded talk available here: https://t.co/ll1EFkpeUW?amp=1
Kubernetes is designed to be an extensible system. But what is the vision for Kubernetes Extensibility? Do you know the difference between webhooks and cloud providers, or between CRI, CSI, and CNI? In this talk we will explore what extension points exist, how they have evolved, and how to use them to make the system do new and interesting things. We’ll give our vision for how they will probably evolve in the future, and talk about the sorts of things we expect the broader Kubernetes ecosystem to build with them.
In this session, we introduce you to a solution for easily running a Docker-powered microservices architecture on AWS using Elastic Beanstalk. We will also cover the fundamentals of Elastic Beanstalk and how it benefits developers looking for a quick and scalable way to get their applications running on AWS with no infrastructure work required. In the second half of the session Sean O’Brien, engineer at Prezi, will share how Prezi is using Elastic Beanstalk to build microservices for its entire development team.
Building a microservices architecture using Docker can require a lot of work, from launching and operating the underlying infrastructure to installing and maintaining cluster management software. With AWS Elastic Beanstalk’s multicontainer support feature, many of these tasks are simplified and abstracted away so you can focus on your application code. AWS Elastic Beanstalk is an easy-to-use service for deploying and scaling web applications and services developed with Java, .NET, PHP, Node.js, Python, Ruby, Go, and Docker. Elastic Beanstalk leverages Amazon EC2 Container Service for its container management capabilities.
AWS DevDay San Francisco, June 21, 2016.
Presenter: Adhiraj Singh, Sr. Product Manager
Similar to Large Scale Kubernetes on AWS at Europe's Leading Online Fashion Platform - Container Days Hamburg (20)
Why I love Kubernetes Failure Stories and you should too - GOTO BerlinHenning Jacobs
Talk held on 2019-10-24 at GOTO Berlin:
Everybody loves failure stories, but maybe for the wrong reasons: Schadenfreude and Internet comment threads are the dark side; continuous improvement through blameless postmortems, sharing incidents, and documenting learnings is what motivated me to compile the list of Kubernetes Failure Stories. Kubernetes gives us a infrastructure platform to talk in the same "language" and foster collaboration across organizations. In this talk, I will walk you through our horror stories of operating 100+ clusters and share the insights we gained from incidents, failures, user reports and general observations. I will highlight why Kubernetes makes sense despite its perceived complexity. Our failure stories will be sourced from recent and past incidents, so the talk will be up-to-date with our latest experiences.
https://gotober.com/2019/sessions/1129/why-i-love-kubernetes-failure-stories-and-you-should-too
Why Kubernetes? Cloud Native and Developer Experience at Zalando - Enterprise...Henning Jacobs
Kubernetes hat sich als defacto Standard für Cloud Native Plattformen etabliert. Doch warum? Welche Vorteile und Fallstricke gibt es in der Praxis? Henning Jacobs zeigt am Beispiel von Zalando wie Kubernetes als Infrastruktur für 1200+ Entwickler dient, welche Aspekte Kubernetes trotz seiner Komplexität einzigartig machen, und was dies für die Developer Experience bedeutet.
Why Kubernetes? Cloud Native and Developer Experience at Zalando - OWL Tech &...Henning Jacobs
Talk held on 2019-09-26 in Paderborn:
Die Keynote:
Warum Kubernetes? Cloud Native und Developer Experience bei Zalando
Kubernetes hat sich als defacto Standard for Cloud Native Plattformen durchgesetzt. Warum? Welche Vorteile und Fallstricke gibt es in der Praxis?
Henning Jacobs zeigt am Beispiel von Zalando wie Kubernetes als Infrastruktur für 1200+ Entwickler dient, welche Aspekte Kubernetes trotz seiner Komplexität einzigartig machen, und was das für die Developer.
Experience bedeutet.
Henning Jacobs ist der Head of Developer Productivity bei Zalando und damit verantwortlich für die Developer Experience von mehr als 200 Zalando Delivery Teams.
Das Kubernetes eine hervorragende Plattform für den Erfahrungsaustausch darstellt, zeigt Henning mit seiner Liste von Kubernetes Failure Stories.
https://teuto.net/owl-tech-innovation-day/
While Go is the language-of-choice in the cloud-native world, Python has a huge community and makes it really easy to extend Kubernetes in only a few lines of code.
This talk shows examples on how to use Python to query the Kubernetes API, how to write simple controllers in only 10 lines of Python, how to build complete web UIs, and how to test everything with py.test and Kind.
Some of the open-source projects which will be covered: pykube-ng, Kubernetes Web View, kube-janitor, and Kopf (Kubernetes Operator Pythonic Framework).
Talk held in Prague on 2019-09-05:
https://www.meetup.com/Cloud-Native-Prague/events/263802447/
Why we don’t use the Term DevOps: the Journey to a Product Mindset - DevOpsCo...Henning Jacobs
While the adoption of DevOps makes teams move faster with reduced dependency on central operations, it can constrain teams who lack the skills to self-manage the full application and infrastructure stack. The way to overcome this challenge is creating an internal platform and treating it as a world-class product offering. “Applying product management to internal platforms means establishing empathy with internal consumers (read: developers) and collaborating with them on the design. Platform product managers establish roadmaps and ensure the platform delivers value to the business and enhances the developer experience”, via ThoughtWorks Technology Radar. In this talk, we will walk you through how Zalando adopted a customer-first mindset with regards to its developer tooling. We will show the effect on developer satisfaction when internal platforms are given the same respect as external product offerings. We will tell our story on how we moved from a classical infrastructure team to a product mindset with strong focus on building a world-class developer experience. We will share both our learnings and challenges going through this transition, and the impact it has on the daily life of our customers (developers).
Why we don’t use the Term DevOps: the Journey to a Product Mindset - Destinat...Henning Jacobs
While the adoption of DevOps makes teams move faster with reduced dependency on central operations, it can constrain teams who lack the skills to self-manage the full application and infrastructure stack.
The way to overcome this challenge is creating an internal platform and treating it as a world-class product offering. “Applying product management to internal platforms means establishing empathy with internal consumers (read: developers) and collaborating with them on the design. Platform product managers establish roadmaps and ensure the platform delivers value to the business and enhances the developer experience”, via ThoughtWorks Technology Radar.
In this talk, Henning Jacobs will walk you through how Zalando adopted a customer-first mindset with regards to its developer tooling. He will show the effect on developer satisfaction when internal platforms are given the same respect as external product offerings. Henning will furthermore tell his story about how Zalando moved from a classical infrastructure team to a product mindset with strong focus on building a world-class developer experience. Henning shares both their learnings and challenges going through this transition, and the impact it has on the daily life of Zalando’s customers (developers).
This talk was given in Aarhus on 4th of June 2019.
Kubernetes Failure Stories - KubeCon Europe BarcelonaHenning Jacobs
Talk given on 2019-05-21 at KubeCon Barcelona: https://kccnceu19.sched.com/event/MPcM/kubernetes-failure-stories-and-how-to-crash-your-clusters-henning-jacobs-zalando-se
Bootstrapping a Kubernetes cluster is easy, rolling it out to nearly 200 engineering teams and operating it at scale is a challenge. In this talk, we are presenting our approach to Kubernetes provisioning on AWS, operations and developer experience for our growing Zalando developer base. We will walk you through our horror stories of operating 100+ clusters and share the insights we gained from incidents, failures, user reports and general observations. Our failure stories will be sourced from recent and past incidents, so the talk will be up-to-date with our latest experiences.
Most of our learnings apply to other Kubernetes infrastructures (EKS, GKE, ..) as well. This talk strives to reduce the audience's unknown unknowns about running Kubernetes in production.
Optimizing Kubernetes Resource Requests/Limits for Cost-Efficiency and Latenc...Henning Jacobs
Talk given at JAX DevOps London on 2019-05-15.
Kubernetes has the concept of resource requests and limits. Pods get scheduled on the nodes based on their requests and optionally limited in how much of the resource they can consume. Understanding and optimizing resource requests/limits is crucial both for reducing resource "slack" and ensuring application performance/low-latency. This talk shows our approach to monitoring and optimizing Kubernetes resources for 90+ clusters to achieve cost-efficiency and reducing impact for latency-critical applications. All shown tools are open source and can be applied to most Kubernetes deployments. Topics covered in the talk include: understanding resource requests and limits, cgroups and CFS quota behavior, contributing factors to cluster costs (in public clouds), and best practices for managing Kubernetes resources.
Talk held at DevOps Gathering 2019 in Bochum on 2019-03-13.
Abstract: This talk will address one of the most common challenges of organizations adopting Kubernetes on a medium to large scale: how to keep cloud costs under control without babysitting each and every deployment and cluster configuration? How to operate 80+ Kubernetes clusters in a cost-efficient way for 200+ autonomous development teams?
This talk provides insights on how Zalando approaches this problem with central cost optimizations (e.g. Spot), cost monitoring/alerting, active measures to reduce resource slack, and automated cluster housekeeping. We will focus on how to ingrain cost efficiency in tooling and developer workflows while balancing rigid cost control with developer convenience and without impacting availability or performance. We will show our use case running Kubernetes on AWS, but all shown tools are open source and can be applied to most other infrastructure environments.
Developer Experience at Zalando - Handelsblatt Strategisches IT-Management 2019Henning Jacobs
Talk given at 25. Handelsblatt Jahrestagung Strategisches IT-Management in Munich on 2019-01-23. Original title (German): "Developer Experience bei Zalando: Entwicklerproduktivität steigern mit Cloud Native Infrastruktur"
- Wie macht man mehr als 1100 Entwickler glücklich und effektiv?
- Entwickler als Kunde: Produktmanagement für Plattformteams
- You build it – you run it: Self-Service-Infrastruktur mit Kubernetes und AWS
- Der Weg vom klassischen Infrastrukturteam zu Developer Productivity als Abteilung
Running Kubernetes in Production: A Million Ways to Crash Your Cluster - DevO...Henning Jacobs
Bootstrapping a Kubernetes cluster is easy, rolling it out to nearly 200 engineering teams and operating it at scale is a challenge. In this talk, we are presenting our approach to Kubernetes provisioning on AWS, operations and developer experience for our growing Zalando developer base.
We will walk you through our horror stories of operating 80+ clusters and share the insights we gained from incidents, failures, user reports and general observations.
Most of our learnings apply to other Kubernetes infrastructures (EKS, GKE, ..) as well.
This talk strives to reduce the audience’s unknown unknowns about running Kubernetes in production.
Optimizing Kubernetes Resource Requests/Limits for Cost-Efficiency and Latenc...Henning Jacobs
Kubernetes has the concept of resource requests and limits. Pods get scheduled on the nodes based on their requests and optionally limited in how much of the resource they can consume. Understanding and optimizing resource requests/limits is crucial both for reducing resource "slack" and ensuring application performance/low-latency. This talk shows our approach to monitoring and optimizing Kubernetes resources for 80+ clusters to achieve cost-efficiency and reducing impact for latency-critical applications. All shown tools are Open Source and can be applied to most Kubernetes deployments.
Running Kubernetes in Production: A Million Ways to Crash Your Cluster - Cont...Henning Jacobs
Bootstrapping a Kubernetes cluster is easy, rolling it out to nearly 200 engineering teams and operating it at scale is a challenge. In this talk, we are presenting our approach to Kubernetes provisioning on AWS, operations and developer experience for our growing Zalando developer base. We will walk you through our horror stories of operating 80+ clusters and share the insights we gained from incidents, failures, user reports and general observations. Most of our learnings apply to other Kubernetes infrastructures (EKS, GKE, ..) as well. This talk strives to reduce the audience’s unknown unknowns about running Kubernetes in production.
https://2018.container.camp/uk/schedule/running-kubernetes-in-production-a-million-ways-to-crash-your-cluster/
STUPS @ AWS Enterprise Web Day Oktober 2015Henning Jacobs
Cross-cutting Teams und Product Teams lösen immer mehr das klassische Pool-Modell ab. Das Motto “You build it, you run it” ist die Basis für DevOps. Zalando geht den Schritt weiter und arbeitet in autonomen Teams als "First Class Entity”. Autonome Teams mögen keine Abhängigkeiten, sonst wären sie nicht wirklich autonom.
Microservices in Verbindung mit einer Cloud Infrastruktur unterstützen Teams in ihrer Autonomie. Wie das in der Praxis mit AWS funktioniert und was dazu organisatorisch und architektonisch notwendig ist, präsentiert Henning Jacobs von Zalando.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
UiPath Test Automation using UiPath Test Suite series, part 3DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 3. In this session, we will cover desktop automation along with UI automation.
Topics covered:
UI automation Introduction,
UI automation Sample
Desktop automation flow
Pradeep Chinnala, Senior Consultant Automation Developer @WonderBotz and UiPath MVP
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Let's dive deeper into the world of ODC! Ricardo Alves (OutSystems) will join us to tell all about the new Data Fabric. After that, Sezen de Bruijn (OutSystems) will get into the details on how to best design a sturdy architecture within ODC.
Neuro-symbolic is not enough, we need neuro-*semantic*Frank van Harmelen
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
UiPath Test Automation using UiPath Test Suite series, part 4DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap.
The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies.
Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques
What will you get from this session?
1. Insights into SAP testing best practices
2. Heatmap utilization for testing
3. Optimization of testing processes
4. Demo
Topics covered:
Execution from the test manager
Orchestrator execution result
Defect reporting
SAP heatmap example with demo
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf91mobiles
91mobiles recently conducted a Smart TV Buyer Insights Survey in which we asked over 3,000 respondents about the TV they own, aspects they look at on a new TV, and their TV buying preferences.
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
Keynote at DIGIT West Expo, Glasgow on 29 May 2024.
Cheryl Hung, ochery.com
Sr Director, Infrastructure Ecosystem, Arm.
The key trends across hardware, cloud and open-source; exploring how these areas are likely to mature and develop over the short and long-term, and then considering how organisations can position themselves to adapt and thrive.
State of ICS and IoT Cyber Threat Landscape Report 2024 previewPrayukth K V
The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development.
The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers:
State of global ICS asset and network exposure
Sectoral targets and attacks as well as the cost of ransom
Global APT activity, AI usage, actor and tactic profiles, and implications
Rise in volumes of AI-powered cyberattacks
Major cyber events in 2024
Malware and malicious payload trends
Cyberattack types and targets
Vulnerability exploit attempts on CVEs
Attacks on counties – USA
Expansion of bot farms – how, where, and why
In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East
Why are attacks on smart factories rising?
Cyber risk predictions
Axis of attacks – Europe
Systemic attacks in the Middle East
Download the full report from here:
https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
2. 2
ZALANDO
15 markets
6 fulfillment centers
20 million active customers
3.6 billion € net sales 2016
165 million visits per month
12,000 employees in Europe
5. 5
FOUR ERAS AT ZALANDO TECH
ZOMCATPHP STUPS KUBERNETES
2010 2015 2016
Data center
WAR
AWS
Docker
Cloud Formation
Low level (AWS API)
AWS
Docker
Kubernetes manifest
High abstraction level
Data center
PHP files
18. 18
ASSIGNING AWS IAM ROLE TO POD
kind: Deployment
spec:
template:
metadata:
annotations:
# annotation for kube2iam
iam.amazonaws.com/role: "app-myapp-role"
spec:
containers:
- name: ...
...
https://github.com/jtblin/kube2iam
⇒ AWS SDKs just work as expected
20. 20
CLUSTER AUTOSCALING
Control # of worker nodes in ASG:
• Satisfy all resource requests
• One spare node per AZ
• No manual config “tweaking”
• Scale down, but not too fast
⇒ we want to be “elastic”
https://github.com/hjacobs/kube-aws-autoscaler
32. 32
GETTING STARTED
Goal: use Kubernetes API as primary interface for AWS
• Mate, External DNS
• Kubernetes Ingress Controller for AWS
• kube2iam
⇒ we wrote new components
to achieve our goal
35. 35
GETTING STARTED
Other questions we asked ourselves..
• Single AZ vs. Multi AZ? ⇒ Multi AZ
• Federation? ⇒ No, not ready yet
• Overlay network? ⇒ Flannel, “rock solid”
• Authnz? ⇒ OAuth, webhook
40. 40
STABILITY: AWS RATE LIMITS
• Ran into the same trap twice (Mate & Ingress Ctrl)
• Kubernetes core causes many calls (e.g. EBS)
• Monitoring (ZMON) needs to poll AWS
⇒ One of our biggest pain points with AWS
(and all workarounds are hard and/or ugly)
41. 41
STABILITY: LIMIT RANGE
kubectl describe limitrange
Name: limits
Namespace: default
Type Resource Min Max Default Req Default Limit Max Limit/Request Ratio
---- -------- --- --- ----------- ------------- -----------------------
Container memory - 64Gi 100Mi 1Gi -
Container cpu - 16 100m 3 -
http://kubernetes-on-aws.readthedocs.io/en/latest/admin-guide/kubernetes-in-production.html#resources
⇒ Mitigate errors on OSI layer 8 ;-)
45. 45
ONBOARDING
• Many new concepts to grasp vs. 200 teams
• Kubernetes Training (2h)
• Documentation
• Recorded Friday Demos
• Support Channels (chat, mail)
53. 53
LINKS
Running Kubernetes in Production on AWS
http://kubernetes-on-aws.readthedocs.io/en/latest/admin-guide/kubernetes-in-production.html
Kube AWS Ingress Controller
https://github.com/zalando-incubator/kube-ingress-aws-controller
External DNS
https://github.com/kubernetes-incubator/external-dns
PostgreSQL Operator
https://github.com/zalando-incubator/postgres-operator
Zalando Cluster Configuration
https://github.com/zalando-incubator/kubernetes-on-aws
List of Organizations using Kubernetes on AWS
https://github.com/hjacobs/kubernetes-on-aws-users