Open Source at Zalando - OSB Open Source Day 2019

•

3 likes•650 views

Talk given at the OSB Open Source Day (https://osb-alliance.de/news/allgemein/open-source-day-2019-seien-sie-dabei) on 2019-11-07.

OPEN SOURCE
AT ZALANDO
OSB OPEN SOURCE DAY
2019-11-07
HENNING JACOBS
@try_except_

2
~ 5.4billion EUR
revenue 2018
> 300
million
visits
per
month
~ 14,000
employees in
Europe
> 80%
of visits via
mobile devices
> 28
million
active customers
> 400,000
product choices
> 2,000
brands
17
countries
as of June 2019
ZALANDO AT A GLANCE

4
2008-2010:
BASED
ON MAGENTO
COMMUNITY EDITION

5
HOME-BREWED SOFTWARE
>1100 developers
>200 development teams
>2000 applications

6
SINCE 2010
• Java (OpenJDK)
• Apache Tomcat
• PostgreSQL
• Python

7
OSS WE BUILD ON
• Java (OpenJDK)
• Apache Tomcat
• PostgreSQL
• Python
• JS, Scala, Go, ..
• Kubernetes

Most open source code can be used in Zalando
projects, but everyone has a responsibility to ensure
that we respect and comply with the licenses, and
that we only adopt mature projects

10
POSTGRESQL AT ZALANDO
> 300
In on-premise data centers
> 700
Databases on AWS/Kubernetes

11
✓ ✓✓✓✓
Cloud native technologies empower organizations to build and run
scalable applications in modern, dynamic environments such as
public, private, and hybrid clouds.
CLOUD NATIVE

108 Companies in the End User Community
Plus 14 non-public members

13
CONTRIBUTING UPSTREAM
• We encourage contributions upstream
• Don’t share confidential information
• Code/non-code contributions: all sanctioned and
encouraged as part of employment at Zalando
• Only sign whitelisted CLAs

THERE IS A CULTURAL UNDERSTANDING
AND APPRECIATION OF OPEN SOURCE WORK
AT ZALANDO.

17
PATRONI
High-availability for PostgreSQL.
Widely used in production
github.com/zalando/patroni

18
POSTGRES OPERATOR
Application to manage
PostgreSQL clusters on
Kubernetes
>500
clusters running
on Kubernetes
github.com/zalando/postgres-operator

Elasticsearch in Kubernetes
Elasticsearch
2.500 vCPUs
1 TB RAM
github.com/zalando-incubator/es-operator/

20
EXTERNAL DNS
Application to manage DNS
records via Kubernetes objects
Part of Kubernetes organization.
184 contributors
github.com/kubernetes-sigs/external-dns

21
API GUIDELINES
opensource.zalando.com/restful-api-guidelines

22
API GUIDELINES
github.com/zalandoresearch/fashion-mnist

23
KUBERNETES CLUSTER CONFIGURATION
github.com/zalando-incubator/kubernetes-on-aws
47+ cluster
components

25
OPEN SOURCE COMMUNITY AT ZALANDO
OPEN SOURCE
REVIEW GROUP
DIVERSITY
OPEN
SOURCE
GUILD
SECURITY
EMPLOYER
BRANDING
COMPLIANCE
MANAGEMENT

27
RELEASING OPEN SOURCE
• Get sign-off by lead
• Be compliant
• Prepare your repository
• Get reviewed

28
BEING COMPLIANT
• MIT license
• Required files
• GitHub.com workflow
• Copyright and ownership

29
GOOD PRACTICES
• Code of Conduct
• Clear expectations
• Documentation
• "good first issue"
• Promote!

30
WE WANT OUR OSS PROJECTS TO BE
• High-quality
• Well documented
• Secure and compliant
• Actively maintained
• Beloved
• Used by the broader community

31
WHY RELEASE OPEN SOURCE?
• Collaboration
• Higher quality
• Sustainability
• Intrinsic motivation
• Employer Branding

“Empowerment of individuals is a key part of
what makes open source work, since in the end,
innovations tend to come from
small groups, not from large, structured efforts”
Tim O’Reilly

33
OPEN SOURCE & MORE
Open Source at Zalando
opensource.zalando.com
More Zalando Tech Talks
github.com/zalando/public-presentations

QUESTIONS?
HENNING JACOBS
SENIOR PRINCIPAL
henning@zalando.de
@try_except_
Illustrations by @01k

Many clusters, many problems? Having many clusters has benefits: reduced blast radius, less vertical scaling of cluster components, and a natural trust boundary. In this session, Zalando shows its approach for running 140+ clusters on AWS, how it does continuous delivery for its cluster infrastructure, and how it created open-source tooling to manage cost efficiency and improve developer experience. The company openly shares its failures and the learnings collected during three years of Kubernetes in production. AWS re:Invent session OPN211 on 2019-12-05

Why I love Kubernetes Failure Stories and you should too - GOTO Berlin

Henning Jacobs

Talk held on 2019-10-24 at GOTO Berlin: Everybody loves failure stories, but maybe for the wrong reasons: Schadenfreude and Internet comment threads are the dark side; continuous improvement through blameless postmortems, sharing incidents, and documenting learnings is what motivated me to compile the list of Kubernetes Failure Stories. Kubernetes gives us a infrastructure platform to talk in the same "language" and foster collaboration across organizations. In this talk, I will walk you through our horror stories of operating 100+ clusters and share the insights we gained from incidents, failures, user reports and general observations. I will highlight why Kubernetes makes sense despite its perceived complexity. Our failure stories will be sourced from recent and past incidents, so the talk will be up-to-date with our latest experiences. https://gotober.com/2019/sessions/1129/why-i-love-kubernetes-failure-stories-and-you-should-too

Why Kubernetes? Cloud Native and Developer Experience at Zalando - OWL Tech &...

Henning Jacobs

Talk held on 2019-09-26 in Paderborn: Die Keynote: Warum Kubernetes? Cloud Native und Developer Experience bei Zalando Kubernetes hat sich als defacto Standard for Cloud Native Plattformen durchgesetzt. Warum? Welche Vorteile und Fallstricke gibt es in der Praxis? Henning Jacobs zeigt am Beispiel von Zalando wie Kubernetes als Infrastruktur für 1200+ Entwickler dient, welche Aspekte Kubernetes trotz seiner Komplexität einzigartig machen, und was das für die Developer. Experience bedeutet. Henning Jacobs ist der Head of Developer Productivity bei Zalando und damit verantwortlich für die Developer Experience von mehr als 200 Zalando Delivery Teams. Das Kubernetes eine hervorragende Plattform für den Erfahrungsaustausch darstellt, zeigt Henning mit seiner Liste von Kubernetes Failure Stories. https://teuto.net/owl-tech-innovation-day/

Why we don’t use the Term DevOps: the Journey to a Product Mindset - DevOpsCo...

Henning Jacobs

While the adoption of DevOps makes teams move faster with reduced dependency on central operations, it can constrain teams who lack the skills to self-manage the full application and infrastructure stack. The way to overcome this challenge is creating an internal platform and treating it as a world-class product offering. “Applying product management to internal platforms means establishing empathy with internal consumers (read: developers) and collaborating with them on the design. Platform product managers establish roadmaps and ensure the platform delivers value to the business and enhances the developer experience”, via ThoughtWorks Technology Radar. In this talk, we will walk you through how Zalando adopted a customer-first mindset with regards to its developer tooling. We will show the effect on developer satisfaction when internal platforms are given the same respect as external product offerings. We will tell our story on how we moved from a classical infrastructure team to a product mindset with strong focus on building a world-class developer experience. We will share both our learnings and challenges going through this transition, and the impact it has on the daily life of our customers (developers).

Kubernetes + Python = ❤ - Cloud Native Prague

Henning Jacobs

While Go is the language-of-choice in the cloud-native world, Python has a huge community and makes it really easy to extend Kubernetes in only a few lines of code. This talk shows examples on how to use Python to query the Kubernetes API, how to write simple controllers in only 10 lines of Python, how to build complete web UIs, and how to test everything with py.test and Kind. Some of the open-source projects which will be covered: pykube-ng, Kubernetes Web View, kube-janitor, and Kopf (Kubernetes Operator Pythonic Framework). Talk held in Prague on 2019-09-05: https://www.meetup.com/Cloud-Native-Prague/events/263802447/

Why we don’t use the Term DevOps: the Journey to a Product Mindset - Destinat...

Henning Jacobs

While the adoption of DevOps makes teams move faster with reduced dependency on central operations, it can constrain teams who lack the skills to self-manage the full application and infrastructure stack. The way to overcome this challenge is creating an internal platform and treating it as a world-class product offering. “Applying product management to internal platforms means establishing empathy with internal consumers (read: developers) and collaborating with them on the design. Platform product managers establish roadmaps and ensure the platform delivers value to the business and enhances the developer experience”, via ThoughtWorks Technology Radar. In this talk, Henning Jacobs will walk you through how Zalando adopted a customer-first mindset with regards to its developer tooling. He will show the effect on developer satisfaction when internal platforms are given the same respect as external product offerings. Henning will furthermore tell his story about how Zalando moved from a classical infrastructure team to a product mindset with strong focus on building a world-class developer experience. Henning shares both their learnings and challenges going through this transition, and the impact it has on the daily life of Zalando’s customers (developers). This talk was given in Aarhus on 4th of June 2019.

Developer Experience at Zalando - CNCF End User SIG-DX

Henning Jacobs

While the adoption of DevOps makes teams move faster with reduced dependency on central operations, it can constrain teams who lack the skills to self-manage the full application and infrastructure stack. The way to overcome this challenge is creating an internal platform and treating it as a world-class product offering. “Applying product management to internal platforms means establishing empathy with internal consumers (read: developers) and collaborating with them on the design. Platform product managers establish roadmaps and ensure the platform delivers value to the business and enhances the developer experience”, via ThoughtWorks Technology Radar. In this talk, Henning Jacobs will walk you through how Zalando adopted a customer-first mindset with regards to its developer tooling. He will show the effect on developer satisfaction when internal platforms are given the same respect as external product offerings. Henning will furthermore tell his story about how Zalando moved from a classical infrastructure team to a product mindset with strong focus on building a world-class developer experience. Henning shares both their learnings and challenges going through this transition, and the impact it has on the daily life of Zalando’s customers (developers). This talk was given in Aarhus on 4th of June 2019.

Developer Experience at Zalando - CNCF End User SIG-DX

Henning Jacobs

Let's talk about Failures with Kubernetes - Hamburg Meetup

Henning Jacobs

Developer Experience at Zalando - Handelsblatt Strategisches IT-Management 2019

Henning Jacobs

Talk given at 25. Handelsblatt Jahrestagung Strategisches IT-Management in Munich on 2019-01-23. Original title (German): "Developer Experience bei Zalando: Entwicklerproduktivität steigern mit Cloud Native Infrastruktur" - Wie macht man mehr als 1100 Entwickler glücklich und effektiv? - Entwickler als Kunde: Produktmanagement für Plattformteams - You build it – you run it: Self-Service-Infrastruktur mit Kubernetes und AWS - Der Weg vom klassischen Infrastrukturteam zu Developer Productivity als Abteilung

Running Kubernetes in Production: A Million Ways to Crash Your Cluster - DevO...

Henning Jacobs

Bootstrapping a Kubernetes cluster is easy, rolling it out to nearly 200 engineering teams and operating it at scale is a challenge. In this talk, we are presenting our approach to Kubernetes provisioning on AWS, operations and developer experience for our growing Zalando developer base. We will walk you through our horror stories of operating 80+ clusters and share the insights we gained from incidents, failures, user reports and general observations. Most of our learnings apply to other Kubernetes infrastructures (EKS, GKE, ..) as well. This talk strives to reduce the audience’s unknown unknowns about running Kubernetes in production.

Kubernetes on AWS at Zalando: Failures & Learnings - DevOps NRW

Henning Jacobs

Distributed tensorflow on kubernetes

inwin stack

Effective Building your Platform with Kubernetes == Keep it Simple

Wojciech Barczyński

Canary Releases on Kubernetes with Spinnaker, Istio, & Prometheus (2020)

Kublr

In a microservices world, applications consist of dozens, hundreds, or even thousands of components. Manually deploying and verifying deployment quality in production is virtually impossible. Kubernetes, which natively supports rolling updates, enables blue-green application deployments with Spinnaker. However, the gradual rollout is a feature that doesn’t come out-of-the-box but can be achieved by adding Istio and Prometheus to the equation. During this meetup, Slava will discuss canary release implementations on Kubernetes with Spinnaker, Istio, and Prometheus. He’ll examine the role of each tool in the process and how they are all connected. During a demo, he will demonstrate a successful and failed canary release, and how these tools enable IT teams, to properly roll out changes to their customer base without any downtime.

Cantainer CI/ CD with Kubernetes

inwin stack

Kubernetes in kubernetes 搭建高可用環境

inwin stack

4K–Kubernetes with Knative, Kafka and Kamel

Red Hat Developers

Zero-downtime deployment of Micro-services with Kubernetes

Wojciech Barczyński

猿でもわかる Helm

Tsuyoshi Miyake

ウェビナーではKubernetesのおさらいとその欠点を確認した後に、それを克服するためにどのようにHelmが誕生したかについてご紹介しました。また本題であるHelmの基礎的内容、３つのコアコンセプトの後、実際にHelmを使ってKubernetesベースのアプリケーションをHelmベースのアプリケーションに移行するための簡単なデモをご覧いただきました。

Distributed tensorflow on kubernetes

inwin stack

AWS Summit Singapore 2019 | Autoscaling Your Kubernetes Workloads

AWS Summits

Kubernetes stack reliability

Oleg Chunikhin

Self-healing does not equal self-healing. There are multiple layers to it, whether a self-healing infrastructure, cluster, pods, or Kubernetes. Kubernetes itself ensures self-healing pods. But how do you ensure your applications, whose reliability depends on every single layer, are truly reliable? In this presentation we discuss aspects of reliability and self-healing in the different layers of a comprehensive container management stack; what Kubernetes does and doesn't do (at least not by default), and what you should look out for to ensure true reliable applications.

Kubernetes x PaaS – コンテナアプリケーションのNoOpsへの挑戦

Yoichi Kawasaki

How to Run Kubernetes in Restrictive Environments

Kublr

Meeting the Needs of Enterprise Governance and Security Installing Kubernetes is easy. Ensuring it complies with your organization’s enterprise governance and security requirements isn’t. During this webinar, Oleg will explain how to use Kubernetes while meeting enterprise requirements. In this technically-focused talk, he’ll summarize common prerequisites for running Kubernetes in production, and how to leverage fine-grained controls and separation of responsibilities to meet enterprise governance and security needs. The presentation will include basic requirements for audit, security, authentication, authorization, integration with existing identity management, logging, and monitoring. Because on-premise Kubernetes deployments don’t come without their challenges, Oleg will cover the limitations of a bare-metal installation, interactions with vSphere’s API, achieving HA, reliability and disaster recovery, as well as handling OS upgrades, security patches, and Kubernetes upgrades. He’ll close with a quick outlook of what’s next, including infrastructure as code, immutable infrastructure, and GitOps.

GitOps A/B testing with Istio and Helm

Weaveworks

Using source code management patterns to configure and secure your Kubernetes...

Giovanni Galloro

In this session we will show how to set up, from scratch, a git repository to centrally manage, with Anthos Config Management, all the configurations and security policies of multiple Kubernetes clusters in different environments, using git as the source of truth and applying the processes typically used in source code lifecycle. We will also explore what is possible to do with ACM Policy Controller, based on Open Policy Agent Gatekeeper, and configure constraints to enforce many of the possible security policies that an enterprise organization would require.

GitOps - Operation By Pull Request

Kasper Nissen

Overpowered Kubernetes: CI/CD for K8s on Enterprise IaaS

J On The Beach

Overpowered Kubernetes: CI/CD for K8s on Enterprise IaaS by Juan Carlos Ruiz Rico Kubernetes has taken the container world by storm, becoming the popular choice for developers and operations teams alike to manage their container deployments at scale. In this session learn how Oracle's managed Kubernetes service combines the power of Kubernetes with the raw performance of Oracle Cloud Infrastructure. See how you can avoid the complexity of standing up and maintaining your own Kubernetes infrastructure while giving your containers direct access to native bare metal performance and empowering development teams to achieve continuous integration and continuous delivery goals with Wercker.

Deploying Containerised Open-Source CSP Platforms

Angel Borroy López

What's hot

Let's talk about Failures with Kubernetes - Hamburg Meetup

Henning Jacobs

Developer Experience at Zalando - Handelsblatt Strategisches IT-Management 2019

Henning Jacobs

Running Kubernetes in Production: A Million Ways to Crash Your Cluster - DevO...

Henning Jacobs

Kubernetes on AWS at Zalando: Failures & Learnings - DevOps NRW

Henning Jacobs

Distributed tensorflow on kubernetes

inwin stack

Effective Building your Platform with Kubernetes == Keep it Simple

Wojciech Barczyński

Canary Releases on Kubernetes with Spinnaker, Istio, & Prometheus (2020)

Kublr

Cantainer CI/ CD with Kubernetes

inwin stack

Kubernetes in kubernetes 搭建高可用環境

inwin stack

4K–Kubernetes with Knative, Kafka and Kamel

Red Hat Developers

Zero-downtime deployment of Micro-services with Kubernetes

Wojciech Barczyński

猿でもわかる Helm

Tsuyoshi Miyake

Distributed tensorflow on kubernetes

inwin stack

AWS Summit Singapore 2019 | Autoscaling Your Kubernetes Workloads

AWS Summits

Kubernetes stack reliability

Oleg Chunikhin

Kubernetes x PaaS – コンテナアプリケーションのNoOpsへの挑戦

Yoichi Kawasaki

How to Run Kubernetes in Restrictive Environments

Kublr

GitOps A/B testing with Istio and Helm

Weaveworks

Using source code management patterns to configure and secure your Kubernetes...

Giovanni Galloro

GitOps - Operation By Pull Request

Kasper Nissen

What's hot (20)

Let's talk about Failures with Kubernetes - Hamburg Meetup

Developer Experience at Zalando - Handelsblatt Strategisches IT-Management 2019

Running Kubernetes in Production: A Million Ways to Crash Your Cluster - DevO...

Kubernetes on AWS at Zalando: Failures & Learnings - DevOps NRW

Distributed tensorflow on kubernetes

Effective Building your Platform with Kubernetes == Keep it Simple

Canary Releases on Kubernetes with Spinnaker, Istio, & Prometheus (2020)

Cantainer CI/ CD with Kubernetes

Kubernetes in kubernetes 搭建高可用環境

4K–Kubernetes with Knative, Kafka and Kamel

Zero-downtime deployment of Micro-services with Kubernetes

猿でもわかる Helm

Distributed tensorflow on kubernetes

AWS Summit Singapore 2019 | Autoscaling Your Kubernetes Workloads

Kubernetes stack reliability

Kubernetes x PaaS – コンテナアプリケーションのNoOpsへの挑戦

How to Run Kubernetes in Restrictive Environments

GitOps A/B testing with Istio and Helm

Using source code management patterns to configure and secure your Kubernetes...

GitOps - Operation By Pull Request

Similar to Open Source at Zalando - OSB Open Source Day 2019

Overpowered Kubernetes: CI/CD for K8s on Enterprise IaaS

J On The Beach

Deploying Containerised Open-Source CSP Platforms

Angel Borroy López

Openstack Benelux Conference 2014 Red Hat Keynote

Microsoft

Cloudfoundry + K8S : Prendre le meilleur des deux mondes

Erwan Bornier

Web Technologies in Automotive & Robotics (BlinkOn 10)

Igalia

Red hat cloud platforms

Giovanni Galloro

Cross Section and Deep Dive into GE Predix

Altoros

Yannis Zarkadas. Enterprise data science workflows on kubeflow

MarynaHoldaieva

Yannis Zarkadas. Stefano Fioravanzo. Enterprise data science workflows on kub...

Lviv Startup Club

MongoDB.local Austin 2018: MongoDB Ops Manager + Kubernetes

MongoDB

Presented by: Jason Mimick Technical Director, MongoDB MongoDB Ops Manager is an enterprise-grade end-to-end database management, monitoring, and backup solution. Kubernetes has clearly won the orchestration-platform "wars". In this session we'll take a deep dive on how you can leverage both these technologies to host your MongoDB deployments within your Kubernetes infrastructure whether that's OpenShift, PKS, Azure AKS, or just upstream. This talk will review the core technologies, such as containers, Kubernetes, and MongoDB Ops Manager. You'll also have a chance to see real-live demos of MongoDB running on Kubernetes and managed with MongoDB Ops Manager with the MongoDB Enterprise Kubernetes Operator.

DevOps LA Meetup Intro to Habitat

Jessica DeVita

The Big Cloud Native FaaS Lebowski

QAware GmbH

Devoxx Poland 2019, Kraków: Talk by Mario-Leander Reimer (@LeanderReimer, Principal Software Architect at QAware) === Please download slides if blurred! === Abstract: Only a few years ago the move towards microservice architecture was the first big disruption in software engineering: instead of running monoliths, systems were now build, composed and run as autonomous services. But this came at the price of added development and infrastructure complexity. Serverless and FaaS seem to be the next disruption, they are the logical evolution trying to address some of the inherent technology complexity we are currently faced when building cloud native apps. FaaS frameworks are currently popping up like mushrooms: Knative, Kubeless, OpenFn, Fission, OpenFaas or Open Whisk are just a few to name. But which one of these is safe to pick and use in your next project? Let's find out. This session will start off by briefly explaining the essence of Serverless application architecture. Leander will then define a criteria catalog for FaaS frameworks and continue by comparing and showcasing the most promising ones.

The Big Cloud native FaaS Lebowski

QAware GmbH

microXchg 2019, Berlin: Talk by Mario-Leander Reimer (@LeanderReimer, Principal Software Architect at QAware) === Please download slides if blurred! === Abstract: Only a few years ago the move towards microservice architecture was the first big disruption in software engineering: instead of running monoliths, systems were now build, composed and run as autonomous services. But this came at the price of added development and infrastructure complexity. Serverless and FaaS seem to be the next disruption, they are the logical evolution trying to address some of the inherent technology complexity we are currently faced when building cloud native apps. FaaS frameworks are currently popping up like mushrooms: Knative, Kubeless, OpenFn, Fission, OpenFaas or Open Whisk are just a few to name. But which one of these is safe to pick and use in your next project? Let's find out. This session will start off by briefly explaining the essence of Serverless application architecture. We will then define a criteria catalog for FaaS frameworks and continue by comparing and showcasing the most promising ones.

Better Operations into the Cloud

Fabio Ferrari

At the helm of kubernetes

Daniel Ramos

ONOS SDN-IP: Tutorial and Use Case for SDX

APNIC

[JOI] TOTVS Developers Joinville - Java #1

Rubens Dos Santos Filho

Completing the Microservices Puzzle: Kubernetes, Prometheus and FreshTracks.io

CA Technologies

The Big Cloud native FaaS Lebowski

QAware GmbH

Linux-Stammtisch Juli 2019, Munich: Talk by Mario-Leander Reimer (@LeanderReimer, Principal Software Architect at QAware) === Please download slides if blurred! === Abstract: Only a few years ago the move towards microservice architecture was the first big disruption in software engineering: instead of running monoliths, systems were now build, composed and run as autonomous services. But this came at the price of added development and infrastructure complexity. Serverless and FaaS seem to be the next disruption, they are the logical evolution trying to address some of the inherent technology complexity we are currently faced when building cloud native apps. FaaS frameworks are currently popping up like mushrooms: Knative, Kubeless, OpenFn, Fission, OpenFaas or Open Whisk are just a few to name. But which one of these is safe to pick and use in your next project? Let's find out. This session will start off by briefly explaining the essence of Serverless application architecture. We will then define a criteria catalog for FaaS frameworks and continue by comparing and showcasing the most promising ones.

IAU workshop 2018 day one

Walid Shaari

Similar to Open Source at Zalando - OSB Open Source Day 2019 (20)

Overpowered Kubernetes: CI/CD for K8s on Enterprise IaaS

Deploying Containerised Open-Source CSP Platforms

Openstack Benelux Conference 2014 Red Hat Keynote

Cloudfoundry + K8S : Prendre le meilleur des deux mondes

Web Technologies in Automotive & Robotics (BlinkOn 10)

Red hat cloud platforms

Cross Section and Deep Dive into GE Predix

Yannis Zarkadas. Enterprise data science workflows on kubeflow

Yannis Zarkadas. Stefano Fioravanzo. Enterprise data science workflows on kub...

MongoDB.local Austin 2018: MongoDB Ops Manager + Kubernetes

DevOps LA Meetup Intro to Habitat

The Big Cloud Native FaaS Lebowski

The Big Cloud native FaaS Lebowski

Better Operations into the Cloud

At the helm of kubernetes

ONOS SDN-IP: Tutorial and Use Case for SDX

[JOI] TOTVS Developers Joinville - Java #1

Completing the Microservices Puzzle: Kubernetes, Prometheus and FreshTracks.io

The Big Cloud native FaaS Lebowski

IAU workshop 2018 day one

More from Henning Jacobs

Kubernetes Failure Stories, or: How to Crash Your Cluster - ContainerDays EU ...

Henning Jacobs

Bootstrapping a Kubernetes cluster is easy, rolling it out to nearly 200 engineering teams and operating it at scale is a challenge. In this talk, we are presenting our approach to Kubernetes provisioning on AWS, operations and developer experience for our growing Zalando developer base. We will walk you through our horror stories of operating 100+ clusters and share the insights we gained from incidents, failures, user reports and general observations. Our failure stories will be sourced from recent and past incidents, so the talk will be up-to-date with our latest experiences.

Kubernetes Failure Stories - KubeCon Europe Barcelona

Henning Jacobs

Talk given on 2019-05-21 at KubeCon Barcelona: https://kccnceu19.sched.com/event/MPcM/kubernetes-failure-stories-and-how-to-crash-your-clusters-henning-jacobs-zalando-se Bootstrapping a Kubernetes cluster is easy, rolling it out to nearly 200 engineering teams and operating it at scale is a challenge. In this talk, we are presenting our approach to Kubernetes provisioning on AWS, operations and developer experience for our growing Zalando developer base. We will walk you through our horror stories of operating 100+ clusters and share the insights we gained from incidents, failures, user reports and general observations. Our failure stories will be sourced from recent and past incidents, so the talk will be up-to-date with our latest experiences. Most of our learnings apply to other Kubernetes infrastructures (EKS, GKE, ..) as well. This talk strives to reduce the audience's unknown unknowns about running Kubernetes in production.

Optimizing Kubernetes Resource Requests/Limits for Cost-Efficiency and Latenc...

Henning Jacobs

Talk given at JAX DevOps London on 2019-05-15. Kubernetes has the concept of resource requests and limits. Pods get scheduled on the nodes based on their requests and optionally limited in how much of the resource they can consume. Understanding and optimizing resource requests/limits is crucial both for reducing resource "slack" and ensuring application performance/low-latency. This talk shows our approach to monitoring and optimizing Kubernetes resources for 90+ clusters to achieve cost-efficiency and reducing impact for latency-critical applications. All shown tools are open source and can be applied to most Kubernetes deployments. Topics covered in the talk include: understanding resource requests and limits, cgroups and CFS quota behavior, contributing factors to cluster costs (in public clouds), and best practices for managing Kubernetes resources.

Ensuring Kubernetes Cost Efficiency across (many) Clusters - DevOps Gathering...

Henning Jacobs

Talk held at DevOps Gathering 2019 in Bochum on 2019-03-13. Abstract: This talk will address one of the most common challenges of organizations adopting Kubernetes on a medium to large scale: how to keep cloud costs under control without babysitting each and every deployment and cluster configuration? How to operate 80+ Kubernetes clusters in a cost-efficient way for 200+ autonomous development teams? This talk provides insights on how Zalando approaches this problem with central cost optimizations (e.g. Spot), cost monitoring/alerting, active measures to reduce resource slack, and automated cluster housekeeping. We will focus on how to ingrain cost efficiency in tooling and developer workflows while balancing rigid cost control with developer convenience and without impacting availability or performance. We will show our use case running Kubernetes on AWS, but all shown tools are open source and can be applied to most other infrastructure environments.

Optimizing Kubernetes Resource Requests/Limits for Cost-Efficiency and Latenc...

Henning Jacobs

Kubernetes has the concept of resource requests and limits. Pods get scheduled on the nodes based on their requests and optionally limited in how much of the resource they can consume. Understanding and optimizing resource requests/limits is crucial both for reducing resource "slack" and ensuring application performance/low-latency. This talk shows our approach to monitoring and optimizing Kubernetes resources for 80+ clusters to achieve cost-efficiency and reducing impact for latency-critical applications. All shown tools are Open Source and can be applied to most Kubernetes deployments.

Running Kubernetes in Production: A Million Ways to Crash Your Cluster - Cont...

Henning Jacobs

API First with Connexion - PyConWeb 2018

Henning Jacobs

Connexion is an open source API first REST framework for Python, built on top of Flask and based on OpenAPI/Swagger, targeted for microservice development. Connexion automagically handles request routing, oauth2 security, request validation and response serialization based on an OpenAPI 2.0 Specification file in YAML, so you don’t have to care about boilerplate anymore. Because it is based on Flask it supports everything that Flask does, including deployment options and extensions. At Zalando we’ve adopted “API First” as one of our key engineering principles, to ensure our API are robust, consistent, general and abstracted from specific implementation and use cases. But when we tried to implement this principle for the first time we were faced with the lack of a python framework to achieve it in a easy fashion - there were several frameworks that produce a swagger definition from the implementation but none that do it the other way around - so we decided to fill that gap. Henning will show how to get started with OpenAPI+Connexion, present some real-world use cases and deployment options such as Kubernetes.

Developer Journey at Zalando - Idea to Production with Containers in the Clou...

Henning Jacobs

Talk held on R-ETAIL:CODE in London on 2018-03-15. - The history of how DevOps evolved at Zalando: from on-premise data centers to autonomous teams, microservices and cluster management in the cloud - How the developer experience looks like for the application lifecycle from idea to production and what our vision for the future is - Challenges and learnings from our past experiences: why architecture principles and constraints are important to lead 200+ engineering teams

Large Scale Kubernetes on AWS at Europe's Leading Online Fashion Platform - C...

Henning Jacobs

Bootstrapping a Kubernetes cluster is easy, rolling it out to nearly 200 engineering teams and operating it at scale is a challenge. In this talk, we are presenting our approach to Kubernetes provisioning on AWS, operations and developer experience for our growing Zalando Technology department. We will highlight in the context of Kubernetes: AWS service integrations, our IAM/OAuth infrastructure, cluster autoscaling, continuous delivery and general developer experience. The talk will cover our most important learnings and we will openly share failure stories. Talk given at Container Days HH (https://containerdays.io/) on 2017-06-20.

From AWS/STUPS to Kubernetes on AWS @Zalando - Berlin Kubernetes Meetup

Henning Jacobs

Kubernetes on AWS @Zalando - Berlin AWS User Group 2017-05-09

Henning Jacobs

In this talk we share our learnings from running Kubernetes on AWS in production and how we are migrating 200+ engineering teams from AWS/STUPS to Kubernetes. This talk was given at the Berlin AWS User Group meetup on 2017-05-09 hosted by NewStore (https://www.meetup.com/aws-berlin/events/236795816/). More information on http://kubernetes-on-aws.readthedocs.io/en/latest/admin-guide/kubernetes-in-production.html

Kubernetes at Zalando - CNCF End User Committee Presentation

Henning Jacobs

Kubernetes on AWS at Europe's Leading Online Fashion Platform

Henning Jacobs

Plan B: Service to Service Authentication with OAuth

Henning Jacobs

In this talk we show how our 85+ Zalando engineering teams are using OAuth 2.0 to secure their growing fleet of microservices running with Docker on AWS. We describe how and why we build our open source “Plan B” OAuth components from scratch to provide a resilient, highly available infrastructure with no single point of failure. We built Plan B as a distributed OAuth provider and validation endpoint by using proven technologies such as Cassandra and JSON Web Tokens (JWT). We show how token revocations are possible without any central token storage and how we avoid network latency on token validation.

Docker Berlin Meetup Nov 2015: Zalando Intro

Henning Jacobs

STUPS @ AWS Enterprise Web Day Oktober 2015

Henning Jacobs

Cross-cutting Teams und Product Teams lösen immer mehr das klassische Pool-Modell ab. Das Motto “You build it, you run it” ist die Basis für DevOps. Zalando geht den Schritt weiter und arbeitet in autonomen Teams als "First Class Entity”. Autonome Teams mögen keine Abhängigkeiten, sonst wären sie nicht wirklich autonom. Microservices in Verbindung mit einer Cloud Infrastruktur unterstützen Teams in ihrer Autonomie. Wie das in der Praxis mit AWS funktioniert und was dazu organisatorisch und architektonisch notwendig ist, präsentiert Henning Jacobs von Zalando.

Python at Zalando Technology @ Python Users Berlin Meetup September 2015

Henning Jacobs

STUPS by Zalando @WHD.local Frankfurt: STUPS.io - an Open Source Cloud Framew...

Henning Jacobs

This talk shed lights on how Zalando, Europe’s leading eCommerce platform for fashion, has developed an architecture and set of open source cloud tools that scales as its technology team (100 autonomous engineering teams and counting) grows and evolves. It also describes the challenges regarding compliance and innovation that we face, and how we built our cloud infrastructure on top of AWS and Docker to manage these challenges. Talk given by Henning Jacobs at the WHD.local event in Frankfurt on September 10th 2015.

More from Henning Jacobs (18)

Kubernetes Failure Stories, or: How to Crash Your Cluster - ContainerDays EU ...

Kubernetes Failure Stories - KubeCon Europe Barcelona

Optimizing Kubernetes Resource Requests/Limits for Cost-Efficiency and Latenc...

Ensuring Kubernetes Cost Efficiency across (many) Clusters - DevOps Gathering...

Optimizing Kubernetes Resource Requests/Limits for Cost-Efficiency and Latenc...

Running Kubernetes in Production: A Million Ways to Crash Your Cluster - Cont...

API First with Connexion - PyConWeb 2018

Developer Journey at Zalando - Idea to Production with Containers in the Clou...

Large Scale Kubernetes on AWS at Europe's Leading Online Fashion Platform - C...

From AWS/STUPS to Kubernetes on AWS @Zalando - Berlin Kubernetes Meetup

Kubernetes on AWS @Zalando - Berlin AWS User Group 2017-05-09

Kubernetes at Zalando - CNCF End User Committee Presentation

Kubernetes on AWS at Europe's Leading Online Fashion Platform

Plan B: Service to Service Authentication with OAuth

Docker Berlin Meetup Nov 2015: Zalando Intro

STUPS @ AWS Enterprise Web Day Oktober 2015

Python at Zalando Technology @ Python Users Berlin Meetup September 2015

STUPS by Zalando @WHD.local Frankfurt: STUPS.io - an Open Source Cloud Framew...

Recently uploaded

Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...

Ramesh Iyer

In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.

To Graph or Not to Graph Knowledge Graph Architectures and LLMs

Paul Groth

Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...

UiPathCommunity

💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™: See how to accelerate model training and optimize model performance with active learning Learn about the latest enhancements to out-of-the-box document processing – with little to no training required Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath. Speakers: 👨‍🏫 Andras Palfi, Senior Product Manager, UiPath 👩‍🏫 Lenka Dulovicova, Product Program Manager, UiPath

From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...

Product School

Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality

Inflectra

In this insightful webinar, Inflectra explores how artificial intelligence (AI) is transforming software development and testing. Discover how AI-powered tools are revolutionizing every stage of the software development lifecycle (SDLC), from design and prototyping to testing, deployment, and monitoring. Learn about: • The Future of Testing: How AI is shifting testing towards verification, analysis, and higher-level skills, while reducing repetitive tasks. • Test Automation: How AI-powered test case generation, optimization, and self-healing tests are making testing more efficient and effective. • Visual Testing: Explore the emerging capabilities of AI in visual testing and how it's set to revolutionize UI verification. • Inflectra's AI Solutions: See demonstrations of Inflectra's cutting-edge AI tools like the ChatGPT plugin and Azure Open AI platform, designed to streamline your testing process. Whether you're a developer, tester, or QA professional, this webinar will give you valuable insights into how AI is shaping the future of software delivery.

GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...

James Anderson

Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management. The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM). Speakers: Bob Boule Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle. Gopinath Rebala Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.

"Impact of front-end architecture on development cost", Viktor Turskyi

Fwdays

I have heard many times that architecture is not important for the front-end. Also, many times I have seen how developers implement features on the front-end just following the standard rules for a framework and think that this is enough to successfully launch the project, and then the project fails. How to prevent this and what approach to choose? I have launched dozens of complex projects and during the talk we will analyze which approaches have worked for me and which have not.

Assuring Contact Center Experiences for Your Customers With ThousandEyes

ThousandEyes

GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...

Sri Ambati

IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx

Abida Shariff

FIDO Alliance Osaka Seminar: Overview.pdf

FIDO Alliance

When stars align: studies in data quality, knowledge graphs, and machine lear...

Elena Simperl

Knowledge engineering: from people to machines and back

Elena Simperl

FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf

FIDO Alliance

Connector Corner: Automate dynamic content and events by pushing a button

DianaGray10

Here is something new! In our next Connector Corner webinar, we will demonstrate how you can use a single workflow to: Create a campaign using Mailchimp with merge tags/fields Send an interactive Slack channel message (using buttons) Have the message received by managers and peers along with a test email for review But there’s more: In a second workflow supporting the same use case, you’ll see: Your campaign sent to target colleagues for approval If the “Approve” button is clicked, a Jira/Zendesk ticket is created for the marketing design team But—if the “Reject” button is pushed, colleagues will be alerted via Slack message Join us to learn more about this new, human-in-the-loop capability, brought to you by Integration Service connectors. And... Speakers: Akshay Agnihotri, Product Manager Charlie Greenberg, Host

FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf

FIDO Alliance

Search and Society: Reimagining Information Access for Radical Futures

Bhaskar Mitra

The field of Information retrieval (IR) is currently undergoing a transformative shift, at least partly due to the emerging applications of generative AI to information access. In this talk, we will deliberate on the sociotechnical implications of generative AI for information access. We will argue that there is both a critical necessity and an exciting opportunity for the IR community to re-center our research agendas on societal needs while dismantling the artificial separation between the work on fairness, accountability, transparency, and ethics in IR and the rest of IR research. Instead of adopting a reactionary strategy of trying to mitigate potential social harms from emerging technologies, the community should aim to proactively set the research agenda for the kinds of systems we should build inspired by diverse explicitly stated sociotechnical imaginaries. The sociotechnical imaginaries that underpin the design and development of information access technologies needs to be explicitly articulated, and we need to develop theories of change in context of these diverse perspectives. Our guiding future imaginaries must be informed by other academic fields, such as democratic theory and critical theory, and should be co-developed with social science scholars, legal scholars, civil rights and social justice activists, and artists, among others.

GraphRAG is All You need? LLM & Knowledge Graph

Guy Korland

Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs. 1. Unifying Large Language Models and Knowledge Graphs: A Roadmap. https://arxiv.org/abs/2306.08302 2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs: https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/

UiPath Test Automation using UiPath Test Suite series, part 3

DianaGray10

Mission to Decommission: Importance of Decommissioning Products to Increase E...

Product School

Recently uploaded (20)