Mesos is a distributed systems kernel that provides efficient resource isolation and sharing across distributed applications or frameworks. It was designed to handle large-scale distributed computing across clusters of servers. Mesos abstracts CPU, memory, storage, and other compute resources away from machines (physical or virtual), enabling fault-tolerant and elastic distributed systems to easily be built and run effectively.
Driving Business and Technical Agility in the Enterprise!
Container World 2017 is the only independent conference offering an exploration of the entire container ecosystem. Over 3 days, you’ll hear from the innovative enterprises, tech giants and startups who are transforming enterprise IT and driving business innovation on such topics as:
Containers and legacy infrastructure
Operations/DevOps
Orchestration & Workloads
Security
Storage/Persistent storage
Standardization and Certification
Emerging technology like serverless, unikernel and beyond
View the brochure for more information: https://goo.gl/OpnoEr
Integrating Docker EE into Société Générale's Existing Enterprise IT SystemsDocker, Inc.
Société Générale knows that containers and the cloud are the future of the IT industry and have been using Docker EE for over a year and a half. In this talk, we will share how Docker EE fits into our global strategy and our architecture for integrating the platform to our existing IT systems. We will go over tradeoffs of how we operationalized the platform to provide a highly available CAAS to our global enterprise. Finally, we will share how we are onboarding development teams and deploying their applications to production.
My @TriangleDevops talk from 2013-10-17. I covered the work that led us to @NetflixOSS (Acme Air), the work we did on the cloud prize (NetflixOSS on IBM SoftLayer/RightScale) and the @NetflixOSS platform (Karyon, Archaius, Eureka, Ribbon, Asgard, Hystrix, Turbine, Zuul, Servo, Edda, Ice, Denominator, Aminator, Janitor/Conformity/Chaos Monkeys of the Simian Army).
Kubernetes is a great tool to run (Docker) containers in a clustered production environment. When deploying often to production we need fully automated blue-green deployments, which makes it possible to deploy without any downtime. We also need to handle external HTTP requests and SSL offloading. This requires integration with a load balancer like Ha-Proxy. Another concern is (semi) auto scaling of the Kubernetes cluster itself when running in a cloud environment. E.g. partially scale down the cluster at night.
In this technical deep dive you will learn how to setup Kubernetes together with other open source components to achieve a production ready environment that takes code from git commit to production without downtime.
Driving Business and Technical Agility in the Enterprise!
Container World 2017 is the only independent conference offering an exploration of the entire container ecosystem. Over 3 days, you’ll hear from the innovative enterprises, tech giants and startups who are transforming enterprise IT and driving business innovation on such topics as:
Containers and legacy infrastructure
Operations/DevOps
Orchestration & Workloads
Security
Storage/Persistent storage
Standardization and Certification
Emerging technology like serverless, unikernel and beyond
View the brochure for more information: https://goo.gl/OpnoEr
Integrating Docker EE into Société Générale's Existing Enterprise IT SystemsDocker, Inc.
Société Générale knows that containers and the cloud are the future of the IT industry and have been using Docker EE for over a year and a half. In this talk, we will share how Docker EE fits into our global strategy and our architecture for integrating the platform to our existing IT systems. We will go over tradeoffs of how we operationalized the platform to provide a highly available CAAS to our global enterprise. Finally, we will share how we are onboarding development teams and deploying their applications to production.
My @TriangleDevops talk from 2013-10-17. I covered the work that led us to @NetflixOSS (Acme Air), the work we did on the cloud prize (NetflixOSS on IBM SoftLayer/RightScale) and the @NetflixOSS platform (Karyon, Archaius, Eureka, Ribbon, Asgard, Hystrix, Turbine, Zuul, Servo, Edda, Ice, Denominator, Aminator, Janitor/Conformity/Chaos Monkeys of the Simian Army).
Kubernetes is a great tool to run (Docker) containers in a clustered production environment. When deploying often to production we need fully automated blue-green deployments, which makes it possible to deploy without any downtime. We also need to handle external HTTP requests and SSL offloading. This requires integration with a load balancer like Ha-Proxy. Another concern is (semi) auto scaling of the Kubernetes cluster itself when running in a cloud environment. E.g. partially scale down the cluster at night.
In this technical deep dive you will learn how to setup Kubernetes together with other open source components to achieve a production ready environment that takes code from git commit to production without downtime.
Practical Design Patterns in Docker NetworkingDocker, Inc.
Migrating an application to Docker creates an opportunity to utilize new networking topologies and features, which can provide new functionality to an existing application. This talk will provide an overview of Docker networking with a focus on the architectural choices when migrating applications. Taking sample applications we will look at the existing networking topology and cover the options available to create a simple migration and provide additional functionality.
Enabling Production Grade Containerized Applications through Policy Based Inf...Docker, Inc.
This session covers the solution addressing the needs of enabling product-grade containerized applications. You will learn how operations teams running containerized applications in a shared infrastructure can define and enforce policies to provide security, monitoring, and performance for network, storage, and computing. You will learn about Contiv and Mantl, open source projects that create a framework for cloud native application development and infrastructure with application intent and operational policies. Contiv integrates Cisco infrastructure (UCS, Nexus, and ACI) with Docker Datacenter to help enterprises adopt containers at a larger scale.
Docker Kubernetes Istio
Understanding Docker and creating containers.
Container Orchestration based on Kubernetes
Blue Green Deployment, AB Testing, Canary Deployment, Traffic Rules based on Istio
Slides from the talk given to the Startup Berlin Slack Group that demonstrates how TruckIN is implementing its continuous delivery workflow using technologies and open-source tools.
Topics that are covered: Automated Cloud Provisioning (Network, Subnets, VMs, Kubernetes Cluster, Firewall, Disks, Credentials, Private Docker Registry); Configuration Management (Salt Stack), Continuous Integration (Jenkins CI), Continuous Delivery/Deployment (Salt API/Reactor + Kubernetes) to a Google Cloud Kubernetes Cluster, Remote Application Debugging, Managing Google Cloud Kubernetes Cluster, Logging, Monitoring and ChatOps (Slack and operable.io)
Sf bay area Kubernetes meetup dec8 2016 - deployment modelsPeter Ss
I talk about deploying complex, multi-layer applications in Kuberentes.
I describe how Kubernetes AppController project (https://github.com/Mirantis/k8s-AppController) can be leveraged to enhance such deployments
K8s in 3h - Kubernetes Fundamentals TrainingPiotr Perzyna
Kubernetes (K8s) is an open-source system for automating deployment, scaling, and management of containerized applications. This training helps you understand key concepts within 3 hours.
Being a cloud native developer requires learning some new language and new skills like circuit-breakers, canaries, service mesh, linux containers, dark launches, tracers, pods and sidecars. In this session, we will introduce you to cloud native architecture by demonstrating numerous principles and techniques for building and deploying Java microservices via Spring Boot, Wildfly Swarm and Vert.x, while leveraging Istio on Kubernetes with OpenShift.
Kubernetes is designed to be an extensible system. But what is the vision for Kubernetes Extensibility? Do you know the difference between webhooks and cloud providers, or between CRI, CSI, and CNI? In this talk we will explore what extension points exist, how they have evolved, and how to use them to make the system do new and interesting things. We’ll give our vision for how they will probably evolve in the future, and talk about the sorts of things we expect the broader Kubernetes ecosystem to build with them.
Kubernetes (K8s) is a powerful, flexible and portable open source framework for distributed containerized applications delivery and management. An important part of the services provided by most Kubernetes clusters is the containers’ networking stack. In most cases and for many applications it “just works”, but this seeming simplicity is backed by a complex stack of technologies that provide many capabilities beyond the basics.
This presentation accompanies the meetup and webinar where Oleg Chunikhin, CTO at Kublr, shows how Kubernetes networking stack works, describes main components, interfaces and extensibility options.
What is covered:
- general notions of Kubernetes networking - Pods and Network Policies
- implementation of Kubernetes networking - CNI, CNI plugins, and Linux network namespaces
- some Kubernetes CNI providers: Calico, Weave, Flanel, and Canal
- K8S networking extensibility for advanced and “exotic” use-cases with Multus CNI plugin as an example
What Is Kubernetes | Kubernetes Introduction | Kubernetes Tutorial For Beginn...Edureka!
***** Kubernetes Certification Training: https://www.edureka.co/kubernetes-certification *****
This Edureka tutorial on "What is Kubernetes" will give you an introduction to one of the most popular Devops tool in the market - Kubernetes, and its importance in today's IT processes. This tutorial is ideal for beginners who want to get started with Kubernetes & DevOps. The following topics are covered in this training session:
1. Need for Kubernetes
2. What is Kubernetes and What it's not
3. How does Kubernetes work?
4. Use-Case: Kubernetes @ Pokemon Go
5. Hands-on: Deployment with Kubernetes
DevOps Tutorial Blog Series: https://goo.gl/P0zAfF
An Introduction to Container Organization with Docker Swarm, Kubernetes, Meso...Neo4j
Interest in Docker has increased significantly since its inception. According to a report compiled by a leading cloud-scale monitoring company, Datadog, two-thirds of the companies that try Docker adopt it, and the adopters have increased their container count by five times over a period of nine months. Neo4j has also embraced Docker by supporting official images and also offering specific images of its own.
While the interest in container technology is growing rapidly, so is the need to deploy containers over a cluster of machines to allow scalability and fault-tolerance. This highlights the need for orchestration which refers to the idea of automating the manual process of deploying, configuring and scaling the containers in an automated manner.
In this talk, we provide a hands-on introduction to the three most popular Docker orchestration tools: Kubernetes, Docker Swarm and Mesos. This talk offers a conceptual understanding of each of these technologies along with an insight into the concepts learned through a series of three demos. The demos will illustrate how to deploy and automatically scale a Neo4j container using each of the three orchestration platforms.
We realize that the scope of the topic in terms of the orchestration tools is too broad. The rationale behind choosing the three specific tools is based on the following two reasons: First is their potential use in our cluster at Cincinnati Children’s Hospital (CCHMC). Secondly, they also fall under the leading orchestration tools.
Scaling OpenStack Networking Beyond 4000 Nodes with Dragonflow - Eshed Gal-Or...Cloud Native Day Tel Aviv
As OpenStack matures, more users move from “dipping a toe” to deploying at large scale, with 1000's of nodes.
OpenStack networking has long been a limiting factor in scaling beyond a few hundreds of nodes, forcing users to turn to cell splitting, or to complete offloading of the networking to the underlay systems and forfeit the overlay network altogether.
Dragonflow is a fully distributed, open source, SDN implementation of Neutron, that handles large scale deployments without splitting to cells.
In testing we've conducted, we were able to scale to 4000+ controllers (each controller is typically deployed on a compute node), while maintaining the same performance we had on a small 30 node environment.
A basic introductory slide set on Kubernetes: What does Kubernetes do, what does Kubernetes not do, which terms are used (Containers, Pods, Services, Replica Sets, Deployments, etc...) and how basic interaction with a Kubernetes cluster is done.
Deploying your first application with KubernetesOVHcloud
Find out how to deploy your first application with Kubernetes on the OVH cloud, and direct questions to the team responsible for our upcoming Kubernetes as-a-Service solution.
WSO2Con US 2015 Kubernetes: a platform for automating deployment, scaling, an...Brian Grant
Kubernetes can run application containers on clusters of physical or virtual machines.
It can also do much more than that.
Kubernetes satisfies a number of common needs of applications running in production, such as co-locating helper processes, mounting storage systems, distributing secrets, application health checking, replicating application instances, horizontal auto-scaling, load balancing, rolling updates, and resource monitoring.
However, even though Kubernetes provides a lot of functionality, there are always new scenarios that would benefit from new features. Ad hoc orchestration that is acceptable initially often requires robust automation at scale. Application-specific workflows can be streamlined to accelerate developer velocity.
This is why Kubernetes was also designed to serve as a platform for building an ecosystem of components and tools to make it easier to deploy, scale, and manage applications. The Kubernetes control plane is built upon the same APIs that are available to developers and users, implementing resilient control loops that continuously drive the current state towards the desired state. This design has enabled Apache Stratos and a number of other Platform as a Service and Continuous Integration and Deployment systems to build atop Kubernetes.
This presentation introduces Kubernetes’s core primitives, shows how some of its better known features are built on them, and introduces some of the new capabilities that are being added.
** Kubernetes Certification Training: https://www.edureka.co/kubernetes-certification **
This Edureka tutorial on "Kubernetes Architecture" will give you an introduction to popular DevOps tool - Kubernetes, and will deep dive into Kubernetes Architecture and its working. The following topics are covered in this training session:
1. What is Kubernetes
2. Features of Kubernetes
3. Kubernetes Architecture and Its Components
4. Components of Master Node and Worker Node
5. ETCD
6. Network Setup Requirements
DevOps Tutorial Blog Series: https://goo.gl/P0zAfF
Practical Design Patterns in Docker NetworkingDocker, Inc.
Migrating an application to Docker creates an opportunity to utilize new networking topologies and features, which can provide new functionality to an existing application. This talk will provide an overview of Docker networking with a focus on the architectural choices when migrating applications. Taking sample applications we will look at the existing networking topology and cover the options available to create a simple migration and provide additional functionality.
Enabling Production Grade Containerized Applications through Policy Based Inf...Docker, Inc.
This session covers the solution addressing the needs of enabling product-grade containerized applications. You will learn how operations teams running containerized applications in a shared infrastructure can define and enforce policies to provide security, monitoring, and performance for network, storage, and computing. You will learn about Contiv and Mantl, open source projects that create a framework for cloud native application development and infrastructure with application intent and operational policies. Contiv integrates Cisco infrastructure (UCS, Nexus, and ACI) with Docker Datacenter to help enterprises adopt containers at a larger scale.
Docker Kubernetes Istio
Understanding Docker and creating containers.
Container Orchestration based on Kubernetes
Blue Green Deployment, AB Testing, Canary Deployment, Traffic Rules based on Istio
Slides from the talk given to the Startup Berlin Slack Group that demonstrates how TruckIN is implementing its continuous delivery workflow using technologies and open-source tools.
Topics that are covered: Automated Cloud Provisioning (Network, Subnets, VMs, Kubernetes Cluster, Firewall, Disks, Credentials, Private Docker Registry); Configuration Management (Salt Stack), Continuous Integration (Jenkins CI), Continuous Delivery/Deployment (Salt API/Reactor + Kubernetes) to a Google Cloud Kubernetes Cluster, Remote Application Debugging, Managing Google Cloud Kubernetes Cluster, Logging, Monitoring and ChatOps (Slack and operable.io)
Sf bay area Kubernetes meetup dec8 2016 - deployment modelsPeter Ss
I talk about deploying complex, multi-layer applications in Kuberentes.
I describe how Kubernetes AppController project (https://github.com/Mirantis/k8s-AppController) can be leveraged to enhance such deployments
K8s in 3h - Kubernetes Fundamentals TrainingPiotr Perzyna
Kubernetes (K8s) is an open-source system for automating deployment, scaling, and management of containerized applications. This training helps you understand key concepts within 3 hours.
Being a cloud native developer requires learning some new language and new skills like circuit-breakers, canaries, service mesh, linux containers, dark launches, tracers, pods and sidecars. In this session, we will introduce you to cloud native architecture by demonstrating numerous principles and techniques for building and deploying Java microservices via Spring Boot, Wildfly Swarm and Vert.x, while leveraging Istio on Kubernetes with OpenShift.
Kubernetes is designed to be an extensible system. But what is the vision for Kubernetes Extensibility? Do you know the difference between webhooks and cloud providers, or between CRI, CSI, and CNI? In this talk we will explore what extension points exist, how they have evolved, and how to use them to make the system do new and interesting things. We’ll give our vision for how they will probably evolve in the future, and talk about the sorts of things we expect the broader Kubernetes ecosystem to build with them.
Kubernetes (K8s) is a powerful, flexible and portable open source framework for distributed containerized applications delivery and management. An important part of the services provided by most Kubernetes clusters is the containers’ networking stack. In most cases and for many applications it “just works”, but this seeming simplicity is backed by a complex stack of technologies that provide many capabilities beyond the basics.
This presentation accompanies the meetup and webinar where Oleg Chunikhin, CTO at Kublr, shows how Kubernetes networking stack works, describes main components, interfaces and extensibility options.
What is covered:
- general notions of Kubernetes networking - Pods and Network Policies
- implementation of Kubernetes networking - CNI, CNI plugins, and Linux network namespaces
- some Kubernetes CNI providers: Calico, Weave, Flanel, and Canal
- K8S networking extensibility for advanced and “exotic” use-cases with Multus CNI plugin as an example
What Is Kubernetes | Kubernetes Introduction | Kubernetes Tutorial For Beginn...Edureka!
***** Kubernetes Certification Training: https://www.edureka.co/kubernetes-certification *****
This Edureka tutorial on "What is Kubernetes" will give you an introduction to one of the most popular Devops tool in the market - Kubernetes, and its importance in today's IT processes. This tutorial is ideal for beginners who want to get started with Kubernetes & DevOps. The following topics are covered in this training session:
1. Need for Kubernetes
2. What is Kubernetes and What it's not
3. How does Kubernetes work?
4. Use-Case: Kubernetes @ Pokemon Go
5. Hands-on: Deployment with Kubernetes
DevOps Tutorial Blog Series: https://goo.gl/P0zAfF
An Introduction to Container Organization with Docker Swarm, Kubernetes, Meso...Neo4j
Interest in Docker has increased significantly since its inception. According to a report compiled by a leading cloud-scale monitoring company, Datadog, two-thirds of the companies that try Docker adopt it, and the adopters have increased their container count by five times over a period of nine months. Neo4j has also embraced Docker by supporting official images and also offering specific images of its own.
While the interest in container technology is growing rapidly, so is the need to deploy containers over a cluster of machines to allow scalability and fault-tolerance. This highlights the need for orchestration which refers to the idea of automating the manual process of deploying, configuring and scaling the containers in an automated manner.
In this talk, we provide a hands-on introduction to the three most popular Docker orchestration tools: Kubernetes, Docker Swarm and Mesos. This talk offers a conceptual understanding of each of these technologies along with an insight into the concepts learned through a series of three demos. The demos will illustrate how to deploy and automatically scale a Neo4j container using each of the three orchestration platforms.
We realize that the scope of the topic in terms of the orchestration tools is too broad. The rationale behind choosing the three specific tools is based on the following two reasons: First is their potential use in our cluster at Cincinnati Children’s Hospital (CCHMC). Secondly, they also fall under the leading orchestration tools.
Scaling OpenStack Networking Beyond 4000 Nodes with Dragonflow - Eshed Gal-Or...Cloud Native Day Tel Aviv
As OpenStack matures, more users move from “dipping a toe” to deploying at large scale, with 1000's of nodes.
OpenStack networking has long been a limiting factor in scaling beyond a few hundreds of nodes, forcing users to turn to cell splitting, or to complete offloading of the networking to the underlay systems and forfeit the overlay network altogether.
Dragonflow is a fully distributed, open source, SDN implementation of Neutron, that handles large scale deployments without splitting to cells.
In testing we've conducted, we were able to scale to 4000+ controllers (each controller is typically deployed on a compute node), while maintaining the same performance we had on a small 30 node environment.
A basic introductory slide set on Kubernetes: What does Kubernetes do, what does Kubernetes not do, which terms are used (Containers, Pods, Services, Replica Sets, Deployments, etc...) and how basic interaction with a Kubernetes cluster is done.
Deploying your first application with KubernetesOVHcloud
Find out how to deploy your first application with Kubernetes on the OVH cloud, and direct questions to the team responsible for our upcoming Kubernetes as-a-Service solution.
WSO2Con US 2015 Kubernetes: a platform for automating deployment, scaling, an...Brian Grant
Kubernetes can run application containers on clusters of physical or virtual machines.
It can also do much more than that.
Kubernetes satisfies a number of common needs of applications running in production, such as co-locating helper processes, mounting storage systems, distributing secrets, application health checking, replicating application instances, horizontal auto-scaling, load balancing, rolling updates, and resource monitoring.
However, even though Kubernetes provides a lot of functionality, there are always new scenarios that would benefit from new features. Ad hoc orchestration that is acceptable initially often requires robust automation at scale. Application-specific workflows can be streamlined to accelerate developer velocity.
This is why Kubernetes was also designed to serve as a platform for building an ecosystem of components and tools to make it easier to deploy, scale, and manage applications. The Kubernetes control plane is built upon the same APIs that are available to developers and users, implementing resilient control loops that continuously drive the current state towards the desired state. This design has enabled Apache Stratos and a number of other Platform as a Service and Continuous Integration and Deployment systems to build atop Kubernetes.
This presentation introduces Kubernetes’s core primitives, shows how some of its better known features are built on them, and introduces some of the new capabilities that are being added.
** Kubernetes Certification Training: https://www.edureka.co/kubernetes-certification **
This Edureka tutorial on "Kubernetes Architecture" will give you an introduction to popular DevOps tool - Kubernetes, and will deep dive into Kubernetes Architecture and its working. The following topics are covered in this training session:
1. What is Kubernetes
2. Features of Kubernetes
3. Kubernetes Architecture and Its Components
4. Components of Master Node and Worker Node
5. ETCD
6. Network Setup Requirements
DevOps Tutorial Blog Series: https://goo.gl/P0zAfF
[Container world 2017] The Questions You're Afraid to Ask about ContainersDustin Kirkland
Use the Right Container Technology for the Job
Application containers, machine containers, process containers, system containers -- what's the difference? 12-factor apps, Microservices, cloud-native application design -- are these real? Docker, Rocket, OCID, LXD -- do I need all of them? Should I run PaaS on top of my IaaS, or my IaaS on top of my PaaS? Do containers fit into PaaS or IaaS? Or both? Neither? Where are the intersections of Kubernetes, Swarm, Mesos, and OpenStack? How do I ensure compatibility across my public and private clouds? And how does bare metal -- from my commodity, scale-out x86 to my powerful, scale-up mainframes fit into all of this? Can any of this stuff actually be used in a highly secure environment? In this session, Dustin Kirkland, Ubuntu Product and Strategy Lead at Canonical, will explain the container ecosystem in clear, concise terms, from real enterprise user experience -- the successes and the failures.
Checking in your deployment configuration as code
Helm is a tool that streamlines the creation, deployment and management of your Kubernetes-native applications. In this talk, we take a look at how Helm enables you to manage your deployment configurations as code, and demonstrate how it can be used to power your continuous delivery (CI/CD) pipeline.
This talk will focus on a brief history, including a demo and overview of how we at Superbalist use Kubernetes, and how Kubernetes uses Docker, does load balancing, deployments, and data migrations.
Talk from Cape Town DevOps meetup on Jun 21, 2016:
https://www.meetup.com/Cape-Town-DevOps/events/231530172/
Code: https://github.com/zoidbergwill/kubernetes-examples
Slides as markdown: http://www.zoidbergwill.com/presentations/2016/kubernetes-1.2-and-spread/index.md
Idea to Production - with Gitlab and KubernetesSimon Dittlmann
Setting up a continuous delivery pipeline form scratch with gitlab.com and Kubernetes (Google Container Service GKE) on Google Cloud Platform.
The entire source code is available at https://github.com/Pindar/gcloud-k8s-express-app
Blog post https://www.itnotes.de/gitlab/kubernetes/k8s/gke/gcloud/2017/03/05/idea-to-production-with-gitlab-and-kubernetes/
More tips and tricks for running containers like a pro - Rancher Online MEetu...Shannon Williams
Whether you’re a longtime container user, or entirely new to Docker, it’s never too late to pick up more tips and tricks for running containers in production. In the February 2017 online Rancher meetup, we covered four topics useful for anyone managing containers and infrastructure at scale.
- Autoscaling Docker containers and infrastructure with webhooks in Rancher
- Secrets Management with Rancher and Vault
- Getting the most out of containers and the Traefik load balancer
- Launching applications with Kubernetes Dashboard and Helm
You can find a recording of the meetup at: http://rancher.com/event/february-2017-online-meetup/
Smuggling Multi-Cloud Support into Cloud-native Applications using Elastic Co...Nane Kratzke
Elastic container platforms (like Kubernetes, Docker Swarm, Apache Mesos) fit very well with existing cloud-native application architecture approaches. So it is more than astonishing, that these already existing and open source available elastic platforms are not considered more consequently for multi-cloud approaches. Elastic container platforms provide inherent multi-cloud support that can be easily accessed. We present a solution proposal of a control process which is able to scale (and migrate as a side effect) elastic container platforms across different public and private cloud-service providers. This control loop can be used in an execution phase of self-adaptive auto-scaling MAPE loops (monitoring, analysis, planning, execution). Additionally, we present several lessons learned from our prototype implementation which might be of general interest for researchers and practitioners. For instance, to describe only the intended state of an elastic platform and let a single control process take care to reach this intended state is far less complex than to define plenty of specific and necessary multi-cloud aware workflows to deploy, migrate, terminate, scale up and scale down elastic platforms or applications.
We're really happy to say that today we made the first meetup about Kubernetes in Russia! Thanks to all speakers and guests! Join us: https://twitter.com/kubernetesMSK
Characterizing and Contrasting Kuhn-tey-ner Awr-kuh-streyt-orsSonatype
Lee Calcote, Solar Winds
Running a few containers? No problem. Running hundreds or thousands? Enter the container orchestrator. Let’s take a look at the characteristics of the four most popular container orchestrators and what makes them alike, yet unique.
Swarm
Nomad
Kubernetes
Mesos+Marathon
We’ll take a structured looked at these container orchestrators, contrasting them across these categories:
Genesis & Purpose
Support & Momentum
Host & Service Discovery
Scheduling
Modularity & Extensibility
Updates & Maintenance
Health Monitoring
Networking & Load-Balancing
High Availability & Scale
OpenStack and Kubernetes - A match made for Telco HeavenTrinath Somanchi
With the advent of Containerization of Telco Clouds for NFV and SDN based deployments, OpenStack with Kubernetes is a best chosen option to solve the challenges is a better way to build a containerized Telco cloud. This involves, "Kubernetes in OpenStack", "OpenStack in Kubernetes" and "Independent OpenStack and Kubernetes". With this complementing collaboration, in the Stadium of OpenStack's Open Infrastructure, Telecom gaints are developing cloud-native solutions to best fit the next generation networking deployments. In this Presentation, we talk about Containerization and benefits, OpenStack and Kubernetes match making and we give a brief overview on Airship and Kata Container projects.
Introduction to containers, k8s, Microservices & Cloud NativeTerry Wang
Slides built to upskill and enable internal team and/or partners on foundational infra skills to work in a containerized world.
Topics covered
- Container / Containerization
- Docker
- k8s / container orchestration
- Microservices
- Service Mesh / Serverless
- Cloud Native (apps & infra)
- Relationship between Kubernetes and Runtime Fabric
Audiences: MuleSoft internal technical team, partners, Runtime Fabric users.
Presentation for the July 2018 @medianetlab meetup at NCSR "Demokritos"
Relative blog post can be found here: https://medianetlab.gr/mnlab-meetup-kubernetes/
and the video: https://www.youtube.com/watch?v=l2ce5U9bh6M
OpenStack Collaboration made in heaven with Heat, Mistral, Neutron and more..Trinath Somanchi
Cross-project collaboration is something OpenStack community has embraced for a long time. Common libraries like Oslo reduces the time and effort to build a new service. Another way this manifests is in new OpenStack services getting built using existing services to solve an higher level use-case.
In this talk we are present how the band of projects comprising of Mistral, Tacker, Neutron, Heat, TOSCA-parser and Barbican came together to build an industry leading ETSI NFV Orchestrator that leveraged the best of these projects. Each of these projects brought in critical functionalities needed towards the final product. You will learn how, when strung together, this solution follows the classic Microservices design pattern that the industry is rapidly adopting.
Introduction to Containers - AWS Startup Day Johannesburg.pdfAmazon Web Services
In this session, we cover all the options for running containers on AWS. This will include an intro of container concepts, and an overview to different services like ECS, EKS, ECR and Fargate. We cover topics like: how to choose the right orchestration platform for your workload, some different tools that are out there to make the process easier, and how to find more information and support as you work.
This presentation covers how app deployment model evolved from bare metal servers to Kubernetes World.
In addition to theoretical information, you will find free KATACODA workshops url to perform practices to understand the details of the each topics.
Container orchestration from theory to practiceDocker, Inc.
"Join Laura Frank and Stephen Day as they explain and examine technical concepts behind container orchestration systems, like distributed consensus, object models, and node topology. These concepts build the foundation of every modern orchestration system, and each technical explanation will be illustrated using SwarmKit and Kubernetes as a real-world example. Gain a deeper understanding of how orchestration systems work in practice and walk away with more insights into your production applications."
Orchestrating Linux Containers while tolerating failuresDocker, Inc.
lthough containers are bringing a refreshing flexibility when deploying services in production, the management of those containers in such an environment still requires special care in order to keep the application up and running. In this regard, orchestration platforms like Docker, Kubernetes and Nomad have been trying to alleviate this responsibility, facilitating the task of deploying and maintaining the entire application stack in its desired state. This ensures that a service will be always running, tolerating machine failures, network erratic behavior or software updates and downtime. The purpose of this talk is to explain the mechanisms and architecture of the Docker Engine orchestration platform (using a framework called swarmkit) to tolerate failures of services and machines, from cluster state replication and leader-election to container re-scheduling logic when a host goes down.
Putting Kafka In Jail – Best Practices To Run Kafka On Kubernetes & DC/OSLightbend
Apache Kafka–part of Lightbend Fast Data Platform–is a distributed streaming platform that is best suited to run close to the metal on dedicated machines in statically defined clusters. For most enterprises, however, these fixed clusters are quickly becoming extinct in favor of mixed-use clusters that take advantage of all infrastructure resources available.
In this webinar by Sean Glover, Fast Data Engineer at Lightbend, we will review leading Kafka implementations on DC/OS and Kubernetes to see how they reliably run Kafka in container orchestrated clusters and reduce the overhead for a number of common operational tasks with standard cluster resource manager features. You will learn specifically about concerns like:
* The need for greater operational knowhow to do common tasks with Kafka in static clusters, such as applying broker configuration updates, upgrading to a new version, and adding or decommissioning brokers.
* The best way to provide resources to stateful technologies while in a mixed-use cluster, noting the importance of disk space as one of Kafka’s most important resource requirements.
* How to address the particular needs of stateful services in a model that natively favors stateless, transient services.
Challenge: Recent success of Docker containers reveals arrival of a new era: the number of CPUs is exploding 10-100 folds up, and cloud networking is already in a new movement of scalability upgrade
Question: To scale UP or OUT? I.e., UPgrade or OUTgrade?
Answer from DaoliCloud’s practice: Better scale OUT, ,,.and Openflow can help
Quantum - Virtual networks for Openstacksalv_orlando
An overview of Quantum, the soon-to-be default Openstack network service.
These slides introduce Quantum, its design goals, and discusses the API. It also tries to address how quantum relates to Software Defined Networking (SDN)
Neutron Done the SDN Way
Dragonflow is an open source distributed control plane implementation of Neutron which is an integral part of OpenStack. Dragonflow introduces innovative solutions and features to implement networking and distributed network services in a manner that is both lightweight and simple to extend, yet targeted towards performance-intensive and latency-sensitive applications. Dragonflow aims at solving the performance
Similar to Container World 2017 - Characterizing and Contrasting Container Orchestrators (20)
Benchmarking Service Meshes - CNCF Networking WGLee Calcote
Presented at the CNCF Networking Working Group in March 2019. A project to provide apples-to-apples comparison of performance overhead induced by different service meshes. Recording - https://www.youtube.com/watch?v=2_JwCc-kLMA
Istio: Using nginMesh as the service proxyLee Calcote
With microservices and containers becoming mainstream, container orchestrators provide much of what the cluster (nodes and containers) needs. With container orchestrators' core focus on scheduling, discovery, and health at an infrastructure level, microservices are left with unmet, service-level needs, such as:
- Traffic management, routing, and resilient and secure communication between services
- Policy enforcement, rate-limiting, circuit breaking
- Visibility and monitoring with metrics, logs, and traces
- Load balancing and rollout/canary deployment support
Service meshes provide for these needs. In this session, we will dive into Istio - its components, capabilities, and extensibility. Istio envelops and integrates with other open source projects to deliver a full-service mesh. We'll explore these integrations and Istio's extensibility in terms of choice of proxies and adapters, such as nginMesh.
CNCF, State of Serverless & Project NuclioLee Calcote
The Serverless working group within the Cloud Native Computing Foundation (CNCF) is one of many. In this talk, we’ll answer why the working group exists and how our efforts help the ecosystem. We'll also take a look at some of the current Serverless and FaaS projects and cover some of the common Serverless myths. Finally, we'll look ahead toward what we foresee as some of Serverless's biggest challenges and best-suited use cases.
Load Balancing in the Cloud using Nginx & KubernetesLee Calcote
Presented on March 16, 2017 through O'Reilly - http://www.oreilly.com/pub/e/3864
Modern day applications bring modern day infrastructure requirements. Whether you bring your own or you use your cloud provider's managed load-balancing services, even moderately sophisticated applications are likely to find their needs underserved.
Overlay/Underlay - Betting on Container NetworkingLee Calcote
Presented at Rackspace Austin (downtown) on July 27th, 2016.
An inherent to component to any distributed application, networking is one of the most complicated and expansive infrastructure technologies. Container networking needs to be developer-friendly. Application-driven and portable. With developers busily adopting container technologies, the time has come for network engineers and operators to prepare for the unique challenges brought on by cloud native applications. What container networking specifications bring to the table and how to leverage them.
A brisk introduction to container runtimes (engines) and an understanding of when container orchestrators enter and what role they play. We’ll look at what makes them alike, yet unique. Presented at ContainerizeThis 2016.
Characterizing and Contrasting Container OrchestratorsLee Calcote
Presented at OpenStack Summit Austin 2016 - Container Day.
Running a few containers? No problem. Running hundreds or thousands? Enter the container orchestrator. Let’s take a look at the characteristics of the three most popular container orchestrators and what makes them alike, yet unique.
Neuro-symbolic is not enough, we need neuro-*semantic*Frank van Harmelen
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on the notifications, alerts, and approval requests using Slack for Bonterra Impact Management. The solutions covered in this webinar can also be deployed for Microsoft Teams.
Interested in deploying notification automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualityInflectra
In this insightful webinar, Inflectra explores how artificial intelligence (AI) is transforming software development and testing. Discover how AI-powered tools are revolutionizing every stage of the software development lifecycle (SDLC), from design and prototyping to testing, deployment, and monitoring.
Learn about:
• The Future of Testing: How AI is shifting testing towards verification, analysis, and higher-level skills, while reducing repetitive tasks.
• Test Automation: How AI-powered test case generation, optimization, and self-healing tests are making testing more efficient and effective.
• Visual Testing: Explore the emerging capabilities of AI in visual testing and how it's set to revolutionize UI verification.
• Inflectra's AI Solutions: See demonstrations of Inflectra's cutting-edge AI tools like the ChatGPT plugin and Azure Open AI platform, designed to streamline your testing process.
Whether you're a developer, tester, or QA professional, this webinar will give you valuable insights into how AI is shaping the future of software delivery.
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf91mobiles
91mobiles recently conducted a Smart TV Buyer Insights Survey in which we asked over 3,000 respondents about the TV they own, aspects they look at on a new TV, and their TV buying preferences.
Essentials of Automations: Optimizing FME Workflows with ParametersSafe Software
Are you looking to streamline your workflows and boost your projects’ efficiency? Do you find yourself searching for ways to add flexibility and control over your FME workflows? If so, you’re in the right place.
Join us for an insightful dive into the world of FME parameters, a critical element in optimizing workflow efficiency. This webinar marks the beginning of our three-part “Essentials of Automation” series. This first webinar is designed to equip you with the knowledge and skills to utilize parameters effectively: enhancing the flexibility, maintainability, and user control of your FME projects.
Here’s what you’ll gain:
- Essentials of FME Parameters: Understand the pivotal role of parameters, including Reader/Writer, Transformer, User, and FME Flow categories. Discover how they are the key to unlocking automation and optimization within your workflows.
- Practical Applications in FME Form: Delve into key user parameter types including choice, connections, and file URLs. Allow users to control how a workflow runs, making your workflows more reusable. Learn to import values and deliver the best user experience for your workflows while enhancing accuracy.
- Optimization Strategies in FME Flow: Explore the creation and strategic deployment of parameters in FME Flow, including the use of deployment and geometry parameters, to maximize workflow efficiency.
- Pro Tips for Success: Gain insights on parameterizing connections and leveraging new features like Conditional Visibility for clarity and simplicity.
We’ll wrap up with a glimpse into future webinars, followed by a Q&A session to address your specific questions surrounding this topic.
Don’t miss this opportunity to elevate your FME expertise and drive your projects to new heights of efficiency.
State of ICS and IoT Cyber Threat Landscape Report 2024 previewPrayukth K V
The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development.
The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers:
State of global ICS asset and network exposure
Sectoral targets and attacks as well as the cost of ransom
Global APT activity, AI usage, actor and tactic profiles, and implications
Rise in volumes of AI-powered cyberattacks
Major cyber events in 2024
Malware and malicious payload trends
Cyberattack types and targets
Vulnerability exploit attempts on CVEs
Attacks on counties – USA
Expansion of bot farms – how, where, and why
In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East
Why are attacks on smart factories rising?
Cyber risk predictions
Axis of attacks – Europe
Systemic attacks in the Middle East
Download the full report from here:
https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/
UiPath Test Automation using UiPath Test Suite series, part 3DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 3. In this session, we will cover desktop automation along with UI automation.
Topics covered:
UI automation Introduction,
UI automation Sample
Desktop automation flow
Pradeep Chinnala, Senior Consultant Automation Developer @WonderBotz and UiPath MVP
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
6. One size does not fit all.
A strict apples-to-apples comparison is inappropriate and not
the objective, hence characterizing and contrasting.
@lcalcote
7. Let's not go here today.
Container orchestrators may be intermixed.
@lcalcote
8. Categorically Speaking
Scheduling
Genesis & Purpose
Support & Momentum
Host & Service Discovery
Modularity & Extensibility
Updates & Maintenance
Health Monitoring
Networking & Load-Balancing
Secrets Management
High Availability & Scale
@lcalcote
9. Core
Capabilities
Cluster Management
Host Discovery
Host Health Monitoring
Scheduling
Orchestrator Updates and Host
Maintenance
Service Discovery
Networking and Load-Balancing
Stateful services
Multi-tenant, multi-region
Additional
Key Capabilities
Application Health & Performance
Monitoring
Application Deployments
Application Secrets
@lcalcote
11. Genesis & Purpose
designed for both long-lived services and short-lived
batch processing workloads.
cluster manager with declarative job specifications.
ensures constraints are satisfied and resource
utilization is optimized by efficient task packing.
supports all major operating systems and virtualized,
containerized or standalone workloads.
written in Go and under the Unix philosophy.
@lcalcote
12. Support & Momentum
Project began June 2015 (19 months old) has 141
contributors
Current release v0.5.4
Nomad Enterprise offering aimed for first half of this
year.
Supported and governed by HashiCorp
HashiConf US '15 had ~300 attendees
HashiConf EU '16 had ~320 attendees
HashiConf US '16 had ~500 attendees
@lcalcote
14. Host &
Service Discovery
Host Discovery
Gossip protocol - Serf is used
Docker multi-host networking and Swarmkit use Serf, too
Servers advertise full set of Nomad servers to clients
heartbeats every 30 seconds
Creating federated clusters is simple
Service Discovery
Nomad integrates with to provide service
discovery and monitoring.
Consul
@lcalcote
15. Scheduling
two distinct phases, feasibility checking and ranking.
optimistically concurrent
enabling all servers to participate in scheduling decisions
which increases the total throughput and reduces latency
three scheduler types used when creating jobs:
service, batch and system
`nomad plan` point-in-time-view of what Nomad will do
@lcalcote
16. Modularity & Extensibility
Task drivers
Used by Nomad clients to execute a task and provide
resource isolation.
By having extensible task drivers are important for
flexibility to support a broad set of workloads (e.g. rkt, lxc).
Does not currently support pluggable task drivers,
Have to implement task driver interface and compile
Nomad binary.
@lcalcote
17. Updates & Maintenance
Nodes
Drain allocations on a running node.
integrates with tools like Packer, Consul, and Terraform
to support building artifacts, service discovery, monitoring and capacity
management.
Applications
Log rotation (stderr and stdout)
no log forward support, yet
Rolling updates (via the `update` block in the job specification).
@lcalcote
18. Health Monitoring
Nodes
Node health monitoring is done via heartbeats, so
Nomad can detect failed nodes and migrate the
allocations to other healthy clients.
Applications
currently http, tcp and script
In the future Nomad will add support for more Consul checks.
`nomad alloc-status` reports actual resource utilization
@lcalcote
20. Secrets Management
Nomad agents provide secure integration with Vault
for all tasks and containers it spins up
gives secure access to Vault secrets through a
workflow which minimizes risk of secret exposure
during bootstrapping.
@lcalcote
21. High Availability & Scale
distributed and highly available, using both leader
election and state replication to provide availability in
the face of failures.
shared state optimistic scheduler
only open source implementation.
1,000,0000 across 5,000 hosts and scheduled in 5 min.
Built for managing multiple clusters / cluster federation.
@lcalcote
22. Easy to use
Single binary for both clients and servers
Supports non-containerized tasks and
multiple container runtimes
Arguably the most advanced scheduler
design
Upfront consideration of federation /
hybrid cloud
Broad OS support
Outside of scheduler, comparatively less
sophisticated
Young project
Less relative momentum
Less relative adoption
Less extensible / pluggable
@lcalcote
25. Genesis & Purpose
Swarm is simple and easy to setup.
Initially responsible for clustering and scheduling
Driving toward application's needs with services,
secrets, etc.
Originally an imperative system, now declarative.
Swarm’s architecture is not complex as those of
Kubernetes and Mesos.
Written in Go, Swarm is lightweight, modular and
somewhat extensible.
@lcalcote
27. Support & Momentum
Contributions:
Standalone: ~3,000 commits, 12 core maintainers (140 contributors)
Swarmkit: ~2,800 commits, ~12 core maintainers (70 contributors)
~289 Docker meetups worldwide
Disclaimer: I organize Docker Austin.
Production-ready:
Standalone announced ~15 months ago (Nov 2015)
Swarmkit announced ~7 months ago (July 2016)
@lcalcote
28. Host & Service Discovery
Host Discovery
Like Nomad, uses Hashicorp's for storing cluster state
Pull model - where worker checks-in with the Manager
Rate Control - of checks-in with Manager may be controlled at
Manager - add jitter
Workers don't need to know which Manager is active; Follower
Managers will redirect Workers to Leader
Service Discovery
Embedded DNS and round robin load-balancing
Services are a new concept
goMemDB
@lcalcote
29. Scheduling
Swarm’s scheduler is pluggable
Swarm scheduling is a combination of strategies and
filters/constraint:
Strategies
Random
Spread*
Binpack
Filters
container constraints (affinity, dependency, port) are defined as
environment variables in the specification file
node constraints (health, constraint) must be specified when starting the
docker daemon and define which nodes a container may be scheduled on.
@lcalcote
30. Modularity & Extensibility
Ability to remove batteries is a strength for Swarm:
Pluggable scheduler
Pluggable network driver
Pluggable distributed K/V store
Docker container engine runtime-only
Pluggable authorization (in docker engine)*
@lcalcote
31. Updates & Maintenance
Nodes
Nodes may be Active, Drained and Paused
Manager weights are used to drain or pause Managers
Manual swarm manager and worker updates
Applications
Rolling updates now supported
--update-delay
--update-parallelism
--update-failure-action
@lcalcote
32. Health Monitoring
Nodes
Swarm monitors the availability and resource usage
of nodes within the cluster
Applications
One health check per container may be run
check container health by running a command inside the container
--interval=DURATION (default: 30s)
--timeout=DURATION (default: 30s)
--retries=N (default: 3)
@lcalcote
33. Networking & Load-
Balancing
Swarm and multi-host networking are simpatico
provides for user-defined overlay networks that are micro-segmentable
uses Hashicorp's Serf gossip protocol for quick convergence of neighbor table
facilitates container name resolution via embedded DNS server (previously via etc/hosts)
Load-balancing based on IPVS
expose Service's port externally
L4 load-balancer; cluster-wide port publishing
Mesh routing
send a request to any one of the nodes and it will be routed automatically
send a request to any one of the nodes and it will be internally load balanced
@lcalcote
34. Secrets Management
@lcalcote
Landed in 1.13
encrypted and kept in Raft store
managed by Swarm Managers
retrieved by Swarm Services (not containers)
via mounted in-memory filesystem on the node
35. High Availability & Scale
Managers may be deployed in a highly-available
configuration
Active/Standby - only one active Leader at-a-time
Maintain odd number of managers
Rescheduling upon node failure
No rebalancing upon node addition to the cluster
Does not support multiple failure isolation regions or
federation
although, with caveats, .
federation is possible
@lcalcote
36. Scaling swarm to 1,000 AWS nodes
and 50,000 containers
@lcalcote
37. Suitable for orchestrating a combination of infrastructure containers
Has only recently added capabilities falling into the application bu
Swarmkit is a young project
advanced features forthcoming
natural expectation of caveats in functionality
No rebalancing, autoscaling or monitoring, yet
Only schedules Docker containers, not containers using other specificat
Does not schedule VMs or non-containerized processes
Does not provide support for batch jobs
Need separate load-balancer for overlapping ingress ports
While dependency and affinity filters are available, Swarm does not pro
the ability to enforce scheduling of two containers onto the same host o
at all.
Filters facilitate sidecar pattern. No “pod” concept.
Swarm works. Swarm is simple and easy to
deploy.
1.12 eliminated need for much, but not all third-party software
Facilitates earlier stages of adoption by organizations viewing
containers as faster VMs
now with built-in functionality for applications
Swarm is easy to extend, if can already know
Docker APIs, you can customize Swarm
Still modular, but has stepped back here.
Moving very fast; eliminating gaps quickly.
39. Genesis & Purpose
an opinionated framework for building distributed
systems
"an open source system for automating deployment, scaling, and operations
of applications."
Written in Go, Kubernetes is lightweight, modular and
extensible
considered a third generation container orchestrator
led by Google, Red Hat and others.
Declaratively, opinionated with many key features
included
bakes in load-balancing, scale, volumes, deployments, secret
management and cross-cluster federated services among other features.
@lcalcote
41. Support & Momentum
Kubernetes is 2 yrs. 20 months old (June 2014)
Announced as production-ready 19 months ago (July 2015)
Project has over 1,000 commits per month (~44,000 total)
reach 1,000 committers (~100 core) Kubernauts in Dec. 2016
~5,000 commits made in each release (1.5 is latest)
~244 Kubernetes meetups worldwide.
Disclaimer: I organize Microservices and Containers Austin.
Under the governance of the Cloud Native Computing
Foundation
KubeCon earlier this year capped at 1,000 attendees
@lcalcote
42. Host & Service Discovery
Host Discovery
by default, the node agent (kubelet) is configured to register
itself with the master (API server)
automating the joining of new hosts to the cluster
Service Discovery
Two primary modes of finding a Service
DNS
SkyDNS is deployed as a cluster add-on
environment variables
environment variables are used as a simple way of providing compatibility
with Docker links-style networking
@lcalcote
43. Scheduling
By default, scheduling is handled by kube-scheduler (pluggable).
Selection criteria used by kube-scheduler to identify the best-fit
node is defined by policy:
Predicates (node resources and characteristics):
PodFitPorts , PodFitsResources, NoDiskConflict , MatchNodeSelector, HostName , ServiceAffinity,
LabelsPresence
Priorities (weighted strategies used to identify “best fit” node):
LeastRequestedPriority, BalancedResourceAllocation, ServiceSpreadingPriority, EqualPriority
@lcalcote
44. Modularity &
Extensibility
One of Kubernetes strengths its pluggable
architecture and it being an extensible platform
Choice of:
database for service discovery or network driver
container runtime - may choose to run docker with rkt containers
Cluster add-ons
optional system components that implement a cluster feature (e.g.
DNS, logging, etc.)
shipped with the Kubernetes binaries and are considered an inherent
part of the Kubernetes clusters
@lcalcote
45. Updates & Maintenance
Applications
`Deployment` objects automate deploying and
rolling updating applications.
Support for rolling back deployments
Kubernetes Components
Consistently backwards compatible
Upgrading the Kubernetes components and hosts is
done via shell script
Host maintenance - mark the node as unschedulable.
existing pods are vacated from the node
prevents new pods from being scheduled on the node
@lcalcote
46. Health Monitoring
Nodes
Failures - actively monitors the health of nodes within the cluster
via Node Controller
Resources - usage monitoring leverages a combination of open
source components:
cAdvisor, Heapster, InfluxDB, Grafana, Prometheus
Applications
three types of user-defined application health-checks and uses the
Kubelet agent as the the health check monitor
HTTP Health Checks, Container Exec, TCP Socket
Cluster-level Logging
collect logs which persist beyond the lifetime of the pod’s container
images or the lifetime of the pod or even cluster
standard output and standard error output of each container can be ingested using a
agent running on each nodeFluentd
47. Networking & Load-
Balancing
…enter the Pod
atomic unit of scheduling
flat networking with each pod receiving an IP address
no NAT required, port conflicts localized
intra-pod communication via localhost
Load-Balancing
Services provide inherent load-balancing via kube-proxy:
runs on each node of a Kubernetes cluster
reflects services as defined in the Kubernetes API
supports simple TCP/UDP forwarding and round-robin and Docker-links-
based service IP:PORT mapping.
@lcalcote
48. Secrets Management
encrypted and stored in etcd
used by containers in a pod either:
1. mounted as data volumes
2. exposed as environment variables
None of the pod’s containers will start until all the pods'
volumes are mounted.
Individual secrets are limited to 1MB in size.
Secrets are created and accessible within a given namespace,
not cross-namespace.
@lcalcote
49. High Availability & Scale
Each master component may be deployed in a highly-
available configuration.
Active/Standby configuration
Federated clusters / multi-region deployments
Scale
v1.2 support for 1,000 node clusters
v1.3 supports 2,000 node clusters
Horizontal Pod Autoscaling (via Replication Controllers).
Cluster Autoscaling (if you're running on GCE with AWS support is
coming soon).
@lcalcote
50. Only runs containerized applications
For those familiar with Docker-only, Kubernetes
requires understanding of new concepts
Powerful frameworks with more moving pieces beget complicated
cluster deployment and management.
Lightweight graphical user interface
Does not provide as sophisticated techniques for
resource utilization as Mesos
Kubernetes can schedule docker or rkt
containers
Inherently opinionated w/functionality built-in.
relatively easy to change its opinion
little to no third-party software needed
builds in many application-level concepts and services
(petsets, jobsets, daemonsets, application packages /
charts, etc.)
advanced storage/volume management
project has most momentum
project is arguably most extensible
thorough project documentation
Supports multi-tenancy
Multi-master, cross-cluster federation, robust
logging & metrics aggregation
@lcalcote
52. Genesis & Purpose
Mesos is a distributed systems kernel
stitches together many different machines into a logical computer
Mesos has been around the longest (launched in 2009)
and is arguably the most stable, with highest (proven) scale currently
Mesos is written mostly in C++
with Java, Python and C++ APIs
Marathon as a Framework
Marathon is one of a number of frameworks (Chronos and Aurora other
examples) that may be run on top of Mesos
Frameworks have a scheduler and executor. Schedulers get resource offers.
Executors run tasks.
Marathon is written in Scala
@lcalcote
54. Support & Momentum
MesosCon 2016 in Denver had ? attendees
MesosCon 2015 in Seattle had 700 attendees
up from 262 attendees in 2014
Mesos has 224 contributors
Marathon has 227 contributors
Mesos under the governance of Apache Foundation
Marathon under governance of Mesosphere
Mesos is used by Twitter, AirBnb, eBay, Apple, Cisco, Yodle
Marathon is used by Verizon and Samsung
@lcalcote
55. Host &
Service Discovery
Mesos-DNS generates an SRV record for each Mesos
task
including Marathon application instances
Marathon will ensure that all dynamically assigned
service ports are unique
Mesos-DNS is particularly useful when:
apps are launched through multiple frameworks (not just Marathon)
you are using an IP-per-container solution like
you use random host port assignments in Marathon
Project Calico
@lcalcote
56. Scheduling
Two-level scheduler
First-level scheduling happens at Mesos master based on
allocation policy, which decides which framework get
resources.
Second-level scheduling happens at Framework scheduler,
which decides what tasks to execute.
Provide reservations, over-subscriptions and preemption.
@lcalcote
57. Modularity & Extensibility
Frameworks
multiple available
may run multiple frameworks concurrently
Modules
extend inner-workings of Mesos by creating and using
shared libraries that are loaded on demand
many types of Modules
Replacement, Isolator, Allocator, Authentication, Hook, Anonymous
@lcalcote
58. Updates & Maintenance
Nodes
- Mesos has maintenance mode.
- Marathon does not.
Mesos API backwards compatible
from v1.0 forward
Applications
Marathon can be instructed to
deploy containers based on that
component using a blue/green
strategy
where old and new versions co-exist for a
time. @lcalcote
59. Health Monitoring
Nodes
Master tracks a set of statistics and metrics to
monitor resource usage
Applications
support for health checks (HTTP and TCP)
an event stream that can be integrated with load-
balancers or for analyzing metrics
@lcalcote
60. Networking & Load-
Balancing
Networking
An IP per Container
No longer share the node's IP
Helps remove port conflicts
Enables 3rd party network drivers
isolator with
MesosContainerizer
Load-Balancing
Marathon offers two TCP/HTTP proxies
A simple shell script and a more complex one called `marathon-lb` that
has more features.
Pluggable (e.g. Traefik for load-balancing)
Container Network Interface (CNI)
@lcalcote
61. Secrets Management
Not yet.
Only supported by Enterprise DC/OS
Stored in ZooKeeper, exposed as ENV variables in Marathon
Secrets shorter than eight characters may not be accepted by Marathon.
By default, you cannot store a secret larger than 1MB.
@lcalcote
62. High Availability & Scale
A strength of Mesos’s architecture
requires masters to form a quorum using ZooKeeper (point of failure)
only one Active (Leader) master at-a-time in Mesos and Marathon
Scale is a strong suit for Mesos. TBD for Marathon.
Autoscale
`marathon-autoscale.py` - autoscales application based on the
utilization metrics from Mesos
- request rate-based autoscaling with Marathon.
Great at short-lived jobs. High availability built-in.
Referred to as the “golden standard” by Solomon Hykes, Docker CTO.
marathon-lb-autoscale
63. Still needs 3rd party tools
Marathon interface could be more Docker friendly
(hard to get at volumes and registry)
May need a dedicated infrastructure IT team
an overly complex solution for small deployments
Universal Containerizer
abstract away from docker, rkt, kurma?, lxc?
Can run multiple frameworks, including Kubernetes and Swarm.
Supports multi-tenancy.
Good for Big Data shops and job / task-oriented workloads.
Good for mixed workloads and with data-locality policies
Mesos is powerful and scalable, battle-tested
Good for multiple large things you need to do 10,000+ node cluster system
Marathon UI is young, but promising.
@lcalcote