An introductory look at Kubernetes and how it leverages AWS IaaS features to provide its own virtual clustering, and demonstration of some of the behaviour inside the cluster that makes Kubernetes a popular choice for microservice deployments.
"[WORKSHOP] K8S for developers", Denis RomanukFwdays
"It seems that perfection is attained, not when there is nothing more to add, but when there is nothing more to take away." - Antoine de Saint Exupéry
We can talk a lot about Kubernetes. But does the whole info is really need for just begining? Let's take away everything, except the really needed, and show a way for ones, who still uses only docker.
Let's consider:
Why do developers need Kubernetes and what does it looks like?
An attempt to automate ops, or why does it was a reinvention of Kubernetes
What does kube consists of?
How to get a Kubernetes?
From docker-compose to c
What is a Helm and why it's difficult without one?
What is "local development on Kubernetes"?
CRD & Operators
AWS re:Invent 2016: Amazon ECR Deep Dive on Image Optimization (CON401)Amazon Web Services
“Are you struggling with bulky images or slow push and pull times? In this session we will walk through the anatomy of a Docker image and provide techniques you can use to optimize images for faster pushes and pulls and reduce your overall storage footprint. We will discuss Docker image building (build containers versus runtime containers to remove unnecessary software), Docker image composition (minimizing the number of layers), the Docker Remote API (optimizing how images are pushed and pulled), and CI/CD Integration (automate building, versioning, and deploying images to production). We’ll also examine the tools that ECR provides to make Docker image management easier so that you can focus on building your application. Finally, we'll hear from Pinterest about how they use ECR and Docker, some valuable experiences gained along the way, and best practices for using ECR with Apache Mesos.”
Jacksonville Users Group Presentation on how to integrate Angular2 and Spring Boot Microservices. It also shows you how to containerize an Angular2 app as a Node.js application and Spring Boot. The code can be found in http://github.com/erwindev.
MuleSoft Manchester Meetup #3 slides 31st March 2020Ieva Navickaite
Francis Edwards from Saint-Gobain Building Distribution presented on design practices for accelerating API delivery using Anypoint Platform. He discussed how to integrate API design with development using RAML and how elements like title, version, and baseUri are used across different Anypoint tools. Venkata Nallapuneni from Rathbone Brothers then presented on DataWeave 2.0 and how it has improved and simplified data transformation compared to Mule Expression Language.
Deploying and Scaling a Rails Application with Docker and FriendsInvisiblelines
Scaling a Docker application across multiple nodes requires leveraging additional tools. Here I outline one approach using Docker Compose, Swarm, Consul and Registrator.
Kubernetes is a container cluster manager that aims to provide a platform for automating deployment, scaling, and operations of application containers across clusters of machines. It uses pods as the basic building block, which are groups of application containers that share storage and networking resources. Kubernetes includes control planes for replication, scheduling, and services to expose applications. It supports deployment of multi-tier applications through replication controllers, services, labels, and pod templates.
Kubernetes is an open-source system for automating deployment, scaling, and management of containerized applications. It was originally developed by Google based on years of experience running production workloads at scale. Kubernetes groups containers into logical units called pods and handles tasks like scheduling, health checking, scaling and rollbacks. The main components include a master node that manages the cluster and worker nodes that run application containers scheduled by the master.
"[WORKSHOP] K8S for developers", Denis RomanukFwdays
"It seems that perfection is attained, not when there is nothing more to add, but when there is nothing more to take away." - Antoine de Saint Exupéry
We can talk a lot about Kubernetes. But does the whole info is really need for just begining? Let's take away everything, except the really needed, and show a way for ones, who still uses only docker.
Let's consider:
Why do developers need Kubernetes and what does it looks like?
An attempt to automate ops, or why does it was a reinvention of Kubernetes
What does kube consists of?
How to get a Kubernetes?
From docker-compose to c
What is a Helm and why it's difficult without one?
What is "local development on Kubernetes"?
CRD & Operators
AWS re:Invent 2016: Amazon ECR Deep Dive on Image Optimization (CON401)Amazon Web Services
“Are you struggling with bulky images or slow push and pull times? In this session we will walk through the anatomy of a Docker image and provide techniques you can use to optimize images for faster pushes and pulls and reduce your overall storage footprint. We will discuss Docker image building (build containers versus runtime containers to remove unnecessary software), Docker image composition (minimizing the number of layers), the Docker Remote API (optimizing how images are pushed and pulled), and CI/CD Integration (automate building, versioning, and deploying images to production). We’ll also examine the tools that ECR provides to make Docker image management easier so that you can focus on building your application. Finally, we'll hear from Pinterest about how they use ECR and Docker, some valuable experiences gained along the way, and best practices for using ECR with Apache Mesos.”
Jacksonville Users Group Presentation on how to integrate Angular2 and Spring Boot Microservices. It also shows you how to containerize an Angular2 app as a Node.js application and Spring Boot. The code can be found in http://github.com/erwindev.
MuleSoft Manchester Meetup #3 slides 31st March 2020Ieva Navickaite
Francis Edwards from Saint-Gobain Building Distribution presented on design practices for accelerating API delivery using Anypoint Platform. He discussed how to integrate API design with development using RAML and how elements like title, version, and baseUri are used across different Anypoint tools. Venkata Nallapuneni from Rathbone Brothers then presented on DataWeave 2.0 and how it has improved and simplified data transformation compared to Mule Expression Language.
Deploying and Scaling a Rails Application with Docker and FriendsInvisiblelines
Scaling a Docker application across multiple nodes requires leveraging additional tools. Here I outline one approach using Docker Compose, Swarm, Consul and Registrator.
Kubernetes is a container cluster manager that aims to provide a platform for automating deployment, scaling, and operations of application containers across clusters of machines. It uses pods as the basic building block, which are groups of application containers that share storage and networking resources. Kubernetes includes control planes for replication, scheduling, and services to expose applications. It supports deployment of multi-tier applications through replication controllers, services, labels, and pod templates.
Kubernetes is an open-source system for automating deployment, scaling, and management of containerized applications. It was originally developed by Google based on years of experience running production workloads at scale. Kubernetes groups containers into logical units called pods and handles tasks like scheduling, health checking, scaling and rollbacks. The main components include a master node that manages the cluster and worker nodes that run application containers scheduled by the master.
Docker at Shopify: From This-Looks-Fun to Production by Simon Eskildsen (Shop...Docker, Inc.
Since July 2014 Shopify's been serving thousands of requests per second of production web traffic from Docker containers. This was an 8 month effort, with multiple pivots of direction from the team—and we're only getting started. This talk covers the lessons learned through the trial and error of an in-flight architecture redesign, spanning hundreds of hosts, as well as the technical vision of the future of our platform.
3 years ago, Meetic chose to rebuild it's backend architecture using microservices and an event driven strategy. As we where moving along our old legacy application, testing features became gradually a pain, especially when those features rely on multiple changes across multiple components. Whatever the number of application you manage, unit testing is easy, as well as functional testing on a microservice. A good gherkin framework and a set of docker container can do the job. The real challenge is set in end-to-end testing even more when a feature can involve up to 60 different components.
To solve that issue, Meetic is building a Kubernetes strategy around testing. To do such a thing we need to :
- Be able to generate a docker container for each pull-request on any component of the stack
- Be able to create a full testing environment in the simplest way
- Be able to launch automated test on this newly created environment
- Have a clean-up process to destroy testing environment after tests To separate the various testing environment, we chose to use Kubernetes Namespaces each containing a variant of the Meetic stack. But when it comes to Kubernetes, managing multiple namespaces can be hard. Yaml configuration files need to be shared in a way that each people / automated job can access to them and modify them without impacting others.
This is typically why Meetic chose to develop it's own tool to manage namespace through a cli tool, or a REST API on which we can plug a friendly UI.
In this talk we will tell you the story of our CI/CD evolution to satisfy the need to create a docker container for each new pull request. And we will show you how to make end-to-end testing easier using Blackbeard, the tool we developed to handle the need to manage namespaces inspired by Helm.
OpenStack is an open source cloud computing platform that provides infrastructure as a service. It abstracts compute, storage, and networking resources from physical hardware into a dashboard that manages these resources as virtual machines, object storage, and virtual networks. OpenStack uses a central dashboard and various components like Nova (compute), Glance (images), Swift (object storage), Neutron (networking), and Keystone (identity) that can work with different underlying hardware and be deployed both publicly or privately. Neutron provides network as a service and tools for building advanced virtual networks using plugins that support technologies like Open vSwitch, Linux bridges, NSX, and OpenDaylight.
Kubernetes deep dive - - Huawei 2015-10Vishnu Kannan
Kubernetes is an open-source container orchestration system that automates deployment, scaling, and management of containerized applications. It was originally designed by Google based on years of experience running containers internally. Kubernetes runs containerized applications across multiple machines, dynamically allocating resources and balancing load. It supports both public and private cloud environments as well as bare metal servers. The system aims to simplify container operations while providing portability and scalability.
Cloud Native Night, January 2018, Munich: Workshop led by Jörg Schad (@joerg_schad, Technical Lead Community Projects at Mesosphere)
Join our Meetup: https://www.meetup.com/de-DE/cloud-native-muc
PLEASE NOTE: During this workshop, Jörg showed many demos and the audience could participate on their laptops. Unfortunately, we can't provide these demos. Nevertheless, Jörg's slides give a deep dive into the topic.
ABSTRACT: Kubernetes has been one of the topics in 2017 and will probably remain so in 2018. In this hands-on technical workshop you will learn how best to deploy, operate, and scale Kubernetes clusters from one to hundreds of nodes using DC/OS. You will learn how to integrate and run Kubernetes alongside traditional applications and fast data services of your choice (e.g. Apache Cassandra, Apache Kafka, Apache Spark, TensorFlow, and more) on any infrastructure.
This workshop best suits operators focussed on keeping their apps and services up and running in production and developers focussed on quickly delivering internal and customer facing apps into production.
You will learn how to:
- Introduction to Kubernetes and DC/OS (including the differences between both)
- Deploy Kubernetes on DC/OS in a secure, highly available, and fault-tolerant manner
- Solve operational challenges of running a large/multiple Kubernetes cluster
- One-click deploy big data stateful and stateless services alongside a Kubernetes cluster
- Play 2.0 is a web framework for Java and Scala that simplifies development by embracing HTTP rather than fighting it
- It takes a new approach to building web apps in Java by not being built on top of servlet APIs and using an asynchronous programming model
- Developing, testing, and deploying a Play app locally and to CloudFoundry involves creating a project, running it locally, and pushing the compiled code to CloudFoundry which automatically detects and supports Play apps
This document discusses implementing microservices using Docker Swarm and Consul. It recommends programming languages and tools for orchestration, databases, load balancing, monitoring, and other functions. Docker Swarm allows clustering Docker hosts into a pool of resources. Consul provides service discovery, configuration, and failure detection across multiple datacenters. Consul-Template listens for Consul updates and configures applications. Registrator automatically registers and deregisters Docker services with Consul. An example scenario shows how services scale across nodes with this architecture.
Orchestration tool roundup kubernetes vs. docker vs. heat vs. terra form vs...Nati Shalom
Video recording: https://www.youtube.com/watch?v=tGlIgUeoGz8
It’s no news that containers represent a portable unit of deployment, and OpenStack has proven an ideal environment for running container workloads. However, where it usually becomes more complex is that many times an application is often built out of multiple containers. What’s more, setting up a cluster of container images can be fairly cumbersome because you need to make one container aware of another and expose intimate details that are required for them to communicate which is not trivial especially if they’re not on the same host.
These scenarios have instigated the demand for some kind of orchestrator. The list of container orchestrators is growing fairly fast. This session will compare the different orchestation projects out there - from Heat to Kubernetes to TOSCA - and help you choose the right tool for the job.
Session link from teh summit: https://openstacksummitmay2015vancouver.sched.org/event/abd484e0dedcb9774edda1548ad47518#.VV5eh5NViko
This presentation includes information on Kubernetes Architecture, Container Orchestration, Internal Routing, External Routing, Configuration Management, Credentials Management, Persistent Volumes, Rolling Out Updates, Autoscaling, Package Management, and a Hello World example using Helm.
One of the main sources of concerns when switching to the container paradigm is security. When dealing with big amounts of sensitive customer data it’s very important to be able to guarantee that the data is transported safely between the different components of the system. This talk will focus on how to setup a Flink cluster to run on a Kubernetes environment taking into account all security aspects to ensure secured communication between the nodes of the cluster, state backend and also for job submission, all taking advantage of Kubernetes tools.
Hands-On Introduction to Kubernetes at LISA17Ryan Jarvinen
This document provides an agenda and instructions for a hands-on introduction to Kubernetes tutorial. The tutorial will cover Kubernetes basics like pods, services, deployments and replica sets. It includes steps for setting up a local Kubernetes environment using Minikube and demonstrates features like rolling updates, rollbacks and self-healing. Attendees will learn how to develop container-based applications locally with Kubernetes and deploy changes to preview them before promoting to production.
(Draft) Kubernetes - A Comprehensive OverviewBob Killen
Kubernetes is an open-source system for automating deployment, scaling, and management of containerized applications. It groups containers that make up an application into logical units for easy management and discovery called pods. Its main components include a master node that manages the cluster and worker nodes that run the applications. It uses labels to organize resources and selectors to group related objects. Common concepts include pods, services for discovery/load balancing, replica controllers for scaling, and namespaces for isolation. It provides mechanisms for configuration, storage, security, and networking out of the box to ensure containers can run reliably and be easily managed at scale.
Microservices, Kubernetes and Istio - A Great Fit!Animesh Singh
Microservices and containers are now influencing application design and deployment patterns. Sixty percent of all new applications will use cloud-enabled continuous delivery microservice architectures and containers. Service discovery, registration, and routing are fundamental tenets of microservices. Kubernetes provides a platform for running microservices. Kubernetes can be used to automate the deployment of Microservices and leverage features such as Kube-DNS, Config Maps, and Ingress service for managing those microservices. This configuration works fine for deployments up to a certain size. However, with complex deployments consisting of a large fleet of microservices, additional features are required to augment Kubernetes.
Exactly Once Delivery with Kafka - Kafka Tel-Aviv MeetupNatan Silnitsky
In this talk I go over the basic theory of messaging in distributed systems, the different message delivery guarantees in Kafka and the to use them.
I focus on exactly once delivery guarantees and the way Kafka implements it with transaction based messaging protocol.
Including a discussion of the latency/throughput trade-offs, resource utilisation and its overall advantages and shortcomings.
Finally, I show a use-case at Wix where exactly once delivery helped us solve a big problem.
This document discusses building a serverless data pipeline using AWS Lambda and other AWS managed services. It provides a 12 step process to create a serverless architecture that collects event data via API calls, stores it in DynamoDB, processes it with Lambda, sends it to Kinesis and Kinesis Firehose, and archives it in S3. This allows building scalable and cost-effective data pipelines and applications without having to manage any servers.
The Operator Pattern - Managing Stateful Services in KubernetesQAware GmbH
Cloud Native Night, January 2018, Mainz: Talk by Jakob Karalus (@krallistic, IT Consultant at codecentric)
Join our Meetup: https://www.meetup.com/de-DE/Cloud-Native-Night
Abstract: While it's easy to deploy stateless application with Kubernetes, it's harder for stateful software. Since applications often require custom functionality that Kubernetes can't provide, developers want to add more specialized patterns like automatic backups, failover or rebalancing to their Kubernetes deployments. In this talk, we will look at the Operator Pattern and other possibilities to extend the functionality of Kubernetes and how to use them to operate stateful applications.
Docker Kubernetes Istio
Understanding Docker and creating containers.
Container Orchestration based on Kubernetes
Blue Green Deployment, AB Testing, Canary Deployment, Traffic Rules based on Istio
This document provides an overview and demonstration of Clocker, an open source tool for managing Docker clouds and deploying composite applications on Docker. It discusses Clocker's components including its use of Brooklyn for application management and jclouds for provisioning. It also covers Clocker's features such as container placement strategies, networking using Weave, and roadmap items like support for Docker Swarm and improved networking.
Kubernetes is an open-source container management platform. It has a master-node architecture with control plane components like the API server on the master and node components like kubelet and kube-proxy on nodes. Kubernetes uses pods as the basic building block, which can contain one or more containers. Services provide discovery and load balancing for pods. Deployments manage pods and replicasets and provide declarative updates. Key concepts include volumes for persistent storage, namespaces for tenant isolation, labels for object tagging, and selector matching.
Istio is an open-source service mesh that provides traffic management, telemetry and security for microservices. It works by injecting Envoy sidecar proxies into applications. The document provides an overview of Istio architecture, setup, and how it can be used for traffic management features like canary releases and advanced load balancing.
Docker at Shopify: From This-Looks-Fun to Production by Simon Eskildsen (Shop...Docker, Inc.
Since July 2014 Shopify's been serving thousands of requests per second of production web traffic from Docker containers. This was an 8 month effort, with multiple pivots of direction from the team—and we're only getting started. This talk covers the lessons learned through the trial and error of an in-flight architecture redesign, spanning hundreds of hosts, as well as the technical vision of the future of our platform.
3 years ago, Meetic chose to rebuild it's backend architecture using microservices and an event driven strategy. As we where moving along our old legacy application, testing features became gradually a pain, especially when those features rely on multiple changes across multiple components. Whatever the number of application you manage, unit testing is easy, as well as functional testing on a microservice. A good gherkin framework and a set of docker container can do the job. The real challenge is set in end-to-end testing even more when a feature can involve up to 60 different components.
To solve that issue, Meetic is building a Kubernetes strategy around testing. To do such a thing we need to :
- Be able to generate a docker container for each pull-request on any component of the stack
- Be able to create a full testing environment in the simplest way
- Be able to launch automated test on this newly created environment
- Have a clean-up process to destroy testing environment after tests To separate the various testing environment, we chose to use Kubernetes Namespaces each containing a variant of the Meetic stack. But when it comes to Kubernetes, managing multiple namespaces can be hard. Yaml configuration files need to be shared in a way that each people / automated job can access to them and modify them without impacting others.
This is typically why Meetic chose to develop it's own tool to manage namespace through a cli tool, or a REST API on which we can plug a friendly UI.
In this talk we will tell you the story of our CI/CD evolution to satisfy the need to create a docker container for each new pull request. And we will show you how to make end-to-end testing easier using Blackbeard, the tool we developed to handle the need to manage namespaces inspired by Helm.
OpenStack is an open source cloud computing platform that provides infrastructure as a service. It abstracts compute, storage, and networking resources from physical hardware into a dashboard that manages these resources as virtual machines, object storage, and virtual networks. OpenStack uses a central dashboard and various components like Nova (compute), Glance (images), Swift (object storage), Neutron (networking), and Keystone (identity) that can work with different underlying hardware and be deployed both publicly or privately. Neutron provides network as a service and tools for building advanced virtual networks using plugins that support technologies like Open vSwitch, Linux bridges, NSX, and OpenDaylight.
Kubernetes deep dive - - Huawei 2015-10Vishnu Kannan
Kubernetes is an open-source container orchestration system that automates deployment, scaling, and management of containerized applications. It was originally designed by Google based on years of experience running containers internally. Kubernetes runs containerized applications across multiple machines, dynamically allocating resources and balancing load. It supports both public and private cloud environments as well as bare metal servers. The system aims to simplify container operations while providing portability and scalability.
Cloud Native Night, January 2018, Munich: Workshop led by Jörg Schad (@joerg_schad, Technical Lead Community Projects at Mesosphere)
Join our Meetup: https://www.meetup.com/de-DE/cloud-native-muc
PLEASE NOTE: During this workshop, Jörg showed many demos and the audience could participate on their laptops. Unfortunately, we can't provide these demos. Nevertheless, Jörg's slides give a deep dive into the topic.
ABSTRACT: Kubernetes has been one of the topics in 2017 and will probably remain so in 2018. In this hands-on technical workshop you will learn how best to deploy, operate, and scale Kubernetes clusters from one to hundreds of nodes using DC/OS. You will learn how to integrate and run Kubernetes alongside traditional applications and fast data services of your choice (e.g. Apache Cassandra, Apache Kafka, Apache Spark, TensorFlow, and more) on any infrastructure.
This workshop best suits operators focussed on keeping their apps and services up and running in production and developers focussed on quickly delivering internal and customer facing apps into production.
You will learn how to:
- Introduction to Kubernetes and DC/OS (including the differences between both)
- Deploy Kubernetes on DC/OS in a secure, highly available, and fault-tolerant manner
- Solve operational challenges of running a large/multiple Kubernetes cluster
- One-click deploy big data stateful and stateless services alongside a Kubernetes cluster
- Play 2.0 is a web framework for Java and Scala that simplifies development by embracing HTTP rather than fighting it
- It takes a new approach to building web apps in Java by not being built on top of servlet APIs and using an asynchronous programming model
- Developing, testing, and deploying a Play app locally and to CloudFoundry involves creating a project, running it locally, and pushing the compiled code to CloudFoundry which automatically detects and supports Play apps
This document discusses implementing microservices using Docker Swarm and Consul. It recommends programming languages and tools for orchestration, databases, load balancing, monitoring, and other functions. Docker Swarm allows clustering Docker hosts into a pool of resources. Consul provides service discovery, configuration, and failure detection across multiple datacenters. Consul-Template listens for Consul updates and configures applications. Registrator automatically registers and deregisters Docker services with Consul. An example scenario shows how services scale across nodes with this architecture.
Orchestration tool roundup kubernetes vs. docker vs. heat vs. terra form vs...Nati Shalom
Video recording: https://www.youtube.com/watch?v=tGlIgUeoGz8
It’s no news that containers represent a portable unit of deployment, and OpenStack has proven an ideal environment for running container workloads. However, where it usually becomes more complex is that many times an application is often built out of multiple containers. What’s more, setting up a cluster of container images can be fairly cumbersome because you need to make one container aware of another and expose intimate details that are required for them to communicate which is not trivial especially if they’re not on the same host.
These scenarios have instigated the demand for some kind of orchestrator. The list of container orchestrators is growing fairly fast. This session will compare the different orchestation projects out there - from Heat to Kubernetes to TOSCA - and help you choose the right tool for the job.
Session link from teh summit: https://openstacksummitmay2015vancouver.sched.org/event/abd484e0dedcb9774edda1548ad47518#.VV5eh5NViko
This presentation includes information on Kubernetes Architecture, Container Orchestration, Internal Routing, External Routing, Configuration Management, Credentials Management, Persistent Volumes, Rolling Out Updates, Autoscaling, Package Management, and a Hello World example using Helm.
One of the main sources of concerns when switching to the container paradigm is security. When dealing with big amounts of sensitive customer data it’s very important to be able to guarantee that the data is transported safely between the different components of the system. This talk will focus on how to setup a Flink cluster to run on a Kubernetes environment taking into account all security aspects to ensure secured communication between the nodes of the cluster, state backend and also for job submission, all taking advantage of Kubernetes tools.
Hands-On Introduction to Kubernetes at LISA17Ryan Jarvinen
This document provides an agenda and instructions for a hands-on introduction to Kubernetes tutorial. The tutorial will cover Kubernetes basics like pods, services, deployments and replica sets. It includes steps for setting up a local Kubernetes environment using Minikube and demonstrates features like rolling updates, rollbacks and self-healing. Attendees will learn how to develop container-based applications locally with Kubernetes and deploy changes to preview them before promoting to production.
(Draft) Kubernetes - A Comprehensive OverviewBob Killen
Kubernetes is an open-source system for automating deployment, scaling, and management of containerized applications. It groups containers that make up an application into logical units for easy management and discovery called pods. Its main components include a master node that manages the cluster and worker nodes that run the applications. It uses labels to organize resources and selectors to group related objects. Common concepts include pods, services for discovery/load balancing, replica controllers for scaling, and namespaces for isolation. It provides mechanisms for configuration, storage, security, and networking out of the box to ensure containers can run reliably and be easily managed at scale.
Microservices, Kubernetes and Istio - A Great Fit!Animesh Singh
Microservices and containers are now influencing application design and deployment patterns. Sixty percent of all new applications will use cloud-enabled continuous delivery microservice architectures and containers. Service discovery, registration, and routing are fundamental tenets of microservices. Kubernetes provides a platform for running microservices. Kubernetes can be used to automate the deployment of Microservices and leverage features such as Kube-DNS, Config Maps, and Ingress service for managing those microservices. This configuration works fine for deployments up to a certain size. However, with complex deployments consisting of a large fleet of microservices, additional features are required to augment Kubernetes.
Exactly Once Delivery with Kafka - Kafka Tel-Aviv MeetupNatan Silnitsky
In this talk I go over the basic theory of messaging in distributed systems, the different message delivery guarantees in Kafka and the to use them.
I focus on exactly once delivery guarantees and the way Kafka implements it with transaction based messaging protocol.
Including a discussion of the latency/throughput trade-offs, resource utilisation and its overall advantages and shortcomings.
Finally, I show a use-case at Wix where exactly once delivery helped us solve a big problem.
This document discusses building a serverless data pipeline using AWS Lambda and other AWS managed services. It provides a 12 step process to create a serverless architecture that collects event data via API calls, stores it in DynamoDB, processes it with Lambda, sends it to Kinesis and Kinesis Firehose, and archives it in S3. This allows building scalable and cost-effective data pipelines and applications without having to manage any servers.
The Operator Pattern - Managing Stateful Services in KubernetesQAware GmbH
Cloud Native Night, January 2018, Mainz: Talk by Jakob Karalus (@krallistic, IT Consultant at codecentric)
Join our Meetup: https://www.meetup.com/de-DE/Cloud-Native-Night
Abstract: While it's easy to deploy stateless application with Kubernetes, it's harder for stateful software. Since applications often require custom functionality that Kubernetes can't provide, developers want to add more specialized patterns like automatic backups, failover or rebalancing to their Kubernetes deployments. In this talk, we will look at the Operator Pattern and other possibilities to extend the functionality of Kubernetes and how to use them to operate stateful applications.
Docker Kubernetes Istio
Understanding Docker and creating containers.
Container Orchestration based on Kubernetes
Blue Green Deployment, AB Testing, Canary Deployment, Traffic Rules based on Istio
This document provides an overview and demonstration of Clocker, an open source tool for managing Docker clouds and deploying composite applications on Docker. It discusses Clocker's components including its use of Brooklyn for application management and jclouds for provisioning. It also covers Clocker's features such as container placement strategies, networking using Weave, and roadmap items like support for Docker Swarm and improved networking.
Kubernetes is an open-source container management platform. It has a master-node architecture with control plane components like the API server on the master and node components like kubelet and kube-proxy on nodes. Kubernetes uses pods as the basic building block, which can contain one or more containers. Services provide discovery and load balancing for pods. Deployments manage pods and replicasets and provide declarative updates. Key concepts include volumes for persistent storage, namespaces for tenant isolation, labels for object tagging, and selector matching.
Istio is an open-source service mesh that provides traffic management, telemetry and security for microservices. It works by injecting Envoy sidecar proxies into applications. The document provides an overview of Istio architecture, setup, and how it can be used for traffic management features like canary releases and advanced load balancing.
Integrating Infrastructure as Code into a Continuous Delivery Pipeline | AWS ...Amazon Web Services
This document discusses integrating infrastructure as code into a continuous delivery pipeline. It covers DevOps principles like collaboration, automation, and monitoring everything. It discusses benefits like increased velocity and reduced risk. Infrastructure as code is realized using tools like Ansible, Chef, and Terraform to define infrastructure in code. This allows infrastructure to be treated like code and integrated into software development pipelines for continuous integration and delivery of both application and infrastructure changes.
The document discusses various ways to extend Kubernetes including custom resources, API server aggregation, CNI plugins, CSI plugins, device plugins, authentication and authorization webhooks, custom scheduler plugins, custom metrics for HPA, kubectl plugins, and more. It provides examples and explanations of how each works, the interfaces involved, and configuration options. The final section promotes the author's book "Mastering Kubernetes 2nd Edition" which covers these extension concepts and capabilities in more detail.
Deploying windows containers with kubernetesBen Hall
The document discusses deploying Windows containers with Kubernetes. It covers building Windows containers, deploying containers on Kubernetes, and operating Kubernetes. Specifically, it shows how to:
- Build a Windows container with SQL Server using Docker
- Deploy a .NET Core app container to Kubernetes and expose it using a load balancer
- Scale the deployment to multiple replicas and observe traffic distribution
- Perform rolling updates to deploy new versions of the application
The document provides an overview of the logical architecture of Kubernetes. It describes the main components that make up the Kubernetes control plane (API server, scheduler, etc.) and Kubernetes workers, as well as core Kubernetes objects like pods, replica sets, deployments, services, ingress and configmaps/secrets. It also touches on controllers, operators, Kubernetes manifests and provides an example manifest configuration.
This document provides an overview of Kubernetes concepts including architecture, fundamental objects like pods and services, and demonstrations. It begins with an agenda then covers Kubernetes architecture including the master node, worker nodes, and control loop. It describes core objects like pods, replica sets, deployments, services, and labels/selectors. The document demonstrates deploying and accessing the guestbook application using these objects. It concludes with asking for questions and describing goals for educational meetups on cloud native technologies.
Kubernetes from Dev to Prod summarizes GoEuro's transition from a legacy environment to using Kubernetes and CI/CD pipelines from development to production. Key points:
- GoEuro transitioned 50+ services across 150 engineers from separate development and operations teams to using Kubernetes, Docker, and CI/CD pipelines in 4 months.
- They developed "hyper-vm" single-node Kubernetes VMs for local development and testing and "y8s" for sharing Kubernetes configurations across environments from development through production.
- CI/CD pipelines were automated using GitLab CI and custom implementations running jobs on "hyper-vm" agents to deploy to environments from preview through production.
- Additional services
Cloud Native Night, April 2018, Mainz: Workshop led by Jörg Schad (@joerg_schad, Technical Community Lead / Developer at Mesosphere)
Join our Meetup: https://www.meetup.com/de-DE/Cloud-Native-Night/
PLEASE NOTE:
During this workshop, Jörg showed many demos and the audience could participate on their laptops. Unfortunately, we can't provide these demos. Nevertheless, Jörg's slides give a deep dive into the topic.
DETAILS ABOUT THE WORKSHOP:
Kubernetes has been one of the topics in 2017 and will probably remain so in 2018. In this hands-on technical workshop you will learn how best to deploy, operate and scale Kubernetes clusters from one to hundreds of nodes using DC/OS. You will learn how to integrate and run Kubernetes alongside traditional applications and fast data services of your choice (e.g. Apache Cassandra, Apache Kafka, Apache Spark, TensorFlow and more) on any infrastructure.
This workshop best suits operators focussed on keeping their apps and services up and running in production and developers focussed on quickly delivering internal and customer facing apps into production.
You will learn how to:
- Introduction to Kubernetes and DC/OS (including the differences between both)
- Deploy Kubernetes on DC/OS in a secure, highly available, and fault-tolerant manner
- Solve operational challenges of running a large/multiple Kubernetes cluster
- One-click deploy big data stateful and stateless services alongside a Kubernetes cluster
Openstack days sv building highly available services using kubernetes (preso)Allan Naim
This document discusses Google Cloud Platform's Kubernetes and how it can be used to build highly available services. It provides an overview of Kubernetes concepts like pods, labels, replica sets, volumes, and services. It then describes how Kubernetes Cluster Federation allows deploying applications across multiple Kubernetes clusters for high availability, geographic scaling, and other benefits. It outlines how to create clusters, configure the federated control plane, add clusters to the federation, deploy federated services and backends, and perform cross-cluster service discovery.
Docker clusters on AWS with Amazon ECS and KubernetesJulien SIMON
This document summarizes and compares Docker container management on AWS using Amazon ECS and Kubernetes. It provides an overview of ECS and ECR services, new features, customer case studies including Coursera and Segment, and resources for learning more. It also introduces Kubernetes as an open source container orchestrator, describes its architecture including pods, labels, replica sets, deployments and services. KOPS is presented as a tool for deploying and managing Kubernetes clusters on AWS. The Cloud Native Computing Foundation is discussed along with AWS' involvement to promote cloud native technologies.
Introduction to Container Storage Interface (CSI)Idan Atias
Among the cool stuff we do at Silk, my colleagues and I develop the Silk CSI Plugin for customers who use our system as the storage layer for their Kubernetes workloads.
Before deep diving into the code and as part of my ramp-up on this subject I prepared some slides that cover some basic and important information on this topic.
These slides start by recapping some basic storage principals in containers and Kubernetes, continues with some more advanced use cases (including an "offline demo" of persisting Redis data on EBS volumes), and ends with a detailed information on the CSI solution itself.
IMHO, reviewing these slides can improve your understanding on this matter and can get you started implementing your own CSI plugin.
The main sources of information I used for preparing these slides are:
* Official CSI docs
* Kubernetes Storage Lingo 101 - Saad Ali, Google
* Container Storage Interface: Present and Future - Jie Yu, Mesosphere, Inc.
Manchester MuleSoft Meetup #6 - Runtime Fabric with Mulesoft Akshata Sawant
Come join us at the Online Meetup to learn more about ServiceNow and Gmail Integration with MuleSoft with detailed Demo. Help us spread the knowledge of Mule!
A brief agenda:
> Networking and Knowledge sharing.
> MuleSoft Latest Product Release Updates.
> Runtime Fabric in depth architecture.
> Finally, we will wrap-up this event with the agenda for the next meetup.
Stay connected to get updates on what's new in MuleSoft.
The document provides an overview of application lifecycle management (ALM) in a serverless world. It discusses key concepts like continuous integration/delivery and testing practices for serverless applications. Serverless architectures using AWS Lambda and API Gateway are highlighted, along with how to manage deployments, configurations, and monitor applications.
The “Twelve-Factor” application model has come to represent twelve best practices for building modern, cloud-native applications. With guidance on things like configuration, deployment, runtime, and multiple service communication, the Twelve-Factor model prescribes best practices that apply to everything from web applications to APIs to data processing applications.
Although serverless computing and AWS Lambda have changed how application development is done, the “Twelve-Factor” best practices remain relevant and applicable in a serverless world. In this talk, Chris will share with you how to apply the “Twelve-Factor” model to serverless application development with AWS Lambda and Amazon API Gateway and show you how these services enable you to build scalable, low cost, and low administration applications.
Webinar: Serverless Architectures with AWS Lambda and MongoDB AtlasMongoDB
It’s easier than ever to power serverless architectures with our managed MongoDB as a service, MongoDB Atlas. In this session, we will explore the rise of serverless architectures and how they’ve rapidly integrated into public and private cloud offerings.
OSDC 2018 | Three years running containers with Kubernetes in Production by T...NETWAYS
The talk gives a state of the art update of experiences with deploying applications in Kubernetes on scale. If in clouds or on premises, Kubernetes took over the leading role as a container operating system. The central paradigm of stateless containers connected to storage and services is the core of Kubernetes. However, it can be extended to distributed databases, Machine Learning, Windows VMs in Kubernetes. All these applications have been considered as edge cases a few years ago, however, are going more and more mainstream today.
UI5con 2024 - Keynote: Latest News about UI5 and it’s EcosystemPeter Muessig
Learn about the latest innovations in and around OpenUI5/SAPUI5: UI5 Tooling, UI5 linter, UI5 Web Components, Web Components Integration, UI5 2.x, UI5 GenAI.
Recording:
https://www.youtube.com/live/MSdGLG2zLy8?si=INxBHTqkwHhxV5Ta&t=0
Microservice Teams - How the cloud changes the way we workSven Peters
A lot of technical challenges and complexity come with building a cloud-native and distributed architecture. The way we develop backend software has fundamentally changed in the last ten years. Managing a microservices architecture demands a lot of us to ensure observability and operational resiliency. But did you also change the way you run your development teams?
Sven will talk about Atlassian’s journey from a monolith to a multi-tenanted architecture and how it affected the way the engineering teams work. You will learn how we shifted to service ownership, moved to more autonomous teams (and its challenges), and established platform and enablement teams.
Enhanced Screen Flows UI/UX using SLDS with Tom KittPeter Caitens
Join us for an engaging session led by Flow Champion, Tom Kitt. This session will dive into a technique of enhancing the user interfaces and user experiences within Screen Flows using the Salesforce Lightning Design System (SLDS). This technique uses Native functionality, with No Apex Code, No Custom Components and No Managed Packages required.
Odoo releases a new update every year. The latest version, Odoo 17, came out in October 2023. It brought many improvements to the user interface and user experience, along with new features in modules like accounting, marketing, manufacturing, websites, and more.
The Odoo 17 update has been a hot topic among startups, mid-sized businesses, large enterprises, and Odoo developers aiming to grow their businesses. Since it is now already the first quarter of 2024, you must have a clear idea of what Odoo 17 entails and what it can offer your business if you are still not aware of it.
This blog covers the features and functionalities. Explore the entire blog and get in touch with expert Odoo ERP consultants to leverage Odoo 17 and its features for your business too.
An Overview of Odoo ERP
Odoo ERP was first released as OpenERP software in February 2005. It is a suite of business applications used for ERP, CRM, eCommerce, websites, and project management. Ten years ago, the Odoo Enterprise edition was launched to help fund the Odoo Community version.
When you compare Odoo Community and Enterprise, the Enterprise edition offers exclusive features like mobile app access, Odoo Studio customisation, Odoo hosting, and unlimited functional support.
Today, Odoo is a well-known name used by companies of all sizes across various industries, including manufacturing, retail, accounting, marketing, healthcare, IT consulting, and R&D.
The latest version, Odoo 17, has been available since October 2023. Key highlights of this update include:
Enhanced user experience with improvements to the command bar, faster backend page loading, and multiple dashboard views.
Instant report generation, credit limit alerts for sales and invoices, separate OCR settings for invoice creation, and an auto-complete feature for forms in the accounting module.
Improved image handling and global attribute changes for mailing lists in email marketing.
A default auto-signature option and a refuse-to-sign option in HR modules.
Options to divide and merge manufacturing orders, track the status of manufacturing orders, and more in the MRP module.
Dark mode in Odoo 17.
Now that the Odoo 17 announcement is official, let’s look at what’s new in Odoo 17!
What is Odoo ERP 17?
Odoo 17 is the latest version of one of the world’s leading open-source enterprise ERPs. This version has come up with significant improvements explained here in this blog. Also, this new version aims to introduce features that enhance time-saving, efficiency, and productivity for users across various organisations.
Odoo 17, released at the Odoo Experience 2023, brought notable improvements to the user interface and added new functionalities with enhancements in performance, accessibility, data analysis, and management, further expanding its reach in the market.
Flutter is a popular open source, cross-platform framework developed by Google. In this webinar we'll explore Flutter and its architecture, delve into the Flutter Embedder and Flutter’s Dart language, discover how to leverage Flutter for embedded device development, learn about Automotive Grade Linux (AGL) and its consortium and understand the rationale behind AGL's choice of Flutter for next-gen IVI systems. Don’t miss this opportunity to discover whether Flutter is right for your project.
Preparing Non - Technical Founders for Engaging a Tech AgencyISH Technologies
Preparing non-technical founders before engaging a tech agency is crucial for the success of their projects. It starts with clearly defining their vision and goals, conducting thorough market research, and gaining a basic understanding of relevant technologies. Setting realistic expectations and preparing a detailed project brief are essential steps. Founders should select a tech agency with a proven track record and establish clear communication channels. Additionally, addressing legal and contractual considerations and planning for post-launch support are vital to ensure a smooth and successful collaboration. This preparation empowers non-technical founders to effectively communicate their needs and work seamlessly with their chosen tech agency.Visit our site to get more details about this. Contact us today www.ishtechnologies.com.au
A neural network is a machine learning program, or model, that makes decisions in a manner similar to the human brain, by using processes that mimic the way biological neurons work together to identify phenomena, weigh options and arrive at conclusions.
Project Management: The Role of Project Dashboards.pdfKarya Keeper
Project management is a crucial aspect of any organization, ensuring that projects are completed efficiently and effectively. One of the key tools used in project management is the project dashboard, which provides a comprehensive view of project progress and performance. In this article, we will explore the role of project dashboards in project management, highlighting their key features and benefits.
How Can Hiring A Mobile App Development Company Help Your Business Grow?ToXSL Technologies
ToXSL Technologies is an award-winning Mobile App Development Company in Dubai that helps businesses reshape their digital possibilities with custom app services. As a top app development company in Dubai, we offer highly engaging iOS & Android app solutions. https://rb.gy/necdnt
Top Benefits of Using Salesforce Healthcare CRM for Patient Management.pdfVALiNTRY360
Salesforce Healthcare CRM, implemented by VALiNTRY360, revolutionizes patient management by enhancing patient engagement, streamlining administrative processes, and improving care coordination. Its advanced analytics, robust security, and seamless integration with telehealth services ensure that healthcare providers can deliver personalized, efficient, and secure patient care. By automating routine tasks and providing actionable insights, Salesforce Healthcare CRM enables healthcare providers to focus on delivering high-quality care, leading to better patient outcomes and higher satisfaction. VALiNTRY360's expertise ensures a tailored solution that meets the unique needs of any healthcare practice, from small clinics to large hospital systems.
For more info visit us https://valintry360.com/solutions/health-life-sciences
Unlock the Secrets to Effortless Video Creation with Invideo: Your Ultimate G...The Third Creative Media
"Navigating Invideo: A Comprehensive Guide" is an essential resource for anyone looking to master Invideo, an AI-powered video creation tool. This guide provides step-by-step instructions, helpful tips, and comparisons with other AI video creators. Whether you're a beginner or an experienced video editor, you'll find valuable insights to enhance your video projects and bring your creative ideas to life.
UI5con 2024 - Boost Your Development Experience with UI5 Tooling ExtensionsPeter Muessig
The UI5 tooling is the development and build tooling of UI5. It is built in a modular and extensible way so that it can be easily extended by your needs. This session will showcase various tooling extensions which can boost your development experience by far so that you can really work offline, transpile your code in your project to use even newer versions of EcmaScript (than 2022 which is supported right now by the UI5 tooling), consume any npm package of your choice in your project, using different kind of proxies, and even stitching UI5 projects during development together to mimic your target environment.
Most important New features of Oracle 23c for DBAs and Developers. You can get more idea from my youtube channel video from https://youtu.be/XvL5WtaC20A
10. www.container-solutions.com | info@container-solutions.com
Pods
● A pod is one or more containers
● Ensures co-location / shared fate
● Pods are scheduled, then do not move between nodes
● Containers share resources within the pod:
➔ Volumes
➔ Network / IP
➔ Port space
➔ CPU / Memory allocations
17. www.container-solutions.com | info@container-solutions.com
Replication Controllers
● Define the number of replicas of a pod
● Will scheduled across all applicable nodes
● Can change replica value to scale up/down
● Which pods are scaled depends on RC selector
● Labels and selectors are used for grouping
● Can do quite complex things with RCs and labels
19. www.container-solutions.com | info@container-solutions.com
Replica Set
Replica Set is the next-generation Replication Controller. The only difference
between a Replica Set and a Replication Controller right now is the selector
support. Replica Set supports the new set-based selector which allow filtering
keys according to a set of values:
- In
- Notin
- exists (only the key identifier)
For example:
environment in (production, qa)
tier notin (frontend, backend)
partition
!partition
20. www.container-solutions.com | info@container-solutions.com
Deployments
A Deployment is responsible for creating
and updating instances of your
application
● Create a Deployment to bring up Pods and a
replica set.
● Check the status of a Deployment to see if it
succeeds or not.
● Later, update that Deployment to recreate the
Pods (for example, to use a new image).
● Rollback to an earlier Deployment revision if
the current Deployment isn’t stable.
● Pause and resume a Deployment.
22. www.container-solutions.com | info@container-solutions.com
Services
“defines a logical set of Pods and a
policy by which to access them”
● As Pods are ephemeral, we can't depend
on Pod IPs
● Services find pods that match certain
selection criteria
● Services can load balance between
multiple Pods
● Services can have a single IP that doesn’t
change
23. www.container-solutions.com | info@container-solutions.com
Services
A group of pods that act as one == Service
- group == selector
Defines access policy
- LoadBalanced, NodePort
Gets a stable virtual IP and Port
- Called the service portal
- Also a DNS name
- On prem additional loadbalancer is needed
VIP is captured by kube-proxy
- Watches the service consistency
- Updates when backend changes
27. www.container-solutions.com | info@container-solutions.com
Architecture
Master Node (“Control Plane”)
Api server
- Point of interaction with the cluster
- Exposes an http endpoint
Controller Manager
- Responsible for most of the important stuff
- Interacts with the api server to retrieve cluster state
- Responsible for configuring networking
- Allocates node CIDRs
- Ensures correct number of pods are running
- Reacts to Nodes being added / deleted
- Manages Service Accounts and security tokens
Scheduler
- Schedules newly created pods to a Node
29. www.container-solutions.com | info@container-solutions.com
kubelet
- Agent for running Pods
- Mounts volumes for Pods where required
- Reports the status of Pods back to rest of system
kube-proxy
- Enforces network rules on each Node (uses iptables)
- Responsible for forwarding packets to correct destination
Architecture
Worker Node
30. www.container-solutions.com | info@container-solutions.com
Master Node (api-server)
- Takes an argument for etcd servers
Master Node (controller-manager)
- Takes an argument for api server
- Creates/defines virtual networks for containers and services
- Takes an argument for cluster node CIDR
- Takes an argument for service CIDR
kubelet
- Configures the Docker bridge
- Takes an address for the cluster DNS
kube-proxy
- Takes an argument for the cluster node CIDR
Architecture
Networking
34. www.container-solutions.com | info@container-solutions.com
We will use CloudFormation to:
- Launch EC2 instances into an existing VPC
- Create a subnet for each kubernetes cluster
- Create a route table for each subnet
- Create Security Groups (firewall rules) for each cluster
- Create Autoscale Groups for Master and Worker nodes
AWS
Today: EC2, VPC and CloudFormation
Instance Configuration:
- Userdata: Instructions to be run by AWS cloud-init system after boot
- Chef: Userdata will instruct instances to bootstrap to Chef server
CloudFormation:
- Method of keeping Infrastructure as Code
- JSON based template that defines AWS Resources
35. www.container-solutions.com | info@container-solutions.com
AWS
Other ways to build
Getting Started guide: http://kubernetes.io/docs/getting-started-guides/aws/
- $ set=something ; wget something | bash
- Great for getting a cluster up and running quickly
- Inflexible for integration into existing VPCs
- Fussy if you put anything else in the VPC it creates
Kops: https://github.com/kubernetes/kops
- “kubectl for clusters”
- Will become the standard way to launch onto AWS
- Still in alpha
Run with your own: https://github.com/kelseyhightower/kubernetes-the-hard-way
- Takes some time
- Expect to reverse-engineer
- You will know exactly how the cluster is put together
36. www.container-solutions.com | info@container-solutions.com
Using the --cloud-provider=aws flag, the kubernetes components can be instructed
to leverage AWS IaaS features.
Master instances (running controller-manager) must have an appropriate IAM role
assigned.
Kubernetes can then
- Create and destroy Elastic Load Balancers (ELBs)
- Add and delete routes from cluster Route Table
- Add and delete firewall rules on cluster Security Group
AWS and Kubernetes
Kubernetes is able to configure AWS
Relevant resources must be appropriately tagged:
- Name: KubernetesCluster
- Value: ClusterId
46. www.container-solutions.com | info@container-solutions.com
What’s happening?
$ ./find-nodes [cluster-id]
x.x.x.x
x.x.x.x
$ ssh ubuntu@x.x.x.x
$ ip route list
$ route -n
$ sudo iptables -L -t nat
# view route table in AWS, note that the pod CIDRs are routed directly to an EC2 NIC