Is there still room for innovation in container orchestration and scheduling
•
1 like•323 views
In the container ecosystem, there is perhaps no technology that has received more focus and attention than orchestration and scheduling. Mesos, Kubernetes, and Swarm have established themselves as the leading technology choices in this space. In this talk, Sheng will discuss what he learned from working directly with hundreds of users who have deployed one of these frameworks. He will look at how these frameworks will continue to evolve and if there’re any gaps and opportunities in container orchestration and scheduling. Sheng will make a case that there are still room for innovation and new orchestration and scheduling frameworks will be created in the future. He will discuss what new frameworks might look like--the features, functionalities, and attributes that differentiate them from the mainstream frameworks today.
Report
Share
Report
Share
Download to read offline
Recommended
Scale Kubernetes to support 50000 services
Kubernetes currently has two load balancing mode: userspace and IPTables. They both have limitation on scalability and performance. We introduced IPVS as third kube-proxy mode which scales kubernetes load balancer to support 50,000 services. Beyond that, control plane needs to be optimized in order to deploy 50,000 services. We will introduce alternative solutions and our prototypes with detailed performance data.
Fully automated kubernetes deployment and management
Fully Automated Kubernetes Deployment and Management (Peng Jiang, Rancher Labs) - Kubernetes is rapidly gaining popularity as a powerful container orchestration and scheduling platform. But deploying and managing Kubernetes clusters is still a challenge for many organizations.How to ensure Kubernetes clusters in different clouds and data centers can communicate with each other? How to automate the deployment of multiple Kubernetes clusters? How to incorporate the new Kubernetes Federation into multi cloud and multi datacenter deployments? How to manage the health of Kubernetes cluster itself? etc.
In this talk, Peng will share his experience on how to automate and simplify Kubernetes deployments, and discuss how some of the latest community projects (such as kubeadm and self-hosting Kubernetes) will help address the problems in the future.
Secure Containers with EPT Isolation
Secure Container solution is to enhance container security by isolating memory between Docker containers inside one VM with Intel VT-x EPT HW, which is highly effective to protect container’s memory and at the meantime defends ret2user privilege escalation attack that exploits kernel vulnerabilities (eg. CVE-2017-6074 UAF (use-after-free) vulnerability). It extends KVM interfaces which the guest OS can leverage to isolate container memory from other containers, and the interfaces rely on Intel VT-x EPT hardware extension and provide memory access protection for the container which sits in an isolated memory region. Each secure container has a dedicated EPT table rather than sharing one EPT table with guest OS, which enforces the cross-EPT memory access protection. The whole solution is user-friendly to fit in the existing cloud server infrastructure with very limited changes.
Running Legacy Applications with Containers
Enterprise data centers have to support a diverse of set of workloads: cloud native, big data, high performance computing, and legacy applications. While cloud native applications are ideal to run in Docker clusters, bare metal and virtualization infrastructures must still be supported in the data center. The result is a proliferation of clusters and technologies running in individual silos, resulting in high management costs and low utilization. This talk describes the challenges and experiences in implementing a shared cluster infrastructure based on Kubernetes to support big data, high performance computing, and VM-based workloads. The talk will show the deployment and scaling of a high performance computing workload manager, Spark, and OpenStack, and how the VM and Docker management can be integrated together.
Get a Taste of 1 k+ Nodes by a Handful of Servers
This document discusses containerized testing of OpenStack scalability. It explores using Docker containers to provision OpenStack components like Neutron and OpenDaylight in a distributed manner. The approach aims to test scalability up to 1,000 nodes using a handful of physical servers. It analyzes component footprints and tasks. Considerations for extending the tests to larger 1,000 node scales include physical server capacity, container placement density, network bandwidth, and more sophisticated test scenarios.
Linuxcon secureefficientcontainerimagemanagementharbor
This document discusses container image management in enterprises and introduces Project Harbor, an open source container registry. It covers key topics like container image basics, Project Harbor features, maintaining consistency of images between environments, security and access control of images, image distribution strategies, and high availability of container registries. Project Harbor provides an enterprise-grade private registry with features for user management, image replication, security, and integration with systems like LDAP. It aims to help organizations securely manage the distribution and storage of container images.
Unikernelized Linux
Presentation given at the 2017 LinuxCon China
Unikernel is a novel software technology that links an application with OS in the form of a library and packages them into a specialized image that facilitates direct deployment on a hypervisor. Comparing to the traditional VM or the recent containers, Unikernels are smaller, more secure and efficient, making them ideal for cloud environments. There are already lots of open source projects like OSv, Rumprun and so on. But why these existing unikernels have yet to gain large popularity broadly? We think Unikernels are facing three major challenges: 1. Compatibility with existing applications; 2. Lack of production support (e.g. monitoring, debugging, logging); 3. Lack of compelling use case. In this presentation, we will review our investigations and exploration of if-how we can convert Linux as Unikernel to eliminate these significant shortcomings, plus some explorations of coordinating and cooperating with hypervisor.
IP Virtual Server(IPVS) 101
In this slide, we discussed the IPVS, including the introduction, demonstration, implementation, and integration in Kubernetes.
IPVS was based on the netfilter and we discussed how it works with iptables and also compares the detail implementation in Kubernetes to show why IPVS has a better performance in IPTABLES.
Recommended
Scale Kubernetes to support 50000 services
Kubernetes currently has two load balancing mode: userspace and IPTables. They both have limitation on scalability and performance. We introduced IPVS as third kube-proxy mode which scales kubernetes load balancer to support 50,000 services. Beyond that, control plane needs to be optimized in order to deploy 50,000 services. We will introduce alternative solutions and our prototypes with detailed performance data.
Fully automated kubernetes deployment and management
Fully Automated Kubernetes Deployment and Management (Peng Jiang, Rancher Labs) - Kubernetes is rapidly gaining popularity as a powerful container orchestration and scheduling platform. But deploying and managing Kubernetes clusters is still a challenge for many organizations.How to ensure Kubernetes clusters in different clouds and data centers can communicate with each other? How to automate the deployment of multiple Kubernetes clusters? How to incorporate the new Kubernetes Federation into multi cloud and multi datacenter deployments? How to manage the health of Kubernetes cluster itself? etc.
In this talk, Peng will share his experience on how to automate and simplify Kubernetes deployments, and discuss how some of the latest community projects (such as kubeadm and self-hosting Kubernetes) will help address the problems in the future.
Secure Containers with EPT Isolation
Secure Container solution is to enhance container security by isolating memory between Docker containers inside one VM with Intel VT-x EPT HW, which is highly effective to protect container’s memory and at the meantime defends ret2user privilege escalation attack that exploits kernel vulnerabilities (eg. CVE-2017-6074 UAF (use-after-free) vulnerability). It extends KVM interfaces which the guest OS can leverage to isolate container memory from other containers, and the interfaces rely on Intel VT-x EPT hardware extension and provide memory access protection for the container which sits in an isolated memory region. Each secure container has a dedicated EPT table rather than sharing one EPT table with guest OS, which enforces the cross-EPT memory access protection. The whole solution is user-friendly to fit in the existing cloud server infrastructure with very limited changes.
Running Legacy Applications with Containers
Enterprise data centers have to support a diverse of set of workloads: cloud native, big data, high performance computing, and legacy applications. While cloud native applications are ideal to run in Docker clusters, bare metal and virtualization infrastructures must still be supported in the data center. The result is a proliferation of clusters and technologies running in individual silos, resulting in high management costs and low utilization. This talk describes the challenges and experiences in implementing a shared cluster infrastructure based on Kubernetes to support big data, high performance computing, and VM-based workloads. The talk will show the deployment and scaling of a high performance computing workload manager, Spark, and OpenStack, and how the VM and Docker management can be integrated together.
Get a Taste of 1 k+ Nodes by a Handful of Servers
This document discusses containerized testing of OpenStack scalability. It explores using Docker containers to provision OpenStack components like Neutron and OpenDaylight in a distributed manner. The approach aims to test scalability up to 1,000 nodes using a handful of physical servers. It analyzes component footprints and tasks. Considerations for extending the tests to larger 1,000 node scales include physical server capacity, container placement density, network bandwidth, and more sophisticated test scenarios.
Linuxcon secureefficientcontainerimagemanagementharbor
This document discusses container image management in enterprises and introduces Project Harbor, an open source container registry. It covers key topics like container image basics, Project Harbor features, maintaining consistency of images between environments, security and access control of images, image distribution strategies, and high availability of container registries. Project Harbor provides an enterprise-grade private registry with features for user management, image replication, security, and integration with systems like LDAP. It aims to help organizations securely manage the distribution and storage of container images.
Unikernelized Linux
Presentation given at the 2017 LinuxCon China
Unikernel is a novel software technology that links an application with OS in the form of a library and packages them into a specialized image that facilitates direct deployment on a hypervisor. Comparing to the traditional VM or the recent containers, Unikernels are smaller, more secure and efficient, making them ideal for cloud environments. There are already lots of open source projects like OSv, Rumprun and so on. But why these existing unikernels have yet to gain large popularity broadly? We think Unikernels are facing three major challenges: 1. Compatibility with existing applications; 2. Lack of production support (e.g. monitoring, debugging, logging); 3. Lack of compelling use case. In this presentation, we will review our investigations and exploration of if-how we can convert Linux as Unikernel to eliminate these significant shortcomings, plus some explorations of coordinating and cooperating with hypervisor.
IP Virtual Server(IPVS) 101
In this slide, we discussed the IPVS, including the introduction, demonstration, implementation, and integration in Kubernetes.
IPVS was based on the netfilter and we discussed how it works with iptables and also compares the detail implementation in Kubernetes to show why IPVS has a better performance in IPTABLES.
Status of Embedded Linux
This document provides a summary of the status of embedded Linux. It discusses recent Linux kernel versions from 4.7 to the upcoming 4.12, highlighting new features. It also covers technology areas like boot time, device tree, graphics, file systems, and security. Several ongoing Linux Foundation projects are mentioned like Long Term Support Initiative, Fuego test framework, and the eLinux wiki. Finally, it lists upcoming conferences and trade associations working on embedded Linux. The document aims to give a quick overview of the current state of embedded Linux topics and projects.
Linux Kernel Development
Presentation delivered at LinuxCon China 2017 by Greg Kroah-Hartman.
The Linux kernel is the largest collaborative software development projects ever. This talk will discuss exactly how Linux is developed, how fast it is happening, who is doing the work, and how we all stay sane keeping up with it. It will discuss the development model used, and how it differs from almost all "traditional" models of software development.
OpenStack on AArch64
Besides huge success in mobile, ARM is also ambitious in server field. Software ecosystem is now a barrier for wide deployment of ARM servers in data center. ARM Shanghai Workloads team is working on clouding and big data software enablement and optimization on ARM64 platform.
In this presentation, Yibo Cai will introduce the status and challenges of running OpenStack on ARM servers, with emphasis on OpenStack compute, storage and networking.
K8s storage-glusterfs-20180210
Kubernetes Storage and GlusterFS
SDN X Cloud Native Taiwan User Group MeetUp #2
Presentation is given on 2018/02/10 at NTUST
Kata Container - The Security of VM and The Speed of Container | Yuntong Jin
Kata Container - The Security of VM and The Speed of Container | Yuntong Jin Vietnam Open Infrastructure User Group
Kata Containers provides container virtualization using lightweight virtual machines to gain the security of virtual machines while maintaining the speed of containers. It uses hypervisor-based isolation to make each container as secure as a virtual machine while integrating seamlessly with container ecosystems. Kata Containers has an open source architecture that supports multiple hypervisors and platforms.Load Balancing 101
Introduce the basic concept of load-balancing, common implementations of load-balancing and the detail fo kubernetes service. In the last, demonstrate how to modify the linux iptable kernel module to fulfill the layer-7 load-balcning for kubernetes
Zephyr: Creating a Best-of-Breed, Secure RTOS for IoT
Presentation delivered at LinuxCon China 2017.
Zephyr is an upstream open source project for places where Linux is too big to fit. This talk will overview the progress we've made in the first year towards the projects goals around incorporating best of breed technologies into the code base, and building up the community to support multiple architectures and development environments. We will share our roadmap, plans and the challenges ahead of the us and give an overview of the major technical challenges we want to tackle in 2017.
Rethinking the OS
Presentation delivered at LinuxCon China 2017. Rethinking the Operating System.
A new wave of Operating Systems optimized for containers appeared on the horizon making us excited and puzzeled at the same time.
"Why do we need anything different for containers when traditional OSs served us well in the last 25+ years?" "Isn't Kubernetes just another package to install on top of my favorite distro?"" Will this obsolete my whole infrastructure?" are some of the questions this talk will shed some light on.
Explore the journey SUSE made in rethinking the OS: From a conservative linux distribution to a platform that goes hand in hand with the needs of Microservices.
You will get an insight at what lessons were learned during the intense development effort that lead to SUSE Containers as a Service Platform, how the obstacles along the way were lifted and why "Upstream first" is - and should always be - the rule.
Building stateful applications on Kubernetes with Rook
Deploying stateful applications such a Wordpress and Jenkins on top of Kubernetes or any other container orchestrator can be a challenging task. In this context, Rook will be used to showcase how to automatically manage the volume's lifecycle through the its Kubernetes operators (operator pattern approach) by leveraging the recently added CSI GA support.
Policy-based Resource Placement
Resource placement is a policy-rich problem, particularly across multi-cluster, multi-geography and multi-cloud environments. Placement may be based on company conventions, external regulation, pricing, performance requirements, or complex combinations of those. Furthermore, placement policies evolve over time and vary across organizations. As a result, it is very difficult to anticipate the policy requirements of all users.
In this presentation, Torin Sandal (Lead Engineer of Open Policy Agent) will present, along with Irfan Ur Rehman, and demonstrate the work they've done integrating OPA into the Kubernetes Cluster Federation Control Plane. This enables high level policies to be expressed in a easy to understand policy language, and automatically enforced across federations of Kubernetes clusters.
See what happened with real time kvm when building real time cloud pezhang@re...
See what happened with real time kvm when building real time cloud pezhang@re...LinuxCon ContainerCon CloudOpen China
Presentation delivered at LinuxCon China 2017
Real-Time is used for deadline-oriented applications and time-sensitive workloads. Real-Time KVM is the extension of KVM(Linux Kernel-based Virtual Machine) to allow the virtual machines(VM) to be a truly Real-Time operating system.Users sometimes need to run low-latency applications(such as audio/video streaming, highly interactive systems, etc) to meet their requirements in clouds. NFV is a new network concept which uses virtualization and software instead of dedicated network appliances. For some use cases of telecommunications, network latency must be within a certain range of values. Real-Time KVM can help NFV meet this requirements.
In this presentation, Pei Zhang will talk about:
(1)Real-Time KVM introduction
(2)Real-Time cloud building
(3)Real-Time KVM in NFV: VM with openvswitch, dpdk and qemu’s vhostuser
(4)Performance testing results show Simplify Networking for Containers
Presentation given at the 2017 LinuxCon China
With the booming of Container technology, it brings obvious advantages for cloud: simple and faster deployment, portability and lightweight cost. But the networking challenges are significant. Users need to restructure their network and support container deployment with current cloud framework, like container and VMs.
In this presentation, we will introduce new container networking solution, which provides one management framework to work with different network componenets through Open/friendly modelling mechnism. iCAN can simplify network deployment and management with most orchestration systems and a variety of data plane components, and design extendsible architect to define and validate Service Level Agreement(SLA) for cloud native applications, which is important factor for enterprise to deliver successful and stable service via containers.
Architecture Overview: Kubernetes with Red Hat Enterprise Linux 7.1
2015/03/12 ver1.0 published.
2015/03/26 ver1.1 fixed: flannel's tunnel type to VXLAN.
2015/04/03 ver1.2 fixed: Flannel's mechanism for VXLAN processing.
Scaling Microservices with Kubernetes
This document provides an overview of using Kubernetes to scale microservices. It discusses the challenges of scaling, monitoring, and discovery for microservices. Kubernetes provides a solution to these challenges through its automation of deployment, scaling, and management of containerized applications. The document then describes Kubernetes architecture and components like the master, nodes, pods, services, deployments and secrets which allow Kubernetes to provide portability, self-healing and a declarative way to manage the desired state of applications.
Rebuild - Simplifying Embedded and IoT Development Using Linux Containers
Rebuild - Simplifying Embedded and IoT Development Using Linux ContainersLinuxCon ContainerCon CloudOpen China
This document introduces Rebuild, a tool that uses Linux containers to provide isolated development environments. It allows developers to easily create, modify, run, and share consistent environments. Rebuild provides a CLI to manage environments locally and publish them to remote registries for sharing. Environments can be created from base images on DockerHub or by importing file systems. Rebuild aims to simplify embedded and IoT development by eliminating issues caused by inconsistent environments.Kubernetes Networking
The document discusses Kubernetes networking. It describes how Kubernetes networking allows pods to have routable IPs and communicate without NAT, unlike Docker networking which uses NAT. It covers how services provide stable virtual IPs to access pods, and how kube-proxy implements services by configuring iptables on nodes. It also discusses the DNS integration using SkyDNS and Ingress for layer 7 routing of HTTP traffic. Finally, it briefly mentions network plugins and how Kubernetes is designed to be open and customizable.
Vault - Enhancement for K8S secret security
The document discusses Vault, a tool for securely managing secrets and sensitive data. It provides centralization of secrets, encryption of data at rest, and auditing of access to credentials. Vault controls access to secrets by authenticating users and services against identity sources and enforcing policies. The document provides guidance on using Vault for development, including mounting volumes, using dedicated service accounts, and adding configuration parameters to access the Vault server.
HA in OpenStack service - meetup #9
The document discusses making OpenStack controller core services highly available. It describes using Pacemaker and Corosync to manage virtual IP addresses and services across multiple nodes. HAProxy is used as a load balancer between the virtual IPs and service instances. The database uses Galera cluster for multi-master replication. RabbitMQ and Memcached are made highly available through clustering as well. Failure scenarios are tested by stopping nodes and services.
Managing Redis with Kubernetes - Kelsey Hightower, Google
Kubernetes can be used to manage Redis clusters and deployments. It schedules Redis instances across nodes for high availability and efficient use of resources. Pods contain Redis containers and shared storage. ReplicaSets ensure the desired number of pods are running. Services provide a stable endpoint for pods. Kubernetes drives the current state towards the desired configuration.
Kubernetes Ingress 101
This presentation explains the basics of Kubernetes ingress traffic management functionality, and how it can be used to simplify managing applications across different environments - in the cloud or on premise.
GPU Acceleration for Containers on Intel Processor Graphics
Presentation delivered at the 2017 LinuxCon China.
Container is a popular technology in cloud with the merits of fast provisioning, high density and near-native performance. New requirements have been rising to further use GPU to accelerate various applications in containers, e.g. media trans-coding, machine learning, etc. However there is still gap of managing GPU in such container usages. This presentation will first review the status of GPU support on both native container and Intel clear container, then provide an anatomy of technical gaps and enabling plans in major areas (orchestration layer, resource isolation and QoS performance isolation), then review our work for GPU virtualization in clear container and prototype work through extensions in docker plugin, cgroup and GPU scheduler to fix those gaps.
Libvirt API Certification
This document discusses the architecture and evolution of Libvirt, an open source software library that provides a standardized interface to various virtualization technologies like Xen, QEMU, KVM, LXC, and others. It outlines Libvirt's public API, driver API, and driver implementations that allow applications to manage virtual machines across different platforms. The document also mentions Libvirt's certification process for hardware and software compatibility and its automated testing framework.
More Related Content
What's hot
Status of Embedded Linux
This document provides a summary of the status of embedded Linux. It discusses recent Linux kernel versions from 4.7 to the upcoming 4.12, highlighting new features. It also covers technology areas like boot time, device tree, graphics, file systems, and security. Several ongoing Linux Foundation projects are mentioned like Long Term Support Initiative, Fuego test framework, and the eLinux wiki. Finally, it lists upcoming conferences and trade associations working on embedded Linux. The document aims to give a quick overview of the current state of embedded Linux topics and projects.
Linux Kernel Development
Presentation delivered at LinuxCon China 2017 by Greg Kroah-Hartman.
The Linux kernel is the largest collaborative software development projects ever. This talk will discuss exactly how Linux is developed, how fast it is happening, who is doing the work, and how we all stay sane keeping up with it. It will discuss the development model used, and how it differs from almost all "traditional" models of software development.
OpenStack on AArch64
Besides huge success in mobile, ARM is also ambitious in server field. Software ecosystem is now a barrier for wide deployment of ARM servers in data center. ARM Shanghai Workloads team is working on clouding and big data software enablement and optimization on ARM64 platform.
In this presentation, Yibo Cai will introduce the status and challenges of running OpenStack on ARM servers, with emphasis on OpenStack compute, storage and networking.
K8s storage-glusterfs-20180210
Kubernetes Storage and GlusterFS
SDN X Cloud Native Taiwan User Group MeetUp #2
Presentation is given on 2018/02/10 at NTUST
Kata Container - The Security of VM and The Speed of Container | Yuntong Jin
Kata Container - The Security of VM and The Speed of Container | Yuntong Jin Vietnam Open Infrastructure User Group
Kata Containers provides container virtualization using lightweight virtual machines to gain the security of virtual machines while maintaining the speed of containers. It uses hypervisor-based isolation to make each container as secure as a virtual machine while integrating seamlessly with container ecosystems. Kata Containers has an open source architecture that supports multiple hypervisors and platforms.Load Balancing 101
Introduce the basic concept of load-balancing, common implementations of load-balancing and the detail fo kubernetes service. In the last, demonstrate how to modify the linux iptable kernel module to fulfill the layer-7 load-balcning for kubernetes
Zephyr: Creating a Best-of-Breed, Secure RTOS for IoT
Presentation delivered at LinuxCon China 2017.
Zephyr is an upstream open source project for places where Linux is too big to fit. This talk will overview the progress we've made in the first year towards the projects goals around incorporating best of breed technologies into the code base, and building up the community to support multiple architectures and development environments. We will share our roadmap, plans and the challenges ahead of the us and give an overview of the major technical challenges we want to tackle in 2017.
Rethinking the OS
Presentation delivered at LinuxCon China 2017. Rethinking the Operating System.
A new wave of Operating Systems optimized for containers appeared on the horizon making us excited and puzzeled at the same time.
"Why do we need anything different for containers when traditional OSs served us well in the last 25+ years?" "Isn't Kubernetes just another package to install on top of my favorite distro?"" Will this obsolete my whole infrastructure?" are some of the questions this talk will shed some light on.
Explore the journey SUSE made in rethinking the OS: From a conservative linux distribution to a platform that goes hand in hand with the needs of Microservices.
You will get an insight at what lessons were learned during the intense development effort that lead to SUSE Containers as a Service Platform, how the obstacles along the way were lifted and why "Upstream first" is - and should always be - the rule.
Building stateful applications on Kubernetes with Rook
Deploying stateful applications such a Wordpress and Jenkins on top of Kubernetes or any other container orchestrator can be a challenging task. In this context, Rook will be used to showcase how to automatically manage the volume's lifecycle through the its Kubernetes operators (operator pattern approach) by leveraging the recently added CSI GA support.
Policy-based Resource Placement
Resource placement is a policy-rich problem, particularly across multi-cluster, multi-geography and multi-cloud environments. Placement may be based on company conventions, external regulation, pricing, performance requirements, or complex combinations of those. Furthermore, placement policies evolve over time and vary across organizations. As a result, it is very difficult to anticipate the policy requirements of all users.
In this presentation, Torin Sandal (Lead Engineer of Open Policy Agent) will present, along with Irfan Ur Rehman, and demonstrate the work they've done integrating OPA into the Kubernetes Cluster Federation Control Plane. This enables high level policies to be expressed in a easy to understand policy language, and automatically enforced across federations of Kubernetes clusters.
See what happened with real time kvm when building real time cloud pezhang@re...
See what happened with real time kvm when building real time cloud pezhang@re...LinuxCon ContainerCon CloudOpen China
Presentation delivered at LinuxCon China 2017
Real-Time is used for deadline-oriented applications and time-sensitive workloads. Real-Time KVM is the extension of KVM(Linux Kernel-based Virtual Machine) to allow the virtual machines(VM) to be a truly Real-Time operating system.Users sometimes need to run low-latency applications(such as audio/video streaming, highly interactive systems, etc) to meet their requirements in clouds. NFV is a new network concept which uses virtualization and software instead of dedicated network appliances. For some use cases of telecommunications, network latency must be within a certain range of values. Real-Time KVM can help NFV meet this requirements.
In this presentation, Pei Zhang will talk about:
(1)Real-Time KVM introduction
(2)Real-Time cloud building
(3)Real-Time KVM in NFV: VM with openvswitch, dpdk and qemu’s vhostuser
(4)Performance testing results show Simplify Networking for Containers
Presentation given at the 2017 LinuxCon China
With the booming of Container technology, it brings obvious advantages for cloud: simple and faster deployment, portability and lightweight cost. But the networking challenges are significant. Users need to restructure their network and support container deployment with current cloud framework, like container and VMs.
In this presentation, we will introduce new container networking solution, which provides one management framework to work with different network componenets through Open/friendly modelling mechnism. iCAN can simplify network deployment and management with most orchestration systems and a variety of data plane components, and design extendsible architect to define and validate Service Level Agreement(SLA) for cloud native applications, which is important factor for enterprise to deliver successful and stable service via containers.
Architecture Overview: Kubernetes with Red Hat Enterprise Linux 7.1
2015/03/12 ver1.0 published.
2015/03/26 ver1.1 fixed: flannel's tunnel type to VXLAN.
2015/04/03 ver1.2 fixed: Flannel's mechanism for VXLAN processing.
Scaling Microservices with Kubernetes
This document provides an overview of using Kubernetes to scale microservices. It discusses the challenges of scaling, monitoring, and discovery for microservices. Kubernetes provides a solution to these challenges through its automation of deployment, scaling, and management of containerized applications. The document then describes Kubernetes architecture and components like the master, nodes, pods, services, deployments and secrets which allow Kubernetes to provide portability, self-healing and a declarative way to manage the desired state of applications.
Rebuild - Simplifying Embedded and IoT Development Using Linux Containers
Rebuild - Simplifying Embedded and IoT Development Using Linux ContainersLinuxCon ContainerCon CloudOpen China
This document introduces Rebuild, a tool that uses Linux containers to provide isolated development environments. It allows developers to easily create, modify, run, and share consistent environments. Rebuild provides a CLI to manage environments locally and publish them to remote registries for sharing. Environments can be created from base images on DockerHub or by importing file systems. Rebuild aims to simplify embedded and IoT development by eliminating issues caused by inconsistent environments.Kubernetes Networking
The document discusses Kubernetes networking. It describes how Kubernetes networking allows pods to have routable IPs and communicate without NAT, unlike Docker networking which uses NAT. It covers how services provide stable virtual IPs to access pods, and how kube-proxy implements services by configuring iptables on nodes. It also discusses the DNS integration using SkyDNS and Ingress for layer 7 routing of HTTP traffic. Finally, it briefly mentions network plugins and how Kubernetes is designed to be open and customizable.
Vault - Enhancement for K8S secret security
The document discusses Vault, a tool for securely managing secrets and sensitive data. It provides centralization of secrets, encryption of data at rest, and auditing of access to credentials. Vault controls access to secrets by authenticating users and services against identity sources and enforcing policies. The document provides guidance on using Vault for development, including mounting volumes, using dedicated service accounts, and adding configuration parameters to access the Vault server.
HA in OpenStack service - meetup #9
The document discusses making OpenStack controller core services highly available. It describes using Pacemaker and Corosync to manage virtual IP addresses and services across multiple nodes. HAProxy is used as a load balancer between the virtual IPs and service instances. The database uses Galera cluster for multi-master replication. RabbitMQ and Memcached are made highly available through clustering as well. Failure scenarios are tested by stopping nodes and services.
Managing Redis with Kubernetes - Kelsey Hightower, Google
Kubernetes can be used to manage Redis clusters and deployments. It schedules Redis instances across nodes for high availability and efficient use of resources. Pods contain Redis containers and shared storage. ReplicaSets ensure the desired number of pods are running. Services provide a stable endpoint for pods. Kubernetes drives the current state towards the desired configuration.
Kubernetes Ingress 101
This presentation explains the basics of Kubernetes ingress traffic management functionality, and how it can be used to simplify managing applications across different environments - in the cloud or on premise.
What's hot (20)
Kata Container - The Security of VM and The Speed of Container | Yuntong Jin
Kata Container - The Security of VM and The Speed of Container | Yuntong Jin
Zephyr: Creating a Best-of-Breed, Secure RTOS for IoT
Zephyr: Creating a Best-of-Breed, Secure RTOS for IoT
Building stateful applications on Kubernetes with Rook
Building stateful applications on Kubernetes with Rook
See what happened with real time kvm when building real time cloud pezhang@re...
See what happened with real time kvm when building real time cloud pezhang@re...
Architecture Overview: Kubernetes with Red Hat Enterprise Linux 7.1
Architecture Overview: Kubernetes with Red Hat Enterprise Linux 7.1
Rebuild - Simplifying Embedded and IoT Development Using Linux Containers
Rebuild - Simplifying Embedded and IoT Development Using Linux Containers
Managing Redis with Kubernetes - Kelsey Hightower, Google
Managing Redis with Kubernetes - Kelsey Hightower, Google
Viewers also liked
GPU Acceleration for Containers on Intel Processor Graphics
Presentation delivered at the 2017 LinuxCon China.
Container is a popular technology in cloud with the merits of fast provisioning, high density and near-native performance. New requirements have been rising to further use GPU to accelerate various applications in containers, e.g. media trans-coding, machine learning, etc. However there is still gap of managing GPU in such container usages. This presentation will first review the status of GPU support on both native container and Intel clear container, then provide an anatomy of technical gaps and enabling plans in major areas (orchestration layer, resource isolation and QoS performance isolation), then review our work for GPU virtualization in clear container and prototype work through extensions in docker plugin, cgroup and GPU scheduler to fix those gaps.
Libvirt API Certification
This document discusses the architecture and evolution of Libvirt, an open source software library that provides a standardized interface to various virtualization technologies like Xen, QEMU, KVM, LXC, and others. It outlines Libvirt's public API, driver API, and driver implementations that allow applications to manage virtual machines across different platforms. The document also mentions Libvirt's certification process for hardware and software compatibility and its automated testing framework.
Building a Better Thermostat
This document outlines Matthew Treinish's project to build a better thermostat using open source tools. It describes how he used a Z-Wave power switch and wireless temperature sensor to control the air conditioner and measure temperature. He implemented this using the Home Assistant home automation platform, setting up a generic thermostat device. Issues with reliability and manual configuration led to improvements such as automating temperature set points based on location tracking and scheduled times. Future work includes adding more sensors and automation functionality.
kdump: usage and_internals
Kdump is a long existing method for acquiring dump of crashed kernel, however very few literatures are available to understand it's usage and internals. We receive a lot of queries on kexec mailing list about different issues related to the kexec/kdump environment.
In this presentation, we talk about basics of kdump usage and some internals about kdump/kexec kernel implementation. It includes end to end flow from kdump kernel configuration to crash analysis. We discuss some of the problem which is frequently faced by kdump users. It also includes related information about ELF structure, so that one can debug if vmcore itself gets corrupted because of any architecture related issue.
Practical CNI
The document provides an overview of Container Network Interface (CNI) which is a standard for configuring network interfaces for Linux containers. It discusses how CNI uses plugins to configure networking for containers and provides examples of plugins like ptp, bridge, and IPvlan. It also covers overlay networking techniques like VXLAN and OVN which can be used to provide networking between containers on different hosts.
High Performance Linux Virtual Machine on Microsoft Azure: SR-IOV Networking ...
High Performance Linux Virtual Machine on Microsoft Azure: SR-IOV Networking ...LinuxCon ContainerCon CloudOpen China
Can we leverage the resource of public cloud for gaming, streaming, transcoding, machine learning and visualized CAD application on demand? Yes if it provides the capability and infrastructure to utilize GPUs. Can we get high performance networking in the cloud as what I have in the bare metal environment? Yes with SR-IOV. How to achieve them? In this presentation we describe Discrete Device Assignment (also known as PCI Pass-through) support for GPU and network adapter in Linux guest and SR-IOV architectures of Linux guest with near-native performance profile running on Hyper-V. We also will share how to integrate accelerated graphics and networking capabilities in Microsoft Azure infrastructure. Releasing a Distribution in the Age of DevOps.
Presentation delivered at LinuxCon China 2017.
Operating systems need to move faster without sacrificing stability. New hardware, new software features, and bugfixes are making it into distribution components every day. To maintain stability, packagers and distribution developers are looking toward lessons learned in the DevOps movement to implement Continuous Integration/Continuous Delivery (CI/CD) workflows that provide quicker test feedback to developers.
This talk highlights some of the coming trends in Fedora such as: streamlined base package sets, userspace applications delivered as containers, continuous validation of individual distro components and the distro as a whole, and collaboration with the CentOS Project.
点融网区块链即服务实践 - The Practice of Blockchain as a Service in Dianrong
点融网区块链即服务实践 - The Practice of Blockchain as a Service in DianrongLinuxCon ContainerCon CloudOpen China
Presentation delivered at LinuxCon China 2017
The practices of Blockchain as a service in Dianrong (Shiyuan Xiao, Dianrong.com) - Blockchain as a Service (BaaS) provides a easy, low-cost and flexible platform for companies to enable their businesses based on blockchain backed by a cloud platform. Shiyuan will introduce the experiences to build such a BaaS platform, what is the architecture, what problems we have met and solved and the best practices we summarized. OCI Support in Mesos
The document discusses support for Open Container Initiative (OCI) images in Mesos. It provides an overview of OCI specifications including the image format specification. It describes how Mesos stores and retrieves OCI images from a default location based on the image layout. It also discusses how the existing Docker isolator was extended to support running containers from both Docker images and OCI images.
Hyperledger Technical Community in China.
The Hyperledger Technical Community in China was founded in December 2016 and has over 10 volunteers. Its goals are to help build the China technical community, promote Hyperledger technical development, and focus on technical development, documentation and education, adoption and collaboration, and event organization. The community believes in open-source technologies and values trust, collaboration, mutual benefit, and openness.
LiteOS
The document discusses Internet of Things (IoT) technologies and applications. It describes how Huawei's LiteOS operating system enables device intelligence and easy connectivity between devices and IoT platforms. LiteOS supports a variety of microcontroller boards and has partnered with STMicroelectronics to launch NB-IoT development kits. The document encourages developers to get involved with the LiteOS open source community through its website, GitHub repository, workshops, and other resources.
Obstacles & Solutions for Livepatch Support on ARM64 Architecture
Obstacles & Solutions for Livepatch Support on ARM64 ArchitectureLinuxCon ContainerCon CloudOpen China
This document discusses obstacles and solutions for supporting live kernel patching (livepatching) on ARM64 architecture. Livepatching allows patching a running kernel without reboots. It is currently only supported on x86, s390 and powerpc. For ARM64, the main obstacle is that the ARM64 calling convention stores return addresses in a link register rather than the stack. The document proposes a solution that extends the profile instrumentation to save and restore the link register, allowing ftrace to be adapted for ARM64 livepatching. It provides an overview of upstream work on gcc and kernel support to enable livepatching for ARM64.OpenDaylight OpenStack Integration
Networking-odl and ODL Neutron Northbound are the key components for integrating OpenStack Neutron and OpenDaylight. They are actively developed open source projects. The document encourages giving the integration a try, providing feedback, and contributing to help further the integration of OpenStack and OpenDaylight networking.
Open Source Software Business Models Redux
The document discusses open source software business models and community practices. It describes how software sharing has occurred since the 1940s through organizations like IBM SHARE and MIT's Project Athena. It notes that writing good software is hard work and companies shared software before the Open Source Definition. The document also discusses how open source projects and communities evolve from individual committers to broader ecosystems including users, developers, products, and services. It emphasizes that open source is about engineering economics and gives Red Hat as an example of a company focused on customer success through open source. Finally, it outlines various community practices that can help projects and software mature.
UEFI HTTP/HTTPS Boot
Presentation delivered at LinuxCon China 2016
UEFI HTTP/HTTPS Boot is a new feature of UEFI 2.5+. In the meantime, this feature is not yet implemented in any Linux bootloader. This Birds of a Feather session will give an introduction to UEFI HTTP/HTTPS Boot, and share a proof-of-concept implementation based on grub2 that works on both the emulator (QEMU/OVMF) and HPE ProLiant Gen10 servers.
For HTTPS, the experience and comparison will be shared between the purely software-based and UEFI-based implementations in the aspects of ease of implementation, security strength, and limitation.
From Resilient to Antifragile Chaos Engineering Primer
Failure injection is somewhat analogous to a vaccine. We want to inject these bad behaviours so our developers can build immunities to them. Can we inject failure scenarios into deployed systems to reduce platform risk Demonstrations of the Simian Army, Chaos Lemur and Locust.io tools will be presented.
Even when all of the individual services in a distributed system are functioning properly, the interactions between those services can cause unpredictable outcomes. Unpredictable outcomes, compounded by rare but disruptive real-world events that affect production environments, make these distributed systems inherently chaotic.
Make Accelerator Pluggable for Container Engine
This document proposes making hardware accelerators like GPUs and FPGAs pluggable for container engines. It discusses identifying accelerator requirements, preparing runtimes, and managing resources. A pluggable accelerator interface is suggested to integrate different plugins for devices like GPUs, DPDK, and FPGAs in a standardized way and manage their lifecycles together with containers. This would allow containers to easily leverage various hardware accelerators.
Introduction to OCI Image Technologies Serving Container
The document introduces OCI image technologies, including the OCI image specification and image tools. It describes the key components of an OCI image like the image manifest, layers, and configuration. It also outlines the features and workflow of image tools, such as creating, unpacking, and validating OCI images. Finally, it discusses concepts like distribution, signatures, and the relationship between image tools and OCI consumers.
Viewers also liked (18)
GPU Acceleration for Containers on Intel Processor Graphics
GPU Acceleration for Containers on Intel Processor Graphics
High Performance Linux Virtual Machine on Microsoft Azure: SR-IOV Networking ...
High Performance Linux Virtual Machine on Microsoft Azure: SR-IOV Networking ...
点融网区块链即服务实践 - The Practice of Blockchain as a Service in Dianrong
点融网区块链即服务实践 - The Practice of Blockchain as a Service in Dianrong
Obstacles & Solutions for Livepatch Support on ARM64 Architecture
Obstacles & Solutions for Livepatch Support on ARM64 Architecture
From Resilient to Antifragile Chaos Engineering Primer
From Resilient to Antifragile Chaos Engineering Primer
Introduction to OCI Image Technologies Serving Container
Introduction to OCI Image Technologies Serving Container
Similar to Is there still room for innovation in container orchestration and scheduling
Introducing Project Longhorn - April 2016 Rancher Online Meetup
In April, Rancher Labs introduced Project Longhorn, a new way to deliver persistent storage for cloud- and container-based deployments. In this online meetup, Rancher labs founders Sheng Liang, Darren Shepherd, and Shannon Williams to introduce and discuss Project Longhorn, a microservices-based solution for distributed, resilient block storage for containers.
To try out Longhorn visit http://github.com/rancher/longhorn
Rancher presentation august 2017
Rancher provides a complete container management platform that simplifies deploying and managing containers in production. It offers robust container orchestration with Kubernetes, allowing deployment of container workloads within five minutes. Rancher provides a centralized interface for managing the entire container stack, including orchestration, security, networking, storage and monitoring tools. It has gained popularity with over 30 million downloads and support for over 100 enterprise customers.
More tips and tricks for running containers like a pro - Rancher Online MEetu...
This document outlines the agenda for a Rancher meetup on tips and tricks for running containers like a pro. The agenda includes presentations on integrated secrets management, autoscaling with Rancher webhooks, using Traefik for load balancing, and the Kubernetes dashboard and Helm. It also provides information on the latest Rancher releases.
Storage for containers and cloud-native deployments - Rancher Online Meetup -...
In our March 2017 Rancher Online Meetup we dove into running persistent storage for containers in Rancher. The presentation included an overview of persistent storage in Rancher, an introduction to Quobyte and an overview of Portworx.
RancherOS July 2017
RancherOS is a lightweight Linux distribution built from containers. It is designed to be simple, easy to use, and customizable. It uses cloud-config files and Docker compose files for primary configuration and management of system services and custom applications. The ros CLI provides commands for managing services, the console, Docker engine, OS upgrades, and configuration.
Deploying and scaling kubernetes using Rancher CNCF meetup June 2017
Everyone knows Kubernetes but not everyone knows how to deploy and scale it easily. How does Rancher help in deploying Kubernetes, and what value does Rancher add to those deployments? Find out how to make your Kubernetes deployment and scaling easy by using Rancher.
Kubernetes One-Click Deployment: Hands-on Workshop (Munich)
Cloud Native Night, January 2018, Munich: Workshop led by Jörg Schad (@joerg_schad, Technical Lead Community Projects at Mesosphere)
Join our Meetup: https://www.meetup.com/de-DE/cloud-native-muc
PLEASE NOTE: During this workshop, Jörg showed many demos and the audience could participate on their laptops. Unfortunately, we can't provide these demos. Nevertheless, Jörg's slides give a deep dive into the topic.
ABSTRACT: Kubernetes has been one of the topics in 2017 and will probably remain so in 2018. In this hands-on technical workshop you will learn how best to deploy, operate, and scale Kubernetes clusters from one to hundreds of nodes using DC/OS. You will learn how to integrate and run Kubernetes alongside traditional applications and fast data services of your choice (e.g. Apache Cassandra, Apache Kafka, Apache Spark, TensorFlow, and more) on any infrastructure.
This workshop best suits operators focussed on keeping their apps and services up and running in production and developers focussed on quickly delivering internal and customer facing apps into production.
You will learn how to:
- Introduction to Kubernetes and DC/OS (including the differences between both)
- Deploy Kubernetes on DC/OS in a secure, highly available, and fault-tolerant manner
- Solve operational challenges of running a large/multiple Kubernetes cluster
- One-click deploy big data stateful and stateless services alongside a Kubernetes cluster
Supercharging CI/CD with GitLab and Rancher - June 2017 Online Meetup
In our June Rancher meetup, GitLab joined us to discuss how to use GitLab tools and the Rancher container management platform for an automated, scalable CI pipeline. We were also joined by the devops team at Kloeckner.i who demonstrated their real-world example of the two together.
Operating Kubernetes at Scale (Australia Presentation)
Kubernetes is an amazing technology, but getting it up and running in your data center or VMs is challenging. In this technical webinar, you will learn how best to deploy, operate, and scale Kubernetes clusters from one to hundreds of nodes using DC/OS.
Jörg Schad and Adrian Smolski from Mesosphere show how to run Kubernetes on DC/OS, as well as how to integrate and run Kubernetes alongside traditional applications and fast data services of your choice (e.g. Apache Cassandra, Apache Kafka, Apache Spark, TensorFlow, and more) on any infrastructure.
You will learn how to:
1. Deploy Kubernetes in a secure, highly available, and fault-tolerant manner on DC/OS
2. Solve operational challenges of running a large/multiple Kubernetes cluster(s)
3. One-click deploy big data stateful and stateless services alongside a Kubernetes cluster
Jörg is a Technical Lead for Community Projects at Mesosphere in San Francisco. His speaking experience includes various Meetups, international conferences, and lecture halls.
Adrian Smolski is the local Field CTO based out of Sydney, Australia. His background is big data, data science and distributed systems.
Webinar slides: How to automate and manage MongoDB & Percona Server for MongoDB
Join us as the Severalnines & Percona teams join forces for a different perspective on how to automate and manage your MongoDB or Percona Server for MongoDB databases. During this webinar, we’ll walk you through the key features of the Percona Server for MongoDB as compared to MongoDB itself as well as how to manage both “flavours” of MongoDB.
There are many generic tools available, both commercial and open source, to aid with the automation of operational tasks. Some of these tools are even deployed in the database world.
However, there are a small number of specialist domain-specific automation tools available, and we are going to compare the MongoDB-relevant functionality of two of these products: MongoDB’s Ops Manager, and ClusterControl from Severalnines. Attendees should take away a clear understanding of the differences between these tools, and how they help automate and manage MongoDB operations.
AGENDA
- Introduction to Percona Server for MongoDB
- How to automate and manage MongoDB
- Installation and maintenance
- Complexity of architecture
- Options for redundancy
- Comparative functionality
- Monitoring, Dashboard, Alerting
- Backing up your deployments
- Automated deployment of advanced configurations
- Upgrading existing deployments
SPEAKERS
Ruairí Newman is passionate about all things cloud and automation and has worked for MongoDB, VMware and Amazon Web Services among others. He has a background in Operational Support Systems and Professional Services.
Prior to joining Severalnines, Ruairí worked for Huawei Ireland as Senior Cloud Solutions Architect on their Web Services project, where he advised on commodity cloud architecture and Monitoring technologies, and deployed and administered a Research & Development Openstack lab.
Prior to joining Percona as a Product Manager, Tyler Duzan spent almost 13 years as an operations and security engineer in a variety of different industries. Deciding to take his analytical mindset and strategic focus into new territory, Tyler is applying his knowledge to solving business problems for Percona customers with inventive solutions combining technology and services.
Graph Day 2017 Spring Boot
A slide deck on an example implementation of the technology we used at ten-x to enable business users to find data in a graph.
Kubernetes basics and hands on exercise
This document provides an overview of Kubernetes concepts including architecture, fundamental objects like pods and services, and demonstrations. It begins with an agenda then covers Kubernetes architecture including the master node, worker nodes, and control loop. It describes core objects like pods, replica sets, deployments, services, and labels/selectors. The document demonstrates deploying and accessing the guestbook application using these objects. It concludes with asking for questions and describing goals for educational meetups on cloud native technologies.
Introduction to Kubernetes - Docker Global Mentor Week 2016
Kubernetes is an open-source system for automating
deployment, scaling, and management of containerized
applications. This presentation will show you overview of Kubernetes concept.
Docker Global Mentor Week 2016 #DockerInThai at Kaidee on November 18, 2016
EDB Postgres with Containers
The use of containers to simplify and speed the deployment and development of applications is taking off. Most container usage is around stateless micro-services, but data and transactions are key components of most applications.
This presentation reviews:
- The purpose of containers and their usage
- How to containerize your EDB Postgres deployment
- How to deal with issues of managing your database and storage
- How to set up a cluster for high availability
- How to build a container with the EDB Postgres Enterprise Manager Agent in the container
Target Audience:
This technical presentation is for DBAs, Data Architects, Developers, DevOps, IT Operations and anyone responsible for supporting a Postgres interested in learning about Containers. It is equally suitable for organizations using community PostgreSQL as well as EDB’s Postgres Plus product family.
To listen to the recording which includes a demonstration, visit EnterpriseDB > Resources > Webcasts
Lunar Way and the Cloud Native "stack"
This talk was given at the Cloud Native Aarhus meetup at May 30rd 2017. Focus is on how we use Cloud Native projects at Lunar Way.
Rancher 2.0 Technical Deep Dive
西脇 雄基（LINE)/Rancher 2.0 Technical Deep Dive
2018/7/28 LINE Developer Meetup in Tokyo #40 -Kubernetes-
https://line.connpass.com/event/92049/
Episode 2: Deploying Kubernetes at Scale
Learn about the challenges the come with deploying and operating Kubernetes at scale and how the Mesosphere DC/OS Kubernetes integration helps solve them.
During this presentation, Joerg Schad discusses:
1. Common challenges associated with getting a Kubernetes cluster up and running
2. The basics of running Kubernetes on Mesosphere DC/OS
3. How failure recovery works with the DC/OS-Kubernetes solution
Using MySQL Containers
This document discusses using MySQL containers and provides an overview of containers and Docker. It begins with definitions of containers and how they isolate applications. It then discusses Docker and how it packages and runs applications in containers. The rest of the document demonstrates how to use official MySQL containers, including running MySQL in containers and using Docker Compose to manage multi-container applications. It concludes by discussing container orchestration with Docker Swarm and Kubernetes.
Docker vs. Kubernetes vs. Serverless
Speakers: Phil Christensen, Sr. DevOps Engineer, Solutions Architect
Phil is an open source software engineer and cloud solutions architect with 15+ years of experience. He focuses on Puppet automation, containers, and Amazon Web Services infrastructure and holds all five AWS Certifications.
If you are interested in learning more about Logicworks' unique approach to cloud migration and management, contact us or talk to a cloud expert at (212) 625-5300.
Austin - Container Days - Docker 101
1) The document discusses Docker containers and provides an introduction to Docker concepts like building images, running containers, storage, networking and linking containers.
2) It includes an agenda for the Docker 101 session which covers container basics, building images, storage, networking and a demo of images, building and networking modes.
3) Examples and exercises are provided around building an image, pushing to a registry, running containers with volumes, linking and networking.
Similar to Is there still room for innovation in container orchestration and scheduling (20)
Introducing Project Longhorn - April 2016 Rancher Online Meetup
Introducing Project Longhorn - April 2016 Rancher Online Meetup
More tips and tricks for running containers like a pro - Rancher Online MEetu...
More tips and tricks for running containers like a pro - Rancher Online MEetu...
Storage for containers and cloud-native deployments - Rancher Online Meetup -...
Storage for containers and cloud-native deployments - Rancher Online Meetup -...
Deploying and scaling kubernetes using Rancher CNCF meetup June 2017
Deploying and scaling kubernetes using Rancher CNCF meetup June 2017
Kubernetes One-Click Deployment: Hands-on Workshop (Munich)
Kubernetes One-Click Deployment: Hands-on Workshop (Munich)
Supercharging CI/CD with GitLab and Rancher - June 2017 Online Meetup
Supercharging CI/CD with GitLab and Rancher - June 2017 Online Meetup
Operating Kubernetes at Scale (Australia Presentation)
Operating Kubernetes at Scale (Australia Presentation)
Webinar slides: How to automate and manage MongoDB & Percona Server for MongoDB
Webinar slides: How to automate and manage MongoDB & Percona Server for MongoDB
Introduction to Kubernetes - Docker Global Mentor Week 2016
Introduction to Kubernetes - Docker Global Mentor Week 2016
Recently uploaded
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift
Overview
Dive into the world of anomaly detection on edge devices with our comprehensive hands-on tutorial. This SlideShare presentation will guide you through the entire process, from data collection and model training to edge deployment and real-time monitoring. Perfect for those looking to implement robust anomaly detection systems on resource-constrained IoT/edge devices.
Key Topics Covered
1. Introduction to Anomaly Detection
- Understand the fundamentals of anomaly detection and its importance in identifying unusual behavior or failures in systems.
2. Understanding Edge (IoT)
- Learn about edge computing and IoT, and how they enable real-time data processing and decision-making at the source.
3. What is ArgoCD?
- Discover ArgoCD, a declarative, GitOps continuous delivery tool for Kubernetes, and its role in deploying applications on edge devices.
4. Deployment Using ArgoCD for Edge Devices
- Step-by-step guide on deploying anomaly detection models on edge devices using ArgoCD.
5. Introduction to Apache Kafka and S3
- Explore Apache Kafka for real-time data streaming and Amazon S3 for scalable storage solutions.
6. Viewing Kafka Messages in the Data Lake
- Learn how to view and analyze Kafka messages stored in a data lake for better insights.
7. What is Prometheus?
- Get to know Prometheus, an open-source monitoring and alerting toolkit, and its application in monitoring edge devices.
8. Monitoring Application Metrics with Prometheus
- Detailed instructions on setting up Prometheus to monitor the performance and health of your anomaly detection system.
9. What is Camel K?
- Introduction to Camel K, a lightweight integration framework built on Apache Camel, designed for Kubernetes.
10. Configuring Camel K Integrations for Data Pipelines
- Learn how to configure Camel K for seamless data pipeline integrations in your anomaly detection workflow.
11. What is a Jupyter Notebook?
- Overview of Jupyter Notebooks, an open-source web application for creating and sharing documents with live code, equations, visualizations, and narrative text.
12. Jupyter Notebooks with Code Examples
- Hands-on examples and code snippets in Jupyter Notebooks to help you implement and test anomaly detection models.
Uni Systems Copilot event_05062024_C.Vlachos.pdf
Unlocking Productivity: Leveraging the Potential of Copilot in Microsoft 365, a presentation by Christoforos Vlachos, Senior Solutions Manager – Modern Workplace, Uni Systems
HCL Notes and Domino License Cost Reduction in the World of DLAU
Webinar Recording: https://www.panagenda.com/webinars/hcl-notes-and-domino-license-cost-reduction-in-the-world-of-dlau/
The introduction of DLAU and the CCB & CCX licensing model caused quite a stir in the HCL community. As a Notes and Domino customer, you may have faced challenges with unexpected user counts and license costs. You probably have questions on how this new licensing approach works and how to benefit from it. Most importantly, you likely have budget constraints and want to save money where possible. Don’t worry, we can help with all of this!
We’ll show you how to fix common misconfigurations that cause higher-than-expected user counts, and how to identify accounts which you can deactivate to save money. There are also frequent patterns that can cause unnecessary cost, like using a person document instead of a mail-in for shared mailboxes. We’ll provide examples and solutions for those as well. And naturally we’ll explain the new licensing model.
Join HCL Ambassador Marc Thomas in this webinar with a special guest appearance from Franz Walder. It will give you the tools and know-how to stay on top of what is going on with Domino licensing. You will be able lower your cost through an optimized configuration and keep it low going forward.
These topics will be covered
- Reducing license cost by finding and fixing misconfigurations and superfluous accounts
- How do CCB and CCX licenses really work?
- Understanding the DLAU tool and how to best utilize it
- Tips for common problem areas, like team mailboxes, functional/test users, etc
- Practical examples and best practices to implement right away
Fueling AI with Great Data with Airbyte Webinar
This talk will focus on how to collect data from a variety of sources, leveraging this data for RAG and other GenAI use cases, and finally charting your course to productionalization.
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Discover how MongoDB Atlas and vector search technology can revolutionize your application's search capabilities. This comprehensive presentation covers:
* What is Vector Search?
* Importance and benefits of vector search
* Practical use cases across various industries
* Step-by-step implementation guide
* Live demos with code snippets
* Enhancing LLM capabilities with vector search
* Best practices and optimization strategies
Perfect for developers, AI enthusiasts, and tech leaders. Learn how to leverage MongoDB Atlas to deliver highly relevant, context-aware search results, transforming your data retrieval process. Stay ahead in tech innovation and maximize the potential of your applications.
#MongoDB #VectorSearch #AI #SemanticSearch #TechInnovation #DataScience #LLM #MachineLearning #SearchTechnology
Infrastructure Challenges in Scaling RAG with Custom AI models
Building Retrieval-Augmented Generation (RAG) systems with open-source and custom AI models is a complex task. This talk explores the challenges in productionizing RAG systems, including retrieval performance, response synthesis, and evaluation. We’ll discuss how to leverage open-source models like text embeddings, language models, and custom fine-tuned models to enhance RAG performance. Additionally, we’ll cover how BentoML can help orchestrate and scale these AI components efficiently, ensuring seamless deployment and management of RAG systems in the cloud.
TrustArc Webinar - 2024 Global Privacy Survey
How does your privacy program stack up against your peers? What challenges are privacy teams tackling and prioritizing in 2024?
In the fifth annual Global Privacy Benchmarks Survey, we asked over 1,800 global privacy professionals and business executives to share their perspectives on the current state of privacy inside and outside of their organizations. This year’s report focused on emerging areas of importance for privacy and compliance professionals, including considerations and implications of Artificial Intelligence (AI) technologies, building brand trust, and different approaches for achieving higher privacy competence scores.
See how organizational priorities and strategic approaches to data security and privacy are evolving around the globe.
This webinar will review:
- The top 10 privacy insights from the fifth annual Global Privacy Benchmarks Survey
- The top challenges for privacy leaders, practitioners, and organizations in 2024
- Key themes to consider in developing and maintaining your privacy program
CAKE: Sharing Slices of Confidential Data on Blockchain
Presented at the CAiSE 2024 Forum, Intelligent Information Systems, June 6th, Limassol, Cyprus.
Synopsis: Cooperative information systems typically involve various entities in a collaborative process within a distributed environment. Blockchain technology offers a mechanism for automating such processes, even when only partial trust exists among participants. The data stored on the blockchain is replicated across all nodes in the network, ensuring accessibility to all participants. While this aspect facilitates traceability, integrity, and persistence, it poses challenges for adopting public blockchains in enterprise settings due to confidentiality issues. In this paper, we present a software tool named Control Access via Key Encryption (CAKE), designed to ensure data confidentiality in scenarios involving public blockchains. After outlining its core components and functionalities, we showcase the application of CAKE in the context of a real-world cyber-security project within the logistics domain.
Paper: https://doi.org/10.1007/978-3-031-61000-4_16
Ocean lotus Threat actors project by John Sitima 2024 (1).pptx
Ocean Lotus cyber threat actors represent a sophisticated, persistent, and politically motivated group that poses a significant risk to organizations and individuals in the Southeast Asian region. Their continuous evolution and adaptability underscore the need for robust cybersecurity measures and international cooperation to identify and mitigate the threats posed by such advanced persistent threat groups.
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...Edge AI and Vision Alliance
For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2024/06/building-and-scaling-ai-applications-with-the-nx-ai-manager-a-presentation-from-network-optix/
Robin van Emden, Senior Director of Data Science at Network Optix, presents the “Building and Scaling AI Applications with the Nx AI Manager,” tutorial at the May 2024 Embedded Vision Summit.
In this presentation, van Emden covers the basics of scaling edge AI solutions using the Nx tool kit. He emphasizes the process of developing AI models and deploying them globally. He also showcases the conversion of AI models and the creation of effective edge AI pipelines, with a focus on pre-processing, model conversion, selecting the appropriate inference engine for the target hardware and post-processing.
van Emden shows how Nx can simplify the developer’s life and facilitate a rapid transition from concept to production-ready applications.He provides valuable insights into developing scalable and efficient edge AI solutions, with a strong focus on practical implementation.Choosing The Best AWS Service For Your Website + API.pptx
Have you ever been confused by the myriad of choices offered by AWS for hosting a website or an API?
Lambda, Elastic Beanstalk, Lightsail, Amplify, S3 (and more!) can each host websites + APIs. But which one should we choose?
Which one is cheapest? Which one is fastest? Which one will scale to meet our needs?
Join me in this session as we dive into each AWS hosting service to determine which one is best for your scenario and explain why!
20240607 QFM018 Elixir Reading List May 2024
Everything I found interesting about the Elixir programming ecosystem in May 2024
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Are you ready to revolutionize how you handle data? Join us for a webinar where we’ll bring you up to speed with the latest advancements in Generative AI technology and discover how leveraging FME with tools from giants like Google Gemini, Amazon, and Microsoft OpenAI can supercharge your workflow efficiency.
During the hour, we’ll take you through:
Guest Speaker Segment with Hannah Barrington: Dive into the world of dynamic real estate marketing with Hannah, the Marketing Manager at Workspace Group. Hear firsthand how their team generates engaging descriptions for thousands of office units by integrating diverse data sources—from PDF floorplans to web pages—using FME transformers, like OpenAIVisionConnector and AnthropicVisionConnector. This use case will show you how GenAI can streamline content creation for marketing across the board.
Ollama Use Case: Learn how Scenario Specialist Dmitri Bagh has utilized Ollama within FME to input data, create custom models, and enhance security protocols. This segment will include demos to illustrate the full capabilities of FME in AI-driven processes.
Custom AI Models: Discover how to leverage FME to build personalized AI models using your data. Whether it’s populating a model with local data for added security or integrating public AI tools, find out how FME facilitates a versatile and secure approach to AI.
We’ll wrap up with a live Q&A session where you can engage with our experts on your specific use cases, and learn more about optimizing your data workflows with AI.
This webinar is ideal for professionals seeking to harness the power of AI within their data management systems while ensuring high levels of customization and security. Whether you're a novice or an expert, gain actionable insights and strategies to elevate your data processes. Join us to see how FME and AI can revolutionize how you work with data!
Things to Consider When Choosing a Website Developer for your Website | FODUU
Choosing the right website developer is crucial for your business. This article covers essential factors to consider, including experience, portfolio, technical skills, communication, pricing, reputation & reviews, cost and budget considerations and post-launch support. Make an informed decision to ensure your website meets your business goals.
Climate Impact of Software Testing at Nordic Testing Days
My slides at Nordic Testing Days 6.6.2024
Climate impact / sustainability of software testing discussed on the talk. ICT and testing must carry their part of global responsibility to help with the climat warming. We can minimize the carbon footprint but we can also have a carbon handprint, a positive impact on the climate. Quality characteristics can be added with sustainability, and then measured continuously. Test environments can be used less, and in smaller scale and on demand. Test techniques can be used in optimizing or minimizing number of tests. Test automation can be used to speed up testing.
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
ABSTRACT: A prima vista, un mattoncino Lego e la backdoor XZ potrebbero avere in comune il fatto di essere entrambi blocchi di costruzione, o dipendenze di progetti creativi e software. La realtà è che un mattoncino Lego e il caso della backdoor XZ hanno molto di più di tutto ciò in comune.
Partecipate alla presentazione per immergervi in una storia di interoperabilità, standard e formati aperti, per poi discutere del ruolo importante che i contributori hanno in una comunità open source sostenibile.
BIO: Sostenitrice del software libero e dei formati standard e aperti. È stata un membro attivo dei progetti Fedora e openSUSE e ha co-fondato l'Associazione LibreItalia dove è stata coinvolta in diversi eventi, migrazioni e formazione relativi a LibreOffice. In precedenza ha lavorato a migrazioni e corsi di formazione su LibreOffice per diverse amministrazioni pubbliche e privati. Da gennaio 2020 lavora in SUSE come Software Release Engineer per Uyuni e SUSE Manager e quando non segue la sua passione per i computer e per Geeko coltiva la sua curiosità per l'astronomia (da cui deriva il suo nickname deneb_alpha).
Full-RAG: A modern architecture for hyper-personalization
Mike Del Balso, CEO & Co-Founder at Tecton, presents "Full RAG," a novel approach to AI recommendation systems, aiming to push beyond the limitations of traditional models through a deep integration of contextual insights and real-time data, leveraging the Retrieval-Augmented Generation architecture. This talk will outline Full RAG's potential to significantly enhance personalization, address engineering challenges such as data management and model training, and introduce data enrichment with reranking as a key solution. Attendees will gain crucial insights into the importance of hyperpersonalization in AI, the capabilities of Full RAG for advanced personalization, and strategies for managing complex data integrations for deploying cutting-edge AI solutions.
Programming Foundation Models with DSPy - Meetup Slides
Prompting language models is hard, while programming language models is easy. In this talk, I will discuss the state-of-the-art framework DSPy for programming foundation models with its powerful optimizers and runtime constraint system.
Recently uploaded (20)
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdf
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Infrastructure Challenges in Scaling RAG with Custom AI models
Infrastructure Challenges in Scaling RAG with Custom AI models
CAKE: Sharing Slices of Confidential Data on Blockchain
CAKE: Sharing Slices of Confidential Data on Blockchain
Ocean lotus Threat actors project by John Sitima 2024 (1).pptx
Ocean lotus Threat actors project by John Sitima 2024 (1).pptx
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
Choosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptx
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Things to Consider When Choosing a Website Developer for your Website | FODUU
Things to Consider When Choosing a Website Developer for your Website | FODUU
Climate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing Days
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Full-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalization
Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
Is there still room for innovation in container orchestration and scheduling
- 1. © 2017 Rancher Labs, Inc.© 2017 Rancher Labs, Inc . Is There Still Room for Innovation in Container Orchestration and Scheduling? Sheng Liang, Rancher Labs LinuxCon China, June 19 2017
- 2. © 2017 Rancher Labs, Inc.2
- 3. © 2017 Rancher Labs, Inc.3 Applications Container Orchestration and Scheduling Linux Linux Linux Linux Linux Linux Infrastructure
- 4. © 2017 Rancher Labs, Inc. Kubernetes internal architecture by Brian Grant 4
- 5. © 2017 Rancher Labs, Inc. Our experience with Kubernetes Rancher 1.0: Kubernetes distro 2016: Make Kubernetes easy to install 2017: Make Kubernetes easy to operate 5
- 6. © 2017 Rancher Labs, Inc. Support multiple orchestrators 6
- 7. © 2017 Rancher Labs, Inc. Cattle orchestrator remains popular 7
- 8. © 2017 Rancher Labs, Inc. Reason for Cattle’s popularity: Simplicity 8
- 9. © 2017 Rancher Labs, Inc. Cattle 2.0: even simpler 9 Service discovery and service aliases Load balancer and DNS Service reconciliation Scaling group Service as basic unit of operation Container as basic unit of operation
- 10. © 2017 Rancher Labs, Inc.10
- 11. © 2017 Rancher Labs, Inc. Future of Container Scheduling and Orchestration 11
- 12. © 2017 Rancher Labs, Inc. The Operator pattern 12 “Orchestrator as Code”
- 13. © 2017 Rancher Labs, Inc.13 Scheduling 1. Allocate resources 2. Usually a relatively simple operation that can be completed in a very short amount of time 3. Requires global knowledge Orchestration 1. Performs configuration operations 2. A potentially long-running operation consisting of many small operations 3. Can usually be partitioned
- 14. © 2017 Rancher Labs, Inc. Other features Authentication Authorization/RBAC/Governance API gateway/server Usage metering and chargeback Plugins and drivers Secret management 14
- 15. © 2017 Rancher Labs, Inc. State management in Orchestration Desired state: usually very small An approximation of actual state: usually (unnecessarily) kept in the database Job state: need not persist 15
- 16. © 2017 Rancher Labs, Inc.16 Host 1 Host 2 Container 3 Controller Replica Replica Container 1 Controller ReplicaReplica Container 2 Controller ReplicaReplica Docker volume Docker volume Docker volume Example: Longhorn storage orchestration How do we manage these?
- 17. © 2017 Rancher Labs, Inc. Longhorn storage orchestration Orchestrator runs on every node without centralized master Desired state: hardcoded, try to keep volumes healthy An approximation of actual state: volume meta data Job state: kept in memory, ephemiral 17
- 18. © 2017 Rancher Labs, Inc. If you have experience operating a distributed system, you have knowledge to write an orchestrator There will be many more orchestrators in the future 18
- 19. © 2017 Rancher Labs, Inc.© 2017 Rancher Labs, Inc . Thank you rancher.com