Presentation given at the 2017 LinuxCon China
With the booming of Container technology, it brings obvious advantages for cloud: simple and faster deployment, portability and lightweight cost. But the networking challenges are significant. Users need to restructure their network and support container deployment with current cloud framework, like container and VMs.
In this presentation, we will introduce new container networking solution, which provides one management framework to work with different network componenets through Open/friendly modelling mechnism. iCAN can simplify network deployment and management with most orchestration systems and a variety of data plane components, and design extendsible architect to define and validate Service Level Agreement(SLA) for cloud native applications, which is important factor for enterprise to deliver successful and stable service via containers.
Presentation delivered at LinuxCon China 2016
UEFI HTTP/HTTPS Boot is a new feature of UEFI 2.5+. In the meantime, this feature is not yet implemented in any Linux bootloader. This Birds of a Feather session will give an introduction to UEFI HTTP/HTTPS Boot, and share a proof-of-concept implementation based on grub2 that works on both the emulator (QEMU/OVMF) and HPE ProLiant Gen10 servers.
For HTTPS, the experience and comparison will be shared between the purely software-based and UEFI-based implementations in the aspects of ease of implementation, security strength, and limitation.
Presentation delivered at LinuxCon China 2017 by Greg Kroah-Hartman.
The Linux kernel is the largest collaborative software development projects ever. This talk will discuss exactly how Linux is developed, how fast it is happening, who is doing the work, and how we all stay sane keeping up with it. It will discuss the development model used, and how it differs from almost all "traditional" models of software development.
Enterprise data centers have to support a diverse of set of workloads: cloud native, big data, high performance computing, and legacy applications. While cloud native applications are ideal to run in Docker clusters, bare metal and virtualization infrastructures must still be supported in the data center. The result is a proliferation of clusters and technologies running in individual silos, resulting in high management costs and low utilization. This talk describes the challenges and experiences in implementing a shared cluster infrastructure based on Kubernetes to support big data, high performance computing, and VM-based workloads. The talk will show the deployment and scaling of a high performance computing workload manager, Spark, and OpenStack, and how the VM and Docker management can be integrated together.
This talks shows how to implement the Application-Based Routing in the common Linux Distribution. We use the NDPI to execute the DPI function to category the packet first, use the linux kernel build-it mark to pass the information from user-space to kernel space and then the policy routing system use that mark to route the packet by different destination or interface.
Kubernetes currently has two load balancing mode: userspace and IPTables. They both have limitation on scalability and performance. We introduced IPVS as third kube-proxy mode which scales kubernetes load balancer to support 50,000 services. Beyond that, control plane needs to be optimized in order to deploy 50,000 services. We will introduce alternative solutions and our prototypes with detailed performance data.
Secure Container solution is to enhance container security by isolating memory between Docker containers inside one VM with Intel VT-x EPT HW, which is highly effective to protect container’s memory and at the meantime defends ret2user privilege escalation attack that exploits kernel vulnerabilities (eg. CVE-2017-6074 UAF (use-after-free) vulnerability). It extends KVM interfaces which the guest OS can leverage to isolate container memory from other containers, and the interfaces rely on Intel VT-x EPT hardware extension and provide memory access protection for the container which sits in an isolated memory region. Each secure container has a dedicated EPT table rather than sharing one EPT table with guest OS, which enforces the cross-EPT memory access protection. The whole solution is user-friendly to fit in the existing cloud server infrastructure with very limited changes.
Presentation delivered at LinuxCon China 2016
UEFI HTTP/HTTPS Boot is a new feature of UEFI 2.5+. In the meantime, this feature is not yet implemented in any Linux bootloader. This Birds of a Feather session will give an introduction to UEFI HTTP/HTTPS Boot, and share a proof-of-concept implementation based on grub2 that works on both the emulator (QEMU/OVMF) and HPE ProLiant Gen10 servers.
For HTTPS, the experience and comparison will be shared between the purely software-based and UEFI-based implementations in the aspects of ease of implementation, security strength, and limitation.
Presentation delivered at LinuxCon China 2017 by Greg Kroah-Hartman.
The Linux kernel is the largest collaborative software development projects ever. This talk will discuss exactly how Linux is developed, how fast it is happening, who is doing the work, and how we all stay sane keeping up with it. It will discuss the development model used, and how it differs from almost all "traditional" models of software development.
Enterprise data centers have to support a diverse of set of workloads: cloud native, big data, high performance computing, and legacy applications. While cloud native applications are ideal to run in Docker clusters, bare metal and virtualization infrastructures must still be supported in the data center. The result is a proliferation of clusters and technologies running in individual silos, resulting in high management costs and low utilization. This talk describes the challenges and experiences in implementing a shared cluster infrastructure based on Kubernetes to support big data, high performance computing, and VM-based workloads. The talk will show the deployment and scaling of a high performance computing workload manager, Spark, and OpenStack, and how the VM and Docker management can be integrated together.
This talks shows how to implement the Application-Based Routing in the common Linux Distribution. We use the NDPI to execute the DPI function to category the packet first, use the linux kernel build-it mark to pass the information from user-space to kernel space and then the policy routing system use that mark to route the packet by different destination or interface.
Kubernetes currently has two load balancing mode: userspace and IPTables. They both have limitation on scalability and performance. We introduced IPVS as third kube-proxy mode which scales kubernetes load balancer to support 50,000 services. Beyond that, control plane needs to be optimized in order to deploy 50,000 services. We will introduce alternative solutions and our prototypes with detailed performance data.
Secure Container solution is to enhance container security by isolating memory between Docker containers inside one VM with Intel VT-x EPT HW, which is highly effective to protect container’s memory and at the meantime defends ret2user privilege escalation attack that exploits kernel vulnerabilities (eg. CVE-2017-6074 UAF (use-after-free) vulnerability). It extends KVM interfaces which the guest OS can leverage to isolate container memory from other containers, and the interfaces rely on Intel VT-x EPT hardware extension and provide memory access protection for the container which sits in an isolated memory region. Each secure container has a dedicated EPT table rather than sharing one EPT table with guest OS, which enforces the cross-EPT memory access protection. The whole solution is user-friendly to fit in the existing cloud server infrastructure with very limited changes.
Presentation given at the 2017 LinuxCon China
Unikernel is a novel software technology that links an application with OS in the form of a library and packages them into a specialized image that facilitates direct deployment on a hypervisor. Comparing to the traditional VM or the recent containers, Unikernels are smaller, more secure and efficient, making them ideal for cloud environments. There are already lots of open source projects like OSv, Rumprun and so on. But why these existing unikernels have yet to gain large popularity broadly? We think Unikernels are facing three major challenges: 1. Compatibility with existing applications; 2. Lack of production support (e.g. monitoring, debugging, logging); 3. Lack of compelling use case. In this presentation, we will review our investigations and exploration of if-how we can convert Linux as Unikernel to eliminate these significant shortcomings, plus some explorations of coordinating and cooperating with hypervisor.
Presentation delivered at LinuxCon China 2017.
Open vSwitch (OVS) is a multilayer open source virtual switch. OVS is designed to enable massive network automation through programmatic extension, while still supporting standard management interfaces. OVN is a new network virtualization project that brings virtual networking to the Open vSwitch user community. OVN includes logical switches and routers, security groups, and L2/L3/L4 ACLs, implemented on top of a tunnel-based overlay network.
In this presentation, we will provide an overview of the current state of the projects and their future plans, such as:
- The current state of the Linux, DPDK, and Hyper-V ports
- A status update on a portable BPF-based datapath
- The latest stateful and OpenFlow features available in OVS
- Performance and debugging enhancement to OVN
- OVN features under development such as ACL logging and encrypted tunnels
Event: inovex Meetup "Let's talk about docker!"
Speaker: Johannes M. Scheuermann
28.04.2016
Weitere Tech-Vorträge: https://www.inovex.de/de/content-pool/vortraege/
In this slide, we discussed the IPVS, including the introduction, demonstration, implementation, and integration in Kubernetes.
IPVS was based on the netfilter and we discussed how it works with iptables and also compares the detail implementation in Kubernetes to show why IPVS has a better performance in IPTABLES.
Container Orchestration from Theory to PracticeDocker, Inc.
Join Laura Frank and Stephen Day as they explain and examine technical concepts behind container orchestration systems, like distributed consensus, object models, and node topology. These concepts build the foundation of every modern orchestration system, and each technical explanation will be illustrated using Docker’s SwarmKit as a real-world example. Gain a deeper understanding of how orchestration systems like SwarmKit work in practice and walk away with more insights into your production applications.
In the container ecosystem, there is perhaps no technology that has received more focus and attention than orchestration and scheduling. Mesos, Kubernetes, and Swarm have established themselves as the leading technology choices in this space.
In this talk, Sheng will discuss what he learned from working directly with hundreds of users who have deployed one of these frameworks. He will look at how these frameworks will continue to evolve and if there’re any gaps and opportunities in container orchestration and scheduling. Sheng will make a case that there are still room for innovation and new orchestration and scheduling frameworks will be created in the future. He will discuss what new frameworks might look like--the features, functionalities, and attributes that differentiate them from the mainstream frameworks today.
Introduce the basic concept of Open vSwitch. In this slide, we talked about how Linux kernel and networking stack worked together to forward and process the network packet and also compare those Linux networking stack functionality with Open vSwitch and Openflow.
At the end of this slide, we talk about the challenge to integrate the Open vSwitch with Kubernetes, what kind of the networking function we need to resolve and what is the benefit we can get from the Open Vswitch.
In this slide, we discuss the concept of IPTABLES/EBTABLES and then show how they work in a simple docker environment.
In order to track the packet flow in those containers communication, we use the LOG module in IPTABLES/EBTABLE to track the information.
Introduce the basic concept of load-balancing, common implementations of load-balancing and the detail fo kubernetes service. In the last, demonstrate how to modify the linux iptable kernel module to fulfill the layer-7 load-balcning for kubernetes
Control Your Network ASICs, What Benefits switchdev Can Bring UsHungWei Chiu
In this slide, I will introduce what is switchdev and what problem it wants to solve. To this day, most of the hardware switch's application-specific integrated circuit (ASIC) only be controlled by the vendor's proprietary binary (SDK) and it's inconvenient for system administrator/developer. In order to break the chip vendor's lock-in situation, the switchdev had been designed to solve this. With the help of switchdev, we can develop a general solution for hardware switch chips and break the connection with vendor's binary-blob (SDK).
In order words. Linux kernel can directly communicate with the vendor's proprietary ASIC now, and the software programmer/system administrator can easily control that ASIC to provide more flexible, powerful and programmable network function.
Docker Meetup: Docker Networking 1.11, by Madhu VenugopalMichelle Antebi
In this talk, Madhu Venugopal will present Docker Networking & Service Discovery features shipped in 1.11 and new Experimental Vlan network drivers introduced in 1.11.
Presentation given at the 2017 LinuxCon China
Unikernel is a novel software technology that links an application with OS in the form of a library and packages them into a specialized image that facilitates direct deployment on a hypervisor. Comparing to the traditional VM or the recent containers, Unikernels are smaller, more secure and efficient, making them ideal for cloud environments. There are already lots of open source projects like OSv, Rumprun and so on. But why these existing unikernels have yet to gain large popularity broadly? We think Unikernels are facing three major challenges: 1. Compatibility with existing applications; 2. Lack of production support (e.g. monitoring, debugging, logging); 3. Lack of compelling use case. In this presentation, we will review our investigations and exploration of if-how we can convert Linux as Unikernel to eliminate these significant shortcomings, plus some explorations of coordinating and cooperating with hypervisor.
Presentation delivered at LinuxCon China 2017.
Open vSwitch (OVS) is a multilayer open source virtual switch. OVS is designed to enable massive network automation through programmatic extension, while still supporting standard management interfaces. OVN is a new network virtualization project that brings virtual networking to the Open vSwitch user community. OVN includes logical switches and routers, security groups, and L2/L3/L4 ACLs, implemented on top of a tunnel-based overlay network.
In this presentation, we will provide an overview of the current state of the projects and their future plans, such as:
- The current state of the Linux, DPDK, and Hyper-V ports
- A status update on a portable BPF-based datapath
- The latest stateful and OpenFlow features available in OVS
- Performance and debugging enhancement to OVN
- OVN features under development such as ACL logging and encrypted tunnels
Event: inovex Meetup "Let's talk about docker!"
Speaker: Johannes M. Scheuermann
28.04.2016
Weitere Tech-Vorträge: https://www.inovex.de/de/content-pool/vortraege/
In this slide, we discussed the IPVS, including the introduction, demonstration, implementation, and integration in Kubernetes.
IPVS was based on the netfilter and we discussed how it works with iptables and also compares the detail implementation in Kubernetes to show why IPVS has a better performance in IPTABLES.
Container Orchestration from Theory to PracticeDocker, Inc.
Join Laura Frank and Stephen Day as they explain and examine technical concepts behind container orchestration systems, like distributed consensus, object models, and node topology. These concepts build the foundation of every modern orchestration system, and each technical explanation will be illustrated using Docker’s SwarmKit as a real-world example. Gain a deeper understanding of how orchestration systems like SwarmKit work in practice and walk away with more insights into your production applications.
In the container ecosystem, there is perhaps no technology that has received more focus and attention than orchestration and scheduling. Mesos, Kubernetes, and Swarm have established themselves as the leading technology choices in this space.
In this talk, Sheng will discuss what he learned from working directly with hundreds of users who have deployed one of these frameworks. He will look at how these frameworks will continue to evolve and if there’re any gaps and opportunities in container orchestration and scheduling. Sheng will make a case that there are still room for innovation and new orchestration and scheduling frameworks will be created in the future. He will discuss what new frameworks might look like--the features, functionalities, and attributes that differentiate them from the mainstream frameworks today.
Introduce the basic concept of Open vSwitch. In this slide, we talked about how Linux kernel and networking stack worked together to forward and process the network packet and also compare those Linux networking stack functionality with Open vSwitch and Openflow.
At the end of this slide, we talk about the challenge to integrate the Open vSwitch with Kubernetes, what kind of the networking function we need to resolve and what is the benefit we can get from the Open Vswitch.
In this slide, we discuss the concept of IPTABLES/EBTABLES and then show how they work in a simple docker environment.
In order to track the packet flow in those containers communication, we use the LOG module in IPTABLES/EBTABLE to track the information.
Introduce the basic concept of load-balancing, common implementations of load-balancing and the detail fo kubernetes service. In the last, demonstrate how to modify the linux iptable kernel module to fulfill the layer-7 load-balcning for kubernetes
Control Your Network ASICs, What Benefits switchdev Can Bring UsHungWei Chiu
In this slide, I will introduce what is switchdev and what problem it wants to solve. To this day, most of the hardware switch's application-specific integrated circuit (ASIC) only be controlled by the vendor's proprietary binary (SDK) and it's inconvenient for system administrator/developer. In order to break the chip vendor's lock-in situation, the switchdev had been designed to solve this. With the help of switchdev, we can develop a general solution for hardware switch chips and break the connection with vendor's binary-blob (SDK).
In order words. Linux kernel can directly communicate with the vendor's proprietary ASIC now, and the software programmer/system administrator can easily control that ASIC to provide more flexible, powerful and programmable network function.
Docker Meetup: Docker Networking 1.11, by Madhu VenugopalMichelle Antebi
In this talk, Madhu Venugopal will present Docker Networking & Service Discovery features shipped in 1.11 and new Experimental Vlan network drivers introduced in 1.11.
Presentation delivered at LinuxCon China 2017
The practices of Blockchain as a service in Dianrong (Shiyuan Xiao, Dianrong.com) - Blockchain as a Service (BaaS) provides a easy, low-cost and flexible platform for companies to enable their businesses based on blockchain backed by a cloud platform. Shiyuan will introduce the experiences to build such a BaaS platform, what is the architecture, what problems we have met and solved and the best practices we summarized.
Kdump is a long existing method for acquiring dump of crashed kernel, however very few literatures are available to understand it's usage and internals. We receive a lot of queries on kexec mailing list about different issues related to the kexec/kdump environment.
In this presentation, we talk about basics of kdump usage and some internals about kdump/kexec kernel implementation. It includes end to end flow from kdump kernel configuration to crash analysis. We discuss some of the problem which is frequently faced by kdump users. It also includes related information about ELF structure, so that one can debug if vmcore itself gets corrupted because of any architecture related issue.
Presentation delivered at LinuxCon China 2017.
Operating systems need to move faster without sacrificing stability. New hardware, new software features, and bugfixes are making it into distribution components every day. To maintain stability, packagers and distribution developers are looking toward lessons learned in the DevOps movement to implement Continuous Integration/Continuous Delivery (CI/CD) workflows that provide quicker test feedback to developers.
This talk highlights some of the coming trends in Fedora such as: streamlined base package sets, userspace applications delivered as containers, continuous validation of individual distro components and the distro as a whole, and collaboration with the CentOS Project.
Can we leverage the resource of public cloud for gaming, streaming, transcoding, machine learning and visualized CAD application on demand? Yes if it provides the capability and infrastructure to utilize GPUs. Can we get high performance networking in the cloud as what I have in the bare metal environment? Yes with SR-IOV. How to achieve them? In this presentation we describe Discrete Device Assignment (also known as PCI Pass-through) support for GPU and network adapter in Linux guest and SR-IOV architectures of Linux guest with near-native performance profile running on Hyper-V. We also will share how to integrate accelerated graphics and networking capabilities in Microsoft Azure infrastructure.
A lot of Internet of things devices use linux as its core. More so with the advent of DIY projects and Internet of things projects. A lot of Raspberry PI's, Beaglebone, Tessel boards are out there with default settings, and all connected to the internet, ready to be taken over. With the recent dyn DNS attack its of prime importance to know how we can keep these end point devices secure and out of the hands of botnet hoarders, attackers. In this presentation Rabimba Karanjai will show how to harden the security on these endpint devices taking a RaspBerry PI as an example. He will explain different techniques with code examples along with a toolkit made specifically for this demo which will make devices considerable harder to compromise. And even when they are, will allow to locate and detect the breach. After all, proetcting the device fially protects us all (prevents another DDOS)
Presentation delivered at the 2017 LinuxCon China.
Build robust blockchain services (Wenjie(Jay) Xie, wutongtree.com) - Blockchain is considered as a great evolution. But the performance, maintainability, and scalability are still confusing many companies. Jay will show you how they reach high availability, scalability, and performance by using hyperledger and container to build robust blockchain services. He will also share their experience on dealing TB data in blockchain and operating a large scale of blockchain services in containers, including linking hyperledger and hbase, service warmup, and much more.
Presentation by Stephen R. Walli at LinuxCon China 2017
There are best practices to understand when building products from open source software, but there are a number of anti-patterns that crop up along the way. Product teams (from engineering to marketing) need to understand these patterns and practices to participate best in open source project communities and deliver products and services to their customers at the same time. These patterns hold regardless of whether the vendor created and owns the project or participates in projects outside their control.
Fully Automated Kubernetes Deployment and Management (Peng Jiang, Rancher Labs) - Kubernetes is rapidly gaining popularity as a powerful container orchestration and scheduling platform. But deploying and managing Kubernetes clusters is still a challenge for many organizations.How to ensure Kubernetes clusters in different clouds and data centers can communicate with each other? How to automate the deployment of multiple Kubernetes clusters? How to incorporate the new Kubernetes Federation into multi cloud and multi datacenter deployments? How to manage the health of Kubernetes cluster itself? etc.
In this talk, Peng will share his experience on how to automate and simplify Kubernetes deployments, and discuss how some of the latest community projects (such as kubeadm and self-hosting Kubernetes) will help address the problems in the future.
There are best practices to understand when building products from open source software, but there are a number of anti-patterns that crop up along the way. Product teams (from engineering to marketing) need to understand these patterns and practices to participate best in open source project communities and deliver products and services to their customers at the same time. These patterns hold regardless of whether the vendor created and owns the project or participates in projects outside their control.
Failure injection is somewhat analogous to a vaccine. We want to inject these bad behaviours so our developers can build immunities to them. Can we inject failure scenarios into deployed systems to reduce platform risk Demonstrations of the Simian Army, Chaos Lemur and Locust.io tools will be presented.
Even when all of the individual services in a distributed system are functioning properly, the interactions between those services can cause unpredictable outcomes. Unpredictable outcomes, compounded by rare but disruptive real-world events that affect production environments, make these distributed systems inherently chaotic.
BKK16-409 VOSY Switch Port to ARMv8 Platforms and ODP IntegrationLinaro
Virtual Open Systems has developed VOSYSwitch, a high-performance user space networking virtual switch solution enabling NFV, based on the open source packet processing framework SnabbSwitch. In this talk, the experience of porting VOSYSwitch from x86 to ARMv8 will be shared, along with the integration of ODP as a driver layer for the available hardware resources. In addition to this presentation, a live demonstration will showcase chained VNFs connected through VOSYSwitch, where an OpenFastPath web server is implemented behind an ODP enabled packet filtering firewall. The targeted platforms are Freescale (NXP) LS2085A and Cavium's ThunderX.
Summit 16: How to Compose a New OPNFV Solution Stack?OPNFV
This session showcases how a new OPNFV solution stack (a.k.a. ""scenario"") is composed and stood up. We'll use a new solution stack framed around a new software forwarder (""VPP"") provided by the FD.io project as example for this session. The session discusses how an evolution/change of upstream components from OpenStack, OpenDaylight and FFD.io are put in place for the scenario, how installers and tests need to be evolved to allow for integration into OPNFV's continuous integration, deployment and test pipeline.
An Introduce of OPNFV (Open Platform for NFV)Mario Cho
OPNFV is Open Platform for Network Function Virtualization.
It lecture are talk on Open Software Conference 2015.
The Lecture of OPNFV explain OPNFV sub-software technology like The Linux Kernel, Virtualization, Software Defined Network, OpenStack, OpenDaylight, and Network Function Virtualization.
Overview of OpenStack nova-networking evolution towards Neutron. Architecture overview of OVS plugin, ML2, and MidoNet Overlay product. Overview and example of Heat templates, along with automation of physical switches using Cumulus
Vector Packet Technologies such as DPDK and FD.io/VPP revolutionized software packet processing initially for discrete appliances and then for NFV use cases. Container based VNF deployments and it's supporting NFV infrastructure is now the new frontier in packet processing and has number of strong advocates among both traditional Comms Service Providers and in the Cloud. This presentation will give an overview of how DPDK and FD.io/VPP project are rising to meet the challenges of the Container dataplane. The discussion will provide an overview of the challenges, recent new features and what is coming soon in this exciting new area for the software dataplane, in both DPDK and FD.io/VPP!
About the speaker: Ray Kinsella has been working on Linux and various other open source technologies for about twenty years. He is recently active in open source communities such as VPP and DPDK but is a constant lurker in many others. He is interested in the software dataplane and optimization, virtualization, operating system design and implementation, communications and networking.
Enterprise Datacenter Virtualization und Cloud Computing stellen neue Anforderungen an das Netzwerk. Traditionsgemäss wurden virtuelle Workloads über als Bridge fungierende virtuelle Switches mit VLANs auf dem physischen Netzwerk verbunden. Mit dem Wachstum der Anfordungen an Skalierung und Automatisierung stossen diese Modelle an Grenzen.
Thomas Graf bot an diesem OpenTuesday einen Einblick in Protokolle und Technologien wie OpenFlow, VXLAN, OpenStack Neutron und Open vSwitch, die eingesetzt werden, um neue automatisierte Netzwerkkonzepte der nächsten Generation, wie Software Defined Networking oder Network Function Virtualization, umzusetzen.
Faced with the dual threats of rising operating costs and declining revenues, network service providers are increasingly turning to network functions virtualization (NFV) to help them keep up with constantly changing market conditions.
In a virtualized Telco environment, service providers can deploy and deliver new network functions, services and capacity on demand—reducing normal rollout time from months and weeks to just hours.
Leveraging the principles of cloud computing, network service providers can deliver a level of responsiveness never before available, easily scaling capacity up or down to meet the evolving needs of their subscribers.
The result is a highly agile system that allows new revenue-generating services to be quickly developed, exhaustively tested and selectively rolled out to targeted groups in a fraction of the time and at a much lower cost than previously thought possible.
In this session, the speaker will present how the solution from Juniper networks look like and how it can be deployed by service provider to improve their agility in delivering services to their customers.
Pushing Packets - How do the ML2 Mechanism Drivers Stack UpJames Denton
Architecting a private cloud to meet the use cases of its users can be a daunting task. How do you determine which of the many L2/L3 Neutron plugins and drivers to implement? Does network performance outweigh reliability? Are overlay networks just as performant as VLAN networks? The answers to these questions will drive the appropriate technology choice.
In this presentation, we will look at many of the common drivers built around the ML2 framework, including LinuxBridge, OVS, OVS+DPDK, SR-IOV, and more, and will provide performance data to help drive decisions around selecting a technology that's right for the situation. We will discuss our experience with some of these technologies, and the pros and cons of one technology over another in a production environment.
OVHcloud Hosted Private Cloud Platform Network use cases with VMware NSXOVHcloud
In this workshop VMware will provide a quick reminder of the main contributions of the NSX network virtualization platform: consistent network and security management, increased application resiliency, rapid migration of workloads to and from the cloud.
VMware and OVH will then move on to practical cases with implementation of micro-segmentation, dynamic routing, automatic deployment of an application, load balancing in the OVH Hosted Private Cloud. This workshop is aimed at a technical audience.
In this talk, Tim Bird will discuss the recent status of the Linux with regard to embedded systems. This will include a review of the last year's worth of mainline kernel releases, as well as topic areas specifically related to embedded, such as boot-up time, security, system size, etc. Tim will also present recent and planned work by the Core Embedded Linux Project of the Linux Foundation, and discuss the current status of Linux in various markets and fields. Tim will go over current areas of work, and discuss remaining challenges faced by Linux in embedded projects.
After returning from a recent trip that occurred during the middle of a heat wave. I arrived home to find my apartment quite hot, at least 45C inside. Needless to say it wasn’t the most comfortable way to come home after 15 days out of town, I decided it was time for me to do something about it to address this so I didn't come home to that unpleasant surprise again. Normally, this problem is solved by having a thermostat which controls the air conditioning. However, my apartment did not have a thermostat. So I decided to build one using open source software.
This talk will cover how I went about solving my problem using existing software and protocols like home-assistant, MQTT, and also some new software that was created for this. It'll also discuss how using open software and home automation I was able to solve my issue but also make cooling my apartment smarter.
The Blockchain for the Internet of Things (IoT) has considered to "change the future." Despite a myriad of studies on the blockchain IoT, few studies have investigated how an IoT blockchain system develops with open source technologies, open standards, web technologies, and a p2p network. In this presentation, Jollen will share the Flowchain case study, an open source IoT blockchain project in Node.js; he will discuss the practice, the technical challenges, and the engineering experiences. Furthermore, to provide the real-time data transaction capabilities for current IoT requirements, he will utilize the "virtual block" idea to facilitate such technical challenges.
Besides huge success in mobile, ARM is also ambitious in server field. Software ecosystem is now a barrier for wide deployment of ARM servers in data center. ARM Shanghai Workloads team is working on clouding and big data software enablement and optimization on ARM64 platform.
In this presentation, Yibo Cai will introduce the status and challenges of running OpenStack on ARM servers, with emphasis on OpenStack compute, storage and networking.
Resource placement is a policy-rich problem, particularly across multi-cluster, multi-geography and multi-cloud environments. Placement may be based on company conventions, external regulation, pricing, performance requirements, or complex combinations of those. Furthermore, placement policies evolve over time and vary across organizations. As a result, it is very difficult to anticipate the policy requirements of all users.
In this presentation, Torin Sandal (Lead Engineer of Open Policy Agent) will present, along with Irfan Ur Rehman, and demonstrate the work they've done integrating OPA into the Kubernetes Cluster Federation Control Plane. This enables high level policies to be expressed in a easy to understand policy language, and automatically enforced across federations of Kubernetes clusters.
Presentation delivered at LinuxCon China 2017
Real-Time is used for deadline-oriented applications and time-sensitive workloads. Real-Time KVM is the extension of KVM(Linux Kernel-based Virtual Machine) to allow the virtual machines(VM) to be a truly Real-Time operating system.Users sometimes need to run low-latency applications(such as audio/video streaming, highly interactive systems, etc) to meet their requirements in clouds. NFV is a new network concept which uses virtualization and software instead of dedicated network appliances. For some use cases of telecommunications, network latency must be within a certain range of values. Real-Time KVM can help NFV meet this requirements.
In this presentation, Pei Zhang will talk about:
(1)Real-Time KVM introduction
(2)Real-Time cloud building
(3)Real-Time KVM in NFV: VM with openvswitch, dpdk and qemu’s vhostuser
(4)Performance testing results show
Presentation delivered at LinuxCon China 2017.
The Libvirt API is cloud industry standard API to manage virtualization hosts on cross platforms.It is widely implemented in renowned cloud system such as openstack,opencloud. The compatibility and fragmentation avoiding of Libvirt APIs will eventually play great impact on what Libvirt can achieve as a whole. For this reason,Libvirt API certification is introduced. Libvirt API certification focuses on testing a technology implementation to make sure that it operates consistently with all other implementations of the same Libvirt technology specification.
In this paper, the author will review current state of Libvirt API certification, discuss the challenges it faces, and look forward to how Libvirt community may address those challenges.
Presentation delivered at LinuxCon China 2017. Rethinking the Operating System.
A new wave of Operating Systems optimized for containers appeared on the horizon making us excited and puzzeled at the same time.
"Why do we need anything different for containers when traditional OSs served us well in the last 25+ years?" "Isn't Kubernetes just another package to install on top of my favorite distro?"" Will this obsolete my whole infrastructure?" are some of the questions this talk will shed some light on.
Explore the journey SUSE made in rethinking the OS: From a conservative linux distribution to a platform that goes hand in hand with the needs of Microservices.
You will get an insight at what lessons were learned during the intense development effort that lead to SUSE Containers as a Service Platform, how the obstacles along the way were lifted and why "Upstream first" is - and should always be - the rule.
Presentation delivered at LinuxCon China 2017.
Zephyr is an upstream open source project for places where Linux is too big to fit. This talk will overview the progress we've made in the first year towards the projects goals around incorporating best of breed technologies into the code base, and building up the community to support multiple architectures and development environments. We will share our roadmap, plans and the challenges ahead of the us and give an overview of the major technical challenges we want to tackle in 2017.
More from LinuxCon ContainerCon CloudOpen China (10)
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
Keynote at DIGIT West Expo, Glasgow on 29 May 2024.
Cheryl Hung, ochery.com
Sr Director, Infrastructure Ecosystem, Arm.
The key trends across hardware, cloud and open-source; exploring how these areas are likely to mature and develop over the short and long-term, and then considering how organisations can position themselves to adapt and thrive.
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Ramesh Iyer
In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
PHP Frameworks: I want to break free (IPC Berlin 2024)Ralf Eggert
In this presentation, we examine the challenges and limitations of relying too heavily on PHP frameworks in web development. We discuss the history of PHP and its frameworks to understand how this dependence has evolved. The focus will be on providing concrete tips and strategies to reduce reliance on these frameworks, based on real-world examples and practical considerations. The goal is to equip developers with the skills and knowledge to create more flexible and future-proof web applications. We'll explore the importance of maintaining autonomy in a rapidly changing tech landscape and how to make informed decisions in PHP development.
This talk is aimed at encouraging a more independent approach to using PHP frameworks, moving towards a more flexible and future-proof approach to PHP development.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
Essentials of Automations: Optimizing FME Workflows with ParametersSafe Software
Are you looking to streamline your workflows and boost your projects’ efficiency? Do you find yourself searching for ways to add flexibility and control over your FME workflows? If so, you’re in the right place.
Join us for an insightful dive into the world of FME parameters, a critical element in optimizing workflow efficiency. This webinar marks the beginning of our three-part “Essentials of Automation” series. This first webinar is designed to equip you with the knowledge and skills to utilize parameters effectively: enhancing the flexibility, maintainability, and user control of your FME projects.
Here’s what you’ll gain:
- Essentials of FME Parameters: Understand the pivotal role of parameters, including Reader/Writer, Transformer, User, and FME Flow categories. Discover how they are the key to unlocking automation and optimization within your workflows.
- Practical Applications in FME Form: Delve into key user parameter types including choice, connections, and file URLs. Allow users to control how a workflow runs, making your workflows more reusable. Learn to import values and deliver the best user experience for your workflows while enhancing accuracy.
- Optimization Strategies in FME Flow: Explore the creation and strategic deployment of parameters in FME Flow, including the use of deployment and geometry parameters, to maximize workflow efficiency.
- Pro Tips for Success: Gain insights on parameterizing connections and leveraging new features like Conditional Visibility for clarity and simplicity.
We’ll wrap up with a glimpse into future webinars, followed by a Q&A session to address your specific questions surrounding this topic.
Don’t miss this opportunity to elevate your FME expertise and drive your projects to new heights of efficiency.
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
Let's dive deeper into the world of ODC! Ricardo Alves (OutSystems) will join us to tell all about the new Data Fabric. After that, Sezen de Bruijn (OutSystems) will get into the details on how to best design a sturdy architecture within ODC.
"Impact of front-end architecture on development cost", Viktor TurskyiFwdays
I have heard many times that architecture is not important for the front-end. Also, many times I have seen how developers implement features on the front-end just following the standard rules for a framework and think that this is enough to successfully launch the project, and then the project fails. How to prevent this and what approach to choose? I have launched dozens of complex projects and during the talk we will analyze which approaches have worked for me and which have not.
Connector Corner: Automate dynamic content and events by pushing a buttonDianaGray10
Here is something new! In our next Connector Corner webinar, we will demonstrate how you can use a single workflow to:
Create a campaign using Mailchimp with merge tags/fields
Send an interactive Slack channel message (using buttons)
Have the message received by managers and peers along with a test email for review
But there’s more:
In a second workflow supporting the same use case, you’ll see:
Your campaign sent to target colleagues for approval
If the “Approve” button is clicked, a Jira/Zendesk ticket is created for the marketing design team
But—if the “Reject” button is pushed, colleagues will be alerted via Slack message
Join us to learn more about this new, human-in-the-loop capability, brought to you by Integration Service connectors.
And...
Speakers:
Akshay Agnihotri, Product Manager
Charlie Greenberg, Host
Search and Society: Reimagining Information Access for Radical FuturesBhaskar Mitra
The field of Information retrieval (IR) is currently undergoing a transformative shift, at least partly due to the emerging applications of generative AI to information access. In this talk, we will deliberate on the sociotechnical implications of generative AI for information access. We will argue that there is both a critical necessity and an exciting opportunity for the IR community to re-center our research agendas on societal needs while dismantling the artificial separation between the work on fairness, accountability, transparency, and ethics in IR and the rest of IR research. Instead of adopting a reactionary strategy of trying to mitigate potential social harms from emerging technologies, the community should aim to proactively set the research agenda for the kinds of systems we should build inspired by diverse explicitly stated sociotechnical imaginaries. The sociotechnical imaginaries that underpin the design and development of information access technologies needs to be explicitly articulated, and we need to develop theories of change in context of these diverse perspectives. Our guiding future imaginaries must be informed by other academic fields, such as democratic theory and critical theory, and should be co-developed with social science scholars, legal scholars, civil rights and social justice activists, and artists, among others.
3. 3
cloud native and containerised micro-services
high density/
dynamic
complex deployment
scenarios
online monitoring
and control
E2E Monitoring
VM Containers
Public
Cloud
Private
Cloud
L2/L3 Overlay Tunnel
SLA (Application to
Application)
more applications and micro services
are deployed in containers
5. 5
deployment complexity
public clouds:
AWS/Azure/HEC
NFV: SR-
IOV/L2/L3
private clouds:
openstack/vmware/
baremetal
simpleflatcontainer
networkmodel:CNI
complexdeployment
scenarios
existing solutions are
suitable for limited cases
with hard-coded “plugins”
require a flexible solution that
always adapts the best technology
based on specific situation
6. 6
Neutron
Kuryr
Bare
mental
Traditional OS
Socket
OpenStack
Backend
TCP/IP
Stack
Container
OS
Socket
TCP/IP
Stack
vSwitch
OVERLAY
NICs
Driver
Underlay
vRouter
Underlay
Kuryr
OVERLAY
Cloud Provider
XEN
KVM
IRONIC
Container OS
Socket
vNIC
DRIVER
TCP/IP
Stack
Container OS
Socket
vNIC
DRIVER
TCP/IP
Stack
Container OS
NIC
DRIVER
Network
Stack (Iaas)
vSwitch
OVERLAY
VPC
vRouter
vSwitch
OVERLAY
vSwitch
OVERLAY
Traditional
OS
Socket
TCP/IP
Stack
Bridge
OVS L2
OVERLAY OVERLAY OVERLAY
Socket
TCP/IP
Stack
SND
backend
HostGW
SuSE12
Socket
Native
Driver
TCP/IP
Stack
Bridge
OVS L2
OVERLAY
Container
OS
L2
Bare
mental
Bare
mental
Hetero
OS
Pass
through
Contain
erOS
DPDK
API
VF
DPDK
PMD
Containe
rOS
DPDK
API
vNIC
DPDK
PMD
VF
DPDK
PMD
VF PassThrough
OVERLAY L2
Container
OS
DPDK
API
vNIC
DPDK
PMD
VF
DPDK
PMD
Cloud
Provider
Bare mental
OVERLAY
Container
OS
Socket
TCP/IP
Stack
vSwitch
OVERLAY
NICs
Driver
L2
Container
OS
vSwitch
OVERLAY
NICs
Driver
Bare Mental Host
Socket
TCP/IP
Stack
vSwitch
L2
NICs
Driver
Socket
TCP/IP
Stack
vSwitch
OVERLAY
vSwitch
L2
Public
Cloud
Private
Cloud
How we deal with so many scenarios for
containers?
7. 7
Kernel Network Stack
vNIC
@Container
vNIC
@Container
vNIC
@Container
Function
feature
Rich, identical to
Kernel
Performance Normal
Compatibility Very good
Customized Network Stack
Customized
Socket Lib
Customized
Socket Lib
Customized
Socket Lib
Function
feature
Normal, according
to Customized Stack
Performance Good, about 3 times
than Kenel
Compatibility Normal, maybe miss
some socket function
DPDK PMD
DPDK
DPDK Client
@ Container
Function
feature
Poor, according to
DPDK application
Performance Very good, identical
to wire speed
Compatibility Poor, only DPDK ENV
DPDK
DPDK Client
@ Container
DPDK
DPDK Client
@ Container
Why we need so many models
8. 8
Our solution: iCAN (intelligent Container
Network)
an extensible framework to
•program various container network data path and
policies
•adapt to different orchestrators
•support end-to-end SLA between containerised
applications
10. 10
CNI Interface Extension
br-intX
Node
PodX
Eth0
br-intY
PodY
Eth1
PhyNet1
PhyNet2
PaaS
①CNI ADD
{
"cniVersion": "0.2.0",
"name": "IDM-M",
"type": "bridge-veth",
// type (plugin) specific
"vlanID": 42,
"ipam": {
"type": "dhcp",
"routes": [ { "dst": "10.3.0.0/16" }, { "dst": "10.4.0.0/16" } ]
}
// args may be ignored by plugins
"args": {
"labels" : {
" phynet " : " Phy_Net1"
}
}
}
{
"cniVersion": "0.2.0",
"name": "IDM-M",
"type": "bridge-veth",
// type (plugin) specific
"vlanID": 42,
"ipam": {
"type": "dhcp",
"routes": [ { "dst": "10.3.0.0/16" }, { "dst": "10.4.0.0/16" } ]
}
// args may be ignored by plugins
"args": {
"labels" : {
" phynet " : " Phy_Net1"
}
}
}
{
"cniVersion": "0.2.0",
"name": "IDM-C",
"type": "bridge-veth",
// type (plugin) specific
"vlanID": 43,
"ipam": {
"type": "dhcp",
"routes": [ { "dst": "10.3.0.0/16" }, { "dst": "10.4.0.0/16" } ]
}
// args may be ignored by plugins
"args": {
"labels" : {
" phynet " : " Phy_Net2"
}
}
}
①CNI ADD
②CNI Network Configuration
②CNI Network Configuration
Once with one ticket Once with multi ticket
1) Parameters on CNI Network Configuration ,
support Once or Multi entry;
2) Reuse the CNI’s common agreement, all
customized fields within ”args” segment;
11. 11
Standard Network Component (SNC) model
abstract for network components in data-path
• interfaces, devices and templates
L2 device
l2 interface
L2 devices:
bridge/macvlan/ovs/…
L3 device
l3 interface
L3 devices:
router/ipvlan/…
L2 dev:linux bridge
L3 dev: IPS
a template for Flannel
data path
l3 interface
l2 paired
interface
l2 interface
l2 paired
interface
12. 12
Unified Framework For Multi Models
Container
MNG
Flannel Plugin Calico Plugin ……
Linux BR
Kernel
VxLAN
Kernel Route
Kernel Route
PGP Route
Sync
Linux BR
Kernel
Route
Container
MNG
Canal Plugin
Linux BR
Kernel VxLAN
Kernel Route
Kernel Route
PGP Route
Sync
GRE Tunnel
IPIP
Tunnel
Flannel Type Calico Type SR-IOV type
User
Stack
SR-IVO
Through
● Existing every Plugins only support its own model
● Though they employ common data module, the
function is isolated
● After deconstruct different data path , we setup a DSL
language to describe them ,using abstracted standard
component
● Unified Framework with Pluggable drivers for additional
vSwitches, Linux BR, SR-IOV, ...
13. 13
Big Pic of Multi-modes && Multi-planes
PHY-NET
PHY-OM PHY-MNG PHY-DATA
NIC NIC
NIC NIC NIC NIC NIC NIC NIC NIC
NICNIC
DPDK
PMD
User
vSwitch
User
Stack
@Host
Container
User
Socket Lib
Container
User
Socket Lib
bonding
DPDK
PMD
User
vSwitch
User
Stack
@Host
Process
APP
Container
User
Socket Lib
Container
User
Socket Lib
Container
DPDK
APP
UIO UIO
DPDK
PMD
Container
DPDK
APP
UIO UIO
DPDK
PMDvSwitch
Container
Kernel
Stack
bonding
Process
APPvSwitch
Container
Kernel
Stack
14. 14
Open stack Neutron Ml2 Solution
Neutron Server
ML2
Plugin
Host A
Linuxbridge
Agent
Host B
Hyper-V
Agent
Host C
Open vSwitch
Agent
Host D
Open vSwitch
Agent
API Network
Neutron Server
ML2
Plugin
Host A
Modular
Agent
Host B
Modular
Agent
Host C
Modular
Agent
Host D
Modular
Agent
API Network
● Existing ML2 Plugin works with existing agents
● Separate agents for Linuxbridge, Open vSwitch,
and Hyper-V
● Combine Open Source Agents, a single agent which can
support Linuxbridge and Open vSwitch
● Pluggable drivers for additional vSwitches, Infiniband, SR-
IOV, ...
15. 15
iCAN Control Plane Integrated with Openstack
Local
Node
Kuberlet
CANAL
Agent
C C C C C C
CANAL Master
Distributed
KV store
(etcd)
Kubernetes
Master
Monitoring
controller
SLA Manager
IPAM
Neutron
controller
Openstack
Neutron Server
Kuryr Agent
Control
Node
Neutron
controller
16. 16
Monitoring based SNC Modeling
pDev
pPort
pIF
pDev
pPort
pIF
vDev
vPort
vIF
pIF
vPort
vIF
vPort
C1
vIF
C2
vIF
vDev
vPort
vIF
pIF
vPort
C3
vIF
•E2E Monitoring
Point Monitor Item
Source Dest
T1
T4
T2
T3
Latency = ((T4 - T1) - (T3 - T2)) / 2
Monitoring on local SNC components :
Latency:
Generate E2E monitoring data in master node:
Monitorin
g Agent
Monitorin
g Agent
… …
Monitoring
Master
•E2Ethrought:minimal throughput
•E2E Drop rate: deviations between RX
and TX
•Throughput Analysis:data from local
node
Bandwidth
Throughput
Status
QoS
CPU utilization
17. 17
Simplify Network SLA modeling
iCAN provides north bound interfaces for orchestration and applications to define their requirements through PG(Pod Group: a group of pods with
the same functions), Linking (network requirement between PG) , SLA Service types and Service LB Type.
Given topology and link bandwidth, evaluate the offers when deploying pods. Essentially a evaluation for pod placement, and validate the
deployment.
2-Tiers Network topology management Underlay Network(Stable and Predictable) and Overlay Network (Customizable and Dynamic)
Support: bandwidth, latency and drop rate
Bandwidth <5%
Latency <10%, more non-deterministic, affected by many factors such as queuing in software switch and hardware, application response,
server IO, etc
Web
Web
DB
DB
Web
Internet
10Mbps (x3)
5Mbps (x6)
Web
Web
DB
DBInternet
10Mbps (x2)
Latency: Low
User 1
User 2
Polices Deployment
Scheduler
validation
Convert link requirement
to node requirement
18. 18
iCAN Container networking
Multi-dimension SLA& Security
Performance Isolation with bandwidth, latency, drop rate(Proactive Network SLA and
Reactive Network SLA )
Security Isolation: VLAN/VXLAN, ACL
Rich Network Support
Powerful network component modeling : SNC and Modeling via Yang
Rich network schemes, support L2, Overlay, NAT, VLAN, L3, BGP, VPC
Accelerated Network Stack
Powerful Monitoring
Implement “monitoring on-demand ”and “E-to-E monitoring” based on the topology
Facilitate on-demand DSL based troubleshooting
Cooperate with the SLA subsystem to assess the SLA quality