Iptables and Netfilter were introduced in 2001 along with Linux 2.4 as the full layer for firewall. The functionalities and the codes changed quite a lot during this decade, but nothing like what has been done with nftables.
The motivation for this change is to overcome the limitations of iptables that was beginning to date both foncionnal level and in the code design: problem with the system update rules (very expensive when the number of rules increases which has become a problem to manage not static rules), code duplication, problematic for code maintenance and users.
Nftables is a replacement for iptables that has been developed since 2008 by Patri ck McHardy who is the head of the Netfilter project. After a period of sleep, the developments around the project resumed in 2012 and a team of developers was formed and is working on the project.
Nftables solves the problem of updates performance using a communication message between the kernel and user space. Infrastructure Netlink was used because it is the basis of the latest major Netfilter developments.
The most notable changes:
incremental update and atomic rules guaranteeing the performance and consistency of the set of rules
expression of the rules using a pseudo machine for avoiding complex operations of writing core modules and additional extensions
Nftables exceeds the limitations of iptables and brings news that should resolve elegant and efficient way many problems. The work is already significant and only the high-level library has not yet been developed. Given the remaining work, the first official release is planned for late 2013.
Open vSwitch Offload: Conntrack and the Upstream KernelNetronome
Offloading all or part of the Open vSwitch datapath to SmartNICs has been shown to not only release CPU resources on the server, but improve traffic processing performance. Recently steps have been made to support such offloading in the upstream Linux kernel. This has focused on creating an OVS datapath using the TC flower filter and utilizing the offload hooks already present here. This presentation focuses on how Connection Tracking (Conntrack) may fit into this model. It describes current work being undertaken with the Netfilter community to allow offloading of Conntrack entries. It continues to link this work with the offloading of Conntrack rules within OVS-TC.
In this session, we’ll review how previous efforts, including Netfilter, Berkley Packet Filter (BPF), Open vSwitch (OVS), and TC, approached the problem of extensibility. We’ll show you an open source solution available within the Red Hat Enterprise Linux kernel, where extending and merging some of the existing concepts leads to an extensible framework that satisfies the networking needs of datacenter and cloud virtualization.
The Next Generation Firewall for Red Hat Enterprise Linux 7 RCThomas Graf
The Linux packet filtering technology, iptables, has its roots in times when networking was relatively simple and network bandwidth was measured in mere megabits. Emerging technologies, such as distributed NAT, overlay networks and containers require enhanced functionality and additional flexibility. In parallel, the next generation of network cards with speeds of 40Gb and 100Gb will put additional pressure on performance.
In the upcoming Red Hat Enterprise Linux 7, a new dynamic firewall service, FirewallD, is planned to provide greater flexibility over iptables by eliminating service disruptions during rule updates, abstraction, and support for different network trust zones. Additionally, a new virtual machine-based packet filtering technology, nftables, addresses the functionality and flexibility requirements of modern network workloads.
In this session you’ll:
Deep dive into the newly introduced packet filtering capabilities of Red Hat Enterprise Linux 7 beta.
Learn best practices.
See the new set of configuration utilities that allow new optimization possibilities.
Replacing iptables with eBPF in Kubernetes with CiliumMichal Rostecki
Cilium is an open source project which provides networking, security and load balancing for application services that are deployed using Linux container technologies by using the native eBPF technology in the Linux kernel. In this presentation we talked about:
- The evolution of the BPF filters and explained the advantages of eBPF Filters and its use cases today in Linux especially on how Cilium networking utilizes the eBPF Filters to secure the Kubernetes workload with increased performance when compared to legacy iptables.
- How Cilium uses SOCKMAP for layer 7 policy enforcement - How Cilium integrates with Istio and handles L7 Network Policies with Envoy Proxies.
- The new features since the last release such as running Kubernetes cluster without kube-proxy, providing clusterwide NetworkPolicies, providing fully distributed networking and security observability platform for cloud native workloads etc.
Open vSwitch Offload: Conntrack and the Upstream KernelNetronome
Offloading all or part of the Open vSwitch datapath to SmartNICs has been shown to not only release CPU resources on the server, but improve traffic processing performance. Recently steps have been made to support such offloading in the upstream Linux kernel. This has focused on creating an OVS datapath using the TC flower filter and utilizing the offload hooks already present here. This presentation focuses on how Connection Tracking (Conntrack) may fit into this model. It describes current work being undertaken with the Netfilter community to allow offloading of Conntrack entries. It continues to link this work with the offloading of Conntrack rules within OVS-TC.
In this session, we’ll review how previous efforts, including Netfilter, Berkley Packet Filter (BPF), Open vSwitch (OVS), and TC, approached the problem of extensibility. We’ll show you an open source solution available within the Red Hat Enterprise Linux kernel, where extending and merging some of the existing concepts leads to an extensible framework that satisfies the networking needs of datacenter and cloud virtualization.
The Next Generation Firewall for Red Hat Enterprise Linux 7 RCThomas Graf
The Linux packet filtering technology, iptables, has its roots in times when networking was relatively simple and network bandwidth was measured in mere megabits. Emerging technologies, such as distributed NAT, overlay networks and containers require enhanced functionality and additional flexibility. In parallel, the next generation of network cards with speeds of 40Gb and 100Gb will put additional pressure on performance.
In the upcoming Red Hat Enterprise Linux 7, a new dynamic firewall service, FirewallD, is planned to provide greater flexibility over iptables by eliminating service disruptions during rule updates, abstraction, and support for different network trust zones. Additionally, a new virtual machine-based packet filtering technology, nftables, addresses the functionality and flexibility requirements of modern network workloads.
In this session you’ll:
Deep dive into the newly introduced packet filtering capabilities of Red Hat Enterprise Linux 7 beta.
Learn best practices.
See the new set of configuration utilities that allow new optimization possibilities.
Replacing iptables with eBPF in Kubernetes with CiliumMichal Rostecki
Cilium is an open source project which provides networking, security and load balancing for application services that are deployed using Linux container technologies by using the native eBPF technology in the Linux kernel. In this presentation we talked about:
- The evolution of the BPF filters and explained the advantages of eBPF Filters and its use cases today in Linux especially on how Cilium networking utilizes the eBPF Filters to secure the Kubernetes workload with increased performance when compared to legacy iptables.
- How Cilium uses SOCKMAP for layer 7 policy enforcement - How Cilium integrates with Istio and handles L7 Network Policies with Envoy Proxies.
- The new features since the last release such as running Kubernetes cluster without kube-proxy, providing clusterwide NetworkPolicies, providing fully distributed networking and security observability platform for cloud native workloads etc.
Unifying Network Filtering Rules for the Linux Kernel with eBPFNetronome
At the core of fast network packet processing lies the ability to filter packets, or in other words, to apply a set of rules on packets, usually consisting of a pattern to match (L2 to L4 source and destination addresses and ports, protocols, etc.) and corresponding actions (redirect to a given queue, or drop the packet, etc.). Over the years, several filtering frameworks have been added to Linux. While at the lower level, ethtool can be used to configure N-tuple rules on the receive side for the hardware, the upper layers of the stack got equipped with rules for firewalling (Netfilter), traffic shaping (TC), or packet switching (Open vSwitch for example).
In this presentation, Quentin Monnet reviewed the needs for those filtering frameworks and the particularities of each one. Then focuses on the changes brought by eBPF and XDP in this landscape: as BPF programs allow for very flexible processing and can be attached very low in the stack—at the driver level, or even run on the NIC itself—they offer filtering capabilities with no precedent in terms of performance and versatility in the kernel. Lastly, the third part explores potential leads in order to create bridges between the different rule formats and to make it easier for users to build their filtering eBPF programs.
SOSCON 2019.10.17
What are the methods for packet processing on Linux? And how fast are each packet processing methods? In this presentation, we will learn how to handle packets on Linux (User space, socket filter, netfilter, tc), and compare performance with analysis of where each packet processing is done in the network stack (hook point). Also, we will discuss packet processing using XDP, an in-kernel fast-path recently added to the Linux kernel. eXpress Data Path (XDP) is a high-performance programmable network data-path within the Linux kernel. The XDP is located at the lowest level of access through SW in the network stack, the point at which driver receives the packet. By using the eBPF infrastructure at this hook point, the network stack can be expanded without modifying the kernel.
Daniel T. Lee (Hoyeon Lee)
@danieltimlee
Daniel T. Lee currently works as Software Engineer at Kosslab and contributing to Linux kernel BPF project. He has interest in cloud, Linux networking, and tracing technologies, and likes to analyze the kernel's internal using BPF technology.
Accelerating Envoy and Istio with Cilium and the Linux KernelThomas Graf
This talk will provide an introduction to injection options of Envoy and then deep dive into ongoing Linux kernel work that enables injecting Envoy while introducing as little latency as possible.
The servicemesh and the sidecar proxy model are on a steep trajectory to redefine many networking and security use cases. This talk explains and demos a new socket redirect Linux kernel technology that allows running Envoy with similar performance as if the sidecar was linked to the application using a UNIX domain socket. The talk will also give an outlook on how Envoy can use the recently merged kernel TLS functionality to gain access to the clear text payload transparently for end to end encrypted applications without requiring to decrypt and re-encrypt any data to further reduce the overhead and latency.
DPDK (Data Plane Development Kit) Overview by Rami Rosen
* Background and short history
* Advantages and disadvantages
- Very High speed networking acceleration in L2
- How this acceleration is achieved (hugepages, optimizations)
- rte_kni (and KCP)
- VPP (and FD.io project) , providing routing and switching.
- TLDK (Transport Layer Development Kit, TCP/UDP)
* Anatomy of a simple DPDK application.
* Development and governance model
* Testpmd: DPDK CLI tool
* DDP - Dynamic Device Profiles
Rami Rosen is a Linux Kernel expert, the author of "Linux Kernel Networking", Apress, 2014.
Rami had published two articles about DPDK in the last year:
"Network acceleration with DPDK"
https://lwn.net/Articles/725254/
"Userspace Networking with DPDK"
https://www.linuxjournal.com/content/userspace-networking-dpdk
Moved to https://speakerdeck.com/ebiken/zebra-srv6-cli-on-linux-dataplane-enog-number-49
Introduction to SRv6, Linux SRv6 implementation and how to add SRv6 CLI to Zebra 2.0 Open Source Network Operation Stack.
Presented at ENOG (Echigo NOG) #49.
USENIX LISA2021 talk by Brendan Gregg (https://www.youtube.com/watch?v=_5Z2AU7QTH4). This talk is a deep dive that describes how BPF (eBPF) works internally on Linux, and dissects some modern performance observability tools. Details covered include the kernel BPF implementation: the verifier, JIT compilation, and the BPF execution environment; the BPF instruction set; different event sources; and how BPF is used by user space, using bpftrace programs as an example. This includes showing how bpftrace is compiled to LLVM IR and then BPF bytecode, and how per-event data and aggregated map data are fetched from the kernel.
BPF of Berkeley Packet Filter mechanism was first introduced in linux in 1997 in version 2.1.75. It has seen a number of extensions of the years. Recently in versions 3.15 - 3.19 it received a major overhaul which drastically expanded it's applicability. This talk will cover how the instruction set looks today and why. It's architecture, capabilities, interface, just-in-time compilers. We will also talk about how it's being used in different areas of the kernel like tracing and networking and future plans.
Kernel Recipes 2014 - What’s new in nftables?Anne Nicolas
Nftables is a new packet filtering framework which aims at replacing iptables. It has been developed by the Netfilter team who wanted after 10 years of development to get rid of iptables. After a successful introduction in the 3.13 kernel and more than one year in vanilla kernel, nftables evolution has been important. If the main concepts did not change a lot of technical problem and improvement have been made.
This talk will do a summary of these changes describing the one being the most challenging at the technical level. It will alsod focus on the decisions and orientations chosen during the Netfilter workshop in July.
Eric Leblond, Stamus Networks
In Network Computing Prototype Using P4 at KSC/KREONET 2019Kentaro Ebisawa
Case Study of P4 applying to CAN (Control Area Network) data pre-processing using FPGA + Netcope P4 compiler.
Presented at KSC / KREONET WORKSHOP 2019 | DAY 1 Session 1: SDN/NFV/P4
http://www.ksc2019.re.kr/
Unifying Network Filtering Rules for the Linux Kernel with eBPFNetronome
At the core of fast network packet processing lies the ability to filter packets, or in other words, to apply a set of rules on packets, usually consisting of a pattern to match (L2 to L4 source and destination addresses and ports, protocols, etc.) and corresponding actions (redirect to a given queue, or drop the packet, etc.). Over the years, several filtering frameworks have been added to Linux. While at the lower level, ethtool can be used to configure N-tuple rules on the receive side for the hardware, the upper layers of the stack got equipped with rules for firewalling (Netfilter), traffic shaping (TC), or packet switching (Open vSwitch for example).
In this presentation, Quentin Monnet reviewed the needs for those filtering frameworks and the particularities of each one. Then focuses on the changes brought by eBPF and XDP in this landscape: as BPF programs allow for very flexible processing and can be attached very low in the stack—at the driver level, or even run on the NIC itself—they offer filtering capabilities with no precedent in terms of performance and versatility in the kernel. Lastly, the third part explores potential leads in order to create bridges between the different rule formats and to make it easier for users to build their filtering eBPF programs.
SOSCON 2019.10.17
What are the methods for packet processing on Linux? And how fast are each packet processing methods? In this presentation, we will learn how to handle packets on Linux (User space, socket filter, netfilter, tc), and compare performance with analysis of where each packet processing is done in the network stack (hook point). Also, we will discuss packet processing using XDP, an in-kernel fast-path recently added to the Linux kernel. eXpress Data Path (XDP) is a high-performance programmable network data-path within the Linux kernel. The XDP is located at the lowest level of access through SW in the network stack, the point at which driver receives the packet. By using the eBPF infrastructure at this hook point, the network stack can be expanded without modifying the kernel.
Daniel T. Lee (Hoyeon Lee)
@danieltimlee
Daniel T. Lee currently works as Software Engineer at Kosslab and contributing to Linux kernel BPF project. He has interest in cloud, Linux networking, and tracing technologies, and likes to analyze the kernel's internal using BPF technology.
Accelerating Envoy and Istio with Cilium and the Linux KernelThomas Graf
This talk will provide an introduction to injection options of Envoy and then deep dive into ongoing Linux kernel work that enables injecting Envoy while introducing as little latency as possible.
The servicemesh and the sidecar proxy model are on a steep trajectory to redefine many networking and security use cases. This talk explains and demos a new socket redirect Linux kernel technology that allows running Envoy with similar performance as if the sidecar was linked to the application using a UNIX domain socket. The talk will also give an outlook on how Envoy can use the recently merged kernel TLS functionality to gain access to the clear text payload transparently for end to end encrypted applications without requiring to decrypt and re-encrypt any data to further reduce the overhead and latency.
DPDK (Data Plane Development Kit) Overview by Rami Rosen
* Background and short history
* Advantages and disadvantages
- Very High speed networking acceleration in L2
- How this acceleration is achieved (hugepages, optimizations)
- rte_kni (and KCP)
- VPP (and FD.io project) , providing routing and switching.
- TLDK (Transport Layer Development Kit, TCP/UDP)
* Anatomy of a simple DPDK application.
* Development and governance model
* Testpmd: DPDK CLI tool
* DDP - Dynamic Device Profiles
Rami Rosen is a Linux Kernel expert, the author of "Linux Kernel Networking", Apress, 2014.
Rami had published two articles about DPDK in the last year:
"Network acceleration with DPDK"
https://lwn.net/Articles/725254/
"Userspace Networking with DPDK"
https://www.linuxjournal.com/content/userspace-networking-dpdk
Moved to https://speakerdeck.com/ebiken/zebra-srv6-cli-on-linux-dataplane-enog-number-49
Introduction to SRv6, Linux SRv6 implementation and how to add SRv6 CLI to Zebra 2.0 Open Source Network Operation Stack.
Presented at ENOG (Echigo NOG) #49.
USENIX LISA2021 talk by Brendan Gregg (https://www.youtube.com/watch?v=_5Z2AU7QTH4). This talk is a deep dive that describes how BPF (eBPF) works internally on Linux, and dissects some modern performance observability tools. Details covered include the kernel BPF implementation: the verifier, JIT compilation, and the BPF execution environment; the BPF instruction set; different event sources; and how BPF is used by user space, using bpftrace programs as an example. This includes showing how bpftrace is compiled to LLVM IR and then BPF bytecode, and how per-event data and aggregated map data are fetched from the kernel.
BPF of Berkeley Packet Filter mechanism was first introduced in linux in 1997 in version 2.1.75. It has seen a number of extensions of the years. Recently in versions 3.15 - 3.19 it received a major overhaul which drastically expanded it's applicability. This talk will cover how the instruction set looks today and why. It's architecture, capabilities, interface, just-in-time compilers. We will also talk about how it's being used in different areas of the kernel like tracing and networking and future plans.
Kernel Recipes 2014 - What’s new in nftables?Anne Nicolas
Nftables is a new packet filtering framework which aims at replacing iptables. It has been developed by the Netfilter team who wanted after 10 years of development to get rid of iptables. After a successful introduction in the 3.13 kernel and more than one year in vanilla kernel, nftables evolution has been important. If the main concepts did not change a lot of technical problem and improvement have been made.
This talk will do a summary of these changes describing the one being the most challenging at the technical level. It will alsod focus on the decisions and orientations chosen during the Netfilter workshop in July.
Eric Leblond, Stamus Networks
In Network Computing Prototype Using P4 at KSC/KREONET 2019Kentaro Ebisawa
Case Study of P4 applying to CAN (Control Area Network) data pre-processing using FPGA + Netcope P4 compiler.
Presented at KSC / KREONET WORKSHOP 2019 | DAY 1 Session 1: SDN/NFV/P4
http://www.ksc2019.re.kr/
Enabling accelerated networking - seminar by Enea at the Embedded Conference ...EneaSoftware
The open source revolution brings a wealth of features and functionality at a rapid growth, and industry leaders come together to shape standardized interfaces, protocols, and ways of working. ARM is such a player and drives several collaborative projects under the Linaro (http://www.linaro.org) umbrella. One such initiative is the ODP (http://www.opendataplane.org/) open-source, cross-platform set of application programming interfaces for the networking data plane.
Kernel Recipes 2017 - EBPF and XDP - Eric LeblondAnne Nicolas
Berkeley Packet Filter is an old friend for most people that deal with network under Linux. But its extended version eBPF is completely redefining the scope of usage and interaction with the kernel. It can indeed be used to instrument most parts of the kernel. This goes from network tracing to process or I/O monitoring.
This talk will provide an overview of eBPF, from concept to tools like BCC. It will then focus on XDP for eXtreme Data Path and the possible applications in term of networking provided by this new framework.
Eric Leblond, Stamus Network
DevOps Fest 2020. Даніель Яворович. Data pipelines: building an efficient ins...DevOps_Fest
Я розповім про досвід будування системи для роботи з великими даними на базі відкритої технологіі Apache Nifi та Kubernetes на прикладі аналізу ресурсів новин з використанням NLP аналізом.
(Embedded Linux Conference Europe 2014)
Linux uses many kind of embedded products. The products include not only consumer electronics but also control systems such as programmable logic controllers. There are many type of infrastructure systems and each system has different technical requirements. The requirements include not only real-time performance but also reliability-related functions. The infrastructure systems have to meet all the requirements. This presentation gives a summary of our study and development to adapt the Linux to infrastructure systems. Then we discuss the direction of future development. Please note, this presentation doesn't focus on a specific product.
Kernel Recipes 2015: Kernel packet capture technologiesAnne Nicolas
Sniffing through the ages
Capturing packets running on the wire to send them to a software doing analysis seems at first sight a simple tasks. But one has not to forget that with current network this can means capturing 30M packets per second. The objective of this talk is to show what methods and techniques have been implemented in Linux and how they have evolved over time.
The talk will cover AF_PACKET capture as well as PF_RING, dpdk and netmap. It will try to show how the various evolution of hardware and software have had an impact on the design of these technologies. Regarding software a special focus will be made on Suricata IDS which is implementing most of these capture methods.
Eric Leblond, Stamus Networks
Kernel Recipes 2014 - kGraft: Live Patching of the Linux KernelAnne Nicolas
The talk introduces the need of live kernel patching. Further, it explains what is kGraft, how it works, what are its limitations, and our plans with the implementation in the future. The presentation includes also a live demo if stars constellation allows.
Jiri Slaby, SUSE
Kernel Recipes 2019 - Driving the industry toward upstream firstAnne Nicolas
Wanting to avoid the Android experience, Google developers always aimed to make their Chrome OS Linux kernels as close to mainline as possible. However, when Chromebooks were first created, Google was left with no choice, the mainline kernel, in some subsystems, still did not have all the functionalities needed by Chromebooks. Hence, similarly to Android, Chrome OS had to develop their own out-of-tree code for the kernel and maintain that for a few different kernel versions.
Luckily, over the last few years a strong and consistent effort has been happening to bring Chromebook devices closer to mainline. It has led to significant improvements that now make it possible to run mainline on Chrome OS devices. And not only Chromebooks, as these significant strides are also improving Arm-based SOCs and other key components of the rich Chromebook hardware ecosystem. In this talk, we will look at how and why upstream support for Chromebooks improved, the current status of various models, and what we expect in the future.
Enric Balletbò i Serra
Kernel Recipes 2019 - No NMI? No Problem! – Implementing Arm64 Pseudo-NMIAnne Nicolas
As the name would suggest, a Non-Maskable Interrupt (NMI) is an interrupt-like feature that is unaffected by the disabling of classic interrupts. In Linux, NMIs are involved in some features such as performance event monitoring, hard-lockup detector, on demand state dumping, etc… Their potential to fire when least expected can fill the most seasoned kernel hackers with dread.
AArch64 (aka arm64 in the Linux tree) does not provide architected NMIs, a consequence being that features benefiting from NMIs see their use limited on AArch64. However, the Arm Generic Interrupt Controller (GIC) supports interrupt prioritization and masking, which, among other things, provides a way to control whether or not a set of interrupts can be signaled to a CPU.
This talk will cover how, using the GIC interrupt priorities, we provide a way to configure some interrupts to behave in an NMI-like manner on AArch64. We’ll discuss the implementation, some of the complications that ensued and also some of the benefits obtained from it.
Julien Thierry
Kernel Recipes 2019 - Hunting and fixing bugs all over the Linux kernelAnne Nicolas
At a rate of almost 9 changes per hour (24/7), the Linux kernel is definitely a scary beast. Bugs are introduced on a daily basis and, through the use of multiple code analyzers, *some* of them are detected and fixed before they hit mainline. Over the course of the last few years, Gustavo has been fixing such bugs and many different issues in every corner of the Linux kernel. Recently, he was in charge of leading the efforts to globally enable -Wimplicit-fallthrough; which appears by default in Linux v5.3. This presentation is a report on all the stuff Gustavo has found and fixed in the kernel with the support of the Core Infrastructure Initiative.
Gustavo A.R. Silva
Kernel Recipes 2019 - Metrics are moneyAnne Nicolas
In I.T. we all use all kinds of metrics. Operations teams rely heavily on these, especially when things go south. These metrics are sometimes overrated. Let’s dive into a few real life stories together.
Aurélien Rougemont
Kernel Recipes 2019 - Kernel documentation: past, present, and futureAnne Nicolas
The Linux kernel project includes a huge amount of documentation, but that information has seen little in the way of care over the
years. The amount of care has increased significantly recently, though, and things are improving quickly. Listen as the kernel’s documentation maintainer discusses the current state of the kernel’s docs, how we got here, where we’re trying to go, and how you can help.
Jonathan Corbet
Embedded Recipes 2019 - Knowing your ARM from your ARSE: wading through the t...Anne Nicolas
Modern SoC designs incorporate technologies from numerous vendors, each with their own inconsistent, confusing, undocumented and even contradictory terminology. The result is a mess of acronyms and product names which have a surprising impact on the ability to develop reusable, modular code thanks to the nature of the underlying IP being obscured.
This presentation will dive into some of the misnomers plaguing the Arm ecosystem, with the aim of explaining why things are like they are, how they fit together under the architectural umbrella and how you, as a developer, can decipher the baffling ingredients list of your next SoC design!
Will Deacon
Kernel Recipes 2019 - GNU poke, an extensible editor for structured binary dataAnne Nicolas
GNU poke is a new interactive editor for binary data. Not limited to editing basic ntities such as bits and bytes, it provides a full-fledged procedural, interactive programming language designed to describe data structures and to operate on them. Once a user has defined a structure for binary data (usually matching some file format) she can search, inspect, create, shuffle and modify abstract entities such as ELF relocations, MP3 tags, DWARF expressions, partition table entries, and so on, with primitives resembling simple editing of bits and bytes. The program comes with a library of already written descriptions (or “pickles” in poke parlance) for many binary formats.
GNU poke is useful in many domains. It is very well suited to aid in the development of programs that operate on binary files, such as assemblers and linkers. This was in fact the primary inspiration that brought me to write it: easily injecting flaws into ELF files in order to reproduce toolchain bugs. Also, due to its flexibility, poke is also very useful for reverse engineering, where the real structure of the data being edited is discovered by experiment, interactively. It is also good for the fast development of prototypes for programs like linkers, compressors or filters, and it provides a convenient foundation to write other utilities such as diff and patch tools for binary files.
This talk (unlike Gaul) is divided into four parts. First I will introduce the program and show what it does: from simple bits/bytes editing to user-defined structures. Then I will show some of the internals, and how poke is implemented. The third block will cover the way of using Poke to describe user data, which is to say the art of writing “pickles”. The presentation ends with a status of the project, a call for hackers, and a hint at future works.
Jose E. Marchesi
Kernel Recipes 2019 - Analyzing changes to the binary interface exposed by th...Anne Nicolas
Operating system distributors often face challenges that are somewhat different from that of upstream kernel developers. For instance, some kernel updates often need to stay at least binary compatible with modules that might be “out of tree” for some time.
In that context, being able to automatically detect and analyze changes to the binary interface exposed by the kernel to its module does have some noticeable value.
The Libabigail framework is capable of analyzing ELF binaries along with their accompanying debug info in the DWARF format, detect and report changes in types, functions, variables and ELF symbols. It has historically supported that for user space shared libraries and application so we worked to make it understand the Linux kernel
binaries.
In this presentation, we are going to present the current support of ABI analysis for Linux Kernel binaries, the challenges we face, how we address them and the plans we have for the future.
Dodji Seketeli, Jessica Yu, Matthias Männich
Embedded Recipes 2019 - Remote update adventures with RAUC, Yocto and BareboxAnne Nicolas
Different upgrade and update strategies exist when it comes to embedded Linux system. If at development time none of these strategies have been chosen, adding them afterwards can be tedious task.
Even harder it gets when the system is already deployed in the field and only accessible via a 3G connection.
This talk is a developer experience of putting in place exactly that. Giving a return of experience on one way of doing it on a system running Barebox and a Yocto-based distribution.
Patrick Boettcher
Embedded Recipes 2019 - Making embedded graphics less specialAnne Nicolas
Traditionally graphics drivers were one of the last hold-outs of proprietary software in an embedded Linux system. This situation is changing with open-source graphics drivers showing up for almost all of the graphics acceleration peripherals on the market right now. This talk will show how open-source graphics drivers are making embedded systems less special, as well as trying to provide an overview of the Linux graphics stack, de-mystifying what is often seen as black magic GPU stuff from outside observers.
Lucas Stach
Embedded Recipes 2019 - Linux on Open Source Hardware and Libre SiliconAnne Nicolas
This talk will explore Open Source Hardware projects relevant to Linux, including boards like BeagleBone, Olimex OLinuXino, Giant board and more. Looking at the benefits and challenges of designing Open Source Hardware for a Linux system, along with BeagleBoard.org’s experience of working with community, manufacturers, and distributors to create an Open Source Hardware platform. In closing also looking at the future, Libre Silicon like RISC-V designs, and where this might take Linux.
Drew Fustini
Embedded Recipes 2019 - From maintaining I2C to the big (embedded) pictureAnne Nicolas
The I2C subsystem is not the shiniest part of the Linux Kernel. For embedded devices, though, it is one of the many puzzle pieces which just have to work. Wolfram Sang has the experience of maintaining this subsystem for nearly 7 years now. This talk gives a short overview of how maintaining works in general and specifically in this subsystem. But mainly, it will highlight noteworthy points in the timeline and lessons learnt from that. It will present trends, not so much regarding I2C but more the Linux Kernel and the embedded ecosystem in general. And of course, there will be plenty of anecdotes and bits from behind the scenes for your entertainment.
Wolfram Sang
Embedded Recipes 2019 - Testing firmware the devops wayAnne Nicolas
ITRenew is selling recertified OCP servers under the Sesame brand, those servers come either with their original UEFI BIOS or with LinuxBoot. The LinuxBoot project is pushing the Linux kernel inside bios flash and using userland programs as bootloader.
To achieve quality on our software stack, as any project, we need to test it. Traditional BIOS are tested by hand, this is 2019 we need to do it automatically! We already presented the hardware setup behind the LinuxBoot CI, this talk will focus on the software.
We use u-root for our userland bootloader; this software is written in Go so we naturally choose to use Go for our testing too. We will present how we are using and extending the Go native test framework `go test` for testing embedded systems (serial console) and improving the report format for integration to a CI.
Julien Viard de Galbert
Embedded Recipes 2019 - Herd your socs become a matchmakerAnne Nicolas
About 60% of the Linux kernel source tree is devoted to drivers for a large variety of supported hardware components. Especially in the embedded world, the number of different SoC families, versions, and revisions, integrating a myriad of “IP cores”, keeps on growing.
In this presentation, Geert will explain how to match drivers against hardware, and how to support a wide variety of (dis)similar devices, without turning platform and driver code into an entangled bowl of spaghetti.tra
Starting with a brief history of driver matching in Linux, he will fast-forward to device-tree based matching. He will discuss ways to handle slight variations of the same hardware devices, and different SoC revisions, each with their own quirks and bugs. Finally, Geert will show best practices for evolving device drivers in a maintainable way, based on his experiences as an embedded Linux kernel developer and maintainer.
Geert Uytterhoeven
Embedded Recipes 2019 - LLVM / Clang integrationAnne Nicolas
Buildroot is a popular and easy to use embedded Linux build system. It generates, in few minutes, lightweight and customized Linux systems, including the cross-compilation toolchain, kernel and bootloader images, as well as a wide variety of userspace libraries and programs.
This talk is about the integration of LLVM/clang into Buildroot.
In 2018, Valentin Korenblit, supervised by Romain Naour, worked on this topic during his internship at Smile ECS. After a short introduction about llvm/clang and Buildroot, this talk will go through the numerous issues discovered while adding llvm/clang componants and how these issues were fixed. Romain will also detail the work in progress and the work to be done based on llvm/clang libraries (OpenCL, Compiler-rt, BCC. Chromium, ldd).
Romain Naour
Embedded Recipes 2019 - Introduction to JTAG debuggingAnne Nicolas
This talk introduces JTAG debugging capabilities, both for debugging hardware and software. Marek first explains what the JTAG stands for and explains the operation of the JTAG state machine. This is followed by an introduction to free software JTAG tools, OpenOCD and urJTAG. Marek shortly explains how to debug software using those tools and how that ties into the JTAG state machine. However, JTAG was designed for testing hardware. Marek explains what boundary scan testing (BST) is, what are BSDL files and their format, and practically demonstrates how to blink an LED using BST and only free software tools.
Marek Vasut
Embedded Recipes 2019 - Pipewire a new foundation for embedded multimediaAnne Nicolas
PipeWire is an open source project that aims to greatly improve audio and video handling under Linux. Utilising a fresh design, it bridges use cases that have been previously addressed by different tools – or not addressed at all -, providing ground for building complex, yet secure and efficient, multimedia systems.
In this talk, Julien is going to present the PipeWire project and the concepts that make up its design. In addition, he is going to give an update of the current and future work going on around PipeWire, both upstream and in Automotive Grade Linux, an early adopter that Julien is actively working on.
Julian Bouzas
Kernel Recipes 2019 - ftrace: Where modifying a running kernel all startedAnne Nicolas
Ftrace’s most powerful feature is the function tracer (and function graph tracer which is built from it). But to have this enabled on production systems, it had to have its overhead be negligible when disabled. As the function tracer uses gcc’s profiling mechanism, which adds a call to “mcount” (or more recently fentry, don’t worry if you don’t know what this is, it will all be explained) at the start of almost all functions, it had to do something about the overhead that causes. The solution was to turn those calls into “nops” (an instruction that the CPU simply ignores). But this was no easy feat. It took a lot to come up with a solution (and also turning a few network cards into bricks). This talk will explain the history of how ftrace came about implementing the function tracer, and brought with it the possibility of static branches and soon static calls!
Steven Rostedt
Kernel Recipes 2019 - Suricata and XDPAnne Nicolas
Suricata is a network threat detection engine using network packets capture to reconstruct the traffic till the application layer and find threats on the network using rules that define behavior to detect. This task is really CPU intensive and discarding non interesting traffic is a solution to enable a scaling of Suricata to 40gbps and other.
This talk will present the latest evolution of Suricata that knows uses eBPF and XDP to bypass traffic. Suricata 5.0 is supporting the hardware XDP to provide ypass with network card such as Netronome. It also takes advantage of pinned maps to get persistance of the bypassed flows. This talk will cover the different usage of XDP and eBPF in Suricata and shows how it impact performance and usability. If development time permit, the talk will also cover AF_XDP and the impact on this new capture method on Suricata.
Eric Leblond
Kernel Recipes 2019 - Marvels of Memory Auto-configuration (SPD)Anne Nicolas
System memory configuration is a transparent operation nowadays, something that we all came to expect to just work out of the box. Still, it does happen behind the scenes every single time we boot our computers. This requires the cooperation of hardware components on the mainboard and on memory modules themselves, as well as firmware code to drive these. While it is possible to just let it happen, having a deeper understanding of how it works makes it possible to access valuable information from the operating system at run-time.
I will take you through the history of system memory configuration from the mid 70s to now. We will explore the different types of memory modules, how their configuration data is stored and how the firmware can access them. We will see which problems had to be solved along the way and how they were solved. Lastly we will see how Linux supports reading the memory configuration information and what you can do with that information.
Jean Delvare
A tale of scale & speed: How the US Navy is enabling software delivery from l...sonjaschweigert1
Rapid and secure feature delivery is a goal across every application team and every branch of the DoD. The Navy’s DevSecOps platform, Party Barge, has achieved:
- Reduction in onboarding time from 5 weeks to 1 day
- Improved developer experience and productivity through actionable findings and reduction of false positives
- Maintenance of superior security standards and inherent policy enforcement with Authorization to Operate (ATO)
Development teams can ship efficiently and ensure applications are cyber ready for Navy Authorizing Officials (AOs). In this webinar, Sigma Defense and Anchore will give attendees a look behind the scenes and demo secure pipeline automation and security artifacts that speed up application ATO and time to production.
We will cover:
- How to remove silos in DevSecOps
- How to build efficient development pipeline roles and component templates
- How to deliver security artifacts that matter for ATO’s (SBOMs, vulnerability reports, and policy evidence)
- How to streamline operations with automated policy checks on container images
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf91mobiles
91mobiles recently conducted a Smart TV Buyer Insights Survey in which we asked over 3,000 respondents about the TV they own, aspects they look at on a new TV, and their TV buying preferences.
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
SAP Sapphire 2024 - ASUG301 building better apps with SAP Fiori.pdfPeter Spielvogel
Building better applications for business users with SAP Fiori.
• What is SAP Fiori and why it matters to you
• How a better user experience drives measurable business benefits
• How to get started with SAP Fiori today
• How SAP Fiori elements accelerates application development
• How SAP Build Code includes SAP Fiori tools and other generative artificial intelligence capabilities
• How SAP Fiori paves the way for using AI in SAP apps
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
PHP Frameworks: I want to break free (IPC Berlin 2024)Ralf Eggert
In this presentation, we examine the challenges and limitations of relying too heavily on PHP frameworks in web development. We discuss the history of PHP and its frameworks to understand how this dependence has evolved. The focus will be on providing concrete tips and strategies to reduce reliance on these frameworks, based on real-world examples and practical considerations. The goal is to equip developers with the skills and knowledge to create more flexible and future-proof web applications. We'll explore the importance of maintaining autonomy in a rapidly changing tech landscape and how to make informed decisions in PHP development.
This talk is aimed at encouraging a more independent approach to using PHP frameworks, moving towards a more flexible and future-proof approach to PHP development.
Generative AI Deep Dive: Advancing from Proof of Concept to ProductionAggregage
Join Maher Hanafi, VP of Engineering at Betterworks, in this new session where he'll share a practical framework to transform Gen AI prototypes into impactful products! He'll delve into the complexities of data collection and management, model selection and optimization, and ensuring security, scalability, and responsible use.
How world-class product teams are winning in the AI era by CEO and Founder, P...
Kernel Recipes 2013 - Nftables, what motivations and what solutions
1. nftables, far more than %s/ip/nf/g
Éric Leblond
Nefilter Coreteam
September 24, 2013
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 1 / 48
2. 1 Introduction
2 Netfilter in 2013
3 Iptables limitations
4 Nftables, an Iptables replacement
5 Advantages of the approach
6 An updated user experience
7 Conclusion
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 2 / 48
3. Éric Leblond
Hacker and contractor
Independant Open Source and Security consultant
Started and developped NuFW, the authenticating firewall
Core developer of Suricata IDS/IPS
Netfilter Coreteam member
Work on kernel-userspace interaction
Kernel hacking
ulogd2 maintainer
Port of Openoffice firewall to Libreoffice
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 4 / 48
4. History
ipchains (1997)
Linux 2.2 firewalling
stateless
Developped by Paul ’Rusty’ Russel
iptables (2000)
Linux 2.4 firewalling
Stateful tracking and full NAT support
in-extremis IPv6 support
Netfilter project
’Rusty’ Russel developed iptables and funded Netfilter project
Netfilter coreteam was created to consolidate the community
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 6 / 48
5. Features
Filtering and logging
Filtering
on protocol fields
on internal state
Packet mangling
Change TOS
Change TTL
Set mark
Connection tracking
Stateful filtering
Helper to support protocol like FTP
Network Address Translation
Destination Network Address Translation
Source Network Address Translation
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 7 / 48
6. Netfilter inside kernel
Hooks
Hooks at different points of network stack
Verdict can be issued and skb can be modified
To each hook correspond at least table
Different families
filter
raw
nat
mangle
Loading a module create the table
Connection tracking tasks
Maintain a hash table with known flows
Detect dynamic connection opening for some protocols
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 8 / 48
7. Major components
Netfilter filtering
In charge of accepting, blocking, transforming packets
Configured by ioctl
Connection tracking
Analyse traffic and maintain flow table
Cost in term of performance
Increase security
iptables
Configuration tools
Update ruleset inside kernel
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 9 / 48
8. The nfnetlink (r)evolution
Nfnetlink
First major evolution of Netfilter (Linux 2.6.14, 2005)
Netfilter dedicated configuration and message passing
mechanism
New interactions
NFLOG: enhanced logging system
NFQUEUE: improved userspace decision system
NFCT: get information and update connection tracking entries
Based on Netlink
datagram-oriented messaging system
passing messages from kernel to user-space and vice-versa
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 10 / 48
10. Components created following 2.6.14
conntrack-tools
conntrackd
connection tracking replication daemon
provide high availability
developped by Pablo Neira Ayuso
conntrack: command line tool to update and query connection
tracking
ulogd2
logging daemon
handle packets and connections logging
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 12 / 48
11. Latest changes
ipset
Efficient set handling
Address list or more complex set
Reach vanilla kernel in 2011 (Linux 2.6.39)
nfacct
Efficient accounting system
Appeared in 2012
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 13 / 48
12. Kernel code
How much code
70000 LOC reside in kernelspace
around 50000 LOC in user-space
Iptables extensions
111 iptables extensions.
Various tasks:
tcp
cluster
bpf
statistic
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 14 / 48
13. Performance
Adding a rule The problem
Atomic replacement
of ruleset
Sent from kernel
to userspace
Modified and
sent back by
userspace
Huge performance
impact
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 16 / 48
14. Dynamic ruleset
Network gets dynamic
Firewall can’t be static anymore
Cloud
IP reputation
Combinatory explosion : one rule per-server and protocol
Set handling
Set handling is made via ipset
Efficient but not as integrated as possible
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 17 / 48
15. Code duplication
Different filtering family
Netfilter classic filtering
Brigde filtering
Arp filtering
IPv4 and IPv6
Matches and target
Similar code in numerous Netfilter module
Nothing is shared
Manual parsing
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 18 / 48
16. Problem due to binary blob usage
ABI breakage
Binary exchange between userspace and kernel
No modification possible without touching kernel
Trusting userspace
Kernel is parsing a binary blob
Possible to break the internal parser
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 19 / 48
17. Integration via exec
Frontend and iptables
No officially available library
Frontend fork iptables command
libiptables
Available inside iptables sources
Not a public library
API and ABI breakage are not checked during version upgrade
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 20 / 48
18. Lack of flexible table and chains configurations
Module loading is the key
Chains are created when module init
Induce a performance cost even without rules
No configuration is possible
Chains are hardcoded
FORWARD is created on a server
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 21 / 48
19. Nftables
A new filtering system
Replace iptables and the filtering infrastructure
No changes in
Hooks
Connection tracking
Helpers
A new language
Based on a grammar
Accessible from a library
Netlink based communication
Atomic modification
Notification system
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 23 / 48
20. History
Introduced in 2008
Developped and
presented by Patrick
McHardy at
NFWS2008
Presentation took 3
hours
Alpha stage in 2008
Development did stop
Patrick McHardy did not finish the code alone
Nobody did join the effort
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 24 / 48
22. What explanations ?
Should have "Release often release early" ?
Started by Patrick McHardy only
Almost complete work presented during NFWS 2008
Complex to enter the project
Too early ?
No user were demanding for that explicitly
Ipset was available and fixing the set issue
Solution for dynamic handling was sufficient
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 26 / 48
23. Development restarted in 2012
Funding by Sophos/Astaro
Pablo Neira Ayuso get funded by Astaro
Work restart in 2012
Gaining momemtum
Tomasz Bursztyka joined the development team
Work on Connman
Lack of libs was painful to him
Start to hack on nftables
Google summer of code
3 students
Some good results
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 27 / 48
24. A filtering based on a pseudo-state machine
Inspired by BPF
4 registers
1 verdict
A extensive instructions set
Add Some Magic ?
reg = pkt.payload[offset, len]
reg = cmp(reg1, reg2, EQ)
reg = pkt.meta(mark)
reg = lookup(set, reg1)
reg = ct(reg1, state)
Easy creation of new matches
reg1 = pkt.payload[offset_src_port, len]
reg2 = pkt.payload[offset_dst_port, len]
reg = cmp(reg1, reg2, EQ)
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 28 / 48
25. Architecture
Kernel
Tables: declared by user and attached to hook
User interface: nfnetlink socket
ADD
DELETE
DUMP
Userspace
libmnl: low level netlink interaction
libnftables: library handling low-level interaction with nftables
Netlink’s API
nftables: command line utility to maintain ruleset
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 29 / 48
26. Dynamic chain loading
Chain are created on-demand
Chain are created via a specific netlink message
Non-user chain are:
Of a specific type
Bound to a given hook
Current chain type
filter: filtering table
route: old mangle table
nat: network address translation table
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 30 / 48
27. From userspace syntax to kernel
Converting user input
Operation is made via a netlink message
The userspace syntax must be converted
From a text message following a grammar
To a binary Netlink message
Linearize
Tokenisation
Parsing
Evaluation
Linearization
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 31 / 48
28. From kernel to userspace syntax
Kernel send netlink message
It must be converted back to text
Conversion
Deliniearization
Postprocessing
Textify
Example
ip f i l t e r output 8 7
[ payload load 4b @ network header + 16 => reg 1 ]
[ bitwise reg 1 = ( reg=1 & 0 x 0 0 f f f f f f ) ^ 0x00000000 ]
[ cmp eq reg 1 0x00500fd9 ]
[ counter pkts 7 bytes 588 ]
is translated to:
ip daddr 217.15.80.0/24 counter packets 7 bytes 588 # handle 8
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 32 / 48
29. Kernel
Atomic ruleset update
atomically commit a set of rule-set updates incrementally
based on a generation counter/mask
00 active in the present, will be active in the next generation.
01 active in the present, needs to zero its future, it becomes 00.
10 inactive in the present, delete now.
xtables compatibility
Possible to use old extensions
Necessary to provide backward compatibility
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 33 / 48
30. Notification
Event based notification
Each rule update trigger an event
Event is sent to userspace via nfnetlink
Userspace usage
Implemented in libnftables
Program can update his view on the ruleset without dump
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 34 / 48
31. A limited in-kernel size
A limited set of operators and instructions
A state machine
No code dedicated to each match
One match on address use same code as a match on port
New matchs are possible without kernel modification
LOC count
50000 LOC in userspace
only 7000 LOC in kernel-space
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 36 / 48
32. Less kernel update
Pseudo state machine instruction
Current instructions cover need found in previous 10 years
New instruction require very limited code
Development in userspace
A new match will not need a new kernel
ICMPv6 implementation is a single userspace patch
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 37 / 48
33. Example of ICMPv6
include/datatype.h | 2 ++
include/payload.h | 14 +++++++++++
src/parser.y | 33 +++++++++++++++++++++++++++---
src/payload.c | 59 +++++++++++++++++++++++++++++++++++++++++++++++++
src/scanner.l | 4 ++
5 files changed, 109 insertions(+), 3 deletions(-)
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 38 / 48
35. Basic utilisation
File mode
nft -f ipv4-filter
Command line mode
nft add rule ip f i l t e r input tcp dport 80 drop
nft l i s t table f i l t e r −a
nft delete rule f i l t e r output handle 10
CLI mode
# nft −i
nft > l i s t table
< c l i >:1:12 −12: Error : syntax error , unexpected end of f i l e , expecting s t r i
l i s t table
^
nft > l i s t table f i l t e r
table f i l t e r {
chain input {
ip saddr 1.2.3.4 counter packets 8 bytes 273
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 41 / 48
36. Set handling
Interests of sets
One single rule evaluation
Simple and readable ruleset
Evolution handling
Anonymous set
nft add rule ip global f i l t e r
ip daddr {192.168.0.0/24 , 192.168.1.4}
tcp dport {22 , 443}
accept
Named set
nft add set global ipv4_ad { type ipv4_address ; }
nft add element global ipv4_ad { 192.168.1.4 , 192.168.1.5}
nft delete element global ipv4_ad { 192.168.1.5}
nft add rule ip global f i l t e r ip saddr @ipv4_ad drop
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 42 / 48
37. Mapping
Principle and interest
Associative mapping linking two notions
A match on the key trigger the use of the value
Using addresses, interfaces, verdicts
Examples
Anonymous mapping:
# nft add rule f i l t e r output ip daddr vmap
{192.168.0.0/24 => drop , 192.168.0.1 => accept }
Named mapping:
# nft −i
nft > add map f i l t e r verdict_map { type ipv4_address => v e r d i c t ; }
nft > add element f i l t e r verdict_map { 1.2.3.5 => drop }
nft > add rule f i l t e r output ip daddr vmap @verdict_map
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 43 / 48
38. Usage example
set web_servers {
type ipv4_address
elements = { 192.168.1.15 , 192.168.1.5}
}
map admin_map {
type ipv4_address => v e r d i c t
elements = { 192.168.0.44 => jump logmetender ,
192.168.0.42 => jump logmetrue , 192.168.0.33 => accept }
}
chain forward {
ct state established accept
ip daddr @web_servers tcp dport ssh ip saddr map @admin_map
ip daddr @web_servers tcp dport http log accept
ip daddr @web_servers tcp dport https accept
counter log drop
}
chain logmetender {
log l i m i t 10/ minute accept
}
chain logmetrue {
counter log accept
}
}
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 44 / 48
39. Transition and evolution
A complete iptables compatibility
iptables-nftables
Binary compatible with iptables
Using nftables framework
Same kernel can be used with two systems
A progressive update
A high level library
To be used by frontends
Or by network manager systems
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 45 / 48
40. Conclusion
A huge evolution
Solving iptables problem
An answer to new usages
Set handling
Complex matches
Availability for end 2013, beginning 2014
Finalizing iptables compatibility
High level library
Debug and some functionalities
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 47 / 48
41. Questions ?
Do you have questions ? Thanks to
Netfilter team
Astaro/Sophos for
financing the development
Google for GSoC 2013
More information
Netfilter :
http://www.netfilter.org
Nftables quick & dirty :
https://t.co/cM4zogob8t
Contact me
Mail:
eric@regit.org
Twitter: @Regiteric
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 48 / 48