Agenda:
In this session, Shmulik Ladkani discusses the kernel's net_device abstraction, its interfaces, and how net-devices interact with the network stack. The talk covers many of the software network devices that exist in the Linux kernel, the functionalities they provide and some interesting use cases.
Speaker:
Shmulik Ladkani is a Tech Lead at Ravello Systems.
Shmulik started his career at Jungo (acquired by NDS/Cisco) implementing residential gateway software, focusing on embedded Linux, Linux kernel, networking and hardware/software integration.
51966 coffees and billions of forwarded packets later, with millions of homes running his software, Shmulik left his position as Jungo’s lead architect and joined Ravello Systems (acquired by Oracle) as tech lead, developing a virtual data center as a cloud service. He's now focused around virtualization systems, network virtualization and SDN.
netfilter is a framework provided by the Linux kernel that allows various networking-related operations to be implemented in the form of customized handlers.
iptables is a user-space application program that allows a system administrator to configure the tables provided by the Linux kernel firewall (implemented as different netfilter modules) and the chains and rules it stores.
Many systems use iptables/netfilter, Linux's native packet filtering/mangling framework since Linux 2.4, be it home routers or sophisticated cloud network stacks.
In this session, we will talk about the netfilter framework and its facilities, explain how basic filtering and mangling use-cases are implemented using iptables, and introduce some less common but powerful extensions of iptables.
Shmulik Ladkani, Chief Architect at Nsof Networks.
Long time network veteran and kernel geek.
Shmulik started his career at Jungo (acquired by NDS/Cisco) implementing residential gateway software, focusing on embedded Linux, Linux kernel, networking and hardware/software integration.
Some billions of forwarded packets later, Shmulik left his position as Jungo's lead architect and joined Ravello Systems (acquired by Oracle) as tech lead, developing a virtual data center as a cloud-based service, focusing around virtualization systems, network virtualization and SDN.
Recently he co-founded Nsof Networks, where he's been busy architecting network infrastructure as a cloud-based service, gazing at internet routes in astonishment, and playing the chkuku.
Linux offers an extensive selection of programmable and configurable networking components from traditional bridges, encryption, to container optimized layer 2/3 devices, link aggregation, tunneling, several classification and filtering languages all the way up to full SDN components. This talk will provide an overview of many Linux networking components covering the Linux bridge, IPVLAN, MACVLAN, MACVTAP, Bonding/Team, OVS, classification & queueing, tunnel types, hidden routing tricks, IPSec, VTI, VRF and many others.
LinuxCon 2015 Linux Kernel Networking WalkthroughThomas Graf
This presentation features a walk through the Linux kernel networking stack for users and developers. It will cover insights into both, existing essential networking features and recent developments and will show how to use them properly. Our starting point is the network card driver as it feeds a packet into the stack. We will follow the packet as it traverses through various subsystems such as packet filtering, routing, protocol stacks, and the socket layer. We will pause here and there to look into concepts such as networking namespaces, segmentation offloading, TCP small queues, and low latency polling and will discuss how to configure them.
High-Performance Networking Using eBPF, XDP, and io_uringScyllaDB
In the networking world there are a number of ways to increase performance over naive use of basic Berkeley sockets. These techniques have ranged from polling blocking sockets, non-blocking sockets controlled by Epoll, all the way through completely bypassing the Linux kernel for maximum network performance where you talk directly to the network interface card by using something like DPDK or Netmap. All these tools have their place, and generally occupy a space from convenience to performance. But in recent years, that landscape has changed massively.. The tools available to the average Linux systems developer have improved from the creation of io_uring, to the expansion of bpf from a simple filtering language to a full-on programming environment embedded directly in the kernel. Along with that came something called XDP (express datapath). This was Linux kernel's answer to kernel-bypass networking. AF_XDP is the new socket type created by this feature, and generally works very similarly to something like DPDK. History lessons out of the way, this talk will look into, and discuss the merits of this technology, it's place in the broader ecosystem and how it can be used to attain the highest level of performance possible. This talk will dive into crucial details, such as how AF_XDP works, how it can be integrated into a larger system and finally more advanced topics such as request sharding/load balancing. There will be detailed look at the design of AF_XDP, the eBpf code used, as well as the userspace code required to drive it all. It will also include performance numbers from this setup compared to regular kernel networking. And most importantly how to put all this together to handle as much data as possible on a single modern multi-core system.
In this talk Jiří Pírko discusses the design and evolution of the VLAN implementation in Linux, the challenges and pitfalls as well as hardware acceleration and alternative implementations.
Jiří Pírko is a major contributor to kernel networking and the creator of libteam for link aggregation.
Agenda:
In this session, Shmulik Ladkani discusses the kernel's net_device abstraction, its interfaces, and how net-devices interact with the network stack. The talk covers many of the software network devices that exist in the Linux kernel, the functionalities they provide and some interesting use cases.
Speaker:
Shmulik Ladkani is a Tech Lead at Ravello Systems.
Shmulik started his career at Jungo (acquired by NDS/Cisco) implementing residential gateway software, focusing on embedded Linux, Linux kernel, networking and hardware/software integration.
51966 coffees and billions of forwarded packets later, with millions of homes running his software, Shmulik left his position as Jungo’s lead architect and joined Ravello Systems (acquired by Oracle) as tech lead, developing a virtual data center as a cloud service. He's now focused around virtualization systems, network virtualization and SDN.
netfilter is a framework provided by the Linux kernel that allows various networking-related operations to be implemented in the form of customized handlers.
iptables is a user-space application program that allows a system administrator to configure the tables provided by the Linux kernel firewall (implemented as different netfilter modules) and the chains and rules it stores.
Many systems use iptables/netfilter, Linux's native packet filtering/mangling framework since Linux 2.4, be it home routers or sophisticated cloud network stacks.
In this session, we will talk about the netfilter framework and its facilities, explain how basic filtering and mangling use-cases are implemented using iptables, and introduce some less common but powerful extensions of iptables.
Shmulik Ladkani, Chief Architect at Nsof Networks.
Long time network veteran and kernel geek.
Shmulik started his career at Jungo (acquired by NDS/Cisco) implementing residential gateway software, focusing on embedded Linux, Linux kernel, networking and hardware/software integration.
Some billions of forwarded packets later, Shmulik left his position as Jungo's lead architect and joined Ravello Systems (acquired by Oracle) as tech lead, developing a virtual data center as a cloud-based service, focusing around virtualization systems, network virtualization and SDN.
Recently he co-founded Nsof Networks, where he's been busy architecting network infrastructure as a cloud-based service, gazing at internet routes in astonishment, and playing the chkuku.
Linux offers an extensive selection of programmable and configurable networking components from traditional bridges, encryption, to container optimized layer 2/3 devices, link aggregation, tunneling, several classification and filtering languages all the way up to full SDN components. This talk will provide an overview of many Linux networking components covering the Linux bridge, IPVLAN, MACVLAN, MACVTAP, Bonding/Team, OVS, classification & queueing, tunnel types, hidden routing tricks, IPSec, VTI, VRF and many others.
LinuxCon 2015 Linux Kernel Networking WalkthroughThomas Graf
This presentation features a walk through the Linux kernel networking stack for users and developers. It will cover insights into both, existing essential networking features and recent developments and will show how to use them properly. Our starting point is the network card driver as it feeds a packet into the stack. We will follow the packet as it traverses through various subsystems such as packet filtering, routing, protocol stacks, and the socket layer. We will pause here and there to look into concepts such as networking namespaces, segmentation offloading, TCP small queues, and low latency polling and will discuss how to configure them.
High-Performance Networking Using eBPF, XDP, and io_uringScyllaDB
In the networking world there are a number of ways to increase performance over naive use of basic Berkeley sockets. These techniques have ranged from polling blocking sockets, non-blocking sockets controlled by Epoll, all the way through completely bypassing the Linux kernel for maximum network performance where you talk directly to the network interface card by using something like DPDK or Netmap. All these tools have their place, and generally occupy a space from convenience to performance. But in recent years, that landscape has changed massively.. The tools available to the average Linux systems developer have improved from the creation of io_uring, to the expansion of bpf from a simple filtering language to a full-on programming environment embedded directly in the kernel. Along with that came something called XDP (express datapath). This was Linux kernel's answer to kernel-bypass networking. AF_XDP is the new socket type created by this feature, and generally works very similarly to something like DPDK. History lessons out of the way, this talk will look into, and discuss the merits of this technology, it's place in the broader ecosystem and how it can be used to attain the highest level of performance possible. This talk will dive into crucial details, such as how AF_XDP works, how it can be integrated into a larger system and finally more advanced topics such as request sharding/load balancing. There will be detailed look at the design of AF_XDP, the eBpf code used, as well as the userspace code required to drive it all. It will also include performance numbers from this setup compared to regular kernel networking. And most importantly how to put all this together to handle as much data as possible on a single modern multi-core system.
In this talk Jiří Pírko discusses the design and evolution of the VLAN implementation in Linux, the challenges and pitfalls as well as hardware acceleration and alternative implementations.
Jiří Pírko is a major contributor to kernel networking and the creator of libteam for link aggregation.
The Linux kernel is undergoing the most fundamental architecture evolution in history and is becoming a microkernel. Why is the Linux kernel evolving into a microkernel? The potentially biggest fundamental change ever happening to the Linux kernel. This talk covers how companies like Facebook and Google use BPF to patch 0-day exploits, how BPF will change the way features are added to the kernel forever, and how BPF is introducing a new type of application deployment method for the Linux kernel.
SOSCON 2019.10.17
What are the methods for packet processing on Linux? And how fast are each packet processing methods? In this presentation, we will learn how to handle packets on Linux (User space, socket filter, netfilter, tc), and compare performance with analysis of where each packet processing is done in the network stack (hook point). Also, we will discuss packet processing using XDP, an in-kernel fast-path recently added to the Linux kernel. eXpress Data Path (XDP) is a high-performance programmable network data-path within the Linux kernel. The XDP is located at the lowest level of access through SW in the network stack, the point at which driver receives the packet. By using the eBPF infrastructure at this hook point, the network stack can be expanded without modifying the kernel.
Daniel T. Lee (Hoyeon Lee)
@danieltimlee
Daniel T. Lee currently works as Software Engineer at Kosslab and contributing to Linux kernel BPF project. He has interest in cloud, Linux networking, and tracing technologies, and likes to analyze the kernel's internal using BPF technology.
DPDK (Data Plane Development Kit) Overview by Rami Rosen
* Background and short history
* Advantages and disadvantages
- Very High speed networking acceleration in L2
- How this acceleration is achieved (hugepages, optimizations)
- rte_kni (and KCP)
- VPP (and FD.io project) , providing routing and switching.
- TLDK (Transport Layer Development Kit, TCP/UDP)
* Anatomy of a simple DPDK application.
* Development and governance model
* Testpmd: DPDK CLI tool
* DDP - Dynamic Device Profiles
Rami Rosen is a Linux Kernel expert, the author of "Linux Kernel Networking", Apress, 2014.
Rami had published two articles about DPDK in the last year:
"Network acceleration with DPDK"
https://lwn.net/Articles/725254/
"Userspace Networking with DPDK"
https://www.linuxjournal.com/content/userspace-networking-dpdk
Pushing Packets - How do the ML2 Mechanism Drivers Stack UpJames Denton
Architecting a private cloud to meet the use cases of its users can be a daunting task. How do you determine which of the many L2/L3 Neutron plugins and drivers to implement? Does network performance outweigh reliability? Are overlay networks just as performant as VLAN networks? The answers to these questions will drive the appropriate technology choice.
In this presentation, we will look at many of the common drivers built around the ML2 framework, including LinuxBridge, OVS, OVS+DPDK, SR-IOV, and more, and will provide performance data to help drive decisions around selecting a technology that's right for the situation. We will discuss our experience with some of these technologies, and the pros and cons of one technology over another in a production environment.
Linux 4.x Tracing: Performance Analysis with bcc/BPFBrendan Gregg
Talk about bcc/eBPF for SCALE15x (2017) by Brendan Gregg. "BPF (Berkeley Packet Filter) has been enhanced in the Linux 4.x series and now powers a large collection of performance analysis and observability tools ready for you to use, included in the bcc (BPF Complier Collection) open source project. BPF nowadays can do system tracing, software defined networks, and kernel fast path: much more than just filtering packets! This talk will focus on the bcc/BPF tools for performance analysis, which make use of other built in Linux capabilities: dynamic tracing (kprobes and uprobes) and static tracing (tracepoints and USDT). There are now bcc tools for measuring latency distributions for file system I/O and run queue latency, printing details of storage device I/O and TCP retransmits, investigating blocked stack traces and memory leaks, and a whole lot more. These lead to performance wins large and small, especially when instrumenting areas that previously had zero visibility. Tracing superpowers have finally arrived, built in to Linux."
Linux のネットワーク設定情報を取得し、描画するツール "plotnetcfg" のインストール方法や描画サンプル。
"How to" for installing / using "plotnetcfg" which scans networking config of Linux machine and plots a diagram of the configuration hierarchy.
Tutorial: Using GoBGP as an IXP connecting routerShu Sugimoto
- Show you how GoBGP can be used as a software router in conjunction with quagga
- (Tutorial) Walk through the setup of IXP connecting router using GoBGP
DockerCon 2017 - Cilium - Network and Application Security with BPF and XDPThomas Graf
This talk will start with a deep dive and hands on examples of BPF, possibly the most promising low level technology to address challenges in application and network security, tracing, and visibility. We will discuss how BPF evolved from a simple bytecode language to filter raw sockets for tcpdump to the a JITable virtual machine capable of universally extending and instrumenting both the Linux kernel and user space applications. The introduction is followed by a concrete example of how the Cilium open source project applies BPF to solve networking, security, and load balancing for highly distributed applications. We will discuss and demonstrate how Cilium with the help of BPF can be combined with distributed system orchestration such as Docker to simplify security, operations, and troubleshooting of distributed applications.
Using the new extended Berkley Packet Filter capabilities in Linux to the improve performance of auditing security relevant kernel events around network, file and process actions.
This presentation features a walk through the Linux kernel networking stack covering the essentials and recent developments a developer needs to know. Our starting point is the network card driver as it feeds a packet into the stack. We will follow the packet as it traverses through various subsystems such as packet filtering, routing, protocol stacks, and the socket layer. We will pause here and there to look into concepts such as segmentation offloading, TCP small queues, and low latency polling. We will cover APIs exposed by the kernel that go beyond use of write()/read() on sockets and will look into how they are implemented on the kernel side.
In this session, we’ll review how previous efforts, including Netfilter, Berkley Packet Filter (BPF), Open vSwitch (OVS), and TC, approached the problem of extensibility. We’ll show you an open source solution available within the Red Hat Enterprise Linux kernel, where extending and merging some of the existing concepts leads to an extensible framework that satisfies the networking needs of datacenter and cloud virtualization.
TRex is an open source, low cost, stateful traffic generator fuelled by DPDK. It generates L4-7 traffic based on pre-processing and a smart replay of real traffic templates. TRex amplifies both client and server side traffic and can scale to 200Gb/sec with one UCS.
This presentation introduces Data Plane Development Kit overview and basics. It is a part of a Network Programming Series.
First, the presentation focuses on the network performance challenges on the modern systems by comparing modern CPUs with modern 10 Gbps ethernet links. Then it touches memory hierarchy and kernel bottlenecks.
The following part explains the main DPDK techniques, like polling, bursts, hugepages and multicore processing.
DPDK overview explains how is the DPDK application is being initialized and run, touches lockless queues (rte_ring), memory pools (rte_mempool), memory buffers (rte_mbuf), hashes (rte_hash), cuckoo hashing, longest prefix match library (rte_lpm), poll mode drivers (PMDs) and kernel NIC interface (KNI).
At the end, there are few DPDK performance tips.
Tags: access time, burst, cache, dpdk, driver, ethernet, hub, hugepage, ip, kernel, lcore, linux, memory, pmd, polling, rss, softswitch, switch, userspace, xeon
Video: https://www.youtube.com/watch?v=JRFNIKUROPE . Talk for linux.conf.au 2017 (LCA2017) by Brendan Gregg, about Linux enhanced BPF (eBPF). Abstract:
A world of new capabilities is emerging for the Linux 4.x series, thanks to enhancements that have been included in Linux for to Berkeley Packet Filter (BPF): an in-kernel virtual machine that can execute user space-defined programs. It is finding uses for security auditing and enforcement, enhancing networking (including eXpress Data Path), and performance observability and troubleshooting. Many new open source tools that have been written in the past 12 months for performance analysis that use BPF. Tracing superpowers have finally arrived for Linux!
For its use with tracing, BPF provides the programmable capabilities to the existing tracing frameworks: kprobes, uprobes, and tracepoints. In particular, BPF allows timestamps to be recorded and compared from custom events, allowing latency to be studied in many new places: kernel and application internals. It also allows data to be efficiently summarized in-kernel, including as histograms. This has allowed dozens of new observability tools to be developed so far, including measuring latency distributions for file system I/O and run queue latency, printing details of storage device I/O and TCP retransmits, investigating blocked stack traces and memory leaks, and a whole lot more.
This talk will summarize BPF capabilities and use cases so far, and then focus on its use to enhance Linux tracing, especially with the open source bcc collection. bcc includes BPF versions of old classics, and many new tools, including execsnoop, opensnoop, funcccount, ext4slower, and more (many of which I developed). Perhaps you'd like to develop new tools, or use the existing tools to find performance wins large and small, especially when instrumenting areas that previously had zero visibility. I'll also summarize how we intend to use these new capabilities to enhance systems analysis at Netflix.
FOSDEM15 SDN developer room talk
DPDK performance
How to not just do a demo with DPDK
The Intel DPDK provides a platform for building high performance Network Function Virtualization applications. But it is hard to get high performance unless certain design tradeoffs are made. This talk focuses on the lessons learned in creating the Brocade vRouter using DPDK. It covers some of the architecture, locking and low level issues that all have to be dealt with to achieve 80 Million packets per second forwarding.
Netronome's half-day tutorial on host data plane acceleration at ACM SIGCOMM 2018 introduced attendees to models for host data plane acceleration and provided an in-depth understanding of SmartNIC deployment models at hyperscale cloud vendors and telecom service providers.
Presenter Bios
Jakub Kicinski is a long term Linux kernel contributor, who has been leading the kernel team at Netronome for the last two years. Jakub’s major contributions include the creation of BPF hardware offload mechanisms in the kernel and bpftool user space utility, as well as work on the Linux kernel side of OVS offload.
David Beckett is a Software Engineer at Netronome with a strong technical background of computer networks including academic research with DDoS. David has expertise in the areas of Linux architecture and computer programming. David has a Masters Degree in Electrical, Electronic Engineering at Queen’s University Belfast and continues as a PhD student studying Emerging Application Layer DDoS threats.
LCU14-101: Coresight Overview
---------------------------------------------------
Speaker: Mathieu Poirier
Date: September 15, 2014
---------------------------------------------------
Coresight is the name given to a set of IP blocks providing hardware assisted tracing for ARM based SoCs. This presentation will give an introduction to the technology, how it works and offer a glimpse of the capabilities it offers. More specifically we will go over the components that are part of the architecture and how they are used. Next will be presented the framework Linaro is working on in an effort to provide consolidation and standardization of interfaces to the coresight subsystem. We will conclude with a status of our current upstreaming efforts and how we see the coming months unfolding.
---------------------------------------------------
★ Resources ★
Zerista: http://lcu14.zerista.com/event/member/137703
Google Event: https://plus.google.com/u/0/events/cvb85kqv10dsc4k3e0hcvbr6i58
Presentation: http://www.slideshare.net/linaroorg/lcu14-101-coresight-overview
Video: https://www.youtube.com/watch?v=NzKPd3FByxI&list=UUIVqQKxCyQLJS6xvSmfndLA
Etherpad: http://pad.linaro.org/p/lcu14-101
---------------------------------------------------
★ Event Details ★
Linaro Connect USA - #LCU14
September 15-19th, 2014
Hyatt Regency San Francisco Airport
---------------------------------------------------
http://www.linaro.org
http://connect.linaro.org
The Linux kernel is undergoing the most fundamental architecture evolution in history and is becoming a microkernel. Why is the Linux kernel evolving into a microkernel? The potentially biggest fundamental change ever happening to the Linux kernel. This talk covers how companies like Facebook and Google use BPF to patch 0-day exploits, how BPF will change the way features are added to the kernel forever, and how BPF is introducing a new type of application deployment method for the Linux kernel.
SOSCON 2019.10.17
What are the methods for packet processing on Linux? And how fast are each packet processing methods? In this presentation, we will learn how to handle packets on Linux (User space, socket filter, netfilter, tc), and compare performance with analysis of where each packet processing is done in the network stack (hook point). Also, we will discuss packet processing using XDP, an in-kernel fast-path recently added to the Linux kernel. eXpress Data Path (XDP) is a high-performance programmable network data-path within the Linux kernel. The XDP is located at the lowest level of access through SW in the network stack, the point at which driver receives the packet. By using the eBPF infrastructure at this hook point, the network stack can be expanded without modifying the kernel.
Daniel T. Lee (Hoyeon Lee)
@danieltimlee
Daniel T. Lee currently works as Software Engineer at Kosslab and contributing to Linux kernel BPF project. He has interest in cloud, Linux networking, and tracing technologies, and likes to analyze the kernel's internal using BPF technology.
DPDK (Data Plane Development Kit) Overview by Rami Rosen
* Background and short history
* Advantages and disadvantages
- Very High speed networking acceleration in L2
- How this acceleration is achieved (hugepages, optimizations)
- rte_kni (and KCP)
- VPP (and FD.io project) , providing routing and switching.
- TLDK (Transport Layer Development Kit, TCP/UDP)
* Anatomy of a simple DPDK application.
* Development and governance model
* Testpmd: DPDK CLI tool
* DDP - Dynamic Device Profiles
Rami Rosen is a Linux Kernel expert, the author of "Linux Kernel Networking", Apress, 2014.
Rami had published two articles about DPDK in the last year:
"Network acceleration with DPDK"
https://lwn.net/Articles/725254/
"Userspace Networking with DPDK"
https://www.linuxjournal.com/content/userspace-networking-dpdk
Pushing Packets - How do the ML2 Mechanism Drivers Stack UpJames Denton
Architecting a private cloud to meet the use cases of its users can be a daunting task. How do you determine which of the many L2/L3 Neutron plugins and drivers to implement? Does network performance outweigh reliability? Are overlay networks just as performant as VLAN networks? The answers to these questions will drive the appropriate technology choice.
In this presentation, we will look at many of the common drivers built around the ML2 framework, including LinuxBridge, OVS, OVS+DPDK, SR-IOV, and more, and will provide performance data to help drive decisions around selecting a technology that's right for the situation. We will discuss our experience with some of these technologies, and the pros and cons of one technology over another in a production environment.
Linux 4.x Tracing: Performance Analysis with bcc/BPFBrendan Gregg
Talk about bcc/eBPF for SCALE15x (2017) by Brendan Gregg. "BPF (Berkeley Packet Filter) has been enhanced in the Linux 4.x series and now powers a large collection of performance analysis and observability tools ready for you to use, included in the bcc (BPF Complier Collection) open source project. BPF nowadays can do system tracing, software defined networks, and kernel fast path: much more than just filtering packets! This talk will focus on the bcc/BPF tools for performance analysis, which make use of other built in Linux capabilities: dynamic tracing (kprobes and uprobes) and static tracing (tracepoints and USDT). There are now bcc tools for measuring latency distributions for file system I/O and run queue latency, printing details of storage device I/O and TCP retransmits, investigating blocked stack traces and memory leaks, and a whole lot more. These lead to performance wins large and small, especially when instrumenting areas that previously had zero visibility. Tracing superpowers have finally arrived, built in to Linux."
Linux のネットワーク設定情報を取得し、描画するツール "plotnetcfg" のインストール方法や描画サンプル。
"How to" for installing / using "plotnetcfg" which scans networking config of Linux machine and plots a diagram of the configuration hierarchy.
Tutorial: Using GoBGP as an IXP connecting routerShu Sugimoto
- Show you how GoBGP can be used as a software router in conjunction with quagga
- (Tutorial) Walk through the setup of IXP connecting router using GoBGP
DockerCon 2017 - Cilium - Network and Application Security with BPF and XDPThomas Graf
This talk will start with a deep dive and hands on examples of BPF, possibly the most promising low level technology to address challenges in application and network security, tracing, and visibility. We will discuss how BPF evolved from a simple bytecode language to filter raw sockets for tcpdump to the a JITable virtual machine capable of universally extending and instrumenting both the Linux kernel and user space applications. The introduction is followed by a concrete example of how the Cilium open source project applies BPF to solve networking, security, and load balancing for highly distributed applications. We will discuss and demonstrate how Cilium with the help of BPF can be combined with distributed system orchestration such as Docker to simplify security, operations, and troubleshooting of distributed applications.
Using the new extended Berkley Packet Filter capabilities in Linux to the improve performance of auditing security relevant kernel events around network, file and process actions.
This presentation features a walk through the Linux kernel networking stack covering the essentials and recent developments a developer needs to know. Our starting point is the network card driver as it feeds a packet into the stack. We will follow the packet as it traverses through various subsystems such as packet filtering, routing, protocol stacks, and the socket layer. We will pause here and there to look into concepts such as segmentation offloading, TCP small queues, and low latency polling. We will cover APIs exposed by the kernel that go beyond use of write()/read() on sockets and will look into how they are implemented on the kernel side.
In this session, we’ll review how previous efforts, including Netfilter, Berkley Packet Filter (BPF), Open vSwitch (OVS), and TC, approached the problem of extensibility. We’ll show you an open source solution available within the Red Hat Enterprise Linux kernel, where extending and merging some of the existing concepts leads to an extensible framework that satisfies the networking needs of datacenter and cloud virtualization.
TRex is an open source, low cost, stateful traffic generator fuelled by DPDK. It generates L4-7 traffic based on pre-processing and a smart replay of real traffic templates. TRex amplifies both client and server side traffic and can scale to 200Gb/sec with one UCS.
This presentation introduces Data Plane Development Kit overview and basics. It is a part of a Network Programming Series.
First, the presentation focuses on the network performance challenges on the modern systems by comparing modern CPUs with modern 10 Gbps ethernet links. Then it touches memory hierarchy and kernel bottlenecks.
The following part explains the main DPDK techniques, like polling, bursts, hugepages and multicore processing.
DPDK overview explains how is the DPDK application is being initialized and run, touches lockless queues (rte_ring), memory pools (rte_mempool), memory buffers (rte_mbuf), hashes (rte_hash), cuckoo hashing, longest prefix match library (rte_lpm), poll mode drivers (PMDs) and kernel NIC interface (KNI).
At the end, there are few DPDK performance tips.
Tags: access time, burst, cache, dpdk, driver, ethernet, hub, hugepage, ip, kernel, lcore, linux, memory, pmd, polling, rss, softswitch, switch, userspace, xeon
Video: https://www.youtube.com/watch?v=JRFNIKUROPE . Talk for linux.conf.au 2017 (LCA2017) by Brendan Gregg, about Linux enhanced BPF (eBPF). Abstract:
A world of new capabilities is emerging for the Linux 4.x series, thanks to enhancements that have been included in Linux for to Berkeley Packet Filter (BPF): an in-kernel virtual machine that can execute user space-defined programs. It is finding uses for security auditing and enforcement, enhancing networking (including eXpress Data Path), and performance observability and troubleshooting. Many new open source tools that have been written in the past 12 months for performance analysis that use BPF. Tracing superpowers have finally arrived for Linux!
For its use with tracing, BPF provides the programmable capabilities to the existing tracing frameworks: kprobes, uprobes, and tracepoints. In particular, BPF allows timestamps to be recorded and compared from custom events, allowing latency to be studied in many new places: kernel and application internals. It also allows data to be efficiently summarized in-kernel, including as histograms. This has allowed dozens of new observability tools to be developed so far, including measuring latency distributions for file system I/O and run queue latency, printing details of storage device I/O and TCP retransmits, investigating blocked stack traces and memory leaks, and a whole lot more.
This talk will summarize BPF capabilities and use cases so far, and then focus on its use to enhance Linux tracing, especially with the open source bcc collection. bcc includes BPF versions of old classics, and many new tools, including execsnoop, opensnoop, funcccount, ext4slower, and more (many of which I developed). Perhaps you'd like to develop new tools, or use the existing tools to find performance wins large and small, especially when instrumenting areas that previously had zero visibility. I'll also summarize how we intend to use these new capabilities to enhance systems analysis at Netflix.
FOSDEM15 SDN developer room talk
DPDK performance
How to not just do a demo with DPDK
The Intel DPDK provides a platform for building high performance Network Function Virtualization applications. But it is hard to get high performance unless certain design tradeoffs are made. This talk focuses on the lessons learned in creating the Brocade vRouter using DPDK. It covers some of the architecture, locking and low level issues that all have to be dealt with to achieve 80 Million packets per second forwarding.
Netronome's half-day tutorial on host data plane acceleration at ACM SIGCOMM 2018 introduced attendees to models for host data plane acceleration and provided an in-depth understanding of SmartNIC deployment models at hyperscale cloud vendors and telecom service providers.
Presenter Bios
Jakub Kicinski is a long term Linux kernel contributor, who has been leading the kernel team at Netronome for the last two years. Jakub’s major contributions include the creation of BPF hardware offload mechanisms in the kernel and bpftool user space utility, as well as work on the Linux kernel side of OVS offload.
David Beckett is a Software Engineer at Netronome with a strong technical background of computer networks including academic research with DDoS. David has expertise in the areas of Linux architecture and computer programming. David has a Masters Degree in Electrical, Electronic Engineering at Queen’s University Belfast and continues as a PhD student studying Emerging Application Layer DDoS threats.
LCU14-101: Coresight Overview
---------------------------------------------------
Speaker: Mathieu Poirier
Date: September 15, 2014
---------------------------------------------------
Coresight is the name given to a set of IP blocks providing hardware assisted tracing for ARM based SoCs. This presentation will give an introduction to the technology, how it works and offer a glimpse of the capabilities it offers. More specifically we will go over the components that are part of the architecture and how they are used. Next will be presented the framework Linaro is working on in an effort to provide consolidation and standardization of interfaces to the coresight subsystem. We will conclude with a status of our current upstreaming efforts and how we see the coming months unfolding.
---------------------------------------------------
★ Resources ★
Zerista: http://lcu14.zerista.com/event/member/137703
Google Event: https://plus.google.com/u/0/events/cvb85kqv10dsc4k3e0hcvbr6i58
Presentation: http://www.slideshare.net/linaroorg/lcu14-101-coresight-overview
Video: https://www.youtube.com/watch?v=NzKPd3FByxI&list=UUIVqQKxCyQLJS6xvSmfndLA
Etherpad: http://pad.linaro.org/p/lcu14-101
---------------------------------------------------
★ Event Details ★
Linaro Connect USA - #LCU14
September 15-19th, 2014
Hyatt Regency San Francisco Airport
---------------------------------------------------
http://www.linaro.org
http://connect.linaro.org
This presentation includes basic information related to sockets ,socket-buffer,cliet-server programs and relationship between them
The files included in the ppt for the variables are taken from linux-2.6.10.
In case of any queriers.
contact souravpunoriyar@gmail.com
CC++ echo serverThis assignment is designed to introduce network .pdfsecunderbadtirumalgi
C/C++ echo server
This assignment is designed to introduce network programing and the socket API. You will build
a simple ECHO server, that is a program that accepts incoming connection requests and then
echoes back any string that is sent.
Required Behavior
1. Your submission must be written in C or C++ and must compile and run on the schools Linux
server.
2. Your code should be clean and easy to read with reasonable comments and variable names.
3. You must provide a makefile that compiles your program. It should compile without errors or
warnings to a binary named echo_s.
4. Your program should take one optional command line argument, -v, that invokes verbose
mode causing the program to print diagnostic messages as it operates.
5. When run the program should:
* Create a socket and prepare it to listen for incoming TCP connections on the port of your
choice.
* Print Using port: X to tell the user what port your program is listening on.
* Begin waiting for a new connection
* When a new connection is established:
- Read a block of data from the connection note the length of the data received (blocks can
be up to 1024 bytes long)
- Check the block to see if it starts with the letters CLOSE or QUIT.
+If CLOSE, close the connection and start waiting for another connection (return to
waiting for a connection).
+If QUIT, close the connection and the listening socket and exit your program.
- Send the block back with a write() call.
- Return to waiting for the next block of data from the client.
6. If run with the v flag you should print a diagnostic message at least
* When the socket is opened
* When the socket is bound.
* Before it blocks waiting for a new connection.
* When a new connection is accepted.
* Before it blocks waiting for new data.
* When you get a block of data.
* If the data included the CLOSE command,
* If the data included the QUIT command.
* Any other time you think it would be useful.
Suggested Approach
There is some skeleton code at the end
The skeleton includes a makefile and everything you need to get started.
Once you have the skeleton setup and running you should start filling in the code. You may of
course do this however you like, but I would recommend the following:
* Fill in the sections of main() required to process command line arguments, create, bind,
establish a listening queue and then close a stream type socket.
- If you run the program without the v flag it should print what port was selected, then exit.
- If you run the program with the v you should see each step of the socket creation process.
* Add code to accept new connection requests and as requests are received call the
processConnection() At this point you can test by connect to the program with telnet.
* At this point you should have the functioning skeleton of a TCP server application. Up to this
point the structure of all server applications are almost identical, regardless of the application
protocol they are going to implement.
* Modify the processCon.
A brief overview of how the ubiquitous 'iptables' command works, under the hood. This is by no means complete, and is intended as a primer. I presented these slides in addition to an interactive demo as a guest lecturer at a computer network security class at DePaul University, Chicago IL.
Let's trace Linux Lernel with KGDB @ COSCUP 2021Jian-Hong Pan
https://coscup.org/2021/en/session/39M73K
https://www.youtube.com/watch?v=L_Gyvdl_d_k
Engineers have plenty of debug tools for user space programs development, code tracing, debugging and analyzing. Except “printk”, do we have any other debug tools for Linux kernel development? The “KGDB” mentioned in Linux kernel document provides another possibility.
Will share how to experiment with the KGDB in a virtual machine. And, use GDB + OpenOCD + JTAG + Raspberry Pi in the real environment as the demo in this talk.
開發 user space 軟體時,工程師們有方便的 debug 工具進行查找、分析、除錯。但在 Linux kernel 的開發,除了 printk 外,還可以有哪些工具可以使用呢?從 Linux kernel document 可以看到 KGDB 相關的資訊,提供了在 kernel 除錯時的另一個可能性。
本次將分享,從建立最簡單環境的虛擬機機開始,到實際使用 GDB + OpenOCD + JTAG + Raspberry Pi 當作展示範例。
"Session ID: BUD17-300
Session Name: Journey of a packet - BUD17-300
Speaker: Maxim Uvarov
Track: LNG
★ Session Summary ★
Describe step by step what components a packet goes through and details cases when components are implemented in hardware or in software. Attendees will have the definite presentation to understand fundamental differences with DPDK and how ODP solves low end and high end networking issues.
---------------------------------------------------
★ Resources ★
Event Page: http://connect.linaro.org/resource/bud17/bud17-300/
Presentation: https://www.slideshare.net/linaroorg/bud17300-journey-of-a-packet
Video: https://youtu.be/wRZXw_xBT20
---------------------------------------------------
★ Event Details ★
Linaro Connect Budapest 2017 (BUD17)
6-10 March 2017
Corinthia Hotel, Budapest,
Erzsébet krt. 43-49,
1073 Hungary
---------------------------------------------------
Keyword: packet, LNG
http://www.linaro.org
http://connect.linaro.org
---------------------------------------------------
Follow us on Social Media
https://www.facebook.com/LinaroOrg
https://twitter.com/linaroorg
https://www.youtube.com/user/linaroorg?sub_confirmation=1
https://www.linkedin.com/company/1026961
Literature Review Basics and Understanding Reference Management.pptxDr Ramhari Poudyal
Three-day training on academic research focuses on analytical tools at United Technical College, supported by the University Grant Commission, Nepal. 24-26 May 2024
An Approach to Detecting Writing Styles Based on Clustering Techniquesambekarshweta25
An Approach to Detecting Writing Styles Based on Clustering Techniques
Authors:
-Devkinandan Jagtap
-Shweta Ambekar
-Harshit Singh
-Nakul Sharma (Assistant Professor)
Institution:
VIIT Pune, India
Abstract:
This paper proposes a system to differentiate between human-generated and AI-generated texts using stylometric analysis. The system analyzes text files and classifies writing styles by employing various clustering algorithms, such as k-means, k-means++, hierarchical, and DBSCAN. The effectiveness of these algorithms is measured using silhouette scores. The system successfully identifies distinct writing styles within documents, demonstrating its potential for plagiarism detection.
Introduction:
Stylometry, the study of linguistic and structural features in texts, is used for tasks like plagiarism detection, genre separation, and author verification. This paper leverages stylometric analysis to identify different writing styles and improve plagiarism detection methods.
Methodology:
The system includes data collection, preprocessing, feature extraction, dimensional reduction, machine learning models for clustering, and performance comparison using silhouette scores. Feature extraction focuses on lexical features, vocabulary richness, and readability scores. The study uses a small dataset of texts from various authors and employs algorithms like k-means, k-means++, hierarchical clustering, and DBSCAN for clustering.
Results:
Experiments show that the system effectively identifies writing styles, with silhouette scores indicating reasonable to strong clustering when k=2. As the number of clusters increases, the silhouette scores decrease, indicating a drop in accuracy. K-means and k-means++ perform similarly, while hierarchical clustering is less optimized.
Conclusion and Future Work:
The system works well for distinguishing writing styles with two clusters but becomes less accurate as the number of clusters increases. Future research could focus on adding more parameters and optimizing the methodology to improve accuracy with higher cluster values. This system can enhance existing plagiarism detection tools, especially in academic settings.
NUMERICAL SIMULATIONS OF HEAT AND MASS TRANSFER IN CONDENSING HEAT EXCHANGERS...ssuser7dcef0
Power plants release a large amount of water vapor into the
atmosphere through the stack. The flue gas can be a potential
source for obtaining much needed cooling water for a power
plant. If a power plant could recover and reuse a portion of this
moisture, it could reduce its total cooling water intake
requirement. One of the most practical way to recover water
from flue gas is to use a condensing heat exchanger. The power
plant could also recover latent heat due to condensation as well
as sensible heat due to lowering the flue gas exit temperature.
Additionally, harmful acids released from the stack can be
reduced in a condensing heat exchanger by acid condensation. reduced in a condensing heat exchanger by acid condensation.
Condensation of vapors in flue gas is a complicated
phenomenon since heat and mass transfer of water vapor and
various acids simultaneously occur in the presence of noncondensable
gases such as nitrogen and oxygen. Design of a
condenser depends on the knowledge and understanding of the
heat and mass transfer processes. A computer program for
numerical simulations of water (H2O) and sulfuric acid (H2SO4)
condensation in a flue gas condensing heat exchanger was
developed using MATLAB. Governing equations based on
mass and energy balances for the system were derived to
predict variables such as flue gas exit temperature, cooling
water outlet temperature, mole fraction and condensation rates
of water and sulfuric acid vapors. The equations were solved
using an iterative solution technique with calculations of heat
and mass transfer coefficients and physical properties.
Sachpazis:Terzaghi Bearing Capacity Estimation in simple terms with Calculati...Dr.Costas Sachpazis
Terzaghi's soil bearing capacity theory, developed by Karl Terzaghi, is a fundamental principle in geotechnical engineering used to determine the bearing capacity of shallow foundations. This theory provides a method to calculate the ultimate bearing capacity of soil, which is the maximum load per unit area that the soil can support without undergoing shear failure. The Calculation HTML Code included.
Cosmetic shop management system project report.pdfKamal Acharya
Buying new cosmetic products is difficult. It can even be scary for those who have sensitive skin and are prone to skin trouble. The information needed to alleviate this problem is on the back of each product, but it's thought to interpret those ingredient lists unless you have a background in chemistry.
Instead of buying and hoping for the best, we can use data science to help us predict which products may be good fits for us. It includes various function programs to do the above mentioned tasks.
Data file handling has been effectively used in the program.
The automated cosmetic shop management system should deal with the automation of general workflow and administration process of the shop. The main processes of the system focus on customer's request where the system is able to search the most appropriate products and deliver it to the customers. It should help the employees to quickly identify the list of cosmetic product that have reached the minimum quantity and also keep a track of expired date for each cosmetic product. It should help the employees to find the rack number in which the product is placed.It is also Faster and more efficient way.
Water billing management system project report.pdfKamal Acharya
Our project entitled “Water Billing Management System” aims is to generate Water bill with all the charges and penalty. Manual system that is employed is extremely laborious and quite inadequate. It only makes the process more difficult and hard.
The aim of our project is to develop a system that is meant to partially computerize the work performed in the Water Board like generating monthly Water bill, record of consuming unit of water, store record of the customer and previous unpaid record.
We used HTML/PHP as front end and MYSQL as back end for developing our project. HTML is primarily a visual design environment. We can create a android application by designing the form and that make up the user interface. Adding android application code to the form and the objects such as buttons and text boxes on them and adding any required support code in additional modular.
MySQL is free open source database that facilitates the effective management of the databases by connecting them to the software. It is a stable ,reliable and the powerful solution with the advanced features and advantages which are as follows: Data Security.MySQL is free open source database that facilitates the effective management of the databases by connecting them to the software.
Forklift Classes Overview by Intella PartsIntella Parts
Discover the different forklift classes and their specific applications. Learn how to choose the right forklift for your needs to ensure safety, efficiency, and compliance in your operations.
For more technical information, visit our website https://intellaparts.com
We have compiled the most important slides from each speaker's presentation. This year’s compilation, available for free, captures the key insights and contributions shared during the DfMAy 2024 conference.
Welcome to WIPAC Monthly the magazine brought to you by the LinkedIn Group Water Industry Process Automation & Control.
In this month's edition, along with this month's industry news to celebrate the 13 years since the group was created we have articles including
A case study of the used of Advanced Process Control at the Wastewater Treatment works at Lleida in Spain
A look back on an article on smart wastewater networks in order to see how the industry has measured up in the interim around the adoption of Digital Transformation in the Water Industry.
2. Agenda
● System calls in Networking world
● Client server model
● Linux networking stack
● Evolution of networking stack
● Driver Interface
● Introduction to Wifi Stack
● Wifi stack as an example
● Future...?
3. Simple router Control plane
Network Driver
Data Plane
Control Plane
Kernel Space
Module 1 Module 2 Module n
Intercard
communication
mechanism
Interconnect
Protocol
Control Plane
User Space
Module 1 to
Module n are
processes on the
CP
4. Problem definition
●
All CP modules are communicating with each other IPC
●
Control plane / Data plan communication happens over
high speed network link
●
Line cards can interact with other line cards or Control
cards.
●
●
●
And the router crashes ???
5. Things to look out for...
●
Is kernel, network driver alive, kernel log, crash dump.
●
see if there is a particular irq screaming in
/proc/interrupts
●
/proc/sys/net/* : networking information
●
Check top, free output if any process is hogging cpu?
●
Check ps to see expected processes/threads are alive :
status of CP processes.
●
Try to get some info from /proc/net/nf_conntrack_stats
to see if a particular type of error packet is being
reported
●
Check firewall rules: iptables -L, ifconfig, route.
●
Kernel/Application log indicating any error:
/var/log/syslog
6. Going deeper...
●
Check files, sockets owned by each process.
●
cat /proc/$PID/* : fd, wchan
●
/proc/net/tcp, /proc/net/udp
●
netstat -apeen
●
lsof (-i for networking)
●
●
Socket Status [socket operation on non-socket]
●
●
Kernel modules to spit information on data structures
like task_struct, struct netdevice
7. Knowing the full stack...
●
In order to understand the complete, kernel knowledge
is necessary.
– User space applications
– Threads, socket types
– Kernel interface through system calls
– TCP IP stack inside the kernel
– Interaction with network device driver.
– And the kernel subsystems.
11. Socket() in kernel
●
For every socket which is created by a userspace
application, there is a corresponding socket struct and
sock struct in the kernel
int socket (int family, int type, int protocol);
●
SOCK_STREAM : TCP, SOCK_DGRAM: UDP, SOCK_RAW.
●
This system call eventually invokes the sock_create() method in the
kernel.
●
struct socket { /* ONLY important members */
socket_state state;
unsigned long flags;
struct fasync_struct *fasync_list;
wait_queue_head_t wait;
struct file *file;
struct sock *sk;
const struct proto_ops *ops;
}
13. bind() in kernel
●
int bind(int sockfd, const struct sockaddr
*addr, socklen_t addrlen);
●
This system call eventually invokes the inet_bind() method in the
kernel.
●
The bind system call associates a local network transport address with a
socket. For a client process, it is not mandatory to issue a bind call. The
kernel takes care of doing an implicit binding when the client process
issues the connect system call.
●
The kernel function sys_bind() does the following:
– sock = sockfd_lookup_light(fd, &err, &fput_needed);
– sock->ops->bind(sock, (struct sockaddr *)address, addrlen);
●
Point to note:
– Binding to unprivileged ports (<1024)
14. listen() in kernel
●
int listen(int sockfd, int backlog);
●
backlog argument defines the maximum length to which the queue of
pending connections for sockfd may grow.
●
Linux uses two queues, a SYN queue (or incomplete connection queue)
and an accept queue (or complete connection queue). Connections in
state SYN RECEIVED are added to the SYN queue and later moved to the
accept queue when their state changes to ESTABLISHED, i.e. when the
ACK packet in the 3-way handshake is received. As the name implies, the
accept call is then implemented simply to consume connections from the
accept queue. In this case, the backlog argument of the listen syscall
determines the size of the accept queue.
●
SYN queue with a size specified by a system wide setting.
– /proc/sys/net/ipv4/tcp_max_syn_backlog.
●
accept queue with a size specified by the application.
●
Implementation is in inet_listen() kernel function.
15. connect() in kernel
●
int connect(int sockfd, const struct sockaddr
*addr, socklen_t addrlen);
●
Calls inet_autobind() to use the available source port as needed.
●
Fills destination in inet_sock and calls ipv4_stream_connect or
ipv4_datagram_connect (for IPV4).
●
Routing is done by ip_route_connect function (L3)
16. accept() in kernel
●
int accept(int sockfd, struct sockaddr *addr,
socklen_t *addrlen);
●
This system call eventually invokes the inet_accept() method in the
kernel.
21. close() in kernel
●
int shutdown(int sockfd, int how);
●
int close(int sockfd);
●
Shutdown can bring down the connection in half duplex mode. At the
point, the queues associated with socket are not purged. Hence, it is
necessary to call the close() function.
23. Socket Data Structures
●
For every socket which is created by a user space application, there
is a corresponding struct socket and struct sock in the kernel.
●
●
struct socket: include/linux/net.h
– Data common to the BSD socket layer
– Has only 8 members
– Any variable “sock” always refers to a struct socket
●
struct sock : include/net/sock.h
– Data common to the Network Protocol layer (i.e., AF_INET)
– Any variable “sk” always refers to a struct sock.
24. AF Interface
●
Main data structures
– struct net_proto_family
– struct proto_ops
●
Key function
sock_register(struct net_proto_family *ops)
●
Each address family:
– Implements the struct net _proto_family.
– Calls the function sock_register( ) when the protocol family is
initialized.
– Implement the struct proto_ops for binding the BSD socket
layer and protocol family layer.
32. sk_buff
●
Kernel buffer that stores packets.
– Contains headers for all network layers.
●
Creation
– Application sends data to socket.
– Packet arrives at network interface.
●
Copying
– Copied from user/kernel space.
– Copied from kernel space to NIC.
– Send: appends headers via skb_reserve().
– Receive: moves ptr from header to header.
33. sk_buff (cont...)
●
sk_buff represents data and headers.
●
sk_buff API (examples)
– sk_buff allocation is done with alloc_skb() or
dev_alloc_skb();
– drivers use dev_alloc_skb();
– (free by kfree_skb() and dev_kfree_skb().
●
unsigned char* data : points to the current header.
●
skb_pull(int len) – removes data from the start of a
buffer by
advancing data to data+len and by decreasing len.
●
Almost always sk_buff instances appear as “skb” in the
kernel code
35. sk_buff functions
●
skb_reserve()
Prototype
void skb_reserve(struct sk_buff *skb, unsigned int len);
Description
adjust headroom. Used to make reservation for the header. When
setting up receive packets that an ethernet device will DMA into,
skb_reserve(skb, NET_IP_ALIGN) is called. This makes it so that, after the
ethernet header, the protocol header will be aligned on at least a 4-byte
boundary
36. sk_buff functions
●
skb_push()
Prototype
unsigned char *skb_push(struct sk_buff *skb, unsigned int
len);
Description
add data to the start of a buffer. skb_push() decrements 'skb-
>data' and increments 'skb->len'. e.g. adding ethernet header
before IP, TCP header.
38. sk_buff functions
●
skb_put()
Prototype
unsigned char *skb_put(struct sk_buff *skb, unsigned int len);
Description
add data to a buffer. skb_put() advances 'skb->tail' by the
specified number of bytes, it also increments 'skb->len' by that
number of bytes as well. Make sure, that enough tailroom is
available, else skb_over_panic()
40. Network device drivers
●
net_device registration
●
hard_start_xmit function pointer
●
Interrupt handler for packet reception
●
●
Bus Interaction (e.g. PCI)
●
NAPI context
41. net_device structure
●
net_device represents a network interface card.
●
It is used to represent physical or virtual devices. e.g.
loopback devices, bonding devices used for load
balancing or high availability.
●
Implemented using the private data of the
device (the void *priv member of net_device);
●
unsigned char* data : points to the current header.
●
skb_pull(int len) – removes data from the start of a
buffer by advancing data to data+len and by
decreasing len.
●
Almost always sk_buff instances appear as “skb” in the
kernel code
42. net_device structure (cont...)
●
unsigned int mtu – Maximum Transmission
Unit: the maximum size of frame the device
can handle.
●
unsigned int flags, dev_addr[6].
●
void *ip_ptr: IPv4 specific data. This pointer is
assigned to a pointer to in_device in
inetdev_init() (net/ipv4/devinet.c)
●
struct in_device: It contains a member named
cnf (which is instance of ipv4_devconf).
Setting /proc/sys/net/ipv4/conf/all/forwarding
43. Packet Transmission
●
TCP/IP stack calls dev_queue_xmit function to
queue the packet in the device queue.
●
The device driver has a Tx handler registered
as hard_start_xmit() function pointer.
●
This function transmits the packet over wire
or air and waits for completion callback.
●
This completion callback is generally used to
free the sk_buff associated with the packet.
44. Packet Transmission (cont...)
●
Handling of sending a packet is done by
ip_route_output_key().
●
Routing lookup also in the case of
transmission.
●
If the packet is for a remote host, set dst
>output to ip_output()
●
ip_output() will call ip_finish_output()
– This is the NF_IP_POST_ROUTING point
45. Packet Reception
●
When working in interrupt-driven model, the
nic registers an interrupt handler with the IRQ
with which the device works by calling
request_irq().
●
This interrupt handler will be called when a
frame is received.
●
The same interrupt handler will be called
when transmission of a frame is finished and
under other conditions like errors.
●
Interrupt handler should verify interrupt
cause
●
Control transferred to TCP/IP stack using
netif_rx() or netif_rx_ni()
46. Packet Reception (cont...)
●
Interrupt handler: sk_buff is allocated by
calling dev_alloc_skb() ; also eth_type_trans()
is called; It also advances the data pointer of
the sk_buff to point to the IP header using
skb_pull(skb, ETH_HLEN).
●
This interrupt handler will be called when a
frame is received.
●
The same interrupt handler will be called
when transmission of a frame is finished and
under other conditions like errors.
●
Interrupt handler should verify interrupt
cause.
48. Physical ( Ethernet ) [L1]
●
NIC generates an Interrupt Request ( IRQ )
●
The card driver is the Interrupt Service
Routine ( ISR ) - disables interrupts
– Allocates a new sk_buff structure
– Fetches packet data from card buffer to freshly
allocated sk_buff ( using DMA )
– Invokes netif_rx()
– When netif_rx() returns, the Interrupts are re-
enabled and the ISR is terminated
49. The picture:
Ethernet Driver
Low Lever Pkt Rx
Deferred pkt rcptn
Other Layer 3 Proc AF_INET ( IP ) AF_PACKET
TCP Processing UDP ICMP
Socket Level
Receiving Process
netif_rx()
net_rx_action()
packet_rcv()ip_rcv()*_rcv()
tcp_rcv() udp_rcv() icmp_rcv()
data_ready()
wake_up_interruptible()
Journey of a packet
50. TCP/IP stack
●
Minimize copying
●
Zero copy technique
●
Page remapping
●
Branch optimization
●
Avoid process migration or cache misses
●
Avoid dynamic assignment of interrupts
to different CPUs
●
Combine Operations within the same
layer to minimize passes to the data
52. Wifi Programming
Steps for programming the wireless extensions:
• Open a network socket.
(PF_INET, SOCK_DGRAM).
• Setup the wireless request using struct iwreq.
Set device name.
Set wireless request data.
Set subioctl_no.
• Invoke device ioctl.
• Wait for the response. [ Blocking Call ]
• Wireless events are received over netlink socket.
( PF_NETLINK )
53. Wifi kernel handling
Kernel space handling:
* When kernel ioctl handler transfers control to the ioctl from
the wireless device driver.
* The driver invokes appropriate wireless extension call based
on the ioctl command.
* The wireless extension call transfers control to wireless
firmware using special command interface over the
USB/SDIO/MMC bus.
* Wireless driver can receive events from firmware.
( e.g.Link_Loss Event)
54. Driver firmware interface
What is a firmware ?
* Firmware is wireless networking software that runs on the
wireless chipset.
* The wireless device driver downloads the firmware to the
wireless chipset, upon initialization.
* All low level wireless operations are performed by the
firmware software.
* It works in two modes
Synchronous Request, response protocol
Asynchronous Events from FW.
* The firmware resides in /lib/firmware/
e.g. /lib/firmware/iwl-3945.ucode
•
55. Need for NGW
●
Next Generation Wireless
●
Centralized control for all wireless work
●
Drivers implement small set of configuration
methods
●
Semantics as per flows in the IEEE specifications
●
Various modes of operation
Station, AP, Monitor, IBSS, WDS, Mesh, P2P
56. Mac80211, cfg80211
●
Mac80211 is Linux kernel subsytem
●
Implements shared code for soft MAC, half MAC
devices
●
Contains MLME (Media Access Control (MAC) Layer Management Entity)
Authenticate, Deauthenticate, Associate, Disassociate
Reassociate , Beacon , Probe
Cfg80211 is the layer between user space and
mac80211.