Linux System Monitoring with eBPF
•
1 like•985 views
Ignite talk given at dodkiel18 and dodzh18. The long version is here: https://www.circonus.com/2018/05/linux-system-monitoring-with-ebpf/
Report
Share
Report
Share
Download to read offline
Recommended
Low Overhead System Tracing with eBPF
In the Container world, there is a need to build observability around apps and backing services running in containers. The observability should allow to capture on demand low level metrics at a low overhead. The proposal is to use ebpf as the tracing technology to capture details at kernel & user level, and generate on demand flamegraphs, heat maps for applications & backing services. The Linux kernel has a built-in BPF JIT compiler, and an in-kernel verifier which is used to validate eBPF programs. This allows user defined instrumentation on a live kernel image that can never crash, hang or interfere with the kernel negatively. eBPF provides in-kernel implementation of storage maps such as histograms and hash-maps, which helps in efficient copy of summarized monitoring data from kernel to user space with low overhead.
These features make eBPF programs safe to run in production and allow admins and engineers to collect crucial data from systems for performance analysis and monitoring.
Understanding eBPF in a Hurry!
eBPF is an exciting new technology that is poised to transform Linux performance engineering. eBPF enables users to dynamically and programatically trace any kernel or user space code path, safely and efficiently. However, understanding eBPF is not so simple. The goal of this talk is to give audiences a fundamental understanding of eBPF, how it interconnects existing Linux tracing technologies, and provides a powerful aplatform to solve any Linux performance problem.
eBPF Basics
This document provides an overview of cBPF and eBPF. It discusses the history and implementation of cBPF, including how it was originally used for packet filtering. It then covers eBPF in more depth, explaining what it is, its history, implementation including different program types and maps. It also discusses several uses of eBPF including networking, firewalls, DDoS mitigation, profiling, security, and chaos engineering. Finally, it introduces XDP and DPDK, comparing XDP's benefits over DPDK.
eBPF Workshop
This document provides instructions for setting up and attending an eBPF workshop. It includes links for setting up the workshop platform, background slides, and code repository. It also lists an agenda with topics that will be covered, including setting up the eBPF lab, an introduction, eBPF 101, writing eBPF programs, BCC, and a tutorial. Attendees are asked to let the presenter know if they have any problems setting up.
Linux Container Basics
Michael Kehoe provides an overview of Linux container basics. Containers isolate processes running within them and provide security and resource control similar to virtual machines but with faster deployment. Key Linux kernel features like namespaces and cgroups are used to isolate containers. Namespaces isolate resources like the network, filesystem and process IDs. Cgroups limit resources like CPU and memory. Copy-on-write is used to improve memory efficiency. Container runtimes like Docker and containerd use these features to package and run applications in containers.
LPC2019 BPF Tracing Tools
This document discusses Brendan Gregg's opinions on various tracing tools including sysdig, perf, ftrace, eBPF, bpftrace, and BPF perf tools. It provides a table comparing the scope, capability, and ease of use of these tools. It then gives an example of using BPF perf tools to analyze readahead performance. Finally, it outlines desired additions to tracing capabilities and BPF helpers as well as challenges in areas like function tracing without frame pointers.
eBPF Perf Tools 2019
The document describes a biolatency tool that traces block device I/O latency using eBPF. It discusses how the tool was originally written in the bcc framework using C/BPF, but has since been rewritten in the bpftrace framework using a simpler one-liner script. It provides examples of the bcc and bpftrace implementations of biolatency.
eBPF maps 101
The document provides an overview of eBPF maps and how they can be used to share data between eBPF programs running in the kernel and userspace applications. It describes how maps are created via the BPF syscall using the BPF_MAP_CREATE command. It also explains how keys and values can be looked up, updated, and deleted from maps using commands like BPF_MAP_LOOKUP_ELEM, BPF_MAP_UPDATE_ELEM, and BPF_MAP_DELETE_ELEM. Finally, it lists the different types of eBPF maps available.
Recommended
Low Overhead System Tracing with eBPF
In the Container world, there is a need to build observability around apps and backing services running in containers. The observability should allow to capture on demand low level metrics at a low overhead. The proposal is to use ebpf as the tracing technology to capture details at kernel & user level, and generate on demand flamegraphs, heat maps for applications & backing services. The Linux kernel has a built-in BPF JIT compiler, and an in-kernel verifier which is used to validate eBPF programs. This allows user defined instrumentation on a live kernel image that can never crash, hang or interfere with the kernel negatively. eBPF provides in-kernel implementation of storage maps such as histograms and hash-maps, which helps in efficient copy of summarized monitoring data from kernel to user space with low overhead.
These features make eBPF programs safe to run in production and allow admins and engineers to collect crucial data from systems for performance analysis and monitoring.
Understanding eBPF in a Hurry!
eBPF is an exciting new technology that is poised to transform Linux performance engineering. eBPF enables users to dynamically and programatically trace any kernel or user space code path, safely and efficiently. However, understanding eBPF is not so simple. The goal of this talk is to give audiences a fundamental understanding of eBPF, how it interconnects existing Linux tracing technologies, and provides a powerful aplatform to solve any Linux performance problem.
eBPF Basics
This document provides an overview of cBPF and eBPF. It discusses the history and implementation of cBPF, including how it was originally used for packet filtering. It then covers eBPF in more depth, explaining what it is, its history, implementation including different program types and maps. It also discusses several uses of eBPF including networking, firewalls, DDoS mitigation, profiling, security, and chaos engineering. Finally, it introduces XDP and DPDK, comparing XDP's benefits over DPDK.
eBPF Workshop
This document provides instructions for setting up and attending an eBPF workshop. It includes links for setting up the workshop platform, background slides, and code repository. It also lists an agenda with topics that will be covered, including setting up the eBPF lab, an introduction, eBPF 101, writing eBPF programs, BCC, and a tutorial. Attendees are asked to let the presenter know if they have any problems setting up.
Linux Container Basics
Michael Kehoe provides an overview of Linux container basics. Containers isolate processes running within them and provide security and resource control similar to virtual machines but with faster deployment. Key Linux kernel features like namespaces and cgroups are used to isolate containers. Namespaces isolate resources like the network, filesystem and process IDs. Cgroups limit resources like CPU and memory. Copy-on-write is used to improve memory efficiency. Container runtimes like Docker and containerd use these features to package and run applications in containers.
LPC2019 BPF Tracing Tools
This document discusses Brendan Gregg's opinions on various tracing tools including sysdig, perf, ftrace, eBPF, bpftrace, and BPF perf tools. It provides a table comparing the scope, capability, and ease of use of these tools. It then gives an example of using BPF perf tools to analyze readahead performance. Finally, it outlines desired additions to tracing capabilities and BPF helpers as well as challenges in areas like function tracing without frame pointers.
eBPF Perf Tools 2019
The document describes a biolatency tool that traces block device I/O latency using eBPF. It discusses how the tool was originally written in the bcc framework using C/BPF, but has since been rewritten in the bpftrace framework using a simpler one-liner script. It provides examples of the bcc and bpftrace implementations of biolatency.
eBPF maps 101
The document provides an overview of eBPF maps and how they can be used to share data between eBPF programs running in the kernel and userspace applications. It describes how maps are created via the BPF syscall using the BPF_MAP_CREATE command. It also explains how keys and values can be looked up, updated, and deleted from maps using commands like BPF_MAP_LOOKUP_ELEM, BPF_MAP_UPDATE_ELEM, and BPF_MAP_DELETE_ELEM. Finally, it lists the different types of eBPF maps available.
Staring into the eBPF Abyss
eBPF (extended Berkeley Packet Filters) is a modern kernel technology that can be used to introduce dynamic tracing into a system that wasn't prepared or instrumented in any way. The tracing programs run in the kernel, are guaranteed to never crash or hang your system, and can probe every module and function -- from the kernel to user-space frameworks such as Node and Ruby.
In this workshop, you will experiment with Linux dynamic tracing first-hand. First, you will explore BCC, the BPF Compiler Collection, which is a set of tools and libraries for dynamic tracing. Many of your tracing needs will be answered by BCC, and you will experiment with memory leak analysis, generic function tracing, kernel tracepoints, static tracepoints in user-space programs, and the "baked" tools for file I/O, network, and CPU analysis. You'll be able to choose between working on a set of hands-on labs prepared by the instructors, or trying the tools out on your own test system.
Next, you will hack on some of the bleeding edge tools in the BCC toolkit, and build a couple of simple tools of your own. You'll be able to pick from a curated list of GitHub issues for the BCC project, a set of hands-on labs with known "school solutions", and an open-ended list of problems that need tools for effective analysis. At the end of this workshop, you will be equipped with a toolbox for diagnosing issues in the field, as well as a framework for building your own tools when the generic ones do not suffice.
UM2019 Extended BPF: A New Type of Software
BPF (Berkeley Packet Filter) has evolved from a limited virtual machine for efficient packet filtering to a new type of software called extended BPF. Extended BPF allows for custom, efficient, and production-safe performance analysis tools and observability programs to be run in the Linux kernel through BPF. It enables new event-based applications running as BPF programs attached to various kernel events like kprobes, uprobes, tracepoints, sockets, and more. Major companies like Facebook, Google, and Netflix are using BPF programs for tasks like intrusion detection, container security, firewalling, and observability with over 150,000 AWS instances running BPF programs. BPF provides a new program model and security features compared
Meet cute-between-ebpf-and-tracing
This document discusses how eBPF (extended Berkeley Packet Filter) can be used for kernel tracing. It provides an overview of BPF and eBPF, how eBPF programs are compiled and run in the kernel, the use of BPF maps, and how eBPF enables new possibilities for dynamic kernel instrumentation through techniques like Kprobes and ftrace.
Tracer Evaluation
Performance evaluation of various implementations of off-cpu wakeup path tracers for Linux kernel, using kprobe, ftrace, and ebpf.
Building Network Functions with eBPF & BCC
eBPF (Extended Berkeley Packet Filter) is an in-kernel virtual machine that allows running user-supplied sandboxed programs inside of the kernel. It is especially well-suited to network programs and it's possible to write programs that filter traffic, classify traffic and perform high-performance custom packet processing.
BCC (BPF Compiler Collection) is a toolkit for creating efficient kernel tracing and manipulation programs. It makes use of eBPF.
BCC provides an end-to-end workflow for developing eBPF programs and supplies Python bindings, making eBPF programs much easier to write.
Together, eBPF and BCC allow you to develop and deploy network functions safely and easily, focusing on your application logic (instead of kernel datapath integration).
In this session, we will introduce eBPF and BCC, explain how to implement a network function using BCC, discuss some real-life use-cases and show a live demonstration of the technology.
About the speaker
Shmulik Ladkani, Chief Technology Officer at Meta Networks,
Long time network veteran and kernel geek.
Shmulik started his career at Jungo (acquired by NDS/Cisco) implementing residential gateway software, focusing on embedded Linux, Linux kernel, networking and hardware/software integration.
Some billions of forwarded packets later, Shmulik left his position as Jungo's lead architect and joined Ravello Systems (acquired by Oracle) as tech lead, developing a virtual data center as a cloud-based service, focusing around virtualization systems, network virtualization and SDN.
Recently he co-founded Meta Networks where he's been busy architecting secure, multi-tenant, large-scale network infrastructure as a cloud-based service.
Beautiful History
The document discusses the history of software and version control using Git. It provides examples of Git commands like git log, git rebase, and git merge. It also discusses best practices for writing commit messages and splitting commits into logical changes.
eBPF Trace from Kernel to Userspace
This document discusses eBPF (extended Berkeley Packet Filter), which allows tracing from the Linux kernel to userspace using BPF programs. It provides an overview of eBPF including extended registers, verification, maps, and probes. Examples are given of using eBPF for tracing functions like kfree_skb() and the C library function malloc. The Berkeley Compiler Collection (BCC) makes it easy to write eBPF programs in C and Python.
Linux kernel-rootkit-dev - Wonokaerun
This document contains the slides from a presentation given by WonoKaerun at the Indonesian Security Conference 2011 in Palembang. The presentation introduces rootkits and techniques for hiding malware at the kernel level on Linux systems. It covers topics like loadable kernel modules, interrupt descriptor table hooking, virtual file system hacking, page fault handler hijacking, debugging register abuse, and kernel instrumentation patching. The goal is to evade detection by security solutions by gaining control of the kernel before anti-rootkit defenses can activate. Throughout, the document emphasizes the cat-and-mouse nature of offensive and defensive security research.
The Next Linux Superpower: eBPF Primer
Imagine you're tackling one of these evasive performance issues in the field, and your go-to monitoring checklist doesn't seem to cut it. There are plenty of suspects, but they are moving around rapidly and you need more logs, more data, more in-depth information to make a diagnosis. Maybe you've heard about DTrace, or even used it, and are yearning for a similar toolkit, which can plug dynamic tracing into a system that wasn't prepared or instrumented in any way.
Hopefully, you won't have to yearn for a lot longer. eBPF (extended Berkeley Packet Filters) is a kernel technology that enables a plethora of diagnostic scenarios by introducing dynamic, safe, low-overhead, efficient programs that run in the context of your live kernel. Sure, BPF programs can attach to sockets; but more interestingly, they can attach to kprobes and uprobes, static kernel tracepoints, and even user-mode static probes. And modern BPF programs have access to a wide set of instructions and data structures, which means you can collect valuable information and analyze it on-the-fly, without spilling it to huge files and reading them from user space.
In this talk, we will introduce BCC, the BPF Compiler Collection, which is an open set of tools and libraries for dynamic tracing on Linux. Some tools are easy and ready to use, such as execsnoop, fileslower, and memleak. Other tools such as trace and argdist require more sophistication and can be used as a Swiss Army knife for a variety of scenarios. We will spend most of the time demonstrating the power of modern dynamic tracing -- from memory leaks to static probes in Ruby, Node, and Java programs, from slow file I/O to monitoring network traffic. Finally, we will discuss building our own tools using the Python and Lua bindings to BCC, and its LLVM backend.
Porting and Optimization of Numerical Libraries for ARM SVE
By Toshiyuki Imamura, RIKEN AICS
RIKEN and Fujitsu are developing ARM-based numerical libraries optimized with the new feature of ARM-SVE. We present porting status of netlib+SSL-II for ARM-SVE and other OSS. Also, we demonstrate some optimization policies and techniques, especially for the basic numerical linear algebra kernels.
Toshiyuki Imamura Bio
Toshiyuki Imamura is currently a team leader of Large-scale Parallel Numerical Computing Technology at Advanced Institute for Computational Science (AICS), RIKEN. He is in charge of the development of numerical libraries for the post-K project. His research interests include high-performance computing, automatic-tuning technology, eigenvalue computation (algorithm/software/applications), etc. He and his colleagues (Japan Atomic Energy Agency (JAEA) team) were nominated as one of the finalists of Gordon Bell Prize in SC05 and SC06. He is a member of IPSJ, JSIAM, and SIAM.
Email
imamura.toshiyuki@riken.jp
For more info on The Linaro High Performance Computing (HPC) visit https://www.linaro.org/sig/hpc/
BPF / XDP 8월 세미나 KossLab
BPF (Berkeley Packet Filter) allows for safe dynamic program injection into the Linux kernel. It provides an in-kernel virtual machine and instruction set for running custom programs. The BPF infrastructure includes a verifier that checks programs for safety, helper functions to access kernel APIs, and maps for inter-process communication. BPF has become a core kernel subsystem and is used for applications like XDP, tracing, networking, and more.
Berkeley Packet Filters
This talk is all about the Berkeley Packet Filters (BPF) and their uses in Linux.
Agenda:
* What is a BPF and why do we need it?
* Writing custom BPFs
* Notes on BPF implementation in the kernel
* Usage examples: SOCKET_FILTER & seccomp
Speaker:
Kfir Gollan, senior embedded software developer, Linux kernel hacker and software team leader.
bcc/BPF tools - Strategy, current tools, future challenges
Brendan Gregg discusses the current state and future potential of BPF and BCC tools for observability in Linux. He outlines 18 areas where BPF support has progressed and 16 areas still needing work. Gregg also discusses challenges like dynamic tracing stability, overhead, ease of coding, and developing visualizations. He proposes finishing ports of his old DTrace tools and links to resources on BPF, BCC, and flame graphs.
Bpf performance tools chapter 4 bcc
Study report about BPF Performance Tools (book)
The video is in https://www.youtube.com/watch?v=GlKtraMn4Ic
Linux kernel tracing superpowers in the cloud
The Linux 4.x series introduced a new powerful engine of programmable tracing (BPF) that allows to actually look inside the kernel at runtime. This talk will show you how to exploit this engine in order to debug problems or identify performance bottlenecks in a complex environment like a cloud. This talk will cover the latest Linux superpowers that allow to see what is happening “under the hood” of the Linux kernel at runtime. I will explain how to exploit these “superpowers” to measure and trace complex events at runtime in a cloud environment. For example, we will see how we can measure latency distribution of filesystem I/O, details of storage device operations, like individual block I/O request timeouts, or TCP buffer allocations, investigating stack traces of certain events, identify memory leaks, performance bottlenecks and a whole lot more.
Spying on the Linux kernel for fun and profit
Do you ever wonder what the kernel is doing while your code is running? This talk will explore some methodologies and techniques (eBPF, ftrace, etc.) to look under the hood of the Linux kernel and understand what it’s actually doing behind the scenes.
Linux 4.x Tracing Tools: Using BPF Superpowers
Talk for USENIX LISA 2016 by Brendan Gregg.
"Linux 4.x Tracing Tools: Using BPF Superpowers
The Linux 4.x series heralds a new era of Linux performance analysis, with the long-awaited integration of a programmable tracer: Enhanced BPF (eBPF). Formally the Berkeley Packet Filter, BPF has been enhanced in Linux to provide system tracing capabilities, and integrates with dynamic tracing (kprobes and uprobes) and static tracing (tracepoints and USDT). This has allowed dozens of new observability tools to be developed so far: for example, measuring latency distributions for file system I/O and run queue latency, printing details of storage device I/O and TCP retransmits, investigating blocked stack traces and memory leaks, and a whole lot more. These lead to performance wins large and small, especially when instrumenting areas that previously had zero visibility. Tracing superpowers have finally arrived.
In this talk I'll show you how to use BPF in the Linux 4.x series, and I'll summarize the different tools and front ends available, with a focus on iovisor bcc. bcc is an open source project to provide a Python front end for BPF, and comes with dozens of new observability tools (many of which I developed). These tools include new BPF versions of old classics, and many new tools, including: execsnoop, opensnoop, funccount, trace, biosnoop, bitesize, ext4slower, ext4dist, tcpconnect, tcpretrans, runqlat, offcputime, offwaketime, and many more. I'll also summarize use cases and some long-standing issues that can now be solved, and how we are using these capabilities at Netflix."
Tracing MariaDB server with bpftrace - MariaDB Server Fest 2021
Bpftrace is a relatively new eBPF-based open source tracer for modern Linux versions (kernels 5.x.y) that is useful for analyzing production performance problems and troubleshooting software. Basic usage of the tool, as well as bpftrace one liners and advanced scripts useful for MariaDB DBAs are presented. Problems of MariaDB Server dynamic tracing with bpftrace and some possible solutions and alternative tracing tools are discussed.
ATO Linux Performance 2018
Talk by Brendan Gregg for All Things Open 2018. "At over one thousand code commits per week, it's hard to keep up with Linux developments. This keynote will summarize recent Linux performance features,
for a wide audience: the KPTI patches for Meltdown, eBPF for performance observability and the new open source tools that use it, Kyber for disk I/O sc
heduling, BBR for TCP congestion control, and more. This is about exposure: knowing what exists, so you can learn and use it later when needed. Get the
most out of your systems with the latest Linux kernels and exciting features."
Linux 4.x Tracing: Performance Analysis with bcc/BPF
Talk about bcc/eBPF for SCALE15x (2017) by Brendan Gregg. "BPF (Berkeley Packet Filter) has been enhanced in the Linux 4.x series and now powers a large collection of performance analysis and observability tools ready for you to use, included in the bcc (BPF Complier Collection) open source project. BPF nowadays can do system tracing, software defined networks, and kernel fast path: much more than just filtering packets! This talk will focus on the bcc/BPF tools for performance analysis, which make use of other built in Linux capabilities: dynamic tracing (kprobes and uprobes) and static tracing (tracepoints and USDT). There are now bcc tools for measuring latency distributions for file system I/O and run queue latency, printing details of storage device I/O and TCP retransmits, investigating blocked stack traces and memory leaks, and a whole lot more. These lead to performance wins large and small, especially when instrumenting areas that previously had zero visibility. Tracing superpowers have finally arrived, built in to Linux."
Cookies for kernel developers
Maintaining stable Linux kernels isn't easy. This talk covers the work we are doing at Red Hat to automate the testing of Linux kernels and find bugs before they make it into stable kernels.
More Related Content
What's hot
Staring into the eBPF Abyss
eBPF (extended Berkeley Packet Filters) is a modern kernel technology that can be used to introduce dynamic tracing into a system that wasn't prepared or instrumented in any way. The tracing programs run in the kernel, are guaranteed to never crash or hang your system, and can probe every module and function -- from the kernel to user-space frameworks such as Node and Ruby.
In this workshop, you will experiment with Linux dynamic tracing first-hand. First, you will explore BCC, the BPF Compiler Collection, which is a set of tools and libraries for dynamic tracing. Many of your tracing needs will be answered by BCC, and you will experiment with memory leak analysis, generic function tracing, kernel tracepoints, static tracepoints in user-space programs, and the "baked" tools for file I/O, network, and CPU analysis. You'll be able to choose between working on a set of hands-on labs prepared by the instructors, or trying the tools out on your own test system.
Next, you will hack on some of the bleeding edge tools in the BCC toolkit, and build a couple of simple tools of your own. You'll be able to pick from a curated list of GitHub issues for the BCC project, a set of hands-on labs with known "school solutions", and an open-ended list of problems that need tools for effective analysis. At the end of this workshop, you will be equipped with a toolbox for diagnosing issues in the field, as well as a framework for building your own tools when the generic ones do not suffice.
UM2019 Extended BPF: A New Type of Software
BPF (Berkeley Packet Filter) has evolved from a limited virtual machine for efficient packet filtering to a new type of software called extended BPF. Extended BPF allows for custom, efficient, and production-safe performance analysis tools and observability programs to be run in the Linux kernel through BPF. It enables new event-based applications running as BPF programs attached to various kernel events like kprobes, uprobes, tracepoints, sockets, and more. Major companies like Facebook, Google, and Netflix are using BPF programs for tasks like intrusion detection, container security, firewalling, and observability with over 150,000 AWS instances running BPF programs. BPF provides a new program model and security features compared
Meet cute-between-ebpf-and-tracing
This document discusses how eBPF (extended Berkeley Packet Filter) can be used for kernel tracing. It provides an overview of BPF and eBPF, how eBPF programs are compiled and run in the kernel, the use of BPF maps, and how eBPF enables new possibilities for dynamic kernel instrumentation through techniques like Kprobes and ftrace.
Tracer Evaluation
Performance evaluation of various implementations of off-cpu wakeup path tracers for Linux kernel, using kprobe, ftrace, and ebpf.
Building Network Functions with eBPF & BCC
eBPF (Extended Berkeley Packet Filter) is an in-kernel virtual machine that allows running user-supplied sandboxed programs inside of the kernel. It is especially well-suited to network programs and it's possible to write programs that filter traffic, classify traffic and perform high-performance custom packet processing.
BCC (BPF Compiler Collection) is a toolkit for creating efficient kernel tracing and manipulation programs. It makes use of eBPF.
BCC provides an end-to-end workflow for developing eBPF programs and supplies Python bindings, making eBPF programs much easier to write.
Together, eBPF and BCC allow you to develop and deploy network functions safely and easily, focusing on your application logic (instead of kernel datapath integration).
In this session, we will introduce eBPF and BCC, explain how to implement a network function using BCC, discuss some real-life use-cases and show a live demonstration of the technology.
About the speaker
Shmulik Ladkani, Chief Technology Officer at Meta Networks,
Long time network veteran and kernel geek.
Shmulik started his career at Jungo (acquired by NDS/Cisco) implementing residential gateway software, focusing on embedded Linux, Linux kernel, networking and hardware/software integration.
Some billions of forwarded packets later, Shmulik left his position as Jungo's lead architect and joined Ravello Systems (acquired by Oracle) as tech lead, developing a virtual data center as a cloud-based service, focusing around virtualization systems, network virtualization and SDN.
Recently he co-founded Meta Networks where he's been busy architecting secure, multi-tenant, large-scale network infrastructure as a cloud-based service.
Beautiful History
The document discusses the history of software and version control using Git. It provides examples of Git commands like git log, git rebase, and git merge. It also discusses best practices for writing commit messages and splitting commits into logical changes.
eBPF Trace from Kernel to Userspace
This document discusses eBPF (extended Berkeley Packet Filter), which allows tracing from the Linux kernel to userspace using BPF programs. It provides an overview of eBPF including extended registers, verification, maps, and probes. Examples are given of using eBPF for tracing functions like kfree_skb() and the C library function malloc. The Berkeley Compiler Collection (BCC) makes it easy to write eBPF programs in C and Python.
Linux kernel-rootkit-dev - Wonokaerun
This document contains the slides from a presentation given by WonoKaerun at the Indonesian Security Conference 2011 in Palembang. The presentation introduces rootkits and techniques for hiding malware at the kernel level on Linux systems. It covers topics like loadable kernel modules, interrupt descriptor table hooking, virtual file system hacking, page fault handler hijacking, debugging register abuse, and kernel instrumentation patching. The goal is to evade detection by security solutions by gaining control of the kernel before anti-rootkit defenses can activate. Throughout, the document emphasizes the cat-and-mouse nature of offensive and defensive security research.
The Next Linux Superpower: eBPF Primer
Imagine you're tackling one of these evasive performance issues in the field, and your go-to monitoring checklist doesn't seem to cut it. There are plenty of suspects, but they are moving around rapidly and you need more logs, more data, more in-depth information to make a diagnosis. Maybe you've heard about DTrace, or even used it, and are yearning for a similar toolkit, which can plug dynamic tracing into a system that wasn't prepared or instrumented in any way.
Hopefully, you won't have to yearn for a lot longer. eBPF (extended Berkeley Packet Filters) is a kernel technology that enables a plethora of diagnostic scenarios by introducing dynamic, safe, low-overhead, efficient programs that run in the context of your live kernel. Sure, BPF programs can attach to sockets; but more interestingly, they can attach to kprobes and uprobes, static kernel tracepoints, and even user-mode static probes. And modern BPF programs have access to a wide set of instructions and data structures, which means you can collect valuable information and analyze it on-the-fly, without spilling it to huge files and reading them from user space.
In this talk, we will introduce BCC, the BPF Compiler Collection, which is an open set of tools and libraries for dynamic tracing on Linux. Some tools are easy and ready to use, such as execsnoop, fileslower, and memleak. Other tools such as trace and argdist require more sophistication and can be used as a Swiss Army knife for a variety of scenarios. We will spend most of the time demonstrating the power of modern dynamic tracing -- from memory leaks to static probes in Ruby, Node, and Java programs, from slow file I/O to monitoring network traffic. Finally, we will discuss building our own tools using the Python and Lua bindings to BCC, and its LLVM backend.
Porting and Optimization of Numerical Libraries for ARM SVE
By Toshiyuki Imamura, RIKEN AICS
RIKEN and Fujitsu are developing ARM-based numerical libraries optimized with the new feature of ARM-SVE. We present porting status of netlib+SSL-II for ARM-SVE and other OSS. Also, we demonstrate some optimization policies and techniques, especially for the basic numerical linear algebra kernels.
Toshiyuki Imamura Bio
Toshiyuki Imamura is currently a team leader of Large-scale Parallel Numerical Computing Technology at Advanced Institute for Computational Science (AICS), RIKEN. He is in charge of the development of numerical libraries for the post-K project. His research interests include high-performance computing, automatic-tuning technology, eigenvalue computation (algorithm/software/applications), etc. He and his colleagues (Japan Atomic Energy Agency (JAEA) team) were nominated as one of the finalists of Gordon Bell Prize in SC05 and SC06. He is a member of IPSJ, JSIAM, and SIAM.
Email
imamura.toshiyuki@riken.jp
For more info on The Linaro High Performance Computing (HPC) visit https://www.linaro.org/sig/hpc/
BPF / XDP 8월 세미나 KossLab
BPF (Berkeley Packet Filter) allows for safe dynamic program injection into the Linux kernel. It provides an in-kernel virtual machine and instruction set for running custom programs. The BPF infrastructure includes a verifier that checks programs for safety, helper functions to access kernel APIs, and maps for inter-process communication. BPF has become a core kernel subsystem and is used for applications like XDP, tracing, networking, and more.
Berkeley Packet Filters
This talk is all about the Berkeley Packet Filters (BPF) and their uses in Linux.
Agenda:
* What is a BPF and why do we need it?
* Writing custom BPFs
* Notes on BPF implementation in the kernel
* Usage examples: SOCKET_FILTER & seccomp
Speaker:
Kfir Gollan, senior embedded software developer, Linux kernel hacker and software team leader.
bcc/BPF tools - Strategy, current tools, future challenges
Brendan Gregg discusses the current state and future potential of BPF and BCC tools for observability in Linux. He outlines 18 areas where BPF support has progressed and 16 areas still needing work. Gregg also discusses challenges like dynamic tracing stability, overhead, ease of coding, and developing visualizations. He proposes finishing ports of his old DTrace tools and links to resources on BPF, BCC, and flame graphs.
Bpf performance tools chapter 4 bcc
Study report about BPF Performance Tools (book)
The video is in https://www.youtube.com/watch?v=GlKtraMn4Ic
Linux kernel tracing superpowers in the cloud
The Linux 4.x series introduced a new powerful engine of programmable tracing (BPF) that allows to actually look inside the kernel at runtime. This talk will show you how to exploit this engine in order to debug problems or identify performance bottlenecks in a complex environment like a cloud. This talk will cover the latest Linux superpowers that allow to see what is happening “under the hood” of the Linux kernel at runtime. I will explain how to exploit these “superpowers” to measure and trace complex events at runtime in a cloud environment. For example, we will see how we can measure latency distribution of filesystem I/O, details of storage device operations, like individual block I/O request timeouts, or TCP buffer allocations, investigating stack traces of certain events, identify memory leaks, performance bottlenecks and a whole lot more.
Spying on the Linux kernel for fun and profit
Do you ever wonder what the kernel is doing while your code is running? This talk will explore some methodologies and techniques (eBPF, ftrace, etc.) to look under the hood of the Linux kernel and understand what it’s actually doing behind the scenes.
Linux 4.x Tracing Tools: Using BPF Superpowers
Talk for USENIX LISA 2016 by Brendan Gregg.
"Linux 4.x Tracing Tools: Using BPF Superpowers
The Linux 4.x series heralds a new era of Linux performance analysis, with the long-awaited integration of a programmable tracer: Enhanced BPF (eBPF). Formally the Berkeley Packet Filter, BPF has been enhanced in Linux to provide system tracing capabilities, and integrates with dynamic tracing (kprobes and uprobes) and static tracing (tracepoints and USDT). This has allowed dozens of new observability tools to be developed so far: for example, measuring latency distributions for file system I/O and run queue latency, printing details of storage device I/O and TCP retransmits, investigating blocked stack traces and memory leaks, and a whole lot more. These lead to performance wins large and small, especially when instrumenting areas that previously had zero visibility. Tracing superpowers have finally arrived.
In this talk I'll show you how to use BPF in the Linux 4.x series, and I'll summarize the different tools and front ends available, with a focus on iovisor bcc. bcc is an open source project to provide a Python front end for BPF, and comes with dozens of new observability tools (many of which I developed). These tools include new BPF versions of old classics, and many new tools, including: execsnoop, opensnoop, funccount, trace, biosnoop, bitesize, ext4slower, ext4dist, tcpconnect, tcpretrans, runqlat, offcputime, offwaketime, and many more. I'll also summarize use cases and some long-standing issues that can now be solved, and how we are using these capabilities at Netflix."
Tracing MariaDB server with bpftrace - MariaDB Server Fest 2021
Bpftrace is a relatively new eBPF-based open source tracer for modern Linux versions (kernels 5.x.y) that is useful for analyzing production performance problems and troubleshooting software. Basic usage of the tool, as well as bpftrace one liners and advanced scripts useful for MariaDB DBAs are presented. Problems of MariaDB Server dynamic tracing with bpftrace and some possible solutions and alternative tracing tools are discussed.
ATO Linux Performance 2018
Talk by Brendan Gregg for All Things Open 2018. "At over one thousand code commits per week, it's hard to keep up with Linux developments. This keynote will summarize recent Linux performance features,
for a wide audience: the KPTI patches for Meltdown, eBPF for performance observability and the new open source tools that use it, Kyber for disk I/O sc
heduling, BBR for TCP congestion control, and more. This is about exposure: knowing what exists, so you can learn and use it later when needed. Get the
most out of your systems with the latest Linux kernels and exciting features."
Linux 4.x Tracing: Performance Analysis with bcc/BPF
Talk about bcc/eBPF for SCALE15x (2017) by Brendan Gregg. "BPF (Berkeley Packet Filter) has been enhanced in the Linux 4.x series and now powers a large collection of performance analysis and observability tools ready for you to use, included in the bcc (BPF Complier Collection) open source project. BPF nowadays can do system tracing, software defined networks, and kernel fast path: much more than just filtering packets! This talk will focus on the bcc/BPF tools for performance analysis, which make use of other built in Linux capabilities: dynamic tracing (kprobes and uprobes) and static tracing (tracepoints and USDT). There are now bcc tools for measuring latency distributions for file system I/O and run queue latency, printing details of storage device I/O and TCP retransmits, investigating blocked stack traces and memory leaks, and a whole lot more. These lead to performance wins large and small, especially when instrumenting areas that previously had zero visibility. Tracing superpowers have finally arrived, built in to Linux."
What's hot (20)
Porting and Optimization of Numerical Libraries for ARM SVE
Porting and Optimization of Numerical Libraries for ARM SVE
bcc/BPF tools - Strategy, current tools, future challenges
bcc/BPF tools - Strategy, current tools, future challenges
Tracing MariaDB server with bpftrace - MariaDB Server Fest 2021
Tracing MariaDB server with bpftrace - MariaDB Server Fest 2021
Linux 4.x Tracing: Performance Analysis with bcc/BPF
Linux 4.x Tracing: Performance Analysis with bcc/BPF
Similar to Linux System Monitoring with eBPF
Cookies for kernel developers
Maintaining stable Linux kernels isn't easy. This talk covers the work we are doing at Red Hat to automate the testing of Linux kernels and find bugs before they make it into stable kernels.
Scaleable PHP Applications in Kubernetes
Kubernetes is also called the "distributed Linux of the cloud" – which implies that it provides fundamental infrastructure, which can solve a lot of challenges. Let’s see how PHP applications fit into this picture. In this presentation, we are going to explore when Kubernetes is a good fit for operating your PHP application and how it can be done in practice. We’ll look at the whole lifecycle: how to build your application, create or choose the right Docker images, deploy and scale, and how to deal with performance and monitoring. At the end you will have a good understanding about all the different stages and building blocks for running a PHP application with Kubernetes in production.
Scientific Computing @ Fred Hutch
At the technology meeting of the Association of Independent Research Centers (http://airi.org): An overview of recent Scientific Computing activities at Fred Hutch, Seattle
Kernel Recipes 2017 - An introduction to the Linux DRM subsystem - Maxime Ripard
Every modern multimedia-oriented ARM SoC usually has a number of display controllers, to drive a screen or an LCD panel, and a GPU, to provide 3D acceleration. The Linux kernel framework of choice to support these controllers is the DRM subsystem.
This talk will walk through the DRM stack, the architecture of a DRM/KMS driver and the interaction between the display and GPU drivers. The presentation is based on the work we have done to develop a DRM driver for the Allwinner SoCs display controller with multiple outputs, such as parallel display interfaces, HDMI or MIPI-DSI. The work done to make the ARM Mali OpenGL driver work on top of a mainline DRM/KMS driver will also be detailed, as well as the more traditional, Mesa-based, solution used in a variety of other platforms.
Maxime Ripard, Free Electrons
CentOS Stream at Facebook
We'll talk about how Facebook is leveraging CentOS Stream to manage our production fleet at scale. We'll cover the latest updates on our fleet migration from CentOS 7, talk about the tooling and processes we've developed and how they've evolved, and how we're working with the CentOS and Fedora communities. This talk is a followup to "Upgrading CentOS on the Facebook fleet" (https://www.youtube.com/watch?v=EajAjFCZz4Q&t=3s) from DevConf.cz 2020.
EMBA - From Firmware to Exploit - BHEU22
See also https://www.blackhat.com/eu-22/arsenal/schedule/index.html#emba--from-firmware-to-exploit-29185
NW/MET 2017 - The Media Ecosystem in Higher Education - Capture
The document discusses the media ecosystem in higher education, focusing on lecture capture. It defines capture as the recording of lecture and course material, which can be done through various technologies from software to hardware to studios. It then summarizes the results of evaluating six hardware lecture capture appliances based on 27 criteria like recording multiple sources, streaming options, and scheduling. The top scoring appliances were the Ncast Presentation Recorder Hydra and Extron SMP-351 based on functionality and affordability.
Saving Human Lives with the IoT
Slides for my talk at Cloud Foundry Summit Europe 2016.
Nearly 1.2 million people die in road crashes each year (WHO - 2015) with additional millions becoming injured or disabled. One big part of this problem is worst road traffic conditions and unless action is taken, road traffic injuries are predicted to become the fifth leading cause of death by 2030. Moreover, although road traffic injuries have been a major cause of mortality for many years, most traffic accidents are both predictable and preventable. In this talk, we want to demonstrate a scalable IoT platform that uses weather data and data from other cars to warn drivers of dangerous conditions. We will show how CF can help to save human lives and the architecture behind this. Additionally, we will also explain the data science that is involved.
Arm - ceph on arm update
講者：
Jeff Chu (Director of Enterprise Solutions, ARM)
Kan Yan Rong (Technical Expert in Storage and Application) Technology, WDC/SanDisk)
概要：
Jeff from ARM will provide a brief update on the activities furthering Ceph on ARM including some recent progress from ARM as well some increased community activity. After that Chris and Yan from Western Digital/San Disk will be presenting the topic on Ceph Block Performance on Cavium ARM and SATA SSDs.
EMBA - Firmware analysis - Black Hat Arsenal USA 2022
IoT (Internet of Things) and OT (Operational Technology) are the current buzzwords for networked devices on which our modern society is based on. In this area, the used operating systems are summarized with the term firmware. The devices themselves, also called embedded devices, are essential in the private and industrial environments as well as in the so-called critical infrastructure.
Penetration testing of these systems is quite complex as we have to deal with different architectures, optimized operating systems and special protocols. EMBA is an open-source firmware analyzer with the goal to simplify and optimize the complex task of firmware security analysis. EMBA supports the penetration tester with the automated detection of 1-day vulnerabilities on binary level. This goes far beyond the plain CVE detection: With EMBA you always know which public exploits are available for the target firmware. Besides the detection of already known vulnerabilities, EMBA also supports the tester on the next 0-day. For this, EMBA identifies critical binary functions, protection mechanisms and services with network behavior on a binary level. There are many other features built into EMBA, such as fully automated firmware extraction, finding file system vulnerabilities, hard-coded credentials, and more.
EMBA is the open-source firmware scanner, created by penetration testers for penetration testers.
Project page: https://github.com/e-m-b-a/emba
Conference page: https://www.blackhat.com/us-22/arsenal/schedule/index.html#emba--open-source-firmware-security-testing-26596
Andrii Soldatenko "The art of data engineering"
As the data space has increased, data engineering has emerged as a separate and related role that works together with data scientists. Usually, data scientists focus on finding new insights from a data set, while data engineers are concerned with the production readiness of that data.
In this talk, I’ll show you how to gather and collect the huge amount of data, store it, do batch processing or real-time processing on it, and how to build a data pipeline using Airflow for processing billions of records per table.
Also, we will discuss what is big data, and why it’s important to be able to process it so quick.
Linux Container Primitives and Runtimes - AWS Summit Sydney
In this session we'll explore the different Linux primitives that are commonly used in implementing container runtimes. We’ll learn about cgroups, namespaces and union filesystems and explain how these are leveraged by container runtimes like Docker to deliver powerful container management system. In this session we’ll demonstrate how Docker uses each of these primitives and show how you can effectively inspect and troubleshoot containers from the host operating system.
Arch linux and whole security concepts in linux explained
this presentation explain about Arch linux and whole security concepts in linux explained for saying thanks to me for this presentation you can follow me on twitter or github that links given in my website link thank you for reaching here .god bless you all
Backups.pptx
This document outlines slides from a lecture on advanced network and system administration backups. It discusses key backup decisions around why, what, when, where, who and how to back up. It also covers backup types, hardware, software, capacity planning, automation, security and references. The goal is to provide a comprehensive overview of planning and implementing a backup strategy and system.
DevOps in Age of Kubernetes
There is a transformation brewing for DevOps in age of Kubernetes. The tools of the trade, configuration management solutions, have been superseded in agility and preference by development teams who want the declarative choreography of containerized applications. The new preference for mixing developer and operations is the site reliability engineering (SRE) model championed by Google. In this new structure, the need to automate doesn’t stop at the containerized application and DevOps professionals should seek to automate the Kubernetes service itself.
CS403: Operating System : Lec 3 Popular types of OS (2).pptx
This document summarizes the topics covered in Lecture 3 of an Operating Systems course. It discusses different types of operating systems including desktop systems like Windows, Mac OS, Linux and Chrome OS as well as server systems like Windows Server and UNIX. It also covers mobile operating systems including iOS and Android. Popular operating systems are described like DOS, Windows, Mac OS, Linux, Chrome OS, Windows Server and UNIX. The advantages of the Linux operating system are listed as being open source, secure, free, lightweight, stable and high performance.
Hortonworks on IBM POWER Analytics / AI
In this session I will tell you what Hortonworks and IBM Power solutions are and how we can realize significant business value development and prompt use of open innovation in future cognitive utilization. In addition, I will introduce the value added unique to IBM that can be provided by IBM and Hortonworks partnership from the viewpoint of storage, analytics, data science and streaming analysis.
Google Cloud Computing on Google Developer 2008 Day
The document discusses the evolution of computing models from clusters and grids to cloud computing. It describes how cluster computing involved tightly coupled resources within a LAN, while grids allowed for resource sharing across domains. Utility computing introduced an ownership model where users leased computing power. Finally, cloud computing allows access to services and data from any internet-connected device through a browser.
Appearances are deceiving: Novel offensive techniques in Windows 10/11 on ARM
In 2017, Microsoft announced the ARM version of Windows. The number of devices with ARM version of Windows is increasing, such as Surface Pro X series and HP ENVY x2, and it is gradually becoming popular.
When using these ARM devices, there is a compatibility issue that existing x86/x64 applications cannot be used.
However, this problem has been addressed by providing x86/x64 emulation capabilities. In recent years, ARM64EC has been announced, allowing for the gradual migration of x64 applications to ARM. The aggressive introduction of these compatibility technologies is a sign of Microsoft's strong will to promote the ARM version of Windows.
On the other hand, doesn't the introduction of new compatibility technologies provide a new avenue of attack for attackers? As far as we know, this point has not even been discussed much at this point. Therefore, we reverse engineered the compatibility technology that exists in Windows on ARM and examined its exploitability.
We found that various techniques are available, such as code injection by modifying XTA cache files, and obfuscation by exploiting newly introduced relocation entries. All of these techniques have in common the characteristic that the binary "appearance" and runtime behavior are different, making them difficult to detect and track. In addition, some of the techniques can be widely exploited to interfere with static analysis or sandbox analysis. Therefore, there is a high possibility that they will become a threat to the ARM version of Windows in the future.
In this presentation, we will explain the details of our new method and its features with demonstrations. We hope that this presentation will be a good opportunity to develop and promote the security research of Windows on ARM.
The PoC code and detailed reverse engineering results will be available on GitHub.
Similar to Linux System Monitoring with eBPF (20)
Kernel Recipes 2017 - An introduction to the Linux DRM subsystem - Maxime Ripard
Kernel Recipes 2017 - An introduction to the Linux DRM subsystem - Maxime Ripard
NW/MET 2017 - The Media Ecosystem in Higher Education - Capture
NW/MET 2017 - The Media Ecosystem in Higher Education - Capture
EMBA - Firmware analysis - Black Hat Arsenal USA 2022
EMBA - Firmware analysis - Black Hat Arsenal USA 2022
Linux Container Primitives and Runtimes - AWS Summit Sydney
Linux Container Primitives and Runtimes - AWS Summit Sydney
Arch linux and whole security concepts in linux explained
Arch linux and whole security concepts in linux explained
CS403: Operating System : Lec 3 Popular types of OS (2).pptx
CS403: Operating System : Lec 3 Popular types of OS (2).pptx
Google Cloud Computing on Google Developer 2008 Day
Google Cloud Computing on Google Developer 2008 Day
Appearances are deceiving: Novel offensive techniques in Windows 10/11 on ARM
Appearances are deceiving: Novel offensive techniques in Windows 10/11 on ARM
More from Heinrich Hartmann
Latency SLOs Done Right @ SREcon EMEA 2019
Latency is a key indicator of service quality, and important to measure and track. However, measuring latency correctly is not easy. In contrast to familiar metrics like CPU utilization or request counts, the "latency" of a service is not easily expressed in numbers. Percentile metrics have become a popular means to measure the request latency, but have several shortcomings, especially when it comes to aggregation. The situation is particularly dire if we want to use them to specify Service Level Objectives (SLOs) that quantify the performance over a longer time horizons. In the talk we will explain these pitfalls, and suggest three practical methods how to implement effective Latency SLOs.
Circonus: Design failures - A Case Study
SRECon18 Keynote https://www.usenix.org/conference/srecon18europe/presentation/schlossnagle
Joint talk Theo Schlossnagle (p1-26), Heinrich Hartmann (p26-44ff).
Statistics for Engineers
1) Heinrich Hartmann presented on statistics and monitoring for engineers. He discussed various methods for API monitoring including external monitoring, log analysis, and measuring latency averages and percentiles.
2) Histograms were presented as another method that involves dividing the latency and time scales into bands and reporting periods to count samples, allowing flexible analysis while enabling aggregation.
3) Takeaways included being wary of line graphs, not aggregating percentiles but instead using histograms, keeping all raw data, and striving for meaningful metrics.
Scalable Online Analytics for Monitoring
Monitoring systems will get smarter in order to keep up with the demands of tomorrow's IT architectures. Features like anomaly detection, root cause analysis, and forecasting tools will be critical components of this next level of monitoring. At the same time, the data that monitoring systems ingest is ever increasing in amount and velocity.
This session covers architectural models for advanced online analytics. We argue that stateful online computations provide a means to realize machine learning on high-velocity data. We show how alerting systems, event engines, stream aggregators, and time-series databases interact to support smart, scalable, and resilient monitoring solutions.
Heinrich Hartmann is the Chief Data Scientist at Circonus. He is driving the development of analytics methods that transform monitoring data into actionable information as part of the Circonus monitoring platform. In his prior life, Heinrich pursued an academic career as a mathematician (PhD in Bonn, Oxford). Later he transitioned into computer science and worked as consultant for a number of different companies and research institutions.
Geometric Aspects of LSA
The document discusses latent semantic analysis (LSA) for determining the similarity of topics between documents. It outlines that LSA uses singular value decomposition to project documents into a lower dimensional space where similar topics are closer. The presentation will cover the geometry of SVD used in LSA and how the extremal vector in the reduced space represents the dominant topic that is common across documents.
Seminar on Complex Geometry
This seminar will cover the basics of complex manifolds and geometry through readings and presentations on topics from reference book [2]. The seminar will be organized as a reading group, with participants reading assigned material each week and one person giving a talk on a particular topic. Some of the topics to be covered include holomorphic functions, complex manifolds, vector bundles, divisors, projective space, differential forms, Kähler manifolds, and applications to curves.
Seminar on Motivic Hall Algebras
This document outlines a seminar on motivic Hall algebras. The seminar will cover topics including Grothendieck rings of varieties, motivic invariants like Euler characteristics and Hodge polynomials, algebraic spaces, and stacks. The main focus will be T. Bridgeland's paper on motivic Hall algebras, which uses motives and stacks in its construction of motivic Hall algebras.
GROUPOIDS, LOCAL SYSTEMS AND DIFFERENTIAL EQUATIONS
This document discusses groupoids, local systems, and their relationships to differential equations on manifolds. Some key points:
1) Groupoids generalize groups by allowing multiple objects and isomorphisms between them. Representations of groupoids correspond to local systems on manifolds.
2) Local systems on a manifold X are sheaves of vector spaces that are locally isomorphic to a constant sheaf. They correspond to representations of the fundamental groupoid of X.
3) Vector bundles with connections on a Riemann surface B are equivalent to local systems on B. Global sections of bundles generate differential equations, whose solutions can be studied via the bundle's local system or groupoid representation.
Topics in Category Theory
This document discusses topics in category theory, including set-functors, adjunctions, and limits. It begins by defining set-functors and natural transformations between them. It notes that a natural transformation is uniquely determined by its value on an initial element of a functor. It then introduces adjunctions and decomposes them into left and right adjoints. It shows that a left adjoint exists if and only if certain set-functors are representable. Finally, it defines limits of diagrams (I-systems) over an index category I. It shows that a limit exists if and only if the cone functor is representable.
Related-Work.net at WeST Oberseminar
This document discusses Related-Work.net, a scientific discussion platform and open database of papers and citations. It describes the vision for the platform to be a social community for scientists with open data and free software. The document outlines the history of Related-Work.net and its merger with OpenCitations.net. It also describes some of the challenges in building the platform, such as matching citations to papers and identifying authors, as well as examples of potential data mining applications.
Komplexe Zahlen
Schülerprojekt 2002
Inhalt:
* Einführung Komplexe Zahlen
* Exponentialfunktion und DeMoiver Formeln
* Penningfallen
* Rollkurven
* Fraktale
Pushforward of Differential Forms
1) The document discusses fiber integrals of differential forms along oriented submersions. It defines the pushforward or fiber integral f* of a differential form along a submersion f, which maps forms on the total space to forms on the base space.
2) Properties of the fiber integral are proven, including normalization, the Fubini theorem, base change formulas, and Stokes' theorem.
3) The document also discusses orientations of vector bundles and submersions, which are needed to define the fiber integral.
Polynomproblem
Beweis der folgenden Aussage:
Sei f ∈ C ∞ (R) mit der Eigenschaft, dass ∀ x ∈ R ∃ n ∈ N mit f (n) (x) = 0. Dann ist f ein Polynom.
Hecke Curves and Moduli spcaes of Vector Bundles
This are the notes of a seminar talk delivered in summer 2008 at Bonn. Let SU (2, 1) be the
moduli space of rank 2 bundles with a fixed determinant of rank 1 over a curve C of genus g ≥ 2.
This is a Fano manifold of Picard rank 1. We discuss the example g = 2 where SU (2, 1) is the
intersection of two quadrics in P5 . In this case the minimal rational curves are lines. There is a
very interesting class of rational curves on SU (2, 1), called Hecke curves, which are constructed
by extending a given bundle by torsion sheaves. In the case g ≥ 3 we will see that Hecke curves
have minimal anti-canonical degree (4) and that any rational curve passing through a generic
point is a Hecke curve.
Local morphisms are given by composition
(1) Local morphisms between function-like ringed spaces are given by composition of functions. A morphism is local if the preimage of the maximal ideal at each point is the maximal ideal of the image point.
(2) A morphism is C-linear if it maps each function to the function composed with the continuous map. For a function-like ringed space, a morphism is local and C-linear if and only if it is given by composition of functions.
(3) The maximal ideal at each point of a function-like ringed space is the intersection of the maximal ideal of vanishing functions and the structure sheaf at that point.
Notes on Intersection theory
Notes on intersection theory written for a seminar in Bonn in 2010.
Following Fulton's book the following topics are covered:
- Motivation of intersection theory
- Cones and Segre Classes
- Chern Classes
- Gauss-Bonet Formula
- Segre classes under birational morphisms
- Flat pull back
More from Heinrich Hartmann (20)
GROUPOIDS, LOCAL SYSTEMS AND DIFFERENTIAL EQUATIONS
GROUPOIDS, LOCAL SYSTEMS AND DIFFERENTIAL EQUATIONS
Recently uploaded
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
Simplify your search for a reliable Python development partner! This list presents the top 10 trusted US providers offering comprehensive Python development services, ensuring your project's success from conception to completion.
TrustArc Webinar - 2024 Global Privacy Survey
How does your privacy program stack up against your peers? What challenges are privacy teams tackling and prioritizing in 2024?
In the fifth annual Global Privacy Benchmarks Survey, we asked over 1,800 global privacy professionals and business executives to share their perspectives on the current state of privacy inside and outside of their organizations. This year’s report focused on emerging areas of importance for privacy and compliance professionals, including considerations and implications of Artificial Intelligence (AI) technologies, building brand trust, and different approaches for achieving higher privacy competence scores.
See how organizational priorities and strategic approaches to data security and privacy are evolving around the globe.
This webinar will review:
- The top 10 privacy insights from the fifth annual Global Privacy Benchmarks Survey
- The top challenges for privacy leaders, practitioners, and organizations in 2024
- Key themes to consider in developing and maintaining your privacy program
20240607 QFM018 Elixir Reading List May 2024
Everything I found interesting about the Elixir programming ecosystem in May 2024
Main news related to the CCS TSI 2023 (2023/1695)
An English 🇬🇧 translation of a presentation to the speech I gave about the main changes brought by CCS TSI 2023 at the biggest Czech conference on Communications and signalling systems on Railways, which was held in Clarion Hotel Olomouc from 7th to 9th November 2023 (konferenceszt.cz). Attended by around 500 participants and 200 on-line followers.
The original Czech 🇨🇿 version of the presentation can be found here: https://www.slideshare.net/slideshow/hlavni-novinky-souvisejici-s-ccs-tsi-2023-2023-1695/269688092 .
The videorecording (in Czech) from the presentation is available here: https://youtu.be/WzjJWm4IyPk?si=SImb06tuXGb30BEH .
Introduction of Cybersecurity with OSS at Code Europe 2024
I develop the Ruby programming language, RubyGems, and Bundler, which are package managers for Ruby. Today, I will introduce how to enhance the security of your application using open-source software (OSS) examples from Ruby and RubyGems.
The first topic is CVE (Common Vulnerabilities and Exposures). I have published CVEs many times. But what exactly is a CVE? I'll provide a basic understanding of CVEs and explain how to detect and handle vulnerabilities in OSS.
Next, let's discuss package managers. Package managers play a critical role in the OSS ecosystem. I'll explain how to manage library dependencies in your application.
I'll share insights into how the Ruby and RubyGems core team works to keep our ecosystem safe. By the end of this talk, you'll have a better understanding of how to safeguard your code.
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Discover the seamless integration of RPA (Robotic Process Automation), COMPOSER, and APM with AWS IDP enhanced with Slack notifications. Explore how these technologies converge to streamline workflows, optimize performance, and ensure secure access, all while leveraging the power of AWS IDP and real-time communication via Slack notifications.
Energy Efficient Video Encoding for Cloud and Edge Computing Instances
Energy Efficient Video Encoding for Cloud and Edge Computing Instances
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for life science domain, where you retriever information from biomedical knowledge graphs using LLMs to increase the accuracy and performance of generated answers
Building Production Ready Search Pipelines with Spark and Milvus
Spark is the widely used ETL tool for processing, indexing and ingesting data to serving stack for search. Milvus is the production-ready open-source vector database. In this talk we will show how to use Spark to process unstructured data to extract vector representations, and push the vectors to Milvus vector database for search serving.
Programming Foundation Models with DSPy - Meetup Slides
Prompting language models is hard, while programming language models is easy. In this talk, I will discuss the state-of-the-art framework DSPy for programming foundation models with its powerful optimizers and runtime constraint system.
How to Get CNIC Information System with Paksim Ga.pptx
Pakdata Cf is a groundbreaking system designed to streamline and facilitate access to CNIC information. This innovative platform leverages advanced technology to provide users with efficient and secure access to their CNIC details.
UiPath Test Automation using UiPath Test Suite series, part 6
Welcome to UiPath Test Automation using UiPath Test Suite series part 6. In this session, we will cover Test Automation with generative AI and Open AI.
UiPath Test Automation with generative AI and Open AI webinar offers an in-depth exploration of leveraging cutting-edge technologies for test automation within the UiPath platform. Attendees will delve into the integration of generative AI, a test automation solution, with Open AI advanced natural language processing capabilities.
Throughout the session, participants will discover how this synergy empowers testers to automate repetitive tasks, enhance testing accuracy, and expedite the software testing life cycle. Topics covered include the seamless integration process, practical use cases, and the benefits of harnessing AI-driven automation for UiPath testing initiatives. By attending this webinar, testers, and automation professionals can gain valuable insights into harnessing the power of AI to optimize their test automation workflows within the UiPath ecosystem, ultimately driving efficiency and quality in software development processes.
What will you get from this session?
1. Insights into integrating generative AI.
2. Understanding how this integration enhances test automation within the UiPath platform
3. Practical demonstrations
4. Exploration of real-world use cases illustrating the benefits of AI-driven test automation for UiPath
Topics covered:
What is generative AI
Test Automation with generative AI and Open AI.
UiPath integration with generative AI
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Generating privacy-protected synthetic data using Secludy and Milvus
During this demo, the founders of Secludy will demonstrate how their system utilizes Milvus to store and manipulate embeddings for generating privacy-protected synthetic data. Their approach not only maintains the confidentiality of the original data but also enhances the utility and scalability of LLMs under privacy constraints. Attendees, including machine learning engineers, data scientists, and data managers, will witness first-hand how Secludy's integration with Milvus empowers organizations to harness the power of LLMs securely and efficiently.
Artificial Intelligence for XMLDevelopment
In the rapidly evolving landscape of technologies, XML continues to play a vital role in structuring, storing, and transporting data across diverse systems. The recent advancements in artificial intelligence (AI) present new methodologies for enhancing XML development workflows, introducing efficiency, automation, and intelligent capabilities. This presentation will outline the scope and perspective of utilizing AI in XML development. The potential benefits and the possible pitfalls will be highlighted, providing a balanced view of the subject.
We will explore the capabilities of AI in understanding XML markup languages and autonomously creating structured XML content. Additionally, we will examine the capacity of AI to enrich plain text with appropriate XML markup. Practical examples and methodological guidelines will be provided to elucidate how AI can be effectively prompted to interpret and generate accurate XML markup.
Further emphasis will be placed on the role of AI in developing XSLT, or schemas such as XSD and Schematron. We will address the techniques and strategies adopted to create prompts for generating code, explaining code, or refactoring the code, and the results achieved.
The discussion will extend to how AI can be used to transform XML content. In particular, the focus will be on the use of AI XPath extension functions in XSLT, Schematron, Schematron Quick Fixes, or for XML content refactoring.
The presentation aims to deliver a comprehensive overview of AI usage in XML development, providing attendees with the necessary knowledge to make informed decisions. Whether you’re at the early stages of adopting AI or considering integrating it in advanced XML development, this presentation will cover all levels of expertise.
By highlighting the potential advantages and challenges of integrating AI with XML development tools and languages, the presentation seeks to inspire thoughtful conversation around the future of XML development. We’ll not only delve into the technical aspects of AI-powered XML development but also discuss practical implications and possible future directions.
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
ABSTRACT: A prima vista, un mattoncino Lego e la backdoor XZ potrebbero avere in comune il fatto di essere entrambi blocchi di costruzione, o dipendenze di progetti creativi e software. La realtà è che un mattoncino Lego e il caso della backdoor XZ hanno molto di più di tutto ciò in comune.
Partecipate alla presentazione per immergervi in una storia di interoperabilità, standard e formati aperti, per poi discutere del ruolo importante che i contributori hanno in una comunità open source sostenibile.
BIO: Sostenitrice del software libero e dei formati standard e aperti. È stata un membro attivo dei progetti Fedora e openSUSE e ha co-fondato l'Associazione LibreItalia dove è stata coinvolta in diversi eventi, migrazioni e formazione relativi a LibreOffice. In precedenza ha lavorato a migrazioni e corsi di formazione su LibreOffice per diverse amministrazioni pubbliche e privati. Da gennaio 2020 lavora in SUSE come Software Release Engineer per Uyuni e SUSE Manager e quando non segue la sua passione per i computer e per Geeko coltiva la sua curiosità per l'astronomia (da cui deriva il suo nickname deneb_alpha).
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
Imagine a world where machines not only perform tasks but also learn, adapt, and make decisions. This is the promise of Artificial Intelligence (AI), a technology that's not just enhancing our lives but revolutionizing entire industries.
Recently uploaded (20)
Nordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptx
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
Introduction of Cybersecurity with OSS at Code Europe 2024
Introduction of Cybersecurity with OSS at Code Europe 2024
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Energy Efficient Video Encoding for Cloud and Edge Computing Instances
Energy Efficient Video Encoding for Cloud and Edge Computing Instances
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracy
Building Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and Milvus
Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
How to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptx
UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
Linux System Monitoring with eBPF
- 1. Heinrich.Hartmann@Circonus.com Linux System Monitoring with eBPF DevOpsDays Kiel, 2018-05-16 Heinrich Hartmann
- 2. Heinrich.Hartmann@Circonus.com System Monitoring is about Kernel & Hardware
- 3. Heinrich.Hartmann@Circonus.com Best Practice: The USE Method https://www.circonus.com/2017/08/system-monitoring-with-the-use-dashboard CPU Memory Network Disks Utilization Saturation Errors
- 4. Heinrich.Hartmann@Circonus.com Best Practice: The USE Method https://www.circonus.com/2017/08/system-monitoring-with-the-use-dashboard CPU Memory Network Disks Utilization Saturation Errors
- 5. Heinrich.Hartmann@Circonus.com Lot’s of Unknowns remaining https://www.circonus.com/2017/08/system-monitoring-with-the-use-dashboard ? ? ? ~ ~ ~ CPU Memory Network Disks Utilization Saturation Errors
- 6. Heinrich.Hartmann@Circonus.com eBPF allows unparalleled insights https://github.com/iovisor/bcc Credits: - Brendan Gregg @ Netflix (Sun) - Sasha Goldshtein @ Sela, Microsoft - Brenden Blanco @ VMWare - Linus Torvalds, et. al.
- 7. Heinrich.Hartmann@Circonus.com eBPF allows unparalleled insights https://github.com/iovisor/bcc Credits: - Brendan Gregg @ Netflix (Sun) - Sasha Goldshtein @ Sela, Microsoft - Brenden Blanco @ VMWare - Linus Torvalds, et. al.
- 11. Heinrich.Hartmann@Circonus.com Disk: Block-I/O Latency over time
- 12. Heinrich.Hartmann@Circonus.com Disk: Block-I/O Latency over time
- 13. Heinrich.Hartmann@Circonus.com Don’t shout in the Datacenter Brendan Gregg (2008) https://www.youtube.com/watch?v=tDacjrSCeq4
- 14. Heinrich.Hartmann@Circonus.com System Calls: The Kernel API Monitor Rate Errors Duration System Call API
- 15. Heinrich.Hartmann@Circonus.com Syscalls: Rate / Count sched_yield (2tn) clock_time (1.5tn) recvfrom (300bn) 394 Metrics
- 17. Heinrich.Hartmann@Circonus.com Syscall durations span >8 orders of magnitude 1s 100 ms 10 us 1.5 tn events total
- 20. Heinrich.Hartmann@Circonus.com Further Reading Slides: @HeinrichHartman / #dodkiel18 Code: https://github.com/circonus-labs/nad/.../bccbpf Blog: http://www.circonus.com/2018/05/linux-system-monitoring-with-ebpf/