In I.T. we all use all kinds of metrics. Operations teams rely heavily on these, especially when things go south. These metrics are sometimes overrated. Let’s dive into a few real life stories together.
Aurélien Rougemont
Kernel Recipes 2019 - Kernel documentation: past, present, and futureAnne Nicolas
The Linux kernel project includes a huge amount of documentation, but that information has seen little in the way of care over the
years. The amount of care has increased significantly recently, though, and things are improving quickly. Listen as the kernel’s documentation maintainer discusses the current state of the kernel’s docs, how we got here, where we’re trying to go, and how you can help.
Jonathan Corbet
Kernel Recipes 2019 - Analyzing changes to the binary interface exposed by th...Anne Nicolas
Operating system distributors often face challenges that are somewhat different from that of upstream kernel developers. For instance, some kernel updates often need to stay at least binary compatible with modules that might be “out of tree” for some time.
In that context, being able to automatically detect and analyze changes to the binary interface exposed by the kernel to its module does have some noticeable value.
The Libabigail framework is capable of analyzing ELF binaries along with their accompanying debug info in the DWARF format, detect and report changes in types, functions, variables and ELF symbols. It has historically supported that for user space shared libraries and application so we worked to make it understand the Linux kernel
binaries.
In this presentation, we are going to present the current support of ABI analysis for Linux Kernel binaries, the challenges we face, how we address them and the plans we have for the future.
Dodji Seketeli, Jessica Yu, Matthias Männich
Kernel Recipes 2019 - CVEs are dead, long live the CVE!Anne Nicolas
For the Linux kernel, CVEs do not work at all given the rate of fixes being applied and rapidly backported and pushed to users through a huge variety of different ways. The average “request to fix” date for Linux CVEs is -100 days, showing that either no one cares about CVEs for Linux, or engineers are using them to game their internal release processes, or no one happens to notice when the kernel developers resolve an issue, or all of the above. This talk will go into the problems with CVEs when it comes to a fast moving project like Linux, and show the decentralized solution that we have been using for the past 14 years instead. All other open source projects are encouraged to use these same methods to help resolve the problems that CVEs have.
Kernel Recipes 2019 - XDP closer integration with network stackAnne Nicolas
XDP (eXpress Data Path) is the new programmable in-kernel fast-path, which is placed as a layer before the existing Linux kernel network stack (netstack).
We claim XDP is not kernel-bypass, as it is a layer before and it can easily fall-through to netstack. Reality is that it can easily be (ab)used to create a kernel-bypass situation, where non of the kernel facilities are used (in form of BPF-helpers and in-kernel tables). The main disadvantage with kernel-bypass, is the need to re-implement everything, even basic building blocks, like routing tables and ARP protocol handling.
It is part of the concept and speed gain, that XDP allows users to avoid calling part of the kernel code. Users have the freedom to do kernel-bypass and re-implement everything, but the kernel should provide access to more in-kernel tables, via BPF-helpers, such that users can leverage other parts of the Open Source ecosystem, like router daemons etc.
This talk is about how XDP can work in-concert with netstack, and proposal on how we can take this even-further. Crazy ideas like using XDP frames to move SKB allocation out of driver code, will also be proposed.
Kernel Recipes 2019 - Marvels of Memory Auto-configuration (SPD)Anne Nicolas
System memory configuration is a transparent operation nowadays, something that we all came to expect to just work out of the box. Still, it does happen behind the scenes every single time we boot our computers. This requires the cooperation of hardware components on the mainboard and on memory modules themselves, as well as firmware code to drive these. While it is possible to just let it happen, having a deeper understanding of how it works makes it possible to access valuable information from the operating system at run-time.
I will take you through the history of system memory configuration from the mid 70s to now. We will explore the different types of memory modules, how their configuration data is stored and how the firmware can access them. We will see which problems had to be solved along the way and how they were solved. Lastly we will see how Linux supports reading the memory configuration information and what you can do with that information.
Jean Delvare
Embedded Recipes 2019 - Testing firmware the devops wayAnne Nicolas
ITRenew is selling recertified OCP servers under the Sesame brand, those servers come either with their original UEFI BIOS or with LinuxBoot. The LinuxBoot project is pushing the Linux kernel inside bios flash and using userland programs as bootloader.
To achieve quality on our software stack, as any project, we need to test it. Traditional BIOS are tested by hand, this is 2019 we need to do it automatically! We already presented the hardware setup behind the LinuxBoot CI, this talk will focus on the software.
We use u-root for our userland bootloader; this software is written in Go so we naturally choose to use Go for our testing too. We will present how we are using and extending the Go native test framework `go test` for testing embedded systems (serial console) and improving the report format for integration to a CI.
Julien Viard de Galbert
Kernel Recipes 2019 - Formal modeling made easyAnne Nicolas
Modeling parts of Linux has become a recurring topic. For instance, the memory model, the model for PREEMPT_RT synchronization, and so on. But the term "formal model" causes panic for most of the developers. Mainly because of the complex notations and reasoning that involves formal languages. It seems to be a very theoretical thing, far from our day-by-day reality.
Believe me. Modeling can be more practical than you might guess!
This talk will discuss the challenges and benefits of modeling, based on the experience of developing the PREEMPT_RT model. It will present a methodology for modeling the Linux behavior as Finite-State Machines (automata), using terms that are very known by kernel developers: tracing events! With the particular focus on how to use models for the formal verification of Linux kernel, at runtime, with low overhead, and in many cases, without even modifying Linux kernel!
Daniel Bristot de Oliveira
Kernel Recipes 2019 - Kernel documentation: past, present, and futureAnne Nicolas
The Linux kernel project includes a huge amount of documentation, but that information has seen little in the way of care over the
years. The amount of care has increased significantly recently, though, and things are improving quickly. Listen as the kernel’s documentation maintainer discusses the current state of the kernel’s docs, how we got here, where we’re trying to go, and how you can help.
Jonathan Corbet
Kernel Recipes 2019 - Analyzing changes to the binary interface exposed by th...Anne Nicolas
Operating system distributors often face challenges that are somewhat different from that of upstream kernel developers. For instance, some kernel updates often need to stay at least binary compatible with modules that might be “out of tree” for some time.
In that context, being able to automatically detect and analyze changes to the binary interface exposed by the kernel to its module does have some noticeable value.
The Libabigail framework is capable of analyzing ELF binaries along with their accompanying debug info in the DWARF format, detect and report changes in types, functions, variables and ELF symbols. It has historically supported that for user space shared libraries and application so we worked to make it understand the Linux kernel
binaries.
In this presentation, we are going to present the current support of ABI analysis for Linux Kernel binaries, the challenges we face, how we address them and the plans we have for the future.
Dodji Seketeli, Jessica Yu, Matthias Männich
Kernel Recipes 2019 - CVEs are dead, long live the CVE!Anne Nicolas
For the Linux kernel, CVEs do not work at all given the rate of fixes being applied and rapidly backported and pushed to users through a huge variety of different ways. The average “request to fix” date for Linux CVEs is -100 days, showing that either no one cares about CVEs for Linux, or engineers are using them to game their internal release processes, or no one happens to notice when the kernel developers resolve an issue, or all of the above. This talk will go into the problems with CVEs when it comes to a fast moving project like Linux, and show the decentralized solution that we have been using for the past 14 years instead. All other open source projects are encouraged to use these same methods to help resolve the problems that CVEs have.
Kernel Recipes 2019 - XDP closer integration with network stackAnne Nicolas
XDP (eXpress Data Path) is the new programmable in-kernel fast-path, which is placed as a layer before the existing Linux kernel network stack (netstack).
We claim XDP is not kernel-bypass, as it is a layer before and it can easily fall-through to netstack. Reality is that it can easily be (ab)used to create a kernel-bypass situation, where non of the kernel facilities are used (in form of BPF-helpers and in-kernel tables). The main disadvantage with kernel-bypass, is the need to re-implement everything, even basic building blocks, like routing tables and ARP protocol handling.
It is part of the concept and speed gain, that XDP allows users to avoid calling part of the kernel code. Users have the freedom to do kernel-bypass and re-implement everything, but the kernel should provide access to more in-kernel tables, via BPF-helpers, such that users can leverage other parts of the Open Source ecosystem, like router daemons etc.
This talk is about how XDP can work in-concert with netstack, and proposal on how we can take this even-further. Crazy ideas like using XDP frames to move SKB allocation out of driver code, will also be proposed.
Kernel Recipes 2019 - Marvels of Memory Auto-configuration (SPD)Anne Nicolas
System memory configuration is a transparent operation nowadays, something that we all came to expect to just work out of the box. Still, it does happen behind the scenes every single time we boot our computers. This requires the cooperation of hardware components on the mainboard and on memory modules themselves, as well as firmware code to drive these. While it is possible to just let it happen, having a deeper understanding of how it works makes it possible to access valuable information from the operating system at run-time.
I will take you through the history of system memory configuration from the mid 70s to now. We will explore the different types of memory modules, how their configuration data is stored and how the firmware can access them. We will see which problems had to be solved along the way and how they were solved. Lastly we will see how Linux supports reading the memory configuration information and what you can do with that information.
Jean Delvare
Embedded Recipes 2019 - Testing firmware the devops wayAnne Nicolas
ITRenew is selling recertified OCP servers under the Sesame brand, those servers come either with their original UEFI BIOS or with LinuxBoot. The LinuxBoot project is pushing the Linux kernel inside bios flash and using userland programs as bootloader.
To achieve quality on our software stack, as any project, we need to test it. Traditional BIOS are tested by hand, this is 2019 we need to do it automatically! We already presented the hardware setup behind the LinuxBoot CI, this talk will focus on the software.
We use u-root for our userland bootloader; this software is written in Go so we naturally choose to use Go for our testing too. We will present how we are using and extending the Go native test framework `go test` for testing embedded systems (serial console) and improving the report format for integration to a CI.
Julien Viard de Galbert
Kernel Recipes 2019 - Formal modeling made easyAnne Nicolas
Modeling parts of Linux has become a recurring topic. For instance, the memory model, the model for PREEMPT_RT synchronization, and so on. But the term "formal model" causes panic for most of the developers. Mainly because of the complex notations and reasoning that involves formal languages. It seems to be a very theoretical thing, far from our day-by-day reality.
Believe me. Modeling can be more practical than you might guess!
This talk will discuss the challenges and benefits of modeling, based on the experience of developing the PREEMPT_RT model. It will present a methodology for modeling the Linux behavior as Finite-State Machines (automata), using terms that are very known by kernel developers: tracing events! With the particular focus on how to use models for the formal verification of Linux kernel, at runtime, with low overhead, and in many cases, without even modifying Linux kernel!
Daniel Bristot de Oliveira
Kernel Recipes 2019 - ftrace: Where modifying a running kernel all startedAnne Nicolas
Ftrace’s most powerful feature is the function tracer (and function graph tracer which is built from it). But to have this enabled on production systems, it had to have its overhead be negligible when disabled. As the function tracer uses gcc’s profiling mechanism, which adds a call to “mcount” (or more recently fentry, don’t worry if you don’t know what this is, it will all be explained) at the start of almost all functions, it had to do something about the overhead that causes. The solution was to turn those calls into “nops” (an instruction that the CPU simply ignores). But this was no easy feat. It took a lot to come up with a solution (and also turning a few network cards into bricks). This talk will explain the history of how ftrace came about implementing the function tracer, and brought with it the possibility of static branches and soon static calls!
Steven Rostedt
Troubleshooting Tips from a Docker Support EngineerJeff Anderson
Troubleshooting is like going on an adventure. Here are some tips for how to tackle unexpected situations when using Docker.
These cases were pulled from the most common issues encountered while helping folks in the Docker community solve issues.
Kernel Recipes 2014 - NDIV: a low overhead network traffic diverterAnne Nicolas
NDIV is a young, very simple, yet efficient network traffic diverter. Its purpose is to help build network applications that intercept packets at line rate with a very low processing overhead. A first example application is a stateless HTTP server reaching line rate on all packet sizes.
Willy Tarreau, HaproxyTech
In this talk Jiří Pírko discusses the design and evolution of the VLAN implementation in Linux, the challenges and pitfalls as well as hardware acceleration and alternative implementations.
Jiří Pírko is a major contributor to kernel networking and the creator of libteam for link aggregation.
DPDK (Data Plane Development Kit) Overview by Rami Rosen
* Background and short history
* Advantages and disadvantages
- Very High speed networking acceleration in L2
- How this acceleration is achieved (hugepages, optimizations)
- rte_kni (and KCP)
- VPP (and FD.io project) , providing routing and switching.
- TLDK (Transport Layer Development Kit, TCP/UDP)
* Anatomy of a simple DPDK application.
* Development and governance model
* Testpmd: DPDK CLI tool
* DDP - Dynamic Device Profiles
Rami Rosen is a Linux Kernel expert, the author of "Linux Kernel Networking", Apress, 2014.
Rami had published two articles about DPDK in the last year:
"Network acceleration with DPDK"
https://lwn.net/Articles/725254/
"Userspace Networking with DPDK"
https://www.linuxjournal.com/content/userspace-networking-dpdk
Kernel Recipes 2019 - Hunting and fixing bugs all over the Linux kernelAnne Nicolas
At a rate of almost 9 changes per hour (24/7), the Linux kernel is definitely a scary beast. Bugs are introduced on a daily basis and, through the use of multiple code analyzers, *some* of them are detected and fixed before they hit mainline. Over the course of the last few years, Gustavo has been fixing such bugs and many different issues in every corner of the Linux kernel. Recently, he was in charge of leading the efforts to globally enable -Wimplicit-fallthrough; which appears by default in Linux v5.3. This presentation is a report on all the stuff Gustavo has found and fixed in the kernel with the support of the Core Infrastructure Initiative.
Gustavo A.R. Silva
netfilter is a framework provided by the Linux kernel that allows various networking-related operations to be implemented in the form of customized handlers.
iptables is a user-space application program that allows a system administrator to configure the tables provided by the Linux kernel firewall (implemented as different netfilter modules) and the chains and rules it stores.
Many systems use iptables/netfilter, Linux's native packet filtering/mangling framework since Linux 2.4, be it home routers or sophisticated cloud network stacks.
In this session, we will talk about the netfilter framework and its facilities, explain how basic filtering and mangling use-cases are implemented using iptables, and introduce some less common but powerful extensions of iptables.
Shmulik Ladkani, Chief Architect at Nsof Networks.
Long time network veteran and kernel geek.
Shmulik started his career at Jungo (acquired by NDS/Cisco) implementing residential gateway software, focusing on embedded Linux, Linux kernel, networking and hardware/software integration.
Some billions of forwarded packets later, Shmulik left his position as Jungo's lead architect and joined Ravello Systems (acquired by Oracle) as tech lead, developing a virtual data center as a cloud-based service, focusing around virtualization systems, network virtualization and SDN.
Recently he co-founded Nsof Networks, where he's been busy architecting network infrastructure as a cloud-based service, gazing at internet routes in astonishment, and playing the chkuku.
Kernel Recipes 2019 - Suricata and XDPAnne Nicolas
Suricata is a network threat detection engine using network packets capture to reconstruct the traffic till the application layer and find threats on the network using rules that define behavior to detect. This task is really CPU intensive and discarding non interesting traffic is a solution to enable a scaling of Suricata to 40gbps and other.
This talk will present the latest evolution of Suricata that knows uses eBPF and XDP to bypass traffic. Suricata 5.0 is supporting the hardware XDP to provide ypass with network card such as Netronome. It also takes advantage of pinned maps to get persistance of the bypassed flows. This talk will cover the different usage of XDP and eBPF in Suricata and shows how it impact performance and usability. If development time permit, the talk will also cover AF_XDP and the impact on this new capture method on Suricata.
Eric Leblond
Kernel Recipes 2016 - Landlock LSM: Unprivileged sandboxingAnne Nicolas
Linux has multiple access-control features, which help to contain the damage from a malicious process. However, it is difficult and complex, especially for unprivileged users, to create a sandboxed application because of the currently administrator-oriented security.
seccomp-bpf was a big step forward in empowering any user with the ability to filter syscalls and therefore limit access to some resources. Nevertheless, it lacks the ability to create a full standalone sandbox (e.g. restrict access to a set of files), unlike Seatbelt/XNU Sandbox or OpenBSD Pledge.
In this talk, we present Landlock, a new Linux Security Module for unprivileged users. This brings some interesting challenges, from architecture design to userland API definition.
Mickaël Salaün
This presentation introduces Data Plane Development Kit overview and basics. It is a part of a Network Programming Series.
First, the presentation focuses on the network performance challenges on the modern systems by comparing modern CPUs with modern 10 Gbps ethernet links. Then it touches memory hierarchy and kernel bottlenecks.
The following part explains the main DPDK techniques, like polling, bursts, hugepages and multicore processing.
DPDK overview explains how is the DPDK application is being initialized and run, touches lockless queues (rte_ring), memory pools (rte_mempool), memory buffers (rte_mbuf), hashes (rte_hash), cuckoo hashing, longest prefix match library (rte_lpm), poll mode drivers (PMDs) and kernel NIC interface (KNI).
At the end, there are few DPDK performance tips.
Tags: access time, burst, cache, dpdk, driver, ethernet, hub, hugepage, ip, kernel, lcore, linux, memory, pmd, polling, rss, softswitch, switch, userspace, xeon
Kernel Recipes 2019 - ftrace: Where modifying a running kernel all startedAnne Nicolas
Ftrace’s most powerful feature is the function tracer (and function graph tracer which is built from it). But to have this enabled on production systems, it had to have its overhead be negligible when disabled. As the function tracer uses gcc’s profiling mechanism, which adds a call to “mcount” (or more recently fentry, don’t worry if you don’t know what this is, it will all be explained) at the start of almost all functions, it had to do something about the overhead that causes. The solution was to turn those calls into “nops” (an instruction that the CPU simply ignores). But this was no easy feat. It took a lot to come up with a solution (and also turning a few network cards into bricks). This talk will explain the history of how ftrace came about implementing the function tracer, and brought with it the possibility of static branches and soon static calls!
Steven Rostedt
Troubleshooting Tips from a Docker Support EngineerJeff Anderson
Troubleshooting is like going on an adventure. Here are some tips for how to tackle unexpected situations when using Docker.
These cases were pulled from the most common issues encountered while helping folks in the Docker community solve issues.
Kernel Recipes 2014 - NDIV: a low overhead network traffic diverterAnne Nicolas
NDIV is a young, very simple, yet efficient network traffic diverter. Its purpose is to help build network applications that intercept packets at line rate with a very low processing overhead. A first example application is a stateless HTTP server reaching line rate on all packet sizes.
Willy Tarreau, HaproxyTech
In this talk Jiří Pírko discusses the design and evolution of the VLAN implementation in Linux, the challenges and pitfalls as well as hardware acceleration and alternative implementations.
Jiří Pírko is a major contributor to kernel networking and the creator of libteam for link aggregation.
DPDK (Data Plane Development Kit) Overview by Rami Rosen
* Background and short history
* Advantages and disadvantages
- Very High speed networking acceleration in L2
- How this acceleration is achieved (hugepages, optimizations)
- rte_kni (and KCP)
- VPP (and FD.io project) , providing routing and switching.
- TLDK (Transport Layer Development Kit, TCP/UDP)
* Anatomy of a simple DPDK application.
* Development and governance model
* Testpmd: DPDK CLI tool
* DDP - Dynamic Device Profiles
Rami Rosen is a Linux Kernel expert, the author of "Linux Kernel Networking", Apress, 2014.
Rami had published two articles about DPDK in the last year:
"Network acceleration with DPDK"
https://lwn.net/Articles/725254/
"Userspace Networking with DPDK"
https://www.linuxjournal.com/content/userspace-networking-dpdk
Kernel Recipes 2019 - Hunting and fixing bugs all over the Linux kernelAnne Nicolas
At a rate of almost 9 changes per hour (24/7), the Linux kernel is definitely a scary beast. Bugs are introduced on a daily basis and, through the use of multiple code analyzers, *some* of them are detected and fixed before they hit mainline. Over the course of the last few years, Gustavo has been fixing such bugs and many different issues in every corner of the Linux kernel. Recently, he was in charge of leading the efforts to globally enable -Wimplicit-fallthrough; which appears by default in Linux v5.3. This presentation is a report on all the stuff Gustavo has found and fixed in the kernel with the support of the Core Infrastructure Initiative.
Gustavo A.R. Silva
netfilter is a framework provided by the Linux kernel that allows various networking-related operations to be implemented in the form of customized handlers.
iptables is a user-space application program that allows a system administrator to configure the tables provided by the Linux kernel firewall (implemented as different netfilter modules) and the chains and rules it stores.
Many systems use iptables/netfilter, Linux's native packet filtering/mangling framework since Linux 2.4, be it home routers or sophisticated cloud network stacks.
In this session, we will talk about the netfilter framework and its facilities, explain how basic filtering and mangling use-cases are implemented using iptables, and introduce some less common but powerful extensions of iptables.
Shmulik Ladkani, Chief Architect at Nsof Networks.
Long time network veteran and kernel geek.
Shmulik started his career at Jungo (acquired by NDS/Cisco) implementing residential gateway software, focusing on embedded Linux, Linux kernel, networking and hardware/software integration.
Some billions of forwarded packets later, Shmulik left his position as Jungo's lead architect and joined Ravello Systems (acquired by Oracle) as tech lead, developing a virtual data center as a cloud-based service, focusing around virtualization systems, network virtualization and SDN.
Recently he co-founded Nsof Networks, where he's been busy architecting network infrastructure as a cloud-based service, gazing at internet routes in astonishment, and playing the chkuku.
Kernel Recipes 2019 - Suricata and XDPAnne Nicolas
Suricata is a network threat detection engine using network packets capture to reconstruct the traffic till the application layer and find threats on the network using rules that define behavior to detect. This task is really CPU intensive and discarding non interesting traffic is a solution to enable a scaling of Suricata to 40gbps and other.
This talk will present the latest evolution of Suricata that knows uses eBPF and XDP to bypass traffic. Suricata 5.0 is supporting the hardware XDP to provide ypass with network card such as Netronome. It also takes advantage of pinned maps to get persistance of the bypassed flows. This talk will cover the different usage of XDP and eBPF in Suricata and shows how it impact performance and usability. If development time permit, the talk will also cover AF_XDP and the impact on this new capture method on Suricata.
Eric Leblond
Kernel Recipes 2016 - Landlock LSM: Unprivileged sandboxingAnne Nicolas
Linux has multiple access-control features, which help to contain the damage from a malicious process. However, it is difficult and complex, especially for unprivileged users, to create a sandboxed application because of the currently administrator-oriented security.
seccomp-bpf was a big step forward in empowering any user with the ability to filter syscalls and therefore limit access to some resources. Nevertheless, it lacks the ability to create a full standalone sandbox (e.g. restrict access to a set of files), unlike Seatbelt/XNU Sandbox or OpenBSD Pledge.
In this talk, we present Landlock, a new Linux Security Module for unprivileged users. This brings some interesting challenges, from architecture design to userland API definition.
Mickaël Salaün
This presentation introduces Data Plane Development Kit overview and basics. It is a part of a Network Programming Series.
First, the presentation focuses on the network performance challenges on the modern systems by comparing modern CPUs with modern 10 Gbps ethernet links. Then it touches memory hierarchy and kernel bottlenecks.
The following part explains the main DPDK techniques, like polling, bursts, hugepages and multicore processing.
DPDK overview explains how is the DPDK application is being initialized and run, touches lockless queues (rte_ring), memory pools (rte_mempool), memory buffers (rte_mbuf), hashes (rte_hash), cuckoo hashing, longest prefix match library (rte_lpm), poll mode drivers (PMDs) and kernel NIC interface (KNI).
At the end, there are few DPDK performance tips.
Tags: access time, burst, cache, dpdk, driver, ethernet, hub, hugepage, ip, kernel, lcore, linux, memory, pmd, polling, rss, softswitch, switch, userspace, xeon
All of Your Network Monitoring is (probably) Wrongice799
Monitorama 2016 talk about network monitoring covering topics like network device drivers, ethtool, and some interesting bugs/features.
For more information about monitoring and tuning the entire Linux network stack, see: blog.packagecloud.io/eng/2016/06/22/monitoring-tuning-linux-networking-stack-receiving-data/
Analyzing OS X Systems Performance with the USE MethodBrendan Gregg
Talk for MacIT 2014. This talk is about systems performance on OS X, and introduces the USE Method to check for common performance bottlenecks and errors. This methodology can be used by beginners and experts alike, and begins by constructing a checklist of the questions we’d like to ask of the system, before reaching for tools to answer them. The focus is resources: CPUs, GPUs, memory capacity, network interfaces, storage devices, controllers, interconnects, as well as some software resources such as mutex locks. These areas are investigated by a wide variety of tools, including vm_stat, iostat, netstat, top, latency, the DTrace scripts in /usr/bin (which were written by Brendan), custom DTrace scripts, Instruments, and more. This is a tour of the tools needed to solve our performance needs, rather than understanding tools just because they exist. This talk will make you aware of many areas of OS X that you can investigate, which will be especially useful for the time when you need to get to the bottom of a performance issue.
I was asked to talk in front of Computer science students at the Bar-Ilan university about "what happens" when you don't care about writing "secured" or "safe" code. A perfect example for that, in my opinion, was the world of embedded computing AKA the IoT. I talked about the history of consumer embedded devices and showed a live demo of an 0day I found in one of the most popular routers in the country.
I am Anne L. I am an Operating System Assignment Expert at programminghomeworkhelp.com. I hold a Ph.D. in Programming, Auburn University, USA. I have been helping students with their homework for the past 8 years. I solve assignments related to Operating systems.
Visit programminghomeworkhelp.com or email support@programminghomeworkhelp.com.
You can also call on +1 678 648 4277 for any assistance with Operating System Assignments.
Time Sensitive Networking in the Linux Kernelhenrikau
Time Sensitive Networking provides mechanisms for sending data accross the network with very low latency, low jitter and low framedrops, opening up a whole range of new applications.
This talk primarily focuses on media, but the driver should be interesting for industrial applications and automotive as well.
HKG15-902: Upstreaming 201
---------------------------------------------------
Speaker: Matt Porter
Date: February 10, 2015
---------------------------------------------------
★ Session Summary ★
This session is an advanced course on Linux kernel upstreaming fundamentals. The course covers how the arm-soc kernel tree is maintained and why that is important to ARM Linux kernel developers. The focus of the course is the explanation of the detailed mechanics of creating and posting patch series to upstream mailing lists for several common cases. Annotated session content is made up of previously upstreamed ARM support captured from emails to the kernel mailing lists. The target audience is both software engineers and engineering managers preparing to upstream software into the kernel. The topic requires a solid background in software configuration management terminology and the git SCM tool as well as a good technical understanding of the Linux kernel itself.
--------------------------------------------------
★ Resources ★
Pathable: https://hkg15.pathable.com/meetings/250797
Video: https://www.youtube.com/watch?v=yhQdSP2436I
Etherpad: N/A
---------------------------------------------------
★ Event Details ★
Linaro Connect Hong Kong 2015 - #HKG15
February 9-13th, 2015
Regal Airport Hotel Hong Kong Airport
---------------------------------------------------
http://www.linaro.org
http://connect.linaro.org
Kernel Recipes 2019 - Driving the industry toward upstream firstAnne Nicolas
Wanting to avoid the Android experience, Google developers always aimed to make their Chrome OS Linux kernels as close to mainline as possible. However, when Chromebooks were first created, Google was left with no choice, the mainline kernel, in some subsystems, still did not have all the functionalities needed by Chromebooks. Hence, similarly to Android, Chrome OS had to develop their own out-of-tree code for the kernel and maintain that for a few different kernel versions.
Luckily, over the last few years a strong and consistent effort has been happening to bring Chromebook devices closer to mainline. It has led to significant improvements that now make it possible to run mainline on Chrome OS devices. And not only Chromebooks, as these significant strides are also improving Arm-based SOCs and other key components of the rich Chromebook hardware ecosystem. In this talk, we will look at how and why upstream support for Chromebooks improved, the current status of various models, and what we expect in the future.
Enric Balletbò i Serra
Kernel Recipes 2019 - No NMI? No Problem! – Implementing Arm64 Pseudo-NMIAnne Nicolas
As the name would suggest, a Non-Maskable Interrupt (NMI) is an interrupt-like feature that is unaffected by the disabling of classic interrupts. In Linux, NMIs are involved in some features such as performance event monitoring, hard-lockup detector, on demand state dumping, etc… Their potential to fire when least expected can fill the most seasoned kernel hackers with dread.
AArch64 (aka arm64 in the Linux tree) does not provide architected NMIs, a consequence being that features benefiting from NMIs see their use limited on AArch64. However, the Arm Generic Interrupt Controller (GIC) supports interrupt prioritization and masking, which, among other things, provides a way to control whether or not a set of interrupts can be signaled to a CPU.
This talk will cover how, using the GIC interrupt priorities, we provide a way to configure some interrupts to behave in an NMI-like manner on AArch64. We’ll discuss the implementation, some of the complications that ensued and also some of the benefits obtained from it.
Julien Thierry
Embedded Recipes 2019 - Knowing your ARM from your ARSE: wading through the t...Anne Nicolas
Modern SoC designs incorporate technologies from numerous vendors, each with their own inconsistent, confusing, undocumented and even contradictory terminology. The result is a mess of acronyms and product names which have a surprising impact on the ability to develop reusable, modular code thanks to the nature of the underlying IP being obscured.
This presentation will dive into some of the misnomers plaguing the Arm ecosystem, with the aim of explaining why things are like they are, how they fit together under the architectural umbrella and how you, as a developer, can decipher the baffling ingredients list of your next SoC design!
Will Deacon
Kernel Recipes 2019 - GNU poke, an extensible editor for structured binary dataAnne Nicolas
GNU poke is a new interactive editor for binary data. Not limited to editing basic ntities such as bits and bytes, it provides a full-fledged procedural, interactive programming language designed to describe data structures and to operate on them. Once a user has defined a structure for binary data (usually matching some file format) she can search, inspect, create, shuffle and modify abstract entities such as ELF relocations, MP3 tags, DWARF expressions, partition table entries, and so on, with primitives resembling simple editing of bits and bytes. The program comes with a library of already written descriptions (or “pickles” in poke parlance) for many binary formats.
GNU poke is useful in many domains. It is very well suited to aid in the development of programs that operate on binary files, such as assemblers and linkers. This was in fact the primary inspiration that brought me to write it: easily injecting flaws into ELF files in order to reproduce toolchain bugs. Also, due to its flexibility, poke is also very useful for reverse engineering, where the real structure of the data being edited is discovered by experiment, interactively. It is also good for the fast development of prototypes for programs like linkers, compressors or filters, and it provides a convenient foundation to write other utilities such as diff and patch tools for binary files.
This talk (unlike Gaul) is divided into four parts. First I will introduce the program and show what it does: from simple bits/bytes editing to user-defined structures. Then I will show some of the internals, and how poke is implemented. The third block will cover the way of using Poke to describe user data, which is to say the art of writing “pickles”. The presentation ends with a status of the project, a call for hackers, and a hint at future works.
Jose E. Marchesi
Embedded Recipes 2019 - Remote update adventures with RAUC, Yocto and BareboxAnne Nicolas
Different upgrade and update strategies exist when it comes to embedded Linux system. If at development time none of these strategies have been chosen, adding them afterwards can be tedious task.
Even harder it gets when the system is already deployed in the field and only accessible via a 3G connection.
This talk is a developer experience of putting in place exactly that. Giving a return of experience on one way of doing it on a system running Barebox and a Yocto-based distribution.
Patrick Boettcher
Embedded Recipes 2019 - Making embedded graphics less specialAnne Nicolas
Traditionally graphics drivers were one of the last hold-outs of proprietary software in an embedded Linux system. This situation is changing with open-source graphics drivers showing up for almost all of the graphics acceleration peripherals on the market right now. This talk will show how open-source graphics drivers are making embedded systems less special, as well as trying to provide an overview of the Linux graphics stack, de-mystifying what is often seen as black magic GPU stuff from outside observers.
Lucas Stach
Embedded Recipes 2019 - Linux on Open Source Hardware and Libre SiliconAnne Nicolas
This talk will explore Open Source Hardware projects relevant to Linux, including boards like BeagleBone, Olimex OLinuXino, Giant board and more. Looking at the benefits and challenges of designing Open Source Hardware for a Linux system, along with BeagleBoard.org’s experience of working with community, manufacturers, and distributors to create an Open Source Hardware platform. In closing also looking at the future, Libre Silicon like RISC-V designs, and where this might take Linux.
Drew Fustini
Embedded Recipes 2019 - From maintaining I2C to the big (embedded) pictureAnne Nicolas
The I2C subsystem is not the shiniest part of the Linux Kernel. For embedded devices, though, it is one of the many puzzle pieces which just have to work. Wolfram Sang has the experience of maintaining this subsystem for nearly 7 years now. This talk gives a short overview of how maintaining works in general and specifically in this subsystem. But mainly, it will highlight noteworthy points in the timeline and lessons learnt from that. It will present trends, not so much regarding I2C but more the Linux Kernel and the embedded ecosystem in general. And of course, there will be plenty of anecdotes and bits from behind the scenes for your entertainment.
Wolfram Sang
Embedded Recipes 2019 - Herd your socs become a matchmakerAnne Nicolas
About 60% of the Linux kernel source tree is devoted to drivers for a large variety of supported hardware components. Especially in the embedded world, the number of different SoC families, versions, and revisions, integrating a myriad of “IP cores”, keeps on growing.
In this presentation, Geert will explain how to match drivers against hardware, and how to support a wide variety of (dis)similar devices, without turning platform and driver code into an entangled bowl of spaghetti.tra
Starting with a brief history of driver matching in Linux, he will fast-forward to device-tree based matching. He will discuss ways to handle slight variations of the same hardware devices, and different SoC revisions, each with their own quirks and bugs. Finally, Geert will show best practices for evolving device drivers in a maintainable way, based on his experiences as an embedded Linux kernel developer and maintainer.
Geert Uytterhoeven
Embedded Recipes 2019 - LLVM / Clang integrationAnne Nicolas
Buildroot is a popular and easy to use embedded Linux build system. It generates, in few minutes, lightweight and customized Linux systems, including the cross-compilation toolchain, kernel and bootloader images, as well as a wide variety of userspace libraries and programs.
This talk is about the integration of LLVM/clang into Buildroot.
In 2018, Valentin Korenblit, supervised by Romain Naour, worked on this topic during his internship at Smile ECS. After a short introduction about llvm/clang and Buildroot, this talk will go through the numerous issues discovered while adding llvm/clang componants and how these issues were fixed. Romain will also detail the work in progress and the work to be done based on llvm/clang libraries (OpenCL, Compiler-rt, BCC. Chromium, ldd).
Romain Naour
Embedded Recipes 2019 - Introduction to JTAG debuggingAnne Nicolas
This talk introduces JTAG debugging capabilities, both for debugging hardware and software. Marek first explains what the JTAG stands for and explains the operation of the JTAG state machine. This is followed by an introduction to free software JTAG tools, OpenOCD and urJTAG. Marek shortly explains how to debug software using those tools and how that ties into the JTAG state machine. However, JTAG was designed for testing hardware. Marek explains what boundary scan testing (BST) is, what are BSDL files and their format, and practically demonstrates how to blink an LED using BST and only free software tools.
Marek Vasut
Embedded Recipes 2019 - Pipewire a new foundation for embedded multimediaAnne Nicolas
PipeWire is an open source project that aims to greatly improve audio and video handling under Linux. Utilising a fresh design, it bridges use cases that have been previously addressed by different tools – or not addressed at all -, providing ground for building complex, yet secure and efficient, multimedia systems.
In this talk, Julien is going to present the PipeWire project and the concepts that make up its design. In addition, he is going to give an update of the current and future work going on around PipeWire, both upstream and in Automotive Grade Linux, an early adopter that Julien is actively working on.
Julian Bouzas
Kernel Recipes 2019 - Kernel hacking behind closed doorsAnne Nicolas
The recent hardware security vulnerabilites exposed the kernel community to unprecedented restrictions and bureaucrazy. Pure software bugs which only affect the Linux kernel are a completely different category and the kernel community has established and well working ways to handle them.
Hardware issues like Meltdown, Spectre, L1TF etc. must be treated differently because they usually affect all Operating Systems and therefore need coordination across different OS vendors, distributions, hardware vendors and other parties. For some of the issues, software mitigations can depend on microcode or firmware updates, which need further coordination.
Meltdown/Spectre hit all affected parties completely unprepared, which was nicely reflected in the resulting chaos all over the place. With that experience the kernel community started to push for workable scenarios to handle these kind of issues as it was entirely clear to everyone that this was just the start and the tip of the iceberg.
This talk will take a look at the difference between hardware and software vulnerabilities, gives insight into the events surrounding Meltdown/Spectre and explains how the later issues, e.g. L1TF, have been dealt with. It also looks at the approach the kernel community has taken to further reduce the annoyance for future issues of that kind
Kernel Recipes 2019 - Faster IO through io_uringAnne Nicolas
Since the dawn of time, Linux has had to make do with inferior IO interfaces. Native Linux AIO supports only a niche application class (O_DIRECT), and even for that use case, it’s far too slow for modern storage. This talk will detail io_uring, a modern IO interface for Linux, that’s both fully featured and performant.
Jens Axboe
Embedded Recipes 2019 - RT is about to make it to mainline. Now what?Anne Nicolas
The PREEMPT_RT (aka real-time patch) started back in 2004. Since then, a lot of it has made it into the kernel. The only part left is the change of turning spinning locks into mutexes, then the merge of the PREEMPT_RT patch will be complete. This is expected to happen no later than the Q1 of 2020. But once it is in, how can you take advantage of it? This talk will discuss the basics of writing a real-time application, and some tricks to play with Linux configured with PREEMPT_RT.
Steven Rostedt
Kernel Recipes 2019 - Hunting and fixing bugs all over the Linux kernelAnne Nicolas
At a rate of almost 9 changes per hour (24/7), the Linux kernel is definitely a scary beast. Bugs are introduced on a daily basis and, through the use of multiple code analyzers, *some* of them are detected and fixed before they hit mainline. Over the course of the last few years, Gustavo has been fixing such bugs and many different issues in every corner of the Linux kernel. Recently, he was in charge of leading the efforts to globally enable -Wimplicit-fallthrough; which appears by default in Linux v5.3. This presentation is a report on all the stuff Gustavo has found and fixed in the kernel with the support of the Core Infrastructure Initiative.
Gustavo A. R. SILVA
Kernel Recipes 2019 - pidfds: Process file descriptors on LinuxAnne Nicolas
Traditionally processes are identified globally via process identifiers (PIDs). Due to how pid allocation works the kernel is free to recycle PIDs once a process has been reaped. As such, PIDs do not allow another process to maintain a private, stable reference on a process. On systems under pressure it is thus possible that a PID is recycled without other (non-parent) processes being aware of it. This becomes rather problematic when (non-parent) processes are in charge of managing other processes as is the case for system managers or userspace implementations of OOM killers.
Over the last months we have been working on solving these and other problems by introducing pidfds – process file descriptors. Among other nice properties, the allow callers to maintain a private, stable reference on a process.
In this talk I will explain the challenges we faces, the different approaches people pushed for. We will see what already has been implement and pushed upstream, look into various implementation details and outline what we have planned for the future.
Christian Brauner
This talk will provide several examples of how Facebook engineers use BPF to scale the networking, prevent denial of service, secure containers, analyze performance. It’s suitable for BPF newbies and experts.
Alexei Starovoitov, Facebook
Kernel Recipes 2019 - The ubiquity but also the necessity of eBPF as a techno...Anne Nicolas
Traditionally operating systems are black boxes designed by system engineers that users simply have to “fit into”. Users do not want to be placed in a box, rather they want to use a system that is flexible enough for their needs. In this sense, traditional system developers are arrogant.
Kernel development has to be done in a careful and well crafted manner, it should not be rushed. But this rate of change absolutely cannot keep up with the continually changing needs of users.
Without the necessary level of flexibility, the resulting rigidity will open the doors for competing operating systems to replace Linux as their platform. These systems would simply be giving the users what they actually want. So we must evolve or die.
Therefore, my talk will be about how eBPF is essential as not just a fancy framework for cool fast networking and interesting tracing and introspection, but rather it is an essential component for the long term longevity and survival of the Linux kernel.
David Miller
Kernel Recipes 2019 - What To Do When Your Device Depends on Another OneAnne Nicolas
Contemporary computer systems are quite complicated. There may be multiple connections between various components in them and the components may depend on each other in various ways. At the same time, however, in many cases it is practical to use a separate device driver for each sufficiently distinct system component, depending on the functionality provided by them. Each driver may then focus on providing the required functionality without worrying about the rest of the system too much, but the dependencies between different system components are still there and need to be taken into account in some situations, for example related to power management.
The Linux kernel’s driver model allows the system to be represented as a hierarchy of device objects with a proper tree structure. In that hierarchy, a device can be a leaf, in which case no other devices should depend on it, or it can be a parent of some other device and, as a rule, device drivers bind to individual devices. That is sufficient if the system topology is a proper tree too, but it may not be entirely adequate otherwise. There are cases in which one device uses resources provided by another device that is not its parent and it may not function correctly if the device providing those resources becomes unavailable, for example due to power management.
The device links framework was added to the Linux kernel with such use cases in mind and it has been revamped recently, which is a good opportunity to look at it again. It allows a dependency between two devices to be represented as a link between the corresponding device objects. If such a link is added, the driver core will automatically take it into account in power management, among other places, so that the device providing resources, referred to as the supplier, is always available as long as the device using them, referred to as the consumer, is active. However, there are two types of device links and there are flags to specify at the link creation time to let the driver core know what to use the link for and how to handle it going forward, so some care needs to be taken in order to get the desired results. I will explain how device links work, what the driver core can be expected to do with then depending on the flags used at the link creation time and why it always is a good idea to check the return value of device_link_add().
Rafael Wysocki, Intel
Innovating Inference - Remote Triggering of Large Language Models on HPC Clus...Globus
Large Language Models (LLMs) are currently the center of attention in the tech world, particularly for their potential to advance research. In this presentation, we'll explore a straightforward and effective method for quickly initiating inference runs on supercomputers using the vLLM tool with Globus Compute, specifically on the Polaris system at ALCF. We'll begin by briefly discussing the popularity and applications of LLMs in various fields. Following this, we will introduce the vLLM tool, and explain how it integrates with Globus Compute to efficiently manage LLM operations on Polaris. Attendees will learn the practical aspects of setting up and remotely triggering LLMs from local machines, focusing on ease of use and efficiency. This talk is ideal for researchers and practitioners looking to leverage the power of LLMs in their work, offering a clear guide to harnessing supercomputing resources for quick and effective LLM inference.
Large Language Models and the End of ProgrammingMatt Welsh
Talk by Matt Welsh at Craft Conference 2024 on the impact that Large Language Models will have on the future of software development. In this talk, I discuss the ways in which LLMs will impact the software industry, from replacing human software developers with AI, to replacing conventional software with models that perform reasoning, computation, and problem-solving.
Code reviews are vital for ensuring good code quality. They serve as one of our last lines of defense against bugs and subpar code reaching production.
Yet, they often turn into annoying tasks riddled with frustration, hostility, unclear feedback and lack of standards. How can we improve this crucial process?
In this session we will cover:
- The Art of Effective Code Reviews
- Streamlining the Review Process
- Elevating Reviews with Automated Tools
By the end of this presentation, you'll have the knowledge on how to organize and improve your code review proces
Gamify Your Mind; The Secret Sauce to Delivering Success, Continuously Improv...Shahin Sheidaei
Games are powerful teaching tools, fostering hands-on engagement and fun. But they require careful consideration to succeed. Join me to explore factors in running and selecting games, ensuring they serve as effective teaching tools. Learn to maintain focus on learning objectives while playing, and how to measure the ROI of gaming in education. Discover strategies for pitching gaming to leadership. This session offers insights, tips, and examples for coaches, team leads, and enterprise leaders seeking to teach from simple to complex concepts.
Navigating the Metaverse: A Journey into Virtual Evolution"Donna Lenk
Join us for an exploration of the Metaverse's evolution, where innovation meets imagination. Discover new dimensions of virtual events, engage with thought-provoking discussions, and witness the transformative power of digital realms."
Climate Science Flows: Enabling Petabyte-Scale Climate Analysis with the Eart...Globus
The Earth System Grid Federation (ESGF) is a global network of data servers that archives and distributes the planet’s largest collection of Earth system model output for thousands of climate and environmental scientists worldwide. Many of these petabyte-scale data archives are located in proximity to large high-performance computing (HPC) or cloud computing resources, but the primary workflow for data users consists of transferring data, and applying computations on a different system. As a part of the ESGF 2.0 US project (funded by the United States Department of Energy Office of Science), we developed pre-defined data workflows, which can be run on-demand, capable of applying many data reduction and data analysis to the large ESGF data archives, transferring only the resultant analysis (ex. visualizations, smaller data files). In this talk, we will showcase a few of these workflows, highlighting how Globus Flows can be used for petabyte-scale climate analysis.
We describe the deployment and use of Globus Compute for remote computation. This content is aimed at researchers who wish to compute on remote resources using a unified programming interface, as well as system administrators who will deploy and operate Globus Compute services on their research computing infrastructure.
Prosigns: Transforming Business with Tailored Technology SolutionsProsigns
Unlocking Business Potential: Tailored Technology Solutions by Prosigns
Discover how Prosigns, a leading technology solutions provider, partners with businesses to drive innovation and success. Our presentation showcases our comprehensive range of services, including custom software development, web and mobile app development, AI & ML solutions, blockchain integration, DevOps services, and Microsoft Dynamics 365 support.
Custom Software Development: Prosigns specializes in creating bespoke software solutions that cater to your unique business needs. Our team of experts works closely with you to understand your requirements and deliver tailor-made software that enhances efficiency and drives growth.
Web and Mobile App Development: From responsive websites to intuitive mobile applications, Prosigns develops cutting-edge solutions that engage users and deliver seamless experiences across devices.
AI & ML Solutions: Harnessing the power of Artificial Intelligence and Machine Learning, Prosigns provides smart solutions that automate processes, provide valuable insights, and drive informed decision-making.
Blockchain Integration: Prosigns offers comprehensive blockchain solutions, including development, integration, and consulting services, enabling businesses to leverage blockchain technology for enhanced security, transparency, and efficiency.
DevOps Services: Prosigns' DevOps services streamline development and operations processes, ensuring faster and more reliable software delivery through automation and continuous integration.
Microsoft Dynamics 365 Support: Prosigns provides comprehensive support and maintenance services for Microsoft Dynamics 365, ensuring your system is always up-to-date, secure, and running smoothly.
Learn how our collaborative approach and dedication to excellence help businesses achieve their goals and stay ahead in today's digital landscape. From concept to deployment, Prosigns is your trusted partner for transforming ideas into reality and unlocking the full potential of your business.
Join us on a journey of innovation and growth. Let's partner for success with Prosigns.
Top 7 Unique WhatsApp API Benefits | Saudi ArabiaYara Milbes
Discover the transformative power of the WhatsApp API in our latest SlideShare presentation, "Top 7 Unique WhatsApp API Benefits." In today's fast-paced digital era, effective communication is crucial for both personal and professional success. Whether you're a small business looking to enhance customer interactions or an individual seeking seamless communication with loved ones, the WhatsApp API offers robust capabilities that can significantly elevate your experience.
In this presentation, we delve into the top 7 distinctive benefits of the WhatsApp API, provided by the leading WhatsApp API service provider in Saudi Arabia. Learn how to streamline customer support, automate notifications, leverage rich media messaging, run scalable marketing campaigns, integrate secure payments, synchronize with CRM systems, and ensure enhanced security and privacy.
OpenFOAM solver for Helmholtz equation, helmholtzFoam / helmholtzBubbleFoamtakuyayamamoto1800
In this slide, we show the simulation example and the way to compile this solver.
In this solver, the Helmholtz equation can be solved by helmholtzFoam. Also, the Helmholtz equation with uniformly dispersed bubbles can be simulated by helmholtzBubbleFoam.
May Marketo Masterclass, London MUG May 22 2024.pdfAdele Miller
Can't make Adobe Summit in Vegas? No sweat because the EMEA Marketo Engage Champions are coming to London to share their Summit sessions, insights and more!
This is a MUG with a twist you don't want to miss.
Enhancing Research Orchestration Capabilities at ORNL.pdfGlobus
Cross-facility research orchestration comes with ever-changing constraints regarding the availability and suitability of various compute and data resources. In short, a flexible data and processing fabric is needed to enable the dynamic redirection of data and compute tasks throughout the lifecycle of an experiment. In this talk, we illustrate how we easily leveraged Globus services to instrument the ACE research testbed at the Oak Ridge Leadership Computing Facility with flexible data and task orchestration capabilities.
Globus Compute wth IRI Workflows - GlobusWorld 2024Globus
As part of the DOE Integrated Research Infrastructure (IRI) program, NERSC at Lawrence Berkeley National Lab and ALCF at Argonne National Lab are working closely with General Atomics on accelerating the computing requirements of the DIII-D experiment. As part of the work the team is investigating ways to speedup the time to solution for many different parts of the DIII-D workflow including how they run jobs on HPC systems. One of these routes is looking at Globus Compute as a way to replace the current method for managing tasks and we describe a brief proof of concept showing how Globus Compute could help to schedule jobs and be a tool to connect compute at different facilities.
Enterprise Resource Planning System includes various modules that reduce any business's workload. Additionally, it organizes the workflows, which drives towards enhancing productivity. Here are a detailed explanation of the ERP modules. Going through the points will help you understand how the software is changing the work dynamics.
To know more details here: https://blogs.nyggs.com/nyggs/enterprise-resource-planning-erp-system-modules/
Unleash Unlimited Potential with One-Time Purchase
BoxLang is more than just a language; it's a community. By choosing a Visionary License, you're not just investing in your success, you're actively contributing to the ongoing development and support of BoxLang.
How to Position Your Globus Data Portal for Success Ten Good PracticesGlobus
Science gateways allow science and engineering communities to access shared data, software, computing services, and instruments. Science gateways have gained a lot of traction in the last twenty years, as evidenced by projects such as the Science Gateways Community Institute (SGCI) and the Center of Excellence on Science Gateways (SGX3) in the US, The Australian Research Data Commons (ARDC) and its platforms in Australia, and the projects around Virtual Research Environments in Europe. A few mature frameworks have evolved with their different strengths and foci and have been taken up by a larger community such as the Globus Data Portal, Hubzero, Tapis, and Galaxy. However, even when gateways are built on successful frameworks, they continue to face the challenges of ongoing maintenance costs and how to meet the ever-expanding needs of the community they serve with enhanced features. It is not uncommon that gateways with compelling use cases are nonetheless unable to get past the prototype phase and become a full production service, or if they do, they don't survive more than a couple of years. While there is no guaranteed pathway to success, it seems likely that for any gateway there is a need for a strong community and/or solid funding streams to create and sustain its success. With over twenty years of examples to draw from, this presentation goes into detail for ten factors common to successful and enduring gateways that effectively serve as best practices for any new or developing gateway.
3. whoami(1)
40 years old nerd
Been pushing buttons on a C64 since i was 9
opensource software user since 1996 ( slackware 3.1 )
Hacked kernel code for the first time in 1999 (ISDN modem)
Wrote a few patches for linux/opensolaris/FreeBSD kernels over the past 19
years
Contributed a few patches for various observability projects
On-call for the last 19 years
Woken up for stupid things for 19 years…
Been happily working for synthesio.com for 2.5 years
5. talk(1)
<Friend> "wow congratulations on making it to
the KR conferences"
<Me> "Thanks !"
<Friend> "i was looking at the KR speakers list.
I saw the usual legends. And you. Good luck with
that. Sincerely"
6. This is not meant to be a public shaming session
Names and bugs were voluntarily removed
Explaining these bugs/patches to most of you would be…
incongruous
You probably wrote or validated the bug… and the fix
motd(5)
8. alarm(2)
500 HTTP error
non zero shell return code
Segfault
Kernel panic
OOM
CRC errors
Network problems
No data
No graph
[...]
9. sleep(1)
HTTP 200 error
Failed shell script returning 0
Segfault hidden by a process supervisor
Silent data corruption
Unknown states
Pattern change
No timeout on probe
[...]
15. free(1)
So ops guys brains working memory are saturated, among other things, by metrics
What if… Even the most basic metrics weren’t what you thought they’d be ?
What if… The same metric did not mean the same from a server to another ?
What if… We were all wrong most part of the time ?
18. top(1)
I have played a game with other mid to senior ops guys : 2 out of 10 were
almost correct.
“Load averages are an industry-critical metric – my company spends millions
auto-scaling cloud instances based on them and other metrics – but on Linux
there's some mystery around them.” Brendan Gregg (2017)
After all it only took around 12 screens to Brendan Gregg to explain linux
load average history.
Oh and good news, linux computes load differently than other kernel/OS
20. irssi(1) /query foo
<foo> is hired, replaces a dying home-made linux-based switch with a very
common one
<foo> adds metrics to this brand new switch and figures out something is wrong
Switch and server are absolutely not giving the same results : at least 50%
drop on all network tx/rx metrics during the usual benchmarks
<foo> examines the dashboard configuration : there’s also a max() function but
that was just an aggravating factor not the root cause
<foo> beorn you know collectd-fu right ?
21. vim(1) ~/collectd core/plugin code
Collectd code was pretty straight forward
Collectd reads data from /proc/net/dev
No voodoo magic here
22. history(3)
The server and the linux-based-old-switch were running the exact same old
linux kernel version
And could not be simply upgraded because of proprietary drivers of specific
components
23. proc(5) /proc/net/dev
When this story happens there was almost no documentation for /proc/net/dev
Gladly there was this old email that gave some useful hints.
24. mail(1)
> How can I find out the /proc/net info
>
> eg: softnet_stat is for what purpose
Much of this is only well-documented in the code. Here's an attempt
at interpreting softnet_stat [no guarantee that it is correct; read the code!]:
% softnet_stat.sh
cpu total dropped squeezed collision
0 1794619684 0 346 0
1 36399632 0 74 2
% softnet_stat.sh -h
usage: softnet_stat.sh [ -h ]
[...]
25. mutt(1)
Output column definitions:
cpu # of the cpu
total # of packets (not including netpoll) received by the interrupt handler
There might be some double counting going on:
net/core/dev.c:1643: __get_cpu_var(netdev_rx_stat).total++;
net/core/dev.c:1836: __get_cpu_var(netdev_rx_stat).total++;
I think the intention was that these were originally on separate
receive paths ...
dropped # of packets that were dropped because netdev_max_backlog was exceeded
squeezed # of times ksoftirq ran out of netdev_budget or time slice with work
remaining
collision # of times that two cpus collided trying to get the device queue lock.
27. git-log(1)
# git log --pretty=oneline --abbrev-commit |grep igb| grep stats
55c05dd0295d igbvf: Use net_device_stats from struct net_device
e66c083aab32 igb: fix stats for i210 rx_fifo_errors
3dbdf96928dc igb: Fix stats output on i210/i211 parts.
0a915b95d67f igb: Add stats output for OS2BMC feature on i350 devices
12dcd86b75d5 igb: fix stats handling
43915c7c9a99 igb: only read phy specific stats if in internal phy mode
128e45eb61b9 igb: Rework how netdev->stats is handled
645a3abd73c2 igb: Remove invalid stats counters
3f9c01648146 igb: only process global stats in igb_update_stats
04a5fcaaf0e1 igb: move alloc_failed and csum_err stats into per rx-ring stat
231835e4163c igb: Fix erroneous display of stats by ethtool -S
8d24e93309d6 igb: Use the instance of net_device_stats from net_device.
cc9073bbc901 igb: remove unused temp variable from stats clearing path
3ea73afafb8c igb: Record host memory receive overflow in net_stats
04fe63583d46 igb: update stats before doing reset in igb_down
e21ed3538f19 igb: update ethtool stats to support multiqueue
28. git-commit(1)
/*
* ENOBUFS = no kernel mem, SOCK_NOSPACE = no sndbuf space. Reporting
* ENOBUFS might not be good (it's not tunable per se), but otherwise
* we don't have a good statistic (IpOutDiscards but it can be too many
* things). We could add another new stat but at least for now that
* seems like overkill.
*/
29. ethtool(8)
Over the years the Linux networking stack had hoarded:
● Tens of (cool) features ( GRO, GSO, RPS, RFS, …)
● Tens of drivers
● Tons of code-paths
● Multiple thousand sysctl entries
● A lot of bugs
Manufacturer’s tech document was 1200 pages long, and is probably bigger today
Documentation was not what it is today
30. sha512sum(1)
To sum it up
● notice the problem
● Fix the graph configuration (bad aggr)
● start reading the userland stats collecting (collectd here)
● realize and make sure the bug was not there
● read your kernel/driver code
● realize the bug is really in the code path that /proc/net/dev hits
● read the 1200 pages tech specs from the manufacturer
● find a few related patches
● rebuild the kernel/driver only 4 times (we were lucky)
● And just reboot production servers for weeks
All that just to read valid tx.rx packets counters !
31. bc(1)
<foo> Last year based on these metrics they doubled network
capacity for more than 2.8M euros
33. We had many servers of a validated type with 10Gbps ixgbe nics
According to capacity planning we order a 240 servers batch
Linux TCP/IP network statistics are bad : tcp retransmits, latencies, …
The new switch metrics were green
Linux did not have signal related statistics for fiber NICs
Another brand/model of SFP+ worked just fine
netstat(1)
34. mutt(1) reseller
<reseller> everything is fine
<me> if only we’ve had those SFP+ DOM registers in kernel/ethtool…
<CTO> you have 10 full days to prove them wrong
<me> Erm it’s the network stack we’re talking about and i’m no real kernel
dev
<CTO> That’s why you have 10 full days to prove them wrong
35. Read everything i could find about optical signal, SFP+ and DOM statistics
Found a microrouter project named bifrost doing just this with a 2.6 kernel
(2012)
Their Patches were never pushed upstream
We needed it to run on 3.4 kernels for features and hardware compatibility
Emailed the guys about a 3.4 patch: no luck
Let’s port this to 3.4
links(1)
36. The network API had major changes between 2.6 and 3.4 on this particular part.
Ended up rewriting the patchset (kernel + ethtool) entirely in 5 days
Patch worked in production for a 3-4 years without a glitch
vim(1) patchset.diff
# ethtool -D eth0
Int-Calbr: Avr RX-Power: Alarm & Warn: TX_DISABLE: TX_FAULT: RX_LOS:
RATE_SELECT MON: RATE_SELECT: Wavelength: 850 nm
Alarms, warnings in beginning of line, Ie. AH = Alarm High, WL == Warn
Low etc
Temp: 45.7 C Thresh: Lo: -45.0/-40.0 Hi: 115.0/125.0 C
Vcc: 3.32 V Thresh: Lo: 2.7/2.9 Hi: 3.7/3.9 V
Tx-Bias: 6.6 mA Thresh: Lo: 1.0/2.0 Hi: 12.0/15.0 mA
TX-pwr: -2.9 dBm ( 0.52 mW) Thresh: Lo: -10.0/-8.3 Hi: 0.8/2.0 dBm
RX-pwr: -1.9 dBm ( 0.64 mW) Thresh: Lo: -16.0/-14.2 Hi: 1.8/3.0 dBm
37. Proud and happy i wrote an email to someone “doing things in the kernel”
<kernelguy> “$#!$#!$$@%$#%#!$%#%@$”
<me> “So what should i fix ?”
<End Of Discussion>
git-format-patch(1)
38. After adding the ethtool output and the patchset into the reseller’s case he
agreed to change the incompatible SFP+ after only 5 days of hard work
480 brand new SFP+ arrived. We changed the faulty SFP+ for weeks.
And that was it
Roughly 200K euros were saved with these metrics
hledger(1)
40. iozone(1)
In a hosting company we built ZFS based SAN/NAS
<coworker> last batch of servers have serious storage
performances issues under load
<SRE> alright let’s dig
41. sha256sum(1)
# iostat -E c0t5000C5004124B687d0
sd31 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: SEAGATE Product: ST2000NM0001 Revision: PS04 Serial No: Z1P1HECD
Size: 2000.40GB <2000398934016 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
# iostat -E c11t50014EE3000E9080d0
sd22 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: WD Product: WD2000FYYG Revision: D1B3 Serial No: WMAWP0192044
Size: 2000.40GB <2000398934016 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 4 Predictive Failure Analysis: 0
Disk had the same labels, same tech specs, but not exactly the
same physical look
42. alpine(1)
<me> Sir it is not the same disk brand/model
<reseller> we do not guarantee anything else that tech specs
<me> [...] Please do something !
43. orion(1)
Device: rrqm/s wrqm/s r/s w/s rMB/s wMB/s avgrq-sz avgqu-sz await svctm %util
sde 0.00 0.00 1.00 246.00 0.00 123.00 1019.89 124.77 127.80 4.05 100.10
sdc 0.00 0.00 1.00 104.00 0.00 52.00 1014.32 120.05 896.32 9.52 100.00
After extensive profiling we are able to reproduce the
problematic workload
Which happens to be a very important workload for ZFS
44. sup(1)
<manufacturer> Our test suite shows that the disks you have sent are fine
<me> except they are not. see the iostat output
[nothing for 1 week]
45. smartctl(1)
/dev/sde: SEAGATE ST2000NM0001 PS04
Direct access device specific parameters: WP=0 DPOFUA=1
Read write error recovery [rw] mode page:
AWRE 1 [cha: y, def: 1, sav: 1] Automatic write reallocation enabled
ARRE 1 [cha: y, def: 1, sav: 1] Automatic read reallocation enabled
TB 0 [cha: y, def: 0, sav: 0] Transfer block
RC 0 [cha: y, def: 0, sav: 0] Read continuous
EER 0 [cha: y, def: 0, sav: 0] Enable early recovery
PER 0 [cha: y, def: 0, sav: 0] Post error
DTE 0 [cha: y, def: 0, sav: 0] Data terminate on error
DCR 0 [cha: y, def: 0, sav: 0] Disable correction
RRC 20 [cha: y, def: 20, sav: 20] Read retry count
COR_S 255 [cha: n, def:255, sav:255] Correction span (obsolete)
HOC 0 [cha: n, def: 0, sav: 0] Head offset count (obsolete)
DSOC 0 [cha: n, def: 0, sav: 0] Data strobe offset count (obsolete)
TPERE 0 [cha: n, def: 0, sav: 0] Thin provisioning error reporting enabled
WRC 5 [cha: y, def: 5, sav: 5] Write retry count
RTL 8000 [cha: y, def:8000, sav:8000] Recovery time limit (ms)
Disconnect-reconnect (SPC + transports) [dr] mode page:
BFR 0 [cha: n, def: 0, sav: 0] Buffer full ratio
BER 0 [cha: n, def: 0, sav: 0] Buffer empty ratio
BIL 0 [cha: n, def: 0, sav: 0] Bus inactivity limit
DTL 0 [cha: n, def: 0, sav: 0] Disconnect time limit
CTL 0 [cha: n, def: 0, sav: 0] Connect time limit
MBS 314 [cha: y, def:314, sav:314] Maximum burst size (512 bytes)
EMDP 0 [cha: n, def: 0, sav: 0] Enable modify data pointers
FA 0 [cha: n, def: 0, sav: 0] Fair arbitration
DIMM 0 [cha: n, def: 0, sav: 0] Disconnect immediate
DTDC 0 [cha: n, def: 0, sav: 0] Data transfer disconnect control
FBS 0 [cha: n, def: 0, sav: 0] First burst size (512 bytes)
Format (SBC) [fo] mode page:
TPZ 48080 [cha: n, def:48080, sav:48080] Tracks per zone
ASPZ 0 [cha: n, def: 0, sav: 0] Alternate sectors per zone
ATPZ 0 [cha: n, def: 0, sav: 0] Alternate tracks per zone
ATPLU 896 [cha: n, def:896, sav:896] Alternate tracks per logical unit
SPT 1220 [cha: n, def:1220, sav:1220] Sectors per track
DBPPS 512 [cha: n, def:512, sav:512] Data bytes per physical sector
INTLV 1 [cha: n, def: 1, sav: 1] Interleave
TSF 156 [cha: n, def:156, sav:156] Track skew factor
CSF 38 [cha: n, def: 38, sav: 38] Cylinder skew factor
SSEC 0 [cha: n, def: 0, sav: 0] Soft sector
HSEC 1 [cha: n, def: 1, sav: 1] Hard sector
RMB 0 [cha: n, def: 0, sav: 0] Removable
SURF 0 [cha: n, def: 0, sav: 0] Surface
Rigid disk (SBC) [rd] mode page:
NOC 249000 [cha: n, def:249000, sav:249000] Number of cylinders
NOH 8 [cha: n, def: 8, sav: 8] Number of heads
SCWP 0 [cha: n, def: 0, sav: 0] Starting cylinder for write
precompensation
SCRWC 0 [cha: n, def: 0, sav: 0] Starting cylinder for reduced write
current
DSR 0 [cha: n, def: 0, sav: 0] Device step rate
LZC 0 [cha: n, def: 0, sav: 0] Landing zone cylinder
RPL 0 [cha: n, def: 0, sav: 0] Rotational position locking
ROTO 0 [cha: n, def: 0, sav: 0] Rotational offset
MRR 7200 [cha: n, def:7200, sav:7200] Medium rotation rate (rpm)
Verify error recovery (SBC) [ve] mode page:
V_EER 0 [cha: y, def: 0, sav: 0] Enable early recovery
V_PER 0 [cha: y, def: 0, sav: 0] Post error
V_DTE 0 [cha: y, def: 0, sav: 0] Data terminate on error
V_DCR 0 [cha: y, def: 0, sav: 0] Disable correction
V_RC 20 [cha: y, def: 20, sav: 20] Verify retry count TSF 156 [cha:
n, def:156, sav:156] Track skew factor
V_COR_S 255 [cha: n, def:255, sav:255] Verify correction span (obsolete)
V_RTL 8000 [cha: y, def:8000, sav:8000] Verify recovery time limit (ms)
Caching (SBC) [ca] mode page:
IC 0 [cha: y, def: 0, sav: 0] Initiator control
ABPF 0 [cha: n, def: 0, sav: 0] Abort pre-fetch
CAP 0 [cha: y, def: 0, sav: 0] Caching analysis permitted
DISC 1 [cha: n, def: 1, sav: 1] Discontinuity
SIZE 0 [cha: n, def: 0, sav: 0] Size enable
WCE 0 [cha: y, def: 0, sav: 0] Write cache enable
MF 0 [cha: n, def: 0, sav: 0] Multiplication factor
RCD 0 [cha: y, def: 0, sav: 0] Read cache disable
DRRP 0 [cha: n, def: 0, sav: 0] Demand read retention priority
WRP 0 [cha: n, def: 0, sav: 0] Write retention priority
DPTL -1 [cha: n, def: -1, sav: -1] Disable pre-fetch transfer length
MIPF 0 [cha: y, def: 0, sav: 0] Minimum pre-fetch
MAPF -1 [cha: y, def: -1, sav: -1] Maximum pre-fetch
MAPFC -1 [cha: n, def: -1, sav: -1] Maximum pre-fetch ceiling
FSW 1 [cha: n, def: 1, sav: 1] Force sequential write
LBCSS 0 [cha: n, def: 0, sav: 0] Logical block cache segment size
DRA 0 [cha: y, def: 0, sav: 0] Disable read ahead
NV_DIS 0 [cha: n, def: 0, sav: 0] Non-volatile cache disable
NCS 32 [cha: n, def: 32, sav: 32] Number of cache segments
CSS 0 [cha: n, def: 0, sav: 0] Cache segment size
Control [co] mode page:
TST 0 [cha: n, def: 0, sav: 0] Task set type
TMF_ONLY 0 [cha: n, def: 0, sav: 0] Task management functions only
D_SENSE 0 [cha: y, def: 0, sav: 0] Descriptor format sense data
GLTSD 0 [cha: y, def: 0, sav: 0] Global logging target save disable
RLEC 0 [cha: y, def: 0, sav: 0] Report log exception condition
QAM 0 [cha: y, def: 0, sav: 0] Queue algorithm modifier
QERR 0 [cha: y, def: 0, sav: 0] Queue error management
RAC 0 [cha: n, def: 0, sav: 0] Report a check
UA_INTLCK 0 [cha: n, def: 0, sav: 0] Unit attention interlocks control
SWP 0 [cha: n, def: 0, sav: 0] Software write protect
ATO 0 [cha: n, def: 0, sav: 0] Application tag owner
TAS 0 [cha: n, def: 0, sav: 0] Task aborted status
AUTOLOAD 0 [cha: n, def: 0, sav: 0] Autoload mode
BTP 0 [cha: n, def: 0, sav: 0] Busy timeout period (100us)
ESTCT 18500 [cha: n, def:18500, sav:18500] Extended self test completion time
(sec)
Protocol specific logical unit [pl] mode page:
LUPID 6 [cha: n, def: 6, sav: 6] Logical unit's (transport) protocol
identifier
Protocol specific port [pp] mode page:
PPID 6 [cha: n, def: 6, sav: 6] Port's (transport) protocol identifier
Power condition [po] mode page:
STANDBY_Y 0 [cha: n, def: 0, sav: 0] Standby_y timer enabled
IDLE_C 0 [cha: n, def: 0, sav: 0] Idle_c timer enabled
IDLE_B 0 [cha: n, def: 0, sav: 0] Idle_b timer active
IDLE 1 [cha: y, def: 1, sav: 1] Idle timer enabled
STANDBY 0 [cha: y, def: 0, sav: 0] Standby timer active
ICT 5 [cha: y, def: 5, sav: 5] Idle condition timer (100 ms)
SCT 36000 [cha: n, def:36000, sav:36000] Standby condition timer (100 ms)
Informational exceptions control [ie] mode page:
PERF 0 [cha: y, def: 0, sav: 0] Performance (impact of ie operations)
EBF 0 [cha: n, def: 0, sav: 0] Enable background function
EWASC 0 [cha: y, def: 0, sav: 0] Enable warning
DEXCPT 0 [cha: y, def: 0, sav: 0] Disable exceptions
TEST 0 [cha: y, def: 0, sav: 0] Test (simulate device failure)
EBACKERR 0 [cha: n, def: 0, sav: 0] Enable background (scan + self test)
error reporting
LOGERR 1 [cha: y, def: 1, sav: 1] Log informational exception errors
MRIE 6 [cha: y, def: 6, sav: 6] Method of reporting informational
exceptions
INTT 6000 [cha: y, def:6000, sav:6000] Interval timer (100 ms)
REPC 0 [cha: n, def: 0, sav: 0] Report count (or Test flag number
[SSC-3])
Background control (SBC) [bc] mode page:
S_L_FULL 0 [cha: n, def: 0, sav: 0] Suspend on log full
LOWIR 0 [cha: n, def: 0, sav: 0] Log only when intervention required
EN_BMS 1 [cha: y, def: 1, sav: 1] Enable background medium scan
EN_PS 0 [cha: n, def: 0, sav: 0] Enable pre-scan
BMS_I 336 [cha: y, def:336, sav:336] Background medium scan interval time
(hour)
BPS_TL 24 [cha: y, def: 24, sav: 24] Background pre-scan time limit (hour)
MIN_IDLE 250 [cha: y, def:250, sav:250] Minumum idle time before background scan
(ms)
MAX_SUSP 0 [cha: y, def: 0, sav: 0] Maximum time to suspend background scan
(ms)
46. :(){ :|:& };:
At the same time they provided us a “fix” firmware
Performances were even better than with the good disk.
Binary diff showed a 1 bit change.
Yes a boolean.
Their firmware was silently enabling write cache (without
battery)
Which is to say the least dangerous
47. dc(1)
--- st2000nm00001_sdparm.txt 2013-02-28 21:12:15.287433646 +0100
+++ wd2000fyyg_sdparm.txt 2013-02-28 21:12:28.823131392 +0100
@@ -1,102 +1,97 @@
- /dev/sde: SEAGATE ST2000NM0001 PS04
+ /dev/sdc: WD WD2000FYYG D1BB
Direct access device specific parameters: WP=0 DPOFUA=1
Read write error recovery [rw] mode page:
- RC 0 [cha: y, def: 0, sav: 0] Read continuous
+ RC 0 [cha: n, def: 0, sav: 0] Read continuous
- RRC 20 [cha: y, def: 20, sav: 20] Read retry count
- COR_S 255 [cha: n, def:255, sav:255] Correction span (obsolete)
+ RRC 255 [cha: y, def:255, sav:255] Read retry count
+ COR_S 0 [cha: n, def: 0, sav: 0] Correction span (obsolete)
- WRC 5 [cha: y, def: 5, sav: 5] Write retry count
+ WRC 255 [cha: y, def:255, sav:255] Write retry count
Verify error recovery (SBC) [ve] mode page:
- V_RC 20 [cha: y, def: 20, sav: 20] Verify retry count
- V_COR_S 255 [cha: n, def:255, sav:255] Verify correction span (obsolete)
+ V_RC 255 [cha: y, def:255, sav:255] Verify retry count
+ V_COR_S 0 [cha: n, def: 0, sav: 0] Verify correction span (obsolete)
Caching (SBC) [ca] mode page:
- FSW 1 [cha: n, def: 1, sav: 1] Force sequential write
+ FSW 0 [cha: y, def: 0, sav: 0] Force sequential write
Informational exceptions control [ie] mode page:
- PERF 0 [cha: y, def: 0, sav: 0] Performance (impact of ie operations)
- EBF 0 [cha: n, def: 0, sav: 0] Enable background function
+ PERF 0 [cha: n, def: 0, sav: 0] Performance (impact of ie operations)
+ EBF 1 [cha: y, def: 1, sav: 1] Enable background function
- EBACKERR 0 [cha: n, def: 0, sav: 0] Enable background (scan + self test)
error reporting
- LOGERR 1 [cha: y, def: 1, sav: 1] Log informational exception errors
+ EBACKERR 0 [cha: y, def: 0, sav: 0] Enable background (scan + self test)
error reporting
+ LOGERR 1 [cha: n, def: 1, sav: 1] Log informational exception errors
- REPC 0 [cha: n, def: 0, sav: 0] Report count (or Test flag number
[SSC-3])
+ REPC 0 [cha: y, def: 0, sav: 0] Report count (or Test flag number
[SSC-3])
Background control (SBC) [bc] mode page:
- S_L_FULL 0 [cha: n, def: 0, sav: 0] Suspend on log full
- LOWIR 0 [cha: n, def: 0, sav: 0] Log only when intervention required
+ S_L_FULL 0 [cha: y, def: 0, sav: 0] Suspend on log full
+ LOWIR 0 [cha: y, def: 0, sav: 0] Log only when intervention required
- EN_PS 0 [cha: n, def: 0, sav: 0] Enable pre-scan
+ EN_PS 0 [cha: y, def: 0, sav: 0] Enable pre-scan
- BPS_TL 24 [cha: y, def: 24, sav: 24] Background pre-scan time limit (hour)
+ BPS_TL 0 [cha: y, def: 0, sav: 0] Background pre-scan time limit (hour)
We proved the disks did not
have the same behavior/specs
using this diff
The reseller changed all the
250 disks 150K euros
We spent the next months
changing and resilvering
arrays already in production
49. sha256sum(1)
Metrics are everywhere in operations at an unprecedented scale and still
growing fast
The vast majority of I.T. professionals do not understand fully what they are
currently graphing
Graphs are meant to trigger a deeper questioning when the behavior changes
To make a costly decision based on metrics without taking the time to ensure
what is exactly this metric is pure folly
Acquiring this knowledge is necessary and time consuming and requires humility
50. task(1) add project:young_me [...]
Kernel code ain’t no saint writing
Macros make things easy if you are not a C guru
Read git history per sub-system it helps a lot
Ask upstream if they are interested in what you plan to write
Propose a (probably stupid) way of doing the change before doing code
Then code and get things upstreamed