DOXLON November 2016: Facebook Engineering on cgroupv2Outlyer
Cgroupv1 (or just "cgroups") has helped revolutionize the way that we manage and use containers over the past 8 years. In kernel 4.5, a complete overhaul is coming -- cgroupv2. This talk will go into why a new control group system was needed, the changes from cgroupv1, and practical uses that you can apply to improve the level of control you have over the processes on your servers.
Erasing Belady's Limitations: In Search of Flash Cache Offline OptimalityYue Cheng
1) Unlike HDDs, flash SSDs erase data in large blocks and have a limited number of erasures before errors occur. When using SSDs as a cache for HDDs, the goal is to balance performance and endurance.
2) Existing flash caching algorithms aim to optimize read hit ratio (RHR) and endurance but it is unknown if they achieve the true offline optimal. The offline optimal maximizes RHR subject to an endurance limit.
3) The authors present a new container-optimized heuristic that maintains the optimal RHR while reducing erasures by up to 67% compared to existing algorithms. It outperforms prior work in balancing performance and endurance.
This document provides an overview of RabbitMQ and Kafka for microservices integration. It discusses the challenges of fully connected microservices and how a message broker can help. RabbitMQ uses AMQP and supports many clients while Kafka is optimized for streaming large volumes of data. The document demonstrates using each for event sourcing. It outlines some common use cases for each and notes that both tools solve different problems so an organization may choose to use both depending on their needs.
Cache coherence is an issue that arises in multiprocessing systems where multiple processors have cached copies of shared memory locations. If a processor modifies its local copy, it can create an inconsistent global view of memory.
There are two main approaches to maintaining cache coherence - snoopy bus protocols and directory schemes. Snoopy bus protocols use a shared bus for processors to monitor memory transactions and invalidate local copies when needed. Directory schemes track which processors are sharing each block of data using a directory structure.
One common snoopy protocol is MESI, which uses cache states of Modified, Exclusive, Shared, and Invalid to track the ownership of cache lines and ensure coherency is maintained when a line is modified.
Agenda:
Have you ever wondered what happens when the kernel fires up? What is going on under the hood before init process is executed?
This talk will go into great depths explaining the entire process. From linker tricks and init sections to mounting and locating the init process to execute.
Speaker:
Boaz Taitler, experienced kernel developer.
This presentation is about Distributed Applications, points:
- Distribution
- Consistency
- Replication
- Distributed KV in Erlang
- Scaling
- Partitioning
- Observability
DOXLON November 2016: Facebook Engineering on cgroupv2Outlyer
Cgroupv1 (or just "cgroups") has helped revolutionize the way that we manage and use containers over the past 8 years. In kernel 4.5, a complete overhaul is coming -- cgroupv2. This talk will go into why a new control group system was needed, the changes from cgroupv1, and practical uses that you can apply to improve the level of control you have over the processes on your servers.
Erasing Belady's Limitations: In Search of Flash Cache Offline OptimalityYue Cheng
1) Unlike HDDs, flash SSDs erase data in large blocks and have a limited number of erasures before errors occur. When using SSDs as a cache for HDDs, the goal is to balance performance and endurance.
2) Existing flash caching algorithms aim to optimize read hit ratio (RHR) and endurance but it is unknown if they achieve the true offline optimal. The offline optimal maximizes RHR subject to an endurance limit.
3) The authors present a new container-optimized heuristic that maintains the optimal RHR while reducing erasures by up to 67% compared to existing algorithms. It outperforms prior work in balancing performance and endurance.
This document provides an overview of RabbitMQ and Kafka for microservices integration. It discusses the challenges of fully connected microservices and how a message broker can help. RabbitMQ uses AMQP and supports many clients while Kafka is optimized for streaming large volumes of data. The document demonstrates using each for event sourcing. It outlines some common use cases for each and notes that both tools solve different problems so an organization may choose to use both depending on their needs.
Cache coherence is an issue that arises in multiprocessing systems where multiple processors have cached copies of shared memory locations. If a processor modifies its local copy, it can create an inconsistent global view of memory.
There are two main approaches to maintaining cache coherence - snoopy bus protocols and directory schemes. Snoopy bus protocols use a shared bus for processors to monitor memory transactions and invalidate local copies when needed. Directory schemes track which processors are sharing each block of data using a directory structure.
One common snoopy protocol is MESI, which uses cache states of Modified, Exclusive, Shared, and Invalid to track the ownership of cache lines and ensure coherency is maintained when a line is modified.
Agenda:
Have you ever wondered what happens when the kernel fires up? What is going on under the hood before init process is executed?
This talk will go into great depths explaining the entire process. From linker tricks and init sections to mounting and locating the init process to execute.
Speaker:
Boaz Taitler, experienced kernel developer.
This presentation is about Distributed Applications, points:
- Distribution
- Consistency
- Replication
- Distributed KV in Erlang
- Scaling
- Partitioning
- Observability
This document summarizes two shared memory architectures - bus-based and directory-based. It describes:
1) Bus-based architectures have CPUs, caches and shared memory connected by a shared bus. The bus bandwidth limits scalability. It discusses the memory coherence problem and snooping protocols like MESI to address it.
2) Directory-based architectures avoid broadcast snooping and scale better using point-to-point messaging. Each block tracks its presence in caches using a directory with processor bits. It brings coherence through directory lookups and targeted invalidations.
This document discusses Google's use of CRIU for task migration at scale. It provides background on Borg, Google's cluster management system, and how tasks run in isolated containers. CRIU is used to checkpoint and restore task state, allowing tasks to be migrated transparently to avoid evictions. While migrations currently take 1-2 minutes, work is ongoing to improve performance and implement live migration to support latency-sensitive tasks. Security around CRIU's use of privileges is also an area of focus. Overall, CRIU has worked well but continued collaboration is needed to address remaining challenges.
The document provides an overview of Logical Volume Management (LVM) in Linux. It discusses what LVM is, its main components like physical volumes, volume groups, logical volumes, and how they relate. It then gives steps to use LVM by creating a physical volume, volume group and logical volume. It also discusses how LVM allows expanding logical volumes and live resizing of file systems.
Control groups (cgroups) allow administrators to allocate CPU, memory, storage, and other system resources to groups of processes running on the system. The document describes testing done using cgroups on a Red Hat Enterprise Linux 6 system with four Oracle database instances running an OLTP workload. It demonstrates how cgroups can be used for application consolidation, performance optimization, dynamic resource management, and application isolation.
This document discusses cache coherence in single and multiprocessor systems. It provides techniques to avoid inconsistencies between cache and main memory including write-through, write-back, and instruction caching. For multiprocessors, it discusses issues with sharing writable data, process migration, and I/O activity. Software solutions involve compiler and OS management while hardware uses coherence protocols like snoopy and directory protocols.
Coherence and consistency models in multiprocessor architectureUniversity of Pisa
Cache coherence and consistency model in multiprocessor architecture. These slide show the introduction of multiprocessor and cache multilevel and then describe the basic mechanism of coherence and consistency protocols. In particular the protocols describe are the following: snooping and directory protocols for the coherence part and sequential protocol for the consistency part. There are also example of (in)consistency and (in)coherence.
Parallelism-Aware Memory Interference Delay Analysis for COTS Multicore SystemsHeechul Yun
In modern Commercial Off-The-Shelf (COTS) multicore systems, each core
can generate many parallel memory requests at a time. The processing of
these parallel requests in the DRAM controller greatly affects the memory
interference delay experienced by running tasks on the platform.
In this paper, we present a new parallelism-aware worst-case memory
interference delay analysis for COTS multicore systems. The analysis
considers a COTS processor that can generate multiple outstanding
requests and a COTS DRAM controller that has a separate read and write
request buffer, prioritizes reads over writes, and supports
out-of-order request processing. Focusing on LLC and DRAM bank
partitioned systems, our analysis computes worst-case upper bounds on
memory-interference delays, caused by competing memory requests.
We validate our analysis on a Gem5 full-system simulator modeling a
realistic COTS multicore platform, with a set of carefully designed
synthetic benchmarks as well as SPEC2006 benchmarks.
The evaluation results show that our analysis produces safe upper
bounds in all tested benchmarks, while the current state-of-the-art analysis significantly
under-estimates the delays.
Taming Non-blocking Caches to Improve Isolation in Multicore Real-Time SystemsHeechul Yun
In this paper, we show that cache partitioning does
not necessarily ensure predictable cache performance in modern
COTS multicore platforms that use non-blocking caches to exploit
memory-level-parallelism (MLP).
Through carefully designed experiments using three real COTS
multicore platforms (four distinct CPU architectures) and a cycleaccurate
full system simulator, we show that special hardware
registers in non-blocking caches, known as Miss Status Holding
Registers (MSHRs), which track the status of outstanding cachemisses,
can be a significant source of contention; we observe up
to 21X WCET increase in a real COTS multicore platform due
to MSHR contention.
We propose a hardware and system software (OS) collaborative
approach to efficiently eliminate MSHR contention for
multicore real-time systems. Our approach includes a low-cost
hardware extension that enables dynamic control of per-core
MLP by the OS. Using the hardware extension, the OS scheduler
then globally controls each core’s MLP in such a way that
eliminates MSHR contention and maximizes overall throughput
of the system.
We implement the hardware extension in a cycle-accurate fullsystem
simulator and the scheduler modification in Linux 3.14
kernel. We evaluate the effectiveness of our approach using a set
of synthetic and macro benchmarks. In a case study, we achieve
up to 19% WCET reduction (average: 13%) for a set of EEMBC
benchmarks compared to a baseline cache partitioning setup.
The document provides an overview of the cgroup subsystem and namespace subsystem in Linux, which form the basis of Linux containers. It discusses how cgroups and namespaces enable lightweight virtualization of processes through isolation of resources and namespaces. It then covers specific aspects of cgroups like the memory, CPU, devices, and PIDs controllers. It also summarizes the key differences and improvements in the cgroup v2 implementation, such as having a single unified hierarchy and consistent controller interfaces.
This document discusses scheduling for multicore processors. It begins by explaining that multicore processors pack multiple CPU cores onto a single chip to increase processing speed. However, traditional C programs only use one CPU, so simply adding more CPUs does not speed programs up. The document then covers several challenges in multicore scheduling, such as cache coherence and affinity. It proposes some solutions like multi-queue scheduling, where each CPU has its own job queue, to help address issues like lack of scalability from single-queue approaches. Common Linux schedulers like the O(1) scheduler and Completely Fair Scheduler that use multiple queues are also mentioned.
The document discusses non-uniform cache architectures (NUCA), cache coherence, and different implementations of directories in multicore systems. It describes NUCA designs that map data to banks based on distance from the controller to exploit non-uniform access times. Cache coherence is maintained using directory-based protocols that track copies of cache blocks. Directories can be implemented off-chip in DRAM or on-chip using duplicate tag stores or distributing the directory among cache banks. Examples of systems like SGI Origin2000 and Tilera Tile64 that use these techniques are also outlined.
The document discusses cache coherence in multiprocessor systems. It describes the cache coherence problem that can arise when multiple processors have caches and can access shared memory. It then summarizes two primary hardware solutions: directory protocols which maintain information about which caches hold which memory lines; and snoopy cache protocols where cache controllers monitor bus traffic to maintain coherence without a directory. Finally it mentions a software-based solution relying on compiler analysis and operating system support.
The document discusses snooping cache coherence protocols. It covers key design issues like when memory gets updated, who responds to requests, and optimizations like adding states. It then describes several specific protocols: a 4-state protocol, the MESI protocol, an update protocol called Dragon, and issues around implementing these with split transaction buses. Key challenges addressed include non-atomic transitions, livelock, and handling multiple outstanding requests.
Agenda:
In this talk we will present various locking mechanisms implemented in the linux kernel.
From System V locks to raw spinlocks and the RT patch.
Speaker:
Mark Veltzer - CTO of Hinbit and a senior instructor at John Bryce. Mark is also a member of the Free Source Foundation and contributes to many free projects.
https://github.com/veltzer
Cache coherence is a technique used in multiprocessing systems to maintain consistency between caches and shared memory. When a processor modifies a variable in its cache that is also stored in another processor's cache, inconsistency arises. There are three main techniques to maintain cache coherence: snoopy-based protocols invalidate or update other caches when a write is observed; directory-based protocols use a directory to control access to shared data and update or invalidate caches; and snarfing-based protocols allow caches to watch addresses and data to update copies when writes are seen. Cache coherence aims to ensure data consistency across caches for shared resources.
Continguous Memory Allocator in the Linux KernelKernel TLV
Agenda:
Continguous Memory Allocator - how to allocate large continguous memory for large scale DMA in the kernel.
Speaker:
Mark Veltzer - CTO of Hinbit and a senior instructor at John Bryce. Mark is also a member of the Free Source Foundation and contributes to many free projects.
The document discusses various techniques for performance tuning a database system, including identifying and eliminating bottlenecks. It describes tuning at the hardware level by adding resources like disks and memory. Database parameters and design can also be tuned, such as schema changes, appropriate indices, and materialized views. Transactions play a role and can be tuned as well. The overall goal is to optimize performance by reducing bottlenecks.
HP-UX 11i LVM Mirroring Features and Multi-threads by Dusan BaljevicCircling Cycle
The document discusses methods used in Logical Volume Manager (LVM) to maintain consistency between mirrored logical volumes. It describes two key methods - the Mirror Write Cache (MWC) and Mirror Consistency Record (MCR). The MWC resides in memory and tracks pending writes, while the MCR on disk is used for recovery after crashes. The document provides details on how these methods work and how they can impact performance and recovery times in different situations.
El término "ademán" proviene del árabe y se refiere a las actitudes corporales que denotan estados emocionales o intenciones de acción. Los ademanes constituyen una forma de comunicación no verbal que puede hacer que un mensaje sea más eficaz o ineficaz. Los ademanes no siempre reemplazan el lenguaje hablado, sino que más bien lo complementan y son el único medio de comunicación para los sordomudos.
Los ademanes son movimientos del cuerpo que forman parte integral de la comunicación no verbal y pueden reforzar o contradecir el mensaje verbal. Los ademanes acompañan al lenguaje hablado a menos que se trate del lenguaje de señas utilizado por los sordomudos. Son importantes porque les dan énfasis visual y emocional a las palabras del orador y permiten transmitir ideas y sentimientos de manera más efectiva.
Este documento presenta conceptos clave relacionados con la investigación como ciencia, teoría, técnica, tecnología e investigación científica. También describe las etapas del proceso de investigación incluyendo la concepción de la idea, descripción del problema, marco de referencia, formulación de hipótesis, método, elaboración de documentos y reportes. Finalmente, discute la relación entre investigación, docencia y universidad.
This document summarizes two shared memory architectures - bus-based and directory-based. It describes:
1) Bus-based architectures have CPUs, caches and shared memory connected by a shared bus. The bus bandwidth limits scalability. It discusses the memory coherence problem and snooping protocols like MESI to address it.
2) Directory-based architectures avoid broadcast snooping and scale better using point-to-point messaging. Each block tracks its presence in caches using a directory with processor bits. It brings coherence through directory lookups and targeted invalidations.
This document discusses Google's use of CRIU for task migration at scale. It provides background on Borg, Google's cluster management system, and how tasks run in isolated containers. CRIU is used to checkpoint and restore task state, allowing tasks to be migrated transparently to avoid evictions. While migrations currently take 1-2 minutes, work is ongoing to improve performance and implement live migration to support latency-sensitive tasks. Security around CRIU's use of privileges is also an area of focus. Overall, CRIU has worked well but continued collaboration is needed to address remaining challenges.
The document provides an overview of Logical Volume Management (LVM) in Linux. It discusses what LVM is, its main components like physical volumes, volume groups, logical volumes, and how they relate. It then gives steps to use LVM by creating a physical volume, volume group and logical volume. It also discusses how LVM allows expanding logical volumes and live resizing of file systems.
Control groups (cgroups) allow administrators to allocate CPU, memory, storage, and other system resources to groups of processes running on the system. The document describes testing done using cgroups on a Red Hat Enterprise Linux 6 system with four Oracle database instances running an OLTP workload. It demonstrates how cgroups can be used for application consolidation, performance optimization, dynamic resource management, and application isolation.
This document discusses cache coherence in single and multiprocessor systems. It provides techniques to avoid inconsistencies between cache and main memory including write-through, write-back, and instruction caching. For multiprocessors, it discusses issues with sharing writable data, process migration, and I/O activity. Software solutions involve compiler and OS management while hardware uses coherence protocols like snoopy and directory protocols.
Coherence and consistency models in multiprocessor architectureUniversity of Pisa
Cache coherence and consistency model in multiprocessor architecture. These slide show the introduction of multiprocessor and cache multilevel and then describe the basic mechanism of coherence and consistency protocols. In particular the protocols describe are the following: snooping and directory protocols for the coherence part and sequential protocol for the consistency part. There are also example of (in)consistency and (in)coherence.
Parallelism-Aware Memory Interference Delay Analysis for COTS Multicore SystemsHeechul Yun
In modern Commercial Off-The-Shelf (COTS) multicore systems, each core
can generate many parallel memory requests at a time. The processing of
these parallel requests in the DRAM controller greatly affects the memory
interference delay experienced by running tasks on the platform.
In this paper, we present a new parallelism-aware worst-case memory
interference delay analysis for COTS multicore systems. The analysis
considers a COTS processor that can generate multiple outstanding
requests and a COTS DRAM controller that has a separate read and write
request buffer, prioritizes reads over writes, and supports
out-of-order request processing. Focusing on LLC and DRAM bank
partitioned systems, our analysis computes worst-case upper bounds on
memory-interference delays, caused by competing memory requests.
We validate our analysis on a Gem5 full-system simulator modeling a
realistic COTS multicore platform, with a set of carefully designed
synthetic benchmarks as well as SPEC2006 benchmarks.
The evaluation results show that our analysis produces safe upper
bounds in all tested benchmarks, while the current state-of-the-art analysis significantly
under-estimates the delays.
Taming Non-blocking Caches to Improve Isolation in Multicore Real-Time SystemsHeechul Yun
In this paper, we show that cache partitioning does
not necessarily ensure predictable cache performance in modern
COTS multicore platforms that use non-blocking caches to exploit
memory-level-parallelism (MLP).
Through carefully designed experiments using three real COTS
multicore platforms (four distinct CPU architectures) and a cycleaccurate
full system simulator, we show that special hardware
registers in non-blocking caches, known as Miss Status Holding
Registers (MSHRs), which track the status of outstanding cachemisses,
can be a significant source of contention; we observe up
to 21X WCET increase in a real COTS multicore platform due
to MSHR contention.
We propose a hardware and system software (OS) collaborative
approach to efficiently eliminate MSHR contention for
multicore real-time systems. Our approach includes a low-cost
hardware extension that enables dynamic control of per-core
MLP by the OS. Using the hardware extension, the OS scheduler
then globally controls each core’s MLP in such a way that
eliminates MSHR contention and maximizes overall throughput
of the system.
We implement the hardware extension in a cycle-accurate fullsystem
simulator and the scheduler modification in Linux 3.14
kernel. We evaluate the effectiveness of our approach using a set
of synthetic and macro benchmarks. In a case study, we achieve
up to 19% WCET reduction (average: 13%) for a set of EEMBC
benchmarks compared to a baseline cache partitioning setup.
The document provides an overview of the cgroup subsystem and namespace subsystem in Linux, which form the basis of Linux containers. It discusses how cgroups and namespaces enable lightweight virtualization of processes through isolation of resources and namespaces. It then covers specific aspects of cgroups like the memory, CPU, devices, and PIDs controllers. It also summarizes the key differences and improvements in the cgroup v2 implementation, such as having a single unified hierarchy and consistent controller interfaces.
This document discusses scheduling for multicore processors. It begins by explaining that multicore processors pack multiple CPU cores onto a single chip to increase processing speed. However, traditional C programs only use one CPU, so simply adding more CPUs does not speed programs up. The document then covers several challenges in multicore scheduling, such as cache coherence and affinity. It proposes some solutions like multi-queue scheduling, where each CPU has its own job queue, to help address issues like lack of scalability from single-queue approaches. Common Linux schedulers like the O(1) scheduler and Completely Fair Scheduler that use multiple queues are also mentioned.
The document discusses non-uniform cache architectures (NUCA), cache coherence, and different implementations of directories in multicore systems. It describes NUCA designs that map data to banks based on distance from the controller to exploit non-uniform access times. Cache coherence is maintained using directory-based protocols that track copies of cache blocks. Directories can be implemented off-chip in DRAM or on-chip using duplicate tag stores or distributing the directory among cache banks. Examples of systems like SGI Origin2000 and Tilera Tile64 that use these techniques are also outlined.
The document discusses cache coherence in multiprocessor systems. It describes the cache coherence problem that can arise when multiple processors have caches and can access shared memory. It then summarizes two primary hardware solutions: directory protocols which maintain information about which caches hold which memory lines; and snoopy cache protocols where cache controllers monitor bus traffic to maintain coherence without a directory. Finally it mentions a software-based solution relying on compiler analysis and operating system support.
The document discusses snooping cache coherence protocols. It covers key design issues like when memory gets updated, who responds to requests, and optimizations like adding states. It then describes several specific protocols: a 4-state protocol, the MESI protocol, an update protocol called Dragon, and issues around implementing these with split transaction buses. Key challenges addressed include non-atomic transitions, livelock, and handling multiple outstanding requests.
Agenda:
In this talk we will present various locking mechanisms implemented in the linux kernel.
From System V locks to raw spinlocks and the RT patch.
Speaker:
Mark Veltzer - CTO of Hinbit and a senior instructor at John Bryce. Mark is also a member of the Free Source Foundation and contributes to many free projects.
https://github.com/veltzer
Cache coherence is a technique used in multiprocessing systems to maintain consistency between caches and shared memory. When a processor modifies a variable in its cache that is also stored in another processor's cache, inconsistency arises. There are three main techniques to maintain cache coherence: snoopy-based protocols invalidate or update other caches when a write is observed; directory-based protocols use a directory to control access to shared data and update or invalidate caches; and snarfing-based protocols allow caches to watch addresses and data to update copies when writes are seen. Cache coherence aims to ensure data consistency across caches for shared resources.
Continguous Memory Allocator in the Linux KernelKernel TLV
Agenda:
Continguous Memory Allocator - how to allocate large continguous memory for large scale DMA in the kernel.
Speaker:
Mark Veltzer - CTO of Hinbit and a senior instructor at John Bryce. Mark is also a member of the Free Source Foundation and contributes to many free projects.
The document discusses various techniques for performance tuning a database system, including identifying and eliminating bottlenecks. It describes tuning at the hardware level by adding resources like disks and memory. Database parameters and design can also be tuned, such as schema changes, appropriate indices, and materialized views. Transactions play a role and can be tuned as well. The overall goal is to optimize performance by reducing bottlenecks.
HP-UX 11i LVM Mirroring Features and Multi-threads by Dusan BaljevicCircling Cycle
The document discusses methods used in Logical Volume Manager (LVM) to maintain consistency between mirrored logical volumes. It describes two key methods - the Mirror Write Cache (MWC) and Mirror Consistency Record (MCR). The MWC resides in memory and tracks pending writes, while the MCR on disk is used for recovery after crashes. The document provides details on how these methods work and how they can impact performance and recovery times in different situations.
El término "ademán" proviene del árabe y se refiere a las actitudes corporales que denotan estados emocionales o intenciones de acción. Los ademanes constituyen una forma de comunicación no verbal que puede hacer que un mensaje sea más eficaz o ineficaz. Los ademanes no siempre reemplazan el lenguaje hablado, sino que más bien lo complementan y son el único medio de comunicación para los sordomudos.
Los ademanes son movimientos del cuerpo que forman parte integral de la comunicación no verbal y pueden reforzar o contradecir el mensaje verbal. Los ademanes acompañan al lenguaje hablado a menos que se trate del lenguaje de señas utilizado por los sordomudos. Son importantes porque les dan énfasis visual y emocional a las palabras del orador y permiten transmitir ideas y sentimientos de manera más efectiva.
Este documento presenta conceptos clave relacionados con la investigación como ciencia, teoría, técnica, tecnología e investigación científica. También describe las etapas del proceso de investigación incluyendo la concepción de la idea, descripción del problema, marco de referencia, formulación de hipótesis, método, elaboración de documentos y reportes. Finalmente, discute la relación entre investigación, docencia y universidad.
El documento habla sobre la oratoria. Explica que la oratoria es el arte de hablar en público de manera elocuente para informar, convencer o persuadir a una audiencia. Señala que la oratoria se desarrolló en la antigua Grecia y que figuras como Sócrates, Demóstenes y Cicerón fueron grandes oradores. El documento luego proporciona consejos para hablar en público de manera efectiva y ganar confianza, así como detalles sobre un concurso de oratoria.
Las 9 reglas de la ORatoria Curso de locucion UCVErika Osteriz
Todos los seres humanos debemos aprender a hablar de forma correcta en todo momento. Existen muchos tips y recomendaciones para lograr la atención de las personas que nos escuchan. Sin embargo, existen 9 reglas básicas de la oratoria que se muestran de forma sencilla en esta presentación.
This document discusses various techniques for improving cache performance, including reducing the miss rate and miss penalty. It describes reducing misses through larger block sizes, higher associativity, victim caches, and prefetching. It also covers reducing miss penalties via read priority on misses, non-blocking caches, and adding a second level cache. The goal is to improve CPU performance by lowering the miss rate, miss penalty, and time to hit in the cache.
Strata+Hadoop 2017 San Jose: Lessons from a year of supporting Apache Kafkaconfluent
The number of deployments of Apache Kafka at enterprise scale has greatly increased in the years since Kafka’s original development in 2010. Along with this rapid growth has come a wide variety of use cases and deployment strategies that transcend what Kafka’s creators imagined when they originally developed the technology. As the scope and reach of streaming data platforms based on Apache Kafka has grown, the need to understand monitoring and troubleshooting strategies has as well.
Dustin Cote and Ryan Pridgeon share their experience supporting Apache Kafka at enterprise-scale and explore monitoring and troubleshooting techniques to help you avoid pitfalls when scaling large-scale Kafka deployments.
Topics include:
- Effective use of JMX for Kafka
- Tools for preventing small problems from becoming big ones
- Efficient architectures proven in the wild
- Finding and storing the right information when it all goes wrong
Visit www.confluent.io for more information.
The document summarizes a talk on container performance analysis. It discusses identifying bottlenecks at the host, container, and kernel level using various Linux performance tools. It then provides an overview of how containers work in Linux using namespaces and control groups (cgroups). Finally, it demonstrates some example commands like docker stats, systemd-cgtop, and bcc/BPF tools that can be used to analyze containers and cgroups from the host system.
Container Performance Analysis Brendan Gregg, NetflixDocker, Inc.
The document summarizes a talk on container performance analysis. It discusses identifying bottlenecks at the host, container, and kernel level using various Linux performance tools. It also provides an overview of how containers work in Linux using namespaces and control groups (cgroups). Specifically, it demonstrates analyzing resource usage and limitations for containers using tools like docker stats, systemd-cgtop, and investigating namespaces.
The document discusses understanding Linux system hangs. It provides an overview of softlockups and hardlockups, how they are detected by the kernel's watchdog mechanism, and examples of improper kernel programming that can cause hangs. It then presents a case study of a cluster node hang when leaving a lockspace group in MDADM. The steps taken to reproduce the hang, capture a kernel dump, and analyze the dump are described. Finally, potential questions for further discussion are listed.
This document discusses memory order consume semantics in C++11. It begins with a recap of acquire and release semantics, then explains that consume semantics are designed to exploit data dependency ordering on weakly-ordered CPUs. However, current compilers like GCC and Clang do not efficiently implement consume, instead using memory barriers. The document ends by discussing proposals to narrow the definition of consume dependencies to guide compiler implementations.
These slides show how to reduce latency on websites and reduce bandwidth for improved user experience.
Covering network, compression, caching, etags, application optimisation, sphinxsearch, memcache, db optimisation
This document summarizes discussions from the Linaro Connect Q4-11 Power Management Working Group about scheduling domains and load balancing on multicore ARM processors. It describes using sched_mc domains to balance load at the multicore level instead of the CPU level. It also discusses using CPU power scaling to emulate different topologies, like packing tasks onto fewer cores at low load to save power on quad-core CPUs or pulling tasks to a single core on a dual-core system. Open issues around idle load balancing and spurious wakeups with these approaches are also outlined. Future work on load balancing for asymmetric big.LITTLE systems is mentioned.
Series of Unfortunate Netflix Container Events - QConNYC17aspyker
Project Titus is Netflix's container runtime on top of Amazon EC2. Titus powers algorithm research through massively parallel model training, media encoding, data research notebooks, ad hoc reporting, NodeJS UI services, stream processing and general micro-services. As an update from last year's talk, we will focus on the lessons learned operating one of the largest container runtimes on a public cloud. We'll cover the migration we've seen of applications and frameworks from VM's to containers. We will cover the operational issues with containers that only showed after we reached the large scale (1000's of container hosts, 100's of thousands of containers launched weekly) we are currently supporting. We'll touch base on the unique features we have added to help both batch and microservices run across a variety of runtimes (Java, R, NodeJS, Python, etc) and how higher level frameworks have taken advantage of Titus's scheduling capabilities.
This document provides an introduction to multiprocessor systems and discusses different multiprocessor architectures including shared memory, distributed memory, and distributed shared memory systems. It describes the key differences between Uniform Memory Access (UMA) and Non-Uniform Memory Access (NUMA) models. Cache coherence problems that can arise in shared memory systems are discussed along with solutions like snooping and directory-based cache coherence protocols.
Supporting Time-Sensitive Applications on a Commodity OSNamHyuk Ahn
1) The document discusses Time-Sensitive Linux (TSL), which aims to support time-sensitive applications on commodity operating systems like Linux.
2) TSL improves kernel latency through an accurate timer mechanism called firm timers, a responsive kernel using lock-breaking preemption, and effective scheduling using proportion-based and priority-based algorithms.
3) Evaluation shows TSL reduces timer latency to under 1ms and preemption latency to under 1ms, improving synchronization of media playback under load compared to standard Linux.
MemGuard: Memory Bandwidth Reservation System for Efficient Performance Isola...Heechul Yun
This document describes MemGuard, an operating system mechanism for providing efficient per-core memory performance isolation on commercial off-the-shelf hardware. MemGuard uses memory bandwidth reservation to guarantee each core's minimum memory bandwidth. It then performs predictive bandwidth donation and on-demand reclaiming to redistribute excess bandwidth, improving overall utilization. Evaluation shows MemGuard isolates performance and eliminates over 50% slowdown of a foreground real-time task due to interference, while maximizing throughput via bandwidth sharing.
Slides presented at Percona Live Europe Open Source Database Conference 2019, Amsterdam, 2019-10-01.
Imagine a world where all Wikipedia articles disappear due to a human error or software bug. Sounds unreal? According to some estimations, it would take an excess of hundreds of million person-hours to be written again. To prevent that scenario from ever happening, our SRE team at Wikimedia recently refactored the relational database recovery system.
In this session, we will discuss how we backup 550TB of MariaDB data without impacting the 15 billion page views per month we get. We will cover what were our initial plans to replace the old infrastructure, how we achieved recovering 2TB databases in less than 30 minutes while maintaining per-table granularity, as well as the different types of backups we implemented. Lastly, we will talk about lessons learned, what went well, how our original plans changed and future work.
This document discusses improvements made to LXC container support in Ganeti through a Google Summer of Code project. Key areas worked on included fixing existing LXC integration, adding unit tests, setting up quality assurance, and adding features like migration. The project was mentored by Hrvoje Ribicic. Future work may include two-level isolation of containers within VMs and live migration of LXC instances.
Kvm performance optimization for ubuntuSim Janghoon
This document discusses various techniques for optimizing KVM performance on Linux systems. It covers CPU and memory optimization through techniques like vCPU pinning, NUMA affinity, transparent huge pages, KSM, and virtio_balloon. For networking, it discusses vhost-net, interrupt handling using MSI/MSI-X, and NAPI. It also covers block device optimization through I/O scheduling, cache mode, and asynchronous I/O. The goal is to provide guidance on configuring these techniques for workloads running in KVM virtual machines.
cache2k is one of the best performing Java in memory cache solution available today. The talk details about the internal design and latest research on the used low overhead eviction algorithm. At the end benchmarks are presented comparing the speed of different cache solutions including EHCache, Infinispan and Google Guava.
Slides from a talk given to coursemates about my university final year project on the UWE CRTS course which involved porting uCLinux to the Pluto 6 gaming control board.
UKUUG presentation about µCLinux on Pluto 6edlangley
Slides from a <a>talk</a> given at the UKUUG 2006 conference derived from my final year project on the UWE CRTS degree which involved porting uCLinux to the Pluto 6 gaming control board.
This document discusses using PFCache to improve storage density for container files by deduplicating identical data. PFCache uses a cache area to store deduplicated file contents, referenced via cache links in container image files. Evaluation showed PFCache improved storage density. Future work includes upstreaming PLoop for containers and pursuing IO deduplication in the Linux kernel for additional benefits.
This document proposes a new interface called task_diag to provide process information in a more efficient way compared to the current /proc interface. Task_diag would use a netlink message format to allow querying process attributes in groups and splitting the response across multiple messages. Performance tests show task_diag can provide process information 2-8 times faster than parsing /proc. The goal is to speed up tools like ps and top.
Live migration: pros, cons and gotchas -- Pavel EmelyanovOpenVZ
Live migrating containers has pros like load balancing and updating hardware, but also cons. It is complex due to needing to save a container's state, transfer it, and restore it on another host while tasks are frozen. This can be done with memory pre-copy, where memory is copied iteratively while tasks run, or post-copy, where memory is transferred after tasks resume. P.Haul uses CRIU to handle the critical state save and restore functions needed for live migrating containers between hosts.
Live migrating a container: pros, cons and gotchas -- Pavel EmelyanovOpenVZ
Live migrating a container: pros, cons and gotchas
Monday, November 16 • 17:20 - 18:05
Pavel Emelyanov
Principal Engineer, Odin
Principal engineer at Odin Server Virtualization team, creator and maintainer of the CRIU project. Joined Parallels in 2004 as junior Linux kernel developer, later became kernel team leader. Now works on architecture of the Odin Server products. | | Pavel tweets at @xemulp.
http://dockerconeu2015.sched.org/event/62e6d2ea7380442a48fafaeee26c9842
В своей презентации мы на примере дистрибутива Linux расскажем об опыте организации процесса тестирования продукта, существенная часть (более 90%) кода которого создается независимыми от компании разработчиками.
https://www.youtube.com/watch?v=AstgrnE7_dI
LibCT: one lib to rule them all -- Andrey VaginOpenVZ
LibCT is a C library that allows building containerized applications by configuring namespaces and cgroups to provide isolation. It aims to simplify the complex low-level container APIs and support different container types like Linux containers, OpenVZ, Solaris Zones, and BSD jails. LibCT hides low-level API changes and provides bindings for other languages. It also serves as an alternative to Libcontainer which is written in Go. The presentation covered the history of Linux containers, namespaces, cgroups, LibCT's API, examples of use, and future integration plans.
Denser containers with PF cache - Pavel EmelyanovOpenVZ
This document discusses using PFCache to improve storage density for container files by deduplicating identical data. PFCache uses a cache area to store deduplicated file contents, referenced via cache links in container image files. Evaluation showed PFCache improved storage density. Future work includes upstreaming PLoop for containers and pursuing IO deduplication in the Linux kernel for additional benefits.
What's missing from upstream kernel containers? - Kir Kolyshkin, Sergey Bronn...OpenVZ
This document discusses containers in the upstream Linux kernel compared to the Virtuozzo (VZ) kernel. It notes that OpenVZ has contributed many features to the Linux kernel over time, with about 60% of the VZ kernel now upstreamed. While much progress has been made, there are still areas needing to be added to the upstream kernel to fully replace the VZ kernel, including memory management and accounting features, network and I/O optimizations, and legacy containerization tools. The presenters from OpenVZ are available to discuss any questions.
Not so brief history of Linux Containers - Kir KolyshkinOpenVZ
Linux containers have evolved from initial ideas in 1999 to widespread use today. Early container technologies included Virtuozzo in 1999-2000 and the introduction of namespaces in 2000-2001. The Linux-VServer project in 2001 helped advance containers. Checkpointing and live migration were implemented in 2002-2003. OpenVZ in 2005 helped popularize containers. Developments from 2006-2010 included new namespaces and use of cgroups. Docker in 2013 and projects like CoreOS and LXC further advanced containers. CRIU, introduced in 2011, helped enable checkpoint/restore in userspace. Containers are now widely used in projects like OpenStack.
This document compares virtual machines (VMs) to containers and discusses their differences. It notes that VMs have higher overhead than containers and can support more instances per server. Containers offer near-native performance and scale better. The document also outlines ongoing and future work to further integrate container technologies into the Linux kernel to provide capabilities like checkpoint/restore and live migration of containers across servers.
E-commerce Development Services- Hornet DynamicsHornet Dynamics
For any business hoping to succeed in the digital age, having a strong online presence is crucial. We offer Ecommerce Development Services that are customized according to your business requirements and client preferences, enabling you to create a dynamic, safe, and user-friendly online store.
Introducing Crescat - Event Management Software for Venues, Festivals and Eve...Crescat
Crescat is industry-trusted event management software, built by event professionals for event professionals. Founded in 2017, we have three key products tailored for the live event industry.
Crescat Event for concert promoters and event agencies. Crescat Venue for music venues, conference centers, wedding venues, concert halls and more. And Crescat Festival for festivals, conferences and complex events.
With a wide range of popular features such as event scheduling, shift management, volunteer and crew coordination, artist booking and much more, Crescat is designed for customisation and ease-of-use.
Over 125,000 events have been planned in Crescat and with hundreds of customers of all shapes and sizes, from boutique event agencies through to international concert promoters, Crescat is rigged for success. What's more, we highly value feedback from our users and we are constantly improving our software with updates, new features and improvements.
If you plan events, run a venue or produce festivals and you're looking for ways to make your life easier, then we have a solution for you. Try our software for free or schedule a no-obligation demo with one of our product specialists today at crescat.io
DDS Security Version 1.2 was adopted in 2024. This revision strengthens support for long runnings systems adding new cryptographic algorithms, certificate revocation, and hardness against DoS attacks.
Hand Rolled Applicative User ValidationCode KataPhilip Schwarz
Could you use a simple piece of Scala validation code (granted, a very simplistic one too!) that you can rewrite, now and again, to refresh your basic understanding of Applicative operators <*>, <*, *>?
The goal is not to write perfect code showcasing validation, but rather, to provide a small, rough-and ready exercise to reinforce your muscle-memory.
Despite its grandiose-sounding title, this deck consists of just three slides showing the Scala 3 code to be rewritten whenever the details of the operators begin to fade away.
The code is my rough and ready translation of a Haskell user-validation program found in a book called Finding Success (and Failure) in Haskell - Fall in love with applicative functors.
E-commerce Application Development Company.pdfHornet Dynamics
Your business can reach new heights with our assistance as we design solutions that are specifically appropriate for your goals and vision. Our eCommerce application solutions can digitally coordinate all retail operations processes to meet the demands of the marketplace while maintaining business continuity.
OpenMetadata Community Meeting - 5th June 2024OpenMetadata
The OpenMetadata Community Meeting was held on June 5th, 2024. In this meeting, we discussed about the data quality capabilities that are integrated with the Incident Manager, providing a complete solution to handle your data observability needs. Watch the end-to-end demo of the data quality features.
* How to run your own data quality framework
* What is the performance impact of running data quality frameworks
* How to run the test cases in your own ETL pipelines
* How the Incident Manager is integrated
* Get notified with alerts when test cases fail
Watch the meeting recording here - https://www.youtube.com/watch?v=UbNOje0kf6E
AI Fusion Buddy Review: Brand New, Groundbreaking Gemini-Powered AI AppGoogle
AI Fusion Buddy Review: Brand New, Groundbreaking Gemini-Powered AI App
👉👉 Click Here To Get More Info 👇👇
https://sumonreview.com/ai-fusion-buddy-review
AI Fusion Buddy Review: Key Features
✅Create Stunning AI App Suite Fully Powered By Google's Latest AI technology, Gemini
✅Use Gemini to Build high-converting Converting Sales Video Scripts, ad copies, Trending Articles, blogs, etc.100% unique!
✅Create Ultra-HD graphics with a single keyword or phrase that commands 10x eyeballs!
✅Fully automated AI articles bulk generation!
✅Auto-post or schedule stunning AI content across all your accounts at once—WordPress, Facebook, LinkedIn, Blogger, and more.
✅With one keyword or URL, generate complete websites, landing pages, and more…
✅Automatically create & sell AI content, graphics, websites, landing pages, & all that gets you paid non-stop 24*7.
✅Pre-built High-Converting 100+ website Templates and 2000+ graphic templates logos, banners, and thumbnail images in Trending Niches.
✅Say goodbye to wasting time logging into multiple Chat GPT & AI Apps once & for all!
✅Save over $5000 per year and kick out dependency on third parties completely!
✅Brand New App: Not available anywhere else!
✅ Beginner-friendly!
✅ZERO upfront cost or any extra expenses
✅Risk-Free: 30-Day Money-Back Guarantee!
✅Commercial License included!
See My Other Reviews Article:
(1) AI Genie Review: https://sumonreview.com/ai-genie-review
(2) SocioWave Review: https://sumonreview.com/sociowave-review
(3) AI Partner & Profit Review: https://sumonreview.com/ai-partner-profit-review
(4) AI Ebook Suite Review: https://sumonreview.com/ai-ebook-suite-review
#AIFusionBuddyReview,
#AIFusionBuddyFeatures,
#AIFusionBuddyPricing,
#AIFusionBuddyProsandCons,
#AIFusionBuddyTutorial,
#AIFusionBuddyUserExperience
#AIFusionBuddyforBeginners,
#AIFusionBuddyBenefits,
#AIFusionBuddyComparison,
#AIFusionBuddyInstallation,
#AIFusionBuddyRefundPolicy,
#AIFusionBuddyDemo,
#AIFusionBuddyMaintenanceFees,
#AIFusionBuddyNewbieFriendly,
#WhatIsAIFusionBuddy?,
#HowDoesAIFusionBuddyWorks
Takashi Kobayashi and Hironori Washizaki, "SWEBOK Guide and Future of SE Education," First International Symposium on the Future of Software Engineering (FUSE), June 3-6, 2024, Okinawa, Japan
Do you want Software for your Business? Visit Deuglo
Deuglo has top Software Developers in India. They are experts in software development and help design and create custom Software solutions.
Deuglo follows seven steps methods for delivering their services to their customers. They called it the Software development life cycle process (SDLC).
Requirement — Collecting the Requirements is the first Phase in the SSLC process.
Feasibility Study — after completing the requirement process they move to the design phase.
Design — in this phase, they start designing the software.
Coding — when designing is completed, the developers start coding for the software.
Testing — in this phase when the coding of the software is done the testing team will start testing.
Installation — after completion of testing, the application opens to the live server and launches!
Maintenance — after completing the software development, customers start using the software.
Neo4j - Product Vision and Knowledge Graphs - GraphSummit ParisNeo4j
Dr. Jesús Barrasa, Head of Solutions Architecture for EMEA, Neo4j
Découvrez les dernières innovations de Neo4j, et notamment les dernières intégrations cloud et les améliorations produits qui font de Neo4j un choix essentiel pour les développeurs qui créent des applications avec des données interconnectées et de l’IA générative.
Transform Your Communication with Cloud-Based IVR SolutionsTheSMSPoint
Discover the power of Cloud-Based IVR Solutions to streamline communication processes. Embrace scalability and cost-efficiency while enhancing customer experiences with features like automated call routing and voice recognition. Accessible from anywhere, these solutions integrate seamlessly with existing systems, providing real-time analytics for continuous improvement. Revolutionize your communication strategy today with Cloud-Based IVR Solutions. Learn more at: https://thesmspoint.com/channel/cloud-telephony
Essentials of Automations: The Art of Triggers and Actions in FMESafe Software
In this second installment of our Essentials of Automations webinar series, we’ll explore the landscape of triggers and actions, guiding you through the nuances of authoring and adapting workspaces for seamless automations. Gain an understanding of the full spectrum of triggers and actions available in FME, empowering you to enhance your workspaces for efficient automation.
We’ll kick things off by showcasing the most commonly used event-based triggers, introducing you to various automation workflows like manual triggers, schedules, directory watchers, and more. Plus, see how these elements play out in real scenarios.
Whether you’re tweaking your current setup or building from the ground up, this session will arm you with the tools and insights needed to transform your FME usage into a powerhouse of productivity. Join us to discover effective strategies that simplify complex processes, enhancing your productivity and transforming your data management practices with FME. Let’s turn complexity into clarity and make your workspaces work wonders!
Top Features to Include in Your Winzo Clone App for Business Growth (4).pptxrickgrimesss22
Discover the essential features to incorporate in your Winzo clone app to boost business growth, enhance user engagement, and drive revenue. Learn how to create a compelling gaming experience that stands out in the competitive market.
Zoom is a comprehensive platform designed to connect individuals and teams efficiently. With its user-friendly interface and powerful features, Zoom has become a go-to solution for virtual communication and collaboration. It offers a range of tools, including virtual meetings, team chat, VoIP phone systems, online whiteboards, and AI companions, to streamline workflows and enhance productivity.
1. List of topics
1. Control loops in userspace
2. Hierarchical Cgroup support in userland
3. Efficient utilization by overcommitment
4. dcache management
5. Memory controller improvements
6. Checkpoint/restart in the userspace
7. Container in a file
8. Proc fs virtualization
9. Entering a container
10.vzctl for LXC
11.Extending CPU accounting stats
2. dcache mgmt 1/2: problems
● Problem 1
● the number of dentries grows uncontrollable
● Entries can be pinned with low effort, leading to all
memory being used and unreclaimable
● Leading to system being unresponsive/stuck
● Problem 2
● Unfair dentry cache usage distribution among
containers
3. dcache mgmt 2/2: Solution
● Account for and limit dcache usage per (smth)
● Currently (smth) is superblock
● Shrink dcache per (smth) once limit it hit
4. checkpoint-restore 1/3
● Checkpoint:
● freeze task(s) with freezer cgroup
● read tasks creds via /proc
– some patches extending kernel API required
● read tasks regs with PTRACE
● dump tasks mem with PTRASE_SEIZE (new)
5. checkpoint-restore 2/3
● Restore:
● Fork task tree
– CLONE_SETPID required
● each task restores its creds
– uid/gid
– open files
– capabilities
– etc.
● each task calls execve() on an image
– ELF binfmt handler extension required
7. Container in a file 1/2
● Currently containers' files reside on host
partition and tasks work in plain chroot
● Problems:
● extX journal is performance bottleneck
● hackish live migration (rsync)
● not effective management (tons of small files)
8. Container in a file 2/2
● Solution – loop-like block device
● Parallels implementation – block device, that
remaps bio-s with FS bmap call
● Possible implementation – DM extension
● block map provided by userspace
● block map got from FS bmap call