Prometheus 2.0
•
1 like•1,045 views
Prometheus 2.0 provides significantly improved performance for highly dynamic environments like Kubernetes with a brand new storage engine. It offers around 3x reduction in CPU usage, 2x less disk space usage, and 100x reduction in I/O compared to previous versions. Additionally, Prometheus 2.0 handles target staleness differently by stopping returning values after a single scrape interval if a target goes down, rather than continuing for 5 minutes, making it simpler to understand.
Report
Share
Report
Share
Download to read offline
Recommended
Automatic Operation Bot for Ceph - You Ji
The document discusses an automatic operation bot for Ceph clusters at eBay. It describes monitoring the clusters with tools like Prometheus and node-exporter. When issues arise, over 95% are due to failed disks, which the bot would automatically remove and replace. It would record failures and perform remediation steps like removing OSDs, offline disks, and lighting indicator LEDs. The bot design includes components for monitoring, alerting, a task queue, and executor to automate common operations and reduce manual work.
OOPs, OOMs, oh my! Containerizing JVM apps
This document discusses best practices for containerizing Java applications to avoid out of memory errors and performance issues. It covers choosing appropriate Java versions, garbage collector tuning, sizing heap memory correctly while leaving room for operating system caches, avoiding swapping, and monitoring applications to detect issues. Key recommendations include using the newest Java version possible, configuring the garbage collector appropriately for the workload, allocating all heap memory at startup, and monitoring memory usage to detect problems early.
Memory, Big Data, NoSQL and Virtualization
In-memory processing has started to become the norm in large scale data handling. This is aclose to the metal analysis of highly important but often neglected aspects of memory accesstimes and how it impacts big data and NoSQL technologies.We cover aspects such as the TLB, the Transparent Huge Pages, the QPI Link, Hyperthreading and the impact of virtualization on high-memory footprint applications. We present benchmarks of various technologies ranging from Cloudera’s Impala to Couchbase and how they are impacted by the underlying hardware.The key takeaway is a better understanding of how to size a cluster, how to choose a cloud provider and an instance type for big data and NoSQL workloads and why not every core or GB of RAM is created equal.
Cassandra Performance Benchmark
The Economics of Scaling Cassandra - By Alex Bordei, Techie Product Manager at Bigstep
This presentation was made during the "Cassandra Summit 2014" Event, in London.
We benchmarked Cassandra on a number of configurations and we show what's the scaling profile. We test Cassandra on Docker as well as Cassandra's In-memory feature.
Follow Alex on Twitter: @alexandrubordei
Bigstep on Twitter: @BigStepInc
If you have any questions, let us know at hello@bigstep.com and we'll do our best to answer.
Stay informed: http://blog.bigstep.com/
Doing QoS Before Ceph Cluster QoS is available - David Byte, Alex Lau
Cephalocon APAC 2018
March 22-23, 2018 - Beijing, China
David Byte, SUSE Senior Technology Strategist
Alex Lau, SUSE Storage Consultant
Performance
Lightning talk showing various aspectos of software system performance. It goes through: latency, data structures, garbage collection, troubleshooting method like workload saturation method, quick diagnostic tools, famegraph and perfview
Docker at OpenDNS
Jessica Gadling is a Software Engineer at OpenDNS. She gave a talk and demo at OpenLate (http://www.meetup.com/OpenLate/) on October 21st, 2014 on why Docker was chosen as a central component in OpenDNS's internal PaaS Quadra.
Ruby on the server
This document discusses using Ruby on Rails web applications on servers. It recommends using Ruby Version Manager (RVM) to manage multiple Ruby versions and environments. Typical web applications use 100,000-200,000 memory slots total, with 2,000-10,000 objects per request. The falcon Ruby patch improves performance. Foreman can be used to launch Resque workers and schedulers, and export the configuration. Capistrano and restarting workers via signals allows for seamless code deployments.
Recommended
Automatic Operation Bot for Ceph - You Ji
The document discusses an automatic operation bot for Ceph clusters at eBay. It describes monitoring the clusters with tools like Prometheus and node-exporter. When issues arise, over 95% are due to failed disks, which the bot would automatically remove and replace. It would record failures and perform remediation steps like removing OSDs, offline disks, and lighting indicator LEDs. The bot design includes components for monitoring, alerting, a task queue, and executor to automate common operations and reduce manual work.
OOPs, OOMs, oh my! Containerizing JVM apps
This document discusses best practices for containerizing Java applications to avoid out of memory errors and performance issues. It covers choosing appropriate Java versions, garbage collector tuning, sizing heap memory correctly while leaving room for operating system caches, avoiding swapping, and monitoring applications to detect issues. Key recommendations include using the newest Java version possible, configuring the garbage collector appropriately for the workload, allocating all heap memory at startup, and monitoring memory usage to detect problems early.
Memory, Big Data, NoSQL and Virtualization
In-memory processing has started to become the norm in large scale data handling. This is aclose to the metal analysis of highly important but often neglected aspects of memory accesstimes and how it impacts big data and NoSQL technologies.We cover aspects such as the TLB, the Transparent Huge Pages, the QPI Link, Hyperthreading and the impact of virtualization on high-memory footprint applications. We present benchmarks of various technologies ranging from Cloudera’s Impala to Couchbase and how they are impacted by the underlying hardware.The key takeaway is a better understanding of how to size a cluster, how to choose a cloud provider and an instance type for big data and NoSQL workloads and why not every core or GB of RAM is created equal.
Cassandra Performance Benchmark
The Economics of Scaling Cassandra - By Alex Bordei, Techie Product Manager at Bigstep
This presentation was made during the "Cassandra Summit 2014" Event, in London.
We benchmarked Cassandra on a number of configurations and we show what's the scaling profile. We test Cassandra on Docker as well as Cassandra's In-memory feature.
Follow Alex on Twitter: @alexandrubordei
Bigstep on Twitter: @BigStepInc
If you have any questions, let us know at hello@bigstep.com and we'll do our best to answer.
Stay informed: http://blog.bigstep.com/
Doing QoS Before Ceph Cluster QoS is available - David Byte, Alex Lau
Cephalocon APAC 2018
March 22-23, 2018 - Beijing, China
David Byte, SUSE Senior Technology Strategist
Alex Lau, SUSE Storage Consultant
Performance
Lightning talk showing various aspectos of software system performance. It goes through: latency, data structures, garbage collection, troubleshooting method like workload saturation method, quick diagnostic tools, famegraph and perfview
Docker at OpenDNS
Jessica Gadling is a Software Engineer at OpenDNS. She gave a talk and demo at OpenLate (http://www.meetup.com/OpenLate/) on October 21st, 2014 on why Docker was chosen as a central component in OpenDNS's internal PaaS Quadra.
Ruby on the server
This document discusses using Ruby on Rails web applications on servers. It recommends using Ruby Version Manager (RVM) to manage multiple Ruby versions and environments. Typical web applications use 100,000-200,000 memory slots total, with 2,000-10,000 objects per request. The falcon Ruby patch improves performance. Foreman can be used to launch Resque workers and schedulers, and export the configuration. Capistrano and restarting workers via signals allows for seamless code deployments.
Scaling Apache Pulsar to 10 Petabytes/Day
This document discusses optimizing an Apache Pulsar cluster to handle 10 PB of data per day for a financial customer. Initial estimates showed the cluster would need over 1000 VMs using HDD storage. Various optimizations were implemented, including eliminating the journal, using direct I/O, compression, and C++ client optimizations. This reduced the estimated number of needed VMs to 200 using L-SSD storage per VM. The optimized cluster can now meet the customer's requirements of processing 10 PB of data per day with 3 hours of retention and zone failure protection.
Extreme HTTP Performance Tuning: 1.2M API req/s on a 4 vCPU EC2 Instance
In this talk I will walk you through the performance tuning steps that I took to serve 1.2M JSON requests per second from a 4 vCPU c5 instance, using a simple API server written in C.
At the start of the journey the server is capable of a very respectable 224k req/s with the default configuration. Along the way I made extensive use of tools like FlameGraph and bpftrace to measure, analyze, and optimize the entire stack, from the application framework, to the network driver, all the way down to the kernel.
I began this wild adventure without any prior low-level performance optimization experience; but once I started going down the performance tuning rabbit-hole, there was no turning back. Fueled by my curiosity, willingness to learn, and relentless persistence, I was able to boost performance by over 400% and reduce p99 latency by almost 80%.
Global deduplication for Ceph - Myoungwon Oh
Global deduplication for Ceph storage clusters can save up to 40% of total storage space by eliminating redundant data blocks. The document discusses two designs for implementing global deduplication in Ceph without adding a centralized metadata server: 1) using a double distribution hash to map data chunks to objects without redirection, and 2) storing deduplication metadata within self-contained objects to avoid complex linking between systems. It also describes the implementation of an extensible tier in Ceph for deduplication using object manifests and ongoing work to contribute these features to the upstream Ceph project. Remaining tasks are listed along with challenges around small chunk sizes and minimizing performance impacts.
Crimson: Ceph for the Age of NVMe and Persistent Memory
Ceph is a mature open source software-defined storage solution that was created over a decade ago.
During that time new faster storage technologies have emerged including NVMe and Persistent memory.
The crimson project aim is to create a better Ceph OSD that is more well suited to those faster devices. The crimson OSD is built on the Seastar C++ framework and can leverage these devices by minimizing latency, cpu overhead, and cross-core communication. This talk will discuss the project design, our current status, and our future plans.
(PFC306) Performance Tuning Amazon EC2 Instances | AWS re:Invent 2014
Netflix tunes Amazon EC2 instances for maximum performance. In this session, you learn how Netflix configures the fastest possible EC2 instances, while reducing latency outliers. This session explores the various Xen modes (e.g., HVM, PV, etc.) and how they are optimized for different workloads. Hear how Netflix chooses Linux kernel versions based on desired performance characteristics and receive a firsthand look at how they set kernel tunables, including hugepages. You also hear about Netflix's use of SR-IOV to enable enhanced networking and their approach to observability, which can exonerate EC2 issues and direct attention back to application performance.
Docker with BGP - OpenDNS
- Project Quadra is a PaaS built on Docker that can run on EC2, OpenStack, and bare metal. It uses an overlay network with GRE tunnels to extend a private network over Amazon VPC.
- The document provides configuration details for setting up GRE tunnels between EC2 instances and routing traffic over these tunnels using BGP. It also discusses Docker networking and Linux network namespaces.
- The document concludes with a demo and discusses possible next steps such as implementing OSPF for routing or setting up automated and fully meshed GRE tunnels.
Linux Block Cache Practice on Ceph BlueStore - Junxin Zhang
This document discusses using Linux block caching with Ceph BlueStore. It explains that BlueStore can better utilize fast storage devices like SSDs compared to FileStore. It tested using Bcache and DM-writeboost to cache BlueStore data on HDDs using SSDs. Bcache performed better overall. Issues found were slow requests when caching and BlueStore used the same SSD, and inconsistency in SSD data management between BlueStore and the cache. Future work could have BlueStore control all raw disks and prioritize data saving to fast devices.
Ceph QoS: How to support QoS in distributed storage system - Taewoong Kim
This document discusses supporting quality of service (QoS) in distributed storage systems like Ceph. It describes how SK Telecom has contributed to QoS support in Ceph, including an algorithm called dmClock that controls I/O request scheduling according to administrator-configured policies. It also details an outstanding I/O-based throttling mechanism to measure and regulate load. Finally, it discusses challenges like queue depth that can be addressed by increasing the number of scheduling threads, and outlines plans to improve and test Ceph's QoS features.
My Learnings on Setting up a Kubernetes Cluster on AWS using Kubernetes Opera...
I recently setup a Kubernetes cluster on Amazon Web Services(AWS) using Kubernetes Operations(KOPS). Here's some of my findings and learnings from a out-of-box-experience of setting up a Kubernetes cluster
RADOS improvements and roadmap - Greg Farnum, Josh Durgin, Kefu Chai
Cephalocon APAC 2018
March 22-23, 2018 - Beijing, China
Greg Farnum，Red Hat RADOS Core Developer
Josh Durgin, Red Hat RADOS Lead
Kefu Chai, Red Hat Senior Software Engineer
Full Stack Load Testing
A talk I gave at the Boston Web Performance Meetup in August 2014.
Performance is one of the most challenging issues in modern web app design, in large part because modeling, testing, and validating performance before deploying to production is so challenging. While many ops teams have nailed down the problem of re-creating pre-production environments that closely mimic production, those environments frequently rely on known-good components beyond the application code itself: AWS ELB, F5 load balancers, CDNs, Varnish, and more.
Testing plug-in components like that can be challenging, because their performance characteristics don't directly align with application metrics.
- How many simultaneous users can my load balancer support? - What sort of network load will I put on my CDN (i.e., how much will it cost?) - How do different user behavior patterns affect performance?
In this meetup, we'll introduce a novel tool in this toolbox: tcpreplay, an open-source tool for replaying packet capture files back at an application. By replaying user traffic to a staging environment, you can test the effects of
- Network saturation to the load balancer - High numbers of users / IPs - Lots of traffic to your other monitoring tools!
Optimizing Kubernetes Resource Requests/Limits for Cost-Efficiency and Latenc...
Kubernetes has the concept of resource requests and limits. Pods get scheduled on the nodes based on their requests and optionally limited in how much of the resource they can consume. Understanding and optimizing resource requests/limits is crucial both for reducing resource "slack" and ensuring application performance/low-latency. This talk shows our approach to monitoring and optimizing Kubernetes resources for 80+ clusters to achieve cost-efficiency and reducing impact for latency-critical applications. All shown tools are Open Source and can be applied to most Kubernetes deployments.
Self Created Load Balancer for MTA on AWS
This document summarizes the creation of a self-managed load balancer on AWS to distribute mail traffic across multiple mail gateway servers. It describes the existing mail system architecture, the need for a load balancer due to traffic volume limitations, and the technical implementation using Linux Virtual Server (LVS) and keepalived for load balancing and iptables for network address translation (SNAT) to support load balancing of SMTP traffic. The results were an increased ability to scale mail gateway servers elastically and observe traffic patterns from email services like Google Apps. A note of caution is provided about network bandwidth limitations based on the EC2 instance type used for the load balancer.
Galaxy CloudMan performance on AWS
Benchmark results for running bioinformatics platform Galaxy on the Amazon Web Services cloud. Results include info about disks, instance types, sizes, and variable data size.
OpenNebulaConf2018 - Why Latency is the #1 Metric of your Cloud - Boyan Krosn...
This document discusses why latency is the most important metric for cloud services. It recommends using 16 vCPUs, 32 GB of RAM, and a dedicated instance if possible for database testing. Benchmark testing should be done using PostgreSQL's pgbench tool with a database size 4 times the RAM.
Unikraft: Fast, Specialized Unikernels the Easy Way
P99 CONF
Unikernels are famous for providing excellent performance in terms of boot times, throughput and memory consumption, to name a few metrics. However, they are infamous for making it hard and extremely time consuming to extract such performance, and for needing significant engineering effort in order to port applications to them. We introduce Unikraft, a novel micro-library OS that (1) fully modularizes OS primitives so that it is easy to customize the unikernel and include only relevant components and (2) exposes a set of composable, performance-oriented APIs in order to make it easy for developers to obtain high performance.
Our evaluation using off-the-shelf applications such as nginx, SQLite, and Redis shows that running them on Unikraft results in a 1.7x-2.7x performance improvement compared to Linux guests. In addition, Unikraft images for these apps are around 1MB, require less than 10MB of RAM to run, and boot in around 1ms on top of the VMM time (total boot time 3ms-40ms). Unikraft is a Linux Foundation open source project and can be found at www.unikraft.org.
Ceph Day Shanghai - Ceph Performance Tools
This document discusses tools for analyzing Ceph performance. It begins by describing common performance issues users encounter with Ceph and potential solutions like tuning configuration values or benchmarking. The rest of the document details various monitoring and benchmarking tools that can help identify bottlenecks like the dispatch layer, object store, or hardware. It provides examples of using tools like dstat, iostat, perf, systemtap, ceph perf dump, and benchmarking tools like Fio, rbd-replay and ceph_perf_local. It concludes with a case study where unaligned partitions and a driver bug were causing low IOPS that were resolved by fixing the partition alignment and downgrading the NVMe driver.
Using SLOs for Continuous Performance Optimizations of Your k8s Workloads
The document discusses how SLOs (Service Level Objectives) can be used for continuous performance optimizations of Kubernetes workloads. It provides an overview of common performance issues like the N+1 query problem. It then describes how Keptn can be used to automate testing, analysis and remediation based on defined SLOs. Keptn establishes an event-driven workflow that orchestrates monitoring, deployments, testing and SLO evaluations to help optimize performance and ensure SLIs meet defined objectives. Real-world examples are provided of how Keptn has been used by companies to validate release readiness and environment stability.
Nitty Gritty of Adaptive Video Transmuxing in JS
This document summarizes adaptive video transmuxing techniques for converting between streaming formats like MPEG-TS and MP4 in a web browser. It discusses using incremental computation and finite state machines to start playback faster by parsing media formats and extracting video data as it is received. The techniques aim to reduce latency, support more features and streams, and improve the developer experience through a cleaner architecture based on communicating sequential processes.
Rust, Wright's Law, and the Future of Low-Latency Systems
The coming decade will see two important changes with profound ramifications for low-latency systems: the rise of Rust-based systems, and the ceding of Moore's Law to Wright's Law. In this talk, we will discuss these two trends, and (especially) their confluence -- and explain why we believe that the future of low-latency systems will include Rust programs in some surprising places.
Prometheus 2.0 Project Update @ CloudNativeCon
Slides from Tom Wilkie's keynote appearance at CloudNativeCon Austin, covering Prometheus, the 2.0 release and the community growth.
Evolving Prometheus for the Cloud Native World (FOSDEM 2018)
As the industry moves towards more cloud based and containerised solutions such as Kubernetes, monitoring tools have to keep up. These new environments are far more dynamic than the hand-maintained machines of old, requiring more sophisticated and scalable approaches. This talk will look at how Prometheus has evolved over the past 5 years to be better able to cope with these challenges, including the 2.0 release and practices that we encourage in a cloud native world.
More Related Content
What's hot
Scaling Apache Pulsar to 10 Petabytes/Day
This document discusses optimizing an Apache Pulsar cluster to handle 10 PB of data per day for a financial customer. Initial estimates showed the cluster would need over 1000 VMs using HDD storage. Various optimizations were implemented, including eliminating the journal, using direct I/O, compression, and C++ client optimizations. This reduced the estimated number of needed VMs to 200 using L-SSD storage per VM. The optimized cluster can now meet the customer's requirements of processing 10 PB of data per day with 3 hours of retention and zone failure protection.
Extreme HTTP Performance Tuning: 1.2M API req/s on a 4 vCPU EC2 Instance
In this talk I will walk you through the performance tuning steps that I took to serve 1.2M JSON requests per second from a 4 vCPU c5 instance, using a simple API server written in C.
At the start of the journey the server is capable of a very respectable 224k req/s with the default configuration. Along the way I made extensive use of tools like FlameGraph and bpftrace to measure, analyze, and optimize the entire stack, from the application framework, to the network driver, all the way down to the kernel.
I began this wild adventure without any prior low-level performance optimization experience; but once I started going down the performance tuning rabbit-hole, there was no turning back. Fueled by my curiosity, willingness to learn, and relentless persistence, I was able to boost performance by over 400% and reduce p99 latency by almost 80%.
Global deduplication for Ceph - Myoungwon Oh
Global deduplication for Ceph storage clusters can save up to 40% of total storage space by eliminating redundant data blocks. The document discusses two designs for implementing global deduplication in Ceph without adding a centralized metadata server: 1) using a double distribution hash to map data chunks to objects without redirection, and 2) storing deduplication metadata within self-contained objects to avoid complex linking between systems. It also describes the implementation of an extensible tier in Ceph for deduplication using object manifests and ongoing work to contribute these features to the upstream Ceph project. Remaining tasks are listed along with challenges around small chunk sizes and minimizing performance impacts.
Crimson: Ceph for the Age of NVMe and Persistent Memory
Ceph is a mature open source software-defined storage solution that was created over a decade ago.
During that time new faster storage technologies have emerged including NVMe and Persistent memory.
The crimson project aim is to create a better Ceph OSD that is more well suited to those faster devices. The crimson OSD is built on the Seastar C++ framework and can leverage these devices by minimizing latency, cpu overhead, and cross-core communication. This talk will discuss the project design, our current status, and our future plans.
(PFC306) Performance Tuning Amazon EC2 Instances | AWS re:Invent 2014
Netflix tunes Amazon EC2 instances for maximum performance. In this session, you learn how Netflix configures the fastest possible EC2 instances, while reducing latency outliers. This session explores the various Xen modes (e.g., HVM, PV, etc.) and how they are optimized for different workloads. Hear how Netflix chooses Linux kernel versions based on desired performance characteristics and receive a firsthand look at how they set kernel tunables, including hugepages. You also hear about Netflix's use of SR-IOV to enable enhanced networking and their approach to observability, which can exonerate EC2 issues and direct attention back to application performance.
Docker with BGP - OpenDNS
- Project Quadra is a PaaS built on Docker that can run on EC2, OpenStack, and bare metal. It uses an overlay network with GRE tunnels to extend a private network over Amazon VPC.
- The document provides configuration details for setting up GRE tunnels between EC2 instances and routing traffic over these tunnels using BGP. It also discusses Docker networking and Linux network namespaces.
- The document concludes with a demo and discusses possible next steps such as implementing OSPF for routing or setting up automated and fully meshed GRE tunnels.
Linux Block Cache Practice on Ceph BlueStore - Junxin Zhang
This document discusses using Linux block caching with Ceph BlueStore. It explains that BlueStore can better utilize fast storage devices like SSDs compared to FileStore. It tested using Bcache and DM-writeboost to cache BlueStore data on HDDs using SSDs. Bcache performed better overall. Issues found were slow requests when caching and BlueStore used the same SSD, and inconsistency in SSD data management between BlueStore and the cache. Future work could have BlueStore control all raw disks and prioritize data saving to fast devices.
Ceph QoS: How to support QoS in distributed storage system - Taewoong Kim
This document discusses supporting quality of service (QoS) in distributed storage systems like Ceph. It describes how SK Telecom has contributed to QoS support in Ceph, including an algorithm called dmClock that controls I/O request scheduling according to administrator-configured policies. It also details an outstanding I/O-based throttling mechanism to measure and regulate load. Finally, it discusses challenges like queue depth that can be addressed by increasing the number of scheduling threads, and outlines plans to improve and test Ceph's QoS features.
My Learnings on Setting up a Kubernetes Cluster on AWS using Kubernetes Opera...
I recently setup a Kubernetes cluster on Amazon Web Services(AWS) using Kubernetes Operations(KOPS). Here's some of my findings and learnings from a out-of-box-experience of setting up a Kubernetes cluster
RADOS improvements and roadmap - Greg Farnum, Josh Durgin, Kefu Chai
Cephalocon APAC 2018
March 22-23, 2018 - Beijing, China
Greg Farnum，Red Hat RADOS Core Developer
Josh Durgin, Red Hat RADOS Lead
Kefu Chai, Red Hat Senior Software Engineer
Full Stack Load Testing
A talk I gave at the Boston Web Performance Meetup in August 2014.
Performance is one of the most challenging issues in modern web app design, in large part because modeling, testing, and validating performance before deploying to production is so challenging. While many ops teams have nailed down the problem of re-creating pre-production environments that closely mimic production, those environments frequently rely on known-good components beyond the application code itself: AWS ELB, F5 load balancers, CDNs, Varnish, and more.
Testing plug-in components like that can be challenging, because their performance characteristics don't directly align with application metrics.
- How many simultaneous users can my load balancer support? - What sort of network load will I put on my CDN (i.e., how much will it cost?) - How do different user behavior patterns affect performance?
In this meetup, we'll introduce a novel tool in this toolbox: tcpreplay, an open-source tool for replaying packet capture files back at an application. By replaying user traffic to a staging environment, you can test the effects of
- Network saturation to the load balancer - High numbers of users / IPs - Lots of traffic to your other monitoring tools!
Optimizing Kubernetes Resource Requests/Limits for Cost-Efficiency and Latenc...
Kubernetes has the concept of resource requests and limits. Pods get scheduled on the nodes based on their requests and optionally limited in how much of the resource they can consume. Understanding and optimizing resource requests/limits is crucial both for reducing resource "slack" and ensuring application performance/low-latency. This talk shows our approach to monitoring and optimizing Kubernetes resources for 80+ clusters to achieve cost-efficiency and reducing impact for latency-critical applications. All shown tools are Open Source and can be applied to most Kubernetes deployments.
Self Created Load Balancer for MTA on AWS
This document summarizes the creation of a self-managed load balancer on AWS to distribute mail traffic across multiple mail gateway servers. It describes the existing mail system architecture, the need for a load balancer due to traffic volume limitations, and the technical implementation using Linux Virtual Server (LVS) and keepalived for load balancing and iptables for network address translation (SNAT) to support load balancing of SMTP traffic. The results were an increased ability to scale mail gateway servers elastically and observe traffic patterns from email services like Google Apps. A note of caution is provided about network bandwidth limitations based on the EC2 instance type used for the load balancer.
Galaxy CloudMan performance on AWS
Benchmark results for running bioinformatics platform Galaxy on the Amazon Web Services cloud. Results include info about disks, instance types, sizes, and variable data size.
OpenNebulaConf2018 - Why Latency is the #1 Metric of your Cloud - Boyan Krosn...
This document discusses why latency is the most important metric for cloud services. It recommends using 16 vCPUs, 32 GB of RAM, and a dedicated instance if possible for database testing. Benchmark testing should be done using PostgreSQL's pgbench tool with a database size 4 times the RAM.
Unikraft: Fast, Specialized Unikernels the Easy Way
P99 CONF
Unikernels are famous for providing excellent performance in terms of boot times, throughput and memory consumption, to name a few metrics. However, they are infamous for making it hard and extremely time consuming to extract such performance, and for needing significant engineering effort in order to port applications to them. We introduce Unikraft, a novel micro-library OS that (1) fully modularizes OS primitives so that it is easy to customize the unikernel and include only relevant components and (2) exposes a set of composable, performance-oriented APIs in order to make it easy for developers to obtain high performance.
Our evaluation using off-the-shelf applications such as nginx, SQLite, and Redis shows that running them on Unikraft results in a 1.7x-2.7x performance improvement compared to Linux guests. In addition, Unikraft images for these apps are around 1MB, require less than 10MB of RAM to run, and boot in around 1ms on top of the VMM time (total boot time 3ms-40ms). Unikraft is a Linux Foundation open source project and can be found at www.unikraft.org.
Ceph Day Shanghai - Ceph Performance Tools
This document discusses tools for analyzing Ceph performance. It begins by describing common performance issues users encounter with Ceph and potential solutions like tuning configuration values or benchmarking. The rest of the document details various monitoring and benchmarking tools that can help identify bottlenecks like the dispatch layer, object store, or hardware. It provides examples of using tools like dstat, iostat, perf, systemtap, ceph perf dump, and benchmarking tools like Fio, rbd-replay and ceph_perf_local. It concludes with a case study where unaligned partitions and a driver bug were causing low IOPS that were resolved by fixing the partition alignment and downgrading the NVMe driver.
Using SLOs for Continuous Performance Optimizations of Your k8s Workloads
The document discusses how SLOs (Service Level Objectives) can be used for continuous performance optimizations of Kubernetes workloads. It provides an overview of common performance issues like the N+1 query problem. It then describes how Keptn can be used to automate testing, analysis and remediation based on defined SLOs. Keptn establishes an event-driven workflow that orchestrates monitoring, deployments, testing and SLO evaluations to help optimize performance and ensure SLIs meet defined objectives. Real-world examples are provided of how Keptn has been used by companies to validate release readiness and environment stability.
Nitty Gritty of Adaptive Video Transmuxing in JS
This document summarizes adaptive video transmuxing techniques for converting between streaming formats like MPEG-TS and MP4 in a web browser. It discusses using incremental computation and finite state machines to start playback faster by parsing media formats and extracting video data as it is received. The techniques aim to reduce latency, support more features and streams, and improve the developer experience through a cleaner architecture based on communicating sequential processes.
Rust, Wright's Law, and the Future of Low-Latency Systems
The coming decade will see two important changes with profound ramifications for low-latency systems: the rise of Rust-based systems, and the ceding of Moore's Law to Wright's Law. In this talk, we will discuss these two trends, and (especially) their confluence -- and explain why we believe that the future of low-latency systems will include Rust programs in some surprising places.
What's hot (20)
Extreme HTTP Performance Tuning: 1.2M API req/s on a 4 vCPU EC2 Instance
Extreme HTTP Performance Tuning: 1.2M API req/s on a 4 vCPU EC2 Instance
Crimson: Ceph for the Age of NVMe and Persistent Memory
Crimson: Ceph for the Age of NVMe and Persistent Memory
(PFC306) Performance Tuning Amazon EC2 Instances | AWS re:Invent 2014
(PFC306) Performance Tuning Amazon EC2 Instances | AWS re:Invent 2014
Linux Block Cache Practice on Ceph BlueStore - Junxin Zhang
Linux Block Cache Practice on Ceph BlueStore - Junxin Zhang
Ceph QoS: How to support QoS in distributed storage system - Taewoong Kim
Ceph QoS: How to support QoS in distributed storage system - Taewoong Kim
My Learnings on Setting up a Kubernetes Cluster on AWS using Kubernetes Opera...
My Learnings on Setting up a Kubernetes Cluster on AWS using Kubernetes Opera...
RADOS improvements and roadmap - Greg Farnum, Josh Durgin, Kefu Chai
RADOS improvements and roadmap - Greg Farnum, Josh Durgin, Kefu Chai
Optimizing Kubernetes Resource Requests/Limits for Cost-Efficiency and Latenc...
Optimizing Kubernetes Resource Requests/Limits for Cost-Efficiency and Latenc...
OpenNebulaConf2018 - Why Latency is the #1 Metric of your Cloud - Boyan Krosn...
OpenNebulaConf2018 - Why Latency is the #1 Metric of your Cloud - Boyan Krosn...
Unikraft: Fast, Specialized Unikernels the Easy Way
Unikraft: Fast, Specialized Unikernels the Easy Way
Using SLOs for Continuous Performance Optimizations of Your k8s Workloads
Using SLOs for Continuous Performance Optimizations of Your k8s Workloads
Rust, Wright's Law, and the Future of Low-Latency Systems
Rust, Wright's Law, and the Future of Low-Latency Systems
Similar to Prometheus 2.0
Prometheus 2.0 Project Update @ CloudNativeCon
Slides from Tom Wilkie's keynote appearance at CloudNativeCon Austin, covering Prometheus, the 2.0 release and the community growth.
Evolving Prometheus for the Cloud Native World (FOSDEM 2018)
As the industry moves towards more cloud based and containerised solutions such as Kubernetes, monitoring tools have to keep up. These new environments are far more dynamic than the hand-maintained machines of old, requiring more sophisticated and scalable approaches. This talk will look at how Prometheus has evolved over the past 5 years to be better able to cope with these challenges, including the 2.0 release and practices that we encourage in a cloud native world.
Flink Forward Berlin 2017: Robert Metzger - Keep it going - How to reliably a...
Let’s be honest: Running a distributed stateful stream processor that is able to handle terabytes of state and tens of gigabytes of data per second while being highly available and correct (in an exactly-once sense) does not work without any planning, configuration and monitoring. While the Flink developer community tries to make everything as simple as possible, it is still important to be aware of all the requirements and implications In this talk, we will provide some insights into the greatest operations mysteries of Flink from a high-level perspective: - Capacity and resource planning: Understand the theoretical limits. - Memory and CPU configuration: Distribute resources according to your needs. - Setting up High Availability: Planning for failures. - Checkpointing and State Backends: Ensure correctness and fast recovery For each of the listed topics, we will introduce the concepts of Flink and provide some best practices we have learned over the past years supporting Flink users in production.
Windows Azure Storage – Architecture View
This presentation provides architecture overview of Azure storage services including blob, table, queue and best practices of storage services.
Storm worker redesign
The document discusses proposed enhancements to improve the performance of Apache Storm 2.0. It analyzes the current messaging architecture and identifies bottlenecks. Preliminary testing shows the redesigned messaging architecture improves latency by 116x and increases throughput by 50% over Storm 1.0 and the 2.0 master branch. Further optimizations to grouping, tuple implementation, and the acking mechanism could potentially yield even higher throughput of 15 million tuples per second. A new threading and execution model is also proposed to improve performance.
RedisConf17- Using Redis at scale @ Twitter
The document discusses Nighthawk, Twitter's distributed caching system which uses Redis. It provides caching services at a massive scale of over 10 million queries per second and 10 terabytes of data across 3000 Redis nodes. The key aspects of Nighthawk's architecture that allow it to scale are its use of a client-oblivious proxy layer and cluster manager that can independently scale and rebalance partitions across Redis nodes. It also employs replication between data centers to provide high availability even in the event of node failures. Some challenges discussed are handling "hot keys" that get an unusually high volume of requests and more efficiently warming up replicas when nodes fail.
AWS CLOUD 2018- Amazon DynamoDB기반 글로벌 서비스 개발 방법 (김준형 솔루션즈 아키텍트)
This document summarizes Amazon DynamoDB features and new capabilities presented at AWS re:Invent 2017. It includes 3 case studies:
1) How Samsung migrated from Cassandra to DynamoDB, improving performance and reducing costs by 50%+.
2) New DynamoDB capabilities like global tables, encryption at rest, on-demand backups were evaluated.
3) Best practices for migration including decreasing threads and item batches to control load are discussed.
OSS Presentation Metro Cluster by Andy Bennett & Roel De Frene
The document describes a stretched metro cluster configuration using NexentaStor software and Coraid SRX storage arrays. Key features include high availability across geographically separate sites, non-disruptive failover for block and file storage, and synchronous replication providing a disaster recovery solution. The configuration was implemented for a Belgian managed services provider, delivering over 10,000 IOPS and 200MB/s of throughput between sites 15km apart with failover times under a minute. A video demo and performance benchmarks are presented.
Toward 10,000 Containers on OpenStack
Charts for the presentation at the OpenStack Summit at Barcelona, October 2016. The video is available at:
https://www.openstack.org/videos/video/toward-10000-containers-on-openstack
What's New with Amazon DynamoDB - SRV311 - Atlanta AWS Summit
Amazon DynamoDB is a serverless database for applications that need a flexible data model with high performance at any scale. In this session, we cover newly announced features and provide an end-to-end view of recent innovations. We also share some of our successful customer stories and use cases. Come to this session to learn all about what's new with DynamoDB!
What's New with Amazon DynamoDB - SRV311 - Chicago AWS Summit
Rick Houlihan presented on What's New with Amazon DynamoDB. The key points included:
1) DynamoDB is a fully managed NoSQL database service that provides fast and flexible performance at any scale.
2) New features of DynamoDB include Global Tables for multi-region replication, encryption at rest, on-demand backups, and VPC endpoints for increased security.
3) DynamoDB Accelerator (DAX) provides in-memory caching of DynamoDB data to reduce response times from milliseconds to microseconds.
4) DynamoDB's auto-scaling capabilities automatically provision throughput and storage capacity to handle varying workloads without performance impacts or required administration.
NFS and Oracle
The document discusses tuning NFS for Oracle databases. It covers network topology issues like routers and switches that can increase latency. NFS configuration is also important, specifically MTU size for jumbo frames, TCP window size, and congestion window settings. Properly configuring these NFS and network settings is necessary to achieve best performance and avoid dropped packets or out of order delivery.
KOCOON – KAKAO Automatic K8S Monitoring
Open & Intelligent Infrastructure kr 2019
Day2 Track2 14:00 ~ 14:40
KOCOON – KAKAO Automatic K8S Monitoring
[Hadoop Meetup] Yarn at Microsoft - The challenges of scale
The document discusses how YARN has been scaled to support very large Hadoop clusters at Microsoft. Key points include:
- YARN supports clusters of over 40,000 nodes through a federation architecture with multiple independent YARN sub-clusters.
- A router and AMRM proxy allow applications to transparently run across sub-clusters while maintaining high availability and reliability.
- Early challenges with scaling included jobs slowing down when a new sub-cluster was added due to imbalanced scheduling across clusters. Monitoring and load balancing fixes were implemented.
- Future work aims to further improve federation, scheduling, and opportunistic containers in very large YARN deployments.
IMCSummit 2015 - Day 2 IT Business Track - 4 Myths about In-Memory Databases ...
IMCSummit 2015 - Day 2 IT Business Track - 4 Myths about In-Memory Databases ...In-Memory Computing Summit
Have you heard that all in-memory databases are equally fast but unreliable, inconsistent and expensive? This session highlights in-memory technology that busts all those myths.
Redis, the fastest database on the planet, is not a simply in-memory key-value data-store; but rather a rich in-memory data-structure engine that serves the world’s most popular apps. Redis Labs’ unique clustering technology enables Redis to be highly reliable, keeping every data byte intact despite hundreds of cloud instance failures and dozens of complete data-center outages. It delivers full CP system characteristics at high performance. And with the latest Redis on Flash technology, Redis Labs achieves close to in-memory performance at 70% lower operational costs. Learn about the best uses of in-memory computing to accelerate everyday applications such as high volume transactions, real time analytics, IoT data ingestion and more.Netflix at-disney-09-26-2014
Slides from a presentation by Monal Daxini at Disney, Glendale CA about Netflix Open Source Software, Cloud Data Persistence, and Cassandra best Practices
Distributed automation selcamp2016
This is about Distributed Automation using Selenium Grid in AWS - fully autoscaled using SeleniumGridScaler plugin
JPrime_JITServer.pptx
For Java developers, the Just-In-Time (JIT) compiler is key to improved performance. However, in a container world, the performance gains are often negated due to CPU and memory consumption constraints. To help solve this issue, the Eclipse OpenJ9 JVM provides JITServer technology, which separates the JIT compiler from the application. JITServer allows the user to employ much smaller containers enabling a higher density of applications, resulting in cost savings for end-users and/or cloud providers. Because the CPU and memory surges due to JIT compilation are eliminated, the user has a much easier task of provisioning resources for his/her application. Additional advantages include: faster ramp-up time, better control over resources devoted to compilation, increased reliability (JIT compiler bugs no longer crash the application) and amortization of compilation costs across many application instances. We will dig into JITServer technology, showing the challenges of implementation, detailing its strengths and weaknesses and illustrating its performance characteristics. For the cloud audience we will show how it can be deployed in containers, demonstrate its advantages compared to a traditional JIT compilation technique and offer practical recommendations about when to use this technology.
Breaking Prometheus (Promcon Berlin '16)
This document discusses strategies for scaling Prometheus monitoring to millions of machines. It describes the stages of scaling from manual Prometheus configuration to per-region Prometheus instances. Issues with input/output and retention led to further scaling techniques like sharding, using Kafka as a message bus, downsampling metrics, and developing an agent and query API. The talk introduces "Vulcan" which implements Prometheus APIs with Cassandra storage and Kafka input to enable monitoring at large scale.
Ensuring Kubernetes Cost Efficiency across (many) Clusters - DevOps Gathering...
Talk held at DevOps Gathering 2019 in Bochum on 2019-03-13.
Abstract: This talk will address one of the most common challenges of organizations adopting Kubernetes on a medium to large scale: how to keep cloud costs under control without babysitting each and every deployment and cluster configuration? How to operate 80+ Kubernetes clusters in a cost-efficient way for 200+ autonomous development teams?
This talk provides insights on how Zalando approaches this problem with central cost optimizations (e.g. Spot), cost monitoring/alerting, active measures to reduce resource slack, and automated cluster housekeeping. We will focus on how to ingrain cost efficiency in tooling and developer workflows while balancing rigid cost control with developer convenience and without impacting availability or performance. We will show our use case running Kubernetes on AWS, but all shown tools are open source and can be applied to most other infrastructure environments.
Similar to Prometheus 2.0 (20)
Evolving Prometheus for the Cloud Native World (FOSDEM 2018)
Evolving Prometheus for the Cloud Native World (FOSDEM 2018)
Flink Forward Berlin 2017: Robert Metzger - Keep it going - How to reliably a...
Flink Forward Berlin 2017: Robert Metzger - Keep it going - How to reliably a...
AWS CLOUD 2018- Amazon DynamoDB기반 글로벌 서비스 개발 방법 (김준형 솔루션즈 아키텍트)
AWS CLOUD 2018- Amazon DynamoDB기반 글로벌 서비스 개발 방법 (김준형 솔루션즈 아키텍트)
OSS Presentation Metro Cluster by Andy Bennett & Roel De Frene
OSS Presentation Metro Cluster by Andy Bennett & Roel De Frene
What's New with Amazon DynamoDB - SRV311 - Atlanta AWS Summit
What's New with Amazon DynamoDB - SRV311 - Atlanta AWS Summit
What's New with Amazon DynamoDB - SRV311 - Chicago AWS Summit
What's New with Amazon DynamoDB - SRV311 - Chicago AWS Summit
[Hadoop Meetup] Yarn at Microsoft - The challenges of scale
[Hadoop Meetup] Yarn at Microsoft - The challenges of scale
IMCSummit 2015 - Day 2 IT Business Track - 4 Myths about In-Memory Databases ...
IMCSummit 2015 - Day 2 IT Business Track - 4 Myths about In-Memory Databases ...
Ensuring Kubernetes Cost Efficiency across (many) Clusters - DevOps Gathering...
Ensuring Kubernetes Cost Efficiency across (many) Clusters - DevOps Gathering...
More from Kausal
Prometheus Monitoring Mixins (Berlin CNCB Meetup)
This document discusses using Jsonnet to package together Prometheus dashboards, alerts and exporters for monitoring Kubernetes clusters. It provides an example of using mixins to define reusable monitoring configurations and templates Grafana dashboards. It demonstrates building a Consul monitoring mixin interactively and emphasizes codifying best practices to share configurable monitoring components.
The RED Method: How To Instrument Your Services
The document describes the RED method for instrumentation and monitoring of services. The RED method involves monitoring the rate, errors, and duration of requests for each service. It can be implemented in Prometheus by instrumenting code to collect metrics on request count, status codes, and response durations. These core metrics provide insight into a service's traffic, errors, and performance. The RED method complements the USE method for resources and the "Four Golden Signals" of latency, traffic, errors, and saturation for monitoring each service.
The RED Method: How To Instrument Your Services
Slides from my talk at London Go User Group.
The RED Method defines three key metrics you should measure for every microservice in your architecture; inspired by the USE Method from Brendan Gregg, it gives developers a template for instrumenting their services and building dashboards in a consistent, repeatable fashion.
In this talk we will discuss patterns of application instrumentation, where and when they are applicable, and how they can be implemented with Prometheus. We’ll cover Google’s Four Golden Signals, the RED Method, the USE Method, and Dye Testing. We’ll also discuss why consistency is an important approach for reducing cognitive load. Finally we’ll talk about the limitations of these approaches and what can be done to overcome them.
gRPC & Kubernetes
In this talk we will discuss the use of gRPC to build a microservices-based application on Kubernetes. We’ll firstly focus on the basic concepts of gRPC: protocol buffers, code generation and HTTP 2.0. We’ll then discuss some more intermediate-level features, such as native Kubernetes load balancing, gRPC streaming, error propagation, HTTP tunneling and instrumentation. Finally we’ll touch on some of the newer, exciting developments in the space: middleware and context propagation for distributed tracing, gRPC Web for client-side rich web apps and gogoproto for blazing-fast generated code. This talk is based on my experience building a hosted, Prometheus-based monitoring service, which itself used gRPC for all intra-server communication and runs on Kubernetes. There’ll be live coding, real world examples and discussion of the benefits (and drawbacks) of gRPC.
Presented at GDG Devfest Ukraine 14th Oct 2017.
Cortex: Prometheus as a Service, One Year On
Presented by Tom Wilkie at PromCon 2017
With Speaker Notes: https://goo.gl/V9qGva
At PromCon 2016, I presented "Project Frankenstein: A multitenant, horizontally scalable Prometheus as a service". It's now one year later, and lots has changed - not least the name! This talk will discuss what we've learnt running a Prometheus service for the past year, the architectural changes we made from the original design, and the improvements we've made to the Cortex user experience.
Things you wish you never knew about the Prometheus Remote Write API.
The document discusses the Prometheus Remote Write API and some things to know about it. It covers four main topics: 1) ordering guarantees may be relaxed in the future, 2) dynamic sharding allows targets to be automatically sharded, 3) the API currently supports snappy batching and streaming writes, and 4) future plans include moving to gRPC and improving remote reads, writes, and supported authentication types. The document provides links to related GitHub pull requests for further information.
More from Kausal (6)
Things you wish you never knew about the Prometheus Remote Write API.
Things you wish you never knew about the Prometheus Remote Write API.
Recently uploaded
UiPath Test Automation using UiPath Test Suite series, part 6
Welcome to UiPath Test Automation using UiPath Test Suite series part 6. In this session, we will cover Test Automation with generative AI and Open AI.
UiPath Test Automation with generative AI and Open AI webinar offers an in-depth exploration of leveraging cutting-edge technologies for test automation within the UiPath platform. Attendees will delve into the integration of generative AI, a test automation solution, with Open AI advanced natural language processing capabilities.
Throughout the session, participants will discover how this synergy empowers testers to automate repetitive tasks, enhance testing accuracy, and expedite the software testing life cycle. Topics covered include the seamless integration process, practical use cases, and the benefits of harnessing AI-driven automation for UiPath testing initiatives. By attending this webinar, testers, and automation professionals can gain valuable insights into harnessing the power of AI to optimize their test automation workflows within the UiPath ecosystem, ultimately driving efficiency and quality in software development processes.
What will you get from this session?
1. Insights into integrating generative AI.
2. Understanding how this integration enhances test automation within the UiPath platform
3. Practical demonstrations
4. Exploration of real-world use cases illustrating the benefits of AI-driven test automation for UiPath
Topics covered:
What is generative AI
Test Automation with generative AI and Open AI.
UiPath integration with generative AI
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...Edge AI and Vision Alliance
For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2024/06/building-and-scaling-ai-applications-with-the-nx-ai-manager-a-presentation-from-network-optix/
Robin van Emden, Senior Director of Data Science at Network Optix, presents the “Building and Scaling AI Applications with the Nx AI Manager,” tutorial at the May 2024 Embedded Vision Summit.
In this presentation, van Emden covers the basics of scaling edge AI solutions using the Nx tool kit. He emphasizes the process of developing AI models and deploying them globally. He also showcases the conversion of AI models and the creation of effective edge AI pipelines, with a focus on pre-processing, model conversion, selecting the appropriate inference engine for the target hardware and post-processing.
van Emden shows how Nx can simplify the developer’s life and facilitate a rapid transition from concept to production-ready applications.He provides valuable insights into developing scalable and efficient edge AI solutions, with a strong focus on practical implementation.National Security Agency - NSA mobile device best practices
Threats to mobile devices are more prevalent and increasing in scope and complexity. Users of mobile devices desire to take full advantage of the features
available on those devices, but many of the features provide convenience and capability but sacrifice security. This best practices guide outlines steps the users can take to better protect personal devices and information.
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
As the digital landscape continually evolves, operating systems play a critical role in shaping user experiences and productivity. The launch of Nitrux Linux 3.5.0 marks a significant milestone, offering a robust alternative to traditional systems such as Windows 11. This article delves into the essence of Nitrux Linux 3.5.0, exploring its unique features, advantages, and how it stands as a compelling choice for both casual users and tech enthusiasts.
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
The choice of an operating system plays a pivotal role in shaping our computing experience. For decades, Microsoft's Windows has dominated the market, offering a familiar and widely adopted platform for personal and professional use. However, as technological advancements continue to push the boundaries of innovation, alternative operating systems have emerged, challenging the status quo and offering users a fresh perspective on computing.
One such alternative that has garnered significant attention and acclaim is Nitrux Linux 3.5.0, a sleek, powerful, and user-friendly Linux distribution that promises to redefine the way we interact with our devices. With its focus on performance, security, and customization, Nitrux Linux presents a compelling case for those seeking to break free from the constraints of proprietary software and embrace the freedom and flexibility of open-source computing.
Communications Mining Series - Zero to Hero - Session 1
This session provides introduction to UiPath Communication Mining, importance and platform overview. You will acquire a good understand of the phases in Communication Mining as we go over the platform with you. Topics covered:
• Communication Mining Overview
• Why is it important?
• How can it help today’s business and the benefits
• Phases in Communication Mining
• Demo on Platform overview
• Q/A
20240609 QFM020 Irresponsible AI Reading List May 2024
Everything I found interesting about the irresponsible use of machine intelligence in May 2024
Climate Impact of Software Testing at Nordic Testing Days
My slides at Nordic Testing Days 6.6.2024
Climate impact / sustainability of software testing discussed on the talk. ICT and testing must carry their part of global responsibility to help with the climat warming. We can minimize the carbon footprint but we can also have a carbon handprint, a positive impact on the climate. Quality characteristics can be added with sustainability, and then measured continuously. Test environments can be used less, and in smaller scale and on demand. Test techniques can be used in optimizing or minimizing number of tests. Test automation can be used to speed up testing.
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Monitoring and observability aren’t traditionally found in software curriculums and many of us cobble this knowledge together from whatever vendor or ecosystem we were first introduced to and whatever is a part of your current company’s observability stack.
While the dev and ops silo continues to crumble….many organizations still relegate monitoring & observability as the purview of ops, infra and SRE teams. This is a mistake - achieving a highly observable system requires collaboration up and down the stack.
I, a former op, would like to extend an invitation to all application developers to join the observability party will share these foundational concepts to build on:
Presentation of the OECD Artificial Intelligence Review of Germany
Consult the full report at https://www.oecd.org/digital/oecd-artificial-intelligence-review-of-germany-609808d6-en.htm
Full-RAG: A modern architecture for hyper-personalization
Mike Del Balso, CEO & Co-Founder at Tecton, presents "Full RAG," a novel approach to AI recommendation systems, aiming to push beyond the limitations of traditional models through a deep integration of contextual insights and real-time data, leveraging the Retrieval-Augmented Generation architecture. This talk will outline Full RAG's potential to significantly enhance personalization, address engineering challenges such as data management and model training, and introduce data enrichment with reranking as a key solution. Attendees will gain crucial insights into the importance of hyperpersonalization in AI, the capabilities of Full RAG for advanced personalization, and strategies for managing complex data integrations for deploying cutting-edge AI solutions.
UiPath Test Automation using UiPath Test Suite series, part 5
Welcome to UiPath Test Automation using UiPath Test Suite series part 5. In this session, we will cover CI/CD with devops.
Topics covered:
CI/CD with in UiPath
End-to-end overview of CI/CD pipeline with Azure devops
Speaker:
Lyndsey Byblow, Test Suite Sales Engineer @ UiPath, Inc.
Video Streaming: Then, Now, and in the Future
In his public lecture, Christian Timmerer provides insights into the fascinating history of video streaming, starting from its humble beginnings before YouTube to the groundbreaking technologies that now dominate platforms like Netflix and ORF ON. Timmerer also presents provocative contributions of his own that have significantly influenced the industry. He concludes by looking at future challenges and invites the audience to join in a discussion.
みなさんこんにちはこれ何文字まで入るの？40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの？えこ...
ここ3000字までしか入らないけどタイトルの方がたくさん文字入ると思います。
How to Get CNIC Information System with Paksim Ga.pptx
Pakdata Cf is a groundbreaking system designed to streamline and facilitate access to CNIC information. This innovative platform leverages advanced technology to provide users with efficient and secure access to their CNIC details.
“I’m still / I’m still / Chaining from the Block”
“An Outlook of the Ongoing and Future Relationship between Blockchain Technologies and Process-aware Information Systems.” Invited talk at the joint workshop on Blockchain for Information Systems (BC4IS) and Blockchain for Trusted Data Sharing (B4TDS), co-located with with the 36th International Conference on Advanced Information Systems Engineering (CAiSE), 3 June 2024, Limassol, Cyprus.
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for life science domain, where you retriever information from biomedical knowledge graphs using LLMs to increase the accuracy and performance of generated answers
Recently uploaded (20)
UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
National Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practices
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
Communications Mining Series - Zero to Hero - Session 1
Communications Mining Series - Zero to Hero - Session 1
20240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 2024
Climate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing Days
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Presentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of Germany
Full-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalization
UiPath Test Automation using UiPath Test Suite series, part 5
UiPath Test Automation using UiPath Test Suite series, part 5
みなさんこんにちはこれ何文字まで入るの？40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの？えこ...
みなさんこんにちはこれ何文字まで入るの？40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの？えこ...
How to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptx
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracy
Prometheus 2.0
- 1. Brian Brazil Founder What's new in Prometheus 2.0 Tom Wilkie GrafanaLabs
- 2. Open source monitoring system, for dynamic environments. Powerful, concise query language. Simple operational model. Efficient local storage. Prometheus
- 3. Prometheus History <0.9.0: • Basic TSDB, primitive PromQL • 50k samples/s 0.9.0: TSDB v2 by Beorn: • 800k/s, Gorilla, 10M timeseries 2.0.0: TSDB v3 by Fabian: • “millions” samples/s • WAL, mmap
- 4. Prometheus 2.0 • Optimized for Kubernetes • Brand new storage engine • Significantly improved performance • Aimed at highly dynamic environments, CI/CD. • Big thanks for Fabian Reinartz, Brian Brazil and Goutham Veeramachaneni for making this happen.
- 5. Prometheus 2.0 ~3x reduction in CPU usage ~2x reduction in disk space ~100x reduction in IO
- 6. Staleness Handling Previously if a target went down, last value would be returned for 5 minutes. In Prometheus 2.0 the value will stop being returned after a scrape interval. Simpler to understand, and no more double counting of fast restarting servers!
- 7. Community
- 8. Community
- 9. 220 attendees - 2.7x bigger than 2016 PromCon 2017: Aug 9th & 10th Community
- 10. Prometheus: The Book Coming in 2018!