This document provides information on various debugging and profiling tools that can be used for Ruby including:
- lsof to list open files for a process
- strace to trace system calls and signals
- tcpdump to dump network traffic
- google perftools profiler for CPU profiling
- pprof to analyze profiling data
It also discusses how some of these tools have helped identify specific performance issues with Ruby like excessive calls to sigprocmask and memcpy calls slowing down EventMachine with threads.
Kernel Recipes 2017 - Modern Key Management with GPG - Werner KochAnne Nicolas
Although GnuPG 2 has been around for nearly 15 years, the old 1.4 version was still in wide use. With Debian and others making 2.1 the default, many interesting things can now be done. In this talk he will explain the advantages of modern key algorithms, like ed25519, and why gpg relaxed some of its more paranoid defaults. The new –quick commands of gpg for easily scriptable key management will be described as well as the new key discovery methods. Finally hints for integration of gpg into other programs will be given.
Werner Koch, g10code
USENIX LISA2021 talk by Brendan Gregg (https://www.youtube.com/watch?v=_5Z2AU7QTH4). This talk is a deep dive that describes how BPF (eBPF) works internally on Linux, and dissects some modern performance observability tools. Details covered include the kernel BPF implementation: the verifier, JIT compilation, and the BPF execution environment; the BPF instruction set; different event sources; and how BPF is used by user space, using bpftrace programs as an example. This includes showing how bpftrace is compiled to LLVM IR and then BPF bytecode, and how per-event data and aggregated map data are fetched from the kernel.
Kernel Recipes 2017 - Modern Key Management with GPG - Werner KochAnne Nicolas
Although GnuPG 2 has been around for nearly 15 years, the old 1.4 version was still in wide use. With Debian and others making 2.1 the default, many interesting things can now be done. In this talk he will explain the advantages of modern key algorithms, like ed25519, and why gpg relaxed some of its more paranoid defaults. The new –quick commands of gpg for easily scriptable key management will be described as well as the new key discovery methods. Finally hints for integration of gpg into other programs will be given.
Werner Koch, g10code
USENIX LISA2021 talk by Brendan Gregg (https://www.youtube.com/watch?v=_5Z2AU7QTH4). This talk is a deep dive that describes how BPF (eBPF) works internally on Linux, and dissects some modern performance observability tools. Details covered include the kernel BPF implementation: the verifier, JIT compilation, and the BPF execution environment; the BPF instruction set; different event sources; and how BPF is used by user space, using bpftrace programs as an example. This includes showing how bpftrace is compiled to LLVM IR and then BPF bytecode, and how per-event data and aggregated map data are fetched from the kernel.
Tracing MariaDB server with bpftrace - MariaDB Server Fest 2021Valeriy Kravchuk
Bpftrace is a relatively new eBPF-based open source tracer for modern Linux versions (kernels 5.x.y) that is useful for analyzing production performance problems and troubleshooting software. Basic usage of the tool, as well as bpftrace one liners and advanced scripts useful for MariaDB DBAs are presented. Problems of MariaDB Server dynamic tracing with bpftrace and some possible solutions and alternative tracing tools are discussed.
Linux Tracing Superpowers by Eugene PirogovPivorak MeetUp
For a long time Linux was far behind operating systems of Unix family from the perspective of debuggability, specifically in a live production systems.
However, over the course of 2016 Linux saw a series of patches that brought it on par with Unix world: an old Linux tool called BPF has risen and extended into powerful new one – eBPF. Some say that eBPF marks the begining of true DTrace for Linux.
During the presentation I'm going to talk about tracing basics, cover a series of events that led to the development of eBPF and will compare eBPF with DTrace from Unix world. Current state of affairs of Linux tracing tools will be explored. Finally, together we'll look at some of the exciting examples of eBPF application.
***
Eugene is well known in our Ruby (and Elixir) communities. Last time when he was at #pivorak he made a very light and interesting intro to the Elixir. You can check his speech out here - http://bit.ly/2evCd9R
Kernel Recipes 2017: Performance Analysis with BPFBrendan Gregg
Talk by Brendan Gregg at Kernel Recipes 2017 (Paris): "The in-kernel Berkeley Packet Filter (BPF) has been enhanced in recent kernels to do much more than just filtering packets. It can now run user-defined programs on events, such as on tracepoints, kprobes, uprobes, and perf_events, allowing advanced performance analysis tools to be created. These can be used in production as the BPF virtual machine is sandboxed and will reject unsafe code, and are already in use at Netflix.
Beginning with the bpf() syscall in 3.18, enhancements have been added in many kernel versions since, with major features for BPF analysis landing in Linux 4.1, 4.4, 4.7, and 4.9. Specific capabilities these provide include custom in-kernel summaries of metrics, custom latency measurements, and frequency counting kernel and user stack traces on events. One interesting case involves saving stack traces on wake up events, and associating them with the blocked stack trace: so that we can see the blocking stack trace and the waker together, merged in kernel by a BPF program (that particular example is in the kernel as samples/bpf/offwaketime).
This talk will discuss the new BPF capabilities for performance analysis and debugging, and demonstrate the new open source tools that have been developed to use it, many of which are in the Linux Foundation iovisor bcc (BPF Compiler Collection) project. These include tools to analyze the CPU scheduler, TCP performance, file system performance, block I/O, and more."
True stories on the analysis of network activity using Pythondelimitry
The presentation from SPbPython community / PiterPy meetup.
The presentation tells about the problems of analysing the network activity of applications on Linux using Python. The following topics are covered: analysis of network packets, analysis of packet filters, packets crafting using Scapy, analysis of open ports.
University of Virginia
cs4414: Operating Systems
http://rust-class.org
What happened with Apple's SSL implementation
How to make sure this doesn't happen to you!
Sharing data
ARCs in Rust
Scheduling
For embedded notes, see:
Kernel Recipes 2019 - Hunting and fixing bugs all over the Linux kernelAnne Nicolas
At a rate of almost 9 changes per hour (24/7), the Linux kernel is definitely a scary beast. Bugs are introduced on a daily basis and, through the use of multiple code analyzers, *some* of them are detected and fixed before they hit mainline. Over the course of the last few years, Gustavo has been fixing such bugs and many different issues in every corner of the Linux kernel. Recently, he was in charge of leading the efforts to globally enable -Wimplicit-fallthrough; which appears by default in Linux v5.3. This presentation is a report on all the stuff Gustavo has found and fixed in the kernel with the support of the Core Infrastructure Initiative.
Gustavo A.R. Silva
Практический опыт профайлинга и оптимизации производительности Ruby-приложенийOlga Lavrentieva
Алексей Туля, Senior Software Developer в Sam Solutions
«Практический опыт профайлинга и оптимизации производительности Ruby-приложений»
В своем докладе Алексей сделает краткий обзор различных реализаций Ruby, попытается найти причины, почему Ruby медленный. Рассмотрит вопрос сборки мусора в Ruby и вызова методов – почему в Ruby это дорого. Расскажет и покажет, что делать, чтобы поднять производительность, проведет обзор утилит для поиска проблемных мест, обзор профайлеров и расскажет, как интерпретировать результаты.
Доклад в основном нацелен на практический подход по поиску проблем. Материал предназначен для пользователей Linux, поэтому все практические советы будут для ОС Linux.
seccomp is a computer security facility in the Linux kernel, pledge is a similar security facility in the OpenBSD kernel. In this presentation Giovanni Bechis will review the development story and progress of both kernel interfaces and will analyze the main differences. There will be some examples of implementations of security patches made for some important open source projects.
Tracing MariaDB server with bpftrace - MariaDB Server Fest 2021Valeriy Kravchuk
Bpftrace is a relatively new eBPF-based open source tracer for modern Linux versions (kernels 5.x.y) that is useful for analyzing production performance problems and troubleshooting software. Basic usage of the tool, as well as bpftrace one liners and advanced scripts useful for MariaDB DBAs are presented. Problems of MariaDB Server dynamic tracing with bpftrace and some possible solutions and alternative tracing tools are discussed.
Linux Tracing Superpowers by Eugene PirogovPivorak MeetUp
For a long time Linux was far behind operating systems of Unix family from the perspective of debuggability, specifically in a live production systems.
However, over the course of 2016 Linux saw a series of patches that brought it on par with Unix world: an old Linux tool called BPF has risen and extended into powerful new one – eBPF. Some say that eBPF marks the begining of true DTrace for Linux.
During the presentation I'm going to talk about tracing basics, cover a series of events that led to the development of eBPF and will compare eBPF with DTrace from Unix world. Current state of affairs of Linux tracing tools will be explored. Finally, together we'll look at some of the exciting examples of eBPF application.
***
Eugene is well known in our Ruby (and Elixir) communities. Last time when he was at #pivorak he made a very light and interesting intro to the Elixir. You can check his speech out here - http://bit.ly/2evCd9R
Kernel Recipes 2017: Performance Analysis with BPFBrendan Gregg
Talk by Brendan Gregg at Kernel Recipes 2017 (Paris): "The in-kernel Berkeley Packet Filter (BPF) has been enhanced in recent kernels to do much more than just filtering packets. It can now run user-defined programs on events, such as on tracepoints, kprobes, uprobes, and perf_events, allowing advanced performance analysis tools to be created. These can be used in production as the BPF virtual machine is sandboxed and will reject unsafe code, and are already in use at Netflix.
Beginning with the bpf() syscall in 3.18, enhancements have been added in many kernel versions since, with major features for BPF analysis landing in Linux 4.1, 4.4, 4.7, and 4.9. Specific capabilities these provide include custom in-kernel summaries of metrics, custom latency measurements, and frequency counting kernel and user stack traces on events. One interesting case involves saving stack traces on wake up events, and associating them with the blocked stack trace: so that we can see the blocking stack trace and the waker together, merged in kernel by a BPF program (that particular example is in the kernel as samples/bpf/offwaketime).
This talk will discuss the new BPF capabilities for performance analysis and debugging, and demonstrate the new open source tools that have been developed to use it, many of which are in the Linux Foundation iovisor bcc (BPF Compiler Collection) project. These include tools to analyze the CPU scheduler, TCP performance, file system performance, block I/O, and more."
True stories on the analysis of network activity using Pythondelimitry
The presentation from SPbPython community / PiterPy meetup.
The presentation tells about the problems of analysing the network activity of applications on Linux using Python. The following topics are covered: analysis of network packets, analysis of packet filters, packets crafting using Scapy, analysis of open ports.
University of Virginia
cs4414: Operating Systems
http://rust-class.org
What happened with Apple's SSL implementation
How to make sure this doesn't happen to you!
Sharing data
ARCs in Rust
Scheduling
For embedded notes, see:
Kernel Recipes 2019 - Hunting and fixing bugs all over the Linux kernelAnne Nicolas
At a rate of almost 9 changes per hour (24/7), the Linux kernel is definitely a scary beast. Bugs are introduced on a daily basis and, through the use of multiple code analyzers, *some* of them are detected and fixed before they hit mainline. Over the course of the last few years, Gustavo has been fixing such bugs and many different issues in every corner of the Linux kernel. Recently, he was in charge of leading the efforts to globally enable -Wimplicit-fallthrough; which appears by default in Linux v5.3. This presentation is a report on all the stuff Gustavo has found and fixed in the kernel with the support of the Core Infrastructure Initiative.
Gustavo A.R. Silva
Практический опыт профайлинга и оптимизации производительности Ruby-приложенийOlga Lavrentieva
Алексей Туля, Senior Software Developer в Sam Solutions
«Практический опыт профайлинга и оптимизации производительности Ruby-приложений»
В своем докладе Алексей сделает краткий обзор различных реализаций Ruby, попытается найти причины, почему Ruby медленный. Рассмотрит вопрос сборки мусора в Ruby и вызова методов – почему в Ruby это дорого. Расскажет и покажет, что делать, чтобы поднять производительность, проведет обзор утилит для поиска проблемных мест, обзор профайлеров и расскажет, как интерпретировать результаты.
Доклад в основном нацелен на практический подход по поиску проблем. Материал предназначен для пользователей Linux, поэтому все практические советы будут для ОС Linux.
seccomp is a computer security facility in the Linux kernel, pledge is a similar security facility in the OpenBSD kernel. In this presentation Giovanni Bechis will review the development story and progress of both kernel interfaces and will analyze the main differences. There will be some examples of implementations of security patches made for some important open source projects.
Tempest is an Openstack test suite which runs against all the OpenStack service endpoints. It makes sure that all the OpenStack components work together properly and that no APIs are changed. Tempest is a "gate" for all commits to OpenStack repositories and will prevent merges if tests fail.
Who carries your container? Zun or Magnum?Madhuri Kumari
There are multiple solution in OpenStack to enable containers. These slides talk about two projects i.e. Magnum and Zun in OpenStack and their use cases.
OSDC 2017 - Werner Fischer - Linux performance profiling and monitoringNETWAYS
Nowadays system administrators have great choices when it comes down to Linux performance profiling and monitoring. The challenge is to pick the appropriate tools and interpret their results correctly.
This talk is a chance to take a tour through various performance profiling and benchmarking tools, focusing on their benefit for every sysadmin.
More than 25 different tools are presented. Ranging from well known tools like strace, iostat, tcpdump or vmstat to new features like Linux tracepoints or perf_events. You will also learn which tools can be monitored by Icinga and which monitoring plugins are already available for that.
At the end the goal is to gather reference points to look at, whenever you are faced with performance problems.
Take the chance to close your knowledge gaps and learn how to get the most out of your system.
Как понять, что происходит на сервере? / Александр Крижановский (NatSys Lab.,...Ontico
Запускаем сервер (БД, Web-сервер или что-то свое собственное) и не получаем желаемый RPS. Запускаем top и видим, что 100% выедается CPU. Что дальше, на что расходуется процессорное время? Можно ли подкрутить какие-то ручки, чтобы улучшить производительность? А если параметр CPU не высокий, то куда смотреть дальше?
Мы рассмотрим несколько сценариев проблем производительности, рассмотрим доступные инструменты анализа производительности и разберемся в методологии оптимизации производительности Linux, ответим на вопрос за какие ручки и как крутить.
USENIX ATC 2017 Performance Superpowers with Enhanced BPFBrendan Gregg
Talk for USENIX ATC 2017 by Brendan Gregg
"The Berkeley Packet Filter (BPF) in Linux has been enhanced in very recent versions to do much more than just filter packets, and has become a hot area of operating systems innovation, with much more yet to be discovered. BPF is a sandboxed virtual machine that runs user-level defined programs in kernel context, and is part of many kernels. The Linux enhancements allow it to run custom programs on other events, including kernel- and user-level dynamic tracing (kprobes and uprobes), static tracing (tracepoints), and hardware events. This is finding uses for the generation of new performance analysis tools, network acceleration technologies, and security intrusion detection systems.
This talk will explain the BPF enhancements, then discuss the new performance observability tools that are in use and being created, especially from the BPF compiler collection (bcc) open source project. These tools provide new insights for file system and storage performance, CPU scheduler performance, TCP performance, and much more. This is a major turning point for Linux systems engineering, as custom advanced performance instrumentation can be used safely in production environments, powering a new generation of tools and visualizations.
Because these BPF enhancements are only in very recent Linux (such as Linux 4.9), most companies are not yet running new enough kernels to be exploring BPF yet. This will change in the next year or two, as companies including Netflix upgrade their kernels. This talk will give you a head start on this growing technology, and also discuss areas of future work and unsolved problems."
Velocity 2017 Performance analysis superpowers with Linux eBPFBrendan Gregg
Talk by for Velocity 2017 by Brendan Gregg: Performance analysis superpowers with Linux eBPF.
"Advanced performance observability and debugging have arrived built into the Linux 4.x series, thanks to enhancements to Berkeley Packet Filter (BPF, or eBPF) and the repurposing of its sandboxed virtual machine to provide programmatic capabilities to system tracing. Netflix has been investigating its use for new observability tools, monitoring, security uses, and more. This talk will investigate this new technology, which sooner or later will be available to everyone who uses Linux. The talk will dive deep on these new tracing, observability, and debugging capabilities. Whether you’re doing analysis over an ssh session, or via a monitoring GUI, BPF can be used to provide an efficient, custom, and deep level of detail into system and application performance.
This talk will also demonstrate the new open source tools that have been developed, which make use of kernel- and user-level dynamic tracing (kprobes and uprobes), and kernel- and user-level static tracing (tracepoints). These tools provide new insights for file system and storage performance, CPU scheduler performance, TCP performance, and a whole lot more. This is a major turning point for Linux systems engineering, as custom advanced performance instrumentation can be used safely in production environments, powering a new generation of tools and visualizations."
When your whole system is unresponsive, how to investigate on this failure ?
We'll see how to get a memory dump for offline analysis with kdump system.
Then how to analyze it with crash utility.
And finally, how to use crash on a running system to modify the kernel memory (at your own risks !)
Presented at LISA18: https://www.usenix.org/conference/lisa18/presentation/babrou
This is a technical dive into how we used eBPF to solve real-world issues uncovered during an innocent OS upgrade. We'll see how we debugged 10x CPU increase in Kafka after Debian upgrade and what lessons we learned. We'll get from high-level effects like increased CPU to flamegraphs showing us where the problem lies to tracing timers and functions calls in the Linux kernel.
The focus is on tools what operational engineers can use to debug performance issues in production. This particular issue happened at Cloudflare on a Kafka cluster doing 100Gbps of ingress and many multiple of that egress.
Kernel Recipes 2017 - Performance analysis Superpowers with Linux BPF - Brend...Anne Nicolas
The in-kernel Berkeley Packet Filter (BPF) has been enhanced in recent kernels to do much more than just filtering packets. It can now run user-defined programs on events, such as on tracepoints, kprobes, uprobes, and perf_events, allowing advanced performance analysis tools to be created. These can be used in production as the BPF virtual machine is sandboxed and will reject unsafe code, and are already in use at Netflix.
Beginning with the bpf() syscall in 3.18, enhancements have been added in many kernel versions since, with major features for BPF analysis landing in Linux 4.1, 4.4, 4.7, and 4.9. Specific capabilities these provide include custom in-kernel summaries of metrics, custom latency measurements, and frequency counting kernel and user stack traces on events. One interesting case involves saving stack traces on wake up events, and associating them with the blocked stack trace: so that we can see the blocking stack trace and the waker together, merged in kernel by a BPF program (that particular example is in the kernel as samples/bpf/offwaketime).
This talk will discuss the new BPF capabilities for performance analysis and debugging, and demonstrate the new open source tools that have been developed to use it, many of which are in the Linux Foundation iovisor bcc (BPF Compiler Collection) project. These include tools to analyze the CPU scheduler, TCP performance, file system performance, block I/O, and more.
Brendan Gregg, Netflix
Performance analysis in a multitenant cloud environment Using Hadoop Cluster ...Orgad Kimchi
Analyzing the performance of a virtualized multitenant cloud environment can be challenging because of the layers of abstraction. This article shows how to use Oracle Solaris 11 to overcome those limitations.
For more information see:
http://www.oracle.com/technetwork/articles/servers-storage-admin/perf-analysis-multitenant-cloud-2082193.html
OSSNA 2017 Performance Analysis Superpowers with Linux BPFBrendan Gregg
Talk by Brendan Gregg for OSSNA 2017. "Advanced performance observability and debugging have arrived built into the Linux 4.x series, thanks to enhancements to Berkeley Packet Filter (BPF, or eBPF) and the repurposing of its sandboxed virtual machine to provide programmatic capabilities to system tracing. Netflix has been investigating its use for new observability tools, monitoring, security uses, and more. This talk will be a dive deep on these new tracing, observability, and debugging capabilities, which sooner or later will be available to everyone who uses Linux. Whether you’re doing analysis over an ssh session, or via a monitoring GUI, BPF can be used to provide an efficient, custom, and deep level of detail into system and application performance.
This talk will also demonstrate the new open source tools that have been developed, which make use of kernel- and user-level dynamic tracing (kprobes and uprobes), and kernel- and user-level static tracing (tracepoints). These tools provide new insights for file system and storage performance, CPU scheduler performance, TCP performance, and a whole lot more. This is a major turning point for Linux systems engineering, as custom advanced performance instrumentation can be used safely in production environments, powering a new generation of tools and visualizations."
UiPath Test Automation using UiPath Test Suite series, part 4DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap.
The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies.
Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques
What will you get from this session?
1. Insights into SAP testing best practices
2. Heatmap utilization for testing
3. Optimization of testing processes
4. Demo
Topics covered:
Execution from the test manager
Orchestrator execution result
Defect reporting
SAP heatmap example with demo
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
PHP Frameworks: I want to break free (IPC Berlin 2024)Ralf Eggert
In this presentation, we examine the challenges and limitations of relying too heavily on PHP frameworks in web development. We discuss the history of PHP and its frameworks to understand how this dependence has evolved. The focus will be on providing concrete tips and strategies to reduce reliance on these frameworks, based on real-world examples and practical considerations. The goal is to equip developers with the skills and knowledge to create more flexible and future-proof web applications. We'll explore the importance of maintaining autonomy in a rapidly changing tech landscape and how to make informed decisions in PHP development.
This talk is aimed at encouraging a more independent approach to using PHP frameworks, moving towards a more flexible and future-proof approach to PHP development.
State of ICS and IoT Cyber Threat Landscape Report 2024 previewPrayukth K V
The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development.
The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers:
State of global ICS asset and network exposure
Sectoral targets and attacks as well as the cost of ransom
Global APT activity, AI usage, actor and tactic profiles, and implications
Rise in volumes of AI-powered cyberattacks
Major cyber events in 2024
Malware and malicious payload trends
Cyberattack types and targets
Vulnerability exploit attempts on CVEs
Attacks on counties – USA
Expansion of bot farms – how, where, and why
In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East
Why are attacks on smart factories rising?
Cyber risk predictions
Axis of attacks – Europe
Systemic attacks in the Middle East
Download the full report from here:
https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/
UiPath Test Automation using UiPath Test Suite series, part 3DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 3. In this session, we will cover desktop automation along with UI automation.
Topics covered:
UI automation Introduction,
UI automation Sample
Desktop automation flow
Pradeep Chinnala, Senior Consultant Automation Developer @WonderBotz and UiPath MVP
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
"Impact of front-end architecture on development cost", Viktor TurskyiFwdays
I have heard many times that architecture is not important for the front-end. Also, many times I have seen how developers implement features on the front-end just following the standard rules for a framework and think that this is enough to successfully launch the project, and then the project fails. How to prevent this and what approach to choose? I have launched dozens of complex projects and during the talk we will analyze which approaches have worked for me and which have not.
Let's dive deeper into the world of ODC! Ricardo Alves (OutSystems) will join us to tell all about the new Data Fabric. After that, Sezen de Bruijn (OutSystems) will get into the details on how to best design a sturdy architecture within ODC.
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
7. strace -cp <pid>
-c
Count time, calls, and errors for each system call and report a
summary on program exit.
-p pid
Attach to the process with the process ID pid and begin tracing.
% time seconds usecs/call calls errors syscall
------ ----------- ----------- --------- --------- ----------------
50.39 0.000064 0 1197 592 read
34.65 0.000044 0 609 writev
14.96 0.000019 0 1226 epoll_ctl
0.00 0.000000 0 4 close
0.00 0.000000 0 1 select
0.00 0.000000 0 4 socket
0.00 0.000000 0 4 4 connect
0.00 0.000000 0 1057 epoll_wait
------ ----------- ----------- --------- --------- ----------------
100.00 0.000127 4134 596 total
8. strace -ttTp <pid> -o <file>
-t
Prefix each line of the trace with the time of day.
-tt
If given twice, the time printed will include the microseconds.
-T
Show the time spent in system calls. This records the time
difference between the beginning and the end of each system call.
-o filename
Write the trace output to the file filename rather than to stderr.
01:09:11.266949 epoll_wait(9, {{EPOLLIN, {u32=68841296, u64=68841296}}}, 4096, 50) = 1 <0.033109>
01:09:11.300102 accept(10, {sa_family=AF_INET, sin_port=38313, sin_addr="127.0.0.1"}, [1226]) = 22 <0.000014>
01:09:11.300190 fcntl(22, F_GETFL) = 0x2 (flags O_RDWR) <0.000007>
01:09:11.300237 fcntl(22, F_SETFL, O_RDWR|O_NONBLOCK) = 0 <0.000008>
01:09:11.300277 setsockopt(22, SOL_TCP, TCP_NODELAY, [1], 4) = 0 <0.000008>
01:09:11.300489 accept(10, 0x7fff5d9c07d0, [1226]) = -1 EAGAIN <0.000014>
01:09:11.300547 epoll_ctl(9, EPOLL_CTL_ADD, 22, {EPOLLIN, {u32=108750368, u64=108750368}}) = 0 <0.000009>
01:09:11.300593 epoll_wait(9, {{EPOLLIN, {u32=108750368, u64=108750368}}}, 4096, 50) = 1 <0.000007>
01:09:11.300633 read(22, "GET / HTTP/1.1r"..., 16384) = 772 <0.000012>
01:09:11.301727 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 <0.000007>
01:09:11.302095 poll([{fd=5, events=POLLIN|POLLPRI}], 1, 0) = 0 (Timeout) <0.000008>
01:09:11.302144 write(5, "1000000-0003SELECT * FROM `table`"..., 56) = 56 <0.000023>
01:09:11.302221 read(5, "25101,20x234m"..., 16384) = 284 <1.300897>
10. stracing ruby: sigprocmask
% time seconds usecs/call calls errors syscall
------ ----------- ----------- --------- --------- ----------------
100.00 0.326334 0 3568567 rt_sigprocmask
0.00 0.000000 0 9 read
0.00 0.000000 0 10 open
0.00 0.000000 0 10 close
0.00 0.000000 0 9 fstat
0.00 0.000000 0 25 mmap
------ ----------- ----------- --------- --------- ----------------
100.00 0.326334 3568685 0 total
debian/redhat compile ruby with --enable-pthread
uses a native thread timer for SIGVTALRM
causes excessive calls to sigprocmask: 30% slowdown!
11. tcpdump
dump traffic on a network
tcpdump -i eth1 -s 1500 -nqA
tcp dst port 80
tcpdump -i eth0 -s 1500 -nqA
tcp dst port 3306
tcpdump -i eth1 -s 1500 -w <file>
tcp dst port 80
12. tcpdump -i <eth> -s <len> -nqA <expr>
tcpdump -i <eth> -w <file> <expr>
-i <eth>
Network interface.
-s <len>
Snarf len bytes of data from each packet.
-n
Don't convert addresses (host addresses, port numbers) to names.
-q
Quiet output. Print less protocol information.
-A
Print each packet (minus its link level header) in ASCII.
-w <file>
Write the raw packets to file rather than printing them out.
<expr>
libpcap expression, for example:
tcp src port 80
tcp dst port 3306
13. tcp dst port 80
19:52:20.216294 IP 24.203.197.27.40105 > 174.37.48.236.80: tcp 438
E...*.@.l.%&.....%0....POx..%s.oP.......GET /poll_images/cld99erh0/logo.png HTTP/1.1
Accept: */*
Referer: http://apps.facebook.com/realpolls/?_fb_q=1
tcp dst port 3306
19:51:06.501632 IP 10.8.85.66.50443 > 10.8.85.68.3306: tcp 98
E..."K@.@.Yy
.UB
.UD.....z....L............
GZ.y3b..[......W....SELECT * FROM `votes` WHERE (`poll_id` = 72621) LIMIT 1
tcpdump -w <file>
17. Profiling MRI
10% of production VM
time spent in
rb_str_sub_bang
String#sub!
called from Time.parse
return unless str.sub!(/A(d{1,2})/, '')
return unless str.sub!(/A( d|d{1,2})/, '')
return unless str.sub!(/A( d|d{1,2})/, '')
return unless str.sub!(/A(d{1,3})/, '')
return unless str.sub!(/A(d{1,2})/, '')
return unless str.sub!(/A(d{1,2})/, '')
switch to third_base gem
ThirdBase: Fast and Easy
Date/DateTime class for
Ruby
18. Profiling EM + threads
Total: 3763 samples
2764 73.5% catch_timer
989 26.3% memcpy
3 0.1% st_lookup
2 0.1% rb_thread_schedule
1 0.0% rb_eval
1 0.0% rb_newobj
1 0.0% rb_gc_force_recycle
known issue: EM+threads =
slow
memcpy??
thread context switches
copy the stack w/ memcpy
EM allocates huge buffer
on the stack
solution: move buffer to
the heap
22. ltrace -F <conf> -b -g -x <sym>
-b
Ignore signals.
-g
Ignore libraries linked at compile time.
-F <conf>
Read prototypes from config file.
-x <sym>
Trace calls to the function sym.
-s <num>
Show first num bytes of string args.
-F ltrace.conf
int mysql_real_query(addr,string,ulong);
void garbage_collect(void);
int memcached_set(addr,string,ulong,string,ulong);
24. gdb
the GNU debugger
gdb <program>
gdb <program> <pid>
Be sure to build with:
-ggdb
-O0
25. gdb walkthrough
% gdb ./test-it start gdb
(gdb) b average set breakpoint on function named average
Breakpoint 1 at 0x1f8e: file test-it.c, line 3.
(gdb) run run program
Starting program: /Users/joe/test-it
Reading symbols for shared libraries ++. done
Breakpoint 1, average (x=5, y=6) at test-it.c:3 hit breakpoint!
3 int sum = x + y;
(gdb) bt show backtrace
#0 average (x=5, y=6) at test-it.c:3
function stack
#1 0x00001fec in main () at test-it.c:12
(gdb) s
4 double avg = sum / 2.0; single step
(gdb) s
5 return avg;
(gdb) p avg
$1 = 5.5 print variables
(gdb) p sum
$2 = 11
26. Ruby VM stack traces
(gdb) where
#0 0x0002a55e in rb_call (klass=1386800, recv=5056455, mid=42, argc=1, argv=0xbfffe5c0,
scope=0, self=1403220) at eval.c:6125
#1 0x000226ef in rb_eval (self=1403220, n=0x1461e4) at eval.c:3493
#2 0x00026d01 in rb_yield_0 (val=5056455, self=1403220, klass=0, flags=0, avalue=0) at
eval.c:5083
#3 0x000270e8 in rb_yield (val=5056455) at eval.c:5168
#4 0x0005c30c in int_dotimes (num=1000000001) at numeric.c:2946
#5 0x00029be3 in call_cfunc (func=0x5c2a0 <int_dotimes>, recv=1000000001, len=0, argc=0,
argv=0x0) at eval.c:5759
#6 0x00028fd4 in rb_call0 (klass=1387580, recv=1000000001, id=5785, oid=5785, argc=0,
argv=0x0, body=0x152b24, flags=0) at eval.c:5911
#7 0x0002a7a7 in rb_call (klass=1387580, recv=1000000001, mid=5785, argc=0, argv=0x0,
scope=0, self=1403220) at eval.c:6158
#8 0x000226ef in rb_eval (self=1403220, n=0x146284) at eval.c:3493
#9 0x000213e3 in rb_eval (self=1403220, n=0x1461a8) at eval.c:3223
#10 0x0001ceea in eval_node (self=1403220, node=0x1461a8) at eval.c:1437
#11 0x0001d60f in ruby_exec_internal () at eval.c:1642
#12 0x0001d660 in ruby_exec () at eval.c:1662
#13 0x0001d68e in ruby_run () at eval.c:1672
#14 0x000023dc in main (argc=2, argv=0xbffff7c4, envp=0xbffff7d0) at main.c:48
rb_eval recursively executes ruby code in 1.8
27. Debugging Ruby Segfaults
test_segv.rb:4: [BUG] Segmentation fault
ruby 1.8.7 (2008-08-11 patchlevel 72) [i686-darwin9.7.0]
def test
#include "ruby.h" require 'segv'
4.times do
VALUE Dir.chdir '/tmp' do
segv() Hash.new{ segv }[0]
{ end
VALUE array[1]; end
array[1000000] = NULL; end
return Qnil;
} sleep 10
test()
void
Init_segv()
{
rb_define_method(rb_cObject, "segv", segv, 0);
}
28. 1. Attach to running process
$ ps aux | grep ruby
joe 23611 0.0 0.1 25424 7540 S Dec01 0:00 ruby test_segv.rb
$ sudo gdb ruby 23611
Attaching to program: ruby, process 23611
0x00007fa5113c0c93 in nanosleep () from /lib/libc.so.6
(gdb) c
Continuing.
Program received signal SIGBUS, Bus error.
segv () at segv.c:7
7 array[1000000] = NULL;
2. Use a coredump
Process.setrlimit Process::RLIMIT_CORE, 300*1024*1024
$ sudo mkdir /cores
$ sudo chmod 777 /cores
$ sudo sysctl kernel.core_pattern=/cores/%e.core.%s.%p.%t
$ sudo gdb ruby /cores/ruby.core.6.23611.1259781224
29. def test
require 'segv'
4.times do
Dir.chdir '/tmp' do
Hash.new{ segv }[0]
end
end (gdb) where
end #0 segv () at segv.c:7
#1 0x000000000041f2be in call_cfunc () at eval.c:5727
test() ...
#13 0x000000000043ba8c in rb_hash_default () at hash.c:521
...
#19 0x000000000043b92a in rb_hash_aref () at hash.c:429
...
#26 0x00000000004bb7bc in chdir_yield () at dir.c:728
#27 0x000000000041d8d7 in rb_ensure () at eval.c:5528
#28 0x00000000004bb93a in dir_s_chdir () at dir.c:816
...
#35 0x000000000041c444 in rb_yield () at eval.c:5142
#36 0x0000000000450690 in int_dotimes () at numeric.c:2834
...
#48 0x0000000000412a90 in ruby_run () at eval.c:1678
#49 0x000000000041014e in main () at main.c:48
32. require 'sinatra'
$ ab -c 1 -n 50 http://127.0.0.1:4567/compute
$ ab -c 1 -n 50 http://127.0.0.1:4567/sleep
get '/sleep' do
sleep 0.25 Sampling profiler:
'done'
end 232 samples total
get '/compute' do 83 samples were in /compute
proc{ |n|
a,b=0,1 118 samples had /compute on
n.times{ a,b = b,a+b } the stack but were in
b another function
}.call(10_000)
'done' /compute accounts for 50%
of process, but only 35% of
end time was in /compute itself
== Sinatra has ended his set (crowd applauds)
PROFILE: interrupts/evictions/bytes = 232/0/2152
Total: 232 samples
83 35.8% 35.8% 118 50.9% Sinatra::Application#GET /compute
56 24.1% 59.9% 56 24.1% garbage_collector
35 15.1% 75.0% 113 48.7% Integer#times
36. gdb.rb
gdb with MRI hooks
gdb.rb <pid>
http://github.com/tmm1/gdb.rb
37. def test
require 'segv'
4.times do
Dir.chdir '/tmp' do
Hash.new{ segv }[0]
end
end
end
(gdb) ruby threads
test()
0xa3e000 main curr thread THREAD_RUNNABLE WAIT_NONE
node_vcall segv in test_segv.rb:5
node_call test in test_segv.rb:5
node_call call in test_segv.rb:5
node_call default in test_segv.rb:5
node_call [] in test_segv.rb:5
node_call test in test_segv.rb:4
node_call chdir in test_segv.rb:4
node_call test in test_segv.rb:3
node_call times in test_segv.rb:3
node_vcall test in test_segv.rb:9
40. mongrel sleeper thread
0x16814c00 thread THREAD_STOPPED WAIT_TIME(0.47) 1522 bytes
node_fcall sleep in lib/mongrel/configurator.rb:285
node_fcall run in lib/mongrel/configurator.rb:285
node_fcall loop in lib/mongrel/configurator.rb:285
node_call run in lib/mongrel/configurator.rb:285
node_call initialize in lib/mongrel/configurator.rb:285
node_call new in lib/mongrel/configurator.rb:285
node_call run in bin/mongrel_rails:128
node_call run in lib/mongrel/command.rb:212
node_call run in bin/mongrel_rails:281
node_fcall (unknown) in bin/mongrel_rails:19
def run
@listeners.each {|name,s|
s.run
}
$mongrel_sleeper_thread = Thread.new { loop { sleep 1 } }
end
41. god memory leaks
(gdb) ruby objects arrays
elements instances
43 God::Process
94310 3
43 God::Watch
94311 3
43 God::Driver
94314 2
43 God::DriverEventQueue
94316 1
43 God::Conditions::MemoryUsage
43 God::Conditions::ProcessRunning
5369 arrays
43 God::Behaviors::CleanPidFile
2863364 member elements
45 Process::Status
86 God::Metric
many arrays with 327 God::System::SlashProcPoller
90k+ elements! 327 God::System::Process
406 God::DriverEvent
5 separate god leaks fixed by Eric
Lindvall with the help of gdb.rb!
42. ruby method cache
(gdb) ruby methodcache
Module#extend wipes the UnboundMethod#arity
Hash#[]=
entire method cache! Class#private
Array#freeze
(gdb) b rb_clear_cache Module#Integer
Breakpoint 1 at 0x41067b: file eval.c, line 351. Fixnum#<
(gdb) c Class#is_a?
Continuing. Fixnum#+
Class#protected
Breakpoint 1, rb_clear_cache () at eval.c:351 Class#>=
351 if (!ruby_running) return;
(gdb) ruby threads 2028 empty slots (99.02%)
0x1623000 main curr thread THREAD_RUNNABLE WAIT_NONE
node_call extend_object in sin.rb:23
node_call extend in sin.rb:23
node_call GET /other in lib/sinatra/base.rb:779
node_call GET /other in lib/sinatra/base.rb:779
node_call call in lib/sinatra/base.rb:779
node_fcall route in lib/sinatra/base.rb:474
44. 191691 total objects
Final heap size 191691 filled, 220961 free
Displaying top 20 most common line/class pairs
89513 __null__:__null__:__node__
41438 __null__:__null__:String
2348 lib/ruby/site_ruby/1.8/rubygems/specification.rb:557:Array
1508 lib/ruby/gems/1.8/specifications/gettext-1.9.gemspec:14:String
1021 lib/ruby/gems/1.8/specifications/heel-0.2.0.gemspec:14:String
951 lib/ruby/site_ruby/1.8/rubygems/version.rb:111:String
935 lib/ruby/site_ruby/1.8/rubygems/specification.rb:557:String
834 lib/ruby/site_ruby/1.8/rubygems/version.rb:146:Array
BleakHouse
installs a patched version of ruby: ruby-bleak-
house
unlike gdb.rb, see where objects were created
(file:line)
create multiple dumps over time with `kill -USR2
<pid>` and compare to find leaks
45. Coming soon:
bleak_house++
no patches or need to recompile
ruby
“assembly metaprogramming” to
setup a trampoline on rb_new_obj
http://timetobleed.com/rewrite-
your-ruby-vm-at-runtime-to-hot-
patch-useful-features/
memory profiler
coredump a production ruby process
load the core to generate profiles
of memory usage and leaks