There are at least 40 to 50 different formats of GC logs. Here, we explained the commonly used GC log formats, tricks, patterns and tools to analyze them effectively.
Are you building high throughput, low latency application? Are you trying to figure out perfect JVM heap size? Are you struggling to choose right garbage collection algorithm and settings? Are you striving to achieve pause less GC? Do you know the right tools & best practices to tame the GC? Do you know to troubleshoot memory problems using GC logs? You will get complete answers to several such questions in this presentation.
7 habits of highly effective Performance TroubleshootersTier1 app
Troubleshooting production performance problems is a combination of art, science, and discipline. Below is the presentation deck shared in the conference which explains, how to forecast the problems?, what to do when the problem is happening?, how to identify the root cause instantly? and how to prevent problems from happening in the future and so on.
Are you building high throughput, low latency application? Are you trying to figure out perfect JVM heap size? Are you struggling to choose right garbage collection algorithm and settings? Are you striving to achieve pause less GC? Do you know the right tools & best practices to tame the GC? Do you know to troubleshoot memory problems using GC logs? You will get complete answers to several such questions in this presentation.
7 habits of highly effective Performance TroubleshootersTier1 app
Troubleshooting production performance problems is a combination of art, science, and discipline. Below is the presentation deck shared in the conference which explains, how to forecast the problems?, what to do when the problem is happening?, how to identify the root cause instantly? and how to prevent problems from happening in the future and so on.
In this session, we will be discussing major outages that happened in major enterprises. We will be analyzing the actual thread dumps, heap dumps, GC logs, and other artifacts captured at the time of the problem. After this session, troubleshooting CPU spikes, OutOfMemoryError, response time degradations, network connectivity issues, application unresponsiveness may not stump you.
There are 600+ arguments that you can pass to JVM just around Garbage collection and memory. It’s way too many arguments for anyone to digest and comprehend. In this session, 7 important JVM arguments that will boost your application performance will be highlighted.
16 artifacts to capture when there is a production problemTier1 app
Production problems are tricky to troubleshoot if proper diagnostic information isn’t captured. In this session, 16 important artifacts that you need to capture and the effective tools that you can use to analyze those artifacts are discussed.
There are 600+ arguments that you can pass to JVM just around Garbage collection and memory. It’s way too many arguments for anyone to digest and comprehend. In this session, 7 important JVM arguments that will boost your application performance will be highlighted.
In this session, sample code snippets that can generate memory leak, thread leak, CPU spike, unresponsiveness, BLOCKED threads, deadlocks, heavy I / O activity are discussed. If you can understand what triggers these problems, diagnosing and solving them might become easier.
In this session, the following topics have been discussed: code snippets that can generate memory leak, thread leak, CPU spike, unresponsiveness, BLOCKED threads, Deadlocks, Heavy I/O activity. If you can understand what triggers these problems, diagnosing and solving them might become easier.
Modern Engineer’s Troubleshooting Tools, Techniques & Tricks at Confoo 2018Tier1app
Learn right tools, tricks and patterns to identify root cause of complex java problems in seconds (not even in minutes). Here is the presentation of Modern Engineer’s Troubleshooting Tools, Techniques & Tricks delivered at Confoo 2018
Java и Linux — особенности эксплуатации / Алексей Рагозин (Дойче Банк)Ontico
HighLoad++ 2017
Зал «Рио-де-Жанейро», 8 ноября, 11:00
Тезисы:
http://www.highload.ru/2017/abstracts/2884.html
Java на Linux встречается повсеместно в информационных системах от больших данных до новомодных serverless архитектур. Как Linux, так и Java имеют свои эксплуатационные нюансы. Понимание этих нюансов важно, чтобы заставить стек Java + Linux работать стабильно и эффективно.
Но на практике "джависты" очень любят мыслить кроссплатформенно и не хотят разбираться с особенностями операционной системы, a "линускоиды" считают JVM чуждым миру Linux процессом, пожирающим всю доступную на сервере память.
А потом появляется Docker, и нюансов становится ещё больше...
Цель доклада - рассказать "джавистам" про Linux и Docker, а "линуксоидам" про JVM.
ok.ru is one of top 10 internet sites of the World, according to similarweb.com. Under the hood, it has several thousand servers. Each of those servers own only fraction of the data or business logic. Shared nothing architecture can be hardly applied to social network, due to its nature, so a lot of communication happens between these servers, diverse in kind and volume. This makes ok.ru one of the largest, complicated, yet highly loaded distributed systems in the world.
This talk is about our experience in building always available, resilient to failures distributed systems in Java, their basic and not so basic failure and recovery scenarios, methods of failure testing and diagnostics. We’ll also discuss on possible disasters and how to prevent or get over them.
De gemiddelde GPU bevat tegenwoordig meer PK's dan de CPU. Naar aanleiding hiervan komen er steeds meer mogelijkheden om computationele problemen te verplaatsen van de CPU naar de GPU. Deze presentatie zal een inleiding zijn hoe je dit in Java kunt doen met behulp van Jogamp JoCL. Aan de hand van enkele simpele problemen wordt aangetoond wanneer een GPU beter ingezet kan worden dan een CPU en vice versa. Dit is ook een van de speerpunten in Java 9 (Project Sumatra) wat o.a. JoCL als inspiratie gebruikt.
Мастер-класс "Логическая репликация и Avito" / Константин Евтеев, Михаил Тюр...Ontico
HighLoad++ 2017
Зал «Кейптаун», 7 ноября, 12:00
Тезисы:
http://www.highload.ru/2017/abstracts/2868.html
В Avito объявления хранятся в базах данных Postgres. При этом уже на протяжении многих лет активно применяется логическая репликация. С помощью неё успешно решаются вопросы роста объема данных и количества запросов к ним, масштабирования и распределения нагрузки, доставки данных в DWH и поисковые подсистемы, меж-базные и меж-сервисные синхронизации данных и пр.
...
Add a bit of ACID to Cassandra. Cassandra Summit EU 2014odnoklassniki.ru
OK.ru is one of the largest social networks for Russian-speaking audiences with 80+ million unique user’s visits monthly. ok.ru uses Cassandra since 2010 and made a number of improvements to C* 2.0 and 2.1 codebase. Until recent time more than 50 TB of data at Ok.ru OLTP systems was managed by Microsoft SQL Sever. It’s very expensive, hard to scale and cannot save us from outage if one of our data centers fail. We wanted a new, fast scalable and reliable storage for these data. These data has requirements to support ACID transactions, so we don’t have to rewrite all application code from scratch. С* does not support these transactions, only lightweight, so we implemented a new storage with ACID and selected features of SQL world by ourselves. Still, it has C* at its heart. We’ll discuss the internals of the new storage, what features of C* we had to alter and which to rewrite from scratch. We’ll also talk about its operational experience in production.
Java In-Process Caching - Performance, Progress and Pittfallscruftex
Alternative approaches for Java In-Progress Caching. Alternative eviction algorithms and why LRU is a bad choice. Performance comparison of latest Java In-Process caching libraries, like EHCache, Google Guava, Caffeine and cache2k.
Talking about Neo4j after 1 year of using it production. This presentation covering db structure(internals), cypher queries, extensions development, db tuning & settings.
Are you building high throughput, low latency application? Are you trying to figure out perfect JVM heap size? Are you struggling to choose right garbage collection algorithm and settings? Are you striving to achieve pause less GC? Do you know the right tools & best practices to tame the GC? Do you know to troubleshoot memory problems using GC logs? You will get complete answers to several such questions in this PPT.
Tier1app CEO & Founder, Ram Lakshmanan, spoke at All Day Devops 2017 about Java GC Logs. In this presentation, you can learn how to enable Java GC logs, commonly used GC log formats, tricks, patterns and tools to analyze them effectively.
In this session, we will be discussing major outages that happened in major enterprises. We will be analyzing the actual thread dumps, heap dumps, GC logs, and other artifacts captured at the time of the problem. After this session, troubleshooting CPU spikes, OutOfMemoryError, response time degradations, network connectivity issues, application unresponsiveness may not stump you.
There are 600+ arguments that you can pass to JVM just around Garbage collection and memory. It’s way too many arguments for anyone to digest and comprehend. In this session, 7 important JVM arguments that will boost your application performance will be highlighted.
16 artifacts to capture when there is a production problemTier1 app
Production problems are tricky to troubleshoot if proper diagnostic information isn’t captured. In this session, 16 important artifacts that you need to capture and the effective tools that you can use to analyze those artifacts are discussed.
There are 600+ arguments that you can pass to JVM just around Garbage collection and memory. It’s way too many arguments for anyone to digest and comprehend. In this session, 7 important JVM arguments that will boost your application performance will be highlighted.
In this session, sample code snippets that can generate memory leak, thread leak, CPU spike, unresponsiveness, BLOCKED threads, deadlocks, heavy I / O activity are discussed. If you can understand what triggers these problems, diagnosing and solving them might become easier.
In this session, the following topics have been discussed: code snippets that can generate memory leak, thread leak, CPU spike, unresponsiveness, BLOCKED threads, Deadlocks, Heavy I/O activity. If you can understand what triggers these problems, diagnosing and solving them might become easier.
Modern Engineer’s Troubleshooting Tools, Techniques & Tricks at Confoo 2018Tier1app
Learn right tools, tricks and patterns to identify root cause of complex java problems in seconds (not even in minutes). Here is the presentation of Modern Engineer’s Troubleshooting Tools, Techniques & Tricks delivered at Confoo 2018
Java и Linux — особенности эксплуатации / Алексей Рагозин (Дойче Банк)Ontico
HighLoad++ 2017
Зал «Рио-де-Жанейро», 8 ноября, 11:00
Тезисы:
http://www.highload.ru/2017/abstracts/2884.html
Java на Linux встречается повсеместно в информационных системах от больших данных до новомодных serverless архитектур. Как Linux, так и Java имеют свои эксплуатационные нюансы. Понимание этих нюансов важно, чтобы заставить стек Java + Linux работать стабильно и эффективно.
Но на практике "джависты" очень любят мыслить кроссплатформенно и не хотят разбираться с особенностями операционной системы, a "линускоиды" считают JVM чуждым миру Linux процессом, пожирающим всю доступную на сервере память.
А потом появляется Docker, и нюансов становится ещё больше...
Цель доклада - рассказать "джавистам" про Linux и Docker, а "линуксоидам" про JVM.
ok.ru is one of top 10 internet sites of the World, according to similarweb.com. Under the hood, it has several thousand servers. Each of those servers own only fraction of the data or business logic. Shared nothing architecture can be hardly applied to social network, due to its nature, so a lot of communication happens between these servers, diverse in kind and volume. This makes ok.ru one of the largest, complicated, yet highly loaded distributed systems in the world.
This talk is about our experience in building always available, resilient to failures distributed systems in Java, their basic and not so basic failure and recovery scenarios, methods of failure testing and diagnostics. We’ll also discuss on possible disasters and how to prevent or get over them.
De gemiddelde GPU bevat tegenwoordig meer PK's dan de CPU. Naar aanleiding hiervan komen er steeds meer mogelijkheden om computationele problemen te verplaatsen van de CPU naar de GPU. Deze presentatie zal een inleiding zijn hoe je dit in Java kunt doen met behulp van Jogamp JoCL. Aan de hand van enkele simpele problemen wordt aangetoond wanneer een GPU beter ingezet kan worden dan een CPU en vice versa. Dit is ook een van de speerpunten in Java 9 (Project Sumatra) wat o.a. JoCL als inspiratie gebruikt.
Мастер-класс "Логическая репликация и Avito" / Константин Евтеев, Михаил Тюр...Ontico
HighLoad++ 2017
Зал «Кейптаун», 7 ноября, 12:00
Тезисы:
http://www.highload.ru/2017/abstracts/2868.html
В Avito объявления хранятся в базах данных Postgres. При этом уже на протяжении многих лет активно применяется логическая репликация. С помощью неё успешно решаются вопросы роста объема данных и количества запросов к ним, масштабирования и распределения нагрузки, доставки данных в DWH и поисковые подсистемы, меж-базные и меж-сервисные синхронизации данных и пр.
...
Add a bit of ACID to Cassandra. Cassandra Summit EU 2014odnoklassniki.ru
OK.ru is one of the largest social networks for Russian-speaking audiences with 80+ million unique user’s visits monthly. ok.ru uses Cassandra since 2010 and made a number of improvements to C* 2.0 and 2.1 codebase. Until recent time more than 50 TB of data at Ok.ru OLTP systems was managed by Microsoft SQL Sever. It’s very expensive, hard to scale and cannot save us from outage if one of our data centers fail. We wanted a new, fast scalable and reliable storage for these data. These data has requirements to support ACID transactions, so we don’t have to rewrite all application code from scratch. С* does not support these transactions, only lightweight, so we implemented a new storage with ACID and selected features of SQL world by ourselves. Still, it has C* at its heart. We’ll discuss the internals of the new storage, what features of C* we had to alter and which to rewrite from scratch. We’ll also talk about its operational experience in production.
Java In-Process Caching - Performance, Progress and Pittfallscruftex
Alternative approaches for Java In-Progress Caching. Alternative eviction algorithms and why LRU is a bad choice. Performance comparison of latest Java In-Process caching libraries, like EHCache, Google Guava, Caffeine and cache2k.
Talking about Neo4j after 1 year of using it production. This presentation covering db structure(internals), cypher queries, extensions development, db tuning & settings.
Are you building high throughput, low latency application? Are you trying to figure out perfect JVM heap size? Are you struggling to choose right garbage collection algorithm and settings? Are you striving to achieve pause less GC? Do you know the right tools & best practices to tame the GC? Do you know to troubleshoot memory problems using GC logs? You will get complete answers to several such questions in this PPT.
Tier1app CEO & Founder, Ram Lakshmanan, spoke at All Day Devops 2017 about Java GC Logs. In this presentation, you can learn how to enable Java GC logs, commonly used GC log formats, tricks, patterns and tools to analyze them effectively.
Description
At Stitch Fix most application logs are output in a structured JSON format for simpler debugging and downstream consumption.
In this talk we’ll cover in more detail why structured logs are useful and provide leverage, caveats to using them, and how simple it is to get one going with Python.
Abstract
At Stitch Fix most application logs are output in a structured JSON format for simpler debugging and downstream consumption. For example, data scientists can add a field to their application log and it will automatically turn up as a parsed field in Elasticsearch for easy dashboarding and querying via Kibana, or be easily found and queried in Presto. In this talk we’ll cover in more detail why structured logs are useful and provide leverage, caveats to using them, and how simple it is to get one going with Python.
Talk for PerconaLive 2016 by Brendan Gregg. Video: https://www.youtube.com/watch?v=CbmEDXq7es0 . "Systems performance provides a different perspective for analysis and tuning, and can help you find performance wins for your databases, applications, and the kernel. However, most of us are not performance or kernel engineers, and have limited time to study this topic. This talk summarizes six important areas of Linux systems performance in 50 minutes: observability tools, methodologies, benchmarking, profiling, tracing, and tuning. Included are recipes for Linux performance analysis and tuning (using vmstat, mpstat, iostat, etc), overviews of complex areas including profiling (perf_events), static tracing (tracepoints), and dynamic tracing (kprobes, uprobes), and much advice about what is and isn't important to learn. This talk is aimed at everyone: DBAs, developers, operations, etc, and in any environment running Linux, bare-metal or the cloud."
In Java 9, Garbage First Garbage Collector (G1 GC) will be the default GC. This presentation makes an effort to help Hotspot VM users to understand the concept of G1 GC as well as provides some tuning advice.
Hadoop Meetup Jan 2019 - Dynamometer and a Case Study in NameNode GCErik Krogen
Erik Krogen of LinkedIn presents regarding Dynamometer, a system open sourced by LinkedIn for scale- and performance-testing HDFS. He discusses one major use case for Dynamometer, tuning NameNode GC, and discusses characteristics of NameNode GC such as why it is important, and how it interacts with various current and future GC algorithms.
This is taken from the Apache Hadoop Contributors Meetup on January 30, hosted by LinkedIn in Mountain View.
CONFidence 2015: DTrace + OSX = Fun - Andrzej Dyjak PROIDEA
Speaker: Andrzej Dyjak
Language: English
In recent years security industry started to grow fond of Apple’s iOS and OS X platforms. This talk will cover one of XNU's flagship debugging utilities: DTrace, a dynamic tracing framework for troubleshooting kernel and application problems on production systems in real time. It will be shown how it can be used in order to ease various tasks within the realm of dynamic binary analysis and beyond.
CONFidence: http://confidence.org.pl/
Docker Logging and analysing with Elastic StackJakub Hajek
Collecting logs from the entire stateless environment is challenging parts of the application lifecycle. Correlating business logs with operating system metrics to provide insights is a crucial part of the entire organization. What aspects should be considered while you design your logging solutions?
Docker Logging and analysing with Elastic Stack - Jakub Hajek PROIDEA
Collecting logs from the entire stateless environment is challenging parts of the application lifecycle. Correlating business logs with operating system metrics to provide insights is a crucial part of the entire organization. We will see the technical presentation on how to manage a large amount of the data in a typical environment with microservices.
Ведущий: Макс Мороз
Обзор системы ClusterFuzz, позволяющей осуществить проверку браузера Chrome на наличие уязвимостей в режиме реального времени и получить воспроизводимые результаты исследования каждого конкретного сбоя. Будут продемонстрированы преимущества использования различных санитайзеров и LibFuzzer, библиотеки для направленного фаззинга. Будет приведена подробная статистика видов уязвимостей, найденных в Chrome. Слушатели узнают о подводных камнях распределенного фаззинга; о том, как можно запустить свои собственные фаззеры в инфраструктуре Google и получить вознаграждение за найденные уязвимости.
Java Garbage Collectors – Moving to Java7 Garbage First (G1) CollectorGurpreet Sachdeva
One of the key strengths of JVM is automatic memory management (Garbage Collection). Its understanding can help in writing better applications. This becomes all the more important as enterprise server applications have large amount of live heap data and significant parallel threads. Until recently, main collectors were parallel collector and concurrent-mark-sweep (CMS) collector. This presentation introduces the various Garbage Collectors and compares the CMS collector against its replacement, a new implementation in Java7 i.e. G1. It is characterized by a single contiguous heap which is split into same-sized regions. In fact if your application is still running on the 1.5 or 1.6 JVM, a compelling argument to upgrade to Java 7 is to leverage G1.
This slide will show you how to use SOFA to do performance analysis of CPU/GPU cooperative programs, especially programs running with deep software stacks like TensorFlow, PyTorch, etc.
source code at:
https://github.com/cyliustack/sofa
MongoDB World 2019: Don't Break the Camel's Back: Running MongoDB as Hard as ...MongoDB
Braze sends over a billion push notifications and emails each day as fast as possible. Each message requires multiple writes to MongoDB. In this talk, CTO Jon Hyman will discuss how Braze has built custom driver patches, monitoring and automated scaling systems to ensure maximal throughput.
This session brings to your attention how several millions of dollars are wasted and what you can do to save money. Optimizing garbage collection performance not only saves money, but also improves the overall customer experience as well.
PHP Frameworks: I want to break free (IPC Berlin 2024)Ralf Eggert
In this presentation, we examine the challenges and limitations of relying too heavily on PHP frameworks in web development. We discuss the history of PHP and its frameworks to understand how this dependence has evolved. The focus will be on providing concrete tips and strategies to reduce reliance on these frameworks, based on real-world examples and practical considerations. The goal is to equip developers with the skills and knowledge to create more flexible and future-proof web applications. We'll explore the importance of maintaining autonomy in a rapidly changing tech landscape and how to make informed decisions in PHP development.
This talk is aimed at encouraging a more independent approach to using PHP frameworks, moving towards a more flexible and future-proof approach to PHP development.
Essentials of Automations: Optimizing FME Workflows with ParametersSafe Software
Are you looking to streamline your workflows and boost your projects’ efficiency? Do you find yourself searching for ways to add flexibility and control over your FME workflows? If so, you’re in the right place.
Join us for an insightful dive into the world of FME parameters, a critical element in optimizing workflow efficiency. This webinar marks the beginning of our three-part “Essentials of Automation” series. This first webinar is designed to equip you with the knowledge and skills to utilize parameters effectively: enhancing the flexibility, maintainability, and user control of your FME projects.
Here’s what you’ll gain:
- Essentials of FME Parameters: Understand the pivotal role of parameters, including Reader/Writer, Transformer, User, and FME Flow categories. Discover how they are the key to unlocking automation and optimization within your workflows.
- Practical Applications in FME Form: Delve into key user parameter types including choice, connections, and file URLs. Allow users to control how a workflow runs, making your workflows more reusable. Learn to import values and deliver the best user experience for your workflows while enhancing accuracy.
- Optimization Strategies in FME Flow: Explore the creation and strategic deployment of parameters in FME Flow, including the use of deployment and geometry parameters, to maximize workflow efficiency.
- Pro Tips for Success: Gain insights on parameterizing connections and leveraging new features like Conditional Visibility for clarity and simplicity.
We’ll wrap up with a glimpse into future webinars, followed by a Q&A session to address your specific questions surrounding this topic.
Don’t miss this opportunity to elevate your FME expertise and drive your projects to new heights of efficiency.
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualityInflectra
In this insightful webinar, Inflectra explores how artificial intelligence (AI) is transforming software development and testing. Discover how AI-powered tools are revolutionizing every stage of the software development lifecycle (SDLC), from design and prototyping to testing, deployment, and monitoring.
Learn about:
• The Future of Testing: How AI is shifting testing towards verification, analysis, and higher-level skills, while reducing repetitive tasks.
• Test Automation: How AI-powered test case generation, optimization, and self-healing tests are making testing more efficient and effective.
• Visual Testing: Explore the emerging capabilities of AI in visual testing and how it's set to revolutionize UI verification.
• Inflectra's AI Solutions: See demonstrations of Inflectra's cutting-edge AI tools like the ChatGPT plugin and Azure Open AI platform, designed to streamline your testing process.
Whether you're a developer, tester, or QA professional, this webinar will give you valuable insights into how AI is shaping the future of software delivery.
Connector Corner: Automate dynamic content and events by pushing a buttonDianaGray10
Here is something new! In our next Connector Corner webinar, we will demonstrate how you can use a single workflow to:
Create a campaign using Mailchimp with merge tags/fields
Send an interactive Slack channel message (using buttons)
Have the message received by managers and peers along with a test email for review
But there’s more:
In a second workflow supporting the same use case, you’ll see:
Your campaign sent to target colleagues for approval
If the “Approve” button is clicked, a Jira/Zendesk ticket is created for the marketing design team
But—if the “Reject” button is pushed, colleagues will be alerted via Slack message
Join us to learn more about this new, human-in-the-loop capability, brought to you by Integration Service connectors.
And...
Speakers:
Akshay Agnihotri, Product Manager
Charlie Greenberg, Host
Search and Society: Reimagining Information Access for Radical FuturesBhaskar Mitra
The field of Information retrieval (IR) is currently undergoing a transformative shift, at least partly due to the emerging applications of generative AI to information access. In this talk, we will deliberate on the sociotechnical implications of generative AI for information access. We will argue that there is both a critical necessity and an exciting opportunity for the IR community to re-center our research agendas on societal needs while dismantling the artificial separation between the work on fairness, accountability, transparency, and ethics in IR and the rest of IR research. Instead of adopting a reactionary strategy of trying to mitigate potential social harms from emerging technologies, the community should aim to proactively set the research agenda for the kinds of systems we should build inspired by diverse explicitly stated sociotechnical imaginaries. The sociotechnical imaginaries that underpin the design and development of information access technologies needs to be explicitly articulated, and we need to develop theories of change in context of these diverse perspectives. Our guiding future imaginaries must be informed by other academic fields, such as democratic theory and critical theory, and should be co-developed with social science scholars, legal scholars, civil rights and social justice activists, and artists, among others.
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
UiPath Test Automation using UiPath Test Suite series, part 3DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 3. In this session, we will cover desktop automation along with UI automation.
Topics covered:
UI automation Introduction,
UI automation Sample
Desktop automation flow
Pradeep Chinnala, Senior Consultant Automation Developer @WonderBotz and UiPath MVP
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
JMeter webinar - integration with InfluxDB and GrafanaRTTS
Watch this recorded webinar about real-time monitoring of application performance. See how to integrate Apache JMeter, the open-source leader in performance testing, with InfluxDB, the open-source time-series database, and Grafana, the open-source analytics and visualization application.
In this webinar, we will review the benefits of leveraging InfluxDB and Grafana when executing load tests and demonstrate how these tools are used to visualize performance metrics.
Length: 30 minutes
Session Overview
-------------------------------------------
During this webinar, we will cover the following topics while demonstrating the integrations of JMeter, InfluxDB and Grafana:
- What out-of-the-box solutions are available for real-time monitoring JMeter tests?
- What are the benefits of integrating InfluxDB and Grafana into the load testing stack?
- Which features are provided by Grafana?
- Demonstration of InfluxDB and Grafana using a practice web application
To view the webinar recording, go to:
https://www.rttsweb.com/jmeter-integration-webinar
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
Keynote at DIGIT West Expo, Glasgow on 29 May 2024.
Cheryl Hung, ochery.com
Sr Director, Infrastructure Ecosystem, Arm.
The key trends across hardware, cloud and open-source; exploring how these areas are likely to mature and develop over the short and long-term, and then considering how organisations can position themselves to adapt and thrive.
Let's dive deeper into the world of ODC! Ricardo Alves (OutSystems) will join us to tell all about the new Data Fabric. After that, Sezen de Bruijn (OutSystems) will get into the details on how to best design a sturdy architecture within ODC.
3. Key Performance Indicators
1. Latency 2. Throughput
99.925%
3. Foot Print
Memory: 2GB
CPU: 30%
GC event’s pause time
Percentage of time spent in processing customer
transactions vs time spent in GC activity.
i.e. productive work vs non-productive work
Memory and CPU utilization of
the application
8. CMS Log formatBefore GC:
Statistics for BinaryTreeDictionary:
------------------------------------
Total Free Space: 2524251
Max Chunk Size: 2519552
Number of Blocks: 13
Av. Block Size: 194173
Tree Height: 8
2016-05-03T04:27:37.503+0000: 30282.678: [ParNew
Desired survivor size 214728704 bytes, new threshold 1 (max 1)
- age 1: 85782640 bytes, 85782640 total
: 3510063K->100856K(3774912K), 0.0516290 secs] 9371816K->6022161K(14260672K)After GC:
Statistics for BinaryTreeDictionary:
------------------------------------
Total Free Space: 530579346
Max Chunk Size: 332576815
Number of Blocks: 7178
Av. Block Size: 73917
Tree Height: 44
After GC:
Statistics for BinaryTreeDictionary:
------------------------------------
Total Free Space: 2524251
Max Chunk Size: 2519552
Number of Blocks: 13
Av. Block Size: 194173
Tree Height: 8
, 0.0552970 secs] [Times: user=0.67 sys=0.00, real=0.06 secs]
18. How to diagnose memory leak
1. Capture Heap Dumps:
a. jmap -dump:live,file=<file-path> <pid>
Example: jmap -dump:live,file=/opt/tmp/AddressBook-heapdump.bin 37320
b. -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/logs/heapdump
2. Best tool to analyze heap dumps: Eclipse MAT
19. Eclipse MAT – Best Practices
1. Use stand-alone version (not plugin version)
2. Increase heap size:
-startup
plugins/org.eclipse.equinox.launcher_1.3.0.v20140415-2008.jar
--launcher.library
plugins/org.eclipse.equinox.launcher.win32.win32.x86_64_1.1.200.v20140603-1326
-vmargs
-Xmx3g
3. Enable ‘keep unreachable objects’:
1. Go to Window > Preferences …
2. Click on ‘Memory Analyzer’
3. Select ‘Keep unreachable objects’
4. Click on ‘OK’ button
20. Reduce GC pause time
https://blog.gceasy.io/2016/11/22/reduce-long-gc-pauses/
21. 1. High Object creation rate
Best Practice: Compare this metric between releases in Performance lab
24. 3. Choice of GC Algorithm
• Serial
• Parallel
• CMS
• G1 GC
• Shenandoah
25. GC Time
• Real is wall clock time – time from start to finish of the call. This is all elapsed
time including time slices used by other processes and time the process spends
blocked (for example if it is waiting for I/O to complete).
• Sys is the amount of CPU time spent in the kernel within the process. This means
executing CPU time spent in system calls within the kernel, as opposed to library
code, which is still running in user-space. Like ‘user’, this is only CPU time used by
the process.
• User is the amount of CPU time spent in user-mode code (outside the kernel)
within the process. This is only actual CPU time used in executing the process.
Other processes and time the process spends blocked do not count towards this
figure.
• User+Sys will tell you how much actual CPU time your process used. Note that
this is across all CPUs, so if the process has multiple threads it could potentially
exceed the wall clock time reported by Real.
[Times: user=11.53 sys=1.38, real=1.03 secs]
27. 4. Process Swapping
• Sometimes due to lack of memory (RAM), Operating system could be
swapping your application from memory.
• Below script will show all the process that are being swapped:
https://blog.gceasy.io/2016/11/22/reduce-long-gc-pauses/
29. 5. Background IO Traffic
• If there is a heavy file system I/O activity (i.e. lot of reads and writes
are happening) it can also cause long GC pauses.
30. 6. Less GC Threads
• WARNING: Adding too many GC threads will consume a lot of CPU
and takes away a resource from your application. Thus you need to
conduct thorough testing before increasing the GC thread count.
31. 7. System.gc() calls
• When System.gc() or Runtime.getRuntime().gc() calls causes stop-the-
world Full GCs.
• What triggers System.gc() calls
• Your own application
• 3rd party libraries, frameworks, sometimes even application servers that you
use could be invoking System.gc() method.
• External tools (like VisualVM) through use of JMX
• RMI
– Dsun.rmi.dgc.server.gcInterval=n
– Dsun.rmi.dgc.client.gcInterval=n
• Can be disabled by -XX:+DisableExplicitGC