Modern Linux Performance Tools for Application Troubleshooting

•

3 likes•1,290 views

Modern Linux Performance Tools for Application Troubleshooting. Mostly demos and focused on application/process troubleshooting, not systemwide summaries.

Technology

gluent.com 1
Modern Linux Performance Tools
Tanel Poder
a long time computer performance geek

gluent.com 2
Intro: About me
• Tanel Põder
• Oracle Database Performance geek (18+ years)
• Exadata Performance geek
• Linux Performance geek
• Hadoop Performance geek
• CEO & co-founder:
Expert Oracle Exadata
book
(2nd edition is out now!)
Instant
promotion

gluent.com 3
Gluent
Oracle
Teradata
NoSQL
Big Data
Sources
MSSQL
App
X
App
Y
App
Z
Gluent as a data
virtualization layer
Open Data
Formats!
Session:
Extending Oracle
Data Warehouses
with Hadoop
Wednesday 10am
Executive Room 6
(small room!)

gluent.com 4
Topics
• Not much about basic stuff like vmstat/iostat
• Also nothing about SystemTap (or Dtrace)
• Practical stuff that you can (relatively) safely run in production
• Measuring / tracing processes
• Measuring IO
• Measuring Memory
• Measuring CPU

gluent.com 5
Measuring Process Activity - 1
• Strace
• man strace
• strace -p <PID>
• https://fritshoogland.files.wordpress.com/2013/03/profiling_of_oracl
e_using_function_calls.pdf
• SysDig
• sysdig fd.name contains tnsnames
• sysdig -p "%evt.arg.name" proc.name=cat and
evt.type=open
• csysdig
• http://www.sysdig.org/wiki/sysdig-examples/

gluent.com 6
Measuring Process Activity - 2
• /proc filesystem
• Proc filesystem in Linux is like V$/X$ views in Oracle
• man proc
• https://www.kernel.org/doc/Documentation/filesystems/proc.txt
• http://blog.tanelpoder.com/2013/02/21/peeking-into-linux-kernel-
land-using-proc-filesystem-for-quickndirty-troubleshooting/
• psnapper (early prototype!)
• ./psnapper <seconds> <pid>
• ./psnapper 5 1234
• http://blog.tanelpoder.com/files/scripts/tools/unix/psnapper

gluent.com 7
Measuring IO
• iostat in RHEL 7
• /proc/<PID>/io
• iotop (or htop, atop)
• blktrace / blkparse (if we have time)
• blktrace -d /dev/sda -o - | blkparse -i -

gluent.com 8
Measuring Memory
• pmap -x <PID>
• http://tech.e2sn.com/oracle/performance/unix-performance-
tools/process-memory-matrix (Solaris example but roughly the same)
• cat /proc/PID/smaps
• grep MMUPageSize: /proc/`pgrep dbw0`/smaps |
sort | uniq –c
• https://www.kernel.org/doc/Documentation/filesystems/proc.txt
• Smem
• sudo smem –u
• https://www.selenic.com/smem/

gluent.com 9
Measuring CPU
• perf top
• https://fritshoogland.wordpress.com/2012/07/10/the-linux-perf-
utility-with-el-6/
• perf record –g && perf report
• http://www.brendangregg.com/perf.html
• http://externaltable.blogspot.jp/2014/05/flame-graphs-for-
oracle.html
• perf stat
• http://blog.tanelpoder.com/2015/08/09/ram-is-the-new-disk-and-
how-to-measure-its-performance-part-1/

gluent.com 10
Linux performance references
• Brendan Gregg:
• http://www.brendangregg.com/linuxperf.html
• Luca Canali’s articles:
• https://db-blog.web.cern.ch/blog/luca-canali/2015-12-extended-
stack-profiling-ideas-tools-and-comments
• Frits Hoogland:
• https://fritshoogland.wordpress.com/

gluent.com 11
Thanks!
http://blog.tanelpoder.com
tanel@tanelpoder.com
@tanelpoder
We are hiring developers & data engineers!!!
http://gluent.com

Tanel Poder has been involved in a number of Exadata migration projects since its introduction, mostly in the area of performance ensurance, troubleshooting and capacity planning. These slides, originally presented at UKOUG in 2010, cover some of the most interesting challenges, surprises and lessons learnt from planning and executing large Oracle database migrations to Exadata v2 platform. This material is not just repeating the marketing material or Oracle's official whitepapers.

Troubleshooting Complex Performance issues - Oracle SEG$ contention

Tanel Poder

Perhaps you have heard the term “In-Memory” but not sure what it means. If you are a SQL Server Professional then you will want to know. Even if you are new to SQL Server, you will want to learn more about this topic. Come learn the basics of how In-Memory OLTP technology in SQL Server 2016 and Azure SQL Database can boost your OLTP application by 30X. We will compare how In-Memory OTLP works vs “normal” disk-based tables. We will discuss what is required to migrate your existing data into memory optimized tables or how to build a new set of data and applications to take advantage of this technology. This presentation will cover the fundamentals of what, how, and why this technology is something every SQL Server Professional should know

Habits of Effective Sqoop Users

Kathleen Ting

January 2015 HUG: Apache Flink: Fast and reliable large-scale data processing

Yahoo Developer Network

Apache Flink (incubating) is one of the latest addition to the Apache family of data processing engines. In short, Flink’s design aims to be as fast as in-memory engines, while providing the reliability of Hadoop. Flink contains (1) APIs in Java and Scala for both batch-processing and data streaming applications, (2) a translation stack for transforming these programs to parallel data flows and (3) a runtime that supports both proper streaming and batch processing for executing these data flows in large compute clusters. Flink’s batch APIs build on functional primitives (map, reduce, join, cogroup, etc), and augment those with dedicated operators for iterative algorithms, and support for logical, SQL-like key attribute referencing (e.g., groupBy(“WordCount.word”). The Flink streaming API extends the primitives from the batch API with flexible window semantics. Internally, Flink transforms the user programs into distributed data stream programs. In the course of the transformation, Flink analyzes functions and data types (using Scala macros and reflection), and picks physical execution strategies using a cost-based optimizer. Flink’s runtime is a true streaming engine, supporting both batching and streaming. Flink operates on a serialized data representation with memory-adaptive out-of-core algorithms for sorting and hashing. This makes Flink match the performance of in-memory engines on memory-resident datasets, while scaling robustly to larger disk-resident datasets. Finally, Flink is compatible with the Hadoop ecosystem. Flink runs on YARN, reads data from HDFS and HBase, and supports mixing existing Hadoop Map and Reduce functions into Flink programs. Ongoing work is adding Apache Tez as an additional runtime backend. This talk presents Flink from a user perspective. We introduce the APIs and highlight the most interesting design points behind Flink, discussing how they contribute to the goals of performance, robustness, and flexibility. We finally give an outlook on Flink’s development roadmap.

Empowering developers to deploy their own data stores

Tomas Doran

Proving out flash storage array performance using swingbench and slob

Kapil Goyal

Top 5 Mistakes to Avoid When Writing Apache Spark Applications

Cloudera, Inc.

Accelerating Shuffle: A Tailor-Made RDMA Solution for Apache Spark with Yuval...

Spark Summit

The opportunity in accelerating Spark by improving its network data transfer facilities has been under much debate in the last few years. RDMA (remote direct memory access) is a network acceleration technology that is very prominent in the HPC (high-performance computing) world, but has not yet made its way to mainstream Apache Spark. Proper implementation of RDMA in network-oriented applications can improve scalability, throughput, latency and CPU utilization. In this talk we are going to present a new RDMA solution for Apache Spark that shows amazing improvements in multiple Spark use cases. The solution is under development in our labs, and is going to be released to the public as an open-source plug-in.

Loading 350M documents into a large Solr cluster: Presented by Dion Olsthoorn...

Lucidworks

ELK Ruminating on Logs (Zendcon 2016)

Mathew Beane

We're talking about serious log crunching and intelligence gathering with Elastic, Logstash, and Kibana. ELK is an end-to-end stack for gathering structured and unstructured data from servers. It delivers insights in real time using the Kibana dashboard giving unprecedented horizontal visibility. The visualization and search tools will make your day-to-day hunting a breeze. During this brief walkthrough of the setup, configuration, and use of the toolset, we will show you how to find the trees from the forest in today's modern cloud environments and beyond.

Lessons PostgreSQL learned from commercial databases, and didn’t

PGConf APAC

Big data 101 for beginners devoxxpl

Duyhai Doan

Oracle Database In-Memory Option in Action

Tanel Poder

Tanel Poder - Troubleshooting Complex Oracle Performance Issues - Part 1

Tanel Poder

What's hot

Moving Data Between Exadata and HadoopEnkitec

Hadoop databases for oracle DBAs

Maxym Kharchenko

Tuning Solr and its Pipeline for Logs: Presented by Rafał Kuć & Radu Gheorghe...

Lucidworks

Inside sql server in memory oltp sql sat nyc 2017

Bob Ward

Advanced Oracle Troubleshooting

Hector Martinez

Big Data and PostgreSQL

PGConf APAC

PostgreSQL WAL for DBAs

PGConf APAC

Presto anatomy

Dongmin Yu

PostgreSQL Enterprise Class Features and Capabilities

PGConf APAC

SQL Server In-Memory OLTP: What Every SQL Professional Should Know

Bob Ward

Habits of Effective Sqoop Users

Kathleen Ting

January 2015 HUG: Apache Flink: Fast and reliable large-scale data processing

Yahoo Developer Network

Empowering developers to deploy their own data stores

Tomas Doran

Proving out flash storage array performance using swingbench and slob

Kapil Goyal

Top 5 Mistakes to Avoid When Writing Apache Spark Applications

Cloudera, Inc.

Accelerating Shuffle: A Tailor-Made RDMA Solution for Apache Spark with Yuval...

Spark Summit

Loading 350M documents into a large Solr cluster: Presented by Dion Olsthoorn...

Lucidworks

ELK Ruminating on Logs (Zendcon 2016)

Mathew Beane

Lessons PostgreSQL learned from commercial databases, and didn’t

PGConf APAC

Big data 101 for beginners devoxxpl

Duyhai Doan

What's hot (20)

Moving Data Between Exadata and Hadoop

Hadoop databases for oracle DBAs

Tuning Solr and its Pipeline for Logs: Presented by Rafał Kuć & Radu Gheorghe...

Inside sql server in memory oltp sql sat nyc 2017

Advanced Oracle Troubleshooting

Big Data and PostgreSQL

PostgreSQL WAL for DBAs

Presto anatomy

PostgreSQL Enterprise Class Features and Capabilities

SQL Server In-Memory OLTP: What Every SQL Professional Should Know

Habits of Effective Sqoop Users

January 2015 HUG: Apache Flink: Fast and reliable large-scale data processing

Empowering developers to deploy their own data stores

Proving out flash storage array performance using swingbench and slob

Top 5 Mistakes to Avoid When Writing Apache Spark Applications

Accelerating Shuffle: A Tailor-Made RDMA Solution for Apache Spark with Yuval...

Loading 350M documents into a large Solr cluster: Presented by Dion Olsthoorn...

ELK Ruminating on Logs (Zendcon 2016)

Lessons PostgreSQL learned from commercial databases, and didn’t

Big data 101 for beginners devoxxpl

Viewers also liked

Oracle Database In-Memory Option in Action

Tanel Poder

Tanel Poder - Troubleshooting Complex Oracle Performance Issues - Part 1

Tanel Poder

Tanel Poder - Troubleshooting Complex Oracle Performance Issues - Part 2

Tanel Poder

SQL in the Hybrid World

Tanel Poder

SQL Monitoring in Oracle Database 12c

Tanel Poder

Connecting Hadoop and Oracle

Tanel Poder

Essential Linux Commands for DBAs

Gokhan Atil

A Second Look at Oracle RAC 12c

Leighton Nelson

Oracle RAC 12c has been touted as the best release so far and with reason. There have been significant enhancements to scalability and high availability with features such as Flex Clusters, Flex ASM, Application Continuity and Transaction Guard to name a few. While these cool features grab the headlines, there are others that are not highlighted but can make significant impact on DBA productivity. In this session we will take a second look at some of these features, including operational support enhancements to srvctl, crsctl commands, ADR support for Grid Infrastructure and tools such orachk and tfa. We will also explore some of the new functionality introduced in 12.1.0.2.0.

Oracle LOB Internals and Performance Tuning

Tanel Poder

Oracle Latch and Mutex Contention Troubleshooting

Tanel Poder

Oracle Enterprise Manager Cloud Control 13c for DBAs

Gokhan Atil

Adding real time reporting to your database oracle db in memory

Zohar Elkayam

201657_Patra_pdflokinisj

Hanganalyze presentation

Leyi (Kamus) Zhang

Oracle 12c r1 installation on solaris 11.1Laurent Leturgez

Applying profilers to my sql (fosdem 2017)

Valeriy Kravchuk

Ukoug15 SIMD outside and inside Oracle 12c (12.1.0.2)

Laurent Leturgez

Oracle Database Advanced Querying (2016)

Zohar Elkayam

This is a presentation from Oracle Week 2016 (Israel). This is a newer version from last year with new 12cR2 features and demo. In the agenda: Aggregative and advanced grouping options Analytic functions, ranking and pagination Hierarchical and recursive queries Regular Expressions Oracle 12c new rows pattern matching XML and JSON handling with SQL Oracle 12c (12.1 + 12.2) new features SQL Developer Command Line tool

Delphix Patching Epiphany

Kellyn Pot'Vin-Gorman

Oracle 12c in memory en action

Laurent Leturgez

Viewers also liked (20)

Oracle Database In-Memory Option in Action

Tanel Poder - Troubleshooting Complex Oracle Performance Issues - Part 1

Tanel Poder - Troubleshooting Complex Oracle Performance Issues - Part 2

SQL in the Hybrid World

SQL Monitoring in Oracle Database 12c

Connecting Hadoop and Oracle

Essential Linux Commands for DBAs

A Second Look at Oracle RAC 12c

Oracle LOB Internals and Performance Tuning

Oracle Latch and Mutex Contention Troubleshooting

Oracle Enterprise Manager Cloud Control 13c for DBAs

Adding real time reporting to your database oracle db in memory

201657_Patra_pdf

Hanganalyze presentation

Oracle 12c r1 installation on solaris 11.1

Applying profilers to my sql (fosdem 2017)

Ukoug15 SIMD outside and inside Oracle 12c (12.1.0.2)

Oracle Database Advanced Querying (2016)

Delphix Patching Epiphany

Oracle 12c in memory en action

Similar to Modern Linux Performance Tools for Application Troubleshooting

Planning for CRAP and entity revisions in Drupal core

Dick Olsson

This is a follow-up on the core conversations in Los Angeles that recived lots of positive feedback when suggesting improvements to the Entity Revision API in core. In this session we will lay out a more concrete and detailed plan of how we can introduce these improvements in Drupal 8.2.x or 9.x. Short background on the topic CRAP stands for Create Read Archive Purge which implies that all changes to an entity creates a new revision, even a delete operation is a new revision (much like Git does it). This creates a system much more capable of managing complex workflows, concurrent editing, distributed content, content staging, audit trails etc.

Systems Performance: Enterprise and the Cloud

Brendan Gregg

My talk for BayLISA, Oct 2013, launching the Systems Performance book. Operating system performance analysis and tuning leads to a better end-user experience and lower costs, especially for cloud computing environments that pay by the operating system instance. This book covers concepts, strategy, tools and tuning for Unix operating systems, with a focus on Linux- and Solaris-based systems. The book covers the latest tools and techniques, including static and dynamic tracing, to get the most out of your systems.

Software Engineering in Startups

Dusan Omercevic

Py tablesAli Hallaji

Large Data Analyze With PyTables

Innfinision Cloud and BigData Solutions

PyTablesAli Hallaji

Monitoring Big Data Systems Done "The Simple Way" - Codemotion Berlin 2017

Demi Ben-Ari

Once you start working with distributed Big Data systems, you start discovering a whole bunch of problems you won’t find in monolithic systems. All of a sudden to monitor all of the components becomes a big data problem itself. In the talk we’ll mention all of the aspects that you should take in consideration when monitoring a distributed system once you’re using tools like: Web Services, Apache Spark, Cassandra, MongoDB, Amazon Web Services. Not only the tools, what should you monitor about the actual data that flows in the system? And we’ll cover the simplest solution with your day to day open source tools, the surprising thing, that it comes not from an Ops Guy.

Silicon Valley Code Camp 2016 - MongoDB in production

Daniel Coupal

PyTables

Ali Hallaji

Hot to build continuously processing for 24/7 real-time data streaming platform?

GetInData

Taboola Road To Scale With Apache Sparktsliwowicz

Multi dimension aggregations using spark and dataframes

Romi Kuntsman

John adams talk cloudy

John Adams

Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017 - D...

Demi Ben-Ari

Demi Ben-Ari - Monitoring Big Data Systems Done "The Simple Way" - Codemotion...

Codemotion

Once you start working with Big Data systems, you discover a whole bunch of problems you won’t find in monolithic systems. Monitoring all of the components becomes a big data problem itself. In the talk we’ll mention all of the aspects that you should take in consideration when monitoring a distributed system using tools like: Web Services,Spark,Cassandra,MongoDB,AWS. Not only the tools, what should you monitor about the actual data that flows in the system? We’ll cover the simplest solution with your day to day open source tools, the surprising thing, that it comes not from an Ops Guy.

Monitoring Big Data Systems Done "The Simple Way" - Demi Ben-Ari - Codemotion...

Codemotion

Once you start working with Big Data systems, you discover a whole bunch of problems you won’t find in monolithic systems. Monitoring all of the components becomes a big data problem itself. In the talk, we’ll mention all of the aspects that you should take into consideration when monitoring a distributed system using tools like Web Services, Spark, Cassandra, MongoDB, AWS. Not only the tools, what should you monitor about the actual data that flows in the system? We’ll cover the simplest solution with your day to day open source tools, the surprising thing, that it comes not from an Ops Guy.

Monitoring Big Data Systems "Done the simple way" - Demi Ben-Ari - Codemotion...

Demi Ben-Ari

Hadoop Operations: Keeping the Elephant Running Smoothly

Michael Arnold

Pune Hadoop Admins Meetup From its beginnings years ago at large Internet sites, Hadoop is spreading everywhere. There are multitudes of cool and interesting things that Hadoop allows your organization to do, but running the actual infrastructure may not be as sexy as the application(s) running on top. Operations can be pure grunt-work, exacerbated by the fact that there is usually one machine out of dozens (or more) that is throwing a wrench in the works. In this talk, I will cover my experiences of running Hadoop, provide some recommended practices to simplify your days and nights in the trenches, and highlight some of the lessons learned along the way.

unix_linux_ORATOP_TechDays2016_presentations

garosgaros

Taboola's experience with Apache Spark (presentation @ Reversim 2014)

tsliwowicz

At taboola we are getting a constant feed of data (many billions of user events a day) and are using Apache Spark together with Cassandra for both real time data stream processing as well as offline data processing. We'd like to share our experience with these cutting edge technologies. Apache Spark is an open source project - Hadoop-compatible computing engine that makes big data analysis drastically faster, through in-memory computing, and simpler to write, through easy APIs in Java, Scala and Python. This project was born as part of a PHD work in UC Berkley's AMPLab (part of the BDAS - pronounced "Bad Ass") and turned into an incubating Apache project with more active contributors than Hadoop. Surprisingly, Yahoo! are one of the biggest contributors to the project and already have large production clusters of Spark on YARN. Spark can run either standalone cluster, or using either Apache mesos and ZooKeeper or YARN and can run side by side with Hadoop/Hive on the same data. One of the biggest benefits of Spark is that the API is very simple and the same analytics code can be used for both streaming data and offline data processing.

Similar to Modern Linux Performance Tools for Application Troubleshooting (20)

Planning for CRAP and entity revisions in Drupal core

Systems Performance: Enterprise and the Cloud

Software Engineering in Startups

Py tables

Large Data Analyze With PyTables

PyTables

Monitoring Big Data Systems Done "The Simple Way" - Codemotion Berlin 2017

Silicon Valley Code Camp 2016 - MongoDB in production

PyTables

Hot to build continuously processing for 24/7 real-time data streaming platform?

Taboola Road To Scale With Apache Spark

Multi dimension aggregations using spark and dataframes

John adams talk cloudy

Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017 - D...

Demi Ben-Ari - Monitoring Big Data Systems Done "The Simple Way" - Codemotion...

Monitoring Big Data Systems Done "The Simple Way" - Demi Ben-Ari - Codemotion...

Monitoring Big Data Systems "Done the simple way" - Demi Ben-Ari - Codemotion...

Hadoop Operations: Keeping the Elephant Running Smoothly

unix_linux_ORATOP_TechDays2016_presentations

Taboola's experience with Apache Spark (presentation @ Reversim 2014)

Recently uploaded

UiPath Test Automation using UiPath Test Suite series, part 6

DianaGray10

Welcome to UiPath Test Automation using UiPath Test Suite series part 6. In this session, we will cover Test Automation with generative AI and Open AI. UiPath Test Automation with generative AI and Open AI webinar offers an in-depth exploration of leveraging cutting-edge technologies for test automation within the UiPath platform. Attendees will delve into the integration of generative AI, a test automation solution, with Open AI advanced natural language processing capabilities. Throughout the session, participants will discover how this synergy empowers testers to automate repetitive tasks, enhance testing accuracy, and expedite the software testing life cycle. Topics covered include the seamless integration process, practical use cases, and the benefits of harnessing AI-driven automation for UiPath testing initiatives. By attending this webinar, testers, and automation professionals can gain valuable insights into harnessing the power of AI to optimize their test automation workflows within the UiPath ecosystem, ultimately driving efficiency and quality in software development processes. What will you get from this session? 1. Insights into integrating generative AI. 2. Understanding how this integration enhances test automation within the UiPath platform 3. Practical demonstrations 4. Exploration of real-world use cases illustrating the benefits of AI-driven test automation for UiPath Topics covered: What is generative AI Test Automation with generative AI and Open AI. UiPath integration with generative AI Speaker: Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP

Climate Impact of Software Testing at Nordic Testing Days

Kari Kakkonen

My slides at Nordic Testing Days 6.6.2024 Climate impact / sustainability of software testing discussed on the talk. ICT and testing must carry their part of global responsibility to help with the climat warming. We can minimize the carbon footprint but we can also have a carbon handprint, a positive impact on the climate. Quality characteristics can be added with sustainability, and then measured continuously. Test environments can be used less, and in smaller scale and on demand. Test techniques can be used in optimizing or minimizing number of tests. Test automation can be used to speed up testing.

GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...

Neo4j

Dr. Sean Tan, Head of Data Science, Changi Airport Group Discover how Changi Airport Group (CAG) leverages graph technologies and generative AI to revolutionize their search capabilities. This session delves into the unique search needs of CAG’s diverse passengers and customers, showcasing how graph data structures enhance the accuracy and relevance of AI-generated search results, mitigating the risk of “hallucinations” and improving the overall customer journey.

Epistemic Interaction - tuning interfaces to provide information for AI support

Alan Dix

Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024 https://alandix.com/academic/papers/synergy2024-epistemic/ As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.

GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024

Neo4j

Free Complete Python - A step towards Data Science

RinaMondal9

FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf

FIDO Alliance

A tale of scale & speed: How the US Navy is enabling software delivery from l...

sonjaschweigert1

Rapid and secure feature delivery is a goal across every application team and every branch of the DoD. The Navy’s DevSecOps platform, Party Barge, has achieved: - Reduction in onboarding time from 5 weeks to 1 day - Improved developer experience and productivity through actionable findings and reduction of false positives - Maintenance of superior security standards and inherent policy enforcement with Authorization to Operate (ATO) Development teams can ship efficiently and ensure applications are cyber ready for Navy Authorizing Officials (AOs). In this webinar, Sigma Defense and Anchore will give attendees a look behind the scenes and demo secure pipeline automation and security artifacts that speed up application ATO and time to production. We will cover: - How to remove silos in DevSecOps - How to build efficient development pipeline roles and component templates - How to deliver security artifacts that matter for ATO’s (SBOMs, vulnerability reports, and policy evidence) - How to streamline operations with automated policy checks on container images

zkStudyClub - Reef: Fast Succinct Non-Interactive Zero-Knowledge Regex Proofs

Alex Pruden

This paper presents Reef, a system for generating publicly verifiable succinct non-interactive zero-knowledge proofs that a committed document matches or does not match a regular expression. We describe applications such as proving the strength of passwords, the provenance of email despite redactions, the validity of oblivious DNS queries, and the existence of mutations in DNA. Reef supports the Perl Compatible Regular Expression syntax, including wildcards, alternation, ranges, capture groups, Kleene star, negations, and lookarounds. Reef introduces a new type of automata, Skipping Alternating Finite Automata (SAFA), that skips irrelevant parts of a document when producing proofs without undermining soundness, and instantiates SAFA with a lookup argument. Our experimental evaluation confirms that Reef can generate proofs for documents with 32M characters; the proofs are small and cheap to verify (under a second). Paper: https://eprint.iacr.org/2023/1886

How to Get CNIC Information System with Paksim Ga.pptx

danishmna97

The Art of the Pitch: WordPress Relationships and Sales

Laura Byrne

Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes? All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.

Introduction to CHERI technology - Cybersecurity

mikeeftimakis1

Communications Mining Series - Zero to Hero - Session 1

DianaGray10

This session provides introduction to UiPath Communication Mining, importance and platform overview. You will acquire a good understand of the phases in Communication Mining as we go over the platform with you. Topics covered: • Communication Mining Overview • Why is it important? • How can it help today’s business and the benefits • Phases in Communication Mining • Demo on Platform overview • Q/A

Mind map of terminologies used in context of Generative AI

Kumud Singh

Enchancing adoption of Open Source Libraries. A case study on Albumentations.AI

Vladimir Iglovikov, Ph.D.

Presented by Vladimir Iglovikov: - https://www.linkedin.com/in/iglovikov/ - https://x.com/viglovikov - https://www.instagram.com/ternaus/ This presentation delves into the journey of Albumentations.ai, a highly successful open-source library for data augmentation. Created out of a necessity for superior performance in Kaggle competitions, Albumentations has grown to become a widely used tool among data scientists and machine learning practitioners. This case study covers various aspects, including: People: The contributors and community that have supported Albumentations. Metrics: The success indicators such as downloads, daily active users, GitHub stars, and financial contributions. Challenges: The hurdles in monetizing open-source projects and measuring user engagement. Development Practices: Best practices for creating, maintaining, and scaling open-source libraries, including code hygiene, CI/CD, and fast iteration. Community Building: Strategies for making adoption easy, iterating quickly, and fostering a vibrant, engaged community. Marketing: Both online and offline marketing tactics, focusing on real, impactful interactions and collaborations. Mental Health: Maintaining balance and not feeling pressured by user demands. Key insights include the importance of automation, making the adoption process seamless, and leveraging offline interactions for marketing. The presentation also emphasizes the need for continuous small improvements and building a friendly, inclusive community that contributes to the project's growth. Vladimir Iglovikov brings his extensive experience as a Kaggle Grandmaster, ex-Staff ML Engineer at Lyft, sharing valuable lessons and practical advice for anyone looking to enhance the adoption of their open-source projects. Explore more about Albumentations and join the community at: GitHub: https://github.com/albumentations-team/albumentations Website: https://albumentations.ai/ LinkedIn: https://www.linkedin.com/company/100504475 Twitter: https://x.com/albumentations

GraphRAG is All You need? LLM & Knowledge Graph

Guy Korland

Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs. 1. Unifying Large Language Models and Knowledge Graphs: A Roadmap. https://arxiv.org/abs/2306.08302 2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs: https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/

GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024

Neo4j

Securing your Kubernetes cluster_ a step-by-step guide to success !

KatiaHIMEUR1

Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster. However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks. In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.

LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...

DanBrown980551

Do you want to learn how to model and simulate an electrical network from scratch in under an hour? Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)! During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook. PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides: - A fully editable and extendable library for grid component modelling; - Visualization tools to display your network; - Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses; The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well. What you will learn during the webinar: - For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills; - For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.

Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...

SOFTTECHHUB

The choice of an operating system plays a pivotal role in shaping our computing experience. For decades, Microsoft's Windows has dominated the market, offering a familiar and widely adopted platform for personal and professional use. However, as technological advancements continue to push the boundaries of innovation, alternative operating systems have emerged, challenging the status quo and offering users a fresh perspective on computing. One such alternative that has garnered significant attention and acclaim is Nitrux Linux 3.5.0, a sleek, powerful, and user-friendly Linux distribution that promises to redefine the way we interact with our devices. With its focus on performance, security, and customization, Nitrux Linux presents a compelling case for those seeking to break free from the constraints of proprietary software and embrace the freedom and flexibility of open-source computing.

Recently uploaded (20)

UiPath Test Automation using UiPath Test Suite series, part 6

Climate Impact of Software Testing at Nordic Testing Days

GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...

Epistemic Interaction - tuning interfaces to provide information for AI support

GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024

Free Complete Python - A step towards Data Science

FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf

A tale of scale & speed: How the US Navy is enabling software delivery from l...

zkStudyClub - Reef: Fast Succinct Non-Interactive Zero-Knowledge Regex Proofs

How to Get CNIC Information System with Paksim Ga.pptx

The Art of the Pitch: WordPress Relationships and Sales

Introduction to CHERI technology - Cybersecurity

Communications Mining Series - Zero to Hero - Session 1

Mind map of terminologies used in context of Generative AI

Enchancing adoption of Open Source Libraries. A case study on Albumentations.AI

GraphRAG is All You need? LLM & Knowledge Graph

GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024

Securing your Kubernetes cluster_ a step-by-step guide to success !

LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...

Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...

Modern Linux Performance Tools for Application Troubleshooting

1. gluent.com 1 Modern Linux Performance Tools Tanel Poder a long time computer performance geek

2. gluent.com 2 Intro: About me • Tanel Põder • Oracle Database Performance geek (18+ years) • Exadata Performance geek • Linux Performance geek • Hadoop Performance geek • CEO & co-founder: Expert Oracle Exadata book (2nd edition is out now!) Instant promotion

3. gluent.com 3 Gluent Oracle Teradata NoSQL Big Data Sources MSSQL App X App Y App Z Gluent as a data virtualization layer Open Data Formats! Session: Extending Oracle Data Warehouses with Hadoop Wednesday 10am Executive Room 6 (small room!)

4. gluent.com 4 Topics • Not much about basic stuff like vmstat/iostat • Also nothing about SystemTap (or Dtrace) • Practical stuff that you can (relatively) safely run in production • Measuring / tracing processes • Measuring IO • Measuring Memory • Measuring CPU

5. gluent.com 5 Measuring Process Activity - 1 • Strace • man strace • strace -p <PID> • https://fritshoogland.files.wordpress.com/2013/03/profiling_of_oracl e_using_function_calls.pdf • SysDig • sysdig fd.name contains tnsnames • sysdig -p "%evt.arg.name" proc.name=cat and evt.type=open • csysdig • http://www.sysdig.org/wiki/sysdig-examples/

6. gluent.com 6 Measuring Process Activity - 2 • /proc filesystem • Proc filesystem in Linux is like V$/X$ views in Oracle • man proc • https://www.kernel.org/doc/Documentation/filesystems/proc.txt • http://blog.tanelpoder.com/2013/02/21/peeking-into-linux-kernel- land-using-proc-filesystem-for-quickndirty-troubleshooting/ • psnapper (early prototype!) • ./psnapper <seconds> <pid> • ./psnapper 5 1234 • http://blog.tanelpoder.com/files/scripts/tools/unix/psnapper

7. gluent.com 7 Measuring IO • iostat in RHEL 7 • /proc/<PID>/io • iotop (or htop, atop) • blktrace / blkparse (if we have time) • blktrace -d /dev/sda -o - | blkparse -i -

8. gluent.com 8 Measuring Memory • pmap -x <PID> • http://tech.e2sn.com/oracle/performance/unix-performance- tools/process-memory-matrix (Solaris example but roughly the same) • cat /proc/PID/smaps • grep MMUPageSize: /proc/`pgrep dbw0`/smaps | sort | uniq –c • https://www.kernel.org/doc/Documentation/filesystems/proc.txt • Smem • sudo smem –u • https://www.selenic.com/smem/

9. gluent.com 9 Measuring CPU • perf top • https://fritshoogland.wordpress.com/2012/07/10/the-linux-perf- utility-with-el-6/ • perf record –g && perf report • http://www.brendangregg.com/perf.html • http://externaltable.blogspot.jp/2014/05/flame-graphs-for- oracle.html • perf stat • http://blog.tanelpoder.com/2015/08/09/ram-is-the-new-disk-and- how-to-measure-its-performance-part-1/

10. gluent.com 10 Linux performance references • Brendan Gregg: • http://www.brendangregg.com/linuxperf.html • Luca Canali’s articles: • https://db-blog.web.cern.ch/blog/luca-canali/2015-12-extended- stack-profiling-ideas-tools-and-comments • Frits Hoogland: • https://fritshoogland.wordpress.com/

11. gluent.com 11 Thanks! http://blog.tanelpoder.com tanel@tanelpoder.com @tanelpoder We are hiring developers & data engineers!!! http://gluent.com

Modern Linux Performance Tools for Application Troubleshooting

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (20)

Similar to Modern Linux Performance Tools for Application Troubleshooting

Similar to Modern Linux Performance Tools for Application Troubleshooting (20)

Recently uploaded

Recently uploaded (20)

Modern Linux Performance Tools for Application Troubleshooting