The document outlines steps for analyzing Java heap dumps:
1. Java heap dumps provide a snapshot of objects in memory and can be obtained automatically or on demand.
2. The Eclipse Memory Analyzer (MAT) can be used to analyze heap dumps and identify memory leaks and inefficient object usage.
3. MAT allows examining the size and relationships between objects to understand memory consumption and find the root causes of leaks and bloat.
UiPath Test Automation using UiPath Test Suite series, part 3DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 3. In this session, we will cover desktop automation along with UI automation.
Topics covered:
UI automation Introduction,
UI automation Sample
Desktop automation flow
Pradeep Chinnala, Senior Consultant Automation Developer @WonderBotz and UiPath MVP
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Neuro-symbolic is not enough, we need neuro-*semantic*Frank van Harmelen
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
UiPath Test Automation using UiPath Test Suite series, part 4DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap.
The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies.
Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques
What will you get from this session?
1. Insights into SAP testing best practices
2. Heatmap utilization for testing
3. Optimization of testing processes
4. Demo
Topics covered:
Execution from the test manager
Orchestrator execution result
Defect reporting
SAP heatmap example with demo
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Connector Corner: Automate dynamic content and events by pushing a buttonDianaGray10
Here is something new! In our next Connector Corner webinar, we will demonstrate how you can use a single workflow to:
Create a campaign using Mailchimp with merge tags/fields
Send an interactive Slack channel message (using buttons)
Have the message received by managers and peers along with a test email for review
But there’s more:
In a second workflow supporting the same use case, you’ll see:
Your campaign sent to target colleagues for approval
If the “Approve” button is clicked, a Jira/Zendesk ticket is created for the marketing design team
But—if the “Reject” button is pushed, colleagues will be alerted via Slack message
Join us to learn more about this new, human-in-the-loop capability, brought to you by Integration Service connectors.
And...
Speakers:
Akshay Agnihotri, Product Manager
Charlie Greenberg, Host
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Tobias Schneck
As AI technology is pushing into IT I was wondering myself, as an “infrastructure container kubernetes guy”, how get this fancy AI technology get managed from an infrastructure operational view? Is it possible to apply our lovely cloud native principals as well? What benefit’s both technologies could bring to each other?
Let me take this questions and provide you a short journey through existing deployment models and use cases for AI software. On practical examples, we discuss what cloud/on-premise strategy we may need for applying it to our own infrastructure to get it to work from an enterprise perspective. I want to give an overview about infrastructure requirements and technologies, what could be beneficial or limiting your AI use cases in an enterprise environment. An interactive Demo will give you some insides, what approaches I got already working for real.
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualityInflectra
In this insightful webinar, Inflectra explores how artificial intelligence (AI) is transforming software development and testing. Discover how AI-powered tools are revolutionizing every stage of the software development lifecycle (SDLC), from design and prototyping to testing, deployment, and monitoring.
Learn about:
• The Future of Testing: How AI is shifting testing towards verification, analysis, and higher-level skills, while reducing repetitive tasks.
• Test Automation: How AI-powered test case generation, optimization, and self-healing tests are making testing more efficient and effective.
• Visual Testing: Explore the emerging capabilities of AI in visual testing and how it's set to revolutionize UI verification.
• Inflectra's AI Solutions: See demonstrations of Inflectra's cutting-edge AI tools like the ChatGPT plugin and Azure Open AI platform, designed to streamline your testing process.
Whether you're a developer, tester, or QA professional, this webinar will give you valuable insights into how AI is shaping the future of software delivery.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
Generating a custom Ruby SDK for your web service or Rails API using Smithyg2nightmarescribd
Have you ever wanted a Ruby client API to communicate with your web service? Smithy is a protocol-agnostic language for defining services and SDKs. Smithy Ruby is an implementation of Smithy that generates a Ruby SDK using a Smithy model. In this talk, we will explore Smithy and Smithy Ruby to learn how to generate custom feature-rich SDKs that can communicate with any web service, such as a Rails JSON API.
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
Keynote at DIGIT West Expo, Glasgow on 29 May 2024.
Cheryl Hung, ochery.com
Sr Director, Infrastructure Ecosystem, Arm.
The key trends across hardware, cloud and open-source; exploring how these areas are likely to mature and develop over the short and long-term, and then considering how organisations can position themselves to adapt and thrive.
Key Trends Shaping the Future of Infrastructure.pdf
Memory and runtime analysis
1.
Click to edit the outline text format
Java Performance
Second Outline Level
− Third Outline Level
Memory and Runtime Analysis format
Click to edit the outline text
Fourth Outline Level
Second Outline Outline Level
− Fifth Level
− Third − Sixth Outline Level
Outline Level
− Seventh Outline Level
Fourth Outline Level
− Eighth Outline Level Viet Nguyen, eXo CT
− Fifth
Ninth Outline LevelClick toOutline Level name, title
edit presenter
− Sixth Outline Level
Click to edit the outline text format March 2012
− Seventh Outline Level
Second Outline Level
− Eighth Outline Level
− Third Outline Level
Ninth Outline LevelClick to edit subtitle
Fourth Outline Level
− Fifth Outline Level
− Sixth Outline Level
− Seventh Outline Level
− Eighth Outline Level
2. Agenda
1 − Introduction the outline text format
Click to edit
− Java Memory Model
Second Outline Level
− Heap dump
− Third Outline Level
−
2
Java Memorythe outline text format
Problems
Click to editFourth Outline Level
− Common problems
−
− Memory Leak Fifth Outline Level
Second Outline Level
− Sixth Outline Level
− Third Outline Level
− Seventh Outline Level
3 − Analysing Heap Dumps with Eclipse Memory Analyzer (MAT)
Click to editFourth Outline Level
the outline text format
− Eighth Outline Level
− Shallow & Retained Size, Dominator Tree
− Fifth Outline Level
− Second Outline Level item
Ninth Outline LevelAgendaMAT 1
− Analysing Heap dumps with
−
− Second level Sixth Outline Level
− Third Outline Level
− Seventh Outline Level
4 −
−
Java runtimeFourth Outline Level
profiling
− JvisualVM & −
− Eighth Outline Level
JProfiler
Fifth Outline Level
Ninth Outline LevelAgenda item 2
− eXo JCR statistic tool
−
− Second level Sixth Outline Level
− Seventh Outline Level
− Eighth Outline Level
− Ninth Outline LevelAgenda item 3 eXo Platform
www.exoplatform.com - Copyright 2012 2
3. Code Performance Questions
− Click to edit the outline text
Did you choose the right data structure ?
− format
Are you using the right sorting algorithm ?
− Is your recursive method TOO recursive ? Outline Level
Second
− Are you throwing away a computation that could prove useful
− Third Outline Level
later ?
− Are you creating too many objects unnecessarily or otherwise
Fourth Outline Level
wasting memory ? − Fifth Outline
Level
− Sixth Outline
Level
− Seventh Outline
Level
− Eighth Outline
Level
− Ninth Outline LevelClick to edit
first level
• Second level
• Third level
www.exoplatform.com - Copyright 2012 eXo Platform
• Fourth level
3
4. Optimization Metrics
− Runtime / CPU usage Click to edit the outline text
format
− What line of code the program is spending the most time in
− What call / invocation paths were used to get to these lines
Second Outline Level
− Third Outline Level
− Memory usage
Fourth Outline Level
− What kind of objects are sitting on the heap −
Fifth Outline
− Where were they allocated Level
− Who is pointing to them now − Sixth Outline
− “Memory leak” Level
− Seventh Outline
Level
− Eighth Outline
Level
− Ninth Outline LevelClick to edit
first level
• Second level
• Third level
www.exoplatform.com - Copyright 2012 eXo Platform
• Fourth level
4
5. Java's Memory Model
Click to edit the outline text
format
Second Outline Level
− Third Outline Level
Fourth Outline Level
−
Fifth Outline
Level
− Sixth Outline
Level
− Seventh Outline
Level
− Eighth Outline
Level
− Ninth Outline LevelClick to edit
first level
• Second level
• Third level
www.exoplatform.com - Copyright 2012 eXo Platform
• Fourth level
5
6. Java's Memory Model 2
Click to edit the outline text
format
Second Outline Level
− Third Outline Level
Fourth Outline Level
−
Fifth Outline
Level
− Sixth Outline
Level
− Seventh Outline
Level
− Eighth Outline
Level
− Ninth Outline LevelClick to edit
first level
• Second level
• Third level
www.exoplatform.com - Copyright 2012 eXo Platform
• Fourth level
6
7. The JVM Heaps
freelist
Native Heap Click to edit the outline text
Size
Java Heap
format Next
Second OutlinefreeLevel
storage
Size
Next
− Third Outline Level Null
free storage
Fourth Outline Level
JIT Compiled Code −
Fifth Outline
Free List
Level
Motif structures − Sixth Outline
‘ Level
Thread Stacks Xms - Active Area of Heap
− Seventh Outline
Level
Buffers
− Eighth Outline
Level
− Ninth Outline LevelClick to edit
first level
• Second level
• Third level
www.exoplatform.com - Copyright 2012 eXo Platform
• Fourth level
7
8. The “Correct” Java Heap Size
• GC will adapt heap size to keep occupancy between
Click to edit the outline text
40% and 70% format
–Heap occupancy over 70% causes frequent GC cycles
Second Outline Level
• Which generally means reduced performance
−
–Heap occupancy below 40% means Third OutlineGC cycles,
infrequent Level
but cycles longer than they needs to beFourth Outline Level
• Which means longer pause times that necessary
− Fifth Outline
• Which generally means reduced performance Level
Sixth Outline −
Level
• The maximum heap size setting should therefore be
− Seventh Outline
43% larger than the maximum occupancy of the
application Level
− Eighth Outline
–Eg. For 70MB occupancy, 100MB Max heap is Level
required (which is 70MB Ninth Outline LevelClick to edit
− + 43% of 70MB)
first level
• Second level
• Third level
www.exoplatform.com - Copyright 2012 eXo Platform
• Fourth level
8
9. Binary Heap Dump
A heap dump is a snapshot of objects that are alive at one point in
time.
It contains:
IOIOII
IOIIOI
OIOIO
IIOIIOI
Objects: fields, references, primitive values, …
Classes: class loader, super class, static fields, …
IOIOII A heap dump does not contain
IOIIOI
OIOIO
IIOIIOI where the objects have been created
which objects have been garbage collected
www.exoplatform.com - Copyright 2012 eXo Platform 9
10. Heap Dumps Are Useful for Analysis
− Heap dumps come “for free” Click to edit the outline text
− They are suitable for production format
− The comprehensive data allows detail analysis
Second Outline Level
− There is a wide platform coverage
− Third
− HPROF dumps from HotSpot based JVMs Outline Level
Fourth Outline Level
− DTFJ and PHD dumps from IBM JVMs −
Fifth Outline
Level
− Sixth Outline
Level
− Seventh Outline
Level
− Eighth Outline
Level
− Ninth Outline LevelClick to edit
first level
• Second level
• Third level
www.exoplatform.com - Copyright 2012 eXo Platform
• Fourth level
10
11. How To Get A Heap Dump
Non-Interactive
-XX:+HeapDumpOnOutOfMemoryError
On Demand
JDK1.4.2_12 and -XX:+HeapDumpOnCtrlBreak
JDK6 and Jconsole
More…
http://wiki.eclipse.org/index.php/MemoryAnalyzer#Getting_a_Heap_Dump
www.exoplatform.com - Copyright 2012 eXo Platform 11
12. Java Memory Problems
Common to edit the outline text format
Click problems
Memory Leak
Second Outline Level
− Third Outline Level
Fourth Outline Level
− Fifth Outline Level
− Sixth Outline Level
− Seventh Outline Level
− Eighth Outline Level
Ninth Outline LevelClick to edit subtitle
13. Java Memory Problems
− Memory leaks − Unnecessarily high memory usage
Click to edit the outline text
• Referencing objects that are no format
• Implementations consuming too
longer used much memory
• Developer forget clear them •
Second Outline Level
Large amount of state information
• Unbound or inefficiently configured
− Third Outline Level
caches
− Inefficient object creation • Large temporary objects: bigLevel
Fourth Outline
• GC must constantly clean up the − Fifth Outline
documents (XML, PDF, images...)
heap are read and processed
Level
• High CPU consumption by GC −
Sixth Outline
• Application response times
Level
− Inefficient garbage collector
increases
behaviour − Seventh Outline
Level
• Missing or wrong configuration of
GC − Eighth Outline
Level
− Ninth Outline LevelClick to edit
first level
• Second level
• Third level
www.exoplatform.com - Copyright 2012 eXo Platform
• Fourth level
13
14. Most Common Memory Leaks
− Thread Local Variables − Mutable static fields and text
Click to edit the outline
• ThreadLocals are often used to track collections
format
state • Most common reason
• Threads are often never garbage Second Outline Level
• Statics often used to hold caches or
collected shareThird Outline Level
− states
• Memory leaks may happen if thread • Memoryleak happens easily if this is
local variable is not clean carefully not done diligently Outline Level
Fourth
−
Fifth Outline
Level
− Circular and complex bi- − Wrong implementation of
− Sixth Outline
directional references equals/hashcode
Level
• All objects will not be garbage • Lead to memory leaks when used as
−
collected a key in a map Seventh Outline
Level
− Eighth Outline
Level
− Ninth Outline LevelClick to edit
first level
• Second level
• Third level
www.exoplatform.com - Copyright 2012 eXo Platform
• Fourth level
14
15. Most Common Memory Leaks
− Classloader Leaks − JNI memory leaks
Click to edit the outline text
• Classes are referenced by their format is used to call native code
• JNI
class loader • Native code can handle, call, or
• They will not get garbage collected create javaOutline Level
Second objects
until classloader itself is collected • These objects are referenced until
− Third Outline Level
• First leak form: an object whose the native method returns
class belongs to the class loader is •
Fourth Outline Level
First leak form: native method run
still referenced by a cache, a thread forever − Fifth Outline
local … • Level
Second leak form: You keep created
• Second leak form: Some objects after − Sixth Outline
native code has ended
frameworks allow dynamic creation
of new classes. If they are forgotten,
Level
PermGen leaks happen − Seventh Outline
Level
− Eighth Outline
Level
− Ninth Outline LevelClick to edit
first level
• Second level
• Third level
www.exoplatform.com - Copyright 2012 eXo Platform
• Fourth level
15
16. How to interpret the Memory Leak Analysis Results
Click to edit the outline text
What is leaking?
format
– What is the object (e.g. a HashMap) holding all the leaking objects
i.e. leak container.
Second Outline Level
− Third Outline Level
– What are the objects getting added to the leak container i.e. leak
unit.
Fourth Outline Level
− Fifth Outline
– Who is holding the leak container in memory? What are the object
types and package names of objects on the chain of Level
references
− Sixth
from a root object to the leak container i.e. owner chain. Outline
Level
− Seventh Outline
Level
− Eighth Outline
Level
− Ninth Outline LevelClick to edit
first level
• Second level
• Third level
www.exoplatform.com - Copyright 2012 eXo Platform
• Fourth level
16
17. View of the leaking data structure
Significant entities Leak Root
Click to edit the outline text
MyClass
format Owner
– An owner chain Chain
Container
Second HashSet
Outline Level
– A leak root − Third Outline Level
– A container Leaking HashMap
Fourth Outline Level
Unit
– The unit of the leak −
Fifth Outline
HM$Entry
Level
HM$Entry HM$Entry
– Leak contents − Sixth Outline
Level
String String String
− Seventh Outline
Level
Contents Char[] Char[] Char[]
− Eighth Outline
Level
− Ninth Outline LevelClick to edit
first level
• Second level
• Third level
www.exoplatform.com - Copyright 2012 eXo Platform
• Fourth level
17
18. Memory Leak Example
public class MyClass { Leak
Click to edit the outline text
Root
format
static HashSet myContainer = new HashSet();
Second Outline Level
public void leak(int numObjects) { − Third Outline Level Leak
Container
for (int i = 0; i < numObjects; ++i) {
Fourth Outline Level
String leakingUnit = new String("this is leaking object: " + i);
−
Fifth Outline
myContainer.add(leakingUnit);
Level
}
− Sixth Outline
}
Leaking
public static void main(String[] args) throws Exception {
Level Units
− Seventh Outline
{
Level
MyClass myObj = new MyClass();
− Eighth Outline
myObj.leak(1000000); // 1 million
Level
}
− Ninth Outline LevelClick to edit
System.gc();
first level
}
• Second level
} • Third level
www.exoplatform.com - Copyright 2012 eXo Platform
• Fourth level
18
19. Analyzing Heap dumps with MAT
Retained to edit the outline text format
Click size, Dominator Tree
MATfeatures
Demos Second Outline Level
− Third Outline Level
Fourth Outline Level
− Fifth Outline Level
− Sixth Outline Level
− Seventh Outline Level
− Eighth Outline Level
Ninth Outline LevelClick to edit subtitle
20. Tools For Analysing Heap Dumps
Click to edit the outline text
format
Second Outline Level
− Third Outline Level
Fourth Outline Level
−
Fifth Outline
Level
− Sixth Outline
Level
− Seventh Outline
Level
− Eighth Outline
Level
− Ninth Outline LevelClick to edit
first level
• Second level
• Third level
www.exoplatform.com - Copyright 2012 eXo Platform
• Fourth level
20
21. Eclipse Memory Analyzer Introduction
The number of memory-related problems is very high
They are extremely difficult to analyze
Analysis requires expertise in the analyzed code
Eclipse Memory Analyzer:
Simplifies memory analysis
Extensible
Free for download @ www.eclipse.org/mat
www.exoplatform.com - Copyright 2012 eXo Platform 21
22. A Little Bit of History
Problem #1: multi-GB heap dumps from 64-bit machines
Quickly open and re-open a heap dump on 32-bit machines
Problem #2: millions of objects. Where is the „needle“?
Dominator Tree: fast calculation of retained memory
Meaningful names for class loaders: see components
SQL-like language: create custom queries
Problem #3: complexity of analysis
Reports on memory leak suspects and checks for known
antipatterns
Problem #4: HPROF heap dumps only
Support for IBM system dumps
www.exoplatform.com - Copyright 2012 eXo Platform 22
23. Shallow And Retained Size
Shallow heap is the memory consumed by one object
Retained set of X is the set of objects that will be garbage collected
if X is garbage collected
Retained heap of X is the sum of shallow sizes of all objects in the
retained set of X, i.e. memory kept alive by X
Set of elements Retained Set
C C, F, G, J
K K
C, K C, F, G, J, K, I
www.exoplatform.com - Copyright 2012 eXo Platform 23
24. Dominator Tree
The Dominator Tree is a Transformation of the Cyclic Object Graph
into a “Keep-Alive” Tree:
Every node in the Tree is directly responsible for keeping alive
its children
Object X dominates object Y if all paths from the roots to Y run
through X
Benefits:
Fast calculation of the
retained size (sum all children)
List of distinct big
objects (first level of the tree)
Fast identification of
responsible objects (just go up
the tree)
www.exoplatform.com - Copyright 2012 eXo Platform 24
25. MAT Features
Memory Leak Hunter
Automatically detect memory leak suspects
Discover if the issue is known (and a fix available)
Collect details for in depth analysis by the code experts
www.exoplatform.com - Copyright 2012 eXo Platform 25
26. MAT Report Overview
One big object:
memory leak suspect
Any up-to-date architecture
loads components with
separate class loaders, be
it OSGi or JEE application
servers.
Extensible to display
meaningful names.
Search by keywords:
identify if problem is
known
Classification for trouble
ticket system: less ping-
pong of trouble tickets.
www.exoplatform.com - Copyright 2012 eXo Platform 26
27. MAT Report Details
The chain of objects and
references which keep
the suspect alive
A significant drop in the
retained sizes shows the
accumulation point
Accumulated
objects
www.exoplatform.com - Copyright 2012 eXo Platform 27
28. MAT Features
Top Components Report
Analyze components, that occupy more memory, than a certain
threshold
Give hints where memory footprint can be optimized
Check for known antipatterns
www.exoplatform.com - Copyright 2012 eXo Platform 28
31. How to Analyze Memory Leaks
− Find the biggest objects Click to edit the outline text
− Analyze why they are kept in memory
format
− Analyze what make them big Second Outline Level
− Third Outline Level
Use MAT To Analyze Leaks
Fourth Outline Level
Fifth Outline −
Level
− Get a “good” heapdump
− Sixth Outline
− Find the biggest objects (in the dominator tree)
Level
− Analyze who kept them alive (using paths) − Seventh Outline
− Analyze what made them big (looking at their retain set)
Level
− Use “Leak report” to automate the analysis − Eighth Outline
Level
− Ninth Outline LevelClick to edit
first level
• Second level
• Third level
www.exoplatform.com - Copyright 2012 eXo Platform
• Fourth level
31
32. How to Analyze a Heavy Request
− Look at the thread attributes – name, class, etc...
Click to edit the outline text
− Look at the local variables format
− Look at the stack of the thread Second Outline Level
− Third Outline Level
Use MAT To Analyze Thread Activity
Fourth Outline Level
Fifth Outline −
Level
− Inspect threads attributes and local variables (in object
− Sixth Outline
explorer)
Level
− Analyze stack trace − Seventh Outline
Level
− Eighth Outline
Level
− Ninth Outline LevelClick to edit
first level
• Second level
• Third level
www.exoplatform.com - Copyright 2012 eXo Platform
• Fourth level
32
33. Find Where Memory Footprint Can Be Optimized
− Inspect the set of retained objectsClick to edit the outline text
− Search for inefficiently used data format
structures
− Look for redundant data Second Outline Level
− Third Outline Level
Use MAT To Analyze Memory FootprintFourth Outline Level
Fifth Outline −
Level
− Analyze retained objects (in dominator tree, retained set)
− Sixth Outline
− Use “Group by Value” to find redundant objects
Level
− Use the commands from the “Collections” group − Seventh Outline
− Use “Component Report” to automate the analysis Level
− Eighth Outline
Level
− Ninth Outline LevelClick to edit
first level
• Second level
• Third level
www.exoplatform.com - Copyright 2012 eXo Platform
• Fourth level
33
34. Java Runtime Profiling
JVisualVM edit the outline text format
Click to
JProfiler
eXo JCR statistic tool Level
Second Outline
− Third Outline Level
Fourth Outline Level
− Fifth Outline Level
− Sixth Outline Level
− Seventh Outline Level
− Eighth Outline Level
Ninth Outline LevelClick to edit subtitle
35. What to do with profiler results
− Observe which methods are being callededit the outline text
Click to the most
format
− These may not necessarily be the “slowest” methods!
Second Outline Level
− Observe which methods are taking the most time relative to
− Third Outline Level
others
Fourth Outline Level
− Common problem: inefficient unbuffered I/O − Fifth Outline
− Common problem: poor choice of data structure Level
− Common problem: recursion call overhead − Sixth Outline
− Common problem: unnecessary re-computation of expensive
Level
information, or unnecessary multiple I/O of same data
− Seventh Outline
Level
− Eighth Outline
Level
− Ninth Outline LevelClick to edit
first level
• Second level
• Third level
www.exoplatform.com - Copyright 2012 eXo Platform
• Fourth level
35
36. Q&A
Click to edit the outline text format
Second Outline Level
− Third Outline Level
http://kohlerm.blogspot.com/search/label/memory
Fourth Outline Level
http://visualvm.java.net/docindex.html
− Fifth Outline Level
http://docs.exoplatform.org/PLF35/index.jsp?topic=/org.exoplatform.do
− Sixth Outline Level
− Seventh Outline Level
− Eighth Outline Level
Ninth Outline LevelClick to edit subtitle
37. How will this session do
− Present common memory & thread-related issues
Click to edit the outline text
− Give hints how to solve them using heap dumps and Eclipse
format
Memory Analyzer
Second Outline Level
− Show demos using real-life heap dumps
− Third Outline
− Runtime Java profiling with JvisualVM, Jprofiler, eXoLevel
JCR
statistics tool (exo.jcr.component.statistics) Fourth Outline Level
−
Fifth Outline
Level
− Sixth Outline
Level
− Seventh Outline
Level
− Eighth Outline
Level
− Ninth Outline LevelClick to edit
first level
• Second level
• Third level
www.exoplatform.com - Copyright 2012 eXo Platform
• Fourth level
37
Editor's Notes
How will this session do Present common memory & thread-related issues Give hints how to solve them using heap dumps and Eclipse Memory Analyzer Show demos using real-life heap dumps Runtime Java profiling with JvisualVM, Jprofiler, eXo JCR statistics tool (exo.jcr.component.statistics)
http://blog.dynatrace.com/2009/08/13/java-memory-problems/ http://blog.dynatrace.com/2011/04/20/the-top-java-memory-problems-part-1/ http://blog.dynatrace.com/2011/12/15/the-top-java-memory-problems-part-2/ Memory Leaks and other memory related problems are among the most prominent performance and scalability problems in Java. Reason enough to discuss this topic in more detail. The Java memory model- or more specifically the garbage collector – has solved many memory problems. At the same time new ones have been created. Especially in J2EE Environments with a large number of parallel users, memory is more and more becoming a critical ressource. In times with cheap memory available, 64bit JVMs and modern garbage collection algorithms this might sound strange at first sight. So let us now take a closer look at Java memory problems. Problems can be categorized into four: groups Memory leaks in Java are created by referencing objects that are no longer used. This easily happens when multiple references to objects exist and developer forget to clear them, when the object is no longer needed. Unnecessarily high memory usage caused by implementations consuming to much memory. This is very often a problem in web applications where a large amount of state information is managed for “user comfort”. When the number of active users increases, memory limits are reached very fast. Unbound or inefficiently configured caches are another source of constant high memory usage. Inefficient object creation easily results in a performance problem when user load increases, as the garbage collector must constantly clean up the heap. This leads to unnecessarily high CPU consumption by the garbage collector. As the CPU is blocked by garbage collection, application response times increases often already under moderate load. This behaviour is also referred to as GC trashing. Inefficient garbage collector behaviour is caused by missing or wrong configuration of the garbage collector. The garbage collector will take care that object are cleaned up. How and when this should happen must however by configured by the programmer or system architect. Very often people simple “forget” to properly configure and tune the garbage collecotr. I was involved in a number of performance workshops where a “simple” parameter change resulted in a performance improvement of up to 25 percent. In most cases memory problems affect not only performance but also scalability. The higher the amount of consumed memory per request, user or session the less parallel transactions can be executed. In some cases memory problems also affect availabilty. When the JVM runs out of memory or it is close to memory limits it will quit with an OutOfMemory error. This is when management enters your office and you know you are in serious trouble. Memory problems are often difficult to resolve for two reasons: In some case analysis will get complex and difficult – especially if you are missing the right methodology to resolve them. Secondly their foundation is often in the architecture of the application. Simple code changes will not help to resolve them. In order to make life easier I present a couple of memory antipatterns which are often found in real world world applications. Those patterns should help to be able to already avoid memory problems during development.
http://blog.dynatrace.com/2009/08/13/java-memory-problems/ http://blog.dynatrace.com/2011/04/20/the-top-java-memory-problems-part-1/ http://blog.dynatrace.com/2011/12/15/the-top-java-memory-problems-part-2/ Memory Leaks and other memory related problems are among the most prominent performance and scalability problems in Java. Reason enough to discuss this topic in more detail. The Java memory model- or more specifically the garbage collector – has solved many memory problems. At the same time new ones have been created. Especially in J2EE Environments with a large number of parallel users, memory is more and more becoming a critical ressource. In times with cheap memory available, 64bit JVMs and modern garbage collection algorithms this might sound strange at first sight. So let us now take a closer look at Java memory problems. Problems can be categorized into four: groups Memory leaks in Java are created by referencing objects that are no longer used. This easily happens when multiple references to objects exist and developer forget to clear them, when the object is no longer needed. Unnecessarily high memory usage caused by implementations consuming to much memory. This is very often a problem in web applications where a large amount of state information is managed for “user comfort”. When the number of active users increases, memory limits are reached very fast. Unbound or inefficiently configured caches are another source of constant high memory usage. Inefficient object creation easily results in a performance problem when user load increases, as the garbage collector must constantly clean up the heap. This leads to unnecessarily high CPU consumption by the garbage collector. As the CPU is blocked by garbage collection, application response times increases often already under moderate load. This behaviour is also referred to as GC trashing. Inefficient garbage collector behaviour is caused by missing or wrong configuration of the garbage collector. The garbage collector will take care that object are cleaned up. How and when this should happen must however by configured by the programmer or system architect. Very often people simple “forget” to properly configure and tune the garbage collecotr. I was involved in a number of performance workshops where a “simple” parameter change resulted in a performance improvement of up to 25 percent. In most cases memory problems affect not only performance but also scalability. The higher the amount of consumed memory per request, user or session the less parallel transactions can be executed. In some cases memory problems also affect availabilty. When the JVM runs out of memory or it is close to memory limits it will quit with an OutOfMemory error. This is when management enters your office and you know you are in serious trouble. Memory problems are often difficult to resolve for two reasons: In some case analysis will get complex and difficult – especially if you are missing the right methodology to resolve them. Secondly their foundation is often in the architecture of the application. Simple code changes will not help to resolve them. In order to make life easier I present a couple of memory antipatterns which are often found in real world world applications. Those patterns should help to be able to already avoid memory problems during development.
http://blog.dynatrace.com/2011/04/20/the-top-java-memory-problems-part-1/ Thread Local Variables ThreadLocals are used to bind a variable or a state to a thread. Each thread has its own instance of the variable. They are very useful but also very dangerous. They are often used to track a state, like the current transaction id, but sometimes they hold a little more information. A thread local variable is referenced by its thread and as such its lifecycle is bound to it. In most application servers threads are reused via thread pools and thus are never garbage collected. If the application code is not carefully clearing the thread local variable you get a nasty memory leak. These kind of memory leaks can easily be discovered with a heap dump. Just take a look at the ThreadLocalMap in the heap dump and follow the references. The heapdump shows that over 4K objects which amount to about 10MB are held by thread locals You can then also look at the name of the thread to figure out which part of your application is responsible for the leak. Mutable static fields and collections The most common reason for a memory leak is the wrong usage of statics. A static variable is held by its class and subsequently by its classloader. While a class can be garbage collected it will seldom happen during an applications lifetime. Very often statics are used to hold cache information or share state across threads. If this is not done diligently it is very easy to get a memory leak. Especially static mutable collections should be avoided at all costs for just that reason. A good architectural rule is not to use mutable static objects at all, most of the time there is a better alternative. Circular and complex bi-directional references This is my favorite memory leak. It is best explained by example: org.w3c.dom.Document doc = readXmlDocument(); org.w3c.dom.Node child = doc.getDocumentElement().getFirstChild(); doc.removeNode(child); doc = null; At the end of the code snippet we would think that the DOM Document will be garbage collected. That is however not the case. A DOM Node object always belongs to a Document. Even when removed from the Document the Node Object still has a reference to its owning document. As long as we keep the child object the document and all other nodes it contains will not be garbage collected. I’ve see this and other similar issues quite often. Wrong implementation of equals/hashcode It might not be obvious on the first glance, but if your equals/hashcode methods violate the equals contract it will lead to memory leaks when used as a key in a map. A HashMap uses the hashcode to lookup an object and verify that it found it by using the equals method. If two objects are equal they must have the same hashcode, but not the other way around. If you do not explicitly implement hashcode yourself this is not the case. The default hashcode is based on object identity. Thus using an object without a valid hashcode implementation as a key in a map, you will be able to add things but you will not find them anymore. Even worse if you re-add it, it will not overwrite the old item but really add a new one. And just like that you have memory leak. You will find it easily enough as it is growing, but the root cause will be hard to determine unless you remember this one. The easiest way to avoid this is to use unit testcases and one of the available frameworks that tests the equals contract of your classes (e.g. http://code.google.com/p/equalsverifier/).
Classloader Leaks When thinking about memory leaks we think mostly about normal java objects. Especially in application servers and OSGi containers there is another form of memory leak, the class loader leak. Classes are referenced by their classloader and normally they will not get garbage collected until the classloader itself is collected. That however only happens when the application gets unloaded by the application server or OSGi container. There are two forms of Classloader Leaks that I can describe off the top of my head. In the first an object whose class belongs to the class loader is still referenced by a cache, a thread local or some other means. In that case the whole class loader, meaning the whole application cannot be garbage collected. This is something that happens quite a lot in OSGi containers nowadays and used to happen in JEE Application Servers frequently as well. As it is only happens when the application gets unloaded or redeployed it does not happen very often. The second form is nastier and was introduced by bytecode manipulation frameworks like BCEL and ASM. These frameworks allow the dynamic creation of new classes. If you follow that thought you will realize that now classes, just like objects, can be forgotten by the developer. The responsible code might create new classes for the same purpose multiple times. As the class is referenced in the current class loader you get a memory leak that will lead to an out of memory in the permanent generation. The real bad news is that most heap analyzer tools do not point out this problem either, we have to analyze it manually, the hard way. This form or memory leak became famous due to an issue in an older version of hibernate and its usage of CGLIB. JNI memory leaks This is a particularly nasty form or memory leak. It is not easily found, unless you have the right tool and it is also not known to a lot of people. JNI is used to call native code from java. This native code can handle, call and also create java objects. Every java object created in a native method begins its life as a so called local reference. That means that the object is referenced until the native method returns. We could say the native method references the java object. So you don’t have a problem unless the native method runs forever. In some cases you want to keep the created object even after the native call has ended. To achieve this you can either ensure that it is referenced by some other java object or you can change the local reference into a global reference. A global reference is a GC root and will never be garbage collected until explicitly deleted by the native code. The only way to discover such a memory leak is to use a heap dump tool that explicitly shows global native references. If you have to use JNI you should rather make sure that you reference these objects normally and forgo global references altogether. You can find this sort of leak when your heap dump analysis tool explicitly marks the GC Root as a native reference, otherwise you will have a hard time.
Classloader Leaks When thinking about memory leaks we think mostly about normal java objects. Especially in application servers and OSGi containers there is another form of memory leak, the class loader leak. Classes are referenced by their classloader and normally they will not get garbage collected until the classloader itself is collected. That however only happens when the application gets unloaded by the application server or OSGi container. There are two forms of Classloader Leaks that I can describe off the top of my head. In the first an object whose class belongs to the class loader is still referenced by a cache, a thread local or some other means. In that case the whole class loader, meaning the whole application cannot be garbage collected. This is something that happens quite a lot in OSGi containers nowadays and used to happen in JEE Application Servers frequently as well. As it is only happens when the application gets unloaded or redeployed it does not happen very often. The second form is nastier and was introduced by bytecode manipulation frameworks like BCEL and ASM. These frameworks allow the dynamic creation of new classes. If you follow that thought you will realize that now classes, just like objects, can be forgotten by the developer. The responsible code might create new classes for the same purpose multiple times. As the class is referenced in the current class loader you get a memory leak that will lead to an out of memory in the permanent generation. The real bad news is that most heap analyzer tools do not point out this problem either, we have to analyze it manually, the hard way. This form or memory leak became famous due to an issue in an older version of hibernate and its usage of CGLIB. JNI memory leaks This is a particularly nasty form or memory leak. It is not easily found, unless you have the right tool and it is also not known to a lot of people. JNI is used to call native code from java. This native code can handle, call and also create java objects. Every java object created in a native method begins its life as a so called local reference. That means that the object is referenced until the native method returns. We could say the native method references the java object. So you don’t have a problem unless the native method runs forever. In some cases you want to keep the created object even after the native call has ended. To achieve this you can either ensure that it is referenced by some other java object or you can change the local reference into a global reference. A global reference is a GC root and will never be garbage collected until explicitly deleted by the native code. The only way to discover such a memory leak is to use a heap dump tool that explicitly shows global native references. If you have to use JNI you should rather make sure that you reference these objects normally and forgo global references altogether. You can find this sort of leak when your heap dump analysis tool explicitly marks the GC Root as a native reference, otherwise you will have a hard time.
Classloader Leaks When thinking about memory leaks we think mostly about normal java objects. Especially in application servers and OSGi containers there is another form of memory leak, the class loader leak. Classes are referenced by their classloader and normally they will not get garbage collected until the classloader itself is collected. That however only happens when the application gets unloaded by the application server or OSGi container. There are two forms of Classloader Leaks that I can describe off the top of my head. In the first an object whose class belongs to the class loader is still referenced by a cache, a thread local or some other means. In that case the whole class loader, meaning the whole application cannot be garbage collected. This is something that happens quite a lot in OSGi containers nowadays and used to happen in JEE Application Servers frequently as well. As it is only happens when the application gets unloaded or redeployed it does not happen very often. The second form is nastier and was introduced by bytecode manipulation frameworks like BCEL and ASM. These frameworks allow the dynamic creation of new classes. If you follow that thought you will realize that now classes, just like objects, can be forgotten by the developer. The responsible code might create new classes for the same purpose multiple times. As the class is referenced in the current class loader you get a memory leak that will lead to an out of memory in the permanent generation. The real bad news is that most heap analyzer tools do not point out this problem either, we have to analyze it manually, the hard way. This form or memory leak became famous due to an issue in an older version of hibernate and its usage of CGLIB. JNI memory leaks This is a particularly nasty form or memory leak. It is not easily found, unless you have the right tool and it is also not known to a lot of people. JNI is used to call native code from java. This native code can handle, call and also create java objects. Every java object created in a native method begins its life as a so called local reference. That means that the object is referenced until the native method returns. We could say the native method references the java object. So you don’t have a problem unless the native method runs forever. In some cases you want to keep the created object even after the native call has ended. To achieve this you can either ensure that it is referenced by some other java object or you can change the local reference into a global reference. A global reference is a GC root and will never be garbage collected until explicitly deleted by the native code. The only way to discover such a memory leak is to use a heap dump tool that explicitly shows global native references. If you have to use JNI you should rather make sure that you reference these objects normally and forgo global references altogether. You can find this sort of leak when your heap dump analysis tool explicitly marks the GC Root as a native reference, otherwise you will have a hard time.
Classloader Leaks When thinking about memory leaks we think mostly about normal java objects. Especially in application servers and OSGi containers there is another form of memory leak, the class loader leak. Classes are referenced by their classloader and normally they will not get garbage collected until the classloader itself is collected. That however only happens when the application gets unloaded by the application server or OSGi container. There are two forms of Classloader Leaks that I can describe off the top of my head. In the first an object whose class belongs to the class loader is still referenced by a cache, a thread local or some other means. In that case the whole class loader, meaning the whole application cannot be garbage collected. This is something that happens quite a lot in OSGi containers nowadays and used to happen in JEE Application Servers frequently as well. As it is only happens when the application gets unloaded or redeployed it does not happen very often. The second form is nastier and was introduced by bytecode manipulation frameworks like BCEL and ASM. These frameworks allow the dynamic creation of new classes. If you follow that thought you will realize that now classes, just like objects, can be forgotten by the developer. The responsible code might create new classes for the same purpose multiple times. As the class is referenced in the current class loader you get a memory leak that will lead to an out of memory in the permanent generation. The real bad news is that most heap analyzer tools do not point out this problem either, we have to analyze it manually, the hard way. This form or memory leak became famous due to an issue in an older version of hibernate and its usage of CGLIB. JNI memory leaks This is a particularly nasty form or memory leak. It is not easily found, unless you have the right tool and it is also not known to a lot of people. JNI is used to call native code from java. This native code can handle, call and also create java objects. Every java object created in a native method begins its life as a so called local reference. That means that the object is referenced until the native method returns. We could say the native method references the java object. So you don’t have a problem unless the native method runs forever. In some cases you want to keep the created object even after the native call has ended. To achieve this you can either ensure that it is referenced by some other java object or you can change the local reference into a global reference. A global reference is a GC root and will never be garbage collected until explicitly deleted by the native code. The only way to discover such a memory leak is to use a heap dump tool that explicitly shows global native references. If you have to use JNI you should rather make sure that you reference these objects normally and forgo global references altogether. You can find this sort of leak when your heap dump analysis tool explicitly marks the GC Root as a native reference, otherwise you will have a hard time.
Classloader Leaks When thinking about memory leaks we think mostly about normal java objects. Especially in application servers and OSGi containers there is another form of memory leak, the class loader leak. Classes are referenced by their classloader and normally they will not get garbage collected until the classloader itself is collected. That however only happens when the application gets unloaded by the application server or OSGi container. There are two forms of Classloader Leaks that I can describe off the top of my head. In the first an object whose class belongs to the class loader is still referenced by a cache, a thread local or some other means. In that case the whole class loader, meaning the whole application cannot be garbage collected. This is something that happens quite a lot in OSGi containers nowadays and used to happen in JEE Application Servers frequently as well. As it is only happens when the application gets unloaded or redeployed it does not happen very often. The second form is nastier and was introduced by bytecode manipulation frameworks like BCEL and ASM. These frameworks allow the dynamic creation of new classes. If you follow that thought you will realize that now classes, just like objects, can be forgotten by the developer. The responsible code might create new classes for the same purpose multiple times. As the class is referenced in the current class loader you get a memory leak that will lead to an out of memory in the permanent generation. The real bad news is that most heap analyzer tools do not point out this problem either, we have to analyze it manually, the hard way. This form or memory leak became famous due to an issue in an older version of hibernate and its usage of CGLIB. JNI memory leaks This is a particularly nasty form or memory leak. It is not easily found, unless you have the right tool and it is also not known to a lot of people. JNI is used to call native code from java. This native code can handle, call and also create java objects. Every java object created in a native method begins its life as a so called local reference. That means that the object is referenced until the native method returns. We could say the native method references the java object. So you don’t have a problem unless the native method runs forever. In some cases you want to keep the created object even after the native call has ended. To achieve this you can either ensure that it is referenced by some other java object or you can change the local reference into a global reference. A global reference is a GC root and will never be garbage collected until explicitly deleted by the native code. The only way to discover such a memory leak is to use a heap dump tool that explicitly shows global native references. If you have to use JNI you should rather make sure that you reference these objects normally and forgo global references altogether. You can find this sort of leak when your heap dump analysis tool explicitly marks the GC Root as a native reference, otherwise you will have a hard time.
Classloader Leaks When thinking about memory leaks we think mostly about normal java objects. Especially in application servers and OSGi containers there is another form of memory leak, the class loader leak. Classes are referenced by their classloader and normally they will not get garbage collected until the classloader itself is collected. That however only happens when the application gets unloaded by the application server or OSGi container. There are two forms of Classloader Leaks that I can describe off the top of my head. In the first an object whose class belongs to the class loader is still referenced by a cache, a thread local or some other means. In that case the whole class loader, meaning the whole application cannot be garbage collected. This is something that happens quite a lot in OSGi containers nowadays and used to happen in JEE Application Servers frequently as well. As it is only happens when the application gets unloaded or redeployed it does not happen very often. The second form is nastier and was introduced by bytecode manipulation frameworks like BCEL and ASM. These frameworks allow the dynamic creation of new classes. If you follow that thought you will realize that now classes, just like objects, can be forgotten by the developer. The responsible code might create new classes for the same purpose multiple times. As the class is referenced in the current class loader you get a memory leak that will lead to an out of memory in the permanent generation. The real bad news is that most heap analyzer tools do not point out this problem either, we have to analyze it manually, the hard way. This form or memory leak became famous due to an issue in an older version of hibernate and its usage of CGLIB. JNI memory leaks This is a particularly nasty form or memory leak. It is not easily found, unless you have the right tool and it is also not known to a lot of people. JNI is used to call native code from java. This native code can handle, call and also create java objects. Every java object created in a native method begins its life as a so called local reference. That means that the object is referenced until the native method returns. We could say the native method references the java object. So you don’t have a problem unless the native method runs forever. In some cases you want to keep the created object even after the native call has ended. To achieve this you can either ensure that it is referenced by some other java object or you can change the local reference into a global reference. A global reference is a GC root and will never be garbage collected until explicitly deleted by the native code. The only way to discover such a memory leak is to use a heap dump tool that explicitly shows global native references. If you have to use JNI you should rather make sure that you reference these objects normally and forgo global references altogether. You can find this sort of leak when your heap dump analysis tool explicitly marks the GC Root as a native reference, otherwise you will have a hard time.
Classloader Leaks When thinking about memory leaks we think mostly about normal java objects. Especially in application servers and OSGi containers there is another form of memory leak, the class loader leak. Classes are referenced by their classloader and normally they will not get garbage collected until the classloader itself is collected. That however only happens when the application gets unloaded by the application server or OSGi container. There are two forms of Classloader Leaks that I can describe off the top of my head. In the first an object whose class belongs to the class loader is still referenced by a cache, a thread local or some other means. In that case the whole class loader, meaning the whole application cannot be garbage collected. This is something that happens quite a lot in OSGi containers nowadays and used to happen in JEE Application Servers frequently as well. As it is only happens when the application gets unloaded or redeployed it does not happen very often. The second form is nastier and was introduced by bytecode manipulation frameworks like BCEL and ASM. These frameworks allow the dynamic creation of new classes. If you follow that thought you will realize that now classes, just like objects, can be forgotten by the developer. The responsible code might create new classes for the same purpose multiple times. As the class is referenced in the current class loader you get a memory leak that will lead to an out of memory in the permanent generation. The real bad news is that most heap analyzer tools do not point out this problem either, we have to analyze it manually, the hard way. This form or memory leak became famous due to an issue in an older version of hibernate and its usage of CGLIB. JNI memory leaks This is a particularly nasty form or memory leak. It is not easily found, unless you have the right tool and it is also not known to a lot of people. JNI is used to call native code from java. This native code can handle, call and also create java objects. Every java object created in a native method begins its life as a so called local reference. That means that the object is referenced until the native method returns. We could say the native method references the java object. So you don’t have a problem unless the native method runs forever. In some cases you want to keep the created object even after the native call has ended. To achieve this you can either ensure that it is referenced by some other java object or you can change the local reference into a global reference. A global reference is a GC root and will never be garbage collected until explicitly deleted by the native code. The only way to discover such a memory leak is to use a heap dump tool that explicitly shows global native references. If you have to use JNI you should rather make sure that you reference these objects normally and forgo global references altogether. You can find this sort of leak when your heap dump analysis tool explicitly marks the GC Root as a native reference, otherwise you will have a hard time.