• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Windows Debugging Tools - JavaOne 2013
 

Windows Debugging Tools - JavaOne 2013

on

  • 900 views

This presentation provides an introduction to the (subjectively) most useful tools for debugging on Windows.

This presentation provides an introduction to the (subjectively) most useful tools for debugging on Windows.

Statistics

Views

Total Views
900
Views on SlideShare
900
Embed Views
0

Actions

Likes
2
Downloads
12
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as OpenOffice

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Windows Debugging Tools - JavaOne 2013 Windows Debugging Tools - JavaOne 2013 Presentation Transcript

    • Matthew Kilner – IBM Java L3 Service – Core team lead 23rd September 2013 Windows Debugging Tools © 2013 IBM Corporation
    • Important Disclaimers THE INFORMATION CONTAINED IN THIS PRESENTATION IS PROVIDED FOR INFORMATIONAL PURPOSES ONLY. WHILST EFFORTS WERE MADE TO VERIFY THE COMPLETENESS AND ACCURACY OF THE INFORMATION CONTAINED IN THIS PRESENTATION, IT IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. ALL PERFORMANCE DATA INCLUDED IN THIS PRESENTATION HAVE BEEN GATHERED IN A CONTROLLED ENVIRONMENT. YOUR OWN TEST RESULTS MAY VARY BASED ON HARDWARE, SOFTWARE OR INFRASTRUCTURE DIFFERENCES. ALL DATA INCLUDED IN THIS PRESENTATION ARE MEANT TO BE USED ONLY AS A GUIDE. IN ADDITION, THE INFORMATION CONTAINED IN THIS PRESENTATION IS BASED ON IBM’S CURRENT PRODUCT PLANS AND STRATEGY, WHICH ARE SUBJECT TO CHANGE BY IBM, WITHOUT NOTICE. IBM AND ITS AFFILIATED COMPANIES SHALL NOT BE RESPONSIBLE FOR ANY DAMAGES ARISING OUT OF THE USE OF, OR OTHERWISE RELATED TO, THIS PRESENTATION OR ANY OTHER DOCUMENTATION. NOTHING CONTAINED IN THIS PRESENTATION IS INTENDED TO, OR SHALL HAVE THE EFFECT OF: - CREATING ANY WARRANT OR REPRESENTATION FROM IBM, ITS AFFILIATED COMPANIES OR ITS OR THEIR SUPPLIERS AND/OR LICENSORS 2 © 2013 IBM Corporation
    • About me  Matthew Kilner   Work for IBM – 13 years working on IBM Java • Memory Management • Class Sharing • RAS – Currently leading the Core customer support team.   Contact info – kilnerm@uk.ibm.com – Twitter: @IBMJTC – Youtube: IBM_JTC   Visit the IBM booth #5112 and meet other IBM developers at JavaOne 2013 3 © 2013 IBM Corporation
    • What should you get from this talk?  Awareness of some excellent windows tools: – A high level understanding of what it is they let you do, – Details of where to obtain the tools, – Details of where to find out how to use them. –  Some useful tips on working with the tools © 2013 IBM Corporation
    • General tips  Most Microsoft tools require symbol files, often known as .pdb, to enable method names and source lines in stack traces.  Always keep your symbols in one place and keep a pre-configured Symbol path to hand. • I will use generic paths for the JDK symbol files and overlay those paths with the correct level of symbol for the JDK I am debugging at the time. • JDK symbol files ship in the majority of the IBM JDK packaging formats.  Remember to include the MS symbol server: • SRV*<local symbol store>*http://msdl.microsoft.com/download/symbols 5 © 2013 IBM Corporation
    • PDB files  PDB files are an essential part of effective debugging   The following link provides deailed information on how to generate them: • http://msdn.microsoft.com/en-us/library/aa260783%28v=vs.60%29.aspx   Requires the addition of compile and link time flags. 6 © 2013 IBM Corporation
    • IBM Presentation Template Full Version The tools  Windows SDK tools: – UMDH – ADPlus  Sysinternals tools: – VMMAP – Procdump  Standard tools: – Performance Monitor  Stand alone tools: – Debug Diagnostics 7 © 2013 IBM Corporation
    • Usefulness is contextual  A tools usefulness is limited to a number of problem types or tasks you are working on.   Most of these tools have been “discovered” during analysis of real world problems. – Predominantly related to Virtual addressing problems. –  Several of the tools have capabilities beyond the use described today. 8 © 2013 IBM Corporation
    • Monitoring the system / application  There are two common problem scenarios where it is desirable to understand more about how a system or application is performing: – Virtual Addressing problems – Performance problems • Slow responsiveness, High CPU usage, etc....   9 © 2013 IBM Corporation
    • Performance Monitor  Where to get it: • Included in you windows installation  What is it: –A powerful monitoring tool  What does it let us do: –Monitor and analyse performance data in real time or from a log file. –Can track almost every aspect of system performance.  How do you use it: –Launched by: • Entering “perfmon” in your run dialog, or • Via the Performance Monitor entry on the start menu –Read the in-built help documentation. 10 © 2013 IBM Corporation
    • Performance Monitor – monitoring process size  The most common use is for accurate measurement of a processes size while debugging virtual addressing issues.   Define a data collector Set to record the following counters on the process object: • Virtual bytes • Private bytes  • 11 © 2013 IBM Corporation
    • Performance Monitor – monitoring process size  For live monitoring add the same counters to the performance monitor view   When loading Data collection sets back into the tool you will see similar output.  • 12 © 2013 IBM Corporation
    • Performance Monitor – other uses  We primarily use it for monitoring process sizes which only scratches the surface of its practical uses.  The ability to monitor nearly anything you want on one or more processes such as: • Monitoring per thread CPU consumption across a process. • Monitoring the power consumption of individual CPU's. 13 © 2013 IBM Corporation
    • Capturing Dumps  Occasionally we see problems where the JDK does not write dumps when we expect it to. – This provides a real challenge for continuing problem analysis. –  We use two tools that help capture dumps in this scenario: – ADPlus (Windows SDK) – Procmon (Sysinternals) –  14 © 2013 IBM Corporation
    • ADPlus  Where to get it: • http://msdn.microsoft.com/en-us/library/windows/desktop/hh852363.aspx • Part of the Windows SDK  What is it: – A useful tool for generating system dumps – Can also be used in hang situations  What does it let us do: – Monitor a process and generate system dumps on: • User request • Process termination • First and second chance exceptions.  How do you use it: – Documentation is included in the install directory • <Path to installation>Windows Kits8.0Debuggersx64ADPlus.doc 15 © 2013 IBM Corporation
    • Procdump  Where to get it: • http://technet.microsoft.com/en-us/sysinternals/dd996900  What is it: –A useful tool for generating system dumps  What does it let us do: –Generate system dumps in a multitude of scenarios: • The process exceeding a threshold on any performance counter • On process exit • On first and second chance exceptions.  How do you use it: –Command line utility • See the link above for examples, or • Run the tool with no options. 16 © 2013 IBM Corporation
    • Procdump – usage examples  Example: Write up to 3 dumps of a process named 'consume' when it exceeds 20% CPU usage for five seconds to the directory c:dumpconsume with the name consume.dmp: • C:>procdump -c 20 -s 5 -n 3 -o consume c:dumpconsume   Example: Write 3 dumps 5 seconds apart: • C:>procdump -s 5 -n 3 notepad.exe notepad.dmp   Example: Launch a process and then monitor it for excessive CPU usage: • C:>procdump -c 30 -s 10 -x consume.exe consume.dmp 17 © 2013 IBM Corporation
    • Understanding Virtual Memory Growth  A common problem we encounter is the virtual address space of processes growing and either: – Causing process termination due to an inability to allocate memory, or – Causing severe performance issues due to paging   There are three excellent tools available on Windows to help understand what is causing the growth: – UMDH (Windows SDK) – Debug Diag ( Stand alone) – VMMAP (Sysinternals)   18 © 2013 IBM Corporation
    • UMDH  Where to get it: • http://msdn.microsoft.com/en-us/library/windows/desktop/hh852363.aspx • Part of the Windows SDK  What is it: –A tool that tracks and analyzes windows heap allocations within a process  What does it let us do: – Take snapshots of memory footprint within a process broken down by allocation stack. – Compare snapshots and summarize the key differences. – Identify stacks for specific allocation sizes.  How do you use it: – Basic usage instructions: • http://msdn.microsoft.com/en-us/library/windows/hardware/ff558947%28v=vs.85%29.aspx – More detail is available in this Microsoft knowledge base article: • http://support.microsoft.com/kb/268343 – 19 © 2013 IBM Corporation
    • UMDH – sample output // Connecting to process 3312 ... // Process 3312 opened handle=48. // Loaded modules:  Snapshot files contain the raw data from a point of execution of the process, you will see: – A summary of loaded modules – A summary fo the data format: – A series of allocations: –  When you have multiple snapshots you run the compare tool. // Base Size Module // 13A0000 // 773B0000 1A9000 C:windowsSYSTEM32ntdll.dll 35000 C:Program FilesIBMCodeRallyjdkjrebinjavaw.exe // Process modules enumerated. // _NT_SYMBOL_PATH set by default to C:windowssymbols // Debug library initialized ... DBGHELP: ntdll - export symbols *- - - - - - - - - - Start of data for heap @ 310000 - - - - - - - - - REQUESTED bytes + OVERHEAD at ADDRESS by BackTraceID STACK if not already dumped. *- - - - - - - - - - Heap 310000 Hogs - - - - - - - - - 28 bytes + 28 at 311310 by BackTrace720C0 7744BBED 773DC2E2 773F357E 773F1937 773DC34E 20 © 2013 IBM Corporation
    • UMDH – comparing snapshots  To run a comparison you execute: • UMDH <log 1> <log 2> > <output file>   Once the compare completes the output file contains a list of the call stacks that have differing memory usage between the dumps sorted by overall size: – + 550376 ( 550776 - 400) 12 allocsBackTrace33599F60 + 11 ( 12 1) BackTrace33599F60 allocations ntdll!MD5Final+A63D J9PRT26!j9mem_allocate_memory+74 (c:cygwinhomeforemansandboxjvm-binsrcj9portcommonj9memtag.c, 184) j9vm26!allocateMemoryForSegment+A0 (c:cygwinhomeforemansandboxjvm-binsrcj9vmsegment.c, 216) j9vm26!allocateVirtualMemorySegmentInListInternal+E6 (c:cygwinhomeforemansandboxjvm-binsrcj9vmsegment.c, 357) j9vm26!allocateFixedMemorySegmentInList+A8 (c:cygwinhomeforemansandboxjvm-binsrcj9vmsegment.c, 319) 21 © 2013 IBM Corporation
    • UMDH – important considerations  The tool uses snapshots of outstanding allocations so a degree of additional interpretation is needed.   Do not assume the top consumer between two snapshots is the cause of the problem • Also consider the volume of allocations, particularly for small chunks which have implications in terms of fragmentation pre Vista/Server 2008. •  Best used in conjunction with performance monitor and over several iterations of snapshots.   Longer durations between snapshots often make it easier to identify leaks 22 © 2013 IBM Corporation
    • Debug Diagnostic Tool  Where to get it: • http://www.microsoft.com/en-us/download/details.aspx?id=26798  What is it: –A tool providing a range of capability for analysis of crash, hang or native memory isssues.  What does it let us do: –Analysis of native memory issues. –Basic analysis of crash or hang scenarios. –Attaches to live processes and can be used to generate dumps at request or by rule configuration. –Designed for minimal performance impact  How do you use it: – Basic Instructions: • http://support.microsoft.com/kb/2580960 –Read the whitepaper: • http://www.microsoft.com/download/en/details.aspx?id=23521 –Use the in-built help documentation. 23 © 2013 IBM Corporation
    • Debug Diagnostic Tool – native memory analysis  Our main use of this tool is for debuging native memory issues. –The key thing to remember is to always inject the leak track dll, without this the reports you generate will be basic. – –The optimal method of analysis involves taking multple dumps of the process as its virtual size grows, understanding the rate of growth between dumps is important and can be monitored using the performance monitor tool. – –Dumps can be generated directly from the UI or rules can be configured to generate dumps after a set amount of time or increase in memory usage. – –Once dumps are generated you load them into the tool and select to run the Memory pressure script. When complete the tool will open the report in a browser window. – 24 © 2013 IBM Corporation
    • Debug Diagnostic Tool – native memory report  The report is broken into distinct sections: • Virtual Memory Analysis Report • Heap Analysis Report • Leak Analysis Report (only with leak tracking) •  The following extracts are taken from results of monitoring a 64 bit process which has a maximum address space of 8 Tbytes.  Each report starts with a summary which contains some key pieces of information:  25 © 2013 IBM Corporation
    • Debug Diagnostic Tool – Virtual Memory Analysis Report  Contains: • Virtual Memory Summary • Virtual Memory Details • Virtual Allocation Summary • Loaded Module Summary • Thread Summary  The most pertinent details are the first three elements • 26 © 2013 IBM Corporation
    • Debug Diagnostic Tool – Heap Analysis Report  The heap analysis report contains information on the Heaps in use within the process.   Not to be confused with the JVM's heap.  27 © 2013 IBM Corporation
    • Debug Diagnostic Tool – Heap Details  For each heap you get: – A summary – Information on the Heap segments – Information on allocations  The Summary:  28 © 2013 IBM Corporation
    • Debug Diagnostic Tool – Heap Details  The Segment info: 29 © 2013 IBM Corporation
    • Debug Diagnostic Tool – Heap Details  The Allocation Summary: 30 © 2013 IBM Corporation
    • Debug Diagnostic Tool – Leak Analysis Report  Only available when the leaktrack dll is injected to the process being monitored.  The most useful aspect is the summary of allocations at the module level: – By Count: –  31 © 2013 IBM Corporation
    • Debug Diagnostic Tool – Leak Analysis Report – and by Size: –  32 © 2013 IBM Corporation
    • Debug Diagnostic Tool – Module Summary  For each module we get a more detailed breakdown including a summary:  33 © 2013 IBM Corporation
    • Debug Diagnostic Tool – Module Summary  We get a summary of functions by count and allocated size:  34 © 2013 IBM Corporation
    • Debug Diagnostic Tool – Module Summary  And for each function additional detail:  35 © 2013 IBM Corporation
    • Debug Diagnostic Tool – Module Summary  Allocation count and size breakdown:   36 © 2013 IBM Corporation
    • Debug Diagnostic Tool – Module Summary  And sample call stacks for prominent sizes:   37 © 2013 IBM Corporation
    • Debug Diagnostic Tool – Drawing conclusions  The tool provides a lot of information, but doesn't do comparison of multiple data sets.  Using a spreadsheet to capture the pertinent data points highlighted over the previous slides will build a picture that allow you to see the key areas that are causing growth. – You can even plot the data for easier visualization   38 © 2013 IBM Corporation
    • VMMAP  Where to get it: • http://technet.microsoft.com/en-us/sysinternals/dd535533  What is it: –A tool for analysis the virtual and physical memory usage of a windows process  What does it let us do: –Attaches to live processes for run time analysis and allows for export of process information in a variety of formats. –Provides a facility to launch a process and trace it which utilizes additional instrumentation that enables us to identify the stacks allocating into specific regions of memory. –Allows us to compare the contents of the address space between two different points in time, when tracing is enabled you can see exactly who is allocating into regions. –At any time you can flush the physical memory the process is using meaning this is a great tool for seeing just how much memory a particular feature of your application uses. – – 39 © 2013 IBM Corporation
    • VMMAP  Having attached or launched a process you will see a window like this:  Each time you refresh the view a new snapshot is created 40 © 2013 IBM Corporation
    • VMMAP – Comparing snapshots  First you have to open the timeline window:       Next you left click and drag to select the timeframe you wish to view: • Snapshots are represented by the small black lines: 41 © 2013 IBM Corporation
    • VMMAP – Comparing snapshots  Once selected you will see: 42 © 2013 IBM Corporation
    • VMMAP – Comparing snapshots  Address ranges that are in the most recent snapshot but not in the earliest are highlighted in bright green.  Address ranges that were in the earliest snapshot but not in the current are highlighted in bright red.  If you started the process under full tracing you can select a “HEAP” and view the allocations made within it between the snapshots, for each allocation you can also view the call stack.   43 © 2013 IBM Corporation
    • VMMAP – other features  When tracing is enabled you have the option to view a trace of key memory operations by selecting the trace option which lists: • Allocation sites • Number of allocations for the site • The number of bytes allocated • The type of operation •  You can also select any site and choose to view its stack. 44 © 2013 IBM Corporation
    • Q&A  Questions 45 © 2013 IBM Corporation
    • IBM@JavaOne http://ibm.co/JavaOne2013 46 © 2013 IBM Corporation