SlideShare a Scribd company logo
Software Trace and
Memory Dump Analysis
Presenter: Dmitry Vostokov
Memory Dump Analysis Services
Prerequisites
Experience in software
troubleshooting and reading
software logs
Advantage: Citrix CDF and
Microsoft ETW trace analysis
including Process Monitor logs
© 2011 Memory Dump Analysis Services
Agenda
 Memory Dump Analysis Services
 Root Cause Analysis Methodology
 Software Traces and Memory Dumps
 Examples
© 2011 Memory Dump Analysis Services
MDA Services
 Memory Dump Analysis Audit
 Software Trace Analysis Audit (New)
 Software Error Reporting Audit
 Remote Training
 Debugging Bureau
 Tool Objects and EasyDbg
Powered by DA+TA
DumpAnalysis.org + TraceAnaysis.org
© 2011 Memory Dump Analysis Services
A.C.P. Root Cause Analysis
© 2011 Memory Dump Analysis Services
Artifacts
Checklists
Patterns
Checklists and patterns
as best practices
Iterative and Incremental
DA+TA
 DA: Dump Artifact / Dump Analysis
Memory snapshots: process, kernel, physical memory dumps
 TA: Trace Artifact / Trace Analysis
Software traces: Event Tracing for Windows, logs
© 2011 Memory Dump Analysis Services
Spatiality vs. Narrativity
© 2011 Memory Dump Analysis Services
Narrativity
Spartiality
Software
Trace
Memory Dump
Software trace as software narrative,
the story of a computation
Tools for Artifact Analysis
Memory dumps:
 WinDbg from Debugging Tools for Windows
 Notepad (textual debugger logs)
Software traces:
 CDFAnalyzer* / CDFControl from Citrix
 Process Monitor* from Microsoft
* supports adjoint threads
© 2011 Memory Dump Analysis Services
Checklists for Analysis
Memory dumps:
http://www.dumpanalysis.org/blog/index.php/2007/06/
20/crash-dump-analysis-checklist/
Software traces:
http://www.dumpanalysis.org/blog/index.php/2011/03/
10/software-trace-analysis-checklist/
© 2011 Memory Dump Analysis Services
Software Behavior Patterns
 Memory dump and software trace
 Examples: Spiking Thread, Discontinuity
 +200 patterns (DA+TA)
 DumpAnalysis.org
© 2011 Memory Dump Analysis Services
DA: Software Behavior
 Memory dump: a memory snapshot
 Definition, partial classification and
historical list
 Pattern identification case studies
© 2011 Memory Dump Analysis Services
TA: Software Behavior
“Imagine you got a software trace from hundreds of modules
you haven’t written or haven’t seen source code of...”
 Software trace: a sequence of memory
fragments ordered in time
 Definition, and historical list
 Pattern identification case studies
© 2011 Memory Dump Analysis Services
CDFAnalyzer Filters
© 2011 Memory Dump Analysis Services
Threads
Time
# PID TID Time Message
Time
# PID TID Time Message
© 2011 Memory Dump Analysis Services
Adjoint Threads
© 2011 Memory Dump Analysis Services
Time
# PID TID Time Message
Time
# PID TID Time Message (ATID)
Significant Event
csrss.exe
winlogon.exe
LogonUI.exe
userinit.exe
…
Custom events: CDFMarker
© 2011 Memory Dump Analysis Services
Time
# PID TID Time Message
Discontinuity
© 2011 Memory Dump Analysis Services
…
14:23:02.146
14:23:02.345
14:31:10.254
14:31:10.341
…
Time
# PID TID Time Message
No Activity
Expecting messages from Module X
Absence of such messages may
suggest that a process or a thread was
hang / blocked
© 2011 Memory Dump Analysis Services
Guest Component
Sudden appearance of an unexpected
module, for example, werfault.exe or
faultrep.dll
© 2011 Memory Dump Analysis Services
Statement Current
The flood of messages
Normal case: 15 msg/s
Abnormal case: 3500 msg/s
May point to a CPU spike
© 2011 Memory Dump Analysis Services
Resources
 DumpAnalysis.org
 Pattern-Driven Memory Dump Analysis
 Memory Dump and Trace Analysis: A Unified Pattern Approach
 Introduction to Pattern-Driven Software Problem Solving
 Advanced Software Debugging Reference:
 OpenTask publishes this talk with extra case studies
(ISBN: 978-1908043238)
© 2011 Memory Dump Analysis Services
More Resources
August remote training season:
 Accelerated Windows Memory Dump Analysis
 Complete Physical Memory Dump Analysis
Visit Memory Dump Analysis Services for registration details:
www.DumpAnalysis.com
© 2011 Memory Dump Analysis Services
Free Summer Webinars
 The Old New Crash: Cloud Memory Dump
Analysis (June 6th)
 Cyber Warfare Memory Dump Analysis
(forthcoming in July-August)
Visit Memory Dump Analysis Services for registration details:
www.DumpAnalysis.com
© 2011 Memory Dump Analysis Services
Q&A
Please send your feedback using the contact
form on DumpAnalysis.com
© 2011 Memory Dump Analysis Services
Thank you!
© 2011 Memory Dump Analysis Services
Join DA+TA Facebook Group

More Related Content

Similar to Software Trace and Memory Dump Analysis: Patterns, Tools, Processes and Best Practices

Lists of tech acronyms
Lists of tech acronymsLists of tech acronyms
Lists of tech acronyms
bc dalai
 
Building an Observability Platform in 389 Difficult Steps
Building an Observability Platform in 389 Difficult StepsBuilding an Observability Platform in 389 Difficult Steps
Building an Observability Platform in 389 Difficult Steps
DigitalOcean
 
Next Gen Data Modeling in the Open Data Platform With Doron Porat and Liran Y...
Next Gen Data Modeling in the Open Data Platform With Doron Porat and Liran Y...Next Gen Data Modeling in the Open Data Platform With Doron Porat and Liran Y...
Next Gen Data Modeling in the Open Data Platform With Doron Porat and Liran Y...
HostedbyConfluent
 
OSMC 2011 | Safed as an agent for supporting a central collection of events w...
OSMC 2011 | Safed as an agent for supporting a central collection of events w...OSMC 2011 | Safed as an agent for supporting a central collection of events w...
OSMC 2011 | Safed as an agent for supporting a central collection of events w...
NETWAYS
 
File000125
File000125File000125
File000125
Desmond Devendran
 
Spug pt session2 - debuggingl
Spug pt session2 - debugginglSpug pt session2 - debuggingl
Spug pt session2 - debuggingl
Comunidade Portuguesa de SharePoiint
 
LogChaos: Challenges and Opportunities of Security Log Standardization
LogChaos: Challenges and Opportunities of Security Log StandardizationLogChaos: Challenges and Opportunities of Security Log Standardization
LogChaos: Challenges and Opportunities of Security Log Standardization
Anton Chuvakin
 
SQL Server 2008 R2 StreamInsight
SQL Server 2008 R2 StreamInsightSQL Server 2008 R2 StreamInsight
SQL Server 2008 R2 StreamInsight
Eduardo Castro
 
FOSDEM2017 - Janus Event Handlers
FOSDEM2017 - Janus Event HandlersFOSDEM2017 - Janus Event Handlers
FOSDEM2017 - Janus Event Handlers
Lorenzo Miniero
 
Responding to extended events in near real time
Responding to extended events in near real timeResponding to extended events in near real time
Responding to extended events in near real time
Gianluca Sartori
 
Methods and Instruments for the new Digital Forensics Environments
Methods and Instruments for the new Digital Forensics EnvironmentsMethods and Instruments for the new Digital Forensics Environments
Methods and Instruments for the new Digital Forensics Environments
piccimario
 
Microsoft SQL Server - StreamInsight Overview Presentation
Microsoft SQL Server - StreamInsight Overview PresentationMicrosoft SQL Server - StreamInsight Overview Presentation
Microsoft SQL Server - StreamInsight Overview Presentation
Microsoft Private Cloud
 
Anti-Forensic Rootkits
Anti-Forensic RootkitsAnti-Forensic Rootkits
Anti-Forensic Rootkits
amiable_indian
 
Open source network forensics and advanced pcap analysis
Open source network forensics and advanced pcap analysisOpen source network forensics and advanced pcap analysis
Open source network forensics and advanced pcap analysis
GTKlondike
 
Production debugging web applications
Production debugging web applicationsProduction debugging web applications
Production debugging web applications
Ido Flatow
 
Using Embedded Linux for Infrastructure Systems
Using Embedded Linux for Infrastructure SystemsUsing Embedded Linux for Infrastructure Systems
Using Embedded Linux for Infrastructure Systems
Yoshitake Kobayashi
 
Malware Narratives
Malware NarrativesMalware Narratives
Malware Narratives
Dmitry Vostokov
 
Log Mining: Beyond Log Analysis
Log Mining: Beyond Log AnalysisLog Mining: Beyond Log Analysis
Log Mining: Beyond Log Analysis
Anton Chuvakin
 
lec4.ppt system calls explained in detail
lec4.ppt system calls explained in detaillec4.ppt system calls explained in detail
lec4.ppt system calls explained in detail
frp60658
 
Digital Forensics
Digital ForensicsDigital Forensics
Digital Forensics
Oldsun
 

Similar to Software Trace and Memory Dump Analysis: Patterns, Tools, Processes and Best Practices (20)

Lists of tech acronyms
Lists of tech acronymsLists of tech acronyms
Lists of tech acronyms
 
Building an Observability Platform in 389 Difficult Steps
Building an Observability Platform in 389 Difficult StepsBuilding an Observability Platform in 389 Difficult Steps
Building an Observability Platform in 389 Difficult Steps
 
Next Gen Data Modeling in the Open Data Platform With Doron Porat and Liran Y...
Next Gen Data Modeling in the Open Data Platform With Doron Porat and Liran Y...Next Gen Data Modeling in the Open Data Platform With Doron Porat and Liran Y...
Next Gen Data Modeling in the Open Data Platform With Doron Porat and Liran Y...
 
OSMC 2011 | Safed as an agent for supporting a central collection of events w...
OSMC 2011 | Safed as an agent for supporting a central collection of events w...OSMC 2011 | Safed as an agent for supporting a central collection of events w...
OSMC 2011 | Safed as an agent for supporting a central collection of events w...
 
File000125
File000125File000125
File000125
 
Spug pt session2 - debuggingl
Spug pt session2 - debugginglSpug pt session2 - debuggingl
Spug pt session2 - debuggingl
 
LogChaos: Challenges and Opportunities of Security Log Standardization
LogChaos: Challenges and Opportunities of Security Log StandardizationLogChaos: Challenges and Opportunities of Security Log Standardization
LogChaos: Challenges and Opportunities of Security Log Standardization
 
SQL Server 2008 R2 StreamInsight
SQL Server 2008 R2 StreamInsightSQL Server 2008 R2 StreamInsight
SQL Server 2008 R2 StreamInsight
 
FOSDEM2017 - Janus Event Handlers
FOSDEM2017 - Janus Event HandlersFOSDEM2017 - Janus Event Handlers
FOSDEM2017 - Janus Event Handlers
 
Responding to extended events in near real time
Responding to extended events in near real timeResponding to extended events in near real time
Responding to extended events in near real time
 
Methods and Instruments for the new Digital Forensics Environments
Methods and Instruments for the new Digital Forensics EnvironmentsMethods and Instruments for the new Digital Forensics Environments
Methods and Instruments for the new Digital Forensics Environments
 
Microsoft SQL Server - StreamInsight Overview Presentation
Microsoft SQL Server - StreamInsight Overview PresentationMicrosoft SQL Server - StreamInsight Overview Presentation
Microsoft SQL Server - StreamInsight Overview Presentation
 
Anti-Forensic Rootkits
Anti-Forensic RootkitsAnti-Forensic Rootkits
Anti-Forensic Rootkits
 
Open source network forensics and advanced pcap analysis
Open source network forensics and advanced pcap analysisOpen source network forensics and advanced pcap analysis
Open source network forensics and advanced pcap analysis
 
Production debugging web applications
Production debugging web applicationsProduction debugging web applications
Production debugging web applications
 
Using Embedded Linux for Infrastructure Systems
Using Embedded Linux for Infrastructure SystemsUsing Embedded Linux for Infrastructure Systems
Using Embedded Linux for Infrastructure Systems
 
Malware Narratives
Malware NarrativesMalware Narratives
Malware Narratives
 
Log Mining: Beyond Log Analysis
Log Mining: Beyond Log AnalysisLog Mining: Beyond Log Analysis
Log Mining: Beyond Log Analysis
 
lec4.ppt system calls explained in detail
lec4.ppt system calls explained in detaillec4.ppt system calls explained in detail
lec4.ppt system calls explained in detail
 
Digital Forensics
Digital ForensicsDigital Forensics
Digital Forensics
 

More from Dmitry Vostokov

Accelerated Windows Debugging 3 training public slides
Accelerated Windows Debugging 3 training public slidesAccelerated Windows Debugging 3 training public slides
Accelerated Windows Debugging 3 training public slides
Dmitry Vostokov
 
Accelerated .NET Memory Dump Analysis training public slides
Accelerated .NET Memory Dump Analysis training public slidesAccelerated .NET Memory Dump Analysis training public slides
Accelerated .NET Memory Dump Analysis training public slides
Dmitry Vostokov
 
Debugging TV Frame 0x1C
Debugging TV Frame 0x1CDebugging TV Frame 0x1C
Debugging TV Frame 0x1C
Dmitry Vostokov
 
Debugging TV Frame 0x1A
Debugging TV Frame 0x1ADebugging TV Frame 0x1A
Debugging TV Frame 0x1A
Dmitry Vostokov
 
Debugging TV Frame 0x34
Debugging TV Frame 0x34Debugging TV Frame 0x34
Debugging TV Frame 0x34
Dmitry Vostokov
 
Debugging TV Frame 0x33
Debugging TV Frame 0x33Debugging TV Frame 0x33
Debugging TV Frame 0x33
Dmitry Vostokov
 
Debugging TV Frame 0x31
Debugging TV Frame 0x31Debugging TV Frame 0x31
Debugging TV Frame 0x31
Dmitry Vostokov
 
Debugging TV Frame 0x25
Debugging TV Frame 0x25Debugging TV Frame 0x25
Debugging TV Frame 0x25
Dmitry Vostokov
 
Debugging TV Frame 0x24
Debugging TV Frame 0x24Debugging TV Frame 0x24
Debugging TV Frame 0x24
Dmitry Vostokov
 
Debugging TV Frame 0x21
Debugging TV Frame 0x21Debugging TV Frame 0x21
Debugging TV Frame 0x21
Dmitry Vostokov
 
Debugging TV Frame 0x20
Debugging TV Frame 0x20Debugging TV Frame 0x20
Debugging TV Frame 0x20
Dmitry Vostokov
 
Debugging TV Frame 0x19
Debugging TV Frame 0x19Debugging TV Frame 0x19
Debugging TV Frame 0x19
Dmitry Vostokov
 
Debugging TV Frame 0x18
Debugging TV Frame 0x18Debugging TV Frame 0x18
Debugging TV Frame 0x18
Dmitry Vostokov
 
Debugging TV Frame 0x17
Debugging TV Frame 0x17Debugging TV Frame 0x17
Debugging TV Frame 0x17
Dmitry Vostokov
 
Debugging TV Frame 0x16
Debugging TV Frame 0x16Debugging TV Frame 0x16
Debugging TV Frame 0x16
Dmitry Vostokov
 
Debugging TV Frame 0x15
Debugging TV Frame 0x15Debugging TV Frame 0x15
Debugging TV Frame 0x15
Dmitry Vostokov
 
Debugging TV Frame 0x14
Debugging TV Frame 0x14Debugging TV Frame 0x14
Debugging TV Frame 0x14
Dmitry Vostokov
 
Debugging TV Frame 0x13
Debugging TV Frame 0x13Debugging TV Frame 0x13
Debugging TV Frame 0x13
Dmitry Vostokov
 
Debugging TV Frame 0x12
Debugging TV Frame 0x12Debugging TV Frame 0x12
Debugging TV Frame 0x12
Dmitry Vostokov
 
Debugging TV Frame 0x11
Debugging TV Frame 0x11Debugging TV Frame 0x11
Debugging TV Frame 0x11
Dmitry Vostokov
 

More from Dmitry Vostokov (20)

Accelerated Windows Debugging 3 training public slides
Accelerated Windows Debugging 3 training public slidesAccelerated Windows Debugging 3 training public slides
Accelerated Windows Debugging 3 training public slides
 
Accelerated .NET Memory Dump Analysis training public slides
Accelerated .NET Memory Dump Analysis training public slidesAccelerated .NET Memory Dump Analysis training public slides
Accelerated .NET Memory Dump Analysis training public slides
 
Debugging TV Frame 0x1C
Debugging TV Frame 0x1CDebugging TV Frame 0x1C
Debugging TV Frame 0x1C
 
Debugging TV Frame 0x1A
Debugging TV Frame 0x1ADebugging TV Frame 0x1A
Debugging TV Frame 0x1A
 
Debugging TV Frame 0x34
Debugging TV Frame 0x34Debugging TV Frame 0x34
Debugging TV Frame 0x34
 
Debugging TV Frame 0x33
Debugging TV Frame 0x33Debugging TV Frame 0x33
Debugging TV Frame 0x33
 
Debugging TV Frame 0x31
Debugging TV Frame 0x31Debugging TV Frame 0x31
Debugging TV Frame 0x31
 
Debugging TV Frame 0x25
Debugging TV Frame 0x25Debugging TV Frame 0x25
Debugging TV Frame 0x25
 
Debugging TV Frame 0x24
Debugging TV Frame 0x24Debugging TV Frame 0x24
Debugging TV Frame 0x24
 
Debugging TV Frame 0x21
Debugging TV Frame 0x21Debugging TV Frame 0x21
Debugging TV Frame 0x21
 
Debugging TV Frame 0x20
Debugging TV Frame 0x20Debugging TV Frame 0x20
Debugging TV Frame 0x20
 
Debugging TV Frame 0x19
Debugging TV Frame 0x19Debugging TV Frame 0x19
Debugging TV Frame 0x19
 
Debugging TV Frame 0x18
Debugging TV Frame 0x18Debugging TV Frame 0x18
Debugging TV Frame 0x18
 
Debugging TV Frame 0x17
Debugging TV Frame 0x17Debugging TV Frame 0x17
Debugging TV Frame 0x17
 
Debugging TV Frame 0x16
Debugging TV Frame 0x16Debugging TV Frame 0x16
Debugging TV Frame 0x16
 
Debugging TV Frame 0x15
Debugging TV Frame 0x15Debugging TV Frame 0x15
Debugging TV Frame 0x15
 
Debugging TV Frame 0x14
Debugging TV Frame 0x14Debugging TV Frame 0x14
Debugging TV Frame 0x14
 
Debugging TV Frame 0x13
Debugging TV Frame 0x13Debugging TV Frame 0x13
Debugging TV Frame 0x13
 
Debugging TV Frame 0x12
Debugging TV Frame 0x12Debugging TV Frame 0x12
Debugging TV Frame 0x12
 
Debugging TV Frame 0x11
Debugging TV Frame 0x11Debugging TV Frame 0x11
Debugging TV Frame 0x11
 

Recently uploaded

What is Continuous Testing in DevOps - A Definitive Guide.pdf
What is Continuous Testing in DevOps - A Definitive Guide.pdfWhat is Continuous Testing in DevOps - A Definitive Guide.pdf
What is Continuous Testing in DevOps - A Definitive Guide.pdf
kalichargn70th171
 
What’s New in VictoriaLogs - Q2 2024 Update
What’s New in VictoriaLogs - Q2 2024 UpdateWhat’s New in VictoriaLogs - Q2 2024 Update
What’s New in VictoriaLogs - Q2 2024 Update
VictoriaMetrics
 
Streamlining End-to-End Testing Automation
Streamlining End-to-End Testing AutomationStreamlining End-to-End Testing Automation
Streamlining End-to-End Testing Automation
Anand Bagmar
 
美洲杯赔率投注网【​网址​🎉3977·EE​🎉】
美洲杯赔率投注网【​网址​🎉3977·EE​🎉】美洲杯赔率投注网【​网址​🎉3977·EE​🎉】
美洲杯赔率投注网【​网址​🎉3977·EE​🎉】
widenerjobeyrl638
 
Baha Majid WCA4Z IBM Z Customer Council Boston June 2024.pdf
Baha Majid WCA4Z IBM Z Customer Council Boston June 2024.pdfBaha Majid WCA4Z IBM Z Customer Council Boston June 2024.pdf
Baha Majid WCA4Z IBM Z Customer Council Boston June 2024.pdf
Baha Majid
 
A Comprehensive Guide on Implementing Real-World Mobile Testing Strategies fo...
A Comprehensive Guide on Implementing Real-World Mobile Testing Strategies fo...A Comprehensive Guide on Implementing Real-World Mobile Testing Strategies fo...
A Comprehensive Guide on Implementing Real-World Mobile Testing Strategies fo...
kalichargn70th171
 
The Power of Visual Regression Testing_ Why It Is Critical for Enterprise App...
The Power of Visual Regression Testing_ Why It Is Critical for Enterprise App...The Power of Visual Regression Testing_ Why It Is Critical for Enterprise App...
The Power of Visual Regression Testing_ Why It Is Critical for Enterprise App...
kalichargn70th171
 
Alluxio Webinar | 10x Faster Trino Queries on Your Data Platform
Alluxio Webinar | 10x Faster Trino Queries on Your Data PlatformAlluxio Webinar | 10x Faster Trino Queries on Your Data Platform
Alluxio Webinar | 10x Faster Trino Queries on Your Data Platform
Alluxio, Inc.
 
The Ultimate Guide to Top 36 DevOps Testing Tools for 2024.pdf
The Ultimate Guide to Top 36 DevOps Testing Tools for 2024.pdfThe Ultimate Guide to Top 36 DevOps Testing Tools for 2024.pdf
The Ultimate Guide to Top 36 DevOps Testing Tools for 2024.pdf
kalichargn70th171
 
一比一原版(sdsu毕业证书)圣地亚哥州立大学毕业证如何办理
一比一原版(sdsu毕业证书)圣地亚哥州立大学毕业证如何办理一比一原版(sdsu毕业证书)圣地亚哥州立大学毕业证如何办理
一比一原版(sdsu毕业证书)圣地亚哥州立大学毕业证如何办理
kgyxske
 
Operational ease MuleSoft and Salesforce Service Cloud Solution v1.0.pptx
Operational ease MuleSoft and Salesforce Service Cloud Solution v1.0.pptxOperational ease MuleSoft and Salesforce Service Cloud Solution v1.0.pptx
Operational ease MuleSoft and Salesforce Service Cloud Solution v1.0.pptx
sandeepmenon62
 
Beginner's Guide to Observability@Devoxx PL 2024
Beginner's  Guide to Observability@Devoxx PL 2024Beginner's  Guide to Observability@Devoxx PL 2024
Beginner's Guide to Observability@Devoxx PL 2024
michniczscribd
 
Building the Ideal CI-CD Pipeline_ Achieving Visual Perfection
Building the Ideal CI-CD Pipeline_ Achieving Visual PerfectionBuilding the Ideal CI-CD Pipeline_ Achieving Visual Perfection
Building the Ideal CI-CD Pipeline_ Achieving Visual Perfection
Applitools
 
ACE - Team 24 Wrapup event at ahmedabad.
ACE - Team 24 Wrapup event at ahmedabad.ACE - Team 24 Wrapup event at ahmedabad.
ACE - Team 24 Wrapup event at ahmedabad.
Maitrey Patel
 
Microsoft-Power-Platform-Adoption-Planning.pptx
Microsoft-Power-Platform-Adoption-Planning.pptxMicrosoft-Power-Platform-Adoption-Planning.pptx
Microsoft-Power-Platform-Adoption-Planning.pptx
jrodriguezq3110
 
Stork Product Overview: An AI-Powered Autonomous Delivery Fleet
Stork Product Overview: An AI-Powered Autonomous Delivery FleetStork Product Overview: An AI-Powered Autonomous Delivery Fleet
Stork Product Overview: An AI-Powered Autonomous Delivery Fleet
Vince Scalabrino
 
Enhanced Screen Flows UI/UX using SLDS with Tom Kitt
Enhanced Screen Flows UI/UX using SLDS with Tom KittEnhanced Screen Flows UI/UX using SLDS with Tom Kitt
Enhanced Screen Flows UI/UX using SLDS with Tom Kitt
Peter Caitens
 
The Comprehensive Guide to Validating Audio-Visual Performances.pdf
The Comprehensive Guide to Validating Audio-Visual Performances.pdfThe Comprehensive Guide to Validating Audio-Visual Performances.pdf
The Comprehensive Guide to Validating Audio-Visual Performances.pdf
kalichargn70th171
 
Strengthening Web Development with CommandBox 6: Seamless Transition and Scal...
Strengthening Web Development with CommandBox 6: Seamless Transition and Scal...Strengthening Web Development with CommandBox 6: Seamless Transition and Scal...
Strengthening Web Development with CommandBox 6: Seamless Transition and Scal...
Ortus Solutions, Corp
 
How GenAI Can Improve Supplier Performance Management.pdf
How GenAI Can Improve Supplier Performance Management.pdfHow GenAI Can Improve Supplier Performance Management.pdf
How GenAI Can Improve Supplier Performance Management.pdf
Zycus
 

Recently uploaded (20)

What is Continuous Testing in DevOps - A Definitive Guide.pdf
What is Continuous Testing in DevOps - A Definitive Guide.pdfWhat is Continuous Testing in DevOps - A Definitive Guide.pdf
What is Continuous Testing in DevOps - A Definitive Guide.pdf
 
What’s New in VictoriaLogs - Q2 2024 Update
What’s New in VictoriaLogs - Q2 2024 UpdateWhat’s New in VictoriaLogs - Q2 2024 Update
What’s New in VictoriaLogs - Q2 2024 Update
 
Streamlining End-to-End Testing Automation
Streamlining End-to-End Testing AutomationStreamlining End-to-End Testing Automation
Streamlining End-to-End Testing Automation
 
美洲杯赔率投注网【​网址​🎉3977·EE​🎉】
美洲杯赔率投注网【​网址​🎉3977·EE​🎉】美洲杯赔率投注网【​网址​🎉3977·EE​🎉】
美洲杯赔率投注网【​网址​🎉3977·EE​🎉】
 
Baha Majid WCA4Z IBM Z Customer Council Boston June 2024.pdf
Baha Majid WCA4Z IBM Z Customer Council Boston June 2024.pdfBaha Majid WCA4Z IBM Z Customer Council Boston June 2024.pdf
Baha Majid WCA4Z IBM Z Customer Council Boston June 2024.pdf
 
A Comprehensive Guide on Implementing Real-World Mobile Testing Strategies fo...
A Comprehensive Guide on Implementing Real-World Mobile Testing Strategies fo...A Comprehensive Guide on Implementing Real-World Mobile Testing Strategies fo...
A Comprehensive Guide on Implementing Real-World Mobile Testing Strategies fo...
 
The Power of Visual Regression Testing_ Why It Is Critical for Enterprise App...
The Power of Visual Regression Testing_ Why It Is Critical for Enterprise App...The Power of Visual Regression Testing_ Why It Is Critical for Enterprise App...
The Power of Visual Regression Testing_ Why It Is Critical for Enterprise App...
 
Alluxio Webinar | 10x Faster Trino Queries on Your Data Platform
Alluxio Webinar | 10x Faster Trino Queries on Your Data PlatformAlluxio Webinar | 10x Faster Trino Queries on Your Data Platform
Alluxio Webinar | 10x Faster Trino Queries on Your Data Platform
 
The Ultimate Guide to Top 36 DevOps Testing Tools for 2024.pdf
The Ultimate Guide to Top 36 DevOps Testing Tools for 2024.pdfThe Ultimate Guide to Top 36 DevOps Testing Tools for 2024.pdf
The Ultimate Guide to Top 36 DevOps Testing Tools for 2024.pdf
 
一比一原版(sdsu毕业证书)圣地亚哥州立大学毕业证如何办理
一比一原版(sdsu毕业证书)圣地亚哥州立大学毕业证如何办理一比一原版(sdsu毕业证书)圣地亚哥州立大学毕业证如何办理
一比一原版(sdsu毕业证书)圣地亚哥州立大学毕业证如何办理
 
Operational ease MuleSoft and Salesforce Service Cloud Solution v1.0.pptx
Operational ease MuleSoft and Salesforce Service Cloud Solution v1.0.pptxOperational ease MuleSoft and Salesforce Service Cloud Solution v1.0.pptx
Operational ease MuleSoft and Salesforce Service Cloud Solution v1.0.pptx
 
Beginner's Guide to Observability@Devoxx PL 2024
Beginner's  Guide to Observability@Devoxx PL 2024Beginner's  Guide to Observability@Devoxx PL 2024
Beginner's Guide to Observability@Devoxx PL 2024
 
Building the Ideal CI-CD Pipeline_ Achieving Visual Perfection
Building the Ideal CI-CD Pipeline_ Achieving Visual PerfectionBuilding the Ideal CI-CD Pipeline_ Achieving Visual Perfection
Building the Ideal CI-CD Pipeline_ Achieving Visual Perfection
 
ACE - Team 24 Wrapup event at ahmedabad.
ACE - Team 24 Wrapup event at ahmedabad.ACE - Team 24 Wrapup event at ahmedabad.
ACE - Team 24 Wrapup event at ahmedabad.
 
Microsoft-Power-Platform-Adoption-Planning.pptx
Microsoft-Power-Platform-Adoption-Planning.pptxMicrosoft-Power-Platform-Adoption-Planning.pptx
Microsoft-Power-Platform-Adoption-Planning.pptx
 
Stork Product Overview: An AI-Powered Autonomous Delivery Fleet
Stork Product Overview: An AI-Powered Autonomous Delivery FleetStork Product Overview: An AI-Powered Autonomous Delivery Fleet
Stork Product Overview: An AI-Powered Autonomous Delivery Fleet
 
Enhanced Screen Flows UI/UX using SLDS with Tom Kitt
Enhanced Screen Flows UI/UX using SLDS with Tom KittEnhanced Screen Flows UI/UX using SLDS with Tom Kitt
Enhanced Screen Flows UI/UX using SLDS with Tom Kitt
 
The Comprehensive Guide to Validating Audio-Visual Performances.pdf
The Comprehensive Guide to Validating Audio-Visual Performances.pdfThe Comprehensive Guide to Validating Audio-Visual Performances.pdf
The Comprehensive Guide to Validating Audio-Visual Performances.pdf
 
Strengthening Web Development with CommandBox 6: Seamless Transition and Scal...
Strengthening Web Development with CommandBox 6: Seamless Transition and Scal...Strengthening Web Development with CommandBox 6: Seamless Transition and Scal...
Strengthening Web Development with CommandBox 6: Seamless Transition and Scal...
 
How GenAI Can Improve Supplier Performance Management.pdf
How GenAI Can Improve Supplier Performance Management.pdfHow GenAI Can Improve Supplier Performance Management.pdf
How GenAI Can Improve Supplier Performance Management.pdf
 

Software Trace and Memory Dump Analysis: Patterns, Tools, Processes and Best Practices

  • 1. Software Trace and Memory Dump Analysis Presenter: Dmitry Vostokov Memory Dump Analysis Services
  • 2. Prerequisites Experience in software troubleshooting and reading software logs Advantage: Citrix CDF and Microsoft ETW trace analysis including Process Monitor logs © 2011 Memory Dump Analysis Services
  • 3. Agenda  Memory Dump Analysis Services  Root Cause Analysis Methodology  Software Traces and Memory Dumps  Examples © 2011 Memory Dump Analysis Services
  • 4. MDA Services  Memory Dump Analysis Audit  Software Trace Analysis Audit (New)  Software Error Reporting Audit  Remote Training  Debugging Bureau  Tool Objects and EasyDbg Powered by DA+TA DumpAnalysis.org + TraceAnaysis.org © 2011 Memory Dump Analysis Services
  • 5. A.C.P. Root Cause Analysis © 2011 Memory Dump Analysis Services Artifacts Checklists Patterns Checklists and patterns as best practices Iterative and Incremental
  • 6. DA+TA  DA: Dump Artifact / Dump Analysis Memory snapshots: process, kernel, physical memory dumps  TA: Trace Artifact / Trace Analysis Software traces: Event Tracing for Windows, logs © 2011 Memory Dump Analysis Services
  • 7. Spatiality vs. Narrativity © 2011 Memory Dump Analysis Services Narrativity Spartiality Software Trace Memory Dump Software trace as software narrative, the story of a computation
  • 8. Tools for Artifact Analysis Memory dumps:  WinDbg from Debugging Tools for Windows  Notepad (textual debugger logs) Software traces:  CDFAnalyzer* / CDFControl from Citrix  Process Monitor* from Microsoft * supports adjoint threads © 2011 Memory Dump Analysis Services
  • 9. Checklists for Analysis Memory dumps: http://www.dumpanalysis.org/blog/index.php/2007/06/ 20/crash-dump-analysis-checklist/ Software traces: http://www.dumpanalysis.org/blog/index.php/2011/03/ 10/software-trace-analysis-checklist/ © 2011 Memory Dump Analysis Services
  • 10. Software Behavior Patterns  Memory dump and software trace  Examples: Spiking Thread, Discontinuity  +200 patterns (DA+TA)  DumpAnalysis.org © 2011 Memory Dump Analysis Services
  • 11. DA: Software Behavior  Memory dump: a memory snapshot  Definition, partial classification and historical list  Pattern identification case studies © 2011 Memory Dump Analysis Services
  • 12. TA: Software Behavior “Imagine you got a software trace from hundreds of modules you haven’t written or haven’t seen source code of...”  Software trace: a sequence of memory fragments ordered in time  Definition, and historical list  Pattern identification case studies © 2011 Memory Dump Analysis Services
  • 13. CDFAnalyzer Filters © 2011 Memory Dump Analysis Services
  • 14. Threads Time # PID TID Time Message Time # PID TID Time Message © 2011 Memory Dump Analysis Services
  • 15. Adjoint Threads © 2011 Memory Dump Analysis Services Time # PID TID Time Message Time # PID TID Time Message (ATID)
  • 16. Significant Event csrss.exe winlogon.exe LogonUI.exe userinit.exe … Custom events: CDFMarker © 2011 Memory Dump Analysis Services Time # PID TID Time Message
  • 17. Discontinuity © 2011 Memory Dump Analysis Services … 14:23:02.146 14:23:02.345 14:31:10.254 14:31:10.341 … Time # PID TID Time Message
  • 18. No Activity Expecting messages from Module X Absence of such messages may suggest that a process or a thread was hang / blocked © 2011 Memory Dump Analysis Services
  • 19. Guest Component Sudden appearance of an unexpected module, for example, werfault.exe or faultrep.dll © 2011 Memory Dump Analysis Services
  • 20. Statement Current The flood of messages Normal case: 15 msg/s Abnormal case: 3500 msg/s May point to a CPU spike © 2011 Memory Dump Analysis Services
  • 21. Resources  DumpAnalysis.org  Pattern-Driven Memory Dump Analysis  Memory Dump and Trace Analysis: A Unified Pattern Approach  Introduction to Pattern-Driven Software Problem Solving  Advanced Software Debugging Reference:  OpenTask publishes this talk with extra case studies (ISBN: 978-1908043238) © 2011 Memory Dump Analysis Services
  • 22. More Resources August remote training season:  Accelerated Windows Memory Dump Analysis  Complete Physical Memory Dump Analysis Visit Memory Dump Analysis Services for registration details: www.DumpAnalysis.com © 2011 Memory Dump Analysis Services
  • 23. Free Summer Webinars  The Old New Crash: Cloud Memory Dump Analysis (June 6th)  Cyber Warfare Memory Dump Analysis (forthcoming in July-August) Visit Memory Dump Analysis Services for registration details: www.DumpAnalysis.com © 2011 Memory Dump Analysis Services
  • 24. Q&A Please send your feedback using the contact form on DumpAnalysis.com © 2011 Memory Dump Analysis Services
  • 25. Thank you! © 2011 Memory Dump Analysis Services Join DA+TA Facebook Group