.NET Debugging Tips and Techniques


Published on

How to fix bugs in live/production code by locating the root causes predictably and quickly?

1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Code reviews and Tracing are the old techniques to find out issues with the code. Unit Testing is the current mantra. But, the good old native debuggers are the way to resolve multi-process, multi-machine server applications. We will chat about using SOS, MDA, Windows Performance Monitors, CLR Profilers and NTSD with zero footprint & no config changes to production environment. The Visual Studio Debugger provides a slew of features that make the task of debugging both easier and more efficient. Learn about time-saving tips and tricks for all versions of the Visual Studio Debugger, including the new debugger features in Visual Studio 2010. Hear about the new mixed-mode debugging feature, the breakpoints window enhancements, the new WPF visualizes, and a number of other features. Also learn about thread debugging enhancements, new features for making stepping into properties easier, and more. Join us as we crack open the toolbox and walk through some of the debugger's best practices. Focus on powerful native debugging tools including WinDBG, NTSD, and CDB. introduce key concepts needed to successfully use .NET’s native debuggers. Next, turn to more sophisticated debugging techniques, using real-world examples that demonstrate many common C# programming errors. For developers who want a deeper understanding of how .NET works, to gain the necessary tools and use them to debug and solve real world problems that cannot be fixed with regular debuggers.
  • A defect that costs $1 to fix on the programmer’s desktop costs $100 to fix once it is incorporated into a complete program and many thousands of dollars if it is identified only after the software has been deployed in the field. (Building a Better Bug Trap – The Economist June 2003) Static Analysis can reduce defects by up to a factor of six! (Capers Jones, Software Productivity Group)
  • Compile-time analysis of the source program like code inspection, but performed by a tool Looks for violations of well-defined constraints procedure contracts Examples of bugs to be found by Static Analysis IoCompleteRequest(p) requires p to be non-NULL: p = NULL ; … ; IoCompleteRequest(p); Completing the same task twice when NOT needed: IoCompleteRequest (Irp); … IoCompleteRequest (Irp);
  • Static Analysis does 2 Things: Finds a defect - without testing Reveals the path - without debugging Path Coverage One test case covers only one path in the driver The path remains unrevealed if no defect found 100 test cases cover < 100 paths? More test cases -> more duplication How many paths remain untested? How long would it take to test all of them? When are you done? Longer than you can afford! Time to market? 100% Coverage In minutes or hours, not weeks or months Employing Computer, not a Test Engineer Targeting a large set of potential violations
  • 1: Introduction to .NET Debugging. Explains the differences between Unmanaged (native) and managed (.NET) debugging, and explore the tools required for successful debugging. 2: CLR Fundamentals. Explores the physical layout of a managed assembly, the native CLR structural hierarchy, managed objects, and the role that JIT compilation plays in managed execution. 3: .NET Memory Management. Examines .NET memory management including the Garbage Collector (GC) and Finalizers. 4: Troubleshooting Hangs. Defines, parses, and troubleshoots “hang” scenarios in WinForms and ASP.NET applications. 5: Troubleshooting Exceptions. Dives into the different types of exceptions, including those that go unhandled in WinForms and ASP.NET applications. 6: Additional Debugging Tools. Reviews additional tools and debugging scenarios.
  • Dbgrsv.exe, kdsrv.exe, dbengprx.exe Debugger protocol remoting tools Discussed later as part of debugger remoting Security Audit Debugging
  • Nothing magic; Understand the code and its underlying illusions
  • Inspire of code auditors, reviews, FxCop... Impossible Business side of things
  • For example: Compiler will not catch an un-initialized pointer but you WILL get a run-time error if you try to use it! Some simple programming techniques * To improve diagnosability of your code * To help support folks get more out of the crash dumps * To enable them determine root cause of an issue from a single crash dump * So they don’t have to ask the customer to reproduce the problem again to get them yet another crash dump Debugging should be data centric not code centric * Especially TRUE for a crash dump * No execution and no execution control * All you have is snapshot of data structures to examine
  • Kdbgctrl.exe Tool to configure the kernel’s debugging options Run it on the target machine Change behavior of DbgPrint, user mode int 3, DbgPrint buffer size, etc.
  • http://blogs.msdn.com/jacdavis/archive/2008/02/20/new-debug-engine-sample.aspx he Visual Studio Debugger is split up architecturally into a few pieces: 1)      The debugger UI: the windows and commands the user actually interacts with.  A good example of this is the watch window or the little red circle that appears in the text editor as a breakpoint. 2)      The SDM (session debug manager): “ A debug engine multiplexer” Admittedly, that’s a confusing explanation. Essentially, the SDM’s job to combine all of the events and commands for the various debug engines into one unified stream for the UI.  The debugger UI only displays one ”view” of what is being debugged at a time.  Even if the user is debugging multiple processes or threads, they are only looking at one of them. 3)      Debug Engines – the components that perform the actual debugging of a debuggee.  For instance, a native debug engine would be responsible for debugging native win32 applications. A script debug engine would be responsible for debugging jscript or vbscript. A CLR debug engine would be responsible for debugging .Net applications running on the CLR.  A hypothetical Perl engine could be responsible for debugging Perl… and on and on Visual Studio Debug Engines implement and interact with a set of interfaces called AD7 which stands for Active Debugging 7. AD7 is publically documented here: http://msdn2.microsoft.com/en-us/library/bb147088.aspx
  • symproxy: a symbol server request proxy Similar to a web server proxy Allows local caching to reduce load on primary server Putting a proxy on a bridge server allows symsrv requests to cross network boundaries agestore: a new tool to manage local caches Simple aging and cleanup of locally-stored files
  • Use Right click on Module:  “Symbol Load Information” to find out where we looked
  • 16 Megs in today's world of .NET framework Production machines can't hv Visual Studio; not updated with latest patches Operations team is diff from dev primarily used for managed code debugging Diff between them WinDBG is not console based; GUI; step through etc NTSD & CDB are console based Security team; Windows HEAT team etc hv written plugins; etc capabilities Ability to write debugging extensions for our programs/projects/products
  • 64 or which architecture? SDK is also available for developing our own extensions Debugger Extensions Dump or analyze complex data structures !process, !devnode, !poolval Leverages the type information from the PDB file Simplify routine tasks !analyze Automate repetitive steps Regularly check state of certain objects Fully control the state of the target Can write a mini-debugger using the extension APIs When Should You Write One? Any task that is repetitive can be sped up by developing an extension Allows other people (testers) to help with basic debugging Can help identify common problems quickly To dump internal data structures in a custom readable format Avoid writing extensions when: Code is still fluctuating a lot Extension must match the code being debugged
  • Analysis Step 1 Use bugcheck or exception parameters to extract basic information Each condition is processed by a separate routine that understands the meaning of each parameter If specific follow-up or faulting code is found, report results Save trap frame, context recording, faulting thread, etc. Analysis Step TWO Use information in step one to get faulting stack Scan the stack for special functions such as Trap0E or UnhandledExceptionFilter to find alternate stack Analyze frames on the final stack to determine most likely culprit Different weights are assigned to routines Internal kernel routines have lowest weight Device drivers have highest weight Fine grain control provided by triage.ini Highest weight frame found on the stack is treated as the culprit Analysis Step Three If stack does not yield an interesting frame, analyze raw stack data Iterate on all stack values using the same weight algorithm The ‘dps’ command will show that output This finds code that corrupts the stack Analysis Step Four Check for presence of memory or pool corrupting drivers Check for corrupted code streams Bad RAM Check for other possible problems, such as invalid call sequences Possible CPU problem Analysis Step Five Generate final bucket ID and follow-up based on all gathered information Determine which fields need to be embedded in the bucket ID !analyze assigns ownership of failure
  • Use Task manager; Get Process ID (TList) Start the app; instead of attaching to already existing Example of Windows NT Heat dependent process; additional information is filled in the blocks. Not Release debugging; So, Debug version is ran
  • show diff between -p (more dlls loaded) and starting manually
  • http://blogs.msdn.com/jacdavis/archive/2006/08/25/using-sos-in-visual-studio.aspx http://http/www.microsoft.com/whdc/devtools/debugging/default.mspx http://msdn.microsoft.com/msdnmag/issues/03/06/Bugslayer/default.aspx How do you go about using SOS in VS? Well first, you have to have the native debugging engine enabled. This is because SOS examines the CLR data structures from the debugger process so it must have access to the native address space. To do this from a console application project or winforms project, right mouse click on the project in solution explorer, choose properties, switch to the Debug tab, and make sure "Enable Unmanaged Code Debugging" is selected. With that done, close the property page add a breakpoint somewhere in your project, and hit F5. Once you've hit your breakpoint, the real fun begins. 1) Open the Immediate window (Debug->Windows->Immediate) 2) Type ".load C:\\windows\\Microsoft.NET\\Framework\\v2.0.50727\\sos.dll" and hit enter (you may need to change the path to point to a different location on your machine). The immediate window should respond by saying "extension C:\\windows\\Microsoft.NET\\Framework\\v2.0.50727\\sos.dll loaded" 3) Type "!help" sos should dump its help contents to the immediate window. It is now ready to use.
  • Symbols are extremely important when debugging native code While managed code are not so much important Source code lines are not present Proprietary information Run tool to strip all Private symbol Publish public symbol files For own symbol files instead of Microsoft refresh the symbol files
  • 03Simple.exe demo for Symbols start with ntsd instead of attaching ntsd 03simple.exe .sysmfix .sympath+ c:\\ .reload
  • Collect all binary files into a Single Location Essential for library vendors Use them from a share (Private symbols) Publish the symbols on a HTTP server (public) Publish the symbols on a Secure HTTP server (Private) Works with other Tools (Process Monitor, profiler) SourceServer is an alternate stream into Private Symbol PDBs Contains information to retrieve the source file from the Source revision control system
  • CLR manages thread differently than Windows Native debuggers don't hv much idea abt Managed Code SO CLR understanding debuggers Threads getting deadlocked kind of scenarios
  • Demo ntsd 03simple.exe g ctrl + C .sysmfix .reload .loadby sos mscorwks !help !help Threads
  • Multi-threading is easy but tricky Hash code of the objects Lock info how many bytes long? 4 or 8 Deadlock issues 200 Threads waiting Instead of going one by one on each of the 200 Execute and get everything for each thread running
  • Run the app - start from cmd 06deadblock.exe TList ntsd -p processID .symfix .reload .loadby sos mscorwks ~*e!ClrStack ~0S (ZEROTH thread) !Dumpstack ~3s !DumpStack Use Syncblock to find out more info !SyncBlk Who is holding locks? Actual object which is holding? Address of the object !do !do on the object open the source code SOSex has the DumpObject command
  • No memory leaks Just new No need to delete in destructor Statistical summary - how many instances Type etc Specify address of the object Size, type etc 2 things Does not hv any references for it ?? Is it eligible for GC If not, who has references for it? (Native resources not getting freed up) Instead of GCRoot - Visual graph Whole lot easier Link to video
  • 03simple.exe .sysmfix - set symbols .reload g ctrl + C load sos dump heap Address of the object - size of the object --- 327 objects Last part - stats 135 instances system.string - how much does it occupy dumpheap -type system.string (filtered on type) Pick an address of size 80 !do on its address !GCRoot on its address No references So it will be cleaned up on next GC Useful - why the reference is held? Who holds it etc
  • Live process is stopped; resumed; break into debugging We controlled the execution Services running in the cloud (credit card transactions) 1000s of requests coming each second Service will be down costs millions of $$ lost Always try to get Full memory dump Otherwise becomes difficult to execute all cmds
  • Automatic way vs .dump command Monitor with conditions for symptoms, problems - ADPLUS Win Error Reporting - Service based solutions version1 is getting used Diff flavors of windows Take the binaries of the product When the crash happens, anywhere Something bad happens at customer site Do you like to report this problem? Windows error reporting will create new entry Forward it to you/Download all the crash data Produce the fix Close the loop with customer Send it up to MS Register the response? - sorry for the inconvenience Take action on your issues Free of cost 3000+ companies signed up Client issues resolved as conditional maintenance
  • SOS debugger extensions loads small piece of native code Asks CLR to give info to debug When live debugging versions are same CLR is present on the system Diff machines when postmortem Hot fixes, service packs, dll hell etc - pain Public symbol server published in Microsoft Contact customer to give the correct version Load it in ur system debugger will launch with -z
  • 08simple.exe Null exception Enable ADPLUS adplus.vbs -crash -pn 08simpleexec.exe (start) pn for process name Crash mode Find crash dumps in location click ok start app Adplus - performance degradation - slow debuggers folder crash mode date Text file Dump files generated Shut down - we are not interested second chance exception ntsd -z path to the file .symfix .reload loadby sos mscorwks !Threads Which generated !pe
  • http://msdn.microsoft.com/en-us/library/bb145915.aspx
  • .NET Debugging Tips and Techniques

    1. 1. .NET Debugging Techniques Bala Subra
    2. 3. Software Bugs are Expensive <ul><li>Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it. - Brian Kernighan </li></ul>
    3. 4. Why Debugging? <ul><li>We need Reliable Software </li></ul><ul><li>Users Choose Reliability over Price </li></ul><ul><li>“ Software bugs or errors are so prevalent and so detrimental that they cost the US economy an estimated $59 billion annually, or about 0.6 percent of the GDP.” </li></ul><ul><ul><li>www.nist.gov/public_affairs/releases/n02-10.htm </li></ul></ul>
    4. 5. Most Commonly Found Defects <ul><li>http://scan.coverity.com/report/Coverity_White_Paper-Scan_Open_Source_Report_2009.pdf </li></ul>
    5. 6. Technical Requirements <ul><li>Understand unmanaged (native) vs. Managed debugging. </li></ul><ul><li>Explore production-environment debugging. </li></ul><ul><li>Deep-dive into CLR fundamentals. </li></ul><ul><li>Understand .NET memory management. </li></ul><ul><li>Debug hangs, crashes, and memory leaks. </li></ul>
    6. 7. Agenda <ul><li>Importance of debugging </li></ul><ul><li>Available Tools </li></ul><ul><li>Basic Tasks and Walkthroughs </li></ul><ul><li>Postmortem Debugging </li></ul>
    7. 8. Common Traits for good debuggers <ul><li>Willingness to venture outside of your &quot;own&quot; code </li></ul><ul><li>Curiosity </li></ul><ul><li>Patience </li></ul><ul><li>Treat dependent code as just that - code </li></ul><ul><li>Ability to see patterns </li></ul>
    8. 9. Importance of debugging <ul><li>Perfect code is an Illusion </li></ul><ul><li>Legacy Code </li></ul><ul><li>Deeper Understanding </li></ul><ul><li>Helps you learn & write better code in the future </li></ul>
    9. 10. Debugging Basics <ul><li>What are you trying to find and fix? </li></ul><ul><ul><li>Two main types of code errors </li></ul></ul><ul><ul><ul><li>Syntax: Compiler catches most if not all of these for you. </li></ul></ul></ul><ul><ul><ul><li>Semantic or logical: Syntactically correct yet program may “crash and burn” at run-time! </li></ul></ul></ul><ul><li>Autos </li></ul><ul><li>Locals </li></ul><ul><li>Watch </li></ul><ul><li>Call Stack </li></ul><ul><li>Command Window </li></ul><ul><li>QuickWatch Dialog </li></ul><ul><li>Breakpoints window </li></ul><ul><li>Threads </li></ul><ul><li>Modules </li></ul><ul><li>Processes </li></ul><ul><li>Memory </li></ul><ul><li>Disassembly </li></ul><ul><li>Registers </li></ul>
    10. 11. Execution Control: Breakpoints <ul><li>Stepping through your code </li></ul><ul><ul><li>Starting / Stopping </li></ul></ul><ul><ul><li>Breaking </li></ul></ul><ul><ul><li>Stepping through your application </li></ul></ul><ul><ul><ul><li>(F10, F11 or Toolbar buttons) </li></ul></ul></ul><ul><ul><li>Run to a specific location </li></ul></ul><ul><ul><ul><li>Run To Cursor (right-click menu) </li></ul></ul></ul><ul><li>Situations under which breakpoints are not feasible </li></ul><ul><ul><li>Timing sensitive issues </li></ul></ul><ul><ul><li>Breakpoint triggers too often </li></ul></ul><ul><ul><li>Live Debug not possible </li></ul></ul><ul><ul><li>Debugging production systems </li></ul></ul>
    11. 12. Tools available for Debugging .NET <ul><li>Visual Studio 2008 (& soon 2010) </li></ul><ul><ul><li>2010: </li></ul></ul><ul><ul><ul><li>Historical Debugging </li></ul></ul></ul><ul><ul><ul><li>Visualizations for locks, threads </li></ul></ul></ul><ul><li>CorDBG / MDbg </li></ul><ul><li>Debugging Tools for Windows </li></ul><ul><ul><li>Focus for Today </li></ul></ul><ul><ul><li>Used by everybody for everybody </li></ul></ul>
    12. 13. Visual Studio 2008: Debugger Tips and Tricks <ul><li>Execution Control </li></ul><ul><li>Breakpoints </li></ul><ul><li>Symbols </li></ul><ul><li>Multi-Threaded debugging </li></ul><ul><li>Post Mortem Debugging </li></ul><ul><li>Behind the Debugger Magic </li></ul>
    13. 14. VS Debugger Overview <ul><li>Debugs many different code Environments </li></ul><ul><ul><li>Native Windows </li></ul></ul><ul><ul><ul><li>X86 </li></ul></ul></ul><ul><ul><ul><li>X64 </li></ul></ul></ul><ul><ul><ul><li>IA64 </li></ul></ul></ul><ul><ul><li>Managed Code </li></ul></ul><ul><ul><ul><li>Windows (32 & 64 bit) </li></ul></ul></ul><ul><ul><ul><li>SQLCLR </li></ul></ul></ul><ul><ul><li>Script </li></ul></ul><ul><ul><li>T-SQL </li></ul></ul><ul><ul><li>Native Device Programs </li></ul></ul>
    14. 15. VS Debugger Architecture VSDebug Package SDM CPDE (Managed) NatDbgDE (Native) Your Engine Here Your EE Here http://msdn.microsoft.com/en-us/library/bb161718.aspx
    15. 16. Execution Control <ul><li>Step Filtering for Managed Code </li></ul><ul><ul><li>Adds support for automatically stepping over simple properties </li></ul></ul><ul><ul><li>Right click to “Step into Specific”: pick Step into Target </li></ul></ul><ul><ul><li>Switch off: How? </li></ul></ul>
    16. 17. Breakpoints <ul><li>Tracepoints </li></ul><ul><ul><li>Print a variety of program state types without stopping </li></ul></ul><ul><ul><ul><li>Program Location, including Stack </li></ul></ul></ul><ul><ul><ul><li>Expressions (including @clk in native for quick timing) </li></ul></ul></ul><ul><ul><ul><li>Thread Info </li></ul></ul></ul><ul><ul><ul><li><Your Message Here> </li></ul></ul></ul><ul><ul><li>Run a Macro </li></ul></ul><ul><ul><ul><li>Useful on stop on certain condition that is difficult to express as a ‘Watch window’ expression. For eg. “Stop if this method is in Call Stack” </li></ul></ul></ul>
    17. 18. Visual Studio 2010 <ul><li>Tagging </li></ul><ul><li>Filtering </li></ul><ul><li>Import from others & Export Breakpoints </li></ul>
    18. 19. Symbols <ul><li>Ensure Symbols are switched on in Final/Retail/Optimized Builds </li></ul><ul><li>Archive Symbols using Symbol Server </li></ul><ul><li>VS 2010 </li></ul><ul><ul><li>Team Build support for Symbol Server </li></ul></ul><ul><ul><li>Add Symbol & Source indexing into the Build’s Workflow </li></ul></ul>
    19. 20. Symbols: Reference Source Support
    20. 21. Symbols: Visual Studio Options
    21. 22. Symbol Loading Internals <ul><li>VS Never loads Mismatched symbols </li></ul><ul><li>Path Plan: </li></ul><ul><ul><li>Where the EXE think it is </li></ul></ul><ul><ul><li>On the path we create from the “Symbols” Dialog </li></ul></ul><ul><ul><li>On the path at HKLM/HKCU </li></ul></ul><ul><ul><ul><li>SoftwareMicrosoftVisualStudioMSPDBSymbolSearchPath </li></ul></ul></ul><ul><ul><li>On the path @ Any of these: </li></ul></ul><ul><ul><ul><li>_NT_ALT_Symbol_Path </li></ul></ul></ul><ul><ul><ul><li>_NT_Symbol_Path </li></ul></ul></ul><ul><ul><ul><li>SystemRoot </li></ul></ul></ul>
    22. 23. Threads <ul><li>Thread Categories </li></ul><ul><li>Flagging Threads in the List (for tracking) </li></ul><ul><li>Using Breakpoint Filters </li></ul><ul><li>Freezing & Thawing </li></ul><ul><li>Stack Tips </li></ul><ul><li>Naming Threads </li></ul><ul><ul><li>Managed: Thread.Name </li></ul></ul><ul><ul><li>Native: Use the SetThreadName execution wrapper </li></ul></ul><ul><ul><li>http://blogs.msdn.com/stevejs/archive/2005/12/19/505815.aspx </li></ul></ul>
    23. 24. Post Mortem Analysis <ul><li>Windows Error Reporting </li></ul><ul><li>http://winqual.microsoft.com </li></ul><ul><li>http://msdn.microsoft.com/en-us/library/aa939342.aspx </li></ul><ul><li>Debugging </li></ul><ul><ul><li>Open the Disassembly Window </li></ul></ul><ul><ul><li>Open the Autos window to see pertinent Registers </li></ul></ul><ul><ul><li>http://blogs.msdn.com/greggm/archive/2004/12/15/315673.aspx </li></ul></ul><ul><ul><li>Trust Statics/Globals </li></ul></ul><ul><ul><li>Trust Stacks when you have Symbols </li></ul></ul>
    24. 25. VS2010: Managed/Interop Support <ul><li>Support for reading & writing Minidumps from Processes with Managed Code </li></ul><ul><li>Support for Mixed Mode debugging on x64 </li></ul>
    25. 26. How does the Debugger do Minidumps <ul><li>Use dbgHelp.dll Method: MiniDumpReadDumpStream to read streams from MiniDump File </li></ul><ul><li>Read the following Streams </li></ul><ul><ul><li>SystemInfoStream </li></ul></ul><ul><ul><li>ThreadListStream </li></ul></ul><ul><ul><li>ModuleListStream </li></ul></ul><ul><ul><li>MemoryListStream </li></ul></ul><ul><li>Create Container Objects in the debugger that wrap the instances from the MiniDump </li></ul><ul><li>Wrap memory as needed by StackWalking or Data Inspection </li></ul>
    26. 27. Benefits of Debugging Tools for Windows <ul><li>Small Footprint </li></ul><ul><ul><li>XCopy Enabled </li></ul></ul><ul><li>Ideal for debugging problems on machines that are locked down </li></ul><ul><li>Frequent releases </li></ul><ul><ul><li>Updated for new versions of Windows </li></ul></ul><ul><li>Which debuggers does it include? </li></ul><ul><ul><li>User Mode Debuggers: windbg/ntsd/cdb </li></ul></ul><ul><ul><li>Kernel Mode Debugger: kd </li></ul></ul><ul><li>Powerful Extensions & Instrumentation </li></ul><ul><li>Extensible by us </li></ul>
    27. 28. Debugging: Package Content <ul><li>Symbol Indexing Tools </li></ul><ul><li>Source Indexing Tools </li></ul><ul><li>Stand-alone Tools </li></ul><ul><ul><li>AgeStore </li></ul></ul><ul><ul><li>AdPlus </li></ul></ul><ul><ul><li>BreakIN </li></ul></ul><ul><ul><li>DbgSrv </li></ul></ul><ul><ul><li>GFlags </li></ul></ul><ul><ul><li>TList </li></ul></ul><ul><ul><li>Remote </li></ul></ul>
    28. 29. Installing Debugging tools for Windows <ul><li>Download Point: </li></ul><ul><ul><li>www.microsoft.com/whds/devtools/debugging/default.msp </li></ul></ul><ul><li>Default options sufficient </li></ul><ul><li>By Default installs into </li></ul><ul><ul><li>C:Program Filesdebugging Tools for Windows </li></ul></ul><ul><li>Directory Listing </li></ul>
    29. 30. Debugger Interaction: 1 st Steps <ul><li>Command Mode or GUI? </li></ul><ul><li>User mode prompt can get us a Head Start </li></ul><ul><li>Get the Exception Code </li></ul><ul><li>Understand the Environment </li></ul><ul><li>Set the Correct Symbols </li></ul><ul><li>Start from the Current Execution Context </li></ul><ul><li>Check the Loaded Module </li></ul>
    30. 31. Basic Tasks : Running the debugger <ul><li>Attaching to Process </li></ul><ul><ul><li>By Process ID: -p <process id> </li></ul></ul><ul><ul><li>By Process Name: -pn <process name> </li></ul></ul><ul><ul><li>TList Command </li></ul></ul><ul><li>Running Under the Debugger </li></ul><ul><ul><li>NTSD.EXE <Command Line> </li></ul></ul><ul><ul><li>NTSD.EXE C:Windowsnotepad.exe </li></ul></ul><ul><ul><li>Caveats: Various Components may go into Debug Mode </li></ul></ul>
    31. 32. Demos
    32. 33. Working with the Target <ul><li>Last Event </li></ul><ul><li>Registers </li></ul><ul><li>Memory </li></ul><ul><li>Variables </li></ul><ul><li>Stack </li></ul><ul><li>Unassemble </li></ul><ul><li>Process Information </li></ul><ul><li>Thread Information </li></ul><ul><li>Address Information </li></ul>
    33. 34. Basic Tasks : Symbols <ul><li>Additional Metadata about the Code </li></ul><ul><li>Managed Types far more self-descriptive </li></ul><ul><li>Private vs Public Symbols </li></ul><ul><li>Microsoft Public Symbol Server </li></ul><ul><li>How to tell the debugger the Location </li></ul><ul><ul><li>Pointing to MS Public Server: .symfix </li></ul></ul><ul><ul><li>Pointing to additional Paths: .sympath+ </li></ul></ul><ul><ul><li>Reloading Symbols: .reload </li></ul></ul><ul><li>Custom Symbol Servers </li></ul>
    34. 35. Walkthrough for Symbols
    35. 36. Symbol Server <ul><li>Large Store of Symbol & Binary Files </li></ul><ul><li>Files are organized based on properties: </li></ul><ul><ul><li>Name </li></ul></ul><ul><ul><li>Type </li></ul></ul><ul><ul><li>Time stamp </li></ul></ul><ul><ul><li>Size of the Image </li></ul></ul><ul><ul><li>RSDS Signature </li></ul></ul><ul><li>Binary files can be stored in different location </li></ul><ul><li>Files can be compressed </li></ul>
    36. 37. Building a Symbol Server <ul><li>Tools: </li></ul><ul><ul><li>PdbCopy.exe </li></ul></ul><ul><ul><li>BinPlace.Exe (WDK) </li></ul></ul><ul><ul><li>SymStore.exe </li></ul></ul><ul><ul><li>Extending the build process (Batch Files) </li></ul></ul>
    37. 38. Basic Tasks : SOS <ul><li>Powerful managed code debugger extension </li></ul><ul><ul><li>Introspect on the internal state of the CLR </li></ul></ul><ul><ul><li>Son of Strike </li></ul></ul><ul><li>Loading SOS </li></ul><ul><ul><li>.NET 2.0 : .loadby sos mscorwks </li></ul></ul><ul><ul><li>.NET 4.0 : .loadby sos clr </li></ul></ul><ul><li>Help Command </li></ul><ul><ul><li>!help displays all commands </li></ul></ul><ul><ul><li>!help <command> displays help for specific command </li></ul></ul><ul><li>SOSEX is another useful debugger extension </li></ul><ul><ul><li>http://www.stevestechspot.com </li></ul></ul>
    38. 39. Debugger Extension walkthroughs
    39. 40. Basic Tasks : Thread <ul><li>Basic Tasks : Thread </li></ul><ul><li>Basic Unit of code Execution </li></ul><ul><li>Before you launch a new thread, think twice </li></ul><ul><li>Sync-blocks </li></ul><ul><ul><li>Plethora of information about objects </li></ul></ul><ul><li>SOS Thread commands </li></ul><ul><ul><li>!Threads: List all managed Thread </li></ul></ul><ul><ul><li>!ClrStack: Displays Managed Callstack for currently active Thread </li></ul></ul><ul><ul><li>~<ThreadNum>s: Switches currently active Thread </li></ul></ul><ul><ul><li>~*e!ClrStack: Shows Callstack for all managed threads </li></ul></ul><ul><ul><li>!syncBlk </li></ul></ul>
    40. 41. Deadlock problem walkthrough
    41. 42. Basic Tasks: Managed Heap & Garbage collection <ul><li>Automatic Memory Management </li></ul><ul><ul><li>Sits on top of the Windows Memory Manager </li></ul></ul><ul><ul><li>Currently consists of 3 generations (0, 1, 2) </li></ul></ul><ul><ul><li>Caveat: Native resources must be explicitly cleaned up </li></ul></ul><ul><li>SOS Commands </li></ul><ul><ul><li>DumpHeap </li></ul></ul><ul><ul><li>DumpObj (do) </li></ul></ul><ul><ul><li>GCRoot </li></ul></ul><ul><li>Visualizing Runtime Object Graphs </li></ul><ul><ul><li>http://www.lovettsoftware.com/blogengine.net/post/2010/01/15/Visualizing-Runtime-Object-Graphs.aspx </li></ul></ul>
    42. 43. Walkthrough for managed heap commands
    43. 44. Resource Leaks <ul><li>What is a Resource? </li></ul><ul><ul><li>Handles </li></ul></ul><ul><ul><ul><li>File Object </li></ul></ul></ul><ul><ul><ul><li>Process Object </li></ul></ul></ul><ul><ul><ul><li>Thread Object </li></ul></ul></ul><ul><ul><ul><li>Isolation layer between User Mode code & Kernel </li></ul></ul></ul><ul><ul><li>Synchronization Primitives </li></ul></ul><ul><ul><li>Heap Memory Allocator </li></ul></ul><ul><ul><li>Virtual Memory Allocator </li></ul></ul><ul><ul><li>COM Allocator </li></ul></ul>
    44. 45. Tools for Heap Memory Tracking <ul><li>UMDH </li></ul><ul><ul><li>Tracks Heap based Memory </li></ul></ul><ul><ul><li>Requires OS Instrumentation to be Enabled (gflags) </li></ul></ul><ul><li>LeakDiag </li></ul><ul><ul><li>Uses Microsoft Detours Library </li></ul></ul><ul><ul><li>Tracks different types of Memory Allocators </li></ul></ul><ul><ul><ul><li>Heap Allocator </li></ul></ul></ul><ul><ul><ul><li>Virtual Memory Allocator </li></ul></ul></ul><ul><ul><ul><li>COM Allocator </li></ul></ul></ul><ul><ul><ul><li>C Runtime Allocator </li></ul></ul></ul><ul><li>Debugger Command: !heap </li></ul><ul><li>Static Source Code Analysis Tools: Prefast (WDK) </li></ul>
    45. 46. Memory Leaks: C Run-Time Functions <ul><li>_CrtDumpMemoryLeaks() </li></ul><ul><ul><li>Performs leak checking where called. You want to place this call at all possible exits of your app. </li></ul></ul><ul><li>_CrtSetDbgFlag () </li></ul><ul><ul><li>Sets debugging flags for the C run-time library. </li></ul></ul><ul><ul><li>_CRTDBG_REPORT_FLAG Gets current flag(s) </li></ul></ul><ul><ul><li>_CRTDBG_LEAK_CHECK_DF Perform automatic leak checking at program exit through a call to _CrtDumpMemoryLeaks </li></ul></ul>
    46. 47. Memory Leaks: Visual Studio <ul><li>_CRTDBG_MAP_ALLOC_ </li></ul><ul><ul><li>Memory allocation number (inside curly braces) </li></ul></ul><ul><ul><li>Block type (normal, client or CRT) </li></ul></ul><ul><ul><li>Memory location in hex </li></ul></ul><ul><ul><li>Size of block in bytes </li></ul></ul><ul><ul><li>Contents of the first 16 bytes in hex </li></ul></ul><ul><ul><li>File name </li></ul></ul><ul><ul><li>Line number </li></ul></ul>
    47. 48. Heap Corruptions <ul><li>Violate the Integrity of Memory allocated on the Heap </li></ul><ul><ul><li>Stray Pointers </li></ul></ul><ul><ul><li>Overruns </li></ul></ul><ul><ul><li>Underruns </li></ul></ul><ul><ul><li>Over-Deletion </li></ul></ul><ul><ul><li>Reuse after Deletion </li></ul></ul><ul><li>One of the toughest problem to Debug </li></ul>
    48. 49. Windows Memory Architecture Application Virtual Memory Manager Heap Manager Default Process Heap C Runtime Heap Other Heaps
    49. 50. Heap Block Structure Current Size Previous Size Seg Index Flags Unused Tag Index Pre-allocation Metadata Post-allocation Metadata User accessible part Pre-allocation Metadata Suffix Bytes Fill Area (debug) Heap Extra Post-allocation Metadata User accessible part User accessible part
    50. 51. Tools for Debugging Heap Corruptions <ul><li>Goal is to Break when the corruption occurs AND not after </li></ul><ul><li>PageHeap helps with that goal </li></ul><ul><ul><li>Annotates heap blocks to trigger fault at the time of write </li></ul></ul><ul><li>Light PageHeap uses Fill Patterns </li></ul><ul><li>Full PageHeap uses Fill Patterns and Guard Pages </li></ul><ul><ul><li>Very Memory Intensive </li></ul></ul>
    51. 52. Postmortem Debugging <ul><li>Scenarios </li></ul><ul><ul><li>Live debugging not feasible </li></ul></ul><ul><ul><li>Reproducing the problem is difficult </li></ul></ul><ul><li>Static Snapshot of a Live Process </li></ul><ul><li>Use the same debugger to debug offline </li></ul><ul><li>Limitations </li></ul><ul><ul><li>It is a snapshot; so you can't control execution </li></ul></ul><ul><ul><li>Depending on type of dumpfile, some SOS commands may not work. </li></ul></ul>
    52. 53. Postmortem Debugging: How to generate dumpfiles <ul><li>Using the debuggers </li></ul><ul><ul><li>.dump /mf c:CoreDump.dmp </li></ul></ul><ul><li>Automatic </li></ul><ul><ul><li>ADPlus </li></ul></ul><ul><ul><li>Windows Error Reporting </li></ul></ul><ul><ul><ul><li>https://winqual.microsoft.com </li></ul></ul></ul><ul><ul><ul><li>Available to everyone </li></ul></ul></ul>
    53. 54. Windows Error Reporting (WER) Architecture Error Sent Dr. Watson Process Crash Crash data over HTTPS Fault response over HTTPS Windows Error Reporting Service ISV Query Fault Data
    54. 55. Postmortem Debugging: How to debug Dumpfiles? <ul><li>Slightly different than Native code debugging </li></ul><ul><ul><li>The data Access Layer (DAC) </li></ul></ul><ul><ul><li>Implemented in mscordacwks.dll </li></ul></ul><ul><ul><li>Different for each version of the CLR </li></ul></ul><ul><li>Debugging Dump files </li></ul><ul><ul><li>use the -z switch with path to the dump file </li></ul></ul>
    55. 56. Walkthroughs with ADPlus & Postmortem debugging
    56. 57. When Not to use Native Debugging <ul><li>During Code Development </li></ul><ul><li>Tracing the Code </li></ul><ul><li>100% managed code </li></ul><ul><li>Need frequent variable inspection </li></ul><ul><li>Need frequent references to the source files </li></ul><ul><li>Debugging Partial Dumps </li></ul><ul><li>Kernel Mode Debugging </li></ul><ul><ul><li>Some pages are paged out </li></ul></ul>
    57. 58. Summary <ul><li>Importance of debugging </li></ul><ul><li>Be aware of Magic </li></ul><ul><li>Tools available for Debugging .NET </li></ul><ul><li>Basic debugging Tasks </li></ul><ul><ul><li>Running the debuggers </li></ul></ul>
    58. 59. Questions? <ul><li>Books </li></ul><ul><ul><li>Advanced .NET Debugging: Mario Hewardt </li></ul></ul><ul><ul><li>Windows Internals: Mark E. Russinovich, David A. Solomon with Alex Ionescu </li></ul></ul><ul><ul><li>Windows via C/C++: Jeffrey M. Richter, Christophe Nasarre </li></ul></ul><ul><li>Blogs </li></ul><ul><ul><li>http://blogs.msdn.com/ms_joc/ </li></ul></ul><ul><ul><li>http://www.wintellect.com/cs/blogs/jrobbins/default.aspx </li></ul></ul>