Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

How to debug machine learning call stacks

563 views

Published on

Python is a popular language for deep learning but debugging calls to existing C/C++ code in shared libraries can be extremely challenging. In this presentation from GPU Tech Conference, we look at how Python-C/C++ transformations combined with a multithreaded, multiprocess debugger helps you understand what’s going on within your deep learning code.

Published in: Software
  • Be the first to comment

  • Be the first to like this

How to debug machine learning call stacks

  1. 1. 1© 2017 Rogue Wave Software, Inc. All Rights Reserved. 1 Debugging machine learning call stacks S7506 GPU Technology Conference
  2. 2. 2© 2017 Rogue Wave Software, Inc. All Rights Reserved. 2 Data modeling is critical for machine learning • Data scientists iterate over huge datasets • Legacy algorithms are critical • Most popular languages – R in scientific communities – Python in engineering • Focusing on Python • Developers use it because – Ease of use – Clean syntax – Extension libraries (many in C/C++)
  3. 3. 3© 2017 Rogue Wave Software, Inc. All Rights Reserved. 3 Calling C/C++ from Python • Legacy libraries are written in C/C++ and Fortran – Runfaster – Rewriting doesn’t make sense • Luckily there are many ways to call between the languages
  4. 4. 4© 2017 Rogue Wave Software, Inc. All Rights Reserved. 4 Debugging multiple languages • Debugging one language is difficult enough – Especially with many threads/processes • The language intersection is tougher – Data comparison – Glue code • Issues are: – Type mismatches – Extraneous stack frames
  5. 5. 5© 2017 Rogue Wave Software, Inc. All Rights Reserved. 5 Python without Filtering Glue code No viewing of Python data & code
  6. 6. 6© 2017 Rogue Wave Software, Inc. All Rights Reserved. 6 Showing C code with mixed data Glue code filtered out Python data available for viewing Shows Python & C++ C++ data Py data
  7. 7. 7© 2017 Rogue Wave Software, Inc. All Rights Reserved. 7 Python with filtering Python code available Program counter shows calling location
  8. 8. 8© 2017 Rogue Wave Software, Inc. All Rights Reserved. 8 Python support in TotalView • Designed for where Python calls C/C++ or vice versa • Glue frames hidden from view – Supported for SWIG – User customizable • Python information shown – Code – Data
  9. 9. 9© 2017 Rogue Wave Software, Inc. All Rights Reserved. 9 Stack Transformation Facility • Hides stack frames • Transforms stack frames • Backbone for: – Python support – OpenMP support • Useful for any glue code you want to hide – Language differences – Wrapper code
  10. 10. 10© 2017 Rogue Wave Software, Inc. All Rights Reserved. 10 Demo
  11. 11. 11© 2017 Rogue Wave Software, Inc. All Rights Reserved. 11 TensorFlow basics • Open source • Numerical computation • Usage in machine learning • Written in C++ – Called from Python
  12. 12. 12© 2017 Rogue Wave Software, Inc. All Rights Reserved. 12 TensorFlow Multi-threaded application Glue code removed Added a rule for some wrappers
  13. 13. 13© 2017 Rogue Wave Software, Inc. All Rights Reserved. 13 TotalView for HPC • Comprehensive multi-core and multi-threaded analysis and debug environment – Thread specific breakpoints – Control individual thread execution – View thread specific stack and data – View complex data types easily • Integrated Reverse debugging • Mixed Lang - Python C/C++ • Track memory leaks in running applications • Supports C/C++ on Linux • Allowing the business to have – Predictable development schedules – Less time spent debugging
  14. 14. 14© 2017 Rogue Wave Software, Inc. All Rights Reserved. 14 Interested in learning more? Download our white paper on: Mixed-language debugging with Python and C/C++
  15. 15. 15© 2017 Rogue Wave Software, Inc. All Rights Reserved. 15 Presenter Martin Bakal Product manager, TotalView & CodeDynamics Rogue Wave Software martin.bakal@roguewave.com Twitter: @mbakal
  16. 16. 16© 2017 Rogue Wave Software, Inc. All Rights Reserved. 16
  17. 17. 17© 2017 Rogue Wave Software, Inc. All Rights Reserved. 17 Numpy

×