Your SlideShare is downloading. ×
0
Visualizing Relationships
Between Python Objects

       Dinu C. Gherman
       gherman@python.net

      EuroPython Confe...
Motivation
• Make Python data structures visible
• Provide a simple tool for text/book authors
• Explore GraphViz capabili...
Typical Questions
• How does that data structure look like?
• How does it change after doing … to it?
• What does this mod...
Background
Example 1




http://inst.eecs.berkeley.edu/
~selfpace/cs9honline/Q2/mutation.html
Example 2




http://www.linux-user.de/ausgabe/
2006/09/080-python-1/
Example 3
                       X = [1, 2, 3]
                       L = ['a', X, 'b']
                       D = {'x':X,...
Example 4
L = ['abc', [(1, 2), ([3], 4)], 5]




http://oreilly.com/catalog/9780596513986
Example 4 revisited
                                 L

        WANTED:                     ref

                       li...
Example 3 revisited



                    L                                                    D


                      ...
PDF References (internal)
                                      2:0       1:0
                                      202   ...
”First System“
                         os




sys   errno          posixpath         UserDict    copy_reg




           ...
”First System“
                                                                                                           ...
GraphViz
Features
• Diagrams of abstract graphs and networks
• Automatic graph drawing
• GUIs, tools, libraries, and language bindi...
Example: Hello World

                     Hello
digraph G {
    Hello -> World
}

                     World
Example: Hello World 2
                                            Hello
digraph G {
    overlap=false;

    Hello [shape=...
Example: FSM
digraph finite_state_machine {
   rankdir=LR;
   size="8,5"
   node [shape = doublecircle]; LR_0 LR_3 LR_4 LR...
Example: FSM
                                LR_4


                       S(A)
                                S(b)


   ...
Pyrels
Features
• “Second system” – really simple, so far
• Recursive func. collecting nodes and edges
• Converting to GraphViz n...
Pyrels Profile
$ cd pyrels
$ fileinfo -p rel -a lc:nmtlines:nclasses:ndefs:ncalls 
  scripts/pyrels $(find src -name "*.py"...
Example: String Caching
                                                           ref
            1                      ...
Example: Singleton None
                                                   digraph G {
        1                          ...
Example: Module Content (pyPdf)
                ref
    utils                                  module: pyPdf.utils
       ...
Future
Issues
• Testing, debugging & refactoring, …
• Detecting cycles
• Fiddling with GraphViz params (overlap, …)
• Adding othe...
Summary
• Development only barely started (v. 0.1)
• Useful already for visualising data structures
• Provides certain ins...
Links
•   http://www.dinu-gherman.net/tmp/pyrels-0.1.0.tar.gz

•   http://www.graphviz.org
Questions?
Questions!
• GraphViz editor with branch fold-in/out?
• How to get full path for DOT edges
  (in plain format)?
Upcoming SlideShare
Loading in...5
×

Visualizing Relationships between Python objects - EuroPython 2008

727

Published on

A presentation about visualizing relationships between Python objects given at EuroPython 2008.

Published in: Technology, Education
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
727
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
50
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Transcript of "Visualizing Relationships between Python objects - EuroPython 2008"

  1. 1. Visualizing Relationships Between Python Objects Dinu C. Gherman gherman@python.net EuroPython Conference 2008-07-08, Vilnius
  2. 2. Motivation • Make Python data structures visible • Provide a simple tool for text/book authors • Explore GraphViz capabilities
  3. 3. Typical Questions • How does that data structure look like? • How does it change after doing … to it? • What does this module contain?
  4. 4. Background
  5. 5. Example 1 http://inst.eecs.berkeley.edu/ ~selfpace/cs9honline/Q2/mutation.html
  6. 6. Example 2 http://www.linux-user.de/ausgabe/ 2006/09/080-python-1/
  7. 7. Example 3 X = [1, 2, 3] L = ['a', X, 'b'] D = {'x':X, 'y':2} http://oreilly.com/catalog/9780596513986
  8. 8. Example 4 L = ['abc', [(1, 2), ([3], 4)], 5] http://oreilly.com/catalog/9780596513986
  9. 9. Example 4 revisited L WANTED: ref list: 0 1 2 str: abc list: 0 1 int: 5 tuple: 0 1 tuple: 0 1 int: 1 int: 2 list: 0 int: 4 int: 3
  10. 10. Example 3 revisited L D ref ref list: 0 1 2 dict: keys X vals WANTED: ref str: a str: b list: 0 1 2 str: y str: x int: 1 int: 3 int: 2
  11. 11. PDF References (internal) 2:0 1:0 202 19 9:0 8:0 205 19 3:0 106 /Page 5:0 97 7:0 10:0 108 53 11:0 /ExtGState /Page 98 24:0 4:0 50 80 22:0 20:0 21:0 /Catalog /Pages 211 3116 21 14:0 15:0 6:0 /FontDescriptor 109 98 173 /Page /Font 25:0 /TrueType trailer 23:0 163 19:0 36 18:0 98 109 26:0 /Page 90 13:0 12:0 208 20 17:0 16:0 208 20
  12. 12. ”First System“ os sys errno posixpath UserDict copy_reg stat Imports of module 'os' (simplified)
  13. 13. ”First System“ fileinfo dir mod mod mod plugins expatbuilder guicocoa compatibility guiwx mod mod fileinfo_plugin_xml mod mod dir minidom mod fileinfo_plugin_py fileinfo_plugin_qt fileinfo_plugin_mp3 xml mod mod mod mod mod __all__ __all__ mod compiler keyword cStringIO tokenize sax test dom mod parsers dir mod mod mod mod mod pycodegen consts transformer mod expatreader _exceptions pulldom examples fileinfo_plugin_test test_fileinfo minicompat NodeFilter domreg xmlbuilder visitor mod mod mod future symbols syntax pyassem symbol parser token fileinfo_plugin_pdf fileinfo_plugin_ttf saxutils expat imports docstrings pythoncode unittest fileinfo copy ast new dis misc pyPdf operator xmlreader urllib urlparse traceback plugging investigator getopt types pdf generic socket string handler linecache glob time datetime struct utils filters _ssl _socket os fnmatch re zlib copy_reg posixpath UserDict stat Module Space
  14. 14. GraphViz
  15. 15. Features • Diagrams of abstract graphs and networks • Automatic graph drawing • GUIs, tools, libraries, and language bindings • Developped by AT&T, Open Source • DOT format (text) ➞ ps, pdf, svg, svgz, fig, mif hpgl, pcl, png, gif, dia, imap, cmapx, …
  16. 16. Example: Hello World Hello digraph G { Hello -> World } World
  17. 17. Example: Hello World 2 Hello digraph G { overlap=false; Hello [shape="box", color="red"]; World [shape="circle", color="blue"]; Hello -> World [color="green"]; } World
  18. 18. Example: FSM digraph finite_state_machine { rankdir=LR; size="8,5" node [shape = doublecircle]; LR_0 LR_3 LR_4 LR_8; node [shape = circle]; LR_0 -> LR_2 [ label = "SS(B)" ]; LR_0 -> LR_1 [ label = "SS(S)" ]; LR_1 -> LR_3 [ label = "S($end)" ]; LR_2 -> LR_6 [ label = "SS(b)" ]; LR_2 -> LR_5 [ label = "SS(a)" ]; LR_2 -> LR_4 [ label = "S(A)" ]; LR_5 -> LR_7 [ label = "S(b)" ]; LR_5 -> LR_5 [ label = "S(a)" ]; LR_6 -> LR_6 [ label = "S(b)" ]; LR_6 -> LR_5 [ label = "S(a)" ]; LR_7 -> LR_8 [ label = "S(b)" ]; LR_7 -> LR_5 [ label = "S(a)" ]; LR_8 -> LR_6 [ label = "S(b)" ]; LR_8 -> LR_5 [ label = "S(a)" ]; }
  19. 19. Example: FSM LR_4 S(A) S(b) S(b) SS(b) LR_6 S(a) LR_2 S(a) LR_8 SS(B) S(a) SS(a) S(b) LR_0 LR_5 S(b) SS(S) S(a) LR_7 LR_1 S($end) LR_3
  20. 20. Pyrels
  21. 21. Features • “Second system” – really simple, so far • Recursive func. collecting nodes and edges • Converting to GraphViz nodes and edges
  22. 22. Pyrels Profile $ cd pyrels $ fileinfo -p rel -a lc:nmtlines:nclasses:ndefs:ncalls scripts/pyrels $(find src -name "*.py") lc nmtlines nclasses ndefs ncalls path 58 14 0 1 9 scripts/pyrels 0 0 0 0 0 src/pyrels/__init__.py 411 70 1 7 163 src/pyrels/pyrels2dot.py 0 0 0 0 0 src/pyrels/test/__init__.py 379 150 0 10 105 src/pyrels/test/pdfposter.py 46 13 0 2 10 src/pyrels/test/test_all.py 99 68 2 3 17 src/pyrels/test/test_basic.py 90 26 1 2 15 src/pyrels/test/test_commandline.py 55 21 1 2 18 src/pyrels/test/test_cyles.py 148 62 1 6 63 src/pyrels/test/test_multi.py 193 66 5 14 50 src/pyrels/test/test_other.py 20 7 0 2 6 src/pyrels/test/testutils.py 12 4 0 1 1 src/pyrels/utils.py 1511 501 11 50 457 total
  23. 23. Example: String Caching ref 1 v str: XXXXXXXXXXXXXXXXXXXXX v = 'X' * 21 ref u str: XXXXXXXXXXXXXXXXXXXXX u = 'X' * 21 s = 'X' * 20 3 t = 'X' * 20 s ref str: XXXXXXXXXXXXXXXXXXXX ref t 2 digraph G { name1775744 [label="v", shape="ellipse"]; str3376528 [label="str: XXXXXXXXXXXXXXXXXXXXX", shape="box"]; name1845536 [label="u", shape="ellipse"]; str3284144 [label="str: XXXXXXXXXXXXXXXXXXXXX", shape="box"]; name1686816 [label="s", shape="ellipse"]; str3376480 [label="str: XXXXXXXXXXXXXXXXXXXX", shape="box"]; name1798720 [label="t", shape="ellipse"]; str3376480 [label="str: XXXXXXXXXXXXXXXXXXXX", shape="box"]; name1686816 -> str3376480 [label="ref"]; name1845536 -> str3284144 [label="ref"]; name1798720 -> str3376480 [label="ref"]; name1775744 -> str3376528 [label="ref"]; }
  24. 24. Example: Singleton None digraph G { 1 2 name3498784 [label="aDict", shape="ellipse"]; dict3660224 [label="dict: | {keys|vals}| {<k0>|<v0>}", shape="record"]; none1 = None none [label="None", shape="box"]; none2 = None none [label="None", shape="box"]; aList = [None] name3499008 [label="aList", shape="ellipse"]; list3662272 [label="list: | <0> 0", aTuple = (None, None) shape="record"]; aDict = {None:None} none [label="None", shape="box"]; name3498976 [label="none2", shape="ellipse"]; none [label="None", shape="box"]; dict: name3498944 [label="none1", shape="ellipse"]; aDict ref keys vals none [label="None", shape="box"]; name3317728 [label="aTuple", shape="ellipse"]; tuple3334392 [label="tuple: | <0> 0|<1> 1", list: shape="Mrecord"]; ref aList none [label="None", shape="box"]; 0 none [label="None", shape="box"]; none2 ref None dict3660224:k0 -> none [label=""]; ref dict3660224:v0 -> none [label=""]; 3 list3662272:0 -> none [label=""]; none1 tuple3334392:0 -> none [label=""]; tuple3334392:1 -> none [label=""]; name3498784 -> dict3660224 [label="ref"]; tuple: name3499008 -> list3662272 [label="ref"]; aTuple ref 0 name3498976 -> none [label="ref"]; name3498944 -> none [label="ref"]; 1 name3317728 -> tuple3334392 [label="ref"]; }
  25. 25. Example: Module Content (pyPdf) ref utils module: pyPdf.utils import pyPdf ref locals().update(pyPdf.__dict__) __doc__ None del pyPdf ref pdf module: pyPdf.pdf ref PdfFileWriter new style class: PdfFileWriter ref __name__ str: 'pyPdf' ref filters module: pyPdf.filters ref __file__ str: '/usr/local/lib/python2.5/site-packages/pyPdf/__init__.pyc' ref generic module: pyPdf.generic ref list: __path__ 0 str: '/usr/local/lib/python2.5/site-packages/pyPdf' ref list: __all__ 0 str: 'pdf' ref PdfFileReader new style class: PdfFileReader
  26. 26. Future
  27. 27. Issues • Testing, debugging & refactoring, … • Detecting cycles • Fiddling with GraphViz params (overlap, …) • Adding other relationships: - Module Imports or not) - Inheritance (UML-like - Call graphs -… • Smart graph pruning
  28. 28. Summary • Development only barely started (v. 0.1) • Useful already for visualising data structures • Provides certain insights for Python newbies • Feedback is warmly welcome!
  29. 29. Links • http://www.dinu-gherman.net/tmp/pyrels-0.1.0.tar.gz • http://www.graphviz.org
  30. 30. Questions?
  31. 31. Questions! • GraphViz editor with branch fold-in/out? • How to get full path for DOT edges (in plain format)?
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×