Visualizing Relationships between Python objects - EuroPython 2008
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share

Visualizing Relationships between Python objects - EuroPython 2008

  • 1,309 views
Uploaded on

A presentation about visualizing relationships between Python objects given at EuroPython 2008.

A presentation about visualizing relationships between Python objects given at EuroPython 2008.

More in: Technology , Education
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
1,309
On Slideshare
1,306
From Embeds
3
Number of Embeds
1

Actions

Shares
Downloads
35
Comments
0
Likes
2

Embeds 3

http://www.slideshare.net 3

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Visualizing Relationships Between Python Objects Dinu C. Gherman gherman@python.net EuroPython Conference 2008-07-08, Vilnius
  • 2. Motivation • Make Python data structures visible • Provide a simple tool for text/book authors • Explore GraphViz capabilities
  • 3. Typical Questions • How does that data structure look like? • How does it change after doing … to it? • What does this module contain?
  • 4. Background
  • 5. Example 1 http://inst.eecs.berkeley.edu/ ~selfpace/cs9honline/Q2/mutation.html
  • 6. Example 2 http://www.linux-user.de/ausgabe/ 2006/09/080-python-1/
  • 7. Example 3 X = [1, 2, 3] L = ['a', X, 'b'] D = {'x':X, 'y':2} http://oreilly.com/catalog/9780596513986
  • 8. Example 4 L = ['abc', [(1, 2), ([3], 4)], 5] http://oreilly.com/catalog/9780596513986
  • 9. Example 4 revisited L WANTED: ref list: 0 1 2 str: abc list: 0 1 int: 5 tuple: 0 1 tuple: 0 1 int: 1 int: 2 list: 0 int: 4 int: 3
  • 10. Example 3 revisited L D ref ref list: 0 1 2 dict: keys X vals WANTED: ref str: a str: b list: 0 1 2 str: y str: x int: 1 int: 3 int: 2
  • 11. PDF References (internal) 2:0 1:0 202 19 9:0 8:0 205 19 3:0 106 /Page 5:0 97 7:0 10:0 108 53 11:0 /ExtGState /Page 98 24:0 4:0 50 80 22:0 20:0 21:0 /Catalog /Pages 211 3116 21 14:0 15:0 6:0 /FontDescriptor 109 98 173 /Page /Font 25:0 /TrueType trailer 23:0 163 19:0 36 18:0 98 109 26:0 /Page 90 13:0 12:0 208 20 17:0 16:0 208 20
  • 12. ”First System“ os sys errno posixpath UserDict copy_reg stat Imports of module 'os' (simplified)
  • 13. ”First System“ fileinfo dir mod mod mod plugins expatbuilder guicocoa compatibility guiwx mod mod fileinfo_plugin_xml mod mod dir minidom mod fileinfo_plugin_py fileinfo_plugin_qt fileinfo_plugin_mp3 xml mod mod mod mod mod __all__ __all__ mod compiler keyword cStringIO tokenize sax test dom mod parsers dir mod mod mod mod mod pycodegen consts transformer mod expatreader _exceptions pulldom examples fileinfo_plugin_test test_fileinfo minicompat NodeFilter domreg xmlbuilder visitor mod mod mod future symbols syntax pyassem symbol parser token fileinfo_plugin_pdf fileinfo_plugin_ttf saxutils expat imports docstrings pythoncode unittest fileinfo copy ast new dis misc pyPdf operator xmlreader urllib urlparse traceback plugging investigator getopt types pdf generic socket string handler linecache glob time datetime struct utils filters _ssl _socket os fnmatch re zlib copy_reg posixpath UserDict stat Module Space
  • 14. GraphViz
  • 15. Features • Diagrams of abstract graphs and networks • Automatic graph drawing • GUIs, tools, libraries, and language bindings • Developped by AT&T, Open Source • DOT format (text) ➞ ps, pdf, svg, svgz, fig, mif hpgl, pcl, png, gif, dia, imap, cmapx, …
  • 16. Example: Hello World Hello digraph G { Hello -> World } World
  • 17. Example: Hello World 2 Hello digraph G { overlap=false; Hello [shape="box", color="red"]; World [shape="circle", color="blue"]; Hello -> World [color="green"]; } World
  • 18. Example: FSM digraph finite_state_machine { rankdir=LR; size="8,5" node [shape = doublecircle]; LR_0 LR_3 LR_4 LR_8; node [shape = circle]; LR_0 -> LR_2 [ label = "SS(B)" ]; LR_0 -> LR_1 [ label = "SS(S)" ]; LR_1 -> LR_3 [ label = "S($end)" ]; LR_2 -> LR_6 [ label = "SS(b)" ]; LR_2 -> LR_5 [ label = "SS(a)" ]; LR_2 -> LR_4 [ label = "S(A)" ]; LR_5 -> LR_7 [ label = "S(b)" ]; LR_5 -> LR_5 [ label = "S(a)" ]; LR_6 -> LR_6 [ label = "S(b)" ]; LR_6 -> LR_5 [ label = "S(a)" ]; LR_7 -> LR_8 [ label = "S(b)" ]; LR_7 -> LR_5 [ label = "S(a)" ]; LR_8 -> LR_6 [ label = "S(b)" ]; LR_8 -> LR_5 [ label = "S(a)" ]; }
  • 19. Example: FSM LR_4 S(A) S(b) S(b) SS(b) LR_6 S(a) LR_2 S(a) LR_8 SS(B) S(a) SS(a) S(b) LR_0 LR_5 S(b) SS(S) S(a) LR_7 LR_1 S($end) LR_3
  • 20. Pyrels
  • 21. Features • “Second system” – really simple, so far • Recursive func. collecting nodes and edges • Converting to GraphViz nodes and edges
  • 22. Pyrels Profile $ cd pyrels $ fileinfo -p rel -a lc:nmtlines:nclasses:ndefs:ncalls scripts/pyrels $(find src -name "*.py") lc nmtlines nclasses ndefs ncalls path 58 14 0 1 9 scripts/pyrels 0 0 0 0 0 src/pyrels/__init__.py 411 70 1 7 163 src/pyrels/pyrels2dot.py 0 0 0 0 0 src/pyrels/test/__init__.py 379 150 0 10 105 src/pyrels/test/pdfposter.py 46 13 0 2 10 src/pyrels/test/test_all.py 99 68 2 3 17 src/pyrels/test/test_basic.py 90 26 1 2 15 src/pyrels/test/test_commandline.py 55 21 1 2 18 src/pyrels/test/test_cyles.py 148 62 1 6 63 src/pyrels/test/test_multi.py 193 66 5 14 50 src/pyrels/test/test_other.py 20 7 0 2 6 src/pyrels/test/testutils.py 12 4 0 1 1 src/pyrels/utils.py 1511 501 11 50 457 total
  • 23. Example: String Caching ref 1 v str: XXXXXXXXXXXXXXXXXXXXX v = 'X' * 21 ref u str: XXXXXXXXXXXXXXXXXXXXX u = 'X' * 21 s = 'X' * 20 3 t = 'X' * 20 s ref str: XXXXXXXXXXXXXXXXXXXX ref t 2 digraph G { name1775744 [label="v", shape="ellipse"]; str3376528 [label="str: XXXXXXXXXXXXXXXXXXXXX", shape="box"]; name1845536 [label="u", shape="ellipse"]; str3284144 [label="str: XXXXXXXXXXXXXXXXXXXXX", shape="box"]; name1686816 [label="s", shape="ellipse"]; str3376480 [label="str: XXXXXXXXXXXXXXXXXXXX", shape="box"]; name1798720 [label="t", shape="ellipse"]; str3376480 [label="str: XXXXXXXXXXXXXXXXXXXX", shape="box"]; name1686816 -> str3376480 [label="ref"]; name1845536 -> str3284144 [label="ref"]; name1798720 -> str3376480 [label="ref"]; name1775744 -> str3376528 [label="ref"]; }
  • 24. Example: Singleton None digraph G { 1 2 name3498784 [label="aDict", shape="ellipse"]; dict3660224 [label="dict: | {keys|vals}| {<k0>|<v0>}", shape="record"]; none1 = None none [label="None", shape="box"]; none2 = None none [label="None", shape="box"]; aList = [None] name3499008 [label="aList", shape="ellipse"]; list3662272 [label="list: | <0> 0", aTuple = (None, None) shape="record"]; aDict = {None:None} none [label="None", shape="box"]; name3498976 [label="none2", shape="ellipse"]; none [label="None", shape="box"]; dict: name3498944 [label="none1", shape="ellipse"]; aDict ref keys vals none [label="None", shape="box"]; name3317728 [label="aTuple", shape="ellipse"]; tuple3334392 [label="tuple: | <0> 0|<1> 1", list: shape="Mrecord"]; ref aList none [label="None", shape="box"]; 0 none [label="None", shape="box"]; none2 ref None dict3660224:k0 -> none [label=""]; ref dict3660224:v0 -> none [label=""]; 3 list3662272:0 -> none [label=""]; none1 tuple3334392:0 -> none [label=""]; tuple3334392:1 -> none [label=""]; name3498784 -> dict3660224 [label="ref"]; tuple: name3499008 -> list3662272 [label="ref"]; aTuple ref 0 name3498976 -> none [label="ref"]; name3498944 -> none [label="ref"]; 1 name3317728 -> tuple3334392 [label="ref"]; }
  • 25. Example: Module Content (pyPdf) ref utils module: pyPdf.utils import pyPdf ref locals().update(pyPdf.__dict__) __doc__ None del pyPdf ref pdf module: pyPdf.pdf ref PdfFileWriter new style class: PdfFileWriter ref __name__ str: 'pyPdf' ref filters module: pyPdf.filters ref __file__ str: '/usr/local/lib/python2.5/site-packages/pyPdf/__init__.pyc' ref generic module: pyPdf.generic ref list: __path__ 0 str: '/usr/local/lib/python2.5/site-packages/pyPdf' ref list: __all__ 0 str: 'pdf' ref PdfFileReader new style class: PdfFileReader
  • 26. Future
  • 27. Issues • Testing, debugging & refactoring, … • Detecting cycles • Fiddling with GraphViz params (overlap, …) • Adding other relationships: - Module Imports or not) - Inheritance (UML-like - Call graphs -… • Smart graph pruning
  • 28. Summary • Development only barely started (v. 0.1) • Useful already for visualising data structures • Provides certain insights for Python newbies • Feedback is warmly welcome!
  • 29. Links • http://www.dinu-gherman.net/tmp/pyrels-0.1.0.tar.gz • http://www.graphviz.org
  • 30. Questions?
  • 31. Questions! • GraphViz editor with branch fold-in/out? • How to get full path for DOT edges (in plain format)?