Profiler Zoo    Alexandre Bergel abergel@dcc.uchile.clPleiad lab, UChile, Chile
2
Test coverageProblem:  Traditional code coverage tools have a binary view of the worldWhy the problem is important:  Which...
Test blueprintC1            Legend for methods (inner boxes)                 # calling methodsd            complexity     ...
Successive improvementVersion 2.2   Version 2.3   Version 2.4   Version 2.5 27.27%        54.54%        87.71%         100...
4 patterns             6
Moose-Test-Core.13                                 Moose-Test-Core.48          Moose-Core.313                             ...
Reducing code complexity Version 1.58.1     Version 1.58.9Coverage: 40.57%   Coverage: 60.60%                             ...
Reducing code complexity        Version 2.10        Version 2.17                           9
Execution profilingProblem:  Traditional code profilers are driven by the method stack, discarding the notion of sending mes...
Structural profiling blueprint                      legend for methods                                     # executions    ...
Structural profiling blueprintbounds                           legend for methods                                          ...
Behavioral profiling blueprint                  legend for methods                                 # executions            ...
Behavioral profiling blueprint                  legend for methods                                 # executions            ...
Code of the bounds methodMOGraphElement>>bounds "Answer the bounds of the receiver." | basicBounds | self shapeBoundsAt: s...
MemoizingMOGraphElement>>bounds "Answer the bounds of the receiver." | basicBounds | boundsCache ifNotNil: [ ^ boundsCache...
Upgrading        MOGraphElement>>boundsA    B                     C                                 17
43%speedup             Upgrading              MOGraphElement>>bounds      A          B                           C        ...
A      UpgradingMOGraphElement>>bounds                             B                                 19
B                  C                      A    cachedabsoluteBounds                                                 instea...
Measuring execution timeProblem:  Traditional code profilers sample program execution at a regular interval. This is inaccu...
Counting messagesWallet >> increaseByOne   money := money + 1Wallet >> increaseBy3   self     increaseByOne;     increaseB...
Execution time and number of                         message sends                        6                 400 x 10messag...
Counting Messages to Identify                      Execution Bottlenecks                              6number of method   ...
Contrasting Execution Sampling with         Message CountingNo need for samplingIndependent from the execution environment...
Counting messages in unit testingCollectionTest>>testInsertion   self     assert: [ Set new add: 1 ]     fasterThan: [Set ...
Counting messages in unit testingMondrianSpeedTest>> testLayout2  | view1 view2 |  view1 := MOViewRenderer new.  view1 nod...
Assessing profiler stability                              28
Identifying redundant computationProblem:  Traditional code profiler cannot determine whether the same computation is reali...
ExampleMOGraphElement>> absoluteBounds   ^ self shape absoluteBoundsFor: selfMOGraphElement>> absoluteBounds   boundsCache...
ExampleAbstractNautilusUI>>packageIconFor: aPackage   |t|   (packageIconCache notNil      and: [ packageIconCache includes...
Identifying memoization candidateA method is candidate for being memoized if  it is executed more than once on a particula...
ExperimentWe took 11 applications and profiled their unit testsWe identified candidates for each of themWe memoized some of ...
34
Execution timeIn some case we reduce the execution time by 20%  e.g., NautilusIn some other case, the execution time incre...
Research questions Is there a general way to identify redundant messagesby monitoring program execution? Can redundant mes...
The Spy profiling frameworkCore                                                                           MethodSpy        ...
Closing wordsLittle innovation in the tools we commonly use  Profilers & debuggers have not significantly evolvesFantastic o...
Thanks toLaurent Laffont, RMoD inria groupall the people who participated in our experiments                              ...
Test coverage with Hapao                     Moose-Test-Core.13                                 Moose-Test-Core.48Profiling...
Upcoming SlideShare
Loading in …5
×

Profilling Zoo

520 views

Published on

Profilling Zoo

ESUG 2011, Edinburgh

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
520
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
7
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Profilling Zoo

  1. 1. Profiler Zoo Alexandre Bergel abergel@dcc.uchile.clPleiad lab, UChile, Chile
  2. 2. 2
  3. 3. Test coverageProblem: Traditional code coverage tools have a binary view of the worldWhy the problem is important: Which method should you test first in order to increase the coverage? Is my code well covered or not?Solution: An intuitive visual representation of a qualitative assessment of the coverage 3
  4. 4. Test blueprintC1 Legend for methods (inner boxes) # calling methodsd complexity # executions c invocation on selfC2 red = not executed blue = abstract 4
  5. 5. Successive improvementVersion 2.2 Version 2.3 Version 2.4 Version 2.5 27.27% 54.54% 87.71% 100% 5
  6. 6. 4 patterns 6
  7. 7. Moose-Test-Core.13 Moose-Test-Core.48 Moose-Core.313 Moose-Core.326 21.42% 100% 56.86% 100% 73.58% 100%68.25% 96.66% 36.78% 64.55% 0% 100% 7
  8. 8. Reducing code complexity Version 1.58.1 Version 1.58.9Coverage: 40.57% Coverage: 60.60% 8
  9. 9. Reducing code complexity Version 2.10 Version 2.17 9
  10. 10. Execution profilingProblem: Traditional code profilers are driven by the method stack, discarding the notion of sending messagesWhy the problem is important: How to answer to “Is there a slow function that is called too often?”Solution: An intuitive visual representation of the execution that visually compare the time spent and the number of executions 10
  11. 11. Structural profiling blueprint legend for methods # executions (color) execution #different time receiver 11
  12. 12. Structural profiling blueprintbounds legend for methods # executions (color) execution #different time receiver 12
  13. 13. Behavioral profiling blueprint legend for methods # executions gray = return m1 m3 self execution m1 time yellow = invokes constant m2 m2 and m3 on return value 13
  14. 14. Behavioral profiling blueprint legend for methods # executions gray = return m1 m3 self execution m1 time yellow = invokes m2 m2 and m3bounds constant on return value 14
  15. 15. Code of the bounds methodMOGraphElement>>bounds "Answer the bounds of the receiver." | basicBounds | self shapeBoundsAt: self shape ifPresent: [ :b | ^ b ]. basicBounds := shape computeBoundsFor: self. self shapeBoundsAt: self shape put: basicBounds. ^ basicBounds 15
  16. 16. MemoizingMOGraphElement>>bounds "Answer the bounds of the receiver." | basicBounds | boundsCache ifNotNil: [ ^ boundsCache ]. self shapeBoundsAt: self shape ifPresent: [ :b | ^ b ]. basicBounds := shape computeBoundsFor: self. self shapeBoundsAt: self shape put: basicBounds. ^ boundsCache := basicBounds 16
  17. 17. Upgrading MOGraphElement>>boundsA B C 17
  18. 18. 43%speedup Upgrading MOGraphElement>>bounds A B C 18
  19. 19. A UpgradingMOGraphElement>>bounds B 19
  20. 20. B C A cachedabsoluteBounds instea A C B C 20
  21. 21. Measuring execution timeProblem: Traditional code profilers sample program execution at a regular interval. This is inaccurate, non portable and non deterministicWhy the problem is important: all profiles are meaningless if I get a new laptop or change the virtual machine cannot profile short execution timeSolution: counting messages as a proxy for execution time 21
  22. 22. Counting messagesWallet >> increaseByOne money := money + 1Wallet >> increaseBy3 self increaseByOne; increaseByOne; increaseByOne.aWallet increaseBy3=> 6 messages sent 22
  23. 23. Execution time and number of message sends 6 400 x 10message sends 400000000 6 300 x 10 300000000 6 200 x 10 200000000 6 100 x 10 100000000 0 0 10000 20000 30000 40000 times (ms) 23
  24. 24. Counting Messages to Identify Execution Bottlenecks 6number of method 10000000 10.0 x 10 invocations 6 7500000 7.5 x 10 6 5.0 x 10 5000000 6 2.5 x 10 2500000 0 0 75 150 225 300 time (ms) 24
  25. 25. Contrasting Execution Sampling with Message CountingNo need for samplingIndependent from the execution environmentStable measurements 25
  26. 26. Counting messages in unit testingCollectionTest>>testInsertion self assert: [ Set new add: 1 ] fasterThan: [Set new add: 1; add: 2 ] 26
  27. 27. Counting messages in unit testingMondrianSpeedTest>> testLayout2 | view1 view2 | view1 := MOViewRenderer new. view1 nodes: (Collection allSubclasses). view1 edgesFrom: #superclass. view1 treeLayout. view2 := MOViewRenderer new. view2 nodes: (Collection withAllSubclasses). view2 edgesFrom: #superclass. view2 treeLayout. self assertIs: [ view1 root applyLayout ] fasterThan: [ view2 root applyLayout ] 27
  28. 28. Assessing profiler stability 28
  29. 29. Identifying redundant computationProblem: Traditional code profiler cannot determine whether the same computation is realized twice or moreWhy the problem is important: Redundant computation cannot be identified and removed without heavily involving programmer imaginationSolution: Finding side-effect-free methods that are executed more than once 29
  30. 30. ExampleMOGraphElement>> absoluteBounds ^ self shape absoluteBoundsFor: selfMOGraphElement>> absoluteBounds boundsCache ifNotNil: [ ^ boundsCache ]. ^ boundsCache := self shape absoluteBoundsFor: selfMONode>> translateBy: realStep boundsCache := nil. ... 30
  31. 31. ExampleAbstractNautilusUI>>packageIconFor: aPackage |t| (packageIconCache notNil and: [ packageIconCache includesKey: aPackage ]) ifTrue: [ ˆ packageIconCache at: aPackage ]. packageIconCache ifNil: [ packageIconCache := IdentityDictionary new ]. aPackage isDirty ifTrue: [ ˆ IconicButton new ]. t := self iconClass iconNamed: #packageIcon. packageIconCache at: aPackage put: t. ˆt 31
  32. 32. Identifying memoization candidateA method is candidate for being memoized if it is executed more than once on a particular object it returns the same value per receiver and arguments it does not any “perceptible” side effect its execution is sufficiently long 32
  33. 33. ExperimentWe took 11 applications and profiled their unit testsWe identified candidates for each of themWe memoized some of the candidatesThe tests are kept green 33
  34. 34. 34
  35. 35. Execution timeIn some case we reduce the execution time by 20% e.g., NautilusIn some other case, the execution time increased (!) This is the case for very short methods 35
  36. 36. Research questions Is there a general way to identify redundant messagesby monitoring program execution? Can redundant messages be removed whilepreserving the overall program behavior? 36
  37. 37. The Spy profiling frameworkCore MethodSpy Profiler methodName packages ClassSpy class currentTest PackageSpy spyClassForMethod originalMethod profile: aBlock spyClassForClass package outgoingCalls runTests: tests packageName superclass incomingCalls spyClassForPackage classes metaclass timeExecution allMethods methods afterRun:with: in: registryName beforeRun:with:in: run:with:in:TestCoverage TCMethod numberOfExectutions TestCoverage receiverTable spyClassForPackage beforeRun:with:in: view TCPackage TCClass numberOfDifferentReceivers ratioExecutedMethods spyClassForClass spyClassForMethod nbOfExecutions ratioCoveredClasses isCovered viewBasicOn: initialize registryName viewBasicOn: 37
  38. 38. Closing wordsLittle innovation in the tools we commonly use Profilers & debuggers have not significantly evolvesFantastic opportunities for improvement 38
  39. 39. Thanks toLaurent Laffont, RMoD inria groupall the people who participated in our experiments 39
  40. 40. Test coverage with Hapao Moose-Test-Core.13 Moose-Test-Core.48Profiling blueprints Moose-Core.313 Moose-Core.326 21.42% 100%Proxy for execution timeIdentifying redundant computation 56.86% 100% 73.58% 100% 68.25% 96.66% 36.78% 64.55% 0% 100% Profiler Zoohttp://bergel.euhttp://hapao.dcc.uchile.clhttp://moosetechnology.org/tools/spy 40

×