2011 ecoop

Counting Messages as a Proxy for AverageExecution Time in Pharo ECOOP 2011 - Lancaster Alexandre Bergel Pleiad lab, DCC, University of Chile http://bergel.eu

The Mondrian Visualization Engine www.pharo-project.org 2

“I like the cool new features of Mondrian, but in my setting,drawing a canvas takes 10 seconds, whereas it took only 7yesterday. Please do something!” -- A Mondrian user, 2009 -- 3

“I like the cool new features of Mondrian, but in my setting,drawing my visualization takes 10 seconds, whereas it tookonly 7 yesterday. Please do something!” -- A Mondrian user, 2009 -- 4

Result of Pharo proﬁler54.8% {11501ms} MOCanvas>>drawOn: 54.8% {11501ms} MORoot(MONode)>>displayOn: 30.9% {6485ms} MONode>>displayOn: | 18.1% {3799ms} MOEdge>>displayOn: ... | 8.4% {1763ms} MOEdge>>displayOn: | | 8.0% {1679ms} MOStraightLineShape>>display:on: | | 2.6% {546ms} FormCanvas>>line:to:width:color: ... 23.4% {4911ms} MOEdge>>displayOn: ... 5

Yesterday version32.9% {6303ms} MOCanvas>>drawOn: 32.9% {6303ms} MORoot(MONode)>>displayOn: 24.4% {4485ms} MONode>>displayOn: | 12.5% {1899ms} MOEdge>>displayOn: ... | 4.2% {1033ms} MOEdge>>displayOn: | | 6.0% {1679ms} MOStraightLineShape>>display:on: | | 2.4% {546ms} FormCanvas>>line:to:width:color: ... 8.5% {2112ms} MOEdge>>displayOn: ... 6

“I like the cool new features of Mondrian, but in my setting,drawing my visualization takes 10 seconds, whereas it tookonly 7 yesterday. Please do something!” -- A Mondrian user, 2009 --On my machine I ﬁnd 11 and 6 seconds. What’s going on? 7

How proﬁlers workSampling the method call stack every 10 msA counter is associated to each frameEach counter is incremented when being sampled 8

How proﬁlers work Sampling the method call stack every 10 ms A counter is associated to each frame Each counter is incremented when being sampled MONode displayOn: (1)method call MORoot displayOn: (1) stack Canvas drawOn: (1) Time = t 9

How proﬁlers work Sampling the method call stack every 10 ms A counter is associated to each frame Each counter is incremented when being sampled MOEdge displayOn: (1) MONode displayOn: (2)method call MORoot displayOn: (2) stack Canvas drawOn: (2) Time = t + 10 ms 10

How proﬁlers work Sampling the method call stack every 10 ms A counter is associated to each frame Each counter is incremented when being sampled MONode setCache (1)method call MORoot displayOn: (3) stack Canvas drawOn: (3) Time = t + 20 ms 11

How proﬁlers work The counter is used to estimate the amount of timespent MONode setCache (1) MOEdge displayOn: (1) MONode displayOn: (2) MORoot displayOn: (3) Canvas drawOn: (3) 12

How proﬁlers work The counter is used to estimate the amount of timespent MONode setCache (1) => 10 ms MOEdge displayOn: (1) => 10 ms MONode displayOn: (2) => 20 ms MORoot displayOn: (3) => 30 ms Canvas drawOn: (3) => 30 ms 13

Problem with execution sampling #1Strongly dependent on the executing environment CPU, memory management, threads, virtual machine, processesListening at a mp3 may perturb your proﬁle 14

Problem with execution sampling #2Non-determinism Even using the same environment does not help“30000 factorial” takes between 3 803 and 3 869 ms 15

Problem with execution sampling #3 Lack of portability Proﬁles are not reusable across platform Buying a new laptop will invalidate the proﬁle youmade yesterday 16

Counting messages to the rescue Pharo is a Smalltalk dialect Intensively based on sending message Almost “Optimization-free compiler” Why not to count messages instead of executiontime? 17

Counting messagesWallet >> increaseByOne money := money + 1Wallet >> addBonus self increaseByOne; increaseByOne; increaseByOne.aWallet addBonus=> 6 messages sent 18

Does this really work? What about the program?MyClass >> main self waitForUserClick We took scenarios from unit tests, which do not relyon user input 19

Experiment A 6 400 x 10 message sends 400000000 6 application 300 x 10 300000000 6 200 x 10 200000000 6 100 x 10 100000000 0 0 10000 20000 30000 40000 times (ms) The number of sent messages related to the averageexecution time over multiple executions 20

Experiment B Application time taken (ms) # sent messages ctime % cmessages % Collections 32 317 334 359 691 16.67 1.05 Mondrian 33 719 292 140 717 5.54 1.44 Nile 29 264 236 817 521 7.24 0.22 Moose 25 021 210 384 157 24.56 2.47 SmallDude 13 942 150 301 007 23.93 0.99 Glamour 10 216 94 604 363 3.77 0.14 Magritte 2 485 37 979 149 2.08 0.85 PetitParser 1 642 31 574 383 46.99 0.52 Famix 1 014 6 385 091 18.30 0.06 DSM 4 012 5 954 759 25.71 0.17 ProfStef 247 3 381 429 0.77 0.10 Network 128 2 340 805 6.06 0.44 AST 37 677 439 1.26 0.46 XMLParser 36 675 205 32.94 0.46 Arki 30 609 633 1.44 0.35 ShoutTests 19 282 313 5.98 0.11 Average 13.95 0.61Table 2.number of sent messages more stable third columns The Applications considered in our experiment (second and than theare average overtime over multiple executions execution 10 runs) 21

Experiment C 6 number of method 10000000 10.0 x 10 invocations 6 method 7500000 7.5 x 10 6 5.0 x 10 5000000 6 2.5 x 10 2500000 0 0 75 150 225 300 time (ms) The number of sent messages as useful as theexecution time to identify an execution bottleneck 22

CompteurCompteurMethod>> run: methodName with: args in: receiver | oldNumberOfCalls v | oldNumberOfCalls := self getNumberOfCalls. v := originalMethod valueWithReceiver: receiver arguments: args. numberOfCalls := (self getNumberOfCalls - oldNumberOfCalls) + numberOfCalls - 5. ˆv 23

New primitive in the VMCompteurMethod>> run: methodName with: args in: receiver | oldNumberOfCalls v | oldNumberOfCalls := self getNumberOfCalls. v := originalMethod valueWithReceiver: receiver arguments: args. numberOfCalls := (self getNumberOfCalls - oldNumberOfCalls) + numberOfCalls - 5. ˆv 24

Cost of the instrumentationOverhead (%) Overhead (%) 3000 10000 2250 1000 1500 100 750 10 0 1 0 10000 20000 30000 40000 0 10000 20000 30000 40000 Execution time (ms) Execution time (ms) (a) Linear scale (b) Logarithmic scale 25

Contrasting Execution Sampling with Message CountingNo need for samplingIndependent from the execution environmentStable measurements 26

Application #1 Counting messages in unit testingCollectionTest>>testInsertion self assert: [ Set new add: 1] fasterThan: [Set new add: 1; add: 2] 27

Application #1 Counting messages in unit testingMondrianSpeedTest>> testLayout2 | view1 view2 | view1 := MOViewRenderer new. view1 nodes: (Collection allSubclasses). view1 edgesFrom: #superclass. view1 treeLayout. view2 := MOViewRenderer new. view2 nodes: (Collection withAllSubclasses). view2 edgesFrom: #superclass. view2 treeLayout. self assertIs: [ view1 root applyLayout ] fasterThan: [ view2 root applyLayout ] 28

Application #2 Differencing proﬁling Comparison of two successive versions of a software(not in the paper) 29

Application #2 Differencing proﬁling Comparison of two successive versions of Mondrian(not in the paper) 30

More in the paper Linear regression model We replay some optimizations we had in our previouswork A methodology to evaluate proﬁler stability overmultiple run All the material to reproduce the experiments 31

Summary Counting method invocation is a more advantageousproﬁling technique, in Pharo Stable correlation between message sending andaverage execution time 32

Closing words The same abstractions are used to proﬁleapplications written in C and in Java Which objects is responsible of a slowdown? Which arguments make a method call slow? ... 33

6 number of method 6 10000000 10.0 x 10 400 x 10 invocations message sends 400000000 6 6 7500000 7.5 x 10 300 x 10 300000000 6 6 200 x 10 200000000 5.0 x 10 5000000 6 6 100 x 10 100000000 2.5 x 10 2500000 0 0 10000 20000 30000 40000 0 0 75 150 225 300 times (ms) time (ms) Counting message as a proxy for average execution time Alexandre Bergel http://bergel.eu Overhead (%) Overhead (%) 3000 10000CollectionTest>>testInsertion 2250 1000self assert: [Set new 1500 1] add: 100 fasterThan: [Set new add: 1; add: 2] 750 10 0 1 0 10000 20000 30000 34 40000 0 10000 20000 30000 40000 Execution time (ms) Execution time (ms)

2011 ecoop

by bergel on Jul 29, 2011

Accessibility

Categories

Tags

Upload Details

Usage Rights

2 Embeds 66

Statistics

2011 ecoop — Presentation Transcript

http://bergel.eu	48
http://www.bergel.eu	18