Modeling History to Understand Software Evolution With Hismo 2008-02-25

Slide 1: Modeling History to Understand Software Evolution Tudor Gîrba www.tudorgirba.com

Slide 2: Modeling History to Understand Software Evolution Inauguraldissertation der Philosophisch-naturwissenschaftlichen Fakultät der Universität Bern vorgelegt von Tudor Gîrba von Rumänien Leiter der Arbeit: Prof. Dr. Stéphane Ducasse Prof. Dr. Oscar Nierstrasz Institut für Informatik und angewandte Mathematik

Slide 3: } } { { } } { { g rin ee gin en d ar rw fo

Slide 4: fo rw ar d en gin ee rin g { { { { { { } { { } } actual development } } } { } } }

Slide 5: fo rw g rin ar ee d gin en gin en ee se rin erv g re { { { { { { } { { } } actual development } } } { } } }

Slide 6: { { } } } { { { } } re v er se en gin ee rin g reverse engineering fo actual development rw ar d en gin ee rin g { { } } { { } }

Slide 7: Most often time is put on the horizontal and a property on the vertical axis. Lehman etal, 2001

Slide 8: Evolution Matrix shows how classes evolve. Time is still on the horizontal axis. Lanza, Ducasse, 2002

Slide 9: Co-change analysis recovers hidden dependencies. Time is the lines. Gall etal, 2003

Slide 10: Evolution information can be mapped on structural information. Eick etal, 2002

Slide 11: Eick etal, 2002 Lehman etal, 2001 ... Lanza, Ducasse, 2002 Gall etal, 2003

Slide 12: Eick etal, 2002 es? niqu tech Lehman etal, 2001 ese ll th da te a om mo we acc can ow H ... Lanza, Ducasse, 2002 Gall etal, 2003

Slide 13: sh ort int er me zzo What is a model?

Slide 14: sh ort int er me zzo A model is a simpliﬁcation of the subject, and its purpose is to answer some particular questions aimed towards the subject. Bezivin, Gerbe, 2001

Slide 15: sh ort int er me zzo what is a meta-model?

Slide 16: sh ort int er me zzo a meta-model is a model that makes statements about what can be expressed in valid models. Seidewitz, 2003

Slide 17: sh ort int er me zzo a good meta-model allows for succinct expression of analyses.

Slide 18: Eick etal, 2002 Lehman etal, 2001 ... Lanza, Ducasse, 2002 Gall etal, 2003

Slide 19: Eick etal, 2002 es? niqu tech Lehman etal, 2001 ese ll th da te a om mo we acc can ow H ... Lanza, Ducasse, 2002 Gall etal, 2003

Slide 20: Evolution Matrix shows class changes. Lanza, Ducasse, 2004 Idle class Pulsar class Supernova class attributes White dwarf class methods Class

Slide 22: Evolution Matrix shows class changes. Lanza, Ducasse, 2004 Idle class entity. st c lass Pulsar da s ﬁr class mo dele to be needs Supernova tio n class Evolu attributes White dwarf class methods Class

Slide 24: Evolution Matrix shows class changes. Idle class history Pulsar class history Supernova class history ClassHistory White dwarf isPulsar class history isIdle ...

Slide 25: System Version Class Version

Slide 26: System Version Class Class History Version

Slide 27: System System History Version Class Class History Version

Slide 29: System System History Version en tity. st c lass 200 5 ya s ﬁr Gir ba, tor l s his m ode H ismo Class Class History Version

Slide 30: Measuring Yesterday’s Time-based history Weather Detection Strategies 1 2 3 4 5 6 Visualizing the evolution Detecting How developers of hierarchies co-change patterns drive evolution

Slide 31: Measuring history 1

Slide 32: What changed? When did it change? ... 2 4 3 5 7 2 2 3 4 9 2 2 1 2 3 2 2 2 2 2 1 5 3 4 4

Slide 33: Evolution of Number of Methods LENOM(C) = ∑ |NOMi(C)-NOMi-1(C)| 2i-n LENOM(C) = 4 + 2 + 1 + 0 = 7 1 5 3 4 4

Slide 34: Latest Evolution of Number of Methods LENOM(C) = ∑ |NOMi(C)-NOMi-1(C)| 2i-n Earliest Evolution of Number of Methods EENOM(C) = ∑ |NOMi(C)-NOMi-1(C)| 22-i -3 -2 -1 0 LENOM(C) = 42 + 22 + 12 + 02 = 1.5 1 5 3 4 4 EENOM(C) = 4 20 + 2 2-1 + 1 2-2 + 0 2-3 = 5.25

Slide 35: ENOM LENOM EENOM 2 4 3 5 7 7 3.5 3.25 2 2 3 4 9 7 5.75 1.37 2 2 1 2 3 3 1 2 2 2 2 2 2 0 0 0 1 5 3 4 4 7 1.25 5.25

Slide 36: ENOM LENOM EENOM balanced changer 7 3.5 3.25 late changer 7 5.75 1.37 3 1 2 dead stable 0 0 0 early changer 7 1.25 5.25

Slide 37: ENOM LENOM EENOM balanced changer 7 3.5 3.25 late changer 7 tity 5.75 nts. 1.37 en ureme lass eas st c gh m s ﬁr rou 3 y a th 1 2 r H isto rison co mpa deadsstable ena ble 0 0 0 early changer 7 1.25 5.25

Slide 38: History can be measured in many ways. Evolution Number of Methods Stability Number of Lines of Code Historical Max of Cyclomatic Complexity Growth Trend Number of Modules ... ...

Slide 39: Yesterday’s weather 2

Slide 40: The recently changed parts are likely to change in the near future. Common wisdom

Slide 41: The recently changed parts are likely to change in the near future. ally? Common wisdom re re they A

Slide 42: 30% 90%

Slide 45: present

Slide 46: past present

Slide 47: past future present

Slide 50: past future prediction hit present

Slide 51: past future YesterdayWeatherHit(present): past:=histories.topLENOM(start, present) future:=histories.topEENOM(present, end) past.intersectWith(future).notEmpty() prediction hit present

Slide 52: Overall Yesterday’s Weather shows the localization of changed in time. Girba etal, 2004 hit hit hit YW = 3 / 8 = 37% hit hit hit hit hit hit hit YW = 7 / 8 = 87%

Slide 53: Time-based Detection Strategies 3

Slide 54: Detection Strategies are metric-based queries to detect design ﬂaws. Lanza, Marinescu 2006 Rule 1 METRIC 1 > Threshold 1 AND Quality problem Rule 2 METRIC 2 < Threshold 2

Slide 55: Example: a God Class centralizes too much intelligence in the system. Class uses directly more than a few attributes of other classes ATFD > FEW Functional complexity of the class is very high AND GodClass WMC ! VERY HIGH Class cohesion is low TCC < ONE THIRD

Slide 56: Example: a God Class centralizes too much intelligence in the system. Class uses directly more than a few attributes of other classes ATFD > FEW tab le? f it is s wh Functional complexity of the at i ut, class is very high B AND GodClass WMC ! VERY HIGH Class cohesion is low TCC < ONE THIRD

Slide 57: History-based Detection Strategies take evolution into account. Ratiu etal, 2004 God Class in the last version isGodClass(last) AND Harmless God Class Stable throughout the history Stability > 90%

Slide 58: History-based Detection Strategies take evolution into account. Ratiu etal, 2004 me. God Class in the last version sa ed the isGodClass(last) eat e tr pace ar an ds AND Harmless God Class e Tim Stable throughout the history Stability > 90%

Slide 59: Visualizing the evolution of hierarchies 4

Slide 60: What happens with inheritance? A A A A A B C B C B C B B D D D E ver .1 ver. 2 ver. 3 ver. 4 ver. 5

Slide 61: History contains too much data. A A A A A A A A A A A A A A A A A A A A B C B C B C B B B C B C B C B B B C B C B C B B B C B C B C B B D D D E D D D E D D D E D D D E ver .1 ver. 2 ver. 3 ver. 4 ver. 5 ver .1 ver. 2 ver. 3 ver. 4 ver. 5 ver .1 ver. 2 ver. 3 ver. 4 ver. 5 ver .1 ver. 2 ver. 3 ver. 4 ver. 5 A A A A A A A A A A A A A A A A A A A A B C B C B C B B B C B C B C B B B C B C B C B B B C B C B C B B D D D E D D D E D D D E D D D E ver .1 ver. 2 ver. 3 ver. 4 ver. 5 ver .1 ver. 2 ver. 3 ver. 4 ver. 5 ver .1 ver. 2 ver. 3 ver. 4 ver. 5 ver .1 ver. 2 ver. 3 ver. 4 ver. 5 A A A A A A A A A A A A A A A A A A A A B C B C B C B B B C B C B C B B B C B C B C B B B C B C B C B B D D D E D D D E D D D E D D D E ver .1 ver. 2 ver. 3 ver. 4 ver. 5 ver .1 ver. 2 ver. 3 ver. 4 ver. 5 ver .1 ver. 2 ver. 3 ver. 4 ver. 5 ver .1 ver. 2 ver. 3 ver. 4 ver. 5 A A A A A A A A A A A A A A A A A A A A B C B C B C B B B C B C B C B B B C B C B C B B B C B C B C B B D D D E D D D E D D D E D D D E ver .1 ver. 2 ver. 3 ver. 4 ver. 5 ver .1 ver. 2 ver. 3 ver. 4 ver. 5 ver .1 ver. 2 ver. 3 ver. 4 ver. 5 ver .1 ver. 2 ver. 3 ver. 4 ver. 5

Slide 63: System System History Version Inheritance Version Class Class History Version

Slide 64: System System History Version Inheritance Inheritance History Version Class Class History Version

Slide 65: A A A A A B C B C B C B B D D D E ver .1 ver. 2 ver. 3 ver. 4 ver. 5 A is persistent, B is stable, C was removed, E is newborn ...

Slide 66: Hierarchy Evolution View encapsulates time. Girba etal, 2005 A changed methods changed age lines C B Removed Removed D E A is persistent, B is stable, C was removed, E is newborn ...

Slide 67: Hierarchy Evolution View reveals patterns. Girba etal, 2005

Slide 68: Hierarchy Evolution View reveals patterns. Girba etal, 2005 ntity . ss e aph t cla a gr ﬁrs to as ing tory mapp His les e nab

Slide 69: Identifying co-change patterns 5

Slide 70: 1 2 3 4 5 6 A A B B E C D E C D Gall etal, ‘98

Slide 71: Co-change patterns are n-ary relationships.

Slide 72: 1 2 3 4 5 6 A B C D Version E

Slide 73: 1 2 3 4 5 6 A B C D Version E changed

Slide 74: 1 2 3 4 5 6 A History changed(i) B C D Version E changed

Slide 75: What is Concept Analysis?

Slide 76: 1 2 3 4 5 6 A B C D E

Slide 77: {A, B, C, D, E} 1 2 3 4 5 6 Ø A {A, D, B} {A, E, C, D} {2} {6} B {D, B} {A, D} {A, E, C} C FCA {2, 4} {2, 6} {5, 6} D {D} {A} {C} E {2, 4, 6} {2, 5, 6} {3, 5, 6} Ø {1, 2, 3, 4, 5, 6}

Slide 78: {A, B, C, D, E} 1 2 3 4 5 6 Ø A {A, D, B} {A, E, C, D} {2} {6} B {D, B} {A, D} {A, E, C} C FCA {2, 4} {2, 6} {5, 6} D {D} {A} {C} E {2, 4, 6} {2, 5, 6} {3, 5, 6} Ø {1, 2, 3, 4, 5, 6} Girba etal, 2007

Slide 79: Parallel Inheritance add simultaneously children to several classes Shotgun Surgery change several classes simultaneously, but do not add methods

Slide 80: {A, B, C, D, E} 1 2 3 4 5 6 Ø A {A, D, B} {A, E, C, D} {2} {6} B {D, B} {A, D} {A, E, C} C FCA {2, 4} {2, 6} {5, 6} D {D} {A} {C} E {2, 4, 6} {2, 5, 6} {3, 5, 6} Ø {1, 2, 3, 4, 5, 6}

Slide 81: {A, B, C, D, E} 1 2 3 4 5 6 Ø A {A, D, B} {A, E, C, D} {2} {6} tity B s en A. st clas {D, B} FC ﬁrFCA g to{2, 4} {A, D} {A, E, C} y as appin C {2, 6} {5, 6} stor s m Hi ble D ena {D} {A} {C} E {2, 4, 6} {2, 5, 6} {3, 5, 6} Ø {1, 2, 3, 4, 5, 6}

Slide 82: How developers drive software evolution 6

Slide 83: CVS shows activity.

Slide 84: Who is responsible for this?

Slide 85: Who is responsible for this?

Slide 86: Alphabetical order is no order.

Slide 87: The Hausdorf metric can be used to compute the similarity between commits. A d(A, B) = ∑ min2{ | a - b | b in B } a in A B

Slide 88: Alphabetical order is no order.

Slide 89: Ownership Map reveals development patterns. Girba etal, 2006

Slide 90: Ownership Map reveals development patterns. Girba etal, 2006 tity anges. s en c ch clas listi rst ho as ﬁ out tory ing ab His son les rea enab

Slide 91: Measuring Yesterday’s Time-based history Weather Detection Strategies 1 2 3 4 5 6 Visualizing the evolution Detecting How developers of hierarchies co-change patterns drive evolution

Slide 92: System System History Version Inheritance Inheritance History Version Class Class History Version

Slide 93: History Version History Version History Version

Slide 94: History Version en tity. st c lass 200 5 as ﬁr Gir ba, his tory History Version mo dels Hi smo History Version

Slide 95: Modeling History to Understand Software Evolution Inauguraldissertation der Philosophisch-naturwissenschaftlichen Fakultät der Universität Bern vorgelegt von Tudor Gîrba von Rumänien Leiter der Arbeit: Prof. Dr. Stéphane Ducasse Prof. Dr. Oscar Nierstrasz Institut für Informatik und angewandte Mathematik www.tudorgirba.com

Slide 96: Tudor Gîrba www.tudorgirba.com creativecommons.org/licenses/by/3.0/

Slideshare.net (beta)