The Next Generation MOP, Jochen Theodorou, GR8Conf 2013


Published on

Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

The Next Generation MOP, Jochen Theodorou, GR8Conf 2013

  1. 1. The next generation MOPJochen "blackdrag" Theodorou
  2. 2. About blackdrag● Working on Groovy Core since about 2005● Almost as long as that, Tech Lead of Groovy● Currently Employed at Pivotal● Responsible for most of the technical side ofGroovyEmail:
  3. 3. About a new MOP● discussions since 2005● good-for-all solution always missing● some ideas open to discussion● and some cleanup duties● we insist on you helping us
  4. 4. Why change
  5. 5. Why change● has many inconsistencies● makes many optimizations impossible● in theory powerful● have to know implementation details● Impossible to extend and spec resistent● the API just plain sucks
  6. 6. Some Basics and History
  7. 7. The Meta Class● Each class has a meta class● Saves all the dynamic and staticproperties/methods for Groovy● Internal control flow involves exceptions● Designed for invoking methods throughthe meta class
  8. 8. The Meta Class● early Groovy meta programming:● MOP methods invokeMethod and get/setProperty● A custom meta class (foo.metaClass = x)● later class based automated meta class lookup● transformed into the meta class creation handle(used by ExpandoMetaClass#enableGlobally())
  9. 9. The Meta Class● Basic principle: Each instance has a meta class● More specified: Only every GroovyObject instance(later we changed that with a global map)● Global registry specifyinginitial meta class on first use
  10. 10. The Meta Class// myMetaClass some custom metaclass// meta class in registry differentdef x1 = new X()assert x1.metaClass != myMetaClassx1.metaClass = myMetaClassassert x1.metaClass == myMetaClassdef x2 = new X()assert x2.metaClass != x1.metaClassX.metaClass = myMetaClassdef phantom = new X()def x3 = new X()assert x3.metaClass == x1.metaClassassert x3.metaClass != x2.metaClass
  11. 11. The Meta ClassX.metaClass = x2.metaClassassert phantom.metaClass == ???
  12. 12. Adding Methods/Properties● Standard meta class: MetaClassImpl● does not support modifications● New meta class for this: ExpandoMetaClass● enabled with ExpandoMetaClass.enableGlobally()● not always equally behaving to MetaClassImpl
  13. 13. More MetaClasses● ProxyMetaClass (intercepting, decorating)● MixinMetaClass (mixins)● ClosureMetaClass (GeneratedClosure)● DelegatingMetaClass (base class)● OwnedMetaClass (related to mixins)● HandleMetaClass (related to mixins)Plus your own custom meta class
  14. 14. DSL not = 1 //defines = {1} //defines methodto use a closure as = = {1}● only for the instance● get metaproperty and set initial value creator
  15. 15. Overriding Super Methodsclass A {def doIt(){two() + done.}def two(){two}}class B extends A {}B.metaClass.two = {my new two!}def b = new B()assert b.two() == my new two!assert b.doIt() == two done.
  16. 16. Overriding Super MethodsTo make it work:class A implements GroovyInterceptable {def doIt(){two() + done.}def two(){two}}class B extends A {}B.metaClass.two = {my new two!}def b = new B()assert b.two() == my new two!assert b.doIt() == my new two! done.
  17. 17. Adding Super Methodsclass A {def doIt(){two() + done.}def methodMissing(String name, args){two}}class B extends A {}def b = new B()assert b.two() == twoassert b.doIt() == two done.A.metaClass.two = {my new two!}assert b.two() == my new two!assert b.doIt() == my new two! done.
  18. 18. Super Methods OverloadDoes not:class A {def doIt(x){two(x) + done.}def two(x) {two}}class B extends A {}def b = new B()assert b.two(1) == twoassert b.doIt(1) == two done.A.metaClass.two = {String s->my new two!}assert b.two(1) == my new two!assert b.doIt(1) == my new two! done.
  19. 19. Private Multi Methodsclass A {def doIt(){two() + done.}}class B extends A {private two(){1}}def b = new B()assert b.two() == 1assert b.doIt() == 1 done.
  20. 20. Speaking of privateclass A {private foo=1def b={foo}}class B extends A {}def b = new B()assert b.b() == 1 //fails● Information loss thorugh Closure#getProperty
  21. 21. get/setMetaClass● persistency framework needs to be aware● transient works for Serialization● what about other frameworks?● seamless integration anyone?
  22. 22. Propertiesclass X extendsorg.xml.sax.helpers.XMLFilterImpl {def foo}● XMLFilterImpl has a get/setProperty● cannot do new X().foo = bar● cannot do println new X().foo
  23. 23. invokeMethodNo such conflict known.... but!● dynamic entry point from Java● as methodMissing● with GroovyInterceptable (EMC too) as upfrontmethodconflicting concepts
  24. 24. What to make better?
  25. 25. … besides fixing those problems
  26. 26. Optimization effortsLesson:Java7 with invokedynamic is much better suited forGroovys dynamic method callsReaction:make Java7 the default (backport); rewriteDefaultGroovyMethods to use indy; throw out a lot ofold code
  27. 27. Optimization effortsLesson:Hotspot is not happy about invoking target methods inthe meta class (mega morphic call sites)Reaction:The meta class only gives back something you can calland does not do the call itself.
  28. 28. Optimization effortsLesson:Synchronization, Locks, volatiles usages on eachmethod call destroy multithread performance as wellas hotspot optimizations. Most applications set up mcchanges on startup.Reaction:metaclass goes immutable; adding methods createsnew meta class; lazy thread update (user levelsynchronization required)
  29. 29. Hot SwappingLesson:Keeping internal state in the class is bad (seetimestamp_xxx, timestamp, $callSiteArray)Reaction:Removal. CallSiteArray not needed anymore, thetimestamps are kept track off by the loader, not in theclass
  30. 30. Optimization effortsLesson:Garbage collecting and recreating meta classes is verybad.Reaction:Keep the base meta class around and reuse everythingpossible to make new meta classes as lean as possible
  31. 31. API DesignLesson:Conventions are good, forcing them is bad(GroovyObject)Reaction:Dont implement GroovyObject by default anymore.
  32. 32. General DesignLesson:Too many ways of doing the same thing is no goodReaction:Most probably only methodMissing/propertyMissinganymore but easy way to „register“ a method to doinvokeMethod.
  33. 33. API DesignLesson:Having multiple, not equivalent entry points is bad.(MetaClassImpl has 3 for methods, multiusage ofinvokeMethod, information loss throughget/setProperty)Reaction:Clean up the API to have only one entry point(removal of MetaClass#invokeMethod)
  34. 34. Possibilities
  35. 35. Internal vs. ExternalInternal usage:class X {def methodMissing(String name, args) {1}}External usage:class X {}X.metaClass.methodMissing = {1}Combined:class X {static {this.metaClass.methodMissing = {1}}}
  36. 36. Dynamic Invoke from JavaBefore:GroovyObject foo = ...;String s = (String)foo.invokeMethod(“bar“, new Object[]{});After:Object foo = ...;String s = MopInterface.invoke(foo, “bar“);● helper class for dynamic interactions with Groovy● similiar for properties
  37. 37. Adding a method from JavaBefore:GroovyObject foo = ...;foo.getMetaClass().registerInstanceMethod(“foo“,new Closure(null){public Object doCall(){1}};After:Object foo = ...;MopInterface.addInstanceMethod(foo, “foo“,new T() {public Object myFancyMethod(){1}});● Doesnt have to be a Closure or MetaMethod● All declared methods used (lambdas too)
  38. 38. Limited Meta Class ChangesUse Case:unrolling all changes a GroovyShell or didBefore:● tracking meta class mutations impossible● „unmutate“ the metaclass certainly is● can only to track newly set meta classes
  39. 39. Limited Meta Class ChangesUse Case:I am a library developer and I dont want intrusivechanges from user code to my library classes,changing the way my library is calling methods.Before:If the change is not effecting the library, then mostprobably because of a bug.
  40. 40. Realms
  41. 41. RealmsA realm here defines how the class in the realm„sees“ the meta classes. Different classes can havedifferent realms.MyClass(y)MyOtherClass(x) metaClass(x)
  42. 42. RealmsMyClass(y)MyOtherClass(x) metaClass(x)from realm(MyClass)<Realm defined by MyClass>
  43. 43. RealmsMyClass(y)MyOtherClass(x) metaClass(y)from realm(MyOtherClass)<Realm defined by MyOtherClass>
  44. 44. RealmsINITDEFAULTrealm(A_n)realm(C_n)realm(A_1)........realm(B_1)realm(C_1)........realm(D_1)
  45. 45. RealmsINIT does not allow changing the meta classINITDEFAULTNon isolatedisolated
  46. 46. RealmsDefault gets all the unspecified changesINITDEFAULTNon isolatedisolated
  47. 47. RealmsIsolated realms dont get unspecific changesINITDEFAULTNon isolatedisolated
  48. 48. RealmsNon isolated realms can override meta classes indefault without impossing themINITDEFAULTNon isolatedisolated
  49. 49. Isolated Realm@Realm(marker=SomeMarkerClass,parentRealm=Realm.INIT)class MyLibraryClass {def foo(){bar()}def bar(){1}} = {2}assert new MyLibraryClass().foo() == 1
  50. 50. Non-Isolated Realm@Realm(marker=SomeMarkerClass)class MyLibraryClass {def foo(){bar()}def bar(){1}} = {2}assert new MyLibraryClass().foo() == 2
  51. 51. Testing Realmclass X {private foo(){1}}class MyTest extends GroovyTestCase {@Realm(marker=SomeMarkerClass,allowPrivate=true)void testPrivateWorking() {def x = new X()assert == 1}void testPrivateNotWorking() {def x = new X()shouldFail(MissingMethodException) {}}}
  52. 52. Limited Meta Class ChangesBefore:Change intrusive, visible to everyoneAfter:def foo(){1}assert foo() == 1realm.withSubRealm { = {2}assert foo() == 2}assert foo() == 1
  53. 53. Q/A?