Meta Object Protocols


Published on

Published in: Technology, Education
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Meta Object Protocols

  1. 1. Metaprogramming, Metaclasses&The Metaobject ProtocolRaymond Pierre de Lacazerpl@lispnyc.orgLispNYC, June 11th 2013
  2. 2. Overview• This talk presents material from The Art of the Metaobject Protocolby Gregor Kiczles, Jim de Rivieres, Daniel G. Bobrow, MIT Press, 1991• Metaprogramming– Programming with macros– Metaprogramming in other languages• Metaclasses– objects, metaobjects and metaclasses– CLOS in a nutshell– Use case of a metaclass: AllegroCache• Metaobject Protocol– motivations underlying a MOP– Part of the implementation of Closette– Designing and Implementing a metaobject protocol
  3. 3. Metaprogramming (Wikipedia)• Metaprogramming is the writing of computerprograms that write or manipulate otherprograms, or themselves, as their data.• The language in which the metaprogram iswritten is called the metalanguage• The ability of a programming language to be itsown metalanguage is called reflextion orreflexivity.• Having the programming language itself as a first-class data type is known as homoicomicity.– Lisp, Forth, Rebol• Homoiconicity: Code = Data
  4. 4. Homoiconicity Examplecode.lisp: (defun add (x y)(+ x y))USER(12): (setf code (with-open-file (file "C:code.lisp") (read file)))(defun add (x y)(+ x y))CL-USER(13): (type-of code)CONSCL-USER(14): (first code)DEFUNCL-USER(15): (fourth code)(+ X Y)CL-USER(16): (setf (second code) subtract)SUBTRACTCL-USER(19): (setf (fourth code) (- x y))(- X Y)CL-USER(20): code(DEFUN SUBTRACT (X Y) (- X Y))CL-USER(21): (eval code)SUBTRACTCL-USER(22): (subtract 5 6)-1
  5. 5. Approaches to Metaprogramming(Wikipedia)• Statically typed functional languages– Usage of dependent types allows proving that generated code isnever invalid.• Template metaprogramming– C "X Macros"– C++ Templates• Staged meta-programming– MetaML– MetaOCaml• Macro systems– Lisp hygienic macros– MacroML– Template Haskell
  6. 6. Part I: LISP MACROS
  7. 7. Why Macros?Say we want to add a new language feature:(unless* <test> <expr>);; Using a function we could do:> (defun unless* (test expr)(if test nil expr))UNLESS*> (unless* nil (print "This should print."))This should print.nil> (unless* t (print "This should not print"))This should not printnilUh oh! The function unless* evaluates <expr> regardless of <test>!
  8. 8. Macros in a Nutshell• Macros do not evaluate their arguments• Expanded at compile-time into anotherprogrammatic form (AST transform) as specifiedby the definition of the macro• It is this resulting programmatic form that isactually compiled by the compiler.• In CL can use back-quote, comma & comma-at asconvenient list manipulators to write macros• In Clojure can use syntax-quote, tilde & tilde-atconvenient list manipulators to write macros
  9. 9. My First Macro> (defmacro unless* (test expr)(list if test nil expr))UNLESS*;; This works as before> (unless* nil (println "This should print."))This should print.nil;; This now behaves correctly> (unless* t (println "This should not print"))nil;; You can use backquote and comma instead of explicit list constructors> (defmacro unless* (test expr)`(if ,test nil ,expr))UNLESS*
  10. 10. Debugging Macros;; This is not correct> (defmacro INVOKE-OP (op arg1 arg2 &rest args)`(funcall ,op ,arg1 ,arg2 ,args))INVOKE-OP> (macroexpand (invoke-op * 3 4 5 6))(funcall * 3 4 (5 6));; This is correct> (defmacro INVOKE-OP (op arg1 arg2 &rest args)`(funcall ,op ,arg1 ,arg2 ,@args))> (macroexpand (invoke-op * 3 4 5 6))(funcall * 3 4 5 6))
  11. 11. Side Effects & Variable Capture in Macros• When you define a macro, it is unsafe to evaluate the argumentsmore than once in the event that they have side effects.• Lisp provides gensym to allow you to define local variables in safeway and also avoid unintentionally repeating side-effects;; This is bad: <expr> is evaluated twice!> (defmacro unless* (test expr)`(let ()(format t “~%Expr: ~a” ,expr)(if ,test nil ,expr)))UNLESS*> (unless* nil (print "foo"))"foo"Expr: foo"foo""foo";; This is good: Use gensym to avoid variable;; capture and avoid repeating side-effecting forms>(defmacro unless* (test expr)(let ((result (gensym)))`(let ((,result ,expr))(format t “~%Expr: ~a” ,result)(if ,test nil ,result))UNLESS*> (unless* nil (print "foo"))"foo"Expr: foo"foo"
  12. 12. Recursive Macros> (defmacro LISP ()`(,(lisp) In Summer Projects))LISP> (lisp)Error: Stack overflow (signal 1000)[condition type: SYNCHRONOUS-OPERATING-SYSTEM-SIGNAL]>(defmacro LISP (n)`(if (> ,n 0) `(,(lisp (1- ,n)) In Summer Projects) LISP)> (lisp 0)LISP> (lisp 1)(LISP IN SUMMER PROJECTS)> (lisp 2)((LISP IN SUMMER PROJECTS) IN SUMMER PROJECTS)> (lisp 3)(((LISP IN SUMMER PROJECTS) IN SUMMER PROJECTS) IN SUMMER PROJECTS)
  13. 13. Macros Summary• Don’t use macros if you can achieve the samething with a simple function• Macros are useful for implementing DSLs oradding new language features• Use macroexpand to debug macros• Leverage backquote, comma, comma-at formacro readability.• Use gensym to write hygienic macros• Paul Graham’s “On Lisp” is the macro book
  14. 14. Part II: Metaclasses
  15. 15. What is a Metaclass?• In an object orient system the term object refersthe domain entities that are being modeled, e.g.a bank-account object.• Programmers have access to these objects• A metaobject refers to the objects the languageuses to model/implement your domain objects ,e.g. a bank-account-class object• Programmers do not have access to thesemetaobjects (typically)• Only Language Implementers have access tothese metaobjects (typically)• A metaclass is the class of a metaobject
  16. 16. Metaobject & Metaclass Example;; In CLOS define a bank account class> (defclass bank-account () ((account-number)(owner)(balance)))#<STANDARD-CLASS BANK-ACCOUNT>;; Create a bank account object> (make-instance bank-account)#<BANK-ACCOUNT {10029ED253}>;; The class of the bank account object is a metaobject> (class-of *)#<STANDARD-CLASS BANK-ACCOUNT>;; The class of this class metaobject is a metaclass> (class-of *)#<STANDARD-CLASS STANDARD-CLASS>
  17. 17. Basic CLOS: defclass> (defclass bank-account ()((account-number :initarg :account-number:accessor bank-account-number)(account-balance :initarg :account-balance:accessor bank-account-balance:initform 0)))#<STANDARD-CLASS BANK-ACCOUNT>> (make-instance bank-account :account-number 123456789 :account-balance 500)#<BANK-ACCOUNT {1002E20513}>> (describe *)#<BANK-ACCOUNT {1002E20513}>[standard-object]Slots with :INSTANCE allocation:ACCOUNT-NUMBER = 123456789ACCOUNT-BALANCE = 500> (bank-account-balance **)500
  18. 18. Basic CLOS: defgeneric & defmethod> (defgeneric PLUS (arg1 arg2))#<STANDARD-GENERIC-FUNCTION PLUS (0)>> (defmethod PLUS ((arg1 NUMBER)(arg2 NUMBER))(+ arg1 arg2))#<STANDARD-METHOD PLUS (NUMBER NUMBER) {100313BFF3}>> (defmethod PLUS ((arg1 STRING)(arg2 STRING))(concatenate string arg1 arg2))#<STANDARD-METHOD PLUS (STRING STRING) {100319C953}>> (plus 1 2)3> (plus "a" ”b”)"ab”
  19. 19. AllegroCache: Object Persistence in Lisp• AllegroCache – A high-performance, dynamicobject caching database system.• Implements full transaction model with long andshort transactions, and meets the classic ACIDcompliancy and maintains referential integrity.• 64-bit real-time data caching•
  20. 20. The Persistent-Class Metaclass> (defclass POINT ()((name :reader point-name :index :any-unique)(x :initarg :x :accessor point-x :index :any)(y :initarg :y :accessor point-x :index :any)(rho :accessor point-rho :allocation :instance)(theta :accessor point-theta :allocation :instance))(:metaclass<DB.ALLEGROCACHE:PERSISTENT-CLASS POINT>> (setf p1 (make-instance point :x 20 :y 30))Error: Attempt to do a database operation without an open database> ( "C:ProjectstrunkLanguagesMOPDB“:if-does-not-exist :create)#<AllegroCache db "C:ProjectstrunkLanguagesMOPDB" @ #x218b8942>> (setf p1 (make-instance point :x 20 :y 30))#<POINT oid: 12, ver 5, trans: NIL, modified @ #x2197943a>
  21. 21. Instances of a Specialized Metaclass> (describe p1)#<POINT oid: 14, ver 7, trans: NIL, modified @ #x21ad2aba> is aninstance of #<DB.ALLEGROCACHE:PERSISTENT-CLASS POINT>:The following slots have :INSTANCE allocation:OID 14SLOTS #((:UNBOUND) (:UNBOUND) (:UNBOUND))NEW-SLOTS #(POINT-1 20 30)DCLASS #<DB.ALLEGROCACHE::FILE-DASH-CLASS @ #x219796a2>VERSION 7PREV-VERSION NILTRANS NILMODIFIED :ENDRHO <unbound>THETA <unbound>The following slots have nonstandard allocation as shown:NAME :PERSISTENT POINT-1X :PERSISTENT 20Y :PERSISTENT 30
  22. 22. Metaclass• Metaclasses allow you to extend existinglanguage constructs. E.g. defclass• This can be more natural than a DSL or aseparate API outside the language• Allow you to safely change or extend thebehavior of the language• Metaclasses are a very powerful underutilizedprogramming paradigm. (IMHO)
  23. 23. Part III: The Metaobject Protocol
  24. 24. Protocol Definitions• An object protocol is a collection of methodsthat operate on some collection of objects.• A metaobject protocol is a collection ofmethods that operate on some collection ofmetaobjects.• A metaclass refers to the class of a metaobject
  25. 25. Implementing defclass;;; All the code on the next few slides is taken from the AMOP book.(defmacro DEFCLASS (name direct-superclasses direct-slots &rest options)`(ensure-class ,name:direct-superclasses ,(canonicalize-direct-superclasses direct-superclasses):direct-slots ,(canonicalize-direct-slots direct-slots),@(canonicalize-defclass-options options)(defun ENSURE-CLASS (name &rest all-keys)(if (find-class name nil)(error "Cant redefine the class named ~S in Closette." name)(let ((class (apply #make-instance standard-class :name name all-keys)))(setf (find-class name) class)class)))(let ((class-table (make-hash-table :test #eq :size 20)))(defun (SETF FIND-CLASS) (new-value symbol)(setf (gethash symbol class-table) new-value)))
  26. 26. Implementing defclass (cont.)(defclass STANDARD-CLASS ()((name :initarg :name:accessor class-name)(direct-superclasses :initarg :direct-superclasses:accessor class-direct-superclasses)(direct-slots :initarg :direct-slots:accessor class-direct-slots)(class-precedence-list :initarg :class-precedence-list:accessor class-precedence-list)(effective-slots :accessor class-slots)(direct-subclasses :initform ():accessor class-direct-subclasses)(direct-methods :initform ():accessor class-direct-methods)))
  27. 27. Implementing defclass (cont.)(defmethod INITIALIZE-INSTANCE :after ((class STANDARD-CLASS)&key direct-superclasses direct-slots)(let ((supers (or direct-superclasses `(,(find-class standard-object)))))(setf (class-direct-superclasses class) supers)(dolist (superclass supers)(push class (class-direct-subclasses superclass))))(let ((slots (mapcar #(lambda (slot-properties)(apply #make-direct-slot-definition slot-properties))direct-slots)))(setf (class-direct-slots class) slots)(dolist (direct-slot slots)(dolist (reader (slot-definition-readers direct-slot))(add-reader-method class reader (slot-definition-name direct-slot))))(dolist (direct-slot slots)(dolist (writer (slot-definition-writers direct-slot))(add-writer-method class writer (slot-definition-name direct-slot)))))(finalize-inheritance))
  28. 28. Implementing defclass (cont.)(defun FINALIZE-INHERITANCE (class)(setf (class-precedence-list class)(compute-class-precedence-list))(setf (class-slots class) (compute-slots class)))(defun COMPUTE-CLASS-PRECEDENCE-LIST (class)(let ((classes-to-order (collect-superclasses* class)))(topological-sort classes-to-order(remove-duplicates(mapappend #local-precedence-ordering classes-to-order))#std-tie-breaker-rule)))(defun COLLECT-SUPERCLASSES* (class)(remove-duplicates(cons class(mapappend #collect-superclasses* (class-direct-superclasses class)))))
  29. 29. Metaobject Protocol Definition• A metaobject protocol is an API that allows you toaccess and manipulate metaobjects.• “A metaobject protocol is an interface to elements of alanguage that are normally hidden from users of thelanguage. Providing such an interface allows users ofthe language to tailor the language to their own needsand in such a way provides a more powerful andflexible language. The CLOS metaobject protocolprovides an interface to program elements such asclasses and methods, thereby allowing users to controlaspects of the language such as how instances of aclass are created or how method dispatching works”.Raymond de Lacaze, JLUGM, 2000
  30. 30. Introspective vs. Intercessory• AMOP has both introspective and intercessoryaspects.• Introspective– Allows programmers to see the on-backstage.Metaobjects can be created, retrieved and examined.• Intercessory– Allows programmers to manipulate the on-backstagethereby affecting the on-stage behavior
  31. 31. Example MOP Usage;; Usage Example 1: An instance-counting metaclass;; Provide a slot in the metaclass to track number of instances(defclass COUNTED-CLASS (STANDARD-CLASS)((counter :initform 0)));; Then add an :after method on make-instance that bumps count(defmethod MAKE-INSTANCE :after ((class COUNTED-CLASS) &key)(incf (slot-value class) ‘counter))
  32. 32. Example MOP Usage (cont.)• Need to define a class with a metaclass other thanstandard-class.• We can’t use defclass, because as previously written italways uses standard-class as the metaclass(setf (find-class ‘counted-rectangle)(make-instance ‘counted-class:name ‘counted-rectangle:direct-superclasses (list (find-class ‘rectangle)):direct-slots nil))
  33. 33. Extending defclass Implementation;;; We would like to simply write the following(defclass COUNTED-RECTANGLE (rectangle)()(:meta-class counted-class));; Simply need to extend the definition of ensure-class(defun ENSURE-CLASS (name &rest all-keys&key (metaclass (find-class ‘standard-class)))(if (find-class name nil)(error "Cant redefine the class named ~S in Closette." name)(let ((class (apply #make-instance metaclass :name name all-keys)))(setf (find-class name) class)class)))
  34. 34. Understanding Method Combination> (defclass a ()())#<STANDARD-CLASS A>> (defclass b (a)())#<STANDARD-CLASS B>> (defclass c (a b)())#<STANDARD-CLASS C>a bc• Add primary methods on a, b, c• Add :before method on a, b, c• Add :after method on a, b, c• Add :around methods on a, b, c
  35. 35. Adding All Possible Methods(defmethod FOO ((obj A))(format t "~%Primary method on class A")(call-next-method))(defmethod FOO ((obj B))(format t "~%Primary method on class B"))(defmethod FOO ((obj C))(format t "~%Primary method on class C")(call-next-method))(defmethod FOO :before ((obj A))(format t "~%Before method on class A"))(defmethod FOO :before ((obj B))(format t "~%Before method on class B"))(defmethod FOO :before ((obj C))(format t "~%Before method on class C"))(defmethod FOO :after ((obj A))(format t "~%After method on class A"))(defmethod FOO :after ((obj B))(format t "~%After method on class B"))(defmethod FOO :after ((obj C))(format t "~%After method on class C"))(defmethod FOO :around ((obj A))(format t "~%Around method (before) on class A")(call-next-method)(format t "~%Around method (after) on class A"))(defmethod FOO :around ((obj B))(format t "~%Around method (before) on class B")(call-next-method)(format t "~%Around method (after) on class B"))(defmethod FOO :around ((obj C))(format t "~%Around method (before) on class C")(call-next-method)(format t "~%Around method (after) on class C"))> (setf x (make-instance c))#<C @ #x209681b2>
  36. 36. CLOS Method CombinationCL-USER(45): (foo x)Around method (before) on class CAround method (before) on class AAround method (before) on class BBefore method on class CBefore method on class ABefore method on class BPrimary method on class CPrimary method on class APrimary method on class BAfter method on class BAfter method on class AAfter method on class CAround method (after) on class BAround method (after) on class AAround method (after) on class CNIL
  37. 37. Changing Method Combination(defclass a ()())(defclass b ()())(defclass c ()())(defclass s (a b)())(defclass r (a c)())(defclass q (r s)())• Different Lisp object systems used different method combination• How can we reuse code written in Flavors, in CLOS?• We would like to be able to change method combination orderFlavors: (q s a b r c standard-object t)Loops: (q s b r a c standard-object t)CLOS: (q s r a b c standard-object t)a b cs rq
  38. 38. Compute-Class-Precedence-List• Do not allow programmers to directly set class-precedence-list slot of the class metaobject but.• Allow them to control the computation that is used when that slot I set• Make compute-class-precedence-list a GF and specialize the original function on standard-class• Require that the class be the first in the list and that standard-object and t be the last two.(defclass flavors-class (standard-class) ())(defmethod compute-class-precedence ((class flavors-class))(append (remove-duplicates (depth-first-preorder-superclasses* class):from-end t)(list (find-class ‘standard-object)(find-class ‘t))))
  39. 39. Additional AMOP Examples• Adding slot attributes– compute-slots (GF)– compute-effective-slots (GF)• Adding default-initargs– ensure-class– make-instance– finalize-inheritance (GF)• Instance Allocation– allocate-instance (GF)Note: In the examples we’ve seen so far we’ve had to rewrite ensure-class twice
  40. 40. Designing a MOP• Design revolves around CLOS metaobjects– Class metaobjects– Method metaobject– Generic function metaobjects– Slot definition metaobjects– Specializer metaobjects– Method combination objects• AMOP is designed to provide an introspective andintercessory API for manipulating andcustomizing these metaobjects
  41. 41. Designing a MOP• Design: MOP should provide mechanisms to extend the syntax andmanipulate the behavior of CLOS without:– Needing to rewrite the programmer user interface macros: defclass,defgeneric and defmethod– Needing to rewrite the functions that implement these macros, namely theensure-x functions and the compute-x functions• Implementation– This is mostly accomplished by making the compute-X functions (on thevarious metaobjects) generic functions, thus using CLOS to implement theimplementation of CLOS.– This approach also provides a level of safety because users of the MOP cannotdirectly set the values of the slots of the metaobjects, they can only controltheir computation which is only performed once and cannot subsequentlychanged.
  42. 42. The Final defclass(defmacro DEFCLASS (name direct-superclasses direct-slots &rest options)(let* ((metaclass-option (find ‘:metaclass options :key #’car))(metaclass-name (or (second metaclass-option) ‘standard-class))(sample-class-metaobject(allocate-instance (find-class metaclass-name)))(canonical-supers(canonicalize-direct-superclasses direct-superclasses))(canonical-slots(canonicalize-direct-slots direct-slots))(canonical-options(canonicalize-defclass-options sample-class-metaobject(remove metaclass-option options))))`(ensure-class ‘,name:direct-superclasses ,canonical-supers: direct-slots ,canonical-slots:metaclass (find-class ‘,metaclass),@canonical-options)))
  43. 43. CLOS Metaclass Hierarchy
  44. 44. When to use a MOP• Any time you need to alter how a metaobject behaves.For instance, instead of doing a method lookup on alocal table, perform an RPC call.• Memoizing method calls• Complex inheritance (this may be of dubious value)• Loosening or strengthening a type system• Implementing Dynamic Dispatch (if necessary)• Persistence• ReplicationThis list is from:
  45. 45. Additional Links••••••••••
  46. 46. Additional Definitions• Dependent Types (Wikipedia)In computer science and logic, a dependent type is a type that depends on a value. Dependent typesplay a central role in intuitionistic type theory and in the design of functional programming languageslike ATS, Agda and Epigram and IdrisAn example is the type of n-tuples of real numbers. This is a dependent type because the type dependson the value n.• Staged meta-programming (Wikipedia)Incremental compiling of new machine code during runtime. Under certain circumstances, significantspeedups are possible using multi-stage programming, because more detailed information about thedata to process is available at runtime than at the regular compile time, so the incremental compiler canoptimize away many cases of condition checking etc.