Meta-Classes in Python
Upcoming SlideShare
Loading in...5
×
 

Meta-Classes in Python

on

  • 2,230 views

 

Statistics

Views

Total Views
2,230
Views on SlideShare
2,052
Embed Views
178

Actions

Likes
5
Downloads
60
Comments
0

3 Embeds 178

http://jujo00obo2o234ungd3t8qjfcjrs3o6k-a-sites-opensocial.googleusercontent.com 176
http://www.slideshare.net 1
http://192.168.6.52 1

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Meta-Classes in Python Meta-Classes in Python Presentation Transcript

  • Dynamic OOP and Meta-Classes in Python Guy Wiener Dept. of Applied Math and Computer Science Weizmann Institute for Science 24/5/2011Guy Wiener (WIS) Dynamic OOP and Meta-Classes in Python 24/5/2011 1 / 57
  • IntroductionOutline1 Introduction2 OOP Hacking3 Meta-Classes4 Examples5 Summary Guy Wiener (WIS) Dynamic OOP and Meta-Classes in Python 24/5/2011 2 / 57
  • IntroductionWhat is OOP?The OOP Manifest (abridged)Alan Kay, 1993 Everything is an object Computation is performed by objects communicating with each other by sending and receiving messages Each object has its own memory (state), which consists of other objects. Every object is an instance of a class. The class is the repository for behavior. All instances of the same class can perform the same actions. Classes are organized into a singly rooted tree, called the inheritance hierarchy. Guy Wiener (WIS) Dynamic OOP and Meta-Classes in Python 24/5/2011 3 / 57
  • IntroductionWhat is an object?An object, according to the OOP manifest: State Memory Behavior Receiving and sending messages Identity Each object is unique Guy Wiener (WIS) Dynamic OOP and Meta-Classes in Python 24/5/2011 4 / 57
  • IntroductionObjects Can Change!Changing objects State Read, write, add and remove field Behavior Create, add and replace methods Identity Remains the same Guy Wiener (WIS) Dynamic OOP and Meta-Classes in Python 24/5/2011 5 / 57
  • IntroductionStatic Objects and ClassesObjects and Classes Example Class A Field descriptors x: int Method desc. and code y: String Hierarchy information foo() {. . . } goo() {. . . } Object Run-time Fields Instance of A type infor- content 5 mation “hello” Guy Wiener (WIS) Dynamic OOP and Meta-Classes in Python 24/5/2011 6 / 57
  • IntroductionDynamic Objects and ClassesObjects and Classes Example Class A A dictionary “foo” ⇒ {. . . } of methods “goo” ⇒ {. . . } and static fields Instance of A Object “x” ⇒ 5 A dictionary of “y” ⇒ “hello” Class fields and bound pointer methods Guy Wiener (WIS) Dynamic OOP and Meta-Classes in Python 24/5/2011 7 / 57
  • OOP HackingOutline1 Introduction2 OOP Hacking3 Meta-Classes4 Examples5 Summary Guy Wiener (WIS) Dynamic OOP and Meta-Classes in Python 24/5/2011 8 / 57
  • OOP HackingEverything is a DictionaryPython’s modules, classes and objects = Dictionaries.The dictionary holds: Classes Methods and class variables Objects Bound methods and fields The “.” operator = Access the inner dictionary The dir command = Lists keys of the inner dictionary Guy Wiener (WIS) Dynamic OOP and Meta-Classes in Python 24/5/2011 9 / 57
  • OOP HackingManipulating the Inner DictionaryThe inner dictionary is accessible and manipulable by string keys!Inner dictionary manipulation getattr(o, ’a’) Get the attribute a from the dictionary of osetattr(o, ’a’, v) Set the attribute a of o to be v obj.foo is same as getattr(obj, ’foo’) obj.foo = val is same as setattr(obj, ’foo’, val) Guy Wiener (WIS) Dynamic OOP and Meta-Classes in Python 24/5/2011 10 / 57
  • OOP HackingManipulating the Inner Dictionary (cont’d)Example>>> a = A()>>> a.x = 5>>> dir(a)[..., ’x’]>>> getattr(a, ’x’)5>>> setattr(a, ’x’, 6)>>> a.x6 Guy Wiener (WIS) Dynamic OOP and Meta-Classes in Python 24/5/2011 11 / 57
  • OOP HackingAdding Fields Adding fields to an object class A: pass >>> a = A() >>> a.x = 5 >>> a.x 5 >>> setattr(a, ’y’, 6) >>> a.y 6 Guy Wiener (WIS) Dynamic OOP and Meta-Classes in Python 24/5/2011 12 / 57
  • OOP HackingAdding Fields Adding fields to an object Gain & Loss class A: pass Gained Flexible object content >>> a = A() Lost Type safety >>> a.x = 5 >>> a.x 5 >>> setattr(a, ’y’, 6) >>> a.y 6 Guy Wiener (WIS) Dynamic OOP and Meta-Classes in Python 24/5/2011 12 / 57
  • OOP HackingReplacing Methods Swapping methods class A: def f(self): print "f" def g(self): print "g" >>> a = A() >>> a.f, a.g = a.g, a.f >>> a.f() g >>> a.g() f Guy Wiener (WIS) Dynamic OOP and Meta-Classes in Python 24/5/2011 13 / 57
  • OOP HackingReplacing Methods Swapping methods Gain & Loss class A: Gained Change an object def f(self): behavior at runtime print "f" Lost Control flow analysis def g(self): print "g" >>> a = A() >>> a.f, a.g = a.g, a.f >>> a.f() g >>> a.g() f Guy Wiener (WIS) Dynamic OOP and Meta-Classes in Python 24/5/2011 13 / 57
  • OOP HackingReplacing Methods for a Class Swapping class methods class A: def f(self): print "f" def g(self): print "g" >>> a = A() >>> A.f, A.g = A.g, A.f >>> a.f() g >>> a.g() f Guy Wiener (WIS) Dynamic OOP and Meta-Classes in Python 24/5/2011 14 / 57
  • OOP HackingReplacing Methods for a Class Swapping class methods Gain & Loss class A: Gained Change a class at def f(self): runtime print "f" Lost Sanity def g(self): print "g" >>> a = A() >>> A.f, A.g = A.g, A.f >>> a.f() g >>> a.g() f Guy Wiener (WIS) Dynamic OOP and Meta-Classes in Python 24/5/2011 14 / 57
  • OOP HackingClasses are Return Values TooA function that returns a classdef PersonClass(salut): class Person(object): def __init__(self, name): self.name = name def greet(self): print salut, self.name return Person Guy Wiener (WIS) Dynamic OOP and Meta-Classes in Python 24/5/2011 15 / 57
  • OOP HackingClasses are Return Values Too (cont’d)Making classes OutputMan = PersonClass("Mr.") for c in a: c.greet()Woman = PersonClass("Ms.")Doctor = PersonClass("Dr.") >>> Ms. Rose Tylar Ms. Amy Ponda = [Woman("Rose Tylar"), Mr. Mickey Smith Woman("Amy Pond"), Mr. Rory Williams Man("Mickey Smith"), Dr. Who Man("Rory Williams"), Doctor("Who")] Guy Wiener (WIS) Dynamic OOP and Meta-Classes in Python 24/5/2011 16 / 57
  • Meta-ClassesOutline1 Introduction2 OOP Hacking3 Meta-Classes4 Examples5 Summary Guy Wiener (WIS) Dynamic OOP and Meta-Classes in Python 24/5/2011 17 / 57
  • Meta-ClassesWhat is Meta-Programming?Meta-ProgramA program that: One of its inputs is a program (possibly itself) Its output is a program Guy Wiener (WIS) Dynamic OOP and Meta-Classes in Python 24/5/2011 18 / 57
  • Meta-ClassesWhat is Meta-OOP?Meta-ClassA class that creates classes Objects are instances of classes Classes are objects Classes are instances of meta-classes Guy Wiener (WIS) Dynamic OOP and Meta-Classes in Python 24/5/2011 19 / 57
  • Meta-ClassesObject Systems3-levels 4-levels Object Object A Class Type A Class Type super: Type[] super: Type[] An Object An Object A MetaClassLegend instance of extends Guy Wiener (WIS) Dynamic OOP and Meta-Classes in Python 24/5/2011 20 / 57
  • Meta-Classes Meta-Classes in PythonOutline1 Introduction2 OOP Hacking3 Meta-Classes Meta-Classes in Python4 Examples Logging Counting Delegation Associations5 Summary Guy Wiener (WIS) Dynamic OOP and Meta-Classes in Python 24/5/2011 21 / 57
  • Meta-Classes Meta-Classes in PythonMeta-Classes in PythonThe type class Base class for meta-classes Creates new classes Dynamic members become static members of the instance classtype. init (cls, name, bases, dict) cls The class object itself (i.e., “self”)name The name of the classbases A list of the base classes dict A dictionary of the methods and static fields of the class Guy Wiener (WIS) Dynamic OOP and Meta-Classes in Python 24/5/2011 22 / 57
  • Meta-Classes Meta-Classes in PythonCreating Classes from Meta-ClassesDeclaring a new class programatically# Meta-Classclass Printable(type): def whoami(self): print "I am ", self.__name__>>> Foo = Printable(’Foo’,(),{}) # Empty new class>>> Foo.whoami()I am Foo>>> Foo.__class__<class ’Printable’>>>> f = Foo() # Object>>> f.__class__<class ’Foo’> Guy Wiener (WIS) Dynamic OOP and Meta-Classes in Python 24/5/2011 23 / 57
  • Meta-Classes Meta-Classes in PythonThe metaclass fieldClass C is an instance of a meta-class M if: 1 C has a static field metaclass 2 One of the ancestors classes of C is an instance of M 3 There is a global variable metaclass 4 Otherwise, the default meta-class type is used Guy Wiener (WIS) Dynamic OOP and Meta-Classes in Python 24/5/2011 24 / 57
  • Meta-Classes Meta-Classes in Python metaclass ExampleDeclaring a meta-class with metaclassclass Bar: __metaclass__ = Printable def foo(self): print ’foo’>>> Bar.whoami()I am a Bar>>> b = Bar()>>> b.foo()foo Guy Wiener (WIS) Dynamic OOP and Meta-Classes in Python 24/5/2011 25 / 57
  • ExamplesOutline1 Introduction2 OOP Hacking3 Meta-Classes4 Examples5 Summary Guy Wiener (WIS) Dynamic OOP and Meta-Classes in Python 24/5/2011 26 / 57
  • Examples LoggingOutline1 Introduction2 OOP Hacking3 Meta-Classes Meta-Classes in Python4 Examples Logging Counting Delegation Associations5 Summary Guy Wiener (WIS) Dynamic OOP and Meta-Classes in Python 24/5/2011 27 / 57
  • Examples LoggingLogging with Meta-ClassesA logger decoratordef log(name, f): def ret(*args): print "enter", name f(*args) print "exit", name return ret Guy Wiener (WIS) Dynamic OOP and Meta-Classes in Python 24/5/2011 28 / 57
  • Examples LoggingLogging with Meta-Classes (cont’d)A logger meta-classclass Logged(type): def init (cls, name, bases, dict): type. init (cls, name, bases, dict) p = re.compile(cls. logmatch ) for attr, item in dict.items(): if callable(item) and p.match(attr): setattr(cls, attr, log(attr, item)) Guy Wiener (WIS) Dynamic OOP and Meta-Classes in Python 24/5/2011 29 / 57
  • Examples LoggingLogging with Meta-Classes (cont’d)A logged classclass Test: metaclass = Logged logmatch = ’.*’ def foo(self): print ’foo’>>> t = Test()>>> t.foo()enter foofooexit foo Guy Wiener (WIS) Dynamic OOP and Meta-Classes in Python 24/5/2011 30 / 57
  • Examples CountingOutline1 Introduction2 OOP Hacking3 Meta-Classes Meta-Classes in Python4 Examples Logging Counting Delegation Associations5 Summary Guy Wiener (WIS) Dynamic OOP and Meta-Classes in Python 24/5/2011 31 / 57
  • Examples CountingCounting InstancesCountable meta-classclass Countable(type): def new_init(cls, init): def ret(self, *args): init(self, *args) cls.count = cls.count + 1 return ret def __init__(cls, name, bases, dct): type.__init__(cls, name, bases, dct) cls.count = 0 cls.__init__ = cls.new_init(cls.__init__) Guy Wiener (WIS) Dynamic OOP and Meta-Classes in Python 24/5/2011 32 / 57
  • Examples CountingCounting ExampleSome classesclass Person: __metaclass__ = Countable def __init__(self, name): self.name = nameclass Book: __metaclass__ = Countable def __init__(self, name, author): self.name = name self.author = author Guy Wiener (WIS) Dynamic OOP and Meta-Classes in Python 24/5/2011 33 / 57
  • Examples CountingCounting Example (cont’d)Testingrah = Person("Robert Anson Heinlin")b1 = Book("Stranger in a Strange Land", rah)b2 = Book("Starship Troopers", rah)print Person.count, Book.count>>> 1 2 Guy Wiener (WIS) Dynamic OOP and Meta-Classes in Python 24/5/2011 34 / 57
  • Examples DelegationOutline1 Introduction2 OOP Hacking3 Meta-Classes Meta-Classes in Python4 Examples Logging Counting Delegation Associations5 Summary Guy Wiener (WIS) Dynamic OOP and Meta-Classes in Python 24/5/2011 35 / 57
  • Examples DelegationAutomatic DelegationDelegate An object of class A that dispatches all message of class B to a target field of type B. Writing a delegate class is a monotonous work But it can be done automatically Guy Wiener (WIS) Dynamic OOP and Meta-Classes in Python 24/5/2011 36 / 57
  • Examples DelegationDelegation Using Meta-ClassesA delegation function decoratordef dlgt(cls, method): def ret(self, *args): method(self.__tgt__, *args) return instancemethod(ret, None, cls)instancemethod takes a function, an object/None and a class andreturns a method of the classAuxiliary – Print class namedef clsname(self): return self.__class__.__name__ Guy Wiener (WIS) Dynamic OOP and Meta-Classes in Python 24/5/2011 37 / 57
  • Examples DelegationDelegation Using Meta-Classes (cont’d)The Delegate meta-classclass Delegate(type): def __init__(cls, name, bases, dict): type.__init__(cls, name, bases, dict) tgtclass = cls.__tgtclass__ for name in dir(tgtclass): val = getattr(tgtclass, name) if callable(val): setattr(cls, name, dlgt(cls, val)) Guy Wiener (WIS) Dynamic OOP and Meta-Classes in Python 24/5/2011 38 / 57
  • Examples DelegationDelegation Using Meta-Classes (cont’d)The delegated classclass A: def bar(self): print clsname(self), ’bar’ def baz(self): print clsname(self), ’baz’ Guy Wiener (WIS) Dynamic OOP and Meta-Classes in Python 24/5/2011 39 / 57
  • Examples DelegationDelegation Using Meta-Classes (cont’d)The delegating classclass B: metaclass = Delegate tgtclass = A def init (self, tgt): self. tgt = tgt def boo(self): print clsname(self), ’boo’ Guy Wiener (WIS) Dynamic OOP and Meta-Classes in Python 24/5/2011 40 / 57
  • Examples DelegationDelegation Using Meta-Classes (cont’d)Delegation test>>> b = B(A())>>> b.bar()A bar>>> b.baz()A baz>>> b.boo()B boo Guy Wiener (WIS) Dynamic OOP and Meta-Classes in Python 24/5/2011 41 / 57
  • Examples DelegationSimplified Syntax for with Meta-Classes with Args.Thanks to Yoav GoldbergUse a function to create the meta-classdef delegate_of(tgtclass): class Delegate(type): def __init__(cls, name, bases, dict): type.__init__(cls, name, bases, dict) for name in dir(tgtclass): val = getattr(tgtclass, name) if callable(val): setattr(cls, name, dlgt(cls,val)) Guy Wiener (WIS) Dynamic OOP and Meta-Classes in Python 24/5/2011 42 / 57
  • Examples DelegationSimplified Syntax for Meta-Classes with Args.Thanks to Yoav GoldbergUse the function to generate a meta-classclass B: metaclass = delegate_of(A) def init (self, tgt): self. tgt = tgt def boo(self): print clsname(self), ’boo’ Guy Wiener (WIS) Dynamic OOP and Meta-Classes in Python 24/5/2011 43 / 57
  • Examples AssociationsOutline1 Introduction2 OOP Hacking3 Meta-Classes Meta-Classes in Python4 Examples Logging Counting Delegation Associations5 Summary Guy Wiener (WIS) Dynamic OOP and Meta-Classes in Python 24/5/2011 44 / 57
  • Examples AssociationsAssociations 0..m ab 0..n A myA myB BGoals Aggregation operations, like a.getMyB().foreach().g() r = b.getMyA().filter().f().map().h() Optimizations: Multithreading, caching No textual code generation Guy Wiener (WIS) Dynamic OOP and Meta-Classes in Python 24/5/2011 45 / 57
  • Examples AssociationsBasic AggregationReplace all public methods with an aggregator wrapperclass Aggr(type): def __init__(cls, name, bases, dct): p = re.compile(’__.*__’) type.__init__(cls, name, bases, {}) for key, item in dct.iteritems(): if callable(item) and not p.match(key): m = instancemethod(cls.wrap(item), None, self) setattr(self, key, m) def wrap(cls, func): def ret(self, *args): return self.__aggr__(func, *args) return ret Guy Wiener (WIS) Dynamic OOP and Meta-Classes in Python 24/5/2011 46 / 57
  • Examples AssociationsBasic Aggregation (cont’d)Make aggrs. that extend impl and wrap ifceclass ListAggr(Aggr): @staticmethod def make(name, impl, ifce): dct = {} for key in dir(ifce): dct[key] = getattr(ifce, key) return ListAggr(name, (impl,), dct) def __init__(self, name, bases, dct): Aggr.__init__(self, name, bases, dct) base = bases[0] ctr = base.__init__ % Use impl’s constructor self.__init__ = instancemethod(ctr, None, self) Guy Wiener (WIS) Dynamic OOP and Meta-Classes in Python 24/5/2011 47 / 57
  • Examples AssociationsList-Based Sequential Aggrs.Store list in private fieldclass ListFunc(object): def __init__(self, data): self.__data__ = data def onAdd(self, other): pass def onRemove(self, other): pass Guy Wiener (WIS) Dynamic OOP and Meta-Classes in Python 24/5/2011 48 / 57
  • Examples AssociationsList-Based Sequential Aggrs. (cont’d)Sequential Foreachclass ListSeqForeach(ListFunc): def __init__(self, data): ListFunc.__init__(self, data) def __aggr__(self, func, *args): for inst in self.__data__: func(inst, *args) Guy Wiener (WIS) Dynamic OOP and Meta-Classes in Python 24/5/2011 49 / 57
  • Examples AssociationsList-Based Sequential Aggrs. (cont’d)Sequential Mapclass ListSeqMap(ListFunc): def __init__(self, data): ListFunc.__init__(self, data) def __aggr__(self, func, *args): ret = [] for inst in self.__data__: ret.append(func(inst, *args)) return ret Guy Wiener (WIS) Dynamic OOP and Meta-Classes in Python 24/5/2011 50 / 57
  • Examples AssociationsBinding It TogetherMaking an aggr. that extends seq. foreachdef makeForeachClass(cls): return ListAggr.make(’...’, ListSeqForeach, cls) ListFunc Aggr ListSeqForeach ListAggr cls ForeachClass Guy Wiener (WIS) Dynamic OOP and Meta-Classes in Python 24/5/2011 51 / 57
  • Examples AssociationsOptimizationsMultithreaded Foreachclass ListSpawnForeach(ListFunc): def __init__(self, data): ListFunc.__init__(self, data) def __aggr__(self, func, *args): n = max(len(self.__data__)/K, 1) ts = [] for c in chunks(self.__data__, n): t = ForeachThread(c, func, args) ts.append(t) t.start() for tr in ts: tr.join() Guy Wiener (WIS) Dynamic OOP and Meta-Classes in Python 24/5/2011 52 / 57
  • Examples AssociationsBinding It Together, Multithreaded VersionMaking an aggr. that extends parallel foreachdef makeForeachClass(cls): return ListAggr.make(’...’, ListSpawnForeach, cls) ListFunc Aggr ListSpawnForeach ListAggr cls ForeachClass Guy Wiener (WIS) Dynamic OOP and Meta-Classes in Python 24/5/2011 53 / 57
  • Examples AssociationsLinksDownload complete codehttp://www.cs.bgu.ac.il/~gwiener/software/associations/Read full paperProceedings of ENASE’2010 / By request Guy Wiener (WIS) Dynamic OOP and Meta-Classes in Python 24/5/2011 54 / 57
  • SummaryOutline1 Introduction2 OOP Hacking3 Meta-Classes4 Examples5 Summary Guy Wiener (WIS) Dynamic OOP and Meta-Classes in Python 24/5/2011 55 / 57
  • SummaryAdvantages of Meta-ClassesTurn this. . . Into this P haarazu-Urh os a cr -Y eku Uru-Ne tah i Ia Dai-Uk -W ka a- Ha kal -U Tru-Saka Guy Wiener (WIS) Dynamic OOP and Meta-Classes in Python 24/5/2011 56 / 57
  • Summary Thank You! wiener.guy@gmail.comGuy Wiener (WIS) Dynamic OOP and Meta-Classes in Python 24/5/2011 57 / 57