Design patterns in Python Glenn Ramsey Kiwi Pycon 2011
Who I am Glenn Ramsey
ME (1993, U of  Auckland) mechanical engineering – control systems
Freelance developer (mainly C++, now Python)
Based at Hikutaia
Row Pro (digitalrowing.com) since 2001
Interest in software is primarily for modelling and simulation
PhD candidate at U of A (FEA horse's hoof, FORTRAN, Perl) thesis submitted Hikutaia
Outline Motivation
General software design / pattern concepts (brief)
Specific examples of “Gang of Four” patterns in Python
Motivation “ 16 of 23 [GoF] patterns have a qualitatively simpler implementation in Lisp or Dylan than in C++, for at least some uses of each pattern” Peter Norvig (http://norvig.com/design-patterns/) “ Patterns are not needed in Python because design patterns are a sign of a deficiency of a language” … ... for the purpose that the design pattern addresses. How is observer implemented in Python?  (equivalent to Boost.Signals, Qt Slot/Signals, .NET events)  Coming from C++ or Java, if you already know the GoF patterns then it would be informative to see how they are implemented in Python. This talk documents part of my journey from C++ to Ptyhon.
Software vs Engineering design Physical construction – E.g. building a house Software construction The code is the design! “ Software may be cheap to build, but it is incredibly expensive to design” J W Reeves, What Is Software Design?, www.developerdotstar.com  Design stage output
Software design How does one design software, compared to physical engineering design? Data + algorithms? - only a part of the solution  Structure and Interpretation of Computer Programs. H Abelson, G Sussman, J Sussman.  http://mitpress.mit.edu/sicp/full-text/book/book.html Software Design Concepts (wikipedia) Abstraction – categorize and group concepts Refinement – convert high level to program statements Modularity – isolate independent features Software architecture – overall structure of the software Control Hierarchy – program structure Structural partitioning – horizontal vs vertical ? Data structure – logical relationship among elements Software procedure – an operation within a module Information hiding -  information contained within a module is inaccessible to others Not especially helpful - too abstract!
Object Oriented Design principles Open Close Principle Software entities like classes, modules and functions should be open for extension but closed for modifications.
Encapsulation – information hiding Dependency Inversion Principle High-level modules should not depend on low-level modules. Both should depend on abstractions.
Abstractions should not depend on details. Details should depend on abstractions.
Loose coupling Interface Segregation Principle Clients should not be forced to depend upon interfaces that they don't use. Single Responsibility Principle A class should have only one job. Liskov's Substitution Principle Derived types must be completely substitutable for their base types. Prefer composition over inheritance http://www.oodesign.com/design-principles.html
What is a design pattern? Wikipedia: In software engineering, a design pattern is a general reusable solution to a commonly occurring problem within a given context in software design. A design pattern is not a finished design that can be transformed directly into code.
Christopher Alexander - Architect
Patterns are discovered – not invented
Patterns are not independent from the programming language. Example: subroutines in assembler. Gamma, Helm, Johnson, Vlissades (1995): Design patterns: elements of reusable object oriented software. Addison Wesley.
Pattern classes Purpose Creational Structural Behavioural Scope Class Factory Method Adapter (class) Interpreter Template Method Object Abstract factory Builder Prototype Singleton Adapter (object) Bridge Composite Decorator Facade Flyweight Proxy Chain of responsibility Command Iterator Mediator Memento Observer State Strategy Visitor Gamma, Helm, Johnson, Vlissades (1995): Design patterns: elements of reusable object oriented software. Addison Wesley. Invisible or simplified in Python due to:  first class types   first class functions  other Fixed at compile time Can change at runtime Object creation Compostion of classes or objects Class and object interactions
Why are they invisible/ simplified? Some patterns are work-arounds for static typing
Python has First class* types
First class* functions An object* is first-class when it: can be stored in variables and data structures
can be passed as a parameter to a subroutine
can be returned as the result of a subroutine
can be constructed at run-time
has intrinsic identity (independent of any given name) *The term "object" is used loosely here, not necessarily referring to objects in object-oriented programming. The simplest scalar data types, such as integer and floating-point numbers, are nearly always first-class. Source:wikipedia
Why are they invisible/simplified? (2) Python has duck typing
Wikipedia: In computer programming with object-oriented programming languages, duck typing is a style of dynamic typing in which an object's current set of methods and properties determines the valid semantics, rather than its inheritance from a particular class or implementation of a specific interface. An object only has to have a method with the right name
This means that a base class is not always needed
Therefore a lot of infrastructure code can be avoided
Why are they invisible/simplified? (3) Override special methods Automatic  delegation Methods that a class does not know about can be passed on to a another class
When to use a class Use a class only: if you need to inherit from it
If you need to do something special. E.g. # Put in const.py...: class   _const : class   ConstError (TypeError):  pass def   __setattr__ ( self ,name,value): if   self .__dict__.has_key(name): raise   self .ConstError,  "Can't  rebind   const (%s)" %name self .__dict__[name]=value import  sys sys.modules[__name__]=_const() # that's all -- now any client-code can import  const # and bind an attribute ONCE: const.magic =  23 # but NOT re-bind it: const.magic =  88   # raises const.ConstError # you may also want to add the obvious __delattr__ Alex Martelli http://code.activestate.com/recipes/65207-constants-in-python/
Iterator – built in Provide a way to access the elements of an aggregate object sequentially without exposing its underlying representation class   Sequence : def   __init__ ( self , size): self .list = [x  for  x  in  xrange(size)] self .index =  0 def   __iter__ ( self ): return   self def   next ( self ): if  len( self .list) ==  self .index: raise  StopIteration current =  self .list[ self .index] self .index +=  1 return  current >>> a = Sequence( 3 ) >>>  for  x  in  a: print  x 0 1 2 >>>  http://www.dofactory.com/Patterns/PatternIterator.aspx
Command Encapsulate a request as an object, thereby letting you parameterize clients with different requests, queue or log requests, and support undoable operations.
Known uses: undo/redo.
OO replacement for callbacks.
Specify, queue and execute requests at different times. http://www.cs.mcgill.ca/~hv/classes/CS400/01.hchen/doc/command/command.html
Command – GoF style Rahul Verma, Chetan Giridhar. Design Patterns in Python. www.testingperspective.com  class   Command : """The Command Abstract class""" def   __init__ ( self ): pass #Make changes  def   execute ( self ): #OVERRIDE raise  NotImplementedError class   FlipUpCommand (Command): """The Command class for turning on the light""" def   __init__ ( self , light): self .__light = light def   execute ( self ): self .__light.turnOn()
Command in Python def   greet (who): print   "Hello %s"  % who greet_command =  lambda : greet( "World" ) # pass the  callable  around, and invoke it later greet_command() class   MoveFileCommand (object): def   __init__ ( self , src, dest): self .src = src self .dest = dest self () def   __call__ ( self ): os.rename( self .src,  self .dest) def   undo ( self ): os.rename( self .dest,  self .src) undo_stack = [] undo_stack.append(MoveFileCommand( 'foo.txt' ,  'bar.txt' )) undo_stack.append(MoveFileCommand( 'bar.txt' ,  'baz.txt' )) # foo.txt is now renamed to baz.txt undo_stack.pop().undo()  # Now it's bar.txt undo_stack.pop().undo()  # and back to foo.txt  http://stackoverflow.com/questions/1494442/general-command-pattern-and-command-dispatch-pattern-in-python  (Ants Aasma) Simple case: Just use a callable More complex case: Use a command object but no need for a base class
Singleton Ensure a class has only one instance, and provide a global point of access to it. Excessive consumption may be harmful because: it overloads your liver
makes you seem stupid
it's global
creates very strong coupling with client classes http://en.csharp-online.net

Patterns in Python

  • 1.
    Design patterns inPython Glenn Ramsey Kiwi Pycon 2011
  • 2.
    Who I amGlenn Ramsey
  • 3.
    ME (1993, Uof Auckland) mechanical engineering – control systems
  • 4.
  • 5.
  • 6.
  • 7.
    Interest in softwareis primarily for modelling and simulation
  • 8.
    PhD candidate atU of A (FEA horse's hoof, FORTRAN, Perl) thesis submitted Hikutaia
  • 9.
  • 10.
    General software design/ pattern concepts (brief)
  • 11.
    Specific examples of“Gang of Four” patterns in Python
  • 12.
    Motivation “ 16of 23 [GoF] patterns have a qualitatively simpler implementation in Lisp or Dylan than in C++, for at least some uses of each pattern” Peter Norvig (http://norvig.com/design-patterns/) “ Patterns are not needed in Python because design patterns are a sign of a deficiency of a language” … ... for the purpose that the design pattern addresses. How is observer implemented in Python? (equivalent to Boost.Signals, Qt Slot/Signals, .NET events) Coming from C++ or Java, if you already know the GoF patterns then it would be informative to see how they are implemented in Python. This talk documents part of my journey from C++ to Ptyhon.
  • 13.
    Software vs Engineeringdesign Physical construction – E.g. building a house Software construction The code is the design! “ Software may be cheap to build, but it is incredibly expensive to design” J W Reeves, What Is Software Design?, www.developerdotstar.com Design stage output
  • 14.
    Software design Howdoes one design software, compared to physical engineering design? Data + algorithms? - only a part of the solution Structure and Interpretation of Computer Programs. H Abelson, G Sussman, J Sussman. http://mitpress.mit.edu/sicp/full-text/book/book.html Software Design Concepts (wikipedia) Abstraction – categorize and group concepts Refinement – convert high level to program statements Modularity – isolate independent features Software architecture – overall structure of the software Control Hierarchy – program structure Structural partitioning – horizontal vs vertical ? Data structure – logical relationship among elements Software procedure – an operation within a module Information hiding - information contained within a module is inaccessible to others Not especially helpful - too abstract!
  • 15.
    Object Oriented Designprinciples Open Close Principle Software entities like classes, modules and functions should be open for extension but closed for modifications.
  • 16.
    Encapsulation – informationhiding Dependency Inversion Principle High-level modules should not depend on low-level modules. Both should depend on abstractions.
  • 17.
    Abstractions should notdepend on details. Details should depend on abstractions.
  • 18.
    Loose coupling InterfaceSegregation Principle Clients should not be forced to depend upon interfaces that they don't use. Single Responsibility Principle A class should have only one job. Liskov's Substitution Principle Derived types must be completely substitutable for their base types. Prefer composition over inheritance http://www.oodesign.com/design-principles.html
  • 19.
    What is adesign pattern? Wikipedia: In software engineering, a design pattern is a general reusable solution to a commonly occurring problem within a given context in software design. A design pattern is not a finished design that can be transformed directly into code.
  • 20.
  • 21.
    Patterns are discovered– not invented
  • 22.
    Patterns are notindependent from the programming language. Example: subroutines in assembler. Gamma, Helm, Johnson, Vlissades (1995): Design patterns: elements of reusable object oriented software. Addison Wesley.
  • 23.
    Pattern classes PurposeCreational Structural Behavioural Scope Class Factory Method Adapter (class) Interpreter Template Method Object Abstract factory Builder Prototype Singleton Adapter (object) Bridge Composite Decorator Facade Flyweight Proxy Chain of responsibility Command Iterator Mediator Memento Observer State Strategy Visitor Gamma, Helm, Johnson, Vlissades (1995): Design patterns: elements of reusable object oriented software. Addison Wesley. Invisible or simplified in Python due to: first class types first class functions other Fixed at compile time Can change at runtime Object creation Compostion of classes or objects Class and object interactions
  • 24.
    Why are theyinvisible/ simplified? Some patterns are work-arounds for static typing
  • 25.
    Python has Firstclass* types
  • 26.
    First class* functionsAn object* is first-class when it: can be stored in variables and data structures
  • 27.
    can be passedas a parameter to a subroutine
  • 28.
    can be returnedas the result of a subroutine
  • 29.
    can be constructedat run-time
  • 30.
    has intrinsic identity(independent of any given name) *The term "object" is used loosely here, not necessarily referring to objects in object-oriented programming. The simplest scalar data types, such as integer and floating-point numbers, are nearly always first-class. Source:wikipedia
  • 31.
    Why are theyinvisible/simplified? (2) Python has duck typing
  • 32.
    Wikipedia: In computerprogramming with object-oriented programming languages, duck typing is a style of dynamic typing in which an object's current set of methods and properties determines the valid semantics, rather than its inheritance from a particular class or implementation of a specific interface. An object only has to have a method with the right name
  • 33.
    This means thata base class is not always needed
  • 34.
    Therefore a lotof infrastructure code can be avoided
  • 35.
    Why are theyinvisible/simplified? (3) Override special methods Automatic delegation Methods that a class does not know about can be passed on to a another class
  • 36.
    When to usea class Use a class only: if you need to inherit from it
  • 37.
    If you needto do something special. E.g. # Put in const.py...: class _const : class ConstError (TypeError): pass def __setattr__ ( self ,name,value): if self .__dict__.has_key(name): raise self .ConstError, "Can't rebind const (%s)" %name self .__dict__[name]=value import sys sys.modules[__name__]=_const() # that's all -- now any client-code can import const # and bind an attribute ONCE: const.magic = 23 # but NOT re-bind it: const.magic = 88 # raises const.ConstError # you may also want to add the obvious __delattr__ Alex Martelli http://code.activestate.com/recipes/65207-constants-in-python/
  • 38.
    Iterator – builtin Provide a way to access the elements of an aggregate object sequentially without exposing its underlying representation class Sequence : def __init__ ( self , size): self .list = [x for x in xrange(size)] self .index = 0 def __iter__ ( self ): return self def next ( self ): if len( self .list) == self .index: raise StopIteration current = self .list[ self .index] self .index += 1 return current >>> a = Sequence( 3 ) >>> for x in a: print x 0 1 2 >>> http://www.dofactory.com/Patterns/PatternIterator.aspx
  • 39.
    Command Encapsulate arequest as an object, thereby letting you parameterize clients with different requests, queue or log requests, and support undoable operations.
  • 40.
  • 41.
  • 42.
    Specify, queue andexecute requests at different times. http://www.cs.mcgill.ca/~hv/classes/CS400/01.hchen/doc/command/command.html
  • 43.
    Command – GoFstyle Rahul Verma, Chetan Giridhar. Design Patterns in Python. www.testingperspective.com class Command : """The Command Abstract class""" def __init__ ( self ): pass #Make changes def execute ( self ): #OVERRIDE raise NotImplementedError class FlipUpCommand (Command): """The Command class for turning on the light""" def __init__ ( self , light): self .__light = light def execute ( self ): self .__light.turnOn()
  • 44.
    Command in Pythondef greet (who): print "Hello %s" % who greet_command = lambda : greet( "World" ) # pass the callable around, and invoke it later greet_command() class MoveFileCommand (object): def __init__ ( self , src, dest): self .src = src self .dest = dest self () def __call__ ( self ): os.rename( self .src, self .dest) def undo ( self ): os.rename( self .dest, self .src) undo_stack = [] undo_stack.append(MoveFileCommand( 'foo.txt' , 'bar.txt' )) undo_stack.append(MoveFileCommand( 'bar.txt' , 'baz.txt' )) # foo.txt is now renamed to baz.txt undo_stack.pop().undo() # Now it's bar.txt undo_stack.pop().undo() # and back to foo.txt http://stackoverflow.com/questions/1494442/general-command-pattern-and-command-dispatch-pattern-in-python (Ants Aasma) Simple case: Just use a callable More complex case: Use a command object but no need for a base class
  • 45.
    Singleton Ensure aclass has only one instance, and provide a global point of access to it. Excessive consumption may be harmful because: it overloads your liver
  • 46.
  • 47.
  • 48.
    creates very strongcoupling with client classes http://en.csharp-online.net

Editor's Notes

  • #5 Personal: needed an observer implementation in Python like Boost.Signal, initially couldn't find one. My engineering background lead me to search for generic design principles in software.
  • #8 Polymorphism
  • #10 Creational patterns: patterns that can be used to create objects. Structural patterns: patterns that can be used to combine objects and classes in order to build structured objects. Behavioral patterns: patterns that can be used to build a computation and to control data flows. Norvig: 16 of 23 patterns are either invisible or simpler, due to: First-class types (6): Abstract-Factory, Flyweight, Factory-Method, State, Proxy, Chain-Of-Responsibility First-class functions (4): Command, Strategy, Template-Method, Visitor Macros (2): Interpreter, Iterator Method Combination (2): Mediator, Observer Multimethods (1): Builder Modules (1): Facade