Python makes loading code dynamically easy, allowing you to configure and extend your application by discovering and loading extensions at runtime. This presentation will discuss the techniques for dynamic code loading used in several well-known applications and weigh the pros and cons of each approach.

1. Dynamic Code Patterns:
Extending Your
Applications with Plugins
Doug Hellmann – @doughellmann – PyCon 2013
Saturday, March 16, 13 1

2. What is a “Plugin”?
• Loaded Dynamically
• Extends Core
• Possibly Unknown Source
Saturday, March 16, 13 2

3. Why Plugins?
• Better API Abstraction
• Reduce Core Dependencies
• Strategy Pattern
• Visitor Pattern
• Indirect Code Contributions
Saturday, March 16, 13 3

4. Ceilometer
• OpenStack Metering
• Measures Clouds
• Varied Billing Requirements
• Deployers Extend and Customize
Saturday, March 16, 13 4

5. Ceilometer Design
Central
Pollsters
Compute Meter Data
Objects Network Volumes Images Compute
Pollsters Message Bus
Event
Listener
Notification
Message Bus
Collector
Saturday, March 16, 13 5

6. Ceilometer Design
Central
Pollsters
Compute Meter Data
Objects Network Volumes Images Compute
Pollsters Message Bus
Event
Listener
Notification
Message Bus
Collector
Saturday, March 16, 13 5

7. Ceilometer Design
Central
Pollsters
Compute Meter Data
Objects Network Volumes Images Compute
Pollsters Message Bus
Event
Listener
Notification
Message Bus
Collector
Saturday, March 16, 13 5

8. Ceilometer Design
Central
Pollsters
Compute Meter Data
Objects Network Volumes Images Compute
Pollsters Message Bus
Event
Listener
Notification
Message Bus
Collector
Saturday, March 16, 13 5

9. Ceilometer Design
Central
Pollsters
Compute Meter Data
Objects Network Volumes Images Compute
Pollsters Message Bus
Event
Listener
Notification
Message Bus
Collector
Saturday, March 16, 13 5

10. Ceilometer Design
Central
Pollsters
Compute Meter Data
Objects Network Volumes Images Compute
Pollsters Message Bus
Event
Listener
Notification
Message Bus
Collector
Saturday, March 16, 13 5

11. Ceilometer Design
Central
Pollsters
Compute Meter Data
Objects Network Volumes Images Compute
Pollsters Message Bus
Event
Listener
Notification
Message Bus
Collector
Saturday, March 16, 13 5

12. Ceilometer Design
Central
Pollsters
Compute Meter Data
Objects Network Volumes Images Compute
Pollsters Message Bus
Event
Listener
Notification
Message Bus
Collector
Saturday, March 16, 13 5

13. Ceilometer Design
Central
Pollsters
Compute Meter Data
Objects Network Volumes Images Compute
Pollsters Message Bus
Event
Listener
Notification
Message Bus
Collector
Saturday, March 16, 13 5

14. Ceilometer Plugins
• Message Bus
• Receiving Notifications
• Polling Compute
• Polling Other
• Storage
Saturday, March 16, 13 6

15. Research
• Blogofile
• Sphinx • Django
• Pyramid
• Mercurial • SQLAlchemy
• cliff
• virtualenvwrapper • Diamond
• Nose • Nova
• Trac
Saturday, March 16, 13 7

16. Discovery
Explicit Scan
Diamond
File Mercurial
Blogofile
Mercurial Trac
Import Django Nose
Pyramid
Reference Sphinx
cliff
virtualenvwrapper
Nova
Saturday, March 16, 13 8

17. Enabling
Explicit Implicit
Django
Pyramid
SQLAlchemy
virtualenvwrapper
Blogofile
cliff
Mercurial
Trac
Sphinx
Saturday, March 16, 13 9

18. Importing
Custom pkg_resources
Django
Pyramid Trac
Sphinx Nose
Diamond SQLAlchemy
Nova Blogofile
Nose cliff
SQLAlchemy virtualenvwrapper
Blogofile
Saturday, March 16, 13 10

19. Integration
Fine Coarse
Nose Sphinx
Prompt virtualenvwrapper Trac
Mercurial
cliff Blogofile
Inspect Diamond Pyramid
Django
Saturday, March 16, 13 11

20. Integration
Fine Coarse
Nose Sphinx
Prompt virtualenvwrapper Trac
Mercurial
cliff Blogofile
Inspect Diamond Pyramid
Django
Saturday, March 16, 13 12

21. API Enforcement
Convention Base Class / Interface
Blogofile
Mercurial Nose (optional)
Django Trac (interface)
Sphinx Diamond
Pyramid cliff (abc)
virtualenvwrapper
Saturday, March 16, 13 13

22. Invocation
Driver Dispatcher Iterator
Mercurial
Nose
SQLAlchemy Django
Diamond
Nova Pyramid
virtualenvwrapper
cliff
Saturday, March 16, 13 14

23. Discovery / Importing
• Entry Points
• Distribute and pkg_resources
• Be Consistent
Saturday, March 16, 13 15

24. Enabling
• Explicit disabling
• Automatic disabling
Saturday, March 16, 13 16

25. Integration
• Fine
• Inspect
• Application Owns Relationship
Saturday, March 16, 13 17

26. API Enforcement
• Abstract Base Classes
• Duck Typing
Saturday, March 16, 13 18

27. Invocation
• Storage – Driver
• Notifications – Dispatcher
• Pollsters – Iterator
Saturday, March 16, 13 19

28. stevedore
• Implements Patterns
• Wraps pkg_resources
Saturday, March 16, 13 20

29. NamedExtensionManager
• Multiple Plugins
• Only Loads Named Plugins
• map()
Saturday, March 16, 13 21

30. EnabledExtensionManager
• Multiple Plugins
• Checks Each with Function on Load
• map()
Saturday, March 16, 13 22

31. DispatchExtensionManager
• Multiple Plugins
• Invokes Subset on map()
Saturday, March 16, 13 23

32. DriverManager
• Single Plugin
• Direct Access
Saturday, March 16, 13 24

33. Dynamic Code Patterns:
Extending Your
Applications with Plugins
Doug Hellmann – @doughellmann – PyCon 2013
http://packages.python.org/distribute/pkg_resources.html
https://github.com/dreamhost/stevedore
https://launchpad.net/ceilometer
http://doughellmann.com
Saturday, March 16, 13 25