Dynamic Code Patterns: Extending Your Applications with Plugins

11,918 views

Published on

Python makes loading code dynamically easy, allowing you to configure and extend your application by discovering and loading extensions at runtime. This presentation will discuss the techniques for dynamic code loading used in several well-known applications and weigh the pros and cons of each approach.

Published in: Technology

Dynamic Code Patterns: Extending Your Applications with Plugins

  1. 1. Dynamic Code Patterns: Extending Your Applications with Plugins Doug Hellmann – @doughellmann – PyCon 2013Saturday, March 16, 13 1
  2. 2. What is a “Plugin”? • Loaded Dynamically • Extends Core • Possibly Unknown SourceSaturday, March 16, 13 2
  3. 3. Why Plugins? • Better API Abstraction • Reduce Core Dependencies • Strategy Pattern • Visitor Pattern • Indirect Code ContributionsSaturday, March 16, 13 3
  4. 4. Ceilometer • OpenStack Metering • Measures Clouds • Varied Billing Requirements • Deployers Extend and CustomizeSaturday, March 16, 13 4
  5. 5. Ceilometer Design Central Pollsters Compute Meter Data Objects Network Volumes Images Compute Pollsters Message Bus Event Listener Notification Message Bus CollectorSaturday, March 16, 13 5
  6. 6. Ceilometer Design Central Pollsters Compute Meter Data Objects Network Volumes Images Compute Pollsters Message Bus Event Listener Notification Message Bus CollectorSaturday, March 16, 13 5
  7. 7. Ceilometer Design Central Pollsters Compute Meter Data Objects Network Volumes Images Compute Pollsters Message Bus Event Listener Notification Message Bus CollectorSaturday, March 16, 13 5
  8. 8. Ceilometer Design Central Pollsters Compute Meter Data Objects Network Volumes Images Compute Pollsters Message Bus Event Listener Notification Message Bus CollectorSaturday, March 16, 13 5
  9. 9. Ceilometer Design Central Pollsters Compute Meter Data Objects Network Volumes Images Compute Pollsters Message Bus Event Listener Notification Message Bus CollectorSaturday, March 16, 13 5
  10. 10. Ceilometer Design Central Pollsters Compute Meter Data Objects Network Volumes Images Compute Pollsters Message Bus Event Listener Notification Message Bus CollectorSaturday, March 16, 13 5
  11. 11. Ceilometer Design Central Pollsters Compute Meter Data Objects Network Volumes Images Compute Pollsters Message Bus Event Listener Notification Message Bus CollectorSaturday, March 16, 13 5
  12. 12. Ceilometer Design Central Pollsters Compute Meter Data Objects Network Volumes Images Compute Pollsters Message Bus Event Listener Notification Message Bus CollectorSaturday, March 16, 13 5
  13. 13. Ceilometer Design Central Pollsters Compute Meter Data Objects Network Volumes Images Compute Pollsters Message Bus Event Listener Notification Message Bus CollectorSaturday, March 16, 13 5
  14. 14. Ceilometer Plugins • Message Bus • Receiving Notifications • Polling Compute • Polling Other • StorageSaturday, March 16, 13 6
  15. 15. Research • Blogofile • Sphinx • Django • Pyramid • Mercurial • SQLAlchemy • cliff • virtualenvwrapper • Diamond • Nose • Nova • TracSaturday, March 16, 13 7
  16. 16. Discovery Explicit Scan Diamond File Mercurial Blogofile Mercurial Trac Import Django Nose Pyramid Reference Sphinx cliff virtualenvwrapper NovaSaturday, March 16, 13 8
  17. 17. Enabling Explicit Implicit Django Pyramid SQLAlchemy virtualenvwrapper Blogofile cliff Mercurial Trac SphinxSaturday, March 16, 13 9
  18. 18. Importing Custom pkg_resources Django Pyramid Trac Sphinx Nose Diamond SQLAlchemy Nova Blogofile Nose cliff SQLAlchemy virtualenvwrapper BlogofileSaturday, March 16, 13 10
  19. 19. Integration Fine Coarse Nose Sphinx Prompt virtualenvwrapper Trac Mercurial cliff Blogofile Inspect Diamond Pyramid DjangoSaturday, March 16, 13 11
  20. 20. Integration Fine Coarse Nose Sphinx Prompt virtualenvwrapper Trac Mercurial cliff Blogofile Inspect Diamond Pyramid DjangoSaturday, March 16, 13 12
  21. 21. API Enforcement Convention Base Class / Interface Blogofile Mercurial Nose (optional) Django Trac (interface) Sphinx Diamond Pyramid cliff (abc) virtualenvwrapperSaturday, March 16, 13 13
  22. 22. Invocation Driver Dispatcher Iterator Mercurial Nose SQLAlchemy Django Diamond Nova Pyramid virtualenvwrapper cliffSaturday, March 16, 13 14
  23. 23. Discovery / Importing • Entry Points • Distribute and pkg_resources • Be ConsistentSaturday, March 16, 13 15
  24. 24. Enabling • Explicit disabling • Automatic disablingSaturday, March 16, 13 16
  25. 25. Integration • Fine • Inspect • Application Owns RelationshipSaturday, March 16, 13 17
  26. 26. API Enforcement • Abstract Base Classes • Duck TypingSaturday, March 16, 13 18
  27. 27. Invocation • Storage – Driver • Notifications – Dispatcher • Pollsters – IteratorSaturday, March 16, 13 19
  28. 28. stevedore • Implements Patterns • Wraps pkg_resourcesSaturday, March 16, 13 20
  29. 29. NamedExtensionManager • Multiple Plugins • Only Loads Named Plugins • map()Saturday, March 16, 13 21
  30. 30. EnabledExtensionManager • Multiple Plugins • Checks Each with Function on Load • map()Saturday, March 16, 13 22
  31. 31. DispatchExtensionManager • Multiple Plugins • Invokes Subset on map()Saturday, March 16, 13 23
  32. 32. DriverManager • Single Plugin • Direct AccessSaturday, March 16, 13 24
  33. 33. Dynamic Code Patterns: Extending Your Applications with Plugins Doug Hellmann – @doughellmann – PyCon 2013 http://packages.python.org/distribute/pkg_resources.html https://github.com/dreamhost/stevedore https://launchpad.net/ceilometer http://doughellmann.comSaturday, March 16, 13 25

×