Successfully reported this slideshow.
Your SlideShare is downloading. ×

Monkey-patching in Python: a magic trick or a powerful tool?

Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Loading in …3
×

Check these out next

1 of 45 Ad

Monkey-patching in Python: a magic trick or a powerful tool?

Download to read offline

Monkey-patching is a dynamic modification of a class or a module at runtime.

The Python gives developers a great opportunity to use monkey-patching almost everywhere. But should developers do it? Is it a magic trick or a powerful tool? In this talk we will try to give the answers to these questions and try to figure out pros and cons of using monkey-patching.

First of all we will learn what is monkey-patching in Python and consider some basic examples of using it.

Of course, monkey-patching may cause some problems in the code. We will consider bad ways to use it and try to learn different types of problems monkey-patching may lead to.

Despite of some bugs that may appear in a patched program, monkey-patching is used in a real life rather often. There are some reasons and motives to do it. We will consider the examples of using monkey-patching in real projects like gevent, in some other libraries and in testing. Also we will learn some monkey-patch tricks that helps to solve real-life problems in the Python debugger which is a part of the PyCharm and the PyDev.

After that we will compare using of monkey-patching in Python to using it in an another dynamic language Ruby. Are there any differences between them? Is our reasoning correct for Ruby?

Finally we will conclude all our thoughts and examples and try to give the answer to the question from title.

Monkey-patching is a dynamic modification of a class or a module at runtime.

The Python gives developers a great opportunity to use monkey-patching almost everywhere. But should developers do it? Is it a magic trick or a powerful tool? In this talk we will try to give the answers to these questions and try to figure out pros and cons of using monkey-patching.

First of all we will learn what is monkey-patching in Python and consider some basic examples of using it.

Of course, monkey-patching may cause some problems in the code. We will consider bad ways to use it and try to learn different types of problems monkey-patching may lead to.

Despite of some bugs that may appear in a patched program, monkey-patching is used in a real life rather often. There are some reasons and motives to do it. We will consider the examples of using monkey-patching in real projects like gevent, in some other libraries and in testing. Also we will learn some monkey-patch tricks that helps to solve real-life problems in the Python debugger which is a part of the PyCharm and the PyDev.

After that we will compare using of monkey-patching in Python to using it in an another dynamic language Ruby. Are there any differences between them? Is our reasoning correct for Ruby?

Finally we will conclude all our thoughts and examples and try to give the answer to the question from title.

Advertisement
Advertisement

More Related Content

Slideshows for you (17)

Similar to Monkey-patching in Python: a magic trick or a powerful tool? (20)

Advertisement

Monkey-patching in Python: a magic trick or a powerful tool?

  1. 1. monkey-patching: amagictrickor apowerfultool? Elizaveta Shashkova
 @lisa_shashkova elizaveta.shashkova@jetbrains.com
  2. 2. bio • Software Developer of PyCharm IDE at JetBrains • Debugger used in PyDev & PyCharm 2
  3. 3. Monkey-patching is a dynamic modification of a class or a module at runtime def safe_sqrt(num): # doesn’t throw exception if num < 0 if num < 0: return math.nan return math.original(num) >>> import math >>> math.original = math.sqrt >>> math.sqrt = safe_sqrt monkey-patching
  4. 4. monkey-patching • Guerilla patch • Does something secretly and incompatibly with others • Gorilla patch • Monkey patch
  5. 5. otherlanguages Ruby class String def upcase self.reverse end end > puts "hello".upcase
 "olleh"
  6. 6. python Explicit is better than implicit
  7. 7. whatcanwepatch? Everything!
  8. 8. whatcanwepatch? Almost Everything! We can’t patch built-ins defined in C
  9. 9. lightsideof monkey-patching 9 darksideof monkey-patching
  10. 10. •Workaround for bugs in third-party code lightside
  11. 11. •Workaround for bugs in third-party code • Don’t forget to fix and create a pull request! lightside
  12. 12. •Workaround for bugs in third-party code • Don’t forget to fix and create a pull request! •Code instrumentation lightside
  13. 13. •Workaround for bugs in third-party code • Don’t forget to fix and create a pull request! •Code instrumentation •Testing lightside
  14. 14. •Workaround for bugs in third-party code • Don’t forget to fix and create a pull request! •Code instrumentation •Testing •Changing of standard modules lightside
  15. 15. examples
  16. 16. lightside:catchingnew process
  17. 17. lightside:newprocess class ProcessManager: def __init__(self, system_fork): self.system_fork = system_fork def do_fork(self): pid = self.system_fork() if pid == 0: # child process start_tracing() return pid
  18. 18. import os manager = ProcessManager(os.fork) os.fork = manager.do_fork
 
 os.fork() #tracing in a new process lightside:newprocess
  19. 19. lightside:importhook
  20. 20. •How to patch a module as soon as possible? •Patch it right after importing lightside
  21. 21. lightside:importhook class ImportHookManager: def __init__(self, system_import): self.system_import = system_import def do_import(self, name): module = self.system_import(name) if name == "os":
 orig_fork = module.fork
 module.fork = ProcessManager(orig_fork).do_fork return module
  22. 22. lightside:importhook import builtins manager = ImportHookManager(builtins.__import__) builtins.__import__ = manager.do_import
 
 
 import os
 # module is patched as soon as it is imported
  23. 23. lightside:gevent
  24. 24. lightside:gevent ● gevent provides asynchronous I/O ● Blocking system calls in the standard library import socket ● Replace with gevent modules from gevent import socket
  25. 25. lightside:gevent ! The simplest way from gevent import monkey
 monkey.patch_socket()
  26. 26. •Workaround for bugs in third-party code •Code instrumentation •Testing •Changing of standard modules lightside
  27. 27. lightsideof monkey-patching 27 darksideof monkey-patching
  28. 28. darkside Implicitness
  29. 29. darkside • Implicit changes
  30. 30. darkside • Implicit changes • Undocumented changes
  31. 31. darkside • Implicit changes • Undocumented changes • Unpredictable behaviour
  32. 32. darkside • Implicit changes • Undocumented changes • Unpredictable behaviour • Patching the same
  33. 33. examples
  34. 34. darkside:gevent
  35. 35. darkside:gevent • Patches standard modules socket, thread, time and others • Should be done as soon as possible
  36. 36. runningcodeunderdebugger • Event loop in debugger Based on threads • Event loop in user’s code Based on gevent loop
 They should be separated
  37. 37. theproblem Save the original versions of modules and use them with the patched versions
  38. 38. importinginpython # very simple version
 def import_module(name): if name in sys.modules: return sys.modules[name] module = internal_import(name) sys.modules[name] = module return module
  39. 39. savemodules saved_modules.py:
 
 import socket # save the module object save = sys.modules.pop('socket', None) # it's not in sys.modules, reimport import socket as socket_saved # put the module available for patching sys.modules['socket'] = save
  40. 40. savemodules from saved_modules import saved_socket
 import socket
 
 # it will patch socket in sys.path
 gevent.patch_socket()
 
 # the patched function
 socket.create_connection() # the original function
 saved_socket.create_connection()
  41. 41. theproblem Save the original versions of modules and use them with the patched versions Solved!
  42. 42. ● Implicit changes ● Undocumented changes ● Broken readability ● Incompatibility with other patches ● Workaround for bugs in third-party code ● Code instrumentation ● Testing ● Changing of standard modules lightsideof monkey-patching darksideof monkey-patching
  43. 43. whenshouldiuseit? If and only if there are no other solutions for the problem
  44. 44. lightsideof monkey-patching 44 darksideof monkey-patching
  45. 45. lightsideof monkey-patching 45 darksideof monkey-patching Elizaveta Shashkova @lisa_shashkova

×