Python speleology
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
1,366
On Slideshare
1,291
From Embeds
75
Number of Embeds
5

Actions

Shares
Downloads
32
Comments
0
Likes
3

Embeds 75

http://localhost 42
http://connectical.com 18
http://www.connectical.com 12
https://www.linkedin.com 2
https://twitter.com 1

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Python Speleology Getting deep in python features
  • 2. 1. The Zen of Python
  • 3. The Zen of Python ○ Beautiful is better than ugly. ○ Explicit is better than implicit. ○ Simple is better than complex. ○ Complex is better than complicated. ○ Flat is better than nested. ○ Sparse is better than dense. ○ Readability counts. ○ Special cases arent special enough to break the rules. ○ Although practicality beats purity. ○ Errors should never pass silently. ○ Unless explicitly silenced. ○ In the face of ambiguity, refuse the temptation to guess. ○ There should be one -and preferably only one- obvious way to do it. ○ Although that way may not be obvious at first unless youre Dutch. ○ Now is better than never. ○ Although never is often better than right now. ○ If the implementation is hard to explain, its a bad idea. ○ If the implementation is easy to explain, it may be a good idea. ○ Namespaces are one honking great idea -lets do more of those!
  • 4. 2. Types and objects
  • 5. Types and objects: To be or not to bea = 256b = 256print (a is b)a = 257b = 257print (a is b)
  • 6. Types and objects: To be or not to bea = 256b = 256print (a is b)a = 257b = 257print (a is b) True False
  • 7. Types and objects: To be or not to bea = 256 a = 256b = 256 b = 256print (a is b) print id(a), id(b)a = 257 a = 257b = 257 b = 257print (a is b) print id(a), id(b) True 22036112 22036112 False 22363568 22363640
  • 8. Types and objects: To be or not to bea = 256 a = 256b = 256 b = 256print (a is b) print id(a), id(b)a = 257 a = 257b = 257 b = 257print (a is b) print id(a), id(b) True False
  • 9. Types and objects: To be or not to bea = 256 a = 256b = 256 b = 256print (a is b) print id(a), id(b)a = 257 a = 257b = 257 b = 257print (a is b) print id(a), id(b) True 22036112 22036112 False 22363568 22363640
  • 10. Types and objects: functions are objectsdef x(f): return fdef y(f): return xprint x(y)(8)(0)
  • 11. Types and objects: functions are objectsdef x(f): return fdef y(f): return xprint x(y)(8)(0) 0
  • 12. Types and objects: functions are objectsdef x(i): if x.enabled: return i else: return "disabled"x.enabled = Trueprint x(8)
  • 13. Types and objects: functions are objectsdef x(i): if x.enabled: 8 return i else: return "disabled"x.enabled = Trueprint x(8)What happened if not set x.enabled?
  • 14. Types and objects: star operatorargs = (3 , 6)print range(*args)args = { "name": "example" }def f(name): print namef(**args)def f(*args, **kwargs): print args print kwargsf(1,2,3, name="example")
  • 15. Types and objects: star operatorargs = (3 , 6) [3, 4, 5] exampleprint range(*args) (1, 2, 3) {name: example}args = { "name": "example" }def f(name): print namef(**args)def f(*args, **kwargs): print args print kwargsf(1,2,3, name="example")
  • 16. 3. Reflection
  • 17. Reflection: get & setclass Example(object): num = 10x = Exampledir(x)hasattr(x, "num") == Truegetattr(x, "num", 0) == 10setattr(x, "num", 20)
  • 18. Reflection: local & globalglobals() Return a dictionary representing the current global symbol table. This is always the dictionary of the current module (inside a function or method, this is the module where it is defined, not the module from which it is called).locals() Update and return a dictionary representing the current local symbol table. Free variables are returned by locals() when it is called in function blocks, but not in class blocks.
  • 19. Reflection: __magic____name__ This is the name of the function. This only have a meaningful value is the function is defined with “def”.__class__ This is a reference to the class a method belongs to.__code__ This is a reference to the code object used in the implementation of python.
  • 20. Reflection: inspectorfrom inspect import getcomments# This is a commentdef f(x): print xprint getcomments(f)
  • 21. Reflection: inspectorfrom inspect import getcomments# This is a commentdef f(x): print xprint getcomments(f) # This is a comment
  • 22. Reflection: inspectorfrom inspect import getsource# This is a commentdef f(x): print xprint getsource(f)
  • 23. Reflection: inspectorfrom inspect import getsource# This is a commentdef f(x): print xprint getsource(f) def f(x): print x
  • 24. Reflection: lets more trickydef f(x): print xprint f.__code__.co_code
  • 25. Reflection: lets more trickydef f(x): print xprint f.__code__.co_code dx01x00GHdx00x00S
  • 26. Reflection: lets more trickydef f(x): print xprint f.__code__.co_code dx01x00GHdx00x00S YEEES!!! THE BYTECODE!!!
  • 27. 4. Context manager
  • 28. Context manager: in the beginning...item = Item()try: item.open() item.do()finally: item.close()
  • 29. Context manager: nowadays...with Item() as item: item.doclass Item(object): def __enter__(self): self.open() return self def __exit__(self,exc_type,exc_value,exc_t): self.close()...
  • 30. Context manager: the real worldwith file("/tmp/test", "w") as f: f.write("hello world")with lock(): # do some concurrentwith sudo("root"): # do some as root
  • 31. 5. Decorations
  • 32. Decorations: bold & italicdef makebold(fn): def wrapped(): return "<b>" + fn() + "</b>" return wrappeddef makeitalic(fn): def wrapped(): return "<i>" + fn() + "</i>" return wrapped@makebold@makeitalicdef hello(): return "hello world"print hello()
  • 33. Decorations: bold & italicdef makebold(fn): def wrapped(): <b><i>hello world</i></b> return "<b>" + fn() + "</b>" return wrappeddef makeitalic(fn): def wrapped(): return "<i>" + fn() + "</i>" return wrapped@makebold@makeitalicdef hello(): return "hello world"print hello()
  • 34. Decorations: complex decordef makestyle(arg): def decorator(f): def wrapper(*args, **kw): return "<" + arg + ">" + f() + "</" + arg + ">" return wrapper return decorator@makestyle("b")def hello(): return "hello world"print hello()
  • 35. Decorations: syntax sugardef makebold(fn): def wrapped(): return "<b>" + fn() + "</b>" return wrappeddef makestyle(arg): def decorator(f): def wrapper(*args, **kw): return "<" + arg + ">" + f() + "</" + arg + ">" return wrapper return decoratormakebold(hello)makestyle("b")(hello)
  • 36. 6. Iterations
  • 37. Iterations: comprehesionssquares = []for x in range(10): squares.append(x**2)squares = [x**2 for x in range(10)][(x, y) for x in [1,2,3] for y in [3,1,4] if x != y]{ (k,v) for k,v in [(1,2)] }
  • 38. Iterations: comprehesionssquares = []for x in range(10): squares.append(x**2)squares = [x**2 for x in range(10)][(x, y) for x in [1,2,3] for y in [3,1,4] if x != y]{ (k,v) for k,v in [(1,2)] } SET NOT DICT!
  • 39. Iterations: comprehesionssquares = []for x in range(10): squares.append(x**2)squares = [x**2 for x in range(10)][(x, y) for x in [1,2,3] for y in [3,1,4] if x != y]{ (k,v) for k,v in [(1,2)] } SET NOT DICT!{ k:v for k,v in [(1,2)] }
  • 40. Iterations: co-routinedef countdown(n): print "Counting down from", n while n > 0: yield n n -= 1 print "Done counting down"for i in countdown(10): print i
  • 41. Iterations: co-routinedef countdown(n): print "Counting down from", n while n > 0: yield n n -= 1 print "Done counting down"for i in countdown(10): print i Counting down from 10 10 9 8 7 6 5 4 3 2 1 Done counting down
  • 42. Iterations: co-routinedef countdown(n): print "Counting down from", n while n > 0: yield n n -= 1 print "Done counting down"print countdown(10) <generator object at 0x4035874c>
  • 43. 7. Overloading
  • 44. Overloading: binary operations __add__(self, other) x+y __sub__(self, other) x-y __mul__(self, other) x*y __div__(self, other) x/y __pow__(self, other) x ** y
  • 45. Overloading: binary operations __radd__(self, other) y+x __rsub__(self, other) y-x __rmul__(self, other) y*x __rdiv__(self, other) y/x __rpow__(self, other) y ** x
  • 46. Overloading: binary operations __radd__(self, other) 1+x __rsub__(self, other) 1-x __rmul__(self, other) 1*x __rdiv__(self, other) 1/x __rpow__(self, other) 1 ** x
  • 47. Overloading: binary operations __iadd__(self, other) x+=y __isub__(self, other) x-=y __imul__(self, other) x*=y __idiv__(self, other) x/=y __ipow__(self, other) x**=y
  • 48. Overloading: unary operations __neg__(self) -x __pos__(self) +x __abs__(self) abs(x) __invert__(self) ~x
  • 49. Overloading: conversion operations __int__(self) int(x) __float__(self) float(x) __complex__(self) complex(x) __str__(self) str(x) __nonzero__(self) bool(x) __unicode__(self) unicode(x)
  • 50. Overloading: comparison operations __eq__(self, other) x == y __lt__(self, other) x<y __le__(self, other) x <= y __gt__(self, other) x>y __ge__(self, other) x >= y __ne__(self, other) x != y
  • 51. Overloading: containers __contains__(self, other) y in x __getitem__(self, other) x[y] __setitem__(self, other,value) x[y] = z __delitem__(self, other) del x[y] __len__(self) len(x) __reversed__(self) reversed(x) __iter__(self) iter(x)
  • 52. 8. The Class Factory
  • 53. The Class Factory: class & objectsclass Example(object): attribute = "this is a class attribute" def __init__(self): self.attribute = "this is an obj attr override class one" self.another = "this is another obj attr, no class"print Example.attributeprint Example().attributeprint Example().anotherprint Example.another
  • 54. The Class Factory: class & objectsclass Example(object): attribute = "this is a class attribute" def __init__(self): self.attribute = "this is an obj attr override class one" self.another = "this is another obj attr, no class"print Example.attributeprint Example().attribute this is a class attributeprint Example().another this is an object attribute and override class one this is another object attribute, no classprint Example.another Traceback (most recent call last): Line 11, in <module> print Example.another AttributeError: type object Example has no attribute another
  • 55. The Class Factory: set & getclass Example(object): def __init__(self): self._name = x @property def name(self): return self._name @name.setter def name(self, value): self._name = value
  • 56. The Class Factory: @property abuseclass Example(object): def __init__(self, host, port): self.host = host self.port = port @property def connect(self): lib.connect(self.host, self.port)
  • 57. The Class Factory: @property abuseclass Example(object): def __init__(self, host, port): self.host = host self.port = port @property def connect(self): lib.connect(self.host, self.port) NEVER TYPE METHODS AS PROPERTIES
  • 58. The Class Factory: methods and more methods@staticmethod Nothing more than a function defined inside a class. It is callable without instantiating the class first. It’s definition is immutable via inheritance.@classmethod Also callable without instantiating the class, but its definition follows Sub class, not Parent class, via inheritance. That’s because the first argument for @classmethod function must always be cls (class).
  • 59. The Class Factory: methods and more methods@staticmethod Nothing more than a function defined inside a class. It is callable without instantiating the class first. It’s definition is immutable via inheritance. @staticmethod def static_method(): print "I do not receive nothing :("@classmethod Also callable without instantiating the class, but its definition follows Sub class, not Parent class, via inheritance. That’s because the first argument for @classmethod function must always be cls (class). @classmethod def class_method(cls): print "Im a class %s" % str(cls)
  • 60. The Class Factory: methods and more methodsclass Example(object): def __init__(self, name): self.name = name @classmethod def class_method(cls, name): return cls(name)x = Example.class_method("example")print x
  • 61. The Class Factory: methods and more methodsclass Example(object): def __init__(self, name): self.name = name @classmethod def class_method(cls, name): return cls(name)x = Example.class_method("example")print x <__main__.Example object at 0x40358b2c>
  • 62. The Class Factory: children and parentsclass Example(object): def __init__(self, name): self.name = name def do_something(self): raise NotImplementedError()class ChildExample(Example): def do_something(self): print self.name
  • 63. The Class Factory: children and parentsclass ExampleA(object): def __init__(self, name): self.name = name def do_something(self): raise NotImplementedError()class ExampleB(object): def do_otherthing(self): raise NotImplementedError()class ChildExample(ExampleB, ExampleA): def do_something(self): print self.name def do_otherthing(self): print self.name
  • 64. The Class Factory: τὰ μετὰ τὰ κλάσηclass Example(object): passx = Example()y = Exampleprint xprint y
  • 65. The Class Factory: τὰ μετὰ τὰ κλάσηclass Example(object): passx = Example()y = Exampleprint xprint y <__main__.Example object at 0x4035894c> <class __main__.Example>
  • 66. The Class Factory: τὰ μετὰ τὰ κλάσηdef example(): class Example(object): pass return Examplex = example()y = x()print xprint y
  • 67. The Class Factory: τὰ μετὰ τὰ κλάσηdef example(): class Example(object): pass return Examplex = example()y = x()print xprint y <class __main__.Example> <__main__.Example object at 0x403589ec>
  • 68. The Class Factory: τὰ μετὰ τὰ κλάσηclass Example(object): passprint type(Example)print type(Example())
  • 69. The Class Factory: τὰ μετὰ τὰ κλάσηclass Example(object): passprint type(Example)print type(Example()) <type type> <class __main__.Example>
  • 70. The Class Factory: τὰ μετὰ τὰ κλάσηclass Example(object): passprint type(Example)print type(Example()) <type type> <class __main__.Example>
  • 71. The Class Factory: τὰ μετὰ τὰ κλάσηclass Example(object): passprint type(Example)print type(Example()) <type type> <class __main__.Example>
  • 72. The Class Factory: τὰ μετὰ τὰ κλάσηclass Example(object): passx = Exampley = type(Example, (object,), {})print xprint yprint (x == y)
  • 73. The Class Factory: τὰ μετὰ τὰ κλάσηclass Example(object): passx = Exampley = type(Example, (object,), {})print xprint yprint (x == y) <class __main__.Example> <class __main__.Example> False
  • 74. The Class Factory: __magic____new__(cls, *args, **kwargs) Is the first method to get called in an objects instantiation, is a @classmethod, and must return a new instance of type cls.__init__(self, *args, **kwargs) Is the initializer for the instance. It gets passed whatever the primary constructor was called with.__del__(self) Is the destructor of the instance, will be invoked before clean the reference to the instance.
  • 75. The Class Factory: __metaclass__def upper_attr(f_class_name, f_class_parents, f_class_attr): attrs = ((name, value) for name, value in f_class_attr.items() if not name.startswith(__)) uppercase_attr = dict((name.upper(), value) for name, value in attrs) return type(f_class_name, f_class_parents, uppercase_attr)class Foo(object): __metaclass__ = upper_attr bar = bipprint hasattr(Foo, bar)print hasattr(Foo, BAR)
  • 76. The Class Factory: __metaclass__def upper_attr(f_class_name, f_class_parents, f_class_attr): attrs = ((name, value) for name, value in f_class_attr.items() if not name.startswith(__)) uppercase_attr = dict((name.upper(), value) for name, value in attrs) return type(f_class_name, f_class_parents, uppercase_attr)class Foo(object): __metaclass__ = upper_attr bar = bipprint hasattr(Foo, bar) Falseprint hasattr(Foo, BAR) True
  • 77. 9. Monkey Patching
  • 78. Monkey patching: first try class Person(object): def speak(self): print "hello" def monkey(foo): print "uh uh uh" Person.speak = monkey x = Person() x.speak()
  • 79. Monkey patching: be evil }:-) import os def monkey(*args, **kwargs): print "no no no" os.system = monkey os.system("ls /tmp")
  • 80. Monkey patching: be evil }:-) import os def monkey(*args, **kwargs): print "no no no" os.system = monkey os.system("ls /tmp") no no no
  • 81. 10. Profiling
  • 82. Profiling: hand made import time class Timer(object): def __init__(self, verbose=False): self.verbose = verbose def __enter__(self): self.start = time.time() return self def __exit__(self, *args): self.end = time.time() self.secs = self.end - self.start self.msecs = self.secs * 1000 # millisecs if self.verbose: print elapsed time: %f ms % self.msecs
  • 83. Profiling: hand made from redis import Redis import time rdb = Redis() with Timer() as t: class Timer(object): rdb.lpush("foo", "bar") def __init__(self, verbose=False): print "=> elasped lpush: %s s" % t. secs self.verbose = verbose with Timer as t: rdb.lpop("foo") def __enter__(self): print "=> elasped lpop: %s s" % t.secs self.start = time.time() return self def __exit__(self, *args): self.end = time.time() self.secs = self.end - self.start self.msecs = self.secs * 1000 # millisecs if self.verbose: print elapsed time: %f ms % self.msecs
  • 84. Profiling: profile & cProfile try: import cProfile as profile except ImportError: import profile def fib(n): if n == 0: return 0 elif n == 1: return 1 else: return fib(n-1) + fib(n-2) def fib_seq(n): seq = [ ] if n > 0: seq.extend(fib_seq(n-1)) seq.append(fib(n)) return seq profile.run(print fib_seq(6); print)
  • 85. Profiling: profile & cProfile try: import cProfile as profile except ImportError: import profile [0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, 233, 377, 610, 987, 1597, 2584, 4181, 6765] 57356 function calls (66 primitive calls) in 0.746 CPU seconds def fib(n): if n == 0: standard name Ordered by: return 0 ncalls tottime percall cumtime percall filename:lineno(function) 21 0.000 0.000 0.000 0.000 :0(append) elif n == 1: 20 0.000 0.000 0.000 0.000 :0(extend) return 1 1 0.001 0.001 0.001 0.001 :0(setprofile) else: 0.000 0.000 0.744 0.744 <string>:1(<module>) 1 1 0.000 0.000 0.746 0.746 profile:0(print fib_seq(20); print) return fib(n-1) + 0.000 0 0.000 fib(n-2) profile:0(profiler) 57291/21 0.743 0.000 0.743 0.035 profile_fibonacci_raw.py:13(fib) 21/1 0.001 0.000 0.744 0.744 profile_fibonacci_raw.py:22(fib_seq) def fib_seq(n): seq = [ ] if n > 0: seq.extend(fib_seq(n-1)) seq.append(fib(n)) return seq profile.run(print fib_seq(6); print)
  • 86. 11. Documentation
  • 87. Documentation: Zen Dont create documentation for your code. Code your documentation. def elements(n): """Return a list of n numbers from 0 to n- 1. """ return range(0,n)
  • 88. Documentation: everything is an object def elements(n): """Return a list of n numbers from 0 to n- 1. """ return range(0,n) print elements.__doc__
  • 89. Documentation: everything is an object def elements(n): """Return a list of n numbers from 0 to n- 1. """ return range(0,n) print elements.__doc__ Return a list of n numbers from 0 to n-1
  • 90. Documentation: style is importantdef elements(n): """Return a list of n numbers from 0 to n-1. :type n: int :param n: the limit upper for the elements to be created (notincluded). :return: a class:`list` with the items. """ return range(0,n)class Example(object): """This is the documentation of the class. Usually you do not need it :) """ def __init__(self, param): """This is the documentation of the instance type. """
  • 91. Documentation: do the work$ sphinx-quickstart
  • 92. Documentation: Bonus... abuse the docclass Command(object): """Undocumented command """class ListCommand(Command): """List command. """def show_help(command): cmd = getattr(globals()[__name__], "%sCommand" % command,None) if cmd is None: return "Invalid command" else: return cmd.__doc__print show_help("List")
  • 93. 12. Future
  • 94. Future: pypy Just in time compiler FAST! With sandboxing Best concurrency support
  • 95. Future: python3 def hello(name: str, age: int) -> str: return name print hello.__anotations__
  • 96. Future: python3 def hello(name: str, age: int) -> str: return name print hello.__anotations__ {return:<classint>,name:<classstr>,age:<classint>}
  • 97. Future: python3 from functools import lru_cache @lru_cache(maxsize=None) def fib(n): if n < 2: return n return fib(n-1)+fib(n-2)
  • 98. Applauses & questions Not necessarily in that order.