Applying functional with Python
Is it possible?
About me
● Position: Data Engineer at 99
● Production experience: Python, Java, Scala, Go and NodeJS
● Looking at: Erlang and Elixir
● twitter.com/@jesuejunior
● github.com/jesuejunior
Agenda
● OOP vs FP (2min)
● About Functional (3min)
● Imperative vs Functional (8min)
● Functions(2min)
● High-Order Functions(10min)
● Lazy Evaluation(8min)
● What did we miss?(4min)
● Next steps?(1min)
OOP vs FP
Psychology
Object-Oriented Programming (Imperative style)
● Often matches with human style of thinking
● Humans automatically parse the world into objects
● Any well-defined something that has a set of properties
● Just like you mentally divide the world into objects
Object-Oriented Programming
Functional Programming
● Does not (necessarily) divide code into objects
● And can lead to unintuitive code
● In cases where this doesn't match our Intuition
Functional Programming
Functional Programming
About Functional
About Functional
● Avoid state
● Immutable data
● First-class functions
● Higher-order functions
● Pure functions
● Recursion, tail recursion
● Iterators, sequences, lazy evaluation, pattern matching, monads....
Imperative mode
vs
Functional Mode
Imperative Mode
expr, res = "28+32+++32++39", 0
for t in expr.split("+"):
if t != "":
res += int(t)
print(res)
Imperative Mode
"28+32+++32++39", 0
"28", 0
"32", 28
"", 60
"", 60
"32", 60
"", 92
"39", 92
131
Functional Mode
from operator import add
expr = "28+32+++32++39"
res = reduce(add, map(int, filter(bool, expr.split("+")))
print(res)
Functional Mode
"28+32+++32++39"
["28","32","","","32","","39"]
["28","32","32","39"]
[28,32,32,39]
131
MAP/REDUCE/FILTER
● Readability VS. conciseness
● Technical aspects VS. operation substance
● Code reusage ("pluggability")
Python Hints
Python Hints - operator
In [3]: import operator
#Equivalent to 5 + 3
In [4]: operator.add(5, 3)
Out[4]: 8
#Equivalent to 2 < 5
In [5]: operator.lt(2,5)
Out[5]: True
#Equivalent to [1,2,3][1]
In [6]: operator.itemgetter(1)([1,2,3])
Out[6]: 2
#Equivalent to 3 ** 3
In [7]: operator.pow(2,3)
Out[7]: 8
Can we avoid loops?
In [17]: name = None
In [18]: while name is None:
...: name = input()
...: if len(name) < 2:
...: name = None
...:
In [25]: def get_name():
...: name = input()
...: return name if len(name) >= 2 else get_Name()
...:
Recursion
Imperative
Tail Recursion
Elixir Python
So sorry...
defmodule Fib do
def fib(0) do 0 end
def fib(1) do 1 end
def fib(n) do
fib(n-1) + fib(n-2)
end
end
IO.puts Fib.fib(10)
Functions
First Class
def add(a,b):
return a + b
add = lambda a,b: a + b
def calculations(a, b):
def add():
return a + b
return a, b, add
High-Order Function
Function as argument
map(lambda x: x^2, [1,2,3,4,5])
Function return function as result
def fill(topic):
print("My baloon is " + topic)
def timer(fn):
def inner(*args, **kwargs):
t = time()
fn(*args, **kwargs)
print("took {time}".format(time=time()-t))
return inner
filler = timer(fill)
filler("FP with Python")
I've already saw this...
@timer
def fill(topic):
print("My baloon is " + topic)
fill("FP with Python")
Partial Function Application
“ The process of fixing a number of
arguments to a function, producing
another function of smaller arity ”
Partial Function Application
import functools
def funcs(a, b, c):
return a + b + c
fp = functools.partial(funcs, c=3)
print(fp(1,2))
Currying
“ The technique of transforming a function
that takes multiple arguments in such a
way that it can be called as a chain of
functions each with a single argument ”
Currying
Currying
def compose(*funcs):
"""
Return a new function s.t.
compose(f,g,...)(x) == f(g(...(x)))
"""
def inner(data, funcs=funcs):
result = data
for f in reversed(funcs):
result = f(result)
return result
return inner
# >>> times2 = lambda x: x*2
# >>> minus3 = lambda x: x-3
# >>> mod6 = lambda x: x%6
# >>> f = compose(mod6, times2, minus3)
# >>> all(f(i)==((i-3)*2)%6 for i in range(1000000))
# True
Currying ( Standard Library)
In [32]: from operator import itemgetter
In [33]: itemgetter(3)([1,2,3,4,5])
Out[33]: 4
In [34]: from operator import attrgetter as attr
In [35]: class Balloon:
...: def __init__(self, name):
...: self.name = "[name] " + name
...:
In [36]: azul = Balloon('Azul')
In [37]: attr('name')(azul)
Out[37]: '[name] Azul'
Lazy Evaluation
List Comprehension
In [43]: res = [x**2 for x in range(10)]
...: print(res)
[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
List Comprehension (TIME)
python -m cProfile -s cumtime not_lazy_ex.py
3 function calls in 13.734 seconds
Ordered by: cumulative time
ncalls tottime percall cumtime percall filename:lineno(function)
1 11.936 11.936 13.734 13.734 partial_ex.py:33(<module>)
1 1.798 1.798 1.798 1.798 {range}
1 0.000 0.000 0.000 0.000 {method 'disable' of
'_lsprof.Profiler' objects}
List Comprehension (GENEXP)
In [44]: res = (x**2 for x in range(10))
...: print(res)
<generator object <genexpr> at 0x10fd98e60>
In [45]: for i in res: print(i, end=" ")
0 1 4 9 16 25 36 49 64 81
List Comprehension (GENEXP)
python -m cProfile -s cumtime lazy_ex.py
3 function calls in 1.812 seconds
Ordered by: cumulative time
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.000 0.000 1.812 1.812 partial_ex.py:30(<module>)
1 1.812 1.812 1.812 1.812 {range}
1 0.000 0.000 0.000 0.000 {method 'disable' of
'_lsprof.Profiler' objects}
Iterable
from collections.abc import Iterable
class Fibonacci(Iterable):
def __init__(self):
self.a, self.b = 0, 1
self.total = 0
def __iter__(self):
return self
def __next__(self):
self.a, self.b = self.b, self.a + self.b
self.total += self.a
return self.a
def running_sum(self):
return self.total
Iterable
>>> fib = Fibonacci()
>>> fib.running_sum()
0
>>> for _, i in zip(range(10), fib):
... print(i, end=" ")
...
1 1 2 3 5 8 13 21 34 55
>>> fib.running_sum()
143
>>> next(fib)
89
Python & FP
Python & FP
PRO
● Functions as first-­class citizens
● Lambda
● Standard library: map/filter/reduce, itertools, operator,
functools
● Generators can be used for lazy­ evaluation (in some
cases)
Python & FP
CON
● Impossible to separate pure / non­ pure
● Mutable variables
● Costly mem copy operations
● Imperative style for cycles
● No optimization for tail recursion
What did we miss?
What did we miss?
ALMOST EVERYTHING
● Errors handling without exceptions
● Pattern matching
● Message passing
● Functional data structures
● Custom data types
● Immutable and mutable
What is the next?
● multipledispatch - A relatively sane approach to multiple dispatch in Python.
○ https://github.com/mrocklin/multipledispatch
● pyrsistent - Contains a number of immutable collections.
○ https://github.com/tobgu/pyrsistent
● toolz - Provides a set of utility functions for iterators, functions and dictionaries.
○ https://github.com/pytoolz/toolz
● hypothesis - Is a library for creating unit tests for finding edge cases in your code you wouldn't have thought to look for.
○ https://github.com/HypothesisWorks/hypothesis-python
● more_itertools - Tries to collect useful compositions of iterators that neither itertools nor the recipes included in its docs address.
○ https://github.com/erikrose/more-itertools
We're hiring
Join our team
carreiras.99app.com
Functional python

Functional python

  • 1.
    Applying functional withPython Is it possible?
  • 2.
    About me ● Position:Data Engineer at 99 ● Production experience: Python, Java, Scala, Go and NodeJS ● Looking at: Erlang and Elixir ● twitter.com/@jesuejunior ● github.com/jesuejunior
  • 3.
    Agenda ● OOP vsFP (2min) ● About Functional (3min) ● Imperative vs Functional (8min) ● Functions(2min) ● High-Order Functions(10min) ● Lazy Evaluation(8min) ● What did we miss?(4min) ● Next steps?(1min)
  • 4.
  • 5.
    Object-Oriented Programming (Imperativestyle) ● Often matches with human style of thinking ● Humans automatically parse the world into objects ● Any well-defined something that has a set of properties ● Just like you mentally divide the world into objects
  • 6.
  • 7.
    Functional Programming ● Doesnot (necessarily) divide code into objects ● And can lead to unintuitive code ● In cases where this doesn't match our Intuition
  • 8.
  • 9.
  • 10.
  • 11.
    About Functional ● Avoidstate ● Immutable data ● First-class functions ● Higher-order functions ● Pure functions ● Recursion, tail recursion ● Iterators, sequences, lazy evaluation, pattern matching, monads....
  • 12.
  • 13.
    Imperative Mode expr, res= "28+32+++32++39", 0 for t in expr.split("+"): if t != "": res += int(t) print(res)
  • 14.
    Imperative Mode "28+32+++32++39", 0 "28",0 "32", 28 "", 60 "", 60 "32", 60 "", 92 "39", 92 131
  • 15.
    Functional Mode from operatorimport add expr = "28+32+++32++39" res = reduce(add, map(int, filter(bool, expr.split("+"))) print(res)
  • 16.
  • 17.
    MAP/REDUCE/FILTER ● Readability VS.conciseness ● Technical aspects VS. operation substance ● Code reusage ("pluggability")
  • 18.
  • 19.
    Python Hints -operator In [3]: import operator #Equivalent to 5 + 3 In [4]: operator.add(5, 3) Out[4]: 8 #Equivalent to 2 < 5 In [5]: operator.lt(2,5) Out[5]: True #Equivalent to [1,2,3][1] In [6]: operator.itemgetter(1)([1,2,3]) Out[6]: 2 #Equivalent to 3 ** 3 In [7]: operator.pow(2,3) Out[7]: 8
  • 20.
    Can we avoidloops? In [17]: name = None In [18]: while name is None: ...: name = input() ...: if len(name) < 2: ...: name = None ...: In [25]: def get_name(): ...: name = input() ...: return name if len(name) >= 2 else get_Name() ...: Recursion Imperative
  • 21.
    Tail Recursion Elixir Python Sosorry... defmodule Fib do def fib(0) do 0 end def fib(1) do 1 end def fib(n) do fib(n-1) + fib(n-2) end end IO.puts Fib.fib(10)
  • 22.
  • 23.
    First Class def add(a,b): returna + b add = lambda a,b: a + b def calculations(a, b): def add(): return a + b return a, b, add
  • 24.
  • 25.
    Function as argument map(lambdax: x^2, [1,2,3,4,5])
  • 26.
    Function return functionas result def fill(topic): print("My baloon is " + topic) def timer(fn): def inner(*args, **kwargs): t = time() fn(*args, **kwargs) print("took {time}".format(time=time()-t)) return inner filler = timer(fill) filler("FP with Python")
  • 27.
    I've already sawthis... @timer def fill(topic): print("My baloon is " + topic) fill("FP with Python")
  • 28.
    Partial Function Application “The process of fixing a number of arguments to a function, producing another function of smaller arity ”
  • 29.
    Partial Function Application importfunctools def funcs(a, b, c): return a + b + c fp = functools.partial(funcs, c=3) print(fp(1,2))
  • 30.
    Currying “ The techniqueof transforming a function that takes multiple arguments in such a way that it can be called as a chain of functions each with a single argument ”
  • 31.
  • 32.
    Currying def compose(*funcs): """ Return anew function s.t. compose(f,g,...)(x) == f(g(...(x))) """ def inner(data, funcs=funcs): result = data for f in reversed(funcs): result = f(result) return result return inner # >>> times2 = lambda x: x*2 # >>> minus3 = lambda x: x-3 # >>> mod6 = lambda x: x%6 # >>> f = compose(mod6, times2, minus3) # >>> all(f(i)==((i-3)*2)%6 for i in range(1000000)) # True
  • 33.
    Currying ( StandardLibrary) In [32]: from operator import itemgetter In [33]: itemgetter(3)([1,2,3,4,5]) Out[33]: 4 In [34]: from operator import attrgetter as attr In [35]: class Balloon: ...: def __init__(self, name): ...: self.name = "[name] " + name ...: In [36]: azul = Balloon('Azul') In [37]: attr('name')(azul) Out[37]: '[name] Azul'
  • 34.
  • 35.
    List Comprehension In [43]:res = [x**2 for x in range(10)] ...: print(res) [0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
  • 36.
    List Comprehension (TIME) python-m cProfile -s cumtime not_lazy_ex.py 3 function calls in 13.734 seconds Ordered by: cumulative time ncalls tottime percall cumtime percall filename:lineno(function) 1 11.936 11.936 13.734 13.734 partial_ex.py:33(<module>) 1 1.798 1.798 1.798 1.798 {range} 1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects}
  • 37.
    List Comprehension (GENEXP) In[44]: res = (x**2 for x in range(10)) ...: print(res) <generator object <genexpr> at 0x10fd98e60> In [45]: for i in res: print(i, end=" ") 0 1 4 9 16 25 36 49 64 81
  • 38.
    List Comprehension (GENEXP) python-m cProfile -s cumtime lazy_ex.py 3 function calls in 1.812 seconds Ordered by: cumulative time ncalls tottime percall cumtime percall filename:lineno(function) 1 0.000 0.000 1.812 1.812 partial_ex.py:30(<module>) 1 1.812 1.812 1.812 1.812 {range} 1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects}
  • 39.
    Iterable from collections.abc importIterable class Fibonacci(Iterable): def __init__(self): self.a, self.b = 0, 1 self.total = 0 def __iter__(self): return self def __next__(self): self.a, self.b = self.b, self.a + self.b self.total += self.a return self.a def running_sum(self): return self.total
  • 40.
    Iterable >>> fib =Fibonacci() >>> fib.running_sum() 0 >>> for _, i in zip(range(10), fib): ... print(i, end=" ") ... 1 1 2 3 5 8 13 21 34 55 >>> fib.running_sum() 143 >>> next(fib) 89
  • 41.
  • 42.
    Python & FP PRO ●Functions as first-­class citizens ● Lambda ● Standard library: map/filter/reduce, itertools, operator, functools ● Generators can be used for lazy­ evaluation (in some cases)
  • 43.
    Python & FP CON ●Impossible to separate pure / non­ pure ● Mutable variables ● Costly mem copy operations ● Imperative style for cycles ● No optimization for tail recursion
  • 44.
  • 45.
    What did wemiss? ALMOST EVERYTHING ● Errors handling without exceptions ● Pattern matching ● Message passing ● Functional data structures ● Custom data types ● Immutable and mutable
  • 46.
    What is thenext? ● multipledispatch - A relatively sane approach to multiple dispatch in Python. ○ https://github.com/mrocklin/multipledispatch ● pyrsistent - Contains a number of immutable collections. ○ https://github.com/tobgu/pyrsistent ● toolz - Provides a set of utility functions for iterators, functions and dictionaries. ○ https://github.com/pytoolz/toolz ● hypothesis - Is a library for creating unit tests for finding edge cases in your code you wouldn't have thought to look for. ○ https://github.com/HypothesisWorks/hypothesis-python ● more_itertools - Tries to collect useful compositions of iterators that neither itertools nor the recipes included in its docs address. ○ https://github.com/erikrose/more-itertools
  • 47.
    We're hiring Join ourteam carreiras.99app.com