Austin Bingham. Transducers in Python. PyCon Belarus

1,805 views

Published on

Understanding Transducers Through Python – Transducers are a new and interesting functional programming concept that comes from the world of Clojure. In this talk we’ll learn about transducers by seeing how to implement them in Python. By using transducers to build familiar functional programming elements like map and filter, we’ll see that transducers are actually simple, elegant, and quite powerful.

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,805
On SlideShare
0
From Embeds
0
Number of Embeds
667
Actions
Shares
0
Downloads
10
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Austin Bingham. Transducers in Python. PyCon Belarus

  1. 1. @sixty_north Understanding Transducers Through Python 1 Austin Bingham @austin_bingham Sunday, January 25, 15
  2. 2. 237 “Transducers are coming” Rich Hickeyhttp://blog.cognitect.com/blog/2014/8/6/transducers-are-coming Photo: Howard Lewis Ship under CC-BY Sunday, January 25, 15
  3. 3. Functions which transform reducers What is a transducer? ‣ Reducer (reducing function) • Any function we can pass to reduce(reducer, iterable[, initial]) • (result, input) → result • add(result, input) reduce(add, [1, 2, 3], 0) → 6 ‣ Transducer (transform reducer) • A function which accepts a reducer, and transforms it in some way, and returns a new reducer • ((result, input) → result) → ((result, input) → result) 3 Sunday, January 25, 15
  4. 4. Transducers are a functional programming technique not restricted to Clojure How does this relate to Python? ‣ Clojure implementation is the prototype/archetype • Heavily uses anonymous functions • Uses overloads on function arity for disparate purposes • Complected with existing approaches ‣ Python implementation is pedagogical (but also useful) • Explicit is better than implicit • Readability counts! • Has all the functional tools we need for transducers • I’m a better Pythonista than Clojurist 4 Sunday, January 25, 15
  5. 5. 5 Review functional programming tools in Sunday, January 25, 15
  6. 6. 6 my_filter(is_prime, my_map(prime_factors, range(32) ) ) iterable sequence sequence Sunday, January 25, 15
  7. 7. 7 def my_map(transform, iterable): def map_reducer(sequence, item): sequence.append(transform(item)) return sequence return reduce(map_reducer, iterable, []) def my_filter(predicate, iterable): def filter_reducer(sequence, item): if predicate(item): sequence.append(item) return sequence return reduce(filter_reducer, iterable, []) Sunday, January 25, 15
  8. 8. 7 def my_map(transform, iterable): def map_reducer(sequence, item): sequence.append(transform(item)) return sequence return reduce(map_reducer, iterable, []) def my_filter(predicate, iterable): def filter_reducer(sequence, item): if predicate(item): sequence.append(item) return sequence return reduce(filter_reducer, iterable, []) reduce() Sunday, January 25, 15
  9. 9. 7 def my_map(transform, iterable): def map_reducer(sequence, item): sequence.append(transform(item)) return sequence return reduce(map_reducer, iterable, []) def my_filter(predicate, iterable): def filter_reducer(sequence, item): if predicate(item): sequence.append(item) return sequence return reduce(filter_reducer, iterable, []) reduce() sequence.append() Sunday, January 25, 15
  10. 10. 7 def my_map(transform, iterable): def map_reducer(sequence, item): sequence.append(transform(item)) return sequence return reduce(map_reducer, iterable, []) def my_filter(predicate, iterable): def filter_reducer(sequence, item): if predicate(item): sequence.append(item) return sequence return reduce(filter_reducer, iterable, []) reduce() sequence.append() Empty list : ‘seed’ must be a mutable sequence Sunday, January 25, 15
  11. 11. 8 >>> reduce(make_mapper(square), range(10), []) [0, 1, 4, 9, 16, 25, 36, 49, 64, 81] >>> reduce(make_filterer(is_prime), range(100), []) [2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47, 53, 59, 61, 67, 71, 73, 79, 83, 89, 97] ‣ make_mapper() and make_filterer() are not composable ‣ to square primes we would need to call reduce() twice ‣ this requires we store the intermediate sequence ‣ the reducers returned by make_mapper() and make_filterer() depend on the seed being a mutable sequence e.g. a list Sunday, January 25, 15
  12. 12. 9 (defn map ([f] (fn [rf] (fn ([] (rf)) ([result] (rf result)) ([result input] (rf result (f input))) ([result input & inputs] (rf result (apply f input inputs)))))) Sunday, January 25, 15
  13. 13. 10 (defn map ;; The tranducer factory... ([f] ;; ...accepts a single argument 'f', the transforming function (fn [rf] ;; The transducer function accepts a reducing function 'rf' (fn ;; This is the reducing function returned by the transducer ([] (rf)) ;; 0-arity : Forward to the zero-arity reducing function 'rf' ([result] (rf result)) ;; 1-arity : Forward to the one-arity reducing function 'rf' ([result input] ;; 2-arity : Perform the reduction with one arg to 'f' (rf result (f input))) ([result input & inputs] ;; n-arity : Perform the reduction with multiple args to 'f' (rf result (apply f input inputs)))))) Sunday, January 25, 15
  14. 14. 11 (defn map ;; The tranducer factory... ([f] ;; ...accepts a single argument 'f', the transforming function (fn [rf] ;; The transducer function accepts a reducing function 'rf' (fn ;; This is the reducing function returned by the transducer ([] (rf)) ;; 0-arity : Return a 'seed' value obtained from 'rf' ([result] (rf result)) ;; 1-arity : Obtain final result from 'rf' and clean-up ([result input] ;; 2-arity : Perform the reduction with one arg to 'f' (rf result (f input))) ([result input & inputs] ;; n-arity : Perform the reduction with multiple args to 'f' (rf result (apply f input inputs)))))) Sunday, January 25, 15
  15. 15. 12 (defn map ;; The tranducer factory... ([f] ;; ...accepts a single argument 'f', the transforming function (fn [rf] ;; The transducer function accepts a reducing function 'rf' (fn ;; This is the reducing function returned by the transducer ([] (rf)) ;; 0-arity : Return a 'seed' value obtained from 'rf' ([result] (rf result)) ;; 1-arity : Obtain final result from 'rf' and clean-up ([result input] ;; 2-arity : Perform the reduction with one arg to 'f' (rf result (f input))) ([result input & inputs] ;; n-arity : Perform the reduction with multiple args to 'f' (rf result (apply f input inputs)))))) To fully implement Clojure’s transducers in Python we also need: ‣ explicit association of the seed value with the reduction operation ‣ support for early termination without reducing the whole series ‣ reduction to a final value and opportunity to clean up state Sunday, January 25, 15
  16. 16. 13 class Reducer: def __init__(self, reducer): # Construct from reducing function pass def initial(self): # Return the initial seed value pass # 0-arity def step(self, result, item): # Next step in the reduction pass # 2-arity def complete(self, result): # Produce a final result and clean up pass # 1-arity Sunday, January 25, 15
  17. 17. 13 class Reducer: def __init__(self, reducer): # Construct from reducing function pass def initial(self): # Return the initial seed value pass # 0-arity def step(self, result, item): # Next step in the reduction pass # 2-arity def complete(self, result): # Produce a final result and clean up pass # 1-arity new_reducer = Reducer(reducer) Sunday, January 25, 15
  18. 18. 13 class Reducer: def __init__(self, reducer): # Construct from reducing function pass def initial(self): # Return the initial seed value pass # 0-arity def step(self, result, item): # Next step in the reduction pass # 2-arity def complete(self, result): # Produce a final result and clean up pass # 1-arity new_reducer = Reducer(reducer) def transducer(reducer): return Reducer(reducer) Sunday, January 25, 15
  19. 19. 13 class Reducer: def __init__(self, reducer): # Construct from reducing function pass def initial(self): # Return the initial seed value pass # 0-arity def step(self, result, item): # Next step in the reduction pass # 2-arity def complete(self, result): # Produce a final result and clean up pass # 1-arity new_reducer = Reducer(reducer) def transducer(reducer): return Reducer(reducer) ⇐ two names for two concepts Sunday, January 25, 15
  20. 20. Python Transducer implementations 14 Cognitect https://github.com/cognitect-labs PyPI: transducers • “official” • Python in the style of Clojure • Only eager iterables (step back from regular Python) • Undesirable Python practices such as hiding built-ins • Also Clojure, Javascript, Ruby, etc. Sixty North http://code.sixty-north.com/python- transducers PyPI: transducer • Pythonic • Eager iterables • Lazy iterables • Co-routine based ‘push’ events • Pull-requests welcome! Sunday, January 25, 15
  21. 21. 15 Thank you! @sixty_north Austin Bingham @austin_bingham http://sixty-north.com/blog/ Sunday, January 25, 15

×