Successfully reported this slideshow.

Building a Functional Stream in Scala

6,228 views

Published on

I gave this presentation to my local Scala Meetup on Feb 12th 2014. It presents an aspect of functional programming by implementing a lazy data structure for storing an infinite collection of data. The act of building the stream is the point - you wouldn't use this in real life. The design for the stream is largely taken from Manning's "Functional Programming in Scala" by Paul Chiusano and Rúnar Bjarnason.

Have a look at the book at http://www.manning.com/bjarnason/.

Published in: Technology

Building a Functional Stream in Scala

  1. 1. Building A F am tre lS ona cti un in Scala Derek Wyatt Twitter: @derekwyatt Email: derek@derekwyatt.org
  2. 2. Structure ctional Data A Fun
  3. 3. Structure ctional Data A Fun Streams are infinite and lazy
  4. 4. Structure ctional Data A Fun Streams are infinite and lazy They can be built using functions
  5. 5. Structure ctional Data A Fun Streams are infinite and lazy They can be built using functions Designing one is fun and instructive
  6. 6. Structure ctional Data A Fun Streams are infinite and lazy They can be built using functions Designing one is fun and instructive ❊ I am not an FP expert (but I do play one in presentations). Much of what you see has been learned from Functional Programming in Scala by Paul Chiusano and Rúnar Bjarnason
  7. 7. Strictness L ziness a vs.
  8. 8. Strictness L ziness a vs. def strict(row: DBRow): Unit = { if (somecondition) // do something with row }
  9. 9. Strictness fetchRow(42) L ziness a vs. def strict(row: DBRow): Unit = { if (somecondition) // do something with row }
  10. 10. Strictness fetchRow(42) L ziness a vs. def strict(row: DBRow): Unit = { if (somecondition) // do something with row } def lazy(row: => DBRow): Unit = { if (somecondition) // do something with row }
  11. 11. Strictness fetchRow(42) L ziness a vs. se fal def strict(row: DBRow): Unit = { = ion if (somecondition) t ndi co // do something with row me so } def lazy(row: => DBRow): Unit = { if (somecondition) // do something with row }
  12. 12. Strictness fetchRow(42) evaluated L ziness a vs. se Not Evaluated fal def strict(row: DBRow): Unit = { = ion if (somecondition) t ndi co // do something with row me so } def lazy(row: => DBRow): Unit = { if (somecondition) // do something with row }
  13. 13. List: A “Strict” Data Type
  14. 14. List: A “Strict” Data Type (1 to 10).toList map { i => println(s”list -> map $i”) i + 10 } filter { i => println(s”list -> filter $i”) i % 2 == 0 } [1, 2, ... 10] toList
  15. 15. List: A “Strict” Data Type (1 to 10).toList map { i => println(s”list -> map $i”) i + 10 } filter { i => println(s”list -> filter $i”) i % 2 == 0 } [1, 2, ... 10] Map [11, 12, ... 20] toList // // // // // list list ... list list -> map 1 -> map 2 -> map 9 -> map 10
  16. 16. List: A “Strict” Data Type (1 to 10).toList map { i => println(s”list -> map $i”) i + 10 } filter { i => println(s”list -> filter $i”) i % 2 == 0 } [1, 2, ... 10] Map [11, 12, ... 20] Filter toList [12, 14, ... 20] // // // // // // // // // // list list ... list list list list ... list list -> map 1 -> map 2 -> -> -> -> map 9 map 10 10 filter 11 filter 12 -> filter 19 -> filter 20
  17. 17. ams: Lazy Ass Seqs Stre
  18. 18. ams: Lazy Ass Seqs Stre Stream(1 to 10) map { i => println(s”stream -> map $i”) i + 10 } filter { i => println(s”stream -> filter $i”) i % 2 == 0 }
  19. 19. ams: Lazy Ass Seqs Stre Stream(1 to 10) map { i => println(s”stream -> map $i”) i + 10 } filter { i => println(s”stream -> filter $i”) i % 2 == 0 } [1, ?] Stream(1 to 10)
  20. 20. ams: Lazy Ass Seqs Stre Stream(1 to 10) map { i => println(s”stream -> map $i”) i + 10 } filter { i => println(s”stream -> filter $i”) i % 2 == 0 } [1, ?] [11, ?] Stream(1 to 10) Map
  21. 21. ams: Lazy Ass Seqs Stre Stream(1 to 10) map { i => println(s”stream -> map $i”) i + 10 } filter { i => println(s”stream -> filter $i”) i % 2 == 0 } [1, ?] [11, ?] Stream(1 to 10) Map [12, ?] Filter The ?‘s are, essentially functions
  22. 22. ams: Lazy Ass Seqs Stre Stream(1 to 10) map { i => println(s”stream -> map $i”) i + 10 } filter { i => println(s”stream -> filter $i”) i % 2 == 0 } [1, ?] [11, ?] Stream(1 to 10) Map [12, ?] Filter The ?‘s are, essentially functions ative purposes only For il ustr
  23. 23. ams: Lazy Ass Seqs Stre Stream(1 to 10) map { i => println(s”stream -> map $i”) i + 10 } filter { i => println(s”stream -> filter $i”) i % 2 == 0 } [1, ?] [11, ?] Stream(1 to 10) Map [12, ?] Filter The ?‘s are, essentially functions ative purposes only For il ustr Stream transformation does not require traversal
  24. 24. ams: Lazy Ass Seqs Stre Stream(1 to 10) map { i => println(s”stream -> map $i”) i + 10 } filter { i => println(s”stream -> filter $i”) i % 2 == 0 } [1, ?] [11, ?] Stream(1 to 10) Map [12, ?] Filter The ?‘s are, essentially functions ative purposes only For il ustr Stream transformation does not require traversal Transformations are applied on-demand as traversal happens
  25. 25. Stream Definition
  26. 26. Stream Definition trait Stream[+A] { def uncons: Option[(A, Stream[A])] }
  27. 27. Stream Definition trait Stream[+A] { def uncons: Option[(A, Stream[A])] } The “data” structure
  28. 28. Stream Definition trait Stream[+A] { def uncons: Option[(A, Stream[A])] } The “data” structure Constructors object Stream { def empty[A]: Stream[A] = new Stream[A] { def uncons = None } def cons[A](hd: => A, tl: => Stream[A]): Stream[A] = new Stream[A] { def uncons = Some((hd, tl)) } }
  29. 29. Stream Definition trait Stream[+A] { def uncons: Option[(A, Stream[A])] } The “data” structure Constructors object Stream { def empty[A]: Stream[A] = new Stream[A] { def uncons = None } def cons[A](hd: => A, tl: => Stream[A]): Stream[A] = new Stream[A] { def uncons = Some((hd, tl)) } }
  30. 30. Constructing Streams
  31. 31. Constructing Streams val streamOfOne = cons(1, empty[Int]) val streamOfThree = cons(1, cons(2, cons(3, empty[Int])))
  32. 32. Constructing Streams val streamOfOne = cons(1, empty[Int]) val streamOfThree = cons(1, cons(2, cons(3, empty[Int]))) Well, that SUCKS...
  33. 33. Constructing Streams val streamOfOne = cons(1, empty[Int]) val streamOfThree = cons(1, cons(2, cons(3, empty[Int]))) Well, that SUCKS... object Stream { def apply[A](as: A*): Stream[A] = {
 if (as.isEmpty) empty[A] else cons(as.head, apply(as.tail:_*)) } }
  34. 34. Constructing Streams val streamOfOne = cons(1, empty[Int]) val streamOfThree = cons(1, cons(2, cons(3, empty[Int]))) Well, that SUCKS... val three = Stream(1, 2, 3) object Stream { def apply[A](as: A*): Stream[A] = {
 if (as.isEmpty) empty[A] else cons(as.head, apply(as.tail:_*)) } }
  35. 35. The Stream... so far
  36. 36. The Stream... so far trait Stream[+A] { def uncons: Option[(A, Stream[A])] } ! object Stream { def empty[A]: Stream[A] = new Stream[A] { def uncons = None } def cons[A](hd: => A, tl: => Stream[A]): Stream[A] = new Stream[A] { def uncons = Some((hd, tl)) } def apply[A](as: A*): Stream[A] = {
 if (as.isEmpty) empty[A] else cons(as.head, apply(as.tail:_*)) } }
  37. 37. Right (does it all) fold (1 to 4).foldRight(z)(f)
  38. 38. Right (does it all) fold (1 to 4).foldRight(z)(f) makes f(1, f(2, f(3, f(4, z))))
  39. 39. Right (does it all) fold (1 to 4).foldRight(z)(f) makes or f(1, f(2, f(3, f(4, z)))) f(1, f(1, f(1, f(1, f(1, f(1, f(1, res4 ... f(2, ... f(2, f(3, ... f(2, f(3, f(4, z)))) f(2, f(3, res1))) f(2, res2)) res3)
  40. 40. Right (does it all) fold (1 to 4).foldRight(z)(f) Builds a functional structure “to the right”, pushing successive evaluation to the second parameter. makes or Each function’s second parameter must be evaluated before its predecessor can be evaluated. f(1, f(2, f(3, f(4, z)))) f(1, f(1, f(1, f(1, f(1, f(1, f(1, res4 ... f(2, ... f(2, f(3, ... f(2, f(3, f(4, z)))) f(2, f(3, res1))) f(2, res2)) res3)
  41. 41. foldRight for Lists
  42. 42. foldRight for Lists case class List[+A](head: A, tail: List[A]) { def foldRight[B](z: B)(f: (A, B) => B): B = if (this.isEmpty) z else f(head, tail.foldRight(z)(f)) }
  43. 43. foldRight for Lists case class List[+A](head: A, tail: List[A]) { def foldRight[B](z: B)(f: (A, B) => B): B = if (this.isEmpty) z else f(head, tail.foldRight(z)(f)) } val sum // f(1, // f(1, // f(1, // f(1, // f(1, // f(1, // f(1, // 10 = (1 to 4).foldRight(0)(_ + _) tail.foldRight... f(2, tail.foldRight... f(2, f(3, tail.foldRight... f(2, f(3, f(4, 0)))) f(2, f(3, 4))) f(2, 7)) 9)
  44. 44. foldRight for Lists case class List[+A](head: A, tail: List[A]) { def foldRight[B](z: B)(f: (A, B) => B): B = if (this.isEmpty) z else f(head, tail.foldRight(z)(f)) } Here A and B are both Ints but they need not be. Note that full recursive expansion takes place at the call site. val sum // f(1, // f(1, // f(1, // f(1, // f(1, // f(1, // f(1, // 10 = (1 to 4).foldRight(0)(_ + _) tail.foldRight... f(2, tail.foldRight... f(2, f(3, tail.foldRight... f(2, f(3, f(4, 0)))) f(2, f(3, 4))) f(2, 7)) 9)
  45. 45. Strictly Folding Right
  46. 46. Strictly Folding Right No application of ‘f’ can complete until all of its parameters ℥ have been applied
  47. 47. Strictly Folding Right No application of ‘f’ can complete until all of its parameters ℥ have been applied Strictly speaking, this means that foldRight must fully ℥ expand into a deeply nested function application
  48. 48. Strictly Folding Right No application of ‘f’ can complete until all of its parameters ℥ have been applied Strictly speaking, this means that foldRight must fully ℥ expand into a deeply nested function application If the collection is infinite, or even significantly large, your ℥ application is doomed
  49. 49. Strictly Folding Right No application of ‘f’ can complete until all of its parameters ℥ have been applied Strictly speaking, this means that foldRight must fully ℥ expand into a deeply nested function application If the collection is infinite, or even significantly large, your ℥ application is doomed
  50. 50. Building foldRight
  51. 51. Building foldRight trait Stream[+A] { def foldRight[B](z: ? B)(f: (A, ? B) => B): B }
  52. 52. Building foldRight trait Stream[+A]Stream[+A] { trait { def uncons: Option[(A, Stream[A])] def foldRight[B](z: ? B)(f: (A, ? B) => B): B ! } def foldRight[B](z: => B)(f: (A, => B) => B): B = uncons match { case None => z case Some((hd, tl)) => f(hd, tl.foldRight(z)(f)) } }
  53. 53. Building foldRight trait Stream[+A]Stream[+A] { trait { def uncons: Option[(A, Stream[A])] def foldRight[B](z: ? B)(f: (A, ? B) => B): B ! } def foldRight[B](z: => B)(f: (A, => B) => B): B = uncons match { case None => z case Some((hd, tl)) => f(hd, tl.foldRight(z)(f)) } } A profound change!
  54. 54. Building foldRight trait Stream[+A]Stream[+A] { trait { def uncons: Option[(A, Stream[A])] def foldRight[B](z: ? B)(f: (A, ? B) => B): B ! } def foldRight[B](z: => B)(f: (A, => B) => B): B = uncons match { case None => z case Some((hd, tl)) => f(hd, tl.foldRight(z)(f)) } } The recursive call is no longer evaluated at the call site because ‘f’ receives it by name
  55. 55. Learning to Relax
  56. 56. Learning to Relax def foldRight[B](z: => B)(f: (A, => B) => B): B
  57. 57. Learning to Relax def foldRight[B](z: => B)(f: (A, => B) => B): B At this point, you’re potentially confused
  58. 58. Learning to Relax def foldRight[B](z: => B)(f: (A, => B) => B): B At this point, you’re potentially confused What good is it to have a lazy => B when foldRight must return a strict B?
  59. 59. Learning to Relax def foldRight[B](z: => B)(f: (A, => B) => B): B At this point, you’re potentially confused What good is it to have a lazy => B when foldRight must return a strict B? def sum(ints: Stream[Int]): Int = ints.foldRight(0)(_ + _)
  60. 60. Learning to Relax def foldRight[B](z: => B)(f: (A, => B) => B): B At this point, you’re potentially confused What good is it to have a lazy => B when foldRight must return a strict B? def sum(ints: Stream[Int]): Int = ints.foldRight(0)(_ + _) val neverGetsHere = sum(streamOfNaturalNumbers)
  61. 61. Learning to Relax ! e yp def foldRight[B](z: => B)(f: (A, => B) => B): B T ” T IC TR S At this point, you’re potentially confused “ a s What good is it to have a lazy => B when foldRight must return a strict B? I i t n def sum(ints: Stream[Int]): Int = ints.foldRight(0)(_ + _) val neverGetsHere = sum(streamOfNaturalNumbers)
  62. 62. Learning to Relax
  63. 63. Learning to Relax Ok, I kinda lied… it’s not that Int is a “strict type” (You can, of course, do something useful with an Int… such as)
  64. 64. Learning to Relax Ok, I kinda lied… it’s not that Int is a “strict type” (You can, of course, do something useful with an Int… such as) def sumToN(ints: Stream[Int], to: Int): Int = ints.take(to).foldRight(0)(_ + _)
  65. 65. Learning to Relax Ok, I kinda lied… it’s not that Int is a “strict type” (You can, of course, do something useful with an Int… such as) def sumToN(ints: Stream[Int], to: Int): Int = ints.take(to).foldRight(0)(_ + _) It’s just that the strict type can cause a bit of confusion because of its non-lazy nature.
  66. 66. Learning to Relax Ok, I kinda lied… it’s not that Int is a “strict type” (You can, of course, do something useful with an Int… such as) def sumToN(ints: Stream[Int], to: Int): Int = ints.take(to).foldRight(0)(_ + _) It’s just that the strict type can cause a bit of confusion because of its non-lazy nature. The more “interesting” stuff, though happens when you can continue beings lazy and stay within the realm of the infinite
  67. 67. Learning to Relax
  68. 68. Learning to Relax def foldRight[B](z: => B)(f: (A, => B) => B): B
  69. 69. Learning to Relax def foldRight[B](z: => B)(f: (A, => B) => B): B Switching to a lazy (by name) parameter gives ‘f’ control over the recursion
  70. 70. Learning to Relax def foldRight[B](z: => B)(f: (A, => B) => B): B Switching to a lazy (by name) parameter gives ‘f’ control over the recursion ‘f’ can decide to delay the execution of its second parameter, eliminating the recursive call, returning control to the caller
  71. 71. Learning to Relax def foldRight[B](z: => B)(f: (A, => B) => B): B Switching to a lazy (by name) parameter gives ‘f’ control over the recursion ‘f’ can decide to delay the execution of its second parameter, eliminating the recursive call, returning control to the caller However, you can’t just delay a computation without shoving it into some sort of context...
  72. 72. Learning to Relax M A E R T t! S x e e t h n T o c t a th is def foldRight[B](z: => B)(f: (A, => B) => B): B Switching to a lazy (by name) parameter gives ‘f’ control over the recursion ‘f’ can decide to delay the execution of its second parameter, eliminating the recursive call, returning control to the caller However, you can’t just delay a computation without shoving it into some sort of context...
  73. 73. A Match Made in Laziness => + Non-Strict Result = Lazy Stream
  74. 74. Enter... Map
  75. 75. Enter... Map def map[T](f: A => T): Stream[T] = foldRight(empty[T]) { (head, tail) => cons(f(head), tail) }
  76. 76. Enter... Map def map[T](f: A => T): Stream[T] = foldRight(empty[T]) { (head, tail) => cons(f(head), tail) } def foldRight[B](z: => B)(f: (A, => B) => B): B
  77. 77. Enter... Map St r ea def map[T](f: A => T): Stream[T] = foldRight(empty[T]) { (head, tail) => cons(f(head), tail) } Stream! St r ea m St r ! ea m m ! ! def foldRight[B](z: => B)(f: (A, => B) => B): B
  78. 78. Enter... Filter
  79. 79. Enter... Filter def filter(p: A => Boolean): Stream[A] = foldRight(empty[A]) { (head, tail) => if (p(head)) cons(head, tail) else tail }
  80. 80. Enter... Filter def filter(p: A => Boolean): Stream[A] = foldRight(empty[A]) { (head, tail) => if (p(head)) cons(head, tail) else tail } def foldRight[B](z: => B)(f: (A, => B) => B): B
  81. 81. Enter... Filter def filter(p: A => Boolean): Stream[A] = foldRight(empty[A]) { (head, tail) => if (p(head)) cons(head, tail) else tail St re } am Stream! ! St r ea St r ea m m ! def foldRight[B](z: => B)(f: (A, => B) => B): B !
  82. 82. Returning Streams
  83. 83. Returning Streams Both map and filter return Streams
  84. 84. Returning Streams Both map and filter return Streams Stream’s constructors eval neither head nor tail
  85. 85. Returning Streams Both map and filter return Streams Stream’s constructors eval neither head nor tail This allows for the chaining of laziness from the by name parameter of foldRight, into the return value
  86. 86. Returning Streams
  87. 87. Returning Streams def map[T](f: A => T): Stream[T] = foldRight(empty[T]) { (head, tail) =>
  88. 88. Returning Streams def map[T](f: A => T): Stream[T] = foldRight(empty[T]) { (head, tail) => Not Eval’d
  89. 89. Returning Streams def map[T](f: A => T): Stream[T] = foldRight(empty[T]) { (head, tail) => ! ! ! Not Eval’d ! Not Eval’d ! cons(f(head), tail) }
  90. 90. Returning Streams def map[T](f: A => T): Stream[T] = foldRight(empty[T]) { (head, tail) => ! ! ! Not Eval’d Not Eval’d ! Not Eval’d ! cons(f(head), tail) }
  91. 91. Returning Streams def map[T](f: A => T): Stream[T] = foldRight(empty[T]) { (head, tail) => ! ! ! Not Eval’d Not Eval’d ! Not Eval’d ! cons(f(head), tail) } Jeez, does this code even do anything!?
  92. 92. Let’s find Out...
  93. 93. Let’s find Out... Stream(1, 2, 3, 4, 5) map { i => println(s"map -> $i") i * i }
  94. 94. Let’s find Out... Stream(1, 2, 3, 4, 5) map { i => println(s"map -> $i") i * i } Which prints... def map[T](f: A => T): Stream[T] = foldRight(empty[T]) { (head, tail) => cons(f(head), tail) } Remember that... and... def foldRight[B](z: => B)(f: (A, => B) => B): B def cons[A](hd: => A, tl: => Stream[A]): Stream[A] and...
  95. 95. This page intentionally left blank
  96. 96. Putting it All Together
  97. 97. Putting it All Together val s = Stream(1 to 10) map { i => println(s”stream -> map $i”) i + 10 } filter { i => println(s”stream -> filter $i”) i % 2 == 0 }
  98. 98. Putting it All Together val s = Stream(1 to 10) map { i => println(s”stream -> map $i”) i + 10 } filter { i => println(s”stream -> filter $i”) i % 2 == 0 } Prints nothing. OK
  99. 99. Putting it All Together val s = Stream(1 to 10) map { i => println(s”stream -> map $i”) i + 10 } filter { i => println(s”stream -> filter $i”) i % 2 == 0 } def toList: List[A] = uncons match { case None => Nil case Some((h, t)) => h :: t.toList } Prints nothing. OK add toList()
  100. 100. Evaluating
  101. 101. Evaluating val numList = s.toList
  102. 102. Evaluating val numList = s.toList // // // // // // // stream stream stream stream ... stream stream -> -> -> -> map 1 filter 11 map 2 filter 12 -> map 10 -> filter 20
  103. 103. Evaluating val numList = s.toList def take(n: Int): Stream[A] = uncons match { case Some((h, t)) if n > 0 => cons(h, t.take(n - 1)) case _ => empty } // // // // // // // stream stream stream stream ... stream stream -> -> -> -> map 1 filter 11 map 2 filter 12 -> map 10 -> filter 20
  104. 104. Evaluating val numList = s.toList def take(n: Int): Stream[A] = uncons match { case Some((h, t)) if n > 0 => cons(h, t.take(n - 1)) case _ => empty } val numList = s.take(1).toList // // // // // // // stream stream stream stream ... stream stream -> -> -> -> map 1 filter 11 map 2 filter 12 -> map 10 -> filter 20
  105. 105. Evaluating val numList = s.toList def take(n: Int): Stream[A] = uncons match { case Some((h, t)) if n > 0 => cons(h, t.take(n - 1)) case _ => empty } val numList = s.take(1).toList // // // // // // // stream stream stream stream ... stream stream // // // // -> -> -> -> map 1 filter 11 map 2 filter 12 -> map 10 -> filter 20 stream stream stream stream -> -> -> -> map 1 filter 11 map 2 filter 12
  106. 106. It’s All About Functions
  107. 107. It’s All About Functions Functions hold the values
  108. 108. It’s All About Functions Functions hold the values Higher order functions pile functions on functions
  109. 109. It’s All About Functions Functions hold the values Higher order functions pile functions on functions Even functions like take() merely store functions
  110. 110. It’s All About Functions Functions hold the values Higher order functions pile functions on functions Even functions like take() merely store functions Nothing “real” happens until you need it to happen
  111. 111. It’s All About Functions Functions hold the values Higher order functions pile functions on functions Even functions like take() merely store functions Nothing “real” happens until you need it to happen ! y in h S
  112. 112. ms rea St ss yA az L i Brought to you Sincere Couch Potato em s Derek Wyatt Twitter: @derekwyatt Email: derek@derekwyatt.org

×