Successfully reported this slideshow.
Upcoming SlideShare
×

# Blazing Fast, Pure Effects without Monads — LambdaConf 2018

Effect monads like IO are the way functional programmers interact with the real world. Yet, monadic effects in programming languages like Scala often perform poorly compared to their Haskell counterparts—as much as 10x slower in some benchmarks. In this presentation, John A. De Goes, author of the Scalaz 8 effect system, dredges up an old paper to cast new light on the question of how to model effects, and comes to the surprising conclusion that in Scala, monads may not be the fastest way to model purely functional effects. Join John as he shows a new model of effects that offers performance improvements without sacrificing the wonderful purity that functional programmers rely on to reason about their software.

• Full Name
Comment goes here.

Are you sure you want to Yes No

### Blazing Fast, Pure Effects without Monads — LambdaConf 2018

1. 1. 1 Blazing Fast, Pure Effects without Monads LambdaConf 2018 By John A. De Goes — @jdegoes
2. 2. 2 1991 1998 2018 - April Eugenio Moggi John Hughes Flavio Brasil F R O M M O N A D S T O A R R O W S A brief history of effects leading to KleisliIO. KleisliIO 2018 - June Notions of computation and monads Generalizing monads to arrows TraneIO: Arrows & tasks in Scala
3. 3. 3 THE TRINITY OF FP Total Deterministic Free of Side-effects
4. 4. 4 THE TRINITY OF FUNCTIONAL PROGRAMMING Total Deterministic Free of Side-effects f : A => B a ∈ A ⇒ f(a) ∈ B
5. 5. 5 THE TRINITY OF FUNCTIONAL PROGRAMMING Total Deterministic Free of Side-effects def notTotal1(a: Int): String = null def notTotal2(a: Int): String = throw new Error(" ") def notTotal3(a: Int): String = notTotal3(a)
6. 6. 6 THE TRINITY OF FUNCTIONAL PROGRAMMING Total Deterministic Free of Side-effects f : A => B b1 = f(a) ∧ b2 = f(a) ⇒ b1 = b2
7. 7. 7 THE TRINITY OF FUNCTIONAL PROGRAMMING Total Deterministic Free of Side-effects def notDeterministic1(a: Int): String = if (Math.random() > 0.5) "Hello" else "Goodbye" def notDeterministic2(a: Int): String = scala.io.StdIn.readLine() def notDeterministic3(a: Int): String = (System.nanoTime() + a).toString
8. 8. 8 THE TRINITY OF FUNCTIONAL PROGRAMMING Total Deterministic Free of Side-effects f : A => B f’s only computational effect is computing B
9. 9. 9 THE TRINITY OF FUNCTIONAL PROGRAMMING Total Deterministic Free of Side-effects def notEffectFree1(a: Int): String = { println("Hello"); a.toString } def notEffectFree2(a: Int): String = { val _ = scala.io.StdIn.readLine() a.toString } def notEffectFree3(a: Int): String = { Logger.log(a); (a * a).toString }
10. 10. 10 THE RICHES OF PURITY Type Reasoning Equality Reasoning
11. 11. 11 Type Reasoning Equality Reasoning def foo1[A](a: A): A def foo2[A](a: A, f: A => A): A def foo3[A, B](f: (A => A) => B): B THE RICHES OF PURITY
12. 12. 12 Type Reasoning Equality Reasoning def println(line: String): Unit def readLine(): String THE RICHES OF PURITY
13. 13. 13 Type Reasoning Equality Reasoning def f = (x: Int) => x * x val b = f(a) val c = b + b // c = b + b // c = f(a) + f(a) // c = (a * a) + (a * a) // c = 2 * a * a THE RICHES OF PURITY
14. 14. 14 Type Reasoning Equality Reasoning def readLine = scala.io.StdIn.readLine() val a = readLine val b = a + a // b = a + a // b = readLine + readLine // !?!?! readLine + readLine != // { val a = readLine; a + a } THE RICHES OF PURITY
15. 15. 15 MONADIC EFFECTS Effects Into Values Values Into Effects The Cost of Value Effects
16. 16. 16
17. 17. 17 EFFECTS INTO VALUES def println(line: String): Unit println IO[Unit] String def println(line: String): IO[Unit]
18. 18. 18 EFFECTS INTO VALUES IO[A] Immutable value produced by the effect Immutable value describing the effect
19. 19. 19 EFFECTS INTO VALUES sealed trait IO[A] { def map[B](f: A => B): IO[B] = flatMap((a: A) => IO.point(f(a))) def flatMap[B](f: A => IO[B]): IO[B] = FlatMap(this, f) } object IO { def point[A](a: A): IO[A] = Point(a) } final case class PrintLn(line: String) extends IO[Unit] final case class ReadLine() extends IO[String] final case class FlatMap[A, B](fa: IO[A], f: A => IO[B]) extends IO[B] final case class Point[A](a: A) extends IO[A]
20. 20. 20 def readLine(): String def println(line: String): Unit val name = readLine() println("Hello " + name + ", how are you?") EFFECTS INTO VALUES println IO[Unit] def readLine: IO[String] def println(line: String): IO[Unit] val program = for { name <- readLine _ <- println("Hello " + name + ", how are you?") } yield () readLine IO[String] String Unit
21. 21. 21 VALUES INTO EFFECTS def unsafePerformIO(io: IO[A]): A = io match { case PrintLn(line) => println(line) case ReadLine() => readLine() case FlatMap(fa, f) => unsafePerformIO(f(unsafePerformIO(fa))) case Point(a) => a }
22. 22. 22 THE COST OF VALUE EFFECTS println F[Unit] class SS { for { name <- readLine _ <- println("Hello " + name + ", how are you?") } yield () readLine F[String] String Unit
23. 23. 23 THE COST OF VALUE EFFECTS readLine.flatMap(name => printLn("Hello, " + name + ", how are you?")) Allocations Allocation Megamorphic Dispatch
24. 24. 24 THE COST OF VALUE EFFECTS 1 Statement in Procedural Programming 4 Extra Allocations, 1 Extra Megamorphic Dispatch in Functional Programming =
25. 25. 25 IO.sync { App.main() } THE COST OF VALUE EFFECTS Monolith Composed Pure Values
26. 26. 26 THE COST OF VALUE EFFECTS Future
27. 27. 27 KLEISLI EFFECTS Effects Into Values Values Into Effects The Cost of Value Effects
28. 28. 28
29. 29. 29 EFFECTS INTO VALUES def println(line: String): Unit println FunctionIO[String, Unit] val println: FunctionIO[String, Unit]
30. 30. 30 EFFECTS INTO VALUES FunctionIO[A, B] Immutable input value to the function Immutable value describing the effectful function Immutable output value from the function
31. 31. 31 EFFECTS INTO VALUES final case class FunctionIO[A, B](apply0: A => IO[B]) extends (A => IO[B]) { def apply(a: A): IO[B] = apply0(a) def andThen[C](f: FunctionIO[B, C]): FunctionIO[A, C] = FunctionIO(a => apply(a).flatMap(f.apply)) } object FunctionIO { def lift[A, B](f: A => B): FunctionIO[A, B] = FunctionIO(IO.point.compose(f)) }
32. 32. 32 def readLine(): String def println(line: String): Unit val name = readLine() println("Hello " + name + ", how are you?") EFFECTS INTO VALUES val readLine: FunctionIO[Unit, String] val println: FunctionIO[String, Unit] val program = readLine .andThen(lift((name: String) => "Hello, " + name + ", how are you?")) .andThen(println) readLine FunctionIO[Unit, String] <anonymous> FunctionIO[String,String] println FunctionIO[String, Unit]
33. 33. 33 VALUES INTO EFFECTS val program: FunctionIO[Unit, Unit] unsafePerformIO(program())
34. 34. 34 THE COST OF EFFECT VALUES readLine.andThen(lift((name: String) => "Hello, " + name + ", how are you?")) .andThen(println) readLine FunctionIO[Unit, String] <anonymous> FunctionIO[String,String] println FunctionIO[String, Unit]
35. 35. 35 Megamorphic Dispatches final case class FunctionIO[A, B](apply: A => IO[B]) THE COST OF EFFECT VALUES readLine.andThen(println) Allocations Allocations
36. 36. 36 THE COST OF EFFECT VALUES 1 Statement in Procedural Programming 6 Extra Allocations, 3 Extra Megamorphic Dispatches in Functional Programming =
37. 37. 37 THE PROMISE OF KLEISLIIO sealed trait FunctionIO[A, B] extends (A => IO[B]) { ... } final class Pure[A, B](val apply0: A => IO[B]) extends FunctionIO[A, B] { def apply(a: A): IO[B] = apply0(a) } final class Impure[A, B](val apply0: A => B) extends FunctionIO[A, B] { def apply(a: A): IO[B] = IO.point(apply0(a)) } val readLine: FunctionIO[Unit, String] = new Impure(_ => scala.io.StdIn.readLine()) val printLn: FunctionIO[String, Unit] = new Impure(println)
38. 38. 38 THE PROMISE OF KLEISLIIO class Impure[A, B](val apply0: A => B) 1 Invocation in Procedural Programming 0 Extra Allocations, 1 Extra Dispatch in Functional Programming =
39. 39. 39 KLEISLIIO Running KleisliIO Constructing KleisliIO Composing KleisliIO Introducing KleisliIO Optimizing KleisliIO Testing KleisliIO
40. 40. 40 KLEISLIIO sealed trait KleisliIO[E, A, B] extends (A => IO[E, B]) { ... } object KleisliIO { class KleisliIOError[E](error: E) extends Throwable { def unsafeCoerce[E2] = error.asInstanceOf[E2] } class Pure[E, A, B](apply0: A => IO[E, B]) extends KleisliIO[E, A, B] { override final def apply(a: A): IO[E, B] = apply0(a) } class Impure[E, A, B](val apply0: A => B) extends KleisliIO[E, A, B] { override final def apply(a: A): IO[E, B] = IO.suspend { try IO.now[E, B](apply0(a)) catch { case e: KleisliIOError[_] => IO.fail[E, B](e.unsafeCoerce[E]) } } } ... }
41. 41. 41 RUNNING KLEISLIIO val printLn: KleisliIO[IOException, String, Unit] printLn("Hello, world!") // IO[IOException, Unit]
42. 42. 42 CONSTRUCTING PURE KLEISLIIO object KleisliIO { ... def pure[E, A, B](f: A => IO[E, B]): KleisliIO[E, A, B] = ??? ... } val printLn: KleisliIO[Void, String, Unit] = KleisliIO.pure((a: String) => IO.sync(println(a)))
43. 43. 43 CONSTRUCTING IMPURE KLEISLIIO object KleisliIO { ... def impure[E, A, B](catcher: PartialFunction[Throwable, E])(f: A => B) ... } val IOExceptions: PartialFunction[Throwable, IOException] = { case io : IOException => io } val printLn: KleisliIO[Void, String, Unit] = KleisliIO.impure(IOExceptions)(println)
44. 44. 44 CONSTRUCTING IMPURE KLEISLIIO object KleisliIO { ... def impureVoid[A, B](f: A => B): KleisliIO[Void, A, B] = ??? ... } val printLn: KleisliIO[Void, String, Unit] = KleisliIO.impureVoid(println)
45. 45. 45 CONSTRUCTING KLEISLIIO object KleisliIO { ... def identity[E, A]: KleisliIO[E, A, A] = ??? ... } A A KleisliIO.identity[Void, Int](1) // IO.point(1)
46. 46. 46 COMPOSING KLEISLIIO trait KleisliIO[E, A, B] extends (A => IO[E, B]) { ... def >>>[C](that: KleisliIO[E, B, C]): KleisliIO[E, A, C] = ??? ... } ... A B C A C readLine >>> printLn
47. 47. 47 COMPOSING KLEISLIIO A B C (B, C) => D A D trait KleisliIO[E, A, B] extends (A => IO[E, B]) { ... def zipWith[C, D](that: KleisliIO[E, A, C])(f: (B, C) => D): KleisliIO[E, A, D] = ??? ... } readLine.zipWith(readLine)((l1, l2) => l1 + l2)
48. 48. 48 COMPOSING KLEISLIIO trait KleisliIO[E, A, B] extends (A => IO[E, B]) { ... def &&&[C]( that: KleisliIO[E, A, C]): KleisliIO[E, A, (B, C)] = ??? ... } A B C A (B,C) readLine &&& readLine
49. 49. 49 COMPOSING KLEISLIIO trait KleisliIO[E, A, B] extends (A => IO[E, B]) { ... def |||[C](that: KleisliIO[E, C, B]): KleisliIO[E, Either[A, C], B] ... } A B C Either[A, C] D (fancyPrintLn ||| standardPrintLn)(Left("Fancied!"))
50. 50. 50 COMPOSING KLEISLIIO trait KleisliIO[E, A, B] extends (A => IO[E, B]) { ... def first: KleisliIO[E, A, (B, A)] = this &&& KleisliIO.identity[E, A] ... } A B A (B, A) readLine >>> printLn.first // (Unit, String)
51. 51. 51 COMPOSING KLEISLIIO trait KleisliIO[E, A, B] extends (A => IO[E, B]) { ... def second: KleisliIO[E, A, (A, B)] = KleisliIO.identity[E, A] &&& this ... } A B A (A, B) readLine >>> printLn.second // (String, Unit)
52. 52. 52 COMPOSING KLEISLIIO trait KleisliIO[E, A, B] extends (A => IO[E, B]) { ... def left[C]: KleisliIO[E, Either[A, C], Either[B, C]] = ??? ... } Either[A, C] Either[B, C] A B printLn.left[String]( Left("Hello")) // Left[Unit]
53. 53. 53 COMPOSING KLEISLIIO trait KleisliIO[E, A, B] extends (A => IO[E, B]) { ... def right[C]: KleisliIO[E, Either[A, C], Either[B, C]] = ??? ... } Either[A, C] Either[B, C] A B printLn.right[String]( Right("Hello")) // Right[Unit]
54. 54. 54 COMPOSING KLEISLIIO trait KleisliIO[E, A, B] extends (A => IO[E, B]) { ... def asEffect: KleisliIO[E, A, A] = self.first >>> KleisliIO._2 ... } A A B printLn.asEffect // String
55. 55. 55 COMPOSING KLEISLIIO object KleisliIO { ... def test[E, A](cond: KleisliIO[E, A, Boolean]): KleisliIO[E, A, Either[A, A]] = ??? ... } test(KleisliIO.lift(_ > 2))(4) // IO[Void, Either[Int, Int]] cond A Bool A Either[A, A] Right(a)Left(a)
56. 56. 56 COMPOSING KLEISLIIO object KleisliIO { ... def ifThenElse[E, A, B](cond : KleisliIO[E, A, Boolean]) (then0: KleisliIO[E, A, B])(else0: KleisliIO[E, A, B]): KleisliIO[E, A, B] = test[E, A](cond) >>> (then0 ||| else0) ... } ifThenElse(KleisliIO.lift(_ == "John"))(printLn)(const(())) A B else0 then0 A BA B cond A Bool
57. 57. 57 COMPOSING KLEISLIIO object KleisliIO { ... def whileDo[E, A](check: KleisliIO[E, A, Boolean]) (body : KleisliIO[E, A, A]): KleisliIO[E, A, A] = ??? ... } A A A Boolean A Abody check readLine >>> whileDo(lift(_ != “John”)) { KleisliIO.point(“Wrong name”) >>> printLn >>> readLine }
58. 58. 58 OPTIMIZING KLEISLIIO final def compose[E, A, B, C](second: KleisliIO[E, B, C], first: KleisliIO[E, A, B]): KleisliIO[E, A, C] = (second, first) match { case (second: Impure[_, _, _], first: Impure[_, _, _]) => new Impure(second.apply0.compose(first.apply0)) case _ => new Pure((a: A) => first(a).flatMap(second)) }
59. 59. 59 OPTIMIZING KLEISLIIO final def ifThenElse[E, A, B](cond: KleisliIO[E, A, Boolean]) (then0: KleisliIO[E, A, B])(else0: KleisliIO[E, A, B]): KleisliIO[E, A, B] = (cond, then0, else0) match { case (cond: Impure[_, _, _], then0: Impure[_, _, _], else0: Impure[_, _, _]) => new Impure[E, A, B](a => if (cond.apply0(a)) then0.apply0(a) else else0.apply0(a)) case _ => test[E, A](cond) >>> (then0 ||| else0) }
60. 60. 60 OPTIMIZING KLEISLIIO final def whileDo[E, A](check: KleisliIO[E, A, Boolean])(body: KleisliIO[E, A, A]): KleisliIO[E, A, A] = (check, body) match { case (check: Impure[_, _, _], body: Impure[_, _, _]) => new Impure[E, A, A]({ (a0: A) => val cond = check.apply0 val update = body.apply0 var a = a0 while (cond(a)) { a = update(a) } a }) case _ => lazy val loop: KleisliIO[E, A, A] = KleisliIO.pure((a: A) => check(a).flatMap((b: Boolean) => if (b) body(a).flatMap(loop) else IO.now(a))) loop }
61. 61. 61 TESTING KLEISLIIO - ARRAY FILL Output: An array filled with 10,000 elements in increasing order.
62. 62. 62 TESTING KLEISLIIO - ARRAY FILL def arrayFill(array: Array[Int]): KleisliIO[Void, Int] = { val condition = KleisliIO.lift(i => i < array.length) val update = KleisliIO.impureVoid{ i => array.update(i, i); i + 1 } KleisliIO.whileDo(condition)(update) } def arrayFill(array: Array[Int])(i: Int): IO[Unit] = if (i >= array.length) IO.unit else IO(array.update(i, i)).flatMap(_ => arrayFill(array)(i + 1)) KleisliIO Array Fill Monadic Array Fill
63. 63. 63 TESTING KLEISLIIO - ARRAY FILL 7958.946 ops/s 3622.744 ops/s 3689.406 ops/s Array Fill 2.18x Faster!
64. 64. 64 TESTING KLEISLIIO - BUBBLE SORT Input: Array of 10000 element with reversed order Output: bubbleSort(array) bubbleSort(array): i <- 1 to 10000 j <- i + 1 to 9999 lessThanEqual = array(i) <= array(j) if (!lessThanEqual) swap(array, i, j)
65. 65. 65 TESTING KLEISLIIO - BUBBLE SORT val sort: KleisliIO[Void, Int, Unit] = KleisliIO .whileDo(outerLoopCheck)( innerLoopStart >>> KleisliIO.whileDo(innerLoopCheck)( extractIJIndexValue >>> KleisliIO.ifNotThen(lessThanEqual)(swapIJ) >>> extractIJAndIncrementJ ) >>> extractIAndIncrementI ) sort(0) def outerLoop(i: Int): IO[Unit] = if (i >= array.length - 1) IO.unit else innerLoop(i, i + 1).flatMap(_ => outerLoop(i + 1)) def innerLoop(i: Int, j: Int): IO[Unit] = if (j >= array.length) IO.unit else IO((array(i), array(j))).flatMap { case (ia, ja) => val maybeSwap = if (lessThanEqual0(ia, ja)) IO.unit else swapIJ(i, ia, j, ja) maybeSwap.flatMap(_ => innerLoop(i, j + 1)) } outerLoop(0) KleisliIO Bubble sort Monadic Bubble sort
66. 66. 66 TESTING KLEISLIIO - BUBBLE SORT 58.811 ops/s 41.545 ops/s 33.229 ops/s 1.57x Faster! Bubble Sort
67. 67. 67 W H E R E F R O M H E R E A brief roadmap for KleisliIO. ? Infinite Composition Recursion Combinator Low-Cost Propagation
68. 68. 68 THANK YOU! Thanks to the staff, volunteers, and speakers of LambdaConf, and to Wiem Zine El Abidine for help with the development and implementation of KleisliIO. Follow me @jdegoes Follow Wiem @wiemzin Join Scalaz at gitter.im/scalaz/scalaz