Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

The Design of the Scalaz 8 Effect System

4,553 views

Published on

Purely functional Scala code needs something like Haskell's IO monad—a construct that allows functional programs to interact with external, effectful systems in a referentially transparent way. To date, most effect systems for Scala have fallen into one of two categories: pure, but slow or inexpressive; or fast and expressive, but impure and unprincipled. In this talk, John A. De Goes, the architect of Scalaz 8’s new effect system, introduces a novel solution that’s up to 100x faster than Future and Cats Effect, in a principled, modular design that ships with all the powerful primitives necessary for building complex, real-world, high-performance, concurrent functional programs.

Thanks to built-in concurrency, high performance, lawful semantics, and rich expressivity, Scalaz 8's effect system may just be the effect system to attract the mainstream Scala developers who aren't familiar with functional programming.

Published in: Technology

The Design of the Scalaz 8 Effect System

  1. 1. TheDesignoftheScalaz8 EffectSystem Scale By The Bay - San Francisco John A. De Goes @jdegoes - http://degoes.net
  2. 2. Agenda · Intro · Tour · Versus · Wrap
  3. 3. AboutMe · I program with functions · I contribute types & functions to FLOSS · I start companies powered by functions
  4. 4. RealityCheck
  5. 5. MostScalaProgrammersDon't ProgramFunctionally !
  6. 6. BusinessScenario
  7. 7. 4MonsterPains 1. Asynchronous 2. Concurrent 3. Resource-Safe 4. Performant
  8. 8. Scalaz8Effectimport scalaz.effect._ Scalaz 8 effect system is a small, composable collection of data types and type classes that help developers build principled, performant, and pragmatic I/O applications that don't leak resources, don't block, and scale across cores.
  9. 9. Scalaz8IO TheHeartofScalaz8 IO[A] is an immutable value that describes an effectful program that either produces an A, fails with a Throwable, or runs forever.
  10. 10. TLDR Scalaz 8 IO helps you quickly build asynchronous, concurrent, leak-free, performant applications.2 2 Which coincidentally happen to be type-safe, purely functional, composable, and easy to reason about.
  11. 11. Tour
  12. 12. Main SafeApp object MyApp extends SafeApp { def run(args: List[String]): IO[Unit] = for { _ <- putStrLn("Hello! What is your name?") n <- getStrLn _ <- putStrLn("Hello, " + n + ", good to meet you!") } yield () }
  13. 13. Core PureValues object IO { ... def apply[A](a: => A): IO[A] = ??? ... } ... val answer: IO[Int] = IO(42)
  14. 14. Core Mapping trait IO[A] { ... def map[B](f: A => IO[B]): IO[B] = ??? } ... IO(2).map(_ * 3) // IO(6)
  15. 15. Core Chaining trait IO[A] { ... def flatMap[B](f: A => IO[B]): IO[B] = ??? } ... IO(2).flatMap(x => IO(3).flatMap(y => IO(x * y)) // IO(6)
  16. 16. Core Failure object IO { ... def fail[A](t: Throwable): IO[A] = ??? ... } ... val failure = IO.fail(new Error("Oh noes!"))
  17. 17. Core Recovery trait IO[A] { ... def attempt: IO[Throwable / A] = ??? ... } ... action.attempt.flatMap { case -/ (error) => IO("Uh oh!") case /-(value) => IO("Yay!") }
  18. 18. Core DerivingAbsolve object IO { ... def absolve[A](io: IO[Throwable / A]): IO[A] = io.flatMap { case -/ (error) => IO.fail(error) case /-(value) => IO(value) } ... } ... IO.absolve(action.attempt)
  19. 19. Core DerivingAlternative trait IO[A] { ... def orElse(that: => IO[A]): IO[A] = self.attempt.flatMap(_.fold(_ => that)(IO(_))) ... } ... val openAnything = openFile("primary.data").orElse(openFile("secondary.data"))
  20. 20. Synchronous ImportingEffects object IO { ... def sync[A](a: => A): IO[A] = ??? ... }
  21. 21. Synchronous ImportingExample def putStrLn(line: String): IO[Unit] = IO.sync(scala.Console.println(line)) def getStrLn: IO[String] = IO.sync(scala.io.StdIn.readLine())
  22. 22. Synchronous EffectExample val program: IO[Unit] = for { _ <- putStrLn("Hello. What is your name?") name <- getStrLn _ <- putStrLn("Hello, " + name + ", good to meet you!") } yield ()
  23. 23. Asynchronous EffectImport:Definition object IO { ... def async0[A](k: (Throwable / A => Unit) => AsyncReturn[A]): IO[A] = ??? ... } ... sealed trait AsyncReturn[+A] object AsyncReturn { final case object Later extends AsyncReturn[Nothing] final case class Now[A](value: A) extends AsyncReturn[A] final case class MaybeLater[A](canceler: Throwable => Unit) extends AsyncReturn[A] }
  24. 24. Asynchronous ImportingEffects def spawn[A](a: => A): IO[A] = IO.async0 { (callback: Throwable / A => Unit) => java.util.concurrent.Executors.defaultThreadFactory.newThread(new Runnable() { def run(): Unit = callback(/-(a)) }) AsyncReturn.Later } def never[A]: IO[A] = IO.async0 { (callback: Throwable / A => Unit) => AsyncReturn.Later }
  25. 25. Asynchronous EffectExample for { response1 <- client.get("http://e.com") limit = parseResponse(response1).limit response2 <- client.get("http://e.com?limit=" + limit) } yield parseResponse(response2)
  26. 26. Asynchronous Sleep IO { ... def sleep(duration: Duration): IO[Unit] = ??? ... }
  27. 27. Asynchronous SleepExample for { _ <- putStrLn("Time to sleep...") _ <- IO.sleep(10.seconds) _ <- putStrLn("Time to wake up!") } yield ()
  28. 28. Asynchronous DerivingDelay trait IO[A] { ... def delay(duration: Duration): IO[A] = IO.sleep(duration).flatMap(_ => self) ... } ... putStrLn("Time to wake up!").delay(10.seconds)
  29. 29. Concurrency Models 1. Threads — Java · OS-level · Heavyweight · Dangerous interruption 2. Green Threads — Haskell · Language-level · Lightweight · Efficient 3. Fibers — Scalaz 8 · Application-level · Lightweight · Zero-cost for pure FP · User-defined semantics
  30. 30. Concurrency Fork/Join trait IO[A] { ... def fork: IO[Fiber[A]] = ??? def fork0(h: Throwable => IO[Unit]): IO[Fiber[A]] = ??? ... } trait Fiber[A] { def join: IO[A] def interrupt(t: Throwable): IO[Unit] }
  31. 31. Concurrency Fork/JoinExample def fib(n: Int): IO[BigInt] = if (n <= 1) IO(n) else for { fiberA <- fib(n-1).fork fiberB <- fib(n-2).fork a <- fiberA.join b <- fiberB.join } yield a + b
  32. 32. Concurrency raceWith trait IO[A] { ... def raceWith[B, C](that: IO[B])( finish: (A, Fiber[B]) / (B, Fiber[A]) => IO[C]): IO[C] = ??? ... }
  33. 33. Concurrency DerivingRace trait IO[A] { ... def race(that: IO[A]): IO[A] = raceWith(that) { case -/ ((a, fiber)) => fiber.interrupt(Errors.LostRace( /-(fiber))).const(a) case /-((a, fiber)) => fiber.interrupt(Errors.LostRace(-/ (fiber))).const(a) } ... }
  34. 34. Concurrency DerivingTimeout trait IO[A] { ... def timeout(duration: Duration): IO[A] = { val err: IO[Throwable / A] = IO(-/(Errors.TimeoutException(duration))) IO.absolve(self.attempt.race(err.delay(duration))) } ... }
  35. 35. Concurrency DerivingPar trait IO[A] { ... def par[B](that: IO[B]): IO[(A, B)] = attempt.raceWith(that.attempt) { case -/ ((-/ (e), fiberb)) => fiberb.interrupt(e).flatMap(_ => IO.fail(e)) case -/ (( /-(a), fiberb)) => IO.absolve(fiberb.join).map(b => (a, b)) case /-((-/ (e), fibera)) => fibera.interrupt(e).flatMap(_ => IO.fail(e)) case /-(( /-(b), fibera)) => IO.absolve(fibera.join).map(a => (a, b)) } ... }
  36. 36. Concurrency DerivingRetry trait IO[A] { ... def retry: IO[A] = this orElse retry def retryN(n: Int): IO[A] = if (n <= 1) this else this orElse (retryN(n - 1)) def retryFor(duration: Duration): IO[A] = IO.absolve( this.retry.attempt race (IO.sleep(duration) *> IO(-/(Errors.TimeoutException(duration))))) ... }
  37. 37. Concurrency MVar trait MVar[A] { def peek: IO[Maybe[A]] = ??? def take: IO[A] = ??? def read: IO[A] = ??? def put(v: A): IO[Unit] = ??? def tryPut(v: A): IO[Boolean] = ??? def tryTake: IO[Maybe[A]] = ??? }
  38. 38. Concurrency MVarExample val action = for { mvar <- MVar.empty // Fiber 1 _ <- mvar.putVar(r).fork // Fiber 2 result <- mvar.takeVar // Fiber 1 } yield result
  39. 39. ComingSoon:RealSTM
  40. 40. ResourceSafety Uninterruptible trait IO[A] { ... def uninterruptibly: IO[A] = ??? ... }
  41. 41. ResourceSafety UninterruptibleExample val action2 = action.uninterruptibly
  42. 42. ResourceSafety Bracket trait IO[A] { ... def bracket[B]( release: A => IO[Unit])( use: A => IO[B]): IO[B] = ??? ... }
  43. 43. ResourceSafety BracketExample def openFile(name: String): IO[File] = ??? def closeFile(file: File): IO[Unit] = ??? openFile("data.json").bracket(closeFile(_)) { file => ... // Use file ... }
  44. 44. ResourceSafety Bracket trait IO[A] { ... def bracket[B]( release: A => IO[Unit])( use: A => IO[B]): IO[B] = ??? ... }
  45. 45. ResourceSafety Deriving'Finally' trait IO[A] { def ensuring(finalizer: IO[Unit]): IO[A] = IO.unit.bracket(_ => finalizer)(_ => this) }
  46. 46. ResourceSafety BrokenErrorModel try { try { try { throw new Error("e1") } finally { throw new Error("e2") } } finally { throw new Error("e3") } } catch { case e4 : Throwable => println(e4.toString()) }
  47. 47. ResourceSafety FixedErrorModel IO.fail(new Error("e1")).ensuring( IO.fail(new Error("e2"))).ensuring( IO.fail(new Error("e3"))).catchAll(e => putStrLn(e.toString()))
  48. 48. ResourceSafety Supervision object IO { ... def supervise[A](io: IO[A]): IO[A] = ??? ... }
  49. 49. ResourceSafety SupervisionExample val action = IO.supervise { for { a <- doX.fork b <- doY.fork ... } yield z }
  50. 50. Principles AlgebraicLaws fork >=> join = id let fiber = fork never in interrupt e fiber >* join fiber = fail e And many more!
  51. 51. Versus
  52. 52. Versus:Performance SCALAZ 8 IO FUTURE CATS IO MONIX TASK Le! Associated flatMap 5061.380 39.088 0.807 3548.260 Narrow flatMap 7131.227 36.504 2204.571 6411.355 Repeated map 63482.647 4599.431 752.771 47235.85 Deep flatMap 1885.480 14.843 131.242 1623.601 Shallow attempt 769.958 CRASHED 643.147 CRASHED Deep attempt 16066.976 CRASHED 16061.906 12207.417 Scalaz 8 IO is up to 6300x faster than Cats (0.4), 195x faster than Future (2.12.4), and consistently faster than Monix Task (3.0.0-RC1).
  53. 53. Versus:Safety SCALAZ 8 IO FUTURE CATS IO MONIX TASK Sync Stack Safety ✓ ✓ ✓ ✓ Async Stack Safety ✓ ✓ ! ✓ Bracket Primitive ✓ ! ! ! No Implicit Executors ✓ ! ! ✓ No Mutable Implicits ✓ ! ! ✓
  54. 54. Versus:Expressiveness SCALAZ 8 IO FUTURE CATS IO MONIX TASK Synchronicity ✓ ! ✓ ✓ Asynchronicity ✓ ✓ ✓ ✓ Concurrency Primitives ✓ ! ! ✓ Async Var ✓ ! ! ✓ Non-Leaky Race ✓ ! ! ✓4 Non-Leaky Timeout ✓ ! ! ✓4 Non-Leaky Parallel ✓ ! ! ✓4 Thread Supervision ✓ ! ! ! 4 Cancellation only occurs at async boundaries.
  55. 55. Versus WhatAboutFS2? IO is not a stream!
  56. 56. Versus FS2:MissingFoundations · Mini-actor library · Mini-FRP library · MVar implementation — Ref · Concurrency primitives · race, bracket, fork, join
  57. 57. Versus FS2:LeakyFoundations package object async { ... def race[F[_]: Effect, A, B](fa: F[A], fb: F[B])( implicit ec: ExecutionContext): F[Either[A, B]] = ref[F, Either[A,B]].flatMap { ref => ref.race(fa.map(Left.apply), fb.map(Right.apply)) >> ref.get } def start[F[_], A](f: F[A])(implicit F: Effect[F], ec: ExecutionContext): F[F[A]] = ref[F, A].flatMap { ref => ref.setAsync(F.shift(ec) >> f).as(ref.get) } def fork[F[_], A](f: F[A])(implicit F: Effect[F], ec: ExecutionContext): F[Unit] = F.liftIO(F.runAsync(F.shift >> f) { _ => IO.unit }) ... }
  58. 58. Versus FS2: Non-Compositional Timeout class Ref[A] { ... def timedGet(timeout: FiniteDuration, scheduler: Scheduler): F[Option[A]] = ??? ... } Scalaz 8: Compositional Timeout mvar.takeVar.timeout(t) mvar.putVar(2).timeout(t) ... what.ev.uh.timeout(t)
  59. 59. ThisisWar
  60. 60. ThankYou Special thanks to Alexy Khrabrov, Twitter, and the wonderful attendees of Scale By The Bay!

×