Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Compositional I/O Stream in Scala

1,055 views

Published on

Video and slides synchronized, mp3 and slide download available at URL http://bit.ly/1TpXAZV.

Scalaz-Stream (soon to be called "FS2" or "Functional Streams for Scala") is a library that lets us write compositional, modular, safe, and efficient asynchronous I/O using an expressive purely functional API. Runar Bjarnason presents how to get started with the library, shows some examples, and how we can combine functional streams into large distributed systems. Filmed at qconlondon.com.

Runar Bjarnason is software engineer in Boston, the author of the book Functional Programming in Scala, and an occasional speaker on topics in functional programming.

Published in: Technology
  • Be the first to comment

Compositional I/O Stream in Scala

  1. 1. Compositional I/Ο Streams in Scala Rúnar Bjarnason, Verizon Labs @runarorama QCon London, March 2016
  2. 2. InfoQ.com: News & Community Site • 750,000 unique visitors/month • Published in 4 languages (English, Chinese, Japanese and Brazilian Portuguese) • Post content from our QCon conferences • News 15-20 / week • Articles 3-4 / week • Presentations (videos) 12-15 / week • Interviews 2-3 / week • Books 1 / month Watch the video with slide synchronization on InfoQ.com! https://www.infoq.com/presentations /funnel-distributed-monitoring
  3. 3. Purpose of QCon - to empower software development by facilitating the spread of knowledge and innovation Strategy - practitioner-driven conference designed for YOU: influencers of change and innovation in your teams - speakers and topics driving the evolution and innovation - connecting and catalyzing the influencers and innovators Highlights - attended by more than 12,000 delegates since 2007 - held in 9 cities worldwide Presented at QCon London www.qconlondon.com
  4. 4. Scalaz-Stream (FS2) Functional Streams for Scala github.com/functional-streams-for-scala
  5. 5. Disclaimer This library is changing.
 
 We’ll talk about the current version (0.8).
  6. 6. Funnel — oncue.github.io/funnel http4s — http4s.org streamz — github.com/krasserm/streamz
  7. 7. Scalaz-Stream (FS2) a purely functional streaming I/O library for Scala
  8. 8. • Streams are essentially “lazy lists” of data and effects. • Immutable and
 referentially transparent
  9. 9. Design goals • compositional • expressive • resource-safe • comprehensible
  10. 10. import scalaz.stream._ io.linesR("testdata/fahrenheit.txt") .filter(s => !s.trim.isEmpty && !s.startsWith("//")) .map(line => fahrenheitToCelsius(line.toDouble).toString) .intersperse("n") .pipe(text.utf8Encode) .to(io.fileChunkW("testdata/celsius.txt"))
  11. 11. Process[Task,A]
  12. 12. scalaz.concurrent.Task • Asynchronous • Compositional • Purely functional
  13. 13. a Task is a first-class program
  14. 14. a Task is a list of instructions
  15. 15. Task is a monad
  16. 16. a Task doesn’t do anything until you call .run
  17. 17. Constructing Tasks
  18. 18. Task.delay(readLine): Task[String] Task.now(42): Task[Int] Task.fail( new Exception("oops!") ): Task[Nothing]
  19. 19. a: Task[A] pool: java.util.concurrent.ExecutorService Task.fork(a)(pool): Task[A]
  20. 20. Combining Tasks
  21. 21. a: Task[A] b: Task[B] val c: Task[(A,B)] = Nondeterminism[Task].both(a,b)
  22. 22. a: Task[A] f: A => Task[B] val b: Task[B] = a flatMap f
  23. 23. val program: Task[Unit] = for { _ <- delay(println("What's your name?")) n <- delay(scala.io.StdIn.readLine) _ <- delay(println(s"Hello $n")) } yield ()
  24. 24. Running Tasks
  25. 25. a: Task[A] a.run: A
  26. 26. a: Task[A] k: (Throwable / A) => Unit a runAsync k: Unit
  27. 27. scalaz.stream.Process
  28. 28. Process[F[_],A]
  29. 29. Process[Task,A]
  30. 30. val halt: Process[Nothing,Nothing] def emit[A](a: A): Process[Nothing,A] def eval[F[_],A](eff: F[A]): Process[F,A] Stream primitives
  31. 31. Process.eval( Task.delay(readLine) ): Process[Task,String]
  32. 32. def IO[A](a: => A): Process[Task,A] = Process.eval(Task.delay(a))
  33. 33. Combining Streams
  34. 34. p1: Process[F,A] p2: Process[F,A] val p3: Process[F,A] = p1 append p2
  35. 35. p1: Process[F,A] p2: Process[F,A] val p3: Process[F,A] = p1 ++ p2
  36. 36. val twoLines: Process[Task,String] = IO(readLine) ++ IO(readLine)
  37. 37. val stdIn: Process[Task,String] = IO(readLine) ++ stdIn
  38. 38. val stdIn: Process[Task,String] = IO(readLine).repeat
  39. 39. val cat: Process[Task,Unit] = stdIn flatMap { s => IO(println(s)) }
  40. 40. val cat: Process[Task,Unit] = for { s <- stdIn _ <- IO(println(s)) } yield ()
  41. 41. def grep(r: Regex): Process[Task,Unit] = { val p = r.pattern.asPredicate.test _ def out(s: String) = IO(println(s)) stdIn filter p flatMap out }
  42. 42. Running Processes
  43. 43. p: Process[Task,A] p.run: Task[Unit]
  44. 44. p: Process[Task,A] p.runLog: Task[List[A]]
  45. 45. p: Process[F,A] B: Monoid f: A => B p runFoldMap f: F[B]
  46. 46. F: Monad p: Process[F,A] p.run: F[Unit]
  47. 47. Sinks
  48. 48. x : Process[F,A] y : Sink[F,A] x to y : Process[F,Unit]
  49. 49. import scalaz.stream.io io.stdInLines: Process[Task,String] io.stdOutLines: Sink[Task,String] val cat = io.stdInLines to io.stdOutLines
  50. 50. A sink is just a stream of functions
  51. 51. type Sink[F[_],A] = Process[F, A => Task[Unit]]
  52. 52. val stdOut: Sink[Task,String] = IO { s => Task.delay(println(s)) }.repeat
  53. 53. Pipes
  54. 54. as: Process[F,A] p: Process1[A,B] as pipe p: Process[F,B]
  55. 55. as: Process[F,A] val p = process1.chunk(10) as pipe p: Process[F,Vector[A]]
  56. 56. Process.await1[A]: Process1[A,A]
  57. 57. def take[I](n: Int): Process1[I,I] = if (n <= 0) halt else await1[I] ++ take(n - 1)
  58. 58. def distinct[A]: Process1[A,A] = { def go(seen: Set[A]): Process1[A,A] = Process.await1[A].flatMap { a => if (seen(a)) go(seen) else Process.emit(a) ++ go(seen + a) } go(Set.empty) }
  59. 59. Multiple sources
  60. 60. as: Process[F,A] bs: Process[F,B] t: Tee[A,B,C] (as tee bs)(t): Process[F,C]
  61. 61. tee.zip: Tee[A,B,(A,B)] tee.interleave: Tee[A,A,A]
  62. 62. val add: Tee[Int,Int,Int] = { for { x <- awaitL[Int] y <- awaitR[Int] } yield x + y }.repeat val sumEach = (p1 tee p2)(add)
  63. 63. as: Process[Task,A] bs: Process[Task,B] y: Wye[A,B,C] (as wye bs)(y): Process[Task,C]
  64. 64. ps: Process[F,Process[F,A]] merge.mergeN(ps): Process[F,A]
  65. 65. scalaz.stream.async
  66. 66. Queues & Signals
  67. 67. trait Queue[A] { ... def enqueue: Sink[Task,A] def dequeue: Process[Task,A] ... }
  68. 68. import scalaz.stream.async._ def boundedQueue[A](n: Int): Queue[A] def unboundedQueue[A]: Queue[A] def circularBuffer[A](n: Int): Queue[A]
  69. 69. trait Signal[A] { ... def get: Task[A] def set(a: A): Task[Unit] ... }
  70. 70. trait Signal[A] { ... def discrete: Process[Task,A] def continuous: Process[Task,A] ... }
  71. 71. Demo: Internet Relay Chat
  72. 72. github.com/runarorama/ircz Server: 38 lines of Scala Client: 14 lines of Scala Uses scalaz-netty
  73. 73. github.com/runarorama/ircz def serve(address: InetSocketAddress) = merge.mergeN { Netty serve address map { client => for { c <- client _ <- IO(clients += c.sink) _ <- c.source to messageQueue.enqueue } yield () } }
  74. 74. github.com/runarorama/ircz val relay = for { message <- messageQueue.dequeue client <- emitAll(clients) _ <- emit(message) to client } yield ()
  75. 75. github.com/runarorama/ircz val main = (serve wye relay)(wye.merge)
  76. 76. github.com/runarorama/ircz client = for { c <- Netty connect Server.address in = c.source .pipe(text.utf8Decode) .to(io.stdOutLines) out = io.stdInLines .pipe(text.utf8Encode) .to(c.sink) _ <- (in wye out)(wye.merge) } yield ()
  77. 77. github.com/functional-streams-for-scala github.com/runarorama/ircz oncue.github.io/funnel
  78. 78. Watch the video with slide synchronization on InfoQ.com! https://www.infoq.com/presentations/funnel- distributed-monitoring

×