This document provides an overview of FS2, a Scala library for functional streaming and asynchronous computation. It explains the key features of FS2, including its task abstraction for handling side effects, the definition of streams, and various stream combinators for processing data. The document includes examples of stream manipulation, resource management, and integrating external IO operations in a purely functional way.

1. FS2
for
Fun & Profit
@adilakhter
1

2. The next 45 minutes
... is about FS2
2

3. @ load.ivy("co.fs2" %% "fs2-core" % "0.9.0-M3")
@ load.ivy("co.fs2" %% "fs2-io" % "0.9.0-M3")
3

4. 4

5. 5

6. fs2.util.
Task
6

7. Task
is a
Monadic context for Asynchronous
computation
7

8. Task[A]
8

9. Task
separates
what to do from when to do
9

10. @ def launchMissile() = ... // side-effect
launchMissile: ()Unit
10

11. @ val launchFuture = Future { launchMissile() }
launchFuture: scala.concurrent.Future[Unit] = ...
// A missile has already launched!
11

12. @ launchFuture
// nothing happens due to side-effect memoization!
12

13. @ val launchTask = Task { launchMissile() }
launchTask: fs2.util.Task[Unit]= ...
13

14. @ launchTask.unsafeRun // <- a missile now launched
14

15. @ launchTask.unsafeRun // missile 1 -> launched
@ launchTask.unsafeRun // missile 2 -> launched
15

16. Task
separates
what to do from when to do
16

17. Task
is
purely functional & compositional
17

18. Task Combinators
4 now[A](a: A): Task[A]
4 delay[A](a: => A): Task[A]
4 fail(e: Throwable): Task[Nothing]
4 fork[A](a: => Task[A]): Task[A]
18

19. Example: Retrieve Tweets
@ import twitter4j._
@ def statuses(query: Query): Task[List[Status]] = Task {
twitterClient.search(query).getTweets.toList
}
19

20. FS2
20

21. Goals
4 Incremental IO & Stream Processing
4 Modularity & Composability
4 Resource-safety
4 Performance
21

22. Goals
4 Incremental IO & Stream Processing
4 Modularity & Composability
4 Resource-safety
4 Performance
22

23. @ import fs2._
@ import fs2.util._
@ implicit val strategy: Strategy = Strategy.fromFixedDaemonPool(8)
@ implicit val scheduler: Scheduler = Scheduler.fromFixedDaemonPool(8)
23

24. File IO using FS2
import java.nio.file.Paths
def fahrenheitToCelsius(f: Double): Double = (f - 32.0) * (5.0/9.0)
val streamConverter: Task[Unit] =
io.file.readAll[Task](Paths.get("testdata/fahrenheit.txt"), 4096)
.through(text.utf8Decode)
.through(text.lines)
.filter(s => !s.trim.isEmpty && !s.startsWith("//"))
.map(line => fahrenheitToCelsius(line.toDouble).toString)
.intersperse("n")
.through(text.utf8Encode)
.through(io.file.writeAll(Paths.get("testdata/celsius.txt")))
.run
streamConverter.unsafeRun()
24

25. FS2
... provides an abstraction to declaratively specify
how to obtain stream of data.
25

26. Stream[F[_], O]
26

27. 27

28. 28

29. 29

30. Example: Pure Streams
@ Stream(1,2,3)
res29: Stream[Nothing, Int] =
Segment(Emit(Chunk(1, 2, 3)))
@ Stream(1,2,3) ++ Stream(4,5,6)
res30: Stream[Nothing, Int] =
append(Segment(Emit(Chunk(1, 2, 3))), Segment(Emit(Chunk(()))).flatMap(<function1>))
30

31. Stream ≡ Emit | Await | Done
31

32. Emit
Emit[F[_],O](
head: Seq[O],
tail: Stream[F, O] = Done)
32

33. Example: Pure Streams
@ import Stream._
import Stream._
@ emit(1) // emits value
res30: Stream[Nothing, Int] = Segment(Emit(Chunk(1)))
@ emits(Seq(1,4,10,20)) // <- emits sequence
res31: Stream[Nothing, Int] = Segment(Emit(Chunk(1, 4, 10, 20)))
33

34. Await
Await[F[_], I, O](
req: F[I], // side-effect
recv: I Stream[F, O]) // continuation of Stream.
34

35. Example: Effectful Stream
@ def statuses(query: Query): Task[List[Status]] = ...
35

36. // Builds a Twitter Search Process for a given Query
@ def tweetStream(query: Query): Stream[Task, Status] = {
val queryTask: Task[List[Status]] = statuses(query)
eval(queryTask).flatMap { statusList
emitAll(statusList)
}
}
36

37. //In essence, it builds: Stream: Await (_ Emit Done)
val searchStream = tweetStream(new Query("#spark"))
37

38. Example: A Stream of Positive Integers
@ import Stream._
import Stream._
@ def integerStream: Stream[Task, Int] = {
def next (i: Int) : Stream[Task, Int] = eval(Task(i)).flatMap { i
emit(i) ++ next(i + 1)
}
next(1)
}
// in essence, it describes Stream of {1, 2, 3, 4 ....}
38

39. Stream[F, O]
... represents a stream of O values
which can interleave external requests
to evaluate computation of form F[_]
39

40. Stream Combinators
4 Stream.constant
4 Stream.eval
4 Stream.repeatEval
4 Stream.evalMap
4 fs2.io.readAll
4 ...
40

41. Evaluating a Stream
41

42. Stream.run
@ integerStream.take(10)
res23: Stream[Task, Int] = ...
@ integerStream.take(10).run
res24: Task[Unit] = Task
@ integerStream.take(10).run.unsafeRun
42

43. Stream.runLog
scala> integerStream.take(10).runLog.unsafeRun
res4: Vector[Int] = Vector(1, 2, 3, 4, 5, 6, 7, 8, 9, 10)
43

44. Stream.runLog
What does happen when we execute
integerStream.runLog.run?
@ def integerStream: Stream[Task, Int] = {
def next (i: Int) : Stream[Task, Int] = eval(Task(i)).flatMap { i
emit(i) ++ next(i + 1)
}
next(1)
}
@ integerStream.runLog.unsafeRun // <--- ?
44

45. Other Run Combinators
> def runFree: fs2.util.Free[F,Unit]
> def runFold[B](z: B)(f: (B, A) => B): F[B]
45

46. Transformations
46

47. Stream.map
@ val stream = integerStream.map(_ * 100)
stream: Stream[Task, Int] =
attemptEval(Task).flatMap(<function1>).flatMap(<function1>).mapChunks(<function1>)
@ stream.take(5).runLog.unsafeRun
res39: Vector[Int] = Vector(100, 200, 300, 400, 500)
47

48. Stream.flatMap
@ integerStream.flatMap { i =>
emits(List.fill(i)(i))
}
.take(10)
.runLog
.unsafeRun
res45: Vector[Int] = Vector(1, 2, 2, 3, 3, 3, 4, 4, 4, 4)
48

49. Stream.zip
@ val zippedStream = integerStream zip emitAll(Seq("A", "B"))
res52: Stream[Task, (Int, String)] =
evalScope(Scope(Bind(Eval(Snapshot),<function1>))).flatMap(<function1>)
@ zippedStream.runLog.unsafeRun
res54: Vector[(Int, String)] = Vector((1, "A"), (2, "B"))
49

50. Composability
50

51. Pipe
> Stream[F, A].through(Pipe[A, B]) => Stream[F, B]
51

52. Pipe
type Pipe[F[_],-I,+O] = Stream[F,I] => Stream[F,O]
52

53. Pipe
@ import pipe._
import pipe._
53

54. Example
@ integerStream |> filter( _%2 == 0)
res59: Stream[Task, Int] = ...
54

55. Example
@ integerStream |> filter( _%2 == 0) |> exists(_ == 10)
res60: Stream[Task, Boolean] = ...
55

56. Pipe2
type Pipe2[F[_],-I,-I2,+O] =
(Stream[F,I], Stream[F,I2]) => Stream[F,O]
56

57. Sink
> Stream[F, A].to(Sink[F, A]) => Stream[F, Unit]
57

58. Sink
type Sink[F[_],-I] = Pipe[F,I,Unit]
58

59. Example
@ def log[A]: Sink[Task, A] = _.evalMap{ x =>
Task{
println(s"$x")
}
}
59

60. Example
scala> val streamWithSink: Stream[Task, Unit]
| = integerStream.take(5) to printInts
scala> streamWithSink.run.unsafeRun
1
2
3
4
5
60

61. Example
Sentiment Analysis of Tweets
61

62. 62

63. Building Source
@ def src: Stream[Task, Query] =
awakeEvery[Task](15 seconds).map { _
new Query("#spark")
}
63

64. Twitter Query Pipe
// defined earlier
def statuses(query: Query): Task[List[Status]] = ...
def twitterPipe: Pipe[Task, Query, List[Status]] =
_.evalMap{ query statuses(query) }
64

65. Stream Pipeline
src
.through(twitterPipe) // Stream[Task, List[Status]]
.flatMap(Stream.emits(_)) // Stream[Task, Status]
.map(s Tweet(s)) // Stream[Task, Tweet]
65

66. Sentiment Analysis Pipe
@ import SentimentAnalyzer._
@ def analyze(t: Tweet): Task[EnrichedTweet] = Task {
EnrichedTweet(
t.author,
t.body,
t.retweetCount,
sentiment(t.body))
}
66

67. @ def analysisPipe: Pipe[Task, Tweet, EnrichedTweet] =
_.evalMap{tweet analyze(tweet)}
67

68. Stream Pipeline (so far)
src
.through(twitterPipe) // Stream[Task, List[Status]]
.flatMap(Stream.emits(_)) // Stream[Task, Status]
.map(s Tweet(s)) // Stream[Task, Tweet]
68

69. Stream Pipeline
src
.through(twitterPipe) // Stream[Task, List[Status]]
.flatMap(Stream.emits(_)) // Stream[Task, Status]
.map(s Tweet(s)) // Stream[Task, Tweet]
.through(analysisPipe) // Stream[Task, EnrichedTweet]
69

70. Stream Pipeline
@ val tweetStream =
src
.through(twitterPipe) // Stream[Task, List[Status]]
.flatMap(Stream.emits(_)) // Stream[Task, Status]
.map(s Tweet(s)) // Stream[Task, Tweet]
.through(analysisPipe) // Stream[Task, EnrichedTweet]
70

71. Evaluating Stream Pipeline
@ val consoleTweetStream =
tweetStream.to(snk) // Stream[Task, Unit]
@ consoleTweetStream.run.unsafeRun
71

72. 72

73. Connecting Stream with WS
// http4s Websocket Route
case r @ GET -> Root / "websocket"
val query = new Query("#spark")
val stream = tweetStream(query).map(Text(_))
WS(Exchange(stream, ...)) // <- connected stream to WS
73

74. 74

75. Merge & Join
75

76. @ val s =
integerStream
.through(randomDelays(1.second))
.through(log("s"))
@ val t =
range(1, 3)
.through(randomDelays(1.second))
.through(log("t"))
76

77. Interleave
@ s interleave t
res89: Stream[Task, Int] = ...
@ (s interleave t).through(log("interleaved")).runLog.unsafeRun
s > 1
t > 1
interleaved > 1
interleaved > 1
s > 2
t > 2
interleaved > 2
interleaved > 2
s > 3
res90: Vector[Int] = Vector(1, 1, 2, 2)
77

78. Merge
@ (s merge t).through(log("merged")).runLog.unsafeRun
s > 1
merged > 1
t > 1
merged > 1
s > 2
merged > 2
s > 3
merged > 3
t > 2
merged > 2
s > 4
merged > 4
s > 5
merged > 5
....
78

79. Either
@ s either t
res22: Stream[Task, Either[Int, Int]] = ...
@ Stream.emit("A, B, C") either t
res23: Stream[Task, Either[String, Int]]
79

80. Join
@ val u = Stream.range(10, 20)
u: Stream[Nothing, Int] = ...
@
@ val streams: Stream[Task, Stream[Task, Int]] = Stream(s, t, u)
streams: Stream[Task, Stream[Task, Int]] = ...
@ concurrent.join(3)(streams)
res27: Stream[Task, Int] = ...
80

81. “There are two types of
libraries: the ones people hate
and the ones nobody use"
— Unknown
81

82. Thank You
82

83. Credits
Credits for the images used in slides: Mario Sixtus.
References and Further Readings
4 FS2: The Official Guide
4 Scalaz-Stream Masterclass by Runar at NE Scala 2016
4 Scalaz Task - the missing documentation
4 scalaz-stream User Notes
4 Comparing akka-stream and scalaz-stream with code examples
4 Additional Resources
83