Fibers are the backbone of the highly performant, asynchronous and concurrent abilities of ZIO. They are lightweight “green threads” implemented by the ZIO runtime system. In this lightening talk you will learn about: * How to handle fiber dying due to unexpected failure * How to guarantee a ZIO fiber is interrupted * How to set up fiber tracking and executing a fiberDump for increased debuggability

1. @NSilnitsky
@NSilnitsky
How to successfully manage a ZIO Fiber
How to successfully manage a
ZIO Fiber’s lifecycle
Natan Silnitsky Backend Infra TL, Wix.com
natansil.com twitter@NSilnitsky linkedin/natansilnitsky github.com/natansil

2. @NSilnitsky
How to successfully manage a ZIO Fiber
>200 Million registered users from 190 countries

3. Agenda 1. What’s a Fiber?
2. Handling unexpected failure
3. Interrupt a Fiber
4. Monitoring Fibers state
How to successfully manage a ZIO Fiber

4. @NSilnitsky
What’s a Fiber?
01
What’s a Fiber?
A lightweight concurrency mechanism.
* little mem, no block, yield, multi task 1 the, GC

5. @NSilnitsky
object WhatIsAFiber extends App {
override def run(args: List[String]): URIO[zio.ZEnv,
ExitCode] =
for {
fiber <- runSomething().fork
_ <- runSomethingElse()
_ <- fiber.join
} yield ExitCode(0)
}
What’s a Fiber?
Fork
Join

6. @NSilnitsky
What’s a Fiber?
01
What’s a Fiber
Usually work with higher level operators:
● ZIO#foreachPar
● ZIO#zipPar
● ZIO#race

7. @NSilnitsky
object WhatIsAFiber extends App {
override def run(args: List[String]): URIO[zio.ZEnv,
ExitCode] =
for {
fiber <- runSomething().forkDaemon
_ <- runSomethingElse()
_ <- fiber.join
} yield ExitCode(0)
}
What’s a
Fiber
What’s a Fiber?
M
Main
Fiber
B
A
Fork
A1 A2 B1
M2
Fork
Daemon
Fork

8. @NSilnitsky
object WhatIsAFiber extends App {
override def run(args: List[String]): URIO[zio.ZEnv,
ExitCode] =
for {
fiber <- runSomething().forkDaemon
_ <- runSomethingElse()
_ <- fiber.join
} yield ExitCode(0)
}
What’s a
Fiber
What’s a Fiber?
Main
Fiber
B
A
Fork
A1 A2 B1
Fork
M2
M
Fork
Daemon

9. @NSilnitsky
object WhatIsAFiber extends App {
override def run(args: List[String]): URIO[zio.ZEnv,
ExitCode] =
for {
fiber <- runSomething().forkDaemon
_ <- runSomethingElse()
_ <- fiber.join
} yield ExitCode(0)
}
What’s a
Fiber
What’s a Fiber?
Main
Fiber
B
A
Fork
A1 A2 B1
Fork
M M2
Fork
Daemon

10. @NSilnitsky
Handling unexpected failures
Make sure the job continues to run periodically
even when an unexpected error occurs
→
02
Handling
unexpected
failures
* imagine - periodic, update state

11. @NSilnitsky
object DefectNotHandledExample extends App {
import zio._
import zio.duration._
override def run(args: List[String]): URIO[zio.ZEnv, ExitCode] =
for {
_ <- runSomePeriodicJob
.catchAll(e => console.putStrLn(s"job failed with $e"))
.repeat(Schedule.spaced(1.seconds))
.forkDaemon
_ <- console.putStrLn("do other stuff")
.repeat(Schedule.spaced(1.seconds))
} yield ExitCode(0)
private def runSomePeriodicJob = {
ZIO.succeed(throw new RuntimeException("unexpected defect")) *> ZIO.effect(println("running job..."))
}
}
Death of a Fiber

12. @NSilnitsky
object DefectNotHandledExample extends App {
...
override def run(args: List[String]): URIO[zio.ZEnv, ExitCode] =
for {
_ <- runSomePeriodicJob
.catchAll(e => console.putStrLn(s"job failed with $e"))
.repeat(Schedule.spaced(1.seconds))
.forkDaemon
_ <- console.putStrLn("do other stuff")
.repeat(Schedule.spaced(1.seconds))
} yield ExitCode(0)
private def runSomePeriodicJob = {
ZIO.succeed(throw new RuntimeException("unexpected defect")) *>
ZIO.effect(println("running job..."))
}
}
Death of a Fiber

13. @NSilnitsky
@NSilnitsky
do other stuff
Fiber failed.
An unchecked error was produced.
java.lang.RuntimeException: unexpected defect
…
do other stuff
do other stuff
do other stuff
…
Handling unexpected failures

14. @NSilnitsky
Handling unexpected failures
catchAll doesn’t catch all failures
catchAllCause does

15. @NSilnitsky
Handling unexpected failures
catchAll doesn’t catch all failures
catchAllCause does
The E type of ZIO[R, E, A] refers only to expected
failures.
Cause[E] is a description of a full story of failure.
Cause can be either Fail[+E](value: E), Die(value:
Throwable), or Interrupt(fiberId: Fiber.Id)

16. @NSilnitsky
object DefectNotHandledExample extends App {
...
override def run(args: List[String]): URIO[zio.ZEnv, ExitCode] =
for {
_ <- runSomePeriodicJob
.catchAllCause(cause => console.putStrLn(s"job failed with $cause"))
.repeat(Schedule.spaced(1.seconds))
.forkDaemon
_ <- console.putStrLn("do other stuff")
.repeat(Schedule.spaced(1.seconds))
} yield ExitCode(0)
private def runSomePeriodicJob = {
ZIO.succeed(throw new RuntimeException("unexpected defect")) *>
ZIO.effect(println("running job..."))
}
}
Death of a Fiber

17. @NSilnitsky
@NSilnitsky
do other stuff
job failed with Traced(Die(...RuntimeException: unexpected defect)…
do other stuff
job failed with Traced(Die(...RuntimeException: unexpected defect)…
…
Cause -> Throwable
Handling unexpected failures
cause.squashTrace

18. @NSilnitsky
runtime.mapPlatform(_.withReportFailure(_ => [report unexpected failure]))
Handling unexpected
failures
* monitoring service

19. @NSilnitsky
Interrupt a Fiber
03
Interrupt a Fiber
There are certain situations where interrupting
a fiber is required.
Releasing a managed resource, that forked a
fiber, when it was acquired.

20. @NSilnitsky
object InterruptExample extends App {
...
override def run(args: List[String]): URIO[zio.ZEnv, ExitCode] =
for {
fiber <- runSomeLongJob
.ensuring(printWithTime("finalizing job!"))
.forkDaemon
_ <- printWithTime("do stuff").repeat(Schedule.spaced(1.seconds) && Schedule.recurs(2))
_ <- fiber.interrupt
_ <- printWithTime("do other stuff").repeat(Schedule.spaced(1.seconds))
} yield ExitCode(0)
private def runSomeLongJob = {
printWithTime("long running job...") *> clock.sleep(100.seconds)
}
}
Interrupt a Fiber

21. @NSilnitsky
object InterruptExample extends App {
...
override def run(args: List[String]): URIO[zio.ZEnv, ExitCode] =
for {
fiber <- runSomeLongJob
.ensuring(printWithTime("finalizing job!"))
.forkDaemon
_ <- printWithTime("do stuff").repeat(Schedule.spaced(1.seconds) && Schedule.recurs(2))
_ <- fiber.interrupt
_ <- printWithTime("do other stuff").repeat(Schedule.spaced(1.seconds))
} yield ExitCode(0)
private def runSomeLongJob = {
printWithTime("long running job...") *> clock.sleep(100.seconds)
}
}
Interrupt a Fiber

22. @NSilnitsky
@NSilnitsky
19:15:38.445 long running job…
19:15:38.501 do stuff
19:15:39.696 do stuff
19:15:40.701 do stuff
19:15:40.862 finalizing job!
19:15:40.871 do other stuff
19:15:41.873 do other stuff
Interrupt a Fiber

23. @NSilnitsky
override def run(args: List[String]): URIO[zio.ZEnv, ExitCode] =
for {
fiber <- runSomeLongJob
.ensuring(printWithTime("finalizing job!"))
.forkDaemon
_ <- printWithTime("do stuff").repeat(Schedule.spaced(1.seconds) && Schedule.recurs(2))
timeout <- fiber.join.resurrect.ignore.disconnect.timeout(5.seconds)
_ <- ZIO.when(timeout.isEmpty)(fiber.interruptFork)
_ <- printWithTime("do other stuff").repeat(Schedule.spaced(1.seconds))
} yield ExitCode(0)
Graceful
Termination

24. @NSilnitsky
override def run(args: List[String]): URIO[zio.ZEnv, ExitCode] =
for {
fiber <- runSomeLongJob
.ensuring(printWithTime("finalizing job!"))
.forkDaemon
_ <- printWithTime("do stuff").repeat(Schedule.spaced(1.seconds) && Schedule.recurs(2))
timeout <- fiber.join.resurrect.ignore.disconnect.timeout(5.seconds)
_ <- ZIO.when(timeout.isEmpty)(fiber.interruptFork)
_ <- printWithTime("do other stuff").repeat(Schedule.spaced(1.seconds))
} yield ExitCode(0)
Graceful
Termination

25. @NSilnitsky
override def run(args: List[String]): URIO[zio.ZEnv, ExitCode] =
for {
fiber <- runSomeLongJob
.ensuring(printWithTime("finalizing job!"))
.forkDaemon
_ <- printWithTime("do stuff").repeat(Schedule.spaced(1.seconds) && Schedule.recurs(2))
timeout <- fiber.join.resurrect.ignore.disconnect.timeout(5.seconds)
_ <- ZIO.when(timeout.isEmpty)(fiber.interruptFork)
_ <- printWithTime("do other stuff").repeat(Schedule.spaced(1.seconds))
} yield ExitCode(0)
Graceful
Termination

26. @NSilnitsky
@NSilnitsky
19:23:01.422 long running job…
19:23:01.492 do stuff
19:23:02.689 do stuff
19:23:03.693 do stuff
19:23:08.960 finalizing job!
19:23:08.963 do other stuff
19:23:09.964 do other stuff
Interrupt a Fiber

27. @NSilnitsky
Monitoring Fiber state
04
Monitoring Fiber
state
Good old Java thread dumps don’t give you
interesting information when it comes to ZIO apps
ZIO fiber dumps can give you stack traces...

28. @NSilnitsky
@NSilnitsky
---- test dump results ----
+---#756 Status: Running()
+---#603480 Status: Suspended(interruptible, 1395652 asyncs,
zio.clock.package$Clock$Service$$anon$1.sleep(package.scala:70))
+---#760 Status: Running()
+---#64569 Status: Suspended(interruptible, 5520272 asyncs,
zio.clock.package$Clock$Service$$anon$1.sleep(package.scala:70))
+---#2399 Status: Suspended(interruptible, 6099394 asyncs,
zio.clock.package$Clock$Service$$anon$1.sleep(package.scala:70))
+---#1478 Status: Suspended(interruptible, 5862206 asyncs,
zio.clock.package$Clock$Service$$anon$1.sleep(package.scala:70))
+---#1654 Status: Suspended(interruptible, 6108699 asyncs,
zio.clock.package$Clock$Service$$anon$1.sleep(package.scala:70))
Monitoring Fiber state

29. @NSilnitsky
@NSilnitsky
#64569 (7h476m28609s28609166ms)
Status: Suspended(interruptible, 5520272 asyncs,
zio.clock.package$Clock$Service$$anon$1.sleep(package.scala:70))
Fiber:Id(1615104333320,64569) was supposed to continue to:
a future continuation at zio.ZIO$._IdentityFn(ZIO.scala:3973)
a future continuation at zio.ZIO.onInterrupt(ZIO.scala:981)
a future continuation at <unknown>.<unknown>(ZIO.scala:0)
a future continuation at zio.ZIO.repeatUntilM(ZIO.scala:1529)
Fiber:Id(1615104333320,64569) execution trace:
at zio.clock.package$Clock$Service$$anon$1.sleep(package.scala:70)
at zio.ZIO$.effectAsyncInterrupt(ZIO.scala:2582)
at zio.ZIO$.effectAsyncInterrupt(ZIO.scala:2582)
at zio.clock.package$.sleep(package.scala:118)
Monitoring Fiber state

30. @NSilnitsky
ZIO-ZMX
val diagnosticsLayer: ZLayer[ZEnv, Throwable, Diagnostics] =
Diagnostics.live(“localhost”, 1111)
val runtime: Runtime[ZEnv] =
Runtime.default.mapPlatform(_.withSupervisor(ZMXSupervisor))
Monitoring Fibers
state

31. @NSilnitsky
Or roll your own...
object RuntimeOps {
implicit class RuntimeEnvOps[R <: Has[_]](val runtime: zio.Runtime[R]) extends AnyVal {
def withFiberTracking(implicit tag: zio.Tag[R]): zio.Runtime[R with FiberTracking] = {
val supervisor = runtime.unsafeRunTask(Supervisor.track(weak = false))
runtime
.mapPlatform(_.withSupervisor(supervisor))
.map(_.union[FiberTracking](FiberTracking.make(supervisor)))
}
}
}
Monitoring Fibers
state

32. Summary
1. What’s a Fiber
2. Handling unexpected failure
3. Interrupt a Fiber
4. Monitoring Fiber state
How to successfully manage a ZIO Fiber
2. catchAllCause, ensuring, withReportFailure
4. Supervisor, fiber dumps
3. resurrect, interruptFork and disconnect

33. @NSilnitsky
https://medium.com/wix-engineering/how-to-successfully-manage-a-zi
o-fibers-lifecycle-96fe9c4ecd06
The blog post
How to successfully manage a ZIO Fiber

34. @NSilnitsky
A Scala ZIO high-level functional SDK for Apache Kafka.
github.com/wix/greyhound
How to successfully manage a ZIO Fiber
0.2 is out!

35. @NSilnitsky
@NSilnitsky
Thank
You!
natansil.com twitter@NSilnitsky linkedin/natansilnitsky github.com/natansil
👉 slideshare.net/NatanSilnitsky
Any questions?
How to successfully manage a ZIO Fiber

36. @NSilnitsky
@NSilnitsky
Q&A
natansil.com twitter@NSilnitsky linkedin/natansilnitsky github.com/natansil
👉 slideshare.net/NatanSilnitsky
Any questions?
How to successfully manage a ZIO Fiber