ZIO Queue John A. De Goes — @jdegoes - http://degoes.net ScalaWave 2018
ZIO Queue
ScalaWave 2018 - Gdańsk, Poland
John A. De Goes
@jdegoes - http://degoes.net
What is ZIO?
Powered by Queues
Troubled Queues
Introducing ZIO Queue
Wrap Up
What is ZIO? Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up
ZIO
ZIO lets you build
high-performance, type-safe,
concurrent, asynchronous
applications that don’t leak
resources or deadlock, and are
easy to reason about
compositionally, test, and refactor.
What is ZIO? Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up
github.com/scalaz/scalaz-zio
ZIO
def main: Unit = {
println("Hello, what is your name?")
val name = readLine()
println(s"Good morning, $name!")
}
Second-Class Effects
✗ Pass to functions
✗ Return from functions
✗ Store in data structures
✗ Async/sync/concurrency
✓ Resource-safety
What is ZIO? Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up
def main: IO[IOException, Unit] = for {
_ <- putStrLn("Hello, what is your name?")
name <- getStrLn
_ <- putStrLn(s"Good morning, $name!")
} yield ()
First-Class Effects
✓ Pass to functions
✓ Return from functions
✓ Store in data structures
✓ Async/sync/concurrency
✓ Resource-safety
What is ZIO? Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up
IO[E, A]
IO[E, A] is an immutable value that
describes an effectful computation,
which may fail with a value of type E, or
compute a value of type A.
What is ZIO? Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up
What is ZIO? Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up
What is ZIO? Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up
What is ZIO? Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up
def runInParallel[E, A, B](
leftIO : IO[E, A],
rightIO: IO[E, B]): IO[E, (A, B)] =
for {
leftFiber <- leftIO.fork
rightFiber <- rightIO.fork
a <- leftFiber.join
b <- rightFiber.join
} yield (a, b)
Fork/Join Concurrency with Fibers
✓ Massive scalability over threads
✓ Non-blocking
✓ Automatically interruptible
✓ Always resource-safe
✓ Powerful composition
What is ZIO? Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up
What is ZIO? Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up
Powered by Queues
What is ZIO? Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up
Producer Consumer
offer take
Queue
1-to-1
What is ZIO? Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up
Producer
Consumer
offer take
Queue
Producer
n-to-1
What is ZIO? Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up
Producer
Consumer
offer take
Queue
Consumer
1-to-m
What is ZIO? Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up
Consumer
offer take
Queue
Consumer
Producer
Producer
n-to-m
What is ZIO? Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up
FIFO
LIFO
PRIO
Bounded
Unbounded
Dropping
Sliding
dimensions of variation
What is ZIO? Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up
queue
actors
send
What is ZIO? Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up
concurrent graph search
What is ZIO? Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up
resource sharing
scarce resource
e.g. threads
Thread 1
take Thread 2offer
What is ZIO? Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up
loose coupling
offer
take
Fast Producers
Slow Consumers
buffer
What is ZIO? Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up
Troubled Queues
What is ZIO? Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up
java.util.concurrent.BlockingQueue
What is ZIO? Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up
java.util.concurrent.BlockingQueue
trait BlockingQueue[A] {
def take: A
def offer(a: A): Boolean
}
What is ZIO? Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up
java.util.concurrent.BlockingQueue
Blocks one thread
until element added
trait BlockingQueue[A] {
def take: A
def offer(a: A): Boolean
}
What is ZIO? Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up
java.util.concurrent.BlockingQueue
Fast Producer Slow Consumer
offer take
Queue
What is ZIO? Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up
java.util.concurrent.BlockingQueue
Fast Producer Slow Consumer
offer
take
Queue
manual
backpressure
What is ZIO? Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up
java.util.concurrent.BlockingQueue
def fastProducer = {
if (!queue.offer(a)) {
// Slow down!!!
} else {
// Keep producing...
}
}
What is ZIO? Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up
java.util.concurrent.BlockingQueue
Fast Producer Slow Consumer
offer
take
Queue
manual
backpressure
manual
backpressure
take
What is ZIO? Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up
java.util.concurrent.ConcurrentLinkedQueue
trait ConcurrentLinkedQueue[A] {
def poll: A
def offer(a: A): Boolean
}
Returns null if empty
What is ZIO? Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up
java.util.concurrent.ConcurrentLinkedQueue
Fast Producer Slow Consumer
offer poll
Queue
What is ZIO? Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up
github.com/scalaz/scalaz-zio
ZIO QUEUE
What is ZIO? Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up
ZIO Queue
trait Queue[A] {
def take: IO[Nothing, A]
def offer(a: A): IO[Nothing, Unit]
def shutdown(ts: Throwable*): IO[Nothing, Unit]
}
object Queue {
def bounded[A](capacity: Int): IO[Nothing, Queue[A]] = ???
def unbounded[A]: IO[Nothing, Queue[A]] = ???
}
What is ZIO? Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up
Example
for {
queue <- Queue.unbounded[String]
_ <- queue.offer("Give me Coffee!").forever.fork
_ <- queue.take.flatMap(putStrLn).forever
} yield ()
What is ZIO? Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up
5. Concurrent
Consumer
offer take
Queue
Consumer
Producer
Producer
ZIO
What is ZIO? Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up
4. Interruptible
for {
queue <- Queue.unbounded[String]
fiber <- queue.offer("Give me Coffee!").forever.fork
_ <- fiber.interrupt
} yield ()
What is ZIO? Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up
3. Clean Shutdown
for {
queue <- Queue.unbounded[String]
fiber1 <- queue.offer("Give me Coffee!").forever.fork
fiber2 <- queue.take.forever.fork
_ <- queue.shutdown
} yield ()
What is ZIO? Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up
2. Asynchronous
for {
queue <- Queue.unbounded[String]
worker = queue.take.flatMap(putStrLn).forever
workers10k = List.fill(10000)(worker)
_ <- IO.forkAll(workers10k)
_ <- queue.offer("More Coffee!").forever.fork
} yield ()
What is ZIO? Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up
2. Asynchronous
trait BlockingQueue[A] {
def take: A ;
def offer(a: A): Boolean
}
What is ZIO? Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up
1. Composable Backpressure
for {
queue <- Queue.bounded[String](10)
worker = queue.offer("Coffee").forever
workers10k = List.fill(10000)(worker)
_ <- IO.forkAll(workers10k)
_ <- queue.take.flatMap(putStrLn).forever.fork
} yield ()
What is ZIO? Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up
Application: Scalaz Actors
case class Actor[+E, -A, +B](run: A => IO[E, B]) { self =>
def ! (a: A): IO[E, B] = run(a)
}
What is ZIO? Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up
Application: Scalaz Actors
def makeActor[S, E, A, B](supervisor: Supervisor)(s: S)(receive: (S, A) => IO[E, (S, B)]):
IO[Nothing, Actor[E, A, B]] =
for {
state <- Ref(s)
queue <- Queue.bounded[(A, Promise[E, B])](QueueSize)
fiber <- (for {
t <- queue.take
s <- state.get
(a, promise) = t
receiver = receive(s, a)
completer = (s: S, b: B) =>
state.set(s) *> promise.complete(b)
_ <- receiver.redeem(
e => supervise(receiver, e)
.redeem(promise.error, completer), completer)
} yield ()).forever.fork
} yield Actor[E, A, B]((a: A) =>
Promise.make[E, B].flatMap(p => queue.offer((a, p)).flatMap(_ => p.get))
What is ZIO? Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up
Application: Scalaz Actors
def makeActor[S, E, A, B](supervisor: Supervisor)(s: S)(receive: (S, A) => IO[E, (S, B)]):
IO[Nothing, Actor[E, A, B]] =
for {
state <- Ref(s)
queue <- Queue.bounded[(A, Promise[E, B])](QueueSize)
fiber <- (for {
t <- queue.take
s <- state.get
(a, promise) = t
receiver = receive(s, a)
completer = (s: S, b: B) =>
state.set(s) *> promise.complete(b)
_ <- receiver.redeem(
e => supervise(receiver, e)
.redeem(promise.error, completer), completer)
} yield ()).forever.fork
} yield Actor[E, A, B]((a: A) =>
Promise.make[E, B].flatMap(p => queue.offer((a, p)).flatMap(_ => p.get))
What is ZIO? Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up
Application: Scalaz Actors
def makeActor[S, E, A, B](supervisor: Supervisor)(s: S)(receive: (S, A) => IO[E, (S, B)]):
IO[Nothing, Actor[E, A, B]] =
for {
state <- Ref(s)
queue <- Queue.bounded[(A, Promise[E, B])](QueueSize)
fiber <- (for {
t <- queue.take
s <- state.get
(a, promise) = t
receiver = receive(s, a)
completer = (s: S, b: B) =>
state.set(s) *> promise.complete(b)
_ <- receiver.redeem(
e => supervise(receiver, e)
.redeem(promise.error, completer), completer)
} yield ()).forever.fork
} yield Actor[E, A, B]((a: A) =>
Promise.make[E, B].flatMap(p => queue.offer((a, p)).flatMap(_ => p.get))
What is ZIO? Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up
Application: Scalaz Actors
def makeActor[S, E, A, B](supervisor: Supervisor)(s: S)(receive: (S, A) => IO[E, (S, B)]):
IO[Nothing, Actor[E, A, B]] =
for {
state <- Ref(s)
queue <- Queue.bounded[(A, Promise[E, B])](QueueSize)
fiber <- (for {
t <- queue.take
s <- state.get
(a, promise) = t
receiver = receive(s, a)
completer = (s: S, b: B) =>
state.set(s) *> promise.complete(b)
_ <- receiver.redeem(
e => supervise(receiver, e)
.redeem(promise.error, completer), completer)
} yield ()).forever.fork
} yield Actor[E, A, B]((a: A) =>
Promise.make[E, B].flatMap(p => queue.offer((a, p)).flatMap(_ => p.get))
What is ZIO? Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up
Application: Scalaz Actors
def makeActor[S, E, A, B](supervisor: Supervisor)(s: S)(receive: (S, A) => IO[E, (S, B)]):
IO[Nothing, Actor[E, A, B]] =
for {
state <- Ref(s)
queue <- Queue.bounded[(A, Promise[E, B])](QueueSize)
fiber <- (for {
t <- queue.take
s <- state.get
(a, promise) = t
receiver = receive(s, a)
completer = (s: S, b: B) =>
state.set(s) *> promise.complete(b)
_ <- receiver.redeem(
e => supervise(receiver, e)
.redeem(promise.error, completer), completer)
} yield ()).forever.fork
} yield Actor[E, A, B]((a: A) =>
Promise.make[E, B].flatMap(p => queue.offer((a, p)).flatMap(_ => p.get))
What is ZIO? Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up
Application: Scalaz Actors
def makeActor[S, E, A, B](supervisor: Supervisor)(s: S)(receive: (S, A) => IO[E, (S, B)]):
IO[Nothing, Actor[E, A, B]] =
for {
state <- Ref(s)
queue <- Queue.bounded[(A, Promise[E, B])](QueueSize)
fiber <- (for {
t <- queue.take
s <- state.get
(a, promise) = t
receiver = receive(s, a)
completer = (s: S, b: B) =>
state.set(s) *> promise.complete(b)
_ <- receiver.redeem(
e => supervise(receiver, e)
.redeem(promise.error, completer), completer)
} yield ()).forever.fork
} yield Actor[E, A, B]((a: A) =>
Promise.make[E, B].flatMap(p => queue.offer((a, p)).flatMap(_ => p.get))
What is ZIO? Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up
Application: Scalaz Actors
def makeActor[S, E, A, B](supervisor: Supervisor)(s: S)(receive: (S, A) => IO[E, (S, B)]):
IO[Nothing, Actor[E, A, B]] =
for {
state <- Ref(s)
queue <- Queue.bounded[(A, Promise[E, B])](QueueSize)
fiber <- (for {
t <- queue.take
s <- state.get
(a, promise) = t
receiver = receive(s, a)
completer = (s: S, b: B) =>
state.set(s) *> promise.complete(b)
_ <- receiver.redeem(
e => supervise(receiver, e)
.redeem(promise.error, completer), completer)
} yield ()).forever.fork
} yield Actor[E, A, B]((a: A) =>
Promise.make[E, B].flatMap(p => queue.offer((a, p)).flatMap(_ => p.get))
What is ZIO? Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up
Application: Scalaz Actors
def makeActor[S, E, A, B](supervisor: Supervisor)(s: S)(receive: (S, A) => IO[E, (S, B)]):
IO[Nothing, Actor[E, A, B]] =
for {
state <- Ref(s)
queue <- Queue.bounded[(A, Promise[E, B])](QueueSize)
fiber <- (for {
t <- queue.take
s <- state.get
(a, promise) = t
receiver = receive(s, a)
completer = (s: S, b: B) =>
state.set(s) *> promise.complete(b)
_ <- receiver.redeem(
e => supervise(receiver, e)
.redeem(promise.error, completer), completer)
} yield ()).forever.fork
} yield Actor[E, A, B]((a: A) =>
Promise.make[E, B].flatMap(p => queue.offer((a, p)).flatMap(_ => p.get))
What is ZIO? Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up
Application: Scalaz Actors
def makeActor[S, E, A, B](supervisor: Supervisor)(s: S)(receive: (S, A) => IO[E, (S, B)]):
IO[Nothing, Actor[E, A, B]] =
for {
state <- Ref(s)
queue <- Queue.bounded[(A, Promise[E, B])](QueueSize)
fiber <- (for {
t <- queue.take
s <- state.get
(a, promise) = t
receiver = receive(s, a)
completer = (s: S, b: B) =>
state.set(s) *> promise.complete(b)
_ <- receiver.redeem(
e => supervise(receiver, e)
.redeem(promise.error, completer), completer)
} yield ()).forever.fork
} yield Actor[E, A, B]((a: A) =>
Promise.make[E, B].flatMap(p => queue.offer((a, p)).flatMap(_ => p.get))
What is ZIO? Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up
Application: Scalaz Actors
def makeActor[S, E, A, B](supervisor: Supervisor)(s: S)(receive: (S, A) => IO[E, (S, B)]):
IO[Nothing, Actor[E, A, B]] =
for {
state <- Ref(s)
queue <- Queue.bounded[(A, Promise[E, B])](QueueSize)
fiber <- (for {
t <- queue.take
s <- state.get
(a, promise) = t
receiver = receive(s, a)
completer = (s: S, b: B) =>
state.set(s) *> promise.complete(b)
_ <- receiver.redeem(
e => supervise(receiver, e)
.redeem(promise.error, completer), completer)
} yield ()).forever.fork
} yield Actor[E, A, B]((a: A) =>
Promise.make[E, B].flatMap(p => queue.offer((a, p)).flatMap(_ => p.get))
What is ZIO? Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up
Application: Scalaz Actors
def makeActor[S, E, A, B](supervisor: Supervisor)(s: S)(receive: (S, A) => IO[E, (S, B)]):
IO[Nothing, Actor[E, A, B]] =
for {
state <- Ref(s)
queue <- Queue.bounded[(A, Promise[E, B])](QueueSize)
fiber <- (for {
t <- queue.take
s <- state.get
(a, promise) = t
receiver = receive(s, a)
completer = (s: S, b: B) =>
state.set(s) *> promise.complete(b)
_ <- receiver.redeem(
e => supervise(receiver, e)
.redeem(promise.error, completer), completer)
} yield ()).forever.fork
} yield Actor[E, A, B]((a: A) =>
Promise.make[E, B].flatMap(p => queue.offer((a, p)).flatMap(_ => p.get))
What is ZIO? Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up
Application: Scalaz Actors
def makeActor[S, E, A, B](supervisor: Supervisor)(s: S)(receive: (S, A) => IO[E, (S, B)]):
IO[Nothing, Actor[E, A, B]] =
for {
state <- Ref(s)
queue <- Queue.bounded[(A, Promise[E, B])](QueueSize)
fiber <- (for {
t <- queue.take
s <- state.get
(a, promise) = t
receiver = receive(s, a)
completer = (s: S, b: B) =>
state.set(s) *> promise.complete(b)
_ <- receiver.redeem(
e => supervise(receiver, e)
.redeem(promise.error, completer), completer)
} yield ()).forever.fork
} yield Actor[E, A, B]((a: A) =>
Promise.make[E, B].flatMap(p => queue.offer((a, p)).flatMap(_ => p.get))
What is ZIO? Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up
Application: Scalaz Actors
def makeCounter: IO[Nothing, Actor[Nothing, Int, Int]] =
makeActor(Supervisor.logDefects(Logging))(0) {
(state: Int, n: Int) =>
IO.now((state + n, state + n))
}
for {
counter <- makeCounter
_ <- counter ! 40
_ <- counter ! -2
v <- counter ! 0
} yield v
What is ZIO? Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up
WRAP UP
What is ZIO? Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up
github.com/scalaz/scalaz-zio
ZIO CFC
What is ZIO? Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up
ZIO ROADMAP
Performance Sliding Dropping
What is ZIO? Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up
https://gitter.im/scalaz/scalaz/
https://gitter.im/scalaz/scalaz-ioqueue/
What is ZIO? Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up
Functional Scala by John A. De Goes
Local Remote Date
Scotland European Time Sep 10 - 14
Toronto East Coast Time Oct 1 - 5
? European Time Oct 22 - 26
SF / SV Pacific Coast Time Nov 18 -
22
What is ZIO? Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up
THANK YOU!
@jdegoes - http://degoes.net

ZIO Queue

  • 1.
    ZIO Queue JohnA. De Goes — @jdegoes - http://degoes.net ScalaWave 2018 ZIO Queue ScalaWave 2018 - Gdańsk, Poland John A. De Goes @jdegoes - http://degoes.net
  • 2.
    What is ZIO? Poweredby Queues Troubled Queues Introducing ZIO Queue Wrap Up
  • 3.
    What is ZIO?Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up ZIO ZIO lets you build high-performance, type-safe, concurrent, asynchronous applications that don’t leak resources or deadlock, and are easy to reason about compositionally, test, and refactor.
  • 4.
    What is ZIO?Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up github.com/scalaz/scalaz-zio ZIO
  • 5.
    def main: Unit= { println("Hello, what is your name?") val name = readLine() println(s"Good morning, $name!") } Second-Class Effects ✗ Pass to functions ✗ Return from functions ✗ Store in data structures ✗ Async/sync/concurrency ✓ Resource-safety What is ZIO? Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up
  • 6.
    def main: IO[IOException,Unit] = for { _ <- putStrLn("Hello, what is your name?") name <- getStrLn _ <- putStrLn(s"Good morning, $name!") } yield () First-Class Effects ✓ Pass to functions ✓ Return from functions ✓ Store in data structures ✓ Async/sync/concurrency ✓ Resource-safety What is ZIO? Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up
  • 7.
    IO[E, A] IO[E, A]is an immutable value that describes an effectful computation, which may fail with a value of type E, or compute a value of type A. What is ZIO? Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up
  • 8.
    What is ZIO?Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up
  • 9.
    What is ZIO?Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up
  • 10.
    What is ZIO?Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up
  • 11.
    def runInParallel[E, A,B]( leftIO : IO[E, A], rightIO: IO[E, B]): IO[E, (A, B)] = for { leftFiber <- leftIO.fork rightFiber <- rightIO.fork a <- leftFiber.join b <- rightFiber.join } yield (a, b) Fork/Join Concurrency with Fibers ✓ Massive scalability over threads ✓ Non-blocking ✓ Automatically interruptible ✓ Always resource-safe ✓ Powerful composition What is ZIO? Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up
  • 12.
    What is ZIO?Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up Powered by Queues
  • 13.
    What is ZIO?Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up Producer Consumer offer take Queue 1-to-1
  • 14.
    What is ZIO?Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up Producer Consumer offer take Queue Producer n-to-1
  • 15.
    What is ZIO?Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up Producer Consumer offer take Queue Consumer 1-to-m
  • 16.
    What is ZIO?Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up Consumer offer take Queue Consumer Producer Producer n-to-m
  • 17.
    What is ZIO?Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up FIFO LIFO PRIO Bounded Unbounded Dropping Sliding dimensions of variation
  • 18.
    What is ZIO?Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up queue actors send
  • 19.
    What is ZIO?Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up concurrent graph search
  • 20.
    What is ZIO?Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up resource sharing scarce resource e.g. threads Thread 1 take Thread 2offer
  • 21.
    What is ZIO?Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up loose coupling offer take Fast Producers Slow Consumers buffer
  • 22.
    What is ZIO?Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up Troubled Queues
  • 23.
    What is ZIO?Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up java.util.concurrent.BlockingQueue
  • 24.
    What is ZIO?Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up java.util.concurrent.BlockingQueue trait BlockingQueue[A] { def take: A def offer(a: A): Boolean }
  • 25.
    What is ZIO?Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up java.util.concurrent.BlockingQueue Blocks one thread until element added trait BlockingQueue[A] { def take: A def offer(a: A): Boolean }
  • 26.
    What is ZIO?Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up java.util.concurrent.BlockingQueue Fast Producer Slow Consumer offer take Queue
  • 27.
    What is ZIO?Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up java.util.concurrent.BlockingQueue Fast Producer Slow Consumer offer take Queue manual backpressure
  • 28.
    What is ZIO?Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up java.util.concurrent.BlockingQueue def fastProducer = { if (!queue.offer(a)) { // Slow down!!! } else { // Keep producing... } }
  • 29.
    What is ZIO?Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up java.util.concurrent.BlockingQueue Fast Producer Slow Consumer offer take Queue manual backpressure manual backpressure take
  • 30.
    What is ZIO?Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up java.util.concurrent.ConcurrentLinkedQueue trait ConcurrentLinkedQueue[A] { def poll: A def offer(a: A): Boolean } Returns null if empty
  • 31.
    What is ZIO?Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up java.util.concurrent.ConcurrentLinkedQueue Fast Producer Slow Consumer offer poll Queue
  • 32.
    What is ZIO?Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up github.com/scalaz/scalaz-zio ZIO QUEUE
  • 33.
    What is ZIO?Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up ZIO Queue trait Queue[A] { def take: IO[Nothing, A] def offer(a: A): IO[Nothing, Unit] def shutdown(ts: Throwable*): IO[Nothing, Unit] } object Queue { def bounded[A](capacity: Int): IO[Nothing, Queue[A]] = ??? def unbounded[A]: IO[Nothing, Queue[A]] = ??? }
  • 34.
    What is ZIO?Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up Example for { queue <- Queue.unbounded[String] _ <- queue.offer("Give me Coffee!").forever.fork _ <- queue.take.flatMap(putStrLn).forever } yield ()
  • 35.
    What is ZIO?Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up 5. Concurrent Consumer offer take Queue Consumer Producer Producer ZIO
  • 36.
    What is ZIO?Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up 4. Interruptible for { queue <- Queue.unbounded[String] fiber <- queue.offer("Give me Coffee!").forever.fork _ <- fiber.interrupt } yield ()
  • 37.
    What is ZIO?Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up 3. Clean Shutdown for { queue <- Queue.unbounded[String] fiber1 <- queue.offer("Give me Coffee!").forever.fork fiber2 <- queue.take.forever.fork _ <- queue.shutdown } yield ()
  • 38.
    What is ZIO?Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up 2. Asynchronous for { queue <- Queue.unbounded[String] worker = queue.take.flatMap(putStrLn).forever workers10k = List.fill(10000)(worker) _ <- IO.forkAll(workers10k) _ <- queue.offer("More Coffee!").forever.fork } yield ()
  • 39.
    What is ZIO?Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up 2. Asynchronous trait BlockingQueue[A] { def take: A ; def offer(a: A): Boolean }
  • 40.
    What is ZIO?Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up 1. Composable Backpressure for { queue <- Queue.bounded[String](10) worker = queue.offer("Coffee").forever workers10k = List.fill(10000)(worker) _ <- IO.forkAll(workers10k) _ <- queue.take.flatMap(putStrLn).forever.fork } yield ()
  • 41.
    What is ZIO?Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up Application: Scalaz Actors case class Actor[+E, -A, +B](run: A => IO[E, B]) { self => def ! (a: A): IO[E, B] = run(a) }
  • 42.
    What is ZIO?Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up Application: Scalaz Actors def makeActor[S, E, A, B](supervisor: Supervisor)(s: S)(receive: (S, A) => IO[E, (S, B)]): IO[Nothing, Actor[E, A, B]] = for { state <- Ref(s) queue <- Queue.bounded[(A, Promise[E, B])](QueueSize) fiber <- (for { t <- queue.take s <- state.get (a, promise) = t receiver = receive(s, a) completer = (s: S, b: B) => state.set(s) *> promise.complete(b) _ <- receiver.redeem( e => supervise(receiver, e) .redeem(promise.error, completer), completer) } yield ()).forever.fork } yield Actor[E, A, B]((a: A) => Promise.make[E, B].flatMap(p => queue.offer((a, p)).flatMap(_ => p.get))
  • 43.
    What is ZIO?Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up Application: Scalaz Actors def makeActor[S, E, A, B](supervisor: Supervisor)(s: S)(receive: (S, A) => IO[E, (S, B)]): IO[Nothing, Actor[E, A, B]] = for { state <- Ref(s) queue <- Queue.bounded[(A, Promise[E, B])](QueueSize) fiber <- (for { t <- queue.take s <- state.get (a, promise) = t receiver = receive(s, a) completer = (s: S, b: B) => state.set(s) *> promise.complete(b) _ <- receiver.redeem( e => supervise(receiver, e) .redeem(promise.error, completer), completer) } yield ()).forever.fork } yield Actor[E, A, B]((a: A) => Promise.make[E, B].flatMap(p => queue.offer((a, p)).flatMap(_ => p.get))
  • 44.
    What is ZIO?Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up Application: Scalaz Actors def makeActor[S, E, A, B](supervisor: Supervisor)(s: S)(receive: (S, A) => IO[E, (S, B)]): IO[Nothing, Actor[E, A, B]] = for { state <- Ref(s) queue <- Queue.bounded[(A, Promise[E, B])](QueueSize) fiber <- (for { t <- queue.take s <- state.get (a, promise) = t receiver = receive(s, a) completer = (s: S, b: B) => state.set(s) *> promise.complete(b) _ <- receiver.redeem( e => supervise(receiver, e) .redeem(promise.error, completer), completer) } yield ()).forever.fork } yield Actor[E, A, B]((a: A) => Promise.make[E, B].flatMap(p => queue.offer((a, p)).flatMap(_ => p.get))
  • 45.
    What is ZIO?Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up Application: Scalaz Actors def makeActor[S, E, A, B](supervisor: Supervisor)(s: S)(receive: (S, A) => IO[E, (S, B)]): IO[Nothing, Actor[E, A, B]] = for { state <- Ref(s) queue <- Queue.bounded[(A, Promise[E, B])](QueueSize) fiber <- (for { t <- queue.take s <- state.get (a, promise) = t receiver = receive(s, a) completer = (s: S, b: B) => state.set(s) *> promise.complete(b) _ <- receiver.redeem( e => supervise(receiver, e) .redeem(promise.error, completer), completer) } yield ()).forever.fork } yield Actor[E, A, B]((a: A) => Promise.make[E, B].flatMap(p => queue.offer((a, p)).flatMap(_ => p.get))
  • 46.
    What is ZIO?Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up Application: Scalaz Actors def makeActor[S, E, A, B](supervisor: Supervisor)(s: S)(receive: (S, A) => IO[E, (S, B)]): IO[Nothing, Actor[E, A, B]] = for { state <- Ref(s) queue <- Queue.bounded[(A, Promise[E, B])](QueueSize) fiber <- (for { t <- queue.take s <- state.get (a, promise) = t receiver = receive(s, a) completer = (s: S, b: B) => state.set(s) *> promise.complete(b) _ <- receiver.redeem( e => supervise(receiver, e) .redeem(promise.error, completer), completer) } yield ()).forever.fork } yield Actor[E, A, B]((a: A) => Promise.make[E, B].flatMap(p => queue.offer((a, p)).flatMap(_ => p.get))
  • 47.
    What is ZIO?Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up Application: Scalaz Actors def makeActor[S, E, A, B](supervisor: Supervisor)(s: S)(receive: (S, A) => IO[E, (S, B)]): IO[Nothing, Actor[E, A, B]] = for { state <- Ref(s) queue <- Queue.bounded[(A, Promise[E, B])](QueueSize) fiber <- (for { t <- queue.take s <- state.get (a, promise) = t receiver = receive(s, a) completer = (s: S, b: B) => state.set(s) *> promise.complete(b) _ <- receiver.redeem( e => supervise(receiver, e) .redeem(promise.error, completer), completer) } yield ()).forever.fork } yield Actor[E, A, B]((a: A) => Promise.make[E, B].flatMap(p => queue.offer((a, p)).flatMap(_ => p.get))
  • 48.
    What is ZIO?Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up Application: Scalaz Actors def makeActor[S, E, A, B](supervisor: Supervisor)(s: S)(receive: (S, A) => IO[E, (S, B)]): IO[Nothing, Actor[E, A, B]] = for { state <- Ref(s) queue <- Queue.bounded[(A, Promise[E, B])](QueueSize) fiber <- (for { t <- queue.take s <- state.get (a, promise) = t receiver = receive(s, a) completer = (s: S, b: B) => state.set(s) *> promise.complete(b) _ <- receiver.redeem( e => supervise(receiver, e) .redeem(promise.error, completer), completer) } yield ()).forever.fork } yield Actor[E, A, B]((a: A) => Promise.make[E, B].flatMap(p => queue.offer((a, p)).flatMap(_ => p.get))
  • 49.
    What is ZIO?Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up Application: Scalaz Actors def makeActor[S, E, A, B](supervisor: Supervisor)(s: S)(receive: (S, A) => IO[E, (S, B)]): IO[Nothing, Actor[E, A, B]] = for { state <- Ref(s) queue <- Queue.bounded[(A, Promise[E, B])](QueueSize) fiber <- (for { t <- queue.take s <- state.get (a, promise) = t receiver = receive(s, a) completer = (s: S, b: B) => state.set(s) *> promise.complete(b) _ <- receiver.redeem( e => supervise(receiver, e) .redeem(promise.error, completer), completer) } yield ()).forever.fork } yield Actor[E, A, B]((a: A) => Promise.make[E, B].flatMap(p => queue.offer((a, p)).flatMap(_ => p.get))
  • 50.
    What is ZIO?Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up Application: Scalaz Actors def makeActor[S, E, A, B](supervisor: Supervisor)(s: S)(receive: (S, A) => IO[E, (S, B)]): IO[Nothing, Actor[E, A, B]] = for { state <- Ref(s) queue <- Queue.bounded[(A, Promise[E, B])](QueueSize) fiber <- (for { t <- queue.take s <- state.get (a, promise) = t receiver = receive(s, a) completer = (s: S, b: B) => state.set(s) *> promise.complete(b) _ <- receiver.redeem( e => supervise(receiver, e) .redeem(promise.error, completer), completer) } yield ()).forever.fork } yield Actor[E, A, B]((a: A) => Promise.make[E, B].flatMap(p => queue.offer((a, p)).flatMap(_ => p.get))
  • 51.
    What is ZIO?Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up Application: Scalaz Actors def makeActor[S, E, A, B](supervisor: Supervisor)(s: S)(receive: (S, A) => IO[E, (S, B)]): IO[Nothing, Actor[E, A, B]] = for { state <- Ref(s) queue <- Queue.bounded[(A, Promise[E, B])](QueueSize) fiber <- (for { t <- queue.take s <- state.get (a, promise) = t receiver = receive(s, a) completer = (s: S, b: B) => state.set(s) *> promise.complete(b) _ <- receiver.redeem( e => supervise(receiver, e) .redeem(promise.error, completer), completer) } yield ()).forever.fork } yield Actor[E, A, B]((a: A) => Promise.make[E, B].flatMap(p => queue.offer((a, p)).flatMap(_ => p.get))
  • 52.
    What is ZIO?Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up Application: Scalaz Actors def makeActor[S, E, A, B](supervisor: Supervisor)(s: S)(receive: (S, A) => IO[E, (S, B)]): IO[Nothing, Actor[E, A, B]] = for { state <- Ref(s) queue <- Queue.bounded[(A, Promise[E, B])](QueueSize) fiber <- (for { t <- queue.take s <- state.get (a, promise) = t receiver = receive(s, a) completer = (s: S, b: B) => state.set(s) *> promise.complete(b) _ <- receiver.redeem( e => supervise(receiver, e) .redeem(promise.error, completer), completer) } yield ()).forever.fork } yield Actor[E, A, B]((a: A) => Promise.make[E, B].flatMap(p => queue.offer((a, p)).flatMap(_ => p.get))
  • 53.
    What is ZIO?Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up Application: Scalaz Actors def makeActor[S, E, A, B](supervisor: Supervisor)(s: S)(receive: (S, A) => IO[E, (S, B)]): IO[Nothing, Actor[E, A, B]] = for { state <- Ref(s) queue <- Queue.bounded[(A, Promise[E, B])](QueueSize) fiber <- (for { t <- queue.take s <- state.get (a, promise) = t receiver = receive(s, a) completer = (s: S, b: B) => state.set(s) *> promise.complete(b) _ <- receiver.redeem( e => supervise(receiver, e) .redeem(promise.error, completer), completer) } yield ()).forever.fork } yield Actor[E, A, B]((a: A) => Promise.make[E, B].flatMap(p => queue.offer((a, p)).flatMap(_ => p.get))
  • 54.
    What is ZIO?Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up Application: Scalaz Actors def makeCounter: IO[Nothing, Actor[Nothing, Int, Int]] = makeActor(Supervisor.logDefects(Logging))(0) { (state: Int, n: Int) => IO.now((state + n, state + n)) } for { counter <- makeCounter _ <- counter ! 40 _ <- counter ! -2 v <- counter ! 0 } yield v
  • 55.
    What is ZIO?Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up WRAP UP
  • 56.
    What is ZIO?Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up github.com/scalaz/scalaz-zio ZIO CFC
  • 57.
    What is ZIO?Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up ZIO ROADMAP Performance Sliding Dropping
  • 58.
    What is ZIO?Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up https://gitter.im/scalaz/scalaz/ https://gitter.im/scalaz/scalaz-ioqueue/
  • 59.
    What is ZIO?Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up Functional Scala by John A. De Goes Local Remote Date Scotland European Time Sep 10 - 14 Toronto East Coast Time Oct 1 - 5 ? European Time Oct 22 - 26 SF / SV Pacific Coast Time Nov 18 - 22
  • 60.
    What is ZIO?Powered by Queues Troubled Queues Introducing ZIO Queue Wrap Up THANK YOU! @jdegoes - http://degoes.net