Beyond Mere Actors

  Concurrent Functional
 Programming with Scalaz

    Rúnar Bjarnason
Traditional Java Concurrency

  Manual creation of threads
  Manual synchronisation
  One thread per process
  Processes communicate by shared mutable state
Traditional Java Concurrency

  Manual creation of threads
  Manual synchronisation
  One thread per process
  Processes communicate by shared mutable state
  Problem: Threads do not compose
java.util.concurrent

Since JDK 5

Includes useful building blocks for making higher-level
abstractions.
Atomic references
Countdown latches
ExecutorService
Callable
Future
Futures

Provided by java.util.concurrent.Future

Future[A] represents a computation of type A, executing
concurrently.

Future.get: A
Futures

ExecutorService is a means of turning Callable[A] into Future
[A]
implicit def toCallable[A](a: => A) =
   new Callable[A] { def call = a }
implicit val s = Executors.newCachedThreadPool

e1 and e2 are evaluated concurrently
val x = s submit { e1 }
val y = e2

Futures can participate in expressions:
val z = f(x.get, y)

No mention of threads in this code. No shared state.
Futures

There's a serious problem. How would we implement this
function?

def fmap[A,B](fu: Future[A], f: A => B)
  (implicit s: ExecutorService):Future[B] =
Futures

We have to call Future.get, blocking the thread.

def fmap[A,B](fu: Future[A], f: A => B)
  (implicit s: ExecutorService):Future[B] =
    s submit f(fu.get)

This is a barrier to composition.

Futures cannot be composed without blocking a thread.
scalaz.concurrent

Strategy - Abstracts over ways of evaluating expressions
concurrently.

Actor - Light-weight thread-like process, communicates by
asynchronous messaging.

Promise - Compose concurrent functions.
Parallel Strategies

ExecutorService: Callable[A] => Future[A]

Callable[A] ~= Function0[A]
Future[A] ~= Function0[A]
Also written: () => A

Strategy[A] ~= Function0[A] => Function0[A]
Also written: (() => A) => () => A

Turns a lazy expression of type A into an expression of the
same type.
scalaz.concurrent.Strategy

Separates the concerns of parallelism and algorithm.
Captures some threading pattern and isolates the rest of your
code from threading.

Executor - wraps the expression in a Callable, turns it into a
Future via an implicit ExecutorService.

Naive - Starts a new thread for each expression.

Sequential - Evaluates each expression in the current thread
(no concurrency).

Identity - Performs no evaluation at all.
Parallel Strategies

You can write your own Strategies.

Some (crazy) ideas:
  Delegate to the fork/join scheduler.
  Run in the Swing thread.
  Call a remote machine.
  Ask a human, or produce a random result.
Actors

Provide asynchronous communication among processes.

Processes receive messages on a queue.

Messages can be enqueued with no waiting.

An actor is always either suspended (waiting for messages) or
working (acting on a message).

An actor processes messages in some (but any) order.
scalaz.concurrent.Actor

These are not scala.actors. Differences:
   Simpler. Scalaz Actors are distilled to the essentials.
   Messages are typed.

Actor is sealed, and instantiated by supplying:
   type A
   effect: A => Unit
   (implicit) strategy: Strategy[Unit]
   (Optional) Error Handler: Throwable => Unit
Strategy + Effect = Actor
Actor Example
Actor: Contravariant Cofunctor

An actor can be composed with a function:
    x.comap(f)

Comap has this type:
   comap: (B => A) => Actor[A] => Actor[B]

Returns a new actor that applies f to its messages and sends
the result to actor x.

x comap f is equivalent to x compose f, but results in an
Actor, as opposed to a function.
Problems with Actors
Problems with Actors

   You have to think about state and process communication.
   An actor can (must?) expose its state.
   It's all about side-effects!

Side-effects absolutely do not compose.
You cannot compose Actors with each other.

Actor[A] ~= (A => Unit)

There's not a lot you can do with Unit.
scalaz.concurrent.Promise

Similar to Future, but non-blocking.

Implements map and flatMap without calling get.
scalaz.concurrent.Promise

Constructed by giving an expression to promise:

lazy val e:String = {Thread sleep 5000; "Hi."}
val p: Promise[String] = promise(e)

Takes an implicit Strategy[Unit]. The expression is
evaluated concurrently by the Strategy.
Think of this as forking a process.

The result is available later by calling p.get. This blocks the
current thread.

But we never have to call it!
On Time-Travel

Promised values are available in the future.

What does it mean to get a value out of the future?
Time-travel into the future is easy. Just wait.
But we don't have to go into the future.
We can give our future-selves instructions.

Instead of getting values out of the future, we send
computations into the future.
Lifting a function into the future

Consider: promise(e).map(f)

map has the following type:
(A => B) => Promise[A] => Promise[B]

We take an ordinary function and turn it into a function that
operates on Promises.

It's saying: Evaluate e concurrently, applying f to the result
when it's ready. Returns a Promise of the final result.
Composing concurrent functions

A concurrent function is of the type A => Promise[B]

Syntax sugar to make any function a concurrent function:
val g = f.promise
promise(f: A => B) = (a:A) => promise(f(a))

We can bind the arguments of concurrent functions to promised
values, using flatMap:
promise(e).flatMap(g)

flatMap has this type:
(A => Promise[B]) => Promise[A] => Promise[B]
Composing concurrent functions

We can compose concurrent functions with each other too.

If f: A => Promise[B] and g: B => Promise[C] then
(f >=> g): A => Promise[C]

(f >=> g)(x) is equivalent to f(x) flatMap g
Joining Promises

join[A]: Promise[Promise[A]] => Promise[A]

(promise { promise { expression } }).join

Join removes the "inner brackets".

A process that forks other processes can join with them later,
without synchronizing or blocking.
A process whose result depends on child processes is still just
a single Promise, and thus can run in a single thread.

Therefore, pure promises cannot deadlock or starve.
Promises - Example
But wait, there's more!

Parallel counterparts of map, flatMap, and zipWith:
parMap, parFlatMap, and parZipWith

x.parMap(f)

Where x can be a List, Stream, Function, Option, Promise, etc.
Scalaz provides parMap on any Functor.

parZipWith for parallel zipping of any Applicative Functor.

parFlatMap is provided for any Monad.
Advanced Topics

If you understand Promise, then you understand monads.
Advanced Topics

Functor is simply this interface:

trait Functor[F[_]] {
  def fmap[A, B](r: F[A], f: A => B): F[B]
}

Functors are "mappable". Any implementation of this interface
is a functor. Here's the Promise functor:
new Functor[Promise] {
   def fmap[A, B](t: Promise[A], f: A => B) =
       t.flatMap(a => promise(f(a)))
}
Advanced Topics

Monad is simply this interface:
trait Monad[M[_]] extends Functor[M] {
  fork[A](a: A): M[A]
  join[A](a: M[M[A]]): M[A]
}

Monads are fork/map/joinable. Any implementation of this
interface is a monad. Here's the Promise monad:
new Monad[Promise] {
   def fork[A](a: A) = promise(a)
   def join[A](a: Promise[Promise[A]]) =
      a.flatMap(Functions.identity)
}
Welcome to Scalaz

Scalaz is a general-purpose library for higher-order
programming.

There's a lot here. Go play around, and ask questions on the
Scalaz Google Group.

For more information:
http://code.google.com/p/scalaz

Documentation is lacking, but we're working on that.

A release of Scalaz 5.0 will coincide with a release of Scala
2.8.
Questions?

Beyond Mere Actors

  • 1.
    Beyond Mere Actors Concurrent Functional Programming with Scalaz Rúnar Bjarnason
  • 2.
    Traditional Java Concurrency Manual creation of threads Manual synchronisation One thread per process Processes communicate by shared mutable state
  • 3.
    Traditional Java Concurrency Manual creation of threads Manual synchronisation One thread per process Processes communicate by shared mutable state Problem: Threads do not compose
  • 4.
    java.util.concurrent Since JDK 5 Includesuseful building blocks for making higher-level abstractions. Atomic references Countdown latches ExecutorService Callable Future
  • 5.
    Futures Provided by java.util.concurrent.Future Future[A]represents a computation of type A, executing concurrently. Future.get: A
  • 6.
    Futures ExecutorService is ameans of turning Callable[A] into Future [A] implicit def toCallable[A](a: => A) = new Callable[A] { def call = a } implicit val s = Executors.newCachedThreadPool e1 and e2 are evaluated concurrently val x = s submit { e1 } val y = e2 Futures can participate in expressions: val z = f(x.get, y) No mention of threads in this code. No shared state.
  • 7.
    Futures There's a seriousproblem. How would we implement this function? def fmap[A,B](fu: Future[A], f: A => B) (implicit s: ExecutorService):Future[B] =
  • 8.
    Futures We have tocall Future.get, blocking the thread. def fmap[A,B](fu: Future[A], f: A => B) (implicit s: ExecutorService):Future[B] = s submit f(fu.get) This is a barrier to composition. Futures cannot be composed without blocking a thread.
  • 9.
    scalaz.concurrent Strategy - Abstractsover ways of evaluating expressions concurrently. Actor - Light-weight thread-like process, communicates by asynchronous messaging. Promise - Compose concurrent functions.
  • 10.
    Parallel Strategies ExecutorService: Callable[A]=> Future[A] Callable[A] ~= Function0[A] Future[A] ~= Function0[A] Also written: () => A Strategy[A] ~= Function0[A] => Function0[A] Also written: (() => A) => () => A Turns a lazy expression of type A into an expression of the same type.
  • 11.
    scalaz.concurrent.Strategy Separates the concernsof parallelism and algorithm. Captures some threading pattern and isolates the rest of your code from threading. Executor - wraps the expression in a Callable, turns it into a Future via an implicit ExecutorService. Naive - Starts a new thread for each expression. Sequential - Evaluates each expression in the current thread (no concurrency). Identity - Performs no evaluation at all.
  • 12.
    Parallel Strategies You canwrite your own Strategies. Some (crazy) ideas: Delegate to the fork/join scheduler. Run in the Swing thread. Call a remote machine. Ask a human, or produce a random result.
  • 13.
    Actors Provide asynchronous communicationamong processes. Processes receive messages on a queue. Messages can be enqueued with no waiting. An actor is always either suspended (waiting for messages) or working (acting on a message). An actor processes messages in some (but any) order.
  • 14.
    scalaz.concurrent.Actor These are notscala.actors. Differences: Simpler. Scalaz Actors are distilled to the essentials. Messages are typed. Actor is sealed, and instantiated by supplying: type A effect: A => Unit (implicit) strategy: Strategy[Unit] (Optional) Error Handler: Throwable => Unit Strategy + Effect = Actor
  • 15.
  • 16.
    Actor: Contravariant Cofunctor Anactor can be composed with a function: x.comap(f) Comap has this type: comap: (B => A) => Actor[A] => Actor[B] Returns a new actor that applies f to its messages and sends the result to actor x. x comap f is equivalent to x compose f, but results in an Actor, as opposed to a function.
  • 17.
  • 18.
    Problems with Actors You have to think about state and process communication. An actor can (must?) expose its state. It's all about side-effects! Side-effects absolutely do not compose. You cannot compose Actors with each other. Actor[A] ~= (A => Unit) There's not a lot you can do with Unit.
  • 19.
    scalaz.concurrent.Promise Similar to Future,but non-blocking. Implements map and flatMap without calling get.
  • 20.
    scalaz.concurrent.Promise Constructed by givingan expression to promise: lazy val e:String = {Thread sleep 5000; "Hi."} val p: Promise[String] = promise(e) Takes an implicit Strategy[Unit]. The expression is evaluated concurrently by the Strategy. Think of this as forking a process. The result is available later by calling p.get. This blocks the current thread. But we never have to call it!
  • 21.
    On Time-Travel Promised valuesare available in the future. What does it mean to get a value out of the future? Time-travel into the future is easy. Just wait. But we don't have to go into the future. We can give our future-selves instructions. Instead of getting values out of the future, we send computations into the future.
  • 22.
    Lifting a functioninto the future Consider: promise(e).map(f) map has the following type: (A => B) => Promise[A] => Promise[B] We take an ordinary function and turn it into a function that operates on Promises. It's saying: Evaluate e concurrently, applying f to the result when it's ready. Returns a Promise of the final result.
  • 23.
    Composing concurrent functions Aconcurrent function is of the type A => Promise[B] Syntax sugar to make any function a concurrent function: val g = f.promise promise(f: A => B) = (a:A) => promise(f(a)) We can bind the arguments of concurrent functions to promised values, using flatMap: promise(e).flatMap(g) flatMap has this type: (A => Promise[B]) => Promise[A] => Promise[B]
  • 24.
    Composing concurrent functions Wecan compose concurrent functions with each other too. If f: A => Promise[B] and g: B => Promise[C] then (f >=> g): A => Promise[C] (f >=> g)(x) is equivalent to f(x) flatMap g
  • 25.
    Joining Promises join[A]: Promise[Promise[A]]=> Promise[A] (promise { promise { expression } }).join Join removes the "inner brackets". A process that forks other processes can join with them later, without synchronizing or blocking. A process whose result depends on child processes is still just a single Promise, and thus can run in a single thread. Therefore, pure promises cannot deadlock or starve.
  • 26.
  • 27.
    But wait, there'smore! Parallel counterparts of map, flatMap, and zipWith: parMap, parFlatMap, and parZipWith x.parMap(f) Where x can be a List, Stream, Function, Option, Promise, etc. Scalaz provides parMap on any Functor. parZipWith for parallel zipping of any Applicative Functor. parFlatMap is provided for any Monad.
  • 28.
    Advanced Topics If youunderstand Promise, then you understand monads.
  • 29.
    Advanced Topics Functor issimply this interface: trait Functor[F[_]] { def fmap[A, B](r: F[A], f: A => B): F[B] } Functors are "mappable". Any implementation of this interface is a functor. Here's the Promise functor: new Functor[Promise] { def fmap[A, B](t: Promise[A], f: A => B) = t.flatMap(a => promise(f(a))) }
  • 30.
    Advanced Topics Monad issimply this interface: trait Monad[M[_]] extends Functor[M] { fork[A](a: A): M[A] join[A](a: M[M[A]]): M[A] } Monads are fork/map/joinable. Any implementation of this interface is a monad. Here's the Promise monad: new Monad[Promise] { def fork[A](a: A) = promise(a) def join[A](a: Promise[Promise[A]]) = a.flatMap(Functions.identity) }
  • 31.
    Welcome to Scalaz Scalazis a general-purpose library for higher-order programming. There's a lot here. Go play around, and ask questions on the Scalaz Google Group. For more information: http://code.google.com/p/scalaz Documentation is lacking, but we're working on that. A release of Scalaz 5.0 will coincide with a release of Scala 2.8.
  • 32.