This document discusses extensible effects in Dotty, a next generation compiler for Scala. It introduces key concepts like Freer monads, which use free functors to represent effects without constraints, and Fast Freer which improves performance using type-aligned queues. Extensible Effects (ExtEff) is implemented as Freer applied to an open union, allowing different effects to be composed. This provides an extensible way to define computations with side effects in Dotty.

1. Extensible Effects
in Dotty
SanshiroYoshida @halcat0x15a
DWANGO Co.,Ltd.

2. Contents
• Dotty
• Introduction of language features
• Extensible Effects
• Explanation of the implementation
DottyとExtEffについて話します

3. About Dotty
• A next generation compiler for Scala
• Formalized in DOT
• Implemented new features
Dottyは次世代のScalaコンパイラです

4. DOT
• Dependent Object Types
• A calculus aimed as a new foundation of
Scala
• Featured path-dependent types, refinement
types, and abstract type members
DOTはScalaの基礎となる計算モデルです。

5. DOT
• Models Scala language features with
minimal calculus.
trait List { self =>
type A
def head: self.A; def tail: List { type A <: self.A }
}
def cons(x: { type A })(hd: x.A)(tl: List { type A <:
x.A }): List { type A <: x.A } =
new List { self =>
type A = x.A
def head = hd; def tail = tl
}
DOTは最小限の計算でScalaの言語機能をモデル化します

6. New language features
• Enums
• Type Lambdas
• Intersection Types
• Union Types
• Implicit Function Types
• etc.
Dottyの新しい言語機能です

7. Enums
• Syntax sugar for enumerations and ADTs.
• The companion object defines utility
methods.
enumは列挙型と代数的データ型を定義するための構文です

8. Enums
• The value of enums is tagged with Int.
enum Color {
case Red, Green, Blue
}
scala> Color.enumValue(0)
val res0: Color = Red
列挙型の値にはInt型のタグが付けられます

9. Enums
• The `enum` supports ADTs.
• ADTs can define fields and methods.
enumキーワードは代数的データ型をサポートします
enum Option[+A] {
case Some(a: A)
case None
def isEmpty: Boolean = this == None
}

10. Type Lambdas
• Representation of higher-kinded types.
Type lambdaは高階型を表現します
type Pair = [A] => (A, A)
val p: Pair[Int] = (0, 1)

11. Type Lambdas
• Possible to write partial application of type
directly
型の部分適用を直接書けるようになりました
// Scala2
type FreeMonad[F[_]] =
Monad[({ type λ[A] = Free[F, A] })#λ]
// Dotty
type FreeMonad[F[_]] = Monad[[A] => Free[F, A]]

12. Dotty
• More expressive types
• More easy-to-write syntax
• More suitable for functional programming
Dottyは関数型プログラミングにより適しています

13. About ExtEff
• Extensible Effects = Freer Monad + Open
Union + Type-aligned Queue
• Freer Monad = Free Monad + Free Functor
(Coyoneda)
• I will talk about these abstractions.
今日はこれらの抽象概念について話します

14. Monad
• Monad is a computation with side-effects.
• For example, Option monad is a
computation for which there may not exist
a value.
• You can write it procedurally with ‘for’
expression.
モナドは副作用付き計算のこと

15. Free
• Free f is a monad if f is a functor.
• Various monads can be represented using f.
Freeはファンクタによって様々なモナドを表現できます

16. Definition of Free
Pureは純粋な計算でImpureは副作用付きの計算を表します
• Pure is a pure computation.
• Impure is a computation with side-effects.
enum Free[F[_], A] {
case Pure(a: A)
case Impure(ffree: F[Free[F, A]])
}
def lift(fa: F[A])(implicit F: Functor[F]): Free[F, A] =
Impure(F.map(fa)(a => Pure(a)))

17. Free Monad
• flatMap has the constraint that F is Functor
flatMapはFがFunctorである制約を持ちます
enum Free[F[_], A] {
def flatMap[B](f: A => Free[F, B])(implicit F:
Functor[F]): Free[F, B] =
this match {
case Pure(a) => f(a)
case Impure(ffree) =>
F.map(ffree)(free => free.flatMap(f))
}
}

18. Free Writer
• Writer is a computation with another
output.
• For example, it is a computation that
outputs logs.
Writerは別の出力を持つ計算です

19. Free Writer
• Definition of Writer monad by Free.
FreeによるWriterモナドの定義です
type Writer[W, A] = Free[[T] => Tell[W, T], A]
// Effect is described CPS
case class Tell[W, A](w: W, a: A) {
def map[B](f: A => B) = Tell(w, f(a))
}
def tell[W](w: W): Writer[W, Unit] =
Free.lift(Tell(w, ()))

20. Free Writer
• An example of handler for Writer.
• Writer can output as List.
WriterはListとして出力することができます
def runAsList[W, A](free: Writer[W, A]): (List[W], A) =
free match {
case Free.Pure(a) => (Nil, a)
case Free.Impure(Tell(w, free)) =>
runAsList(free).map {
case (ws, a) => (w :: ws, a)
}
}

21. Free Writer
• Writer can output asVector.
WriterはVectorとしても出力できます
def runAsVec[W, A](free: Writer[W, A]): (Vector[W], A) =
{
def go(acc: Vector[W], free: Writer[W, A]):
(Vector[W], A) =
free match {
case Free.Pure(a) => (acc, a)
case Free.Impure(Tell(w, free)) =>
go(acc :+ w, free)
}
go(Vector.empty, free)
}

22. Free Writer
• Multiple interpretations can be made for
one expression.
一つの式に対して複数の解釈が可能です
val e = for {
_ <- tell("hoge")
_ <- tell("fuga")
} yield ()
scala> runAsList(e)
val res0: (List[String], Unit) = (List(hoge, fuga),())
scala> runAsVec(e)
val res1: (Vector[String], Unit) = (Vector(hoge, fuga),
())

23. Free
• Free represents various monads.
• Free has a functor constraint.
• Free is free to interpret.
Freeは様々なモナドを表現でき、自由に解釈できます

24. Freer
• Freer is Free applied to Coyoneda.
• Freer becomes a monad without
constraints.
• Freer uses a tree in flatMap.
Freerは制約なしにモナドになります

25. Coyoneda
• Coyoneda is Free Functor.
• For all f, Coyoneda f is a functor.
任意fについてCoyoneda fはファンクタです

26. Definition of Coyoneda
• FMap has a signature similar to map.
FMapはmapと似たシグネチャをもちます
enum Coyoneda[F[_], A] {
case FMap[F[_], A, B](fa: F[A], k: A => B) extends
Coyoneda[F, B]
}
def lift[F[_], A](fa: F[A]): Coyoneda[F, A] =
Coyoneda.FMap(fa, a => a)

27. Coyoneda Functor
• Coyoneda becomes a functor without
constraints.
Coyonedaは制約なしにファンクタになります
enum Coyoneda[F[_], A] {
def map[B](f: A => B): Coyoneda[F, B] =
this match {
case Coyoneda.FMap(fi, k) =>
Coyoneda.FMap(fi, k andThen f)
}
}

28. Coyoneda
• Using Coyoneda seems to be able to map
everything.
Coyonedaは全てをmapできるように見えます
case class Box[A](a: A)
val box = Coyoneda.lift(Box(0))
.map(i => i + 1)
.map(i => i.toString)

29. Coyoneda
• Coyoneda does not apply to the value of
the Box.
• Coyoneda#map is only composing
functions.
Coyonedaは関数の合成をしているだけで適用をしていません

30. Definition of Freer
• Expands the definition of Free Coyoneda.
• Impure has a signature similar to flatMap.
Free Coyonedaの定義を展開したものです
enum Freer[F[_], A] {
case Pure(a: A)
case Impure[F[_], A, B](fa: F[A], k: A => Freer[F, B])
extends Freer[F, B]
}
def lift[F[_], A](fa: F[A]): Freer[F, A] =
Impure(fa, a => Pure(a))

31. Freer Monad
• flatMap is a free from constraints of
Functor.
flatMapからFunctorの制約がなくなりました
enum Freer[F[_], A] {
def flatMap[B](f: A => Freer[F, B]): Freer[F, B] =
this match {
case Freer.Pure(a) => f(a)
case Freer.Impure(fa, k) =>
Freer.Impure(fa, a => k(a).flatMap(f))
}
}

32. Freer Monad
• This implementation of flatMap is slow.
• flatMap(f_0).flatMap(f_1)…flatMap(f_n) is
O(n^2).
このflatMapの実装は遅いです

33. Type-aligned Queue
• FTCQ is a sequence of functions.
• It is implemented with a tree.
FTCQは関数の列を表します
val `A => F[C]`: FTCQ[F, A, C] =
Node(
Leaf(f: A => F[B]),
Leaf(g: B => F[C])
)

34. Type-aligned Queue
• The composition of functions is constant-
time.
• The application of function is stack-safe.
合成は定数時間で、適用はスタックセーフに行われます
val `A => F[D]`: FTCQ[F, A, D] =
Node(
`A => F[C]`,
Leaf(h: C => F[D])
)

35. Definition of Fast Freer
• Impure represents a continuation with
FTCQ.
Impureは継続をFTCQで表現します
type Arrs[F, A, B] = FTCQ[[T] => Freer[F, T], A, B]
enum Freer[F[_], A] {
case Pure(a: A)
case Impure[F[_], A, B](fa: F[A], k: Arrs[F, A, B])
extends Freer[F, B]
}

36. Fast Freer Monad
• flatMap is constant-time.
flatMapは定数時間で実行されます
enum Freer[F[_], A] {
def flatMap[B](f: A => Freer[F, B]): Freer[F, B] =
this match {
case Freer.Pure(a) => f(a)
case Freer.Impure(fa, k) =>
Freer.Impure(fa, k :+ f)
}
}

37. Freer Reader
• Reader is a computation with environment.
• For example, it is a computation that takes
the configuration.
Readerは環境を持つような計算です

38. Freer Reader
• Definition of Reader monad by Freer.
FreerによるReaderモナドの定義です
type Reader[I, A] = Freer[[T] => Ask[I, T], A]
case class Ask[I, A](k: I => A)
def ask[I]: Reader[I, I] =
Freer.lift(Ask(i => i))

39. Freer Reader
• The handler of Freer applies continuation.
Freerのハンドラは継続の適用を行います
def runReader[I, A](freer: Reader[I, A], i: I): A =
freer match {
case Freer.Pure(a) => a
case Freer.Impure(Ask(f), k) =>
runReader(k(f(i)), i)
}

40. Freer Reader
• An example of a Reader monad.
Readerモナドの例です
val e = for {
x <- ask[Int]
y <- ask[Int]
} yield x + y
scala> runReader(e, 1)
val res0: Int = 2

41. Freer
• Freer has no constraints of Functor.
• Freer is faster than Free by using Type-
aligned Queue.
FreerはFunctorの制約がなくFreeよりも高速です

42. Extensible Effects
• ExtEff is Freer applied to Open Union.
• ExtEff can compose various effects using
Open Union.
ExtEffはOpen Unionを使って様々な副作用を合成できます

43. Open Union
• Representation of extensible sum of types
• Automatic construction by typeclass
Open Unionは拡張可能な型の和を表します

44. Open Union
• Union seems to be a higher-kinded type
version of Either.
Unionは高階型を使ったEitherのような定義です
enum Union[F[_], G[_], A] {
case Inl(value: F[A])
case Inr(value: G[A])
}

45. Open Union
• An alias for writing in infix notation is
useful.
中置記法で記述するための別名があると便利です
type :+:[F[_], G[_]] = [A] => Union[F, G, A]
type ListOrOption = List :+: Option :+: Nothing

46. Member
MemberはUnionに値をinjectします
trait Member[F[_], R[_]] {
def inject[A](fa: F[A]): R[A]
}
• Member injects a value into an union.

47. Member
• It is uniquely derived if a value is in Inl.
値がInlにあることでインスタンスを一意に導出できます
implicit def leftMember[F[_], G[_]] =
new Member[F, F :+: G] {
def inject[A](fa: F[A]) = Union.Inl(fa)
}
implicit def rightMember[F[_], G[_], H[_]](implicit F:
Member[F, H]) =
new Member[F, G :+: H] {
def inject[A](fa: F[A]) = Union.Inr(F.inject(fa))
}

48. Member
• An example of constructing Union using
Member.
Memberを使ってUnionを構成する例です
def inject[F[_], R[_], A](fa: F[A])(implicit F:
Member[F, R]): R[A] =
F.inject(fa)
scala> val opt: ListOrOption[Int] = inject(Option(0))
val opt: ListOrOption[Int] = Inr(Inl(Some(0)))

49. Definition of ExtEff
• lift injects effects using Member.
liftはMemberを使ってエフェクトを注入します
enum Eff[R[_], A] {
case Pure(a: A)
case Impure[R[_], A, B](union: R[A], k: Arrs[F, A, B])
extends Eff[R, B]
}
def lift[F[_], R[_], A](fa: F[A])(implicit F: Member[F,
R]): Eff[R, A] =
Impure(F.inject(fa), Arrs(a => Pure(a)))

50. ExtEff Writer
コンストラクタからモナドの値が決まります
• No longer necessary to write in CPS.
• A value of a monad is determined from the
constructor.
enum Writer[W, A] {
case Tell[W](w: W) extends Writer[W, Unit]
}
def tell[R[_], W](w: W)(implicit ev: Member[[A] =>
Writer[W, A], R]): Eff[R, Unit] = Eff.lift(Tell(w))

51. ExtEff Reader
プリミティブの関数はコンストラクタを持ち上げるだけです
• Primitive functions only lifts constructors.
enum Reader[I, A] {
case Ask[I]() extends Reader[I, I]
}
def ask[R[_], I](implicit ev: Member[[A] => Reader[I,
A], R]): Eff[R, I] = Eff.lift(Ask[I])

52. ExtEff Reader
• If another effect appears, transfer it.
他のエフェクトがあらわれた場合は処理を移譲します
def runReader[R[_], I, A](eff: Eff[([T] => Reader[I, T])
:+: R, A], i: I): Eff[R, A] =
eff match {
case Eff.Pure(a) => Free.Pure(a)
case Eff.Impure(Union.Inl(Reader.Ask()), k) =>
runReader(k(i), i)
case Eff.Impure(Union.Inr(r), k) =>
Eff.Impure(r, a => runReader(k(a), i))
}

53. ExtEff Handler
• Handlers can be generalized.
ハンドラは一般化することができます
def handleRelay[F[_], R[_], A, B]
(eff: Eff[F :+: R, A])
(pure: A => Eff[R, B])
(bind: F[A] => (A => Eff[R, B]) => Eff[R, B])
: Eff[R, B] = eff match {
case Eff.Pure(a) => pure(a)
case Eff.Impure(Union.Inl(fa), k) =>
bind(fa)(a => handleRelay(k(a))(pure)(bind))
case Eff.Impure(Union.Inr(r), k) =>
Eff.Impure(r, a => handleRelay(k(a))(pure)(bind))
}

54. ExtEff Writer
• Just write pure and flatMap with a handler.
handleRelayを使えばpureとflatMapを書くだけです。
def runWriter[R[_], W, A](eff: Eff[([T] => Writer[W, T])
:+: R, A]): Eff[R, (List[W], A)] =
handleRelay(eff)(a => (Nil, a)) {
case Writer.Tell(w) =>
k => k(()).map { case (ws, a) => (w :: ws, a) }
}

55. Run ExtEff
• If effects is Nothing, no instance of Impure
exists.
def run[A](eff: Eff[Nothing, A]): A =
eff match {
case Eff.Pure(a) => a
}
エフェクトがNothingならImpureのインスタンスは存在しない

56. ExtEff Example
• You can write two monads in one ‘for’
二つのモナドを一つのfor式に書けます
def e[R[_]](implicit r: Member[[T] => Reader[Int, T],
R], w: Member[[T] => Writer[Int, T], R]): Eff[R, Int] =
for {
x <- Reader.ask
_ <- Writer.tell(x + 1)
} yield x

57. ExtEff Example
• To run the Eff requires explicit monad stack
実行にはエフェクトスタックの明示が必要です
type Stack = ([T] => Reader[Int, T]) :+: ([T] =>
Writer[Int, T]) :+: Nothing
scala> run(runWriter(runReader(e[Stack], 0))))
val res0: (List[Int], Int) = (List(1),0)

58. Problems of ExtEff
• Requires description of type parameters
• We can not make use of type inference.
型パラメータを引き回す必要があります

59. ExtEff with Subtyping
• Make type parameters covariant.
• Replace Open Union with Dotty’s Union
types.
Open UnionをDottyのUnion typesで置き換えます

60. Union types
• Values of type A | B are all values of type A
and type B
A ¦ BはAとBの値すべてをとります
val x: String | Int =
if util.Random.nextBoolean() then "hoge" else 0

61. Tagged Union
• Tagged Union is tagged to identify the value
of Union types.
Tagged Unionは値を識別するためにタグが付けられます
case class Union[+A](tag: Tag[_], value: A)
object Union {
def apply[F[_], A](value: F[A])(implicit F: Tag[F]) =
new Union(F, value)
}

62. Tag
• Tag is implemented with ClassTag.
• It makes unique identifiers from types.
case class Tag[F[_]](value: String)
object Tag {
implicit def __[F[_, _], T](implicit F: ClassTag[F[_,
_]], T: ClassTag[T]): Tag[[A] => F[T, A]] =
Tag(s"${F}[${T}, _]")
}
型から一意な識別子を作ります

63. New ExtEff
• The type parameter R becomes covariant.
• Impure is replaced Open Union with Tagged
Union.
ImpureはOpen UnionをTagged Unionで置き換えます
enum Eff[+R[_], A] {
case Pure(a: A)
case Impure[R[_], A, B](union: Union[R[A]], k: Arrs[R,
A, B]) extends Eff[R, B]
}
def lift[F[_]: Tag, A](fa: F[A]): Eff[F, A] =
Impure(Union(fa), a => Pure(a))

64. New ExtEff
• flatMap returns an union of effects.
flatMapはエフェクトの和を返します
enum Eff[+R[_], A] {
def flatMap[S[_], B](f: A => Eff[S, B])
: Eff[[T] => R[T] | S[T], B] =
this match {
case Eff.Pure(a) => f(a)
case Eff.Impure(u, k) =>
Eff.Impure(u, k :+ f)
}
}

65. New Handler
• Uses Tag to identify an effect.
Tagを使ってエフェクトを識別します
def handleRelay[F[_], R[_], A, B]
(eff: Eff[[T] => F[T] | R[T], A])
(pure: A => Eff[R, B])
(flatMap: F[A] => (A => Eff[R, B]) => Eff[R, B])
(implicit F: Tag[F]): Eff[R, B] =
eff match {
case Eff.Pure(a) => pure(a)
case Eff.Impure(Union(`F`, fa: F[A]), k) =>
flatMap(fa)(a => handleRelay(k(a))(pure)(flatMap))
case Eff.Impure(u: Union[R[A]], k) =>
Eff.Impure(r, a => handleRelay(k(a))(pure)(flatMap))
}

66. New ExtEff Example
• You can write more simply.
• Type inference works.
よりシンプルな記述が可能になりました
val e: Eff[[T] => Reader[Int, T] | Writer[Int, T], Int]
=
for {
x <- Reader.ask[Int]
_ <- Writer.tell(x + 1)
} yield x
scala> run(runWriter(runReader(e, 0)))
val res0: (Int, Int) = (1,0)

67. Benchmarks
• Comparison of ExtEff and
MonadTransformer.
• Count up with State monad in 1,000,000
loops.
• The effect stack gets deeper.
ExtEffとMTを比較します

68. Benchmarks in ExtEff
def benchEff(ns: Seq[Int])
: Eff[[A] => State[Int, A], Int] =
ns.foldLeft(Eff.Pure(1)) { (acc, n) =>
if n % 5 == 0 then for {
acc <- acc
s <- Reader.ask[Int]
_ <- Writer.tell(s + 1)
} yield acc max n
else acc.map(_ max n)
}
ExtEffのベンチマークコードです

69. Benchmarks in MT
MTのベンチマークコードです
def benchTrans[F[_]: Monad](ns: Seq[Int])
: StateT[F, Int, Int] = {
val m = StateT.stateTMonadState[Int, F]
ns.foldLeft(StateT.stateT(1)) { (acc, n) =>
if n % 5 == 0 then for {
acc <- acc
s <- m.get
_ <- m.put(s + 1)
} yield acc max n
else acc.map(_ max n)
}
}

70. Benchmarks
• Overlays State effects
Stateモナドを重ねます
def benchEff(): (Int, Int) =
Eff.run(State.run(0)(Bench.benchEff(1 to N)))
def benchEffS(): (String, (Int, Int)) =
Eff.run(State.run("")(State.run(0)(Bench.benchEff(1 to
N))))
def benchTrans(): (Int, Int) =
Bench.benchTrans[Id](1 to N).runRec(0)
def benchTransS(): (String, (Int, Int)) =
Bench.benchTrans[[A] => StateT[Id, String, A]](1 to
N).runRec(0).runRec("")

71. Benchmarks
• Run JMH with 20 warmups,100 iterations.
ExtEffの実行は線形時間、MTの実行は二乗時間です
Benchmark Mode Cnt Score Error Units
BenchJmh.benchEffJmh thrpt 100 5.724 ± 0.220 ops/s
BenchJmh.benchEffSJmh thrpt 100 5.133 ± 0.154 ops/s
BenchJmh.benchEffSSJmh thrpt 100 4.578 ± 0.175 ops/s
BenchJmh.benchTransJmh thrpt 100 4.928 ± 0.155 ops/s
BenchJmh.benchTransSJmh thrpt 100 1.848 ± 0.107 ops/s
BenchJmh.benchTransSSJmh thrpt 100 0.852 ± 0.042 ops/s

72. Benchmarks
• The run-time of ExtEff is linear.
• The run-time of MT is quadratic.
ExtEffはUnion Typesを使うことでシンプルにかけます
ops/s
0
1.5
3
4.5
6
ExtEff MT

73. Conclusions
• You can compose multiple effects by ExtEff.
• Using the union types makes writing ExtEff
simpler.
• If the monad stack is deep, ExtEff is faster
than MT.
ExtEffはUnion Typesを使うことでシンプルにかけます

74. Thank you for listening