flatMap(Oslo) 2018

1. Advanced Tagless Final -
Saying farewell to Free
Luka Jacobowitz • flatMap(Oslo)

2. Software Developer at
codecentric
Co-organizer of ScalaDus
and IdrisDus
Maintainer of cats,
cats-effect, cats-mtl,
OutWatch
Enthusiastic about FP
About me

3. Overview
● Motivation (Free vs
Tagless Final)
● Program optimization
● Interpreter
transformations
● Stack-safety
● Conclusions

4. Motivation
● Tagless Final is cool
● Certain problems are still very hard to solve
while staying within the constraints of the
interpreter pattern
● Learn about new libraries to help with these
problems
● Have fun along the way!

5. Free vs. Tagless

6. Free vs. Tagless (For interpreter pattern)
Advantage Free
● Free Applicative allows us
to inspect the inner
structure of programs and
optimize
● Stack safe by default
Advantage Tagless Final
● Almost no boilerplate
● Much more performant
● Not married to the
Monad/Applicative
constraint

7. Program optimization
“Optimization in general requires peek ahead which
requires data structures”
With Free we have a data structure, how could it
possibly work for Tagless Final?

8. Program optimization
Solution:
Interpret it twice!
With a little help from sphynx!

9. Program optimization - example
trait KVStore[F[_]] {
def get(key: String): F[Option[String]]
def put(key: String, a: String): F[Unit]
}
def program[F[_]: Applicative](gets: List[String])
(puts: List[(String, String)])
(F: KVStore[F]): F[List[String]] =
puts.traverse(t => F.put(t._1, t._2))
*> gets.traverse(F.get).map(_.flatten)

10. Program optimization - example
What are some potential optimizations for this
program?
● Run actions in parallel
● Remove duplicates
● Put and then Get with same key should not perform
a Get action

11. First step: Extracting information from our program
We pre-interpret our program to get the information
we need to optimize
● To do so, we need an Applicative F[_]
● We can “lift” any Monoid into an Applicative
using Const.
● Our pre-interpreter should be of type
KVStore[Const[M, ?]]

12. Extraction
case class KVStoreInfo(gets: Set[String], puts: Map[String, String])
val extractor = new KVStore[Const[KVStoreInfo, ?]] {
def get(key: String): Const[KVStoreInfo, Option[String]] =
Const(KVStoreInfo(Set(key), Map.empty))
def put(key: String, v: String): Const[KVStoreInfo, Unit] =
Const(KVStoreInfo(Set.empty, Map(key -> v)))
}
val extracted: KVStoreInfo = program(gs, ps)(extractor).getConst

13. Next step: Defining a new interpreter using our new info
Now that we have the information we desire, we can
use it to define an optimized interpreter
● We could precompute values and store them
● That way our interpreter only has to look up the
values
● Since this will be effectful, it will be of type
IO[KVStore[IO]] meaning an IO that will compute a
new IO-interpreter for KVStore

14. Optimizing
val optimizedInterp = info.gets.filterNot(info.puts.contains)
.parTraverse(key => interp.get(key).map(_.map(s => (key, s))))
.map { list: List[Option[(String, String)]] =>
val table: Map[String, String] = list.flatten.toMap
new KVStore[IO] {
def get(key: String): IO[Option[String]] =
table.get(key).orElse(info.puts.get(key)) match {
case Some(a) => Option(a).pure[IO]
case None => interp.get(key)
}
def put(key: String, v: String): IO[Unit] = interp.put(key, v)
}
}

15. Let’s put it all together
val interp: KVStore[IO] = ???
val gets = List("Dog", "Bird", "Mouse", "Bird")
val puts = List("Cat" -> "Cat!", "Dog" -> "Dog!")
val info: KVStoreInfo = program(gets, puts)(extractor).getConst
val result: IO[List[String]] = program(gets, puts)(optimizedInterp)
val naiveResult: IO[List[String]] = program(gets, puts)(interp)

16. Result: initial naive interpreter
naiveResult.unsafeRunSync()
// Hit Network for: Put Cat -> Cat!
// Hit Network for: Put Dog -> Dog!
// Hit Network for: Get Dog
// Hit Network for: Get Bird
// Hit Network for: Get Mouse
// Hit Network for: Get Bird

17. Result: optimized interpreter
result.unsafeRunSync()
// Hit Network for: Get Bird
// Hit Network for: Get Mouse
// Hit Network for: Put Cat -> Cat!
// Hit Network for: Put Dog -> Dog!

18. Can we do better?
More precisely, can we generalize this?
Yes!

19. The Sphynx Optimizer
trait Optimizer[Alg[_[_]], F[_]: Monad] {
type M: Monoid
def extract: Alg[Const[M, ?]]
def rebuild(m: M, interpreter: Alg[F]): F[Alg[F]]
def optimize[A](p: Alg[F] => F[A]): Alg[F] => F[A] =
{ (interpreter: Alg[F]) =>
val m: M = p(extract).getConst
rebuild(m, interpreter).flatMap(interp => p(interp))
}
}

20. Lifting into a larger context
import sphynx.Optimizer
import sphynx.syntax._
def monadicProgram[F[_]: Monad](F: KVStore[F])
(implicit O: Optimizer[KVStore, F]): F[Unit] = for {
cat <- F.get("Cat")
list <- program(cat.toList, List.empty).optimize(F)
_ <- F.put("Birds", list.headOption.getOrElse("Finch"))
} yield ()

21. More cool sphynx features
● StaticOptimizer
● SemigroupalOptimizer

22. Interpreter transformations

23. Interpreter transformation
Given an algebra of the form Alg[_[_]] and appropriate
interpreter will have the form Alg[F] where F is the type
we’re interpreting into.
How can we turn an Alg[F] into an Alg[G]?
FunctorK!

24. Mainecoon - FunctorK
trait FunctorK[A[_[_]] {
def mapK[F[_], G[_]](a: A[F])(n: F ~> G): A[G]
}
@autoFunctorK
trait KVStore[F[_]] {
def get(key: String): F[Option[String]]
def put(key: String, a: String): F[Unit]
}

25. Mainecoon - FunctorK
val toTask: IO ~> Task = λ[IO ~> Task](_.to[Task])
val taskInterp: KVStore[Task] =
KVStore[IO].mapK(toTask)

26. Stack safety
Tagless final programs are only stack safe when their target
monad is stack safe.
Free Monads on the other hand guarantee this to be the case.
Solution?
Interpret program into Free
Made super easy with Mainecoon!

27. Mainecoon - Stack safety
def toFree[F[_]]: F ~> Free[F, ?] =
λ[F ~> Free[F, ?]](t => Free.liftF(t))
program(tryInterp.mapK(toFree))
.foldMap(FunctionK.id)

28. More cool Mainecoon features
● InvariantK, ContravariantK
● CartesianK (SemigroupalK)

29. Conclusions
Tagless final is great for separating problem description
from problem solution.
With these additional approaches we can keep this layer of
abstraction, while sacrificing none of the performance or
stack safety.
Scala as a language isn’t quite there yet to make full use
of some of these advanced techniques, but we can find
workarounds.

30. Thank you for
listening!
Twitter: @LukaJacobowitz
GitHub: LukaJCB