- 1. Implementing Higher- Kinded Types in Dotty Martin Odersky Scala Symposium, October 2016, Amsterdam joint work with Guillaume Martres, Dmitry Petrashko
- 2. Dotty • The new Scala compiler developed at LAMP • Based on DOT as a theoretical foundation • Long-standing question: How to implement higher-kinded types on these foundations? • The paper and talk investigate 4 (!) different solutions to do so.
- 4. Foundations: DOT The DOT calculus is intended to be a minimal foundation of Scala. Its type structure is a blueprint for the types used internally in the compiler.
- 5. DOT Terms • Translated to Scala notation, the language covered by DOT is: Value v = (x: T) => t Function new { x: T => d } Object Definition d = def a = t Method definition type A = T Type Term t = v Value x Variable t1(t2) Application t.a Selection { val x = t1; t2 } Local definition.
- 6. DOT Types The Types covered by DOT are: Type T = Any Top type Nothing Bottom type x.A Selection (x: T1) => T2 Function { def a: T } Method declaration { type T >: T1 <: T2 } Type declaration T1 & T2 Intersection { x => T } Recursion
- 7. DOT Types The Types covered by DOT are: Type T = Any Top type Nothing Bottom type x.A Selection (x: T1) => T2 Function { def a: T } Method declaration { type T >: T1 <: T2 } Type declaration T1 & T2 Intersection { x => T } Recursion Should Scala have these?
- 8. DOT Types The Types covered by DOT are: Type T = Any Top type Nothing Bottom type x.A Selection (x: T1) => T2 Function { def a: T } Method declaration { type T >: T1 <: T2 } Type declaration T1 & T2 Intersection { x => T } RecursionScala has only refinements T { d1 … dn} with this as self reference.
- 10. Lists, Encoded
- 11. A Variant of Variance
- 13. Why Encode Parameterization? • One fundamental concept (type member selection) instead of two. • Clarifies interaction between parameterization and membership. E.g. List[_ <: Number] = List { type T <: Number }
- 15. Higher-Kinded Types Problem: How to encode type C[X] = List[X] type C[+X <: Number] <: List[X] ? Idea: Generalize definition of type parameters: • of a class: designated member types • of an alias type: type params of its alias • of an abstract type: type params of upper bound
- 16. The Simple Encoding Problem: How to encode type C[X] <: List[X] type D[X <: Number] = Cons[X] ? This leads to: type C <: List type D = List { type T <: Number }
- 17. The Simple Encoding Problem: How to encode type C[X] <: List[X] type D[X <: Number] = Cons[X] ? This leads to: type C <: List List[_] type D = List { type T <: Number } List[_ <: Number] Note that these encodings can also be seen as wildcard (existential) types!
- 18. Limitations of the Simple Encoding Unfortunately, there are things that can’t be encoded that way: type Rep[T] = T type LL[T] = List[List[T]] type RMap[V, K] = Map[K, V] What To Do?
- 20. The Projection Encoding Idea (originally due to Adriaan Moors): Simulate type application using type selection. E.g. type C[X] <: List[X] becomes type C <: Lambda$P { type $hk0 type $Apply <: List { type T = $hk0 } }
- 21. The Projection Encoding Idea (originally due to Adriaan Moors): Simulate type application using type selection. Then C[String] becomes C { type $hk0 = String } # $Apply
- 22. The Projection Encoding Idea (originally due to Adriaan Moors): Simulate type application using type selection. If we instantiate C to List, say, we get: type C = Lambda$P { type $hk0 type $Apply = List { type T = $hk0 } }
- 23. The Projection Encoding Hence, C[String] = C { type $hk0 = String } # $Apply = Lambda$P { type $hk0; type $Apply = List { type T = $hk0 } } { type $hk0 = String } # $Apply =:= List { type $hk0 = String } = List[String] Where “=:=” is interpreted as “being mutual subtype of”.
- 24. Problems with the Projection Enc. • There were several, but the killer was: It relies on type projection T#U, which is unsound in general and will therefore be phased out. • See the paper for a discussion in detail.
- 26. The Refinement Encoding Idea: Encode type C[X] <: List[X] C[String] as type C <: { z => List { type T = $hk0 } type $hk0 } C { type $hk0 = String } This also had several problems which are described in the paper.
- 27. In the end… … we represented hk types directly, using special type forms for • Type lambda: [X] -> T TypeLambda • Higher-kinded type application: C[T] HKApply • A beta reduction rule: ([X] -> T)[U] à [X := U]T • Rules for subtyping, type inference …
- 28. Implementation Data Simple Encoding Projection Encoding Refinement Encoding Direct Repr. Total type- checker + core: 28’000 cloc
- 29. Evaluation • The simple encoding is a modest extension for a language with wildcards or variance that wants to support basic concepts associated with hk types. • Cheap way to get a lot of the advantages of hk. • Difficulty: Explain what’s legal and what is not. • Other encodings were less successful than they looked at first. • A direct representation is needed for robust support of full higher kinded types.
- 30. Still Missing • A theory that explains what the compiler did in terms of DOT. • Hopefully, we can find something that’s small and fits well.
- 31. A Take Away • Compiler hacking is a possible way to validate ideas • But it may not be the most efficient one. • If we would have a better theory how higher- kinded types fit into DOT, we could have avoided some costly experiments.