Demystifying
Shapeless
Jared Roesch
@roeschinc
https://github.com/jroesch
1

Who I am
2

What I do:
PL Lab
3

What is Shapeless?
• https://github.com/milessabin/shapeless
• A library from Miles Sabin and co.
• A playground for advanced typed FP
4

It has lots of features
5

object size extends Poly1 {
implicit def int = at[Int] (x => 1)
implicit def string = at[String] (s => s.length)
implicit def tuple[A, B] = at[(A, B)] (t => 2)
}
!
val list = 1 :: “foo” :: (‘x’, ‘a’) :: HNil
list.map(size) // => 1 :: 3 :: 2 :: HNil
We can do flexible things like:
6

Inspiration
• Scrap Your Boilerplate
• Generic Deriving in Haskell
• Dependently typed languages
7

Static reasoning
• allows for us to implement powerful compile time
abstractions
• you only pay a minor cost with extra compile time,
and passing around proof terms
8

Type safe casting of an arbitrary List to Product:
9
case class Point(x: Int, y: Int)
val xs: List[Any] = List(1,2)
xs.as[Point]
https://gist.github.com/jroesch/52727c6d77a9d98458d5

scala> val q = sql("select name, age from person")
scala> q() map (_ get "age")
res0: List[Int] = List(36, 14)
compile time checked SQL:
from: https://github.com/jonifreeman/sqltyped
10

Dependent Types
• relaxation of the “phase distinction”
• simply: remove the distinction between types and
values (allow types to depend on values)
• many things billed as type level programming are
attempts at emulating dependent types
11

The phase distinction exists in Scala:
!
Vect : (Nat, Type) => Type
!
we would like to write something like this:
!
trait Vect[N <: Nat, A]
!
we are still writing this:
!
Vect : (Type, Type) => Type
12

Types as Logic
• type systems == logical framework
• types correspond to propositions, and programs to
proofs
• types can be boring (i.e Int, String, User); not all
proofs are interesting
13

How do we emulate
dependent types?
• We still have the phase distinction to get around
• We need to promote values to the type level (i.e 0
can either act as a value or type)
• Vect[0, Int] and val x = 0
14

Nat
• Natural numbers (0, 1 …)
• Let’s look at both the value level and the type level
• We need these to reason about numeric constraints
like size, ordering, indexing, and so on.
• Usually represented by a Zero element and a
Successor function.
15

sealed trait Nat
case object Zero extends Nat
case class Succ(n : Nat) extends Nat
16
A value representing a Natural number:

How do we encode a type level representation of naturals?
17

Prerequisites
‣ implicit arguments
‣ sub-typing, and bounded polymorphism
‣ type members
‣ structural refinement types
‣ path dependent types
18

implicit val x: Int = 10
!
def needsImplicit(implicit ev: Int) = ???
implicit arguments allow us to pass extra parameters
around with help of the compiler:
19

type members allow for values to have type information:
trait User { type Email; val email : Email }
!
20

def userWithEmail[E](e : E) = new User {
type Email = E
val email = e
}
!
21
We make a type part of the value:

val user = userWithEmail(“jroesch@invoca.com”)
val email : user.Email = user.email
!
def takesUser(u: User) =
/* the type of email is hidden here */
22
The type is now existential:

also we make use of structural refinement types:
sealed trait Database
!
sealed trait Postgres extends Database
case object Postgres extends Postgres
!
sealed trait MySQL extends Database
case object MySQL extends MySQL
!
trait DataStore { type DB <: Database … }
23

// Refined type
type PostgreSQL = DataStore { type DB = Postgres }
!
def writeJSONB(ds: PostgreSQL, jsonb: JSONB) = ???
def dontCare(ds: DataStore, …) = ???
!
val postgres: PostgresSQL = new DataStore { … }
!
val otherDataStore = new DataStore { … }
!
writeJSONB(postgres, value) //works
writeJSONB(otherDataStore, value) //fails
24
We can use this to make our type more specific:

trait Unrefined
type Refined = Unrefined { type T = String }
!
implicitly[Refined <:< Unrefined]
25
refined types are subtypes of unrefined types:

trait ObligationFor[N <: Nat]
!
def proof[N <: Nat](implicit ev: ObligationFor[N])
26
we can use this sub-typing rule and type bounds to our
advantage during implicit selection:
vs
def proof(implicit ev: ObligationFor[Nat])

/* Shapeless Typelevel Natural */
trait Nat {
type N <: Nat
}
!
class _0 extends Nat {
type N = _0
}
!
class Succ[P <: Nat]() extends Nat {
type N = Succ[P]
}
27

implicit def intToNat(i: Int): Nat = …
!
val n: Nat = 1
!
type One = n.N // Succ[_0]
28
We can add an implicit conversion for Naturals:

def lessThan(n: Nat, m: Nat): Bool =
match (n, m) {
case (Zero, Succ(_)) =>
true
case (Succ(np), Succ(mp)) =>
lessThan(np, mp)
case (_, _) =>
false
}
How do we translate a value level algorithm to one
that constructs a proof object instead
29

How do we translate this piece of code?
30

trait LessThan[N <: Nat, M <: Nat]
31
Take our proof and translate it to a type:

// Typelevel LessThan
trait LessThan[N <: Nat, M <: Nat]
!
// We need ways to construct our proofs.
implicit def lessThanBase[M <: Nat] =
new LessThan[_0, Succ[M]] {}
!
implicit def lessThanStep[N <: Nat, M <: Nat]
(implicit lessThanN: LessThan[N, M]) =
new LessThan[Succ[N], Succ[M]] {}
!
def lessThan(n : Nat, m : Nat)
(implicit lessThan: LessThan[n.N, m. N]): Boolean =
true
32

HList
sealed trait HList
!
case class ::[+H, +T <: HList](head : H, tail : T)
extends HList
!
sealed trait HNil extends HList
case object HNil extends HNil
33

import shapeless._
!
val xs : Int :: String :: HNil = 1 :: “foo” :: HNil
34

trait IsHCons[L <: HList] {
type H
type T <: HList
def head(l : L) : H
def tail(l : L) : T
}
35

object IsHCons {
…
!
type Aux[L <: HList, Head, Tail <: HList] =
IsHCons[L] { type H = Head; type T = Tail }
!
implicit def hlistIsHCons[Head, Tail <: HList] =
new IsHCons[Head :: Tail] {
type H = Head
type T = Tail
def head(l : Head :: Tail) : H = l.head
def tail(l : Head :: Tail) : T = l.tail
}
}
36

def head(implicit c : IsHCons[L]) : c.H = c.head(l)
!
def tail(implicit c : IsHCons[L]) : c.T = c.tail(l)
We then demand proof when we implement methods on
HList’s:
37

Proofs as black boxes
• Proof objects can be treated as black boxes, we
only need to know what relationship they express,
not proof details.
• We can use shapeless as a standard library of
useful tools.
38

case class Point(x: Int, y: Int)
!
val generic = Generic.Aux[Point, Int :: Int :: HNil] =
Generic[Point]
!
val point = Point(1,2)
!
val list: Int :: Int :: HNil = generic.to(point)
!
assert(generic.from(list) == point)
39

Applying it
• We can build things using many of the same ideas
• typed SQL, JSON with schema, static string
encoding, and plenty of other uses (ex. Spray)
40

41
sealed trait Encoding
…
!
trait EncodedString[E <: Encoding] { … }
…
!
def staticEncoding[E <: Encoding](enc: E, s: String)
= macro StaticEncoding.encodeAs[E]

42
trait Transcode[Initial <: Encoding] {
type Result <: Encoding
!
def transcode(s: EncodedString[Initial]):
EncodedString[Result]
}

43
trait Concatable[Prefix <: Encoding, Suffix <: Encoding] {
type Result <: Encoding
/* Concat is a little verbose, we just ask for
both our strings. */
def concat(s1: EncodedString[Prefix],
s2: EncodedString[Suffix])
/* We get proof that a transcode can happen for both */
(implicit t1: Transcode.Aux[Prefix, Result]
t2: Transcode.Aux[Suffix, Result]):
/* And we get the result */
EncodedString[Result]
}

44
def concat[E1 <: Encoding, E2 <: Encoding]
(s1: EncodedString[E1], s2: EncodedString[E2])
(implicit c: Concatable[E1, E2]) =
c.concat(s1, s2)

An extended example
• Let’s encode a proof that one HList is a subset of
another HList.
• But is it useful?
45

Imagine our own implementation of a SQL DSL:
!
case class User(
id: Id
name: String,
age: Int,
email: String,
deviceId: Long
)
!
// Create Table
SQL.create[User]
!
SQL.insert(
User(1, “Jared”, 21, “jroesch@cs.ucsb.edu”, 1)
)
!
// successful update
SQL.update[User](“id” ->> 1, “age” ->> 22)
!
// fails to compile
SQL.update[User](“id” ->> 1, bogusField” ->> 1337)
!
… // Queries and so on
46

// successful update
SQL.update[User](“id” ->> 1, “age” ->> 22)
!
// fails to compile
SQL.update[User](“id” ->> 1, bogusField” ->> 1337)
47

48
def subList[A](sub: List[A], list: List[A]): Boolean =
(sub, list) match {
case (Nil, _) =>
true
case (x :: xs, y :: ys) if x == y =>
true && subList(xs, ys)
case (subp, first :: remanning) =>
subList(subp, remaining)
}

49
trait SubSeq[Sub <: HList, Super <: HList]

50
object SubSeq extends LowPrioritySubSeq {
type Aux[L <: HList, S <: HList] = SubSeq[L] {
type Sub = S
}
…
}

51
/* Low priority case where we just keep scanning the list. */
trait LowPrioritySubSeq {
implicit def hconsSubSeq[Sub <: HList, SH, ST <: HList]
(implicit subseq: SubSeq.Aux[Sub, ST]) =
new SubSeq[Sub] {
type Sub = SH :: ST
}
}

52
object SubSeq extends LowPrioritySubSeq {
…
/* HNil is a SubSeq of any HList */
implicit def hnilSubSeq[H <: HList] =
new SubSeq[HNil] {
type Sub = H
}
!
…
}

53
object SubSeq extends LowPrioritySubSeq {
…
implicit def hconsSubSeqIso
[H, SH, T <: HList, ST <: HList]
(implicit iso: H =:= SH,
subseq: SubSeq.Aux[T, ST]) =
new SubSeq[H :: T] {
type Sub = SH :: ST
}
}

54
There are few ways to improve upon our last example, I’ll
leave it as a fun puzzle for you.

55
https://gist.github.com/jroesch/db2674d0ef3e49d43154
!
https://github.com/jroesch/scala-by-the-bay-2014

Acknowledgements
• Thanks to Miles Sabin, the PL Lab, Adelbert
Chang, and Pete Cruz.
56

Thank you for your time, questions?
57