Upcoming SlideShare
×

# Monad Transformers In The Wild

3,592 views

Published on

12 Likes
Statistics
Notes
• Full Name
Comment goes here.

Are you sure you want to Yes No
Your message goes here
• Be the first to comment

Views
Total views
3,592
On SlideShare
0
From Embeds
0
Number of Embeds
31
Actions
Shares
0
51
0
Likes
12
Embeds 0
No embeds

No notes for slide

### Monad Transformers In The Wild

1. 1. MONADTRANSFORMERSIn The Wild
3. 3. TWITTER: @_JRWEST GITHUB.COM/JRWESTBLOG.LOOPEDSTRANGE.COM
4. 4. SF SCALA May 2012* http://marakana.com/s/scala_typeclassopedia_with_john_kodumal_of_atlassian_video,1198/index.html
5. 5. trait Monad[F[_]] extends Applicative[F] { def ﬂatMap[A, B](fa: F[A])(f :A=>F[B]):F[B] }* monad type class* ﬂatMap also called bind, >>=
6. 6. def point[A](a: => A): M[A] def map[A,B](ma: M[A])(f: A => B): M[B] def ﬂatMap[A,B](ma: M[A])(f: A => M[B]): M[B]* the functions we care about* lift pure value, lift pure function, chain “operations”
7. 7. scala> import scalaz.Monad scala> import scalaz.std.option._ scala> val a = Monad[Option].point(1) a: Option[Int] = Some(1) scala> Monad[Option].map(a)(_.toString + "hi") res2: Option[java.lang.String] = Some(1hi) scala> Monad[Option].bind(a)(i => if (i < 0) None else Some(i + 1)) res4: Option[Int] = Some(2)* explicit type class usage in scalaz seven
8. 8. scala> import scalaz.syntax.monad._ import scalaz.syntax.monad._ scala> Option(1).ﬂatMap(i => if (i < 0) None else Some(i+1)) res6: Option[Int] = Some(2) scala> 1.point[Option].ﬂatMap(...) res7: Option[Int] = Some(2)* implicit type class usage in scalaz7 using syntax extensions
9. 9. “A MONADIC FOR COMPREHENSION IS AN EMBEDDED PROGRAMMING LANGUAGE WITH SEMANTICS DEFINED BY THE MONAD”* “one intuition of monads” - john
10. 10. MULTIPLE EFFECTSComposition
11. 11. Option[A]* it may not exist
12. 12. SIDE NOTE: SEMANTICS* to an extent, you can “choose” the meaning of a monad* Option -- anon. exceptions -- more narrowly, the exception that something is not there. Validation - monad/not monad - canmean different things in different contexts
13. 13. IO[Option[A]]* but side-effects are needed to even look for that value
14. 14. IO[Validation[Throwable,Option[A]]* and looking for that value may throw exceptions (or fail in some way)
15. 15. IO[(List[String], Validation[Throwable,Option[A])]* and logging what is going on is necessary
16. 16. MULTIPLE EFFECTSA Problem
17. 17. MONADS DO NOT COMPOSE* the problem in theory (core issue)
18. 18. “COMPOSE”?
19. 19. FUNCTORS DO COMPOSE* as well as applicatives
20. 20. trait Functor[F[_]] { def map[A, B](fa: F[A])(f :A=>B):F[B]}
21. 21. def composeFunctor[M[_],N[_]](implicit m: Functor[M], n: Functor[N]) = new Functor[({type MN[A]=[M[N[A]]]})#MN] { def map[A,B](mna: M[N[A]])(f: A => B): M[N[B]] = ... }* generic function that composes any two functors M[_] and N[_]
22. 22. def composeFunctor[M[_],N[_]](implicit m: Functor[M], n: Functor[N]) = new Functor[({type MN[A]=[M[N[A]]]})#MN] { def map[A,B](mna: M[N[A]])(f: A => B): M[N[B]] = { M.map(mna)(na => N.map(na)(f)) } }
23. 23. scala> Option("abc").map(f) res1: Option[Int] = Some(3) scala> List(Option("abc"), Option("d"), Option("ef")).map2(f) res2: List[Option[Int]] = List(Some(3), Some(1), Some(2))* can compose functors inﬁnitely deep but...* scalaz provides method to compose 2, with nice syntatic sugar, easily (map2)
24. 24. def notPossible[M[_],N[_]](implicit m: Monad[M], n: Monad[N]) = new Monad[({type MN[A]=[M[N[A]]]})#MN] { def ﬂatMap[A,B](mna: M[N[A]])(f: A => M[N[B]]): M[N[B]] = ... }* cannot write the same function for any two monads M[_], N[_]
25. 25. IT ! def notPossible[M[_],N[_]](implicit m: Monad[M], n: Monad[N]) = Y new Monad[({type MN[A]=[M[N[A]]]})#MN] { R def ﬂatMap[A,B](mna: M[N[A]])(f: A => M[N[B]]): M[N[B]] = ... } T* best way to understand this is attempt to write it yourself* it won’t compile
26. 26. http://blog.tmorris.net/monads-do-not-compose/* good resource to dive into this in more detail* some of previous slides based on above* provides template, in the form of a gist, for trying this stuff out
27. 27. STAIR STEPPING* the problem in practice*http://www.ﬂickr.com/photos/caliperstudio/2667302181/
28. 28. val a: IO[Option[MyData]] = ... val b: IO[Option[MyData]] = ...* have two values that require we communicate w/ outside world to fetch* those values may not exist (alternative meaning, fetching may result in exceptions that are anonymous)
29. 29. for { data1 <- a data2 <- b } yield { data1 merge data2 // fail }* want to merge the two pieces of data if they both exist
30. 30. for { // weve escaped IO, fail d1 <- a.unsafePerformIO d2 <- b.unsafePerformIO } yield d1 merge d2* don’t want to perform the actions until later (don’t escape the IO monad)
31. 31. for { od1 <- a for { od2 <- b od1 <- a } yield (od1,od2) match { od2 <- b case (Some(d1),Some(d2) => } yield for { Option(d1 merge d2) d1 <- od1 case (a@Some(d1),_)) => a d2 <- od2 case (_,a@Some(d2)) => a case _ => None } yield d1 merge d2 }* may notice the semi-group here* can also write it w/ an applicative* this is a contrived example
32. 32. BUT WHAT IF... def b(data: MyData): IO[Option[MyData]* even w/ simple example, this minor change throws a monkey wrench in things
33. 33. for { ):   readRes <- readIO(domain)   res <- readRes.fold(    success = _.cata(     some = meta => if (meta.enabledStatus /== status) { writeIO(meta.copy(enabledStatus = status)) } else meta.successNel[BarneyException].pure[IO],      none = new ReadFailure(domain).failNel[AppMetadata].pure[IO]     ),     failure = errors => errors.fail[AppMetadata].pure[IO]   ) } yield res* example of what not to do from something I wrote a while back
34. 34. MULTIPLE EFFECTSA Solution
35. 35. case class IOOption[A](run: IO[Option[A]])deﬁne type that boxes box the value, doesn’t need to be a case class, similar to haskell newtype.
36. 36. new Monad[IOOption] { def point[A](a: => A): IOOption[A] = IOOption(a.point[Option].point[IO]) def map[A,B](fa: IOOption[A])(f: A => B): IOOption[B] = IOOption(fa.run.map(opt => opt.map(f))) def ﬂatMap[A, B](fa: IOOption[A])(f :A=>IOOption[B]):IOOption[B] = IOOption(fa.run.ﬂatMap((o: Option[A]) => o match { case Some(a) => f(a).run case None => (None : Option[B]).point[IO] })) }* can deﬁne a Monad instance for new type
37. 37. val a: IOOption[MyData] = ... val b: IOOption[MyData] = ... val c: IOOption[MyData] = for { data1 <- a data2 <- b } yield { data1 merge data2 } val d: IO[Option[MyData]] = c.runcan use new type to improve previous contrived example
38. 38. type MyState[A] = State[StateData,A] case class MyStateOption[A](run: MyState[Option[A]])* what if we don’t need effects, but state we can read and write to produce a ﬁnal optional value and some new state* State[S,A] where S is ﬁxed is a monad* can deﬁne a new type for that as well
39. 39. new Monad[MyStateOption] { new Monad[IOOption] { def map[A,B](fa: MyStateOption[A])(f: A => B): MyStateOption[B] = def map[A,B](fa: IOOption[A])(f: A => B): IOOption[B] = MyStateOption(Functor[MyState].map(fa)(opt => opt.map(f))) IOOption(Functor[IO].map(fa)(opt => opt.map(f))) def ﬂatMap[A, B](fa: MyStateOption[A])(f :A=>IOOption[B]) = def ﬂatMap[A, B](fa: IOOption[A])(f :A=>IOOption[B]) = MyStateOption(Monad[MyState]].bind(fa)((o: Option[A]) => o match { IOOption(Monad[IO]].bind(fa)((o: Option[A]) => o match { case Some(a) => f(a).run case Some(a) => f(a).run case None => (None : Option[B]).point[MyState] case None => (None : Option[B]).point[IO] })) })) } }* opportunity for more abstraction* if you were going to do this, not exactly the way you would deﬁne these in real code, cheated a bit using {Functor,Monad}.apply
40. 40. case class OptionT[M[_], A](run: M[Option[A]])deﬁne a new type parameterized * -> * and *.
41. 41. case class OptionT[M[_], A](run: M[Option[A]]) { def map[B](f: A => B)(implicit F: Functor[M]): OptionT[M,B] def ﬂatMap[B](f: A => OptionT[M,B])(implicit M: Monad[M]): OptionT[M,B] }* deﬁne map/ﬂatMap a little differently, can be done like previous as typeclass instance but convention is to deﬁne the interfaceon the transformer and later deﬁne typeclass instance using the interface
42. 42. case class OptionT[M[_], A](run: M[Option[A]]) { def map[B](f: A => B)(implicit F: Functor[M]): OptionT[M,B] = OptionT[M,B](F.map(run)((o: Option[A]) => o map f)) def ﬂatMap[B](f: A => OptionT[M,B])(implicit M: Monad[M]): OptionT[M,B] = OptionT[M,B](M.bind(run)((o: Option[A]) => o match { case Some(a) => f(a).run case None => M.point((None: Option[B])) })) }* implementations resemble what has already been shown
43. 43. new Monad[IOOption] { case class OptionT[M[_], A](run: M[Option[A]]) { def map[A,B](fa: IOOption[A])(f: A => B): IOOption[B] = def map[B](f: A => B)(implicit F: Functor[M]): OptionT[M,B] = OptionT[M,B](F.map(run)((o: Option[A]) => o map f)) IOOption(Functor[IO].map(fa)(opt => opt.map(f))) def ﬂatMap[B](f: A => OptionT[M,B])(implicit M: Monad[M]) = def ﬂatMap[A, B](fa: IOOption[A])(f :A=>IOOption[B]) = OptionT[M,B](M.bind(run)((o: Option[A]) => o match { IOOption(Monad[IO]].bind(fa)((o: Option[A]) => o match { case Some(a) => f(a).run case Some(a) => f(a).run case None => M.point((None: Option[B])) })) case None => (None : Option[B]).point[IO] } })) }* it the generalization of what was written before
44. 44. type FlowState[A] = State[ReqRespData, A] val f: Option[String] => FlowState[Boolean] = (etag: Option[String]) => { val a: OptionT[FlowState, Boolean] = for { // string <- OptionT[FlowState,String]      e <- optionT[FlowState](etag.point[FlowState]) // wrap FlowState[Option[String]] in OptionT      matches <- optionT[FlowState]((requestHeadersL member IfMatch))    } yield matches.split(",").map(_.trim).toList.contains(e) a getOrElse false // FlowState[Boolean] }* check existence of etag in an http request, data lives in state* has minor bug, doesn’t deal w/ double quotes as written* https://github.com/stackmob/scalamachine/blob/master/core/src/main/scala/scalamachine/core/v3/WebmachineDecisions.scala#L282-285
45. 45. val reqCType: OptionT[FlowState,ContentType] = for {       contentType <- optionT[FlowState]( (requestHeadersL member ContentTypeHeader) )       mediaInfo <- optionT[FlowState]( parseMediaTypes(contentType).headOption.point[FlowState] ) } yield mediaInfo.mediaRange* determine content type of the request, data lives in state, may not be speciﬁed* https://github.com/stackmob/scalamachine/blob/master/core/src/main/scala/scalamachine/core/v3/WebmachineDecisions.scala#L772-775
46. 46. scala> type EitherTString[M[_],A] = EitherT[M,String,A] deﬁned type alias EitherTString scala> val items = eitherT[List,String,Int](List(1,2,3,4,5,6).map(Right(_))) items: scalaz.EitherT[List,String,Int] = ...* adding features to a “embedded language”
47. 47. for { i <- items } yield print(i) // 123456 for { i <- items _ <- if (i > 4) leftT[List,String,Unit]("fail") else rightT[List,String,Unit](()) } yield print(i) // 1234* adding error handling, and early termination to non-deterministic computation
48. 48. MONADTRANSFORMERS In General
50. 50. NAMING CONVENTION MyMonadT[M[_], A]* transformer name ends in T
51. 51. BOXES A VALUE run: M[MyMonad[A]* value is typically called “run” in scalaz7* often called “value” in scalaz6 (because of NewType)
52. 52. A MONAD TRANSFORMER IS A MONAD TOO* i mean, its thats kinda the point of this whole exercise isn’t it :)
53. 53. def optTMonad[M[_] : Monad] = new Monad[({type O[X]=OptionT[M,X]]})#O) { def point[A](a: => A): OptionT[M,A] = OptionT(a.point[Option].point[M]) def map[A,B](fa: OptionT[M,A])(f: A => B): OptionT[M,B] = fa map f def ﬂatMap[A, B](fa: OptionT[M,A])(f :A=> OptionT[M,B]): OptionT[M, B] = fa ﬂatMap f }* monad instance deﬁnition for OptionT
54. 54. HAS INTERFACE RESEMBLING UNDERLYING MONAD’S INTERFACE* can interact with the monad transformer in a manner similar to working with the actual monad* same methods, slightly different type signatures* different from haskell, “feature” of scala, since we can deﬁne methods on a type
55. 55. case class OptionT[M[_], A](run: M[Option[A]]) { def getOrElse[AA >: A](d: => AA)(implicit F: Functor[M]): M[AA] = F.map(run)((_: Option[A]) getOrElse default) def orElse[AA >: A](o: OptionT[M,AA])(implicit M: Monad[M]): OptionT[M,AA] = OptionT[M,AA](M.bind(run) { case x@Some(_) => M.point(x) case None => o.run }}
57. 57. TRANSFORMER IS A MONAD TRANSFORMER CAN WRAP ANOTHER TRANSFORMER* at the start, the goal was to stack effects (not just stack 2 effects)* this makes it possible
58. 58. type VIO[A] = ValidationT[IO,Throwable,A] def doWork(): VIO[Option[Int]] = ... val r: OptionT[VIO,Int] = optionT[VIO](doWork())* wrap the ValidationT with success type Option[A] in an OptionT* deﬁne type alias for connivence -- avoids nasty type lambda syntax inline
59. 59. val action: OptionT[VIO, Boolean] = for { devDomain <- optionT[VIO] {     validationT(        bucket.fetch[CName]("%s.%s".format(devPreﬁx,hostname))        ).mapFailure(CNameServiceException(_))    } _ <- optionT[VIO] { validationT(deleteDomains(devDomain)).map(_.point[Option]) } } yield true* code (slightly modiﬁed) from one of stackmob’s internal services* uses Scaliak to fetch hostname data from riak and then remove them* possible to clean this code up a bit, will discuss shortly (monadtrans)
60. 60. KEEP ON STACKIN’ ON* don’t have to stop at 2 levels deep, our new stack is monad too* each monad/transformer we add to the stack compose more types of effects
61. 61. “ORDER” MATTERS* how stack is built, which transformers wrap which monads, determines the overall semantics of the entire stack* changing that order can, and usually does, change semantics
62. 62. OptionT[FlowState, A] vs. StateT[Option,ReqRespData,A]* what is the difference in semantics between the two?* type FlowState[A] = State[ReqRespData,A]
63. 63. FlowState[Option[A]] vs. Option[State[ReqRespData,A]* unboxing makes things easier to see* a state action that returns an optional value vs a state action that may not exist* the latter probably doesn’t make as much sense in the majority of cases
64. 64. MONADTRANS The Type Class* type classes beget more type classes
65. 65. REMOVING REPETITION === MORE ABSTRACTION* previous examples have had a repetitive, annoying, & verbose task* can be abstracted away...by a type class of course
66. 66. optionT[VIO](validationT(deleteDomains(devDomain)).map(_.point[Option])) eitherT[List,String,Int](List(1,2,3,4,5,6).map(Right(_))) resT[FlowState](encodeBodyIfSet(resource).map(_.point[Res]))* some cases require lifting the value into the monad and then wrap it in the transformer* from previous examples
67. 67. M[A] -> M[N[A]] -> NT[M[N[_]], A]* this is basically what we are doing every time* taking some monad M[A], lifting A into N, a monad we have a transformer for, and then wrapping all of that in N’s monadtransformer
68. 68. trait MonadTrans[F[_[_], _]] {   def liftM[G[_] : Monad, A](a: G[A]): F[G, A] }* liftM will do this for any transformer F[_[_],_] and any monad G[_] provided an instance of it is deﬁned for F[_[_],_]
69. 69.  def liftM[G[_], A](a: G[A])(implicit G: Monad[G]): OptionT[G, A] =     OptionT[G, A](G.map[A, Option[A]](a)((a: A) => a.point[Option]))* full deﬁnition requires some type ceremony* https://github.com/scalaz/scalaz/blob/scalaz-seven/core/src/main/scala/scalaz/OptionT.scala#L155-156
70. 70. def liftM[G[_], A](ga: G[A])(implicit G: Monad[G]): ResT[G,A] =       ResT[G,A](G.map(ga)(_.point[Res]))* implementation for scalamachine’s Res monad* https://github.com/stackmob/scalamachine/blob/master/scalaz7/src/main/scala/scalamachine/scalaz/res/ResT.scala#L75-76
71. 71. encodeBodyIfSet(resource).liftM[OptionT] List(1,2,3).liftM[EitherTString] validationT(deleteDomains(devDomain)).liftM[OptionT]* cleanup of previous examples* method-like syntax requires a bit more work: https://github.com/scalaz/scalaz/blob/scalaz-seven/core/src/main/scala/scalaz/syntax/MonadSyntax.scala#L9
72. 72. for { media <- (metadataL >=> contentTypeL).map(_ | ContentType("text/plain")).liftM[ResT]    charset <- (metadataL >=> chosenCharsetL).map2(";charset=" + _).getOrElse("")).liftM[ResT]    _ <- (responseHeadersL += (ContentTypeHeader, media.toHeader + charset)).liftM[ResT]    mbHeader <- (requestHeadersL member AcceptEncoding).liftM[ResT]    decision <- mbHeader >| f7.point[ResTFlow] | chooseEncoding(resource, "identity;q=1.0,*;q=0.5") } yield decision* https://github.com/stackmob/scalamachine/blob/master/core/src/main/scala/scalamachine/core/v3/WebmachineDecisions.scala#L199-205
73. 73. MONADTRANSFORMERS In Review
74. 74. STACKING MONADS COMPOSES EFFECTS* when monads are stacked an embedded language is being built with multiple effects* this is not the only intuition of monads/transformers
75. 75. CAN NOT COMPOSE MONADS GENERICALLY* cannot write generic function to compose any two monads M[_], N[_] like we can for any two functors
76. 76. MONAD TRANSFORMERS COMPOSE M[_] : MONAD WITH ANY N[_] : MONAD* can’t compose any two, but can compose a given one with any other
77. 77. MONAD TRANSFORMERS WRAP OTHER MONAD TRANSFORMERS* monad transformers are monads* so they can be the N[_] : Monad that the transformer composes with its underlying monad
78. 78. MONADTRANS REDUCES REPETITION* often need to take a value that is not entirely lifted into a monad transformer stack and do just that
79. 79. STACK MONADS DON’T STAIR-STEP* monad transformers reduce ugly, stair-stepping or nested code and focuses on core task* focuses on intuition of mutiple effects instead of handling things haphazardly
80. 80. THANK YOU* stackmob, markana, john & atlassian, other sponsors, cosmin
81. 81. QUESTIONS?