Monad Transformers In The Wild

  • 1,540 views
Uploaded on

 

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
1,540
On Slideshare
0
From Embeds
0
Number of Embeds
1

Actions

Shares
Downloads
33
Comments
0
Likes
9

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. MONADTRANSFORMERSIn The Wild
  • 2. speakerdeck.com/u/jrwest/p/monad-transformers
  • 3. TWITTER: @_JRWEST GITHUB.COM/JRWESTBLOG.LOOPEDSTRANGE.COM
  • 4. SF SCALA May 2012* http://marakana.com/s/scala_typeclassopedia_with_john_kodumal_of_atlassian_video,1198/index.html
  • 5. trait Monad[F[_]] extends Applicative[F] { def flatMap[A, B](fa: F[A])(f :A=>F[B]):F[B] }* monad type class* flatMap also called bind, >>=
  • 6. def point[A](a: => A): M[A] def map[A,B](ma: M[A])(f: A => B): M[B] def flatMap[A,B](ma: M[A])(f: A => M[B]): M[B]* the functions we care about* lift pure value, lift pure function, chain “operations”
  • 7. scala> import scalaz.Monad scala> import scalaz.std.option._ scala> val a = Monad[Option].point(1) a: Option[Int] = Some(1) scala> Monad[Option].map(a)(_.toString + "hi") res2: Option[java.lang.String] = Some(1hi) scala> Monad[Option].bind(a)(i => if (i < 0) None else Some(i + 1)) res4: Option[Int] = Some(2)* explicit type class usage in scalaz seven
  • 8. scala> import scalaz.syntax.monad._ import scalaz.syntax.monad._ scala> Option(1).flatMap(i => if (i < 0) None else Some(i+1)) res6: Option[Int] = Some(2) scala> 1.point[Option].flatMap(...) res7: Option[Int] = Some(2)* implicit type class usage in scalaz7 using syntax extensions
  • 9. “A MONADIC FOR COMPREHENSION IS AN EMBEDDED PROGRAMMING LANGUAGE WITH SEMANTICS DEFINED BY THE MONAD”* “one intuition of monads” - john
  • 10. MULTIPLE EFFECTSComposition
  • 11. Option[A]* it may not exist
  • 12. SIDE NOTE: SEMANTICS* to an extent, you can “choose” the meaning of a monad* Option -- anon. exceptions -- more narrowly, the exception that something is not there. Validation - monad/not monad - canmean different things in different contexts
  • 13. IO[Option[A]]* but side-effects are needed to even look for that value
  • 14. IO[Validation[Throwable,Option[A]]* and looking for that value may throw exceptions (or fail in some way)
  • 15. IO[(List[String], Validation[Throwable,Option[A])]* and logging what is going on is necessary
  • 16. MULTIPLE EFFECTSA Problem
  • 17. MONADS DO NOT COMPOSE* the problem in theory (core issue)
  • 18. “COMPOSE”?
  • 19. FUNCTORS DO COMPOSE* as well as applicatives
  • 20. trait Functor[F[_]] { def map[A, B](fa: F[A])(f :A=>B):F[B]}
  • 21. def composeFunctor[M[_],N[_]](implicit m: Functor[M], n: Functor[N]) = new Functor[({type MN[A]=[M[N[A]]]})#MN] { def map[A,B](mna: M[N[A]])(f: A => B): M[N[B]] = ... }* generic function that composes any two functors M[_] and N[_]
  • 22. def composeFunctor[M[_],N[_]](implicit m: Functor[M], n: Functor[N]) = new Functor[({type MN[A]=[M[N[A]]]})#MN] { def map[A,B](mna: M[N[A]])(f: A => B): M[N[B]] = { M.map(mna)(na => N.map(na)(f)) } }
  • 23. scala> Option("abc").map(f) res1: Option[Int] = Some(3) scala> List(Option("abc"), Option("d"), Option("ef")).map2(f) res2: List[Option[Int]] = List(Some(3), Some(1), Some(2))* can compose functors infinitely deep but...* scalaz provides method to compose 2, with nice syntatic sugar, easily (map2)
  • 24. def notPossible[M[_],N[_]](implicit m: Monad[M], n: Monad[N]) = new Monad[({type MN[A]=[M[N[A]]]})#MN] { def flatMap[A,B](mna: M[N[A]])(f: A => M[N[B]]): M[N[B]] = ... }* cannot write the same function for any two monads M[_], N[_]
  • 25. IT ! def notPossible[M[_],N[_]](implicit m: Monad[M], n: Monad[N]) = Y new Monad[({type MN[A]=[M[N[A]]]})#MN] { R def flatMap[A,B](mna: M[N[A]])(f: A => M[N[B]]): M[N[B]] = ... } T* best way to understand this is attempt to write it yourself* it won’t compile
  • 26. http://blog.tmorris.net/monads-do-not-compose/* good resource to dive into this in more detail* some of previous slides based on above* provides template, in the form of a gist, for trying this stuff out
  • 27. STAIR STEPPING* the problem in practice*http://www.flickr.com/photos/caliperstudio/2667302181/
  • 28. val a: IO[Option[MyData]] = ... val b: IO[Option[MyData]] = ...* have two values that require we communicate w/ outside world to fetch* those values may not exist (alternative meaning, fetching may result in exceptions that are anonymous)
  • 29. for { data1 <- a data2 <- b } yield { data1 merge data2 // fail }* want to merge the two pieces of data if they both exist
  • 30. for { // weve escaped IO, fail d1 <- a.unsafePerformIO d2 <- b.unsafePerformIO } yield d1 merge d2* don’t want to perform the actions until later (don’t escape the IO monad)
  • 31. for { od1 <- a for { od2 <- b od1 <- a } yield (od1,od2) match { od2 <- b case (Some(d1),Some(d2) => } yield for { Option(d1 merge d2) d1 <- od1 case (a@Some(d1),_)) => a d2 <- od2 case (_,a@Some(d2)) => a case _ => None } yield d1 merge d2 }* may notice the semi-group here* can also write it w/ an applicative* this is a contrived example
  • 32. BUT WHAT IF... def b(data: MyData): IO[Option[MyData]* even w/ simple example, this minor change throws a monkey wrench in things
  • 33. for { ):   readRes <- readIO(domain)   res <- readRes.fold(    success = _.cata(     some = meta => if (meta.enabledStatus /== status) { writeIO(meta.copy(enabledStatus = status)) } else meta.successNel[BarneyException].pure[IO],      none = new ReadFailure(domain).failNel[AppMetadata].pure[IO]     ),     failure = errors => errors.fail[AppMetadata].pure[IO]   ) } yield res* example of what not to do from something I wrote a while back
  • 34. MULTIPLE EFFECTSA Solution
  • 35. case class IOOption[A](run: IO[Option[A]])define type that boxes box the value, doesn’t need to be a case class, similar to haskell newtype.
  • 36. new Monad[IOOption] { def point[A](a: => A): IOOption[A] = IOOption(a.point[Option].point[IO]) def map[A,B](fa: IOOption[A])(f: A => B): IOOption[B] = IOOption(fa.run.map(opt => opt.map(f))) def flatMap[A, B](fa: IOOption[A])(f :A=>IOOption[B]):IOOption[B] = IOOption(fa.run.flatMap((o: Option[A]) => o match { case Some(a) => f(a).run case None => (None : Option[B]).point[IO] })) }* can define a Monad instance for new type
  • 37. val a: IOOption[MyData] = ... val b: IOOption[MyData] = ... val c: IOOption[MyData] = for { data1 <- a data2 <- b } yield { data1 merge data2 } val d: IO[Option[MyData]] = c.runcan use new type to improve previous contrived example
  • 38. type MyState[A] = State[StateData,A] case class MyStateOption[A](run: MyState[Option[A]])* what if we don’t need effects, but state we can read and write to produce a final optional value and some new state* State[S,A] where S is fixed is a monad* can define a new type for that as well
  • 39. new Monad[MyStateOption] { new Monad[IOOption] { def map[A,B](fa: MyStateOption[A])(f: A => B): MyStateOption[B] = def map[A,B](fa: IOOption[A])(f: A => B): IOOption[B] = MyStateOption(Functor[MyState].map(fa)(opt => opt.map(f))) IOOption(Functor[IO].map(fa)(opt => opt.map(f))) def flatMap[A, B](fa: MyStateOption[A])(f :A=>IOOption[B]) = def flatMap[A, B](fa: IOOption[A])(f :A=>IOOption[B]) = MyStateOption(Monad[MyState]].bind(fa)((o: Option[A]) => o match { IOOption(Monad[IO]].bind(fa)((o: Option[A]) => o match { case Some(a) => f(a).run case Some(a) => f(a).run case None => (None : Option[B]).point[MyState] case None => (None : Option[B]).point[IO] })) })) } }* opportunity for more abstraction* if you were going to do this, not exactly the way you would define these in real code, cheated a bit using {Functor,Monad}.apply
  • 40. case class OptionT[M[_], A](run: M[Option[A]])define a new type parameterized * -> * and *.
  • 41. case class OptionT[M[_], A](run: M[Option[A]]) { def map[B](f: A => B)(implicit F: Functor[M]): OptionT[M,B] def flatMap[B](f: A => OptionT[M,B])(implicit M: Monad[M]): OptionT[M,B] }* define map/flatMap a little differently, can be done like previous as typeclass instance but convention is to define the interfaceon the transformer and later define typeclass instance using the interface
  • 42. case class OptionT[M[_], A](run: M[Option[A]]) { def map[B](f: A => B)(implicit F: Functor[M]): OptionT[M,B] = OptionT[M,B](F.map(run)((o: Option[A]) => o map f)) def flatMap[B](f: A => OptionT[M,B])(implicit M: Monad[M]): OptionT[M,B] = OptionT[M,B](M.bind(run)((o: Option[A]) => o match { case Some(a) => f(a).run case None => M.point((None: Option[B])) })) }* implementations resemble what has already been shown
  • 43. new Monad[IOOption] { case class OptionT[M[_], A](run: M[Option[A]]) { def map[A,B](fa: IOOption[A])(f: A => B): IOOption[B] = def map[B](f: A => B)(implicit F: Functor[M]): OptionT[M,B] = OptionT[M,B](F.map(run)((o: Option[A]) => o map f)) IOOption(Functor[IO].map(fa)(opt => opt.map(f))) def flatMap[B](f: A => OptionT[M,B])(implicit M: Monad[M]) = def flatMap[A, B](fa: IOOption[A])(f :A=>IOOption[B]) = OptionT[M,B](M.bind(run)((o: Option[A]) => o match { IOOption(Monad[IO]].bind(fa)((o: Option[A]) => o match { case Some(a) => f(a).run case Some(a) => f(a).run case None => M.point((None: Option[B])) })) case None => (None : Option[B]).point[IO] } })) }* it the generalization of what was written before
  • 44. type FlowState[A] = State[ReqRespData, A] val f: Option[String] => FlowState[Boolean] = (etag: Option[String]) => { val a: OptionT[FlowState, Boolean] = for { // string <- OptionT[FlowState,String]      e <- optionT[FlowState](etag.point[FlowState]) // wrap FlowState[Option[String]] in OptionT      matches <- optionT[FlowState]((requestHeadersL member IfMatch))    } yield matches.split(",").map(_.trim).toList.contains(e) a getOrElse false // FlowState[Boolean] }* check existence of etag in an http request, data lives in state* has minor bug, doesn’t deal w/ double quotes as written* https://github.com/stackmob/scalamachine/blob/master/core/src/main/scala/scalamachine/core/v3/WebmachineDecisions.scala#L282-285
  • 45. val reqCType: OptionT[FlowState,ContentType] = for {       contentType <- optionT[FlowState]( (requestHeadersL member ContentTypeHeader) )       mediaInfo <- optionT[FlowState]( parseMediaTypes(contentType).headOption.point[FlowState] ) } yield mediaInfo.mediaRange* determine content type of the request, data lives in state, may not be specified* https://github.com/stackmob/scalamachine/blob/master/core/src/main/scala/scalamachine/core/v3/WebmachineDecisions.scala#L772-775
  • 46. scala> type EitherTString[M[_],A] = EitherT[M,String,A] defined type alias EitherTString scala> val items = eitherT[List,String,Int](List(1,2,3,4,5,6).map(Right(_))) items: scalaz.EitherT[List,String,Int] = ...* adding features to a “embedded language”
  • 47. for { i <- items } yield print(i) // 123456 for { i <- items _ <- if (i > 4) leftT[List,String,Unit]("fail") else rightT[List,String,Unit](()) } yield print(i) // 1234* adding error handling, and early termination to non-deterministic computation
  • 48. MONADTRANSFORMERS In General
  • 49. MyMonad[A]
  • 50. NAMING CONVENTION MyMonadT[M[_], A]* transformer name ends in T
  • 51. BOXES A VALUE run: M[MyMonad[A]* value is typically called “run” in scalaz7* often called “value” in scalaz6 (because of NewType)
  • 52. A MONAD TRANSFORMER IS A MONAD TOO* i mean, its thats kinda the point of this whole exercise isn’t it :)
  • 53. def optTMonad[M[_] : Monad] = new Monad[({type O[X]=OptionT[M,X]]})#O) { def point[A](a: => A): OptionT[M,A] = OptionT(a.point[Option].point[M]) def map[A,B](fa: OptionT[M,A])(f: A => B): OptionT[M,B] = fa map f def flatMap[A, B](fa: OptionT[M,A])(f :A=> OptionT[M,B]): OptionT[M, B] = fa flatMap f }* monad instance definition for OptionT
  • 54. HAS INTERFACE RESEMBLING UNDERLYING MONAD’S INTERFACE* can interact with the monad transformer in a manner similar to working with the actual monad* same methods, slightly different type signatures* different from haskell, “feature” of scala, since we can define methods on a type
  • 55. case class OptionT[M[_], A](run: M[Option[A]]) { def getOrElse[AA >: A](d: => AA)(implicit F: Functor[M]): M[AA] = F.map(run)((_: Option[A]) getOrElse default) def orElse[AA >: A](o: OptionT[M,AA])(implicit M: Monad[M]): OptionT[M,AA] = OptionT[M,AA](M.bind(run) { case x@Some(_) => M.point(x) case None => o.run }}
  • 56. MONADTRANSFORMERSStacked Effects
  • 57. TRANSFORMER IS A MONAD TRANSFORMER CAN WRAP ANOTHER TRANSFORMER* at the start, the goal was to stack effects (not just stack 2 effects)* this makes it possible
  • 58. type VIO[A] = ValidationT[IO,Throwable,A] def doWork(): VIO[Option[Int]] = ... val r: OptionT[VIO,Int] = optionT[VIO](doWork())* wrap the ValidationT with success type Option[A] in an OptionT* define type alias for connivence -- avoids nasty type lambda syntax inline
  • 59. val action: OptionT[VIO, Boolean] = for { devDomain <- optionT[VIO] {     validationT(        bucket.fetch[CName]("%s.%s".format(devPrefix,hostname))        ).mapFailure(CNameServiceException(_))    } _ <- optionT[VIO] { validationT(deleteDomains(devDomain)).map(_.point[Option]) } } yield true* code (slightly modified) from one of stackmob’s internal services* uses Scaliak to fetch hostname data from riak and then remove them* possible to clean this code up a bit, will discuss shortly (monadtrans)
  • 60. KEEP ON STACKIN’ ON* don’t have to stop at 2 levels deep, our new stack is monad too* each monad/transformer we add to the stack compose more types of effects
  • 61. “ORDER” MATTERS* how stack is built, which transformers wrap which monads, determines the overall semantics of the entire stack* changing that order can, and usually does, change semantics
  • 62. OptionT[FlowState, A] vs. StateT[Option,ReqRespData,A]* what is the difference in semantics between the two?* type FlowState[A] = State[ReqRespData,A]
  • 63. FlowState[Option[A]] vs. Option[State[ReqRespData,A]* unboxing makes things easier to see* a state action that returns an optional value vs a state action that may not exist* the latter probably doesn’t make as much sense in the majority of cases
  • 64. MONADTRANS The Type Class* type classes beget more type classes
  • 65. REMOVING REPETITION === MORE ABSTRACTION* previous examples have had a repetitive, annoying, & verbose task* can be abstracted away...by a type class of course
  • 66. optionT[VIO](validationT(deleteDomains(devDomain)).map(_.point[Option])) eitherT[List,String,Int](List(1,2,3,4,5,6).map(Right(_))) resT[FlowState](encodeBodyIfSet(resource).map(_.point[Res]))* some cases require lifting the value into the monad and then wrap it in the transformer* from previous examples
  • 67. M[A] -> M[N[A]] -> NT[M[N[_]], A]* this is basically what we are doing every time* taking some monad M[A], lifting A into N, a monad we have a transformer for, and then wrapping all of that in N’s monadtransformer
  • 68. trait MonadTrans[F[_[_], _]] {   def liftM[G[_] : Monad, A](a: G[A]): F[G, A] }* liftM will do this for any transformer F[_[_],_] and any monad G[_] provided an instance of it is defined for F[_[_],_]
  • 69.  def liftM[G[_], A](a: G[A])(implicit G: Monad[G]): OptionT[G, A] =     OptionT[G, A](G.map[A, Option[A]](a)((a: A) => a.point[Option]))* full definition requires some type ceremony* https://github.com/scalaz/scalaz/blob/scalaz-seven/core/src/main/scala/scalaz/OptionT.scala#L155-156
  • 70. def liftM[G[_], A](ga: G[A])(implicit G: Monad[G]): ResT[G,A] =       ResT[G,A](G.map(ga)(_.point[Res]))* implementation for scalamachine’s Res monad* https://github.com/stackmob/scalamachine/blob/master/scalaz7/src/main/scala/scalamachine/scalaz/res/ResT.scala#L75-76
  • 71. encodeBodyIfSet(resource).liftM[OptionT] List(1,2,3).liftM[EitherTString] validationT(deleteDomains(devDomain)).liftM[OptionT]* cleanup of previous examples* method-like syntax requires a bit more work: https://github.com/scalaz/scalaz/blob/scalaz-seven/core/src/main/scala/scalaz/syntax/MonadSyntax.scala#L9
  • 72. for { media <- (metadataL >=> contentTypeL).map(_ | ContentType("text/plain")).liftM[ResT]    charset <- (metadataL >=> chosenCharsetL).map2(";charset=" + _).getOrElse("")).liftM[ResT]    _ <- (responseHeadersL += (ContentTypeHeader, media.toHeader + charset)).liftM[ResT]    mbHeader <- (requestHeadersL member AcceptEncoding).liftM[ResT]    decision <- mbHeader >| f7.point[ResTFlow] | chooseEncoding(resource, "identity;q=1.0,*;q=0.5") } yield decision* https://github.com/stackmob/scalamachine/blob/master/core/src/main/scala/scalamachine/core/v3/WebmachineDecisions.scala#L199-205
  • 73. MONADTRANSFORMERS In Review
  • 74. STACKING MONADS COMPOSES EFFECTS* when monads are stacked an embedded language is being built with multiple effects* this is not the only intuition of monads/transformers
  • 75. CAN NOT COMPOSE MONADS GENERICALLY* cannot write generic function to compose any two monads M[_], N[_] like we can for any two functors
  • 76. MONAD TRANSFORMERS COMPOSE M[_] : MONAD WITH ANY N[_] : MONAD* can’t compose any two, but can compose a given one with any other
  • 77. MONAD TRANSFORMERS WRAP OTHER MONAD TRANSFORMERS* monad transformers are monads* so they can be the N[_] : Monad that the transformer composes with its underlying monad
  • 78. MONADTRANS REDUCES REPETITION* often need to take a value that is not entirely lifted into a monad transformer stack and do just that
  • 79. STACK MONADS DON’T STAIR-STEP* monad transformers reduce ugly, stair-stepping or nested code and focuses on core task* focuses on intuition of mutiple effects instead of handling things haphazardly
  • 80. THANK YOU* stackmob, markana, john & atlassian, other sponsors, cosmin
  • 81. QUESTIONS?