Building a Cloud API Server using Play(SCALA) & Riak

2,751
-1

Published on

Megam's Cloud API is a stateless REST based cloud server. Our SDKs abstract away the REST interface and provide a set of handy methods you can interact with when deleting, creating, updating and deleting objects. We've also provided information on how we use HMAC to create authorization headers for use when accessing Megam.

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,751
On Slideshare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
19
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Building a Cloud API Server using Play(SCALA) & Riak

  1. 1. Building a Cloud API Server using Play(SCALA) & Riak RESTful API for Megam Cloud
  2. 2. We'll Cover Architecture of our API ➔ Authentication using HMAC ➔ How to handle JSON Requests/Response ➔ How to handle Errors ➔ How do you interface with Riak. ➔ © 2012-2013 Megam Systems
  3. 3. Response (json) Request (json) Auth OK ? Native API Wrappers Ruby HMAC Cloud API Server Router Herk Riak Snowflake ID © 2012-2013 Megam Systems Funnel Request Funnel Response
  4. 4. We'll Use Play 2.2.0 setup => Link SBT 0.13.0 Migration => Link play2-auth Authentication Scala 2.10.3 Play 2.2.0 SBT 0.13.0 Riak 1.4.2 © 2012-2013 Megam Systems
  5. 5. We'll also use scalaz "org.scalaz" %% "scalaz-core" % "7.0.3" Code is weaved with Functional Programming using scalaz © 2012-2013 Megam Systems
  6. 6. Code : https://github.com/indykish/megam_play.git © 2012-2013 Megam Systems
  7. 7. Beta Launch of Megam Cloud (Polygot PaaS) Our PaaS design => Link Register http://www.megam.co for an invite Twitter : @indykish © 2012-2013 Megam Systems
  8. 8. Screencast illustrating the Cloud API Servers working live
  9. 9. Play2-Auth : Setup play2-auth : offers Authentication and Authorization features to play framework applications. Add a dependency declaration into your Build.scala file: val appDependencies = Seq( "jp.t2v" %% "play2.auth" % "0.11.0-SNAPSHOT", "jp.t2v" %% "play2.auth.test" % "0.11.0-SNAPSHOT" % "test" ) © 2012-2013 Megam Systems
  10. 10. Authentication play2-auth uses Stackable Controller. This is handy for Authentication. All you need to do is use StackAction in your Controller. Your Controller will first call StackAction operation and then compose it with other Actions. © 2012-2013 Megam Systems
  11. 11. Scenario /nodes HTTP Request to Nodes shall be authenticated using HMAC Customer onboarded and has a email/api_key (or) private cert. © 2012-2013 Megam Systems
  12. 12. Let us create a Nodes controller object Nodes extends Controller with APIAuthElement © 2012-2013 Megam Systems
  13. 13. Controller - Nodes def post = StackAction(parse.tolerantText) { implicit request => (Validation.fromTryCatch[SimpleResult] { reqFunneled match { case Success(succ) => { val freq = succ.getOrElse(throw new Error("Request wasn't funneled. Verify the header.")) val email = freq.maybeEmail.getOrElse(throw new Error("Email not found (or) invalid.")) val clientAPIBody = freq.clientAPIBody.getOrElse(throw new Error("Body not found (or) invalid.")) models.Nodes.create(email, clientAPIBody) match { case Success(succ) => val tuple_succ = succ.getOrElse(("Nah", "Bah", "Gah")) } case Failure(err) => { val rn: FunnelResponse = new HttpReturningError(err) Status(rn.code)(rn.toJson(true)) } } } case Failure(err) => { val rn: FunnelResponse = new HttpReturningError(err) Status(rn.code)(rn.toJson(true)) } } }).fold(succ = { a: SimpleResult => a }, fail = { t: Throwable => Status(BAD_REQUEST) (t.getMessage) }) } © 2012-2013 Megam Systems Full code
  14. 14. What is happening ? A HTTPRequest that comes to Cloud API Server gets funneled implicitly. FunneledRequest as seen in next page. © 2012-2013 Megam Systems
  15. 15. FunneledRequest(FR) A Case class case class FunneledRequest(maybeEmail: Option[String], clientAPIHmac: Option[String], clientAPIDate: Option[String], clientAPIPath: Option[String], clientAPIBody: Option[String]) { val wowEmail = { val EmailRegex = """^[a-z0-9_+-]+(.[a-z0-9_+-]+)*@[a-z0-9-]+(.[a-z0-9-]+)*. ([a-z]{2,4})$""".r maybeEmail.flatMap(x => EmailRegex.findFirstIn(x)) } match { case Some(succ) => Validation.success[Throwable, Option[String]](succ.some) case None => Validation.failure[Throwable, Option[String]](new MalformedHeaderError(maybeEmail.get, """Email is blank or invalid. Kindly provide us an email in the standard format.n" eg: goodemail@megam.com""")) } val mkSign = { val ab = ((clientAPIDate ++ clientAPIPath ++ calculateMD5(clientAPIBody)) map { a: String => a }).mkString("n") play.api.Logger.debug(("%-20s -->[%s]").format("FunnelRequest:mkSign", ab)) ab } } Full code FunneledRequestBuilder creates FR © 2012-2013 Megam Systems
  16. 16. FR Builder Takes a rawheader HMAC and creates FR //Look for the X_Megam_HMAC field. If not the FunneledRequest will be None. private lazy val frOpt: Option[FunneledRequest] = (for { hmac <- rawheader.get(X_Megam_HMAC) trimmed <- hmac.trim.some res <- trimmed.some if (res.indexOf(":") > 0) } yield { val res1 = res.split(":").take(2) FunneledRequest(res1(0).some, res1(1).some, clientAPIReqDate, clientAPIReqPath, clientAPIReqBody) }) Full code © 2012-2013 Megam Systems
  17. 17. Sample Processed FR FunneledRequest [email =Some(steve@olympics.com)] [apiHMAC =Some(6010ab91b07ee680aee8bd8075591e1f4fc8bc58)] [apiDATE =Some(2013-10-12 19:04)] [apiPATH =Some(/v1/predefclouds)] [apiBody =Some()] Full code © 2012-2013 Megam Systems
  18. 18. APIAuthElement APIAuthElement is sub trait for stackable controller. APIAuthElememt trait will handle our auth operation and will chain to next action only on success © 2012-2013 Megam Systems
  19. 19. APIElement trait APIAuthElement extends StackableController { self: Controller => case object APIAccessedKey extends RequestAttributeKey[Option[String]] override def proceed[A](req: RequestWithAttributes[A])(f: RequestWithAttributes[A] => Future[SimpleResult]): Future[SimpleResult] = { SecurityActions.Authenticated(req) match { case Success(rawRes) => super.proceed(req.set(APIAccessedKey, rawRes))(f) case Failure(err) => { val g = Action { implicit request => val rn: FunnelResponse = new HttpReturningError(err) //implicitly loaded. SimpleResult(header = ResponseHeader(rn.code, Map(CONTENT_TYPE -> "text/plain")), body = Enumerator(rn.toJson(true).getBytes(UTF8Charset) )) } val origReq = req.asInstanceOf[Request[AnyContent]] g(origReq) } } } implicit def reqFunneled[A](implicit req: RequestWithAttributes[A]): ValidationNel[Throwable, Option[FunneledRequest]] = req2FunnelBuilder(req).funneled implicit def apiAccessed[A](implicit req: RequestWithAttributes[A]): Option[String] = req.get(APIAccessedKey).get } Full code © 2012-2013 Megam Systems
  20. 20. Securing API Uses SecurityActions to authenticate a FunneledRequest Get FR from controller Extract information from the FR calculate a HMAC and compares the computed HMAC from Riak. Authentication Error if match fails. © 2012-2013 Megam Systems
  21. 21. Respond back As JSON We respond back as JSON using FunnelResponse – Code (HTTP Status Code : 404, 503.. ) – Message (A String message) – More (Detail info like support links) – JSON_CLAZ (A String understood by an unmarshaller or receiver) © 2012-2013 Megam Systems
  22. 22. FunnelResponse case class FunnelResponse(code: Int, msg: String, more: String, json_claz: String, msg_type: String = "error", links: String = tailMsg) { def toJValue: JValue = { import net.liftweb.json.scalaz.JsonScalaz.toJSON import controllers.funnel.FunnelResponseSerialization val funser = new FunnelResponseSerialization() toJSON(this)(funser.writer) } def toJson(prettyPrint: Boolean = false): String = if (prettyPrint) { pretty(render(toJValue)) } else { compactRender(toJValue) } } © 2012-2013 Megam Systems
  23. 23. Funnel Errors ResourceItemNotFound CannotAuthenticate JSONParsingError HTTPReturningError MalformedBody MalformedHeader ServiceUnAvailable © 2012-2013 Megam Systems
  24. 24. Funnel Errors Object Case class *Errors in FunnelErrors object FunnelErrors { val tailMsg = """Forum :https://groups.google.com/forum/?fromgroups=#!forum/megamlive. |API :https://api.megam.co |Docs :http://docs.megam.co |Support :http://support.megam.co""".stripMargin case class CannotAuthenticateError(input: String, msg: String, httpCode: Int = BAD_REQUEST) extends java.lang.Error(msg) …. } HTTPReturningError folds the App defined error case class HttpReturningError(errNel: NonEmptyList[Throwable]) extends Exception { def mkMsg(err: Throwable): String = { err.fold( a => """Authentication failure using the email/apikey combination. %n'%s' |Verify the email and api key combination. """.format(a.input).stripMargin, … } © 2012-2013 Megam Systems
  25. 25. RichThrowable, implicit error to json implicit class RichThrowable(thrownExp: Throwable) { def fold[T]( cannotAuthError: CannotAuthenticateError => T, malformedBodyError: MalformedBodyError => T, malformedHeaderError: MalformedHeaderError => T, serviceUnavailableError: ServiceUnavailableError => T, resourceNotFound: ResourceItemNotFound => T, anyError: Throwable => T): T = thrownExp match { case a @ CannotAuthenticateError(_, _, _) => cannotAuthError(a) case m @ MalformedBodyError(_, _, _) => malformedBodyError(m) case h @ MalformedHeaderError(_, _, _) => malformedHeaderError(h) case c @ ServiceUnavailableError(_, _, _) => serviceUnavailableError(c) case r @ ResourceItemNotFound(_, _, _) => resourceNotFound(r) case t @ _ => anyError(t) } } implicit def err2FunnelResponse(hpret: HttpReturningError) = new FunnelResponse(hpret.code.getOrElse(BAD_REQUEST), hpret.msg, hpret.more.getOrElse(new String("none")), "Megam::Error", hpret.severity) implicit def err2FunnelResponses(hpret: HttpReturningError) = hpret.errNel.map { err: Throwable => err.fold(a => new FunnelResponse(hpret.mkCode(a).getOrElse(BAD_REQUEST), hpret.mkMsg(a), hpret.mkMore(a), "Megam::Error", hpret.severity), m => new FunnelResponse(hpret.mkCode(m).getOrElse(BAD_REQUEST), hpret.mkMsg(m), hpret.mkMore(m), "Megam::Error", hpret.severity), h => new FunnelResponse(hpret.mkCode(h).getOrElse(BAD_REQUEST), hpret.mkMsg(h), hpret.mkMore(h), "Megam::Error", hpret.severity), c => new FunnelResponse(hpret.mkCode(c).getOrElse(BAD_REQUEST), hpret.mkMsg(c), hpret.mkMore(c), "Megam::Error", hpret.severity), r => new FunnelResponse(hpret.mkCode(r).getOrElse(BAD_REQUEST), hpret.mkMsg(r), hpret.mkMore(r), "Megam::Error", hpret.severity), t => new FunnelResponse(hpret.mkCode(t).getOrElse(BAD_REQUEST), hpret.mkMsg(t), hpret.mkMore(t), "Megam::Error", hpret.severity)) © 2012-2013 Megam Systems }.some
  26. 26. Interface to RiaK Scaliak library to Interface with Riak "com.stackmob" % "scaliak_2.10" % "0.8.0" GSRiak - A Wrapper on top of Scaliak "com.github.indykish" % "megam_common_2.10" % "0.1.0-SNAPSHOT", © 2012-2013 Megam Systems
  27. 27. Code for megam_common : https://github.com/indykish/megam_common.git © 2012-2013 Megam Systems
  28. 28. Interface to Riak The model class which wishes to store stuff in Riak has : GSRiak("http://localhost:6999/riak", "firstbucket") © 2012-2013 Megam Systems
  29. 29. Interface to RiaK Every model provides its "bucketName". The RIAK Base URL will be pulled from the play configuration. © 2012-2013 Megam Systems
  30. 30. Find All List of Nodes By Name GET : /nodes © 2012-2013 Megam Systems
  31. 31. def findByNodeName(nodeNameList: Option[List[String]]): ValidationNel[Throwable, NodeResults] = { play.api.Logger.debug(("%-20s -->[%s]").format("models.Node", "findByNodeName:Entry")) play.api.Logger.debug(("%-20s -->[%s]").format("nodeNameList", nodeNameList)) (nodeNameList map { _.map { nodeName => play.api.Logger.debug(("%-20s -->[%s]").format("nodeName", nodeName)) (riak.fetch(nodeName) leftMap { t: NonEmptyList[Throwable] => new ServiceUnavailableError(nodeName, (t.list.map(m => m.getMessage)).mkString("n")) }).toValidationNel.flatMap { xso: Option[GunnySack] => xso match { case Some(xs) => { //JsonScalaz.Error doesn't descend from java.lang.Error or Throwable. Screwy. (NodeResult.fromJson(xs.value) leftMap { t: NonEmptyList[net.liftweb.json.scalaz.JsonScalaz.Error] => JSONParsingError(t) }).toValidationNel.flatMap { j: NodeResult => play.api.Logger.debug(("%-20s -->[%s]").format("noderesult", j)) Validation.success[Throwable, NodeResults](nels(j.some)).toValidationNel //screwy kishore, every element in a list ? } } case None => { Validation.failure[Throwable, NodeResults](new ResourceItemNotFound(nodeName, "")).toValidationNel } } } } // -> VNel -> fold by using an accumulator or successNel of empty. +++ => VNel1 + VNel2 } map { _.foldRight((NodeResults.empty).successNel[Throwable])(_ +++ _) }).head //return the folded element in the head. } © 2012-2013 Megam Systems
  32. 32. Notice the below code riak.fetch(nodeName) leftMap { t: NonEmptyList[Throwable] => new ServiceUnavailableError(nodeName, (t.list.map(m => m.getMessage)).mkString("n")) }).toValidationNel.flatMap { xso: Option[GunnySack] => Which Fetches data from Riak. Where riak private def riak: GSRiak = GSRiak(MConfig.riakurl, "nodes") © 2012-2013 Megam Systems
  33. 33. What does GSRiak Do ? Connect to the riak system using the scaliak client. private lazy val client: ScaliakClient = Scaliak.httpClient(uri) © 2012-2013 Megam Systems
  34. 34. And Fetch value(V) from Riak for a key(K) Create the bucket using following syntax. client.bucket(bucketName) fetch(key) function fetches value by riak. © 2012-2013 Megam Systems
  35. 35. private def fetchIO(key: String): IO[Validation[Throwable, Option[GunnySack]]] = { logger.debug("_/-->fetchIO:" + key) bucketIO flatMap { mgBucket => //mgBucket is ValidationNel[Throwable, ScaliakBucket] mgBucket match { case Success(realMeat) => (realMeat.fetch(key) flatMap { x => x match { case Success(res) => Validation.success[Throwable, Option[GunnySack]] (res).pure[IO] case Failure(err) => Validation.failure[Throwable, Option[GunnySack]] (RiakError(err)).pure[IO] } }) case Failure(nahNoBucket) => Validation.failure[Throwable, Option[GunnySack]] (RiakError(nels(BucketFetchError(uri, bucketName, key)))).pure[IO] } } } //old code val fetchResult: ValidationNel[Throwable, Option[GunnySack]] = bucket.fetch(key).unsafePerformIO() def fetch(key: String) = fetchIO(key).unsafePerformIO.toValidationNel © 2012-2013 Megam Systems
  36. 36. FetchIO fetchIO method which when interpreted will result in a fetch operation of a bucket using a key. The "key : String, value: Option[GunnySack] are the input and output. Merely calling this method doesn't fetch results in a fetch operation. It just results in scalaz's IO[x]. © 2012-2013 Megam Systems
  37. 37. Beta Launch of Megam Cloud (Polygot PaaS) Our PaaS design => Link Register http://www.megam.co for an invite Twitter : @indykish © 2012-2013 Megam Systems
  38. 38. Screencast illustrating the Cloud API Servers working
  39. 39. Thank you for watching © 2012-2013 Megam Systems

×