• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Building a Cloud API Server using  Play(SCALA) & Riak
 

Building a Cloud API Server using Play(SCALA) & Riak

on

  • 2,070 views

Megam's Cloud API is a stateless REST based cloud server. Our SDKs abstract away the REST interface and provide a set of handy methods you can interact with when deleting, creating, updating and ...

Megam's Cloud API is a stateless REST based cloud server. Our SDKs abstract away the REST interface and provide a set of handy methods you can interact with when deleting, creating, updating and deleting objects. We've also provided information on how we use HMAC to create authorization headers for use when accessing Megam.

Statistics

Views

Total Views
2,070
Views on SlideShare
1,430
Embed Views
640

Actions

Likes
2
Downloads
10
Comments
0

3 Embeds 640

http://blog.megam.co 607
https://twitter.com 30
http://webcache.googleusercontent.com 3

Accessibility

Categories

Upload Details

Uploaded via as OpenOffice

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Building a Cloud API Server using  Play(SCALA) & Riak Building a Cloud API Server using Play(SCALA) & Riak Presentation Transcript

    • Building a Cloud API Server using Play(SCALA) & Riak RESTful API for Megam Cloud
    • We'll Cover Architecture of our API ➔ Authentication using HMAC ➔ How to handle JSON Requests/Response ➔ How to handle Errors ➔ How do you interface with Riak. ➔ © 2012-2013 Megam Systems
    • Response (json) Request (json) Auth OK ? Native API Wrappers Ruby HMAC Cloud API Server Router Herk Riak Snowflake ID © 2012-2013 Megam Systems Funnel Request Funnel Response
    • We'll Use Play 2.2.0 setup => Link SBT 0.13.0 Migration => Link play2-auth Authentication Scala 2.10.3 Play 2.2.0 SBT 0.13.0 Riak 1.4.2 © 2012-2013 Megam Systems
    • We'll also use scalaz "org.scalaz" %% "scalaz-core" % "7.0.3" Code is weaved with Functional Programming using scalaz © 2012-2013 Megam Systems
    • Code : https://github.com/indykish/megam_play.git © 2012-2013 Megam Systems
    • Beta Launch of Megam Cloud (Polygot PaaS) Our PaaS design => Link Register http://www.megam.co for an invite Twitter : @indykish © 2012-2013 Megam Systems
    • Screencast illustrating the Cloud API Servers working live
    • Play2-Auth : Setup play2-auth : offers Authentication and Authorization features to play framework applications. Add a dependency declaration into your Build.scala file: val appDependencies = Seq( "jp.t2v" %% "play2.auth" % "0.11.0-SNAPSHOT", "jp.t2v" %% "play2.auth.test" % "0.11.0-SNAPSHOT" % "test" ) © 2012-2013 Megam Systems
    • Authentication play2-auth uses Stackable Controller. This is handy for Authentication. All you need to do is use StackAction in your Controller. Your Controller will first call StackAction operation and then compose it with other Actions. © 2012-2013 Megam Systems
    • Scenario /nodes HTTP Request to Nodes shall be authenticated using HMAC Customer onboarded and has a email/api_key (or) private cert. © 2012-2013 Megam Systems
    • Let us create a Nodes controller object Nodes extends Controller with APIAuthElement © 2012-2013 Megam Systems
    • Controller - Nodes def post = StackAction(parse.tolerantText) { implicit request => (Validation.fromTryCatch[SimpleResult] { reqFunneled match { case Success(succ) => { val freq = succ.getOrElse(throw new Error("Request wasn't funneled. Verify the header.")) val email = freq.maybeEmail.getOrElse(throw new Error("Email not found (or) invalid.")) val clientAPIBody = freq.clientAPIBody.getOrElse(throw new Error("Body not found (or) invalid.")) models.Nodes.create(email, clientAPIBody) match { case Success(succ) => val tuple_succ = succ.getOrElse(("Nah", "Bah", "Gah")) } case Failure(err) => { val rn: FunnelResponse = new HttpReturningError(err) Status(rn.code)(rn.toJson(true)) } } } case Failure(err) => { val rn: FunnelResponse = new HttpReturningError(err) Status(rn.code)(rn.toJson(true)) } } }).fold(succ = { a: SimpleResult => a }, fail = { t: Throwable => Status(BAD_REQUEST) (t.getMessage) }) } © 2012-2013 Megam Systems Full code
    • What is happening ? A HTTPRequest that comes to Cloud API Server gets funneled implicitly. FunneledRequest as seen in next page. © 2012-2013 Megam Systems
    • FunneledRequest(FR) A Case class case class FunneledRequest(maybeEmail: Option[String], clientAPIHmac: Option[String], clientAPIDate: Option[String], clientAPIPath: Option[String], clientAPIBody: Option[String]) { val wowEmail = { val EmailRegex = """^[a-z0-9_+-]+(.[a-z0-9_+-]+)*@[a-z0-9-]+(.[a-z0-9-]+)*. ([a-z]{2,4})$""".r maybeEmail.flatMap(x => EmailRegex.findFirstIn(x)) } match { case Some(succ) => Validation.success[Throwable, Option[String]](succ.some) case None => Validation.failure[Throwable, Option[String]](new MalformedHeaderError(maybeEmail.get, """Email is blank or invalid. Kindly provide us an email in the standard format.n" eg: goodemail@megam.com""")) } val mkSign = { val ab = ((clientAPIDate ++ clientAPIPath ++ calculateMD5(clientAPIBody)) map { a: String => a }).mkString("n") play.api.Logger.debug(("%-20s -->[%s]").format("FunnelRequest:mkSign", ab)) ab } } Full code FunneledRequestBuilder creates FR © 2012-2013 Megam Systems
    • FR Builder Takes a rawheader HMAC and creates FR //Look for the X_Megam_HMAC field. If not the FunneledRequest will be None. private lazy val frOpt: Option[FunneledRequest] = (for { hmac <- rawheader.get(X_Megam_HMAC) trimmed <- hmac.trim.some res <- trimmed.some if (res.indexOf(":") > 0) } yield { val res1 = res.split(":").take(2) FunneledRequest(res1(0).some, res1(1).some, clientAPIReqDate, clientAPIReqPath, clientAPIReqBody) }) Full code © 2012-2013 Megam Systems
    • Sample Processed FR FunneledRequest [email =Some(steve@olympics.com)] [apiHMAC =Some(6010ab91b07ee680aee8bd8075591e1f4fc8bc58)] [apiDATE =Some(2013-10-12 19:04)] [apiPATH =Some(/v1/predefclouds)] [apiBody =Some()] Full code © 2012-2013 Megam Systems
    • APIAuthElement APIAuthElement is sub trait for stackable controller. APIAuthElememt trait will handle our auth operation and will chain to next action only on success © 2012-2013 Megam Systems
    • APIElement trait APIAuthElement extends StackableController { self: Controller => case object APIAccessedKey extends RequestAttributeKey[Option[String]] override def proceed[A](req: RequestWithAttributes[A])(f: RequestWithAttributes[A] => Future[SimpleResult]): Future[SimpleResult] = { SecurityActions.Authenticated(req) match { case Success(rawRes) => super.proceed(req.set(APIAccessedKey, rawRes))(f) case Failure(err) => { val g = Action { implicit request => val rn: FunnelResponse = new HttpReturningError(err) //implicitly loaded. SimpleResult(header = ResponseHeader(rn.code, Map(CONTENT_TYPE -> "text/plain")), body = Enumerator(rn.toJson(true).getBytes(UTF8Charset) )) } val origReq = req.asInstanceOf[Request[AnyContent]] g(origReq) } } } implicit def reqFunneled[A](implicit req: RequestWithAttributes[A]): ValidationNel[Throwable, Option[FunneledRequest]] = req2FunnelBuilder(req).funneled implicit def apiAccessed[A](implicit req: RequestWithAttributes[A]): Option[String] = req.get(APIAccessedKey).get } Full code © 2012-2013 Megam Systems
    • Securing API Uses SecurityActions to authenticate a FunneledRequest Get FR from controller Extract information from the FR calculate a HMAC and compares the computed HMAC from Riak. Authentication Error if match fails. © 2012-2013 Megam Systems
    • Respond back As JSON We respond back as JSON using FunnelResponse – Code (HTTP Status Code : 404, 503.. ) – Message (A String message) – More (Detail info like support links) – JSON_CLAZ (A String understood by an unmarshaller or receiver) © 2012-2013 Megam Systems
    • FunnelResponse case class FunnelResponse(code: Int, msg: String, more: String, json_claz: String, msg_type: String = "error", links: String = tailMsg) { def toJValue: JValue = { import net.liftweb.json.scalaz.JsonScalaz.toJSON import controllers.funnel.FunnelResponseSerialization val funser = new FunnelResponseSerialization() toJSON(this)(funser.writer) } def toJson(prettyPrint: Boolean = false): String = if (prettyPrint) { pretty(render(toJValue)) } else { compactRender(toJValue) } } © 2012-2013 Megam Systems
    • Funnel Errors ResourceItemNotFound CannotAuthenticate JSONParsingError HTTPReturningError MalformedBody MalformedHeader ServiceUnAvailable © 2012-2013 Megam Systems
    • Funnel Errors Object Case class *Errors in FunnelErrors object FunnelErrors { val tailMsg = """Forum :https://groups.google.com/forum/?fromgroups=#!forum/megamlive. |API :https://api.megam.co |Docs :http://docs.megam.co |Support :http://support.megam.co""".stripMargin case class CannotAuthenticateError(input: String, msg: String, httpCode: Int = BAD_REQUEST) extends java.lang.Error(msg) …. } HTTPReturningError folds the App defined error case class HttpReturningError(errNel: NonEmptyList[Throwable]) extends Exception { def mkMsg(err: Throwable): String = { err.fold( a => """Authentication failure using the email/apikey combination. %n'%s' |Verify the email and api key combination. """.format(a.input).stripMargin, … } © 2012-2013 Megam Systems
    • RichThrowable, implicit error to json implicit class RichThrowable(thrownExp: Throwable) { def fold[T]( cannotAuthError: CannotAuthenticateError => T, malformedBodyError: MalformedBodyError => T, malformedHeaderError: MalformedHeaderError => T, serviceUnavailableError: ServiceUnavailableError => T, resourceNotFound: ResourceItemNotFound => T, anyError: Throwable => T): T = thrownExp match { case a @ CannotAuthenticateError(_, _, _) => cannotAuthError(a) case m @ MalformedBodyError(_, _, _) => malformedBodyError(m) case h @ MalformedHeaderError(_, _, _) => malformedHeaderError(h) case c @ ServiceUnavailableError(_, _, _) => serviceUnavailableError(c) case r @ ResourceItemNotFound(_, _, _) => resourceNotFound(r) case t @ _ => anyError(t) } } implicit def err2FunnelResponse(hpret: HttpReturningError) = new FunnelResponse(hpret.code.getOrElse(BAD_REQUEST), hpret.msg, hpret.more.getOrElse(new String("none")), "Megam::Error", hpret.severity) implicit def err2FunnelResponses(hpret: HttpReturningError) = hpret.errNel.map { err: Throwable => err.fold(a => new FunnelResponse(hpret.mkCode(a).getOrElse(BAD_REQUEST), hpret.mkMsg(a), hpret.mkMore(a), "Megam::Error", hpret.severity), m => new FunnelResponse(hpret.mkCode(m).getOrElse(BAD_REQUEST), hpret.mkMsg(m), hpret.mkMore(m), "Megam::Error", hpret.severity), h => new FunnelResponse(hpret.mkCode(h).getOrElse(BAD_REQUEST), hpret.mkMsg(h), hpret.mkMore(h), "Megam::Error", hpret.severity), c => new FunnelResponse(hpret.mkCode(c).getOrElse(BAD_REQUEST), hpret.mkMsg(c), hpret.mkMore(c), "Megam::Error", hpret.severity), r => new FunnelResponse(hpret.mkCode(r).getOrElse(BAD_REQUEST), hpret.mkMsg(r), hpret.mkMore(r), "Megam::Error", hpret.severity), t => new FunnelResponse(hpret.mkCode(t).getOrElse(BAD_REQUEST), hpret.mkMsg(t), hpret.mkMore(t), "Megam::Error", hpret.severity)) © 2012-2013 Megam Systems }.some
    • Interface to RiaK Scaliak library to Interface with Riak "com.stackmob" % "scaliak_2.10" % "0.8.0" GSRiak - A Wrapper on top of Scaliak "com.github.indykish" % "megam_common_2.10" % "0.1.0-SNAPSHOT", © 2012-2013 Megam Systems
    • Code for megam_common : https://github.com/indykish/megam_common.git © 2012-2013 Megam Systems
    • Interface to Riak The model class which wishes to store stuff in Riak has : GSRiak("http://localhost:6999/riak", "firstbucket") © 2012-2013 Megam Systems
    • Interface to RiaK Every model provides its "bucketName". The RIAK Base URL will be pulled from the play configuration. © 2012-2013 Megam Systems
    • Find All List of Nodes By Name GET : /nodes © 2012-2013 Megam Systems
    • def findByNodeName(nodeNameList: Option[List[String]]): ValidationNel[Throwable, NodeResults] = { play.api.Logger.debug(("%-20s -->[%s]").format("models.Node", "findByNodeName:Entry")) play.api.Logger.debug(("%-20s -->[%s]").format("nodeNameList", nodeNameList)) (nodeNameList map { _.map { nodeName => play.api.Logger.debug(("%-20s -->[%s]").format("nodeName", nodeName)) (riak.fetch(nodeName) leftMap { t: NonEmptyList[Throwable] => new ServiceUnavailableError(nodeName, (t.list.map(m => m.getMessage)).mkString("n")) }).toValidationNel.flatMap { xso: Option[GunnySack] => xso match { case Some(xs) => { //JsonScalaz.Error doesn't descend from java.lang.Error or Throwable. Screwy. (NodeResult.fromJson(xs.value) leftMap { t: NonEmptyList[net.liftweb.json.scalaz.JsonScalaz.Error] => JSONParsingError(t) }).toValidationNel.flatMap { j: NodeResult => play.api.Logger.debug(("%-20s -->[%s]").format("noderesult", j)) Validation.success[Throwable, NodeResults](nels(j.some)).toValidationNel //screwy kishore, every element in a list ? } } case None => { Validation.failure[Throwable, NodeResults](new ResourceItemNotFound(nodeName, "")).toValidationNel } } } } // -> VNel -> fold by using an accumulator or successNel of empty. +++ => VNel1 + VNel2 } map { _.foldRight((NodeResults.empty).successNel[Throwable])(_ +++ _) }).head //return the folded element in the head. } © 2012-2013 Megam Systems
    • Notice the below code riak.fetch(nodeName) leftMap { t: NonEmptyList[Throwable] => new ServiceUnavailableError(nodeName, (t.list.map(m => m.getMessage)).mkString("n")) }).toValidationNel.flatMap { xso: Option[GunnySack] => Which Fetches data from Riak. Where riak private def riak: GSRiak = GSRiak(MConfig.riakurl, "nodes") © 2012-2013 Megam Systems
    • What does GSRiak Do ? Connect to the riak system using the scaliak client. private lazy val client: ScaliakClient = Scaliak.httpClient(uri) © 2012-2013 Megam Systems
    • And Fetch value(V) from Riak for a key(K) Create the bucket using following syntax. client.bucket(bucketName) fetch(key) function fetches value by riak. © 2012-2013 Megam Systems
    • private def fetchIO(key: String): IO[Validation[Throwable, Option[GunnySack]]] = { logger.debug("_/-->fetchIO:" + key) bucketIO flatMap { mgBucket => //mgBucket is ValidationNel[Throwable, ScaliakBucket] mgBucket match { case Success(realMeat) => (realMeat.fetch(key) flatMap { x => x match { case Success(res) => Validation.success[Throwable, Option[GunnySack]] (res).pure[IO] case Failure(err) => Validation.failure[Throwable, Option[GunnySack]] (RiakError(err)).pure[IO] } }) case Failure(nahNoBucket) => Validation.failure[Throwable, Option[GunnySack]] (RiakError(nels(BucketFetchError(uri, bucketName, key)))).pure[IO] } } } //old code val fetchResult: ValidationNel[Throwable, Option[GunnySack]] = bucket.fetch(key).unsafePerformIO() def fetch(key: String) = fetchIO(key).unsafePerformIO.toValidationNel © 2012-2013 Megam Systems
    • FetchIO fetchIO method which when interpreted will result in a fetch operation of a bucket using a key. The "key : String, value: Option[GunnySack] are the input and output. Merely calling this method doesn't fetch results in a fetch operation. It just results in scalaz's IO[x]. © 2012-2013 Megam Systems
    • Beta Launch of Megam Cloud (Polygot PaaS) Our PaaS design => Link Register http://www.megam.co for an invite Twitter : @indykish © 2012-2013 Megam Systems
    • Screencast illustrating the Cloud API Servers working
    • Thank you for watching © 2012-2013 Megam Systems