![SLICK MASTER SLAVE SUPPORT
•
Routing of database queries:
write
read
•
master
slave
Slick pull request pending
def createWishlist(name: String) = readWrite {
sqlu"INSERT INTO wishlist (name) VALUES ($name)".execute
}
def getWishlist(wishlistId: String) = readOnly {
sql"SELECT * FROM wishlist WHERE id = $wishlistId".as[Wishlist].firstOption
}](https://image.slidesharecdn.com/scaladoesthecatwalk23-nonotes-140205063014-phpapp02/85/Scala-does-the-Catwalk-46-320.jpg)
Uploaded byAriel Kogan
Scala does the Catwalk
The document outlines a discussion led by Ian Forsey and Ariel Kogan about application architecture using Scala and Akka, emphasizing design evolution and handling common problems such as timeout errors and actor management. It also highlights the benefits of using flow pipelines for request handling and provides insights on stress testing and optimization strategies. Additionally, it includes examples and recommendations for improving the performance and reliability of services built with these technologies.
More Related Content
Scala does the Catwalk
- 1. SCALA does the catwalk IanForsey @theon github.com/theon Ariel Kogan @arikogan github.com/arikogan
- 2. AGENDA • Who are we?What are we building? • Architecture • Lessons learnt – Design evolution – Flow: Embrace the pipelines – Stress is good – Optimise
- 3. WHO ARE WE? WHATARE WE BUILDING?
- 8.
- 9.
- 10.
- 11.
- 12.
- 13. APPLICATION ARCHITECTURE REST ROUTING APPLICATIONCORE DTO REST CLIENT
- 14. Should I useActor Tells or Actor Asks? ! ?
- 15. ACTOR REFRESHER Tells: wishlistClient !GetItemsForWishlist(123) def receive = { case items: Items => ... } Asks: implicit val t = Timeout(5 seconds) val f = wishlistClient ? GetItemsForWishlist(123) f.map(items => { ... })
- 16. FIRST DESIGNS: ASKS AppCore Routing Clients ask ask ROUTING WISHLIS T ask GET WISHLIST ITEMS PRODUCT ask ALERT
- 17. THREE PROBLEMS WITH THISDESIGN
- 18. PROBLEM 1 Timeout errorsnot helpful: akka.pattern.AskTimeoutException: Timed out at akka.pattern.PromiseActorRef$$anonfun$1.apply$mcV$sp(AskSupport.scala:312) at akka.actor.DefaultScheduler$$anon$8.run(Scheduler.scala:191) at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:137) at akka.dispatch.ForkJoinExecutorConfigurator$MailboxExecutionTask.exec(…) at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:262) at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(…) at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1478) at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(…)
- 19. PROBLEM 1: SHORT-TERMFIX Better timeout failures: f recoverWith { case t: TimeoutException => Future.failed( new TimeoutException(”Alerts API Timeout”) ) }
- 20. PROBLEM 2 Routing App Core Clients 5sec 5 sec WISHLIS T 5 sec ROUTING GET WISHLIST ITEMS PRODUCT 5 sec 5 sec ALERT
- 21. PROBLEM 2: SHORT-TERMFIX Be careful with timeouts • Larger at the routing • Smaller the deeper into your app core Extra logging for belt and braces: f recoverWith { case t: TimeoutException => { log.error(”Alerts API took too long”) Future.failed( new TimeoutException(”Alerts API Timeout”) ) } }
- 22. PROBLEM 3 Actors –We’re doing it wrong! Good things from Actors: • Concurrency and State • Supervision Hierarchies
- 23.
- 24.
- 25. SECOND DESIGN –PER REQUEST ACTOR App Core Routing Clients tell tell ROUTING PER REQUEST WISHLIS T tell GET WISHLIST ITEMS PRODUCT tell ALERT
- 26. SUPERVISION HIERARCHY Routing App Core Clients WISHLIST ROUTING PER REQUEST GETWISHLIST ITEMS PRODUCT A C B ALERTS
- 27. TIMEOUTS App Core Routing Clients WISHLIST ROUTING PER REQUEST 5 sec GETWISHLIST ITEMS 10 sec PRODUCT A C B 5 sec ALERTS 5 sec
- 28. DOWNSIDES • Performance Impact? – Creating Actorsis cheap, but not free – Benchmark it – Horizontally scale out servers
- 29. PER REQUEST ACTOR EXAMPLEAPPLICATION github.com/net-a-porter/spray-actor-perrequest
- 30.
- 31. SPRAY CLIENT PIPELINES Outgoingrequests • Authentication WISHLIST AGGREGATIO N WISHLIST • Internationalisation • Marshalling • Logging • Traceability • Error handling
- 32. SPRAY CLIENT PIPELINES TheBenefits • Write your own! • Predefined flow of operations • Cross-cutting concerns • Holistic view of what’s happening
- 33. FLOW. EMBRACE THEPIPELINES. def pipeline(proxyCtx: ProxyContext) = ( keepQueryStringFor(GET, DELETE) ~> whitelistHeaders(allowedHeaders) ~> whitelistQueryString(allowedParams) ~> addHeader("X-Authenticated-User", proxyCtx.userId) ~> requestIdHeader(proxyCtx.requestId) ~> updateHost(scheme, host, port) ~> meToOwner(proxyCtx.userId) ~> logRequest ~> sendReceive(transport) ~> transformConnectionFailure ~> logResponse ~> checkSuccessResponse ~> parseAsJson
- 34. DIRECTIVES Incoming requests • WISHLIST AGGREGATIO N Log request • Errorhandling • Extract – Transaction tracing info – Pagination params • Authorisation • Validation
- 35. SPRAY ROUTING DIRECTIVES • Keepyour routing clean – Write your own directives • Cross-cutting concerns (e.g. validation, error handling) • Improves readability • Holistic view of what’s happening
- 36. DIRECTIVES GET www.net-a-porter.com/newcollection?authToken=TOKEN path(Segment) {segment => ctx => log.info(”Received a request: ” + ctx.request) val authToken = ctx.request.uri.query.get("authToken") val isAuthorised = authToken.map(_ == “TOKEN").orElse(Some(false)).get if (isAuthorised) { ctx.complete("You have been authorised to access /" + segment) } else { ctx.reject(AuthorizationFailedRejection) } }
- 37. DIRECTIVES GET www.net-a-porter.com/newcollection?authToken=TOKEN path(Segment) {segment => ctx => logRequest { val authToken = ctx.request.uri.query.get("authToken") val isAuthorised = authToken.map(_ == “TOKEN").orElse(Some(false)).get if (isAuthorised) { ctx.complete("You have been authorised to access /" + segment) } else { ctx.reject(AuthorizationFailedRejection) } } }
- 38. DIRECTIVES GET www.net-a-porter.com/newcollection?authToken=TOKEN path(Segment) {segment => ctx => logRequest { withAuth { ctx.complete("You have been authorised to access /" + segment) } } }
- 39. DIRECTIVES logReq { handleExceptions(myExceptionHandler()) { handleRejections(myRejectionHandler()){ respondWithMediaType(`application/json`) { pathPrefix("api") { withAuth { authCtx => wishlistRoute(authCtx) ~ itemsRoute(authCtx) }~ withoutAuth { authCtx => schemaRoute(authCtx) ~ corsRoute() ~ swaggerDocs() } ... }
- 40. STRESS IS GOOD • Scenarioswritten in Scala • Nice DSL • Integration with Jenkins • Open Source
- 41. STRESS IS GOOD ExampleScenario: val httpConf = http .baseURL("http://www.net-a-porter.com") .disableFollowRedirect val scn = scenario("NAP homepage") .exec( http("NAP desktop homepage") .get("/") .queryParam("deviceType", "Desktop") .check(status.is(200))) setUp(scn.inject(ramp(10) over (5))) .protocols(httpConf)
- 42. GATLING TIPS • Use Gatling2 • Start by hitting a specific resource and then run a whole scenario • Log your KOs: val httpConf = http .baseURL("http://www.net-a-porter.com") .extraInfoExtractor { case (KO, _, req, res) => req.getUrl :: res.getResponseBody() :: Nil case _ } => Nil
- 43.
- 44.
- 45. PRODUCTION MONITORING • Be carefulwhen tools say they support Scala • Many Scala frameworks are not supported – E.g. Requests are not being tracked if you don’t use one of their supported HTTP clients. • Only very basic information is provided – E.g. No knowledge of the actor system
- 46. SLICK MASTER SLAVESUPPORT • Routing of database queries: write read • master slave Slick pull request pending def createWishlist(name: String) = readWrite { sqlu"INSERT INTO wishlist (name) VALUES ($name)".execute } def getWishlist(wishlistId: String) = readOnly { sql"SELECT * FROM wishlist WHERE id = $wishlistId".as[Wishlist].firstOption }
- 47.
- 48.
- 49.
- 50.
- 51. TRANSITIONING TO SCALA • In-houseworkshops • Coursera courses – Functional Programming Principles in Scala – Principles of Reactive Programming
- 52. CHECK OUT OUR TECHBLOG techblog.net-a-porter.com