Scala does the Catwalk

834 views
680 views

Published on

Slides from Ian Forsey and Ariel Kogan's session at Skill Matter's Scala Exchange 2013.

-------------------------------------------------

In this session we will share our experience at Net-a-porter, creating our first reactive Scala/Akka/Spray service in a company with a long-standing Java codebase and production infrastructure.

We've heard how Twitter and LinkedIn adopted Scala on greenfield initiatives and we're excited to use a more expressive language running on a robust, familiar VM. But is the ecosystem ready to support the demands of a long-established enterprise infrastructure, mission-critical (non-Scala!) middleware and the traditional dev-test-release workflow?

We'll start by exposing what drove our decision to dive into Scala. Next: We'll talk about some of the challenges we faced designing, building, load testing and debugging our service. We will discuss some of the patterns we used moving to a more reactive platform, the availability/maturity of the tooling and some of the framework code we had to write. Finally we'll outline the benefits gained through embarking on this project and any prices we have paid for doing so.

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
834
On SlideShare
0
From Embeds
0
Number of Embeds
13
Actions
Shares
0
Downloads
0
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Scala does the Catwalk

  1. 1. SCALA does the catwalk Ian Forsey @theon github.com/theon Ariel Kogan @arikogan github.com/arikogan
  2. 2. AGENDA • Who are we? What are we building? • Architecture • Lessons learnt – Design evolution – Flow: Embrace the pipelines – Stress is good – Optimise
  3. 3. WHO ARE WE? WHAT ARE WE BUILDING?
  4. 4. ARCHITECTURE
  5. 5. DOMAINS OF KNOWLEDGE WISHLIST PRODUCT ALERT
  6. 6. SERVICES WISHLIST PRESENTATIO N WISHLIST AGGREGATIO N WISHLIST PRODUCT ALERT WISHLIST DB
  7. 7. TECHNOLOGY SBT REST
  8. 8. DESIGN EVOLUTION
  9. 9. APPLICATION ARCHITECTURE REST ROUTING APPLICATION CORE DTO REST CLIENT
  10. 10. Should I use Actor Tells or Actor Asks? ! ?
  11. 11. ACTOR REFRESHER Tells: wishlistClient ! GetItemsForWishlist(123) def receive = { case items: Items => ... } Asks: implicit val t = Timeout(5 seconds) val f = wishlistClient ? GetItemsForWishlist(123) f.map(items => { ... })
  12. 12. FIRST DESIGNS: ASKS App Core Routing Clients ask ask ROUTING WISHLIS T ask GET WISHLIST ITEMS PRODUCT ask ALERT
  13. 13. THREE PROBLEMS WITH THIS DESIGN
  14. 14. PROBLEM 1 Timeout errors not helpful: akka.pattern.AskTimeoutException: Timed out at akka.pattern.PromiseActorRef$$anonfun$1.apply$mcV$sp(AskSupport.scala:312) at akka.actor.DefaultScheduler$$anon$8.run(Scheduler.scala:191) at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:137) at akka.dispatch.ForkJoinExecutorConfigurator$MailboxExecutionTask.exec(…) at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:262) at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(…) at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1478) at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(…)
  15. 15. PROBLEM 1: SHORT-TERM FIX Better timeout failures: f recoverWith { case t: TimeoutException => Future.failed( new TimeoutException(”Alerts API Timeout”) ) }
  16. 16. PROBLEM 2 Routing App Core Clients 5 sec 5 sec WISHLIS T 5 sec ROUTING GET WISHLIST ITEMS PRODUCT 5 sec 5 sec ALERT
  17. 17. PROBLEM 2: SHORT-TERM FIX Be careful with timeouts • Larger at the routing • Smaller the deeper into your app core Extra logging for belt and braces: f recoverWith { case t: TimeoutException => { log.error(”Alerts API took too long”) Future.failed( new TimeoutException(”Alerts API Timeout”) ) } }
  18. 18. PROBLEM 3 Actors – We’re doing it wrong! Good things from Actors: • Concurrency and State • Supervision Hierarchies
  19. 19. AKKA SURVEY https://typesafe.com/blog/akka-survey-2013-and-roadmap-update
  20. 20. WHAT TO DO? http://www.flickr.com/photos/laenulfean/5943132296/
  21. 21. SECOND DESIGN – PER REQUEST ACTOR App Core Routing Clients tell tell ROUTING PER REQUEST WISHLIS T tell GET WISHLIST ITEMS PRODUCT tell ALERT
  22. 22. SUPERVISION HIERARCHY Routing App Core Clients WISHLIST ROUTING PER REQUEST GET WISHLIST ITEMS PRODUCT A C B ALERTS
  23. 23. TIMEOUTS App Core Routing Clients WISHLIST ROUTING PER REQUEST 5 sec GET WISHLIST ITEMS 10 sec PRODUCT A C B 5 sec ALERTS 5 sec
  24. 24. DOWNSIDES • Performance Impact? – Creating Actors is cheap, but not free – Benchmark it – Horizontally scale out servers
  25. 25. PER REQUEST ACTOR EXAMPLE APPLICATION github.com/net-a-porter/spray-actor-perrequest
  26. 26. FLOW: EMBRACE THE PIPELINES
  27. 27. SPRAY CLIENT PIPELINES Outgoing requests • Authentication WISHLIST AGGREGATIO N WISHLIST • Internationalisation • Marshalling • Logging • Traceability • Error handling
  28. 28. SPRAY CLIENT PIPELINES The Benefits • Write your own! • Predefined flow of operations • Cross-cutting concerns • Holistic view of what’s happening
  29. 29. FLOW. EMBRACE THE PIPELINES. def pipeline(proxyCtx: ProxyContext) = ( keepQueryStringFor(GET, DELETE) ~> whitelistHeaders(allowedHeaders) ~> whitelistQueryString(allowedParams) ~> addHeader("X-Authenticated-User", proxyCtx.userId) ~> requestIdHeader(proxyCtx.requestId) ~> updateHost(scheme, host, port) ~> meToOwner(proxyCtx.userId) ~> logRequest ~> sendReceive(transport) ~> transformConnectionFailure ~> logResponse ~> checkSuccessResponse ~> parseAsJson
  30. 30. DIRECTIVES Incoming requests • WISHLIST AGGREGATIO N Log request • Error handling • Extract – Transaction tracing info – Pagination params • Authorisation • Validation
  31. 31. SPRAY ROUTING DIRECTIVES • Keep your routing clean – Write your own directives • Cross-cutting concerns (e.g. validation, error handling) • Improves readability • Holistic view of what’s happening
  32. 32. DIRECTIVES GET www.net-a-porter.com/newcollection?authToken=TOKEN path(Segment) { segment => ctx => log.info(”Received a request: ” + ctx.request) val authToken = ctx.request.uri.query.get("authToken") val isAuthorised = authToken.map(_ == “TOKEN").orElse(Some(false)).get if (isAuthorised) { ctx.complete("You have been authorised to access /" + segment) } else { ctx.reject(AuthorizationFailedRejection) } }
  33. 33. DIRECTIVES GET www.net-a-porter.com/newcollection?authToken=TOKEN path(Segment) { segment => ctx => logRequest { val authToken = ctx.request.uri.query.get("authToken") val isAuthorised = authToken.map(_ == “TOKEN").orElse(Some(false)).get if (isAuthorised) { ctx.complete("You have been authorised to access /" + segment) } else { ctx.reject(AuthorizationFailedRejection) } } }
  34. 34. DIRECTIVES GET www.net-a-porter.com/newcollection?authToken=TOKEN path(Segment) { segment => ctx => logRequest { withAuth { ctx.complete("You have been authorised to access /" + segment) } } }
  35. 35. DIRECTIVES logReq { handleExceptions(myExceptionHandler()) { handleRejections(myRejectionHandler()) { respondWithMediaType(`application/json`) { pathPrefix("api") { withAuth { authCtx => wishlistRoute(authCtx) ~ itemsRoute(authCtx) }~ withoutAuth { authCtx => schemaRoute(authCtx) ~ corsRoute() ~ swaggerDocs() } ... }
  36. 36. STRESS IS GOOD • Scenarios written in Scala • Nice DSL • Integration with Jenkins • Open Source
  37. 37. STRESS IS GOOD Example Scenario: val httpConf = http .baseURL("http://www.net-a-porter.com") .disableFollowRedirect val scn = scenario("NAP homepage") .exec( http("NAP desktop homepage") .get("/") .queryParam("deviceType", "Desktop") .check(status.is(200))) setUp(scn.inject(ramp(10) over (5))) .protocols(httpConf)
  38. 38. GATLING TIPS • Use Gatling 2 • Start by hitting a specific resource and then run a whole scenario • Log your KOs: val httpConf = http .baseURL("http://www.net-a-porter.com") .extraInfoExtractor { case (KO, _, req, res) => req.getUrl :: res.getResponseBody() :: Nil case _ } => Nil
  39. 39. OPTIMISE
  40. 40. TYPESAFE CONSOLE
  41. 41. PRODUCTION MONITORING • Be careful when tools say they support Scala • Many Scala frameworks are not supported – E.g. Requests are not being tracked if you don’t use one of their supported HTTP clients. • Only very basic information is provided – E.g. No knowledge of the actor system
  42. 42. SLICK MASTER SLAVE SUPPORT • Routing of database queries: write read • master slave Slick pull request pending def createWishlist(name: String) = readWrite { sqlu"INSERT INTO wishlist (name) VALUES ($name)".execute } def getWishlist(wishlistId: String) = readOnly { sql"SELECT * FROM wishlist WHERE id = $wishlistId".as[Wishlist].firstOption }
  43. 43. OPTIMISE DISPATCHER
  44. 44. OPTIMISE ? DISPATCHER
  45. 45. OPTIMISE DISPATCHER A DISPATCHER B
  46. 46. TRANSITIONING TO SCALA
  47. 47. TRANSITIONING TO SCALA • In-house workshops • Coursera courses – Functional Programming Principles in Scala – Principles of Reactive Programming
  48. 48. CHECK OUT OUR TECH BLOG techblog.net-a-porter.com

×