Scala does the Catwalk

  • 367 views
Uploaded on

Slides from Ian Forsey and Ariel Kogan's session at Skill Matter's Scala Exchange 2013. …

Slides from Ian Forsey and Ariel Kogan's session at Skill Matter's Scala Exchange 2013.

-------------------------------------------------

In this session we will share our experience at Net-a-porter, creating our first reactive Scala/Akka/Spray service in a company with a long-standing Java codebase and production infrastructure.

We've heard how Twitter and LinkedIn adopted Scala on greenfield initiatives and we're excited to use a more expressive language running on a robust, familiar VM. But is the ecosystem ready to support the demands of a long-established enterprise infrastructure, mission-critical (non-Scala!) middleware and the traditional dev-test-release workflow?

We'll start by exposing what drove our decision to dive into Scala. Next: We'll talk about some of the challenges we faced designing, building, load testing and debugging our service. We will discuss some of the patterns we used moving to a more reactive platform, the availability/maturity of the tooling and some of the framework code we had to write. Finally we'll outline the benefits gained through embarking on this project and any prices we have paid for doing so.

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
367
On Slideshare
0
From Embeds
0
Number of Embeds
3

Actions

Shares
Downloads
0
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. SCALA does the catwalk Ian Forsey @theon github.com/theon Ariel Kogan @arikogan github.com/arikogan
  • 2. AGENDA • Who are we? What are we building? • Architecture • Lessons learnt – Design evolution – Flow: Embrace the pipelines – Stress is good – Optimise
  • 3. WHO ARE WE? WHAT ARE WE BUILDING?
  • 4. ARCHITECTURE
  • 5. DOMAINS OF KNOWLEDGE WISHLIST PRODUCT ALERT
  • 6. SERVICES WISHLIST PRESENTATIO N WISHLIST AGGREGATIO N WISHLIST PRODUCT ALERT WISHLIST DB
  • 7. TECHNOLOGY SBT REST
  • 8. DESIGN EVOLUTION
  • 9. APPLICATION ARCHITECTURE REST ROUTING APPLICATION CORE DTO REST CLIENT
  • 10. Should I use Actor Tells or Actor Asks? ! ?
  • 11. ACTOR REFRESHER Tells: wishlistClient ! GetItemsForWishlist(123) def receive = { case items: Items => ... } Asks: implicit val t = Timeout(5 seconds) val f = wishlistClient ? GetItemsForWishlist(123) f.map(items => { ... })
  • 12. FIRST DESIGNS: ASKS App Core Routing Clients ask ask ROUTING WISHLIS T ask GET WISHLIST ITEMS PRODUCT ask ALERT
  • 13. THREE PROBLEMS WITH THIS DESIGN
  • 14. PROBLEM 1 Timeout errors not helpful: akka.pattern.AskTimeoutException: Timed out at akka.pattern.PromiseActorRef$$anonfun$1.apply$mcV$sp(AskSupport.scala:312) at akka.actor.DefaultScheduler$$anon$8.run(Scheduler.scala:191) at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:137) at akka.dispatch.ForkJoinExecutorConfigurator$MailboxExecutionTask.exec(…) at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:262) at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(…) at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1478) at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(…)
  • 15. PROBLEM 1: SHORT-TERM FIX Better timeout failures: f recoverWith { case t: TimeoutException => Future.failed( new TimeoutException(”Alerts API Timeout”) ) }
  • 16. PROBLEM 2 Routing App Core Clients 5 sec 5 sec WISHLIS T 5 sec ROUTING GET WISHLIST ITEMS PRODUCT 5 sec 5 sec ALERT
  • 17. PROBLEM 2: SHORT-TERM FIX Be careful with timeouts • Larger at the routing • Smaller the deeper into your app core Extra logging for belt and braces: f recoverWith { case t: TimeoutException => { log.error(”Alerts API took too long”) Future.failed( new TimeoutException(”Alerts API Timeout”) ) } }
  • 18. PROBLEM 3 Actors – We’re doing it wrong! Good things from Actors: • Concurrency and State • Supervision Hierarchies
  • 19. AKKA SURVEY https://typesafe.com/blog/akka-survey-2013-and-roadmap-update
  • 20. WHAT TO DO? http://www.flickr.com/photos/laenulfean/5943132296/
  • 21. SECOND DESIGN – PER REQUEST ACTOR App Core Routing Clients tell tell ROUTING PER REQUEST WISHLIS T tell GET WISHLIST ITEMS PRODUCT tell ALERT
  • 22. SUPERVISION HIERARCHY Routing App Core Clients WISHLIST ROUTING PER REQUEST GET WISHLIST ITEMS PRODUCT A C B ALERTS
  • 23. TIMEOUTS App Core Routing Clients WISHLIST ROUTING PER REQUEST 5 sec GET WISHLIST ITEMS 10 sec PRODUCT A C B 5 sec ALERTS 5 sec
  • 24. DOWNSIDES • Performance Impact? – Creating Actors is cheap, but not free – Benchmark it – Horizontally scale out servers
  • 25. PER REQUEST ACTOR EXAMPLE APPLICATION github.com/net-a-porter/spray-actor-perrequest
  • 26. FLOW: EMBRACE THE PIPELINES
  • 27. SPRAY CLIENT PIPELINES Outgoing requests • Authentication WISHLIST AGGREGATIO N WISHLIST • Internationalisation • Marshalling • Logging • Traceability • Error handling
  • 28. SPRAY CLIENT PIPELINES The Benefits • Write your own! • Predefined flow of operations • Cross-cutting concerns • Holistic view of what’s happening
  • 29. FLOW. EMBRACE THE PIPELINES. def pipeline(proxyCtx: ProxyContext) = ( keepQueryStringFor(GET, DELETE) ~> whitelistHeaders(allowedHeaders) ~> whitelistQueryString(allowedParams) ~> addHeader("X-Authenticated-User", proxyCtx.userId) ~> requestIdHeader(proxyCtx.requestId) ~> updateHost(scheme, host, port) ~> meToOwner(proxyCtx.userId) ~> logRequest ~> sendReceive(transport) ~> transformConnectionFailure ~> logResponse ~> checkSuccessResponse ~> parseAsJson
  • 30. DIRECTIVES Incoming requests • WISHLIST AGGREGATIO N Log request • Error handling • Extract – Transaction tracing info – Pagination params • Authorisation • Validation
  • 31. SPRAY ROUTING DIRECTIVES • Keep your routing clean – Write your own directives • Cross-cutting concerns (e.g. validation, error handling) • Improves readability • Holistic view of what’s happening
  • 32. DIRECTIVES GET www.net-a-porter.com/newcollection?authToken=TOKEN path(Segment) { segment => ctx => log.info(”Received a request: ” + ctx.request) val authToken = ctx.request.uri.query.get("authToken") val isAuthorised = authToken.map(_ == “TOKEN").orElse(Some(false)).get if (isAuthorised) { ctx.complete("You have been authorised to access /" + segment) } else { ctx.reject(AuthorizationFailedRejection) } }
  • 33. DIRECTIVES GET www.net-a-porter.com/newcollection?authToken=TOKEN path(Segment) { segment => ctx => logRequest { val authToken = ctx.request.uri.query.get("authToken") val isAuthorised = authToken.map(_ == “TOKEN").orElse(Some(false)).get if (isAuthorised) { ctx.complete("You have been authorised to access /" + segment) } else { ctx.reject(AuthorizationFailedRejection) } } }
  • 34. DIRECTIVES GET www.net-a-porter.com/newcollection?authToken=TOKEN path(Segment) { segment => ctx => logRequest { withAuth { ctx.complete("You have been authorised to access /" + segment) } } }
  • 35. DIRECTIVES logReq { handleExceptions(myExceptionHandler()) { handleRejections(myRejectionHandler()) { respondWithMediaType(`application/json`) { pathPrefix("api") { withAuth { authCtx => wishlistRoute(authCtx) ~ itemsRoute(authCtx) }~ withoutAuth { authCtx => schemaRoute(authCtx) ~ corsRoute() ~ swaggerDocs() } ... }
  • 36. STRESS IS GOOD • Scenarios written in Scala • Nice DSL • Integration with Jenkins • Open Source
  • 37. STRESS IS GOOD Example Scenario: val httpConf = http .baseURL("http://www.net-a-porter.com") .disableFollowRedirect val scn = scenario("NAP homepage") .exec( http("NAP desktop homepage") .get("/") .queryParam("deviceType", "Desktop") .check(status.is(200))) setUp(scn.inject(ramp(10) over (5))) .protocols(httpConf)
  • 38. GATLING TIPS • Use Gatling 2 • Start by hitting a specific resource and then run a whole scenario • Log your KOs: val httpConf = http .baseURL("http://www.net-a-porter.com") .extraInfoExtractor { case (KO, _, req, res) => req.getUrl :: res.getResponseBody() :: Nil case _ } => Nil
  • 39. OPTIMISE
  • 40. TYPESAFE CONSOLE
  • 41. PRODUCTION MONITORING • Be careful when tools say they support Scala • Many Scala frameworks are not supported – E.g. Requests are not being tracked if you don’t use one of their supported HTTP clients. • Only very basic information is provided – E.g. No knowledge of the actor system
  • 42. SLICK MASTER SLAVE SUPPORT • Routing of database queries: write read • master slave Slick pull request pending def createWishlist(name: String) = readWrite { sqlu"INSERT INTO wishlist (name) VALUES ($name)".execute } def getWishlist(wishlistId: String) = readOnly { sql"SELECT * FROM wishlist WHERE id = $wishlistId".as[Wishlist].firstOption }
  • 43. OPTIMISE DISPATCHER
  • 44. OPTIMISE ? DISPATCHER
  • 45. OPTIMISE DISPATCHER A DISPATCHER B
  • 46. TRANSITIONING TO SCALA
  • 47. TRANSITIONING TO SCALA • In-house workshops • Coursera courses – Functional Programming Principles in Scala – Principles of Reactive Programming
  • 48. CHECK OUT OUR TECH BLOG techblog.net-a-porter.com