Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

アドテク×Scala @Dynalyst

3,937 views

Published on

How to build Spray
How to build Akka
How to Monitor them

Published in: Engineering
  • Be the first to comment

アドテク×Scala @Dynalyst

  1. 1. Scala {Culture}
  2. 2. Who Am I • Now: Dynalyst • Lang: Korean(30+ years) Japanese(8+ years) JAVA(8.2+ years) Scala(a year) • AdTech: AMoAd DSP CAA Reward
  3. 3. What Dynalyst • Dynamic Retargeting + Analyst • ユーザーの趣味嗜好に適した広告配 信を行う国内初のスマートフォンに 特化したダイナミックリターゲティ ング広告
  4. 4. Scala USAGE
  5. 5. Today’s Talk •How to build Spray •How to build Akka •How to Monitor them
  6. 6. continuous importing D mark ad bid imp click cv redshift request logging importing stored tracking
  7. 7. continuous reporting D redshift date imp click cv 2014/09 10,000 500 10 2014/08 15,000 700 25 reporting & stored summaring bidding optimization
  8. 8. Benchmark
  9. 9. What is Spray? http://spray.io
  10. 10. What is Spray? spray is an open-source toolkit for building REST/HTTP-based integration layers on top of Scala and Akka. Being asynchronous, actor-based, fast, lightweight, modular and testable it's a great way to connect your Scala applications to the world.
  11. 11. Concurrency
  12. 12. Future[T] implicit val ec: ExecutionContext val f = Future { “hello world” } // Future[String]
  13. 13. Future[T] implicit val ec: ExecutionContext for { greeting ← Future(“hello world”) name ← Future(“Han”) } yield s”$greeting, $name” // Future[String]
  14. 14. FutureDirectives • spray-routing directives • non-blocking actor thread
  15. 15. FutureDirectives def onComplete[T] (future: ⇒ Future[T]) (implicit ec: ExecutionContext) :Directive1[Try[T]]
  16. 16. FutureDirectives implicit val ec: ExecutionContext def respondWithBidding = { onComplete(bidding) { case Success(bid: Bid) ⇒ … case Success(noBid: NoBid) ⇒ … case Failure(e) ⇒ … } }
  17. 17. Type of bidding def bidding:Future[A<:BidResponse]
  18. 18. Implementation def bidding:Future[A<:BidResponse] ! • simple but bad implementation Future { // All bidding computations :( }
  19. 19. Application Layers Spray Router Handle raw level http request & response Bidding Business Logic External Resources" Fetch bidding candidates from data stores
  20. 20. Application Layers Global Dispatcher Spray Router Handle raw level http request & response Bidding Business Logic External Resources" Fetch bidding candidates from data stores
  21. 21. Application Layers Spray Router Handle raw level http request & response Bidding Business Logic External Resources" Router Dispatcher Service Dispatcher Fetch bidding candidates from data stores Repository Dispatcher
  22. 22. Implementation trait Router { implicit val ec = RouterDispatcher def handle: Future[BidResponse] } trait Service { implicit val ec = ServiceDispatcher def bidding: Future[Bidder] } trait Repository { implicit val ec = RepositoryDispatcher def candidates: Future[List[Candidate]] }
  23. 23. How to test those Future[T] things • Specs2 - Matcher[Future[T]] bidding must beEqual(expected).await bidding must beEqual(expected).await ( retries = 2, timeout = 100.millis )
  24. 24. Abstract Future http://logji.blogspot.jp/2014/02/ the-abstract-future.html
  25. 25. Abstract Future trait Router[M[_]] { implicit val M: Monad[M] def handle: M[BidResponse] } trait Service[M[_]] { implicit val M: Monad[M] def bidding: M[Bidder] = M.point(bidder) def bidder = … } trait Repository[M[_]] { implicit val M: Monad[M] def candidates: M[List[Candidate]] }
  26. 26. Type class & Implicit object Service { implicit def fs = new Service[Future] { implicit val ec = ServiceDispatcher val M = Monad[Future] } implicit def ids = new Service[Id] { val M = Monad[Id] } } // type Id[+A] = A
  27. 27. Test Again val s = Service.ids s.bidding must beEqual(expected)
  28. 28. M[_] for Monad • Monad is a structure that represents computations defined as sequences of steps • Don’t dive into “Monad” here!
  29. 29. What about Akka? http://akka.io
  30. 30. continuous importing D redshift mark ad bid imp click cv request logging importing stored tracking
  31. 31. continuous reporting D redshift date imp click cv 2014/09 10,000 500 10 2014/08 15,000 700 25 reporting & stored summaring bidding optimization
  32. 32. Master/Worker http://letitcrash.com/post/ 29044669086/balancing-workload-across- nodes-with-akka-2
  33. 33. Overview
  34. 34. Overview
  35. 35. How to Monitor Akka • Typesafe Console http://resources.typesafe.com/docs/ console/manual/overview.html • Kamon http://kamon.io/
  36. 36. Typesafe console • Great user interface • Support by Typesafe • Expensive
  37. 37. Kamon • Open Source - Free • Metrics: Akka, Spray, Play (over 50 categories) • StatsD, New Relic, Datadog • Migration could be hard
  38. 38. Kamon meets Zabbix
  39. 39. Running Thread Count
  40. 40. Summary • Future Directive on Spray • Abstract Future (Type class & Implicit) • Akka master/worker pattern • Monitoring Akka with Kamon
  41. 41. Best Practices? • https://github.com/alexandru/ scala-best-practices • So many “MUST NOT, SHOULD NOT” • In my opinion, Trial and Error to find right patterns
  42. 42. We are Hiring! • 一緒に働く仲間を募集しています! Scalaに興味のある方はぜひ! adtech_scala@cyberagent.co.jp
  43. 43. Thanks han_sangwon@cyberagent.co.jp

×