Julio Capote, Twitter

1,619 views

Published on

HighLoad++ 2013

Published in: Technology
0 Comments
4 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,619
On SlideShare
0
From Embeds
0
Number of Embeds
460
Actions
Shares
0
Downloads
23
Comments
0
Likes
4
Embeds 0
No embeds

No notes for slide

Julio Capote, Twitter

  1. 1. Distributing Systems in Scala with Finagle Julio Capote
  2. 2. About me • • • Julio Capote API team at Twitter @capotej
  3. 3. This talk • • • • Services Scala Finagle Examples
  4. 4. Scaling
  5. 5. Monolithic System • • • Limited to a single language and runtime Can only support few developers Deploys get more dangerous
  6. 6. Solution?
  7. 7. Distributed System • • • Each service written in appropriate language Can only support many developers Granular deploys
  8. 8. But, this is complex...
  9. 9. Quick Scala Intro
  10. 10. What is Scala? • • • • JVM language Interoperable with Java Functional But Object Oriented too
  11. 11. Values
  12. 12. val foo: String = "bar"
  13. 13. val age: Int = 27
  14. 14. Type inference
  15. 15. val foo = "bar"
  16. 16. val foo: String = "bar"
  17. 17. Collections
  18. 18. List(61,31,34)
  19. 19. Seq(1,2,3)
  20. 20. Dictionaries / Hashes
  21. 21. Map("foo" -> "bar")
  22. 22. Functional
  23. 23. val addOne = { x: Int => x + 1 }
  24. 24. addOne.apply(2) => 3
  25. 25. addOne(2) => 3
  26. 26. val list = Seq(1,2,3,4)
  27. 27. list.map { x => addOne(x) }
  28. 28. list.map(addOne(_))
  29. 29. list.map(addOne)
  30. 30. list map(addOne)
  31. 31. list map addOne
  32. 32. list map addOne filter { item: Int => item % 2 == 0 }
  33. 33. Function Composition
  34. 34. val addOne = { x: Int => x + 1 } val addTwo = { x: Int => x + 2 } val addThree = addOne compose addTwo
  35. 35. val addOne = { x: Int => x + 1 } val addTwo = { x: Int => x + 2 } val addThree = addOne compose addTwo
  36. 36. addThree(2) => 5
  37. 37. Object oriented
  38. 38. class Dog extends Animal def bark = { println("woof") } }
  39. 39. Options
  40. 40. Map("foo" -> "bar").get("foo")
  41. 41. Some(bar)
  42. 42. Map("foo" -> "bar").get("zzz")
  43. 43. None
  44. 44. val a = Map("foo" -> "bar") val opt = a.get("zzz") val result = opt.getOrElse("default")
  45. 45. val a = Map("foo" -> "bar") val opt = a.get("zzz") val result = opt match { case Some(x) => x case None => "default" }
  46. 46. val a = Map("foo" -> "bar") val opt = a.get("zzz") opt foreach { result => println(result) }
  47. 47. Finagle
  48. 48. What is Finagle? • • • • Client/Server RPC Framework Asynchronous Functional Protocol Agnostic
  49. 49. Futures
  50. 50. def fetchUrl(url: String) = Future[Response]
  51. 51. val f: Future[Response] = fetchUrl("google.ru") f map { response => println(response.content) }
  52. 52. val f: Future[Response] = fetchUrl("google.ru") f onSuccess { response => println(response.content) } onFailure { case e => log.error(e) }
  53. 53. val f: Future[Response] = fetchUrl("google.ru") f onSuccess { response => println(response.content) } onFailure { case e => log.error(e) }
  54. 54. Services
  55. 55. “Your Server as a Function”
  56. 56. type Service[Req, Rep] = Req => Future[Rep]
  57. 57. Request => Future[Response]
  58. 58. Http Client
  59. 59. val client: Service[Request, Response] = Http.newService("www.google.ru:80")
  60. 60. Making a request
  61. 61. val req = Request("/") client(req) onSuccess { resp => println(resp.content) }
  62. 62. val req = Request("/") client.apply(req) onSuccess { resp => println(resp.content) }
  63. 63. val req = Request("/") val resp = client(req) resp onSuccess { resp => println(resp.content) } Await.ready(resp)
  64. 64. val req = Request("/") val resp = client(req) resp onSuccess { resp => println(resp.content) } Await.ready(resp)
  65. 65. Http Server
  66. 66. class MyServer extends Service[Request, Response] { def apply(request: Request) = { Future.value(new Response("hi")) } }
  67. 67. class MyServer extends Service[Request, Response] { def apply(request: Request) = { Future.value(new Response("hi")) } }
  68. 68. Request => Future[Response]
  69. 69. Starting the server
  70. 70. val service = new MyServer val server = Http.serve(":8080", service) Await.ready(server)
  71. 71. Http Proxy
  72. 72. object Proxy extends App { val client: Service[Request, Response] = Http.newService("www.google.ru:80") val server = Http.serve(":8080", client) Await.ready(server) }
  73. 73. object Proxy extends App { val client: Service[Request, Response] = Http.newService("www.google.ru:80") val server = Http.serve(":8080", client) Await.ready(server) }
  74. 74. object Proxy extends App { val client: Service[Request, Response] = Http.newService("www.google.ru:80") val server = Http.serve(":8080", client) Await.ready(server) }
  75. 75. Filters
  76. 76. Filter[InRequest, InResponse, OutRequest, OutResponse]
  77. 77. Filter[InRequest, InResponse, OutRequest, OutResponse]
  78. 78. Filter[InRequest, InResponse, OutRequest, OutResponse]
  79. 79. SimpleFilter[Request, Response]
  80. 80. def apply(request: Req, service: Service[Req, Rep]): Future[Rep]
  81. 81. def apply(request: Req, service: Service[Req, Rep]): Future[Rep]
  82. 82. def apply(request: Req, service: Service[Req, Rep]): Future[Rep]
  83. 83. def apply(request: Req, service: Service[Req, Rep]): Future[Rep]
  84. 84. class ScreamFilter[Req, Rep] extends SimpleFilter[Req, Rep] def apply(request: Req, service: Service[Req, Rep]) = { service(request) onSuccess { response => response.setContent(response.content + "!") response } } }
  85. 85. class ScreamFilter[Req, Rep] extends SimpleFilter[Req, Rep] def apply(request: Req, service: Service[Req, Rep]) = { service(request) onSuccess { response => response.setContent(response.content + "!") response } } }
  86. 86. class ScreamFilter[Req, Rep] extends SimpleFilter[Req, Rep] def apply(request: Req, service: Service[Req, Rep]) = { service(request) onSuccess { response => response.setContent(response.content + "!") response } } }
  87. 87. class ScreamFilter[Req, Rep] extends SimpleFilter[Req, Rep] def apply(request: Req, service: Service[Req, Rep]) = { service(request) onSuccess { response => response.setContent(response.content + "!") response } } }
  88. 88. class ScreamFilter[Req, Rep] extends SimpleFilter[Req, Rep] def apply(request: Req, service: Service[Req, Rep]) = { service(request) onSuccess { response => response.setContent(response.content + "!") response } } }
  89. 89. class ScreamFilter[Req, Rep] extends SimpleFilter[Req, Rep] def apply(request: Req, service: Service[Req, Rep]) = { service(request) onSuccess { response => response.setContent(response.content + "!") response } } }
  90. 90. val service = new MyServer val server = Http.serve(":8080", service) Await.ready(server)
  91. 91. val service = new MyServer val myFilter = new MyFilter val newService = myFilter andThen service val server = Http.serve(":8080", newService) Await.ready(server)
  92. 92. val service = new MyServer val myFilter = new MyFilter val newService = myFilter andThen service val server = Http.serve(":8080", newService) Await.ready(server)
  93. 93. val service = new MyServer val myFilter = new MyFilter val newService = myFilter andThen service val server = Http.serve(":8080", newService) Await.ready(server)
  94. 94. Metrics
  95. 95. curl http://localhost:9990/admin/metrics.json
  96. 96. Add your own
  97. 97. class MyServer(stats: StatsReceiver) extends Service[Request, Response] { def apply(request: Request) = { stats.counter("my counter").incr() stats.time("fetch") { expensiveOperation() } Future.value(new Response("hi")) } }
  98. 98. class MyServer(stats: StatsReceiver) extends Service[Request, Response] { def apply(request: Request) = { stats.counter("my counter").incr() stats.time("fetch") { expensiveOperation() } Future.value(new Response("hi")) } }
  99. 99. class MyServer(stats: StatsReceiver) extends Service[Request, Response] { def apply(request: Request) = { stats.counter("my counter").incr() stats.time("fetch") { expensiveOperation() } Future.value(new Response("hi")) } }
  100. 100. Finatra
  101. 101. Finagle with Sinatra
  102. 102. class Example extends Controller { get("/") { request => render.plain("hi").toFuture } }
  103. 103. class Example extends Controller { get("/") { request => render.plain("hi").toFuture } }
  104. 104. class MyServer extends FinatraServer { val controller = new Example() register(controller) }
  105. 105. http://finatra.info
  106. 106. Questions? • • • • http://tinyurl.com/finagle-guide http://tinyurl.com/server-as-a-function http://finatra.info @capotej

×