High Performance RPC with Finagle

@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #finagle #scala
High Performance RPC with
Sam BESSALAH
@samklr

RPC Redux
• Control of clients and servers
• Need for predictable performance
• Communication between different languages
• Isolated computations
• Fan out, Fan In distributed systems
• ...

Open sourced by Twitter in 2011 to solve this :

Asynchronous
Non-Blocking
Protocol Agnostic
Full Stack RPC
built on top of

http://monkey.org/~marius/funsrv.pdf

Load Balancing : Heap Balancer
Client Backoff Strategy
BackPressure
Failure Detection
Failover/Retry
Connection Pooling
Tracing (via ZipKin)
...
Production ready distributed toolbox

• Box
• Foursquare
• Nest
• Uber
• Pinterest
• NY Times
• Rdio
• PagerDuty
• Soundcloud
• Strava
• StumbleUpon
• ...
Known Users ...

@nivdul

Networking Layer
@nivdul

Building Blocks
Futures
Services
Filters

Futures
• Concurrency primitive in Finagle
• Different from scala.concurrent.Future and
java.util.concurrent.Future
• Less context-switching, easier cancelability
• From com.twitter.util.Future
• Composables
• Error and Exception Handling

Futures Combinators
• Can be composed with map, flatMap, handle and rescue
• Exception handling via:

Service

Services
class DummyService extends Service[httpx.Request, httpx.Response]
{
def apply(req: httpx.Request): Future[httpx.Response] = {
val response = req.response
response.setStatusCode(200)
response.setContentTypeJson()
response.write(requestHandler(req.getContentString()))
Future.value(response)
}

Filters
class Filter[-ReqIn, +RepOut, +ReqOut, -RepIn] extends
((ReqIn, Service[ReqOut, RepIn]) => Future[RepOut])
or
type Filter[Req, Rep] =
(Req, Service[Req, Rep]) => Future[Rep]

Filters
• Wrap services to modify inputs or outputs (or both) …
• Protocol and Network agnostic
• Composable

Filters
val myTimeoutFilter =
new TimeoutFilter[String,Int](1.second,DefaultTimer.twitter)

Filters
new TimeoutFilter[String, Int](1.second,DefaultTimer.twitter)
val myService = new DummyService

Filters
new TimeoutFilter[String, Int](1.second,DefaultTimer.twitter)
val server = myTimeOutFilter andThen myService

Filters
new TimeoutFilter[String,Int](
1.second, DefaultTimer.twitter)
val service = myService andThen myTimeOutFilter
val server = Httpx.serve( “:8080”, service)
Await.ready(server)

Filters
Can be used for:
• Retries
• Logging
• Serialization
• Authentication
• Retries
• Timeouts
• ...

Clients

Clients
Finagle client are just full stack services
val myClient: Service[Request, Response] =
Httpx.newService("rtb-bidder.com:8081")
val bidder = Httpx.serve(":8080", myClient)

Finagle in a nutshell

Protocols
• Thrift
• Memcache
• Redis
• HTTP
• SMTP
• MySQL
• Zookeeper
• Mux
Protocol.newService( … )

Finagle Mux
• RPC Protocol Multiplexing
• One network connection per client-server session
• Maximization of available bandwidth without incurring the cost of
opening additional sockets
• Elimination of head-of-line blocking
• Explicit queue management
• Pure session layer (OSI )

Finagle Serial

Finagle Serial
Use binary format. Scodec, Protobuf, Thrift, Avro, etc.

Service discovery
• Finagle ServerSets via ”Names”
• Uses Zookeeper as Name server
val client = Http.newService(
“zk!myzkhost.home.com:2181!/my/zk/path” )

Twitter-Server
• Template for servers at Twitter built on top of Finagle
• Flags for configuration
• Logging and Metrics
• Admin HTTP UI
• Lifecycle management endpoints, handy for Docker and/or Mesos
• Tracing via ZipKin

Finagle Java
Service<Request, Response> service = new Service<Request, Response>() {
public Future<Response> apply(Request request) {
Response response = Response.apply();
response.setContent(ChannelBuffers.wrappedBuffer("yo".getBytes()));
Future<Response> future = Future.value(response);
return future;
}};
ServerBuilder.safeBuild(
service,
ServerBuilder.get()
.codec(Http.get())
.bindTo(new InetSocketAddress(InetAddress.getLoopbackAddress(), 0))
.name("HttpServer"));
}

Finatra
• Library to easily build API Services on top of the Twitter Stack (Finagle,
Twitter-Server, Twitter-util, etc.)
• Supports Http and Thrift services
• Brings Sinatra like Style to Finagle for Rest Services

Finatra
class HelloWorldController extends Controller {
get("/hi") { request: Request =>
"Hello " + request.params.getOrElse("name", "unnamed")
}
post("/hi") { hiRequest: HiRequest =>
"Hello " + hiRequest.name + " with id " + hiRequest.id
}
}

Finatra
object HelloWorldServerMain extends HelloWorldServer
class HelloWorldServer extends HttpServer {
override def modules = Seq(Slf4jBridgeModule)
override def configureHttp(router: HttpRouter) {
router
.filter[LoggingMDCFilter[Request, Response]]
.filter[TraceIdMDCFilter[Request, Response]]
.filter[CommonFilters]
.add[HelloWorldController]
}
}

● Thin layer of purely functional basic blocks, to build
composable REST APIs.
● Built on top of Finagle
● Functional wrapper of Finagle-http
● Performance close to Bare Metal Finagle
http://github.com/finagle/finch

Zipkin
• Zipkin is a distributed tracing system that helps us gather timing data
for all the disparate services at Twitter. It manages both the collection
and lookup of this data through a Collector and a Query service.
• Implementation of Google’s Dapper paper
• Moved to OpenZipkin

ZipKin - Open ZipKin

Finagle : Cons
• Stuck to old versions of Netty. Currently Netty 3 and now moving to 4
• API Changes since 6.x. No more Netty API Leaking through Finagle’s
• Mismatch between scala.concurrent.Future and com.twitter.util.Future
• Slow to update
• Documentation not exhaustive. Thanks for gitter.im and the Mailing List
• Can be painful to plug to other Metrics and Monitoring systems. Hopefully
it’s changing

Quick Demo

Conclusion

High Performance RPC with Finagle

More Related Content

What's hot

Viewers also liked

Similar to High Performance RPC with Finagle

More from Samir Bessalah

Recently uploaded

High Performance RPC with Finagle

Editor's Notes